Skip to content

sglang profiling - #189

Open
AakarshAMD wants to merge 1 commit into
ROCm:developfrom
AakarshAMD:sglang_profiling
Open

sglang profiling#189
AakarshAMD wants to merge 1 commit into
ROCm:developfrom
AakarshAMD:sglang_profiling

Conversation

@AakarshAMD

Copy link
Copy Markdown

Motivation

Add an opt-in profiling workflow for SGLang disaggregated serving using rocprofv3, ROCtx, and MoRI instrumentation, while preserving the normal non-profiling workflow.

Technical Details

  • Added a dedicated ROCm 7.2.3 profiling Dockerfile with SGLang and MoRI instrumentation patches.
  • Added benchmark_xPyD_profile.sh for profiling-specific request IDs, timing manifests, warmup control, and multi-sweep isolation.
  • Added conditional rocprofv3 kernel/marker capture and graceful worker finalization.
  • Added post-processing for clean traces, request statistics, MoRI correlation, kernel categorization, and local-rank-0 analysis.
  • Added validation for capture completeness, request identity, malformed durations, and MoRI transfer attribution.
  • Profiling is enabled with RUN_PROFILE=1; the standard benchmark and Dockerfile remain unchanged for normal runs.

Test Plan

  • Build the profiling image with ENABLE_ROCTX=1.
  • Run profiled 1P1D and 2P2D configurations across TP and EP models.
  • Verify benchmark completion, worker capture finalization, request-ID consistency, MoRI attribution, clean trace generation, and kernel-analysis outputs.

Test Result

All tested configurations passed:

  • DeepSeek-V3 — 1P1D, EP8
  • DeepSeek-V3 — 2P2D, EP16
  • DeepSeek-R1 — 1P1D, EP8
  • DeepSeek-R1 — 2P2D, EP16
  • Llama-3.1-8B-Instruct — 1P1D, TP8
  • Llama-3.3-70B-Instruct — 1P1D, TP8
  • Mixtral-8x7B-Instruct-v0.1 — 1P1D, TP8

Submission Checklist

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant