Skip to content

From NVIDIA Megatron-LM for visibility - #18

Open
RaymondLi0 wants to merge 7447 commits into
bigcode-project:multi-query-attentionfrom
NVIDIA:main
Open

From NVIDIA Megatron-LM for visibility#18
RaymondLi0 wants to merge 7447 commits into
bigcode-project:multi-query-attentionfrom
NVIDIA:main

Conversation

@RaymondLi0

@RaymondLi0 RaymondLi0 commented Jan 24, 2023

Copy link
Copy Markdown
Collaborator

No description provided.

@RaymondLi0
RaymondLi0 changed the base branch from multi-query-attention to before-merge June 20, 2023 20:12
@RaymondLi0
RaymondLi0 changed the base branch from before-merge to multi-query-attention June 20, 2023 20:12
balasaajay and others added 27 commits June 18, 2026 21:24
@balasaajay
Revert "Remove checkpoint-time GPU cache reclaim workaround (#5170)" (#…
be82829 
…5366)

Signed-off-by: Ajay Balasa <abalasa@nvidia.com>
@balasaajay
Update goldens for weekly tests after pytorch and TE bumps. (#5399)
a50252b 
Signed-off-by: Ajay Balasa <abalasa@nvidia.com>
@yashaswikarnati @claude
Add MIMO runtime setup: per-role RNG seeding and DDP wrapping (#5285)
d1410e1 
Signed-off-by: ykarnati <ykarnati@nvidia.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
@tdene @jalbericiola
Add --mamba-training-ssm-states-dtype argument (#5309)
6142ee4 
Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>
Co-authored-by: Jorge Albericio <jalbericiola@nvidia.com>
@github-actions
chore(beep boop 🤖): Bump (main) (2026-06-22)
2cb1b80
@lmcafee-nvidia
Fix Mamba prefix match for chunked prefill (#4758)
57c484e 
Signed-off-by: Lawrence McAfee <lmcafee@nvidia.com>
@wdykas @claude
Disag MR2: Refit into multiple destination pools and tied-embedding +…
ae2efd5 
… UVM fixes (#5187)

Signed-off-by: wdykas <wdykas@nvidia.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
@wdykas @claude
Disag MR1: Add inference shard specs and pg-collection building (#5186)
fc4597c 
Signed-off-by: wdykas <wdykas@nvidia.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
@yashaswikarnati @claude
Support the MIMO cross-grid path in training loop (#5373)
e1c4495 
Signed-off-by: ykarnati <ykarnati@nvidia.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
@shanmugamr1992
Stabilize hybrid_2b GB200 perf test against run-to-run noise (#5364)
6bd392f
@Phlip79
Consistent oncall schedule (#5404)
b6b44a7 
Signed-off-by: Philip Petrakian <ppetrakian@nvidia.com>
@wdykas
Disag MR3: Add heterogeneous KV/Mamba reshard planners (#5188)
93a7642 
Signed-off-by: wdykas <wdykas@nvidia.com>
@yashaswikarnati @claude
Add RADIO vision encoder wrapper for MIMO example (#5397)
2a46893 
Signed-off-by: ykarnati <ykarnati@nvidia.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
@santhnm2
Clean up MTP inference control flow (#5418)
76f6ccc 
Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>
@yashaswikarnati @claude
Add MIMO dual gradient finalization (colocated + non-colocated) (#5286)
f8170b4 
Signed-off-by: ykarnati <ykarnati@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@lauradang
Add RL rollout submission and consumption granularity controls (#5306)
a58373f 
Signed-off-by: Laura Dang <laurad@nvidia.com>
@ko3n1g @claude
Add --functional-test-name to trigger_internal_ci (#5449)
f66c28f 
Signed-off-by: oliver könig <okoenig@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@deepakn94 @claude
Rename CP batch helpers to describe balancing granularity (#5403)
8fa1831 
Signed-off-by: Deepak Narayanan <dnarayanan@nvidia.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
@ko3n1g @claude
build: point flash_mla at the nv_dev branch (#5448)
06ae6a9 
Signed-off-by: oliver könig <okoenig@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@tdene
Add logprobs_mode (raw/processed) to inference config (#5419)
a2bb5e5 
Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>
@cuichenx @shanmugamr1992 @claude
Support SWA and sink attention in dynamic inference (#5249)
fcbb6ed 
Signed-off-by: shanmugamr1992 <shanmugamr1992@gmail.com>
Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com>
Co-authored-by: shanmugamr1992 <shanmugamr1992@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com>
@yashaswikarnati @claude
Add hetero grid args and MoE process groups for MIMO example (#5375)
b1884d1 
Signed-off-by: ykarnati <ykarnati@nvidia.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
@chtruong814
ci: Set test_save_verify_integrity_manifest_directly as flaky (#5468)
b549290 
Signed-off-by: Charlie Truong <chtruong@nvidia.com>
@wujingyue
Remove DBuffer mesh axis validation (#5441)
47cb413 
Signed-off-by: Jingyue Wu <wujingyue@gmail.com>
@shanmugamr1992
feat(inference): default use_coordinator to True in high-level APIs (#…
a27b040 
…5326)
@Phlip79
Support HybridModel feature specs in ModelOpt (#5354)
811bd29 
Signed-off-by: Philip Petrakian <ppetrakian@nvidia.com>
@wujingyue
Add experimental Megatron-FSDP fully_shard implementation (#5387)
e7af860 
Signed-off-by: Jingyue Wu <wujingyue@gmail.com>
balasaajay and others added 30 commits July 25, 2026 00:24
@balasaajay
fix: add additional error checks for flaky failures (#6029)
21fe0fe 
Signed-off-by: Ajay Balasa <abalasa@nvidia.com>
@github-actions
Update copy-pr-bot.yaml [skip ci]
7ee2580
@santhnm2 @sidsingh-nvidia
Prevent FlashInfer sampling from running with CUDA graphs (#5791)
650b783 
Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>
Signed-off-by: Siddharth Singh <sidsingh@nvidia.com>
Co-authored-by: Siddharth Singh <sidsingh@nvidia.com>
@svcnemo-autobot
fix(inference): AUT-980 disable fp8 primary weights in graph tests (#…
10c9383 
…6038)

Signed-off-by: svcnemo-autobot <svcnemo-autobot@nvidia.com>
@svcnemo-autobot @balasaajay
chore(deps): AUT-967 stabilize Transformer Engine 2.18 upgrade (#5997)
7e6a045 
Signed-off-by: Ajay Balasa <abalasa@nvidia.com>
Signed-off-by: svcnemo-autobot <svcnemo-autobot@nvidia.com>
Co-authored-by: Ajay Balasa <abalasa@nvidia.com>
@fanshiqing @jiemingz
@deepakn94 @claude @JF-D
[feat] Generalized Tensor Parallelism (GTP) (#4967)
c5ff22b 
Signed-off-by: Shiqing Fan <shiqingf@nvidia.com>
Signed-off-by: Deepak Narayanan <deepakn94@gmail.com>
Signed-off-by: Jiangfei Duan <jiangfeid@nvidia.com>
Co-authored-by: Jieming Zhang <jiemingz@nvidia.com>
Co-authored-by: Deepak Narayanan <deepakn94@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Jiangfei Duan <jiangfeid@nvidia.com>
@philipcmonk
Fix gradient-norm undercounting when using EP and TP (#5916)
cd4afff 
Signed-off-by: Philip Monk <pmonk@nvidia.com>
@svcnemo-autobot
fix(inference): MCORE-536 report dropped prompt token lengths (#6051)
7ec1bce 
Signed-off-by: svcnemo-autobot <svcnemo-autobot@nvidia.com>
@fanshiqing
[GTP][Feat] Add one-block-ahead prefetch for GTP grouped-expert weigh…
543b803 
…ts (#6057)

Signed-off-by: Shiqing Fan <shiqingf@nvidia.com>
@svcnemo-autobot
chore(codeowners): AUT-1094 add GTP owners (#6062)
076f61f 
Signed-off-by: svcnemo-autobot <svcnemo-autobot@nvidia.com>
@Phlip79
Deprecate GPTModel in favor of HybridModel (#5911)
10cd30c 
Signed-off-by: Philip Petrakian <ppetrakian@nvidia.com>
@jiemingz
Fix CUDA graph correctness issues due to memory bugs (#5975)
4b18b26 
Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com>
@github-actions
Update copy-pr-bot.yaml [skip ci]
876016a
@svcnemo-autobot
fix(dist-ckpt): AUT-1100 restore default strategy factories (#6065)
fc57daa 
Signed-off-by: svcnemo-autobot <svcnemo-autobot@nvidia.com>
@xuantengh
refactor: extract and split common logic between GDN & GDN2 (#5843)
07e958f 
Signed-off-by: Xuanteng Huang <xuantengh@nvidia.com>
@guihong-nv
Add load-time GPT-to-Hybrid checkpoint translation (#5675) (#5792)
c4b4d59 
Signed-off-by: guihong-nv <guihongl@nvidia.com>
@asolergi-nv @claude
dist_ckpt: add --stream-ckpt-dequant to fix OOM on large FP8/MXFP8 lo…
866efa4 
…ads with --fp8-param-gather (#4451)

Signed-off-by: asolergi-nv <asolergibert@nvidia.com>
Signed-off-by: Antoni-Joan Solergibert <asolergibert@nvidia.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@ilml @claude
Populate dp process group in auto-built ProcessGroupCollection in pip…
6f7bcd4 
…eline schedules (#5901)

Signed-off-by: ilml <tolong@nvidia.com>
Signed-off-by: Tom Long <tolong@nvidia.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
@svcnemo-autobot
fix(cuda-graphs): MB-928 align DDP initialization with capture stream (…
bacd340 
…#6021)

Signed-off-by: svcnemo-autobot <svcnemo-autobot@nvidia.com>
@deepakn94
Enforce that the number of optimizer shards used in layout computatio…
8a424b8 
…n is the same used during the training iteration (#6048)

Signed-off-by: Deepak Narayanan <dnarayanan@nvidia.com>
@YangFei1990 @claude @yaox12
NCCL EP zero copy (#5735)
4da9212 
Signed-off-by: YangFei1990 <feiw@nvidia.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Xin Yao <xiny@nvidia.com>
@sraman-rgb
Optimize unit metadata for fused shared experts (#6053)
7557c02 
Signed-off-by: Siddhartha Raman Sundara Raman <sraman@nvidia.com>
@svcnemo-autobot @kunlunl
build: AUT-1117 serialize uv dependency installation (#6090)
053fa07 
Signed-off-by: kunlunl <kunlunl@nvidia.com>
Signed-off-by: svcnemo-autobot <svcnemo-autobot@nvidia.com>
Co-authored-by: kunlunl <kunlunl@nvidia.com>
@lmcafee-nvidia
Extend dynamic inference asynchronous scheduling support (#5939)
541d5ee 
Signed-off-by: Lawrence McAfee <lmcafee@nvidia.com>
@tdene @jalbericiola @lauradang
Add support for non-Gym multi-turn environments (#5312)
c922805 
Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>
Co-authored-by: Jorge Albericio <jalbericiola@nvidia.com>
Co-authored-by: Laura Dang <laurad@nvidia.com>
@tdene
Correct prefix-caching ref-count accounting (#6047)
2576c15 
Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>
@wdykas
send pg group for distributed checkpoint validation (#6092)
818b258 
Signed-off-by: William Dykas <wdykas@nvidia.com>
@svcnemo-autobot
fix(ci): AUT-1080 retry live NCCL watchdog timeouts (#6054)
2658704 
Signed-off-by: svcnemo-autobot <svcnemo-autobot@nvidia.com>
@tdene @jalbericiola
Prevent coordinator crash if an engine disconnects (#6025)
157c023 
Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>
Co-authored-by: Jorge Albericio <jalbericiola@nvidia.com>
@github-actions
Update copy-pr-bot.yaml [skip ci]
bc67abd
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.