Skip to content

[ASM] [HIP] [CK] feat(mha): gfx950 hd256 FP8 LINEAR paged-varlen asm prefill - #4971

Open
MiloLurati wants to merge 4 commits into
mainfrom
milurati/fmha_batch_prefill_hd256_fp8_paged
Open

[ASM] [HIP] [CK] feat(mha): gfx950 hd256 FP8 LINEAR paged-varlen asm prefill#4971
MiloLurati wants to merge 4 commits into
mainfrom
milurati/fmha_batch_prefill_hd256_fp8_paged

Conversation

@MiloLurati

@MiloLurati MiloLurati commented Aug 24, 2026

Copy link
Copy Markdown

Motivation

SGLang chunked prefill uses a paged KV cache (page size 64, LINEAR [N, 64, H, D]). The gfx950 hd256 FP8 asm prefill kernel only accepted contiguous KV, so that path fell back to CK. This PR selects a PAGED_VARLEN asm kernel for that layout.

Technical Details

  • op_tests/test_batch_prefill.py: test_batch_prefill_hd256_fp8_page64_asm (LINEAR, causal/non-causal, including one randomized Q/KV length case) and test_batch_prefill_hd256_fp8_page16_stays_ck.
  • --pagesize accepts 64.

Test Plan

  • op_tests/test_batch_prefill.py: test_batch_prefill_hd256_fp8_page64_asm (gfx950-only skip; LINEAR; causal/non-causal; equal lengths, Q=128/KV=1024, randomized Q/KV with KV ≥ Q, GQA 16:2).
  • --pagesize choices include 64 (-p 64 --kv_layout linear for a CLI bench). 64 is not in the default [1, 16, 1024] sweep

Test Result

  • page64 asm: 8/8
  • CK smoke (LINEAR fp8 hd256 ps=16): 3/3 (SGLang non-causal, SGLang causal+softcap, vLLM)

Submission Checklist

Select the PAGED_VARLEN asm kernel for gfx950 FP8 hd256 page_size=64, then fall back to CK.
@MiloLurati MiloLurati self-assigned this Aug 24, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🏷️ CI Guide

Runs automatically on every PR:

  • ✅ Pre-checks (submodule verification, code formatting)
  • ✅ Aiter op tests (gfx942 + gfx950)
  • ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label Tests
ci:gfx1250-ffm-triton Run the five-shard gfx1250 FFM Triton test suite
ci:triton-300x Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
ci:sglang SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 4971 --add-label <label>

PR title tags:
Component tags ([Triton/Gluon], [HIP], [CK], [ASM], ...) are added to the PR title automatically from the changed files and re-synced on every push — change-type tags like [fix]/[Perf] and op tags like [MLA] are left untouched. Add the no-auto-title label to opt this PR out of title tagging.

MiloLurati and others added 3 commits August 25, 2026 09:07
Skip page64 asm off gfx950, drop the redundant page16 case, clamp empty-page seqlen_k, and use the file's FP8 threshold.
Fold the -1 ladder into one compound if like fmha_fwd_v3, and restore the CK kUseGlobalLoad comment.
@MiloLurati
MiloLurati marked this pull request as ready for review August 25, 2026 11:33
@MiloLurati
MiloLurati requested a review from a team August 25, 2026 11:33
@github-actions github-actions Bot changed the title feat(mha): gfx950 hd256 FP8 LINEAR paged-varlen asm prefill [ASM] [HIP] [CK] feat(mha): gfx950 hd256 FP8 LINEAR paged-varlen asm prefill Aug 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant