Source Reading 010 · ROCm/AITER

AITER CI Kernel Surface

A source-level map of what ROCm/AITER actually runs in CI: which workflows are active, which kernel families are exercised, and which shapes deserve attention when we mirror their coverage.

这是一张 ROCm/AITER 当前 CI 的源码级地图： 哪些 workflow 还在主线， 哪些 kernel family 会被跑到， 以及我们对标时最该盯住哪些 shape。

RepoROCm/aiter
Snapshotmain @ ff61bb39977907f5e2071894ab20bf6b2a2a9e77
Categorycode · source reading
Date2026-06-11

The Short Version先给结论

AITER's main CI is not one benchmark workflow. It is a layered test surface: top-level operator scripts, Triton pytest shards, multi-GPU collectives, FlashAttention integration, and end-to-end vLLM / SGLang / ATOM gates.

AITER 主线 CI 不是一个 benchmark workflow。 它是一层层测试面： top-level operator 脚本、Triton pytest shard、multi-GPU collective、FlashAttention 集成， 以及 vLLM / SGLang / ATOM 下游门禁。

77top-level AITER operator test filestop-level AITER operator 测试文件
99Triton pytest filesTriton pytest 文件
19multi-GPU test filesmulti-GPU 测试文件
Reading rule

For top-level AITER tests, CI executes python3 file.py through .github/scripts/aiter_test.sh. For Triton tests, CI executes pytest shards. Treat CLI defaults and explicit shell invocations as the hard CI contract; pytest-only parametrization is a broader local test surface unless the workflow calls pytest.

top-level AITER 测试在 CI 里通过 .github/scripts/aiter_test.sh 执行 python3 file.py。 Triton 测试则跑 pytest shard。 所以判断“CI 真跑什么”时， CLI 默认参数和 shell 里的显式调用是硬合同；pytest-only 参数矩阵只有在 workflow 调 pytest 时才算主线覆盖。

The Example Is Historical你给的例子是历史痕迹

aiter-benchmark.yaml is visible in GitHub Actions history, but it is not present in current main. The file came from closed, unmerged PR #2115 — its benchmark_release branch targeting main, closed 2026-03-18 without merging. It delegated to ATOM's benchmark workflow with deepseek-r1-0528=true, --kv_cache_dtype fp8 -tp 8, and param_list="1024,1024,128,0.8".

aiter-benchmark.yaml 在 GitHub Actions 历史里还能看到， 但当前 main 没有这个文件。 它来自已关闭、未合入的 PR #2115： 源分支 benchmark_release 指向 main， 2026-03-18 关闭未合。 那个 workflow 只是委托 ATOM benchmark， 参数是 deepseek-r1-0528=true、--kv_cache_dtype fp8 -tp 8、param_list="1024,1024,128,0.8"。

Plate IActions history versus main
Historical benchmark workflow A diagram showing that aiter-benchmark lives in Actions history, not current main, and delegates to ATOM. Actions history Aiter Benchmark aiter-benchmark.yaml Current main .github/workflows file absent PR #2115 closed / unmerged delegates to ATOM shape-like payload: 1024,1024,128,0.8 + fp8 KV + TP8

The benchmark file is useful evidence, but not a current mainline gate.

这个 benchmark 文件可以当历史证据看， 但不能算当前主线门禁。

The Active CI Map当前主线 CI 地图

The active surface splits cleanly into three levels: kernel correctness, integration correctness, and downstream model gates. The source path matters because each level answers a different risk question.

当前活跃测试面可以分成三层： kernel correctness、integration correctness、downstream model gate。 这些路径回答的是不同风险问题， 不能混成一个“benchmark 是否通过”。

Plate IIWorkflow stack
AITER workflow stack A three-layer map from operator tests to integration tests and downstream gates. Kernel layer aiter-test.yaml triton-test.yaml split into 8 shards Integration layer FlashAttention OPUS wheel smoke Model layer vLLM SGLang ATOM / Kimi 77 AITER python3 file.py 99 Triton pytest -v 19 Multi-GPU main-only pool TP8 models latency / gsm8k

AITER's CI is not flat: a failure in a kernel shard and a failure in Kimi accuracy mean different things.

AITER 的 CI 不是一张平表： kernel shard 挂掉和 Kimi accuracy 挂掉， 含义完全不同。

Workflow What it covers Trigger shape
aiter-test.yaml Core top-level operator tests, MI35X + MI300X, 8 shards; main also runs multi-GPU tests and publishes the tuned-op baseline (the regression compare runs on PRs).核心 top-level operator tests， MI35X + MI300X， 8 shard；main 还跑 multi-GPU tests 并发布 tuned-op 基线（回归对比在 PR 上跑）。 PR / main / daily / manual
triton-test.yaml Triton tests under op_tests/triton_tests, pytest shards, MI35X by default and MI300X when requested.op_tests/triton_tests 下的 Triton tests， pytest shard， 默认 MI35X， 需要时跑 MI300X。 Triton path PR / main / manual
flash_attention_integration.yaml Dao-AILab FlashAttention integration with representative PR filters.Dao-AILab FlashAttention 集成， PR 上只跑代表性过滤条件。 2048-2048, 1024-1024, 16-20000
vllm_benchmark.yaml 8-GPU vLLM latency for GPT-OSS-120B, DeepSeek-R1-0528, and Kimi-K2.5.8-GPU vLLM latency， 覆盖 GPT-OSS-120B、DeepSeek-R1-0528、Kimi-K2.5。 batch=123 input=456 output=78 tp=8
kimi-*, sglang_*, atom-test Downstream accuracy and throughput gates. These are model-system tests, not just kernel unit tests.下游 accuracy 和 throughput 门禁。 这些是 model-system tests， 不只是 kernel unit tests。 labels / schedule / main

Refs: aiter_test.sh:14-24, aiter_test.sh:73-78, triton-test.yaml:208-210.

引用：aiter_test.sh:14-24、aiter_test.sh:73-78、triton-test.yaml:208-210。

Kernel SurfaceKernel 覆盖面

The weighted shard list is the best quick proxy for what AITER treats as expensive and important. The heaviest top-level files are fused QK norm + cache quant, MLA, batch prefill, MHA, PA, rope, and top-k.

weighted shard list 是判断 AITER 认为哪些测试“又贵又重要”的最快入口。 top-level 最重的是 fused QK norm + cache quant、MLA、batch prefill、MHA、PA、rope、top-k。

GEMM a16w16 GEMM a8w8 GEMM blockscale Batched GEMM MoE MoE 2-stage MLA / persistent / sparse MHA / varlen Paged Attention Batch Prefill RoPE TopK KV cache Quant RMSNorm / LayerNorm Multi-GPU collectives
Plate IIIWeighted kernel attention
Kernel test weight bars A bar chart of representative long-running AITER and Triton tests. AITER top-level weights fused qk norm mrope cache quant MLA MLA persistent batch prefill MHA / PA / varlen rope + topk Triton weights pa decode gluon causal conv1d attention/mha_v3

The shard weights are not correctness thresholds, but they expose where CI time is spent.

shard weight 不是 correctness 阈值， 但它暴露了 CI 时间主要花在哪里。

Refs: split_tests.sh:68-87, split_tests.sh:147-169, split_tests.sh:274-311.

引用：split_tests.sh:68-87、split_tests.sh:147-169、split_tests.sh:274-311。

Shape Pressure PointsShape 压力点

AITER's shapes are not random unit-test shapes. They cluster around decode-side skinny GEMM, large-model projection GEMM, MoE token extremes, long-context MLA, and attention boundary lengths.

AITER 的 shape 不是随便挑的 unit-test shape。 它们集中在 decode-side skinny GEMM、大模型 projection GEMM、MoE token 极值、long-context MLA、以及 attention 边界长度。

Kernel Shapes to notice
GEMM a8w8 M=[1..16384] over projection forms such as N=1280,K=8192 and N=8192,K=1024; hipmm preshuffle includes 7424 x 8192 cases.projection 形态覆盖 M=[1..16384]， 例如 N=1280,K=8192 和 N=8192,K=1024；hipmm preshuffle 还有 7424 x 8192 cases。
GEMM blockscale M=1..10240, default N=24576,K=1536, with ck_preshuffle true and false.M=1..10240， 默认 N=24576,K=1536， 并且 ck_preshuffle true / false 都跑。
MLA Default contexts 21,64,256,512,1200,3200,5200,8192; CI adds 49152, 98304, 10000/100000, and boundary 1,21,63,64,65,256.默认 context 是 21,64,256,512,1200,3200,5200,8192；CI 额外加 49152、98304、10000/100000， 以及边界 1,21,63,64,65,256。
MoE moe_2stage uses hidden=7168, inter=256, expert=257, topk=9, and tokens up to 163840. Blockscale uses expert=256, topk=8.moe_2stage 使用 hidden=7168、inter=256、expert=257、topk=9， tokens 到 163840。 blockscale 使用 expert=256、topk=8。
PA / TopK Paged attention contexts include 7,26,57,66,109,128,257,282,4097; top-k per row contexts go to 128000, with top_k=512/1024/2048.Paged attention context 包括 7,26,57,66,109,128,257,282,4097；top-k per row context 到 128000，top_k=512/1024/2048。
Plate IVShape pressure lanes
Shape pressure lanes Four lanes showing the shape ranges emphasized by AITER tests. GEMM MLA MoE TopK M=1 M=32 M=1024 M=16384 49k / 98k / 100k ctx 163840 tokens context 128000, k 2048

The important shape fact is not only maximum size; it is the mix of tiny decode cases, model-like projections, and long-context tails.

重要的不只是最大 shape， 而是 tiny decode case、model-like projection 和 long-context tail 混在一起。

Refs: test_gemm_a8w8.py:505-546, test_gemm_a8w8_blockscale.py:233-283, test_mla.py:756-764, aiter_test.sh:106-114.

引用：test_gemm_a8w8.py:505-546、test_gemm_a8w8_blockscale.py:233-283、test_mla.py:756-764、aiter_test.sh:106-114。

Downstream Gates Are Part Of The Surface下游门禁也是测试面的一部分

The model gates show what AITER is optimizing for at system level. vLLM uses fixed latency inputs; Kimi performance sweeps concurrency; Kimi accuracy gates GSM8K; ATOM carries frontier-model smoke. These tests do not identify a single bad kernel, but they catch integration drift that unit tests can miss.

model gate 暴露的是 AITER 在系统层面想守住什么。 vLLM 用固定 latency input；Kimi performance sweep concurrency；Kimi accuracy gate GSM8K；ATOM 承担 frontier-model smoke。 这些测试不一定能定位某个坏 kernel， 但能抓住 unit test 容易漏掉的集成漂移。

Gate Model / shape Signal
vLLM benchmark GPT-OSS-120B fp8 KV, DeepSeek-R1-0528, Kimi-K2.5; batch=123,input=456,output=78,tp=8 Latency smoke under 8-GPU serving.8-GPU serving 下的 latency smoke。
Kimi perf ISL=1024, OSL=1024, concurrency 4/8/16/32/64 Throughput floor at high concurrency.高 concurrency 下的吞吐下限。
Kimi accuracy Kimi-K2.5 MXFP4, GSM8K 3-shot Flexible-extract threshold 0.92.flexible-extract 阈值 0.92。
SGLang / ATOM DeepSeek-R1-MXFP4, Qwen 3.5, GPT-OSS-120B, Kimi-K2.5 Backend-specific integration and accuracy smoke.backend-specific 集成和 accuracy smoke。

Refs: vllm_benchmark.yaml:144-159, vllm_benchmark.yaml:261, kimi_vllm_perf.sh:10-15, kimi-downstream.yaml:73-74.

引用：vllm_benchmark.yaml:144-159、vllm_benchmark.yaml:261、kimi_vllm_perf.sh:10-15、kimi-downstream.yaml:73-74。

What We Should Mirror First我们优先对标什么

For FlyDSL or any AMD kernel stack trying to compare against AITER, the practical coverage target is not "run every file." The target is to match the pressure points that AITER protects.

如果 FlyDSL 或其他 AMD kernel stack 要对标 AITER， 实际目标不是“把每个文件都跑一遍”。 目标是先覆盖 AITER 真正保护的压力点。

01 GEMM

Cover skinny decode GEMM around M=1..16, then projection-like M=1..16k with model N/K pairs.

先覆盖 M=1..16 的 skinny decode GEMM， 再覆盖 M=1..16k 的 projection-like model N/K pairs。

02 MoE

Include token extremes up to 163840, top-k routing, blockscale variants, and the large-model hidden dimension 7168.

包括 tokens 到 163840 的极端值、top-k routing、blockscale variants， 以及大模型常见的 hidden=7168。

03 Attention

Keep MLA long contexts 49k/98k/100k, boundary contexts around powers of two, and PA's small odd contexts.

保留 MLA long context 49k/98k/100k、power-of-two 附近的边界 context， 以及 PA 的小型奇数 context。

04 System

Use one TP8 downstream smoke for a latency path and one accuracy path; otherwise kernel-only CI can look healthy while serving integration drifts.

至少保留一个 TP8 downstream latency smoke 和一个 accuracy path；否则 kernel-only CI 可能看起来健康， 但 serving integration 已经漂移。

Source References源码引用

All source links below are pinned to the audited commit.

下面的源码链接都固定到本次审计使用的 commit。