Skip to content

Add stream-dse fused SwiGLU-prefill operator#122

Open
asyms wants to merge 5 commits into
amd:develfrom
KULeuven-MICAS:stream-dse-fused-swiglu
Open

Add stream-dse fused SwiGLU-prefill operator#122
asyms wants to merge 5 commits into
amd:develfrom
KULeuven-MICAS:stream-dse-fused-swiglu

Conversation

@asyms

@asyms asyms commented Jun 18, 2026 •
edited
Loading

Copy link
Copy Markdown

Adds SwiGLUPrefillStream, a fused SwiGLU-prefill operator whose single MLIR design (gate/up GEMMs + SiLU + elementwise-mul + down GEMM) is generated by stream-dse and compiled to one xclbin, instead of chaining separately-compiled sub-operators.

Its per-kernel operand layouts (the tiled-strided DMA tiling) are authored on the IRON side and injected into stream-dse code generation via optimize_allocation_co(kernels=...) — so IRON owns the layouts while stream keeps kernel construction and the MLIR rewrite, instead of the layouts being hand-copied on both sides.

Added

  • SwiGLUPrefillStream (iron/operators/swiglu_prefill_stream/): fused stream-dse design → one xclbin; MLIR generated at build time by stream_design.py.
  • iron.common.layout: a TiledStridedLayout type (with to_snaxc()) for handing IRON-authored operand layouts to stream-dse.
  • stream_kernels.py: injects IRON's operand layouts into codegen through the kernels= override, replacing only operand_layouts() on stream's own kernels (requires stream-dse ≥ 1.13.4).
  • requirements_stream.txt (optional dependency stream-dse>=1.13.4); the operator's test skips when stream-dse is absent.
  • Minimal demo under demos/swiglu_prefill_stream/.

Changed

  • Importing iron.operators no longer requires an NPU runtime: lazy XRT/pyxrt import and PEP 562 lazy operator exports, so the package loads (and tests collect) on hosts without XRT/pyxrt.

Removed

  • None.

Running the test

Prerequisites: the XDNA driver + XRT installed (/opt/xilinx/xrt) and an npu2 device. From a fresh clone of this branch:

python3 -m venv .venv && source .venv/bin/activate
source /opt/xilinx/xrt/setup.sh            # provides pyxrt
pip install --upgrade pip
pip install -r requirements.txt            # IRON + mlir_aie/llvm-aie toolchain + torch
pip install -r requirements_stream.txt     # stream-dse>=1.13.4 (PyPI)
stream-setup-aie                           # required: installs snaxc / xdsl-aie / aie-python-extras
python iron/operators/swiglu_prefill_stream/test.py

This generates the fused design with stream-dse, compiles it to an xclbin, and runs it once on the NPU (≈2 ms for the 256×512×2048 shape). stream-setup-aie is required: it installs the AIE codegen packages stream-dse needs that cannot be plain PyPI dependencies.

Licensing note

The new IRON-side files — iron/common/layout.py, iron/operators/swiglu_prefill_stream/stream_kernels.py, and demos/swiglu_prefill_stream/demo.py — carry a KU Leuven (MICAS) copyright header (Apache-2.0), as they were authored by MICAS; all other touched files keep their existing AMD headers. We can discuss this further.

PR Merge Checklist

  1. The PR is rebased on the latest devel commit and pointing to devel.
  2. Your PR has been reviewed and approved.
  3. All checks are passing.

@andrej

andrej commented Jun 22, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Hi Arne, sorry for the CI failures, if you rebase on #125 maybe once it's merged hopefully these should pass

@asyms
asyms force-pushed the stream-dse-fused-swiglu branch from 7f54de3 to 5103ce5 Compare June 22, 2026 19:55
@github-actions

github-actions Bot commented Jun 23, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

CI Test Results

a1ee5e5 (2026_07_06_21_01_36)

IRON - CI Summary

Examples

iron/applications/llama_3.2_1b
Test Krackan Status Krackan Phoenix Status Phoenix
test_llama_3_2_1b[llama_3.2_1b_prompt_1024_tokens_1] ✅ - - -
test_llama_3_2_1b[llama_3.2_1b_prompt_1024_tokens_40] ✅ - - -
test_llama_3_2_1b[llama_3.2_1b_prompt_13_tokens_1] ✅ - - -
test_llama_3_2_1b[llama_3.2_1b_prompt_13_tokens_40] ✅ - - -

Small

iron/operators/axpy
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_axpy[input_length_2048-num_aie_columns_1-tile_size_2048-scalar_factor_3.0] ✅ 175.40 ✅ 411.60
test_axpy[input_length_2048-num_aie_columns_2-tile_size_1024-scalar_factor_3.0] ✅ 135.00 ✅ 348.30
test_axpy[input_length_2048-num_aie_columns_4-tile_size_512-scalar_factor_3.0] ✅ 140.24 ✅ 431.16
test_axpy[input_length_2048-num_aie_columns_8-tile_size_256-scalar_factor_3.0] ✅ 168.80 - -
iron/operators/dequant
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_dequant[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-group_size_32] ✅ 149.62 ✅ 294.58
test_dequant[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-group_size_32] ✅ 152.64 ✅ 269.60
test_dequant[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-group_size_32] ✅ 159.12 ✅ 268.38
test_dequant[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-group_size_32] ✅ 165.34 ✅ 266.16
test_dequant[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-group_size_32] ✅ 161.04 ✅ 291.04
test_dequant[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-group_size_32] ✅ 167.96 ✅ 262.18
test_dequant[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256-group_size_32] ✅ 163.54 - -
test_dequant[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128-group_size_32] ✅ 195.84 - -
iron/operators/elementwise_add
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_elementwise_add[input_length_2048-num_aie_columns_1-tile_size_2048] ✅ 145.78 ✅ 286.74
test_elementwise_add[input_length_2048-num_aie_columns_2-tile_size_1024] ✅ 117.06 ✅ 270.76
test_elementwise_add[input_length_2048-num_aie_columns_4-tile_size_512] ✅ 106.98 ✅ 343.72
test_elementwise_add[input_length_2048-num_aie_columns_8-tile_size_256] ✅ 124.82 - -
iron/operators/elementwise_mul
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_elementwise_mul[input_length_2048-num_aie_columns_1-tile_size_2048] ✅ 154.26 ✅ 299.16
test_elementwise_mul[input_length_2048-num_aie_columns_2-tile_size_1024] ✅ 136.46 ✅ 348.70
test_elementwise_mul[input_length_2048-num_aie_columns_4-tile_size_512] ✅ 136.00 ✅ 313.10
test_elementwise_mul[input_length_2048-num_aie_columns_8-tile_size_256] ✅ 173.90 - -
iron/operators/gelu
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_gelu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048] ✅ 116.46 ✅ 383.10
test_gelu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024] ✅ 128.66 ✅ 322.50
test_gelu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024] ✅ 130.08 ✅ 424.28
test_gelu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512] ✅ 114.06 ✅ 443.18
test_gelu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512] ✅ 112.74 ✅ 386.08
test_gelu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256] ✅ 140.22 ✅ 464.66
test_gelu[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256] ✅ 140.64 - -
test_gelu[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128] ✅ 153.20 - -
iron/operators/gemm
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_gemm[M_1792-K_896-N_1152-num_aie_columns_8-b_col_maj_False-c_col_maj_True-m_64-k_32-n_48-trace_size_0-partition_N_1] ✅ 1930.36 - -
test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_False-c_col_maj_False-m_48-k_96-n_16-trace_size_0-partition_N_1] ✅ 187.60 ✅ 471.24
test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_True-c_col_maj_True-m_48-k_96-n_16-trace_size_0-partition_N_1] ✅ 157.44 ✅ 591.76
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_1-b_col_maj_False-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1] ✅ 48302.54 ✅ 84732.84
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_2-b_col_maj_True-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1] ✅ 27155.28 ✅ 26139.28
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_8-b_col_maj_True-c_col_maj_True-m_64-k_64-n_64-trace_size_0-partition_N_1] ✅ 7278.52 - -
test_gemm[M_384-K_1536-N_1792-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_32-k_48-n_64-trace_size_0-partition_N_1] ✅ 1924.18 ✅ 3271.46
test_gemm[M_64-K_512-N_256-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_16-k_64-n_64-trace_size_0-partition_N_4] ✅ 2349.14 ✅ 6468.38
test_gemm[M_896-K_1792-N_640-num_aie_columns_8-b_col_maj_False-c_col_maj_True-m_32-k_64-n_80-trace_size_0-partition_N_1] ✅ 1344.70 - -
iron/operators/gemv
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_gemv[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128] ✅ 0.21 ✅ 0.12
test_gemv[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048] ✅ 13.16 ✅ 3.57
test_gemv[M_2048-K_8192-num_aie_columns_2-tile_size_input_1-tile_size_output_1024] ✅ 23.47 ✅ 6.48
test_gemv[M_2048-K_8192-num_aie_columns_4-tile_size_input_1-tile_size_output_512] ✅ 32.17 ✅ 9.09
test_gemv[M_2048-K_8192-num_aie_columns_8-tile_size_input_1-tile_size_output_256] ✅ 36.69 - -
test_gemv[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024] ✅ 12.87 ✅ 3.49
test_gemv[M_8192-K_2048-num_aie_columns_2-tile_size_input_4-tile_size_output_1024] ✅ 22.18 ✅ 6.71
test_gemv[M_8192-K_2048-num_aie_columns_4-tile_size_input_4-tile_size_output_1024] ✅ 31.70 ✅ 10.90
test_gemv[M_8192-K_2048-num_aie_columns_8-tile_size_input_4-tile_size_output_1024] ✅ 35.76 - -
iron/operators/layer_norm
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048] ✅ 266.10 ✅ 423.72
test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024] ✅ 241.26 ✅ 342.70
test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024] ✅ 238.72 ✅ 455.30
test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512] ✅ 248.94 ✅ 476.38
test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512] ✅ 189.70 ✅ 412.78
test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256] ✅ 219.56 ✅ 541.90
test_layer_norm[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256] ✅ 259.28 - -
test_layer_norm[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128] ✅ 292.28 - -
iron/operators/mem_copy
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_mem_copy[input_length_2048-num_cores_1-num_channels_1-bypass_False-tile_size_2048] ✅ 3687.78 ✅ 406.68
test_mem_copy[input_length_2048-num_cores_16-num_channels_2-bypass_False-tile_size_128] ✅ 1368.94 - -
test_mem_copy[input_length_2048-num_cores_2-num_channels_1-bypass_False-tile_size_1024] ✅ 198.40 ✅ 327.86
test_mem_copy[input_length_2048-num_cores_2-num_channels_2-bypass_False-tile_size_1024] ✅ 941.50 ✅ 428.78
test_mem_copy[input_length_2048-num_cores_4-num_channels_1-bypass_False-tile_size_512] ✅ 2902.78 ✅ 375.46
test_mem_copy[input_length_2048-num_cores_4-num_channels_2-bypass_False-tile_size_512] ✅ 584.52 ✅ 431.08
test_mem_copy[input_length_2048-num_cores_8-num_channels_1-bypass_False-tile_size_256] ✅ 799.06 - -
test_mem_copy[input_length_2048-num_cores_8-num_channels_2-bypass_False-tile_size_256] ✅ 616.48 ✅ 557.76
iron/operators/mha
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_mha[seq_len_16384-dim_64-num_heads_1-num_pipelines_8-num_kv_heads_0] ✅ 40629.24 - -
iron/operators/relu
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048] ✅ 273.28 ✅ 349.56
test_relu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024] ✅ 284.00 ✅ 389.88
test_relu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024] ✅ 219.52 ✅ 362.12
test_relu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512] ✅ 260.24 ✅ 651.52
test_relu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512] ✅ 271.20 ✅ 756.40
test_relu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256] ✅ 248.40 ✅ 750.02
test_relu[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256] ✅ 220.64 - -
test_relu[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128] ✅ 238.10 - -
iron/operators/rms_norm
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_False] ✅ 2874.68 ✅ 418.88
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_True] ✅ 228.42 ✅ 314.06
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_False] ✅ 3227.02 ✅ 350.62
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_True] ✅ 228.86 ✅ 433.08
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_False] ✅ 3226.58 ✅ 479.84
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_True] ✅ 214.60 ✅ 364.18
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_False] ✅ 225.32 ✅ 483.18
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_True] ✅ 923.42 ✅ 386.16
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_False] ✅ 241.84 ✅ 452.54
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_True] ✅ 3677.14 ✅ 430.18
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-weighted_False] ✅ 230.60 ✅ 417.42
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-weighted_True] ✅ 907.12 - -
test_rms_norm[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256-weighted_False] ✅ 201.74 - -
test_rms_norm[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256-weighted_True] ✅ 1355.62 - -
test_rms_norm[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128-weighted_False] ✅ 283.72 - -
iron/operators/rope
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_1-method_type_0] ✅ 3347.92 ✅ 512.28
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_2-method_type_0] ✅ 3495.82 ✅ 433.82
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_4-method_type_0] ✅ 3461.76 ✅ 445.22
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_8-method_type_0] ✅ 2085.48 - -
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_1-method_type_0] ✅ 3534.36 ✅ 365.46
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_2-method_type_0] ✅ 3521.26 ✅ 411.52
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_4-method_type_0] ✅ 3434.38 ✅ 596.70
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_8-method_type_0] ✅ 3446.76 - -
iron/operators/sigmoid
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_sigmoid[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048] ✅ 953.60 ✅ 316.18
test_sigmoid[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024] ✅ 3316.92 ✅ 397.44
test_sigmoid[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024] ✅ 10781.78 ✅ 391.56
test_sigmoid[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512] ✅ 722.30 ✅ 378.26
test_sigmoid[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512] ✅ 814.38 ✅ 435.14
test_sigmoid[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256] ✅ 453.44 ✅ 472.56
test_sigmoid[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256] ✅ 647.08 - -
test_sigmoid[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128] ✅ 1012.18 - -
iron/operators/silu
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_silu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048] ✅ 820.38 ✅ 323.24
test_silu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024] ✅ 462.44 ✅ 332.28
test_silu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512] ✅ 504.36 ✅ 494.26
test_silu[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256] ✅ 634.46 - -
iron/operators/softmax
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_1024] ✅ 290.74 ✅ 523.32
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_2048] ✅ 261.74 ✅ 432.56
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_512] ✅ 317.68 ✅ 454.12
iron/operators/swiglu_decode
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_swiglu_decode[embedding_dim_1024-hidden_dim_3584] ✅ 11119.84 ✅ 13160.98
test_swiglu_decode[embedding_dim_2048-hidden_dim_2048] ✅ 8886.98 ✅ 11547.31
iron/operators/swiglu_prefill
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_swiglu_prefill[seq_len_256-embedding_dim_2048-hidden_dim_2048-prio_accuracy_False] ✅ 8482.23 ✅ 23924.00
iron/operators/swiglu_prefill_stream
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_swiglu_prefill_stream[seq_len_256-embedding_dim_512-hidden_dim_2048-seq_tile_32-embedding_tile_32-hidden_tile_64] ❌ - - -
test_swiglu_prefill_stream_k2[seq_len_256-embedding_dim_512-hidden_dim_2048-seq_tile_32-embedding_tile_32-hidden_tile_64] ❌ 3010.20 - -
iron/operators/tanh
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_tanh[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048] ✅ 186.22 ✅ 392.62
test_tanh[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024] ✅ 168.40 ✅ 440.66
test_tanh[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024] ✅ 207.36 ✅ 444.40
test_tanh[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512] ✅ 189.48 ✅ 414.82
test_tanh[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512] ✅ 197.58 ✅ 512.64
test_tanh[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256] ✅ 197.84 ✅ 424.80
test_tanh[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256] ✅ 209.72 - -
test_tanh[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128] ✅ 223.08 - -
iron/operators/transpose
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_1] ✅ 198.80 ✅ 428.68
test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_2] ✅ 218.70 ✅ 529.20
test_transpose[M_2048-N_64-aie_columns_1-channels_2-m_64-n_64-s_8-num_batches_1] ✅ 200.72 ✅ 486.34
Krackan - Small

IRON

Tested on 2026_07_06_21_01_36 at commit a1ee5e5.

iron/operators/axpy
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_axpy[input_length_2048-num_aie_columns_1-tile_size_2048-scalar_factor_3.0]✅ 5/5175.400.07n/a
test_axpy[input_length_2048-num_aie_columns_2-tile_size_1024-scalar_factor_3.0]✅ 5/5135.000.09n/a
test_axpy[input_length_2048-num_aie_columns_4-tile_size_512-scalar_factor_3.0]✅ 5/5140.240.09n/a
test_axpy[input_length_2048-num_aie_columns_8-tile_size_256-scalar_factor_3.0]✅ 5/5168.800.08n/a
iron/operators/dequant
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_dequant[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-group_size_32]✅ 5/5149.620.04n/a
test_dequant[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-group_size_32]✅ 5/5152.640.04n/a
test_dequant[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-group_size_32]✅ 5/5159.120.03n/a
test_dequant[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-group_size_32]✅ 5/5165.340.04n/a
test_dequant[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-group_size_32]✅ 5/5161.040.04n/a
test_dequant[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-group_size_32]✅ 5/5167.960.03n/a
test_dequant[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256-group_size_32]✅ 5/5163.540.03n/a
test_dequant[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128-group_size_32]✅ 5/5195.840.03n/a
iron/operators/elementwise_add
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_elementwise_add[input_length_2048-num_aie_columns_1-tile_size_2048]✅ 5/5145.780.09n/a
test_elementwise_add[input_length_2048-num_aie_columns_2-tile_size_1024]✅ 5/5117.060.11n/a
test_elementwise_add[input_length_2048-num_aie_columns_4-tile_size_512]✅ 5/5106.980.12n/a
test_elementwise_add[input_length_2048-num_aie_columns_8-tile_size_256]✅ 5/5124.820.10n/a
iron/operators/elementwise_mul
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_elementwise_mul[input_length_2048-num_aie_columns_1-tile_size_2048]✅ 5/5154.260.08n/a
test_elementwise_mul[input_length_2048-num_aie_columns_2-tile_size_1024]✅ 5/5136.460.10n/a
test_elementwise_mul[input_length_2048-num_aie_columns_4-tile_size_512]✅ 5/5136.000.10n/a
test_elementwise_mul[input_length_2048-num_aie_columns_8-tile_size_256]✅ 5/5173.900.08n/a
iron/operators/gelu
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_gelu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5116.460.07n/a
test_gelu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5128.660.07n/a
test_gelu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5130.080.07n/a
test_gelu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5114.060.08n/a
test_gelu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5112.740.08n/a
test_gelu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5140.220.06n/a
test_gelu[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256]✅ 5/5140.640.06n/a
test_gelu[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128]✅ 5/5153.200.06n/a
iron/operators/gemm
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_gemm[M_1792-K_896-N_1152-num_aie_columns_8-b_col_maj_False-c_col_maj_True-m_64-k_32-n_48-trace_size_0-partition_N_1]✅ 5/51930.364.871917.43
test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_False-c_col_maj_False-m_48-k_96-n_16-trace_size_0-partition_N_1]✅ 5/5187.601.2553.20
test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_True-c_col_maj_True-m_48-k_96-n_16-trace_size_0-partition_N_1]✅ 5/5157.441.4662.09
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_1-b_col_maj_False-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]✅ 5/548302.540.52355.67
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_2-b_col_maj_True-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]✅ 5/527155.280.93632.65
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_8-b_col_maj_True-c_col_maj_True-m_64-k_64-n_64-trace_size_0-partition_N_1]✅ 5/57278.523.462360.42
test_gemm[M_384-K_1536-N_1792-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_32-k_48-n_64-trace_size_0-partition_N_1]✅ 5/51924.184.191099.46
test_gemm[M_64-K_512-N_256-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_16-k_64-n_64-trace_size_0-partition_N_4]✅ 5/52349.140.5328.69
test_gemm[M_896-K_1792-N_640-num_aie_columns_8-b_col_maj_False-c_col_maj_True-m_32-k_64-n_80-trace_size_0-partition_N_1]✅ 5/51344.705.681755.21
iron/operators/gemv
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_gemv[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128]✅ 5/5n/a0.210.21
test_gemv[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048]✅ 5/5n/a13.1613.15
test_gemv[M_2048-K_8192-num_aie_columns_2-tile_size_input_1-tile_size_output_1024]✅ 5/5n/a23.4723.46
test_gemv[M_2048-K_8192-num_aie_columns_4-tile_size_input_1-tile_size_output_512]✅ 5/5n/a32.1732.15
test_gemv[M_2048-K_8192-num_aie_columns_8-tile_size_input_1-tile_size_output_256]✅ 5/5n/a36.6936.67
test_gemv[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024]✅ 5/5n/a12.8712.86
test_gemv[M_8192-K_2048-num_aie_columns_2-tile_size_input_4-tile_size_output_1024]✅ 5/5n/a22.1822.17
test_gemv[M_8192-K_2048-num_aie_columns_4-tile_size_input_4-tile_size_output_1024]✅ 5/5n/a31.7031.68
test_gemv[M_8192-K_2048-num_aie_columns_8-tile_size_input_4-tile_size_output_1024]✅ 5/5n/a35.7635.74
iron/operators/layer_norm
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5266.100.04n/a
test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5241.260.04n/a
test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5238.720.04n/a
test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5248.940.04n/a
test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5189.700.05n/a
test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5219.560.04n/a
test_layer_norm[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256]✅ 5/5259.280.03n/a
test_layer_norm[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128]✅ 5/5292.280.03n/a
iron/operators/mem_copy
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_mem_copy[input_length_2048-num_cores_1-num_channels_1-bypass_False-tile_size_2048]✅ 5/53687.780.02n/a
test_mem_copy[input_length_2048-num_cores_16-num_channels_2-bypass_False-tile_size_128]✅ 5/51368.940.03n/a
test_mem_copy[input_length_2048-num_cores_2-num_channels_1-bypass_False-tile_size_1024]✅ 5/5198.400.05n/a
test_mem_copy[input_length_2048-num_cores_2-num_channels_2-bypass_False-tile_size_1024]✅ 5/5941.500.03n/a
test_mem_copy[input_length_2048-num_cores_4-num_channels_1-bypass_False-tile_size_512]✅ 5/52902.780.03n/a
test_mem_copy[input_length_2048-num_cores_4-num_channels_2-bypass_False-tile_size_512]✅ 5/5584.520.03n/a
test_mem_copy[input_length_2048-num_cores_8-num_channels_1-bypass_False-tile_size_256]✅ 5/5799.060.03n/a
test_mem_copy[input_length_2048-num_cores_8-num_channels_2-bypass_False-tile_size_256]✅ 5/5616.480.02n/a
iron/operators/mha
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_mha[seq_len_16384-dim_64-num_heads_1-num_pipelines_8-num_kv_heads_0]✅ 5/540629.240.21n/a
iron/operators/relu
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5273.280.03n/a
test_relu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5284.000.03n/a
test_relu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5219.520.04n/a
test_relu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5260.240.03n/a
test_relu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5271.200.04n/a
test_relu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5248.400.04n/a
test_relu[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256]✅ 5/5220.640.04n/a
test_relu[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128]✅ 5/5238.100.04n/a
iron/operators/rms_norm
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_False]✅ 5/52874.680.03n/a
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_True]✅ 5/5228.420.06n/a
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_False]✅ 5/53227.020.03n/a
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_True]✅ 5/5228.860.05n/a
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_False]✅ 5/53226.580.03n/a
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_True]✅ 5/5214.600.05n/a
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_False]✅ 5/5225.320.04n/a
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_True]✅ 5/5923.420.03n/a
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_False]✅ 5/5241.840.04n/a
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_True]✅ 5/53677.140.02n/a
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-weighted_False]✅ 5/5230.600.04n/a
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-weighted_True]✅ 5/5907.120.03n/a
test_rms_norm[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256-weighted_False]✅ 5/5201.740.04n/a
test_rms_norm[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256-weighted_True]✅ 5/51355.620.03n/a
test_rms_norm[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128-weighted_False]✅ 5/5283.720.04n/a
iron/operators/rope
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_1-method_type_0]✅ 5/53347.920.38n/a
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_2-method_type_0]✅ 5/53495.820.38n/a
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_4-method_type_0]✅ 5/53461.760.33n/a
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_8-method_type_0]✅ 5/52085.480.24n/a
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_1-method_type_0]✅ 5/53534.360.24n/a
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_2-method_type_0]✅ 5/53521.260.24n/a
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_4-method_type_0]✅ 5/53434.380.24n/a
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_8-method_type_0]✅ 5/53446.760.22n/a
iron/operators/sigmoid
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_sigmoid[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5953.600.03n/a
test_sigmoid[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/53316.920.04n/a
test_sigmoid[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/510781.780.04n/a
test_sigmoid[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5722.300.03n/a
test_sigmoid[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5814.380.03n/a
test_sigmoid[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5453.440.04n/a
test_sigmoid[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256]✅ 5/5647.080.04n/a
test_sigmoid[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128]✅ 5/51012.180.04n/a
iron/operators/silu
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_silu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5820.380.03n/a
test_silu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5462.440.03n/a
test_silu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5504.360.02n/a
test_silu[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256]✅ 5/5634.460.02n/a
iron/operators/softmax
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_1024]✅ 5/5290.740.46n/a
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_2048]✅ 5/5261.740.57n/a
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5317.680.41n/a
iron/operators/swiglu_decode
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_swiglu_decode[embedding_dim_1024-hidden_dim_3584]✅ 5/511119.840.00n/a
test_swiglu_decode[embedding_dim_2048-hidden_dim_2048]✅ 5/58886.980.00n/a
iron/operators/swiglu_prefill
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_swiglu_prefill[seq_len_256-embedding_dim_2048-hidden_dim_2048-prio_accuracy_False]✅ 5/58482.230.25n/a
iron/operators/swiglu_prefill_stream
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_swiglu_prefill_stream[seq_len_256-embedding_dim_512-hidden_dim_2048-seq_tile_32-embedding_tile_32-hidden_tile_64]❌ 0/5n/an/an/a
test_swiglu_prefill_stream_k2[seq_len_256-embedding_dim_512-hidden_dim_2048-seq_tile_32-embedding_tile_32-hidden_tile_64]❌ 0/53010.200.17n/a
iron/operators/tanh
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_tanh[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5186.220.05n/a
test_tanh[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5168.400.05n/a
test_tanh[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5207.360.04n/a
test_tanh[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5189.480.04n/a
test_tanh[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5197.580.04n/a
test_tanh[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5197.840.04n/a
test_tanh[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256]✅ 5/5209.720.04n/a
test_tanh[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128]✅ 5/5223.080.04n/a
iron/operators/transpose
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_1]✅ 5/5198.802.67n/a
test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_2]✅ 5/5218.704.86n/a
test_transpose[M_2048-N_64-aie_columns_1-channels_2-m_64-n_64-s_8-num_batches_1]✅ 5/5200.722.64n/a

Trends:

IRON Trends

iron/operators/axpy

test_axpy[input_length_2048-num_aie_columns_1-tile_size_2048-scalar_factor_3.0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.08 (-19.89%)0.07 (+6.43%)0.07 (+0.38%)0.06 (+82.02%)0.01 (-67.83%)207.90 (-45.07%)175.40 (-18.87%)172.70 (-0.40%)150.80 (+24.83%)22.25 (-78.45%)
573678d — 2026-06-29 18:46:460.10 (n/a)0.07 (n/a)0.07 (n/a)0.03 (n/a)0.03 (n/a)378.50 (n/a)216.20 (n/a)173.40 (n/a)120.80 (n/a)103.24 (n/a)

test_axpy[input_length_2048-num_aie_columns_2-tile_size_1024-scalar_factor_3.0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.11 (-0.37%)0.09 (+22.80%)0.10 (+42.84%)0.07 (+9.70%)0.02 (-5.98%)183.60 (-8.84%)135.00 (-19.08%)122.90 (-29.97%)110.40 (+0.36%)30.66 (-10.03%)
573678d — 2026-06-29 18:46:460.11 (n/a)0.08 (n/a)0.07 (n/a)0.06 (n/a)0.02 (n/a)201.40 (n/a)166.84 (n/a)175.50 (n/a)110.00 (n/a)34.07 (n/a)

test_axpy[input_length_2048-num_aie_columns_4-tile_size_512-scalar_factor_3.0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.13 (+27.33%)0.09 (+24.89%)0.08 (+9.20%)0.08 (+102.83%)0.02 (-10.51%)162.70 (-50.70%)140.24 (-26.50%)153.60 (-8.41%)93.50 (-21.49%)27.70 (-67.02%)
573678d — 2026-06-29 18:46:460.10 (n/a)0.07 (n/a)0.07 (n/a)0.04 (n/a)0.03 (n/a)330.00 (n/a)190.80 (n/a)167.70 (n/a)119.10 (n/a)83.99 (n/a)

test_axpy[input_length_2048-num_aie_columns_8-tile_size_256-scalar_factor_3.0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.10 (+39.38%)0.08 (+27.50%)0.08 (+32.75%)0.06 (+56.54%)0.02 (+19.81%)206.20 (-36.12%)168.80 (-22.85%)158.20 (-24.70%)124.40 (-28.26%)34.03 (-44.20%)
573678d — 2026-06-29 18:46:460.07 (n/a)0.06 (n/a)0.06 (n/a)0.04 (n/a)0.01 (n/a)322.80 (n/a)218.80 (n/a)210.10 (n/a)173.40 (n/a)60.99 (n/a)
iron/operators/dequant

test_dequant[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.06 (+52.16%)0.04 (+19.63%)0.03 (+5.36%)0.02 (-13.95%)0.01 (+191.44%)223.40 (+16.23%)149.62 (-10.29%)153.10 (-5.08%)91.90 (-34.31%)48.83 (+117.42%)
573678d — 2026-06-29 18:46:460.04 (n/a)0.03 (n/a)0.03 (n/a)0.03 (n/a)0.00 (n/a)192.20 (n/a)166.78 (n/a)161.30 (n/a)139.90 (n/a)22.46 (n/a)

test_dequant[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.05 (+17.60%)0.04 (-7.77%)0.03 (-9.09%)0.03 (-18.70%)0.01 (+104.74%)186.60 (+23.01%)152.64 (+12.68%)158.70 (+10.06%)97.30 (-15.02%)33.91 (+105.63%)
573678d — 2026-06-29 18:46:460.05 (n/a)0.04 (n/a)0.04 (n/a)0.03 (n/a)0.01 (n/a)151.70 (n/a)135.46 (n/a)144.20 (n/a)114.50 (n/a)16.49 (n/a)

test_dequant[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.04 (+15.10%)0.03 (+8.69%)0.03 (+2.15%)0.03 (+19.06%)0.00 (+3.82%)175.50 (-15.99%)159.12 (-8.22%)162.70 (-2.11%)134.40 (-13.12%)16.03 (-25.61%)
573678d — 2026-06-29 18:46:460.03 (n/a)0.03 (n/a)0.03 (n/a)0.03 (n/a)0.00 (n/a)208.90 (n/a)173.38 (n/a)166.20 (n/a)154.70 (n/a)21.55 (n/a)

test_dequant[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.05 (+21.90%)0.04 (-4.22%)0.03 (-19.85%)0.02 (-26.12%)0.01 (+214.81%)225.30 (+35.32%)165.34 (+16.08%)176.20 (+24.79%)103.30 (-17.95%)59.39 (+242.71%)
573678d — 2026-06-29 18:46:460.04 (n/a)0.04 (n/a)0.04 (n/a)0.03 (n/a)0.00 (n/a)166.50 (n/a)142.44 (n/a)141.20 (n/a)125.90 (n/a)17.33 (n/a)

test_dequant[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.05 (+21.77%)0.04 (-0.87%)0.03 (-2.77%)0.02 (-17.48%)0.01 (+92.11%)230.20 (+21.16%)161.04 (+6.82%)153.20 (+2.82%)96.80 (-17.90%)49.22 (+87.16%)
573678d — 2026-06-29 18:46:460.04 (n/a)0.04 (n/a)0.04 (n/a)0.03 (n/a)0.01 (n/a)190.00 (n/a)150.76 (n/a)149.00 (n/a)117.90 (n/a)26.30 (n/a)

test_dequant[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.06 (+28.86%)0.03 (+5.04%)0.03 (-0.82%)0.02 (-8.30%)0.01 (+76.77%)219.00 (+9.06%)167.96 (+0.17%)179.40 (+0.84%)91.80 (-22.40%)46.91 (+39.50%)
573678d — 2026-06-29 18:46:460.04 (n/a)0.03 (n/a)0.03 (n/a)0.03 (n/a)0.01 (n/a)200.80 (n/a)167.68 (n/a)177.90 (n/a)118.30 (n/a)33.63 (n/a)

test_dequant[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.05 (+41.72%)0.03 (+19.44%)0.03 (+19.15%)0.02 (-2.49%)0.01 (+94.25%)226.40 (+2.58%)163.54 (-12.55%)158.50 (-16.05%)106.00 (-29.43%)47.57 (+39.88%)
573678d — 2026-06-29 18:46:460.03 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)220.70 (n/a)187.00 (n/a)188.80 (n/a)150.20 (n/a)34.01 (n/a)

test_dequant[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.04 (+11.20%)0.03 (+2.01%)0.03 (-4.18%)0.02 (-8.60%)0.01 (+51.08%)243.70 (+9.38%)195.84 (-0.62%)199.50 (+4.34%)148.00 (-10.03%)34.31 (+44.41%)
573678d — 2026-06-29 18:46:460.03 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.00 (n/a)222.80 (n/a)197.06 (n/a)191.20 (n/a)164.50 (n/a)23.76 (n/a)
iron/operators/elementwise_add

test_elementwise_add[input_length_2048-num_aie_columns_1-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.12 (n/a)0.09 (n/a)0.09 (n/a)0.06 (n/a)0.03 (n/a)218.00 (n/a)145.78 (n/a)144.30 (n/a)102.80 (n/a)46.07 (n/a)

test_elementwise_add[input_length_2048-num_aie_columns_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.12 (n/a)0.11 (n/a)0.12 (n/a)0.08 (n/a)0.02 (n/a)146.60 (n/a)117.06 (n/a)106.80 (n/a)101.70 (n/a)18.43 (n/a)

test_elementwise_add[input_length_2048-num_aie_columns_4-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.14 (n/a)0.12 (n/a)0.11 (n/a)0.10 (n/a)0.02 (n/a)126.30 (n/a)106.98 (n/a)107.00 (n/a)90.80 (n/a)15.00 (n/a)

test_elementwise_add[input_length_2048-num_aie_columns_8-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.13 (n/a)0.10 (n/a)0.10 (n/a)0.09 (n/a)0.02 (n/a)142.40 (n/a)124.82 (n/a)128.30 (n/a)93.40 (n/a)19.45 (n/a)
iron/operators/elementwise_mul

test_elementwise_mul[input_length_2048-num_aie_columns_1-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.10 (n/a)0.08 (n/a)0.09 (n/a)0.06 (n/a)0.02 (n/a)204.40 (n/a)154.26 (n/a)144.20 (n/a)121.80 (n/a)36.19 (n/a)

test_elementwise_mul[input_length_2048-num_aie_columns_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.14 (n/a)0.10 (n/a)0.08 (n/a)0.07 (n/a)0.03 (n/a)165.60 (n/a)136.46 (n/a)152.60 (n/a)86.90 (n/a)34.84 (n/a)

test_elementwise_mul[input_length_2048-num_aie_columns_4-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.14 (n/a)0.10 (n/a)0.10 (n/a)0.06 (n/a)0.04 (n/a)212.40 (n/a)136.00 (n/a)123.10 (n/a)89.10 (n/a)52.49 (n/a)

test_elementwise_mul[input_length_2048-num_aie_columns_8-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.12 (n/a)0.08 (n/a)0.06 (n/a)0.05 (n/a)0.03 (n/a)225.60 (n/a)173.90 (n/a)193.00 (n/a)105.30 (n/a)57.07 (n/a)
iron/operators/gelu

test_gelu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.09 (n/a)0.07 (n/a)0.08 (n/a)0.05 (n/a)0.02 (n/a)161.00 (n/a)116.46 (n/a)100.60 (n/a)89.80 (n/a)29.26 (n/a)

test_gelu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.09 (n/a)0.07 (n/a)0.06 (n/a)0.05 (n/a)0.01 (n/a)160.30 (n/a)128.66 (n/a)131.30 (n/a)95.20 (n/a)23.23 (n/a)

test_gelu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.09 (n/a)0.07 (n/a)0.07 (n/a)0.05 (n/a)0.02 (n/a)159.70 (n/a)130.08 (n/a)122.90 (n/a)89.30 (n/a)29.46 (n/a)

test_gelu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.09 (n/a)0.08 (n/a)0.08 (n/a)0.05 (n/a)0.02 (n/a)165.90 (n/a)114.06 (n/a)104.60 (n/a)89.30 (n/a)30.02 (n/a)

test_gelu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.09 (n/a)0.08 (n/a)0.08 (n/a)0.05 (n/a)0.02 (n/a)176.20 (n/a)112.74 (n/a)98.50 (n/a)87.10 (n/a)36.49 (n/a)

test_gelu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.07 (n/a)0.06 (n/a)0.06 (n/a)0.05 (n/a)0.01 (n/a)166.40 (n/a)140.22 (n/a)147.40 (n/a)112.10 (n/a)26.09 (n/a)

test_gelu[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.08 (n/a)0.06 (n/a)0.06 (n/a)0.05 (n/a)0.01 (n/a)171.80 (n/a)140.64 (n/a)148.00 (n/a)102.40 (n/a)28.06 (n/a)

test_gelu[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.08 (n/a)0.06 (n/a)0.06 (n/a)0.04 (n/a)0.02 (n/a)231.50 (n/a)153.20 (n/a)132.30 (n/a)108.50 (n/a)50.83 (n/a)
iron/operators/gemm

test_gemm[M_1792-K_896-N_1152-num_aie_columns_8-b_col_maj_False-c_col_maj_True-m_64-k_32-n_48-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
a1ee5e5 — 2026-07-06 20:51:535.09 (+16.92%)4.87 (+21.02%)4.84 (+20.28%)4.76 (+30.41%)0.13 (-49.94%)1976.60 (-23.32%)1930.36 (-17.59%)1942.40 (-16.86%)1847.00 (-14.47%)48.91 (-67.61%)2002.88 (+16.92%)1917.43 (+21.02%)1904.50 (+20.28%)1871.60 (+30.41%)49.87 (-49.94%)
573678d — 2026-06-29 18:46:464.35 (n/a)4.03 (n/a)4.03 (n/a)3.65 (n/a)0.25 (n/a)2577.60 (n/a)2342.44 (n/a)2336.40 (n/a)2159.60 (n/a)151.02 (n/a)1712.99 (n/a)1584.41 (n/a)1583.36 (n/a)1435.20 (n/a)99.61 (n/a)

test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_False-c_col_maj_False-m_48-k_96-n_16-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
a1ee5e5 — 2026-07-06 20:51:531.76 (+72.50%)1.25 (+30.13%)1.02 (+5.51%)0.99 (+8.06%)0.35 (+701.00%)224.50 (-7.46%)187.60 (-18.87%)217.50 (-5.19%)125.80 (-42.00%)45.79 (+339.72%)75.04 (+72.50%)53.20 (+30.13%)43.40 (+5.51%)42.04 (+8.06%)14.87 (+701.00%)
573678d — 2026-06-29 18:46:461.02 (n/a)0.96 (n/a)0.96 (n/a)0.91 (n/a)0.04 (n/a)242.60 (n/a)231.22 (n/a)229.40 (n/a)216.90 (n/a)10.41 (n/a)43.50 (n/a)40.88 (n/a)41.13 (n/a)38.91 (n/a)1.86 (n/a)

test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_True-c_col_maj_True-m_48-k_96-n_16-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
a1ee5e5 — 2026-07-06 20:51:531.76 (+53.54%)1.46 (+62.13%)1.45 (+71.37%)1.13 (+76.19%)0.30 (+50.91%)195.10 (-43.25%)157.44 (-38.69%)152.10 (-41.63%)125.60 (-34.89%)33.10 (-44.14%)75.13 (+53.54%)62.09 (+62.13%)62.06 (+71.37%)48.36 (+76.19%)12.87 (+50.91%)
573678d — 2026-06-29 18:46:461.15 (n/a)0.90 (n/a)0.85 (n/a)0.64 (n/a)0.20 (n/a)343.80 (n/a)256.80 (n/a)260.60 (n/a)192.90 (n/a)59.25 (n/a)48.93 (n/a)38.30 (n/a)36.22 (n/a)27.45 (n/a)8.53 (n/a)

test_gemm[M_2048-K_2048-N_2048-num_aie_columns_1-b_col_maj_False-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
a1ee5e5 — 2026-07-06 20:51:530.52 (+0.73%)0.52 (+0.72%)0.52 (+0.67%)0.52 (+0.77%)0.00 (-1.62%)48328.90 (-0.76%)48302.54 (-0.72%)48319.90 (-0.67%)48271.10 (-0.73%)28.47 (-3.11%)355.90 (+0.73%)355.67 (+0.72%)355.54 (+0.67%)355.48 (+0.77%)0.21 (-1.61%)
573678d — 2026-06-29 18:46:460.52 (n/a)0.52 (n/a)0.52 (n/a)0.52 (n/a)0.00 (n/a)48698.80 (n/a)48651.26 (n/a)48644.30 (n/a)48625.70 (n/a)29.38 (n/a)353.31 (n/a)353.12 (n/a)353.17 (n/a)352.78 (n/a)0.21 (n/a)

test_gemm[M_2048-K_2048-N_2048-num_aie_columns_2-b_col_maj_True-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
a1ee5e5 — 2026-07-06 20:51:530.93 (+4.71%)0.93 (+5.22%)0.93 (+5.13%)0.92 (+5.78%)0.00 (-62.43%)27210.00 (-5.47%)27155.28 (-4.97%)27174.60 (-4.88%)27067.30 (-4.50%)56.23 (-66.08%)634.71 (+4.71%)632.65 (+5.22%)632.20 (+5.13%)631.38 (+5.78%)1.31 (-62.43%)
573678d — 2026-06-29 18:46:460.89 (n/a)0.88 (n/a)0.88 (n/a)0.87 (n/a)0.01 (n/a)28783.60 (n/a)28574.08 (n/a)28568.40 (n/a)28342.90 (n/a)165.78 (n/a)606.14 (n/a)601.26 (n/a)601.36 (n/a)596.86 (n/a)3.49 (n/a)

test_gemm[M_2048-K_2048-N_2048-num_aie_columns_8-b_col_maj_True-c_col_maj_True-m_64-k_64-n_64-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
a1ee5e5 — 2026-07-06 20:51:533.48 (+1.06%)3.46 (+5.62%)3.46 (+6.49%)3.43 (+7.41%)0.02 (-80.27%)7337.00 (-6.90%)7278.52 (-5.39%)7277.30 (-6.10%)7226.40 (-1.05%)41.35 (-81.69%)2377.36 (+1.06%)2360.42 (+5.62%)2360.75 (+6.49%)2341.55 (+7.41%)13.39 (-80.27%)
573678d — 2026-06-29 18:46:463.45 (n/a)3.27 (n/a)3.25 (n/a)3.19 (n/a)0.10 (n/a)7880.60 (n/a)7693.12 (n/a)7749.80 (n/a)7302.80 (n/a)225.78 (n/a)2352.51 (n/a)2234.74 (n/a)2216.80 (n/a)2180.03 (n/a)67.84 (n/a)

test_gemm[M_384-K_1536-N_1792-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_32-k_48-n_64-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
a1ee5e5 — 2026-07-06 20:51:534.35 (+2.33%)4.19 (+22.27%)4.11 (+25.12%)4.08 (+48.29%)0.13 (-78.96%)1974.50 (-32.56%)1924.18 (-20.24%)1959.60 (-20.08%)1854.10 (-2.28%)59.40 (-86.07%)1140.12 (+2.33%)1099.46 (+22.27%)1078.75 (+25.12%)1070.61 (+48.29%)34.31 (-78.96%)
573678d — 2026-06-29 18:46:464.25 (n/a)3.43 (n/a)3.29 (n/a)2.75 (n/a)0.62 (n/a)2928.00 (n/a)2412.46 (n/a)2451.90 (n/a)1897.40 (n/a)426.57 (n/a)1114.14 (n/a)899.19 (n/a)862.16 (n/a)721.97 (n/a)163.11 (n/a)

test_gemm[M_64-K_512-N_256-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_16-k_64-n_64-trace_size_0-partition_N_4]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
a1ee5e5 — 2026-07-06 20:51:530.57 (+8.90%)0.53 (+55.77%)0.55 (+78.21%)0.48 (+87.72%)0.04 (-64.29%)2581.90 (-46.73%)2349.14 (-39.38%)2258.00 (-43.89%)2202.50 (-8.17%)170.95 (-82.14%)30.47 (+8.90%)28.69 (+55.77%)29.72 (+78.21%)25.99 (+87.72%)2.02 (-64.29%)
573678d — 2026-06-29 18:46:460.52 (n/a)0.34 (n/a)0.31 (n/a)0.26 (n/a)0.11 (n/a)4846.70 (n/a)3875.42 (n/a)4024.20 (n/a)2398.40 (n/a)957.27 (n/a)27.98 (n/a)18.42 (n/a)16.68 (n/a)13.85 (n/a)5.67 (n/a)

test_gemm[M_896-K_1792-N_640-num_aie_columns_8-b_col_maj_False-c_col_maj_True-m_32-k_64-n_80-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
a1ee5e5 — 2026-07-06 20:51:536.97 (+40.48%)5.68 (+26.53%)6.34 (+35.95%)2.56 (-27.13%)1.77 (+215.87%)2596.40 (+37.23%)1344.70 (-10.57%)1049.50 (-26.45%)954.50 (-28.82%)701.27 (+218.08%)2153.19 (+40.48%)1755.21 (+26.53%)1958.19 (+35.95%)791.56 (-27.13%)546.83 (+215.87%)
573678d — 2026-06-29 18:46:464.96 (n/a)4.49 (n/a)4.66 (n/a)3.52 (n/a)0.56 (n/a)1892.00 (n/a)1503.56 (n/a)1426.90 (n/a)1340.90 (n/a)220.47 (n/a)1532.72 (n/a)1387.16 (n/a)1440.38 (n/a)1086.24 (n/a)173.12 (n/a)
iron/operators/gemv

test_gemv[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
a1ee5e5 — 2026-07-06 20:51:530.26 (+27.59%)0.21 (+18.49%)0.20 (+7.09%)0.14 (+14.95%)0.04 (+39.29%)0.26 (+27.59%)0.21 (+18.49%)0.20 (+7.09%)0.14 (+14.95%)0.04 (+39.29%)
573678d — 2026-06-29 18:46:460.20 (n/a)0.18 (n/a)0.19 (n/a)0.12 (n/a)0.03 (n/a)0.20 (n/a)0.17 (n/a)0.19 (n/a)0.12 (n/a)0.03 (n/a)

test_gemv[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
a1ee5e5 — 2026-07-06 20:51:5313.24 (-0.15%)13.16 (+5.18%)13.17 (+5.07%)13.05 (+9.54%)0.07 (-86.59%)13.24 (-0.15%)13.15 (+5.18%)13.16 (+5.07%)13.04 (+9.54%)0.07 (-86.58%)
573678d — 2026-06-29 18:46:4613.26 (n/a)12.51 (n/a)12.54 (n/a)11.91 (n/a)0.53 (n/a)13.25 (n/a)12.50 (n/a)12.53 (n/a)11.90 (n/a)0.53 (n/a)

test_gemv[M_2048-K_8192-num_aie_columns_2-tile_size_input_1-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
a1ee5e5 — 2026-07-06 20:51:5324.55 (+2.22%)23.47 (-0.38%)24.32 (+1.93%)20.05 (-10.09%)1.93 (+171.77%)24.53 (+2.22%)23.46 (-0.38%)24.31 (+1.93%)20.04 (-10.09%)1.93 (+171.77%)
573678d — 2026-06-29 18:46:4624.01 (n/a)23.56 (n/a)23.86 (n/a)22.30 (n/a)0.71 (n/a)24.00 (n/a)23.55 (n/a)23.85 (n/a)22.29 (n/a)0.71 (n/a)

test_gemv[M_2048-K_8192-num_aie_columns_4-tile_size_input_1-tile_size_output_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
a1ee5e5 — 2026-07-06 20:51:5339.97 (+0.86%)32.17 (-14.01%)38.76 (-1.56%)9.74 (-68.22%)12.84 (+233.69%)39.95 (+0.86%)32.15 (-14.01%)38.74 (-1.56%)9.74 (-68.22%)12.83 (+233.69%)
573678d — 2026-06-29 18:46:4639.63 (n/a)37.41 (n/a)39.38 (n/a)30.65 (n/a)3.85 (n/a)39.61 (n/a)37.39 (n/a)39.35 (n/a)30.63 (n/a)3.84 (n/a)

test_gemv[M_2048-K_8192-num_aie_columns_8-tile_size_input_1-tile_size_output_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
a1ee5e5 — 2026-07-06 20:51:5342.43 (-5.00%)36.69 (-13.81%)41.65 (-3.84%)15.66 (-59.87%)11.77 (+432.73%)42.41 (-5.00%)36.67 (-13.81%)41.63 (-3.84%)15.65 (-59.87%)11.76 (+432.73%)
573678d — 2026-06-29 18:46:4644.67 (n/a)42.57 (n/a)43.32 (n/a)39.02 (n/a)2.21 (n/a)44.64 (n/a)42.55 (n/a)43.29 (n/a)39.00 (n/a)2.21 (n/a)

test_gemv[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
a1ee5e5 — 2026-07-06 20:51:5313.27 (+0.41%)12.87 (+3.30%)13.19 (+6.70%)11.47 (-4.18%)0.78 (+71.46%)13.26 (+0.41%)12.86 (+3.30%)13.18 (+6.70%)11.46 (-4.18%)0.78 (+71.46%)
573678d — 2026-06-29 18:46:4613.21 (n/a)12.46 (n/a)12.36 (n/a)11.97 (n/a)0.46 (n/a)13.20 (n/a)12.45 (n/a)12.35 (n/a)11.96 (n/a)0.46 (n/a)

test_gemv[M_8192-K_2048-num_aie_columns_2-tile_size_input_4-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
a1ee5e5 — 2026-07-06 20:51:5324.82 (+1.99%)22.18 (-6.36%)23.90 (+0.15%)14.39 (-36.00%)4.38 (+519.27%)24.80 (+1.99%)22.17 (-6.36%)23.88 (+0.15%)14.38 (-36.00%)4.38 (+519.27%)
573678d — 2026-06-29 18:46:4624.33 (n/a)23.69 (n/a)23.86 (n/a)22.48 (n/a)0.71 (n/a)24.32 (n/a)23.67 (n/a)23.84 (n/a)22.46 (n/a)0.71 (n/a)

test_gemv[M_8192-K_2048-num_aie_columns_4-tile_size_input_4-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
a1ee5e5 — 2026-07-06 20:51:5340.24 (-1.36%)31.70 (-9.96%)38.76 (+2.81%)2.04 (-91.73%)16.59 (+151.90%)40.22 (-1.36%)31.68 (-9.96%)38.73 (+2.81%)2.04 (-91.73%)16.58 (+151.90%)
573678d — 2026-06-29 18:46:4640.80 (n/a)35.21 (n/a)37.70 (n/a)24.68 (n/a)6.59 (n/a)40.77 (n/a)35.18 (n/a)37.67 (n/a)24.67 (n/a)6.58 (n/a)

test_gemv[M_8192-K_2048-num_aie_columns_8-tile_size_input_4-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
a1ee5e5 — 2026-07-06 20:51:5343.02 (-5.48%)35.76 (-14.95%)41.92 (-0.69%)12.38 (-68.16%)13.16 (+416.90%)42.99 (-5.48%)35.74 (-14.95%)41.89 (-0.69%)12.37 (-68.16%)13.15 (+416.90%)
573678d — 2026-06-29 18:46:4645.51 (n/a)42.04 (n/a)42.21 (n/a)38.89 (n/a)2.55 (n/a)45.48 (n/a)42.02 (n/a)42.18 (n/a)38.86 (n/a)2.54 (n/a)
iron/operators/layer_norm

test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.07 (n/a)0.04 (n/a)0.03 (n/a)0.02 (n/a)0.02 (n/a)407.00 (n/a)266.10 (n/a)276.90 (n/a)111.10 (n/a)106.82 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.08 (n/a)0.04 (n/a)0.04 (n/a)0.02 (n/a)0.02 (n/a)397.10 (n/a)241.26 (n/a)228.80 (n/a)100.60 (n/a)112.54 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.05 (n/a)0.04 (n/a)0.03 (n/a)0.03 (n/a)0.01 (n/a)314.80 (n/a)238.72 (n/a)257.10 (n/a)171.20 (n/a)62.81 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.05 (n/a)0.04 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)331.00 (n/a)248.94 (n/a)242.30 (n/a)150.60 (n/a)72.08 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.07 (n/a)0.05 (n/a)0.05 (n/a)0.03 (n/a)0.02 (n/a)302.80 (n/a)189.70 (n/a)149.40 (n/a)111.00 (n/a)84.12 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.07 (n/a)0.04 (n/a)0.04 (n/a)0.02 (n/a)0.02 (n/a)336.00 (n/a)219.56 (n/a)192.40 (n/a)122.40 (n/a)83.42 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.05 (n/a)0.03 (n/a)0.03 (n/a)0.03 (n/a)0.01 (n/a)324.00 (n/a)259.28 (n/a)266.50 (n/a)177.50 (n/a)53.86 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.05 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)502.10 (n/a)292.28 (n/a)265.80 (n/a)164.10 (n/a)125.30 (n/a)
iron/operators/mem_copy

test_mem_copy[input_length_2048-num_cores_1-num_channels_1-bypass_False-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.06 (-7.64%)0.02 (-54.64%)0.02 (-53.20%)0.00 (-98.77%)0.02 (+129.71%)15796.30 (+8004.82%)3687.78 (+2171.92%)364.40 (+113.60%)131.20 (+8.25%)6804.60 (+21867.73%)
573678d — 2026-06-29 18:46:460.07 (n/a)0.05 (n/a)0.05 (n/a)0.04 (n/a)0.01 (n/a)194.90 (n/a)162.32 (n/a)170.60 (n/a)121.20 (n/a)30.98 (n/a)

test_mem_copy[input_length_2048-num_cores_16-num_channels_2-bypass_False-tile_size_128]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.03 (-33.33%)0.03 (-28.40%)0.03 (-5.31%)0.00 (-94.14%)0.01 (+19.54%)5817.70 (+1607.57%)1368.94 (+457.43%)261.40 (+5.62%)238.00 (+50.06%)2486.96 (+3204.02%)
573678d — 2026-06-29 18:46:460.05 (n/a)0.04 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)340.70 (n/a)245.58 (n/a)247.50 (n/a)158.60 (n/a)75.27 (n/a)

test_mem_copy[input_length_2048-num_cores_2-num_channels_1-bypass_False-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.08 (+48.17%)0.05 (-0.80%)0.04 (-23.55%)0.03 (-20.77%)0.02 (+240.95%)274.30 (+26.23%)198.40 (+13.06%)225.80 (+30.82%)104.20 (-32.51%)70.96 (+185.09%)
573678d — 2026-06-29 18:46:460.05 (n/a)0.05 (n/a)0.05 (n/a)0.04 (n/a)0.01 (n/a)217.30 (n/a)175.48 (n/a)172.60 (n/a)154.40 (n/a)24.89 (n/a)

test_mem_copy[input_length_2048-num_cores_2-num_channels_2-bypass_False-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.06 (+0.38%)0.03 (-39.43%)0.03 (-50.46%)0.00 (-95.38%)0.02 (+256.58%)3809.40 (+2065.66%)941.50 (+529.77%)295.50 (+101.84%)128.50 (-0.39%)1604.95 (+8777.21%)
573678d — 2026-06-29 18:46:460.06 (n/a)0.06 (n/a)0.06 (n/a)0.05 (n/a)0.01 (n/a)175.90 (n/a)149.50 (n/a)146.40 (n/a)129.00 (n/a)18.08 (n/a)

test_mem_copy[input_length_2048-num_cores_4-num_channels_1-bypass_False-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.05 (-22.14%)0.03 (-41.15%)0.04 (-33.36%)0.00 (-98.66%)0.02 (+221.23%)13629.00 (+7359.77%)2902.78 (+1733.72%)230.90 (+50.03%)181.70 (+28.41%)5996.29 (+33812.52%)
573678d — 2026-06-29 18:46:460.06 (n/a)0.05 (n/a)0.05 (n/a)0.04 (n/a)0.01 (n/a)182.70 (n/a)158.30 (n/a)153.90 (n/a)141.50 (n/a)17.68 (n/a)

test_mem_copy[input_length_2048-num_cores_4-num_channels_2-bypass_False-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.06 (+11.17%)0.03 (-40.79%)0.03 (-44.41%)0.00 (-90.88%)0.02 (+379.40%)1941.50 (+996.27%)584.52 (+270.70%)288.20 (+79.90%)129.10 (-10.03%)761.76 (+5502.48%)
573678d — 2026-06-29 18:46:460.06 (n/a)0.05 (n/a)0.05 (n/a)0.05 (n/a)0.00 (n/a)177.10 (n/a)157.68 (n/a)160.20 (n/a)143.50 (n/a)13.60 (n/a)

test_mem_copy[input_length_2048-num_cores_8-num_channels_1-bypass_False-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.04 (-34.40%)0.03 (-47.71%)0.03 (-42.99%)0.00 (-94.11%)0.01 (+304.73%)2958.30 (+1596.27%)799.06 (+386.22%)288.40 (+75.43%)226.60 (+52.39%)1207.41 (+11205.07%)
573678d — 2026-06-29 18:46:460.06 (n/a)0.05 (n/a)0.05 (n/a)0.05 (n/a)0.00 (n/a)174.40 (n/a)164.34 (n/a)164.40 (n/a)148.70 (n/a)10.68 (n/a)

test_mem_copy[input_length_2048-num_cores_8-num_channels_2-bypass_False-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.04 (-35.56%)0.02 (-49.57%)0.03 (-47.46%)0.00 (-87.40%)0.01 (+42.78%)1915.40 (+694.11%)616.48 (+247.51%)311.50 (+90.40%)225.40 (+55.23%)727.83 (+1836.88%)
573678d — 2026-06-29 18:46:460.06 (n/a)0.05 (n/a)0.05 (n/a)0.03 (n/a)0.01 (n/a)241.20 (n/a)177.40 (n/a)163.60 (n/a)145.20 (n/a)37.58 (n/a)
iron/operators/mha

test_mha[seq_len_16384-dim_64-num_heads_1-num_pipelines_8-num_kv_heads_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.21 (+3.46%)0.21 (+0.84%)0.21 (+3.46%)0.20 (-3.65%)0.01 (+3510.60%)42514.70 (+3.79%)40629.24 (-0.72%)39571.20 (-3.35%)39494.20 (-3.34%)1503.50 (+3508.64%)
573678d — 2026-06-29 18:46:460.21 (n/a)0.20 (n/a)0.20 (n/a)0.20 (n/a)0.00 (n/a)40961.20 (n/a)40924.92 (n/a)40941.50 (n/a)40859.60 (n/a)41.66 (n/a)
iron/operators/rms_norm

test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.05 (-5.65%)0.03 (-40.72%)0.03 (-37.14%)0.00 (-98.52%)0.02 (+391.42%)13371.30 (+6656.59%)2874.68 (+1516.26%)277.40 (+59.06%)171.70 (+5.99%)5867.97 (+43257.96%)
573678d — 2026-06-29 18:46:460.05 (n/a)0.05 (n/a)0.05 (n/a)0.04 (n/a)0.00 (n/a)197.90 (n/a)177.86 (n/a)174.40 (n/a)162.00 (n/a)13.53 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.09 (-6.81%)0.06 (-29.84%)0.05 (-43.37%)0.04 (-50.81%)0.03 (+144.45%)344.90 (+103.24%)228.42 (+60.72%)249.10 (+76.67%)130.70 (+7.31%)88.34 (+403.09%)
573678d — 2026-06-29 18:46:460.10 (n/a)0.09 (n/a)0.09 (n/a)0.07 (n/a)0.01 (n/a)169.70 (n/a)142.12 (n/a)141.00 (n/a)121.80 (n/a)17.56 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.06 (-6.03%)0.03 (-46.10%)0.03 (-45.74%)0.00 (-98.90%)0.02 (+205.80%)15145.60 (+8969.22%)3227.02 (+2085.74%)288.70 (+84.36%)131.70 (+6.38%)6663.16 (+36358.50%)
573678d — 2026-06-29 18:46:460.07 (n/a)0.06 (n/a)0.05 (n/a)0.05 (n/a)0.01 (n/a)167.00 (n/a)147.64 (n/a)156.60 (n/a)123.80 (n/a)18.28 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.07 (+2.24%)0.05 (-18.65%)0.05 (-17.54%)0.03 (-35.71%)0.01 (+165.50%)301.00 (+55.56%)228.86 (+31.56%)216.00 (+21.28%)148.80 (-2.17%)68.18 (+326.72%)
573678d — 2026-06-29 18:46:460.07 (n/a)0.06 (n/a)0.06 (n/a)0.05 (n/a)0.01 (n/a)193.50 (n/a)173.96 (n/a)178.10 (n/a)152.10 (n/a)15.98 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.06 (-2.04%)0.03 (-31.52%)0.03 (-36.21%)0.00 (-98.63%)0.02 (+118.56%)15244.50 (+7194.02%)3226.58 (+1739.14%)285.70 (+56.81%)130.30 (+2.04%)6718.58 (+19035.10%)
573678d — 2026-06-29 18:46:460.06 (n/a)0.05 (n/a)0.04 (n/a)0.04 (n/a)0.01 (n/a)209.00 (n/a)175.44 (n/a)182.20 (n/a)127.70 (n/a)35.11 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.08 (+2.90%)0.05 (-19.28%)0.05 (-24.36%)0.03 (-36.60%)0.02 (+63.39%)322.80 (+57.77%)214.60 (+33.99%)195.00 (+32.20%)129.20 (-2.86%)77.99 (+155.18%)
573678d — 2026-06-29 18:46:460.08 (n/a)0.07 (n/a)0.07 (n/a)0.05 (n/a)0.01 (n/a)204.60 (n/a)160.16 (n/a)147.50 (n/a)133.00 (n/a)30.56 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.06 (+4.99%)0.04 (-17.33%)0.04 (-9.55%)0.03 (-33.63%)0.01 (+69.79%)313.60 (+50.70%)225.32 (+30.94%)196.90 (+10.56%)130.20 (-4.75%)78.24 (+159.59%)
573678d — 2026-06-29 18:46:460.06 (n/a)0.05 (n/a)0.05 (n/a)0.04 (n/a)0.01 (n/a)208.10 (n/a)172.08 (n/a)178.10 (n/a)136.70 (n/a)30.14 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.05 (-39.10%)0.03 (-52.41%)0.03 (-50.07%)0.00 (-95.44%)0.02 (+84.72%)3588.60 (+2093.52%)923.42 (+535.79%)288.40 (+100.28%)188.10 (+64.14%)1491.00 (+7530.47%)
573678d — 2026-06-29 18:46:460.08 (n/a)0.06 (n/a)0.06 (n/a)0.06 (n/a)0.01 (n/a)163.60 (n/a)145.24 (n/a)144.00 (n/a)114.60 (n/a)19.54 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.06 (-5.96%)0.04 (-17.87%)0.03 (-33.18%)0.03 (-23.82%)0.01 (+43.53%)323.60 (+31.28%)241.84 (+30.37%)273.00 (+49.67%)144.00 (+6.35%)81.08 (+99.30%)
573678d — 2026-06-29 18:46:460.06 (n/a)0.05 (n/a)0.04 (n/a)0.03 (n/a)0.01 (n/a)246.50 (n/a)185.50 (n/a)182.40 (n/a)135.40 (n/a)40.68 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.04 (-38.56%)0.02 (-55.56%)0.03 (-40.82%)0.00 (-98.21%)0.02 (+63.92%)16128.80 (+5482.83%)3677.14 (+1773.22%)301.20 (+68.93%)250.00 (+62.76%)6979.11 (+13049.00%)
573678d — 2026-06-29 18:46:460.06 (n/a)0.05 (n/a)0.05 (n/a)0.03 (n/a)0.01 (n/a)288.90 (n/a)196.30 (n/a)178.30 (n/a)153.60 (n/a)53.08 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.05 (-8.30%)0.04 (-18.33%)0.04 (-19.25%)0.03 (-25.25%)0.01 (+22.90%)302.80 (+33.75%)230.60 (+24.95%)210.80 (+23.85%)176.30 (+9.10%)50.81 (+82.72%)
573678d — 2026-06-29 18:46:460.05 (n/a)0.05 (n/a)0.05 (n/a)0.04 (n/a)0.01 (n/a)226.40 (n/a)184.56 (n/a)170.20 (n/a)161.60 (n/a)27.81 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.04 (-29.32%)0.03 (-42.91%)0.03 (-29.76%)0.00 (-93.40%)0.01 (+164.88%)3399.80 (+1414.39%)907.12 (+362.91%)280.60 (+42.36%)234.90 (+41.51%)1394.19 (+6233.65%)
573678d — 2026-06-29 18:46:460.05 (n/a)0.04 (n/a)0.04 (n/a)0.04 (n/a)0.01 (n/a)224.50 (n/a)195.96 (n/a)197.10 (n/a)166.00 (n/a)22.01 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.07 (+30.01%)0.04 (-0.56%)0.04 (-3.43%)0.03 (-24.41%)0.02 (+177.58%)275.60 (+32.31%)201.74 (+7.94%)187.70 (+3.59%)120.20 (-23.10%)60.49 (+176.39%)
573678d — 2026-06-29 18:46:460.05 (n/a)0.04 (n/a)0.05 (n/a)0.04 (n/a)0.01 (n/a)208.30 (n/a)186.90 (n/a)181.20 (n/a)156.30 (n/a)21.89 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.04 (-37.82%)0.03 (-36.72%)0.04 (-15.33%)0.00 (-96.17%)0.02 (+62.21%)5820.80 (+2513.74%)1355.62 (+606.13%)242.30 (+18.08%)220.20 (+60.85%)2496.17 (+7412.30%)
573678d — 2026-06-29 18:46:460.06 (n/a)0.05 (n/a)0.04 (n/a)0.04 (n/a)0.01 (n/a)222.70 (n/a)191.98 (n/a)205.20 (n/a)136.90 (n/a)33.23 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.06 (+36.60%)0.04 (+25.00%)0.03 (+10.96%)0.01 (-29.52%)0.02 (+91.62%)564.00 (+41.89%)283.72 (-5.58%)258.20 (-9.88%)137.80 (-26.78%)170.50 (+87.14%)
573678d — 2026-06-29 18:46:460.04 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)397.50 (n/a)300.50 (n/a)286.50 (n/a)188.20 (n/a)91.11 (n/a)
iron/operators/rope

test_rope[rows_32-cols_512-angle_rows_32-aie_columns_1-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.86 (+9.74%)0.38 (-32.82%)0.35 (-39.13%)0.01 (-98.57%)0.31 (+122.50%)15785.60 (+6915.82%)3347.92 (+1769.51%)279.00 (+64.31%)113.70 (-8.82%)6953.25 (+17446.03%)
573678d — 2026-06-29 18:46:460.79 (n/a)0.57 (n/a)0.58 (n/a)0.44 (n/a)0.14 (n/a)225.00 (n/a)179.08 (n/a)169.80 (n/a)124.70 (n/a)39.63 (n/a)

test_rope[rows_32-cols_512-angle_rows_32-aie_columns_2-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:531.05 (+27.97%)0.38 (-42.97%)0.31 (-52.68%)0.01 (-98.96%)0.39 (+335.82%)16331.00 (+9551.89%)3495.82 (+2262.68%)313.90 (+111.24%)93.50 (-21.82%)7176.15 (+39178.60%)
573678d — 2026-06-29 18:46:460.82 (n/a)0.67 (n/a)0.66 (n/a)0.58 (n/a)0.09 (n/a)169.20 (n/a)147.96 (n/a)148.60 (n/a)119.60 (n/a)18.27 (n/a)

test_rope[rows_32-cols_512-angle_rows_32-aie_columns_4-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.60 (+2.09%)0.33 (-34.56%)0.33 (-33.12%)0.01 (-98.47%)0.21 (+190.64%)16289.00 (+6415.60%)3461.76 (+1638.35%)295.00 (+49.52%)164.00 (-2.03%)7170.86 (+22620.39%)
573678d — 2026-06-29 18:46:460.59 (n/a)0.50 (n/a)0.50 (n/a)0.39 (n/a)0.07 (n/a)250.00 (n/a)199.14 (n/a)197.30 (n/a)167.40 (n/a)31.56 (n/a)

test_rope[rows_32-cols_512-angle_rows_32-aie_columns_8-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.39 (-36.23%)0.24 (-53.90%)0.38 (-24.92%)0.02 (-96.10%)0.20 (+209.34%)5797.40 (+2461.82%)2085.48 (+978.66%)260.00 (+33.20%)254.00 (+56.79%)2594.91 (+10856.49%)
573678d — 2026-06-29 18:46:460.61 (n/a)0.51 (n/a)0.50 (n/a)0.43 (n/a)0.06 (n/a)226.30 (n/a)193.34 (n/a)195.20 (n/a)162.00 (n/a)23.68 (n/a)

test_rope[rows_32-cols_512-angle_rows_8-aie_columns_1-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.53 (-21.47%)0.24 (-46.35%)0.22 (-52.27%)0.00 (-98.49%)0.19 (+31.84%)16561.10 (+6540.38%)3534.36 (+1918.94%)334.90 (+109.57%)140.00 (+27.39%)7282.62 (+13803.79%)
573678d — 2026-06-29 18:46:460.67 (n/a)0.45 (n/a)0.46 (n/a)0.30 (n/a)0.14 (n/a)249.40 (n/a)175.06 (n/a)159.80 (n/a)109.90 (n/a)52.38 (n/a)

test_rope[rows_32-cols_512-angle_rows_8-aie_columns_2-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.53 (+1.93%)0.24 (-38.42%)0.24 (-33.49%)0.00 (-98.38%)0.19 (+85.94%)16442.70 (+6067.55%)3521.26 (+1684.72%)301.10 (+50.32%)138.80 (-1.91%)7224.17 (+13912.54%)
573678d — 2026-06-29 18:46:460.52 (n/a)0.39 (n/a)0.37 (n/a)0.28 (n/a)0.10 (n/a)266.60 (n/a)197.30 (n/a)200.30 (n/a)141.50 (n/a)51.56 (n/a)

test_rope[rows_32-cols_512-angle_rows_8-aie_columns_4-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.42 (-13.52%)0.24 (-41.64%)0.26 (-39.49%)0.00 (-98.63%)0.15 (+145.82%)16154.60 (+7216.39%)3434.38 (+1816.51%)289.00 (+65.24%)174.10 (+15.60%)7111.01 (+25335.02%)
573678d — 2026-06-29 18:46:460.49 (n/a)0.42 (n/a)0.42 (n/a)0.33 (n/a)0.06 (n/a)220.80 (n/a)179.20 (n/a)174.90 (n/a)150.60 (n/a)27.96 (n/a)

test_rope[rows_32-cols_512-angle_rows_8-aie_columns_8-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.30 (-51.71%)0.22 (-50.40%)0.27 (-36.45%)0.00 (-98.59%)0.12 (+10.09%)16133.20 (+6982.18%)3446.76 (+1851.95%)271.50 (+57.39%)248.90 (+107.07%)7091.97 (+17770.64%)
573678d — 2026-06-29 18:46:460.61 (n/a)0.44 (n/a)0.43 (n/a)0.32 (n/a)0.11 (n/a)227.80 (n/a)176.58 (n/a)172.50 (n/a)120.20 (n/a)39.69 (n/a)
iron/operators/softmax

test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.62 (-30.05%)0.46 (-35.36%)0.43 (-36.66%)0.41 (-33.39%)0.09 (-15.69%)317.70 (+50.14%)290.74 (+55.96%)307.00 (+57.84%)211.40 (+42.93%)44.63 (+80.33%)
573678d — 2026-06-29 18:46:460.89 (n/a)0.71 (n/a)0.67 (n/a)0.62 (n/a)0.11 (n/a)211.60 (n/a)186.42 (n/a)194.50 (n/a)147.90 (n/a)24.75 (n/a)

test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:531.05 (+10.06%)0.57 (-20.01%)0.44 (-33.62%)0.41 (-27.52%)0.27 (+87.61%)315.90 (+37.95%)261.74 (+37.14%)297.90 (+50.68%)125.20 (-9.14%)78.66 (+135.27%)
573678d — 2026-06-29 18:46:460.95 (n/a)0.71 (n/a)0.66 (n/a)0.57 (n/a)0.14 (n/a)229.00 (n/a)190.86 (n/a)197.70 (n/a)137.80 (n/a)33.43 (n/a)

test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.44 (-44.54%)0.41 (-40.10%)0.41 (-41.09%)0.39 (-27.55%)0.02 (-79.12%)335.60 (+37.99%)317.68 (+64.50%)316.30 (+69.78%)298.10 (+80.34%)14.83 (-49.69%)
573678d — 2026-06-29 18:46:460.79 (n/a)0.69 (n/a)0.70 (n/a)0.54 (n/a)0.09 (n/a)243.20 (n/a)193.12 (n/a)186.30 (n/a)165.30 (n/a)29.49 (n/a)
iron/operators/swiglu_decode

test_swiglu_decode[embedding_dim_1024-hidden_dim_3584]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.00 (-9.09%)0.00 (-44.00%)0.00 (-70.00%)0.00 (-66.67%)0.00 (+257.77%)16340.03 (+253.93%)11119.84 (+167.46%)15245.47 (+265.68%)3953.85 (+4.99%)6255.45 (+1614.37%)
573678d — 2026-06-29 18:46:460.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)4616.74 (n/a)4157.53 (n/a)4169.13 (n/a)3765.84 (n/a)364.88 (n/a)

test_swiglu_decode[embedding_dim_2048-hidden_dim_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.00 (-8.70%)0.00 (-25.26%)0.00 (+11.11%)0.00 (-66.67%)0.00 (+147.95%)16152.66 (+191.01%)8886.98 (+100.50%)4195.64 (-7.58%)3905.63 (+9.48%)6619.73 (+742.60%)
573678d — 2026-06-29 18:46:460.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)5550.60 (n/a)4432.38 (n/a)4539.79 (n/a)3567.47 (n/a)785.64 (n/a)
iron/operators/swiglu_prefill

test_swiglu_prefill[seq_len_256-embedding_dim_2048-hidden_dim_2048-prio_accuracy_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.27 (-3.78%)0.25 (-0.75%)0.24 (-10.96%)0.24 (+42.66%)0.01 (-76.08%)8734.71 (-29.91%)8482.23 (-2.73%)8603.81 (+12.29%)7848.30 (+3.91%)360.25 (-82.94%)
573678d — 2026-06-29 18:46:460.28 (n/a)0.25 (n/a)0.27 (n/a)0.17 (n/a)0.05 (n/a)12461.92 (n/a)8720.16 (n/a)7661.84 (n/a)7552.69 (n/a)2112.03 (n/a)
iron/operators/swiglu_prefill_stream

test_swiglu_prefill_stream[seq_len_256-embedding_dim_512-hidden_dim_2048-seq_tile_32-embedding_tile_32-hidden_tile_64]

No metrics available.

test_swiglu_prefill_stream_k2[seq_len_256-embedding_dim_512-hidden_dim_2048-seq_tile_32-embedding_tile_32-hidden_tile_64]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:530.18 (n/a)0.17 (n/a)0.18 (n/a)0.17 (n/a)0.00 (n/a)3113.09 (n/a)3010.20 (n/a)2984.47 (n/a)2936.11 (n/a)66.95 (n/a)
iron/operators/transpose

test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:533.18 (+8.47%)2.67 (+10.26%)2.56 (+4.21%)2.30 (+24.39%)0.34 (-15.94%)227.70 (-19.60%)198.80 (-10.36%)205.00 (-4.07%)164.90 (-7.83%)24.24 (-38.97%)
573678d — 2026-06-29 18:46:462.93 (n/a)2.42 (n/a)2.45 (n/a)1.85 (n/a)0.41 (n/a)283.20 (n/a)221.78 (n/a)213.70 (n/a)178.90 (n/a)39.73 (n/a)

test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_2]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:535.97 (+1.75%)4.86 (+7.41%)4.57 (-1.72%)4.41 (+28.48%)0.65 (-36.05%)237.90 (-22.15%)218.70 (-9.53%)229.50 (+1.73%)175.60 (-1.73%)25.87 (-52.78%)
573678d — 2026-06-29 18:46:465.87 (n/a)4.52 (n/a)4.65 (n/a)3.43 (n/a)1.02 (n/a)305.60 (n/a)241.74 (n/a)225.60 (n/a)178.70 (n/a)54.79 (n/a)

test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
4bb8427 — 2026-06-23 23:08:223.72 (+20.32%)3.12 (+16.02%)3.04 (+6.32%)2.73 (+32.06%)0.43 (+8.15%)192.20 (-24.30%)170.32 (-14.26%)172.50 (-5.94%)141.00 (-16.86%)22.57 (-32.31%)
4d4b803 — 2026-06-22 17:54:573.09 (n/a)2.69 (n/a)2.86 (n/a)2.07 (n/a)0.40 (n/a)253.90 (n/a)198.64 (n/a)183.40 (n/a)169.60 (n/a)33.34 (n/a)

test_transpose[M_2048-N_64-aie_columns_1-channels_2-m_64-n_64-s_8-num_batches_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:533.00 (-5.28%)2.64 (-2.95%)2.74 (-0.26%)2.24 (+3.63%)0.29 (-35.82%)233.80 (-3.51%)200.72 (+1.70%)191.50 (+0.26%)174.90 (+5.62%)22.96 (-32.70%)
573678d — 2026-06-29 18:46:463.17 (n/a)2.72 (n/a)2.74 (n/a)2.16 (n/a)0.46 (n/a)242.30 (n/a)197.36 (n/a)191.00 (n/a)165.60 (n/a)34.11 (n/a)

test_transpose[M_2048-N_64-aie_columns_1-channels_2-m_64-n_64-s_8]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
4bb8427 — 2026-06-23 23:08:224.36 (+7.95%)3.26 (+13.04%)3.33 (+22.37%)2.38 (+18.97%)0.79 (-14.92%)220.70 (-15.96%)168.40 (-14.68%)157.40 (-18.28%)120.30 (-7.39%)40.78 (-34.38%)
4d4b803 — 2026-06-22 17:54:574.04 (n/a)2.89 (n/a)2.72 (n/a)2.00 (n/a)0.93 (n/a)262.60 (n/a)197.38 (n/a)192.60 (n/a)129.90 (n/a)62.14 (n/a)
Krackan - Examples

IRON

Tested on 2026_07_06_20_58_00 at commit a1ee5e5.

iron/applications/llama_3.2_1b
TestChecksTTFT (mean)TPS (mean)
test_llama_3_2_1b[llama_3.2_1b_prompt_1024_tokens_1]✅ 5/51.99n/a
test_llama_3_2_1b[llama_3.2_1b_prompt_1024_tokens_40]✅ 5/52.064.08
test_llama_3_2_1b[llama_3.2_1b_prompt_13_tokens_1]✅ 5/51.89n/a
test_llama_3_2_1b[llama_3.2_1b_prompt_13_tokens_40]✅ 5/51.924.06

Trends:

IRON Trends

iron/applications/llama_3.2_1b

test_llama_3_2_1b[llama_3.2_1b_prompt_1024_tokens_1]

Commit/Date TTFT (max)TTFT (mean)TTFT (median)TTFT (min)TTFT (stddev)
a1ee5e5 — 2026-07-06 20:52:022.11 (-1.36%)1.99 (-5.84%)1.97 (-6.68%)1.95 (-7.18%)0.07 (+362.06%)
573678d — 2026-06-29 18:14:412.14 (n/a)2.12 (n/a)2.11 (n/a)2.10 (n/a)0.01 (n/a)

test_llama_3_2_1b[llama_3.2_1b_prompt_1024_tokens_40]

Commit/Date TPS (max)TPS (mean)TPS (median)TPS (min)TPS (stddev)TTFT (max)TTFT (mean)TTFT (median)TTFT (min)TTFT (stddev)
a1ee5e5 — 2026-07-06 20:52:024.34 (+3.46%)4.08 (-2.33%)4.01 (-3.98%)3.87 (-6.91%)0.20 (+1109.51%)2.16 (-5.14%)2.06 (-4.38%)2.10 (-1.13%)1.94 (-8.51%)0.09 (+30.72%)
573678d — 2026-06-29 18:14:414.20 (n/a)4.17 (n/a)4.17 (n/a)4.16 (n/a)0.02 (n/a)2.28 (n/a)2.16 (n/a)2.13 (n/a)2.12 (n/a)0.07 (n/a)

test_llama_3_2_1b[llama_3.2_1b_prompt_13_tokens_1]

Commit/Date TTFT (max)TTFT (mean)TTFT (median)TTFT (min)TTFT (stddev)
a1ee5e5 — 2026-07-06 20:52:021.92 (-8.75%)1.89 (-9.04%)1.89 (-9.67%)1.88 (-8.52%)0.02 (-7.99%)
573678d — 2026-06-29 18:14:412.10 (n/a)2.08 (n/a)2.09 (n/a)2.05 (n/a)0.02 (n/a)

test_llama_3_2_1b[llama_3.2_1b_prompt_13_tokens_40]

Commit/Date TPS (max)TPS (mean)TPS (median)TPS (min)TPS (stddev)TTFT (max)TTFT (mean)TTFT (median)TTFT (min)TTFT (stddev)
a1ee5e5 — 2026-07-06 20:52:024.32 (+3.52%)4.06 (-2.38%)3.97 (-4.54%)3.84 (-7.59%)0.24 (+2541.48%)1.99 (-4.74%)1.92 (-7.38%)1.90 (-8.12%)1.87 (-8.74%)0.05 (+179.35%)
573678d — 2026-06-29 18:14:414.17 (n/a)4.16 (n/a)4.16 (n/a)4.15 (n/a)0.01 (n/a)2.09 (n/a)2.07 (n/a)2.07 (n/a)2.05 (n/a)0.02 (n/a)
Phoenix - Small

IRON

Tested on 2026_07_06_20_54_46 at commit a1ee5e5.

iron/operators/axpy
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_axpy[input_length_2048-num_aie_columns_1-tile_size_2048-scalar_factor_3.0]✅ 5/5411.600.03n/a
test_axpy[input_length_2048-num_aie_columns_2-tile_size_1024-scalar_factor_3.0]✅ 5/5348.300.04n/a
test_axpy[input_length_2048-num_aie_columns_4-tile_size_512-scalar_factor_3.0]✅ 5/5431.160.03n/a
iron/operators/dequant
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_dequant[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-group_size_32]✅ 5/5294.580.02n/a
test_dequant[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-group_size_32]✅ 5/5269.600.02n/a
test_dequant[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-group_size_32]✅ 5/5268.380.02n/a
test_dequant[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-group_size_32]✅ 5/5266.160.02n/a
test_dequant[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-group_size_32]✅ 5/5291.040.02n/a
test_dequant[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-group_size_32]✅ 5/5262.180.02n/a
iron/operators/elementwise_add
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_elementwise_add[input_length_2048-num_aie_columns_1-tile_size_2048]✅ 5/5286.740.05n/a
test_elementwise_add[input_length_2048-num_aie_columns_2-tile_size_1024]✅ 5/5270.760.05n/a
test_elementwise_add[input_length_2048-num_aie_columns_4-tile_size_512]✅ 5/5343.720.04n/a
iron/operators/elementwise_mul
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_elementwise_mul[input_length_2048-num_aie_columns_1-tile_size_2048]✅ 5/5299.160.05n/a
test_elementwise_mul[input_length_2048-num_aie_columns_2-tile_size_1024]✅ 5/5348.700.04n/a
test_elementwise_mul[input_length_2048-num_aie_columns_4-tile_size_512]✅ 5/5313.100.04n/a
iron/operators/gelu
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_gelu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5383.100.02n/a
test_gelu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5322.500.03n/a
test_gelu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5424.280.02n/a
test_gelu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5443.180.02n/a
test_gelu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5386.080.02n/a
test_gelu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5464.660.02n/a
iron/operators/gemm
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_False-c_col_maj_False-m_48-k_96-n_16-trace_size_0-partition_N_1]✅ 5/5471.240.5021.52
test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_True-c_col_maj_True-m_48-k_96-n_16-trace_size_0-partition_N_1]✅ 5/5591.760.3816.39
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_1-b_col_maj_False-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]✅ 5/584732.840.30202.84
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_2-b_col_maj_True-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]✅ 5/526139.280.96658.18
test_gemm[M_384-K_1536-N_1792-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_32-k_48-n_64-trace_size_0-partition_N_1]✅ 5/53271.462.70707.43
test_gemm[M_64-K_512-N_256-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_16-k_64-n_64-trace_size_0-partition_N_4]✅ 5/56468.380.2111.24
iron/operators/gemv
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_gemv[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128]✅ 5/5n/a0.120.11
test_gemv[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048]✅ 5/5n/a3.573.57
test_gemv[M_2048-K_8192-num_aie_columns_2-tile_size_input_1-tile_size_output_1024]✅ 5/5n/a6.486.47
test_gemv[M_2048-K_8192-num_aie_columns_4-tile_size_input_1-tile_size_output_512]✅ 5/5n/a9.099.09
test_gemv[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024]✅ 5/5n/a3.493.49
test_gemv[M_8192-K_2048-num_aie_columns_2-tile_size_input_4-tile_size_output_1024]✅ 5/5n/a6.716.70
test_gemv[M_8192-K_2048-num_aie_columns_4-tile_size_input_4-tile_size_output_1024]✅ 5/5n/a10.9010.89
iron/operators/layer_norm
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5423.720.02n/a
test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5342.700.03n/a
test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5455.300.02n/a
test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5476.380.02n/a
test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5412.780.02n/a
test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5541.900.02n/a
iron/operators/mem_copy
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_mem_copy[input_length_2048-num_cores_1-num_channels_1-bypass_False-tile_size_2048]✅ 5/5406.680.02n/a
test_mem_copy[input_length_2048-num_cores_2-num_channels_1-bypass_False-tile_size_1024]✅ 5/5327.860.03n/a
test_mem_copy[input_length_2048-num_cores_2-num_channels_2-bypass_False-tile_size_1024]✅ 5/5428.780.02n/a
test_mem_copy[input_length_2048-num_cores_4-num_channels_1-bypass_False-tile_size_512]✅ 5/5375.460.02n/a
test_mem_copy[input_length_2048-num_cores_4-num_channels_2-bypass_False-tile_size_512]✅ 5/5431.080.02n/a
test_mem_copy[input_length_2048-num_cores_8-num_channels_2-bypass_False-tile_size_256]✅ 5/5557.760.02n/a
iron/operators/relu
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5349.560.03n/a
test_relu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5389.880.02n/a
test_relu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5362.120.03n/a
test_relu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5651.520.02n/a
test_relu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5756.400.02n/a
test_relu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5750.020.02n/a
iron/operators/rms_norm
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_False]✅ 5/5418.880.02n/a
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_True]✅ 5/5314.060.04n/a
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_False]✅ 5/5350.620.03n/a
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_True]✅ 5/5433.080.03n/a
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_False]✅ 5/5479.840.02n/a
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_True]✅ 5/5364.180.03n/a
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_False]✅ 5/5483.180.02n/a
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_True]✅ 5/5386.160.03n/a
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_False]✅ 5/5452.540.02n/a
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_True]✅ 5/5430.180.02n/a
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-weighted_False]✅ 5/5417.420.02n/a
iron/operators/rope
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_1-method_type_0]✅ 5/5512.280.20n/a
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_2-method_type_0]✅ 5/5433.820.25n/a
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_4-method_type_0]✅ 5/5445.220.22n/a
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_1-method_type_0]✅ 5/5365.460.21n/a
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_2-method_type_0]✅ 5/5411.520.20n/a
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_4-method_type_0]✅ 5/5596.700.13n/a
iron/operators/sigmoid
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_sigmoid[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5316.180.03n/a
test_sigmoid[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5397.440.02n/a
test_sigmoid[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5391.560.02n/a
test_sigmoid[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5378.260.02n/a
test_sigmoid[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5435.140.02n/a
test_sigmoid[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5472.560.02n/a
iron/operators/silu
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_silu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5323.240.03n/a
test_silu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5332.280.03n/a
test_silu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5494.260.02n/a
iron/operators/softmax
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_1024]✅ 5/5523.320.29n/a
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_2048]✅ 5/5432.560.32n/a
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5454.120.32n/a
iron/operators/swiglu_decode
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_swiglu_decode[embedding_dim_1024-hidden_dim_3584]✅ 5/513160.980.00n/a
test_swiglu_decode[embedding_dim_2048-hidden_dim_2048]✅ 5/511547.310.00n/a
iron/operators/swiglu_prefill
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_swiglu_prefill[seq_len_256-embedding_dim_2048-hidden_dim_2048-prio_accuracy_False]✅ 5/523924.000.09n/a
iron/operators/tanh
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_tanh[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5392.620.02n/a
test_tanh[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5440.660.02n/a
test_tanh[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5444.400.02n/a
test_tanh[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5414.820.02n/a
test_tanh[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5512.640.02n/a
test_tanh[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5424.800.02n/a
iron/operators/transpose
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_1]✅ 5/5428.681.31n/a
test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_2]✅ 5/5529.202.08n/a
test_transpose[M_2048-N_64-aie_columns_1-channels_2-m_64-n_64-s_8-num_batches_1]✅ 5/5486.341.17n/a

Trends:

IRON Trends

iron/operators/axpy

test_axpy[input_length_2048-num_aie_columns_1-tile_size_2048-scalar_factor_3.0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.04 (-25.57%)0.03 (-14.96%)0.03 (-24.35%)0.02 (+5.44%)0.01 (-34.29%)541.60 (-5.17%)411.60 (+11.36%)413.70 (+32.17%)280.80 (+34.35%)119.98 (-18.27%)
573678d — 2026-06-29 18:24:150.06 (n/a)0.04 (n/a)0.04 (n/a)0.02 (n/a)0.01 (n/a)571.10 (n/a)369.60 (n/a)313.00 (n/a)209.00 (n/a)146.79 (n/a)

test_axpy[input_length_2048-num_aie_columns_2-tile_size_1024-scalar_factor_3.0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.05 (-7.31%)0.04 (+16.51%)0.04 (+79.48%)0.02 (-1.78%)0.01 (-21.49%)561.20 (+1.81%)348.30 (-17.87%)290.70 (-44.28%)240.80 (+7.89%)129.47 (-15.32%)
573678d — 2026-06-29 18:24:150.06 (n/a)0.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)551.20 (n/a)424.08 (n/a)521.70 (n/a)223.20 (n/a)152.88 (n/a)

test_axpy[input_length_2048-num_aie_columns_4-tile_size_512-scalar_factor_3.0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.04 (-22.73%)0.03 (-23.22%)0.03 (-40.28%)0.02 (-9.52%)0.01 (-51.85%)600.90 (+10.52%)431.16 (+18.54%)423.60 (+67.43%)308.60 (+29.45%)108.34 (-31.86%)
573678d — 2026-06-29 18:24:150.05 (n/a)0.04 (n/a)0.05 (n/a)0.02 (n/a)0.01 (n/a)543.70 (n/a)363.74 (n/a)253.00 (n/a)238.40 (n/a)159.01 (n/a)
iron/operators/dequant

test_dequant[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.02 (+2.06%)0.02 (+7.12%)0.02 (+11.45%)0.01 (+32.49%)0.00 (-28.17%)374.20 (-24.51%)294.58 (-10.24%)274.70 (-10.26%)241.50 (-2.03%)55.26 (-45.98%)
573678d — 2026-06-29 18:24:150.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.00 (n/a)495.70 (n/a)328.18 (n/a)306.10 (n/a)246.50 (n/a)102.30 (n/a)

test_dequant[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.03 (+34.65%)0.02 (+43.71%)0.02 (+48.86%)0.02 (+92.37%)0.00 (-25.69%)331.10 (-48.01%)269.60 (-36.97%)279.70 (-32.83%)195.80 (-25.72%)48.76 (-70.83%)
573678d — 2026-06-29 18:24:150.02 (n/a)0.01 (n/a)0.01 (n/a)0.01 (n/a)0.01 (n/a)636.90 (n/a)427.76 (n/a)416.40 (n/a)263.60 (n/a)167.14 (n/a)

test_dequant[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.02 (+4.79%)0.02 (+19.07%)0.02 (+25.99%)0.02 (+28.12%)0.00 (-48.48%)288.70 (-21.95%)268.38 (-17.46%)268.70 (-20.62%)240.20 (-4.57%)19.26 (-62.20%)
573678d — 2026-06-29 18:24:150.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.00 (n/a)369.90 (n/a)325.14 (n/a)338.50 (n/a)251.70 (n/a)50.96 (n/a)

test_dequant[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.02 (+13.15%)0.02 (+46.43%)0.02 (+44.05%)0.02 (+74.51%)0.00 (-44.82%)311.90 (-42.70%)266.16 (-35.32%)261.90 (-30.59%)232.30 (-11.61%)30.96 (-72.19%)
573678d — 2026-06-29 18:24:150.02 (n/a)0.01 (n/a)0.01 (n/a)0.01 (n/a)0.00 (n/a)544.30 (n/a)411.50 (n/a)377.30 (n/a)262.80 (n/a)111.30 (n/a)

test_dequant[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.02 (-15.57%)0.02 (+22.98%)0.02 (+38.37%)0.02 (+46.37%)0.00 (-87.02%)304.70 (-31.68%)291.04 (-22.63%)289.00 (-27.71%)280.10 (+18.44%)8.99 (-88.86%)
573678d — 2026-06-29 18:24:150.02 (n/a)0.01 (n/a)0.01 (n/a)0.01 (n/a)0.00 (n/a)446.00 (n/a)376.18 (n/a)399.80 (n/a)236.50 (n/a)80.66 (n/a)

test_dequant[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.03 (+40.80%)0.02 (+37.79%)0.02 (+53.69%)0.02 (+40.96%)0.01 (+19.29%)340.20 (-29.07%)262.18 (-29.47%)270.40 (-34.92%)164.10 (-28.99%)64.75 (-43.23%)
573678d — 2026-06-29 18:24:150.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)0.01 (n/a)479.60 (n/a)371.74 (n/a)415.50 (n/a)231.10 (n/a)114.05 (n/a)
iron/operators/elementwise_add

test_elementwise_add[input_length_2048-num_aie_columns_1-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.07 (n/a)0.05 (n/a)0.05 (n/a)0.03 (n/a)0.02 (n/a)397.00 (n/a)286.74 (n/a)266.20 (n/a)170.40 (n/a)98.62 (n/a)

test_elementwise_add[input_length_2048-num_aie_columns_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.06 (n/a)0.05 (n/a)0.05 (n/a)0.03 (n/a)0.01 (n/a)374.50 (n/a)270.76 (n/a)258.70 (n/a)207.00 (n/a)62.15 (n/a)

test_elementwise_add[input_length_2048-num_aie_columns_4-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.06 (n/a)0.04 (n/a)0.04 (n/a)0.02 (n/a)0.02 (n/a)611.80 (n/a)343.72 (n/a)279.70 (n/a)202.20 (n/a)164.70 (n/a)
iron/operators/elementwise_mul

test_elementwise_mul[input_length_2048-num_aie_columns_1-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.07 (n/a)0.05 (n/a)0.05 (n/a)0.02 (n/a)0.02 (n/a)525.50 (n/a)299.16 (n/a)261.30 (n/a)185.60 (n/a)130.74 (n/a)

test_elementwise_mul[input_length_2048-num_aie_columns_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.05 (n/a)0.04 (n/a)0.04 (n/a)0.03 (n/a)0.01 (n/a)477.10 (n/a)348.70 (n/a)291.60 (n/a)251.90 (n/a)110.98 (n/a)

test_elementwise_mul[input_length_2048-num_aie_columns_4-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.06 (n/a)0.04 (n/a)0.05 (n/a)0.02 (n/a)0.02 (n/a)501.30 (n/a)313.10 (n/a)252.90 (n/a)193.90 (n/a)123.32 (n/a)
iron/operators/gelu

test_gelu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.03 (n/a)0.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)539.90 (n/a)383.10 (n/a)418.70 (n/a)246.00 (n/a)120.34 (n/a)

test_gelu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.03 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)447.20 (n/a)322.50 (n/a)293.20 (n/a)237.50 (n/a)89.27 (n/a)

test_gelu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)590.90 (n/a)424.28 (n/a)433.60 (n/a)249.00 (n/a)154.55 (n/a)

test_gelu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.02 (n/a)0.02 (n/a)0.02 (n/a)0.02 (n/a)0.00 (n/a)457.00 (n/a)443.18 (n/a)447.10 (n/a)419.80 (n/a)15.26 (n/a)

test_gelu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.03 (n/a)0.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)541.90 (n/a)386.08 (n/a)386.90 (n/a)249.00 (n/a)126.40 (n/a)

test_gelu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)625.30 (n/a)464.66 (n/a)462.00 (n/a)247.60 (n/a)143.11 (n/a)
iron/operators/gemm

test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_False-c_col_maj_False-m_48-k_96-n_16-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
a1ee5e5 — 2026-07-06 20:51:500.68 (+10.46%)0.50 (+21.36%)0.52 (+30.79%)0.35 (+99.55%)0.15 (-10.08%)624.10 (-49.88%)471.24 (-25.89%)424.60 (-23.55%)324.50 (-9.46%)141.83 (-59.77%)29.08 (+10.46%)21.52 (+21.36%)22.23 (+30.79%)15.12 (+99.55%)6.27 (-10.08%)
573678d — 2026-06-29 18:24:150.62 (n/a)0.42 (n/a)0.40 (n/a)0.18 (n/a)0.16 (n/a)1245.30 (n/a)635.90 (n/a)555.40 (n/a)358.40 (n/a)352.52 (n/a)26.33 (n/a)17.73 (n/a)16.99 (n/a)7.58 (n/a)6.98 (n/a)

test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_True-c_col_maj_True-m_48-k_96-n_16-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
a1ee5e5 — 2026-07-06 20:51:500.52 (-7.06%)0.38 (-3.43%)0.35 (+1.51%)0.33 (+67.25%)0.08 (-48.83%)677.10 (-40.21%)591.76 (-7.95%)623.80 (-1.48%)425.10 (+7.59%)98.14 (-67.09%)22.20 (-7.06%)16.39 (-3.43%)15.13 (+1.51%)13.94 (+67.25%)3.33 (-48.83%)
573678d — 2026-06-29 18:24:150.56 (n/a)0.40 (n/a)0.35 (n/a)0.20 (n/a)0.15 (n/a)1132.40 (n/a)642.84 (n/a)633.20 (n/a)395.10 (n/a)298.25 (n/a)23.88 (n/a)16.97 (n/a)14.90 (n/a)8.33 (n/a)6.51 (n/a)

test_gemm[M_2048-K_2048-N_2048-num_aie_columns_1-b_col_maj_False-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
a1ee5e5 — 2026-07-06 20:51:500.31 (+0.35%)0.30 (-0.30%)0.30 (+0.31%)0.29 (-1.11%)0.01 (+16.36%)86875.00 (+1.12%)84732.84 (+0.32%)84892.30 (-0.31%)81910.10 (-0.35%)1969.97 (+17.05%)209.74 (+0.35%)202.84 (-0.30%)202.37 (+0.31%)197.75 (-1.11%)4.76 (+16.36%)
573678d — 2026-06-29 18:24:150.31 (n/a)0.30 (n/a)0.30 (n/a)0.29 (n/a)0.01 (n/a)85909.50 (n/a)84466.00 (n/a)85155.20 (n/a)82194.40 (n/a)1682.98 (n/a)209.02 (n/a)203.46 (n/a)201.75 (n/a)199.98 (n/a)4.09 (n/a)

test_gemm[M_2048-K_2048-N_2048-num_aie_columns_2-b_col_maj_True-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
a1ee5e5 — 2026-07-06 20:51:501.01 (-2.93%)0.96 (-4.99%)0.95 (-7.95%)0.92 (-2.27%)0.04 (-4.21%)27420.70 (+2.32%)26139.28 (+5.25%)26453.90 (+8.63%)24961.60 (+3.02%)1100.39 (-0.20%)688.25 (-2.93%)658.18 (-4.99%)649.43 (-7.95%)626.53 (-2.27%)27.85 (-4.21%)
573678d — 2026-06-29 18:24:151.04 (n/a)1.01 (n/a)1.03 (n/a)0.94 (n/a)0.04 (n/a)26798.50 (n/a)24836.36 (n/a)24352.00 (n/a)24229.80 (n/a)1102.58 (n/a)709.04 (n/a)692.75 (n/a)705.48 (n/a)641.08 (n/a)29.07 (n/a)

test_gemm[M_384-K_1536-N_1792-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_32-k_48-n_64-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
a1ee5e5 — 2026-07-06 20:51:503.76 (-4.86%)2.70 (-12.08%)2.42 (-28.98%)1.79 (+2.65%)0.91 (+6.56%)4500.10 (-2.58%)3271.46 (+14.66%)3325.50 (+40.82%)2143.60 (+5.10%)1062.64 (+2.32%)986.17 (-4.86%)707.43 (-12.08%)635.66 (-28.98%)469.76 (+2.65%)238.70 (+6.56%)
573678d — 2026-06-29 18:24:153.95 (n/a)3.07 (n/a)3.41 (n/a)1.75 (n/a)0.85 (n/a)4619.20 (n/a)2853.28 (n/a)2361.60 (n/a)2039.50 (n/a)1038.58 (n/a)1036.52 (n/a)804.63 (n/a)895.11 (n/a)457.64 (n/a)224.01 (n/a)

test_gemm[M_64-K_512-N_256-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_16-k_64-n_64-trace_size_0-partition_N_4]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
a1ee5e5 — 2026-07-06 20:51:500.34 (+6.21%)0.21 (-17.71%)0.19 (-34.18%)0.14 (-20.90%)0.07 (+18.55%)8768.80 (+26.42%)6468.38 (+24.71%)6560.20 (+51.94%)3708.40 (-5.85%)1820.09 (+30.28%)18.10 (+6.21%)11.24 (-17.71%)10.23 (-34.18%)7.65 (-20.90%)4.00 (+18.55%)
573678d — 2026-06-29 18:24:150.32 (n/a)0.25 (n/a)0.29 (n/a)0.18 (n/a)0.06 (n/a)6936.30 (n/a)5186.80 (n/a)4317.60 (n/a)3938.80 (n/a)1397.04 (n/a)17.04 (n/a)13.66 (n/a)15.54 (n/a)9.68 (n/a)3.37 (n/a)
iron/operators/gemv

test_gemv[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
a1ee5e5 — 2026-07-06 20:51:500.15 (+8.91%)0.12 (+9.49%)0.11 (-3.74%)0.07 (+8.53%)0.03 (+16.83%)0.15 (+8.91%)0.11 (+9.49%)0.11 (-3.74%)0.07 (+8.53%)0.03 (+16.83%)
573678d — 2026-06-29 18:24:150.14 (n/a)0.11 (n/a)0.11 (n/a)0.06 (n/a)0.03 (n/a)0.13 (n/a)0.10 (n/a)0.11 (n/a)0.06 (n/a)0.03 (n/a)

test_gemv[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
a1ee5e5 — 2026-07-06 20:51:503.94 (-0.89%)3.57 (-3.12%)3.46 (-5.91%)3.35 (-3.04%)0.26 (+38.91%)3.93 (-0.89%)3.57 (-3.12%)3.46 (-5.91%)3.35 (-3.04%)0.26 (+38.91%)
573678d — 2026-06-29 18:24:153.97 (n/a)3.69 (n/a)3.67 (n/a)3.45 (n/a)0.18 (n/a)3.97 (n/a)3.69 (n/a)3.67 (n/a)3.45 (n/a)0.18 (n/a)

test_gemv[M_2048-K_8192-num_aie_columns_2-tile_size_input_1-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
a1ee5e5 — 2026-07-06 20:51:507.62 (-1.19%)6.48 (+4.63%)6.34 (+12.68%)5.67 (+14.16%)0.72 (-36.94%)7.62 (-1.19%)6.47 (+4.63%)6.34 (+12.68%)5.66 (+14.16%)0.72 (-36.94%)
573678d — 2026-06-29 18:24:157.72 (n/a)6.19 (n/a)5.63 (n/a)4.96 (n/a)1.14 (n/a)7.71 (n/a)6.19 (n/a)5.62 (n/a)4.96 (n/a)1.14 (n/a)

test_gemv[M_2048-K_8192-num_aie_columns_4-tile_size_input_1-tile_size_output_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
a1ee5e5 — 2026-07-06 20:51:5012.99 (+33.87%)9.09 (+13.97%)8.45 (+13.84%)7.17 (-1.09%)2.25 (+117.12%)12.99 (+33.87%)9.09 (+13.97%)8.45 (+13.84%)7.17 (-1.09%)2.25 (+117.12%)
573678d — 2026-06-29 18:24:159.71 (n/a)7.98 (n/a)7.43 (n/a)7.25 (n/a)1.04 (n/a)9.70 (n/a)7.98 (n/a)7.42 (n/a)7.25 (n/a)1.04 (n/a)

test_gemv[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
a1ee5e5 — 2026-07-06 20:51:503.85 (-2.56%)3.49 (-7.19%)3.65 (-4.33%)2.87 (-17.64%)0.39 (+114.92%)3.85 (-2.56%)3.49 (-7.19%)3.65 (-4.33%)2.87 (-17.64%)0.39 (+114.92%)
573678d — 2026-06-29 18:24:153.95 (n/a)3.77 (n/a)3.82 (n/a)3.49 (n/a)0.18 (n/a)3.95 (n/a)3.76 (n/a)3.81 (n/a)3.49 (n/a)0.18 (n/a)

test_gemv[M_8192-K_2048-num_aie_columns_2-tile_size_input_4-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
a1ee5e5 — 2026-07-06 20:51:507.44 (-1.71%)6.71 (+1.21%)6.79 (-6.96%)5.70 (+24.48%)0.75 (-41.44%)7.44 (-1.71%)6.70 (+1.21%)6.78 (-6.96%)5.70 (+24.48%)0.75 (-41.44%)
573678d — 2026-06-29 18:24:157.57 (n/a)6.63 (n/a)7.29 (n/a)4.58 (n/a)1.29 (n/a)7.57 (n/a)6.62 (n/a)7.29 (n/a)4.58 (n/a)1.28 (n/a)

test_gemv[M_8192-K_2048-num_aie_columns_4-tile_size_input_4-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
a1ee5e5 — 2026-07-06 20:51:5014.07 (+2.54%)10.90 (+7.79%)9.21 (-0.24%)8.41 (-1.99%)2.81 (+33.51%)14.06 (+2.54%)10.89 (+7.79%)9.20 (-0.24%)8.41 (-1.99%)2.81 (+33.51%)
573678d — 2026-06-29 18:24:1513.72 (n/a)10.11 (n/a)9.23 (n/a)8.58 (n/a)2.10 (n/a)13.71 (n/a)10.10 (n/a)9.22 (n/a)8.58 (n/a)2.10 (n/a)
iron/operators/layer_norm

test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)555.50 (n/a)423.72 (n/a)464.30 (n/a)274.20 (n/a)112.75 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.03 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)454.60 (n/a)342.70 (n/a)300.80 (n/a)266.60 (n/a)87.69 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)642.90 (n/a)455.30 (n/a)471.60 (n/a)302.70 (n/a)134.45 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.00 (n/a)558.10 (n/a)476.38 (n/a)471.50 (n/a)391.30 (n/a)63.02 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.03 (n/a)0.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)537.20 (n/a)412.78 (n/a)478.80 (n/a)239.10 (n/a)129.14 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.03 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)0.01 (n/a)797.60 (n/a)541.90 (n/a)552.00 (n/a)254.60 (n/a)207.71 (n/a)
iron/operators/mem_copy

test_mem_copy[input_length_2048-num_cores_1-num_channels_1-bypass_False-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.03 (-45.05%)0.02 (-22.37%)0.02 (-10.02%)0.02 (-23.16%)0.01 (-51.81%)539.60 (+30.12%)406.68 (+20.58%)436.20 (+11.13%)263.50 (+81.98%)130.06 (+16.06%)
573678d — 2026-06-29 18:24:150.06 (n/a)0.03 (n/a)0.02 (n/a)0.02 (n/a)0.02 (n/a)414.70 (n/a)337.28 (n/a)392.50 (n/a)144.80 (n/a)112.07 (n/a)

test_mem_copy[input_length_2048-num_cores_2-num_channels_1-bypass_False-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.04 (-14.05%)0.03 (-14.85%)0.03 (-8.12%)0.02 (-7.58%)0.01 (-16.48%)468.40 (+8.20%)327.86 (+16.38%)292.60 (+8.85%)232.00 (+16.35%)91.17 (+2.13%)
573678d — 2026-06-29 18:24:150.04 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)432.90 (n/a)281.72 (n/a)268.80 (n/a)199.40 (n/a)89.26 (n/a)

test_mem_copy[input_length_2048-num_cores_2-num_channels_2-bypass_False-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.03 (-10.93%)0.02 (-5.35%)0.02 (+10.44%)0.01 (-15.35%)0.01 (-6.82%)589.30 (+18.12%)428.78 (+7.62%)381.10 (-9.46%)256.10 (+12.28%)140.34 (+37.33%)
573678d — 2026-06-29 18:24:150.04 (n/a)0.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)498.90 (n/a)398.42 (n/a)420.90 (n/a)228.10 (n/a)102.19 (n/a)

test_mem_copy[input_length_2048-num_cores_4-num_channels_1-bypass_False-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.03 (-10.75%)0.02 (+12.88%)0.02 (+26.75%)0.02 (+173.08%)0.01 (-54.61%)500.20 (-63.38%)375.46 (-36.74%)372.90 (-21.11%)274.60 (+12.04%)88.92 (-80.62%)
573678d — 2026-06-29 18:24:150.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)1365.90 (n/a)593.54 (n/a)472.70 (n/a)245.10 (n/a)458.86 (n/a)

test_mem_copy[input_length_2048-num_cores_4-num_channels_2-bypass_False-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.03 (+10.22%)0.02 (-4.40%)0.02 (-13.42%)0.01 (-3.28%)0.01 (+25.53%)603.00 (+3.40%)431.08 (+9.27%)423.90 (+15.50%)242.90 (-9.26%)162.59 (+24.34%)
573678d — 2026-06-29 18:24:150.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)583.20 (n/a)394.52 (n/a)367.00 (n/a)267.70 (n/a)130.76 (n/a)

test_mem_copy[input_length_2048-num_cores_8-num_channels_2-bypass_False-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.04 (+43.69%)0.02 (+8.58%)0.02 (-3.35%)0.01 (-37.54%)0.01 (+99.93%)1049.00 (+60.10%)557.76 (+10.16%)494.30 (+3.47%)215.00 (-30.40%)305.89 (+110.18%)
573678d — 2026-06-29 18:24:150.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)655.20 (n/a)506.34 (n/a)477.70 (n/a)308.90 (n/a)145.54 (n/a)
iron/operators/rms_norm

test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.03 (+14.70%)0.02 (+17.16%)0.03 (+60.11%)0.01 (+6.48%)0.01 (+31.84%)632.20 (-6.08%)418.88 (-9.47%)308.00 (-37.54%)240.50 (-12.80%)192.69 (+20.81%)
573678d — 2026-06-29 18:24:150.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)673.10 (n/a)462.70 (n/a)493.10 (n/a)275.80 (n/a)159.50 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.06 (+6.34%)0.04 (+19.20%)0.05 (+26.23%)0.03 (+29.37%)0.01 (-0.16%)458.20 (-22.69%)314.06 (-18.65%)265.50 (-20.79%)220.10 (-5.98%)106.07 (-28.85%)
573678d — 2026-06-29 18:24:150.05 (n/a)0.04 (n/a)0.04 (n/a)0.02 (n/a)0.01 (n/a)592.70 (n/a)386.04 (n/a)335.20 (n/a)234.10 (n/a)149.09 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.03 (+0.66%)0.03 (+3.32%)0.03 (+5.92%)0.02 (+3.65%)0.01 (+4.33%)494.40 (-3.53%)350.62 (-2.85%)279.00 (-5.58%)245.40 (-0.65%)116.39 (-0.14%)
573678d — 2026-06-29 18:24:150.03 (n/a)0.02 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)512.50 (n/a)360.92 (n/a)295.50 (n/a)247.00 (n/a)116.55 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.04 (+15.72%)0.03 (+12.61%)0.02 (+13.49%)0.02 (+41.33%)0.01 (+6.11%)577.70 (-29.25%)433.08 (-14.95%)458.80 (-11.89%)231.20 (-13.60%)129.11 (-38.01%)
573678d — 2026-06-29 18:24:150.04 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)816.50 (n/a)509.18 (n/a)520.70 (n/a)267.60 (n/a)208.27 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.04 (+16.64%)0.02 (-13.37%)0.01 (-46.32%)0.01 (+323.51%)0.01 (-8.77%)594.80 (-76.39%)479.84 (-35.71%)548.00 (+86.27%)208.40 (-14.27%)156.08 (-84.28%)
573678d — 2026-06-29 18:24:150.03 (n/a)0.02 (n/a)0.03 (n/a)0.00 (n/a)0.01 (n/a)2519.00 (n/a)746.32 (n/a)294.20 (n/a)243.10 (n/a)992.55 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.05 (+38.09%)0.03 (+19.89%)0.04 (+14.72%)0.02 (+13.17%)0.02 (+43.85%)595.80 (-11.64%)364.18 (-12.57%)265.00 (-12.83%)189.60 (-27.61%)178.42 (-2.89%)
573678d — 2026-06-29 18:24:150.04 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)674.30 (n/a)416.56 (n/a)304.00 (n/a)261.90 (n/a)183.72 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.03 (-28.11%)0.02 (-16.34%)0.02 (-9.83%)0.01 (-12.80%)0.01 (-22.35%)669.60 (+14.68%)483.18 (+19.52%)460.90 (+10.90%)322.50 (+39.13%)165.38 (+24.78%)
573678d — 2026-06-29 18:24:150.04 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)583.90 (n/a)404.28 (n/a)415.60 (n/a)231.80 (n/a)132.53 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.04 (+33.13%)0.03 (+26.83%)0.02 (+13.31%)0.02 (+25.46%)0.01 (+62.85%)554.20 (-20.28%)386.16 (-19.13%)386.30 (-11.74%)262.80 (-24.87%)121.37 (-8.83%)
573678d — 2026-06-29 18:24:150.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.00 (n/a)695.20 (n/a)477.50 (n/a)437.70 (n/a)349.80 (n/a)133.12 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.04 (+11.92%)0.02 (-16.43%)0.02 (-41.22%)0.01 (-12.92%)0.01 (+21.73%)599.60 (+14.84%)452.54 (+24.15%)498.40 (+70.16%)231.20 (-10.66%)151.66 (+23.14%)
573678d — 2026-06-29 18:24:150.03 (n/a)0.02 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)522.10 (n/a)364.50 (n/a)292.90 (n/a)258.80 (n/a)123.16 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.03 (-7.58%)0.02 (+9.67%)0.03 (+51.48%)0.01 (-10.59%)0.01 (+11.23%)660.60 (+11.85%)430.18 (-4.11%)319.50 (-33.97%)271.50 (+8.21%)191.85 (+36.73%)
573678d — 2026-06-29 18:24:150.04 (n/a)0.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)590.60 (n/a)448.60 (n/a)483.90 (n/a)250.90 (n/a)140.32 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.04 (+73.93%)0.02 (+47.06%)0.02 (+7.15%)0.01 (+335.76%)0.01 (+45.09%)563.10 (-77.05%)417.42 (-51.20%)476.30 (-6.68%)205.60 (-42.52%)151.55 (-83.09%)
573678d — 2026-06-29 18:24:150.02 (n/a)0.02 (n/a)0.02 (n/a)0.00 (n/a)0.01 (n/a)2453.70 (n/a)855.38 (n/a)510.40 (n/a)357.70 (n/a)895.94 (n/a)
iron/operators/rope

test_rope[rows_32-cols_512-angle_rows_32-aie_columns_1-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.25 (-44.00%)0.20 (-18.92%)0.18 (-24.61%)0.16 (+203.72%)0.04 (-76.04%)632.90 (-67.08%)512.28 (-26.93%)534.90 (+32.63%)392.80 (+78.55%)91.62 (-86.95%)
573678d — 2026-06-29 18:24:150.45 (n/a)0.24 (n/a)0.24 (n/a)0.05 (n/a)0.15 (n/a)1922.40 (n/a)701.04 (n/a)403.30 (n/a)220.00 (n/a)702.07 (n/a)

test_rope[rows_32-cols_512-angle_rows_32-aie_columns_2-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.37 (-8.45%)0.25 (-2.29%)0.21 (+5.59%)0.16 (+0.06%)0.09 (-20.40%)625.00 (-0.06%)433.82 (-2.60%)460.40 (-5.29%)267.30 (+9.19%)151.67 (-15.86%)
573678d — 2026-06-29 18:24:150.40 (n/a)0.26 (n/a)0.20 (n/a)0.16 (n/a)0.12 (n/a)625.40 (n/a)445.40 (n/a)486.10 (n/a)244.80 (n/a)180.26 (n/a)

test_rope[rows_32-cols_512-angle_rows_32-aie_columns_4-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.27 (-36.82%)0.22 (-6.50%)0.21 (+11.46%)0.21 (+318.19%)0.03 (-81.62%)478.80 (-76.09%)445.22 (-38.60%)471.40 (-10.28%)360.00 (+58.31%)49.72 (-93.20%)
573678d — 2026-06-29 18:24:150.43 (n/a)0.24 (n/a)0.19 (n/a)0.05 (n/a)0.16 (n/a)2002.40 (n/a)725.14 (n/a)525.40 (n/a)227.40 (n/a)731.54 (n/a)

test_rope[rows_32-cols_512-angle_rows_8-aie_columns_1-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.27 (+46.76%)0.21 (+38.01%)0.20 (+37.24%)0.17 (+45.99%)0.04 (+49.88%)438.40 (-31.49%)365.46 (-27.38%)377.80 (-27.12%)277.60 (-31.86%)68.03 (-28.06%)
573678d — 2026-06-29 18:24:150.18 (n/a)0.15 (n/a)0.14 (n/a)0.12 (n/a)0.03 (n/a)639.90 (n/a)503.24 (n/a)518.40 (n/a)407.40 (n/a)94.56 (n/a)

test_rope[rows_32-cols_512-angle_rows_8-aie_columns_2-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.33 (+121.46%)0.20 (+67.55%)0.16 (+30.49%)0.14 (+96.60%)0.08 (+170.64%)540.80 (-49.13%)411.52 (-37.91%)463.70 (-23.37%)221.50 (-54.84%)134.64 (-41.20%)
573678d — 2026-06-29 18:24:150.15 (n/a)0.12 (n/a)0.12 (n/a)0.07 (n/a)0.03 (n/a)1063.20 (n/a)662.82 (n/a)605.10 (n/a)490.50 (n/a)228.99 (n/a)

test_rope[rows_32-cols_512-angle_rows_8-aie_columns_4-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.16 (-50.40%)0.13 (-38.42%)0.13 (-11.97%)0.08 (-42.49%)0.03 (-64.40%)869.20 (+73.91%)596.70 (+51.95%)550.70 (+13.59%)463.60 (+101.65%)165.63 (+21.10%)
573678d — 2026-06-29 18:24:150.32 (n/a)0.21 (n/a)0.15 (n/a)0.15 (n/a)0.09 (n/a)499.80 (n/a)392.70 (n/a)484.80 (n/a)229.90 (n/a)136.77 (n/a)
iron/operators/softmax

test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.54 (+12.84%)0.29 (+20.74%)0.22 (+6.06%)0.18 (+242.58%)0.15 (-5.02%)712.10 (-70.81%)523.32 (-41.84%)591.30 (-5.71%)241.70 (-11.40%)182.42 (-79.11%)
573678d — 2026-06-29 18:24:150.48 (n/a)0.24 (n/a)0.21 (n/a)0.05 (n/a)0.15 (n/a)2439.40 (n/a)899.78 (n/a)627.10 (n/a)272.80 (n/a)873.13 (n/a)

test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.39 (-19.13%)0.32 (-3.45%)0.32 (+18.81%)0.24 (+0.53%)0.08 (-31.63%)546.10 (-0.53%)432.56 (+0.09%)407.40 (-15.83%)333.20 (+23.64%)105.51 (-16.92%)
573678d — 2026-06-29 18:24:150.49 (n/a)0.33 (n/a)0.27 (n/a)0.24 (n/a)0.11 (n/a)549.00 (n/a)432.16 (n/a)484.00 (n/a)269.50 (n/a)126.99 (n/a)

test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.49 (-6.35%)0.32 (-9.01%)0.26 (-30.75%)0.21 (+5.45%)0.11 (-16.95%)614.70 (-5.17%)454.12 (+4.96%)507.80 (+44.38%)268.10 (+6.77%)140.71 (-21.91%)
573678d — 2026-06-29 18:24:150.52 (n/a)0.35 (n/a)0.37 (n/a)0.20 (n/a)0.14 (n/a)648.20 (n/a)432.66 (n/a)351.70 (n/a)251.10 (n/a)180.18 (n/a)
iron/operators/swiglu_decode

test_swiglu_decode[embedding_dim_1024-hidden_dim_3584]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.00 (+100.00%)0.00 (+46.15%)0.00 (+50.00%)0.00 (+0.00%)0.00 (+166.93%)19484.76 (+0.81%)13160.98 (-17.58%)13714.49 (-18.83%)5301.73 (-51.43%)5067.23 (+57.36%)
573678d — 2026-06-29 18:24:150.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)19328.08 (n/a)15968.12 (n/a)16896.51 (n/a)10916.30 (n/a)3220.18 (n/a)

test_swiglu_decode[embedding_dim_2048-hidden_dim_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.00 (+0.00%)0.00 (-25.86%)0.00 (-30.77%)0.00 (-33.33%)0.00 (+28.97%)19292.62 (+37.01%)11547.31 (+47.50%)8695.40 (+38.30%)6257.51 (+2.25%)6006.30 (+71.71%)
573678d — 2026-06-29 18:24:150.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)14081.45 (n/a)7828.86 (n/a)6287.46 (n/a)6119.72 (n/a)3497.88 (n/a)
iron/operators/swiglu_prefill

test_swiglu_prefill[seq_len_256-embedding_dim_2048-hidden_dim_2048-prio_accuracy_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:500.12 (-15.11%)0.09 (-3.94%)0.09 (+7.94%)0.07 (-7.26%)0.02 (-28.69%)28294.05 (+7.82%)23924.00 (+2.65%)23376.15 (-7.32%)17865.88 (+17.82%)4393.68 (-5.30%)
573678d — 2026-06-29 18:24:150.14 (n/a)0.09 (n/a)0.08 (n/a)0.08 (n/a)0.02 (n/a)26241.28 (n/a)23305.43 (n/a)25221.66 (n/a)15163.43 (n/a)4639.82 (n/a)
iron/operators/transpose

test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:501.58 (-3.83%)1.31 (+3.52%)1.50 (+1.25%)0.78 (-2.04%)0.34 (-12.37%)668.50 (+2.08%)428.68 (-4.99%)348.40 (-1.25%)332.10 (+3.98%)142.24 (-7.83%)
573678d — 2026-06-29 18:24:151.64 (n/a)1.27 (n/a)1.49 (n/a)0.80 (n/a)0.38 (n/a)654.90 (n/a)451.18 (n/a)352.80 (n/a)319.40 (n/a)154.32 (n/a)

test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_2]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:502.57 (+4.62%)2.08 (+40.55%)2.08 (+66.40%)1.37 (+365.49%)0.48 (-46.15%)763.50 (-78.52%)529.20 (-57.38%)504.30 (-39.91%)408.20 (-4.40%)142.83 (-89.11%)
573678d — 2026-06-29 18:24:152.46 (n/a)1.48 (n/a)1.25 (n/a)0.30 (n/a)0.89 (n/a)3554.20 (n/a)1241.76 (n/a)839.20 (n/a)427.00 (n/a)1311.77 (n/a)

test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
4bb8427 — 2026-06-25 20:01:371.58 (-2.67%)1.21 (+5.65%)1.29 (+41.89%)0.66 (-20.05%)0.38 (-2.53%)792.90 (+25.08%)479.12 (-3.80%)406.80 (-29.52%)331.10 (+2.76%)189.18 (+26.62%)
4bb8427 — 2026-06-23 22:46:491.63 (n/a)1.14 (n/a)0.91 (n/a)0.83 (n/a)0.38 (n/a)633.90 (n/a)498.02 (n/a)577.20 (n/a)322.20 (n/a)149.40 (n/a)

test_transpose[M_2048-N_64-aie_columns_1-channels_2-m_64-n_64-s_8-num_batches_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
a1ee5e5 — 2026-07-06 20:51:501.60 (-20.84%)1.17 (-15.66%)1.05 (-29.15%)0.75 (+0.61%)0.37 (-23.19%)695.80 (-0.61%)486.34 (+14.74%)499.60 (+41.13%)326.90 (+26.31%)154.62 (-10.66%)
573678d — 2026-06-29 18:24:152.03 (n/a)1.39 (n/a)1.48 (n/a)0.75 (n/a)0.49 (n/a)700.10 (n/a)423.86 (n/a)354.00 (n/a)258.80 (n/a)173.06 (n/a)

test_transpose[M_2048-N_64-aie_columns_1-channels_2-m_64-n_64-s_8]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
4bb8427 — 2026-06-25 20:01:371.75 (-15.44%)1.32 (-0.50%)1.36 (+22.47%)0.77 (-19.20%)0.36 (-21.75%)678.10 (+23.76%)430.02 (-0.09%)385.10 (-18.34%)300.10 (+18.29%)146.96 (+19.46%)
4bb8427 — 2026-06-23 22:46:492.07 (n/a)1.32 (n/a)1.11 (n/a)0.96 (n/a)0.46 (n/a)547.90 (n/a)430.40 (n/a)471.60 (n/a)253.70 (n/a)123.02 (n/a)
Phoenix - Examples

IRON

Tested on 2026_07_06_20_52_10 at commit a1ee5e5.

Trends:

IRON Trends

andrej
andrej requested changes Jun 23, 2026
View reviewed changes
Comment thread iron/operators/swiglu_prefill_stream/test.py
@asyms
asyms force-pushed the stream-dse-fused-swiglu branch from 96bbfd8 to 8e4c6c4 Compare June 29, 2026 13:54
@asyms
asyms force-pushed the stream-dse-fused-swiglu branch from 0a900fb to 5334fb5 Compare July 15, 2026 23:22
@asyms

asyms commented Jul 16, 2026

Copy link
Copy Markdown
Author

@andrej Could you re-run the workflows? I have rebased on top of devel and fixed the timeout issue (I think).

andrej
andrej reviewed Jul 24, 2026
View reviewed changes
Comment thread .github/actions/prereqs/action.yaml Outdated
Comment on lines +25 to +28
# stream-dse's AIE codegen deps (snax-mlir/snaxc, xdsl-aie, aie-python-extras)
# are not PyPI dependencies; this console script installs them so the
# stream-backed operators can generate their MLIR at build time.
stream-setup-aie

@andrej andrej Jul 24, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this might potentially be the source of CI failures. If stream-setup-aie installs aie-python-extras, it might overwrite the version of those extras installed by the mlir-aie wheel (which is now called EUDSL I believe).

@asyms
asyms force-pushed the stream-dse-fused-swiglu branch 2 times, most recently from b589a5c to 8170f70 Compare July 24, 2026 20:26
asyms added 5 commits July 24, 2026 22:35
@asyms
iron/common: forward full-ELF aiecc flags
1fc0d9e 
FullElfArtifact/OperatorSequence gain an extra_flags list forwarded into the
aiecc --generate-full-elf command (mirrors XclbinArtifact), so placed/routed
whole-array designs can pass --dynamic-objFifos and not overflow AIE2p program
memory; empty by default, so other sequences are unaffected.
@asyms
iron/common: add TiledStridedLayout for tiled-strided operand layouts
b66f3d7 
Small IRON-side tiled-strided layout (Stride / TiledStride / TiledStridedLayout + tiled_2d) with a to_snaxc() bridge, used to author kernel operand layouts for stream-dse-backed operators.
@asyms
ci: package stream-dse for stream-backed operators
6fb2417 
Optional stream-dse dependency set (requirements_stream.txt) plus the CI bits to build stream-backed operators: run stream-setup-aie in prereqs, install graphviz in the docker image (stream-dse codegen shells out to dot), and tolerate a missing GitHub token in test_docker_ci.sh.
@asyms
swiglu_prefill_stream: stream-dse-backed SwiGLU-prefill operator (k=1…
85e5208 
… and k=2)

Fused SwiGLU-prefill whose MLIR is generated by stream-dse and deployed as a single full-ELF via OperatorSequence. Two variants share one _SwiGLUStreamGroup child: SwiGLUPrefillStream fuses the whole block into one design; SwiGLUPrefillStreamK2 splits it into a gate/up/SiLU/mul group and a down-projection group, with the hidden state h kept on device between them. Passes --dynamic-objFifos so the whole-array design does not overflow program memory. Verified on npu2 (Strix).
@asyms
ci: require stream-dse>=1.13.7 so stream-setup-aie stops clobbering m…
0a2151b 
…lir_aie

stream-setup-aie (from stream-dse) runs after requirements.txt in the prereqs
action and, up to 1.13.6, unconditionally reinstalled an older pinned
mlir_aie/llvm-aie, downgrading the wheels requirements.txt had just installed
(mlir_aie 1.3.5 -> 0.0.1, numpy 2.x -> 1.26.4). Every operator test then failed
against the stale bindings (AnyComputeTile.col, Worker(tile=...),
ObjectFifoHandle.split(tile=...), aie.iron.ScratchpadParameter): 525 failures.

stream-dse 1.13.7 makes stream-setup-aie install only the pure-Python codegen
dialects (xdsl-aie, snax-mlir) and leaves mlir_aie/llvm-aie to requirements.txt.
Bump the pin and correct the now-inaccurate comments/README.
@asyms
asyms force-pushed the stream-dse-fused-swiglu branch from 8170f70 to 0a2151b Compare July 24, 2026 20:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@andrej andrej andrej requested changes
@jgmelber jgmelber Awaiting requested review from jgmelber jgmelber is a code owner
@hunhoffe hunhoffe Awaiting requested review from hunhoffe hunhoffe is a code owner

Requested changes must be addressed to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@asyms @andrej