[Regression][Jetson AGX Thor] Layer-0 V tensor corruption in v0.9.1; v0.4.0 works correctly on the same JetPack 7.1 system

[mengdxiran]

The same Jetson AGX Thor system running JetPack 7.1 produces
correct results with TensorRT Edge-LLM v0.4.0.

TensorRT Edge-LLM v0.9.1 reproduces the layer-0 V-cache corruption
on the same device and software stack.

This suggests an Edge-LLM regression introduced after v0.4.0,
rather than a general JetPack 7.1 compatibility issue.

I am seeing a reproducible numerical correctness issue on Jetson AGX Thor with TensorRT Edge-LLM v0.9.1.

The issue was initially observed as invalid LLM output, but layer-by-layer validation isolates the first divergence to:

decoder layer 0
V projection → AttentionPlugin → V KV cache

The V projection itself is correct. However, in the full graph, the V cache is severely corrupted.

Most importantly, if the V projection tensor is marked as an additional ONNX graph output, the issue completely disappears.

This suggests a possible TensorRT/Myelin tensor lifetime, buffer reuse, or materialization issue around the V tensor.

Environment
Device: Jetson AGX Thor
Jetson Linux: R38.4
JetPack: 7.1
CUDA: 13.0.48
TensorRT: 10.13.3.9
TensorRT Edge-LLM: v0.9.1
Commit: 7f061f2
Target: jetson-thor
Architecture artifact: aarch64/sm_110
ENABLE_CUTE_DSL=ALL

The repository and submodules are complete.

The problem is reproducible in FP16, so NVFP4 quantization is not the root cause.

The original Hugging Face/PyTorch model runs correctly on the same Thor device.

First numerical divergence

Layer 0 V cache:

cosine = 0.015065519
max_abs = 13.1431694
mean_abs = 0.255735815
shape = [8, 22, 128]
dtype = FP16 vs FP16
NaN/Inf = 0 / 0

Layer 0 K cache from the same execution is essentially correct:

cosine ≈ 1.0
V projection isolation

Testing layer 0 V projection independently:

TensorRT vs PyTorch

cosine = 0.999999881
max_abs = 0.00390625
mean_abs = 0.000058087

So the V projection weights/GEMM themselves appear correct.

Critical observation
Normal full graph

Without exposing the V intermediate tensor:

V cache vs PyTorch V projection

cosine = 0.015077175
Mark V projection as graph output

If ONNX linear_2 (V projection) is marked as an additional graph output:

TensorRT V projection vs PyTorch:

cosine = 0.999998927

And:

V cache vs TensorRT V projection:

cosine = 0.999999881
max_abs = 0

The corruption completely disappears.

Control experiments:

mark Q output → FAIL
mark K output → FAIL
mark V output → PASS

No model weights, inputs, or operator parameters are changed.

CuTe DSL A/B

I also rebuilt Edge-LLM with:

ENABLE_CUTE_DSL=OFF

and confirmed the CuTe DSL FMHA symbol is absent.

The issue still reproduces:

layer 0 V cosine ≈ 0.01507

Therefore this does not appear to be caused by the CuTe DSL FMHA path.

compute-sanitizer

Running:

compute-sanitizer --tool racecheck

on the non-CuTe-DSL build reports:

4 errors
48 warnings

inside the Myelin-generated kernel_mha path.

I cannot confirm that these reports are directly causal, but they appear consistent with the observed behavior.

Expected behavior

The V cache should numerically agree with the PyTorch reference regardless of whether the intermediate V tensor is marked as a graph output.

Actual behavior
V not exposed:
cosine ≈ 0.015

V exposed as graph output:
cosine ≈ 1.0

This makes the issue look like a TensorRT/Myelin optimization, memory lifetime, or buffer reuse problem before AttentionPlugin consumes the V tensor.

[hungho77]

I also hit this on my Jetson AGX Thor as well, and I think I found the root cause — it also
explains the V-tensor behaviour you isolated.

TensorRT's Myelin horizontal fully-connected fusion miscompiles at maxBatchSize == 1
on SM110 with TRT 10.13.3.9. It targets the projections that share a layernorm output — per
decoder layer that is q/k/v (3 MatMuls on the input_layernorm output) and gate/up
(2 MatMuls on the post_attention_layernorm output).

applyMyelinCompileWorkarounds() in cpp/builder/llmBuilder.cpp already disables this
fusion, but the guard restricts it to TRT >= 10.15:

#if NV_TENSORRT_MAJOR == 10 && (NV_TENSORRT_MINOR == 13 || NV_TENSORRT_MINOR == 14)
    appendLunowudFlag(lunowudFlags, "-peep:match_dual_gemm=off");
#endif
#if NV_TENSORRT_MAJOR >= 11 || (NV_TENSORRT_MAJOR == 10 && NV_TENSORRT_MINOR >= 15)
    ...
    if (maxBatchSize == 1)
        appendLunowudFlag(lunowudFlags, "-peep:fc_h_fusion=off");   // never applied on 10.13
#endif

JetPack 7.1 ships TRT 10.13.3.9, which falls in the gap between the two branches, so the
flag is never applied on Jetson Thor.

Why marking the V projection as a graph output fixes it: a graph output cannot be fused
away, so V drops out of the horizontal q/k/v group and the miscompiled fusion no longer
applies — hence cosine ~0.015 -> ~1.0. That makes this a fusion defect rather than a
buffer-lifetime or memory-reuse problem, and it is consistent with your finding that
disabling CuTe DSL does not help: we see the failure identically under both CuTe DSL FMHA
and FMHA_v2, and both are correct once the fusion is off.

Quantized engines were never affected because Q/DQ nodes sit between the shared activation
and the projections, so the horizontal pattern never matches. Precision was a red herring —
the bug is fusion-pattern-dependent.

Workaround (no runtime rebuild, no re-export)

applyMyelinCompileWorkarounds appends to any pre-existing __LUNOWUD, so exporting it
before llm_build picks up both flags:

export __LUNOWUD="-peep:fc_h_fusion=off"
./build/examples/llm/llm_build --onnxDir ... --engineDir ... --maxBatchSize 1
# log: Using __LUNOWUD=-peep:fc_h_fusion=off -peep:match_dual_gemm=off

Only the engine needs rebuilding. On our Thor this turns a non-quantized FP16
Qwen2.5-VL-3B-Instruct engine from gibberish into correct image captions, and
Qwen2.5-0.5B-Instruct (text-only, so no vision tower and no mRoPE involved) into
The capital of France is Paris.

Still present on 0.9.1 — the code is unchanged from 0.8.0.

Proposed fix

Widen the guard for this one flag to cover 10.13/10.14:

 #if NV_TENSORRT_MAJOR >= 11 || (NV_TENSORRT_MAJOR == 10 && NV_TENSORRT_MINOR >= 15)
     appendLunowudFlag(lunowudFlags, "-mlir:autotune:num_threads=1");
     appendLunowudFlag(lunowudFlags, "-mlir:collective:fp4=off");
     appendLunowudFlag(lunowudFlags, "-cask_fusion:async_policy=1");
+#endif
+    // Horizontal fully-connected fusion miscompiles at batch size 1, corrupting the
+    // projections that share a layernorm output (q/k/v, and gate/up). On SM110 with
+    // TRT 10.13.3.9 this makes a non-quantized FP16 LLM engine emit garbage tokens;
+    // quantized engines are unaffected because Q/DQ nodes keep the pattern from
+    // matching. Previously gated to TRT >= 10.15, which left 10.13/10.14 broken.
+#if NV_TENSORRT_MAJOR >= 11 || (NV_TENSORRT_MAJOR == 10 && NV_TENSORRT_MINOR >= 13)
     if (maxBatchSize == 1)
     {
         appendLunowudFlag(lunowudFlags, "-peep:fc_h_fusion=off");
     }
 #endif

Branch, built and verified end to end on 0.9.1 with no __LUNOWUD in the environment, so the
source change alone is under test:
https://github.com/hungho77/TensorRT-Edge-LLM/tree/fix/fc-h-fusion-trt1013

Happy to open a PR with benchmarks and the full list of hypotheses we eliminated, once this
is triaged.

Note this only widens an existing workaround; the underlying miscompile is inside Myelin and
would still need a TensorRT-side fix.

Environment: Jetson AGX Thor, JetPack 7.1, aarch64, SM110, TensorRT 10.13.3.9, CUDA 13.0,
Release build, TensorRT Edge-LLM 0.8.0 (f9cc746) and 0.9.1 (7f061f2).

[taoz27]

@hungho77 Thanks for the great analysis. I tried your workaround and it works on my side as well.

I’m curious about your debugging process. How did you narrow this down to TensorRT’s fc_h_fusion optimization causing the V cache corruption? Were there any specific tools, logs, or experiments that helped identify the problematic fusion pass?

I would really appreciate it if you could share your approach, as it would be very helpful for debugging similar TensorRT issues in the future.

[hungho77]

@hungho77 Thanks for the great analysis. I tried your workaround and it works on my side as well.

I’m curious about your debugging process. How did you narrow this down to TensorRT’s fc_h_fusion optimization causing the V cache corruption? Were there any specific tools, logs, or experiments that helped identify the problematic fusion pass?

I would really appreciate it if you could share your approach, as it would be very helpful for debugging similar TensorRT issues in the future.

Glad it works on your side too — that's a useful second confirmation.

Honestly the path was not a straight line; it took eliminating a lot of wrong guesses first
(FP16 dynamic range, the ONNX export, dependency versions, the attention kernel, engine
size, RMSNorm overflow). What actually cracked it was two things.

1. The FP8 vs FP16 split. Same checkpoint, same pipeline, same runtime: FP8 correct,
non-quantized FP16 garbage. I diffed the two exported ONNX graphs and the op histograms are
identical apart from quantization nodes — FP16 has 1276 nodes, FP8 has 2068 = 1276 + 396
DequantizeLinear + 144 QuantizeLinear + 252 Transpose. Same 253 MatMul, same 36
AttentionPlugin, same norms. So the math was the same and only the graph shape differed,
which pointed at a graph-rewrite/fusion problem rather than a numerical one.

2. Subgraph bisection — every part correct, the whole wrong. I cut pieces out of the
exported ONNX with onnx.utils.extract_model, built each with
trtexec --stronglyTyped, and compared against an fp32 reference:

Subgraph	Result
inputs_embeds -> RMSNorm -> q/k/v MatMul+bias	cos 1.000000
AttentionPlugin alone, real RoPE table	cos 1.000000 (0.999731 at deep-layer magnitudes)
o_proj -> residual -> RMSNorm -> SwiGLU MLP -> residual	cos 0.999998
standalone FP16 MatMul at the model's shapes	cos 1.000000 / 0.999992 / 0.999998

Every component is correct in isolation while the assembled engine is wrong. That is the
signature of a fusion-level defect: the individual ops are fine, the cross-op rewrite is not.

At that point the shared-input/multi-GEMM shape was the obvious suspect (per layer: 3 MatMuls
on the input_layernorm output, 2 on post_attention_layernorm), and
applyMyelinCompileWorkarounds() in cpp/builder/llmBuilder.cpp already ships
-peep: flags to disable specific Myelin fusions — including -peep:fc_h_fusion=off, gated
to TRT >= 10.15 and maxBatchSize == 1. The name and the shape matched. Toggling it via
__LUNOWUD confirmed it with zero code changes.

That also explains your V-projection experiment, and I'd frame it slightly differently than
"V cache corruption": the KV cache itself isn't corrupted. V is simply one of the three
outputs of the horizontally-fused q/k/v GEMM, and marking it as a graph output makes it
unfusable, so it drops out of the group and the fusion no longer applies.

Practical notes if you repeat the bisection:

• Shrink the repro first. I got it down to Qwen/Qwen2.5-0.5B-Instruct, text-only — that
  alone eliminated anything VLM-specific, mRoPE, and engine size (988 MB engine still fails).
• To build a subgraph containing AttentionPlugin, trtexec needs
  --preview=+aliasedPluginIO1003 or it refuses with an aliased-I/O error.
• past_key_values_* must be allocated at full KV capacity. Present aliases past, so a
  zero-size buffer gives a spurious error code: 700 illegal memory access that looks like a
  kernel bug but is a test artifact.
• To test the plugin without knowing the RoPE table layout, feed a table of all zeros:
  q and k both become 0, every score is 0, causal softmax is uniform, so the expected output
  is just the running causal mean of v. Layout-independent reference, no PyTorch needed.

[nvluxiaoz]

Thanks. The proposed fix will be applied in the next release.