feat: add host range profiling#817
Draft
voltjia wants to merge 1 commit into
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
INFINI_OPS_ENABLE_HOST_RANGE_PROFILING, disabled by default.--host-range-profile, cold/warm JSONL reports, calibration/control tooling, and focused regression tests.Motivation
The existing pytest benchmark path measures end-to-end operator latency but cannot attribute CPU-side overhead across Python binding, conversion, dispatch, cache, operator, and backend-submission layers.
This PR adds coarse host attribution without introducing a separate C++ benchmark framework or timing device execution. It is an opt-in diagnostic facility, not a kernel profiler or performance gate.
Related issue: N/A - follows the performance-testing design discussion.
Type of Change
feat- new feature / new operator / new platformfix- bug fixperf- performance improvement (no behavioral change)refactor- code restructuring without behavior changetest- adding or fixing tests onlydocs- documentation onlybuild/ci- build system or CI configurationchore- tooling, formatting, or other non-code changesPlatforms Affected
WITH_CPU)WITH_NVIDIA)WITH_ILUVATAR)WITH_METAX)WITH_CAMBRICON)WITH_MOORE)WITH_ASCEND)WITH_TORCH)Smoke Test Result
Validation ran in
accelerator-dev/nvidia:latestonssh nvidiaagainst final commitc4ff40f163781c4e558dacf71739bf07bed027ac(tree4dc01eb03896f84d4cfe6f061963d12523c2fd62).GitHub CI after PR creation:
Representative commands:
Test Results on Supported Platforms
54 passed, 8 skipped)41 passed); full suite not runixsmi; no code failure observedtests/test_topk_softmax.pyrequiring unavailabletorch.uint32;74 passed, 22 skipped, 4 collection errorsFull focused and smoke pytest output
Benchmark / Performance Impact
Final-report measurements used one NVIDIA A100-SXM4-80GB (
CUDA_VISIBLE_DEVICES=0), host CPU 32, aRelWithDebInfobuild, and synchronization only outside collection windows.(13, 4), implementation 0, 1,500 warm calls.(4, 48, 64) x (4, 64, 6), cuBLASLt implementation 1, 6,000 warm calls.end_to_endbinding.bodyinclusivedispatch.callinclusiveoperator.invokeinclusivebackend.submitinclusiveThe two final JSONL reports contain 39 rows each.
add-final.jsonlSHA256 is9b3c43bcc7538a425820e6cdde64b6f075398c83eda19e198644f8e85125600d;gemm-final.jsonlSHA256 is67e7d178932e79b9f17c792e834c99f6e363bd8cf5b00065e639f669584dc4d9.An earlier same-process alternating control on the same A100 characterized active-collection overhead at approximately
+9.7%for Add and+6.6%to+7.0%for GEMM, with a conservative complete-scope calibration of 134.12 ns. Those control values came from the pre-rebase experiment commit and are observer-effect guidance, not final-tree performance claims.Complete `add-final.jsonl` output
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"binding.body","metric":"inclusive","count":1,"unit":"ns","mean":4279182.0,"median":4279182.0} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"binding.body","metric":"self","count":1,"unit":"ns","mean":13293.0,"median":13293.0} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"binding.tensor_conversion","metric":"inclusive","count":3,"unit":"ns","mean":76600.0,"median":38533.0} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"binding.tensor_conversion","metric":"self","count":3,"unit":"ns","mean":76600.0,"median":38533.0} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"binding.device_conversion","metric":"inclusive","count":1,"unit":"ns","mean":82549.0,"median":82549.0} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"binding.device_conversion","metric":"self","count":1,"unit":"ns","mean":82549.0,"median":82549.0} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"dispatch.call","metric":"inclusive","count":1,"unit":"ns","mean":3953540.0,"median":3953540.0} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"dispatch.call","metric":"self","count":1,"unit":"ns","mean":1820.0,"median":1820.0} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"operator.call","metric":"inclusive","count":1,"unit":"ns","mean":3951720.0,"median":3951720.0} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"operator.call","metric":"self","count":1,"unit":"ns","mean":1892790.0,"median":1892790.0} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"cache.key","metric":"inclusive","count":1,"unit":"ns","mean":11811.0,"median":11811.0} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"cache.key","metric":"self","count":1,"unit":"ns","mean":11811.0,"median":11811.0} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"cache.lookup","metric":"inclusive","count":1,"unit":"ns","mean":2073.0,"median":2073.0} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"cache.lookup","metric":"self","count":1,"unit":"ns","mean":2073.0,"median":2073.0} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"cache.construct","metric":"inclusive","count":1,"unit":"ns","mean":1737664.0,"median":1737664.0} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"cache.construct","metric":"self","count":1,"unit":"ns","mean":1737664.0,"median":1737664.0} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"operator.invoke","metric":"inclusive","count":1,"unit":"ns","mean":307382.0,"median":307382.0} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"operator.invoke","metric":"self","count":1,"unit":"ns","mean":7357.0,"median":7357.0} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"backend.submit","metric":"inclusive","count":1,"unit":"ns","mean":300025.0,"median":300025.0} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"backend.submit","metric":"self","count":1,"unit":"ns","mean":300025.0,"median":300025.0} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"binding.body","metric":"inclusive","count":1500,"unit":"ns","mean":93085.79933333333,"median":90530.0} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"binding.body","metric":"self","count":1500,"unit":"ns","mean":5505.358666666667,"median":2270.0} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"binding.tensor_conversion","metric":"inclusive","count":4500,"unit":"ns","mean":14654.111333333334,"median":12948.5} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"binding.tensor_conversion","metric":"self","count":4500,"unit":"ns","mean":14654.111333333334,"median":12948.5} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"binding.device_conversion","metric":"inclusive","count":1500,"unit":"ns","mean":5168.288666666666,"median":3749.0} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"binding.device_conversion","metric":"self","count":1500,"unit":"ns","mean":5168.288666666666,"median":3749.0} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"dispatch.call","metric":"inclusive","count":1500,"unit":"ns","mean":38449.818,"median":35994.5} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"dispatch.call","metric":"self","count":1500,"unit":"ns","mean":377.0726666666667,"median":186.0} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"operator.call","metric":"inclusive","count":1500,"unit":"ns","mean":38072.74533333333,"median":35747.0} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"operator.call","metric":"self","count":1500,"unit":"ns","mean":3082.5526666666665,"median":846.5} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"cache.key","metric":"inclusive","count":1500,"unit":"ns","mean":954.5826666666667,"median":382.0} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"cache.key","metric":"self","count":1500,"unit":"ns","mean":954.5826666666667,"median":382.0} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"cache.lookup","metric":"inclusive","count":1500,"unit":"ns","mean":519.2473333333334,"median":194.0} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"cache.lookup","metric":"self","count":1500,"unit":"ns","mean":519.2473333333334,"median":194.0} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"operator.invoke","metric":"inclusive","count":1500,"unit":"ns","mean":33516.36266666667,"median":31308.0} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"operator.invoke","metric":"self","count":1500,"unit":"ns","mean":7148.822,"median":5606.0} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"backend.submit","metric":"inclusive","count":1500,"unit":"ns","mean":26367.540666666668,"median":24399.0} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"backend.submit","metric":"self","count":1500,"unit":"ns","mean":26367.540666666668,"median":24399.0} {"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"end_to_end","metric":"inclusive","count":1500,"unit":"ns","mean":140217.05587704974,"median":140659.45520997047}Complete `gemm-final.jsonl` output
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"binding.body","metric":"inclusive","count":1,"unit":"ns","mean":1048207.0,"median":1048207.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"binding.body","metric":"self","count":1,"unit":"ns","mean":10318.0,"median":10318.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"binding.tensor_conversion","metric":"inclusive","count":3,"unit":"ns","mean":42226.333333333336,"median":11812.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"binding.tensor_conversion","metric":"self","count":3,"unit":"ns","mean":42226.333333333336,"median":11812.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"binding.device_conversion","metric":"inclusive","count":1,"unit":"ns","mean":57562.0,"median":57562.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"binding.device_conversion","metric":"self","count":1,"unit":"ns","mean":57562.0,"median":57562.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"dispatch.call","metric":"inclusive","count":1,"unit":"ns","mean":853648.0,"median":853648.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"dispatch.call","metric":"self","count":1,"unit":"ns","mean":2979.0,"median":2979.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"operator.call","metric":"inclusive","count":1,"unit":"ns","mean":850669.0,"median":850669.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"operator.call","metric":"self","count":1,"unit":"ns","mean":5816.0,"median":5816.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"cache.key","metric":"inclusive","count":1,"unit":"ns","mean":19492.0,"median":19492.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"cache.key","metric":"self","count":1,"unit":"ns","mean":19492.0,"median":19492.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"cache.lookup","metric":"inclusive","count":1,"unit":"ns","mean":2051.0,"median":2051.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"cache.lookup","metric":"self","count":1,"unit":"ns","mean":2051.0,"median":2051.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"cache.construct","metric":"inclusive","count":1,"unit":"ns","mean":34832.0,"median":34832.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"cache.construct","metric":"self","count":1,"unit":"ns","mean":34832.0,"median":34832.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"operator.invoke","metric":"inclusive","count":1,"unit":"ns","mean":788478.0,"median":788478.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"operator.invoke","metric":"self","count":1,"unit":"ns","mean":1817.0,"median":1817.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"backend.submit","metric":"inclusive","count":1,"unit":"ns","mean":786661.0,"median":786661.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"backend.submit","metric":"self","count":1,"unit":"ns","mean":786661.0,"median":786661.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"binding.body","metric":"inclusive","count":6000,"unit":"ns","mean":29986.799666666666,"median":21776.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"binding.body","metric":"self","count":6000,"unit":"ns","mean":1087.2481666666667,"median":635.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"binding.tensor_conversion","metric":"inclusive","count":18000,"unit":"ns","mean":4048.639277777778,"median":3006.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"binding.tensor_conversion","metric":"self","count":18000,"unit":"ns","mean":4048.639277777778,"median":3006.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"binding.device_conversion","metric":"inclusive","count":6000,"unit":"ns","mean":1653.6136666666666,"median":1203.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"binding.device_conversion","metric":"self","count":6000,"unit":"ns","mean":1653.6136666666666,"median":1203.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"dispatch.call","metric":"inclusive","count":6000,"unit":"ns","mean":15100.02,"median":10284.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"dispatch.call","metric":"self","count":6000,"unit":"ns","mean":229.51383333333334,"median":185.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"operator.call","metric":"inclusive","count":6000,"unit":"ns","mean":14870.506166666666,"median":10085.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"operator.call","metric":"self","count":6000,"unit":"ns","mean":645.4595,"median":370.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"cache.key","metric":"inclusive","count":6000,"unit":"ns","mean":1022.6486666666667,"median":608.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"cache.key","metric":"self","count":6000,"unit":"ns","mean":1022.6486666666667,"median":608.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"cache.lookup","metric":"inclusive","count":6000,"unit":"ns","mean":230.21066666666667,"median":204.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"cache.lookup","metric":"self","count":6000,"unit":"ns","mean":230.21066666666667,"median":204.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"operator.invoke","metric":"inclusive","count":6000,"unit":"ns","mean":12972.187333333333,"median":8865.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"operator.invoke","metric":"self","count":6000,"unit":"ns","mean":1049.1403333333333,"median":539.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"backend.submit","metric":"inclusive","count":6000,"unit":"ns","mean":11923.047,"median":8318.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"backend.submit","metric":"self","count":6000,"unit":"ns","mean":11923.047,"median":8318.0} {"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"end_to_end","metric":"inclusive","count":6000,"unit":"ns","mean":33592.056017369025,"median":31603.74891012907}Notes for Reviewers
backend.submitmeasures host API work through API return; it does not measure device execution or kernel duration. CUDA APIs may still block on queue/device progress.end_to_endand replayed C++ ranges use separate populations and must not be subtracted from one another.pytest-xdistis intentionally rejected when host-range output is enabled.torch.uint32compatibility problem intests/test_topk_softmax.py. No unrelated compatibility fix is included here.docs/superpowers/were removed from the final branch.