Adding llama configs, subtests and moving common functions to the common library

[amd-droy]

This PR extends the SGLang inference suite with Llama 70B sample configs (single-node, distributed, and disaggregated), adds pytest subtests for per-node/per-metric performance verification, and refactors duplicated logic from the SGLang controller libs into sglang_common.py.

It also improves reporting by introducing a benchmark metric registry and perf-metric table rendering, wired into the SGLang report presets and report plugins so threshold results show up more clearly in HTML output.

[hnimra-amd]

Config review notes on threshold_file and model_num_params.

[hnimra-amd]

This points to the DeepSeek R1 threshold file, but the disaggregated config uses mi30x_sglang_llama_70b_threshold.json. Single and distributed would run Llama 70B with DeepSeek thresholds, and those files have different throughput values. Should this be the Llama threshold file instead?

[amd-droy]

updated.

[hnimra-amd]

Same as single: this still points to the DeepSeek R1 threshold file while disaggregated uses the Llama file. Single/distributed would pick up DeepSeek thresholds, which differ from the Llama values. Can we align this with mi30x_sglang_llama_70b_threshold.json?

[amd-droy]

updated.

[hnimra-amd]

This looks copied from the DeepSeek configs (671B). For Llama 3.1 70B it should be closer to 70000000000. It only affects MFU calculation, not the core perf gates, but worth fixing.

[amd-droy]

updated.

[hnimra-amd]

Same DeepSeek copy-paste: 671000000000 is 671B params. For Llama 3.1 70B use something like 70000000000. Lower priority since this only affects MFU, not throughput/latency gates.

[amd-droy]

updated.

[hnimra-amd]

Same here: 671000000000 looks like it came from the DeepSeek configs. For Llama 3.1 70B this should be closer to 70000000000. Only affects MFU, not the main perf thresholds.

[amd-droy]

updated.

[atnair-amd]

Code review

Found 2 issues:

1. New library code has no unit tests (AGENTS.md, this repo's root CLAUDE.md doc, says under Unit Testing Standards: "All library functions and classes must have corresponding unit tests in the tested module's own unittests/ dir" and under Do Not: "Never add library functions without unit tests"). This PR adds substantial new logic to sglang_common.py (parse_inference_bench_results, verify_inference_results, poll_for_inference_completion, metric_threshold_violation, etc.) plus two new files, benchmark_metric_registry.py and perf_metric_table.py, with no corresponding tests in cvs/lib/inference/unittests/ or cvs/lib/report/unittests/ — both of which are active, populated test directories for these packages already (e.g. test_vllm_parsing.py, test_atom_parsing.py), so this isn't a case of the module having no prior test convention.

cvs/cvs/lib/inference/sglang/sglang_common.py

Lines 1 to 30 in f9bb9d4

	'''Shared helpers for SGLang single-node and disaggregated inference libs.'''
	from __future__ import annotations
	import base64
	import json
	import re
	import shlex
	import time
	from typing import Any, Callable, Mapping, Optional
	from cvs.lib import globals
	from cvs.lib.utils.model_query_lib import LmEvalBenchmark, OpenAIProbe
	from cvs.lib.utils_lib import fail_test
	from cvs.lib.utils.verdict import ThresholdViolation, _check_one, evaluate_all
	log = globals.log
	DEFAULT_GPU_MEM_THRESHOLD_MB = 5000
	AMD_SMI_METRIC_CMD = "sudo amd-smi metric --json"
	_SERVER_READY_RE = re.compile(
	r"server is fired up and ready to roll",
	re.I,
	)
	def textwrap_for_yml(msg_string: str) -> str:
	return '\n'.join([m.lstrip() for m in msg_string.split('\n')])

cvs/cvs/lib/report/benchmark_metric_registry.py

Lines 1 to 30 in f9bb9d4

	'''Shared storage for benchmark metric pass/fail rows used by pytest-html hooks.
	Metric rows must live in a normal import path (``cvs.lib...``), not only on the
	sglang ``conftest`` module. Pytest loads directory ``conftest.py`` files under a
	different module name than ``cvs.tests.inference.sglang.conftest``, so a dict on
	the conftest module is invisible to hooks while ``record_benchmark_metric_rows()``
	writes to the package import path.
	'''
	from __future__ import annotations
	import html
	import json
	import re
	from pathlib import Path
	from typing import Any
	from _pytest.stash import StashKey
	from cvs.lib.report.render.perf_metric_table import (
	dedupe_metric_rows,
	is_benchmark_metrics_extra,
	render_benchmark_metrics_html,
	)
	BENCHMARK_METRIC_ROWS_KEY = StashKey[list[dict[str, Any]]]()
	BENCHMARK_METRIC_ROWS_USER_PROPERTY = 'cvs_benchmark_metric_rows'
	DEFAULT_BENCHMARK_TEST_NAME = 'test_run_performance_benchmark_test'
	_ROWS_BY_NODEID: dict[str, list[dict[str, Any]]] = {}

2. Global HTML-report CSS/JS regression outside SGLang's scope. HtmlReportManager.inject_style_overrides is invoked unconditionally for every suite via the root cvs/conftest.py's pytest_html_results_summary hook, not just SGLang. This PR removes .collapse { display: none !important; } and .collapsible td:not(.col-links) { cursor: default !important; } from REPORT_STYLE_OVERRIDES and replaces them only with rules scoped to the new cvs-benchmark-collapsible class used by SGLang's benchmark-metric rows. Nothing re-establishes the suppression for the general case, so pytest-html's built-in "Show all details / Hide all details" control bar and per-row toggle cursor will reappear in every other suite's report (vLLM, ATOM, RCCL, training, etc.) as an apparently unintended side effect of an SGLang-only feature. The docstring on inject_style_overrides ("Inject CSS to hide show/hide details UI elements") is also now stale, since it no longer does that for non-benchmark rows and doesn't mention the new REPORT_BENCHMARK_METRICS_SCRIPT JS injection.

cvs/cvs/lib/report_plugins.py

Lines 630 to 640 in f9bb9d4

	inf_res_dict=inf_res_dict,
	lifecycle_report=store.get("lifecycle_report") or {},
	report_manager=self,
	pytest_config=session.config,
	)
	@staticmethod
	def inject_style_overrides(prefix):
	"""Inject CSS to hide show/hide details UI elements."""
	prefix.extend([REPORT_STYLE_OVERRIDES, REPORT_BENCHMARK_METRICS_SCRIPT])

[amd-droy]

Code review

Found 2 issues:

1. New library code has no unit tests (AGENTS.md, this repo's root CLAUDE.md doc, says under Unit Testing Standards: "All library functions and classes must have corresponding unit tests in the tested module's own unittests/ dir" and under Do Not: "Never add library functions without unit tests"). This PR adds substantial new logic to sglang_common.py (parse_inference_bench_results, verify_inference_results, poll_for_inference_completion, metric_threshold_violation, etc.) plus two new files, benchmark_metric_registry.py and perf_metric_table.py, with no corresponding tests in cvs/lib/inference/unittests/ or cvs/lib/report/unittests/ — both of which are active, populated test directories for these packages already (e.g. test_vllm_parsing.py, test_atom_parsing.py), so this isn't a case of the module having no prior test convention.

cvs/cvs/lib/inference/sglang/sglang_common.py

Lines 1 to 30 in f9bb9d4

	'''Shared helpers for SGLang single-node and disaggregated inference libs.'''
	from __future__ import annotations
	import base64
	import json
	import re
	import shlex
	import time
	from typing import Any, Callable, Mapping, Optional
	from cvs.lib import globals
	from cvs.lib.utils.model_query_lib import LmEvalBenchmark, OpenAIProbe
	from cvs.lib.utils_lib import fail_test
	from cvs.lib.utils.verdict import ThresholdViolation, _check_one, evaluate_all
	log = globals.log
	DEFAULT_GPU_MEM_THRESHOLD_MB = 5000
	AMD_SMI_METRIC_CMD = "sudo amd-smi metric --json"
	_SERVER_READY_RE = re.compile(
	r"server is fired up and ready to roll",
	re.I,
	)
	def textwrap_for_yml(msg_string: str) -> str:
	return '\n'.join([m.lstrip() for m in msg_string.split('\n')])

cvs/cvs/lib/report/benchmark_metric_registry.py

Lines 1 to 30 in f9bb9d4

	'''Shared storage for benchmark metric pass/fail rows used by pytest-html hooks.
	Metric rows must live in a normal import path (``cvs.lib...``), not only on the
	sglang ``conftest`` module. Pytest loads directory ``conftest.py`` files under a
	different module name than ``cvs.tests.inference.sglang.conftest``, so a dict on
	the conftest module is invisible to hooks while ``record_benchmark_metric_rows()``
	writes to the package import path.
	'''
	from __future__ import annotations
	import html
	import json
	import re
	from pathlib import Path
	from typing import Any
	from _pytest.stash import StashKey
	from cvs.lib.report.render.perf_metric_table import (
	dedupe_metric_rows,
	is_benchmark_metrics_extra,
	render_benchmark_metrics_html,
	)
	BENCHMARK_METRIC_ROWS_KEY = StashKey[list[dict[str, Any]]]()
	BENCHMARK_METRIC_ROWS_USER_PROPERTY = 'cvs_benchmark_metric_rows'
	DEFAULT_BENCHMARK_TEST_NAME = 'test_run_performance_benchmark_test'
	_ROWS_BY_NODEID: dict[str, list[dict[str, Any]]] = {}

2. Global HTML-report CSS/JS regression outside SGLang's scope. HtmlReportManager.inject_style_overrides is invoked unconditionally for every suite via the root cvs/conftest.py's pytest_html_results_summary hook, not just SGLang. This PR removes .collapse { display: none !important; } and .collapsible td:not(.col-links) { cursor: default !important; } from REPORT_STYLE_OVERRIDES and replaces them only with rules scoped to the new cvs-benchmark-collapsible class used by SGLang's benchmark-metric rows. Nothing re-establishes the suppression for the general case, so pytest-html's built-in "Show all details / Hide all details" control bar and per-row toggle cursor will reappear in every other suite's report (vLLM, ATOM, RCCL, training, etc.) as an apparently unintended side effect of an SGLang-only feature. The docstring on inject_style_overrides ("Inject CSS to hide show/hide details UI elements") is also now stale, since it no longer does that for non-benchmark rows and doesn't mention the new REPORT_BENCHMARK_METRICS_SCRIPT JS injection.

cvs/cvs/lib/report_plugins.py

Lines 630 to 640 in f9bb9d4

	inf_res_dict=inf_res_dict,
	lifecycle_report=store.get("lifecycle_report") or {},
	report_manager=self,
	pytest_config=session.config,
	)
	@staticmethod
	def inject_style_overrides(prefix):
	"""Inject CSS to hide show/hide details UI elements."""
	prefix.extend([REPORT_STYLE_OVERRIDES, REPORT_BENCHMARK_METRICS_SCRIPT])

1. 3 unit tests added.
2. Handled. Now, vLLM, ATOM, RCCL, training, and the rest of the reports should look the same as before this PR.

[hnimra-amd]

LGTM