Hnimrama/rundeck reports

[hnimra-amd]

inferencex_atom_run_deck.html
inferencex_atom_run_deck.json
inferencex_atom_run_deck_viewer.html
rccl_perf_run_deck.html
rccl_perf_run_deck.json
vllm_run_deck.html
vllm_run_deck.json
vllm_run_deck_viewer.html

[solaiys]

Review: blocking issues

The Run Deck framework itself is mostly additive and self-contained (the module-global _SESSION store in registry.py is only mutated sequentially by pytest hooks in a single process — no new threads/async — so no data-race concerns there). The blocking issues are in the two RCCL test modules, where this PR reverts previously-fixed, documented behavior unrelated to reporting.

🔴 Reverts the documented dmesg-window fix — RCCL error detection silently drops the last minute of each test

cvs/tests/rccl/rccl_perf.py (start_time / end_time) and cvs/tests/rccl/rccl_regression.py (same)

The PR changes start_time/end_time from date +"%a %b %e %H:%M:%S" back to date +"%a %b %e %H:%M" (dropping seconds) and removes the explanatory comment. That comment warned that verify_dmesg_for_errors' node-scraper path treats analysis_range_end as an exclusive cutoff, so a minute-truncated end time silently drops the final minute of the test's dmesg window — real GPU HW errors / segfaults in that last minute go undetected (false PASS).

Fix: restore the :%S seconds precision on both start_time and end_time in both files.

🔴 Reverts till_end_flag=False → True, re-introducing a cross-test false-failure cascade

cvs/tests/rccl/rccl_perf.py and cvs/tests/rccl/rccl_regression.py (verify_dmesg_for_errors(...) call)

The PR flips verify_dmesg_for_errors(phdl, start_time, end_time, till_end_flag=True) and deletes the comment explaining why it must be False: with till_end_flag=True the scan runs from start_time to the end of the dmesg buffer, so an earlier parametrized test's kernel event (e.g. a scatter_perf segfault) repeatedly fails every subsequent collective in the sweep — a known false-failure cascade.

Fix: restore till_end_flag=False in both files so each test is bounded to its own start..end window.

These two reverts look unrelated to the reporting feature (likely an accidental rebase / branch-point regression) and should be dropped from the PR.

---

Non-blocking (not gating): after this change test_gen_graph no longer writes an RCCL HTML report directly — it only fills cvs_results_dict, and the deck is produced at session finish via generate_rundeck, which returns early when --html/htmlpath is unset. Fine if cvs run always passes --html.

[solaiys]

@hnimra-amd , Please check if the changes in tests/rccl/ is intentional or by mistake.

[cijohnson]

Hi @hnimra-amd Overall the json interface and framework look good.

i have couple of minor comments related to rccl changed, also i have following recommendation.

1. unittests are using pytest, we decided to used unittest framework for UT's and pytest for the actual test suites. so that the run_all_unittests.py can discover only the unitests based UT's and run during "make test". Also the UT's has to be under respective dirs. refer AGENTS.md (if you can instruct your coding agent to refer AGENTS.md while writing UT's it will take care of it for you). Can you migrate the UT's to unittest framework please

2. I see functional programming style used in this cvs/report/* library, Can you please move to Object Oriented design, so that we no need to pass around a lot of args.
   I have modified cvs/lib/report/render/cell_card.py to use Object Oriented design and have come with it own UT in this commit (2e1eae0 in your branch, you can use that as a reference and improve on it.