schema: document switch scope rule + v4.0 test-infrastructure-switch idea - #846
Merged
Merged
Conversation
…idea Comment-only. Records the review-chairs ruling (2026-08-14) that product_switches is for switches that talk exclusively to storage-solution nodes -- a switch the benchmark clients attach to is test infrastructure and must not feed the rule-11 total_rack_units sum or the derived Provisioned Power column. Also records the v4.0 enhancement idea prompted by the Everpure v3.0 submission (a dedicated-but-client-attached fabric with no structured place to be documented): either a sibling test_infrastructure_switches list excluded from both sums, or a scope field on switch_description. Deliberately not implemented now; revisit when gearing up for v4.0.
|
MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Comment-only change — no behavior, no schema semantics touched.
Records two things at the
product_switchesdeclaration inschema.yamland onSwitchDescriptioninschema_validator.py:The switch scope rule adopted by the review chairs during the v3.0 review (2026-08-14): a switch belongs in
product_switchesonly if it talks exclusively to storage-solution nodes. A switch the benchmark clients attach to is test infrastructure and must be omitted, because entries there feed the rule-11total_rack_unitssum and reportgen's derived "Provisioned Power (W)" column. (Applied across the v3.0 tree: Everpure PR mlcommons/submissions_storage_v3.0#200, plus guidance to Suzhou and InspurData in their submission issues mlpstorage checkpointing datasize run failed #198/cosmoflow data load too slow #199.)A v4.0 enhancement idea, deliberately not implemented: submitters have no structured slot for documenting test-infrastructure switches — the v3.0 Everpure submission wanted to honestly report a dedicated-but-client-attached 8-switch fabric and could only do so in PDF prose and YAML comments. Options sketched: a sibling
test_infrastructure_switcheslist excluded from rule 11 and power derivation, or ascope: solution|test_infrastructurefield onswitch_description. To be picked up when gearing up for the v4.0 submission round.