rules(cli): a listing answers from the published index, and never multiplies - #552
Open
elronbandel wants to merge 1 commit into
Open
rules(cli): a listing answers from the published index, and never multiplies#552elronbandel wants to merge 1 commit into
elronbandel wants to merge 1 commit into
Conversation
…tiplies Rule 14 said `list` MUST read Docker image labels, with no separate index. The one workable reading is what the code did — docker images plus docker inspect — so `list` could only describe what you had already pulled: nothing on a clean machine, and never what the fleet publishes, which is the question a listing is asked. The release now writes that down (#519). The index is not the list the rule forbids: it is derived by the publisher, from the registry and from the components' own labels, so nothing is kept that a component does not declare. The rule now says where a listing may read from — the index, or the checkout under --local — and keeps the prohibition that matters: the CLI keeps no list of its own. The last sentence is the lesson the dashboard paid for. It offered the product of a benchmark list and an agent list, so an agent published for one benchmark was offered for ninety-nine, and an image whose manifest list was never stitched (latest-amd64, no latest) was offered although nothing could pull it. A listing offers a pair only if that pair is published under the tag a run pulls. curl joins the underlying tools for the three HTTP calls that read the index. Closes #512 Signed-off-by: Elron Bandel <elron.bandel@ibm.com> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Sep 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #512. Rules only — the code is #515's successor (#552), per contributing rule 2.
Before
One workable reading, and it is what the code did:
docker images+docker inspect. Solistcould only describe what you had already pulled — nothing ona clean machine, and never what the fleet publishes.
After
A listing answers from the published index (
evals/index:latest, written byfleet-index.pyfrom the registry itself and the components' own labels), orfrom the checkout under
--local. The prohibition that matters is kept: the CLIkeeps no list of its own.
Plus the sentence the dashboard paid for:
It offered the product of two lists, so
openhands— published for one benchmark— was offered for ninety-nine; and an image whose manifest list was never
stitched (
latest-amd64, nolatest) was offered although nothing could pullit. 719 of 7649 artifacts were in that state.
curljoins the underlying tools for the three HTTP calls that read the index.Rules checked against
.agents/meta/rules/RULES.md— an inspectable property of the finishedartifact, with the change and its reason in this file's changelog.
.agents/contributing/RULES.md1 (resolves rfc: .agents/src/RULES.md#14 — a listing should read the published catalog, not only images you pulled #512), 2 (rules only),3 (this section), 4 (no tool attribution).
🤖 Generated with Claude Code