Name the repositories the flip covers, and the one it must not - #116
Merged
Conversation
`VERSIONING.md` §6.1 said "the seven repositories become public". That was written this morning and is now wrong twice. **`.repo-template` was deleted**, not flipped. It existed to scaffold a per-component repository, which the four-distribution model retired; publishing an empty template for a layout the project no longer uses would be publishing a wrong instruction. Mirrored to the local backup first, with its one pull request exported, the same way RM-DIST-05 handled the nineteen. **`astro-mine/embargo` must stay private, and no document said so.** It holds the sealed held-out benchmark seed sets Bench discloses only at evaluation time -- the anti-gaming property the leaderboard rests on. Publishing it destroys that property permanently: unlike a leaked credential, a disclosed held-out seed cannot be rotated into safety, because the value of the secret *is* that nobody has trained against it. Before this commit the repository was invisible to the spec. `bench.md` described embargoed held-out seeds as a concept without saying where they live, `$ASTRO_MINE_BENCH_EMBARGO_ROOT` appeared only in code, and the string "embargo" did not occur in `src/docs` at all. The one place the exclusion was recorded was the repository's own GitHub description, which is not a document anyone reads before a flip. So §6.1 now enumerates the six repositories that become public rather than counting them, and `bench.md` names where the sealed sets live and why that one stays behind.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
VERSIONING.md§6.1 said "the seven repositories become public". I wrote that this morning in docs#115, and it is now wrong in two directions..repo-templatewas deleted, not flippedIt existed to scaffold a per-component repository — the thing the four-distribution model retired. Publishing an empty template for a layout the project no longer uses is publishing a wrong instruction.
Mirrored to
files/archived-repos-backup/first with its one pull request exported, the same procedureRM-DIST-05used for the nineteen.fsckclean, 3 commits, 20 mirrors now held.astro-mine/embargomust stay private — and no document said soThis is the part worth reading. It holds the sealed held-out benchmark seed sets Bench discloses only at evaluation time, which is the anti-gaming property the whole leaderboard rests on.
Publishing it destroys that property permanently, and not in the way a leaked credential does. A disclosed held-out seed cannot be rotated into safety — the value of the secret is precisely that nobody has trained against it. Rotation gives you new seeds; it does not give you back the population of methods that never saw the old ones.
Before this commit the repository was invisible to the spec:
bench.mddescribed embargoed held-out seeds as a concept without saying where they live;$ASTRO_MINE_BENCH_EMBARGO_ROOTappeared only in_eval.py, in no document;embargodid not occur anywhere insrc/docs.The only place the exclusion was recorded was the repository's own GitHub description — "Stays private after the public flip" — which is not a document anybody reads before flipping an org.
I found it by listing the org after deleting
.repo-template, not by reading anything. A "make the repositories public" step executed against the org rather than against a named list would have published it.What changed
bench.mdnamesastro-mine/embargo, the env var, and why it stays behind.Related, not fixed here
The secret scan on .github#9 covered seven repositories: the six flipping plus
.repo-template. It did not coverembargo— correctly, since that repo is not being published, but the scan's "all seven repositories" phrasing now means a different seven than the tracker's. Noted on #9 rather than rescanned, because scanning a repo that stays private is not a flip prerequisite.Test plan
python3 scripts/check_docs.py— clean, 69 filespython3 -m unittest discover -s scripts -t scripts— 37 passedembargofsck-clean before deletion, PR and issue JSON exported