Feat/path url contract - #155
Merged
Merged
Conversation
…vice. Caddy (and uvicorn, and clamd's wrapper) run as PID 1 and never wait() for orphaned children, so each 30-second healthcheck probe left one Z-state process behind; measured live on the VPS front door 2026-08-18 (48 min up = 96 intervals = 97 zombies, tripping doctor's warn threshold at 50). Docker's tini becomes PID 1 and reaps; no other behaviour changes. gw-runner stays init-less: no healthcheck by design and its loop reaps its own job children. Pinned by a new deploy test that fails for any healthchecked service missing init: true in either compose file.
…stations/<ausmt_id> and /collections/<id> are the published URL contract (tier 1, owner ruling 2026-08-18). The canonical block maps each shape onto its SPA hash route with a permanent 301: handle_path strips the prefix (raw path updated in lock-step, so the entity id rides byte-for-byte, never decoded or re-encoded) and the target is rebuilt from two lazily-evaluated maps over the stripped URI ({ausmt_pathurl_rest} = the raw id, {ausmt_qs} = the query; a handle wrapping its own strip would fire redir before it under the default directive order). Query strings are PRESERVED before the fragment (/surveys/x?utm=1 -> /?utm=1#/survey/x, a pinned decision, not an accident); a bare /surveys[,/] lands on the portal root, never an empty-fragment URL; the legacy chain stays two hops (the {uri}-preserving host 301, then this mapping). Analytics decision, stated and pinned rather than coded: the hop lands in the canonical block's masked log but aggregate_stats.py already excludes it twice over (only /data/* paths classify, and 301 is outside every admitted status set), so a path-link visit counts once, at the SPA boot. doctor.sh gains the pathurl leg (pinned vulcan-2022, explicit https with --resolve, clean PASS-labelled skip when the edge gives no response); the runbook documents the shapes, the query decision, the two-hop chain and the deferred tiers 2/3, each promising no published URL changes when it comes. Textual pins run everywhere; real-Caddy runtime pins (CI class, like the cutover lane's) prove every shape, the byte-preservation, the bare-prefix rule, the two-hop chain, the logged hop, and a red-proof that stripping the section makes path links fall through to the reader. The two site-address test helpers now remove placeholder tokens before counting braces: a map opener line carries balanced placeholder braces beside one structural opener, and raw counting would have hidden every later site block from the pins (the pinned properties themselves are unchanged).
…oin it - per-survey URLs become <base>/surveys/<slug>, per-station <base>/stations/<ausmt_id>, and per-collection URLs are ADDED as <base>/collections/<id> (the sitemap emitted no collection links at all before). The hash-fragment forms leave the sitemap and the Atom feed entry <link> moves to the path form too: the path shape is the published contract now (the front door 301s it into the SPA, tier 1), and crawlers ignore fragments anyway. The per-survey URL now carries the AUTHORITATIVE survey.yaml slug the build stamps into smeta/ausmt_id/product paths, never a re-slugified display label (the old slugify(lbl) sitemap could advertise an id the portal router cannot resolve whenever a declared slug differs from the label's slugification; slugify remains only as the raw-mode fallback). The stale sitemap caveat comment is rewritten to the tier-1 reality: path URLs redirect into the SPA today, prerendered per-entity pages (tier 3) will serve the same URLs later, and the advertised contract does not change when they come. RED-proven: the new pins in test_sitemap_pathurls.py failed against the fragment-emitting build first, and test_feed's link pin failed against the path form before being re-pinned. Portal-side comment truth-ups only (main.js SLUG_TO_SURVEY + interaction_test route F name the path URLs that 301 into the hash route); no router change, per the lane's out-of-scope line.
…y.json freezes every published survey slug (26), station ausmt_id (2584) and collection id (1), seeded from the live published corpus (https://ausmt.auscope.org.au/data/mtcat.json, 2026-08-18, dated note in _meta). extract/url_registry.py regenerates the set from a BUILT data tree (mtcat.json - the engine's own id derivations are the single authority; re-walking raw inputs here would be the second-derivation divergence risk the build refuses elsewhere) and enforces the freeze: a REMOVED or CHANGED id fails with the prescribed message ('a published URL id moved - add a redirect entry and a dated registry note, never rename silently.'), ADDITIONS pass and are auto-recorded into the registry, and the sitemap can never advertise an unpinned id (path form AND the legacy fragment form, so a regression cannot smuggle ids past the pin); --update seeds/extends and refuses to bless a removal. The registry header states each id's derivation: survey slugs are DECLARED (survey.yaml slug, defaulting to the folder name, safe_component-sanitised), ausmt_ids are au.<slug>.<DATAID[.variant]>, collection ids are declared verbatim in member survey.yamls (exact-string grouping, near-duplicate warning at build, Add Survey datalist autofill). Tests: fixture-driven freeze/rename/addition/sitemap pins (the rename RED prints the prescribed message, proven live against the seeded registry: mutating vulcan-2022 fails exit 1), CLI end-to-end seed/check/auto-record, a well-formedness pin on the committed registry that also freezes the two cross-pinned ids (doctor.sh's vulcan-2022 probe slug, the auslamp collection), and a real-data tier (AUSMT_URL_REGISTRY_DATA) that checks the committed registry against an actual built tree and skips cleanly elsewhere - proven green against the live corpus data on this box.
…its own comments and docstring with plain dashes, per the repo's no-em-dash rule for added lines. Content unchanged.
…ripwire. The path-URL lane added test_url_registry.py whose real-corpus leg skips when AUSMT_URL_REGISTRY_DATA names no built data tree - legitimately unreachable in every CI engine lane (they build no corpus; same dev-box-only class as the realdata and preflight-corpus entries). The first CI run of the batch PR tripped the tripwire on exactly this skip in both engine lanes - the tripwire doing precisely its job; the lane's battery gap was not feeding the new test's -rs output through ci_check_skips, now done: repro red before this entry, green after, and a non-matching --allow red control still fails. The freeze invariant stays guarded in CI by the module's fixture tests; this entry excuses the real-corpus leg only.
…ly on any checkout. The in-image engine-full-tests lane red-lined because test_url_registry's checked-in-registry test hard-asserted portal/data/url_registry.json exists - but the engine image ships no portal tree by designed topology. Guard added in the test_mtcat_version_parity.py pattern exactly: probe a SET of other portal surfaces; none present = the image, skip with the reason already on the tripwire allow-list ('engine image build: portal tree not shipped...'); any present = a checkout, the guard opens and a missing registry FAILS the test. Three arms proven locally: image-sim skips + tripwire green (2 accounted skips); checkout runs green; checkout with the registry hidden FAILS not skips. The registry file itself is deliberately not in the probe set.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.