feat: Pre-Phase 2 Sprint B — cloud transport, offline spool, and evidence bundles - #35
Merged
Conversation
The envelope, the five event types, the full error-code registry, idempotency by event_id, batch semantics, and the client-side hashing contract were all implied by the code and written down nowhere. That is fine while the only implementation is this repo; it stops being fine the moment a hosted backend, a TypeScript SDK, or a community connector has to agree with it — and it is the document a design partner's security reviewer asks for first. Publishing it before building the cloud transport is deliberate: the spec is what the transport implements, not a description written afterwards. Sprint B W1.
…ulary The SDK can now target the hosted ingest endpoint (rootsign[cloud]) ahead of the backend existing, and — more importantly — it can survive that endpoint being unreachable without losing records or interrupting the agent. What lands: * HttpIngestClient (ADR-013). Seals the chain client-side so the server verifies rather than computes; owns its retry budget (owns_retry=True, so BufferedIngestClient stands down to one attempt per flush rather than multiplying 3x3); maps every HTTP status onto the published error registry and honours the wire `retryable` flag over its own reading of a code, which is what lets the registry grow without stranding fielded clients. * The offline spool. On retryable exhaustion the client fails over to the ADR-011 JSONL writer under $ROOTSIGN_DATA_DIR/spool/ — one WARNING for the outage, not one per record — so spooled sessions are ordinary session files that `rootsign verify --local` reads while still offline. Chain continuity across the outage comes from seal-then-route plus adoption in the writer, so the store ends up with one session rather than two fragments. * `rootsign-admin sync` replays the spool through the batch endpoint, sharing a transport-agnostic core (rootsign/replay.py) with the replay-pending command still to come. DUPLICATE_EVENT counts as delivered, so a partial upload resumes; the walk stops at the first hard failure because these envelopes are a chain. The replay client sets enable_spool=False: a sync that failed over would append the records it is reading back into the file it is uploading. * The loss ladder (ADR-013 D4a) for when the disk fails too. Telemetry drops with accounting — one CRITICAL, an in-memory ledger, and a chain gap that makes the loss provable — while control records (pending actions, approvals) raise HiTLPersistenceError before the gated tool runs. An approval is the control, not telemetry. * Three verdicts (ADR-013 D4b). VALID / TAMPERED / INCOMPLETE, with one shared precedence rule in rootsign/verdict.py used by both verifiers: a break explained by a sequence gap reads INCOMPLETE, a break between contiguous records reads TAMPERED, and both together read TAMPERED with the gaps still reported. Verifiers re-anchor after a gap-explained break instead of stopping, so one dropped record cannot mask every alteration after it. `valid` survives as a derived property; exit codes are 0/1/2 at both verify sites. Wire version moves to 1.1 (the sealed chain fields on ACTION_RECORD), defined once in rootsign.ingest.schemas. Postgres rejects a sealed record rather than silently recomputing — recompute would fork the chain and leave the client holding a self_hash the store never stored. pyproject carries the new `cloud` extra; the core install stays at four dependencies. Sprint B W2 (T2.1-T2.8). ADR-013.
A verified hash chain answers a developer's question. The people who actually need this artefact ask a different one — what did the agent do, who approved it, and can I trust this file — and cannot read JSONL to find out. That artefact gap, not the capture gap, is what keeps RootSign filed as a developer tool inside partner organisations. `rootsign export` writes one directory per session: three JSON documents (verification, timeline, redaction) plus a manifest, with report.md and report.html rendered from that JSON and nothing else. If a fact is not in the machine truth it cannot appear in the human rendering, which also means the Phase 2 dashboard starts from a schema that already exists rather than inventing one under deadline. The honesty rules are the design, not decoration: * Records after a chain break are `unverified`, never `verified` — the walk stopped there and proved nothing about them. * A hash-only session says "payload previews not retained for this session" in words rather than showing empty fields. * redaction.json reports the sentinel paths it can see and explicitly does not claim which rule fired, because that is not recorded anywhere and never will be for records already captured. * --redact-previews withholds every field carrying stored content — not just the two obvious previews, but the context a human was shown at a checkpoint and captured decision summaries too — and names what it withheld, so a recipient knows what to ask for. The bundle is self-verifying and honest about the limit: manifest.json hashes every other file, and `--check` re-hashes them plus flags files that are present but unlisted. That proves internal consistency only — an attacker who edits a file and updates the manifest passes it — so the manifest's own digest is printed at export and on both --check outcomes, because comparing it out of band is the actual verification step. tests/fixtures/evidence_bundle_v1.json freezes the schema, generated from a pinned session that includes a rejected HiTL action; the verdict vocabulary is pinned as schema rather than as a value, so the first gap-bearing session cannot force a bundle v1.1. CONTRIBUTING documents regeneration and when a bump is required instead. Also here, because they share files with the export surface: the CI no-extras block now exercises `rootsign export --local` / `--check` and `rootsign-admin sync` on a bare install, and rootsign/sdk/cli.py carries the W2 verdict exit codes alongside the new command. Sprint B W3 (T3.1-T3.6). ADR-014.
Two new sections and one corrected claim. "Evidence bundles" shows the command, the bundle contents, and two screenshots of a real report — the verdict banner and chain table, then further down a GBP 250,000 transfer the agent proposed, the human who refused it, and the context they were shown. That second frame is the product argument in one image. "Offline & sync" tells the kill-network story: the agent never notices, spooled records verify offline, `rootsign-admin sync` uploads them when connectivity returns, and the chain spans the outage. It also states the uncomfortable half — a file that spooled mid-session verifies INCOMPLETE, not VALID, because the records from before the outage are not in it, and "this file is the whole session" is not a claim anyone can make offline. Corrected: verify's exit codes are 0/1/2, not 0/1, and HttpIngestClient is described as shipping rather than "lands in Phase 2". The quickstart gains its final line (`rootsign export --local ...`), and test_show_hn_quickstart.py — the gate that keeps README claims honest — now covers it: the bundle's file set must match the contents table, and the anchor printed at export must be the value `--check` reports back. Screenshots are embedded by raw.githubusercontent URL, matching demo.gif, because PyPI renders this README but does not resolve relative links. Sprint B W4 (T4.1).
The example stopped at `verify`, which is where a developer's interest ends and everyone else's begins. It now prints the export command too, and the demo tape has a fourth beat so the GIF ends on the manifest hash rather than on a verdict — that hash is the artefact a reader actually has to do something with. Two fixes found by running the thing rather than reading it: * The example printed a JSONL path unconditionally. With ROOTSIGN_BACKEND=postgres (a `.env`, an exported variable) the records go to the database and that path names a file which does not exist — followed by two commands that then fail. It now prints the session-id form when the backend is not jsonl. * `export` printed its "record this hash" instruction as one 130-column line, which wrapped mid-word in any normal terminal — including the GIF. Split into two lines; the quoted output in both READMEs follows. demo.gif re-recorded (560px tall now, four beats, 0.22 MB). Sprint B W4 (T4.2).
Both ADRs shipped as designed and move to Accepted. ADR-014 gains one implementation note rather than a quiet edit: its scope note promised `redaction.json` would carry "the active rule-set identity from configuration", and that turned out to be too generous — RedactionConfig is passed per call site, never read from settings, so at export time nothing knows what ran. The field ships null with the reason in `rule_set.provenance`, because naming a plausible rule set would be the bundle inventing provenance, which is the failure that decision exists to prevent. The release notes lead with what the two features are *for* — the artefact gap and an unreachable endpoint — and carry the three things a reader has to act on: `verify` now exits 2 for INCOMPLETE (a check written `== 1` stops catching missing records), `VerifyResult.valid` is derived rather than stored, and a store speaking the wire protocol must deploy 1.1 before a client sends sealed records. Known limits are stated rather than omitted: cloud-sourced export waits on a server read API, cloud HiTL is unsupported, a mid-session spool file verifies INCOMPLETE, and content hashes prove integrity but not authorship. The version bump stays out of this commit — per the release process it belongs to the release/vX.Y.Z PR that triggers the OIDC publish. Sprint B W4 (T4.3).
…es not exist T2.4c added `db.expunge_all()` after the Core UPDATE/DELETE in the verdict-parity damage helper, on the theory that `expire_on_commit=False` left the identity map holding pre-damage instances — and recorded that tests/integration/test_cross_backend_hash.py only passed because the weak identity map GC'd its stale objects first. That theory is wrong. SQLAlchemy 2.0 treats `session.execute(update(Model)…)` as ORM-enabled DML and synchronizes the identity map (`synchronize_session="auto"`). A probe holding a strong reference to the row observes the new value in memory with no expire, no expunge and no commit, and a re-select returns that same synchronized object. So the call was unnecessary, and worse, its comment documented a hazard that does not exist — which is how the same wrong conclusion gets reached twice. Both are gone, replaced by a note recording what actually happens and naming the case that would need care: a raw `text()` UPDATE is not ORM-enabled and does not synchronize. No change to test_cross_backend_hash.py: it was never at risk. Sprint B W4 (T4.5).
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
Both changes come from code-quality findings on PR #35. `on_progress` accepts a sync or an async callback, and the code decided which by `result is not None`. That is the wrong question: a sync callback that happens to return a value — `lambda i, r: seen.append(i) or True`, a logger that returns self — would be awaited and raise "object … can't be used in 'await' expression", crashing the caller inside their own progress reporting. `inspect.isawaitable` asks the question that was actually meant. The existing tests passed only because `list.append` returns None; two new ones cover a returning sync callback and a genuinely async one, and the first fails against the old predicate. The bot's "statement has no effect" on `await result` is a false positive — awaiting a coroutine is the effect — but it pointed at the line worth looking at, which is the useful outcome. Second finding, on tests/unit/test_chain_state.py: the module was imported both ways (`from rootsign.chain_state import …` at the top, a local `import rootsign.chain_state as chain_state` inside one test) to hand-roll a save/restore of `uuid4`. `monkeypatch.setattr` with a string target needs no second import and restores the attribute even when an assertion fails partway — the same single-import-style fix as 02afa70.
oabolade
added a commit
that referenced
this pull request
Aug 22, 2026
Evidence bundles, the cloud transport with its offline spool, and the published wire spec (PR #35). Release notes are already on main at docs/releases/v0.3.0.md. README status moves with it: the version line stops promising v0.3.0 as future work, and the 1.6 row flips from 🚧 to ✅. Verified locally before pushing: `uv build` produces rootsign-0.3.0-{whl,tar.gz}, the wheel carries Version: 0.3.0, the `cloud` extra resolves to httpx, all eight migrations ship inside the package, and the seven new modules are present. `rootsign version` reports 0.3.0 after a reinstall, so SDK_VERSION — which reads distribution metadata — will stamp envelopes correctly.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Targets v0.3.0. Additive throughout: a new
cloudextra, new commands, no default changes. Existing installs need no migration and no code changes.Two things ship.
rootsign exportturns a session into a directory a compliance officer can read — the artefact gap, not the capture gap, is what keeps RootSign filed as a developer tool inside partner organisations. And the cloud transport arrives ahead of the backend it talks to, with an offline spool that means an unreachable endpoint costs nothing.Seven commits, one per workstream, reviewable in order:
e44fa15docs/ingest-spec-v1.mdas the wire contract5e281cfHttpIngestClient, offline spool,rootsign-admin sync, loss ladder, three verdicts (ADR-013)6df093a--check, frozen schema (ADR-014)8b7ac2960893f53395367docs/releases/v0.3.0.md3b5a0fdexpunge_allthat guarded a hazard that does not existWhat to look at closely
pyproject.tomlno longer omitsrootsign/cli.pyfrom coverage. That omission predated real logic living on the operator CLI;syncchanged it. Total is ~91% against the 85 gate, andcli.pyitself is 62% — every remaining miss is a docker/alembic wrapper needing a live daemon. This is a gate-policy change, not a code change, so it deserves a deliberate yes.verifyexit codes are now 0/1/2.2is INCOMPLETE (a record is missing) alongside1for TAMPERED (a record was altered). Anything checking!= 0is unaffected; anything checking== 1silently stops catching incomplete chains. Called out in the release notes.VerifyResult.validis derived, not stored — two fields that must agree eventually disagree, and this one decides whether an auditor trusts a log. Constructing one now takesverdict=.Wire version moves to 1.1 (the sealed chain fields on
ACTION_RECORD). Per spec §9 that deploys store-first: a 1.0-era store rejects an envelope actually carrying the new fields, because per-event payloads forbid extras.Design points worth knowing
syncthe store holds one unbroken session, not two fragments.require_approval=Truecall raises before the gated tool runs. An approval is the control, not telemetry.unverified, notverified.--checkprints the manifest hash on both outcomes, because re-hashing files proves internal consistency only — an attacker who also updates the manifest passes it.export --checkandexport --localare DB-free, asserted by the no-extras contract suite running the real commands with sqlalchemy hidden.Testing
847 passed, 6 skipped. New: 3 contract/cloud tests, verdict parity across both verifiers, kill-network and dead-spool integration suites, the export bundle/report/check/CLI/golden suites, and a golden fixture freezing the bundle schema at v1.0 (
ROOTSIGN_UPDATE_GOLDEN=1regenerates; CONTRIBUTING documents when a version bump is required instead).tests/integration/test_show_hn_quickstart.py— the gate that keeps README claims honest — now covers the export step: the bundle's file set must match the README's contents table, and the anchor printed at export must be the value--checkreports back.Deliberately not here
The
pyprojectversion bump. Per the release process that belongs to therelease/v0.3.0PR whose merge fires the OIDC publish;docs/releases/v0.3.0.mdis written and ready for--notes-file.Website copy for v0.3.0 is staged on a branch in the site repo and must not merge until the tag exists — the notification bar links to
/releases/tag/v0.3.0.🤖 Generated with Claude Code