Skip to content

The traceability completeness gate has never fired: 60/60 release artifacts across v0.56-v0.60 have no linked sys-verification, and 'rivet coverage' prints instead of asserting #1091

Description

@avrabe

The gate the methodology names as the release blocker has never blocked anything

release-execution's traceability completeness gate is defined as the release-time audit: every approved/implemented artifact must have the full V closed — requirement → architecture → implementation, and up the right side (tests via verifies). "If a step here was skipped, that gate is where it surfaces as a blocker."

Measured on main with the CI-pinned rivet 0.23.0, externals stripped (this job's own conditions):

release-v0.56:   8/8  artifacts have NO linked sys-verification
release-v0.57:  16/16
release-v0.58:  11/11
release-v0.59:  17/17
release-v0.60:   8/8
                -----
                60/60   — five consecutive releases, 100%
sys2-has-verification    system-req    61 / 152    40.1%
V-closure: system-req (all 2 rules)    40.1%  [61/152]
Overall (weighted)                     83.8%

So v0.60 is not a regression — it matches five releases of practice exactly, which is why this is a process finding and not a release blocker.

Why nobody noticed

rivet coverage does run, inside the required Rivet Validation job:

      - name: Check coverage
        run: rivet coverage

No threshold, no assertion, and rivet coverage exits 0 at 40.1% (verified). It prints a report into a log nobody reads and the job goes green. The required job's real gates are the ERROR:-grep and the ARTIFACT_FLOOR — neither of which can see an artifact with no verifies edge, because an unlinked artifact produces no error and still loads.

This is the same shape as #1012 (0 broken cross-refs beside 50 link errors) and #1064 (a whole release file skipped on a WARN): a validator reporting success about work it never did. Third instance, and the first where the un-run work is the one the methodology calls the release gate.

What is NOT true here

The verification exists. These artifacts carry unicorn execution differentials, ld.lld-linked run evidence, Rocq proofs with red-first transcripts, mutation-killed gates, non-vacuity floors. v0.60's own RQ-60-VFPPRESSURE reached verified on hardware execution by a sibling repo.

What is missing is the typed link that makes any of it queryable. The mechanism works elsewhere — 61 of 152 system-req artifacts are covered, so sys-verification + verifies is a live, functioning pattern that release artifacts simply never adopted. This is "verification invisible to the trace graph", not "unverified work". The distinction matters because the fix is cheap and the alternative reading is alarming and wrong.

Proposed

  1. Make the coverage step assert. A floor on sys2-has-verification (or on V-closure: system-req), pinned like ARTIFACT_FLOOR and raised in the PR that improves it. A number that may only rise. Without this, any backfill silently rots again.
  2. Backfill v0.60's eight as the pilot — each already has its evidence named in done-when/verified-by, so the sys-verification artifact is a transcription, not new work. That also tells us the real per-artifact cost before committing to 60.
  3. Then decide about the other 52 deliberately: backfill, or state in the methodology that release artifacts are exempt and why. Either is defensible; the current state — a documented gate that has never once fired — is not.

Not a v0.60 blocker: the gap predates it by four releases and the underlying evidence is real and cited. But v0.60's release notes should say the V is not closed on the right-hand side rather than let "traceability completeness" be assumed.

Found while running the release-execution traceability gate before cutting v0.60 — the first time it appears to have actually been run at release time.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions