Skip to content

Bind reports to exact target identity - #95

Open
pdurlej wants to merge 2 commits into
openclaw:mainfrom
pdurlej:codex/target-identity
Open

Bind reports to exact target identity#95
pdurlej wants to merge 2 commits into
openclaw:mainfrom
pdurlej:codex/target-identity

Conversation

@pdurlej

@pdurlej pdurlej commented Aug 23, 2026

Copy link
Copy Markdown

Summary

  • bind exact npm:<semver> targets to the installed OCM runtime's npm integrity
  • require a canonical SHA-512 SRI that decodes to exactly 64 bytes; malformed or truncated values remain version_only
  • add targetIdentity to reports, run and matrix receipts, and bundle manifests
  • preserve report generation instead of inferring exactness from the requested selector

This lets downstream evidence consumers prove that the tested candidate is the candidate they intend to install.

Verification

  • PATH=<local-ocm>:$PATH npm run check:full — 280/280 checks passed
  • npm run pack:release — release artifact SHA-256 b51b21f0eb69ca0b0e6d8c2a9ef0d0aea3d81b34e16c5afddf35a3cf12531125
  • focused self-check rejects both truncated and non-canonical SHA-512 SRI values

Redacted OCM behavior proof

Exact HEAD: b909e045cac8e7276fd32aee680366efb8ce3079

A disposable OCM-backed execution ran release-runtime-startup against exact target npm:2026.7.1-2:

  • run ID: kova-260823-221103-b5f536
  • receipt, report, and bundle manifest emitted the same requested selector, resolved version, and integrity
  • integrity algorithm: sha512; decoded length: 64 bytes
  • redacted integrity fingerprint: sha256:9fb2f91a06e694c9eea4309901ecb170c584e1f09f85689fe13a312ea4ae47ca
  • report identity equals bundle-manifest identity: true
  • bundle SHA-256: a659fa8d03c2206e214aac5a29f994ede92eee6b3ddf2e39fa8b1ba94fdac60e
  • all 15 required scenario evidence-ledger entries passed

The scenario itself remained FAIL because cold readiness was 32,927 ms against a 30,000 ms performance threshold. That unrelated performance failure is retained rather than presented as a green scenario.

@clawsweeper

clawsweeper Bot commented Aug 23, 2026

Copy link
Copy Markdown

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

@clawsweeper clawsweeper Bot added merge-risk: 🚨 other 🚨 Merging this PR has meaningful risk outside the owned taxonomy. P2 Normal priority bug or improvement with limited blast radius. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Aug 23, 2026
@clawsweeper

clawsweeper Bot commented Aug 23, 2026

Copy link
Copy Markdown

Codex review: needs changes before merge. Reviewed August 26, 2026, 4:29 PM ET / 20:29 UTC.

ClawSweeper review

What this changes

The PR adds integrity-backed identity metadata for exact npm runtime targets to Kova reports, JSON receipts, and artifact bundle manifests.

Merge readiness

⚠️ Ready for maintainer review - 3 items remain

Keep open: the integrity-binding implementation and OCM-backed proof are useful, but the still-unfixed P3 documentation example promises a null field that moving-selector reports do not emit.

Priority: P2
Reviewed head: b909e045cac8e7276fd32aee680366efb8ce3079

Review scores

Measure Result What it means
Overall readiness 🐚 platinum hermit (4/6) The real runtime evidence and focused implementation are solid; one small documentation-contract correction remains.
Proof confidence 🐚 platinum hermit (4/6) Sufficient (live_output): The PR body supplies redacted after-fix OCM-backed evidence that an exact npm target produced matching identity data in the receipt, report, and bundle manifest.
Patch quality 🐚 platinum hermit (4/6) 1 actionable review finding remain.

Verification

Check Result Evidence
Real behavior Verified Sufficient (live_output): The PR body supplies redacted after-fix OCM-backed evidence that an exact npm target produced matching identity data in the receipt, report, and bundle manifest.
Evidence reviewed 5 items PR scope: The branch changes nine files with 283 additions, including report construction, both CLI receipts, bundle manifests, contract checks, and schema documentation.
Moving-selector contract: The self-check explicitly asserts that a moving release selector has no target identity, while the report builder conditionally omits the property when identity resolution returns null.
Documentation mismatch: The generic top-level example uses target runtime:stable but adds targetIdentity: null, contradicting the documented and tested omission for moving selectors.
Findings 1 actionable finding [P3] Remove the nonexistent moving-target identity field
Security None None.

Live Verification

Command: npm run test:snapshots

Result: FAIL (failed) — execution before step 1 run: sh -lc pnpm install --ignore-scripts --frozen-lockfile failed: ! Corepack is about to download https://registry.npmjs.org/pnpm/-/pnpm-11.24.0.tgz

sh -lc pnpm install --ignore-scripts --frozen-lockfile failed: ! Corepack is about to download https://registry.npmjs.org/pnpm/-/pnpm-11.24.0.tgz

Assertions:

  • FAIL expect_output: summary:

How this fits together

Kova executes OpenClaw runtime scenarios and emits reports, CLI receipts, and artifact bundles for evidence consumers. Target selection feeds an optional identity record into each output surface.

flowchart LR
  A[Target selector] --> B[Target planning]
  B --> C[Runtime identity lookup]
  C --> D[Run report]
  D --> E[JSON receipt]
  D --> F[Artifact bundle]
  E --> G[Evidence consumer]
  F --> G
Loading

Before merge

  • Remove the nonexistent moving-target identity field (P3) - This remains the prior finding: the generic runtime:stable report example shows targetIdentity: null, but moving selectors produce no identity and buildRunReport omits the property. Remove this field from the example so consumers do not learn an output shape they will not receive.
  • Resolve merge risk (P1) - Merging as-is leaves consumers with documentation that depicts targetIdentity as a null field for runtime:stable even though moving-selector reports and receipts omit it.
  • Complete next step (P2) - The sole remaining P3 finding is a mechanical, one-file documentation repair suitable for an automated patch attempt.

Findings

  • [P3] Remove the nonexistent moving-target identity field — docs/REPORT_SCHEMA.md:27
Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Changed surface 9 files, +283/-0 lines The identity record is propagated across reports, receipts, manifests, self-checks, snapshots, and documentation.
Production versus test growth production +96, tests +145, docs +42 Most added lines are validation and contract coverage, proportionate to a new evidence schema.

Merge-risk options

Maintainer options:

  1. Align the optional-field documentation (recommended)
    Remove the field from the generic moving-selector example so the documented JSON shape matches serialized reports and receipts.
Copy recommended automerge instruction
@clawsweeper automerge

Special instructions:
Remove targetIdentity from the generic runtime:stable report example; retain the exact npm target examples and do not change report serialization.

Technical review

Best possible solution:

Keep identity optional for moving selectors and remove targetIdentity from the generic runtime:stable report example while retaining the exact npm examples.

Do we have a high-confidence way to reproduce the issue?

Yes. Source inspection establishes that moving selectors resolve to no identity while the generic moving-selector documentation shows a null-valued field.

Is this the best way to solve the issue?

No. The implementation and schema example need one consistent optional-field contract; omitting the field from the generic example is the narrowest repair.

Full review comments:

  • [P3] Remove the nonexistent moving-target identity field — docs/REPORT_SCHEMA.md:27
    This remains the prior finding: the generic runtime:stable report example shows targetIdentity: null, but moving selectors produce no identity and buildRunReport omits the property. Remove this field from the example so consumers do not learn an output shape they will not receive.
    Confidence: 0.99

Overall correctness: patch is correct
Overall confidence: 0.98

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 1fe2f4081877.

Labels

Label changes:

  • add merge-risk: 🚨 compatibility: The documented moving-selector schema does not match the conditional serialized output.

Label justifications:

  • P2: This is a bounded evidence-contract improvement with downstream consumer impact.
  • merge-risk: 🚨 compatibility: The documented moving-selector schema does not match the conditional serialized output.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The PR body supplies redacted after-fix OCM-backed evidence that an exact npm target produced matching identity data in the receipt, report, and bundle manifest.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body supplies redacted after-fix OCM-backed evidence that an exact npm target produced matching identity data in the receipt, report, and bundle manifest.

Evidence

Acceptance criteria:

  • [P1] npm run test:snapshots.

What I checked:

  • PR scope: The branch changes nine files with 283 additions, including report construction, both CLI receipts, bundle manifests, contract checks, and schema documentation. (src/target-identity.mjs:20, b909e045cac8)
  • Moving-selector contract: The self-check explicitly asserts that a moving release selector has no target identity, while the report builder conditionally omits the property when identity resolution returns null. (src/selfcheck.mjs:26238, b909e045cac8)
  • Documentation mismatch: The generic top-level example uses target runtime:stable but adds targetIdentity: null, contradicting the documented and tested omission for moving selectors. (docs/REPORT_SCHEMA.md:27, b909e045cac8)
  • OCM contract dependency: The implementation directly executes OCM's runtime-list JSON command and reads its installed-runtime integrity fields, so its exact-target claim depends on that OCM output contract. (src/target-identity.mjs:26, b909e045cac8)
  • Current-main status: The PR head is not an ancestor of current main, so the central target-identity behavior is not already implemented on the default branch. (b909e045cac8)

Likely related people:

  • Vincent Koc: Current-main blame attributes the report-construction contract to this author, and their release-contract work also touched the report-schema history. (role: original report-output author; confidence: high; commits: 80a88fc760ba; files: src/run/report-finalization.mjs, docs/REPORT_SCHEMA.md)

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

  • Remove targetIdentity from the generic moving-selector schema example.

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (9 earlier review cycles; latest 8 shown)
  • reviewed 2026-08-23T22:16:56.943Z sha b909e04 :: needs maintainer review before merge. :: none
  • reviewed 2026-08-24T04:08:58.742Z sha b909e04 :: needs maintainer review before merge. :: none
  • reviewed 2026-08-24T10:07:51.631Z sha b909e04 :: needs changes before merge. :: [P3] Remove the nonexistent moving-target identity field from the schema example
  • reviewed 2026-08-24T15:07:45.287Z sha b909e04 :: needs changes before merge. :: [P3] Remove the moving-target identity field from the generic report example
  • reviewed 2026-08-24T22:12:17.866Z sha b909e04 :: needs changes before merge. :: [P3] Remove the invalid moving-target identity field
  • reviewed 2026-08-25T08:03:57.112Z sha b909e04 :: needs changes before merge. :: [P3] Remove the nonexistent moving-target identity field
  • reviewed 2026-08-25T22:09:30.465Z sha b909e04 :: needs changes before merge. :: [P3] Remove the nonexistent moving-target identity field
  • reviewed 2026-08-26T12:02:48.178Z sha b909e04 :: needs changes before merge. :: [P3] Remove the nonexistent moving-target identity field

@pdurlej

pdurlej commented Aug 23, 2026

Copy link
Copy Markdown
Author

Fixed on exact HEAD b909e045cac8e7276fd32aee680366efb8ce3079: SHA-512 integrity must now be canonical and decode to exactly 64 bytes, with truncated and non-canonical regression coverage. The PR body now includes redacted after-fix OCM execution proof across the receipt, report, and bundle manifest.

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Aug 23, 2026

Copy link
Copy Markdown

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event exact_review_queue).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

@clawsweeper clawsweeper Bot added proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. merge-risk: 🚨 other 🚨 Merging this PR has meaningful risk outside the owned taxonomy. labels Aug 23, 2026
@pdurlej

pdurlej commented Aug 23, 2026

Copy link
Copy Markdown
Author

Contributor-side work is complete on exact HEAD b909e045cac8e7276fd32aee680366efb8ce3079.

  • npm run check:full: 280/280 passed with the local OCM in PATH
  • npm run pack:release: passed
  • disposable OCM-backed behavior proof is documented in the PR body
  • the completed ClawSweeper review artifact reports: patch correct, security cleared, real behavior proof sufficient, no actionable findings

The durable ClawSweeper comment/labels still show the previous head because its publication step did not update the PR, but the review run itself completed successfully. Upstream CI is currently action_required for this external-fork PR.

We are waiting for a Kova maintainer to approve/run the upstream CI and perform the normal merge review. No further contributor changes are currently indicated.

@clawsweeper clawsweeper Bot added the merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. label Aug 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. P2 Normal priority bug or improvement with limited blast radius. proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant