Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 38 additions & 16 deletions docs/evidence/issue-62/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,15 @@ payloads are redacted.
Gateway filtering beyond the observed prefix. The exact-version Desktop core
and Code Mode app-server controls pass.

The source snapshot is OpenAI Codex commit
9e552e9d15ba52bed7077d5357f3e18e330f8f38. At that revision, the dynamic
The retained Desktop capture is historical: it was captured on 2026-07-12
with Codex CLI `0.144.0-alpha.4`, source commit
`9e552e9d15ba52bed7077d5357f3e18e330f8f38`. It must not be relabeled as the
later CLI 0.146.0 release. The separate
`codex-0.146-source-contract.json` records the 0.146.0 source contract (tag
`rust-v0.146.0`, attested commit `e363b08c9175ac1cbe5893615dd2cb9ddf95043b`,
and exact binary hash) with `capture_status=not_observed` and
`qualification_status=unqualified`. At that
historical revision, the dynamic
tool protocol defines optional deferLoading; the dynamic handler maps true to
Deferred and missing or false to Direct. ToolExposure keeps Direct,
DirectModelOnly, Deferred, and Hidden distinct. Tool search is planned only
Expand Down Expand Up @@ -90,6 +97,10 @@ session, task, turn, call, item, request, or response identifiers.
The bounded audit establishes these additional facts without a restart,
reconnect, configuration write, or production-handler change:

Its candidate provenance is explicitly `capture_status=not_observed` and is
bound to the same 0.146 source contract; the retained historical capture
metadata remains recorded separately rather than being promoted to 0.146.

- Forty-three retained Sol transport rows resolve to three actual
model-visible planner surfaces. The largest retained surface includes the
base functions, collaboration namespace, goal functions, image generation,
Expand Down Expand Up @@ -123,6 +134,7 @@ Run the sanitizer with explicit bounded inputs and observation cutoffs:

```powershell
python scripts/audit_issue_62_runtime_artifacts.py `
--source-contract docs/evidence/issue-62/codex-0.146-source-contract.json `
--codex-log-db <codex-log-db> `
--gateway-db <gateway-telemetry-db> `
--model gpt-5.6-sol `
Expand All @@ -146,18 +158,28 @@ records one per-scope disposition for every taxonomy item the Codex CLI
exposes over the core Responses contract and the explicitly-deferred advanced
capabilities.

The artifact is bound to CLI floor `0.145.0` and to the candidate identity
derived from the existing sanitized artifacts (`cli_version=0.144.0-alpha.4`,
source commit `9e552e9d15ba52bed7077d5357f3e18e330f8f38`, official Responses
route). Because the captured CLI is below the floor, the generated
`qualification.ready_for_beta1` is `false` and the candidate is explicitly
marked `legacy_below_floor`; this evidence cannot be used as the beta.1
candidate. The generator rejects an explicitly supplied CLI/source value that
does not match the trace, binds route/provider/model fields across trace and
wire fixtures (including pre/post models, catalog binding, and route profile),
and records a canonical-LF SHA-256 manifest for all three input artifacts. It
never fabricates a capability disposition for a gate the artifacts do not
qualify.
The artifact is bound to CLI floor `0.146.0` and to the candidate identity
from the unobserved 0.146 source contract (`cli_version=0.146.0`, source
commit `e363b08c9175ac1cbe5893615dd2cb9ddf95043b`, candidate revision
`accab8ff6eb4d6ebd93cda84585fb5f6cb89da82`, official Responses route). The
historical trace and wire fixture remain explicitly bound as 0.144.0 evidence.
The audit carries the 0.146 source-contract provenance with
`capture_status=not_observed` and nested historical-capture metadata; it is not
a 0.146 runtime capture. The candidate is version-eligible, but
`qualification.ready_for_beta2` remains
`false`: planner completeness, clean current-binding cold start,
independently fingerprinted full pre/post request and response bodies,
non-streaming/terminal/error/hosted/unknown controls, and wire replay evidence
are still incomplete. This is evidence for #62's downstream gates, not a #65
qualification or capability unlock. The generator rejects an explicitly
supplied CLI/source value that does not match the trace, binds route/provider/
model fields across trace and wire fixtures (including pre/post models, catalog
binding, and route profile), and records a canonical-LF SHA-256 manifest for
all four input artifacts (including the source contract). The audit sanitizer
must receive that source-contract path so reruns retain the exact
`capture_status=not_observed` 0.146 provenance and nested historical 0.144
capture metadata. It never fabricates a capability disposition for a gate the
artifacts do not qualify.

The qualification also has a separate `wire_identity_replay` gate. A full
request/response fingerprint is not treated as replay proof by itself: a
Expand Down Expand Up @@ -219,7 +241,7 @@ reconciliation invokes the same drift check, then independently checks the
input fingerprints, rejects duplicate scopes, and requires each core scope to
point at its declared evidence path. A zero
`unclassified_core_items` count therefore describes vocabulary validity only;
`qualification.ready_for_beta1` is the separate completion gate. That gate also
`qualification.ready_for_beta2` is the separate completion gate. That gate also
consumes planner completeness, current-binding cold-start, full-wire
fingerprinting, non-streaming, and identity-replay statuses; item dispositions
alone cannot make an incomplete evidence set ready.
Expand Down Expand Up @@ -260,7 +282,7 @@ Evidence may use `cli_source_commit: null` with
not acceptable. If the npm provenance attestation has been independently
verified, its exact SLSA resolved-dependency commit may instead be recorded
with status `published` (for `0.146.0`, the attested release commit is
`e363b08c9175ac1cbe5893615dd2cb9ddf95043b2`).
`e363b08c9175ac1cbe5893615dd2cb9ddf95043b`).

## Isolated live-evidence sidecar lane

Expand Down
179 changes: 179 additions & 0 deletions docs/evidence/issue-62/codex-0.146-source-contract.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,179 @@
{
"schema_version": 1,
"fixture_kind": "codex_cli_source_contract",
"capture_status": "not_observed",
"qualification_status": "unqualified",
"captured_at": null,
"provenance": {
"cli_version": "0.146.0",
"source_commit": "e363b08c9175ac1cbe5893615dd2cb9ddf95043b",
"cli_source_tag": "rust-v0.146.0",
"cli_source_commit_status": "published_attested",
"cli_binary_sha256": "bc343ba420dc2e2e9f59e6fc5e5bf0aae1cd8c771fc319665241fc9c0271fddb",
"candidate_revision": "accab8ff6eb4d6ebd93cda84585fb5f6cb89da82"
},
"runtime_wire_surface": {
"source": "Codex CLI 0.146.0 ToolSpec and ResponseItem source contract",
"declaration_family_order": [
"plain_function",
"custom_freeform",
"namespace",
"client_executed_tool_discovery",
"selected_provider_hosted",
"unknown_future_kind"
],
"declaration_families": [
{"family": "plain_function", "runtime_type": "function", "wire_declaration_type": "function", "observed": false, "observation": "not_observed_source_contract_only", "executor": "codex_client", "loss_boundary": "preserve declaration and inverse call/result/history IDs"},
{"family": "custom_freeform", "runtime_type": "custom", "wire_declaration_type": "custom", "observed": false, "observation": "not_observed_source_contract_only", "executor": "codex_client", "loss_boundary": "preserve declaration and inverse call/result/history IDs"},
{"family": "namespace", "runtime_type": "namespace", "wire_declaration_type": "namespace", "observed": false, "observation": "not_observed_source_contract_only", "executor": "codex_client", "loss_boundary": "preserve declaration and inverse call/result/history IDs"},
{"family": "client_executed_tool_discovery", "runtime_type": "tool_search", "wire_declaration_type": "tool_search", "observed": false, "observation": "not_observed_source_contract_only", "executor": "codex_client", "loss_boundary": "discovery request/result stays client-executed"},
{"family": "selected_provider_hosted", "runtime_type": "web_search", "wire_declaration_type": "web_search", "observed": false, "observation": "not_observed_selected_provider_control_required", "executor": "selected_provider", "loss_boundary": "optional unsupported hosted capability is omitted; required capability fails visibly"},
{"family": "unknown_future_kind", "runtime_type": "unknown", "wire_declaration_type": "<unknown>", "observed": false, "observation": "opaque_sentinel_only", "executor": "unknown", "loss_boundary": "retain tag and opaque payload; do not normalize"}
],
"request_shape": {
"protocol": "responses",
"streaming_fields": [
"model",
"instructions",
"input",
"tools",
"tool_choice",
"parallel_tool_calls",
"reasoning",
"store",
"stream",
"stream_options",
"include",
"service_tier",
"prompt_cache_key",
"text",
"client_metadata"
],
"representative": {
"model": "gpt-5.6-sol",
"input": "<redacted>",
"tools": "<redacted>",
"tool_choice": "auto",
"parallel_tool_calls": false,
"stream": true,
"store": false
},
"non_streaming_control": {
"stream": false,
"response_body": "<redacted>",
"captured": false,
"status": "unqualified"
}
},
"response_shape": {
"response_item_types": [
"message",
"reasoning",
"function_call",
"function_call_output",
"custom_tool_call",
"custom_tool_call_output",
"tool_search_call",
"tool_search_output",
"web_search_call",
"local_shell_call",
"compaction",
"context_compaction",
"unknown"
],
"stream_event_order": [
"response.created",
"response.in_progress",
"response.output_item.added",
"response.output_text.delta",
"response.function_call_arguments.delta",
"response.custom_tool_call_input.delta",
"response.function_call_arguments.done",
"response.custom_tool_call_input.done",
"response.output_item.done",
"response.reasoning_summary_part.added",
"response.reasoning_summary_text.delta",
"response.reasoning_summary_text.done",
"response.completed"
],
"terminal_events": [
"response.completed",
"response.incomplete",
"response.failed"
],
"error_shape": {
"event": "response.failed",
"response": {"id": "response_error_001", "status": "failed", "error": "<redacted>"},
"classification": "unqualified"
}
},
"declaration_family_examples": {
"plain_function": {
"declaration": {"type": "function", "name": "function_alpha", "parameters": "<redacted>"},
"call": {"type": "function_call", "item_id": "item_call_function_001", "call_id": "call_function_001", "arguments": "<redacted>"},
"result": {"type": "function_call_output", "item_id": "item_output_function_001", "call_id": "call_function_001", "output": "<redacted>"},
"history": {"call_id": "call_function_001", "call_item_id": "item_call_function_001", "output_item_id": "item_output_function_001"},
"streaming": {"added": "response.output_item.added", "delta": "response.function_call_arguments.delta", "arguments_done": "response.function_call_arguments.done", "done": "response.output_item.done", "terminal": "response.completed", "event_order": ["response.output_item.added", "response.function_call_arguments.delta", "response.function_call_arguments.done", "response.output_item.done", "response.completed"]},
"terminal": {"event": "response.completed", "classification": "not_observed"},
"error": {"event": "response.failed", "classification": "unqualified"},
"loss_boundary": "preserve declaration and inverse call/result/history IDs"
},
"custom_freeform": {
"declaration": {"type": "custom", "name": "custom_alpha", "format": "<redacted>"},
"call": {"type": "custom_tool_call", "item_id": "item_call_custom_001", "call_id": "call_custom_001", "input": "<redacted>"},
"result": {"type": "custom_tool_call_output", "item_id": "item_output_custom_001", "call_id": "call_custom_001", "output": "<redacted>"},
"history": {"call_id": "call_custom_001", "call_item_id": "item_call_custom_001", "output_item_id": "item_output_custom_001"},
"streaming": {"added": "response.output_item.added", "delta": "response.custom_tool_call_input.delta", "input_done": "response.custom_tool_call_input.done", "done": "response.output_item.done", "terminal": "response.completed", "event_order": ["response.output_item.added", "response.custom_tool_call_input.delta", "response.custom_tool_call_input.done", "response.output_item.done", "response.completed"]},
"terminal": {"event": "response.completed", "classification": "not_observed"},
"error": {"event": "response.failed", "classification": "unqualified"},
"loss_boundary": "preserve declaration and inverse call/result/history IDs"
},
"namespace": {
"declaration": {"type": "namespace", "name": "namespace_alpha", "tools": [{"type": "function", "name": "function_beta", "parameters": "<redacted>"}]},
"call": {"type": "function_call", "item_id": "item_call_namespace_001", "call_id": "call_namespace_001", "namespace": "namespace_alpha", "arguments": "<redacted>"},
"result": {"type": "function_call_output", "item_id": "item_output_namespace_001", "call_id": "call_namespace_001", "output": "<redacted>"},
"history": {"call_id": "call_namespace_001", "call_item_id": "item_call_namespace_001", "output_item_id": "item_output_namespace_001", "namespace": "namespace_alpha"},
"streaming": {"added": "response.output_item.added", "delta": "response.function_call_arguments.delta", "arguments_done": "response.function_call_arguments.done", "done": "response.output_item.done", "terminal": "response.completed", "event_order": ["response.output_item.added", "response.function_call_arguments.delta", "response.function_call_arguments.done", "response.output_item.done", "response.completed"]},
"terminal": {"event": "response.completed", "classification": "not_observed"},
"error": {"event": "response.failed", "classification": "unqualified"},
"loss_boundary": "preserve declaration and inverse call/result/history IDs"
},
"client_executed_tool_discovery": {
"declaration": {"type": "tool_search", "execution": "client", "parameters": "<redacted>"},
"call": {"type": "tool_search_call", "item_id": "item_call_search_001", "call_id": "call_search_001", "execution": "client", "arguments": "<redacted>"},
"result": {"type": "tool_search_output", "item_id": "item_output_search_001", "call_id": "call_search_001", "execution": "client", "tools": "<redacted>"},
"history": {"call_id": "call_search_001", "call_item_id": "item_call_search_001", "output_item_id": "item_output_search_001", "executor": "codex_client"},
"streaming": {"added": null, "delta": null, "done": "response.output_item.done", "terminal": "response.completed", "event_order": ["response.output_item.done", "response.completed"]},
"terminal": {"event": "response.completed", "classification": "not_observed"},
"error": {"event": "response.failed", "classification": "unqualified"},
"loss_boundary": "discovery request/result stays client-executed"
},
"selected_provider_hosted": {
"declaration": {"type": "web_search", "executor": "selected_provider", "provider_scope": "selected_provider_only"},
"call": {"type": "web_search_call", "item_id": "item_call_hosted_001", "status": "<redacted>", "action": "<redacted>"},
"result": {"type": "web_search_call", "item_id": "item_output_hosted_001", "status": "<redacted>", "provider_scope": "selected_provider_only"},
"history": {"call_id": null, "call_item_id": "item_call_hosted_001", "output_item_id": "item_output_hosted_001", "executor": "selected_provider", "cross_provider_proxy": "forbidden"},
"streaming": {"added": "response.output_item.added", "delta": "<provider-defined>", "done": "response.output_item.done", "terminal": "response.completed", "event_order": ["response.output_item.added", "<provider-defined>", "response.output_item.done", "response.completed"]},
"terminal": {"event": "response.completed", "classification": "not_observed"},
"error": {"event": "response.failed", "classification": "unqualified"},
"loss_boundary": "optional unsupported hosted capability is omitted; required capability fails visibly",
"observed": false,
"status": "selected_provider_control_required",
"provider_scope": "selected_provider_only",
"cross_provider_proxy": "forbidden"
},
"unknown_future_kind": {
"declaration": {"type": "unknown", "tag": "unknown", "opaque_payload": "<redacted>"},
"call": {"type": "unknown", "tag": "unknown", "opaque_payload": "<redacted>"},
"result": {"type": "unknown", "tag": "unknown", "opaque_payload": "<redacted>"},
"history": {"call_id": null, "call_item_id": null, "output_item_id": null, "loss_rule": "retain opaque sentinel"},
"streaming": {"added": "unknown.future_event", "delta": "unknown.future_delta", "done": "unknown.future_done", "terminal": "response.completed", "event_order": ["unknown.future_event", "unknown.future_delta", "unknown.future_done", "response.completed"]},
"terminal": {"event": "response.completed", "classification": "not_observed"},
"error": {"event": "response.failed", "classification": "unqualified"},
"loss_boundary": "retain tag and opaque payload; do not normalize",
"observed": false,
"status": "opaque_sentinel_only"
}
}
}
}
1 change: 1 addition & 0 deletions docs/evidence/issue-62/codexhub-runtime-wire-fixture.json
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
"provenance": {
"cli_version": "0.144.0-alpha.4",
"source_commit": "9e552e9d15ba52bed7077d5357f3e18e330f8f38",
"captured_at": "2026-07-12T14:57:55+08:00",
"capture_id": "sanitized-current-side-capture"
},
"exposure_state_tags": ["Direct", "DirectModelOnly", "Deferred", "Hidden", "hosted-only", "host-unavailable"],
Expand Down
Loading