Skip to content

Count unreadable records in three, not one - #6

Merged
UnbreakableMJ merged 1 commit into
mainfrom
fix/split-unreadable-record-counters
Aug 7, 2026
Merged

Count unreadable records in three, not one#6
UnbreakableMJ merged 1 commit into
mainfrom
fix/split-unreadable-record-counters

Conversation

@UnbreakableMJ

Copy link
Copy Markdown
Contributor

unknown_record counted three unrelated things: an unrecognized record type, a line that is not valid JSON, and a conversation record with no uuid. Only the first is what the counter is documented to signal.

Why this matters

The merge made torn lines indistinguishable from format changes. A session that read a transcript while its harness was still appending reported 56 unknown records — every one a partial line that was complete minutes later, with nothing about the format having moved. That report reached a user three times as "the transcript format may have changed."

Same transcript, after this change:

{"unknown_record": 0, "torn_line": 1, "missing_uuid": 0}

A signal that is wrong two times in three stops being read, which costs precisely the early warning the counter exists to give. It has earned its keep before — it surfaced three miscategorized Codex tool types — and that is worth protecting.

The split

counter meaning what to do
unknown_record unrecognized record type act on it — format moved, extend an allowlist
torn_line not valid JSON; interrupted write nothing to fix here; often transient
missing_uuid conversation record with no uuid the only one where a real turn was lost

Both readers report torn lines to the new counter. Only the Claude Code reader can produce missing_uuid — the Codex reader keys on record index, not uuid.

Docs

  • plugins/engram/commands/save-chat.md told the reader that any non-zero count meant the format had changed. That instruction is how the misreport reached a user; it now names all three counters and what each implies.
  • CLAUDE.md documented the merge as a deliberate decision ("it correctly flags genuinely torn lines"), so it is amended rather than left contradicting the code.

Compatibility

Breaking for envelope consumers. filtered.unknown_record returns a smaller number for the same transcript; the three counters sum to the old value. Noted in CHANGELOG.md under Unreleased.

Verification

cargo fmt --check, cargo clippy --all-targets -D warnings, 245 tests (181 unit + 64 integration) — all pass. Three tests cover the new behaviour, including one per reader asserting a torn line does not register as a format change.

🤖 Generated with Claude Code

https://claude.ai/code/session_01FCJbPm4FWS99y9utbUe13z

`unknown_record` counted three unrelated things: a record `type` engram does
not recognize, a line that is not valid JSON, and a conversation record with no
`uuid`. Only the first is what the counter is documented to signal — a format
change in a file engram does not own.

The merge made torn lines indistinguishable from format changes. A session that
read a transcript while its harness was still appending reported 56 unknown
records; every one was a partial line that was complete minutes later, and
nothing about the format had moved. Acting on that report meant looking for a
change that did not exist. A signal wrong two times in three stops being read,
which costs exactly the early warning the counter exists to give.

Split into `unknown_record` (unrecognized `type`, worth acting on),
`torn_line` (interrupted write, nothing to fix here, often transient), and
`missing_uuid` (a real turn dropped for want of a stable id). Both readers
report torn lines to the new counter; only the Claude Code reader can produce
`missing_uuid`, since the Codex reader keys on record index rather than uuid.

The slash command told a reader that any non-zero count meant the format had
changed, which is how the misreport reached a user in the first place; it now
names all three and what each implies. CLAUDE.md carried the merge as a
deliberate decision, so it is amended rather than left contradicting the code.

Callers reading `filtered.unknown_record` see a smaller number for the same
transcript; the three counters sum to the old value.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FCJbPm4FWS99y9utbUe13z
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@UnbreakableMJ
UnbreakableMJ merged commit 8cdb044 into main Aug 7, 2026
5 checks passed
@UnbreakableMJ
UnbreakableMJ deleted the fix/split-unreadable-record-counters branch August 7, 2026 21:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant