Skip to content

logmessaged: drop records too big for the msgq queue instead of dying - #312

Closed
PeterPhuTran wants to merge 10000 commits into
FrogAi:MAKE-PRS-HEREfrom
PeterPhuTran:logmessaged-queue-guard
Closed

logmessaged: drop records too big for the msgq queue instead of dying#312
PeterPhuTran wants to merge 10000 commits into
FrogAi:MAKE-PRS-HEREfrom
PeterPhuTran:logmessaged-queue-guard

Conversation

@PeterPhuTran

Copy link
Copy Markdown

Description

logmessaged re-publishes every log record onto the logMessage msgq socket. msgq
queues are 256KiB and msgq_msg_send() asserts that a message fits in a third of
the queue — so any record over ~85KB is a SIGABRT. The existing guard only rejected
records over 2MB, leaving an 85KB–2MB band where a single large log record kills the
daemon. And since a crashed logmessaged takes down the whole logging pipeline, the
crash itself can never be logged: the only symptom is a permanent
"Process Not Running: logmessaged" and silent loss of all swaglogs for the drive.

Hit in the field on 2026-07-10 (same device as #309/#311): updated logged a 130KB
git diff output record for a dirty working tree; logmessaged died seconds after
boot and stayed dead all day.

Fix: lower the guard to fit the actual queue capacity (MAX_PUBLISH_BYTES = 80KB).
Oversized records are still written to the on-disk log (the file write happens before
the publish) — they're only dropped from the msgq stream, exactly like the existing

2MB behavior.

Verification

New test_medium_log in system/tests/test_logmessaged.py, validated on a comma three:

  • red on stock: a 200KB record kills logmessaged (msgq assertion)
  • green with fix: logmessaged survives, later records still flow on the socket,
    and both records land in the on-disk log

Existing test_simple_log passes. Note: test_big_log fails on-device with and
without this change (it globs the live /data/log while a running logmessaged
rotates files — a pre-existing test-environment race; it passes in isolated CI).

Related: #311 flags logmessaged restart_if_crash as defense in depth; this PR
removes the crash trigger itself.

🤖 Generated with Claude Code

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.