Bound evidence per kind instead of failing the packet - #62
Merged
Conversation
The evidence builder raised as soon as the aggregate 400-line or 64 KiB limit was reached, which aborted the entire capture including healthy jobs in the same run. Give each evidence kind a line and byte budget, clamped by what the packet has already used, and keep a deterministic prefix that fits both. Trimming is recorded in sanitization.truncated and declared in missing_evidence rather than discarding the packet. Verified against run 31668261705 attempt 2: both packets stay inside Bar's budgets, are byte-identical across repeated live collections, and are accepted by Bar's ingestion validator.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bounds collector evidence per kind so a large job log trims instead of
failing the capture.
Problem
EvidenceBuilder.validate_limitsraised once the aggregate 400-line or64 KiB ceiling was reached. That aborted
build_packet_for_job, andbecause
collect_packetsbuilds every packet in one comprehension, asingle oversized job discarded the packets for healthy jobs in the same
run.
Change
consumed, so the aggregate ceilings cannot be exceeded by construction.
bounded_evidence_textkeeps a deterministic prefix satisfying both.EvidenceBuilder.addreturns a boolean instead of raising, so an itemthat sanitizes to nothing is skipped rather than killing the packet.
sanitization.truncatedand adds amissing_evidencedeclaration.
Budgets total exactly the aggregate ceilings: 3 log windows at 56 lines
and 12 KiB, one workflow snippet at 48 and 8 KiB, four diffs at 40 and
5 KiB.
Verification against run 31668261705 attempt 2
Packets built from live GitHub data, not fixtures.
WinError 32,page-fts-v1.sqlite, andPermissionErrorin the Windows packet;Forced include not foundand
.linkignorein the package packet.lines 100-159 trimmed to 56 lines cites 100-155, and the last retained
line is exactly line 155.
other job's packet.
Full suite 1041 passed, ruff clean, actionlint clean, release hygiene
passed.
Known follow-ups, not addressed here
sanitization.truncatedis effectively always true, because per-kindcaps trim routinely. Both packets above report truncation while using
28% and 15% of the byte budget, so the signal does not distinguish
a real ceiling from a routine per-item cap.
validate_limitsno longer guarantees at least one evidence item,while Bar rejects an empty evidence array.
job still prevents the other job's packet from being sent.