fix: validate frame Dictionary_ID against the supplied dictionary - #58
Conversation
…ionary Real libzstd sets a non-zero Dictionary_ID by default when compressing with a proper Zstandard-format dictionary (RFC 8878 §5), and decoders are expected to validate it (RFC 8878 §3.1.1.3). kzstd read the field but discarded it, so decoding with the wrong dictionary surfaced as a confusing generic corruption error instead of a clear mismatch. ParsedDictionary now captures a trained dictionary's own embedded Dictionary_ID (0 for raw content dictionaries, matching the wire's own "not specified" sentinel). The decoder compares it against a frame's declared Dictionary_ID and throws a ZstdException naming the mismatch when both are present and differ. Frames with no declared ID (kzstd's own encoder never sets one) or dictionaries with no embedded ID skip validation entirely, so existing callers are unaffected. Signed-off-by: James Rich <james.a.rich@gmail.com>
|
Warning Review limit reached
Next review available in: 59 minutes Limit details: You’ve used all 1 included review currently available under your plan. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (6)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Closes an RFC 8878 conformance gap found in a parity audit: the decoder read a frame's Dictionary_ID field (§3.1.1.3) but never validated it.
Real libzstd sets a real Dictionary_ID by default when compressing with a proper Zstandard-format dictionary (
zstd -D dictfile) and checks it on decode. kzstd previously decoded with whateverZstdDictionarythe caller passed regardless of what the frame declared, producing a generic corruption error deep in entropy decoding on a wrong-dictionary frame instead of a clear one.What changed
ParsedDictionarynow captures a trained dictionary's own embedded Dictionary_ID (0 for raw-content dictionaries, matching the wire's "no ID" sentinel).ZstdExceptionwith a message that names it as a dictionary-ID mismatch (not generic corruption).Testing
DictionaryIdValidationTest: pure parsing checks on every target (trained dict captures its embedded ID, raw-content/empty dicts read as ID 0, kzstd's own zero-ID frames still decode against a dictionary with a different ID).KzstdLibzstdInteropTest(zstd-jni oracle, real libzstd): compresses with real libzstd + the existing trained-dict fixture (already has a non-zero embedded ID), decodes with the correct dictionary (succeeds) and a different one (clearZstdException).No public API change — no
apiDumpneeded../gradlew buildpasses on every target except native test-binary linking, which hits a pre-existing, unrelated crash on this dev host (reproduced on unmodifiedmain, tracked in #56).