Optional value capture with default-on sensitive-key redaction - #10
Merged
Conversation
dump_graph/spawn_dump grow values='none'|'builtins'|'repr' (default 'none': behavior and dump format unchanged, one falsy check per object), plus value_limit=128 and redact=True. Captured reprs land in a new object_repr table (object PK, repr, truncated, redacted); meta gains a capture_values column so dumps are self-describing. - tier 'builtins': exact str/bytes/int/float/complex/bool only; C-level repr, never runs user code. str/bytes slice before repr; int guarded against 3.11+ int_max_str_digits; stored text hard-bounded at value_limit. - tier 'repr': everything else via a bounded reprlib.Repr; user __repr__ runs mid-dump (accepted risk, fork isolation via spawn_dump), exceptions skip the row. - redaction post-pass (default on when capturing): any object referenced under a sensitive-looking edge label (key/secret/password/token/auth/..., attr and dict-key edges alike) has its repr replaced by a length-bucketed placeholder '<redacted len<=N>' (buckets 100, 500, 1000, 5000, ...) built from object.len; keys themselves stay readable, one sensitive inbound edge wins for aliased objects. Redaction is dump-time: the artifact never contains the secret text. - Reader.obj_repr (feature-detected via _table_names, old dumps fine), repr in object_summary, display-capped repr snippets in both Reader.object_label and Console._obj_label, Repr row in the web object panel. Known hole inherited from traversal: CPython untracks atomic-only dicts, so their key edges never materialize (values still captured, un-redacted under tier 'repr' container reprs). Tests keep the fixture dict tracked.
…as values CPython 3.14 introduced internal dicts that map interned identifier strings to themselves. The redaction pass saw 'api_token' as a value under a key whose repr contains 'token' (sensitive pattern), and replaced the key string's repr with a redacted placeholder. This broke the design invariant: key labels must stay readable so the dump shows *what* was redacted. Fix: after collecting sensitive destinations from dict-key edges, subtract any object ids that are themselves sensitive-pattern keys. These are label strings (like 'password', 'api_token'), never actual secrets. The test now filters to redacted-only rows when checking placeholder format, tolerating the unredacted self-referencing entries.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
optional value capture with default-on sensitive-key redaction
dump_graph/spawn_dump grow values='none'|'builtins'|'repr' (default 'none':
behavior and dump format unchanged, one falsy check per object), plus
value_limit=128 and redact=True. Captured reprs land in a new object_repr
table (object PK, repr, truncated, redacted); meta gains a capture_values
column so dumps are self-describing.
repr, never runs user code. str/bytes slice before repr; int guarded
against 3.11+ int_max_str_digits; stored text hard-bounded at value_limit.
runs mid-dump (accepted risk, fork isolation via spawn_dump), exceptions
skip the row.
under a sensitive-looking edge label (key/secret/password/token/auth/...,
attr and dict-key edges alike) has its repr replaced by a length-bucketed
placeholder '<redacted len<=N>' (buckets 100, 500, 1000, 5000, ...) built
from object.len; keys themselves stay readable, one sensitive inbound
edge wins for aliased objects. Redaction is dump-time: the artifact never
contains the secret text.
repr in object_summary, display-capped repr snippets in both
Reader.object_label and Console._obj_label, Repr row in the web object
panel.
Known hole inherited from traversal: CPython untracks atomic-only dicts,
so their key edges never materialize (values still captured, un-redacted
under tier 'repr' container reprs). Tests keep the fixture dict tracked.