exporter: scrape dicts via items(), not keys() + __getitem__ (KeyError on mutated keys) - #9
Merged
Conversation
A dict key mutated after insertion (hash no longer matches its bucket) iterates fine but raises KeyError on lookup by key, killing the whole dump. Seen in the wild with openpyxl's StyleArray (an array subclass with a content-based hash that openpyxl mutates in place after using it as a dict key): dump_graph on any heap containing a loaded-and- modified workbook died with KeyError(StyleArray(...)). dict.items() reads key and value straight off the bucket, so no re-lookup happens; it's also one C call instead of one per key.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
dump_graphdies withKeyError(<key>)on any heap containing a dict whose key was mutated after insertion (hash no longer matches its bucket). Iteration over such a dict works fine, but lookup by the key raises, and the exporter scraped dicts viakeys()+dict.__getitem__(obj, key).Seen in the wild via openpyxl:
StyleArrayis anarraysubclass with a content-based hash that openpyxl mutates in place after using it as a dict key. Any worker heap containing a loaded-and-modified workbook made the dump fail:Minimal repro against master:
Fix
Scrape dicts with
dict.items(obj): reads key and value straight off the bucket, no re-lookup; also one C call instead of one per key.Test
Added
MutableHashKey(a pure-Python stand-in for StyleArray) to the shared fixture heap with a post-insert mutation, plustest_dump_graph_survives_mutated_dict_keysfollowing the existingsurvives_*pattern. Pre-fix, the fixture dump subprocess crashes (22 errors); post-fix, the full suite passes (30 passed).