⚡ Bolt: avoid eager list materialization in AST traversals - #138
⚡ Bolt: avoid eager list materialization in AST traversals#138tachyon-beep wants to merge 1 commit into
Conversation
Co-authored-by: tachyon-beep <544926+tachyon-beep@users.noreply.github.com>
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
There was a problem hiding this comment.
Pull request overview
This PR reduces memory allocations during recursive Python AST traversals in the taint scanner by avoiding repeated list(ast.iter_child_nodes(...)) materialization and instead passing the child-node iterator through the recursion.
Changes:
- Updated
_assignment_calleeand_collect_return_pathsto acceptIterable[ast.AST]and recurse usingast.iter_child_nodes(node)directly. - Added a Bolt note documenting the “avoid eager AST child list materialization” performance guideline.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
src/wardline/scanner/taint/variable_level.py |
Switches recursive AST traversal helpers to take Iterable and passes ast.iter_child_nodes() directly to reduce per-level allocations. |
.jules/bolt.md |
Documents the AST traversal performance guideline that motivates the change. |
Suppressed comments (1)
src/wardline/scanner/taint/variable_level.py:2512
- The new
_assignment_callee(...)call is inlined onto one long line, while nearby calls in this module use multi-line formatting. Reformatting this call improves readability and avoids potential line-length/lint issues without reintroducing eager list materialization.
# ⚡ Bolt: pass generator directly instead of list(ast.iter_child_nodes()) to avoid memory allocations
nested = _assignment_callee(ast.iter_child_nodes(node), name, worst, function_taint, taint_map, var_taints)
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| def _assignment_callee( | ||
| nodes: list[ast.AST], | ||
| nodes: Iterable[ast.AST], | ||
| name: str, |
💡 What: Updated recursive AST traversals in
_assignment_calleeand_collect_return_pathsto acceptIterable[ast.AST]and passast.iter_child_nodes()directly instead of eagerly materializing it into lists usinglist().🎯 Why: Calling
list(ast.iter_child_nodes())allocates a new list in memory at every level of recursive AST descent. Since Python ASTs can be quite deep and numerous across a large project, passing the underlying generator directly prevents unnecessary memory allocations and garbage collector pressure.📊 Impact: Reduces intermediate object allocations for AST branches during recursive tree walks. Faster, lower-memory taint analysis.
🔬 Measurement: Verifiable via memory profiling on large ASTs or running standard
pytestbenchmarks (performance parity check). Tests run fully cleanly after this adjustment.PR created automatically by Jules for task 1881551180393145932 started by @tachyon-beep