⚡ Bolt: Remove eager list() materializations in AST traversals - #134
⚡ Bolt: Remove eager list() materializations in AST traversals#134tachyon-beep wants to merge 1 commit into
Conversation
- Replaced explicit list() materialization with direct passing of Iterables (`func_node.body` and `ast.iter_child_nodes()`) to core AST traversal functions `_assignment_callee` and `_collect_return_paths`. - Updated parameter typing from `list[ast.AST]` to `Iterable[ast.AST]`. - Implemented in `src/wardline/scanner/taint/variable_level.py`. Co-authored-by: tachyon-beep <544926+tachyon-beep@users.noreply.github.com>
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
There was a problem hiding this comment.
Pull request overview
This PR reduces unnecessary memory allocations during Level 2 taint analysis by avoiding eager list(...) materialization when traversing Python AST nodes, and updates traversal function type annotations accordingly.
Changes:
- Removed
list(func_node.body)andlist(ast.iter_child_nodes(...))materializations in return/assignment traversal helpers. - Updated helper function parameter types to accept
Iterable[ast.AST]for generator-friendly traversal. - Added a Jules “Bolt” note documenting the AST traversal performance guideline.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| src/wardline/scanner/taint/variable_level.py | Switch traversal helpers to accept/consume Iterable and pass AST iterators directly to reduce allocations. |
| .jules/bolt.md | Adds a project note capturing the “avoid eager list materialization in AST traversals” guidance. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| ## 2024-05-24 - Initial | ||
| ## 2024-05-24 - Removing Eager `list()` Materializations in AST Traversals |
💡 What:
Removed eager list materialization of AST generators (
ast.iter_child_nodes) and sequence properties (func_node.body) when passed to L2 taint traversal functions. Updated type annotations to acceptIterable[ast.AST].🎯 Why:
Eagerly converting generators or AST node lists into brand new Python lists
list(...)before iterating over them in recursive functions forces unnecessary synchronous memory allocations. Since the iteration functions justfor node in nodes:walk over them sequentially exactly once, intermediate lists provide zero value but cost O(N) allocation per layer, heavily penalizing deeply nested code evaluation.📊 Impact:
Reduces intermediate list allocations during static analysis traversals proportionally to the sheer volume of AST nodes in each function analyzed, thus decreasing both peak memory usage and generator-materialization overhead.
🔬 Measurement:
Tests (
make test), Typechecks (make typecheck), and Lints (make lint) all pass natively, validating that iteration works perfectly without materialized lists. Measured by reviewing memory footprint on large nested code samples.PR created automatically by Jules for task 10794879021442726230 started by @tachyon-beep