Restore leak attribution on modern CPython: hybrid traversal, scenario suite, CI revamp

[mahmoud]

Why

objex's core promise is leak attribution: "what is still holding these objects, and from which module/frame root." That promise was quietly broken on exactly the leak classes where you need it most. The exporter discovers edges by scraping __dict__/__slots__ and a handful of named handlers, so C-implemented types had zero outbound edges: coroutines, async generators, tracebacks, cells, every C iterator, slice, super, weakrefs, ContextVar, GenericAlias, asyncio.Task. Exceptions and functools.partial only showed their __dict__. The result: objects pinned by a suspended coroutine, a stored exception's frame chain, or an lru_cache showed up as spurious "roots," and find_path_to_module/find_path_to_frame silently returned nothing. These gaps are structural, not version drift — verified identical on 3.10 and 3.12.

Worse, modern CPython added a second silent hole: since 3.11, running stacks materialize frame objects lazily, after the exporter's gc.get_objects() snapshot. Live thread and coroutine frames got pyframe rows but were never traversed — zero .locals[...] edges, so thread-held and coroutine-held leaks were entirely unattributable on 3.11+.

None of this was caught because nothing tested the actual product claim (a path to a root through a tricky edge), and CI ran unittest, which ignores pytest markers — so every push ran the slow in-process dumps and nothing else meaningful.

What

Exporter — hybrid traversal. Named handlers (with human-legible labels, objex's UX advantage) for the high-value types: coroutine .cr_frame/.cr_code/.cr_await, async generator, traceback .tb_frame/.tb_next, cell .cell_contents, slice, super, weakref.ref (.__callback__ only — the referent is deliberately a weak edge, matching gc semantics), BaseException (.args/.__traceback__/.__cause__/.__context__/.__notes__), generator .gi_yieldfrom, function .__kwdefaults__. Then a generic gc.get_referents (tp_traverse) fallback labeled <gc> for everything not already covered: this is ground truth from the C level, and it future-proofs against types nobody enumerated (iterators, Task, ContextVar, lru_cache internals, whatever 3.15 adds). Exact builtin containers are exempted so the hot path stays hot: bare-interpreter dump goes 0.855s → 0.97s (+13%, budget was 25%).

Exporter — lazy-frame fix. Frames encountered anywhere (sys._current_frames(), f_back, cr_frame, ...) are queued and traversed at the end of add_all, restoring locals attribution on 3.11+.

Tests that defend the promise. tests/test_scenarios.py builds 7 leaks in bare subprocesses — module global, exception-pinned frame chain, closure cell, suspended coroutine task, parked thread frame, lru_cache, iterator-pinned dict — and asserts a payload instance is attributable: a non-empty path to a module/frame root passing through the scenario's anchor edge (.__traceback__/.tb_frame, .locals['payloads'], <gc>). tests/test_gc_coverage.py is the Python-upgrade regression guard: every tp_traverse edge among seen objects must exist in the reference table, and failures print a readable per-type miss list.

CI that runs what it claims. pytest with markers honored: a fast test job (non-slow + scenarios) and an integration job (-m slow, the in-process full-dump smokes), uv-managed 3.10–3.14 matrix, concurrency cancellation. The non-slow suite dropped 629s → ~35s: almost all of the old runtime was Reader being opened on a raw unindexed collection DB, which triggers the one-time attributed-size build without indices (pathological; noted in TODO.md rather than "fixed," since users are directed through make-analysis-db). The same pathology turned out to hang the forked-dump slow test for >55 min under pytest; it now routes through make_analysis_db like users do.

Agent/scripting interface. The explorer is a human REPL; scripts and agents need one-shot JSON. New q subcommand wraps the existing dispatch_request seam (no duplicated query logic): python -m objex q analysis.db path-to-module id=4211, plus a read-only sql escape hatch (sqlite mode=ro, enforced by test).

Positioning. docs/2026-08-06_leak_tool_landscape.md records where objex sits among leak tools (memray/tracemalloc answer where is it allocated; objex answers who still holds it) — no maintained tool combines fork-and-dump capture, a portable SQLite artifact, offline SQL analysis, and labeled path-to-root attribution. README gains when-to-use and agent-usage sections.

Verification

• Non-slow suite green on 3.10, 3.11, 3.12, 3.13, 3.14 (29 passed each, ~35s).
• Slow suite green on 3.12 (2m49s), 3.13 (3m04s), 3.14 (6m01s).
• Zoo audit: every previously-edgeless type now has outbound reference rows.
• Perf: +13% bare-dump wall time, within the 25% budget.

[kurtbrose]

let me know if you want me to merge :-)

[kurtbrose]

eh tests all pass we can just merge