Skip to content

summarize's sessions_examined double-counts sessions that span more than one agent_id #609

Description

@anshss

Severity: Medium

Where: core/optimize/analyzers/summarize.py, _load_profile().

Observed: The query does GROUP BY agent_id and sums each group's COUNT(DISTINCT session_id) into sessions_total. A session that emitted spans under two agent_ids is therefore counted once per agent. Measured on a real 90d store: sessions_total reports 7,084 against 5,686 genuinely distinct sessions, because 1,352 sessions (24%) span more than one agent_id. Two effects: (a) the user-visible sessions_examined figure and the CLI line "...across the 7084 session(s) in this window" overstate by ~25%; (b) calls_per_session = calls_total/sessions_total correspondingly understates, and because _price_reduction charges the FIRST read of each session at the full input rate and later reads at the cache-read rate, splitting one session into two moves a re-read onto the expensive input rate and inflates estimated_recoverable_usd. The token figure is roughly invariant (sessions x reads ~= calls_total), so this does NOT break the token/dollar same-basis property, but it does inflate the dollar side.

Fix direction: Compute sessions_total as a single COUNT(DISTINCT session_id) over the whole window rather than a sum of per-agent counts, and derive calls_per_session from that; keep the per-repo maps for scope matching but attribute each session to one repo (e.g. its dominant agent_id) so sessions_by_repo also partitions rather than overlaps.

Done when: sessions_examined equals the window's distinct session count, per-repo session counts sum to no more than that total, and a test seeds one session emitting under two agent_ids and asserts it is counted once.

Ref: Found while verifying the summarize token/dollar basis fix in #589.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions