Context
All usage data is derived on the agent by re-parsing transcript JSONLs (usage_report() in hub-agent.py), reported per session with a per-day breakdown capped at HISTORY_DAYS = 60, and held by the hub only as part of the latest heartbeat payload in its state file. The history page renders straight from /api/agents.
Problem
The historical record is fragile and bounded:
- Delete erases history: a deleted session leaves
sessions[], so its cost disappears retroactively from every total and from the history chart. Fleet spend for "last month" changes depending on what you've since cleaned up.
- 60-day cap on the per-day breakdown; the payload-size concern that motivated it is real (every beat re-ships the whole
days map for every session), but it means no quarter/year view, ever.
- Host prune: an agent gone for a week is pruned (
PRUNE_AFTER_MS), taking all of its sessions' history with it.
- The heartbeat payload grows linearly with session count × history days — a data-model smell independent of features.
Proposal
- Hub-side append-only ledger: on each beat, upsert
(day, host, sessionId, repo, label) → {tokens…, cost} into a persistent store — a JSONL/JSON file beside state.json fits the stdlib-only, no-deps constraint fine at this scale (one row per session-day; even years of a busy fleet is trivial).
- The ledger is upsert-by-key, so re-reports of the same day are idempotent, and rows persist when the session, or the whole host, disappears.
/api/history?from=&to= serves the chart/table from the ledger instead of the live payload; the history page gains a date-range picker and repo/host rollup views (sum by repo across sessions — "what has AgentHub-the-repo cost this month").- Follow-up (separate change): once the ledger is authoritative, the agent can stop shipping the full 60-day
days map every beat — today + a hash/watermark would do — which shrinks the heartbeat considerably. Kept out of scope here to avoid coupling.
Touchpoints
agent-hub/server.js — ledger write on heartbeat, /api/history endpointagent-hub/public/history.html — fetch from the new endpoint, range picker, rollupsagent/hub-agent.py — none initially (payload-diet follow-up only)
Context
All usage data is derived on the agent by re-parsing transcript JSONLs (
usage_report()inhub-agent.py), reported per session with a per-day breakdown capped atHISTORY_DAYS = 60, and held by the hub only as part of the latest heartbeat payload in its state file. The history page renders straight from/api/agents.Problem
The historical record is fragile and bounded:
sessions[], so its cost disappears retroactively from every total and from the history chart. Fleet spend for "last month" changes depending on what you've since cleaned up.daysmap for every session), but it means no quarter/year view, ever.PRUNE_AFTER_MS), taking all of its sessions' history with it.Proposal
(day, host, sessionId, repo, label) → {tokens…, cost}into a persistent store — a JSONL/JSON file besidestate.jsonfits the stdlib-only, no-deps constraint fine at this scale (one row per session-day; even years of a busy fleet is trivial)./api/history?from=&to=serves the chart/table from the ledger instead of the live payload; the history page gains a date-range picker and repo/host rollup views (sum by repo across sessions — "what has AgentHub-the-repo cost this month").daysmap every beat — today + a hash/watermark would do — which shrinks the heartbeat considerably. Kept out of scope here to avoid coupling.Touchpoints
agent-hub/server.js— ledger write on heartbeat,/api/historyendpointagent-hub/public/history.html— fetch from the new endpoint, range picker, rollupsagent/hub-agent.py— none initially (payload-diet follow-up only)