Skip to content

feat(budget): cap what an agent may spend in money, tokens, or working time#3702

Open
dwin-gharibi wants to merge 20 commits into
docker:mainfrom
dwin-gharibi:feat/budget-limits
Open

feat(budget): cap what an agent may spend in money, tokens, or working time#3702
dwin-gharibi wants to merge 20 commits into
docker:mainfrom
dwin-gharibi:feat/budget-limits

Conversation

@dwin-gharibi

Copy link
Copy Markdown
Contributor

Adds budgets that stop an agent once it crosses a configured ceiling, and tracks every budget live in the TUI, broken down per agent.

# One ceiling for the whole session, charged for every agent.
budget:
  max_cost: 0.50      # USD
  max_tokens: 100000  # cumulative input+output
  max_time: 10m       # time agents spend working

# Named budgets an agent opts into by name.
budgets:
  shell-work:
    max_cost: 0.03
  research:
    max_time: 1m

agents:
  root:
    budgets: [shell-work]
  researcher:
    budgets: [shell-work, research]   # shares shell-work with root

Every limit is optional, unset means unlimited, and declaring nothing leaves runs unbudgeted. Existing configs are unaffected - the feature is inert unless opted into.

docker agent budget

The named form follows the existing top-level mcps / rag / toolsets reference-by-name convention. An agent may list several budgets; all apply, and the first exhausted stops the run.

Closes #3701.

Why this needs runtime code

The runtime already prices every response (computeMessageCost) and hands that exact figure to after_llm_call hooks, which the docs suggest using for "a sidecar cost ledger".

But a ledger can only record. executeAfterLLMCallHooks (hooks.go) discards the hook Result - both callsites invoke it as a bare statement, and after_llm_call_test.go locks that in. Continue: false is only honored in executor.go, which the event never reaches. So spend can be watched but not acted on; enforcement has to live in the loop.

This PR adds no new pricing math. It reuses the value the session already bills, so the ceiling can never disagree with the invoice.

Design decisions

Each of these is a place a naive implementation is silently wrong, so they're called out for review rather than buried.

A budget name is one shared pot. root and researcher above both reference shell-work, so together they cannot exceed $0.03. Per-agent copies would let a run spend max_cost × N by fanning out to N sub-agents. Distinct names give independent pots. Sub-sessions likewise share the root's wallets - they run on the same LocalRuntime (loop.go already computes rootStream := !sess.IsSubSession()). This intentionally differs from max_iterations, where per-child caps (agent_delegation.go) are correct: money doesn't behave like iterations.

A budget spans the session, not a message. RunStream runs once per user message, so building the trackers there would silently redefine max_cost as "per message" - a session could re-spend the whole ceiling on every turn. ensureBudget builds once and leaves them alone. A budgetStarted flag distinguishes "not built yet" from "built, nothing to budget", so unbudgeted runs don't rebuild every message.

max_time measures working time, not wall-clock. Follows directly from the above: a session sits idle while the user reads and types, so wall-clock would let a budget expire during a coffee break - leave the TUI open ten minutes and the next message instantly trips max_time: 2m. It accumulates turn durations instead. This also made the tracker simpler: no start timestamp, no clock.

max_tokens uses its own accumulator, not session.Usage(). ApplyCompaction resets the session counters because they measure context length, not spend. Reading them would make max_tokens silently stop working on precisely the long runs it exists for.

Unpriced models can't be enforced against, and say so. computeMessageCost returns nil for a model with no pricing table. There's no honest number to add, so it can't count towards max_cost. Failing closed would break local/custom-endpoint users; counting $0 would read reassuringly low precisely because it's incomplete. Instead: a one-shot warning, an (unpriced spend) marker in the TUI, and docs pointing at the model-level cost: block.

Terminal, not resumable. No "continue?" prompt, unlike max_iterations. A budget is a ceiling set on purpose; offering to raise it defeats it. This also keeps resumeChan and the headless consumers (chatserver, embeddedchat) untouched.

Boundary-checked. Checked between turns beside enforceMaxIterations, so a run overshoots by at most the turn in flight and max_time won't interrupt an in-flight call. Documented explicitly rather than left for users to discover.

Changes

Config

  • BudgetConfig, Config.Budget (run-wide), Config.Budgets (named map), AgentConfig.Budgets (name list).
  • Validation rejects negative limits and unknown budget names - a name that resolves to nothing would silently leave the agent uncapped.
  • agent-schema.json: budget via $ref, a budgets map, and the agent-level budgets array. Plus an HCL block rule so budgets "tight" { ... } maps correctly.
  • max_time reuses the existing latest.Duration type, so 10m / 30s / 1h30m all work.
  • No migration needed: v0–v11 → latest is a generic JSON round-trip (CloneThroughJSON), so additive fields arrive as zero values. Version stays "12".

Runtime

  • pkg/runtime/budget.go: budgetTracker (one per budget name) and budgetSet (the run-wide budget plus each agent's named budgets). Nil-safe throughout, so the loop needs no nil checks.
  • Enforcement beside enforceMaxIterations; new turnEndReasonBudgetExceeded so a budget-killed run is distinguishable from a completed one in telemetry.
  • budget_usage (one BudgetStatus per active budget, each with a per-agent breakdown) and budget_exceeded (carrying budget, limit, used, max, config_path), registered in client.go for wire decoding.
  • Reuses the notification hook rather than adding on_budget_exceeded - budget_exceeded already carries the structured payload. Easy to add later if asked.

TUI

  • The sidebar lists every budget by name with consumption against each ceiling, breaking a budget down per agent when more than one draws on it (biggest spender first, with cost/tokens/active time). Each reading warns past 80%.
run        $0.12/$0.50 · 12.3K/100.0K · 2m14s/10m
  developer  $0.09 · 8.0K · 1m12s
  root       $0.03 · 4.3K · 1m02s
shell-work $0.09/$0.10 · 4.3K/20.0K
  • The reading is emitted at stream start too, so a configured budget is visible immediately rather than only after the first priced turn - a run that dies on its first call still shows its budget was active.
  • budget_exceeded surfaces as a warning, not a dialog (nothing to ask).

Docs / example

  • docs/configuration/budget/index.md (weight 75, no collision), covering shared pots, the unpriced-model caveat, boundary granularity, and that budget max_tokens ≠ model max_tokens.
  • Runnable examples/budget.yaml - OpenRouter via base_url with an explicit cost: block (so max_cost is enforceable on a custom endpoint), two agents sharing one named budget, and deliberately low limits so the stop is easy to see.

Verification

The Go CI jobs don't appear to run on this branch, so I ran the repo's own linters and tests locally. The go 1.26.5 toolchain is unfetchable here (403 from proxy.golang.org), so tests ran against cached go1.26.4 via a local-only -modfile; go.mod is untouched.

  • golangci-lint run (v2.12.2, repo's own .golangci.yml) over every touched package → 0 issues
  • go test ./pkg/runtime/ ./pkg/config/... → passing, including the schema drift guards (TestSchemaMatchesGoTypes, TestHCLBlockRulesCoverSchemaMaps, TestJsonSchemaWorksForExamples, TestDocYAMLSnippetsAreValid)
  • go test -race on the shared trackers → clean (sub-sessions and background agents record concurrently)
  • markdownlint-cli2@0.22.1 from docs/0 errors, 111 files
  • Binary builds; end-to-end through the real CLI: the example parses, an unknown budget name is rejected (agents.root: budgets: unknown budget "nope"), and a negative limit is rejected (budgets.shell-work: max_cost must not be negative, got -3).

Things I verified are load-bearing rather than assuming:

  1. Added "budget"/"budgets" to topLevelConfigKeys (doc_yaml_test.go) - without it, doc snippets containing them are silently skipped rather than validated. Proved by breaking a snippet: schema errors: [budget.max_cost: Invalid type. Expected: number, given: string]; restored → passes.
  2. Used $ref for the budget property rather than inlining. TestHCLBlockRulesCoverSchemaMaps doesn't resolve refs and types AdditionalProperties as any, so an inlined "additionalProperties": false would demand a nonexistent HCL rule.
  3. The budgets map genuinely needed an HCL rule - the drift guard caught its absence and failed until one was added.

Pre-existing failures unrelated to this PR: pkg/runtime's TestListGatewayModels_* / TestAvailableModels_Gateway* fail identically on a clean origin/main worktree with none of this code present (verified, not assumed). pkg/tui/components/sidebar tests cannot build under the -modfile workaround (it resolves a cellbuf incompatible with the pinned ansi); the pre-existing sidebar tests fail the same way, so it's environmental - the package compiles and those tests will run in CI on go 1.26.5.

Try it

export OPENROUTER_API_KEY=<your-key>
docker agent run examples/budget.yaml \
  "count from 1 to 40, calling the shell once per number"

Watch the sidebar track each budget until the run stops.

…er agent core components like agent loops and runtime and client
… into tui sidebar to announcing them realtime
@dwin-gharibi
dwin-gharibi requested a review from a team as a code owner July 17, 2026 06:59
@dwin-gharibi

Copy link
Copy Markdown
Contributor Author

@Sayt-0

@aheritier aheritier added area/config For configuration parsing, YAML, environment variables area/docs Documentation changes area/runtime Runtime engine, agent loop execution, tool dispatch, loop detection area/tui For features/issues/fixes related to the TUI kind/feat PR adds a new feature (maps to feat:). Use on PRs only. labels Jul 17, 2026
@dwin-gharibi

Copy link
Copy Markdown
Contributor Author

@Sayt-0 Deps fixed.

@Sayt-0

Sayt-0 commented Jul 17, 2026

Copy link
Copy Markdown
Member

#3714 already cover a part of this pr, but I like the idea of budget limit

@dwin-gharibi

Copy link
Copy Markdown
Contributor Author

#3714 already cover a part of this pr, but I like the idea of budget limit

Thanks! I checked #3714.
I'll update this PR to remove the overlapping changes and keep it focused on the budget limit feature.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/config For configuration parsing, YAML, environment variables area/docs Documentation changes area/runtime Runtime engine, agent loop execution, tool dispatch, loop detection area/tui For features/issues/fixes related to the TUI kind/feat PR adds a new feature (maps to feat:). Use on PRs only.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add budgets to cap what an agent may spend in money, tokens, or working time

3 participants