epic: modernize LLM providers and harden the public library boundary

[devlikebear]

Goal

Three related tracks, in dependency order:

1. Provider modernization — make caching, reasoning, and context-size controls actually work against current frontier models.
2. Public library promotion — make pkg/* a real, documentable, dependency-light API surface that external consumers (e.g. linetta) can depend on.
3. Boundary and simplification — enforce layering in-repo instead of splitting into tars-app / tars-cli repositories, and narrow the application surface.

Based on repository analysis at d9f9861 (v0.35.0).

Findings That Motivate This Epic

Verified against the tree, not assumed:

#	Finding	Evidence	Issue
1	Prompt caching is defeated on every turn, for every provider — the system prompt's first line is a second-resolution timestamp, so no prefix ever matches.	internal/prompt/builder.go:70	#920
2	Anthropic requests carry cache breakpoints on the system block and last tool only — never on message history, so long transcripts are re-charged in full each turn.	internal/llm/anthropic.go:110-125	#921
3	reasoning_effort is silently ignored by the Anthropic client. The shipped default config already sets it on a kind: anthropic tier, so a no-op ships today.	internal/llm/anthropic.go:106, config/default.yaml:46	#922
4	Anthropic assistant turns drop thinking blocks and signatures on the way back to the API. gemini-native already round-trips thoughtSignature correctly — Anthropic was simply never updated.	internal/llm/anthropic.go:461, cf. internal/llm/gemini_native_convert.go:161	#922
5	Anthropic max_tokens is pinned at 4096 and cannot be raised: no tier field exists and the router never passes one.	internal/llm/provider.go:373, internal/config/types.go:141, internal/tarsserver/helpers_llm_router.go:34	#923
6	anthropic-beta is a single hardcoded constant, occupying the only slot with a header that has since gone GA — 1M context, interleaved thinking, and extended cache TTL cannot be opted into.	internal/llm/anthropic.go:16	#923
7	Cost accounting prices every Anthropic model through one anthropic/* wildcard at Sonnet rates.	internal/usage/tracker_cost.go:33	#924
8	No tier carries a context window, so history budgeting and compaction do not change when a tier moves between a 200k and a 1M model.	internal/config/llm_resolve.go:22	#925
9	Provider tests are per-provider and per-file, so capability divergence between providers is unobservable.	internal/llm/*_test.go	#926
10	Importing pkg/agentloop pulls 25 internal packages including modernc.org/sqlite and robfig/cron. pkg/llm alone pulls 4. The cause is internal/tool bundling app-specific tools with generic primitives.	go list -deps ./pkg/agentloop	#927
11	pkg/* is a pure type-alias facade over internal/*. pkg.go.dev cannot render fields of internal types, so the public API is effectively undocumented, and no gate exists to catch breaking changes.	pkg/llm/exports.go	#928, #929
12	pkg/* does not import internal/tarsserver at all — the server/core split already holds in the import direction. The real coupling problem is horizontal, not vertical.	go list -deps ./pkg/...	#930

Product Direction

Preserve:

• Go single-binary, local-first operation
• One repository, one release train
• Skills + companion CLIs over built-in tools that inflate the system prompt

Add:

• Provider capability parity across anthropic / openai / gemini-native, enforced by a shared conformance suite
• Tier-level control of output limits, context window, and provider beta features
• A public API surface that owns its own types, renders on pkg.go.dev, and is gated in CI
• Compiler-enforced layering (cmd/ → internal/app/* → internal/core/* → pkg/)

Phases And Work Orders

• Phase 0 — Provider correctness and cost
  • fix(prompt): stop invalidating provider prompt caches on every turn #920 — LP-001: stop invalidating prompt caches every turn
  • feat(llm): add rolling cache breakpoints to Anthropic message history #921 — LP-002: rolling cache breakpoints on Anthropic message history
  • fix(llm): reach reasoning parity on the Anthropic provider #922 — LP-003: Anthropic reasoning parity (reasoning_effort + thinking round-trip)
  • feat(llm): make output limits and provider beta features tier-configurable #923 — LP-004: tier-configurable output limits and provider beta features
  • fix(usage): price Anthropic models individually instead of one wildcard #924 — LP-005: per-model Anthropic pricing
• Phase 1 — Context sizing and regression guards
  • feat(llm): make context-window budgeting model-aware #925 — LP-006: model-aware context-window budgeting
  • test(llm): add a cross-provider conformance suite #926 — LP-007: cross-provider conformance suite
• Phase 2 — Public library boundary
  • refactor(tool): split internal/tool into core primitives and TARS app tools #927 — LP-008: split internal/tool into core primitives and TARS app tools
  • refactor(pkg): invert the public API aliases so pkg owns the exported types #928 — LP-009: invert the pkg/* aliases so pkg owns the exported types
  • ci(pkg): gate the public API surface and document its stability policy #929 — LP-010: gate and document the public API surface
• Phase 3 — Layering and application surface
  • chore(arch): enforce layer boundaries in-repo and defer the multi-repo split #930 — LP-011: enforce layer boundaries in-repo; defer the multi-repo split
  • refactor(console): narrow the console surface to chat and observability #931 — LP-012: narrow the console surface

Dependency Order

• Independent, parallelizable: fix(prompt): stop invalidating provider prompt caches on every turn #920, feat(llm): add rolling cache breakpoints to Anthropic message history #921, fix(usage): price Anthropic models individually instead of one wildcard #924, refactor(console): narrow the console surface to chat and observability #931
• fix(llm): reach reasoning parity on the Anthropic provider #922 → feat(llm): make output limits and provider beta features tier-configurable #923 (thinking budget must fit under a configurable max_tokens)
• feat(llm): make output limits and provider beta features tier-configurable #923 → feat(llm): make context-window budgeting model-aware #925 (context window and output limit land on the same tier binding)
• fix(prompt): stop invalidating provider prompt caches on every turn #920 + feat(llm): add rolling cache breakpoints to Anthropic message history #921 + fix(llm): reach reasoning parity on the Anthropic provider #922 + feat(llm): make output limits and provider beta features tier-configurable #923 + feat(llm): make context-window budgeting model-aware #925 → test(llm): add a cross-provider conformance suite #926 (the suite encodes the behavior these establish)
• refactor(tool): split internal/tool into core primitives and TARS app tools #927 → refactor(pkg): invert the public API aliases so pkg owns the exported types #928 → ci(pkg): gate the public API surface and document its stability policy #929
• refactor(tool): split internal/tool into core primitives and TARS app tools #927 → chore(arch): enforce layer boundaries in-repo and defer the multi-repo split #930 (layer rules need the package split to be satisfiable)

Suggested Starting Point

#920 is a few lines and affects every turn of every session. Start there.

Roadmap Gates

• Caching: a second identical-prefix turn reports non-zero cache_read_input_tokens.
• Reasoning: every provider either honors reasoning_effort or reports it as unsupported at resolve time — never silently.
• Context: changing a tier's model changes its history budget without a code change.
• Cost: Haiku, Sonnet, and Opus traffic price differently.
• Library: pkg/llm and pkg/agentloop render usable docs on pkg.go.dev and pull no storage or scheduling dependencies.
• Layering: a reverse import from internal/core/* to internal/app/* fails CI.

Non-goals

• Splitting the repository into tars-app / tars-cli / tars-console. See chore(arch): enforce layer boundaries in-repo and defer the multi-repo split #930 for the decision and the criteria under which it is revisited.
• Adding providers. This epic makes the existing ones correct.
• A v1.0 API freeze. The goal is a documented policy, not a permanent contract.
• New built-in tools.

Completion

• Every linked phase issue is complete.
• A multi-turn chat on an Anthropic tier reports cache reads after the first turn.
• Reasoning and output-limit settings on any tier have an observable effect or a loud diagnostic.
• go list -deps ./pkg/agentloop contains no storage, scheduler, or server package.
• The public API surface is covered by a CI gate and a documented stability policy.
• The repository-split question is answered in writing with explicit re-evaluation criteria.