feat(subscriptions): a recovered subscription stops claiming it is rate-limited, and the row says when the limit returns (abilityai/trinity-enterprise#447)

[trinity-ability]

Summary

Three defects stacked on one surface. Only the third is what an operator sees, so they ship together — fixing any one alone leaves the tile still lying.

• A recovered subscription kept claiming it was rate-limited. rate_limited_now was db_2h_predicate OR fresh_provider_verdict, and nothing clears a failure row on success, so a fresh "allowed · 32% used · resets 19:10" from the provider was structurally powerless against a 2-hour-old inference. Observed live: two subscriptions wearing LIMIT while every agent assigned to them answered test messages normally.
• Nothing asked the provider again. The fresh-verdict arm is only reachable if a probe runs, and ambient refresh is demand-driven — an unwatched instance never re-checked at all.
• The reset time was on the wire and rendered nowhere but a hover — and on the row that needs it, the old code path structurally could not reach it.

Changes

A — resolve_rate_limited_now is three-state, not an OR (services/subscription_headroom_service.py)

fresh verdict says limited  -> True    (unchanged)
fresh verdict says allowed  -> False   (the missing arm)
no usable verdict           -> the 2h db predicate (unchanged)

One gate, consumed by both decorate_usage and pressure_states, so the tile and the per-agent chip cannot disagree about one subscription (#2157 rule).

_headroom_indicates_healthy is deliberately not the negation of _headroom_indicates_limited: "not limited" is also true for a stale snapshot, a rejected token and a transport error — none of which are evidence of headroom — so all three fall through to the predicate rather than clearing it (#2353's rule, preserved).

Auto-switch is untouched. It reads the kind-blind has_recent_subscription_failures (#2352's split), so a just-recovered subscription is still skipped as a switch target until its failures age out — #444's ping-pong cannot return through this door.

B — recovery probe (services/subscription_recovery_service.py, new)

Sweeps every 300s and probes only subscriptions currently presented as limited — normally an empty set, so the cycle costs nothing. Reuses the existing max_tokens=1 Haiku probe (~a dozen tokens each) and the Settings-surfaced subscription_headroom_auto_refresh toggle rather than adding a second knob, since that setting already answers may Trinity probe on its own?.

Property	Why
Cannot feed itself	_probe records a 429 into the snapshot only, never subscription_rate_limit_events — otherwise the loop would manufacture the very db rows that keep a subscription limited (pinned by a static test)
Leadership fail-open	A duplicated probe wastes a dozen tokens; failing closed would silently stop recovery detection — the mode the operator cannot see
Probe fail-closed on Redis	Without a readable cache the result could not be stored for anyone to see, so it would be quota spent for nothing (the ambient path's rule)

C — the reset on the row face (utils/subscriptionPressureTile.js)

resetReading reads headroom.*.resets_at directly, bypassing windowReadings and therefore its freshness gate — the same asymmetry headroomStatus documents (#2353), for the same reason one level along: a number decays, an instant does not. That gate is exactly what hid it: a 429 probe sets status: 'rate_limited', so decorate_usage never promotes source to anthropic, so windowReadings returns null — while resets_at sits populated in the same object.

Binding window: representative_claim → any non-allowed window → the fullest. The claim comes first because it was live-verified populated, and both windows can report the same utilization (observed: 32% / 32%), so a "fullest wins" tiebreak has no answer there and would pick by array order.

Honest states, none blank: resets 19:10 · reset due (lapsed — only another probe confirms the roll) · reset unknown (limited, nothing in the payload) · nothing at all for a rejected token, because a dead credential has no quota clock.

Placement is decided in the pure module, so the SFC template is unchanged and every rule stays reachable by the node-environment suite: the reset takes whichever text slot is free — joining the headline when there are no bars, leading the second line when there are, since the fixed-width bars cannot take more without overflowing a row that clips silently.

Row alignment (operator-reported alongside, SubscriptionPressureTile.vue, CSS only)

The two window groups now form columns down the tile. Previously 0% / 6% / 24% had different advances and pushed the 7d group left by a different amount per row — the bars were always 30px but landed at three x positions and read as three lengths. min-width not width, so an overage plan reporting >100% overflows rather than being clipped (the barWidthPct clamp-only-geometry rule).

Test Plan

• New backend tests pass: pytest tests/unit/test_447_subscription_reset_and_recovery.py -v — 21 passed
• New frontend tests pass: npx vitest run tests/unit/subscriptionPressureTile.spec.js — 81 passed (65 pre-existing + 16 new)
• Sibling suites unaffected: test_471_* + test_2352_* — 45 passed
• Broader backend sweep: pytest tests/unit -k "subscription or headroom or pressure" — 234 passed, 4 skipped
• Full frontend unit suite: 1103 passed (52 files)
• Live-probed a real instance to confirm resets_at + representative_claim are populated on the wire

Scope note

The issue was filed as C only and grew during investigation, operator-directed (2026-08-21) after test messages to agents on two LIMIT subscriptions came back fine. The issue description has been rewritten to match what shipped. The alignment fix is a genuinely separate concern bundled deliberately: it is CSS in the same block this PR already touches, and splitting it would have guaranteed a conflict.

Residual, stated honestly

• Code-verified, not live-verified: that a 429 response carries the unified rate-limit headers (so a limited subscription has a reset to show). _probe()'s contract asserts it and the code path preserves the parsed windows on a 429, but no rate-limited subscription was available to observe. If false, the feature degrades to reset unknown on exactly that row — an already-specified state, not a wrong answer.
• Out of scope: the per-agent chip's batch row carries utilization_5h_pct but no reset field; surfacing one there is a backend change, left out to keep C zero-backend.

Fixes abilityai/trinity-enterprise#447

Generated with Claude Code

[vybe]

Validated via /validate-pr.

Found and fixed one blocker: the #1920 static guard was newly failing on HEAD — this service is the fifth leader lease and its set(..., nx=True, ex=ttl) was unlisted. Not adoptable onto SingleFlightLock (it mints a unique token per acquire, so a lease could never refresh its own grant across cycles), so applied the guard's prescribed second option: one justified allowlist row (9d58c4e).

Regression diff: ✅ no new failures — HEAD reports 0 failures across all three seeds. The pytest (base, seed 99999) red is a runner timeout on dev, not this PR.

One follow-up, not blocking: SUBSCRIPTION_RECOVERY_PROBE_SECONDS and SUBSCRIPTION_RECOVERY_DISABLED_POLL_SECONDS are wired into neither compose file nor .env.example, so the advertised env-tunability is inert on deploy (#1056 class). Left as-is because the sibling SUBSCRIPTION_HEADROOM_REFRESH_SECONDS from #471 is unwired the same way — worth one follow-up covering both rather than diverging here.