piper: RSS climbs across the process lifetime on the Pi 4 (v5.2.1: 262 → 601 MiB in 11 days) — cap-bounce projected ~09-04

[genwave-radio]

🐛 Sighting (Pi 4 mjolnir, home-v5.2.1, piper-only topology)

The repo-owned piper container's resident memory climbs across the life of the process. Recipe checks since the 2026-08-17 upgrade (fresh baseline 262.5 MiB):

Day	Date	RSS	Δ
0	08-17	262.5 MiB	fresh
1	08-18	393.9 MiB	+131
3	08-20	442.1 MiB	+48
5	08-22	454.9 MiB	+13
11	08-28	601.4 MiB / 768 (78%)	+146

It is real heap, not page cache: RssAnon 603 MB vs RssFile 20 MB, dead flat over a 30 s double-sample, single python -u /app/server.py process, 7 threads, 0 restarts, OOMKilled=false, piper log silent. Workload is ~774 renders/day (TTS render outcome lines on the api), so ~8.5k renders since boot — roughly 40 KB/render on average, but the shape is steps and plateaus, not a slope: +131 on day one, near-flat days 3–5, then +146 by day 11.

The same shape showed on the previous lineage: home-v3.3.2 sat at ~405–411 MiB for six days of its 7-day soak, then stepped to ~498 MiB on the final day (HARDWARE.md, Pi 4 row + soak runbook). That was recorded as a watch item and retired by the upgrade; this is the same behaviour on the new build, further along.

⚠️ Why it matters (and why it's P2, not P1)

• Linear projection hits the 768 MiB mem_limit around 2026-09-04. The cap is a designed backstop — the container bounces, restarts=1, and comes back at ~262 MiB. Music never stops.
• But on the piper-only topology piper is the only voice: the bounce is a brief no-TTS window (renders fail → segments dropped per F92.4 until the health probe passes again), and every box in that topology will do it on a cadence set by its render volume. An appliance that "restarts its TTS every ~2 weeks" is a wart, not a crash.
• It's growth by the soak runbook's own discriminator (real heap, monotonic between plateaus) — the watch item deserves a tracked home instead of living in check-in notes.

🔬 Suspects (in likelihood order)

1. onnxruntime's arena allocator — ORT's BFC arena grows to the largest activation footprint it has ever needed and never shrinks. The longest text yet seen (a long back-announce, a long announcement) = a step up, then a plateau until a longer one arrives. Fits the step-and-plateau shape exactly. Levers: SessionOptions → enable_mem_pattern=False / arena config (arena_extend_strategy=kSameAsRequested), or bounding input length before synthesis.
2. glibc malloc arenas — ThreadingHTTPServer spawns a thread per request; each new thread can land on a fresh malloc arena, and freed ORT/numpy buffers fragment across them. Classic mitigation: MALLOC_ARENA_MAX=2 (and/or MALLOC_TRIM_THRESHOLD_) in the container env — zero code, one compose line.
3. espeak-ng phonemizer state inside piper-tts 1.6.0 — least likely, but the render lock means it's exercised serially and its allocations are small.

🔧 Plan

1. Discriminate on the box, read-only: correlate the next step with the api's render log (longest text length that day) — if steps track new-longest inputs, it's (1). Snapshot /proc/<pid>/smaps_rollup + malloc_info via py-spy dump/gdb if reachable; RssAnon per check stays the trend line.
2. Try the free lever first: MALLOC_ARENA_MAX=2 on the piper service (compose + the piper-only overlay), rebuild, re-baseline on the Pi 4 for a week. If the slope flattens, ship it and record the new plateau in HARDWARE.md.
3. If not, ORT session options in piper/server.py (the image is repo-owned — piper-tts==1.6.0 exposes the session via PiperVoice), bounded input length as the belt.
4. Either way: a piper RSS line on the admin resources tile is already there (DockerContainerStatsSource) — no new observability needed; the soak runbook's piper reference numbers get refreshed.

Not in scope: raising the 768 MiB cap (it's the backstop that makes this harmless), or a cron restart (masking, not fixing).

Refs: HARDWARE.md Pi 4 row + soak runbook (the v3.3.2 498 MiB step), #559 (soak results), #242 (piper-only topology), #241 (the repo-owned piper image).