Skip to content

fix: one state file, many instances — an orphan you cannot stop - #10

Merged
lntvan166 merged 1 commit into
mainfrom
fix/one-state-file-many-instances
Aug 21, 2026
Merged

fix: one state file, many instances — an orphan you cannot stop#10
lntvan166 merged 1 commit into
mainfrom
fix/one-state-file-many-instances

Conversation

@lntvan166

Copy link
Copy Markdown
Owner

paddock stop answered "not running" while a paddock held the port, so
paddock stop && paddock start failed on "port 8787 is already in use". The
report was "always have this issue", and it is not a version mix-up: it is one
bug with two halves.

Root cause

There is ONE state file per config dir, but paddock is per-PORT — a
PADDOCK_PORT=… instance, a --demo on a spare port and paddock tunnel are
all separate serving processes writing the same file.

  • recordState wrote unconditionally, so the second instance to start
    silently took over the first's record.
  • removeState deleted unconditionally, so the first instance to exit
    wiped the file.

Whatever still held the dashboard's port was then untrackable for the rest of
its life. Reproduced end to end before fixing:

Step State file paddock status
Start A on 8795 pid A, port 8795 running (A)
Start B on 8796 pid B, port 8796 — A orphaned running (B)
Ctrl-C B gone paddock — not running
A still serving stop → "not running", A survives; start → port in use

The fix

First instance wins. recordState refuses to write over a running
record belonging to a different pid, and says so. Only that case blocks the
write — a stale record, a mismatch, garbage, and this pid's own earlier record
must all stay claimable, or one leftover file would lock out every future
start. Servers clear their record with removeOwnState, which deletes only a
record it can positively identify as its own; removeState stays
unconditional for stop/status, which delete only after deciding the record
is theirs to clear.

Existing orphans — a SIGKILL leaves no cleanup either — are no longer
reported as absent. status and stop probe the port when no record exists
and name what is serving. stop returns non-zero there, so stop && start
halts with the reason instead of at the bind. No signal is sent: the pid lived
in the record that is gone, and finding a pid from a port is
platform-specific, so it hands over the command rather than guessing at a
kill.

paddock update replaces the binary but cannot restart the process running
it — /proc/<pid>/exe reads "… (deleted)" and the old build keeps serving. It
now reports the pid, port and version still running, with the restart command.
Told, not done, by the operator's choice: restarting would drop every
connected phone mid-session to finish a command run for the binary's sake.

Note on one test change

tests/cli.test.ts spawned verbs without PADDOCK_PORT, so they inherited
8787 — where a developer's own paddock listens — and the new probe
legitimately found it. It now asks the OS for a free port, the same fix
docs/gotchas.md already records for port ranges that "look unused". No
assertion was weakened.

Verification

make check / make check-clean   clean
bun test                        1022 pass, 0 fail, 101 files

Live: B refuses to record over A and says why; B's Ctrl-C leaves A's record
intact; A stays trackable and stoppable. Against a real orphan, status went
from paddock — not running to naming the untracked instance and the version
it was still serving. The real dashboard was then restarted and is tracked
again on 0.8.2.

🤖 Generated with Claude Code

Copilot AI lite review requested due to automatic review settings August 21, 2026 02:11

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

`paddock stop` answered "not running" while a paddock held the port, so
`paddock stop && paddock start` failed on "port 8787 is already in use". The
operator's report was "always have this issue", and it was not a version
mix-up: it is one bug with two halves.

There is ONE state file per config dir, but paddock is per-PORT — a
`PADDOCK_PORT=…` instance, a `--demo` on a spare port, a dev or test server.
`recordState` wrote unconditionally, so the second instance to start silently
took over the first's record; `removeState` deleted unconditionally, so the
first instance to EXIT wiped the file. Whatever still held the dashboard's
port was then untrackable for the rest of its life.

paddock's own advice walks an operator into it: "port 8787 is already in use …
choose another port: PADDOCK_PORT=8788 paddock". Do that, then stop the second
instance, and the first is orphaned. (`paddock tunnel` is NOT one of these
cases, though an earlier draft of this message said so — it serves the
dashboard itself and its preflight refuses to start beside a recorded
instance. Verified by running it against a live one.)

Reproduced end to end before fixing: start A on one port, start B on another,
Ctrl-C B, and A serves on with `status` and `stop` both saying "not running"
and `start` refusing A's port.

FIRST INSTANCE WINS. `recordState` refuses to write over a `running` record
belonging to a different pid, and says so. Only that case blocks the write — a
stale record, a mismatch, garbage and this pid's own earlier record must all
stay claimable, or one leftover file would lock out every future start. A
conflict check that THROWS also refuses, and is caught: this runs at top level
right after the bind, where an escaping rejection would kill a paddock that is
already serving, and "cannot tell who holds the record" must not become
"overwrite it". Servers now clear their record with `removeOwnState`, which
deletes only a record it can positively identify as its own; `removeState`
stays unconditional for `stop` and `status`, which delete only after deciding
the record is theirs to clear.

That fixes new orphans. For the ones already out there — a SIGKILL leaves no
cleanup either — `status` and `stop` now probe the port when no record exists
and name what is serving, instead of reporting a live process as "not
running". `stop` returns non-zero there so `stop && start` halts with the
reason rather than at the bind. No signal is sent: the pid lived in the record
that is gone, and finding a pid from a port is platform-specific, so it hands
over the command instead of guessing at a kill.

Separately, `paddock update` replaces the binary but cannot restart the
process running it — `/proc/<pid>/exe` reads "… (deleted)" and the old build
keeps serving. It now reports the pid, port and version still running, with
the restart command. Told, NOT done, by the operator's choice: restarting
would drop every connected phone mid-session to finish a command they ran for
the binary's sake.

One existing test changed rather than an assertion: tests/cli.test.ts spawned
verbs without PADDOCK_PORT, so they inherited 8787 — where the developer's own
paddock listens — and the new probe legitimately found it. It now asks the OS
for a free port, the same fix docs/gotchas.md already records for port ranges
that "look unused".

  make check / make check-clean   clean
  bun test                        1023 pass, 0 fail, 101 files

Verified live: B refuses to record over A and says why, B's Ctrl-C leaves A's
record intact, A stays trackable and stoppable. Against the real orphan on
this machine, `status` went from "paddock — not running" to naming the
untracked instance and the version it was still serving; that instance was
then restarted and is tracked again on 0.8.2.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@lntvan166
lntvan166 force-pushed the fix/one-state-file-many-instances branch from 872cbd6 to 115b8bd Compare August 21, 2026 02:17
@lntvan166
lntvan166 merged commit 73104e3 into main Aug 21, 2026
1 check passed
@lntvan166
lntvan166 deleted the fix/one-state-file-many-instances branch August 21, 2026 02:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants