Skip to content

Teardown matches the socket spelling the engine carries, and verifies after the cascade (refs #344, refs #345) - #363

Merged
vyskocilm merged 7 commits into
mainfrom
issue-344-reap-matcher
Aug 22, 2026
Merged

Teardown matches the socket spelling the engine carries, and verifies after the cascade (refs #344, refs #345)#363
vyskocilm merged 7 commits into
mainfrom
issue-344-reap-matcher

Conversation

@vyskocilm

@vyskocilm vyskocilm commented Aug 22, 2026

Copy link
Copy Markdown
Contributor

refs #344, refs #345

Teardown's verification leg matched a string no process on the machine carries,
so it could not fail. Fixed, plus the four regression tests that hold it, plus
the measurement showing the fifth cannot exist.

redteam round: OWED, deferred by maintainer ruling, NOT waived. See the
last section — it names the round's target so whoever runs it does not have to
re-derive it.

The defect

Engine.paths() returned the HOST socket /tmp/snug-<uid>-<pid>/sock/podman-<pid>.sock.
Since Tier C the engine is exec'd with an argv written in its DERIVED view —
Spec builds "unix://" + guestSock — so its host-visible cmdline names
/snug/engine/sock/podman-<pid>.sock. engine.go says the two names must not be
tidied into one; reap.go was never told.

So waitQuiet returned nil on its first poll of every run, signalOwned was
never reached, the SIGKILL escalation never happened, and the "this sandbox's
containers did not die with it" warning was unreachable. #167's lesson —
host-side code reading engine-namespaced pids — unlearned for paths.

paths() now returns the recorded GUEST spelling, and Spec is the one place
that records it: a second derivation of the host→guest mapping in reap.go
would agree with Spec until the day it did not. It returns nil rather than
falling back to e.sock, because that spelling appears in no command line on the
machine, so a fallback is indistinguishable from not sweeping while looking like
a sweep. Emptiness has its own failure mode, so stopLocked reconciles it
against the lifeline.

The position

Stop split into Detach (drop the keepalive, at payload exit) and Stop
(verify, reap, remove the run directory, from the caller's own deferred
cleanup). Also fixes a latent bug: step 4 removed the run directory, holding the
live engine's socket and generated conf, while the engine was still serving it.

Be precise about what the split buys, because the loose version is false and
this PR measured it.
Real podman 5.8.4 bundle, wall clock payload-exit to
snug-exit:

fixed wiring    15ms
shipped wiring  15ms, 16ms, 15ms

No difference. The engine is Pdeathsig'd to P1, the stage, not to P0
(internal/stage/enginefork.go:156), and MainServe returns the moment
runOneSandbox has sent the "exited" event, whereupon exitOnStageError
calls os.Exit(0). P1 waits for nothing P0 does. On a clean run the shipped
wiring costs nothing.

What the position actually buys is a GUARANTEE where there was a RACE:

  • Stage.Wait() returns on recvEvent — when the event BYTES ARRIVE — and does
    NOT wait for P1 to exit. At payload exit, P0 and a still-exiting P1 race on
    separate cores. P1 wins 7/7 on an idle laptop and nothing MAKES it win.
  • Stage.Close() ends with cmd.Process.Wait(), which BLOCKS until P1 is
    reaped, and the kernel delivers a child's pdeathsig in
    forget_original_parent before that wait wakes. After it, the engine's
    SIGKILL is already queued.

A race it usually won, versus a barrier it cannot lose.

The 15.3s was measured on a decoy

The round that found #344 recorded "15.3s of added teardown with a decoy
standing in for the engine
". A decoy started by a harness is not forked by the
stage and carries no Pdeathsig, so it CANNOT die with P1: it survives,
waitQuiet burns the whole budget, and 15.3s is exactly right for that
process
. What it models is an engine outside the cascade's reach — the rare
case the sweep exists for — never an ordinary engine on a clean run.

General form, which is the transferable part: a substitute in a round models
the subject only in the dimensions it shares with it, and the round must say
which dimensions those are.
This decoy shared the cmdline (the dimension under
test) and not the parentage (the dimension that decides whether it dies).

quietBudget is REWRITTEN, not renumbered: 2s, covering scheduling and a walk of
/proc for a SIGKILL already delivered, with the relation to idleTimeout
deleted rather than kept as a coincidence. killBudget (3s) names the second
wait, which was a bare literal.

The accident sentence, inverted

The old text accepted one risk: the only way this reaps something else is a
foreign command line containing the host socket path. With the mark fixed the
string is /snug/engine/sock/podman-<pid>.socka string the PAYLOAD can
author
, since it can put candidate pids on its own argv and payload processes
are visible in the host /proc. What stops that is not the matcher but WHEN the
sweep runs: after the stage is reaped, the sandbox's pid namespace has collapsed
and no payload process is alive to match. Stated in the file, because moving the
sweep back re-opens it.

Tests

Four of five specified, plus a documented impossibility for the fifth.

test what it holds
TestTeardownMatchesTheArgvTheEngineIsStartedWith every element of paths() appears in the argv Spec produced; fails on the code that shipped
TestStopEscalatesToSIGKILLWhenTheEngineOutlivesTheCascade drives stopLocked step 3 for the first time in this repo's history
TestPathsIsEmptyUntilSpecHasRun paths() empty before Spec, non-empty after (the positive control)
TestStopLockedReconcilesADialledLifelineWithNoMark both arms of the reconciliation warning — a warning that always fires is as useless as one that never does
TestSweepDoesNotMatchTheHostSocketSpelling a live decoy naming the HOST spelling is not swept, with a mandatory guest-spelling decoy that must be, at both cmdlineNamesPath and ownedPIDs
test/guard/enginereapordering_test.go (3 tests) the three-package wiring: container.go wires Detach not Stop, main.go defers cleanup before sandbox.Run, exec.go defers st.Close
TestDescribeSanitisesACommandLine landed in #362

A mutation that trips a test's own guard has graded the guard, not the assertion

Stated as a method, because it generalises to every negative test in this repo
and it is not obvious: a mutation is only evidence about the assertion it
actually reaches.

The host-spelling negative needed two. Reverting paths() to e.sock — the
pre-#344 spelling, the obvious mutation — does fail the test, but at the VACUITY
GUARD: host and guest collapse to the same string and the guard fires first.
That grades the guard. It says the test refuses to run vacuously, and nothing
whatsoever about whether the negative assertion works. Reported as sufficient, it
would have looked like proof.

paths() returning {e.guestSock, e.sock} keeps the two spellings distinct, so
the guard passes and the mutation reaches the assertion — which then fails at
both cmdlineNamesPath and ownedPIDs, while the guest-spelling positive
control still holds. That is the one that grades the test.

Before believing a mutation, check WHICH assertion it tripped.

Every one mutation-proved. The escalation test's decoy blocks on a shell read
nothing closes, so it outlives quietBudget by construction rather than by
timing; SIGKILLSIGCONT fails it and prints the never-before-rendered "did
not die with it" warning. The ordering guard proves each regex matches today's
source before trusting it to prove absence, and mustFindOne fails on zero AND
on more than one, with distinct messages.

The ordering guard's third test is the weakest and says so in its first
sentence.
opts.OnPayloadExit() is called synchronously inside runStaged's
guard.wait callback, so it always precedes that function's own deferred
st.Close() whichever line comes first textually. It pins a written convention,
not a runtime ordering.

The fifth test cannot exist, and that is the deliverable

test/integration/enginereapteardown_test.go was written as the fifth slot and
the mandatory mutation (onPayloadExit: eng.Detacheng.Stop) DOES NOT FAIL
IT — measurement above. No threshold was tuned to make it pass. On a clean run
no black-box wall-clock test can distinguish the fixed wiring from the shipped
bug's.

Kept as a narrower real-engine ratchet: clean teardown stays well under
quietBudget, positive-controlled by a real /v1.41/version 200 in the same run
being timed. Its threshold is quietBudget/2, computed by parsing
internal/engine/engine.go's own const quietBudget at test time
rather than
copied, because that value already went stale once. The comment names the trap:
raising it toward 3s is not a relaxation, it is a disablement.

The property doing the clearing is named, so a later change re-opens the
question instead of inheriting a stale pass (CLAUDE.md's rule, and #313's round
cleared a route on a property #276 then deleted). The slot is unfillable because,
and only because, the engine is Pdeathsig'd to P1 rather than P0 AND
MainServe waits on nothing P0 does. Change either and the wall-clock test
becomes both writable and necessary.

Also here

SNUG_REAP_SOCK deleted. reaperScript has read only $SNUG_REAP_DIR since
#167 deleted the reaper's host-side podman stop; the variable survived that
deletion unread — a path handed to a process designed to outlive snug, for no
reason anyone can state. The sentence survives as a RULE: every run-specific path
travels in the environment, never argv, because two /proc sweeps match on path
strings and a path in argv makes snug's own cleanup match its own sweep. The
test's negative named sock alone, so deleting the parameter would have turned
it green by deletion — it now asserts the SET (run directory and socket
path).

#345's surviving second copy (refs #345). orphansweep.go's doc comment is
correct since #362 — the number IS recyclable while the fd is held, measured —
and twelve lines below its closing warning, inside the function that warning
governs, sat the original false claim: "the number cannot be recycled while
pidfd is held, so the checks below and the kill all name the same task."
#270's
shape: a rule fixed at the site the reporter quoted while a pre-existing copy
survives. No behavioural consequence today — the kill goes through the pin and
fails closed with ESRCH.

Swept the SET, not the site, before fixing. No third copy. reap.go:195 and
teardown.go:444 both state it correctly, so the header's claim about its two
neighbours is true — verified rather than inherited. Nuance worth carrying: they
state the same GUARANTEE, not the same WORDS, so a fixed-string sweep would
report two sites and miss the third, and a sweep that finds nothing looks
identical to one that proves absence.

Five stale inheritances in one branch, and what caught each

All the same shape — prose or a spec copying state that lived somewhere else,
each true where it was written and false where it was read. Recorded as a base
rate
, not a confession, so the catches are counted alongside the defects:
excluding the ones caught in-lane would quietly imply this mechanism only
produces defects that ship.

# the stale statement caught by
1 "nothing verified against a real podman" — true of /usr/bin/podman (the distrobox shim P1 refuses by name), false of the pinned 5.8.4 bundle the suite drives checking a caveat before repeating it
2 the review's under 3s — derived from quietBudget = 15s, which this same commit series rewrote to 2s, so the threshold passed on the bug reading the constant the number came from
3 the 15.3s — a decoy's cost read as an ordinary engine's measuring it
4 orphansweep.go's inline pin claim — corrected at the header in #362, left standing twelve lines below, inside the function that header governs verifying the tree after a merge
5 a new test's doc comment calling itself "#344's fifth and last regression test" — the fifth is the one that cannot exist review, in-lane, before it shipped

The distribution is the interesting part. One was caught by a mechanical
guard (TestEveryDesignDocACommentCitesExists, on a separate citation defect in
the same lane). One by checking the tree after a merge. Three only because
someone refused to repeat a number without checking it.
That is an argument for
the habit, not an apology for the branch — and it is the same measured base rate
CLAUDE.md already records for @git-ro binding ~/.gitconfig for a full
milestone before a mechanical sweep, not a redteam round, caught it.

Engines used

fakepodman via $SNUG_PODMAN for the matcher measurements behind 80ca6b2.
The pinned static podman 5.8.4 bundle for the integration timing test and its
positive control. Per-claim, not per-branch: /usr/bin/podman here is distrobox's
distrobox-host-exec shim, which preflight P1 refuses by name.

Gate

make gate green on every commit. Integration green with a real engine.

The owed redteam round

#344 owes a redteam round. It was deferred by maintainer ruling, not
waived.
Definition-of-done rule 3 applies in full: this is engine/stage and it
changes what gets SIGKILLed and when. The round is scheduled against MERGED code,
in its own lane.

The reasoning is worth stating rather than referencing: a round is worth running
against the code that SHIPS, and a lane in flight is not that code yet. Running
one mid-lane spends it on a diff that is still moving and produces the one thing
a round record must never be — a note saying a surface was attacked when what was
attacked was a draft.

The round's target, written here so it does not have to be re-derived:

  1. The matcher. Decoys naming the GUEST spelling
    /snug/engine/sock/podman-<pid>.sock against decoys naming the HOST spelling
    /tmp/snug-<uid>-<pid>/sock/podman-<pid>.sock. The guest string is one a
    PAYLOAD can author on its own argv, which the host string was not.
  2. The SIGKILL escalation in stopLocked step 3 — unreachable before this
    branch, so it has one unit test and no adversarial exposure at all.
  3. Whether the ORDERING that makes the guest sock non-forgeable actually
    holds.
    The sweep runs only after the stage is reaped and the sandbox's pid
    namespace has collapsed, so no payload process should be alive to match. That
    is the claim; it has not been attacked.

Engines, per-claim and never per-branch. Everything behind 80ca6b2 was
measured against test/integration/testdata/fakepodman via $SNUG_PODMAN. The
new integration work drives the pinned static podman 5.8.4 bundle at
~/.local/opt/podman-static. /usr/bin/podman on this host is distrobox's
distrobox-host-exec shim, which preflight P1 refuses by name.

… verifies after the cascade (refs #344)

Two halves of one defect, and they only work together.

THE MATCHER. reap.go identifies this run's engine by looking for a socket path
in a host-visible command line. Engine.paths() returned e.sock, the HOST path
/tmp/snug-<uid>-<pid>/sock/podman-<pid>.sock. Since Tier C the engine is exec'd
with an argv written in its DERIVED view — engine.go's Spec builds
"unix://" + guestSock — so its host-visible cmdline names
/snug/engine/sock/podman-<pid>.sock. engine.go:547-550 says the two names must
not be tidied into one; reap.go was never told.

Measured by the owed redteam round, twice: a decoy naming the GUEST path — what
a leaked engine looks like — SURVIVED teardown, while a decoy naming the HOST
path was killed and burned the full 15s budget. So waitQuiet returned nil on its
first poll on every run, signalOwned was never reached, the SIGKILL fallback
never happened, and the "this sandbox's containers did not die with it" warning
was unreachable. #167's lesson — host-side code reading engine-namespaced pids —
unlearned for paths.

paths() now returns the recorded GUEST spelling, and Spec is the one place that
records it: a second derivation of the host->guest mapping in reap.go would
agree with Spec until the day it did not. It returns nil rather than falling
back to e.sock when Spec never ran, because that spelling appears in no command
line on the machine and falling back to it is indistinguishable from not
sweeping while looking like a sweep. Emptiness has its own failure mode, so
stopLocked reconciles it: a dialled lifeline with no mark is named as a wiring
bug rather than swept vacuously.

THE POSITION, which is why the matcher could not simply be swapped. stopLocked
ran from OnPayloadExit — inside runStaged, BEFORE its deferred st.Close(), so
before the Pdeathsig cascade P1 -> engine has fired. The engine is ALIVE there by
construction, so the only thing that could make a working sweep go quiet is the
engine's own idle timeout: measured 15.3s of added teardown on every clean run.
quietBudget was idleTimeout + 5s precisely because of that, i.e. snug's own exit
waited out a timeout whose entire purpose is to cover snug NOT BEING THERE.

So Engine.Stop is split. Detach (drop the keepalive, idempotent) is what
OnPayloadExit gets; Stop — verify, reap, remove the run directory — stays wired
to the caller's own cleanup, which main.go registers with `defer ctr.cleanup()`
BEFORE it calls sandbox.Run and therefore runs strictly after runStaged's
deferred st.Close(). st.Close() does not return until P1 has been reaped, and
the kernel delivers a child's pdeathsig in forget_original_parent before
do_notify_parent wakes that wait, so by the time Stop sweeps the engine's SIGKILL
is already queued. That is the first position from which the question has an
answer other than "yes, obviously": a process the cascade killed is gone or a
zombie, and a zombie reads back an empty cmdline, which cmdlineNamesPath already
answers "not ours" to. What survives is an engine outside the cascade's reach —
the case this sweep was written for.

It also fixes a latent bug: step 4 removed the run directory, holding the live
engine's socket and generated conf, while the engine was still serving it.

quietBudget is REWRITTEN, not renumbered: 2s, covering scheduling and a walk of
/proc for a SIGKILL already delivered, with the relation to idleTimeout deleted
rather than kept as a coincidence. killBudget (3s) names the second wait, which
was a bare literal. waitQuiet's doc loses the "quietBudget is idleTimeout + 5s"
clause, which is now false.

THE ACCIDENT SENTENCE INVERTED, and the new one says what actually holds. The
old text accepted one risk: the only way this reaps something else is a foreign
command line containing the host socket path. With the mark fixed the string is
/snug/engine/sock/podman-<pid>.sock — a string the PAYLOAD can author, since it
can put candidate pids on its own argv and payload processes are visible in the
host /proc. What stops that is not the matcher but WHEN the sweep runs: after
the stage is reaped, the sandbox's pid namespace has collapsed and no payload
process is alive to match. Stated in the file, because moving the sweep back
re-opens it.

describe() now escapes through policy.VisibleText. internal/cli sweeps its own
screens for forging runes and that sweep does not reach internal/engine, which is
why this could print a raw command line for as long as it did. The warning is a
screen a human reads especially carefully — it is the one telling them what to
kill -9 — and what it renders is a container's or a process's own argv, not
snug's text. Escaped BEFORE the length clamp, so a cut cannot leave a raw 0x9b
behind.

TESTS

TestTeardownMatchesTheArgvTheEngineIsStartedWith — the ratchet, pure and cheap.
Build an Engine, call Spec, assert every element of paths() appears in the argv
Spec produced, plus the negative (the HOST spelling must not be a mark when the
derived view gives a different name) and an empty-paths() control, since an empty
mark set satisfies every other assertion while meaning the sweep matches nothing.
It fails on the code that shipped, printing both spellings.

TestTheTeardownWarningEscapesTheCommandLinesItPrints — a real process with a
poisoned argv, read back through /proc the way describe() reads it. The poison
goes in argv[0]: `sleep 60 <poison>` exits before /proc can be read, and
`sh -c 'sleep 60' <poison>` is exec-optimised by the shell and silently loses the
marker (the harness caveat on #344). Control that an ordinary command line still
renders as itself, because a describe() that quoted everything would pass the
escape assertion and make the warning unreadable.

TestOwnedPIDsMatchesOnlyThisEnginesPaths now runs Spec first and builds its
positive marker from the guest spelling. Without Spec its paths() is empty and
every negative in it would pass vacuously.

make gate green.

PARKED, NOT FINISHED — read this before picking it up.

Session stopped at the maintainer's call with this branch incomplete. What is
here is coherent and green, but it is NOT ready to land, and the missing half is
named rather than left to be rediscovered.

DONE:
  - the matcher: paths() returns the recorded GUEST spelling, nil before Spec
  - the position: Stop split into Detach (payload exit) and Stop (after the
    cascade), wired in internal/cli/container.go
  - quietBudget rewritten to 2s with killBudget named, and the stale prose in
    main.go, exec.go and reap.go corrected
  - describe() escapes through policy.VisibleText
  - the Spec-argv ratchet, which is the deliverable: it fails on the code that
    shipped and passes here
  - make gate green; go test ./... green

NOT DONE, and each is a real gap:
  - THE MANDATORY redteam ROUND HAS NOT RUN. Definition-of-done rule 3 applies —
    this is engine/stage, a host-integration surface, and it changes what gets
    SIGKILLed and when. It was deliberately not started, not skipped by
    oversight. Nothing here may land before it does.
  - The regression tests specified by the host-bridge review are missing:
    paths() empty until Spec has run plus the reconciliation warning;
    a decoy naming the HOST spelling must NOT match (the negative that makes
    guest-only a fact rather than an implementation detail); the SIGKILL
    escalation, which no test has ever exercised because it was unreachable;
    a source-text test pinning the three-package ordering (container.go wires
    Detach not Stop, main.go defers cleanup before sandbox.Run, exec.go defers
    st.Close); and the integration test that would have caught the 15s a naive
    matcher fix introduces — wall clock from payload exit to snug exit under 3s,
    with a mandatory positive control that the engine actually served.
  - SNUG_REAP_SOCK is dead (reaperScript reads only $SNUG_REAP_DIR since #167).
    The review says to delete the parameter and the variable in a separate
    commit while KEEPING the sentence as a rule about the reaper's argv. Not
    done here.
  - Unverified against a real podman: this host's /usr/bin/podman is distrobox's
    host-exec shim, which preflight P1 refuses, so every measurement behind this
    change used the committed fakepodman stand-in via $SNUG_PODMAN. The claim
    that a zombie engine reads back an empty cmdline, and the end-to-end timing,
    both want a run against a real engine.
…er's argv (refs #344)

reaperScript has read only $SNUG_REAP_DIR since #167 deleted the reaper's
host-side `podman stop`. SNUG_REAP_SOCK survived that deletion unread: a
path handed to a process that by design outlives snug, for no reason anyone
can state. startReaper's sock parameter goes with it.

WHAT SURVIVES IS THE RULE, not the variable. The old comment said "the path
travels in the ENVIRONMENT, not in argv, so the reaper's own command line
does not name the socket the sweep in reap.go matches on". That is one
variable's fact. Restated forward: EVERY run-specific path this helper is
given travels in the environment. Its argv is `/bin/sh -c <script>` and the
script names only variables, never values. Two /proc sweeps match on path
strings — reap.go's cmdlineNamesPath and internal/sandbox/teardown.go's
confirmTeardown — so a path in argv makes snug's own cleanup match its own
sweep: reported as a leaked engine by the first, killed as one by the second
were it not exempted by pid (#113).

THE TEST ASSERTS THE SET, NOT THE SITE. TestReaperFiresOnEOF...'s negative
named `sock` alone, so deleting the parameter would have turned it green by
deletion rather than by the property holding. It now checks both the run
directory (which startReaper is handed) and the socket path (which it is
not, any more), with a distinguishable message per path.

MUTATION OBSERVED. Adding runDir to the argv --
exec.Command("/bin/sh", "-c", reaperScript, runDir) -- fails it on both arms:

    the reaper's command line names this run's directory
    ("/tmp/.../snug-1000-508950"); a /proc sweep matching on path strings
    will match snug's own cleanup process.

The empty-cmdline control (#317's arg_start window) is unchanged and still
asks its question separately: an empty read means the assertion never ran.

Two mentions of $SNUG_REAP_SOCK stay in engine_test.go as history (`rm -f
"$SNUG_REAP_SOCK"`, `rm -rf "$(dirname "$SNUG_REAP_SOCK")"`) because that
history is why the target is STATED rather than DERIVED. Marked as history
at the first, so a grep finding no live one is not read as the comment being
stale. ArmReaper's doc said it "needs only this run's socket path" -- now
false, it needs the run DIRECTORY.

Engine unchanged: fakepodman via $SNUG_PODMAN throughout, never a real
podman (this host's /usr/bin/podman is distrobox's shim, refused by
preflight P1).

make gate green.
…ring (refs #344)

Two of the five regression tests the host-bridge review specified. Both
guard code that has never been exercised, for the same reason: #344's
matcher named a string no process on the machine carried, so everything
downstream of it was dead code that looked green.

THE ESCALATION (internal/engine/reapescalation_test.go).
stopLocked step 3 is waitQuiet -> signalOwned(SIGKILL) -> waitQuiet ->
WARNING. The OUTER condition could never be true before this branch, so
signalOwned, the second waitQuiet and the warning had never run once.
TestStopEscalatesToSIGKILLWhenTheEngineOutlivesTheCascade builds an Engine,
runs Spec (paths() is empty until it has), starts a decoy through the
existing marker() helper -- /bin/sh -c 'read x' <mark>, which blocks forever
because nothing closes its stdin, so it survives quietBudget (2s) by
construction rather than by timing -- and asserts the decoy died BY SIGKILL.

Three positive controls before Stop runs, each removing a way the test could
pass vacuously: e.paths() non-empty (an empty mark set sweeps for nothing),
the decoy's cmdline really names the mark, the decoy really alive. The wait
is BOUNDED (5s, channel + select) rather than a bare Process.Wait: a broken
escalation leaves the decoy alive forever, so a bare Wait would HANG the
suite instead of failing it.

MUTATION OBSERVED, run twice independently:
signalOwned(..., syscall.SIGKILL) -> syscall.SIGCONT

    --- FAIL: TestStopEscalatesToSIGKILLWhenTheEngineOutlivesTheCascade (10.06s)
        the decoy (pid 525598) was still alive 5s after Stop() returned:
        Stop's SIGKILL escalation did not reach it

That mutation also printed "snug: WARNING - this sandbox's containers did
not die with it" with the decoy's real pid and cmdline -- the first time
that branch has ever rendered.

The final warning branch is NOT covered, stated in the file rather than
faked: no process survives SIGKILL, and the one thing that does (a zombie)
reads back an empty cmdline, which cmdlineNamesPath correctly answers "not
ours" to, so it would make waitQuiet report quiet rather than reach the
warning. No honest fixture exists for "signalled, still there, still
identifiable".

THE ORDERING (test/guard/enginereapordering_test.go). The fix's other half
is a position, spread over three files in three packages that no compiler,
vet, or in-package test sees together:

  1. internal/cli/container.go   cleanup calls eng.Stop(), onPayloadExit is
                                 eng.Detach -- never the reverse
  2. internal/cli/main.go        `defer ctr.cleanup()` textually BEFORE the
                                 sandbox.Run( call (LIFO: registered first,
                                 fires last)
  3. internal/sandbox/exec.go    `defer st.Close()` textually BEFORE the
                                 opts.OnPayloadExit() call

Any one reintroduced alone silently reopens #344, and internal/engine's own
tests cannot notice: they call stopLocked directly and never touch the
wiring that decides WHEN it runs relative to the stage's collapse.

Every regex is proved to match today's source before it is trusted to prove
absence. mustFindOne fails on zero matches AND on more than one, with a
message distinct from the ordering failure, so a rename fails loudly instead
of the sweep finding nothing and reporting a clean pass. Test 1 additionally
runs its negative pattern against a synthetic pre-#344 fixture first.

MUTATIONS OBSERVED (one verified independently, all reverted):
  container.go onPayloadExit: eng.Detach -> eng.Stop
    "onPayloadExit set to eng.Detach (pattern onPayloadExit:\s*eng\.Detach)
     was not found at all"
  main.go `defer ctr.cleanup()` moved after sandbox.Run(
    "(byte offset 22182) does not appear before the sandbox.Run( call (byte
     offset 22125)"
  exec.go `defer st.Close()` moved after opts.OnPayloadExit()
    "(byte offset 26807) does not appear before the opts.OnPayloadExit()
     call (byte offset 26785)"

TEST 3 IS THE WEAKEST OF THE THREE AND SAYS SO FIRST. opts.OnPayloadExit()
is called synchronously inside runStaged's guard.wait callback, so it always
runs before that function's own deferred st.Close() fires whichever line
comes first textually. Reversing the two changes nothing the runtime does.
What it pins is which window the call is WRITTEN to belong to, so a later
edit making OnPayloadExit order-sensitive cannot silently inherit the wrong
position. A reader who assumes it carries the same weight as tests 1 and 2
would trust it for something it does not do.

Two stale references corrected before commit: the ratchet lives in
reapmark_test.go, not reapscreen_test.go, and the escalation test is in
internal/engine, not test/guard.

No non-test source file touched. Engine throughout is fakepodman via
$SNUG_PODMAN, never a real podman.

make gate green.
…measurement (refs #344)

The review's fifth slot wanted an integration test catching "the 15s a naive
matcher fix adds": wall clock payload-exit to snug-exit, under 3s. Written
against a real podman 5.8.4 bundle. THE MANDATORY MUTATION DOES NOT FAIL IT,
and that is the deliverable rather than a shortfall.

MEASURED, mutation = internal/cli/container.go's onPayloadExit: eng.Detach
rewired to eng.Stop, the exact shape #344 shipped with. Real engine, real
/v1.41/version 200 positive control in the same run being timed:

    fixed wiring    15ms
    mutated wiring  15ms, 16ms, 15ms   (three runs)

Threshold 1s, quietBudget 2s. No difference. Run twice independently, once
with temporary stderr instrumentation in stopLocked (reverted): the first
waitQuiet found ZERO processes owning the engine's socket on its FIRST poll,
~8-16ms after onPayloadExit ran, on BOTH wirings, across seven runs.

WHY. internal/stage/serve.go's MainServe returns as soon as runOneSandbox has
sent the "exited" event, and internal/cli/main.go's exitOnStageError calls
os.Exit(0) the instant it returns. The engine is Pdeathsig'd to P1 (the
stage), not to P0 -- internal/stage/enginefork.go:156. So P1 exits on its own,
unconditionally and near-instantly, once the payload is reaped; it waits for
nothing P0 does. By the time P0 wakes from its blocking read of that event and
calls onPayloadExit, the cascade has already felled the engine. On a clean run
the shipped bug costs nothing, so no black-box clean-run timing test can
discriminate.

THE 15.3s WAS MEASURED ON A DECOY, which is the whole discrepancy. A decoy
started by a harness is not forked by the stage and carries no Pdeathsig, so
it CANNOT die with P1: it survives, waitQuiet burns the whole budget, 15.3s is
correct for that process. It models an engine OUTSIDE the cascade's reach --
the rare case the sweep exists for -- not an ordinary engine on a clean run.
The review read a decoy's cost as every clean run's cost and the spec
inherited it.

THE POSITION FIX IS STILL RIGHT, on a narrower claim than the review made, and
the file says so at length so nobody reverts it off the back of this
measurement. Stage.Wait() (stage.go:484) returns on recvEvent -- when the
event BYTES ARRIVE -- and does NOT wait for P1 to exit, so onPayloadExit sits
in a RACE with a still-exiting P1 that P1 happens to win 7/7 on an idle
laptop. Stage.Close() (stage.go:503) ends with cmd.Process.Wait(), which
BLOCKS until P1 is reaped, and the kernel delivers a child's pdeathsig in
forget_original_parent before do_notify_parent wakes that wait. Shipped wiring:
a race it usually won. Fixed wiring: a barrier it cannot lose. Real property,
not one a wall clock can see, because the losing side of the race is the rare
side.

WHAT THE TEST IS KEPT AS. A real-engine, positive-controlled ratchet that
clean container teardown stays well under quietBudget -- worth catching for
any reason -- with the package comment stating in its first section that it
does NOT close #344's fifth slot. Threshold is quietBudget/2, computed by
parsing internal/engine/engine.go's own `const quietBudget` at test time
rather than copying the value, because that value already went stale once
(15s to 2s, in this same commit series). The comment names the trap: raising
this toward 3s is not a relaxation, it is a disablement.

Three corrections made to the agent's draft before commit. It claimed the
shipped bug's "cost floor IS quietBudget" while its own headline measurement
refutes that -- the bound is about what a threshold can RESOLVE, never about
what the bug costs. It framed P1's exit as making the split unnecessary,
missing that Wait() returns on the event and Close() on the reap -- race
versus guarantee. And it cited OWED-REDTEAM.md, which is gitignored:
TestEveryDesignDocACommentCitesExists caught it ("a citation a reader cannot
follow"), now cites the issue instead.

No non-test source file touched. make gate green; the new test green under
SNUG_REQUIRE_SANDBOX=1 with a real engine.
… expires loudly (refs #344)

"Unwritable" is a fact about today's architecture, not a law. CLAUDE.md's own
rule -- when a round clears a route, RECORD WHICH PROPERTY DID THE CLEARING --
exists because #313's round cleared a route on a property #276 then deleted,
and the pass was silently inherited.

The package comment said the fifth slot is not closed, which is the fact. It
now says WHY, which is the part that expires. The slot is unfillable because,
and only because, BOTH hold:

  1. the engine is Pdeathsig'd to P1, the stage, not to P0
     (internal/stage/enginefork.go:156), so P1's exit alone fells it;
  2. MainServe waits on nothing P0 does -- it returns the moment
     runOneSandbox has sent the "exited" event, and exitOnStageError then
     calls os.Exit(0).

Pdeathsig the engine to P0, or give MainServe anything to wait for, and the
shipped bug acquires a real per-run cost -- at which point the wall-clock test
becomes both writable and NECESSARY. The paragraph is addressed to whoever
makes that change.

make gate green.
…344, refs #345)

Two conflicts, both prose, both in paragraphs #362 and this branch edited for
opposite reasons. Neither side taken wholesale.

reap.go, conflict 1 -- the accident sentence. #362 describes the matcher as
UNFIXED and says so outright ("The matcher is not corrected here; #344 carries
it"). This branch corrects it, so HEAD's substance wins: the mark is
/snug/engine/sock/podman-<pid>.sock, a string the PAYLOAD can author, and what
stops that is WHEN the sweep runs, not the matcher. Kept from #362: the host
spelling is /tmp/snug-<uid>-<pid>/sock/podman-<pid>.sock and NOT under
$XDG_RUNTIME_DIR, where an earlier design put it before #63 Tier B moved the
run directory to /tmp. That fact now earns its place for a new reason -- it is
the other half of paths() returning nil rather than falling back to e.sock: no
process carries that string, so a fallback is indistinguishable from not
sweeping.

reap.go, conflict 2 -- describe()'s escaping. HEAD's substance again (the
match is now a payload-authorable string, so escaping stopped being
theoretical), plus #362's naming of WHICH test covers the sink, which HEAD
omitted: TestNoSnugScreenEmitsARawControlCharacter drives the --dry-run screen
and does not reach internal/engine; TestDescribeSanitisesACommandLine does.

container.go -- not a real disagreement. main added `started = true` (disarming
the early-cleanup defer), this branch added the Detach/Stop rationale. Both
kept. The rationale is also CORRECTED by this branch's own measurement: it no
longer says the shipped wiring makes snug wait out the engine's idle timeout,
because that is false -- measured 15ms on both wirings against a real podman
5.8.4. It says what is true instead: Stage.Wait returns on recvEvent (the
bytes ARRIVING), not on P1's exit, so payload-exit is a RACE that P1 merely
happens to win; st.Close ends with cmd.Process.Wait, after which it cannot
lose. A guarantee where there was a race.

Five test call sites updated for main's two signature changes: Spec's fifth
parameter (noSignaturePolicy(t) -- "this host configured none", never nil,
which #307 makes an error naming the skipped ProjectHostSignaturePolicy) and
New now taking a *policy.Policy (testPol(...)).

#345's SECOND COPY, which is the reason this touches orphansweep.go at all.
The doc comment there is correct since #362 -- the number IS recyclable while
the fd is held, measured, with a closing warning that a future edit acting on
the number is not made safe by the pin. Twelve lines below that warning, inside
the very function it governs:

    // pid is pinned from here: the number cannot be recycled while pidfd is
    // held, so the checks below and the kill all name the same task.

The original false claim, in its second location, and exactly the licence the
corrected header withdraws. #270's shape: a rule fixed at the site the reporter
quoted while a pre-existing copy survives. No behavioural consequence today --
the kill goes through the pin and fails closed with ESRCH.

SWEPT THE SET, NOT THE SITE, before fixing. All three files the header claims
"state the same guarantee the same way" were checked. NO THIRD COPY:
  internal/engine/reap.go:195       correct -- "pins the task the number named
                                    AT OPEN TIME, so pidfd_send_signal can
                                    never land on a later reuse"
  internal/sandbox/teardown.go:444  correct -- "pins whatever process holds pid
                                    RIGHT NOW"; names the recycled-occupant
                                    case explicitly and skips rather than kills
So the header's claim about its two neighbours is TRUE, verified rather than
inherited. Same guarantee, not the same words.

The replacement says what the pin does guarantee, and adds the part the old
text obscured: procStartTime and procNamespaceInodes below read /proc/<pid> BY
NUMBER, so identity comes from what they COMPARE against the state file, never
from the pin.

make gate green.
…elling negative (refs #344)

Completes the four that can be written. The fifth was the integration timing
test; test/integration/enginereapteardown_test.go carries the measurement
showing it cannot exist on this architecture.

reapvacuity_test.go -- EMPTINESS, AND ITS RECONCILIATION.

TestPathsIsEmptyUntilSpecHasRun: paths() is empty on a fresh Engine and
non-empty after Spec. The second half is the positive control -- without it,
"empty before Spec" passes on a paths() that returns nil always.

TestStopLockedReconcilesADialledLifelineWithNoMark: both arms of
`e.dialled && len(e.paths()) == 0`. A warning that always fires is as useless
as one that never does, so the ordinary case (Spec ran, mark exists) is
asserted silent. Stderr is captured by a local os.Pipe swap restored
unconditionally; NO test-only writer field or hook was added to engine.go. The
helper documents that it stops being safe if this package ever adds
t.Parallel.

reaphostspelling_test.go -- THE NEGATIVE THAT MAKES GUEST-ONLY A FACT.

A live decoy naming the HOST spelling /tmp/snug-<uid>-<pid>/sock/podman-<pid>.sock
must not be swept. That is exactly the string paths() returned before the fix,
and exactly why the sweep matched nothing on every run.

Asserted at BOTH levels: cmdlineNamesPath (the predicate) and ownedPIDs (what
signalOwned and waitQuiet actually call).

Two guards, because this test degrades into a vacuous one in two different
ways:
  - MANDATORY GUEST-SPELLING POSITIVE CONTROL. A second decoy naming
    e.paths()[0] must be MATCHED. Without it, "the host decoy was not matched"
    is satisfied by a matcher that matches nothing at all -- #344's own defect,
    which would otherwise pass its own regression test.
  - host != guest asserted up front, as a Fatal. If those spellings ever
    became one string, both decoys become indistinguishable and every
    assertion holds for the wrong reason.

MUTATIONS, both run independently rather than taken on report.

1. paths() -> []string{e.sock}, the pre-#344 spelling. Fails all three new
   tests, but for test B it fires the VACUITY GUARD rather than the negative:

     e.Socket() ("/tmp/snug-1000-619877/sock/podman-619877.sock") and
     e.paths()[0] (...same...) are identical -- ... makes the whole test
     vacuous

   Correct behaviour, and not sufficient proof: it shows the test refuses to
   run vacuously, not that the negative assertion works.

2. paths() -> []string{e.guestSock, e.sock}. Host and guest stay distinct, the
   guard passes, and the NEGATIVE fails for the right reason at both levels
   while the guest positive control still holds:

     cmdlineNamesPath matched decoy pid 620437, whose command line names the
     HOST socket spelling ... the engine is never exec'd with this string
     ownedPIDs(paths(), exclude) reports the host-spelling decoy pid 620437
     as owned by this run

   That is the one that proves the assertion, and it is why mutation 1 alone
   would have been a weaker claim than it looks.

Reconciliation-condition mutations (forced false, forced true) fail the
respective arms of the stopLocked test.

No production source file touched. make gate green; the three new tests green
10x with no flakiness from the arg_start-window polling.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant