docs(specs): specify clojure.test conformance, clojure.test.tap, and stack-trace primitives

[nnunley]

Adds docs/specs/clojure-test-conformance.md, an NLSpec-style specification for porting clojure.test to Clojure 1.12.5 semantics so unmodified harnesses (kaocha, cognitect test-runner, #738) can drive let-go: metadata-based test discovery, the report multimethod and do-report, *test-out* / with-test-out, per-namespace fixtures, test-var / test-vars / test-ns, runners that return the summary map, and a port of clojure.test.tap with the plan printed last as the JVM oracle does.

It also specifies the runtime pieces the port needs: structured stack frames captured on every throw (including across the Go boundary), current-stack-trace / ex-trace / Throwable->map and a clojure.stacktrace namespace, namespace metadata, exposing the reader's FormSource positions through meta / &form / def, *file*, *out* accepting io/buffer, and a Go harness that first consumes the summary and then bridges report events into testing.T subtests.

It names #673, #671, #754 as the work whose behavior survives and whose mechanism (the registry) it replaces, relies on the typed-catch dispatch from #472/#476 as-is, and lists every deviation from the JVM oracle in an appendix with the oracle transcripts.

testing-and-conformance.md yields clojure-test-api-design to the new spec via superseded-by, and the docs README gains a topical-map row. Docs only; no code.

Pushed past the pre-push ratchet: it is red on current main independent of this change, see #791.

Refs #739 (fully implement the clojure.test.* namespaces) and #738 (run the cognitect test-runner unmodified). This PR is the specification for both; it does not close them. The implementation lands in the three delivery slices the spec defines (traces; runtime extensions plus the clojure.test / clojure.test.tap port; the Go harness bridge), and the slice that makes the harnesses run will carry the closing reference.

[mparrett]

I reviewed docs-only head ba70c59 against the current runtime and the Clojure 1.12.5 oracle. CI is green and the document is impressively comprehensive, but I found five blocking correctness/design issues:

1. The trace contract has no viable ownership model for arbitrary thrown values. Sections 5.2-5.3 require ex-trace to work for every value, distinguish a never-thrown value, preserve a trace on rethrow, and assign a new trace to a newly thrown value. Today ThrownError owns only the raw Value, and handleError discards that wrapper when it pushes errorToValue(err) into a catch. A side table keyed by Value cannot satisfy the contract: some values (for example ArrayVector) are not Go-comparable, and equal scalars such as two independent throws of "s" have no identity with which to select the right trace. Please specify the trace-bearing representation/provenance that survives catch and rethrow while preserving the caught value's observable type/equality, and add an acceptance case for two equal scalar values thrown at different sites (including a nested catch).

2. The specified clojure.stacktrace output is not the 1.12.5 output and the differences are absent from Appendix A. The oracle prints ex-data on a new line, uses " at " only for the first frame and four spaces for later frames, and prints " at [empty stack trace]" for an empty trace. Sections 5.4 and 16.4 instead put ex-data on the header line, prefix every frame with " at ", omit the empty-stack line, and require two at lines at depth 2. Please either match the tagged implementation or explicitly document/test each intentional deviation. Oracle: https://github.com/clojure/clojure/blob/clojure-1.12.5/src/clj/clojure/stacktrace.clj#L40-L70

3. The *test-out* initialization order cannot produce the promised WASM behavior as written. Section 10.2 says bootstrap captures the root after an embedder installs its writer and cites pkg/rt/wasm/rendermain.go, but compiler/runtime package initialization (pkg/compiler/init.go) finishes before generated main, while that template installs HostWriter only inside main. No hook or generated-main update is specified to refresh *test-out* after *out*.SetRoot. The proposed capture therefore freezes the old writer and contradicts the Section 16.9 WASM acceptance case. Please define an explicit post-host-install operation (and require every root-writer replacement path to use it), or have the generated host setup update both roots atomically.

4. The report bridge protocol is not self-contained and can misattribute or deadlock valid runs. ReportEvent carries no snapshot/rendered form of *testing-vars*, *testing-contexts*, or trace output, yet fail_text/error_text read those dynamic vars and invoke let-go formatting on the harness goroutine, contradicting the stated rule that only the runner goroutine touches let-go state. In addition, composed deftests are explicitly supported, but consume_var treats nested BEGIN_TEST_VAR as an ELSE and returns on the nested END_TEST_VAR, so the rest of the outer test escapes its Go subtest. Finally, the goroutine pseudocode has no deferred close/recover/result send; an unexpected panic leaves consume_events blocked forever despite the behavior paragraph promising propagation. Please make the envelope self-contained on the runner goroutine, define nesting/depth handling, and define a deferred terminal result that closes the event stream and transports panics/errors to the harness goroutine.

5. print-tap-diagnostic preserves a trailing empty field that Java's oracle drops. Section 11.1 explicitly uses keep_trailing_empty = true; Clojure 1.12.5 calls Java String.split("\\n"), which discards trailing empty strings. For example, the oracle renders "a\\n" as only "# a\\n", whereas the specified algorithm adds a second "# \\n" line. This also affects multiline error text captured with with-out-str. Please use the oracle split semantics and add trailing-newline coverage. Oracle: https://github.com/clojure/clojure/blob/clojure-1.12.5/src/clj/clojure/test/tap.clj#L51-L57

Once these contracts are resolved, the rest of the document gives a strong implementation and acceptance-test roadmap.

[nnunley]

Updated and restacked in ed0edff4502e6679b1261c3c9583602e1edf8edc onto current main a6763e774fd05bc90ce2d1663fcfde7354046c1e.

All five review blockers are addressed:

1. Throw ownership: the spec now defines occurrence-owned ThrowOccurrence/EvalDatum transport, a context-preserving DatumFn invocation ABI, typed LoweredDatum<T> adapters, one-time ErrorToDatum normalization, datum-preserving collections/refs/natives/closures, and cause-datum storage in ExInfo. Acceptance covers non-comparable vectors, independent equal strings/integers, nested catches, rethrow, cause wrapping, and interpreter/lowered-Go specialized-path parity.
2. clojure.stacktrace: ex-data is on its own line; only the first frame has at; later frames use four spaces; empty traces and n <= 0 match the 1.12.5 oracle.
3. *test-out*: install-host-output-roots plus register-test-out handles either initialization order. WASM uses the paired root operation, while api.WithStdout binds both vars per execution context and temporary captures still bind only *out*.
4. Go bridge: runner-side freezing produces immutable Go-only envelopes; scope IDs recursively preserve composed deftests; t.Error/t.Log avoid format-string interpretation; and buffered terminal, panic recovery, cancellation, sequence validation, and drain-on-protocol-fault rules prevent stranded producers/deadlocks.
5. TAP splitting: the algorithm now specifies Java String.split("\\n") limit-zero behavior. Direct Java 26 and Clojure 1.12.5 probes confirmed that trailing empty fields are dropped, an interior empty field is retained, "\n" emits nothing, and "" emits "# \n". String.strip is unrelated and is not used.

The source/oracle audit also corrected canonical :test/*-fixtures keys, thrown-string trace rendering, Throwable->map root/omission/phase rules, NO_SOURCE_PATH, the explicit read-string metadata deviation, one-pass namespace discovery, the exact are root-cause class/message, and the Appendix B TAP exception transcript. Each now has a falsifiable Definition-of-Done case.

Verification on the exact pushed head:

• git diff --check: pass
• docs frontmatter for all three changed docs: pass
• internal spec contract/heading/link validator: pass
• make check-generated: pass (using a disposable in-workspace module cache because the sandbox-private external cache cannot be source-loaded by Go 1.26)
• independent adversarial source/oracle review performed; every reported finding was incorporated

@mparrett, please re-review when convenient.

[mparrett]

I reviewed the current restacked head (ed0edff4502e6679b1261c3c9583602e1edf8edc). The earlier five blockers appear substantially addressed, but I found three remaining correctness/design blockers:

1. [P1] Scalar is forms can fail during macroexpansion. Section 6.2 sends non-sequential forms to :default, but the normative expansion then evaluates function?(first(form)) without first checking sequential?. As written, (is true), (is false), (is :ok), and (is x) can error instead of going through assert-any. Clojure 1.12.5 uses (and (sequential? form) (function? (first form))). Please restore that guard and add scalar acceptance cases.

2. [P1] The trace design still lacks an execution-wide frame carrier. current-stack-trace and capture_live_frames(ec, err) require the caller's live frames, but DatumFn receives only ExecContext, whose current contract contains bindings and scope rather than a logical frame chain. Interpreter parent links exist only inside one VM run, native-to-Lisp callbacks start another run, and lowered Go has no VM Frame objects. At the same time, ThrowOccurrence is immutable and an existing occurrence is never normalized or captured again, so outer native/caller frames cannot be appended after a callback returns. That makes the cross-Go-boundary requirement in Section 5.2(4), and the parity cases in 16.4, unrealizable as specified. Please define an execution-wide logical-frame stack, including interpreter, native, callback, and generated-Go push/pop/finalization rules.

3. [P1] The mandatory no test namespace failure breaks the existing corpus. The current TestRunner walk includes 18 .lg files with no ns form: all 16 test/gold-aot/*.lg fixtures, test/top_level_do_test.lg, and test/in_ns_auto_refer_test.lg. The last intentionally uses only in-ns, which Section 16.11 explicitly says must leave saw_ns_form false. Implementing Sections 12.2 and 16.11 literally therefore makes go test ./test/... fail, contradicting the Definition of Done. Please specify whether these are excluded/load-only or enumerate safe namespace migrations for them.

CI is green and git diff --check passes, but I do not think the specification is implementation-safe until these contracts are resolved.

[mparrett]

Agent-drafted, posted by @mparrett. This continues the 2026-09-04 review on this PR (the five blockers). The 2026-09-08 review came from the same account through a different tool; this comment builds on it rather than restating it.

Round-1 blockers: resolved

I re-read the restacked head ed0edff4502e6679b1261c3c9583602e1edf8edc against the runtime and the Clojure 1.12.5 sources. All five are closed:

1. Trace ownership. Section 3.5 now names an owner, a ThrowOccurrence allocated per throw and carried by EvalDatum, instead of a table keyed by Value. Section 16.4 covers the cases the old design could not: non-comparable vectors, two equal scalars thrown at distinct sites, nested catch, rethrow, and cause retention.
2. clojure.stacktrace. Checked against the tagged source. Ex-data on its own line, at on the first frame only, four spaces on the rest, [empty stack trace], and take (dec n) all match.
3. *test-out* initialization. install-host-output-roots plus register-test-out is order-independent, and treating api.WithStdout as a per-execution binding matches what it already does.
4. Report bridge. Freezing on the runner goroutine, scope IDs for composed deftests, sequence validation, drain-on-fault, and a buffered terminal close the misattribution and stranded-producer paths. I traced the abandonment cases, and every early return drains first.
5. TAP splitting. Matches String.split at limit zero, and Section 11.2's print-diagnostics matches tap.clj exactly, including the :pass versus non-:pass branch.

Three further findings

All three are about Section 3.5, and they build on the frame-carrier finding in the 2026-09-08 review rather than competing with it. If the capture side gains a logical-frame stack, the transport side still needs these answered.

[P1] The datum carrier widens every typed lowered value, with no stated budget. Section 3.5 makes LoweredDatum<T> the representation of typed parameters, captures, locals, and results in lowered Go, and EvalDatum the representation of operand-stack slots and locals in the interpreter. This is not a reboxing claim: the arithmetic stays a native int64 or float64 operation on .value, with no interface header and no allocation. Two costs survive that. A typed local goes from one register to two, and a slot that was pointer-free becomes pointer-containing, so locals and typed slices the collector skips today join the scan set. Frame.stack is a separate widening, from 16 to 24 bytes on an already-pointerful slot. make bench-ratchet is a blocking pre-push hook at 5% anchor-normalized, so Slice 1 has to clear it, and Section 14 already prices a cheaper case when it declines construction-time capture to avoid a stack walk per ex-info. I have measured none of this, which is why the ask is a budget rather than a number: please state the accepted budget and add a Definition of Done case that runs the ratchet, or narrow the carrier to values reachable from a catch binding.

[P2] Datum-carrying collections change the exported pkg/vm surface, and no migration row covers Go embedders. Section 3.5 lists collection key, value, and element storage as datum storage, and Section 16.4 requires a caught vector to keep its occurrence through [caught] and first. Since EvalDatum never implements Value, vm.ArrayVector (declared []Value in pkg/vm/vector.go) has to become a slice of datums, along with the Value-typed slots in PersistentMap and vm.Var's root and binding stack, which pkg/api writes directly. ArrayVector alone appears in 80 Go files, and docs/guide/go-interop.md documents this surface. Section 13's migration table covers only Lisp-visible changes. This is a completeness gap rather than a defect, so it does not bear on the verdict: please say what the exported surface looks like after Slice 1 and add the embedder row.

[P3] The datum-preserving set is illustrative rather than closed. Section 3.5 introduces the DatumFn operations with "including" and routes everything else through the legacy []Value path, which drops provenance. Nothing states the rule that decides membership, so (first [caught]) rethrows with the original trace while (first (vec [caught])) allocates a fresh one, because vec is an ordinary native. Both are user-visible through ex-trace. The same boundary carries an unpriced cost: handing a datum-carrying collection to a legacy native means projecting it down to []Value per call. Please make the set normative and closed, or state the rule that determines membership.

The existing verdict stands. The corpus finding in the 2026-09-08 review is a real blocker, and none of the three above changes that.