Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 38 additions & 5 deletions .github/workflows/schema-validation.yml
Original file line number Diff line number Diff line change
@@ -1,15 +1,48 @@
name: Validate JSON Schemas
name: Schema Validation

# This workflow is named a required check in docs/BRANCH-PROTECTION.md. Its entire
# body used to be:
#
# echo "Validating JSON schemas..."
#
# A required check that cannot fail reads as coverage and is worse than none —
# it is the shape of gate this repository has now found four times in one day.
#
# It now asks the weakest question worth asking: does every tracked JSON file
# parse. That question is cheap, carries no theory that could itself be wrong,
# and it immediately found a file a test actually loads:
# clara-bridge/audit-trail/experience-schema.json had a literal `...` on line 40,
# and clara-bridge/tests/run_tests.py:152 does json.load() on it — 3 of its 11
# tests were failing, and no workflow runs clara-bridge.
#
# Known-unparseable files live in tools/json_parse_baseline.txt as debts, one per
# line. Remove a line when the file is fixed and the gate then holds it fixed.

on:
pull_request:
branches: [master]
push:
branches: [master]
workflow_dispatch:

jobs:
# The job id is the status-check CONTEXT that branch protection matches on.
# It was `validate` and must stay `validate`: renaming it makes the required
# context silently stop reporting, and the PR is BLOCKED with every visible
# check green. Found the hard way on the PR that introduced this file.
validate:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"

# A gate nobody has seen fail is not a gate. This plants a malformed file
# and an empty one and proves the scan reports both while staying silent
# on a valid file.
- name: Negative control
run: python3 tools/check_json_parses.py --self-check

- name: Validate schemas
run: |
echo "Validating JSON schemas..."
- name: Every tracked JSON parses
run: python3 tools/check_json_parses.py
2 changes: 1 addition & 1 deletion clara-bridge/audit-trail/experience-schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@
"episode_id": "reference to original episode",
"verdict": "toxic (only for mistakes)",
"error_type": "regression | invariant_violation",
"blocked_modules": ["nn/attention", "nn/hslm", ...],
"blocked_modules": ["nn/attention", "nn/hslm"],
"explanation": "Changed phi constant broke trinity invariant in downstream",
"blocked_until": "episode_id:verification-resolution",
"quarantine_timestamp": "ISO-8601 datetime"
Expand Down
12 changes: 12 additions & 0 deletions docs/NOW.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,15 @@
# NOW -- two required checks were a single echo (2026-08-18)

Last updated: 2026-08-18

## ci: give schema-validation a body, and say what seal-coverage should assert -- I do not know (Closes #2191)

- **`seal-coverage.yml` and `schema-validation.yml` are each one `echo`**, and both are named required in `docs/BRANCH-PROTECTION.md`. `phi-loop-ci.yml`, described there as the main test suite, asserts `abs(phi**2 + phi**-2 - 3) < 1e-10` -- true of an empty repository -- alongside one real grep lint. A required check that cannot fail reads as coverage and is worse than none
- **`schema-validation` now asks the weakest question worth asking:** does every tracked JSON parse. Cheap, and carrying no theory that could itself be wrong
- **It found a broken file that a test actually loads.** `clara-bridge/audit-trail/experience-schema.json` had a literal `...` on line 40; `clara-bridge/tests/run_tests.py:152` does `json.load()` on it. **3 of its 11 tests were failing** -- measured by reverting the fix and re-running, not assumed -- and no workflow runs `clara-bridge` at all. Fixed; the suite is 11/11
- Six empty JSON artefacts recorded as debts in `tools/json_parse_baseline.txt`. `external/` excluded: tsconfig is JSONC by convention, and flagging it would be this gate making the mistake it exists to catch
- **`seal-coverage` deliberately untouched.** `.trinity/seals/` holds 1714 files keyed on TYPE names (`Account`, `AXI4_Testbench`, `"[]const u8"`), not spec names. My first attempt matched seal filenames against spec filenames and reported "1668 orphans of 1714" -- a finding about my assumption, not the repository. `t27c` has no `seal` subcommand. Wrote neither a check nor a deletion

# NOW -- finish the diagnostic repair; a crash is not a numeric mismatch (2026-08-18)

Last updated: 2026-08-18
Expand Down
127 changes: 127 additions & 0 deletions tools/check_json_parses.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
"""Does every JSON file this repository ships actually parse?

`schema-validation.yml` was named a required check in docs/BRANCH-PROTECTION.md
and its entire body was:

echo "Validating JSON schemas..."

A required check that cannot fail reads as coverage and is worse than none. This
replaces it with the weakest question worth asking -- does the file parse at all
-- because that one is cheap, has no false theory behind it, and it already finds
something the echo was hiding:

clara-bridge/audit-trail/experience-schema.json contains a literal `...` on
line 40 and cannot parse. clara-bridge/tests/run_tests.py:152 does
`json.load()` on exactly that path, so load_experience_schema() raises every
time it is called -- and no workflow runs clara-bridge, so nothing said so.

Empty files are reported separately from malformed ones: an empty artefact is
usually a build that died, not a syntax error, and the two want different fixes.

`external/` is excluded. It is vendored, and TypeScript's tsconfig.json is JSONC
by convention -- comments there are correct, and flagging them would be this
gate making the same mistake it exists to catch.

Usage:
tools/check_json_parses.py gate
tools/check_json_parses.py --self-check negative control
tools/check_json_parses.py --update-baseline

Exits non-zero on any new unparseable file.
"""
import json
import os
import pathlib
import subprocess
import sys

ROOT = pathlib.Path(__file__).resolve().parent.parent
BASELINE = ROOT / "tools/json_parse_baseline.txt"
EXCLUDE = ("external/",)


def tracked_json(root=ROOT):
try:
out = subprocess.run(["git", "ls-files", "-z", "*.json"], cwd=root,
capture_output=True, check=True).stdout
names = [f.decode() for f in out.split(b"\0") if f]
except Exception:
names = [str(p.relative_to(root)) for p in root.rglob("*.json")]
return [n for n in names if not any(n.startswith(x) for x in EXCLUDE)]


def scan(root=ROOT):
empty, bad = [], []
for rel in tracked_json(root):
p = root / rel
try:
raw = p.read_bytes()
except OSError as e:
bad.append((rel, f"unreadable: {e}"))
continue
if not raw.strip():
empty.append(rel)
continue
try:
json.loads(raw.decode("utf-8", "replace"))
except Exception as e:
bad.append((rel, str(e)[:90]))
return empty, bad


def baseline():
if not BASELINE.exists():
return set()
return {l.split("|")[0].strip() for l in BASELINE.read_text().splitlines()
if l.strip() and not l.startswith("#")}


def self_check():
"""Plant a malformed file and prove the scan reports it."""
import tempfile
with tempfile.TemporaryDirectory() as td:
t = pathlib.Path(td)
(t / "good.json").write_text('{"a": 1}')
(t / "bad.json").write_text('{"a": 1,,}')
(t / "empty.json").write_text("")
empty, bad = scan(t)
ok = ([b[0] for b in bad] == ["bad.json"]) and (empty == ["empty.json"])
print(f" self-check: malformed caught = {[b[0] for b in bad] == ['bad.json']}, "
f"empty caught = {empty == ['empty.json']}, good file silent = {len(bad) + len(empty) == 2}")
return 0 if ok else 1


def main():
if "--self-check" in sys.argv:
return self_check()
empty, bad = scan()
total = len(tracked_json())

if "--update-baseline" in sys.argv:
BASELINE.write_text(
"# JSON files that do not parse today. Each line is a debt, not a rule.\n"
"# Remove the line when the file is fixed; the gate then holds it fixed.\n"
+ "".join(f"{rel} | {why}\n" for rel, why in sorted(bad))
+ "".join(f"{rel} | empty file\n" for rel in sorted(empty)))
print(f" baseline written: {len(bad) + len(empty)} entries")
return 0

known = baseline()
new_bad = [(r, w) for r, w in bad if r not in known]
new_empty = [r for r in empty if r not in known]
if not new_bad and not new_empty:
print(f"OK: {total} tracked JSON files, none newly unparseable "
f"({len(bad) + len(empty)} known, listed in {BASELINE.name})")
return 0
print(f"FAIL: {len(new_bad) + len(new_empty)} JSON file(s) do not parse\n")
for rel, why in new_bad:
print(f" {rel}\n {why}")
for rel in new_empty:
print(f" {rel}\n empty file — a build that died, or a placeholder never filled")
print("\n Fix the file. If it is a documentation example that only looks like JSON,")
print(" rename it (.jsonc, .md) rather than adding it to the baseline.")
return 1


if __name__ == "__main__":
sys.exit(main())
8 changes: 8 additions & 0 deletions tools/json_parse_baseline.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# JSON files that do not parse today. Each line is a debt, not a rule.
# Remove the line when the file is fixed; the gate then holds it fixed.
.trinity/icarus-baselines/specs/scratch/w743_bench_module_305x2p6_aos_var_call_write.json | empty file
.trinity/icarus-baselines/specs/scratch/w744_bench_module_307x2p6_aos_var_call_write.json | empty file
.trinity/icarus-baselines/specs/scratch/w745_bench_module_309x2p6_aos_var_call_write.json | empty file
.trinity/icarus-baselines/specs/scratch/w746_bench_module_311x2p6_aos_var_call_write.json | empty file
.trinity/icarus-baselines/specs/scratch/w747_bench_module_313x2p6_aos_var_call_write.json | empty file
research/toda_derivation.json | empty file
Loading