A rewrite mangled slice types in 63 spec files, and it is the floor under every field-set measurement

[gHashTag]

While giving reason-codes to the field sets that moved under #2151, sixteen of the eighteen turned out to have nothing to do with the parser change: the source lines they parse are corrupt. This is a corpus defect, and it is bigger than the change that exposed it.

The count

63 spec files, 115 field lines of the form : [[]…". All under specs/tri/, densest in tri/collections (15 files), then tri/trees, tri/graph, tri/encoding (7 each).

specs/tri/crypto/crypto.t27:13:

pub const KeyPair = struct {
    public_key : [[]U8",
    private_key : [[]U8",
};

specs/tri/utils/logger.t27:17:

pub const LogEntry = struct {
    timestamp : "Instant",
    level : "Level",
    message : [[]Const u8",
};

An unbalanced [[, a ] with no opener to match, and a closing quote with no opening quote. 93 of the 115 lines carry exactly one " character — the quote is unpaired on the overwhelming majority.

The shapes, which say what happened

shape	count
[[]X",	52
[[]X X",	37
[[]X [,	10
[[][,	7
[[]X" = "[2, 2]";	2
other	7

[[]Const u8" is []const u8 with [] doubled, const title-cased, and one quote left behind. [[]Const [, is the same accident applied to a nested slice, truncated mid-type. This is the residue of an automated rewrite that ran over the corpus and mangled slice types — it is not something anyone typed. The 121 files that use quoted type texts (: "Ident") are probably from the same generation and should be checked in the same pass, but they at least have balanced quotes.

Why it matters beyond tidiness

Any measurement over these files measures the parser's recovery from corrupt input, not the parser. Concretely: the "14 changed field sets" on #2151 look like a behavioural change in the parser, and 16 of 18 are not — the old collector truncated the type on such a line, the new one absorbs the following name : type pair into the type text, and neither is correct because the input has no correct reading. That is why those files could only be labelled accepted-malformed-tradeoff rather than improved. As long as these lines are in the tree, every field-set differential over specs/tri/ has a floor of noise it cannot get under.

This also puts a caveat on the corpus census in #2152. That census classifies 590 of 618 files as SOURCE and attributes 168 blocked specs to a real defect population. At least 63 SOURCE files contain lines that no reading of the grammar accepts, so some part of the 144 NOPARSE — and possibly some of the 168 — is corrupt input rather than a compiler defect. The census numbers are not wrong, but the NOPARSE bucket needs splitting by input validity before any of it is attributed to the parser.

What is not being claimed

I have not repaired anything. A mechanical repair is temptingly easy ([[]Const u8" → []const u8), and that is exactly why it should not be done in the same change as anything else: it moves 115 lines in 63 files, it will move conformance output, and it needs to be a reviewable diff of its own. The 18 [[]… lines with zero quotes and the 4 with three quotes are not the same shape and must not be swept into one regex.

Nor is it established that a repair makes any of the 168 specs pass. That is the point of separating the two: repair first, re-measure after, and do not attribute the delta before it is observed.

Bounded next steps

1. Enumerate all 115 lines with file, line and shape; commit the inventory before touching a byte.
2. Repair only the 93 single-quote lines, by shape, one shape per commit.
3. Handle the 18 zero-quote and 4 three-quote lines by hand.
4. Re-run the field-set differential over the corpus and re-classify the parser: nesting decides the separator, never the terminator (#2127) #2151 tradeoff files — the expectation is that most reduce to same.
5. Only then re-run the census and split NOPARSE into corrupt-input and genuine parse defect.

Refs #2151, refs #2152.