[parser] skills — a "Language: Fluent in Spanish" proficiency row is admitted as a skill token

[s-annam]

Context — what a knownWrong entry is

Each corpus fixture under tests/fixtures/pdfs/ may carry a hand-authored ground-truth sidecar (*.truth.json, minted under #654). Its knownWrong block records, per field, a place where the parser disagrees with what the page actually draws, with a status of open (live bug, issue must be open), accepted (written-down tradeoff), or unfiled (measured, never filed — issue: null).

unfiled is capped at UNFILED_TRUTH_CEILING = 10 (src/lib/heuristics/corpus.test.ts:186) and is saturated at 10/10 today. This issue files half of one such entry — see the note about the shared field at the bottom.

What's wrong

A spoken-language proficiency statement is admitted into skills as if it were a skill token.

tests/fixtures/pdfs/google-docs/google-docs-skia-proxy-role-first-experience.pdf draws:

Skills
Programming Languages: C, HTML, CSS, Java, VHDL, Assembly
Applications: Eclipse, Xilinx, ModelSim, PSpice, Microsoft Word/Excel/PowerPoint
Language: Fluent in Spanish

Observed: Fluent in Spanish appears in the skills array and as a Language category.

That fixture's truth sidecar excludes it deliberately, and says why in its provenance: "the 'Language: Fluent in Spanish' row is a language proficiency, not a skill token, and is deliberately excluded."

This is a precision defect, and precision is where the skills list is load-bearing beyond display: skills feeds the JD-match keyword surface (src/lib/jd-match/) and the job-search query builder's resume-derived keyword string (src/lib/job-search/providers/keywords.ts). A sentence-shaped pseudo-skill pollutes both.

Root cause

The Skills-section extractor has exactly one label denylist, src/lib/heuristics/extract/skills.ts:279-280:

const NON_SKILL_SUBLABEL_RE =
  /^(?:personal\s+|other\s+)?(?:interests?|hobbies|hobby|activities|pastimes)\s*$/i;

consulted by matchCellLabel (skills.ts:610-614), which returns undefined for a denied label so tokenizeCell drops the whole cell.

Language / Languages is not on that list — and must not be added to it. The docblock immediately above states the opposite intent, correctly: "a real skill sub-label like Languages: / Frameworks: never matches." On the overwhelming majority of engineering résumés Languages: heads the programming-language row, and denying the label would delete the single most valuable row on the page. The multi-degree-coursework.pdf fixture in this same corpus proves the collision: its Languages row is programming languages.

So the label is genuinely ambiguous and the discriminator has to be the body, not the label:

• Languages: Python, Go, TypeScript — a delimited token list. Skills.
• Language: Fluent in Spanish — a proficiency predication about one language. Not a skill token.

Nothing in the extractor looks at body shape today. tokenizeCell (skills.ts:306) splits on the delimiters and hands every fragment to isSkillToken (skills.ts:141), whose filters are all token-local — length, digits-only, contact-link, date-range, and a >6-word cap. Fluent in Spanish is three words, so it clears the word cap and is admitted.

Implementation plan

1. src/lib/heuristics/extract/skills.ts — add a proficiency-row predicate. Fires only on a cell whose captured label is language-ish AND whose body reads as a proficiency predication rather than a delimited list. Both conjuncts are load-bearing: the label alone would kill programming-language rows, and the body pattern alone would reject a legitimate skill that happens to contain in (Certified in AWS).

   /** A Skills sub-label that MAY head a spoken-language row. Deliberately NOT
    *  added to NON_SKILL_SUBLABEL_RE: on most engineering résumés `Languages:`
    *  heads the PROGRAMMING-language row, which is the single highest-value row
    *  on the page. The label is ambiguous, so it only gates — the BODY decides.
    *  See LANGUAGE_PROFICIENCY_BODY_RE. */
   const LANGUAGE_LABEL_RE = /^languages?$/i;
   
   /** A spoken-language PROFICIENCY predication — "Fluent in Spanish",
    *  "Native speaker", "Conversational French", "Spanish (C1)". Distinguished
    *  from a token LIST by the proficiency word, never by the label. */
   const LANGUAGE_PROFICIENCY_BODY_RE =
     /\b(fluent|native|bilingual|conversational|proficient|intermediate|beginner|basic|working\s+proficiency|mother\s+tongue)\b/i;

2. Gate on both, in matchCellLabel's neighbourhood. matchCellLabel currently sees only the label; it needs the body too, or the decision moves to the single call site at skills.ts:647. Prefer keeping the decision in one predicate — isLanguageProficiencyCell(cell) — and calling it where NON_SKILL_SUBLABEL_RE is consulted, so a denied cell is dropped by exactly the same path (whole cell, no tokens, no category). Do not special-case it later in extractSkills; two drop paths for the same concept is the divergence this file already fights (see matchCellLabel's docblock on keeping the two decisions aligned).

3. Do not silently discard the information. Dropping the row entirely is correct for skills — but confirm nothing else in the canonical model wants spoken languages. If there is no home for them today (there is not, as of this writing), the drop is the right answer and a languages field is out of scope for this issue; do not add one here.

4. Flip the ground-truth entry. tests/fixtures/pdfs/google-docs/google-docs-skia-proxy-role-first-experience.truth.json → skills.

   ⚠️ That entry records TWO independent disagreements on one field: this one, and a separate C-token drop caused by isSkillToken's 2-character floor (filed as its own issue). The knownWrong schema is keyed per FIELD, so the entry can carry only one issue: number. Whichever issue lands first takes the citation; the note must keep naming both. The entry only clears once BOTH are fixed — the truth scoreboard compares the whole skills multiset.

5. UNFILED_TRUTH_CEILING (src/lib/heuristics/corpus.test.ts:186) is not lowered by this issue on its own — the shared entry only stops being unfiled once it carries an issue number, and the sibling C-drop issue already accounts for it. Do not double-count.

Acceptance criteria

• google-docs-skia-proxy-role-first-experience.pdf parses skills with no Fluent in Spanish entry and no Language category.
• Languages: Python, Go, TypeScript still parses to three skills under a Languages category. Unit test.
• Languages: Spanish, French, Mandarin — a bare spoken-language LIST with no proficiency word — is still admitted. This is a deliberate, stated limit of the body-shape discriminator: a list without a proficiency word is indistinguishable from a programming-language list by shape alone, and guessing from the language names themselves would need a gazetteer this issue does not add. Assert the current behaviour so the limit is visible rather than accidental.
• Certified in AWS Solutions Architecture under a non-language label is unaffected — the label conjunct keeps the body pattern from firing.
• multi-degree-coursework.pdf's Languages row (programming languages) is unchanged.
• npx vitest run src/lib/heuristics/corpus.test.ts passes; skills precision improves on the target fixture and falls on none.
• npx vitest run src/lib/heuristics/corpus-roundtrip.test.ts passes with no new KNOWN_FAILURES baseline rows.
• npm run verify passes.

[s-annam]

The sibling issue sharing this fixture's skills truth entry is #832 (the single-letter C token dropped by isSkillToken's 2-character floor). The knownWrong schema is keyed per field, so google-docs-skia-proxy-role-first-experience.truth.json → skills can carry only one issue: number — whichever lands first takes it, and the note must keep naming both. The entry only clears once both are fixed.