fix: don't create duplicate languages from retired ISO 639-3 codes (BL-15916)

[andrew-polk]

Follow-up to #149. Found while auditing the QA test steps on the card: the langtags.json API 1.4 update introduced a user-visible bad language entry that no test step covered.

The bug

langtags.json sometimes keeps a code that ISO has since retired in an entry's iso639_3 field even though the entry's own tag already uses the surviving code — e.g. tag: "enm-Latn-IE" with iso639_3: "yol", or tag: "jkp-Zyyy-MM" with iso639_3: "kpp". Because the pipeline keys generated entries off iso639_3, we filed those under the retired code and minted a second, duplicate card for a language we already list under its current code.

Where the surviving code belongs to a different language, the duplicate also inherited the wrong name and subtag. That was the user-visible symptom: yol (Yola, a language of Ireland) was showing up in the language chooser as:

Middle English (1100-1500) — A language of Ireland

carrying the language subtag enm. Three consequences, all confirmed in the running react-mui demo:

• selecting it would have written the tag enm onto a collection;
• searching "Yola" found nothing at all;
• enm is on our deliberate historic-language exclusion list, but because the entry was keyed yol it slipped past the filter — so we were showing a historic language we intend to suppress.

The new iso-639-3.tab that came with the API 1.4 update retires yol, which is what triggered it.

The fix

In parseLangtagsJson: when an entry's iso639_3 code is no longer in iso-639-3.tab but the tag's language subtag is, trust the tag. Entries whose retired code matches their own tag (33 of them, e.g. aoh/Arma) are deliberately left alone — there is no surviving code to fold them into, and they behave exactly as before.

Regenerating languageData.json removes exactly five duplicates, folding their regions, scripts and tags into the surviving entries and changing nothing else across the other 7,953:

Removed duplicate	Folded into
yol "Middle English (1100-1500)" [enm]	enm
kpp Paku Karen [jkp]	jkp
jeg Oy [oyb]	oyb
dek Suma [sqm]	sqm
tpw Tupinambá [tpn]	tpn

Retired codes stay searchable via alternativeTags, so kpp, jeg, dek, tpw (and skk, thx) still find the right language.

Yola is no longer offered at all. The current iso-639-3.tab has retired yol, and no source we have still carries the name "Yola", so there is no correct entry we can build for it. Suppressing it beats offering it under another language's name.

Verification

• find-language 191 tests, language-chooser-controller 97, language-chooser-react-hook 16, language-chooser-react-mui e2e 114 — all pass.
• tsc typecheck clean; eslint 0 errors.
• Two new tests in languageSearch.spec.ts: one covering all five retired/surviving pairs (no duplicate entry, retired code still folded in and still searchable), one guarding the specific enm/yol symptom.
• Checked by hand in the running react-mui demo that "Middle English" no longer offers the Ireland card and that "Paku Karen" returns a single card.

Also on the card, not in this PR

Auditing the test steps turned up three that were simply wrong (they asked testers to confirm country lists on the English and French cards, which are stripped on purpose; and to search for a "Kurdish Sign Language" that does not exist in the data). Those have been corrected on BL-15916, and the newly-added dyl Bhutanese Sign Language entry reading "A language of Turkey" has been reported upstream — that one is upstream source data, not ours.

Ref: https://issues.bloomlibrary.org/youtrack/issue/BL-15916

🤖 Generated with Claude Code

---

This change is 

---

Devin review

[andrew-polk]

[Claude Opus 5 from Andrew Polk's machine during preflight] Consulted Devin on 2026-07-29 up to commit 6b43348.

Clean: no Bugs and no Investigate flags, so nothing was mirrored as a review thread. Four Informational items, and it is worth saying what they were rather than just counting them:

• Two were confirmations rather than concerns — that the retired-code remap only fires on codes genuinely missing from iso-639-3.tab (leaving the 33 aoh-style entries alone), and that macrolanguage codes are correctly excluded from it.
• One noted that when a retired-code entry folds into its surviving code, the merged entry keeps whichever name langtags happened to list first. That cannot affect any of the five current pairs — both entries in each pair carry the same name — but it is a latent version of the very bug this PR fixes, so it is now written down as a comment in langtagProcessing.ts telling whoever regenerates the data next to re-check the folded entries' names.
• One noted that the reworked e2e scroll helper deliberately keeps sweeping until its timeout, so using it to assert a card is absent would waste the full 15 seconds. Both current callers assert presence; that constraint is now a comment in e2eHelpers.ts.

Both of those last two are recorded in commit 6b43348, which is comments only — regenerating languageData.json afterwards produces a byte-identical file.

CI (run-tests) passed for this commit. This repo has no comment-posting review bots configured (no Greptile/CodeRabbit), so Devin and CI are the full remote panel.