[i18n] Polish-language support: voice dictation, deterministic NLU, and a UI language switch

[grzanka]

Feature request

• A language switch in the UI — a flag icon/button (🇬🇧 ⇄ 🇵🇱) — that puts the app into Polish mode: typed
  queries and mic dictation are interpreted as Polish, and the answer comes back in Polish.
• Several Polish-speaking colleagues are willing to record voice samples for a Polish eval/training set,
  the same way the existing English speakers did (eval/RECORDING.md, eval/audio/{km,lg,mn}/).
• An LLM could plausibly help draft the Polish sentences needed for such a set, rather than hand-writing
  all of them.
• Underlying question: what accuracy can we actually expect from the models on Polish, and what has
  to change in the app to support it end-to-end?

This issue is a feasibility investigation answering that question, not an implementation — it audits every
pipeline stage this repo already has (ASR worker, deterministic NLU matcher, alias tables, NLG templates,
eval harness, model hosting) with file:line citations, adds external research on Whisper/LLM/TTS Polish
performance, and proposes a scoped path forward. Per this repo's own convention for research/spike issues,
whoever picks this up should land measured findings in a new docs/polish-language-feasibility.md before
closing it — this issue is the proposal, not the final report.

Why this is a natural next step, not a stretch

• docs/model-hosting-cyfronet.md:42-43 already justifies the Cyfronet S3 mirror partly on "Cyfronet is a
  Polish/EU host... for the project's actual user base" — the intended users already skew Polish.
• The whisper-small checkpoint already mirrored in production (onnx-community/whisper-small, per
  docs/model-hosting-cyfronet.md:9-13) is OpenAI's standard multilingual checkpoint (not the
  .en-suffixed English-only variant) — it already covers ~99 languages, including Polish. No new model
  needs downloading or mirroring just to attempt Polish ASR.
• Piper — already the planned TTS engine per docs/design.md ("NLG + TTS: templated sentence →
  SpeechSynthesis / Piper") — has real, working Polish voice models today.

None of this makes Polish support free — the NLU and NLG layers are the real cost, detailed below — but
the ASR and TTS legs are cheaper than they might look at first glance.

What performance can we actually expect?

Short answer: generic Polish speech recognition is in a good tier; domain accuracy (the number that
actually matters here) is presently unmeasured and cannot be estimated from generic numbers alone.

• ASR, generic speech. Third-party aggregated reporting on Whisper large-v3 (Common Voice 15) puts
  Polish in the "high accuracy" tier (WER < 10%), in the same band as German/Portuguese/Dutch, well above
  genuinely low-resource languages (Arabic/Hindi/Russian/Turkish ≈ 10–20%, and 25%+ for the long tail). I
  could not independently verify this against OpenAI's own primary per-language table — web fetches to
  primary sources (arxiv.org, the Whisper GitHub repo, Wikipedia) all returned 403 in this research
  session — so treat the <10% figure as directionally indicative, not authoritative, until someone confirms
  it against the actual paper. Separately, Mozilla Common Voice's Polish corpus (cv-corpus-25.0,
  2026-03) has 188.84h recorded / 176.61h validated from 3,465 speakers — a mid/well-resourced language for
  ASR purposes, not a low-resource one.
• ASR, domain jargon — the actual open question. This repo's own numbers show generic WER is a poor
  proxy for domain accuracy: English whisper-small needed domain-prompt biasing (+7.6pp raw slot-token
  accuracy, docs/voice-pipeline-feasibility.md §2.4) and an extended corrector (+9pp more) to go from a
  mediocre raw transcript to 89% E2E audio→intent on physics vocabulary (keV/MeV/GeV, per-nucleon, dE/dx,
  ASTAR/PSTAR, isotope names, material trade names like Lucite/PMMA). There's no reason to expect Polish
  physics vocabulary ("zdolność/moc hamowania" for stopping power, "zasięg" for range, element names,
  material names) to be any different — expect a similarly large gap between generic Polish WER and domain
  E2E accuracy until a Polish domain-prompt string, a Polish corrector, and a Polish eval set exist. This
  is exactly the gap the offered recordings would let us measure and close — nobody can quote a
  trustworthy domain-accuracy number today because it doesn't exist yet; the honest answer is "unknown but
  cheaply measurable," not "X%."
• On-device LLM fallback (if this project keeps one at all — see docs/voice-pipeline-feasibility.md
  §5.6, currently scoped as a rare last resort, English-only, unshipped). The models already benchmarked
  here (Qwen2.5-0.5B/1.5B, §2.5) claim "multilingual support across 29+ languages," but I found no verified
  Polish-specific benchmark for them. A better-fitting candidate: Bielik (SpeakLeash + ACK Cyfronet
  AGH — trained on the same Cyfronet HPC infrastructure this repo already has model-mirroring access to),
  a Polish-native-tuned open LLM family at 1.5B/4.5B/7B/11B. Bielik-11B-v2 scores 58.14 avg on the Open PL
  LLM Leaderboard (best under 20B params, +8.75pp over 2nd place); Bielik-11B-v3 scores 65.93 on the same
  leaderboard and 71.83% on the Polish Linguistic & Cultural Competency Benchmark. The 1.5B/4.5B variants
  are reported competitive with models 2–3× their size, but — same caveat as Qwen — nobody has run this
  repo's actual constrained-single-token-classification task against them. Needs its own small bench, not
  an assumption; low priority given the fallback itself is unshipped for English.
• TTS. Piper has real Polish voice models today (e.g. pl_PL-darkman-medium,
  community-trained pl_PL-jarvis_wg_glos-medium / pl_PL-justyna_wg_glos-medium /
  pl_PL-meski_wg_glos-medium / pl_PL-zenski_wg_glos-medium — HF WitoldG/polish_piper_models, also
  mirrored e.g. as csukuangfj/vits-piper-pl_PL-zenski_wg_glos-medium). Browser SpeechSynthesis also
  works across Chrome/Edge on Windows/macOS, but whether a Polish voice is actually installed depends on
  OS/browser state — unverified with a specific voice name here, needs a real-device check before relying
  on it. Piper is the lower-risk choice since its Polish weights are known and shippable exactly like the
  Whisper weights already are.
• LLM-generated Polish sentences (for building the eval set). High confidence this works well for raw
  fluency — Polish is a mid/high-resource language for frontier LLMs, not a low-resource one where
  synthetic generation quality degrades. The real risk isn't fluency, it's domain naturalness (would a
  Polish medical physicist actually phrase a query that way?) — that needs native-speaker review, i.e.
  exactly the friends who offered to help. Treat LLM output as a first draft, not ground truth.

What exists today, and exactly where English is hardcoded

Stage	File(s)	Finding
ASR language token	src/lib/asr/transcribe.ts:229	`const languageEnglish = Number(langToId["<
ASR domain prompt	src/lib/asr/transcribe.ts:75-78	DOMAIN_PROMPT — the English biasing string ("MeV, keV, GeV, ..., stopping power, Lucite, adipose tissue"). Needs a Polish counterpart; units (MeV/keV) are language-invariant, words like "stopping power" aren't.
ASR model	src/lib/models/manifest.ts:34-41	One whisper entry, onnx-community/whisper-small q8 — already the standard multilingual checkpoint. No per-language variant concept in ModelManifestEntry, and none needed for a first pass.
NLU matcher	src/lib/intent/matcher.ts (705 lines, all of it)	Entirely inline English regex/Set literals, not data-driven — e.g. INDIRECT_IDIOMS (73-102, /\bhow far\b/ → csdaRange), DIRECT_STOPPING/DIRECT_RANGE (104-106), detectInverse() (109-124, `/\b(?:what
QueryIntent schema	src/lib/intent/query-intent.ts, src/lib/intent/coverage.ts	No lang/locale field anywhere in QueryIntent, EvalExample, EVAL_TAGS, or the coverage types. Language would need to be threaded in as new state, not slotted into an existing field.
Alias tables	src/lib/aliases/materials.ts, particles.ts, elements.ts	Flat alias → id maps (MATERIAL_ALIAS_INDEX, PARTICLE_ALIAS_INDEX). Adding Polish aliases (["węgiel", 6], ["woda", 276], a Polish protony particle entry, …) fits the existing shape with no structural change to the lookup mechanism itself. Fun accident: elements.ts:163 already has ["wolfram", 74] as an English variant name for tungsten — that's also the standard Polish word for tungsten, a free hit. Caveat: normalize.ts's NFKD stripping (61-70) folds Polish ć/ń/ó/ś/ą/ę to their base letters fine, but "ł" (U+0142) does not NFKD-decompose and survives as a distinct letter — an unaccented "l" typed by a user won't match a stored "ł" alias without an explicit extra variant. Recommend a locale-scoped index (reusing the same lookup.ts fuzzy-match machinery) rather than merging Polish aliases into the same flat map as English/trade names, to avoid cross-language collisions.
NLG templates	src/lib/nlg/render.ts (216 lines, all of it)	Hardcoded English template literals throughout, e.g. `The ${QUANTITY_PHRASE[quantity]} of ${energyLabel(...)} ${particle} in ${material} is ${value} (...)`. — English word order and prepositions baked in; no lang parameter on renderAnswer(). The harder problem isn't translation, it's Polish grammatical case: "in water" is "w wodzie" (locative), not "w woda" — a naive string-table swap doesn't handle noun declension. Recommend a v1 that sidesteps this with a labeled/colon-style Polish output ("Zdolność hamowania: 12.3 MeV·cm²/g (materiał: woda, cząstka: proton, energia: 100 MeV)") instead of attempting fully natural declined sentences, deferring a real declension table to a stretch goal.
UI strings / i18n scaffolding	src/routes/*.svelte (3 files), src/lib/components/**/*.svelte (10 files), plus answer-status.svelte.ts, asr-status.svelte.ts, model-status.svelte.ts	No i18n library anywhere (package.json has exactly one runtime dependency, @huggingface/transformers; no svelte-i18n/paraglide/i18next/etc.). No centralized strings module — every .svelte file inlines its own English text. Roughly 45–55 distinct user-facing strings across all of it (button labels, status messages, the "Sorry, I couldn't understand that..." fallback, mic states "Warming up…"/"Listening…", download/cache dialogs). src/app.html:2 hardcodes <html lang="en"> — a real fix needed (a11y-relevant), not a false positive. Two earlier locale/translate grep hits were confirmed false positives: DarkModeToggle.svelte:35-36 (translate-x-* Tailwind transform classes) and coverage.ts:220 (localeCompare for sorting tag strings).
Eval set	eval/intents.jsonl, scripts/validate-intents.ts	No lang field in the EvalExample schema, so the format itself is language-neutral — but validate-intents.ts:17 hardcodes the file path (../eval/intents.jsonl) and line 21 hardcodes MIN_EXAMPLES = 100. A Polish set needs a genuinely separate frozen file (eval/intents.pl.jsonl, matching CLAUDE.md's rule that the existing intents.jsonl is a frozen regression suite not to be touched) plus a parameterized validator or a second script with its own (lower, initially) minimum-count threshold. EVAL_TAGS categories (phrasing/quantity/comparison/unit/ambiguity/special) are language-agnostic and should be reusable as-is; per CLAUDE.md, LLM-drafted/hard Polish examples must be tagged "adversarial", never "stress-test" (reserved for the two existing §7 sentences).
Model hosting	docs/model-hosting-cyfronet.md:9-13,25-29, src/lib/models/remote.ts:12	Only whisper (the multilingual checkpoint already discussed) is mirrored; qwen/llama aren't yet. Because the mirrored Whisper weights already cover Polish, basic Polish ASR needs zero new model mirroring — the change is code-level (parametrize the `"<

The one strategic fork worth deciding early

Option A — native Polish pipeline. Polish ASR (task: "transcribe", <|pl|> token) → a Polish
variant of the matcher → Polish-aware alias lookups → Polish NLG. Full fidelity: the user sees their own
Polish question echoed back for confirmation (this repo's whole "trust UX" design depends on that echo,
per docs/voice-pipeline-feasibility.md §4/§5.4). Highest cost: forking/parametrizing all 705 lines of
matcher.ts, plus the NLG declension problem above.

Option B — Whisper's built-in translate task as a shortcut. Whisper's decoder supports a task
token alongside the language token — "transcribe" (current: Polish audio → Polish text) vs.
"translate" (Polish audio → English text, directly, in one decode pass). If Polish queries are run
through task: "translate" instead, the output could flow through the existing, unmodified English
matcher/aliases/NLG entirely — sidestepping the expensive matcher fork completely. Real tradeoffs, all
unmeasured: (1) translation is a strictly harder task than transcription, so domain-jargon accuracy on
this path is unknown and needs its own benchmark pass, same as the transcribe-task English work needed;
(2) the "echoed" query shown to the user would be in English even though they spoke Polish — plausibly
fine for bilingual physicist users (this project's actual audience), but a real trust/UX regression worth
a deliberate decision, not an accident; (3) it only solves dictation, not the flag-toggle's promise of a
fully Polish-language app — NLG/UI still need Polish localization regardless.

Suggested pragmatic path: a hybrid — use Option B to shortcut the hard NLU parsing step for a v1 (Polish
speech in → Whisper translate → existing English matcher/compute pipeline), while still localizing NLG
output, TTS voice, and UI chrome to Polish (Polish in, Polish out; English only in the invisible middle).
Defer the full Option A matcher fork unless the translate-task's domain accuracy or the trust-UX regression
turns out to be unacceptable once measured. This isn't free either — it needs its own accuracy benchmark —
but it avoids committing to the single largest cost item (the matcher fork) before knowing if it's
necessary.

Data collection plan (ties the two feature-request ideas together)

1. Draft ~30 Polish sentences covering the same coverage matrix eval/RECORDING.md used for English
   (direct stopping-power/range queries, indirect phrasing, conversational filler, multi-material/particle/
   energy comparisons, program-name comparison, tricky units, isotope names, an inverse query, a material
   alias) — an LLM can produce this first draft.
2. Native-speaker review pass (the colleagues who offered to help): correct anything that reads as
   unnatural or a literal translation rather than how a Polish physicist would actually ask. This is the
   step that actually de-risks the "LLM generates sentences" idea — treat model output as a draft, not
   ground truth.
3. Record sessions mirroring scripts/record-session.sh (same WAV/sample-rate conventions), 2–3+ speakers,
   into a new eval/audio/<speaker>/ set kept separate from the English speakers.
4. Hand-label gold QueryIntents for each sentence into a new eval/intents.pl.jsonl (same schema, new
   file — see "Eval set" row above).
5. Run the existing pipeline scripts (asr-transcribe.mjs, asr-score-slots.mjs, e2e-audio-intents.ts)
   against the Polish set once the ASR language plumbing exists, to get the first real Polish domain
   E2E number — the thing nobody can quote yet.

Smallest useful next step, doable this week: record even 10–15 of these sentences from 1–2 friends,
run them through whisper-small forced to <|pl|> with no Polish domain prompt yet, and get a first
honest raw-Polish-domain-WER baseline — exactly how the English work started, before any tuning existed.

Effort/risk at a glance

Component	Relative effort	Why
ASR language plumbing (thread a language param through worker → transcribe)	Low	One hardcoded lookup key to parametrize; no new model.
Polish domain prompt + corrector	Medium	Needs real Polish recordings to tune against — data-bound, not design-bound.
NLU matcher	High	705 lines of inline English regex/grammar; no data-driven shortcut (Option B above is the way to defer this).
Alias tables (materials/particles)	Low	Same alias → id shape already supports this; mainly data entry + native-speaker verification.
NLG templates	Medium–High	Polish case declension is a real linguistic problem, not just translation; a labeled/non-declined v1 output sidesteps it.
UI strings / i18n scaffolding	Medium	No library exists yet; ~45–55 short strings to extract; recommend a minimal messages.ts + a runes-based locale.svelte.ts store (matching the existing *.svelte.ts store convention, e.g. model-status.svelte.ts) over pulling in a full i18n framework for 2 languages.
TTS (Piper Polish voice)	Low	Voices already exist; mirrors the same model-hosting pattern already used for Whisper.
Eval tooling	Low–Medium	New file + a parameterized/duplicated validator script.
On-device Polish LLM fallback	Medium, low priority	Needs its own Bielik-vs-Qwen bench; the fallback itself is unshipped for English too.

Open questions (need a human decision, not more research)

1. Option A vs. B vs. the hybrid above — is showing an English-translated echo of a Polish spoken
   query acceptable for v1, or does trust UX require the literal Polish transcript even if it costs the
   full matcher fork?
2. NLG v1 style — is a labeled/colon-style Polish answer ("materiał: woda") acceptable initially, or
   is naturally-declined Polish a hard requirement from day one?
3. UI toggle design — a flag-emoji button (🇬🇧/🇵🇱, as originally suggested) or a switch control matching
   the existing DarkModeToggle.svelte pattern? Does switching language also need to persist across
   sessions (localStorage, matching how ASR progress calibration already persists)?
4. Auto-detect vs. explicit toggle — should the app ever attempt to detect Polish vs. English from
   speech automatically, or should the toggle always be an explicit, user-driven choice? (Recommend the
   latter for v1 — simpler, and matches how the ASR domain-prompt mechanism already assumes a known,
   fixed language per request.)
5. How many speakers/sentences is "enough" for a first Polish eval set? The English set used 3
   speakers × ~30 sentences as its first iteration (eval/RECORDING.md) and grew from there — is that the
   right starting scale here too, given however many friends actually volunteer?
6. Physicist/native-speaker sign-off, same as docs/design.md already flags for English isotope/
   energy conventions — Polish terminology choices (e.g. "moc hamowania" vs. "zdolność hamowania" for
   stopping power) need a real decision from a Polish-speaking physicist, not a model guess.

References

Gathered during this research pass — general/background reading, not load-bearing citations for any
specific number quoted above (see caveats inline where a figure is unverified):

• speakleash/Bielik-11B-v2, Bielik-11B-v2.3-Instruct — Polish LLM family, Cyfronet-trained
• Bielik V3 Small: Technical Report, Bielik 11B v3
• bardsai/whisper-medium-pl — existing Polish-finetuned Whisper (over this project's size budget, but evidence the fine-tune path works)
• WitoldG/polish_piper_models, vits-piper-pl_PL mirror — Piper Polish voices
• Common Voice dataset metadata, Common Voice datasets — Polish corpus size

---

Filed from an investigation session; no code changes made. Scope intentionally left as an open proposal —
see "Open questions" before picking an implementation path.

[grzanka]

Follow-up after checking PR #71 (Qwen3-TTS eval-audio pilot) against this issue's scope.

Qwen3-TTS doesn't cover Polish. PR #71 §6–7 ran Qwen/Qwen3-TTS-12Hz-1.7B (-VoiceDesign/-CustomVoice) successfully to generate 100 English audio clips, but the model's own card lists only 10 official languages (Chinese, English, Japanese, Korean, German, French, Russian, Portuguese, Spanish, Italian) — Polish isn't one of them, confirmed by still-open asks on the Qwen3-TTS repo itself (discussion #116, #166). A community fine-tune exists (agnostic/Qwen3-TTS-Polish / -r10 on HF) but it's unofficial/unverified — same due-diligence PR #71 already did for the official model would be needed before trusting it. Coqui XTTS-v2 — already considered and passed over in issue #30/PR #71 in favor of Kokoro — does officially support Polish (one of ~16-17 languages) and is the more credible same-tier option if Qwen3-TTS-style free-text voice/emotion design is wanted for Polish too; otherwise Piper (already covered above in this issue) is the lower-risk pick and needs no such workaround.

The more important blocker, independent of which TTS model gets picked: we need the matcher first. PR #71 §7's entire method is "validate every candidate sentence against the app's real matcher + WASM compute (scripts/tts-sentence-check.ts) before synthesizing it" — that's what makes its 100 clips trustworthy ground truth rather than just plausible-sounding audio. There's no Polish equivalent of that check today, because there's no Polish matcher for it to call (this issue's "NLU matcher" row above is still the highest-effort item, 705 lines of inline English regex). So generating Polish eval audio isn't usefully unblocked by picking a Polish-capable TTS model alone — without a Polish matcher (or the Option B translate-task shortcut already proposed above) to validate against, a Polish audio batch can only produce clips whose "expected" intent was never actually confirmed resolvable, exactly the trap tts-sentence-check.ts was built to avoid for English. Sequencing: the matcher (or its translate-task shortcut) comes before a Polish audio-generation pass, not in parallel with it.

---

Generated by Claude Code

[grzanka]

Progress update from a recent session's work — nothing here is Polish-specific yet (no pl pack, no
UI toggle, no ASR language plumbing), but two of the concrete blockers this issue's own audit table
named are now addressed, and the architecture the rest of this issue depends on is in place.

Directly fixes a bug this issue's own table flagged

The ł NFKD bug, called out above under "Alias tables": "normalize.ts's NFKD stripping folds
Polish ć/ń/ó/ś/ą/ę fine, but 'ł' (U+0142) does not NFKD-decompose and survives... an unaccented 'l'
typed by a user won't match a stored 'ł' alias." Fixed in #88 —
src/lib/aliases/normalize.ts:65 now maps ł→l/Ł→L explicitly before the NFKD strip, with a
regression test (ołów/łuk in src/lib/aliases/aliases.test.ts). One less real bug waiting for
whoever adds the first Polish alias entries.

The architectural seam this issue's "High effort" NLU-matcher row was waiting on

This issue's own table said: "Entirely inline English regex/Set literals, not data-driven...
A Polish variant realistically means forking/parametrizing the whole matcher, not dropping in a
translated table." #88 did exactly that refactor (as a verified no-behavior-change split,
pnpm coverage:intents unchanged at 93.4%): src/lib/intent/matcher.ts is now the language-neutral
core (span arithmetic, number+unit grammar, entity resolution, confidence scoring) and every
English-specific table (INDIRECT_IDIOMS, DIRECT_STOPPING/DIRECT_RANGE, the material stopword
set, list connectors, particle-head words) moved verbatim into src/lib/intent/lang/en.ts. A pl.ts
pack is now the natural next step — data, not a fork.

The same split happened on the ASR-correction side (issue #28, this issue's "Medium effort" Polish
domain-prompt/corrector row): #89 split the regex correction layer into
src/lib/asr/correct/core.ts + en.ts and wired it into the live transcript path (previously the
.mjs corrector scripts weren't wired into the app at all); #90 added a closed-lexicon
edit-distance pass on top for mishearings no one's written a rule for yet. Both are shipped and
validated — first against the 3-speaker set, then at 1000-sentence scale in #91
(docs/tts-eval-1000-v2.md), where the phonetic pass alone lifted clip-pass from 70.5% raw to 77.7%.

Still exactly where this issue's audit left it

• ASR language token still hardcoded — src/lib/asr/transcribe.ts:218,
  langToId["<|en|>"] — and DOMAIN_PROMPT (line 78) is still English-only. No lang/locale field
  exists in TranscribeOptions, the worker protocol, or QueryIntent yet.
• No pl.ts pack for either the matcher or the corrector — the seam exists, the content doesn't.
• No UI toggle, no i18n scaffolding, no eval/intents.pl.jsonl.
• NLG (src/lib/nlg/render.ts) untouched — the Polish-declension problem this issue flagged is still
  open.

Also relevant

Two unrelated-but-adjacent things landed in the same span of work: the matcher now recognizes "LET"/
"linear energy transfer" as a stopping-power synonym (#86) — the earlier Polish recording plan
(#84/#85, eval/RECORDING.pl.md) already calls out "LET" as a synonym for "zdolność hamowania" in its
own domain-review conventions, so matcher and Polish-recording-plan agree on that term independently.
And the regenerated 1000-sentence TTS batch (#91) is an English-only eval artifact, but the same
generate→validate→sbatch pipeline it exercises is the template a Polish TTS eval batch would reuse.

Net effect: the two "fork the whole thing" cost items in this issue's effort table (NLU matcher, ASR
corrector) are now pack-seamed rather than monolithic, and one concrete bug is gone. The ASR-language-
plumbing, alias-data, NLG, and UI-toggle rows are all still open exactly as scoped.