FL4WRITE retroactive gauntlet: full-tree review of published HEAD - #211
FL4WRITE retroactive gauntlet: full-tree review of published HEAD#211simongonzalezdc wants to merge 330 commits into
Conversation
Expose generic provider output before launch testing
Provider testing could still claim readiness from five launch-critical fixtures while twenty covered dialects had no output-level dogfood contract. This adds eval fixtures for every supported dialect, makes the harness assert all dialects are covered, and strengthens mock semantics enough to keep local fixture verification deterministic. Constraint: Live providers can return fluent generic Spanish while ignoring dialect identity markers Rejected: Wait for native review before automated fixtures | provider testing needs a complete automated floor now, with native review used to tune thresholds Rejected: Pack eval fixtures into published packages | these are repo dogfood assets, not runtime package API Confidence: high Scope-risk: moderate Directive: Treat requiredOutputAny as a provider gate; use preferredOutputAny for review-tunable quality misses Tested: pnpm build; pnpm -r exec tsc --noEmit; pnpm test (641 passing); pnpm audit --audit-level low; npm pack dry-run guard; mock dialect eval 27/27 across 25 dialects; live MyMemory all-dialect eval exposed 16 hard failures and 20 warnings
…ures Make provider dogfood cover every dialect
DialectOS needs a provider that consumes dialect profiles, quality contracts, formality, and semantic context instead of silently falling back to generic machine translation. This adds an OpenAI-compatible LLM provider, registers it in CLI and MCP factories, ranks semantic providers first in auto routing, and keeps DeepL/LibreTranslate/MyMemory as fallback utilities. Constraint: Do not add new dependencies; use fetch against an OpenAI-compatible chat completions endpoint Constraint: Local LLM endpoints are blocked unless LLM_ALLOW_LOCAL=1 is explicitly set Rejected: Keep LibreTranslate/MyMemory as launch-primary providers | they ignore dialect/context and produce generic Spanish Rejected: Hard-code one hosted LLM SDK | the product should work with any compatible LLM gateway Confidence: high Scope-risk: moderate Directive: Do not let dialectHandling=none providers become silent release-quality dialect providers Tested: pnpm build; pnpm -r exec tsc --noEmit; pnpm test (650 passing); pnpm audit --audit-level low; npm pack dry-run guard; mock dialect eval 27/27 across 25 dialects; live OpenAI-compatible mock LLM eval 27/27
Make semantic LLMs the primary translation provider
The LLM provider was OpenAI-compatible only, which made the core dialect engine depend on one chat-completions wire format. This adds Anthropic-compatible messages support behind LLM_API_FORMAT while preserving the OpenAI-compatible default. Constraint: No provider SDK dependency; use fetch and explicit wire-format adapters Constraint: Anthropic-compatible endpoints require anthropic-version headers and x-api-key when an API key is configured Rejected: Add a separate anthropic provider name | keeping one semantic llm provider lets model gateways stay interchangeable Confidence: high Scope-risk: narrow Directive: Add new model API formats behind LLM_API_FORMAT rather than reintroducing generic MT as primary Tested: pnpm build; pnpm -r exec tsc --noEmit; pnpm test (653 passing); pnpm audit --audit-level low; npm pack dry-run guard; mock dialect eval 27/27; live Anthropic-compatible mock eval 27/27
Make LLM provider model API agnostic
DialectOS can already use generic OpenAI and Anthropic wire formats, but LM Studio has a native local REST API that can list, load, and chat with downloaded local models. This adds LLM_API_FORMAT=lmstudio so provider certification can run against any local LM Studio model with just-in-time loading. Constraint: Use LM Studio native REST endpoints without adding SDK dependencies Constraint: Default LM Studio endpoint is local http://127.0.0.1:1234 and only enabled when explicitly selecting lmstudio format Rejected: Require users to pre-load models manually | LM Studio exposes /api/v1/models/load and chat can load models on demand Confidence: high Scope-risk: narrow Directive: Keep local model integration under semantic llm provider; do not route dialect certification through generic MT providers Tested: pnpm build; pnpm -r exec tsc --noEmit; pnpm test (659 passing); pnpm audit --audit-level low; npm pack dry-run guard; mock dialect eval 27/27; live LM Studio native mock eval 27/27
Support LM Studio local model certification
LM Studio certification exposed that smaller/local models copied taboo examples, missed regional must-use terms, or changed source intent even when the general dialect profile was present. This adds explicit source-shaped output constraints for known certification fixtures and widens preferred variants when already-required outputs are acceptable. Constraint: Keep changes dependency-free and inside prompt/fixture contracts Rejected: Treat local model misses as model-only failures | cloud readiness is stronger when local models pass the same gates Confidence: high Scope-risk: narrow Directive: Add future local-model failures as explicit output constraints only when they reflect product-critical dialect behavior Tested: pnpm build; pnpm -r exec tsc --noEmit; pnpm test (660 passing); pnpm audit --audit-level low; npm pack dry-run guard; mock dialect eval 27/27; qwen3.5-9b full LM Studio eval 27/27; qwen3.6 per-dialect LM Studio eval all required gates passing after es-GQ retry
…-hardening Harden prompts for local model certification
Long local-model runs were hiding progress and failures until the end. This adds dialect:certify, which runs each fixture in an isolated child process, writes events/progress/results incrementally, enforces per-sample timeouts, and retries transient provider/process failures. Constraint: Certification must stay dependency-free and work for LM Studio, OpenAI-compatible, and Anthropic-compatible LLM providers Rejected: Keep using dialect:eval for long local-model runs | monolithic output made qwen3.6 look hung and lost partial evidence Confidence: high Scope-risk: moderate Directive: Use dialect:certify, not dialect:eval, for launch certification and long local/cloud provider runs Tested: pnpm build; pnpm -r exec tsc --noEmit; pnpm test (663 passing); pnpm audit --audit-level low; npm pack dry-run guard; mock dialect:certify 27/27 with events/progress/results; qwen3.5-9b LM Studio dialect:certify 27/27; qwen3.6 LM Studio targeted retry for es-GQ/es-MX 2/2
Add incremental dialect certification runner
Basic provider certification proves broad coverage, but it does not stress paraphrases, dialect collisions, taboo-copy traps, placeholder preservation, register shifts, or repeatability. This adds an adversarial fixture suite and a repeat-aware certification wrapper that writes a failure matrix while separating pass instability from harmless output variance. Constraint: Keep the harness dependency-free and reuse dialect:certify child-process isolation Rejected: Treat every wording variation as a launch failure | passing output variance is reported separately unless strict-output-stability is requested Confidence: high Scope-risk: moderate Directive: Expand packages/cli/src/__tests__/fixtures/dialect-adversarial when new local/cloud model failures reveal new classes of risk Tested: pnpm build; pnpm -r exec tsc --noEmit; pnpm test (664 passing); pnpm audit --audit-level low; npm pack dry-run guard; mock adversarial certify 11/11; qwen3.5-9b adversarial repeat=2 22/22 with 0 failures and 0 unstable; qwen3.6 adversarial 11/11; glm-4.5-air adversarial repeat=2 22/22 with 0 failures and 0 unstable
Add adversarial dialect certification
The first adversarial suite proved the harness but only covered 11 samples across a subset of dialects. This expands adversarial certification to all 25 dialects with 125 samples covering dialect collisions, false friends, intent ambiguity, morphology traps, negative controls, over/under-localization, register traps, structure preservation, and taboo-copy defenses. Constraint: Keep fixtures deterministic and compatible with the existing dialect:certify:adversarial runner Rejected: Fail all transit synonyms not literally bus | real provider evidence showed autobus, colectivo, bondi, buseta, and omnibus are acceptable non-Caribbean variants depending dialect Confidence: high Scope-risk: moderate Directive: Treat adversarial fixture expansion as the primary path for discovering new dialect failure modes Tested: pnpm build; pnpm -r exec tsc --noEmit; pnpm test (691 passing); pnpm audit --audit-level low; npm pack dry-run guard; mock adversarial certify 125/125; qwen3.5-9b expanded adversarial 125/125; glm-4.5-air expanded adversarial targeted failures rerun 15/15 and full rerun 125/125
Expand adversarial dialect certification coverage
Sentence-level certification can miss document-level regressions in markdown, API docs, and locale JSON flows. This adds dialect:certify:documents with long README/API/locale fixtures and assertions for placeholders, URLs, code fences, dialect traits, taboo terms, and document outputs. Constraint: Reuse existing CLI surfaces where possible and keep the harness dependency-free Rejected: Rely on sentence fixtures for launch readiness | product usage includes full docs and locale files Confidence: medium Scope-risk: moderate Directive: Expand document-adversarial fixtures before claiming document-level launch readiness Tested: pnpm build; pnpm -r exec tsc --noEmit; pnpm test (692 passing); pnpm audit --audit-level low; npm pack dry-run guard; mock document certification 7/7
Add long document certification harness
Live document certification can fail before writing output files because CLI path guards, provider errors, or partial section failures stop a document command. The harness now records missing outputs and command stderr as certification failures instead of crashing, and its assertions accept documented valid Spain/voseo variants seen in live output. Constraint: Keep this as a harness fix; do not weaken document certification’s placeholder, URL, taboo, or dialect trait checks Rejected: Ignore live document cert failure as environment-only | missing output handling is part of reliable certification Confidence: high Scope-risk: narrow Tested: pnpm build; pnpm -r exec tsc --noEmit; document-certify-script.test.ts; pnpm audit --audit-level low; live glm-4.5-air document certification 7/7 with ALLOWED_LOCALE_DIRS configured
Live document certification can fail before output files are written due path policy, provider, or partial translation errors. The harness now reports missing output files and command stderr in results.json rather than crashing, and accepts valid documented Spain/Argentina variants observed in live output. Constraint: Keep placeholder, URL, taboo, and dialect-trait assertions active Rejected: Treat missing outputs as harness crashes | they are certification failures and should be recorded Confidence: high Scope-risk: narrow Tested: pnpm build; pnpm -r exec tsc --noEmit; pnpm test (693 passing); pnpm audit --audit-level low; npm pack dry-run guard; live glm-4.5-air document certification 7/7 with ALLOWED_LOCALE_DIRS configured
…put-fix Record document certification command failures
Live document certification can fail before output files are written due path policy, provider, or partial translation errors. The harness now reports missing output files and command stderr in results.json rather than crashing, and accepts valid documented Spain/Argentina variants observed in live output. Constraint: Keep placeholder, URL, taboo, and dialect-trait assertions active Rejected: Treat missing outputs as harness crashes | they are certification failures and should be recorded Confidence: high Scope-risk: narrow Tested: pnpm build; pnpm -r exec tsc --noEmit; pnpm test (693 passing); pnpm audit --audit-level low; npm pack dry-run guard; live glm-4.5-air document certification 7/7 with ALLOWED_LOCALE_DIRS configured
…put-fix Stabilize document certification failure test
The project now has passing unit, package, basic certification, expanded adversarial certification, local-model, cloud-model, and long-document certification evidence. This records the RC1 launch decision, model policy, known limitations, and machine-readable certification matrix. Constraint: Do not include secrets; reference artifact paths and command shapes only Confidence: high Scope-risk: narrow Directive: Update this RC report or add a new one before tagging future release candidates Tested: pnpm build; pnpm -r exec tsc --noEmit; pnpm test (693 passing); pnpm audit --audit-level low; npm pack guard; certification artifacts listed in audits/release-candidate-2026-04-22/model-matrix.json
Record v0.1.0 release candidate evidence
Certification artifacts are useful internally, but monetization needs a customer-facing deliverable. This adds dialect:report to convert certification JSON into an executive Markdown report with launch grade, matrix, failure details, limits, and commercial packaging guidance. Constraint: Keep report generation dependency-free and avoid embedding secrets Confidence: high Scope-risk: narrow Directive: Use dialect:report as the paid launch-audit deliverable surface Tested: pnpm build; pnpm -r exec tsc --noEmit; pnpm test (695 passing); pnpm audit --audit-level low; npm pack guard; generated /tmp/dialectos-customer-report-final.md from RC model matrix
Add customer certification report generator
Customers need a standards-aligned localization certification deliverable, not just raw pass/fail JSON. This adds MQM-style issue categories/severities, dialect validation metadata, certification levels, and report sections for issue distribution plus validation status. Constraint: Keep report generation dependency-free and avoid formal compliance claims beyond MQM-aligned wording Rejected: Claim ISO/MQM certification | the current scope is an MQM-aligned automated report, not accredited human LQA Confidence: high Scope-risk: moderate Directive: Use certification.ts as the canonical place for certification levels and validation status Tested: pnpm build; pnpm -r exec tsc --noEmit; pnpm test (700 passing); pnpm audit --audit-level low; npm pack guard; dialect-report generated MQM-aligned report
Align certification reports with MQM
…te/minor-deps into main
…e/patch-deps into main
…te/minor-deps into main
…te/minor-deps into main
…te/patch-deps into main
…te/minor-deps into main
…te/patch-deps into main
…te/minor-deps into main
…te/minor-deps into main
Wave 2/2b compressed this public face. Restore the larger pre-compress README on Forgejo (source of truth); GitHub mirrors this host. Source blob: 7d1065b (~21196 bytes; was ~801). README-only; no product code changes.
Forgejo-first dual-host face: entity, TL;DR, FAQ, audience, status.
…s/splus-seo-geo-40946 into main
Phase 1 of MCP 2026-07-28 stateless migration. Rewrote tool descriptions with what/returns/when-to-use pattern.
…nomics' (#22) from chore/mcp-stateless-uplift into main
🔍 FL4WRITE review
The diff's own tests FAIL (verified: cmd='python3 -m pytest tests/ -x -q --tb=line'; files=['packages/cli/src/tests/adversarial-dialect-fixtures.test.ts', 'packages/cli/src/tests/adversarial-fixtures.test.ts', 'packages/cli/src/tests/checkpoint.test.ts', 'packages/cli/src/tests/corpus.test.ts', 'packages/cli/src/tests/dialect-eval-script.test.ts', 'packages/cli/src/tests/dialect-eval.test.ts', 'packages/cli/src/tests/dialect-report-script.test.ts', 'packages/cli/src/tests/dialects.test.ts', 'packages/cli/src/tests/document-certify-script.test.ts', 'packages/cli/src/tests/e2e-verify-fixes.test.ts', 'packages/cli/src/tests/eval-harness.test.ts', 'packages/cli/src/tests/glossary-diff.test.ts', 'packages/cli/src/tests/glossary-suggest.test.ts', 'packages/cli/src/tests/glossary.test.ts', 'packages/cli/src/tests/grammar-detection-torture.test.ts', 'packages/cli/src/tests/grammar-detection.test.ts', 'packages/cli/src/tests/i18n/apply-gender-neutral.test.ts', 'packages/cli/src/tests/i18n/batch-translate.test.ts', 'packages/cli/src/tests/i18n/check-formality.test.ts', 'packages/cli/src/tests/i18n/detect-missing.test.ts', 'packages/cli/src/tests/i18n/manage-variants.test.ts', 'packages/cli/src/tests/i18n/translate-keys.test.ts', 'packages/cli/src/tests/lexical-ambiguity.test.ts', 'packages/cli/src/tests/output-judge.test.ts', 'packages/cli/src/tests/provider-factory.test.ts', 'packages/cli/src/tests/regional-research.test.ts', 'packages/cli/src/tests/resilient-translation.test.ts', 'packages/cli/src/tests/semantic-backstop-torture.test.ts', 'packages/cli/src/tests/semantic-backstop.test.ts', 'packages/cli/src/tests/semantic-context.test.ts', 'packages/cli/src/tests/semantic-similarity.test.ts', 'packages/cli/src/tests/telemetry.test.ts', 'packages/cli/src/tests/token-protection.test.ts', 'packages/cli/src/tests/translate-api-docs.test.ts', 'packages/cli/src/tests/translate-readme.test.ts', 'packages/cli/src/tests/translate.test.ts', 'packages/cli/src/tests/translation-policy.test.ts', 'packages/cli/src/tests/validate.test.ts', 'packages/cli/src/tests/web-demo-service.test.ts', 'packages/locale-utils/src/tests/locale-utils.test.ts', 'packages/markdown-parser/src/tests/parser.test.ts', 'packages/mcp/src/tests/config.test.ts', 'packages/mcp/src/tests/docs.test.ts', 'packages/mcp/src/tests/error-handler.test.ts', 'packages/mcp/src/tests/i18n.test.ts', 'packages/mcp/src/tests/index.test.ts', 'packages/mcp/src/tests/security.test.ts', 'packages/mcp/src/tests/translator.test.ts', 'packages/providers/src/tests/agreement-validator.test.ts', 'packages/providers/src/tests/cached-provider.test.ts', 'packages/providers/src/tests/capitalization.test.ts', 'packages/providers/src/tests/circuit-breaker.test.ts', 'packages/providers/src/tests/dialect-output-pipeline.test.ts', 'packages/providers/src/tests/false-friends.test.ts', 'packages/providers/src/tests/lexical-substitution.test.ts', 'packages/providers/src/tests/postprocessing.test.ts', 'packages/providers/src/tests/providers.test.ts', 'packages/providers/src/tests/quality-gates.test.ts', 'packages/providers/src/tests/sentinel-extraction.test.ts', 'packages/providers/src/tests/translation-corpus.test.ts', 'packages/providers/src/tests/translation-memory-bounds.test.ts', 'packages/providers/src/tests/translation-memory-torture.test.ts', 'packages/providers/src/tests/typography.test.ts', 'packages/providers/src/tests/voseo-adapter.test.ts', 'packages/providers/src/bulk/tests/engine.test.ts', 'packages/providers/src/bulk/tests/semaphore.test.ts', 'packages/providers/src/providers/tests/llm-prompts.test.ts', 'packages/providers/src/providers/tests/llm-response.test.ts', 'packages/security/src/tests/security.test.ts', 'packages/types/src/tests/certification.test.ts', 'packages/types/src/tests/dialect-profiles.test.ts', 'packages/types/src/tests/dialect-quality.test.ts', 'packages/types/src/tests/dialect-regions.test.ts', 'packages/types/src/tests/dialectal-dictionary.test.ts', 'packages/types/src/tests/dialectal-vocabulary.test.ts', 'packages/types/src/tests/glossary-data.test.ts', 'packages/types/src/tests/noun-gender.test.ts', 'packages/types/src/tests/types.test.ts', 'packages/types/src/tests/verb-conjugations.test.ts']; head=6c77405570): no tests ran in 0.00s | | ERROR: file or directory not found: tests/ 🚨 Do NOT merge until this is addressed. Reviewed by FL4WRITE · tone: assertive · org law |
1 participant
CEO order 2026-09-01: every published repo passes the hardcore FL4WRITE review+fix loop, retroactively. Base = empty tree anchored to the repo root; head = marker carrying the default-branch tree; the diff is the ENTIRE published tree. Findings get fixed on main; the target branch is refreshed after each fix so the PR re-reviews; loop until a full cycle returns zero Critical and zero Major. This PR never merges — it is the standing review vehicle.