Conversation
… pipeline The grammar in typescript2/pkg-grammar now feeds every external consumer through the BoundaryML/textMate-baml mirror, with no manual steps: - build.ts also emits dist/index.js + dist/index.d.ts with the grammar inlined as a JS object literal (Shiki LanguageRegistration), so npm consumers never import JSON (Metro can't parse import attributes) - language-configuration.json moves into pkg-grammar as the source of truth; sync.mjs mirrors it into app-vscode-ext like the grammar - scripts/assemble-mirror.mjs assembles the complete mirror repo layout (npm package @boundaryml/baml-grammar + Linguist vendor dir + the mirror's own npm publish workflow) from the templates in mirror/ - sync-grammar-mirror workflow pushes the assembled mirror to BoundaryML/textMate-baml on grammar changes on canary, stamps a patch-bumped version only when content actually changed, and fails loudly if the npm release doesn't appear - new linguist-compile-gate CI job runs GitHub Linguist's own grammar-compiler (Oniguruma->PCRE conversion) against the freshly built grammar, so a change that would break .baml highlighting on github.com fails the PR instead of Linguist's next quarterly release - new tests: external-consumer invariants (scopeName/fileTypes/ self-containedness), dist-vs-raw drift, strict-JSON language config, and a whole-repo corpus tokenization pass (1293 files) that catches crashes and catastrophic-backtracking regexes - grammar-tests CI job now rebuilds and fails if committed artifacts drift from src/ Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
⏭️ Performance benchmarks were skippedPerf benchmarks (CodSpeed) are opt-in on pull requests — they no longer run on every push. They always run automatically after merge to To run them on this PR, do any of the following, then push a commit (or re-run CI):
|
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (2)
📝 WalkthroughWalkthroughGrammar packaging now emits typed ESM artifacts and language configuration files, synchronizes mirrors, publishes an external npm package, and adds corpus, artifact, Linguist, and generated-file consistency checks to CI. ChangesGrammar artifact pipeline
Estimated code review effort: 4 (Complex) | ~45 minutes Possibly related PRs
Poem
Sequence Diagram(s)sequenceDiagram
participant GrammarSource
participant BuildAndTest
participant MirrorWorkflow
participant MirrorRepository
participant NpmRegistry
GrammarSource->>BuildAndTest: build and validate grammar artifacts
BuildAndTest->>MirrorWorkflow: provide committed generated artifacts
MirrorWorkflow->>MirrorRepository: assemble, version, commit, and push mirror
MirrorRepository->>NpmRegistry: publish package
MirrorWorkflow->>NpmRegistry: poll for published version
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Binary size checks passed✅ 7 passed
Generated by |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
typescript2/pkg-grammar/mirror/publish.yml (1)
33-36: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueConsider pinning
node-versioninstead of usinglatest.
node-version: latestresolves to the newest Node.js release on the runner, which can change without notice and potentially break the workflow. Consider pinning to a specific major (e.g.,22) or usinglts/*for more reproducible behavior while still meeting the npm ≥ 11.5.1 requirement.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@typescript2/pkg-grammar/mirror/publish.yml` around lines 33 - 36, Update the node-version setting in the setup-node step to use a stable pinned Node.js major or lts/* value instead of latest, while retaining compatibility with the npm ≥ 11.5.1 requirement.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@typescript2/pkg-grammar/language-configuration.json`:
- Line 14: Update the bracket-pair configuration in language-configuration.json
so the `{#` opener uses `#}` as its closing token instead of `}`; leave the
other bracket pairs unchanged.
---
Nitpick comments:
In `@typescript2/pkg-grammar/mirror/publish.yml`:
- Around line 33-36: Update the node-version setting in the setup-node step to
use a stable pinned Node.js major or lts/* value instead of latest, while
retaining compatibility with the npm ≥ 11.5.1 requirement.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: c04e1918-2240-4121-89c5-5feeb54591e0
⛔ Files ignored due to path filters (1)
typescript2/pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (15)
.github/workflows/ci.yaml.github/workflows/sync-grammar-mirror.yml.pre-commit-config.yamltypescript2/app-vscode-ext/language-configuration.jsontypescript2/pkg-grammar/README.mdtypescript2/pkg-grammar/language-configuration.jsontypescript2/pkg-grammar/mirror/README.mdtypescript2/pkg-grammar/mirror/package.jsontypescript2/pkg-grammar/mirror/publish.ymltypescript2/pkg-grammar/package.jsontypescript2/pkg-grammar/scripts/assemble-mirror.mjstypescript2/pkg-grammar/scripts/build.tstypescript2/pkg-grammar/scripts/sync.mjstypescript2/pkg-grammar/tests/package-artifacts.test.tstypescript2/pkg-grammar/tests/repo-corpus.test.ts
💤 Files with no reviewable changes (1)
- typescript2/app-vscode-ext/language-configuration.json
{# ... #} is the template comment form, so typing {# should insert #},
not a bare brace.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Merged
3 tasks
meefs
pushed a commit
to meefs/baml
that referenced
this pull request
Jul 13, 2026
…ghlight.js, tree-sitter (BoundaryML#3996) Extends the grammar mirror pipeline (BoundaryML#3986) so every syntax-highlighting registry on the internet is fed from this monorepo, with mirrors as write-only build artifacts. Follows the plan in `typescript2/pkg-grammar/DISTRIBUTION.md` (included). ## What's here **`pkg-grammar` (extended)** - `baml.sublime-syntax` generated from the TextMate grammar by `scripts/emit-sublime.ts` (wired into `build`, drift-guard test). Validated against real syntect: `bat` renders every fixture with zero errors. - `syntaxes/baml.xml` — hand-authored KDE KSyntaxHighlighting definition (Kate, Pandoc via skylighting). Validates against the official KDE `language.xsd`. Dynamic rules support arbitrary raw-string `#` depth. - `tests/fixtures/showcase__golden_sample.baml` — canonical showcase sample, mirrored as `samples/baml.sample` for registry submissions (Shiki etc.). - Mirror hardening: `SUPPORT.md` registry matrix, frozen-path contract + read-only footer in the mirror README, `DISTRIBUTION.md` rollout playbook. **`pkg-grammar-hljs` (new)** → mirrors to [BoundaryML/baml-highlightjs](https://github.com/BoundaryML/baml-highlightjs), npm `@boundaryml/baml-highlightjs` - highlight.js 11 language definition with Jinja `{{ }}`/`{% %}` highlighting inside prompt bodies, raw/backtick/byte strings, full keyword taxonomy from the real lexer. - Tested against every `pkg-grammar` fixture (82 tests); browser/CDN dist build generated at mirror-assembly time. **`pkg-grammar-treesitter` (new)** → mirrors to [BoundaryML/baml-treesitter](https://github.com/BoundaryML/baml-treesitter) (nvim-treesitter / Zed / Helix) - Modern tree-sitter grammar for the full current language (interfaces, generics, lambdas, match patterns, `let`/`const` destructuring, backtick `${}` bodies). No external scanner. - **80/80 fixtures parse with zero ERROR/MISSING**; 63 corpus tests; highlight queries + jinja injection into prompt bodies. - The old `BoundaryML/tree-sitter-baml` (15 months stale) is untouched; the Zed pin migrates later. **Automation** - `sync-grammar-mirror.yml` fans out to all three mirrors; npm-published mirrors get `v*` tags (bat/Package Control/Linguist pinning). - `grammar-tests` CI runs all three packages as a conformance gate: every port validates against the shared fixture corpus, so a language change a port doesn't handle fails the PR. ## Bootstrap status (already done via gh) - [x] `BoundaryML/baml-highlightjs` + `BoundaryML/baml-treesitter` created and seeded with assembled content (hljs tagged `v0.1.0`) - [x] Write deploy keys added; `BAML_HIGHLIGHTJS_DEPLOY_KEY` / `BAML_TREESITTER_DEPLOY_KEY` secrets set on this repo - [x] First manual `npm publish` of `@boundaryml/baml-highlightjs` (v0.1.0 on npm) + trusted publisher configured 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Highlight.js and Tree-sitter BAML grammar support with expanded automated read-only mirror syncing and npm publishing. * Expanded the generated syntax-highlighting artifacts across the grammar family (including KDE and Sublime outputs). * Added support for constructor field shorthand (fields without `:`) in syntax highlighting. * **Bug Fixes** * Improved CI change detection to ensure grammar-impacting updates trigger full conformance runs. * **Tests** * Added/extended Highlight.js and Tree-sitter test suites with new fixtures and snapshot/golden validation. * **Documentation** * Added documentation for the new grammar packages and distribution/mirroring outputs. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements B-870 (with the release mechanism moved out of the monorepo, per design discussion).
Architecture
typescript2/pkg-grammarstays the single source of truth. Every external consumer is fed through the BoundaryML/textMate-baml mirror, entirely by automation:What's in this PR
build.tsemitsdist/index.js+.d.tswith the grammar inlined as a JS object literal, typed as a ShikiLanguageRegistration. npm consumers never touch JSON import attributes (Metro/React Native can't parse them).sync.mjsmirrors it into app-vscode-ext (pre-commit + CI guarded).mirror/templates: publishable package.json, README, LICENSE,grammars/baml.tmLanguage.json(Linguist's stable path), and the mirror's own npm publish workflow..bamlhighlighting on github.com breaking at Linguist's next quarterly release.scopeName: source.baml,.bamlfileType, grammar self-containedness), dist-vs-raw drift, strict-JSON language configuration, and a whole-repo corpus pass (1293.bamlfiles) catching crashes and catastrophic-backtracking regexes.src/.Deployment state
TEXTMATE_BAML_DEPLOY_KEYsecret on this repo. ✅npm publish --access publicof@boundaryml/baml-grammarfrom a mirror checkout (OIDC can't create a new package), then configure the trusted publisher (repoBoundaryML/textMate-baml, workflowpublish.yml) in the npm package settings.shikijs/textmate-grammars-themespointing at the mirror's raw grammar URL.🤖 Generated with Claude Code
Summary by CodeRabbit
{# ... #}pairs.