Skip to content

feat(cli): background-agnostic describe highlighting with LSP-exact tokens - #4017

Merged
codeshaunted merged 2 commits into
canaryfrom
avery/describe-paint
Jul 14, 2026
Merged

feat(cli): background-agnostic describe highlighting with LSP-exact tokens#4017
codeshaunted merged 2 commits into
canaryfrom
avery/describe-paint

Conversation

@codeshaunted

@codeshaunted codeshaunted commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Follow-up to #4011 (now on canary). Fixes three gaps that made baml describe highlighting look worse than the same code in an editor:

  1. Background-agnostic palette. Colors are restricted to the terminal's named ANSI palette (slots 0–15, always theme-controlled) plus dim/bold attributes. The fixed 256-color-cube values and the forced-white variables are gone: ordinary names render at the default foreground, comments/operators are dim. Output is legible on light and dark terminals alike, styled by the user's own scheme.
  2. Semantic-token modifiers are honored. Declarations render bold, defaultLibrary (stdlib) entities italic, deprecated ones struck through — the same signals editor themes derive from the LSP modifier bitset, previously discarded. Dim-base token types trade dim for bold at their declaration instead of stacking contradictory weights.
  3. Fragments use the exact LSP classifier. Synthesized snippets (keyword-doc examples, interface member signatures) are parsed as a scratch file in a thread-local in-memory project and rendered from real semantic_tokens — replacing the hand-rolled lexer scanner that colored only keywords/primitives/literals. Names a context-free fragment can't resolve stay neutral, exactly like an editor. The scanner (PRIMITIVE_TYPES, decl_keyword_kind, is_keyword) is deleted and Highlighter::range/highlight_str now share one render loop.

Tests

  • New unit tests: variables keep the default foreground; declaration/defaultLibrary modifier overlays; a rendered body contains no fixed-cube 256-color codes and no forced white/black; fragments classify via the compiler (bold declaration + italic stdlib type — signals the old lexer could never emit); unresolvable fragment names stay neutral.
  • baml_cli suite 343/343 (describe snapshots are colorless and unaffected); clippy clean.
  • Verified end-to-end on a scratch project and on the describe defer keyword page with forced color.

Summary by CodeRabbit

  • Bug Fixes
    • Improved syntax highlighting so both real source code and generated fragments use the same semantic-token-based styling.
    • Correctly honors semantic modifiers (e.g., declaration emphasis, default-library italics, deprecation strikethrough, and dimming for non-declarations).
    • Keeps unresolved names neutral and avoids highlighting errors/crashes.
    • Preserves terminal default colors—no forced black/white and no fixed palette indices—while ensuring modifier overlays render correctly.

…okens

Three fixes to baml describe's terminal rendering, all in paint.rs:

- Background-agnostic palette: colors are now restricted to the terminal's
  named ANSI palette (slots 0-15, which the terminal theme controls) plus
  dim/bold attributes. The fixed 256-color-cube values (173/179/214/244/245)
  and the forced white for variables are gone - ordinary names render at the
  default foreground, comments and operators use dim.
- Modifiers are honored: declarations render bold, defaultLibrary (stdlib)
  entities italic, deprecated ones struck through. A dim-base token type
  trades dim for bold at its declaration instead of stacking both weights.
- Synthesized fragments (keyword-doc examples, interface member signatures)
  now run through the exact compiler classifier the LSP uses - parsed as a
  scratch file in a thread-local in-memory project - instead of the
  hand-rolled lexer scanner, which colored only keywords/primitives/literals
  and left every other identifier plain. Names a fragment cannot resolve stay
  at the default foreground, matching editor behavior for unresolved names.

The lexer fallback (PRIMITIVE_TYPES, decl_keyword_kind, is_keyword) is
deleted; Highlighter::range and highlight_str share one token-render loop.
@vercel

vercel Bot commented Jul 14, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
beps Ready Ready Preview, Comment Jul 14, 2026 6:11pm
promptfiddle2 Ready Ready Preview, Comment Jul 14, 2026 6:11pm
1 Skipped Deployment
Project Deployment Actions Updated (UTC)
promptfiddle Skipped Skipped Jul 14, 2026 6:11pm

Request Review

@vercel
vercel Bot temporarily deployed to Preview – promptfiddle July 14, 2026 17:34 Inactive
@github-actions

Copy link
Copy Markdown

⏭️ Performance benchmarks were skipped

Perf benchmarks (CodSpeed) are opt-in on pull requests — they no longer run on every push. They always run automatically after merge to canary/main.

To run them on this PR, do any of the following, then push a commit (or re-run CI):

  • Add RUN_CODSPEED=1 to the PR description, or
  • Include run-perf or /perf in the PR title or any commit message.

@vercel
vercel Bot temporarily deployed to Preview – beps July 14, 2026 17:35 Inactive
@coderabbitai

coderabbitai Bot commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 664cf418-7bfa-437f-94fe-decd9dc75180

📥 Commits

Reviewing files that changed from the base of the PR and between 2da5b9b and 134e0eb.

📒 Files selected for processing (1)
  • baml_language/crates/baml_cli/src/paint.rs
🚧 Files skipped from review as they are similar to previous changes (1)
  • baml_language/crates/baml_cli/src/paint.rs

📝 Walkthrough

Walkthrough

paint.rs now uses compiler semantic tokens for terminal highlighting, applies declaration/default-library/deprecated modifiers, shares token rendering between fragments and source ranges, and adds tests for styling, classification, terminal colors, and unresolved names.

Changes

Semantic highlighting

Layer / File(s) Summary
Modifier-aware style mapping
baml_language/crates/baml_cli/src/paint.rs
Semantic token types and modifiers now map to named terminal styles, while plain text uses a modifier-free styling path.
Compiler token rendering
baml_language/crates/baml_cli/src/paint.rs
Synthesized fragments are classified through a scratch ProjectDatabase, and fragments and source ranges share overlap-aware semantic-token rendering.
Highlighting behavior validation
baml_language/crates/baml_cli/src/paint.rs
Tests cover neutral variable styling, modifier attributes, compiler classification, terminal colors, default-library italics, and unresolved names.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Painter
  participant ProjectDatabase
  participant styled_from_tokens
  Painter->>ProjectDatabase: Parse synthesized fragment and request semantic_tokens
  ProjectDatabase-->>Painter: Return sorted tokens and modifiers
  Painter->>styled_from_tokens: Apply token styles and render plain gaps
  styled_from_tokens-->>Painter: Return styled fragment
Loading

Possibly related PRs

Poem

I hop through tokens, bright and neat,
With bold little ears and italic feet.
Deprecated marks swing through the night,
Plain gaps stay calm, unresolved names light.
The compiler paints each trail just right.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately reflects the CLI highlighting refactor toward theme-aware, LSP semantic-token-based rendering.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch avery/describe-paint

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@vercel
vercel Bot temporarily deployed to Preview – promptfiddle2 July 14, 2026 17:44 Inactive
@github-actions

github-actions Bot commented Jul 14, 2026

Copy link
Copy Markdown

Binary size checks passed

7 passed

Artifact Platform File Gzip Gated on Baseline Delta Status
baml-cli Linux 🔒 22.1 MB 9.4 MB file 22.1 MB +2.1 KB (+0.0%) OK
packed-program Linux 🔒 15.8 MB 6.7 MB file 15.8 MB +0 B (+0.0%) OK
baml-cli macOS 🔒 17.0 MB 8.2 MB file 17.0 MB +0 B (+0.0%) OK
packed-program macOS 🔒 12.2 MB 5.9 MB file 12.2 MB +0 B (+0.0%) OK
baml-cli Windows 🔒 18.5 MB 8.4 MB file 18.5 MB -512 B (-0.0%) OK
packed-program Windows 🔒 13.1 MB 6.0 MB file 13.1 MB +0 B (+0.0%) OK
bridge_wasm WASM 14.6 MB 🔒 4.1 MB gzip 4.1 MB +1 B (+0.0%) OK

🔒 = the size this artifact is GATED on (ceiling + delta). Binaries gate on file size (installed binary); WASM gates on gzip (download size). The other size is shown for information only.


Generated by cargo size-gate · workflow run

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@baml_language/crates/baml_cli/src/paint.rs`:
- Around line 100-140: Update tests covering style_for to validate rendered
styling behavior rather than searching output for individual SGR fragments such
as bold or color escapes. Account for console::Style combining multiple
attributes into one escape sequence, while preserving coverage for declaration,
color, italic, and strikethrough behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 788b25d5-7602-4854-b3f1-5c503a65a634

📥 Commits

Reviewing files that changed from the base of the PR and between 4a8ed19 and 2da5b9b.

📒 Files selected for processing (1)
  • baml_language/crates/baml_cli/src/paint.rs

Comment thread baml_language/crates/baml_cli/src/paint.rs
CodeRabbit review on #4017: console::Style may batch attributes into one
escape sequence (\x1b[1;33m vs \x1b[1m\x1b[33m), so substring checks on
individual fragments could reject valid output. The paint tests now decode
SGR parameter lists into effects (attribute / palette-indexed / truecolor)
and assert on the effects active at a given rendered name, which is
encoding-shape independent. The stricter assertions also surfaced that
constructor-position names classify syntactically even when unresolved, so
the neutrality test now pins that behavior and checks true neutrality on a
member of an unresolvable receiver instead.
@vercel
vercel Bot temporarily deployed to Preview – promptfiddle July 14, 2026 18:03 Inactive
@vercel
vercel Bot temporarily deployed to Preview – beps July 14, 2026 18:04 Inactive
@vercel
vercel Bot temporarily deployed to Preview – promptfiddle2 July 14, 2026 18:11 Inactive
@codeshaunted
codeshaunted enabled auto-merge July 14, 2026 18:12
@codeshaunted
codeshaunted added this pull request to the merge queue Jul 14, 2026
Merged via the queue into canary with commit 7a4e540 Jul 14, 2026
54 checks passed
@codeshaunted
codeshaunted deleted the avery/describe-paint branch July 14, 2026 18:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant