Skip to content

feat: replace syntect with arborium for syntax highlighting - #1

Merged
abusch merged 1 commit into
mainfrom
replace-syntect-with-arborium
Aug 26, 2026
Merged

feat: replace syntect with arborium for syntax highlighting#1
abusch merged 1 commit into
mainfrom
replace-syntect-with-arborium

Conversation

@abusch

@abusch abusch commented Aug 25, 2026

Copy link
Copy Markdown
Owner

Replaces syntect + two-face with arborium's tree-sitter grammars, using its structured span API (highlight_spans + spans_to_flat_tokens) rather than its HTML or ANSI renderers.

Why

  • Drops both RUSTSEC ignores from deny.toml. yaml-rust (RUSTSEC-2024-0320) and bincode (RUSTSEC-2025-0141) reached the tree only through syntect. cargo deny check now reports advisories ok, bans ok, licenses ok, sources ok with no ignores and no warnings.
  • ~4x faster highlighting: 107µs/block vs 410µs measured on the same input. This matters because page.rs re-highlights on every render and resize.
  • Faster startup: grammars compile lazily on first use, replacing syntect's eager ~4.3ms syntax-set load.

Trade-off

Only the Rust grammar is compiled in. Rustdoc code blocks are overwhelmingly Rust, and each additional grammar costs binary size. Other fences (toml, json, sh, c, cpp, py) now render as plain text instead of highlighted.

code_language() is unchanged, so re-enabling any of them is a feature flag plus one line in the language mapping.

Release binary grows 1.4MB → 7.6MB; locked packages drop 208 → 200.

Implementation notes

Highlighter wraps arborium::Highlighter in a RefCell — arborium needs &mut to parse, but rendering only ever holds &Highlighter. This keeps highlight(&self, ..) intact, so page.rs/app.rs/ui.rs are untouched.

Raw arborium spans are unordered and overlapping, so the conversion has to: flatten them into a disjoint token stream (spans_to_flat_tokens, innermost match wins), emit the untagged gaps between tokens, and split tokens that straddle newlines.

The theme moves from syntect's base16-eighties.dark to arborium's monokai.

Testing

62 tests pass, including the existing method_docs_are_rendered_and_highlighted. Four new tests cover the risk areas of the span conversion:

  • source text is preserved exactly through highlighting (guards the gap/newline handling)
  • Rust code actually receives colour
  • non-Rust falls back to plain text
  • empty code blocks don't panic

clippy and cargo fmt --check are clean.

🤖 Generated with Claude Code

Swap syntect and two-face for arborium's tree-sitter grammars, using the
structured span API rather than its HTML or ANSI renderers.

This drops both RUSTSEC ignores from deny.toml: yaml-rust and bincode
reached the tree only through syntect.

Rust is the only grammar enabled. Rustdoc code blocks are overwhelmingly
Rust, and each extra grammar costs binary size; other languages now render
as plain text.
@abusch
abusch merged commit 349a886 into main Aug 26, 2026
5 checks passed
@abusch
abusch deleted the replace-syntect-with-arborium branch August 26, 2026 00:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant