feat: replace syntect with arborium for syntax highlighting - #1
Merged
Conversation
Swap syntect and two-face for arborium's tree-sitter grammars, using the structured span API rather than its HTML or ANSI renderers. This drops both RUSTSEC ignores from deny.toml: yaml-rust and bincode reached the tree only through syntect. Rust is the only grammar enabled. Rustdoc code blocks are overwhelmingly Rust, and each extra grammar costs binary size; other languages now render as plain text.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces
syntect+two-facewitharborium's tree-sitter grammars, using its structured span API (highlight_spans+spans_to_flat_tokens) rather than its HTML or ANSI renderers.Why
deny.toml.yaml-rust(RUSTSEC-2024-0320) andbincode(RUSTSEC-2025-0141) reached the tree only through syntect.cargo deny checknow reportsadvisories ok, bans ok, licenses ok, sources okwith no ignores and no warnings.page.rsre-highlights on every render and resize.Trade-off
Only the Rust grammar is compiled in. Rustdoc code blocks are overwhelmingly Rust, and each additional grammar costs binary size. Other fences (
toml,json,sh,c,cpp,py) now render as plain text instead of highlighted.code_language()is unchanged, so re-enabling any of them is a feature flag plus one line in the language mapping.Release binary grows 1.4MB → 7.6MB; locked packages drop 208 → 200.
Implementation notes
Highlighterwrapsarborium::Highlighterin aRefCell— arborium needs&mutto parse, but rendering only ever holds&Highlighter. This keepshighlight(&self, ..)intact, sopage.rs/app.rs/ui.rsare untouched.Raw arborium spans are unordered and overlapping, so the conversion has to: flatten them into a disjoint token stream (
spans_to_flat_tokens, innermost match wins), emit the untagged gaps between tokens, and split tokens that straddle newlines.The theme moves from syntect's
base16-eighties.darkto arborium'smonokai.Testing
62 tests pass, including the existing
method_docs_are_rendered_and_highlighted. Four new tests cover the risk areas of the span conversion:clippy and
cargo fmt --checkare clean.🤖 Generated with Claude Code