Make trivia first class member of token - #219
Conversation
|
|
||
| #[derive(Clone, Copy, PartialEq, Eq, Hash, Debug)] | ||
| pub enum TriviaPieceKind { | ||
| Newline, |
There was a problem hiding this comment.
It is not the job of rowan to define this. Trivias should just have a SyntaxKind. From rowan's point of view, they're tokens attached to tokens. The rest will be handled by rust-analyzer. This will also require fewer changes.
| self.raw.text() | ||
| } | ||
|
|
||
| pub fn text_trimmed(&self) -> &str { |
There was a problem hiding this comment.
I believe text() and text_range() should exclude trivia, and we should have a text_including_trivia() method (and ditto for text_range()). While I don't know for sure, code in r-a today does not handle trivia so I expect this to be easier.
There was a problem hiding this comment.
I also think it'll be easier if trivias will store their text (like regular tokens).
There was a problem hiding this comment.
Yup, having both text and text_range with including trivia variants.
|
|
||
| let start: usize = leading_len.into(); | ||
| let end: usize = (total_len - trailing_len).into(); | ||
| let text = unsafe { std::str::from_utf8_unchecked(self.data.slice()) }; |
There was a problem hiding this comment.
| let text = unsafe { std::str::from_utf8_unchecked(self.data.slice()) }; | |
| let text = self.text(); |
Save the unsafe.
| @@ -0,0 +1,134 @@ | |||
| use std::{ | |||
There was a problem hiding this comment.
I'm not pleased with the amount of public functions here. We're going to remove green trees so we need to minimize their usage. Things that do not need to be public should remain private.
There was a problem hiding this comment.
Ah!! sorry, lazy me. Should be good now.
| } | ||
|
|
||
| impl<L: Language> SyntaxToken<L> { | ||
| pub fn new_detached<Leading, Trailing>( |
There was a problem hiding this comment.
Why do we need this? A token cannot be a root, this feels wrong.
There was a problem hiding this comment.
Removed. I was copy pasting from biome back then.
| // endregion | ||
|
|
||
| #[derive(PartialEq, Eq, Clone, Hash)] | ||
| pub struct SyntaxTrivia { |
There was a problem hiding this comment.
I don't think we need this type. We can just have fn leading_trivia(&self) -> impl Iterator<Item = SyntaxToken> om SyntaxToken, and ditto for trailing_trivia(). Just store a list (even Box<[GreenToken]>) of leading trivia in a token, and ditto for trailing trivia.
There was a problem hiding this comment.
In particular, using regular tokens for trivias simplifies both the implementation and the changes in r-a.
There was a problem hiding this comment.
Its much better. Again, took inspiration from biome initially. Removed.
ebcb103 to
c97bf3d
Compare
c97bf3d to
7e9daab
Compare
2 participants
Still making changes to RA