macros can observe raw identifier state [discuss]

[nikomatsakis]

In #48942, @Lymia landed raw identifiers; "equality" in this implementation includes observing the "raw" state. Therefore, macros can observe if r#foo was used or not:

#![feature(raw_identifiers)]

macro_rules! foo {
    (a) => {
        1
    };
    
    (r#a) => { 2 };
}

fn main() {
    println!("{}", foo!(a));
    println!("{}", foo!(r#a));
}

This makes a certain measure of sense, but also makes me nervous, and I wanted to open an issue to discuss. For example, @Manishearth proposed making identifiers "raw" in macros when they are serialized, thus ensuring consistent interpretation as we cross editions.

I'm not entirely sure of the implications of this but it does make me nervous for users to be able to distinguish foo and r#foo.

[petrochenkov]

Thanks for creating this issue, this is an interesting case.

[petrochenkov]

My reasoning (#48942 (comment)) when merging this behavior was that identifiers on the left side of a macro are keywords in the grammar introduced by that macro (each macro introduces a new grammar and doesn't care about keywords defined by the general language), so raw identifiers provide opt-out from those keywords in the same way like they provide opt-out for the large grammar of the whole Rust language.

"Raw keywords" introduced by macros (r#a in the example above) are especially interesting - we don't have an analogue to them in the general Rust grammar. Maybe it makes sense to prohibit them on macros' left side, at least initially.

[petrochenkov]

One example:

macro my_macro {
    (struct { $i: ident }) => {}
    (construct { $i: ident }) => {}
}

From my_macro's point of view struct and construct are treated equivalently, regardless of their keyword status in Rust because it's irrelevant - my_macro just tries to match tokens provided in my_macro!(....) to tokens written in its LHS.
If this token matching ignores the rawness property, then my_macro should accept both r#struct { A } and r#construct { A }, if matching doesn't ignore the rawness property, then it should only accept struct { A } and construct { A }.

[nikomatsakis]

@petrochenkov yeah, that makes a lot of sense, and I'm coming around to this POV

"Raw keywords" introduced by macros (r#a in the example above) are especially interesting - we don't have an analogue to them in the general Rust grammar. Maybe it makes sense to prohibit them on macros' left side, at least initially.

I agree with this. This would resolve a lot of my concerns, I think.

[glmdgrielson]

Wait, it feels a lot more natural to simply not care. Like you said the grammar doesn't care about keywords or any other bit of syntax:

/// I should be able to do this because it's a macro
macro_rules! blank {
  () => { ;}
}

So this shouldn't make any difference.