1773f60ea5
rustc_ast: Visit tokens stored in AST nodes in mutable visitor After #77271 token visiting is enabled only for one visitor in `rustc_expand\src\mbe\transcribe.rs` which applies hygiene marks to tokens produced by declarative macros (`macro_rules` or `macro`), so this change doesn't affect anything else. When a macro has some interpolated token from an outer macro in its output ```rust macro inner() { $interpolated } ``` we can use the usual interpretation of interpolated tokens in token-based model - a None-delimited group - to write this macro in an equivalent form ```rust macro inner() { ⟪ a b c d ⟫ } ``` When we are expanding the macro `inner` we need to apply hygiene marks to all tokens produced by it, including the tokens inside the group. Before this PR we did this by visiting the AST piece inside the interpolated token and applying marks to all spans in it. I'm not sure this is 100% correct (ideally we should apply the marks to tokens and then re-parse the AST from tokens), but it's a very good approximation at least. We didn't however apply the marks to actual tokens stored in the nonterminal, so if we used the nonterminal as a token rather than as an AST piece (e.g. passed it to a proc macro), then we got hygiene bugs. This PR applies the marks to tokens in addition to the AST pieces thus fixing the issue. r? `@Aaron1011` |
||
|---|---|---|
| .. | ||
| src | ||
| Cargo.toml | ||
| README.md | ||
The rustc_ast crate contains those things concerned purely with syntax
– that is, the AST ("abstract syntax tree"), parser, pretty-printer,
lexer, macro expander, and utilities for traversing ASTs.
For more information about how these things work in rustc, see the rustc dev guide: