This commit does the following.
- Renames `collect_tokens_trailing_token` as `collect_tokens`, because
(a) it's annoying long, and (b) the `_trailing_token` bit is less
accurate now that its types have changed.
- In `collect_tokens`, adds a `Option<CollectPos>` argument and a
`UsePreAttrPos` in the return type of `f`. These are used in
`parse_expr_force_collect` (for vanilla expressions) and in
`parse_stmt_without_recovery` (for two different cases of expression
statements). Together these ensure are enough to fix all the problems
with token collection and assoc expressions. The changes to the
`stringify.rs` test demonstrate some of these.
- Adds a new test. The code in this test was causing an assertion
failure prior to this commit, due to an invalid `NodeRange`.
The extra complexity is annoying, but necessary to fix the existing
problems.
`missing_fragment_specifier` has been a future compatibility warning
since 2017. Uplifting it to an unconditional hard error was attempted in
2020, but eventually reverted due to fallout.
Make it an error only in edition >= 2024, leaving the lint for older
editions. This change will make it easier to support more macro syntax
that relies on usage of `$`.
Fixes <https://github.com/rust-lang/rust/issues/40107>
Do not use question as label
We don't want to have questions in the diagnostic output. Instead, we use wording that communicates uncertainty, like "might":
```
error[E0432]: unresolved import `spam`
--> $DIR/import-from-missing-star-3.rs:2:9
|
LL | use spam::*;
| ^^^^ you might be missing crate `spam`
|
= help: consider adding `extern crate spam` to use the `spam` crate
```
We don't want to have questions in the diagnostic output. Instead, we use wording that communicates uncertainty, like "might":
```
error[E0432]: unresolved import `spam`
--> $DIR/import-from-missing-star-3.rs:2:9
|
LL | use spam::*;
| ^^^^ you might be missing crate `spam`
|
= help: consider adding `extern crate spam` to use the `spam` crate
```
We are moving toward forbidding `missing_fragment_specifier` either in
edition 2024 or unconditionally. Make a first step toward this by
ensuring crates that rely on the old behavior are reported when used as
dependencies.
Tracking issue: <https://github.com/rust-lang/rust/issues/128143>
When finding item gated behind a `cfg` flag, point at it
Previously we would only mention that the item was gated out, and opportunisitically mention the feature flag name when possible. We now point to the place where the item was gated, which can be behind layers of macro indirection, or in different modules.
```
error[E0433]: failed to resolve: could not find `doesnt_exist` in `inner`
--> $DIR/diagnostics-cross-crate.rs:18:23
|
LL | cfged_out::inner::doesnt_exist::hello();
| ^^^^^^^^^^^^ could not find `doesnt_exist` in `inner`
|
note: found an item that was configured out
--> $DIR/auxiliary/cfged_out.rs:6:13
|
LL | pub mod doesnt_exist {
| ^^^^^^^^^^^^
note: the item is gated here
--> $DIR/auxiliary/cfged_out.rs:5:5
|
LL | #[cfg(FALSE)]
| ^^^^^^^^^^^^^
```
Previously we would only mention that the item was gated out, and opportunisitically mention the feature flag name when possible. We now point to the place where the item was gated, which can be behind layers of macro indirection, or in different modules.
```
error[E0433]: failed to resolve: could not find `doesnt_exist` in `inner`
--> $DIR/diagnostics-cross-crate.rs:18:23
|
LL | cfged_out::inner::doesnt_exist::hello();
| ^^^^^^^^^^^^ could not find `doesnt_exist` in `inner`
|
note: found an item that was configured out
--> $DIR/auxiliary/cfged_out.rs:6:13
|
LL | pub mod doesnt_exist {
| ^^^^^^^^^^^^
note: the item is gated here
--> $DIR/auxiliary/cfged_out.rs:5:5
|
LL | #[cfg(FALSE)]
| ^^^^^^^^^^^^^
```
Go over all structured parser suggestions and make them verbose style.
When suggesting to add or remove delimiters, turn them into multiple suggestion parts.
[`macro_metavar_expr_concat`] Add support for literals
Adds support for things like `${concat($variable, 123)}` or `${concat("hello", "_world")}` .
cc #124225
Remove `__rust_force_expr`.
This was added (with a different name) to improve an error message. It is no longer needed -- removing it changes the error message, but overall I think the new message is no worse:
- the mention of `#` in the first line is a little worse,
- but the extra context makes it very clear what the problem is, perhaps even clearer than the old message,
- and the removal of the note about the `expr` fragment (an internal detail of `__rust_force_expr`) is an improvement.
Overall I think the error is quite clear and still far better than the old message that prompted #61933, which didn't even mention patterns.
The motivation for this is #124141, which will cause pasted metavariables to be tokenized and reparsed instead of the AST node being cached. This change in behaviour occasionally has a non-zero perf cost, and `__rust_force_expr` causes the tokenize/reparse step to occur twice. Removing `__rust_force_expr` greatly reduces the extra overhead for the `deep-vector` benchmark.
r? ```@oli-obk```
This was added (with a different name) to improve an error message. It
is no longer needed -- removing it changes the error message, but overall
I think the new message is no worse:
- the mention of `#` in the first line is a little worse,
- but the extra context makes it very clear what the problem is, perhaps
even clearer than the old message,
- and the removal of the note about the `expr` fragment (an internal
detail of `__rust_force_expr`) is an improvement.
Overall I think the error is quite clear and still far better than the
old message that prompted #61933, which didn't even mention patterns.
The motivation for this is #124141, which will cause pasted
metavariables to be tokenized and reparsed instead of the AST node being
cached. This change in behaviour occasionally has a non-zero perf cost,
and `__rust_force_expr` causes the tokenize/reparse step to occur twice.
Removing `__rust_force_expr` greatly reduces the extra overhead for the
`deep-vector` benchmark.
Print `token::Interpolated` with token stream pretty printing.
This is a step towards removing `token::Interpolated` (#124141). It unavoidably changes the output of the `stringify!` macro, generally for the better.
r? `@petrochenkov`
Remove ignore-cross-compile directive from ui/macros/proc_macro
All the other proc-macro tests don't have this, presumably this was forgotten when the restriction got lifted as it does test just fine
r? `@pietroalbini`
Spruce up the diagnostics of some early lints
Implement the various "*(note to myself) in a follow-up PR we should turn parts of this message into a subdiagnostic (help msg or even struct sugg)*" drive-by comments I left in #124417 during my review.
For context, before #124417, only a few early lints touched/decorated/customized their diagnostic because the former API made it a bit awkward. Likely because of that, things that should've been subdiagnostics were just crammed into the primary message. This PR rectifies this.
Instead of using AST pretty printing.
This is a step towards removing `token::Interpolated`, which will
eventually (in #124141) be replaced with a token stream within invisible
delimiters.
This changes (improves) the output of the `stringify!` macro in some
cases. This is allowed. As the `stringify!` docs say: "Note that the
expanded results of the input tokens may change in the future. You
should be careful if you rely on the output."
Test changes:
- tests/ui/macros/stringify.rs: this used to test both token stream
pretty printing and AST pretty printing via different ways of invoking
of `stringify!` (i.e. `$expr` vs `$tt`). But those two different
invocations now give the same result, which is a nice consistency
improvement. This removes the need for all the `c2*` macros. The AST
pretty printer now has more thorough testing thanks to #125236.
- tests/ui/proc-macro/*: minor improvements where small differences
between `INPUT (DISPLAY)` output and `DEEP-RE-COLLECTED (DISPLAY)`
output disappear.
Add tests for `-Zunpretty=expanded` ported from stringify's tests
This PR adds a new set of tests for the AST pretty-printer.
Previously, pretty-printer edge cases were tested by way of `stringify!` in [tests/ui/macros/stringify.rs](https://github.com/rust-lang/rust/blob/1.78.0/tests/ui/macros/stringify.rs), such as the tests added by 419b26931b and 527e2eac17.
Those tests will no longer provide effective coverage of the AST pretty-printer after #124141. `Nonterminal` and `TokenKind::Interpolated` are being removed, and a consequence is that `stringify!` will perform token stream pretty printing, instead of AST pretty printing, in all of the `stringify!` cases including $:expr and all other interpolations.
This PR adds 2 new ui tests with `compile-flags: -Zunpretty=expanded`:
- **tests/ui/unpretty/expanded-exhaustive.rs** — this test aims for exhaustive coverage of all the variants of `ExprKind`, `ItemKind`, `PatKind`, `StmtKind`, `TyKind`, and `VisibilityKind`. Some parts could use being fleshed out further, but the current state is roughly on par with what exists in the old stringify-based tests.
- **tests/ui/unpretty/expanded-interpolation.rs** — this test covers tricky macro metavariable edge cases that require the AST pretty printer to synthesize parentheses in order for the printed code to be valid Rust syntax.
r? `@nnethercote`
Fix println! ICE when parsing percent prefix number
This PR fixes#125002 ICE occurring, for example, with `println!("%100000", 1)` or `println!("% 100000", 1)`.
## Test Case/Change Explanation
The return type of `Num::from_str` has been changed to `Option<Self>` to handle errors when parsing large integers fails.
1. The first `println!` in the test case covers the change of the first `Num::from_str` usage in `format_foreign.rs:426`.
2. The second `println!` in the test case covers the change of the second `Num::from_str` usage in line 460.
3. The 3rd to 5th `Num::from_str` usages behave the same as before.
The 3rd usage would cause an ICE when `num > u16::MAX` in the previous version, but this commit does not include a fix for the ICE in `println!("{:100000$}")`. I think we need to emit an error in the compiler and have more discussion in another issue/PR.
Update `expr` matcher for Edition 2024 and add `expr_2021` nonterminal
This commit adds a new nonterminal `expr_2021` in macro patterns, and `expr_fragment_specifier_2024` feature flag.
This change also updates `expr` so that on Edition 2024 it will also match `const { ... }` blocks, while `expr_2021` preserves the current behavior of `expr`, matching expressions without `const` blocks.
Joint work with `@vincenzopalazzo.`
Issue #123742
Rename `${length()}` to `${len()}`
Implements the rename suggested in https://github.com/rust-lang/rust/pull/122808#issuecomment-2047722187
> I brought this up in the doc PR but it belongs here – `length` should probably be renamed `len` before stabilization. The latter is de facto standard in the standard library, whereas the former is only used in a single unstable API. These metafunctions aren’t library items of course, but should presumably still be consistent with established names.
r? `@c410-f3r`
Ralf Jung