Experimental feature postfix match
This has a basic experimental implementation for the RFC postfix match (rust-lang/rfcs#3295, #121618). [Liaison is](https://rust-lang.zulipchat.com/#narrow/stream/213817-t-lang/topic/Postfix.20Match.20Liaison/near/423301844) ```@scottmcm``` with the lang team's [experimental feature gate process](https://github.com/rust-lang/lang-team/blob/master/src/how_to/experiment.md).
This feature has had an RFC for a while, and there has been discussion on it for a while. It would probably be valuable to see it out in the field rather than continue discussing it. This feature also allows to see how popular postfix expressions like this are for the postfix macros RFC, as those will take more time to implement.
It is entirely implemented in the parser, so it should be relatively easy to remove if needed.
This PR is split in to 5 commits to ease review.
1. The implementation of the feature & gating.
2. Add a MatchKind field, fix uses, fix pretty.
3. Basic rustfmt impl, as rustfmt crashes upon seeing this syntax without a fix.
4. Add new MatchSource to HIR for Clippy & other HIR consumers
Add asm goto support to `asm!`
Tracking issue: #119364
This PR implements asm-goto support, using the syntax described in "future possibilities" section of [RFC2873](https://rust-lang.github.io/rfcs/2873-inline-asm.html#asm-goto).
Currently I have only implemented the `label` part, not the `fallthrough` part (i.e. fallthrough is implicit). This doesn't reduce the expressive though, since you can use label-break to get arbitrary control flow or simply set a value and rely on jump threading optimisation to get the desired control flow. I can add that later if deemed necessary.
r? ``@Amanieu``
cc ``@ojeda``
Add an ErrorGuaranteed to ast::TyKind::Err (attempt 2)
This makes it more like `hir::TyKind::Err`, and avoids a `has_errors` assertion in `LoweringContext::lower_ty_direct`.
r? ```@oli-obk```
This mostly works well, and eliminates a couple of delayed bugs.
One annoying thing is that we should really also add an
`ErrorGuaranteed` to `proc_macro::bridge::LitKind::Err`. But that's
difficult because `proc_macro` doesn't have access to `ErrorGuaranteed`,
so we have to fake it.
This makes it more like `hir::TyKind::Err`, and avoids a
`span_delayed_bug` call in `LoweringContext::lower_ty_direct`.
It also requires adding `ast::TyKind::Dummy`, now that
`ast::TyKind::Err` can't be used for that purpose in the absence of an
error emission.
There are a couple of cases that aren't as neat as I would have liked,
marked with `FIXME` comments.
Invert diagnostic lints.
That is, change `diagnostic_outside_of_impl` and `untranslatable_diagnostic` from `allow` to `deny`, because more than half of the compiler has been converted to use translated diagnostics.
This commit removes more `deny` attributes than it adds `allow` attributes, which proves that this change is warranted.
r? ````@davidtwco````
That is, change `diagnostic_outside_of_impl` and
`untranslatable_diagnostic` from `allow` to `deny`, because more than
half of the compiler has be converted to use translated diagnostics.
This commit removes more `deny` attributes than it adds `allow`
attributes, which proves that this change is warranted.
Further improve `space_between`
`space_between` is used by `print_tts` to decide when spaces should be put between tokens. This PR improves it in two ways:
- avoid unnecessary spaces before semicolons, and
- don't omit some necessary spaces before/after some punctuation symbols.
r? `@petrochenkov`
There are a number of cases where we erroneously omit the space between
two tokens, all involving an exception to a more general case. The
affected tokens are `$`, `!`, `.`, `,`, and `let` followed by a
parenthesis.
This fixes a lot of FIXME comments.
Fix parenthesization of subexprs containing statement boundary
This PR fixes a multitude of false negatives and false positives in the AST pretty printer's parenthesis insertion related to statement boundaries — statements which terminate unexpectedly early if there aren't parentheses.
Without this fix, the AST pretty printer (including both `stringify!` and `rustc -Zunpretty=expanded`) is prone to producing output which is not syntactically valid Rust. Invalid output is problematic because it means Rustfmt is unable to parse the output of `cargo expand`, for example, causing friction by forcing someone trying to debug a macro into reading poorly formatted code.
I believe the set of bugs fixed in this PR account for the most prevalent reason that `cargo expand` produces invalid output in real-world usage.
Fixes#98790.
## False negatives
The following is a correct program — `cargo check` succeeds.
```rust
macro_rules! m {
($e:expr) => {
match () { _ => $e }
};
}
fn main() {
m!({ 1 } - 1);
}
```
But `rustc -Zunpretty=expanded main.rs` produces output that is invalid Rust syntax, because parenthesization is needed and not being done by the pretty printer.
```rust
fn main() { match () { _ => { 1 } - 1, }; }
```
Piping this expanded code to rustfmt, it fails to parse.
```console
error: unexpected `,` in pattern
--> <stdin>:1:38
|
1 | fn main() { match () { _ => { 1 } - 1, }; }
| ^
|
help: try adding parentheses to match on a tuple...
|
1 | fn main() { match () { _ => { 1 } (- 1,) }; }
| + +
help: ...or a vertical bar to match on multiple alternatives
|
1 | fn main() { match () { _ => { 1 } - 1 | }; }
| ~~~~~
```
Fixed output after this PR:
```rust
fn main() { match () { _ => ({ 1 }) - 1, }; }
```
## False positives
Less problematic, but worth fixing (just like #118726).
```rust
fn main() {
let _ = match () { _ => 1 } - 1;
}
```
Output of `rustc -Zunpretty=expanded lib.rs` before this PR. There is no reason parentheses would need to be inserted there.
```rust
fn main() { let _ = (match () { _ => 1, }) - 1; }
```
After this PR:
```rust
fn main() { let _ = match () { _ => 1, } - 1; }
```
## Alternatives considered
In this PR I opted to parenthesize only the leading subexpression causing the statement boundary, rather than the entire statement. Example:
```rust
macro_rules! m {
($e:expr) => {
$e
};
}
fn main() {
m!(loop { break [1]; }[0] - 1);
}
```
This PR produces the following pretty-printed contents for fn main:
```rust
(loop { break [1]; })[0] - 1;
```
A different equally correct output would be:
```rust
(loop { break [1]; }[0] - 1);
```
I chose the one I did because it is the *only* approach used by handwritten code in the standard library and compiler. There are 4 places where parenthesization is being used to prevent a statement boundary, and in all 4, the developer has chosen to parenthesize the smallest subexpression rather than the whole statement:
b37d43efd9/compiler/rustc_codegen_cranelift/example/alloc_system.rs (L102)b37d43efd9/compiler/rustc_parse/src/errors.rs (L1021-L1029)b37d43efd9/library/core/src/future/poll_fn.rs (L151)b37d43efd9/library/core/src/ops/range.rs (L824-L828)
Clairify `ast::PatKind::Struct` presese of `..` by using an enum instead of a bool
The bool is mainly used for when a `..` is present, but it is also set on recovery to avoid errors. The doc comment not describes both of these cases.
See cee794ee98/compiler/rustc_parse/src/parser/pat.rs (L890-L897) for the only place this is constructed.
r? ``@compiler-errors``
Rid the AST & HIR pretty printer of cruft
Found while working on #119163.
For `trait Trait: ?Sized {}` (semantically malformed), we currently output `trait Trait for ? Sized {}` (sic!) / `trait Trait for ? Sized { }` (sic!) if `-Zunpretty=expanded` / `-Zunpretty=hir` is passed.
`trait Tr for Sized? {}` (#15521) and later also `trait Tr for ?Sized {}` (I guess, #20194) is former Rust syntax. Hence I'm removing these outdated branches.
~~This will conflict with #119163, therefore marking this PR as blocked.~~ Rebased
Add support for `for await` loops
This adds support for `for await` loops. This includes parsing, desugaring in AST->HIR lowering, and adding some support functions to the library.
Given a loop like:
```rust
for await i in iter {
...
}
```
this is desugared to something like:
```rust
let mut iter = iter.into_async_iter();
while let Some(i) = loop {
match core::pin::Pin::new(&mut iter).poll_next(cx) {
Poll::Ready(i) => break i,
Poll::Pending => yield,
}
} {
...
}
```
This PR also adds a basic `IntoAsyncIterator` trait. This is partly for symmetry with the way `Iterator` and `IntoIterator` work. The other reason is that for async iterators it's helpful to have a place apart from the data structure being iterated over to store state. `IntoAsyncIterator` gives us a good place to do this.
I've gated this feature behind `async_for_loop` and opened #118898 as the feature tracking issue.
r? `@compiler-errors`
Refactor AST trait bound modifiers
Instead of having two types to represent trait bound modifiers in the parser / the AST (`parser::ty::BoundModifiers` & `ast::TraitBoundModifier`), only to map one to the other later, just use `parser::ty::BoundModifiers` (moved & renamed to `ast::TraitBoundModifiers`).
The struct type is more extensible and easier to deal with (see [here](https://github.com/rust-lang/rust/pull/119099/files#r1430749981) and [here](https://github.com/rust-lang/rust/pull/119099/files#r1430752116) for context) since it more closely models what it represents: A compound of two kinds of modifiers, constness and polarity. Modeling this as an enum (the now removed `ast::TraitBoundModifier`) meant one had to add a new variant per *combination* of modifier kind, which simply isn't scalable and which lead to a lot of explicit non-DRY matches.
NB: `hir::TraitBoundModifier` being an enum is fine since HIR doesn't need to worry representing invalid modifier kind combinations as those get rejected during AST validation thereby immensely cutting down the number of possibilities.
Do not parenthesize exterior struct lit inside match guards
Before this PR, the AST pretty-printer injects parentheses around expressions any time parens _could_ be needed depending on what else is in the code that surrounds that expression. But the pretty-printer did not pass around enough context to understand whether parentheses really _are_ needed on any particular expression. As a consequence, there are false positives where unneeded parentheses are being inserted.
Example:
```rust
#![feature(if_let_guard)]
macro_rules! pp {
($e:expr) => {
stringify!($e)
};
}
fn main() {
println!("{}", pp!(match () { () if let _ = Struct {} => {} }));
}
```
**Before:**
```console
match () { () if let _ = (Struct {}) => {} }
```
**After:**
```console
match () { () if let _ = Struct {} => {} }
```
This PR introduces a bit of state that is passed across various expression printing methods to help understand accurately whether particular situations require parentheses injected by the pretty printer, and it fixes one such false positive involving match guards as shown above.
There are other parenthesization false positive cases not fixed by this PR. I intend to address these in follow-up PRs. For example here is one: the expression `{ let _ = match x {} + 1; }` is pretty-printed as `{ let _ = (match x {}) + 1; }` despite there being no reason for parentheses to appear there.
This is an extension of the previous commit. It means the output of
something like this:
```
stringify!(let a: Vec<u32> = vec![];)
```
goes from this:
```
let a: Vec<u32> = vec![] ;
```
With this PR, it now produces this string:
```
let a: Vec<u32> = vec![];
```
`tokenstream::Spacing` appears on all `TokenTree::Token` instances,
both punct and non-punct. Its current usage:
- `Joint` means "can join with the next token *and* that token is a
punct".
- `Alone` means "cannot join with the next token *or* can join with the
next token but that token is not a punct".
The fact that `Alone` is used for two different cases is awkward.
This commit augments `tokenstream::Spacing` with a new variant
`JointHidden`, resulting in:
- `Joint` means "can join with the next token *and* that token is a
punct".
- `JointHidden` means "can join with the next token *and* that token is a
not a punct".
- `Alone` means "cannot join with the next token".
This *drastically* improves the output of `print_tts`. For example,
this:
```
stringify!(let a: Vec<u32> = vec![];)
```
currently produces this string:
```
let a : Vec < u32 > = vec! [] ;
```
With this PR, it now produces this string:
```
let a: Vec<u32> = vec![] ;
```
(The space after the `]` is because `TokenTree::Delimited` currently
doesn't have spacing information. The subsequent commit fixes this.)
The new `print_tts` doesn't replicate original code perfectly. E.g.
multiple space characters will be condensed into a single space
character. But it's much improved.
`print_tts` still produces the old, uglier output for code produced by
proc macros. Because we have to translate the generated code from
`proc_macro::Spacing` to the more expressive `token::Spacing`, which
results in too much `proc_macro::Along` usage and no
`proc_macro::JointHidden` usage. So `space_between` still exists and
is used by `print_tts` in conjunction with the `Spacing` field.
This change will also help with the removal of `Token::Interpolated`.
Currently interpolated tokens are pretty-printed nicely via AST pretty
printing. `Token::Interpolated` removal will mean they get printed with
`print_tts`. Without this change, that would result in much uglier
output for code produced by decl macro expansions. With this change, AST
pretty printing and `print_tts` produce similar results.
The commit also tweaks the comments on `proc_macro::Spacing`. In
particular, it refers to "compound tokens" rather than "multi-char
operators" because lifetimes aren't operators.
tidy error: /git/rust/compiler/rustc_ast_pretty/src/pprust/state.rs:1165: unexplained "```ignore" doctest; try one:
* make the test actually pass, by adding necessary imports and declarations, or
* use "```text", if the code is not Rust code, or
* use "```compile_fail,Ennnn", if the code is expected to fail at compile time, or
* use "```should_panic", if the code is expected to fail at run time, or
* use "```no_run", if the code should type-check but not necessary linkable/runnable, or
* explain it like "```ignore (cannot-test-this-because-xxxx)", if the annotation cannot be avoided.
tidy error: /git/rust/compiler/rustc_ast_pretty/src/pprust/state.rs:1176: unexplained "```ignore" doctest; try one:
* make the test actually pass, by adding necessary imports and declarations, or
* use "```text", if the code is not Rust code, or
* use "```compile_fail,Ennnn", if the code is expected to fail at compile time, or
* use "```should_panic", if the code is expected to fail at run time, or
* use "```no_run", if the code should type-check but not necessary linkable/runnable, or
* explain it like "```ignore (cannot-test-this-because-xxxx)", if the annotation cannot be avoided.
In all four of Break, Closure, Ret, Yeet, the needs_par_as_let_scrutinee
is guaranteed to return true because the .precedence().order() of those
expr kinds is <= AssocOp::LAnd.precedence().
The relevant functions in rustc_ast::util::parser are:
fn needs_par_as_let_scrutinee(order: i8) -> bool {
order <= prec_let_scrutinee_needs_par() as i8
}
fn prec_let_scrutinee_needs_par() -> usize {
AssocOp::LAnd.precedence()
}
The .precedence().order() of Closure is PREC_CLOSURE (-40) and of Break,
Ret, Yeet is PREC_JUMP (-30).
The value of AssocOp::LAnd.precedence() is 6.
So this commit causes no change in behavior, only potentially
performance by doing a redundant call to contains_exterior_struct_lit in
those four cases. This is fine because Break, Closure, Ret, Yeet should
be exceedingly rare in the position of a let scrutinee.
Introduce support for `async gen` blocks
I'm delighted to demonstrate that `async gen` block are not very difficult to support. They're simply coroutines that yield `Poll<Option<T>>` and return `()`.
**This PR is WIP and in draft mode for now** -- I'm mostly putting it up to show folks that it's possible. This PR needs a lang-team experiment associated with it or possible an RFC, since I don't think it falls under the jurisdiction of the `gen` RFC that was recently authored by oli (https://github.com/rust-lang/rfcs/pull/3513, https://github.com/rust-lang/rust/issues/117078).
### Technical note on the pre-generator-transform yield type:
The reason that the underlying coroutines yield `Poll<Option<T>>` and not `Poll<T>` (which would make more sense, IMO, for the pre-transformed coroutine), is because the `TransformVisitor` that is used to turn coroutines into built-in state machine functions would have to destructure and reconstruct the latter into the former, which requires at least inserting a new basic block (for a `switchInt` terminator, to match on the `Poll` discriminant).
This does mean that the desugaring (at the `rustc_ast_lowering` level) of `async gen` blocks is a bit more involved. However, since we already need to intercept both `.await` and `yield` operators, I don't consider it much of a technical burden.
r? `@ghost`
Matthias Krüger