The pretty printer algorithm involves 2 VecDeques: a ring-buffer of
tokens and a deque of ring-buffer indices. Confusingly, those two deques
were being grown in opposite directions for no good reason. Ring-buffer
pushes would go on the "back" of the ring-buffer (i.e. higher indices)
while scan_stack pushes would go on the "front" (i.e. lower indices).
This commit flips the scan_stack accesses to grow the scan_stack and
ring-buffer in the same direction, where push does the same
operation as a Vec push i.e. inserting on the high-index end.
Pretty printer algorithm revamp step 2
This PR follows #92923 as a second chunk of modernizations backported from https://github.com/dtolnay/prettyplease into rustc_ast_pretty.
I've broken this up into atomic commits that hopefully are sensible in isolation. At every commit, the pretty printer is compilable and has runtime behavior that is identical to before and after the PR. None of the refactoring so far changes behavior.
The general theme of this chunk of commits is: the logic in the old pretty printer is doing some very basic things (pushing and popping tokens on a ring buffer) but expressed in a too-low-level way that I found makes it quite complicated/subtle to reason about. There are a number of obvious invariants that are "almost true" -- things like `self.left == self.buf.offset` and `self.right == self.buf.offset + self.buf.data.len()` and `self.right_total == self.left_total + self.buf.data.sum()`. The reason these things are "almost true" is the implementation tends to put updating one side of the invariant unreasonably far apart from updating the other side, leaving the invariant broken while unrelated stuff happens in between. The following code from master is an example of this:
e5e2b0be26/compiler/rustc_ast_pretty/src/pp.rs (L314-L317)
In this code the `advance_right` is reserving an entry into which to write a next token on the right side of the ring buffer, the `check_stack` is doing something totally unrelated to the right boundary of the ring buffer, and the `scan_push` is actually writing the token we previously reserved space for. Much of what this PR is doing is rearranging code to shrink the amount of stuff in between when an invariant is broken to when it is restored, until the whole thing can be factored out into one indivisible method call on the RingBuffer type.
The end state of the PR is that we can entirely eliminate `self.left` (because it's now just equal to `self.buf.offset` always) and `self.right` (because it's equal to `self.buf.offset + self.buf.data.len()` always) and the whole `Token::Eof` state which used to be the value of tokens that have been reserved space for but not yet written.
I found without these changes the pretty printer implementation to be hard to reason about and I wasn't able to confidently introduce improvements like trailing commas in `prettyplease` until after this refactor. The logic here is 43 years old at this point (Graydon translated it as directly as possible from the 1979 pretty printing paper) and while there are advantages to following the paper as closely as possible, in `prettyplease` I decided if we're going to adapt the algorithm to work better for Rust syntax, it was worthwhile making it easier to follow than the original.
Add manifest docs fallback.
This adds a fallback so that the rustup manifest will contain the rust-docs component for all hosts. There is a mapping so that the docs that get downloaded are roughly close to the actual host. There inevitably will be things that don't match. Ideally the standard library docs would be the same for every platform (`cfg(doc)` goes a long way towards this), but there are still lots of minor differences.
Closes#69525
Change PhantomData type for `BuildHasherDefault` (and more)
Changes `PhantomData<H>` to `PhantomData<fn() -> H>` for `BuildHasherDefault`. This preserves the covariance of `H`, while it lifts the currently inferred unnecessary bounds like [`H: Send` for `BuildHasherDefault<H>: Send`](https://doc.rust-lang.org/1.57.0/std/hash/struct.BuildHasherDefault.html#impl-Send), etc.
_Edit:_ Also does a similar change for `iter::Empty` and `future::Pending`.
mangling_v0: Skip extern blocks during mangling
There's no need to include the dummy `Nt` into the symbol name, items in extern blocks belong to their parent modules for all purposes except for inheriting the ABI and attributes.
Follow up to https://github.com/rust-lang/rust/pull/92032
(There's also a drive-by fix to the `rust-demangler` tool's tests, which don't run on CI, I initially attempted using them for testing this PR.)
Rollup of 10 pull requests
Successful merges:
- #88642 (Formally implement let chains)
- #89621 (doc: guarantee call order for sort_by_cached_key)
- #91278 (Use iterator instead of recursion in `codegen_place`)
- #92124 (Little improves in CString `new` when creating from slice)
- #92783 (Annotate dead code lint with notes about ignored derived impls)
- #92797 (Remove horizontal lines at top of page)
- #92920 (Move expr- and item-related pretty printing functions to modules)
- #93041 (Remove some unused ordering derivations based on `DefId`)
- #93051 (Add Option::is_some_with and Result::is_{ok,err}_with)
- #93062 (Update books)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
Update books
## nomicon
1 commits in c05c452b36358821bf4122f9c418674edd1d713d..66d097d3d80e8f88c288c6879c7c2b909ecf8ad4
2021-12-13 15:23:48 +0900 to 2022-01-05 05:45:21 +0900
- Fix typo / type error in FFI code example (rust-lang/nomicon#327)
## reference
8 commits in f8ba2f12df60ee19b96de24ae5b73af3de8a446b..4dee6eb63d728ffb9e7a2ed443e9ada9275c69d2
2022-01-03 11:02:08 -0800 to 2022-01-18 09:26:33 -0800
- (minor) Remove Expression Path sub-types splits in Pattern specs (rust-lang/reference#1138)
- Document destructuring assignment (rust-lang/reference#1116)
- Document the 2021 edition changes to macros-by-example `pat` metavariables (rust-lang/reference#1135)
- Improve the documentation of macros-by-example metavariable names (rust-lang/reference#1130)
- trait-bounds.md: add pronoun 'that' (rust-lang/reference#1131)
- Say that macros-by-example `ident` metavariables can match raw identifiers (rust-lang/reference#1133)
- State in the UAX31 profile description that a lone `_` is not an identifier (rust-lang/reference#1129)
- Document syntax reserved in Rust 2021 (rust-lang/reference#1128)
## book
17 commits in d3740fb7aad0ea4a80ae20f64dee3a8cfc0c5c3c..f17df27fc14696912c48b8b7a7a8fa49e648088d
2022-01-03 21:46:04 -0500 to 2022-01-18 17:46:28 -0500
- Add a notice to the top of all nostarch snapshots
- Fix quotes
- Grammar (minor): 'or' → 'and' for enum variants
- Propagate edits of chapter 8 to src
- Replies to nostarch edits
- more edits
- ch8 from nostarch
- Fix grammar and line wrapping
- Merge remote-tracking branch 'origin/pr/2880'
- Remove wikipedia link
- Merge remote-tracking branch 'origin/pr/2927'
- Snapshot of ch14 for nostarch
- Backport fixes to chapter 14 noticed while doing nostarch snapshot
- Fix usage of find piped into xargs
- Adjust some more line numbers of Cargo.toml includes
- Merge branch '2909'
- Merge remote-tracking branch 'parkerziegler/fix/ch14-add-one-naming'
## rustc-dev-guide
7 commits in 875464457c4104686faf667f47848aa7b0f0a744..78dd6a4684cf8d6b72275fab6d0429ea40b66338
2021-12-28 22:17:49 -0600 to 2022-01-18 14:44:26 -0300
- Reorganize and expand the testing chapters. (rust-lang/rustc-dev-guide#1281)
- Add inline assembly internals (rust-lang/rustc-dev-guide#1266)
- Spelling: Rename `rust` to `Rust` (rust-lang/rustc-dev-guide#1288)
- Clean up section about FCPs (rust-lang/rustc-dev-guide#1287)
- Address more review comments in rust-lang/rustc-dev-guide#1286.
- Address review comments in rust-lang/rustc-dev-guide#1286.
- Streamline "Getting Started" some more.
Remove some unused ordering derivations based on `DefId`
Like #93018, this removes some unused/unneeded ordering derivations as part of ongoing work on #90317. Here, these changes are aimed at making https://github.com/rust-lang/rust/pull/90749 easier to review, test, and merge.
r? `@cjgillot`
Move expr- and item-related pretty printing functions to modules
Currently *compiler/rustc_ast_pretty/src/pprust/state.rs* is 2976 lines on master. The `tidy` limit is 3000, which is blocking #92243.
This PR adds a `mod expr;` and `mod item;` to move logic related to those AST nodes out of the single huge file.
Little improves in CString `new` when creating from slice
Old code already contain optimization for cases with `&str` and `&[u8]` args. This commit adds a specialization for `&mut[u8]` too.
Also, I added usage of old slice in search for zero bytes instead of new buffer because it produce better code for constant inputs on Windows LTO builds. For other platforms, this wouldn't cause any difference because it calls `libc` anyway.
Inlined `_new` method into spec trait to reduce amount of code generated to `CString::new` callers.
Use iterator instead of recursion in `codegen_place`
This PR fixes the FIXME in `codegen_place` about using iterator instead of recursion when processing the `projection` field in `mir::PlaceRef`. At the same time, it also reduces the right drift.
doc: guarantee call order for sort_by_cached_key
`slice::sort_by_cached_key` takes a caching function `f: impl FnMut(&T) -> K`, which means that the order that calls to the caching function are made is user-visible. This adds a clause to the documentation to promise the current behavior, which is that `f` is called on all elements of the slice from left to right, unless the slice has len < 2 in which case `f` is not called.
For example, this can be used to ensure that the following code is a correct way to involve the index of the element in the sort key:
```rust
let mut index = 0;
slice.sort_by_cached_key(|x| (my_key(index, x), index += 1).0);
```
Formally implement let chains
## Let chains
My longest and hardest contribution since #64010.
Thanks to `@Centril` for creating the RFC and special thanks to `@matthewjasper` for helping me since the beginning of this journey. In fact, `@matthewjasper` did much of the complicated MIR stuff so it's true to say that this feature wouldn't be possible without him. Thanks again `@matthewjasper!`
With the changes proposed in this PR, it will be possible to chain let expressions along side local variable declarations or ordinary conditional expressions. In other words, do much of what the `if_chain` crate already does.
## Other considerations
* `if let guard` and `let ... else` features need special care and should be handled in a following PR.
* Irrefutable patterns are allowed within a let chain context
* ~~Three Clippy lints were already converted to start dogfooding and help detect possible corner cases~~
cc #53667
David Tolnay