mirror of https://github.com/rust-lang/rust.git
925f7fad57
`tokenstream::Spacing` appears on all `TokenTree::Token` instances, both punct and non-punct. Its current usage: - `Joint` means "can join with the next token *and* that token is a punct". - `Alone` means "cannot join with the next token *or* can join with the next token but that token is not a punct". The fact that `Alone` is used for two different cases is awkward. This commit augments `tokenstream::Spacing` with a new variant `JointHidden`, resulting in: - `Joint` means "can join with the next token *and* that token is a punct". - `JointHidden` means "can join with the next token *and* that token is a not a punct". - `Alone` means "cannot join with the next token". This *drastically* improves the output of `print_tts`. For example, this: ``` stringify!(let a: Vec<u32> = vec![];) ``` currently produces this string: ``` let a : Vec < u32 > = vec! [] ; ``` With this PR, it now produces this string: ``` let a: Vec<u32> = vec![] ; ``` (The space after the `]` is because `TokenTree::Delimited` currently doesn't have spacing information. The subsequent commit fixes this.) The new `print_tts` doesn't replicate original code perfectly. E.g. multiple space characters will be condensed into a single space character. But it's much improved. `print_tts` still produces the old, uglier output for code produced by proc macros. Because we have to translate the generated code from `proc_macro::Spacing` to the more expressive `token::Spacing`, which results in too much `proc_macro::Along` usage and no `proc_macro::JointHidden` usage. So `space_between` still exists and is used by `print_tts` in conjunction with the `Spacing` field. This change will also help with the removal of `Token::Interpolated`. Currently interpolated tokens are pretty-printed nicely via AST pretty printing. `Token::Interpolated` removal will mean they get printed with `print_tts`. Without this change, that would result in much uglier output for code produced by decl macro expansions. With this change, AST pretty printing and `print_tts` produce similar results. The commit also tweaks the comments on `proc_macro::Spacing`. In particular, it refers to "compound tokens" rather than "multi-char operators" because lifetimes aren't operators. |
||
|---|---|---|
| .. | ||
| auxiliary | ||
| asm.pp | ||
| asm.rs | ||
| ast-stmt-expr-attr.rs | ||
| async.rs | ||
| attr-derive.rs | ||
| attr-fn-inner.rs | ||
| attr-literals.rs | ||
| attr-tokens-raw-ident.rs | ||
| auto-trait.rs | ||
| blank-lines.rs | ||
| block-comment-multiple-asterisks.rs | ||
| block-comment-trailing-whitespace.rs | ||
| block-comment-trailing-whitespace2.rs | ||
| block-comment-wchar.pp | ||
| block-comment-wchar.rs | ||
| block-disambig.rs | ||
| cast-lt.pp | ||
| cast-lt.rs | ||
| closure-reform-pretty.rs | ||
| delimited-token-groups.rs | ||
| disamb-stmt-expr.rs | ||
| do1.rs | ||
| doc-comments.rs | ||
| dollar-crate.pp | ||
| dollar-crate.rs | ||
| empty-impl.rs | ||
| empty-lines.rs | ||
| enum-variant-vis.rs | ||
| example1.rs | ||
| example2.pp | ||
| example2.rs | ||
| expanded-and-path-remap-80832.pp | ||
| expanded-and-path-remap-80832.rs | ||
| fn-return.rs | ||
| fn-types.rs | ||
| fn-variadic.rs | ||
| for-comment.rs | ||
| format-args-str-escape.pp | ||
| format-args-str-escape.rs | ||
| gat-bounds.rs | ||
| hir-fn-variadic.pp | ||
| hir-fn-variadic.rs | ||
| hir-pretty-loop.pp | ||
| hir-pretty-loop.rs | ||
| if-attr.rs | ||
| import-renames.rs | ||
| issue-4264.pp | ||
| issue-4264.rs | ||
| issue-12590-a.rs | ||
| issue-12590-b.rs | ||
| issue-12590-c.pp | ||
| issue-12590-c.rs | ||
| issue-19077.rs | ||
| issue-25031.rs | ||
| issue-30731.rs | ||
| issue-31073.pp | ||
| issue-31073.rs | ||
| issue-68710-field-attr-proc-mac-lost.rs | ||
| issue-73626.rs | ||
| issue-74745.rs | ||
| issue-85089.pp | ||
| issue-85089.rs | ||
| let.rs | ||
| lifetime.rs | ||
| macro.rs | ||
| macro_rules.rs | ||
| match-block-expr.rs | ||
| match-naked-expr-medium.rs | ||
| match-naked-expr.rs | ||
| nested-item-vis-defaultness.rs | ||
| offset_of.rs | ||
| path-type-bounds.rs | ||
| qpath-associated-type-bound.rs | ||
| raw-address-of.rs | ||
| raw-str-nonexpr.rs | ||
| stmt_expr_attributes.rs | ||
| struct-pattern.rs | ||
| struct-tuple.rs | ||
| tag-blank-lines.rs | ||
| tests-are-sorted.pp | ||
| tests-are-sorted.rs | ||
| top-level-doc-comments.rs | ||
| trait-inner-attr.rs | ||
| trait-polarity.rs | ||
| trait-safety.rs | ||
| unary-op-disambig.rs | ||
| use-tree.rs | ||
| vec-comments.pp | ||
| vec-comments.rs | ||
| where-clauses.rs | ||
| yeet-expr.rs | ||