This shortens VtPipeTerm quite a bit, which used to have various debug
flags and modes. I kept the `--out` flag to redirect the output to a
file, but I removed the `--debug` (pipe the output through WSL and
show escape sequences visually) and `--headless` (hide conpty) flags.
I did this, because VtPipeTerm always used the system ConPTY API
but I needed it to use my local OpenConsole. I also wanted it to
use overlapped IO for testing but found that it was too difficult
to refactor make that work.
I also noticed that the project was the only holdout for
`conpty.h` which had to be kept in sync with `winconpty.h`.
This adds logic to get the DA1 report from the hosting terminal on
startup. We then use the information to figure out if it supports
rectangular area operations. If so, we can use DECCRA/DECFRA to
implement ScrollConsoleScreenBuffer.
This additionally changes `ScrollConsoleScreenBuffer` to always
forbid control characters as the fill character, even in conhost
(via `VtIo::SanitizeUCS2`). My hope is that this makes the API
more consistent and robust as it avoids another source for
invisible control characters in the text buffer.
Part of #17643
## Validation Steps Performed
* New tests ✅
Adds functionality throughout the settings model to keep track of which
settings have been set.
There are two entry points:
- AppLogic.cpp: this is where we perform a settings reload by loading
the JSON
- MainPage.cpp: this is where the Save button is clicked in the settings
UI
Both of these entry points call into
`CascadiaSettings::LogSettingChanges()` where we aggregate the list of
changes (specifically, _which_ settings changed, not _what_ their value
is).
Just about all of the settings model objects now have a
`LogSettingChanges(std::set& changes, std::string_view context)` on
them.
- `changes` is where we aggregate all of the changes to. In it being a
set, we don't need to worry about duplicates and can do things like
iterate across all of the profiles.
- `context` prepends a string to the setting. This'll allow us to better
identify where a setting was changes (i.e. "global.X" are global
settings). We also use this to distinguish between settings set in the
~base layer~ profile defaults vs individual profiles.
The change log in each object is modified via two ways:
- `LayerJson()` changes: this is useful for detecting JSON changes! All
we're doing is checking if the setting has a value (due to inheritance,
just about everything is an optional here!). If the value is set, we add
the json key to the change log
- `INHERITABLE_SETTING_WITH_LOGGING` in IInheritable.h: we already use
this macro to define getters and setters. This new macro updates the
setter to check if the value was set to something different. If so, log
it!
Other notes:
- We're not distinguishing between `defaultAppearance` and
`unfocusedAppearance`
- We are distinguishing between `profileDefaults` and `profile` (any
other profile)
- New Tab Menu Customization:
- we really just care about the entry types. Handled in
`GlobalAppSettings`
- Font:
- We still have support for legacy values here. We still want to track
them, but just use the modern keys.
- `Theme`:
- We don't do inheritance here, so we have to approach it differently.
During the JSON load, we log each setting. However, we don't have
`LayerJson`! So instead, do the work in `CascadiaSettings` and store the
changes there. Note that we don't track any changes made via setters.
This is fine for now since themes aren't even in the settings UI, so we
wouldn't get much use out of it anyways.
- Actions:
- Actions are weird because we can have nested and iterable actions too,
but `ActionsAndArgs` as a whole add a ton of functionality. I handled it
over in `Command::LogSettingChanges` and we generally just serialize it
to JSON to get the keys. It's a lot easier than dealing with the object
model.
Epic: #10000
Auto-Save (ish): #12424
This pull request adds support for querying all of the "dynamic
resource" colors (foreground, background, cursor) as well as the entire
color palette using OSC 4, 10, 11 and 12 with the `?` color specifier.
To ease integration and to make it easier to extend later, I have
consolidated `SetDefaultForeground`, `SetDefaultBackground` and
`SetCursorColor` into one function `SetXtermColorResource`, plus its
analog `RequestXtermColorResource`.
Those functions will map xterm resource OSC numbers to color table
entries and optionally color _alias_ entries using a constant table. The
alias mappings are required to support reassigning the default
foreground and background to their indexed entries after a `DECAC`.
While there are only three real entries in the mapping table right now,
I have designs on bringing in selection background (xterm "highlight")
and foreground (xterm "highlightText").
We can also extend this to support resetting via OSC 110-119. However,
at the adapter layer we do not have the requisite information to restore
any of the colors (even the cursor color!) to the user's defaults.
`OSC 10` and `OSC 11` queries report the final values of
`DECAC`-reassigned entries, under the assumption that an application
asking for them wants to make a determination regardless of their
internal meaning to us (that is: they read through the aliased color to
its final destination in the color palette.)
I've tested this with lsix, which detects the background color before
generating sixel previews. It works great!
ConPTY does not currently pass OSC sequences received on the input
handle, so work was required to make it do so.
Closes#3718
Between fmt 7.1.3 and 11.0.2 a lot has happened. `wchar_t` support is
now more limited and implicit conversions don't work anymore.
Furthermore, even the non-`FMT_COMPILE` API is now compile-time checked
and so it fails to work in our UI code which passes `hstring` format
strings which aren't implicitly convertible to the expected type.
`fmt::runtime` was introduced for this but it also fails to work for
`hstring` parameters. To solve this, a new `RS_fmt` macro was added
to abstract the added `std::wstring_view` casting away.
Finally, some additional changes to reduce `stringstream` usage
have been made, whenever `format_to`, etc., is available.
This mostly affects `ActionArgs.cpp`.
Closes#16000
## Validation Steps Performed
* Compiles ✅
* Settings page opens ✅
The answerback feature allows for the user to define a message that the
terminal will transmit to the host whenever an `ENQ` (enquiry) control
character is received.
## Detailed Description of the Pull Request / Additional comments
In Windows Terminal, the message can be configured at the profile level
of the settings file, as a string property named `AnswerbackMessage`.
In ConHost, the message can be configured in the registry, again as a
string value with the name `AnswerbackMessage`.
## Validation Steps Performed
I've confirmed that the control is working as intended in both Windows
Terminal and ConHost using Vttest.
Closes#11946
The idea is that we can translate Console API calls directly to VT at
least as well as the current VtEngine setup can. For instance, a call
to `SetConsoleCursorPosition` clearly translates directly to a `CUP`
escape sequence. Effectively, instead of translating output
asynchronously in the renderer thread, we'll do it synchronously
right during the Console API call.
Most importantly, the this means that any VT output that an
application generates will now be given to the terminal unmodified.
Aside from reducing our project's complexity quite a bit and opening
the path towards various interesting work like sixels, Device Control
Strings, buffer snapshotting, synchronized updates, and more, it also
improves performance for mixed text output like enwik8.txt in conhost
to 1.3-2x and in Windows Terminal via ConPTY to roughly 20x.
This adds support for overlapped IO, because now that output cannot
be "skipped" anymore (VtEngine worked like a renderer after all)
it's become crucial to block conhost as little as possible.
⚠️ Intentionally unresolved changes/quirks:
* To force a delayed EOL wrap to wrap, `WriteCharsLegacy` emits a
`\r\n` if necessary. This breaks text reflow on window resize.
We cannot emit ` \r` the way readline does it, because this would
overwrite the first column in the next row with a whitespace.
The alternative is to read back the affected cell from the buffer
and emit that character and its attributes followed by a `\r`.
I chose to not do that, because buffer read-back is lossy (= UCS2).
Unless the window is resized, the difference is unnoticeable
and historically, conhost had no support for buffer reflow anyway.
* If `ENABLE_VIRTUAL_TERMINAL_PROCESSING` is set while
`DISABLE_NEWLINE_AUTO_RETURN` is reset, we'll blindly replace all
LF with CRLF. This may hypothetically break DCS sequences, but it's
the only way to do this without parsing the given VT string and
thus the only way we can achieve passthrough mode in the future.
* `ENABLE_WRAP_AT_EOL_OUTPUT` is translated to `DECAWM`.
Between Windows XP and Windows 11 21H2, `ENABLE_WRAP_AT_EOL_OUTPUT`
being reset would cause the cursor position to reset to wherever
a write started, _if_ the write, including expanded control chars,
was less than 100 characters long. If it was longer than that,
the cursor position would end up in an effectively random position.
After lengthy research I believe that this is a bug introduced in
Windows XP and that the original intention was for this mode to be
equivalent to `DECAWM`. This is compounded by MSDN's description
(emphasis mine):
> If this mode is disabled, the **last character** in the row is
> overwritten with any subsequent characters.
⚠️ Unresolved issues/quirks:
* Focus/Unfocus events are injected into the output stream without
checking whether the VT output is currently in a ground state.
This may break whatever VT sequence is currently ongoing.
This is an existing issue.
* `VtIo::Writer::WriteInfos` should properly verify the width of
each individual character.
* Using `SetConsoleActiveScreenBuffer` destroys surrogate pairs
and extended (VT) attributes. It could be translated to VT pages
in the long term.
* Similarly, `ScrollConsoleScreenBuffer` results in the same and
could be translated to `DECCRA` and `DECFRA` in the near term.
This is important because otherwise `vim` output may loose
its extended attributes during scrolling.
* Reflowing a long line until it wraps results in the cooked read
prompt to be misaligned vertically.
* `SCREEN_INFORMATION::s_RemoveScreenBuffer` should trigger a
buffer switch similar to `SetConsoleActiveScreenBuffer`.
* Translation of `COMMON_LVB_GRID_HORIZONTAL` to `SGR 53` was dropped
and may be reintroduced alongside `UNDERSCORE` = `SGR 4`.
* Move the `OSC 0 ; P t BEL` sequence to `WriteWindowTitle`
and swap the `BEL` with the `ST` (`ESC \`).
* PowerShell on Windows 10 ships with PSReadLine 2.0.0-beta2
which emits SGR 37/40 instead of 39/49. This results in black
spaces when typing and there's no good way to fix that.
* A test is missing that ensures that `FillConsoleOutputCharacterW`
results in a `CSI n J` during the PowerShell shim.
* A test is missing that ensures that `PtySignal::ClearBuffer`
does not result in any VT being generated.
Closes#262Closes#1173Closes#3016Closes#4129Closes#5228Closes#8698Closes#12336Closes#15014Closes#15888Closes#16461Closes#16911Closes#17151Closes#17313
Removes the GitHub action that provides the functionality for the
similar issues bot prototype. We can onboard to the more official
prototype instead to conserve functionality.
This pull request removes the following vendored open source, in favor
of getting it from vcpkg:
- CLI11 2.4
- jsoncpp 1.9
- fmt 7.1.3
- gsl 3.1 (not vendored, but submoduled--arguably worse!)
Now that Visual Studio 2022 includes a built-in workload for vcpkg, the
onboarding process is much smoother. Terminal should only require the
vcpkg workload.
I've added some build rules that detect vcpkg via VS and via the user's
environment before falling back to a location in the source tree. The CI
pipeline will fall back to installing and bootstrapping vcpkg in
dep/vcpkg if necessary.
Some OSS has not been (and will not be) migrated:
- wyhash: ours is included directly in til/hash
- pcg_random: we have a stripped down copy compared to vcpkg
- stb_rect: vcpkg only ships *all of STB*; ours is a stripped down copy
- chromium numerics: vcpkg does not ship Chromium, especially not this
tiny fraction of Chromium
- dynamic_bitset and libpopcnt: removing in #17510
- interval_tree: no vcpkg equivalent
To support the needs of the inbox Windows build, I've split up our vcpkg
manifest into dependencies for all projects and dependencies just for
Terminal. To support this, we now offer a `terminal` feature. The vcpkg
rules in `common.build.pre.props` are set up to turn it on, whereas the
build rules we eventually write for the OS will not be.
Most of the work is concentrated in `common.build.pre.props`.
Split off from #17510:
* `HandleWantsOverlappedIo` can be used to check if a handle requires
overlapped IO. This is important, as `ReadFile` and `WriteFile` are
documented to not work correctly if an overlapped handle is used
without overlapped IO and vice versa.
In my tests with pipes, this appears to be true.
* `CreatePipe` creates a synchronous, unidirectional pipe.
* `CreateOverlappedPipe` does what it says on the tin, while allowing
you to specify the direction of the pipe (in, out, duplex).
* `GetOverlappedResultSameThread` is largely the same as
`GetOverlappedResult`, but adds back a neat optimization from
the time before Windows 7. I thought it was neat.
This also updates the localization pipeline to check in translations for
the PDPs.
Right now, the primary source for PDPs is the Terminal.Internal
repository. They are submitted from there, and pulled back in as though
they were destined for the internal repo. We rename them on disk prior
to loc check-in to pretend they live in this repo.
Once I submit a change request to the Touchdown team to update the paths
in their backend, I will follow up with another pull request that
updates the remaining build steps to account for that.
## Summary of the Pull Request
This PR introduces basic support for the Sixel graphics protocol in
conhost, limited to the GDI renderer.
## References and Relevant Issues
This is a first step towards supporting Sixel graphics in Windows
Terminal (#448), but that will first require us to have some form of
ConPTY passthrough (#1173).
## Detailed Description of the Pull Request / Additional comments
There are three main parts to the architecture:
* The `SixelParser` class takes care of parsing the incoming Sixel `DCS`
sequence.
* The resulting image content is stored in the text buffer in a series
of `ImageSlice` objects, which represent per-row image content.
* The renderer then takes care of painting those image slices for each
affected row.
The parser is designed to support multiple conformance levels so we can
one day provide strict compatibility with the original DEC hardware. But
for now the default behavior is intended to work with more modern Sixel
applications. This is essentially the equivalent of a VT340 with 256
colors, so it should still work reasonably well as a VT340 emulator too.
## Validation Steps Performed
Thanks to the work of @hackerb9, who has done extensive testing on a
real VT340, we now have a fairly good understanding of how the original
Sixel hardware terminals worked, and I've tried to make sure that our
implementation matches that behavior as closely as possible.
I've also done some testing with modern Sixel libraries like notcurses
and jexer, but those typically rely on the terminal implementing certain
proprietary Xterm query sequences which I haven't included in this PR.
---------
Co-authored-by: Dustin L. Howett <dustin@howett.net>
First, this adds `GraphemeTableGen` which
* parses `ucd.nounihan.grouped.xml`
* computes the cluster break property for each codepoint
* computes the East Asian Width property for each codepoint
* compresses everything into a 4-stage trie
* computes a LUT of cluster break rules between 2 codepoints
* and serializes everything to C++ tables and helper functions
Next, this adds `GraphemeTestTableGen` which
* parses `GraphemeBreakTest.txt`
* splits each test into graphemes and break opportunities
* and serializes everything to a C++ table for use as unit tests
`CodepointWidthDetector.cpp` was rewritten from scratch to
* use an iterator struct (`GraphemeState`) to maintain state
* accumulate codepoints until a break opportunity arises
* accumulate the total width of a grapheme
* support 3 different measurement modes: Grapheme clusters,
`wcswidth`-style, and a mode identical to the old conhost
With this in place the following changes were made:
* `ROW::WriteHelper::_replaceTextUnicode` now uses the new
grapheme cluster text iterators
* The same function was modified to join new text with existing
contents of the current cell if they join to form a cluster
* Otherwise, a ton of places were modified to funnel the selection
of the measurement mode over from WT's settings to ConPTY
This is part of #1472
## Validation Steps Performed
* So many tests ✅
* https://github.com/apparebit/demicode works fantastic ✅
* UTF8-torture-test.txt works fantastic ✅
This specs out a lot of plans for snippets. We've already got these in
the sxnui as "tasks", but we can do so very much more.
This spec is a few years old now, but it's time for it to get promoted
out of my draft branch.
References:
* #1595
* #7039
* #3121
* #10436
* #12927
* #12857
* #5790
* #15845
---------
Co-authored-by: Dustin L. Howett <duhowett@microsoft.com>
I'm planning to use the `dark2` color palette in the upcoming
cooked read rewrite as a debug aid to paint dirty regions.
Now that it's going to be used in more than one place I figured
it may be time to properly add it to the NOTICE file even if
it still won't be shipped with the final product.
Quick Fix will be a new UI surface that allows the user to interact with
the terminal and leverage the context of a specific command being
executed. This new UI surface will be a home for features like WinGet
Command Not Found.
#16599
With the move to Action IDs, it doesn't quite make sense anymore for a
`Command` to know which keys map to it. This PR removes all `Keys` from
`Command`, and any callers to that now instead query the `ActionMap` for
that Command's keys.
Closes#17160Closes#13943
I was talking with @plante-msft this week at Build and we agreed that
.75 is just a bit too chatty. .8 seems like it's a better threshold -
sure, it'll miss a few of the harder edge cases, but it'll chime in less
frequently when it's just wrong.
This PR adds support for multiples pages in the VT architecture, along
with new operations for moving between those pages: `NP` (Next Page),
`PP` (Preceding Page), `PPA` (Page Position Absolute), `PPR` (Page
Position Relative), and `PPB` (Page Position Back).
There's also a new mode, `DECPCCM` (Page Cursor Coupling Mode), which
determines whether or not the active page is also the visible page, and
a new query sequence, `DECRQDE` (Request Displayed Extent), which can be
used to query the visible page.
## References and Relevant Issues
When combined with `DECCRA` (Copy Rectangular Area), which can copy
between pages, you can layer content on top of existing output, and
still restore the original data afterwards. So this could serve as an
alternative solution to #10810.
## Detailed Description of the Pull Request / Additional comments
On the original DEC terminals that supported paging, you couldn't have
both paging and scrollback at the same time - only the one or the other.
But modern terminals typically allow both, so we support that too.
The way it works, the currently visible page will be attached to the
scrollback, and any content that scrolls off the top will thus be saved.
But the background pages will not have scrollback, so their content is
lost if it scrolls off the top.
And when the screen is resized, only the visible page will be reflowed.
Background pages are not affected by a resize until they become active.
At that point they just receive the traditional style of resize, where
the content is clipped or padded to match the new dimensions.
I'm not sure this is the best way to handle resizing, but we can always
consider other approaches once people have had a chance to try it out.
## Validation Steps Performed
I've added some unit tests covering the new operations, and also done a
lot of manual testing.
Closes#13892
Tests added/passed
Something something code changes, fixes issue. Events need throwing,
some events don't need throwing, some events need throttling to
not overwhelm the flurble function within the gumbies component.
This is all boilerplate and it works now.
Closes#17171Closes#17172
## Validation Steps Performed
* Resetting the font axis/feature expander keeps
the add new button flyout sorted ✅
* Changing a font axis/feature value updates the preview ✅
* After changing the font, features/axes can still be edited ✅
This required me to push a bunch more parameters through the build
pipeline, but it gave me the opportunity to define them as variables
that can be set at queue time.
This adds a system message which displays the time at which the
buffer snapshot was written to disk.
Additionally, this PR moves the snapshot loading into a background
thread, so that the UI thread is unblocked and that multiple
tabs/panes can load simultaneously.
Closes#17031Closes#17074
## Validation Steps Performed
Repeatedly closing and opening WT adds more and more messages.
Currently, the messages get somewhat corrupted due to a bug
in our line-wrap handling, or some similar part.
Next in the popular series of minor refactorings:
Out with the old, in with the new!
This PR removes all of the existing TSF code, both for conhost and
Windows Terminal. conhost's TSF implementation was awful:
It allocated an entire text buffer _per line_ of input.
Additionally, its implementation spanned a whopping 40 files and
almost 5000 lines of code. Windows Terminal's implementation was
absolutely fine in comparison, but it was user unfriendly due to
two reasons: Its usage of the `CoreTextServices` WinRT API indirectly
meant that it used a non-transitory TSF document, which is not the
right choice for a terminal. A `TF_SS_TRANSITORY` document (-context)
indicates to TSF that it cannot undo a previously completed composition
which is exactly what we need: Once composition has completed we send
the result to the shell and we cannot undo this later on.
The WinRT API does not allow us to use `TF_SS_TRANSITORY` and so it's
unsuitable for our application. Additionally, the implementation used
XAML to render the composition instead of being part of our text
renderer, which resulted in the text looking weird and hard to read.
The new implementation spans just 8 files and is ~1000 lines which
should make it significantly easier to maintain. The architecture is
not particularly great, but it's certainly better than what we had.
The implementation is almost entirely identical between both conhost
and Windows Terminal and thus they both also behave identical.
It fixes an uncountable number of subtle bugs in the conhost TSF
implementation, as it failed to check for status codes after calls.
It also adds several new features, like support for wavy underlines
(as used by the Japanese IME), dashed underlines (the default for
various languages now, like Vietnamese), colored underlines,
colored foreground/background controlled by the IME, and more!
I have tried to replicate the following issues and have a high
confidence that they're resolved now:
Closes#1304Closes#3730Closes#4052Closes#5007 (as it is not applicable anymore)
Closes#5110Closes#6186Closes#6192Closes#13805Closes#14349Closes#14407Closes#16180
For the following issues I'm not entirely sure if it'll fix it,
but I suspect it's somewhat likely:
#13681#16305#16817
Lastly, there's one remaining bug that I don't know how to resolve.
However, that issue also plagues conhost and Windows Terminal
right now, so it's at least not a regression:
* Press Win+. (emoji picker) and close it
* Move the window around
* Press Win+.
This will open the emoji picker at the old window location.
It also occurs when the cursor moves within the window.
While this is super annoying, I could not find a way to fix it.
## Validation Steps Performed
* See the above closed issues
* Use Vietnamese Telex and type "xin choaf"
Results in "xin chào" ✅
* Use the MS Japanese IME and press Alt+`
Toggles between the last 2 modes ✅
* Use the MS Japanese IME, type "kyouhaishaheiku", and press Space
* The text is converted, underlined and the first part is
doubly underlined ✅
* Left/Right moves between the 3 segments ✅
* Home/End moves between start/end ✅
* Esc puts a wavy line under the current segment ✅
* Use the Korean IME, type "gksgks"
This results in "한한" ✅
* Use the Korean IME, type "gks", and press Right Ctrl
Opens a popup which allows you to navigate with Arrow/Tab keys ✅
This gets rid off the implicit dependency on `IsGlyphFullWidth`
for the IRM and DECSWL/DECDWL/DECDHL implementations.
## Validation Steps Performed
In pwsh:
* ``"`e[31mab`e[m`b`e[4h`e[32m$('*'*10)`e[m`e[4l"``
prints a red "a", 10 green "*" and a red "b" ✅
* ``"`e[31mab`e[m`b`e[4h`e[32m$('*'*1000)`e[m`e[4l"``
prints a red "a" and a couple lines of green "*" ✅
* ``"`e[31mf$('o'*70)`e[m`e#6`e#5"``
the right half of the row is erased ✅
Work is ongoing to remove individually-authenticated service accounts
from some pipelines. This moves us closer to that goal.
Tested in Nightly 2403.28002.
This changeset allows Windows Terminal to dump its buffer contents as
UTF-16LE VT text onto disk and restore it later. This functionality is
enabled whenever `persistedWindowLayout` is being used.
Closes#961Closes#16741
## Validation Steps Performed
* Open multiple windows with multiple tabs and restart the app
Everything's restored ✅
* Reopen a tab with output from `RenderingTests.exe`
Everything's restored ✅
* Closing tabs and windows with Ctrl+W deletes their buffer dumps ✅
* Closing tabs doesn't create buffer dumps ✅
This adds support for specifying more than one font family using a
syntax that is similar to CSS' `font-family` property.
The implementation is straight-forward and is effectively
just a wrapper around `IDWriteFontFallbackBuilder`.
Closes#2664
## PR Checklist
* Font fallback
* Write "「猫」"
* Use "Consolas" and remember the shape of the glyphs
* Use "Consolas, MS Gothic" and check that it changed ✅
* Settings UI autocompletion
* It completes ✅
* It filters ✅
* It recognizes commas and starts a new name ✅
* All invalid font names are listed in the warning message ✅
---------
Co-authored-by: Dustin L. Howett <duhowett@microsoft.com>
The `nLength` parameter of `ReadConsoleOutputCharacterW` indicates
the number of columns that should be read. For single-column (narrow)
surrogate pairs this previously clipped a trailing character of the
returned string. In the major Unicode support update in #13626
surrogate pairs truly got stored as atomic units for the first time.
This now meant that a 120 column read with such codepoints resulted
in 121 characters. Other parts of conhost still assume UCS2 however,
and so this results in the entire read failing.
This fixes the issue by turning surrogate pairs into U+FFFD
which makes it UCS2 compatible.
Closes#16892
## Validation Steps Performed
* Write U+F15C0 and read it back with `ReadConsoleOutputCharacterW`
* Read succeeds with a single U+FFFD ✅
Right now, the localization submission pipeline runs every night and
sends our localizable resources over to Touchdown. Later, release builds
pick up the localizations directly from Touchdown, move them into place,
and consume them.
This allowed us to avoid having localized content in the repository, but
it came with too many downsides:
- Users could not contribute additional localizations very easily
- We use the same release pipeline and Touchdown configuration for every
branch, so strings needed to either slightly match or _entirely match_
across an entire set of active release branches
- Building from day to day can pull in different strings, making the
product not reproduceable
- Calling TDBuild during release builds requires network access from the
build machine (so does restoring NuGet packages, but that's neither
here nor there)
- Local developers and users could not test out other languages
This pull request moves all localization processing into the nightly
build phase and introduces support for checking loc in and submitting a
pull request. The pull request will not be created anew if one already
exists which has not been merged.
Anything we needed to do on release is now done overnight. This includes
moving loc files into the right places and merging the Cascadia
resources with the Context Menu resources (so that we can work around a
relatively lower amount of translations being chosen for the app versus
the context menu; see #12491 for more info.)
There are some smaller downsides to this approach and its
implementation:
- The first commit is going to be huge
- Right now, it only manages a single branch and uses a force push; if a
PR is not reviewed timely, it will be force-pushed and you cannot see
the day-to-day changes in the strings. Hopefully there won't be any.
I've taken great care to ensure that repeated runs of this new pipeline
will not result in unnecessary whitespace changes. This required
changing how we merge ContextMenu.resw into CascadiaPackage to always
use the .NET XmlWriter with specific flags.
NOTE that this does not allow users to _contribute_ translation fixes
for the 10 languages which we are importing. We will still need to pull
changes out of those files and submit them as bugs to the localization
team separately, and hope they come back around in another nightly
build. However, there is no reason users cannot contribute
_non-Touchdown_ languages.
This PR adds support for the `OSC 21` sequence used on DEC terminals to
set the window title. It's just an alias of the `OSC 2` title sequence
used by XTerm.
This PR also corrects the handling of blank title sequences, which are
supposed to reset the title to its default value, but were previously
ignored.
## Detailed Description of the Pull Request / Additional comments
To handle the blank title parsing correctly, I had to make some changes
to the state machine. Previously it would not have dispatched an `OSC`
sequence unless it received a semicolon following the `OSC` number, but
when there's a blank string, that semicolon should not be required.
I also took this opportunity to simplify the `OSC` parsing in the state
machine, and eliminate the `_GetOscTitle` method which no longer served
any purpose.
## Validation Steps Performed
I've manually confirmed that the title sequences are now working as
expected, and added some state machine unit tests covering the blank
value handling for these sequences.
I also had to update one of the existing state machine tests to account
for the changes I made to allow the semicolon to be omitted.
Closes#16783Closes#16784
This pull request introduces the module Microsoft.Terminal.UI.dll, and
moves into it the following things:
- Any `IDirectKeyListener`
- All XAML converter helpers from around the project
- ... including `IconPathConverter` from TerminalSettingsModel
- ... but not `EmptyStringVisibilityConverter`, which has died
It also adds a XAML Markup Extension named `mtu:ResourceString`, which
will allow us to refer to string resources directly from XAML. It will
allow us to remove all of the places in the code where we manually set
resources on XAML controls.
---------
Co-authored-by: Leonard Hecker <lhecker@microsoft.com>
## Summary of the Pull Request
Allow editing of font features and axes in the SUI to get the UI closer
towards JSON parity
The allowed font axes are obtained directly from the currently selected
font, and their display names are presented to the user in the user's
current locale (if it exists). Otherwise, we just display the axis tag
to the user.
## References and Relevant Issues
#10000
## Validation Steps Performed
- [x] Font Axes can be added/changed/removed from the Settings UI
## PR Checklist
- [ ] Closes #xxx
- [ ] Tests added/passed
- [ ] Documentation updated
- If checked, please file a pull request on [our docs
repo](https://github.com/MicrosoftDocs/terminal) and link it here: #xxx
- [ ] Schema updated (if necessary)
Up until now, we have treated inbox, fragment and user color schemes the
same: we load them all into one big map and when we save the settings
file we write them *all* out. It's been a big annoyance pretty much
forever.
In addition to cluttering the user's settings file, it prevents us from
making changes to the stock color schemes (like to change the cursor
color, or to adjust the colors in Tango Dark, or what have you) because
they're already copied in full in the user settings. It also means that
we need some special UI affordances for color schemes that you are
allowed to view but not to delete or rename.
We also have a funny hardcoded list of color scheme names and we use
that to determine whether they're "inbox" for UI purposes.
Because of all that, we are hesitant to add *more* color schemes to the
default set.
This pull request resolves all of those issues at once.
It:
- Adds an "origin" to color schemes indicating where they're from
(Inbox, Fragment, User, ...)
- Replaces the Edit UI with a much simpler version that pretty much only
has a "duplicate this color scheme to start editing it" button
- Deletes color schemes that we consider to be equivalent to inbox ones;
this allows us to finally disentangle the user's preferences from the
terminal's.
- Migrates all user settings that referred to schemes they may have
modified (even implicitly!) to their modified versions.
The equivalence check intentionally leaves out the cursor and selection
colors, so that we have the freedom to change them in the future.
The Origin is part of a new interface, `ISettingsModelObject`, which we
can use in the future for things like Themes and Actions.
`ConsoleBench` is capable of launching conhost instances and measuring
their performance characteristics. It writes these out as an HTML file
with violin graphs (using plotly.js) for easy comparison.
Currently, only a small number of tests have been added, but the code
is structured in such a way that more tests can be added easily.
## Summary of the Pull Request
This PR adds support for the missing operations that are required in the
8-bit interface architecture extension.
`DECAUPSS` - Assign User-Preference Supplemental Set
`DECRQUPSS` - Request User-Preference Supplemental Set
`ACS` - Announce Code Structure for ANSI levels 1, 2, and 3
## Detailed Description of the Pull Request / Additional comments
For the UPSS assignment there's a new `wstring_view` in `TerminalOutput`
that tracks the current mapping table for the character set, and a new
`VTID` field tracking the ID so it can be queried. The ANSI conformance
just required a couple of calls to existing methods to designate the
appropriate character sets and GL/GR maps.
And now that we've implemented everything that's required, I've updated
our device attributes report (`DA1`) to indicate support for the 8-bit
interface architecture (feature 14).
This PR also addresses some issues with the way the 8-bit GR capability
is managed. Previously a soft reset (`DECSTR`) would return the code
page to its startup state, and the restore cursor operation (`DECRC`)
could have a similar effect. This was a problem for 8-bit apps that
weren't expecting that behavior.
I've now made it so those operations no longer have any effect on the
code page, and the same applies to the C1 parsing option (`DECAC1`). The
only way to restore the code page and C1 parsing to their startup state
now is with a hard reset (`RIS`).
## Validation Steps Performed
I've added unit tests covering the `DECAUPSS` and `DECRQUPSS` sequences,
and I've also manually tested their behavior in Vttest.
## PR Checklist
- [x] Closes#16546
- [x] Tests added/passed
This is a specification for a way to customize console allocations.
The new manifest type `consoleAllocationPolicy` and the new
`AllocConsoleWithOptions` API were already added to the console
client library internally.
Closes#7335
This contains all the parts of #16598 that aren't specific to session
restore, but are required for the code in #16598:
* Adds new GUID<>String functions that remove the `{}` brackets.
* Adds `SessionId` to the `ITerminalConnection` interface.
* Flush the `ApplicationState` before we terminate the process.
* Not monitoring `state.json` for changes is important as it prevents
disturbing the session state while session persistence is ongoing.
That's because when `ApplicationState` flushes to disk, the FS
monitor will be triggered and reload the `ApplicationState` again.
The primary reason for this refactoring was to simplify the management
of VT input sequences that vary depending on modes, adding support for
the missing application keypad sequences, and preparing the way for
future extensions like `S8C1T`.
However, it also includes fixes for a number of keyboard related bugs,
including a variety of missing or incorrect mappings for the `Ctrl` and
`Ctrl`+`Alt` key combinations,
## References and Relevant Issues
This PR also includes a fix for #10308, which was previously closed as a
duplicate of #10551. I don't think those bugs were related, though, and
although they're both supposed to be fixed in Windows 11, this PR fixes
the issue in Windows 10.
## Detailed Description of the Pull Request / Additional comments
The way the input now works, there's a single keyboard map that takes a
virtual key code combined with `Ctrl`, `Alt`, and `Shift` modifier bits
as the lookup key, and the expected VT input sequence as the value. This
map is initially constructed at startup, and then regenerated whenever a
keyboard mode is changed.
This map takes care of the cursor keys, editing keys, function keys, and
keys like `BkSp` and `Return` which can be affected by mode changes. The
remaining "graphic" key combinations are determined manually at the time
of input.
The order of precedence looks like this:
1. If the virtual key is `0` or `VK_PACKET`, it's considered to be a
synthesized keyboard event, and the `UnicodeChar` value is used
exactly as given.
2. If it's a numeric keypad key, and `Alt` is pressed (but not `Ctrl`),
then it's assumedly part of an Alt-Numpad composition, so the key
press is ignored (the generated character will be transmitted when
the `Alt` is released).
3. If the virtual key combined with modifier bits is found in the key
map described above, then the matched escape sequence will be used
used as the output.
4. If a `UnicodeChar` value has been provided, that will be used as the
output, but possibly with additional Ctrl and Alt modifiers applied:
a. If it's an `AltGr` key, and we've got either two `Ctrl` keys
pressed or a left `Ctrl` key that is distinctly separate from a
right `Alt` key, then we will try and convert the character into
a C0 control code.
b. If an `Alt` key is pressed (or in the case of an `AltGr` value,
both `Alt` keys are pressed), then we will convert it into an
Alt-key sequence by prefixing the character with an `ESC`.
5. If we don't have a `UnicodeChar`, we'll use the `ToUnicodeEx` API to
check whether the current keyboard state reflects a dead key, and if
so, return nothing.
6. Otherwise we'll make another `ToUnicodeEx` call but with any `Ctrl`
and `Alt` modifiers removed from the state to determine the base key
value. Once we have that, we can apply the modifiers ourself.
a. If the `Ctrl` key is pressed, we'll try and convert the base value
into a C0 control code. But if we can't do that, we'll try again
with the virtual key code (if it's alphanumeric) as a fallback.
b. If the `Alt` key is pressed, we'll convert the base value (or
control code value) into an Alt-key sequence by prefixing it with
an `ESC`.
For step 4-a, we determine whether the left `Ctrl` key is distinctly
separate from the right `Alt` key by recording the time that those keys
are pressed, and checking for a time gap greater than 50ms. This is
necessary to distinguish between the user pressing `Ctrl`+`AltGr`, or
just pressing `AltGr` alone, which triggers a fake `Ctrl` key press at
the same time.
## Validation Steps Performed
I created a test script to automate key presses in the terminal window
for every relevant key, along with every Ctrl/Alt/Shift modifier, and
every relevant mode combination. I then compared the generated input
sequences with XTerm and a DEC VT240 terminal. The idea wasn't to match
either of them exactly, but to make sure the places where we differed
were intentional and reasonable.
This mostly dealt with the US keyboard layout. Comparing international
layouts wasn't really feasible because DEC, Linux, and Windows keyboard
assignments tend to be quite different. However, I've manually tested a
number of different layouts, and tried to make sure that they were all
working in a reasonable manner.
In terms of unit testing, I haven't done much more than patching the
ones that already existed to get them to pass. They're honestly not
great tests, because they aren't generating events in the form that
you'd expect for a genuine key press, and that can significantly affect
the results, but I can't think of an easy way to improve them.
## PR Checklist
- [x] Closes#16506
- [x] Closes#16508
- [x] Closes#16509
- [x] Closes#16510
- [x] Closes#3483
- [x] Closes#11194
- [x] Closes#11700
- [x] Closes#12555
- [x] Closes#13319
- [x] Closes#15367
- [x] Closes#16173
- [x] Tests added/passed
`TextBuffer::GenHTML` and `TextBuffer::GenRTF` now read directly from
the TextBuffer.
- Since we're reading from the buffer, we can now read _all_ the
attributes saved in the buffer. Formatted copy now copies most (if not
all) font/color attributes in the requested format (RTF/HTML).
- Use `TextBuffer::CopyRequest` to pass all copy-related options into
text generation functions as one unit.
- Helper function `TextBuffer::CopyRequest::FromConfig()` generates a
copy request based on Selection mode and user configuration.
- Both formatted text generation functions now use `std::string` and
`fmt::format_to` to generate the required strings. Previously, we were
using `std::ostringstream` which is not recommended due to its potential
overhead.
- Reading attributes from `ROW`'s attribute RLE simplified the logic as
we don't have to track attribute change between the text.
- On the caller side, we do not have to rebuild the plain text string
from the vector of strings anymore. `TextBuffer::GetPlainText()` returns
the entire text as one `std::string`.
- Removed `TextBuffer::TextAndColors`.
- Removed `TextBuffer::GetText()`. `TextBuffer::GetPlainText()` took its
place.
This PR also fixes two bugs in the formatted copy:
- We were applying line breaks after each selected row, even though the
row could have been a Wrapped row. This caused the wrapped rows to break
when they shouldn't.
- We mishandled Unicode text (\uN) within the RTF copy. Every next
character that uses a surrogate pair or high codepoint was missing in
the copied text when pasted to MSWord. The command `\uc4` should have
been `\uc1`, which is used to tell how many fallback characters are used
for each Unicode codepoint (\u). We always use one `?` character as the
fallback.
Closes#16191
**References and Relevant Issues**
- #16270
**Validation Steps Performed**
- Casual copy-pasting from Terminal or OpenConsole to word editors works
as before.
- Verified HTML copy by copying the generated HTML string and running it
through an HTML viewer.
[Sample](https://codepen.io/tusharvickey/pen/wvNXbVN)
- Verified RTF copy by copy-pasting the generated RTF string into
MSWord.
- SingleLine mode works (<kbd>Shift</kbd>+ copy)
- BlockSelection mode works (<kbd>Alt</kbd> selection)
This changeset ensures that the message queue of frozen windows is
always being serviced. This should ensure that it won't fill up and
lead to deadlocks, freezes, or similar. I've tried _a lot_ of different
approaches before settling on this one. Introducing a custom `WM_APP`
message has the benefit of being the least intrusive to the existing
code base.
The approach that I would have favored the most would be to never
destroy the `AppHost` instance in the first place, as I imagined that
this would be more robust in general and resolve other (rare) bugs.
However, I found that this requires rewriting some substantial parts
of the code base around `AppHost` and it could be something that may
be of interest in the future.
Closes#16332
Depends on #16587 and #16575
Leonard Hecker