Make conhost act in VtIo mode earlier in startup (#15298)

We need to act like a ConPTY just a little earlier in startup. My relevant notes start here: https://github.com/microsoft/terminal/issues/15245#issuecomment-1536150388.

Basically, we'd create the first screen buffer with 9001 rows, because it would be created _before_ VtIo would be in a state to say "yes, we're a conpty". Then, if a CLI app emits an entire screenful of text _before_ the terminal has a chance to resize the conpty, then the conpty will explode during `_DoLineFeed`. That method is absolutely not expecting the buffer to get resized (and the old text buffer deallocated).

Instead, this will treat the console as in ConPty mode as soon as `VtIo::Initialize` is called (this is during `ConsoleCreateIoThread`, which is right at the end of `ConsoleEstablishHandoff`, which is before the API server starts to process the client connect message).  THEORETICALLY, `VtIo` could `Initialize` then fail to create objects in `CreateIoHandlers` (which is what we used to treat as the moment that we were in conpty mode). However, if we do fail out of `CreateIoHandlers`, then the console itself will fail to start up, and just die. So I don't think that's needed.

This fixes #15245. I think this is PROBABLY also the solution to #14512, but I'm not gonna explicitly mark closed. We'll loop back on it.

(cherry picked from commit 6ad8cd0a63)
Service-Card-Id: 89112504
Service-Version: 1.17
This commit is contained in:
Mike Griese 2023-05-10 07:16:44 -05:00 committed by Dustin L. Howett
parent a6ce08a4cf
commit 63addccfb0
2 changed files with 3 additions and 6 deletions

View File

@ -25,7 +25,6 @@ using namespace Microsoft::Console::Interactivity;
VtIo::VtIo() :
_initialized(false),
_objectsCreated(false),
_lookingForCursorPosition(false),
_IoMode(VtIoMode::INVALID)
{
@ -223,13 +222,12 @@ VtIo::VtIo() :
}
CATCH_RETURN();
_objectsCreated = true;
return S_OK;
}
bool VtIo::IsUsingVt() const
{
return _objectsCreated;
return _initialized;
}
// Routine Description:
@ -245,7 +243,7 @@ bool VtIo::IsUsingVt() const
[[nodiscard]] HRESULT VtIo::StartIfNeeded()
{
// If we haven't been set up, do nothing (because there's nothing to start)
if (!_objectsCreated)
if (!_initialized)
{
return S_FALSE;
}
@ -510,7 +508,7 @@ void VtIo::EndResize()
// - <none>
void VtIo::EnableConptyModeForTests(std::unique_ptr<Microsoft::Console::Render::VtEngine> vtRenderEngine)
{
_objectsCreated = true;
_initialized = true;
_pVtRenderEngine = std::move(vtRenderEngine);
}
#endif

View File

@ -63,7 +63,6 @@ namespace Microsoft::Console::VirtualTerminal
VtIoMode _IoMode;
bool _initialized;
bool _objectsCreated;
bool _lookingForCursorPosition;