Skip to content

sys/win: display a useful error message when SetConsoleMode fails#639

Merged
DHowett merged 4 commits intomainfrom
dev/duhowett/maybe
Sep 9, 2025
Merged

sys/win: display a useful error message when SetConsoleMode fails#639
DHowett merged 4 commits intomainfrom
dev/duhowett/maybe

Conversation

@DHowett
Copy link
Member

@DHowett DHowett commented Sep 8, 2025

edit will now display specific error messages when the console fails to support ENABLE_VIRTUAL_TERMINAL_INPUT. On Windows 8.1, the user will be gently reprimanded to update to Windows 10. On Windows 10, they will be coerced into using the modern console host.

We only check the version after this fails because it is technically possible to run edit on OpenConsole (or another third-party console host!)--even on Windows 8.1--where it will work properly.

@DHowett
Copy link
Member Author

DHowett commented Sep 8, 2025

image

assuming you run OpenConsole, it works fine.

| Console::ENABLE_VIRTUAL_TERMINAL_INPUT,
))?;
)) {
Err(apperr::Error::Sys(e)) if e == 0x80070000 | ERROR_INVALID_PARAMETER => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could call gle_to_apperr(ERROR_INVALID_PARAMETER) instead here. It's a const function so it should get optimized away.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

at first I tried to match Sys(0x80070057) directly and it didn't work. i resorted to this somewhat tortured syntax because it was the only thing that did work. i can test it again.

..mem::zeroed()
};
let condition =
VerSetConditionMask(0, VER_BUILDNUMBER, 3 /* VER_GREATER_EQUAL */);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can import VER_GREATER_EQUAL from windows_sys::Win32::System::SystemInformation.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image

I can, in fact, not do that.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I didn't go out of my way to do it the more difficult, harder to document and more annoying to review way on purpose 😁)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, it's in windows_sys::Win32::System::SystemServices for some reason. 🥴
https://docs.rs/windows-sys/0.61.0/windows_sys/Win32/System/SystemServices/constant.VER_GREATER_EQUAL.html

Err(apperr::Error::Sys(
match VerifyVersionInfoW(&mut osv, VER_BUILDNUMBER, condition) {
Foundation::TRUE => ERROR_UNSUPPORTED_LEGACY_CONSOLE,
Foundation::FALSE => ERROR_UNSUPPORTED_WINDOWS_VERSION,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still not 100% sure if the distinction between the two errors is meaningful nowadays, given how rarely older versions of Windows are still used.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, that's fair. It is technically true that "On windows 8.1, the legacy console is in use"

@DHowett DHowett changed the title sys/win: display useful error messages when our console modes don't work sys/win: display a useful error message when SetConsoleMode fails Sep 9, 2025
@DHowett DHowett merged commit bb56984 into main Sep 9, 2025
6 checks passed
@DHowett DHowett deleted the dev/duhowett/maybe branch September 9, 2025 20:24
This was referenced Sep 25, 2025
Lou32Verbose pushed a commit to Lou32Verbose/edit that referenced this pull request Jan 11, 2026
…crosoft#639)

edit will now display specific error messages when the console fails to
support `ENABLE_VIRTUAL_TERMINAL_INPUT`. The user will be gently
reprimanded for not using the modern console host.

It is technically possible to run edit on OpenConsole (or another third-
party console host!)--even on Windows 8.1--where it will work properly.
jenia90 pushed a commit to jenia90/edit that referenced this pull request Jan 30, 2026
…crosoft#639)

edit will now display specific error messages when the console fails to
support `ENABLE_VIRTUAL_TERMINAL_INPUT`. The user will be gently
reprimanded for not using the modern console host.

It is technically possible to run edit on OpenConsole (or another third-
party console host!)--even on Windows 8.1--where it will work properly.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants