Dev docs for Keyboard Manager (#7340)

* Create kbm md file

* Documented KBM main module (without links)

* Completed docs for backend except links

* Added docs for UI and created doc file for KBM Common

* Added docs for KBM common and for tests

* Add table of contents for all files

* Started updating links

* Added some more links

* Updated links

* testing link

* Resolve all links on keyboardmanager.md

* Added links in KBM Common

* Added comments for keyboardeventhandlers

* Added links for KBM-UI

* Added mention of shortcut to key

* Add section link for settings update code.

* Include more information on updating remap tables
This commit is contained in:
Arjun Balgovind 2020-10-21 08:30:40 -07:00 committed by GitHub
parent 9c341cc9bc
commit 147b78ffce
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
6 changed files with 474 additions and 0 deletions

View File

@ -0,0 +1,6 @@
# Table of Contents
The devdocs for Keyboard Manager have been divided into the following modules:
1. [Keyboard Manager Module](keyboardmanager.md)
2. [Keyboard Event Handlers](keyboardeventhandlers.md)
3. [Keyboard Manager UI](keyboardmanagerui.md)
4. [Keyboard Manager Common](keyboardmanagercommon.md)

Binary file not shown.

After

Width:  |  Height:  |  Size: 22 KiB

View File

@ -0,0 +1,84 @@
# Keyboard Event Handlers (Remapping Logic)
This file contains documentation for all the methods involved in key/shortcut remapping.
## Table of Contents:
1. [HandleSingleKeyRemapEvent](#HandleSingleKeyRemapEvent)
2. [HandleShortcutRemapEvent](#HandleShortcutRemapEvent)
3. [HandleOSLevelShortcutRemapEvent](#HandleOSLevelShortcutRemapEvent)
4. [HandleAppSpecificShortcutRemapEvent](#HandleAppSpecificShortcutRemapEvent)
5. [HandleSingleKeyToggleToModEvent (Obsolete))](#HandleSingleKeyToggleToModEvent-(Obsolete---Code-from-PoC-which-is-commented-out))
6. [Tests](#Tests)
1. [MockedInput](#MockedInput)
2. [Tests for single key remaps and shortcut remaps](#Tests-for-single-key-remaps-and-shortcut-remaps)
## HandleSingleKeyRemapEvent
[This method](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/KeyboardEventHandlers.cpp#L13-L124) is used for handling the key to key and key to shortcut remapping logic. The general logic is as follows:
- Check if the `dwExtraInfo` field contains the `KEYBOARDMANAGER_INJECTED_FLAG` bit set. This bit is used to indicate that the key event was generated by KBM using `SendInput`. This ensures that we don't read events generated by the key or shortcut remap methods.
- Check if the current key is present in the list of remaps. If it isn't, return 0 (i.e. do not suppress the event).
- If it is remapped to Disable, suppress the event.
- If it is remapped to a key, we send the key down/up message for the target key and suppress the current key event. We have a check for filtering artificial keys, such as `VK_WIN` (which is a keycode added by us), so that it is translated to `VK_LWIN` instead.
- If it is remapped to a shortcut, for key down we set the target modifiers first, followed by the target action key, and for key up we release the action key first, followed by the modifiers.
- All the remapped key events that we send above are sent with `KEYBOARDMANAGER_SINGLEKEY_FLAG` on the `dwExtraInfo` field.
## HandleShortcutRemapEvent
[This method](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/KeyboardEventHandlers.cpp#L178-L739) is used for handling the shortcut to shortcut and shortcut to key remapping logic. The general logic is as follows:
- Check if any shortcut remap is currently invoked. This is required to ensure that two remaps don't occur simultaneously at a time, and we send key up events for the shortcuts only if they are actually invoked and not for artificial key up events. In addition to that, while a remap is in the middle of execution, the keyboard state will not match the physical keys, so we do not want a remap <kbd>Ctrl+A</kbd> to <kbd>Ctrl+V</kbd> to also trigger the remap from <kbd>Ctrl+V</kbd> to <kbd>Alt+V</kbd> on pressing <kbd>Ctrl+A</kbd> on the keyboard.
- Get the remap table as per the the `activatedApp` argument (i.e. if it is empty, we get the global shortcut remap table and otherwise we get the corresponding app-specific shortcut remap table).
- Iterate over the list of remaps in descending order of number of keys in the shortcut. This is required **for shortcut to key remaps** to ensure that if a user has both <kbd>Ctrl+A</kbd> and <kbd>Ctrl+Shift+A</kbd> remapped to some keys, and the user presses <kbd>Ctrl+Shift+A</kbd>, then we prefer the <kbd>Ctrl+Shift+A</kbd> remap. This logic would not be required if there were only shortcut to shortcut remaps, as they are invoked only on exact match.
- If any shortcut was found to be invoked (from the first step), then we skip till we find the matching shortcut remap. If not we check if the modifiers of the original shortcut are pressed down. If they are, we check if the current key event is a key down event and it matches the action key of the original shortcut. For shortcut to shortcut and for disabling a shortcut [we have an additional step](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/KeyboardEventHandlers.cpp#L208-L212) where we check if any other key is pressed apart from the original shortcut. This is required because for these two features we allow the remaps only if those exact keys are pressed. The method used for this is described in detail [here](keyboardmanagercommon.md#IsKeyboardStateClearExceptShortcut). If a win key was pressed, we store whether it was the left or the right one, in order to determine which key to set for remaps from/to the common Win key code which we added. This is so that pressing and releasing Left Win key results in that Win key getting modified and not the Right Win key.
- If the remap is to a key, we send a dummy key event followed by releasing the original shortcut's modifiers and setting the target key (or doing nothing if it is remapped to disable) and we suppress the event.
- If the remap is to a shortcut, if the modifiers in the original shortcut are present in the target, we only set the additional modifiers and the action key of the target. If it isn't, we send a dummy key event followed by releasing the modifiers which are not common, and setting the remaining ones in the target along with the action key.
- For both cases, we set the `isShortcutInvoked` flag to true, and set the `KeyboardManagerState.activatedApp` if it is an app-specific shortcut remap.
- For the `isShortcutInvoked` is true scenario (i.e. the initial remap keydown section is done) there are several cases depending on the key pressed or released:
- [**Case 1:**](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/KeyboardEventHandlers.cpp#L339-L430) If a modifier in the original shortcut is released, we need to reset back to the physical keys pressed.
- For remap to shortcut, we release the target action key if it is currently pressed, and depending on whether all the modifiers of the original shortcut are present in the target, we release the target modifiers that are not common, and set the remaining original shortcut modifiers except the one that was released. We do not need to send the original action key as that will get generate it's own key event if it is held down.
- For remap to key, we release the target key if it is pressed (and it is not remapped to Disable), and we set the original shortcut modifiers.
- For both the cases we send a dummy key event at the end, since we are setting modifiers without any other key after that, and we reset all the remap variables.
- [**Case 2:**](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/KeyboardEventHandlers.cpp#L435-L461) If the original shortcut's action key is pressed again, we send the target shortcut's action key or the target key again (or for disable we just suppress the event).
- [**Case 3:**](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/KeyboardEventHandlers.cpp#L463-L527) If the original shortcut's action key is released
- For remap to shortcut, we just release the target shortcut's action key
- For remap to disable, we suppress the event
- For remap to key, we check if any other keys are pressed apart from the target key. If not, we just release the target key. If there are, we reset back to the physical keys by releasing the target key and setting the original shortcut's modifiers along with a dummy key, and we reset all the remap variables. This behavior is different from remap to shortcut because if the action key is released while other keys are pressed the remap should be inactive, but such a state can't occur for shortcut to shortcut remaps since they happen only when the exact keys are pressed.
- [**Case 4:**](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/KeyboardEventHandlers.cpp#L529-L551) If a modifier in the original shortcut is pressed, suppress the event
- [**Case 5:**](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/KeyboardEventHandlers.cpp#L553-L732) If any other key is pressed
- For remap to shortcut, we need to reset back to physical keys as the shortcut remaps can't be pressed in combination with other keys. We release the target action key if it was pressed, and we release the modifier keys of the target shortcut that are not common and set the remaining ones in the original shortcut. We then send the original shortcut's action key if the target action key was found to be pressed, and we send the current key press at the end.
- For remap to key, if it is remapped to disable or if the target key is not found to be pressed, we reset to the physical keys, we set the original shortcut's modifiers and if is remap to Disable and the original shortcut's action key is physically pressed (this is checked by the `isOriginalActionKeyPressed` flag which we keep track of whenever the action key is pressed or released for remap to Disable), then we set the original shortcut's action key, followed by the current key press. If it is not remapped to disable and the target key is pressed, then we don't suppress the event as we allow shortcut to key remappings to be pressed along with other keys.
- For all the above cases, dummy key isn't required as we want the current key press to behave like a normal key.
- [**Case 6:**](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/KeyboardEventHandlers.cpp#L733) If any other key is released, do not suppress the event as this event didn't appear with a corresponding key down event (such as an app sending a key up event) or we processed the key down and let it continue (for shortcut to key scenario).
- All the remapped key events that we send above are sent with `KEYBOARDMANAGER_SHORTCUT_FLAG` on the `dwExtraInfo` field, except the usage of the current key press in Case 5, for which we don't send any extra info so that it is considered as a normal key event which may in turn invoke some other remap.
**Note:** Shortcuts are considered valid if they have modifiers and an action key. The reason why we haven't supported key combinations of just modifiers (which is requested in this [issue](https://github.com/microsoft/PowerToys/issues/5670)) (like remapping <kbd>Ctrl+Alt</kbd>) is because this would require more cases and handling as these remappings have to take place only on press and release and if there is no key pressed in between similar to what Start Menu does. The remapping would have to be invoked only for this specific sequence <kbd>Ctrl</kbd> key down, <kbd>Alt</kbd> key down, <kbd>Alt</kbd> key up, <kbd>Ctrl</kbd> key up (ordering between Ctrl and Alt can be swapped). If any other key is pressed in between it shouldn't be invoked, and since this logic requires tracking exact states instead of using GetAsyncKeyStates, this could cause false positives if a user is not running as admin.
## HandleOSLevelShortcutRemapEvent
[This method](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/KeyboardEventHandlers.cpp#L741-L752) is used for handling global shortcut to shortcut and shortcut to key remaps. The general logic is as follows:
- Check if the `dwExtraInfo` field is set to `KEYBOARDMANAGER_SHORTCUT_FLAG`. This indicates that the key event was generated by the KBM shortcut remap method using `SendInput`. This ensures that we don't read events generated by the shortcut remap method, but we still read events which are generated by the key remap method.
- Call `HandleShortcutRemapEvent` without the `activatedApp` argument so that global shortcut remapping takes place if it applies for the current key event.
## HandleAppSpecificShortcutRemapEvent
[This method](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/KeyboardEventHandlers.cpp#L754-L809) is used for handling app-specific shortcut to shortcut and shortcut to key remaps. The general logic is as follows:
- Check if the `dwExtraInfo` field is set to `KEYBOARDMANAGER_SHORTCUT_FLAG`. This indicates that the key event was generated by the KBM shortcut remap method using `SendInput`. This ensures that we don't read events generated by the shortcut remap method, but we still read events which are generated by the key remap method.
- Get the name of the process in the foreground. This is done using `GetCurrentApplication` which uses `GetForegroundWindow` to get the window handle and `get_process_path` from the common lib. This approach can fail for UWP apps in full screen, so for that scenario we use the `GetGUIThreadInfo` approach to find the correct window handle, and hence the correct process name. This method is described in more detail [here](keyboardmanagercommon.md#Foreground-app-detection)
- By checking `KeyboardManagerState.GetActivatedApp` we check if an app-specific shortcut is currently invoked. If so, we consider this application to be the activated app. This is required because some shortcut remaps could cause the current app to lose focus and hence until the shortcut is completely released we should allow that remap to continue, otherwise the user could end up in a state where some keys do not get released. For example: remap <kbd>Ctrl+A</kbd> to <kbd>Alt+Tab</kbd> for Edge, when a user presses <kbd>Ctrl+A</kbd> the window loses focus as <kbd>Alt+Tab</kbd> gets executed.
- If there is no app-specific shortcut currently invoked, we check if the foreground process is present in the list of app-specific remaps, either with or without the file extension and case insensitive. If it is, this is considered to be the activated app.
- Call `HandleShortcutRemapEvent` with the `activatedApp` argument so that app-specific shortcut remapping takes place if it applies for the current key event.
## HandleSingleKeyToggleToModEvent (Obsolete - Code from PoC which is commented out)
[This method](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/KeyboardEventHandlers.cpp#L126-L176) was added to support a feature for converting the behavior of a key from behaving like a toggle (like Caps Lock/Num Lock) to a modifier (like Ctrl), such that when you hold Caps Lock it would behave as if Caps Lock was active, and when it was not pressed Caps Lock would be off. For Caps Lock this would be similar to behaving like Shift, but for Num Lock there is no existing key which can substitute for this. This was added while testing out remapping for the KBM PoC, but wasn't added as a feature since it wasn't a priority.
## Tests
In order to test the remapping logic, a mocked keyboard input handler had to be created because otherwise the tests would process and send actual key events. For this the [`InputInterface`](https://github.com/microsoft/PowerToys/blob/master/src/modules/keyboardmanager/common/InputInterface.h) was made, and in production code the methods are implemented using [`SendInput`](https://docs.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-sendinput) and [`GetAsyncKeyState`](https://docs.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-getasynckeystate). In addition to this, [`GetCurrentApplication`](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/common/Helpers.cpp#L226-L268) had to be mocked so that app-specific remapping can be tested.
### MockedInput
The [`MockedInput`](https://github.com/microsoft/PowerToys/blob/master/src/modules/keyboardmanager/test/MockedInput.h) class uses a 256 size `bool` vector to store the key state for each key code. Identifying the foreground process is mocked by simply setting and getting a string value for the name of the current process.
[To mock the `SendInput` method](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/test/MockedInput.cpp#L10-L110), the steps for processing the input are as follows. This implementation is based on public documentation for SendInput and the behavior of key messages and keyboard hooks:
- Iterate over all the inputs in the INPUT array argument
- If the event is a key up event, then it is considered [`WM_SYSKEYUP`](https://docs.microsoft.com/en-us/windows/win32/inputdev/wm-syskeyup) if Alt is held down, otherwise it is `WM_KEYUP`.
- If the event is a key down event, then it is considered [`WM_SYSKEYDOWN`](https://docs.microsoft.com/en-us/windows/win32/inputdev/wm-syskeydown) if either Alt is held down or if it is F10, otherwise it is `WM_KEYDOWN`.
- An optional function which can be set on the `MockedInput` handler can be used to test for the number of times a key event is received by the system with a particular condition using [`sendVirtualInputCallCondition`](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/test/MockedInput.cpp#L48-L52).
- The hook logic for a low level hook which returns 0 or 1 can be set on the `MockedInput` handler such that it behaves like a low level hook would behave with actual keyboard input. If the method returns 1, then the keyboard state is not updated, and if it returns 0 the corresponding key event is used to update the key state. This works in the recursive way as well similar to low level hooks, as `SendVirtualInput` can be called from within the hook, thus simulating identical behavior to calling `SendInput` in a low level hook (as soon as SendInput is called, the low level hook is called for the new input event, and only after those are processed it returns back to the current event, check this [blog](https://devblogs.microsoft.com/oldnewthing/20140213-00/?p=1773) for more details).
- For updating the keyboard state, KEYUP messages result in the state for that key code being set to false, and KEYDOWN result in the state for that key code being set to true.
- For modifiers the behavior is slightly different as if the key state of the L/R version is modified, it should also modify the common version, and if a common version is released, it should release both the L and R versions.
### Tests for single key remaps and shortcut remaps
Using the MockedInput handler, all the expected (and known) key scenarios that can occur for while pressing a [remapped key](https://github.com/microsoft/PowerToys/blob/master/src/modules/keyboardmanager/test/SingleKeyRemappingTests.cpp) or [remapped shortcut](https://github.com/microsoft/PowerToys/blob/master/src/modules/keyboardmanager/test/OSLevelShortcutRemappingTests.cpp) are tested. The foreground app behavior which is specific to app-specific shortcuts is tested [here](https://github.com/microsoft/PowerToys/blob/master/src/modules/keyboardmanager/test/AppSpecificShortcutRemappingTests.cpp).

View File

@ -0,0 +1,199 @@
# Keyboard Manager module
This file contains the documentation for the KeyboardManager PowerToy module which is called by the runner.
## Table of Contents:
1. [Class members](#Class-members)
2. [Enable/Disable](#Enable/disable)
3. [Settings format](#Settings-format)
4. [Loading settings](#Loading-settings)
5. [Low level keyboard hook handler](#Low-level-keyboard-hook-handler)
6. [Custom Action to launch KBM UI](#Custom-Action-to-launch-KBM-UI)
7. [SendInput Special Scenarios](#SendInput-Special-Scenarios)
1. [Extended keys](#Extended-keys)
2. [Scan code](#Scan-code)
8. [Special Scenarios](#Special-Scenarios)
1. [Dummy key events](#Dummy-key-events)
2. [Suppressing Num Lock in a keyboard hook](#Suppressing-Num-Lock-in-a-keyboard-hook)
3. [Modifier-Caps Lock interaction on Japanese IME keyboards](#Modifier-Caps-Lock-interaction-on-Japanese-IME-keyboards)
4. [UIPI Issues (not resolved)](#UIPI-Issues-(not-resolved))
9. [Other remapping approaches](#Other-remapping-approaches)
1. [Registry approach](#Registry-approach)
2. [Driver approach](#Driver-approach)
10. [Telemetry](#Telemetry)
## Class members
The `KeyboardManager` module has [3 main class members](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/dllmain.cpp#L54-L61):
- A static pointer to the current object of `KeyboardManager`. This is required for using the `KeyboardManager` object in the low level keyboard hook handler as that method must be static. This is described in more detail in [this section](#Low-level-keyboard-hook-handler).
- An object of type `Input`, which is used for all the operations that involving getting or setting keyboard states. This is wrapped in an object to allow testing the remapping methods.
- An object of type `KeyboardManagerState`. This object contains all the data related to remappings and is also used in the sense of a View Model as it used to communicate common data that is shared between the KBM UI and the backend. This class is described in more detail [here](keyboardmanagercommon.md#keyboardmanagerstate).
## Enable/Disable
On enabling KBM, the low level keyboard hook is started, and it is unhooked on disable. This is done to allow users to manually restart KBM if some other application which registers a keyboard hook was launched after PowerToys, so that it can be brought back to the highest priority hook (as the last hook to be registered receives the input first as mentioned [here](https://docs.microsoft.com/en-us/windows/win32/winmsg/about-hooks#hook-procedures)).
In addition to stopping the hook, any active KBM UI windows are also closed on disabling. This is done because the KBM UI uses the same keyboard hook for the Type button where you can type a key/shortcut, so if KBM is disabled the windows would not be completely functional.
The enable/disable code can be found [here](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/dllmain.cpp#L301-L322)
## Settings format
KBM uses two sets of settings files.
- The main `settings.json` is of the following format:
{
"properties":
{
"activeConfiguration":
{
"value":"default"
},
"keyboardConfigurations":
{
"value":["default"]
}
},
"name":"Keyboard Manager",
"version":"1"
}
- The `activeConfiguration` attribute stores the current remapping profile, while `keyboardConfigurations` stores all the profiles that the user has. This was added to avoid any future breaking changes for the [profiles feature](https://github.com/microsoft/PowerToys/issues/1881), which would allow users to switch between remappings
- The profile format (`default.json`) is of the following format:
{
"remapKeys":
{
"inProcess":
[
{
"originalKeys":"91",
"newRemapKeys":"162;70"
},
{
"originalKeys":"92",
"newRemapKeys":"162;70"
}
]
},
"remapShortcuts":
{
"global":
[
{
"originalKeys":"164;37",
"newRemapKeys":"162;65"
},
{
"originalKeys":"162;68",
"newRemapKeys":"91"
}
],
"appSpecific":
[
{
"originalKeys":"91;162;65",
"newRemapKeys":"162;86",
"targetApp":"msedge"
}
]
}
}
- `originalKeys` stores the key/shortcut which is to be pressed for the remap, and `newKeys` stores the key/shortcut which is to be executed.
- Both contain semi-colon separated virtual key codes. For `remapKeys`, `originalKeys` must have only one key code, whereas for `remapShortcuts` it must have atleast two key codes.
- `inProcess` sub-key was added in `remapKeys` because there was a possibility of adding the registry based remapping approach (used by [SharpKeys](https://github.com/randyrants/sharpkeys)), so that would be under a separate sub-key while `inProcess` would be for keyboard hook based remaps. This was deprioritized as there weren't enough requests for it.
- `remapShortcuts` is split into `global` and `appSpecific`, where `global` remaps would apply to all applications, whereas `appSpecific` would apply on when the `targetApp` is in focus. `targetApp` must be the process name of the app (with or without it's extension), e.g. `msedge` or `msedge.exe` for Microsoft Edge.
## Loading settings
KBM settings are loaded only on the C++ side only at start up, in the [constructor](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/dllmain.cpp#L67-L68). The settings file may get modified from the KBM UI on applying new remappings, but the the file is not read again. The files are read from the PowerToys Settings process whenever a change is made to the file (using a FileWatcher) or whenever the KBM page is opened. The settings are updated only when the user presses the OK button from either of the Remap Keys or Remap Shortcuts windows. This is described in more detail [here](keyboardmanagerui.md#ok-and-cancel-button).
## Low level keyboard hook handler
Since the [`hook_proc`](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/dllmain.cpp#L330-L349) cannot be a member function in the class, this is declared `static` and a `static pointer` to the `KeyboardManager` project is used ([`keyboardmanager_object_ptr`](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/dllmain.cpp#L54-L55)).
As seen in the code for `hook_proc`, similar to other keyboard hooks in PowerToys it consists of a main method `HandleKeyboardHookEvent` which computes whether the key needs to be suppressed and accordingly returns 1 or calls the `CallNextHook` method.
`HandleKeyboardHookEvent` is covered in the [next section](#HandleKeyboardHookEvent). The `SetNumLockToPreviousState` code in the above snippet is required for a special scenario with keyboard input, which is covered in [this section](#Suppressing-Num-Lock-in-a-keyboard-hook).
## HandleKeyboardHookEvent
The [`HandleKeyboardHookEvent`](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/dllmain.cpp#L384-L458) is the method which calls the corresponding remapping methods in the required order. The following checks are executed in order:
- **`KeyboardManagerState.AreRemappingsEnabled`:** This returns false while the KBM remap tables are getting updated. If it is in this state, `HandleKeyboardHookEvent` returns `0`, i.e. the key event is not suppressed and is forwarded normally.
- **Check for `KEYBOARDMANAGER_SUPPRESS_FLAG`:** If the key event has the suppress flag, the method returns 1 to suppress the key event.
- **[`KeyboardManagerState.DetectSingleRemapKeyUIBackend`](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/dllmain.cpp#L399-L408):** This method is used for handling hook operations for the single key Type UI in the Remap keys window. If the Remap keys window is open, then `HandleKeyboardHookEvent` returns `0` and the key event is forwarded normally. If the left column Type button is clicked on the Remap keys window and and the window is in focus, then the key event is suppressed and the UI is updated with the latest key from the recent key events. This method is described in more detail [here](keyboardmanagercommon.md#DetectSingleRemapKeyUIBackend-and-DetectShortcutUIBackend).
- **[`KeyboardManagerState.DetectShortcutUIBackend(data, true)`](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/dllmain.cpp#L410-L419):** This method is used for handling hook operations for the shortcut Type UI in the Remap keys window (when `isRemapKey` arg is `true`). If the Remap keys window is open, then `HandleKeyboardHookEvent` returns `0` and the key event is forwarded normally. If the right column Type button is clicked on the Remap keys window and and the window is in focus, then the key event is suppressed and the UI is updated with the shortcut from the recent key events. This method is described in more detail [here](keyboardmanagercommon.md#DetectSingleRemapKeyUIBackend-and-DetectShortcutUIBackend).
- **`HandleSingleKeyRemapEvent`:** This method handles the single key remap logic. If a remapping takes place, the key event is suppressed. This method is described in more detail [here](keyboardeventhandlers.md#HandleSingleKeyRemapEvent).
- **[`KeyboardManagerState.DetectShortcutUIBackend(data, false)`](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/dllmain.cpp#L430-L439):** This method is used for handling hook operations for the shortcut Type UI in the Remap shortcuts window (when `isRemapKey` arg is `false`). If the Remap shortcuts window is open, then `HandleKeyboardHookEvent` returns `0` and the key event is forwarded normally. If the Type button is clicked on the Remap shortcuts window and and the window is in focus, then the key event is suppressed and the UI is updated with the shortcut from the recent key events. **Since this is executed after the single key remap method, all single key remappings are applied when the user is on the Remap shortcuts window.**
- **`HandleAppSpecificShortcutRemapEvent`:** This method handles the app-specific shortcut remap logic. If a remapping takes place, the key event is suppressed. This method is described in more detail [here](keyboardeventhandlers.md#HandleAppSpecificShortcutRemapEvent). **Since this is executed after the single key remap method, single key remappings have precedence over shortcut remaps and are correspondingly reflected in shortcut remaps.**
- **`HandleOSLevelShortcutRemapEvent`:** This method handles the global shortcut remap logic. If a remapping takes place, the key event is suppressed. This method is described in more detail [here](keyboardeventhandlers.md#HandleOSLevelShortcutRemapEvent). The app-specific remap method is executed before this because if a shortcut is remapped to different keys/shortcuts for a particular app and globally, the app-specific variant should be preferred if that app is in focus. **Since this is executed after the single key remap method, single key remappings have precedence over shortcut remaps and are correspondingly reflected in shortcut remaps.**
**Note:** Single key remaps need to be executed before shortcut remaps, because otherwise there can be several logical issues. For example if a user has Ctrl remapped to X and Ctrl+A remapped to Y, we can't detect Ctrl+A because the moment Ctrl is pressed it would be remapped to X before the system ever sees Ctrl+A. This is why the design decision was made to separate Remap keys and Remap shortcuts, and all key remaps are reflected in the shortcut remaps.
## Custom Action to launch KBM UI
KBM uses the [`call_custom_action`](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/dllmain.cpp#L249-L280) method from the `PowertoyModuleIface` in order to launch the KBM UI when the user clicks the Remap a key or Remap a shortcut button from the KBM settings page. On clicking the button, we check if there is already any active KBM UI window, and if there is it is brought to the foreground. If not, the corresponding KBM UI window is launched on a separate detached thread. The UI is described in more detail [here](keyboardmanagerui.md).
## SendInput Special Scenarios
### Extended keys
Certain keys such as the arrow keys, <kbd>right Ctrl/Alt</kbd>, and <kbd>Del/Home/Ins</kbd>, etc need to be sent with the `KEYEVENTF_EXTENDEDKEY` flag because otherwise the NumPad versions get sent, which can cause weird behavior when NumLock is on. The code can be found [here](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/common/Helpers.cpp#L190-L194) and the list of extended keys in code can be found [here](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/common/Helpers.cpp#L73-L98). Docs about extended keys can be found [here](https://docs.microsoft.com/en-us/windows/win32/inputdev/about-keyboard-input#extended-key-flag).
The weird behavior that is caused by this can be found at these issues:
- https://github.com/microsoft/PowerToys/issues/3478
- https://github.com/microsoft/PowerToys/issues/3647
- https://github.com/microsoft/PowerToys/issues/3981
### Scan code
Certain applications (such as Windows Terminal) may filter out key events which are set to scan code 0. Even though the `KEYEVENTF_SCANCODE` flag is not set, the `wScan` field is still sent, which defaults to 0. To avoid this issue we use the `MapVirtualKey` API to find the scan code from the virtual key code. Code can be found [here](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/common/Helpers.cpp#L196-L198).
## Special Scenarios
Since we are using low level keyboard hooks and not actual OS level input handling certain scenarios with input require workarounds as do they not interact well with the OS input logic directly. These are covered in the sub-sections below.
### Dummy key events
To prevent the behavior that some modifiers have that occur when you press and release the modifier without pressing any key in between, we need to send a dummy key event in between the two states. Some examples of this behavior are Win key for Start Menu and Alt key to focus the menu bar. We need to send the dummy key events at any point where an unintentional modifier press/release sequence may occur. We use the undocumented `0xFF` virtual key code for this as we haven't found any side effects of using this key code yet. Initially we used only a key up message, but it has been tweaked now to send a key down followed by key up (code can be found (here)[https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/common/Helpers.cpp#L201-L208]), as without the key down there were [compatibility issues with some apps](https://github.com/microsoft/PowerToys/issues/7133) (like Slack).
The dummy key event is currently used in the following places (the linked code snippets contains an example scenario of why it is required):
- https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/KeyboardEventHandlers.cpp#L251-L253
- https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/KeyboardEventHandlers.cpp#L289-L291
- https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/KeyboardEventHandlers.cpp#L382-L383
- https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/KeyboardEventHandlers.cpp#L409-L410
- https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/KeyboardEventHandlers.cpp#L509-L510
### Suppressing Num Lock in a keyboard hook
The <kbd>Num Lock</kbd> key state is updated by the OS before it is intercepted by low level hooks. This causes the issue that even if you suppress a <kbd>Num Lock</kbd> key event, <kbd>Num Lock</kbd> will still get toggled. In order to work around this, in the [`hook_proc`](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/dllmain.cpp#L340-L344) whenever we suppress a <kbd>Num Lock</kbd> key down event, we send an additional <kbd>Num Lock</kbd> key up followed by key down so that the <kbd>Num Lock</kbd> state is reverted to it's previous value before the suppressed event. These are sent with a `KEYBOARDMANAGER_SUPPRESS_FLAG` in the `dwExtraInfo` field, so that we suppress them at the start of the hook (see code [here](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/KeyboardEventHandlers.cpp#L811-L825)). Since these events will update the <kbd>Num Lock</kbd> state before the low level hooks, by suppressing them we ensure that these are not sent to any other hooks/applications and hence are only processed by the OS.
This assumes that KBM is the last hook to be registered (since another hook-based app like AutoHotkey could remap NumLock to some other key which could mess up this logic).
### Modifier-Caps Lock interaction on Japanese IME keyboards
While using Japanese IME on Windows, shortcuts like <kbd>Shift/Alt/Ctrl</kbd> + <kbd>Caps Lock</kbd> can be used to switch IME options.
![japanese-ime](japanese-ime.png)
These shortcuts are detected before low level hooks, and hence cause issues while remapping <kbd>Caps Lock</kbd> to <kbd>Shift/Alt/Ctrl</kbd> or vice-versa, as there could be an intermediate state where the system detects both the keys as being pressed. This results in a state where the modifier key does not get released since the OS suppresses the key up messages before they reach the low level hooks.
In order to work around this when a key down for the modifier is being processed, we send a key up for the modifier key with the `KEYBOARDMANAGER_SUPPRESS_FLAG` in the `dwExtraInfo` field, so that we suppress them at the start of the hook, and this key event would only be processed by the OS, without getting forwarded to other hooks/apps. The approach is described in more detail at [this comment](https://github.com/microsoft/PowerToys/issues/3397#issuecomment-640136416), as discussed with the AutoHotkey team. The code for the workaround can be found [here](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/KeyboardEventHandlers.cpp#L827-L846). Tests for these scenarios have also been added at:
- [Tests for workaround on single key remaps](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/test/SingleKeyRemappingTests.cpp#L110-L219)
- [Tests for workaround on shortcut remaps](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/test/OSLevelShortcutRemappingTests.cpp#L1935-L2144)
For example, while [remapping <kbd>Ctrl</kbd> to <kbd>Caps Lock</kbd>](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/KeyboardEventHandlers.cpp#L59-L63), before sending the <kbd>Caps Lock</kbd> key down message and suppressing <kbd>Ctrl</kbd>, we send a suppressed <kbd>Ctrl</kbd> key up, so that the OS doesn't see <kbd>Ctrl</kbd> and <kbd>Caps Lock</kbd> pressed together at any point. For the [<kbd>Caps Lock</kbd> to <kbd>Ctrl</kbd> scenario](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/KeyboardEventHandlers.cpp#L104-L116), we send a suppressed C<kbd>Ctrl</kbd>trl key up message after sending <kbd>Ctrl</kbd> key down before <kbd>Caps Lock</kbd> suppressed. Similar logic is added for such scenarios in shortcut remapping.
While the above work around fixes most of the cases, there are still some scenarios where the modifier can get stuck, mentioned at this [comment](https://github.com/microsoft/PowerToys/issues/3397#issuecomment-663729278), which is why the issue is still open. This occurs if a modifier is pressed after the remap has been invoked before releasing the remapped key and it is a harder scenario to solve which requires refactoring the single key remap code.
### UIPI Issues (not resolved)
`SendInput` does not work directly with certain key codes such as Play/Pause Media, Calculator key, etc as it requires UAC privileges to be injected to the OS and accordingly play the active media app or launch the Calculator app. In order to resolve this the correct approach is that the executable which calls `SendInput` needs to have the [UIAccess flag](https://docs.microsoft.com/en-us/windows/win32/winauto/uiauto-securityoverview) set to true, which will also avoid the requirement of KBM having to run as administrator to intercept key events when an elevated window is in focus. The UIAccess flag has many constraints such as it must be a signed executable and must be located in a protected path like Program Files. Since KBM currently runs out of the runner process, it would make more sense to do this work after KBM is moved to a separate executable, and it could be enabled by a separate toggle in settings only if PowerToys is installed in Program Files. [This comment](https://github.com/microsoft/PowerToys/issues/3192#issuecomment-646323661) has more details on this approach and (this)[https://github.com/microsoft/PowerToys/issues/3255] is the tracking issue.
## Other remapping approaches
Other approaches for remapping which were deprioritized are:
### Registry approach
This method is used by [SharpKeys](https://github.com/randyrants/sharpkeys) and involves using the [Microsoft Keyboard Scancode mapper registry key](https://github.com/randyrants/sharpkeys) to remap keys based on their scan codes. This has the advantage of being applied in all scenarios and not facing any elevation or UAC issues, however the disadvantages are that for modifying the settings the process must run elevated (as it modifies HKLM registry) and it requires a reboot to get applied. Another issue which is an advantage/disadvantage for users is that the process does not need to be running, so the remaps are applied all the time, including at the password prompt on logging in to the user's Windows account, which could get a user stuck if they orphaned a key in their password. This registry doesn't have any support for remapping shortcuts either, so the hook approach was prioritized over this.
### Driver approach
Using a driver approach has the benefit of not depending on precedence orders as KBM could always run before low level hooks, and it also has the benefit of differentiating between different keyboards, allowing [multi keyboard-specific remaps](https://github.com/microsoft/PowerToys/issues/1460). The disadvantages are however that any bug or crash could have system level consequences. [Interception](https://github.com/oblitum/Interception) is an open source driver that could be used for implementing this. The approach was deprioritized due to the potential side effects.
## Telemetry
Keyboard Manager emits the following telemetry events (implemented in [trace.h](https://github.com/microsoft/PowerToys/blob/master/src/modules/keyboardmanager/common/trace.h) and [trace.cpp](https://github.com/microsoft/PowerToys/blob/master/src/modules/keyboardmanager/common/trace.cpp)):
- **`KeyboardManager_EnableKeyboardManager`:** Logs a `boolean` value storing the KBM toggle state. It is logged whenever KBM is enabled or disabled (emitted [here](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/dllmain.cpp#L305-L316)).
- **`KeyboardManager_KeyRemapCount`:** Logs the number of key to key and key to shortcut remaps (i.e. all the remaps on the Remap a key window). This gets logged on saving new settings in the Remap a key window (emitted [here](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/LoadingAndSavingRemappingHelper.cpp#L159-L163)).
- **`KeyboardManager_OSLevelShortcutRemapCount`:** Logs the number of global shortcut to shortcut and shortcut to key remaps. This gets logged on saving new settings in the Remap a shortcut window (emitted [here](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/LoadingAndSavingRemappingHelper.cpp#L220)).
- **`KeyboardManager_AppSpecificShortcutRemapCount`:** Logs the number of app-specific shortcut to shortcut and shortcut to key remaps. This gets logged on saving new settings in the Remap a shortcut window (emitted [here](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/LoadingAndSavingRemappingHelper.cpp#L221)).
- **`KeyboardManager_KeyToKeyRemapInvoked`:** Logs an event when a key to key remap is invoked (emitted [here](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/KeyboardEventHandlers.cpp#L101-L102)).
- **`KeyboardManager_KeyToShortcutRemapInvoked`:** Logs an event when a key to shortcut remap is invoked (emitted [here](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/KeyboardEventHandlers.cpp#L101-L102)).
- **`KeyboardManager_OSLevelShortcutToShortcutRemapInvoked`:** Logs an event when a global shortcut to shortcut remap is invoked (emitted [here](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/KeyboardEventHandlers.cpp#L320-L321)).
- **`KeyboardManager_OSLevelShortcutToKeyRemapInvoked`:** Logs an event when a global shortcut to key remap is invoked (emitted [here](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/KeyboardEventHandlers.cpp#L320-L321)).
- **`KeyboardManager_AppSpecificShortcutToShortcutRemapInvoked`:** Logs an event when an app-specific shortcut to shortcut remap is invoked (emitted [here](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/KeyboardEventHandlers.cpp#L320-L321)).
- **`KeyboardManager_AppSpecificShortcutToKeyRemapInvoked`:** Logs an event when an app-specific shortcut to key remap is invoked (emitted [here](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/KeyboardEventHandlers.cpp#L320-L321)).
- **`KeyboardManager_Error`:** Logs the occurrence of an error in KBM with the name of the method, error code and the corresponding error message. This is currently used only for logging `SetWindowsHookEx` failures (emitted [here](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/dllmain.cpp#L364-L369)).

View File

@ -0,0 +1,63 @@
# Keyboard Manager Common
This project contains any code that is to be shared between the backend and UI projects. This file covers any functionality in this project which hasn't been covered along with the other modules.
## Table of Contents
1. [KeyboardManagerState](#KeyboardManagerState)
1. [UI States](#UI-States)
2. [DetectSingleRemapKeyUIBackend and DetectShortcutUIBackend](#DetectSingleRemapKeyUIBackend-and-DetectShortcutUIBackend)
3. [HandleKeyDelayEvent](#HandleKeyDelayEvent)
4. [Saving remappings to file](#Saving-remappings-to-file)
5. [Concurrent Access to remap tables](#Concurrent-Access-to-remap-tables)
2. [KeyDelay](#KeyDelay)
3. [Shortcut and RemapShortcut classes](#Shortcut-and-RemapShortcut-classes)
1. [IsKeyboardStateClearExceptShortcut](#IsKeyboardStateClearExceptShortcut)
2. [CheckModifiersKeyboardState](#CheckModifiersKeyboardState)
3. [Tests](#Tests)
4. [Helpers](#Helpers)
1. [Foreground App Detection](#Foreground-App-Detection)
## KeyboardManagerState
[This class](https://github.com/microsoft/PowerToys/blob/master/src/modules/keyboardmanager/common/KeyboardManagerState.cpp) stores all the data related to remappings and is also used in the sense of a View Model as it used to communicate common data that is shared between the KBM UI and the backend. They are accessed on the UI controls using static class members of `SingleKeyRemapControl` and `ShortcutControl`.
### UI States
[UI states](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/common/KeyboardManagerState.h#L27-L42) are used to keep track in which step of the UI flow is the user at, such as which Remap window they are on, or if they have one of the Type windows open. This is required because the hook needs to suppress input and update UI in some cases, and in some cases remappings have to be disabled altogether.
### DetectSingleRemapKeyUIBackend and DetectShortcutUIBackend
[These methods](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/common/KeyboardManagerState.cpp#L374-L446) are [called on the low level hook](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/dllmain.cpp#L399-L408) in the main keyboard event handler. When the user opens any UI window the UI states are updated and in this case some remappings have to be disabled. On the Remap keys window, all remappings are disabled, while on the Remap shortcuts window, shortcut remappings are disabled.
In addition to this, if the user has opened the Type window, and the window is in focus, [whenever a key event is received we have to update the set of selected keys in the TextBlock](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/common/KeyboardManagerState.cpp#L266-L329) in the ContentDialog. These methods also call the `KeyDelay` handlers to check if the input is Esc/Enter and accordingly handle the Accessibility events for the Type window. When the user clicks the Type button, [variables in the KeyboardManagerState store the corresponding TextBlocks](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/SingleKeyRemapControl.cpp#L375-L376) which appear in the ContentDialog, so that these UI controls can be updated from the hook on the dispatcher thread.
### HandleKeyDelayEvent
[This method](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/common/KeyboardManagerState.cpp#L482-L498) checks if the UI is in the foreground, and if so runs the key delay handlers that have been registered.
### Saving remappings to file
The [`SaveConfigToFile`](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/common/KeyboardManagerState.cpp#L500-L607) method is called on clicking the OK button on the EditKeyboardWindow or EditShortcutsWindow. Since PowerToys Settings also reads the config JSON file, [a named mutex is used](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/common/KeyboardManagerState.cpp#L582-L600) before accessing the file, with a 1 second timeout. If the mutex is obtained the settings are written to the default.json file.
### Concurrent Access to remap tables
To prevent the UI thread and low level hook thread from concurrently accessing the remap tables we use an [`atomic bool` variable](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/common/KeyboardManagerState.h#L91-L92), which is set to `true` while the tables are getting updated. When this is `true` the hook will skip all remappings. Use of mutexes in the hook were removed to prevent re-entrant mutex bugs.
## KeyDelay
[This class](https://github.com/microsoft/PowerToys/blob/master/src/modules/keyboardmanager/common/KeyDelay.cpp) implements a queue based approach for processing key events and based on the time difference between key down and key up events [executes separate methods for `ShortPress`, `LongPress` or `LongPressReleased`](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/common/KeyDelay.h#L69-L72). The class is used for the hold Enter/Esc functionality required for making the Type window accessible and prevent keyboard traps (see [this](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/SingleKeyRemapControl.cpp#L273-L292) for an example of it's usage). The `KeyEvents` are added to the queue from the hook thread of KBM, and a separate [`DelayThread`](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/common/KeyDelay.cpp#L142-L166) is used to process the key events by checking the `time` member in the key event. The thresholds for short vs long press and hold wait timeouts are `static` constants, but if the module is extended for other purposes these could be made into arguments.
**Note:** [Deletion of the `KeyDelay`](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/common/KeyDelay.cpp#L4-L12) object should never be called from the `DelayThread` i.e. from within one of the 3 handlers, as it can re-enter the mutex and would lead to a deadlock. This can be avoided by either deleting it on a separate thread or as done in the KBM UI, on the dispatcher thread. See [this PR](https://github.com/microsoft/PowerToys/pull/6959#issue-496583547) for more details on this issue.
## Shortcut and RemapShortcut classes
The [`Shortcut` class](https://github.com/microsoft/PowerToys/blob/master/src/modules/keyboardmanager/common/Shortcut.h) is a data structure for storing key combinations which are valid shortcuts and it contains several methods which are used for shortcut specific operations. [`RemapShortcut`](https://github.com/microsoft/PowerToys/blob/master/src/modules/keyboardmanager/common/RemapShortcut.h) consists of a shortcut/key union (`std::variant`), along with other boolean flags which are required on the hook side for storing any relevant keyboard states mid-execution.
### IsKeyboardStateClearExceptShortcut
[This method](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/common/Shortcut.cpp#L665-L813) is used by the `HandleShortcutRemapEvent` to check if any other keys on the keyboard have been pressed apart from the keys in the shortcut. This is required because shortcut to shortcut remaps should not be applied if the shortcut is pressed with other keys. The method iterates over all the possible key codes, except any keys that are considered reserved, unassigned, OEM-specific or undefined, as well as mouse buttons (see list [here](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/common/Shortcut.cpp#L628-L663)).
### CheckModifiersKeyboardState
[This method](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/common/Shortcut.cpp#L517-L614) uses `GetVirtualKeyState` (internally calls `GetAsyncKeyState` in production code), to check if all the modifiers of the current shortcut are being pressed. Since Win doesn't have a non-L/R key code we check this by checking both LWIN and RWIN.
### Tests
Tests for some methods in the `Shortcut` class can be found [here](https://github.com/microsoft/PowerToys/blob/master/src/modules/keyboardmanager/test/ShortcutTests.cpp).
## Helpers
[This namespace](https://github.com/microsoft/PowerToys/blob/master/src/modules/keyboardmanager/common/Helpers.cpp) has any methods which are used across either UI or the backend which aren't specific to either. Some of these methods have tests [here](https://github.com/microsoft/PowerToys/blob/master/src/modules/keyboardmanager/test/SetKeyEventTests.cpp).
### Foreground App Detection
[`GetCurrentApplication`](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/common/Helpers.cpp#L226-L268) is used for detecting the foreground process for App-specific shortcuts. The logic is very similar to that used for FZ's app exception feature, involving `GetForegroundWindow` and `get_process_path`. The one additional case which has been added is for full-screen UWP apps, where the above method fails and returns `ApplicationFrameHost.exe`. The [`GetFullscreenUWPWindowHandle`](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/common/Helpers.cpp#L210-L224) uses `GetGUIThreadInfo` API to find the window linked to the GUI thread. This logic is based on [this stackoverflow answer](https://stackoverflow.com/questions/39702704/connecting-uwp-apps-hosted-by-applicationframehost-to-their-real-processes/55353165#55353165).
**Note:** The [`GetForegroundProcess` method](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/dll/Input.cpp#L17-L21) performs string allocation in a weird way because of exceptions that were occurring while running tests as a result of memory being allocated or deallocated across dll boundaries. Here's the comment from the PR where this was added
> To make app-specific logic test-able, a GetForegroundProcess was added to the input interface which internally calls GetCurrentApplication. This allows us to mock this method in the test project by just setting some process name as the foreground process for that function. When I set this to just return the string name, it would goes runtime errors on the test project in debug_heap with `__acrt_first_block == header`. Based on [this stackoverflow answer](https://stackoverflow.com/a/35311928), this would happen if allocation happens in one dll's code space and deallocation happens in another. One way to avoid this is to change both the projects to MD (multi threaded dll) instead of MT(multi threaded), however that results in many compile-time errors since all the PT projects are configured as MT. To solve this, the GetForegroundProcess was rewritten such that its argument is the output variable, and we allocate memory for that string within the AppSpecificHandler method rather than in that function.

View File

@ -0,0 +1,122 @@
# Keyboard Manager UI
## Table of Contents:
1. [C++ XAML Islands](#c---xaml-islands)
1. [Debugging exceptions in XAML Islands](#debugging-exceptions-in-xaml-islands)
2. [Build times](#build-times)
3. [Setting custom backgrounds for Xaml Controls using brushes](#setting-custom-backgrounds-for-xaml-controls-using-brushes)
2. [UI Structure](#ui-structure)
3. [EditKeyboardWindow/EditShortcutsWindow](#editkeyboardwindow-editshortcutswindow)
1. [OK and Cancel button](#ok-and-cancel-button)
2. [Delete button](#delete-button)
3. [Handling common modifiers in EditKeyboardWindow](#handling-common-modifiers-in-editkeyboardwindow)
4. [SingleKeyRemapControl](#singlekeyremapcontrol)
5. [ShortcutControl](#shortcutcontrol)
6. [KeyDropDownControl](#keydropdowncontrol)
1. [Localized key names](#localized-key-names)
2. [Single Key ComboBox Selection Handler](#single-key-combobox-selection-handler)
3. [Shortcut ComboBox Selection Handler](#shortcut-combobox-selection-handler)
## C++ XAML Islands
The KBM UI is implemented as a C++ XAML Island, but all the controls are implemented in code behind rather than .xaml and .xaml.cs files. This was done as per a XAML Island Code sample and it didn't require a separate UWP project, which could be limited in terms of using hooks. There is a [tech debt item](https://github.com/microsoft/PowerToys/issues/2027) for moving this to XAML. The reason it wasn't implemented in the C# Settings was because it required communication with the low level hook thread, which could be too slow if IPC is used, since the UI needs to update on every key event.
**Note:** For functions which take a XAML component as argument, pass it by value and not by reference. This is because `winrt` WinUI classes store their own internal references, so they are supposed to be passed by value (and internally ref counts are incremented). Passing by reference can lead to weird behavior where the object is `null`.
The windows are [created as C++ windows](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/EditKeyboardWindow.cpp#L128-L140) and the window sizes are set to default by [scaling them as per DPI](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/EditKeyboardWindow.cpp#L120-L126) using the `DPIAware::Convert` API from common lib. Since the UI is launched on a new thread, the window may not be in the foreground, so [we call `SetForegroundWindow`](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/EditKeyboardWindow.cpp#L146-L150).
`DesktopWindowXamlSource` has to be declared and [it is initialized](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/EditKeyboardWindow.cpp#L159-L162) using the [`XamlBridge`](https://github.com/microsoft/PowerToys/blob/master/src/modules/keyboardmanager/ui/XamlBridge.cpp), and [a second window handle](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/EditKeyboardWindow.cpp#L161-L162) is generated for the internal Xaml Island window. Most of the code was based on the [Xaml Island Sample](https://github.com/microsoft/Xaml-Islands-Samples/blob/master/Samples/Win32/SampleCppApp/XamlBridge.cpp). The `XamlBridge` class contains code which handles initializing the Xaml Island containers as well as handling special messages like keyboard navigation, and focus between islands and between the C++ window and the island. It also has methods for clearing the xaml islands and closing the window.
Once the UI controls are created, the parent container is set as the content for the `DesktopWindowXamlSource` and the `XamlBridge.MessageLoop` is executed. Messages are processed by the C++ window handler like [`EditKeyboardWindowProc`](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/EditKeyboardWindow.cpp#L364-L404). The general structure we use for this is, for any `WM_PAINT` or `WM_SIZE` message we resize the Xaml Island window. For `WM_GETMINMAXINFO` we set minimum widths so that the window cannot be resized beyond a minimum height and width. This is done to prevent the WinUI elements from overlapping and getting cropped. If it is neither of these cases we send the message to the [`XamlBridge.MessageHandler`](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/XamlBridge.cpp#L291-L301) which handles Destroy, Activation and Focus. If `WM_NCDESTROY` is received when the `XamlBridge` is `nullptr`, the window thread is terminated.
**Note:** `ContentDialog` in Xaml Islands requires manually settings a `XamlRoot`. This can generally be done by passing the XamlRoot from a component in the main window, such as the button used to open the dialog ([`sender.as<Button>().XamlRoot()`](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/ShortcutControl.cpp#L31-L32)). [These docs]((https://docs.microsoft.com/en-us/uwp/api/windows.ui.xaml.controls.contentdialog#contentdialog-in-appwindow-or-xaml-islands)) have more details about this.
### Debugging exceptions in XAML Islands
Sometimes if an exception occurs in XAML Islands, the stack trace may not always point to the correct code causing the exception and instead it will point to the Xaml Island message loop. In these cases the output window in VS will generally show the correct exception.
### Build times
C++ Xaml Islands generally take several minutes to build because the `pch` which contains the WinUI headers takes longer to build and compiles to a file of several GBs. To minimize the build times, multi-processor compilation within the projects have been enabled (files are distributed for compilation to the processors), and references to the Xaml headers have been removed from the .h headers files as much as possible. Since several classes of ours had class members with UI controls like `StackPanel` (which requires definitions of the classes in order to compile), we worked around this by declaring them as `IInspectable` (the equivalent of an object pointer in winrt), and initializing them to the actual control like `StackPanel` in the constructor and accessing all their member functions by inline typecasting (for `IInspectable x;` we do `x = StackPanel();` and `x.as<StackPanel>().MemberFunction()`). Check [this](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/ShortcutControl.cpp#L19-L25) for this type of usage in `ShortcutControl`.
### Setting custom backgrounds for Xaml Controls using brushes
To access the brushes available on C# Xaml, it has to be done with the `Resources.Lookup` syntax:
`primaryButton.Background(Windows::UI::Xaml::Application::Current().Resources().Lookup(box_value(L"SystemControlBackgroundBaseMediumLowBrush")).as<Windows::UI::Xaml::Media::SolidColorBrush>());`
## UI Structure
The KBM UI consists of a [`Grid` with several columns](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/EditKeyboardWindow.cpp#L200-L218). Rows are added dynamically when [the add button is pressed](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/EditKeyboardWindow.cpp#L305-L309). [A vector of vector of unique pointers to `SingleKeyRemapControl`/`ShortcutControl`](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/EditKeyboardWindow.cpp#L248-L249) is created so that references to the UI components and their data are not lost until the window is closed. [`SingleKeyRemapControl`](https://github.com/microsoft/PowerToys/blob/master/src/modules/keyboardmanager/ui/SingleKeyRemapControl.cpp) is the UI class for each row of the Remap keys table, and [`ShortcutControl`](https://github.com/microsoft/PowerToys/blob/master/src/modules/keyboardmanager/ui/ShortcutControl.cpp) is the UI class for each row of the Remap shortcuts table. [`KeyDropDownControl`](https://github.com/microsoft/PowerToys/blob/master/src/modules/keyboardmanager/ui/KeyDropDownControl.cpp) is used for handling the ComboBox operations. Each of these two classes [have vectors of unique pointers to the `KeyDropDownControl` objects](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/ShortcutControl.h#L44-L45) so that references to the objects are active until the control is deleted.
When the UI windows are activated the `KeyboardManagerState` object [sets the `UIState` variable](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/EditKeyboardWindow.cpp#L251-L252) which is used for distinguishing if the UI is up from the keyboard hook thread. The [states are also updated](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/SingleKeyRemapControl.cpp#L53) on opening and closing the Type window.
Clicking the Type Button [opens a content dialog](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/SingleKeyRemapControl.cpp#L206-L380) which registers key delays using the [`KeyDelay` class](keyboardmanagercommon.md#KeyDelay) for Enter and Esc keys and sets the UI states such that when a key event occurs the TextBlocks on the ContentDialog are updated accordingly. On accepting the dialog the selected keys are copied into the ComboBoxes from the TextBlocks, and on closing the window the key delays are unregistered and UI states are reset.
Since ComboBoxes are added dynamically, handlers have been added which [update the accessible names for these controls](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/KeyDropDownControl.cpp#L69-L74), which get executed whenever a drop down is added or removed.
When the `EditKeyboardWindow`/`EditShortcutsWindow` is created, [we iterate through the remappings](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/EditKeyboardWindow.cpp#L254-L262) stored in `KeyboardManagerState` and add rows to the UI Grid. For both the windows we have `static` buffers [`singleKeyRemapBuffer`](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/SingleKeyRemapControl.h#L39-L40) and [`shortcutRemapBuffer`](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/ShortcutControl.h#L42-L43) which store the corresponding key/shortcuts as per the selections in the UI if they are valid with no warnings.
## EditKeyboardWindow/EditShortcutsWindow
### OK and Cancel button
[On pressing the OK button](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/EditKeyboardWindow.cpp#L66-L89) in `EditKeyboardWindow`, first the [`CheckIfRemappingsAreValid` method](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/LoadingAndSavingRemappingHelper.cpp#L10-L44) is executed which performs basic validity checks on the current remappings in the remap buffer (`static SingleKeyRemapControl::singleKeyRemapBuffer`), such as if there are no NULL columns and none of the source keys are repeated. All other validity checks are assumed to happen while the user adds the remapping. If this is found to be invalid a ContentDialog is displayed which shows that some remappings are invalid and if the user proceeds only the valid ones will be applied. If it is valid [`GetOrphanedKeys`](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/LoadingAndSavingRemappingHelper.cpp#L46-L75) is executed which checks if any keys are orphaned (i.e. the key has been remapped and no other key has been remapped to it, so there is no way to send that key code), and a dialog is shown for notifying the user with a list of orphaned keys. After this the settings are [applied by adding it to the `KeyboardManagerState.singleKeyReMap` member](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/LoadingAndSavingRemappingHelper.cpp#L102-L164) and they are saved to the JSON file. `EditShortcutsWindow` differs slightly from this, as there is no orphaned keys check, and [on pressing OK](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/EditShortcutsWindow.cpp#L32-L47) both the global and app-specific shortcuts are validated and [updated](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/LoadingAndSavingRemappingHelper.cpp#L166-L223).
The code used for updating the remapping tables in `KeyboardManagerState` can be found [here](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/common/KeyboardManagerState.cpp#L104-L183). For shortcut remaps, the `sortedKeys` vectors are updated and re-sorted whenever an element is added to them (like [this](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/common/KeyboardManagerState.cpp#L135-L136)).
On pressing OK (after confirmation dialogs) or Cancel, the window is closed and UI states are reset.
### Delete button
Since there is no single method to delete the elements in a row for a Grid, [the logic](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/SingleKeyRemapControl.cpp#L143-L188) we use involves decrementing the rowIndex for all the UI controls that appearing after the row to be deleted, and removing each of the items of the row from the Grid, followed by deleting that row definition. We also update the accessible names for all the rows since the indexing has changed. After this the corresponding row in the remap buffer is also deleted, and `SingleKeyRemapControl`/`ShortcutControl` objects are deleted from the vector.
### Handling common modifiers in EditKeyboardWindow
In the SingleKeyRemap table for a remapping of the form Ctrl->X, where Ctrl is the common version and not L/R, we can't store it directly as Ctrl->X because when the hook receives the key event it only gets LCtrl or RCtrl specifically and not `VK_CONTROL`. To simplify the backend code, when [single key remappings are applied](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/LoadingAndSavingRemappingHelper.cpp#L116-L143), any remapping of the form Ctrl->X is split into Ctrl(L)->X and Ctrl(R)->X (i.e. both L and R versions are remapped to the same target), and when remappings are loaded in EditKeyboardWindow, we [pre-process the remap table](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/LoadingAndSavingRemappingHelper.cpp#L77-L100) such that if the L and R versions of a modifier are remapped to the same key/shortcut we combine them. This also results in the behavior where a user adds LCtrl->X and RCtrl->X and after closing and re-opening KBM UI it appears combined as Ctrl->X.
## SingleKeyRemapControl
The left drop down column uses a single ComboBox and the Type button is linked to `createDetectKeyWindow`, whereas the right column is linked to `createDetectShortcutWindow` as the column can accept a key or a shortcut (required to support key to key and key to shortcut). The `KeyDropDownControl` for the left column in the window uses a smaller key list, without `None`.
## ShortcutControl
Both the columns in are linked to `createDetectShortcutWindow`, however the drop down selection handlers differ in their logic as the left column only allows shortcuts, and the drop downs do not contain the `Disable` key, whereas the right column allows you to select both shortcuts and keys (to support shortcut to shortcut and shortcut to key), and it allows selection of `Disable`.
For the app-specific shortcut target app text-box, we had to validate that the shortcut row is still valid when the target app is changed (for example, <kbd>Ctrl+A</kbd> is remapped for Chrome, and another remapping for <kbd>Ctrl+A</kbd> was remapped to Edge, but the target was changed to Chrome.). For this we didn't use the TextChanged handler as every time a letter is typed it would get executed. Instead we used the [`LostFocus` handler](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/ShortcutControl.cpp#L108-L167) which gets executed whenever you focus into the box (by clicking or tabbing) and then tabbing or clicking out. In the method we perform the same shortcut buffer validation used for selections in the drop down menus and update the buffers accordingly.
## KeyDropDownControl
Each ComboBox has a linked flyout, which is used to show warnings to the users whenever a user selects an invalid key from the drop down. When the warning is displayed the ComboBox is also reset to -1, i.e. no selection.
For selection handlers on the ComboBoxes we couldn't use just the SelectionChanged handler directly as it gets executed even on searching for elements in the drop down. Instead we used DropDownClosed (when a user opens the drop down and searches and selects something) and SelectionChanged when the drop down is not open (for setting selections programmatically or selection made by searching with tab focus on the drop down without opening it). This was required because if we execute the selection handlers while users are searching, it could cause false positive flyout warnings if the search causes an invalid value to be selected, and flyouts cause the drop down to close leading to bad UI experience.
### Localized key names
For getting localized key names and symbols for each virtual key code, whenever the key lists are accessed, i.e. [whenever the drop down is opened](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/KeyDropDownControl.cpp#L56-L60) or when `GetKeyName` is called in the Type window, the current `KeyboardLayout` is retrieved to ensure that the displayed key names are always updated. Since the `WM_INPUTLANGCHANGED` event was having some issues with XAML islands we weren't able to use this to update the keyboard layout. In addition to this we do not refresh the UI, so the key lists get updated only on opening/interacting with them.
### Single Key ComboBox Selection Handler
On making a selection in the drop down, [the selection handler](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/KeyDropDownControl.cpp#L91-L130) validates the input with the buffer from the other column and other rows. Error messages are shown using flyouts if the selection is not considered valid and the drop down and buffer for that entry are reset to empty selection. The errors that can occur on the single key ComboBox are:
- Remap to same key (A->A)
- Same key previously remapped (A->B and A->C)
- Conflicting modifier previously remapped (Ctrl->A and Ctrl(left)->B, since Ctrl also includes Ctrl(left))
If the selection is found to be valid, the `singleKeyRemapBuffer` is updated accordingly.
For handling `Shortcut` and key in the remap buffer for the right column, we use `std::variant`, which allows us to store either of the two types and check which one of them is present in the buffer by using the `index` method.
[`ValidateAndUpdateKeyBufferElement`](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/BufferValidationHelpers.cpp#L8-L66) does not reference any UI components and instead takes all the relevant data as arguments. This method [has tests](https://github.com/microsoft/PowerToys/blob/master/src/modules/keyboardmanager/test/BufferValidationTests.cpp) which covers all the cases that could arise from making selections on the UI.
### Shortcut ComboBox Selection Handler
On making a selection in the drop down, [the selection handler](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/KeyDropDownControl.cpp#L215-L295) validates the input with the buffer from the other column and other rows. Error messages are shown using flyouts if the selection is not considered valid and the drop down and buffer for that entry are reset to empty selection.
This differs from the Single Key ComboBox handler in the sense that after validating the current selection we may perform [a set of actions](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/KeyDropDownControl.cpp#L160-L204) such as:
- Adding a drop down (if a modifier is selected)
- Removing a drop down (if None is selected)
- Clearing terminal empty drop downs (if an action key is selected in a non-last drop down and the remaining ones are empty)
After performing the corresponding action, if any, we check if the drop down resulted in an error, in which case we do [a second level of validation](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/KeyDropDownControl.cpp#L224-L229) on all the drop downs in that list of drop downs. This is done because there can be cases where an error in one drop down results us setting it to empty, and the remaining selection is also invalid. For example, you have Ctrl+A -> Ctrl+Shift+A, and you change Shift to Ctrl. This would show a warning for having two Ctrls in the shortcut being invalid, after which setting that to empty would result in Ctrl+A->Ctrl+Empty+A, which is a case of remapping a shortcut to itself.
Once this second level of validation is done, we proceed with [updating the buffer](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/KeyDropDownControl.cpp#L231-L251). Depending on the number of drop downs with valid values, this could be either a key or a shortcut (for the right columns). We also [set the buffer value for the target app](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/KeyDropDownControl.cpp#L252-L267) while doing this.
Unlike the Single Key handler, there is a different set of errors that can occur here which are related to making a selection that is considered as a valid shortcut. The `isHybridControl` argument is used to distinguish between the differing behaviors for the two types of columns (shortcut only or shortcut/key column). The errors that can occur for this handler are:
- Shortcut must start with modifier (selecting A on the first drop for the left column is invalid)
- Shortcut can't have a repeated modifier (Ctrl+Ctrl(left)+A is not a shortcut)
- Shortcut can only have upto 2 modifiers (Ctrl+Shift+Alt is not supported as we have enforced a 3 key constraint (**not a backend limitation, there is [an issue](https://github.com/microsoft/PowerToys/issues/3936) requesting to remove this**))
- Shortcut must contain an action key (Ctrl+A and change A to None, only for left column)
- Shortcut must have atleast two keys (Ctrl+A and change Ctrl to None, only for left column)
- Disable can't be a modifier or action key (Ctrl+Disable is invalid)
- Shortcut can't have more than one action key (Ctrl+Shift+A, change Shift to B)
- Remap to same shortcut(Ctrl+A->Ctrl+A)
- Same shortcut previously remapped for same target app (Ctrl+A->B and Ctrl+A->C)
- Conflicting shortcut previously remapped for same target app (Ctrl+A->B and Ctrl(left)+A->C, since Ctrl also includes Ctrl(left))
- Illegal shortcut remaps like Win+L or Ctrl+Alt+Del (since these cannot be remapped using LL hooks)
[`ValidateShortcutBufferElement`](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/BufferValidationHelpers.cpp#L68-L304) does not reference any UI components and instead takes all the relevant data as arguments. This method [has tests](https://github.com/microsoft/PowerToys/blob/master/src/modules/keyboardmanager/test/BufferValidationTests.cpp) which covers all the cases that could arise from making selections on the UI.
**Note:** After updating the buffer we have [code to handle a special case](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/KeyDropDownControl.cpp#L269-L279), which was required to prevent scenarios where a drop down can get deleted but the corresponding `KeyDropDownControl` object isn't deleted. The code checks if the drop down is still linked to the parent and accordingly deletes the `KeyDropDownControl` object from the vector.
**IgnoreKeyToShortcutWarning special case:** [An additional](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/KeyDropDownControl.cpp#L177-L181) check was added to ignore the Map to Same key error when an existing remapping is loaded. This was because a remapping like Ctrl->Ctrl+A has an intermediate step of Ctrl->Ctrl, which could lead to an error of invalid input, even though Ctrl+A is valid. The only way to actually add this is from the Type button or by adding them in a different order (like typing Shift+A and then changing Shift to Ctrl). Since the intermediate check could fail, this was causing the app to crash since the Xaml Island wouldn't be completely loaded at that point and the Flyout can't be displayed. [This](https://github.com/microsoft/PowerToys/issues/6695) is the linked issue which describes the repro scenario.