<Swing Dev> Incorrect / unexpected handling of keybindings using non-standard keyboard layouts

[Semyon Sadetsky]

Hi Matthijs,

Thank you for your e-mail and the analysis. We really appreciate your 
contribution.
I ran your test on Windows and Linux. In both cases with Belgian(Dutch) 
keyboard layout I got single key combinations only, just keys were 
swapped Q->A, W->Z,  Z->W. (I don't have the Belgium physical keyboard, 
I only switched the OS layout.) It looks like that this is really Mac 
specific problem.

--Semyon


On 12/10/2015 12:24 PM, Matthijs Kooijman wrote:
> (Please keep my CC'd, I'm not subscribed to the list)
>
> Hi folks,
>
> while going over the Java keybinding code (trying to solve a different
> problem), I found some strangeness in how KeyboardManager and JComponent
> handle keystrokes. In particular, they try to match any registered
> keybindings to both the normal keycode, as well as the extended keycode
> from the KeyEvent.
>
> According to the documentation, the normal keycode relates to the
> physical position of the key on the keyboard and is not influenced by
> the keyboard layout. The extended keycode relates to the key pressed,
> according to the current keyboard layout. When a US QWERTY layout is
> used, both keycodes will be identical.
>
> Matching the extended keycode makes sense, if a keybinding is CTRL-Q,
> then you want it to work whenever you press the key labeled "Q" on the
> keyboard. Matching the normal keycode makes less sense to me - why would
> the "key-that-is-Q-on-a-QWERTY-keyboard" trigger a Ctrl-Q binding? I
> presume this is a sort of backward compatibility feature, but I wonder
> if the application should be in control of this? In any case, in the
> current implementation, this approach causes a problem, which I will
> describe below.
>
> Handling of keybindings happens by letting each JComponent track a
> number of "input maps", mapping keystrokes to actions. There is a map
> for when the component is focused, when it is the ancestor of a focused
> component, and when the component is in a focused window. The first two
> mappings are resolved by traversing the component hierarchy and letting
> each component check its maps, the latter map is resolved by
> KeyboardManager, which keeps one big map of bindings for each window.
>
> In summary, resolving keypress works like this:
>
>   1. SwingUtilities.processKeyBindings() / JComponent.processKeyBindings():
>      Starting with the focused component, going upwards, each component
>      checks its keybindings for the current keyEvent. Each component first
>      checks against the extended keycode, then the normal keycode.
>
>   2. KeyboardManager.fireKeyboardAction(), via JComponent.processKeyBindingsForAllComponents():
>      In the map of WHEN_IN_FOCUSED_WINDOW bindings kept by
>      KeyboardManager for the current window, the key event is looked up
>      and, if found, forwarded to the right JComponent. Again, this checks
>      the extended keycode first, then the normal keycode.
>
>   3. KeyboardManager.fireKeyboardAction() / JMenuBar.processKeyBinding():
>      For each JMenuBar in the current window (tracked in a separate map
>      by KeyboardManager), JMenuBar.processKeyBinding() is called, which
>      recurses through the menu hierarchy to check the
>      WHEN_IN_FOCUSED_WINDOW keybindings of each menu item. I assume this
>      is needed because children of JMenus are not added to the hierarchy
>      directly, but through a (detached) JPopupMenu instance.
>
> Note that in step 1, it actually seems like the component hierarchy is
> traversed in both SwingUtilities.processKeyBindings() as well as in
> JComponent.processKeyBindings(), which seems like it is doing double
> work (and applies the WHEN_FOCUSED maps even to ancestors of the focused
> component). However, this doesn't seem relevant to the subject of this
> mail, so I'll not investigate this further here.
>
> In my examples below, I will be using the Belgian AZERTY layout. For
> these examples, the only relevant changes are that the Q is swapped with
> the A, and the W is swapped with the Z.
>
> This resolution process has, AFAICS, two problems:
>
>   1. A given shortcut can be triggered by two keys. For example, when
>      using the Belgian AZERTY layout, a Ctrl-Q shortcut will be triggered
>      by both CTRL-Q (matching the extended keycode) as well as by CTRL-A
>      (the key in the same place as the Q on a QWERTY keyboard, by
>      matching the normal keycode).
>
>      I guess this is really intended as a feature, but it might be
>      confusing. One unintended side effect seems to be that, on OSX, some
>      keyboard shortcuts (such as Ctrl-, to open preferences) are handled
>      by the windowing system and passed to the application using a
>      seperate notification (e.g. "Open your preferences"), in *addition*
>      to passing the original keystroke to the application. This was
>      reported in https://bugs.openjdk.java.net/browse/JDK-8019498 where
>      a keybinding for Ctrl-M was triggered by pressing Ctrl-, in
>      *addition* to opening the preferences. Note that the preferences
>      event handling uses some OSX specific code, e.g.
>      http://hg.openjdk.java.net/jdk9/jdk9/jdk/file/c021b855f51e/src/java.desktop/macosx/classes/com/apple/eawt/_AppEventHandler.java#l192
>
>   2. The second problem is that a component that is checked earlier in
>      the resolution order can "hijack" keystrokes from later components.
>      Normally, this is intentional - WHEN_FOCUSED and
>      WHEN_ANCESTOR_OF_FOCUSED_COMPONENTS favor the focused component, or
>      components close to them, and for WHEN_IN_FOCUSED_WINDOW bindings,
>      no duplicate bindings should exist.
>
>      However, because each step in the resolution matches both the
>      extended and normal keycode, a component can hijack the even using
>      the "backward compatible" normal keycode matching, even when a later
>      component would have succesfully matched against the extended
>      keycode.
>
>      For example, when using a Belgian AZERTY layout, a button in the
>      window binds Ctrl-Z, and a menu item binds Ctrl-W, you would expect
>      both keypresses to trigger the corresponding action. However, in
>      practice the button will match the Ctrl-W keypress too (by matching
>      the normal keycode), preventing the menu bar (which comes later in
>      the resolution procedure) from matching the keypress, so both
>      Ctrl-W and Ctrl-Z trigger the button.
>
> The latter problem is of course the bigger one. The obvious solution is
> to run through the entire resolution twice, once matching the extended
> keycode, then once more matching the normal keycode. To fix the first
> problem too, the last step could perhaps be made optional, or perhaps
> KeyStroke can be modified to match the normal keycode, extended keycode,
> or both. Not sure if this can be done in a compatible way, though, since
> some of the methods involved are public or protected.
>
> To confirm my analisis, I wrote a small testcase (inline below, and at
> https://gist.github.com/matthijskooijman/4d016e7a9e3fb07d0699). It shows
> both of the problems outlined above. It contains a menu item bound to
> Ctrl-Q, which triggers on both Ctrl-Q and Ctrl-A when using the Belgian
> AZERTY layout.  It also contains a button binding to Ctrl-Z, which
> prevents a menu item bound to Ctrl-W from working when using the Belgian
> AZERTY layout. I have tested this on Linux, by selecting a different
> keyboard layout ("Input source") in the Gnome keyboard settings (without
> actually switching the keyboard).
>
> There seem to be a few related bug reports already. I suspect that most
> or all of these are caused by this problem, though not all of them have
> enough details to be sure.
>
> https://bugs.openjdk.java.net/browse/JDK-8087915
> https://bugs.openjdk.java.net/browse/JDK-8022079
> https://bugs.openjdk.java.net/browse/JDK-8019498
> https://bugs.openjdk.java.net/browse/JDK-8141096
>
> Interestingly, all of these reports are about OSX, but I'm pretty sure
> the problem I'm describing will occur on all platforms, being caused by
> platform-independent code. There might be additional OSX-specific
> problems, of course, but I do not have access to OSX to test.
>
> Gr.
>
> Matthijs
>
>
>
> import javax.swing.*;
> import java.awt.event.*;
> import java.awt.*;
>
> class Item extends JMenuItem {
>      public void addNotify() {
>          System.out.println("Add\n");
>      }
>      public Item(String s) { super(s); }
> }
>
> public class Test {
>      public static void main(String[] args) {
>          SwingUtilities.invokeLater(() -> {
>              JFrame frame = new JFrame();
>              JMenuBar bar = new JMenuBar();
>              JButton button = new JButton("Button Ctrl-Z");
>              JTextArea text = new JTextArea("");
>              JMenu menu = new JMenu("Menu");
>              JMenuItem item1 = new Item( "Item 1" );
>              JMenuItem item2 = new JMenuItem( "Item 2" );
>
>              item1.addActionListener((ActionEvent e) -> text.append("Menuitem 1\n"));
>              item2.addActionListener((ActionEvent e) -> text.append("Menuitem 2\n"));
>              button.addActionListener((ActionEvent e) -> text.append("Button click\n"));
>
>              item1.setAccelerator(KeyStroke.getKeyStroke('Q', KeyEvent.CTRL_MASK));
>              item2.setAccelerator(KeyStroke.getKeyStroke('W', KeyEvent.CTRL_MASK));
>              KeyStroke keyStroke = KeyStroke.getKeyStroke('Z', KeyEvent.CTRL_MASK);
>              ActionListener action = (ActionEvent) -> text.append("Button key\n");
>              button.registerKeyboardAction(action, "action", keyStroke, JComponent.WHEN_IN_FOCUSED_WINDOW);
>
>              menu.add(item1);
>              menu.add(item2);
>              bar.add(menu);
>
>              text.setEnabled(false);
>              frame.setLayout(new BorderLayout());
>              frame.add(button, BorderLayout.SOUTH);
>              frame.add(text, BorderLayout.CENTER);
>              frame.setJMenuBar(bar);
>              frame.setDefaultCloseOperation(WindowConstants.EXIT_ON_CLOSE);
>              frame.setSize(200,200);
>              frame.setVisible(true);
>          });
>      }
> }