KeyCode in the javafx.scene.robot.Robot interface

[Martin Fox]

Mark,

You may already know this but before JavaFX 21 the Mac and Windows Robot code had some long-standing bugs with non-US keyboards. Linux is in better shape but you can encounter problems if the user has installed multiple layouts (I have a PR pending to fix that). That might explain some of the flakey behavior you’ve been seeing. Your approach might also invoke an unexpected dead key depending on what modifiers you’re probing.

With one exception JavaFX doesn’t have any facility for querying the keyboard layout. It relies on the platform code to take in keyboard events and translate them to JavaFX events on the fly. The exception is the internal call Toolkit.getKeyCodeForChar which attempts to map from a character to a KeyCode and is used to match shortcuts specified as KeyCharacterCombinations. Unfortunately the current implementation on Mac and Linux can’t really be extended to cover the sort of testing you’re trying to do.

(On any OS the keyboard machinery is designed to take a series of physical keystrokes and produce a series of characters. Attempting to go in the opposite direction is fraught with complications particularly for punctuation and symbols. It’s faster and more reliable to wait for the user to type something and record what happens. That’s how getKeyCodeForChar is implemented on the Mac; until the user types something it yields no results. Linux is currently buggy but the PR to fix it adopts the same approach as the Mac.)

JavaFX needs a better framework for testing text entry and I’ve thought about how that might look. But my priorities might be different than yours. Based on the bugs I’ve seen the biggest issue is that developers only test on the platforms and layouts they use. Much of this is due to lack of awareness but it’s also a big hurdle to have to buy Mac, Windows, and Linux boxes and then manually switch between layouts to test. My ideal framework would be based on emulated keyboards so, say, a U.S. Windows developer could generate a key event stream that corresponds to a Spanish Mac layout.

That approach wouldn’t involve sending platform events through the OS. Extending the Robot to make it easier to, say, target punctuation and symbols via platform events would be nice to have but not essential (IMHO).

So, no, you’re not missing anything exposed or under the hood. The approach you’re taking is probably the best that can be done with the current API.

Martin

> On May 12, 2024, at 3:16 AM, Mark Raynsford <org.openjdk at io7m.com> wrote:
> 
> Hello!
> 
> I maintain a test harness for JavaFX applications:
> 
> https://www.github.com/io7m-com/xoanon
> 
> I expose an interface that uses the javafx.scene.robot.Robot
> interface internally, but I expose a slightly higher level
> API that allows for (amongst other things) typing text as strings
> on components rather than sending individual key codes.
> 
> The problem I immediately run into is that KeyCodes are keyboard
> layout specific, and there doesn't appear to be any way to detect
> what keyboard layout is in use. If I, for example, send the KeyCode for
> the `@` symbol, I'll actually get `"` on some keyboard layouts. This
> can cause some types of tests to fail, even though the code is correct.
> 
> Consider a test where an email address is typed into a text field, and
> then the form containing the field is checked to see if the text field
> accepted/rejected the text as being a valid email address. On some
> keyboard layouts, a test that sends the string "someone at example.com"
> will fail because what ends up in the text field is actually
> "someone"example.com".
> 
> I provide a workaround for this: At the start of the test suite run,
> xoanon will send every KeyCode to a text field one at a time and see
> what character appeared in the text field as a result. It then builds a
> mapping between characters to KeyCodes, and it can then do the
> translations as necessary when running the rest of the test suite:
> 
> https://github.com/io7m-com/xoanon?tab=readme-ov-file#keymap-generation
> 
> Unfortunately, this is still rather flaky. This often seems to fail more
> or less at random for reasons that aren't clear to me.
> 
> Is there some way the Robot interface could be extended to allow for
> typing strings? Failing that, is there some way JavaFX could advertise
> the underlying keyboard map? I feel like some underlying component must
> know the information I'm trying to infer, but it doesn't seem to be
> exposed anywhere that I've been able to find.
> 
> -- 
> Mark Raynsford | https://www.io7m.com