Swing works, but OpenJFX applications not working with screen readers

[Philip Race]

Those tools as I think you've worked out are Windows-only tools which 
use the Windows-only
Java Access Bridge API that ships with JDK.
AccessBridge was designed and built 25 years ago (!) to enable Assistive 
Technologies to
work for Swing applications at a time when no platform could do this.
ATs like JAWS (and others) implemented support for AccessBridge and 
still provide it today
(we still get bug reports from customers using it)
So Java + Swing was a long way ahead of the platforms.

But JavaFX does not use AccessBridge.
By the time it was developed platforms were starting to add AT APIs.
So instead JavaFX implements support for the platform accessibility APIs 
meaning
Narrator on Windows and VoiceOver on macOS.

Therefore JavaFX will not work with those JDK tools which are tied to 
AccessBridge.

 > with the screen reader JAWS running, the 
Platform.accessibilityActiveProperty() is "true

I guess this is because JAWS registers itself with windows as the active AT.
Whether JAWS interjects itself into Narrator APIs would depend on 
whether Windows
supports that. If it doesn't then JAWS probably has to do old-fashioned 
screen scraping
and I don't know how well that would work.

A11Y is usually tested at the application level - whereas JavaFX and 
Swing are both
toolkits with which to build applications - but we do try to make things 
work well
with the ATs that each of them use, so Narrator is supposed to work for 
JavaFX
applications on Windows.


-phil.

On 3/5/23 12:36 AM, Don Johnson wrote:
> Hi All,
>
> This is accessibility API question. Swing applications like IntelliJ 
> IDEA appear in the utilities jaccesswalker, jaccessinspector (JDK/bin) 
> and access-bridge-explorer (.NET app at 
> https://github.com/google/access-bridge-explorer.git). However, no 
> scene graph or accessibility messages in these utilities appear for 
> any OpenJFX application I have written. A screen reader needs both to 
> know the scene graph and be able to send and receive accessibility 
> messages to perform automation and read control descriptions.
>
> I have put these values in the "accessibility.properties" file in the 
> JDK lib folder:
>
>     assistive_technologies=com.sun.java.accessibility.AccessBridge
>     screen_magnifier_present=true
>     screen_reader_present=true
>
> In different test OpenJFX applications running on Windows machines, 
> with the screen reader JAWS running, the 
> Platform.accessibilityActiveProperty() is "true." When JAWS is not 
> running, the value is "false." However, the screen reader cannot 
> communicate with the OpenJFX applications.
>
> Why are the OpenJFX applications not communicating across the 
> communication channel facilitated by the WindowsAccessBridge-64.dll 
> and JavaAccessBridge.dll? Incidentally, the Java Access Bridge is 
> enabled with jabswitch.exe. There is some type of Windows IPC used by 
> the bridge (COM?) once it is enabled. This bridge facilitates 
> communication between the screen reader and a Java application running 
> inside of a JVM.
>
> I've also tried explicitly setting roles etc. even though OpenJFX 
> graphical object already have default values. For example in the 
> "bouncing ball" sample program I set,
>
>         setAccessibleRole(AccessibleRole.BUTTON);
>         setAccessibleRoleDescription("Bouncing Ball");
>         setAccessibleText("I'm a bouncing ball");
>         setAccessibleHelp("This is bouncing ball, use the primary 
> action to start animation");
>
> Does anyone have an OpenJFX sample program the shows up in the 
> jaccesswalker and jaccessinspector utilities and works with a screen 
> reader like JAWS or NVDA? This YouTube presentation implies 
> accessibility is already built into OpenJFX, but I can't get it to 
> work, see https://youtu.be/iUPPEkD9H1I "The New JavaFX Accessibility 
> API." My applications must comply with Section 508 of the 
> Rehabilitation Act of 1973 but cannot so far because of this OpenJFX 
> accessibility problem.
>
> Thanks,
> Don