Swing works, but OpenJFX applications not working with screen readers

[Don Johnson]

Hi Phil,

I understand now that JavaFX does not work with the 25 year-old Java 
Access Bridge API. As far as JAWS using Windows Narrator to communicate 
with our Java/JavaFX geospatial application, I am going to have to do 
further investigation. We are in communication with Vispero/Freedom 
Scientific. Thank you for replying and for all the hard work you JavaFX 
developers put into improving the JavaFX API.

Don

On 3/5/2023 12:16 PM, Philip Race wrote:
> Those tools as I think you've worked out are Windows-only tools which 
> use the Windows-only
> Java Access Bridge API that ships with JDK.
> AccessBridge was designed and built 25 years ago (!) to enable 
> Assistive Technologies to
> work for Swing applications at a time when no platform could do this.
> ATs like JAWS (and others) implemented support for AccessBridge and 
> still provide it today
> (we still get bug reports from customers using it)
> So Java + Swing was a long way ahead of the platforms.
>
> But JavaFX does not use AccessBridge.
> By the time it was developed platforms were starting to add AT APIs.
> So instead JavaFX implements support for the platform accessibility 
> APIs meaning
> Narrator on Windows and VoiceOver on macOS.
>
> Therefore JavaFX will not work with those JDK tools which are tied to 
> AccessBridge.
>
> > with the screen reader JAWS running, the 
> Platform.accessibilityActiveProperty() is "true
>
> I guess this is because JAWS registers itself with windows as the 
> active AT.
> Whether JAWS interjects itself into Narrator APIs would depend on 
> whether Windows
> supports that. If it doesn't then JAWS probably has to do 
> old-fashioned screen scraping
> and I don't know how well that would work.
>
> A11Y is usually tested at the application level - whereas JavaFX and 
> Swing are both
> toolkits with which to build applications - but we do try to make 
> things work well
> with the ATs that each of them use, so Narrator is supposed to work 
> for JavaFX
> applications on Windows.
>
>
> -phil.
>
> On 3/5/23 12:36 AM, Don Johnson wrote:
>> Hi All,
>>
>> This is accessibility API question. Swing applications like IntelliJ 
>> IDEA appear in the utilities jaccesswalker, jaccessinspector 
>> (JDK/bin) and access-bridge-explorer (.NET app at 
>> https://github.com/google/access-bridge-explorer.git). However, no 
>> scene graph or accessibility messages in these utilities appear for 
>> any OpenJFX application I have written. A screen reader needs both to 
>> know the scene graph and be able to send and receive accessibility 
>> messages to perform automation and read control descriptions.
>>
>> I have put these values in the "accessibility.properties" file in the 
>> JDK lib folder:
>>
>> assistive_technologies=com.sun.java.accessibility.AccessBridge
>>     screen_magnifier_present=true
>>     screen_reader_present=true
>>
>> In different test OpenJFX applications running on Windows machines, 
>> with the screen reader JAWS running, the 
>> Platform.accessibilityActiveProperty() is "true." When JAWS is not 
>> running, the value is "false." However, the screen reader cannot 
>> communicate with the OpenJFX applications.
>>
>> Why are the OpenJFX applications not communicating across the 
>> communication channel facilitated by the WindowsAccessBridge-64.dll 
>> and JavaAccessBridge.dll? Incidentally, the Java Access Bridge is 
>> enabled with jabswitch.exe. There is some type of Windows IPC used by 
>> the bridge (COM?) once it is enabled. This bridge facilitates 
>> communication between the screen reader and a Java application 
>> running inside of a JVM.
>>
>> I've also tried explicitly setting roles etc. even though OpenJFX 
>> graphical object already have default values. For example in the 
>> "bouncing ball" sample program I set,
>>
>>         setAccessibleRole(AccessibleRole.BUTTON);
>>         setAccessibleRoleDescription("Bouncing Ball");
>>         setAccessibleText("I'm a bouncing ball");
>>         setAccessibleHelp("This is bouncing ball, use the primary 
>> action to start animation");
>>
>> Does anyone have an OpenJFX sample program the shows up in the 
>> jaccesswalker and jaccessinspector utilities and works with a screen 
>> reader like JAWS or NVDA? This YouTube presentation implies 
>> accessibility is already built into OpenJFX, but I can't get it to 
>> work, see https://youtu.be/iUPPEkD9H1I "The New JavaFX Accessibility 
>> API." My applications must comply with Section 508 of the 
>> Rehabilitation Act of 1973 but cannot so far because of this OpenJFX 
>> accessibility problem.
>>
>> Thanks,
>> Don
>