RFR: 8323664: java/awt/font/JNICheck/FreeTypeScalerJNICheck.java still fails with JNI warning on some Windows configurations

[Christoph Langer]

On Fri, 12 Jan 2024 17:47:23 GMT, Christoph Langer <clanger at openjdk.org> wrote:

> This picks up fixing the issue of [JDK-8276809](https://bugs.openjdk.org/browse/JDK-8276809) again. A fix had been integrated with #17224 but @prrace had concerns and so it was backed out.
> 
> I have now spent quite some thoughts into the problem and end up with the [initial commit](https://github.com/openjdk/jdk/pull/6306/commits/5d18a76cb967e9ede6394cbd6c28bb61facf785c) of #6306 as the most elegant and least intrusive solution.
> 
> Why is this?
> 
> The JNI warning we observe in the test is:
> `[WARNING in native method: JNI call made without checking exceptions when required to from CallStaticVoidMethodV
> 	at sun.awt.Win32GraphicsEnvironment.initDisplay(java.desktop at 22.0.1-internal/Native Method)
> 	at sun.awt.Win32GraphicsEnvironment.initDisplayWrapper(java.desktop at 22.0.1-internal/Win32GraphicsEnvironment.java:95)
> 	at sun.awt.Win32GraphicsEnvironment.<clinit>(java.desktop at 22.0.1-internal/Win32GraphicsEnvironment.java:63)
>         ...
>         at FreeTypeScalerJNICheck.runTest(FreeTypeScalerJNICheck.java:53)
> 	at FreeTypeScalerJNICheck.main(FreeTypeScalerJNICheck.java:44)`
> 
> This happens because obviously the test FreeTypeScalerJNICheck runs with `-Xcheck:jni` and in the scenario where we're observing the warning, a missing exception check for the JNI call to `sun.awt.Win32GraphicsEnvironment::dwmCompositionChanged` at awt_Win32GraphicsEnv.cpp#L129
> https://github.com/openjdk/jdk/blob/c5e72450966ad50d57a8d22e9d634bfcb319aee9/src/java.desktop/windows/native/libawt/windows/awt_Win32GraphicsEnv.cpp#L129 is airing up. Omitting the exception check would not be a problem if it could be guaranteed that after this call no other JNI->Java call was being made. But seemingly in this very particular configuration on some of our Windows servers, there must be JNI->Java calls that follow the call to `sun.awt.Win32GraphicsEnvironment::dwmCompositionChanged`, likely from the subsequent call to [initScreens](https://github.com/openjdk/jdk/blob/c5e72450966ad50d57a8d22e9d634bfcb319aee9/src/java.desktop/windows/native/libawt/windows/awt_Win32GraphicsEnv.cpp#L149) in `sun.awt.Win32GraphicsEnvironment::initDisplay`. https://github.com/openjdk/jdk/blob/8b6293f6bfb7b7628c6604e6c44401fc96d85cf4/src/java.desktop/windows/native/libawt/windows/awt_Win32GraphicsEnv.cpp#L141 Maybe the usual control flow would call the wrapping native method `DWMIsCompositi
 onEnabled()` from somewhere else initially such that the initialization of `Win32GraphicsEnvironment` would not go through `initScreens` or similar.
> ...

OK, finally it's understood what's happening. Here's the story:

We basically run into the assertion reported in [JDK-8185862](https://bugs.openjdk.org/browse/JDK-8185862). It causes the JNI warning through a [CallStaticBooleanMethod](https://github.com/openjdk/jdk/blob/bccd823c8e40863bed70ff5b24772843203871a5/src/java.desktop/windows/native/libawt/windows/awt_Debug.cpp#L163C25-L163C48) in its depths. In detail: Via [initScreens](https://github.com/openjdk/jdk/blob/bccd823c8e40863bed70ff5b24772843203871a5/src/java.desktop/windows/native/libawt/windows/awt_Win32GraphicsEnv.cpp#L40), called by [initDisplay](https://github.com/openjdk/jdk/blob/bccd823c8e40863bed70ff5b24772843203871a5/src/java.desktop/windows/native/libawt/windows/awt_Win32GraphicsEnv.cpp#L149), we call [Devices::UpdateInstance](https://github.com/openjdk/jdk/blob/bccd823c8e40863bed70ff5b24772843203871a5/src/java.desktop/windows/native/libawt/windows/Devices.cpp#L216) that triggers [AwtWin32GraphicsDevice::Initialize](https://github.com/openjdk/jdk/blob/bccd823c8e40863bed70ff5b2477284
 3203871a5/src/java.desktop/windows/native/libawt/windows/awt_Win32GraphicsDevice.cpp#L163) and here we run into the assertion. The assertion is fired by [this VERIFY](https://github.com/openjdk/jdk/blob/bccd823c8e40863bed70ff5b24772843203871a5/src/java.desktop/windows/native/libawt/windows/awt_Win32GraphicsDevice.cpp#L183). In Windows we have an [assertion callback](https://github.com/openjdk/jdk/blob/bccd823c8e40863bed70ff5b24772843203871a5/src/java.desktop/windows/native/libawt/windows/awt_Debug.cpp#L142) installed and it calls Java in [isHeadless()](https://github.com/openjdk/jdk/blob/bccd823c8e40863bed70ff5b24772843203871a5/src/java.desktop/windows/native/libawt/windows/awt_Debug.cpp#L183) which brings up the assertion.

So, how to continue? I think the assertion itself should be addressed through [JDK-8185862](https://bugs.openjdk.org/browse/JDK-8185862). We could maybe make the assert [callback](https://github.com/openjdk/jdk/blob/bccd823c8e40863bed70ff5b24772843203871a5/src/java.desktop/windows/native/libawt/windows/awt_Debug.cpp#L171) more robust by checking for a pending exception before [calling the isHeadless function](https://github.com/openjdk/jdk/blob/bccd823c8e40863bed70ff5b24772843203871a5/src/java.desktop/windows/native/libawt/windows/awt_Debug.cpp#L163C25-L163C48). But I think the currently proposed exception check on calling dwmCompositionChanged is also a viable solution. Or we could remove the dwmCompositionChanged call altogether and set the isDWMCompositionEnabled field through JNI's setField function directly. Furthermore, the test FreeTypeScalerJNICheck could also check for "Assertion" in its output, in addition to "Warning". Then it would bail out on both, JNI Warnings and AWT a
 ssertions.

WDYT?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/17404#issuecomment-1908491604