RFR: 8346388: Cannot use DllMain in libawt for static builds

[Magnus Ihse Bursie]

On Wed, 18 Dec 2024 19:35:09 GMT, Phil Race <prr at openjdk.org> wrote:

> I'm not sure it seems right to have every static destructor need to know what needs to be done to exit the DLL.

That is an over-generalization of the problem at hand that is not really true. What we have is an unfortunate coupling between the API from one DLL (JVM.DLL) and another (AWT.DLL), and that only happens in three specific static objects, each of which are very specialized.

Another way of describing the problem is that the AWT debug system should not use JVM raw monitors, since they could happen to be used when the JVM is unloaded and the calls will fail. In fact, my first solution attempt for this bug was to replace the locking mechanism with an own implementation of pthread mutexes/critical sections. (Basically a simple re-implementation of JVM raw monitors.)

The current code tries to solve this by "disabling" the locks when awt.dll is unloaded, but technically it should be disabled when the jvm.dll module is unloaded -- the fact that this work is just due to the circumstance that they typically are unloaded together when the process ends. I don't know if it would be possible to trigger some kind of race, where jvm.dll is unloaded but some thread is still running the awt debug code, like some of the `operator delete` stuff (which does take a lock).

I think the entire design leaves a lot to be desired, but I thought this was the most elegant to the solution to the problem at hand. As a second hand alternative, I'd suggest replacing JVM raw monitors with a hand-rolled implementation.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/22790#issuecomment-2552165713