RFR: 8346433: Cannot use DllMain in hotspot for static builds

[Thomas Stuefe]

On Thu, 19 Dec 2024 07:58:52 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> To be able to properly support static builds on Windows in [JDK-8346377](https://bugs.openjdk.org/browse/JDK-8346377), we cannot use DllMain, for two reasons:
>> 
>> 1) This is not called for statically linked libraries, and
>> 2) There are multiple DllMain definitions throughout the JDK native libraries, causing name collisions.
>> 
>> While it could have been possible to keep the DllMain function for non-static builds and just use an alternative solution for static builds, I think it is preferable to have a single solution that works as well for both static and dynamic builds.
>> 
>> The DllMain in hotspot is doing work both at DLL load time, and at DLL unload time. Let's go through them to see why this patch is safe.
>> 
>> During DLL load time, the library handle is set, and the hi-res timer is initialized. The `pre_initialize` method from `WindowsDbgHelp` and `SymbolEngine` is called. These two are basically identical; both setup a critical section (a Windows mutex). However, this mutex is only used from a stack local guard object, which is allocated only when calls are made into these classes, which are not happening at bootstrapping time, so this shift in initialization time is harmless.
>> 
>> That shift in time is also very small. The `DllMain` method is called by Windows when the DLL is loaded (by libjli), and the very first thing that JLI does after that is to call `JNI_CreateJavaVM`, which ends up calling  `Threads::create_vm`, which calls `os::init` early on.
>
> Thinking about this, decorating all DllMain in JDK code with something like `DllMain_<libname>`, then in the static launcher dlsym'ing them and calling them in the typical order they would be called in were the JVM started normally... would that be a solution? This would be simple to maintain.
> 
> Another solution would be global RAII objects - but that only works in C++ and does not offer the fine grained order-of-init control a simple list of called DllMains would give us.

> @tstuefe @dholmes-ora Does this look okay now?

Hmm, I don't think os::init is early enough. DllMain gets called at library loading time, long before JVM_CreateJavaVM is called. os::init gets called in the (early) mids of CreateJavaVM. At the very least, it should be called at the entry of CreateJavaVM.

I am not only thinking of the things DllMain on Windows does now. This also can serve as a blueprint to handle C++ dynamic initialization: global C++ objects (of which we have a number) are instantiated in random order when the dynamic library loads. With static linking, they will be created when the static binary is loaded. That is a change in behavior. We may run into cases where that is too early. And even if it is not, we may want to shift things that are currently randomly initialized to the new preinit anyway, just to have a clear deterministic order of initialization for these things.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/22793#issuecomment-2590008366