Questions about the Hermetic Java project

Thu Jun 5 08:33:08 UTC 2025

On 5/06/2025 1:33 am, Jiangli Zhou wrote:
> Ok, still thanks for the thoughts, David.
> 
> To summarize, here is what we can do for the current step:
> 
> - Allow JNI_OnLoad_L and etc for JDK internal native libraries with
> the dynamic linking case.
> - JNI spec already allows JNI_OnLoad_L and etc for dynamically linked
> native libraries. No spec change is required to the above.

Sorry but where does the spec allow this? I thought we had agreed that 
this was a limitation of the implementation in regard to detecting 
whether a library is loaded dynamically or statically, and that we 
needed to document this case in the spec. ??

David
-----

> - JNI_OnLoad_L and etc should not be called for dynamically linked
> native libraries when the library is loaded due to loadLibrary() by
> default. If a dynamic library is already loaded as a dependency of
> other native libraries, when loadLibrary() is called for the library,
> JNI_OnLoad_L can be called. That is an existing behavior since JDK 8.
> Need to document (in spec or release notes?).
> 
> Best,
> Jiangli
> 
> On Tue, Jun 3, 2025 at 9:31 PM David Holmes <david.holmes at oracle.com> wrote:
>>
>> On 4/06/2025 5:00 am, Jiangli Zhou wrote:
>>> On Mon, Jun 2, 2025 at 6:22 PM David Holmes <david.holmes at oracle.com> wrote:
>>>>
>>>> On 3/06/2025 9:29 am, Jiangli Zhou wrote:
>>>>> On Sun, Jun 1, 2025 at 7:55 PM David Holmes <david.holmes at oracle.com> wrote:
>>>>>>
>>>>>> On 31/05/2025 7:20 am, Jiangli Zhou wrote:
>>>>>>> On Thu, May 29, 2025 at 11:54 PM David Holmes <david.holmes at oracle.com> wrote:
>>>>>>>>
>>>>>>>> On 30/05/2025 9:26 am, Jiangli Zhou wrote:
>>>>>>>>>
>>>>>>>>> I just thought of one more thing related to the discussion now. Any
>>>>>>>>> concern if the implementation does not ignore JNI_OnLoad_L and etc if
>>>>>>>>> they are defined application's dynamically linked native libraries? Or
>>>>>>>>> that's unspecified behavior and it's up to the implement to decide?
>>>>>>>>
>>>>>>>> For Internal libraries or external? For external you have to follow the
>>>>>>>> spec - if both methods exist you only want to execute one of them.
>>>>>>>
>>>>>>> It's for the external (non-JDK) library that I'm a bit more cautious.
>>>>>>>
>>>>>>> In the existing code in JDK mainline,
>>>>>>> https://github.com/openjdk/jdk/blob/3cc630985d47be6ba4cf991698e999f17dbde203/src/java.base/share/classes/jdk/internal/loader/NativeLibraries.java#L117,
>>>>>>> loadLibrary() first tries to find the built-in library using
>>>>>>> JNI_OnLoad_L symbol (L is the library name). When dlsym is called to
>>>>>>> find the symbol from the main process, any of the already loaded
>>>>>>> shared libraries are also searched, as described by the dlsym man page
>>>>>>> (included related part below).
>>>>>>>
>>>>>>> https://man7.org/linux/man-pages/man3/dlsym.3.html:
>>>>>>>            RTLD_DEFAULT
>>>>>>>                   Find the first occurrence of the desired symbol using the
>>>>>>>                   default shared object search order.  The search will
>>>>>>>                   include global symbols in the executable and its
>>>>>>>                   dependencies, as well as symbols in shared objects that
>>>>>>>                   were dynamically loaded with the RTLD_GLOBAL flag.
>>>>>>>
>>>>>>> I think it would be rare, it is possible to construct such case:
>>>>>>>
>>>>>>> There are user JNI libraries A and B, with B is built as a dependency
>>>>>>> of A. A defines JNI_OnLoad_A and JNI_OnLoad. B defines JNI_OnLoad_B
>>>>>>> and JNI_OnLoad. When A is being loaded using loadLibrary(),
>>>>>>> loadLibrary() tries first to lookup JNI_OnLoad_A, which is not found.
>>>>>>> A is then loaded dynamically, which causes B being loaded implicitly
>>>>>>> as a dependency of A. Later when loadLibrary() is called for B,
>>>>>>> JNI_OnLoad_B would be found and then called. This is an existing
>>>>>>> behavior. I think it's an unspecified behavior and we don't need to
>>>>>>> add any additional checks to prevent JNI_OnLoad_B from being called.
>>>>>>
>>>>>> That sounds like a significant design flaw to me. You can't specify that
>>>>>> JNI_OnLoad_L will only be called if L is statically linked, if the
>>>>>> existence of JNI_OnLoad_L is used to infer that L is statically linked!
>>>>>> I would expect libraries to have both versions of the OnLoad functions
>>>>>> to allow for them being statically or dynamically linked - which the
>>>>>> spec allows for by saying the alternate variant is ignored. But then the
>>>>>> JDK will execute the wrong method if it finds JNI_OnLoad_L in a
>>>>>> dynamically linked library.
>>>>>
>>>>> JNI_OnLoad_L is used to determine if a requested JNI native library is
>>>>> a built-in (statically linked) library. Thus, it can avoid the
>>>>> operation of explicit loading for the shared library, e.g. with dlopen
>>>>> on Linux. JNI_OnLoad_L is expected to provide the same implementation
>>>>> as JNI_OnLoad besides being used as an identifier of a built-in
>>>>> library, IIUC.
>>>>>
>>>>> In the scenario that I described in the previous message, when a JNI
>>>>> shared library is already implicitly loaded as a dependency of another
>>>>> native library, dlopen for explicitly loading the shared library is
>>>>> not necessary. From the implementation point of view, the code seems
>>>>> to have been doing the right thing since JDK 8. I did some search and
>>>>> found https://stackoverflow.com/questions/32302262/does-dlopen-re-load-already-loaded-dependencies-if-so-what-are-the-implication,
>>>>> which point out to the following in POSIX spec (latest
>>>>> https://pubs.opengroup.org/onlinepubs/9799919799/):
>>>>>
>>>>> "Only a single copy of an executable object file shall be brought into
>>>>> the address space, even if dlopen() is invoked multiple times in
>>>>> reference to the executable object file, and even if different
>>>>> pathnames are used to reference the executable object file."
>>>>>
>>>>> Then avoiding calling dlopen for the already loaded native library
>>>>> doesn't cause any undesired side effects.
>>>>>
>>>>> Perhaps we can update the JNI spec to include the "already loaded" JNI
>>>>> native libraries case, in addition to the built-in native libraries,
>>>>> regarding JNI_OnLoad_L. Also clarify the "these functions will be
>>>>> ignored" part in JNI spec for the dynamically linked libraries.
>>>>>
>>>>> "If dynamically linked library defines JNI_OnLoad_L and/or
>>>>> JNI_OnUnload_L functions, these functions will be ignored."
>>>>>
>>>>> Thoughts?
>>>>
>>>> The problem is, as I see it, that the spec assumes that if it finds the
>>>> JNI_OnLoad_L symbol then L must be a statically linked library. But that
>>>> ignores the case you highlight where L was implicitly dynamically loaded
>>>> as a dependency on another library. Hence the existence test for the
>>>> symbol is not sufficient to determine if L was statically linked.
>>>
>>> Right.
>>>
>>>>
>>>> If JNI_OnLoad_L and JNI_OnLoad were guaranteed to always do exactly the
>>>> same thing it would not make any practical difference, but surely that
>>>> is not always the case? I can certainly postulate the existence of a
>>>> library that only needs the "on load" hook for the statically linked
>>>> case, in which case invoking it when actually dynamically linked would
>>>> be incorrect and potentially harmful.
>>>
>>> Hmmm, I haven't thought of such a case. David, could you please give a
>>> concrete example for the case?
>>
>> I don't have a concrete example, that's why I just said I could
>> "postulate the existence" of such a case. :)
>>
>>>>
>>>> It seems we lack a way to know if a given library is truly statically
>>>> linked, or to be advised when a library is implicitly loaded as a
>>>> dependency. I am no expert on linking but I've been unable to locate any
>>>> information on programmatically determining these conditions.
>>>
>>> I haven't found anything either.
>>>
>>>>
>>>> If there is no real solution then documenting the problem may be all we
>>>> can do.
>>>
>>> That sounds reasonable to me.
>>>
>>> I mentioned the new mailing list discussions to Ron, Alan and Magnus
>>> today during the hermetic Java meeting. They may have follow up
>>> thoughts.
>>
>> Okay.
>>
>> David
>> -----
>>
>>
>>> Thanks!
>>> Jiangli
>>>
>>>>
>>>> David
>>>> -----
>>>>
>>>>> Thanks!
>>>>> Jiangli
>>