Questions about the Hermetic Java project

Thu Jun 5 15:23:47 UTC 2025

On Thu, Jun 5, 2025 at 1:33 AM David Holmes <david.holmes at oracle.com> wrote:
>
> On 5/06/2025 1:33 am, Jiangli Zhou wrote:
> > Ok, still thanks for the thoughts, David.
> >
> > To summarize, here is what we can do for the current step:
> >
> > - Allow JNI_OnLoad_L and etc for JDK internal native libraries with
> > the dynamic linking case.
> > - JNI spec already allows JNI_OnLoad_L and etc for dynamically linked
> > native libraries. No spec change is required to the above.
>
> Sorry but where does the spec allow this? I thought we had agreed that
> this was a limitation of the implementation in regard to detecting
> whether a library is loaded dynamically or statically, and that we
> needed to document this case in the spec. ??

It's based on the following (referenced in
https://mail.openjdk.org/pipermail/leyden-dev/2025-May/002144.html message)
specified in
https://docs.oracle.com/en/java/javase/21/docs/specs/jni/invocation.html#support-for-statically-linked-libraries
:

Support for Statically Linked Libraries

"If dynamically linked library defines JNI_OnLoad_L and/or
JNI_OnUnload_L functions, these functions will be ignored."

There are two parts to the above. The first part indicates that the spec
allows JNI_OnLoad_L and/or JNI_OnUnload_L being defined in a dynamically
linked library.

The second part stating "these functions will be ignored" however does not
reflect the limitation of the implementation when handling implicitly
loaded native libraries. That part needs to be documented.

Please let me know if that's more clear.

Best,

Jiangli

>
> David
> -----
>
> > - JNI_OnLoad_L and etc should not be called for dynamically linked
> > native libraries when the library is loaded due to loadLibrary() by
> > default. If a dynamic library is already loaded as a dependency of
> > other native libraries, when loadLibrary() is called for the library,
> > JNI_OnLoad_L can be called. That is an existing behavior since JDK 8.
> > Need to document (in spec or release notes?).
> >
> > Best,
> > Jiangli
> >
> > On Tue, Jun 3, 2025 at 9:31 PM David Holmes <david.holmes at oracle.com>
wrote:
> >>
> >> On 4/06/2025 5:00 am, Jiangli Zhou wrote:
> >>> On Mon, Jun 2, 2025 at 6:22 PM David Holmes <david.holmes at oracle.com>
wrote:
> >>>>
> >>>> On 3/06/2025 9:29 am, Jiangli Zhou wrote:
> >>>>> On Sun, Jun 1, 2025 at 7:55 PM David Holmes <david.holmes at oracle.com>
wrote:
> >>>>>>
> >>>>>> On 31/05/2025 7:20 am, Jiangli Zhou wrote:
> >>>>>>> On Thu, May 29, 2025 at 11:54 PM David Holmes <
david.holmes at oracle.com> wrote:
> >>>>>>>>
> >>>>>>>> On 30/05/2025 9:26 am, Jiangli Zhou wrote:
> >>>>>>>>>
> >>>>>>>>> I just thought of one more thing related to the discussion now.
Any
> >>>>>>>>> concern if the implementation does not ignore JNI_OnLoad_L and
etc if
> >>>>>>>>> they are defined application's dynamically linked native
libraries? Or
> >>>>>>>>> that's unspecified behavior and it's up to the implement to
decide?
> >>>>>>>>
> >>>>>>>> For Internal libraries or external? For external you have to
follow the
> >>>>>>>> spec - if both methods exist you only want to execute one of
them.
> >>>>>>>
> >>>>>>> It's for the external (non-JDK) library that I'm a bit more
cautious.
> >>>>>>>
> >>>>>>> In the existing code in JDK mainline,
> >>>>>>>
https://github.com/openjdk/jdk/blob/3cc630985d47be6ba4cf991698e999f17dbde203/src/java.base/share/classes/jdk/internal/loader/NativeLibraries.java#L117
,
> >>>>>>> loadLibrary() first tries to find the built-in library using
> >>>>>>> JNI_OnLoad_L symbol (L is the library name). When dlsym is called
to
> >>>>>>> find the symbol from the main process, any of the already loaded
> >>>>>>> shared libraries are also searched, as described by the dlsym man
page
> >>>>>>> (included related part below).
> >>>>>>>
> >>>>>>> https://man7.org/linux/man-pages/man3/dlsym.3.html:
> >>>>>>>            RTLD_DEFAULT
> >>>>>>>                   Find the first occurrence of the desired symbol
using the
> >>>>>>>                   default shared object search order.  The search
will
> >>>>>>>                   include global symbols in the executable and its
> >>>>>>>                   dependencies, as well as symbols in shared
objects that
> >>>>>>>                   were dynamically loaded with the RTLD_GLOBAL
flag.
> >>>>>>>
> >>>>>>> I think it would be rare, it is possible to construct such case:
> >>>>>>>
> >>>>>>> There are user JNI libraries A and B, with B is built as a
dependency
> >>>>>>> of A. A defines JNI_OnLoad_A and JNI_OnLoad. B defines
JNI_OnLoad_B
> >>>>>>> and JNI_OnLoad. When A is being loaded using loadLibrary(),
> >>>>>>> loadLibrary() tries first to lookup JNI_OnLoad_A, which is not
found.
> >>>>>>> A is then loaded dynamically, which causes B being loaded
implicitly
> >>>>>>> as a dependency of A. Later when loadLibrary() is called for B,
> >>>>>>> JNI_OnLoad_B would be found and then called. This is an existing
> >>>>>>> behavior. I think it's an unspecified behavior and we don't need
to
> >>>>>>> add any additional checks to prevent JNI_OnLoad_B from being
called.
> >>>>>>
> >>>>>> That sounds like a significant design flaw to me. You can't
specify that
> >>>>>> JNI_OnLoad_L will only be called if L is statically linked, if the
> >>>>>> existence of JNI_OnLoad_L is used to infer that L is statically
linked!
> >>>>>> I would expect libraries to have both versions of the OnLoad
functions
> >>>>>> to allow for them being statically or dynamically linked - which
the
> >>>>>> spec allows for by saying the alternate variant is ignored. But
then the
> >>>>>> JDK will execute the wrong method if it finds JNI_OnLoad_L in a
> >>>>>> dynamically linked library.
> >>>>>
> >>>>> JNI_OnLoad_L is used to determine if a requested JNI native library
is
> >>>>> a built-in (statically linked) library. Thus, it can avoid the
> >>>>> operation of explicit loading for the shared library, e.g. with
dlopen
> >>>>> on Linux. JNI_OnLoad_L is expected to provide the same
implementation
> >>>>> as JNI_OnLoad besides being used as an identifier of a built-in
> >>>>> library, IIUC.
> >>>>>
> >>>>> In the scenario that I described in the previous message, when a JNI
> >>>>> shared library is already implicitly loaded as a dependency of
another
> >>>>> native library, dlopen for explicitly loading the shared library is
> >>>>> not necessary. From the implementation point of view, the code seems
> >>>>> to have been doing the right thing since JDK 8. I did some search
and
> >>>>> found
https://stackoverflow.com/questions/32302262/does-dlopen-re-load-already-loaded-dependencies-if-so-what-are-the-implication
,
> >>>>> which point out to the following in POSIX spec (latest
> >>>>> https://pubs.opengroup.org/onlinepubs/9799919799/):
> >>>>>
> >>>>> "Only a single copy of an executable object file shall be brought
into
> >>>>> the address space, even if dlopen() is invoked multiple times in
> >>>>> reference to the executable object file, and even if different
> >>>>> pathnames are used to reference the executable object file."
> >>>>>
> >>>>> Then avoiding calling dlopen for the already loaded native library
> >>>>> doesn't cause any undesired side effects.
> >>>>>
> >>>>> Perhaps we can update the JNI spec to include the "already loaded"
JNI
> >>>>> native libraries case, in addition to the built-in native libraries,
> >>>>> regarding JNI_OnLoad_L. Also clarify the "these functions will be
> >>>>> ignored" part in JNI spec for the dynamically linked libraries.
> >>>>>
> >>>>> "If dynamically linked library defines JNI_OnLoad_L and/or
> >>>>> JNI_OnUnload_L functions, these functions will be ignored."
> >>>>>
> >>>>> Thoughts?
> >>>>
> >>>> The problem is, as I see it, that the spec assumes that if it finds
the
> >>>> JNI_OnLoad_L symbol then L must be a statically linked library. But
that
> >>>> ignores the case you highlight where L was implicitly dynamically
loaded
> >>>> as a dependency on another library. Hence the existence test for the
> >>>> symbol is not sufficient to determine if L was statically linked.
> >>>
> >>> Right.
> >>>
> >>>>
> >>>> If JNI_OnLoad_L and JNI_OnLoad were guaranteed to always do exactly
the
> >>>> same thing it would not make any practical difference, but surely
that
> >>>> is not always the case? I can certainly postulate the existence of a
> >>>> library that only needs the "on load" hook for the statically linked
> >>>> case, in which case invoking it when actually dynamically linked
would
> >>>> be incorrect and potentially harmful.
> >>>
> >>> Hmmm, I haven't thought of such a case. David, could you please give a
> >>> concrete example for the case?
> >>
> >> I don't have a concrete example, that's why I just said I could
> >> "postulate the existence" of such a case. :)
> >>
> >>>>
> >>>> It seems we lack a way to know if a given library is truly statically
> >>>> linked, or to be advised when a library is implicitly loaded as a
> >>>> dependency. I am no expert on linking but I've been unable to locate
any
> >>>> information on programmatically determining these conditions.
> >>>
> >>> I haven't found anything either.
> >>>
> >>>>
> >>>> If there is no real solution then documenting the problem may be all
we
> >>>> can do.
> >>>
> >>> That sounds reasonable to me.
> >>>
> >>> I mentioned the new mailing list discussions to Ron, Alan and Magnus
> >>> today during the hermetic Java meeting. They may have follow up
> >>> thoughts.
> >>
> >> Okay.
> >>
> >> David
> >> -----
> >>
> >>
> >>> Thanks!
> >>> Jiangli
> >>>
> >>>>
> >>>> David
> >>>> -----
> >>>>
> >>>>> Thanks!
> >>>>> Jiangli
> >>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/leyden-dev/attachments/20250605/501874c3/attachment.htm>