java 17 libsvml.so conflict with user app

[Vitaly Davidovich]

Thanks all! At some point, we'd actually like to play around with the
Vector API but need to get our baseline ducks in a row first :).

On Tue, Oct 26, 2021 at 2:19 PM Paul Sandoz <paul.sandoz at oracle.com> wrote:

> I created https://bugs.openjdk.java.net/browse/JDK-8276025
>
> Magnus, I assigned to you so you can comment regarding build/infra, and we
> can adjust the issue’s component accordingly.
>
> Paul.
>
> > On Oct 26, 2021, at 10:56 AM, Paul Sandoz <paul.sandoz at oracle.com>
> wrote:
> >
> > Unfortunately, UseVectorStubs is set by modules.cpp after the stub
> generation code is run in stubGenerator_x86_64.cpp, so we cannot reliably
> use that flag to control the opening of the library. I was hoping that by
> default we could avoid stub loading with that flag triggered by the module
> not being visible. Hence you would not have to declare any flags.
> >
> > I am not that familiar with hotspot runtime, but there are two phases to
> compute stubs. We would need to introduce a third, which seems intrusive
> and I don’t know what the implications are.
> >
> > I think we need to explore the route Magnus is suggesting, and making
> this more private to HotSpot.
> >
> > Paul.
> >
> >> On Oct 26, 2021, at 9:40 AM, Vitaly Davidovich <vitalyd at gmail.com>
> wrote:
> >>
> >> Hi Paul,
> >>
> >>
> >> On Tue, Oct 26, 2021 at 11:54 AM Paul Sandoz <paul.sandoz at oracle.com>
> wrote:
> >> Hi,
> >>
> >> The Vector API is only enabled when the jdk.incibator.vector module is
> installed in the image and is visible. If the module is not installed then
> the svml library will not be present. That is one possible work around.
> >>
> >> If you run a simple program with logging we should see:
> >>
> >> $java --add-modules=jdk.incubator.vector -Xlog:compilation:stdout A
> >> [0.051s][info][compilation] EnableVectorSupport=true
> >> [0.051s][info][compilation] EnableVectorReboxing=true
> >> [0.051s][info][compilation] EnableVectorAggressiveReboxing=true
> >> [0.051s][info][compilation] UseVectorStubs=true
> >> WARNING: Using incubator modules: jdk.incubator.vector
> >> Hello World
> >>
> >> The code Maurizio linked to is not guarded by the flag UseVectorStubs.
> I think that is an issue and you should be able to use that flag at least
> as a work around. I will explore that.
> >> UseVectorStubs appears to be marked experimental.  While we could try
> that as a workaround (I take it you're suggesting setting
> -XX:+UnlockExperimentalVMOptions -XX:-UseVectorStubs), we don't generally
> open up experimental options.
> >>
> >> Magnus, is there anything we can do regarding library loading code.
> Renaming the library file as you suggested and perhaps other aspects?
> >> I think renaming the packaged svml lib and changing its symbol names
> (e.g. adding some jdk-specific prefix) would be the most robust solution.
> >>
> >> For now, we'll look into either removing the packaged libsvml.so (the
> linked code Maurizio showed seems to handle the library missing just fine)
> or removing the jdk.incubator.vector module, just to make progress on our
> jdk 17 testing.
> >>
> >> Thanks
> >>
> >>
> >> Intel contributed the SVML assembly source files to the source code
> base. That was a compromise given the original nature of the sources and
> their license.
> >>
> >> Paul.
> >>
> >>
> >>> On Oct 26, 2021, at 7:25 AM, Vitaly Davidovich <vitalyd at gmail.com>
> wrote:
> >>>
> >>> Hi Maurizio,
> >>>
> >>>
> >>> On Tue, Oct 26, 2021 at 6:16 AM Maurizio Cimadamore <
> >>> maurizio.cimadamore at oracle.com> wrote:
> >>>
> >>>> Looking at the code, it seems like the library is loaded whenever the
> >>>> "jdk.incubator.foreign" module is added on the command line.
> >>>>
> >>>> Code loading the library seems to be deep in hotspot:
> >>>>
> >>>>
> >>>>
> https://github.com/openjdk/jdk/blob/a8edd1b360d4e5f35aff371a91fda42eeb00d395/src/hotspot/cpu/x86/stubGenerator_x86_64.cpp#L7834
> >>>
> >>> Thanks for the code pointer!
> >>>
> >>>>
> >>>>
> >>>> And the code seems not to be guarded by flag. The only think which
> >>>> surrounds it is an ifdef for C2, so I suppose disabling C2 causes the
> >>>> library not to be loaded.
> >>>>
> >>> As I mentioned in my reply to Magnus, would it make sense to introduce
> a
> >>> JVM option to skip loading libsvml/Vector API support for (say a 17.0.2
> >>> release?
> >>>
> >>>>
> >>>> But I could be wrong and of course that's not a workaround.
> >>>>
> >>>> In Panama, for different reasons, on Windows systems we create a
> library
> >>>> which contains an array of function pointers to symbols in
> >>>> msvcrt/ucrtbase - this allows us to be relatively agnostic as to
> whether
> >>>> the symbols will be defined in one or the other library. I wonder if
> the
> >>>> vector API should add a similar level of indirection - e.g. have a
> >>>> "shim" library which depends on "svml" - but not bundle "svml" with
> the
> >>>> JDK (but this assumes the library can always be found installed in the
> >>>> system when needed by C2, which I don't know if it can be relied
> upon).
> >>>>
> >>> It looks like
> >>>
> https://github.com/openjdk/jdk/tree/master/src/jdk.incubator.vector/linux/native/libsvml
> >>> defines the various assembly routines making up the libsvml packaged
> with
> >>> the JDK.  Could these perhaps be built up by C2's MacroAssembler
> instead?
> >>> i.e. rather than loading them from a shared lib, have C2 generate the
> >>> assembly (and stub routines) on-the-fly.
> >>>
> >>> Thanks
> >>>
> >>>>
> >>>> Maurizio
> >>>>
> >>>>
> >>>> On 26/10/2021 00:43, Vitaly Davidovich wrote:
> >>>>> Hi all,
> >>>>>
> >>>>> We're testing some of our code on Java 17 (17.0.1)/linux and hit an
> issue
> >>>>> related to libsvml.so.  It seems this library is now part of 17 to
> >>>> support
> >>>>> the (incubating) Vector API.  We have a java library backed (via
> JNI) by
> >>>>> NAG, which itself links against libsvml.so.  The issue arises due to
> >>>>> java.lang.UnsatisfiedLinkError when our java library is trying to
> call
> >>>> into
> >>>>> a NAG function which in turn is looking for a certain symbol from
> libsvml
> >>>>> (__svml_exp2_ha_mask in particular).
> >>>>>
> >>>>> It looks like the JDK is eagerly loading symbols from its packaged
> >>>> libsvml
> >>>>> (is there a way to disable that for now?).  That version of the
> library
> >>>> is
> >>>>> also in conflict with the one we want to load, as witnessed by the
> >>>> missing
> >>>>> symbol (there're probably others but we stopped testing at this
> point).
> >>>>>
> >>>>> Is this a known issue/compatibility hazard? Happy to hear
> >>>> thoughts/opinions
> >>>>> on this and provide further info, if needed.
> >>>>>
> >>>>> Thanks
> >>>>
> >
>
>