Regression for Native Library Loading in 11.0.5 on Linux
David Holmes
david.holmes at oracle.com
Wed Feb 12 13:39:48 UTC 2020
Hi Marc,
I'm cc'ing the build folk on build-dev as they likely have a better
understanding of linker symbols.
On 12/02/2020 7:54 pm, Marc Streckfuß wrote:
> Hey David and Martijn,
> I've now spent more than a week on the issue, and I am reporting back
> because I have a question (as something has changed since java 8) and
> because it seems the Maintainers/Packagers at Ubuntu are rather busy and
> the topic is to complicated for most others to help me.
>
> Now it seems only libjawt.so is affected, a tiny wrapper around libawt.so.
> Historically, it defined itself as libjawt.so with the version
> "SUNWprivate_1.1".
> This seems gone with java 11 (might be gone with java 9 already), there
> is no version _definition_ anymore.
In JDK 11 we stopped using mapfiles:
https://bugs.openjdk.java.net/browse/JDK-8200178
which is where the SUNWprivate_1.1 label previously came from.
> lwjgl2 however was built against java 8's jni, which made the linker put
> a SUNWprivate_1.1 dependency in.
I'm not at all sure that is a valid configuration - for an external
library compiled/linked against a Java 8 JDK to be used with a JDK 11
binary.
David
-----
> Since it works with AdoptOpenJDK and others, I assume if the version
> can't be found, it still loads the method (JAWT_GetAWT).
>
> When I now compare the Ubuntu libjawt to the Debian libjawt again, I see
> that Debian still has a version need on glibc and thus comes with a
> version symbol table.
> I _suspect_ that Ubuntu tries to be clever and sees that there is no
> version definition, thus it can omit the version tables, which would be
> true, if there wasn't someone relying on the version symbols.
> If that's a dead end, then the question is, why Debian links against
> "__cxa_finalize at GLIBC_2.2.5" and Ubuntu does not, Ubuntu just imports
> whatever __cxa_finalize is there.
> I suspect that this versioned import is the only reason the versioning
> table is kept.
>
> Do you know more about the omission of SUNWprivate_1.1?
> Also can you maybe help me, or point me towards someone that can, in
> order to fix this issue.
> I also don't know where the issue actually is (in glibc when loading?
> binutils when linking? specific makefile patches?)
>
> Cheers,
> Marc
>
> Am 04.02.20 um 19:53 schrieb Martijn Verburg:
>> Hi Marc,
>>
>> reporting to Ubuntu seems like the correct thing to do here - thanks!.
>>
>> Cheers,
>> Martijn
>>
>>
>> On Mon, 3 Feb 2020 at 22:16, Marc Streckfuß <marc.streckfuss at gmail.com
>> <mailto:marc.streckfuss at gmail.com>> wrote:
>>
>> Hey David and Martijn,
>> I've now done some testing locally with three JDKs and I am afraid
>> this is even more specific: An Ubuntu issue.
>> I think one of our monkeys also said that they had it on
>> Arch-Linux as well, which makes this issue more obscure, but:
>>
>> FAILING:
>>
>> openjdk version "11.0.6" 2020-01-14
>> OpenJDK Runtime Environment (build
>> 11.0.6+10-post-Ubuntu-1ubuntu118.04.1)
>> OpenJDK 64-Bit Server VM (build
>> 11.0.6+10-post-Ubuntu-1ubuntu118.04.1, mixed mode, sharing)
>>
>> WORKING:
>>
>> openjdk version "11.0.6" 2020-01-14
>> OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.6+10)
>> OpenJDK 64-Bit Server VM AdoptOpenJDK (build 11.0.6+10, mixed mode)
>>
>> openjdk version "11.0.6" 2020-01-14
>> OpenJDK Runtime Environment 18.9 (build 11.0.6+10)
>> OpenJDK 64-Bit Server VM 18.9 (build 11.0.6+10, mixed mode)
>>
>> Now I also got the second one from AdoptOpenJDK (upstream), but I
>> assume this is a problme specifically from
>> "-post-Ubuntu-1ubuntu118.04.1"
>>
>> I guess I should address this directly on launchpad.net
>> <https://urldefense.com/v3/__http://launchpad.net__;!!GqivPVa7Brio!ILKOUmt20SkUgMFjnMzicAW4_SK7C4SCf-ZKvIHJfOxuxVkigbgGTKIVsdakemmi_A$>?
>>
>> Thanks for your help so far,
>> Marc
>>
>> Am 03.02.20 um 03:39 schrieb Martijn Verburg:
>>> Hi Marc,
>>>
>>> I’m one of the AdoptOpenJDK representatives. Please to come and
>>> report it to us (openjdk-support repo on our github) sounds like
>>> an issue that a packaging workaround may be able to solve. If
>>> it’s genuinely a generic openjdk issue we can help triage
>>> regardless 🙂
>>>
>>> Cheers,
>>> Martijn
>>>
>>> On Mon, 3 Feb 2020 at 14:46, David Holmes
>>> <david.holmes at oracle.com <mailto:david.holmes at oracle.com>> wrote:
>>>
>>> On 3/02/2020 10:04 am, Marc Streckfuß wrote:
>>> > Hi David,
>>> > Thanks in Advance for that swift analysis!
>>> >
>>> > Actually I am asking on behalf of jMonkeyEngine, but didn't
>>> see the
>>> > precise post you've outlined.
>>> > The hint that this seems to be related to the build
>>> environment is a
>>> > good one, actually we switched from Oracle 8 to
>>> AdoptOpenJDK 11.0.5, so
>>> > this may come into play.
>>> > My intent for posting here was at least to get some eyes on
>>> the
>>> > situation, as searching the net is mostly a list of
>>> confusing reports
>>> > with the only solution being going back to 8.
>>> >
>>> > I wonder one thing, though: Does the JVM statically link
>>> against glibc?
>>>
>>> I'm not sure sorry - the detailed build config is not
>>> something I'm
>>> intimately familiar with. It may be configurable.
>>>
>>> > As usually when the glibc is linked in dynamically, there
>>> shouldn't be
>>> > many ways this can trigger a bug, like it depends on the
>>> runtime
>>> > environment and not on the build environment.
>>> > Apart from the suggestions on our forums, did you find some
>>> clear
>>> > statement on the glibc bug? e.g. which version could've
>>> triggered it?
>>>
>>> Some of the links I was following were actually for different
>>> failure
>>> modes so drew blanks. This one is interesting though:
>>>
>>> https://bugs.launchpad.net/ubuntu/+source/gcc-7/+bug/1764701
>>> <https://urldefense.com/v3/__https://bugs.launchpad.net/ubuntu/*source/gcc-7/*bug/1764701__;Kys!!GqivPVa7Brio!ILKOUmt20SkUgMFjnMzicAW4_SK7C4SCf-ZKvIHJfOxuxVkigbgGTKIVsdaknoFiCw$>
>>>
>>> Particularly the last post regarding Ubuntu OpenJDK build versus
>>> AdoptOpenJDK. I can't quite glean exactly where they think
>>> the problem
>>> is though.
>>>
>>> > If it really depends on the build environment/JDK
>>> Distribution, we can
>>> > mark this as solved. I'll do some research and report this
>>> to the
>>> > AdoptOpenJDK team and lets see what happens.
>>>
>>> It would be really good if someone who can reproduce this can
>>> try with
>>> 11.0.4 and 11.0.5 from OS distribution, AdoptOpenJDK build
>>> and Oracle
>>> JDK build.
>>>
>>> Cheers,
>>> David
>>> -----
>>>
>>> >
>>> > Thanks for your time,
>>> > Marc
>>> >
>>> > Am Mo., 3. Feb. 2020 um 00:18 Uhr schrieb David Holmes
>>> > <david.holmes at oracle.com <mailto:david.holmes at oracle.com>
>>> <mailto:david.holmes at oracle.com
>>> <mailto:david.holmes at oracle.com>>>:
>>> >
>>> > Hi Marc,
>>> >
>>> > On 1/02/2020 9:56 pm, Marc Streckfuß wrote:
>>> > > Dear Sirs or Madams,
>>> > > Since posting to the Java Bug System is restricted
>>> to OpenJDK
>>> > Authors, I'm
>>> > > asking here for help and maybe someone can create a
>>> tracking
>>> > issue on my
>>> > > behalf.
>>> > >
>>> > > We're seeing a problem where loading the "lwjgl2" (
>>> > > https://github.com/LWJGL/lwjgl
>>> <https://urldefense.com/v3/__https://github.com/LWJGL/lwjgl__;!!GqivPVa7Brio!ILKOUmt20SkUgMFjnMzicAW4_SK7C4SCf-ZKvIHJfOxuxVkigbgGTKIVsdbadJxVJw$>)
>>> native dependencies fails ONLY on
>>> > Linux and
>>> > > ONLY on JDK/JVM 11.
>>> > > I've been told that this only happens on 11.0.5,
>>> 11.0.4 should be
>>> > fine,
>>> > > however research shows, that this also happens on 11.0.3
>>> > > https://unix.stackexchange.com/q/532054
>>> <https://urldefense.com/v3/__https://unix.stackexchange.com/q/532054__;!!GqivPVa7Brio!ILKOUmt20SkUgMFjnMzicAW4_SK7C4SCf-ZKvIHJfOxuxVkigbgGTKIVsdYPDBzvfA$>.
>>> > >
>>> > > The issue is:
>>> > >
>>> > > Inconsistency detected by ld.so: dl-lookup.c: 111:
>>> check_match:
>>> > > Assertion `version->filename == NULL || !
>>> _dl_name_match_p
>>> > > (version->filename, map)' failed!
>>> > >
>>> > > Technically this issue stems from glib, here:
>>> > >
>>> >
>>> https://github.com/lattera/glibc/blob/master/elf/dl-lookup.c#L111
>>> <https://urldefense.com/v3/__https://github.com/lattera/glibc/blob/master/elf/dl-lookup.c*L111__;Iw!!GqivPVa7Brio!ILKOUmt20SkUgMFjnMzicAW4_SK7C4SCf-ZKvIHJfOxuxVkigbgGTKIVsdbtcEOjKQ$>,
>>> > but it
>>> > > has to be somehow related to the way the JVM is
>>> interacting with
>>> > glibc, as
>>> > > this behavior wasn't there with Java 8.
>>> > > I can't really reliable comment on the state for
>>> versions 9, 10
>>> > and up
>>> > > until 11.0.4, but 11.0.5 has the problem and 8 doesn't.
>>> > >
>>> > > Sorry for the vague information on this one, but
>>> maybe someone
>>> > has an idea
>>> > > or could give this a quick look?
>>> > > I guess if you already have a dev env setup and can
>>> step through
>>> > with a
>>> > > debugger, the issue could be trivial.
>>> >
>>> > I've done a bit of a google search on this problem and
>>> it seems to be a
>>> > somewhat confused situation. I've seen reports of 8 not
>>> working, but
>>> > falling back to 8 as fixing it. I've seen 11.0.5 is
>>> broken but 11.0.4
>>> > works, but someone else said 11.0.4 was fine. I've
>>> followed other
>>> > reports that indicate this all relates to a glibc bug.
>>> >
>>> >
>>> https://hub.jmonkeyengine.org/t/solved-jme-does-not-work-at-all-on-modern-java-due-to-a-regression/42112/14
>>> <https://urldefense.com/v3/__https://hub.jmonkeyengine.org/t/solved-jme-does-not-work-at-all-on-modern-java-due-to-a-regression/42112/14__;!!GqivPVa7Brio!ILKOUmt20SkUgMFjnMzicAW4_SK7C4SCf-ZKvIHJfOxuxVkigbgGTKIVsdZhXS8GXA$>
>>> >
>>> > but also suggests a different solution:
>>> >
>>> > "you either need openJDK 11.0.4 (not 11.0.5) or lwjgl3
>>> instead of
>>> > lwjgl2
>>> > unfortunately."
>>> >
>>> > From what I can tell from the reports that I have
>>> found is that the
>>> > problem seems to come and go with OpenJDK builds from
>>> different OpenJDK
>>> > distributors. I did not see any reports that
>>> conclusively indicated the
>>> > problem was seen on Oracle JDK. My suspicion is that
>>> whether or not
>>> > this
>>> > problem appears depends on how the JDK was built i.e.
>>> which version of
>>> > glibc it has been linked against. The Oracle JDK
>>> binaries for 11.0.4
>>> > and
>>> > 11.0.5 seems to have the exact same build environment.
>>> I can't comment
>>> > on any binaries from other places e.g. AdoptOpenJDK.
>>> This conclusion
>>> > was
>>> > also proposed on the jmonkeyengine post above:
>>> >
>>> > "If they’re right, it’s going to depend on what version
>>> of glibc you
>>> > jvm
>>> > is using, and possibly what was present when your
>>> native payload was
>>> > compiled."
>>> >
>>> > Not sure what can be done at the OpenJDK end.
>>> >
>>> > Cheers,
>>> > David
>>> >
>>> > > Thanks in Advance,
>>> > > Marc Streckfuß
>>> > >
>>> >
>>>
>>> --
>>> Cheers, Martijn (Sent from Gmail Mobile)
>>
More information about the build-dev
mailing list