RFR: JDK-8214063: OpenJDK will not build on AIX while using the xlc 13.1 compiler

Wed Nov 21 14:07:07 UTC 2018

On Wed, Nov 21, 2018 at 1:46 PM Adam Farley8 <adam.farley at uk.ibm.com> wrote:
>
> Hi Volker,
>
> The NativeImageBuffer.cpp changes are best explained by the full text of
> the referenced GitHub Pull Request, copied here for simplicity:
>
> -----------------------------------------
> Define JNIEXPORT and JNIIMPORT for xlc version 13.1 or newer. Without this,
> almost no symbols are exported from shared libraries due to use of
> -qvisibility=hidden as specified in make/lib/LibCommon.gmk. The symptoms
> are reported in eclipse/openj9#2468.
>
> Unfortunately, this encounters a bug in xlc: it fails to parse what seems
> to be reasonable code.

Sorry, but I don't see how this answers my question.

1. Which "reasonable code" does xlc fails to parse. A stand-alone
example would be nice.

2. Have you reported this as bug to the xlc developers? What did they say?

3. "jdk_internal_jimage_NativeImageBuffer.h" doesn't seem to be
special. It's a plain, generated JNI header file as generated by
'javah' or 'javac -h'. If XLC 13 has problems parsing it, there should
be much more places which need fixing. So what's special about
"jdk_internal_jimage_NativeImageBuffer.h".

In the referenced pull request
(https://github.com/eclipse/openj9/issues/2468) I can only see linker
errors (and no compiler errors). The linker errors are for both
libjsig and libjava. They are related to the symbol ".sigaction" in
jsig.o and I don't see how this should be related to
NativeImageBuffer.cpp or "jdk_internal_jimage_NativeImageBuffer.h".
NativeImageBuffer.cpp is only used to create libjimage and not related
in any way to libjsig or libjava.

It seems wired to do the change to NativeImageBuffer.cpp which you've
proposed without understanding the real cause of the problem.

Regards,
Volker

> A workaround is required in just one place:
> src/java.base/share/native/libjimage/NativeImageBuffer.cpp.
> -----------------------------------------
>
> Best Regards
>
> Adam Farley
> IBM Runtimes
>
>
> Volker Simonis <volker.simonis at gmail.com> wrote on 20/11/2018 17:50:41:
>
> > From: Volker Simonis <volker.simonis at gmail.com>
> > To: "Stuefe, Thomas" <thomas.stuefe at gmail.com>
> > Cc: adam.farley at uk.ibm.com, Java Core Libs <core-libs-dev at openjdk.java.net>
> > Date: 20/11/2018 17:59
> > Subject: Re: RFR: JDK-8214063: OpenJDK will not build on AIX while
> > using the xlc 13.1 compiler
> >
> > On Tue, Nov 20, 2018 at 6:15 PM Thomas Stüfe <thomas.stuefe at gmail.com> wrote:
> > >
> > > On Tue, Nov 20, 2018 at 6:12 PM Adam Farley8 <adam.farley at uk.ibm.com> wrote:
> > > >
> > > > Heya Tom,
> > > >
> > > > "In JDK11 and JDK12, source files are compiled with -qvisibility=hidden
> > > > when using xlc version other than 12.1. That doesn't seem to play well
> > > > with link option -bexpall. "
> > > >
> > > > Found that buried in one of the associated Git issues. It appears that
> > > > it's OpenJDK's use of that option that's causing the problem, though
> > > > I couldn't speculate as to why it was added in the first place.
> > > >
> > > > I see this has also been noted in https://
> > urldefense.proofpoint.com/v2/url?
> > u=https-3A__bugs.openjdk.java.net_browse_JDK-2D8204541&d=DwIFaQ&c=jf_iaSHvJObTbx-
> > siA1ZOg&r=P5m8KWUXJf-
> > CeVJc0hDGD9AQ2LkcXDC0PMV9ntVw5Ho&m=SD6UdjysISJRBlWUm8pEzF5lRZ5opfbrKzEh_jrOras&s=5qDEdIfg8qZ-
> > vCglsZ9qNDTEPMnCkj-mVPVah6eEDLE&e=
> > > >
> > > > Does that answer your question?
> > > >
> > >
> > > Yes, Thank you. Odd. Will have to do archeology on that one.
> > >
> >
> > No I begin to understand the problem as well :)
> >
> > It was actually change "8202322: AIX: symbol visibility flags not
> > support on xlc 12.1" [1] which introduced "-qvisibility=hidden" for
> > XLC version not equal to 12.1. That's kind of a weak check and I
> > suppose nobody has ever tested this change with an XLC version other
> > than 12.1 (until you came along :). Maybe that check should be a more
> > precisly check for >= 13.1 (but I know such version checks are hard to
> > do in Makefile syntax)?
> >
> > The thing I don't understand about your patch (the changes in
> > "jni_md.h" look good although I haven't tested them) is why you need
> > the extra changes in NativeImageBuffer.cpp?
> > "jdk_internal_jimage_NativeImageBuffer.h" is a plain, generated JNI
> > header file. If XLC 13 has problems to parse it, there should be much
> > more places which need fixing. I think that part of your change needs
> > a closer evaluation.
> >
> > Thank you and best regards,
> > Volker
> >
> > [1] https://urldefense.proofpoint.com/v2/url?
> > u=https-3A__bugs.openjdk.java.net_browse_JDK-2D8202322&d=DwIFaQ&c=jf_iaSHvJObTbx-
> > siA1ZOg&r=P5m8KWUXJf-
> > CeVJc0hDGD9AQ2LkcXDC0PMV9ntVw5Ho&m=SD6UdjysISJRBlWUm8pEzF5lRZ5opfbrKzEh_jrOras&s=JAEK6rePGMPinZzOquHBzj5oc7vA3kaFt9x0WIIUzvk&e=
> >
> > > ..Thomas
> > >
> > > > Best Regards
> > > >
> > > > Adam Farley
> > > > IBM Runtimes
> > > >
> > > >
> > > > "Thomas Stüfe" <thomas.stuefe at gmail.com> wrote on 20/11/2018 16:44:07:
> > > >
> > > > > From: "Thomas Stüfe" <thomas.stuefe at gmail.com>
> > > > > To: Adam Farley8 <adam.farley at uk.ibm.com>
> > > > > Cc: Java Core Libs <core-libs-dev at openjdk.java.net>
> > > > > Date: 20/11/2018 16:48
> > > > > Subject: Re: RFR: JDK-8214063: OpenJDK will not build on AIX while
> > > > > using the xlc 13.1 compiler
> > > > >
> > > > > Hi Adam,
> > > > >
> > > > > On Tue, Nov 20, 2018 at 5:12 PM Adam Farley8
> > <adam.farley at uk.ibm.com> wrote:
> > > > > >
> > > > > > Hi Tom,
> > > > > >
> > > > > > Sounds reasonable. I've added a webex to the bug, and here's a
> > > > > link to the bug.
> > > > > >
> > > > > > https://urldefense.proofpoint.com/v2/url?
> > > > >
> > u=https-3A__bugs.openjdk.java.net_browse_JDK-2D8214063&d=DwIFaQ&c=jf_iaSHvJObTbx-
> > > > > siA1ZOg&r=P5m8KWUXJf-
> > > > >
> > CeVJc0hDGD9AQ2LkcXDC0PMV9ntVw5Ho&m=z8YYwBXEfN7UtX1suPjpp9CZSHf8v0GrIMK3XGIC9VY&s=81TP9mIjhYD2Hmt8g7p2EHWRZXgiep21hxKLYRU7zIQ&e=
> > > > > >
> > > > > > This patch is required because otherwise, when building on AIX
> > > > > using xlc 3.1,
> > > > > > the build fails with this error:
> > > > > >
> > > > > > "Visibility is not allowed on a reference to an imported symbol."
> > > > > >
> > > > > > We believe this is caused by JNIEXPORT and JNIIMPORT not being
> > > > > defined. Without
> > > > > > this, almost no symbols are exported from shared libraries
> > due to use of
> > > > > > -qvisibility=hidden as specified in make/lib/LibCommon.gmk.
> > > > >
> > > > > Yes but what I try to understand is why does this happen now with
> > > > > xlc13? Did xlc change the rules for -qvisibility from v12 to v13 ?
> > > > > That would be quite a break in backward compatibility.
> > > > >
> > > > > >
> > > > > > For convenience, here's a summary of the diffs:
> > > > > >
> > > > > > --------------------------------------
> > > > > > File 1 of 2) src/java.base/share/native/libjimage/
> > NativeImageBuffer.cpp
> > > > > >
> > > > > >  #include "osSupport.hpp"
> > > > > >
> > > > > > +#if defined(__xlC__) && (__xlC__ >= 0x0d01)
> > > > > > +/*
> > > > > > + * Version 13.1.3 of xlc seems to have trouble parsing the
> > `__attribute__`
> > > > > > + * annotation in the generated header file we're about to
> > > > > include. Repeating
> > > > > > + * the forward declaration (without the braces) here avoids
> > the diagnostic:
> > > > > > + *   1540-0040 (S) The text "void" is unexpected.  "visibility"
> > > > > may be undeclared or ambiguous.
> > > > > > + */
> > > > > > +extern "C" JNIEXPORT jobject JNICALL
> > > > > Java_jdk_internal_jimage_NativeImageBuffer_getNativeMap(JNIEnv *,
> > > > > jclass, jstring);
> > > > > > +#endif
> > > > > > +
> > > > > > #include "jdk_internal_jimage_NativeImageBuffer.h"
> > > > > > --------------------------------------
> > > > > > File 2 of 2) src/java.base/unix/native/include/jni_md.h
> > > > > >
> > > > > >      #define JNIIMPORT     __attribute__((visibility("default")))
> > > > > >   #endif
> > > > > > +#elif defined(__xlC__) && (__xlC__ >= 0x0d01) /* xlc version 13.1
> > > > > or better required */
> > > > > > +  #define JNIEXPORT       __attribute__((visibility("default")))
> > > > > > +  #define JNIIMPORT       __attribute__((visibility("default")))
> > > > > > #else
> > > > > >   #define JNIEXPORT
> > > > > > --------------------------------------
> > > > > >
> > > > >
> > > > > Thank you.
> > > > >
> > > > > Cheers, Thomas
> > > > >
> > > > > > Best Regards
> > > > > >
> > > > > > Adam Farley
> > > > > > IBM Runtimes
> > > > > >
> > > > > >
> > > > > > "Thomas Stüfe" <thomas.stuefe at gmail.com> wrote on 19/11/201818:11:34:
> > > > > >
> > > > > > > From: "Thomas Stüfe" <thomas.stuefe at gmail.com>
> > > > > > > To: Adam Farley8 <adam.farley at uk.ibm.com>
> > > > > > > Cc: Java Core Libs <core-libs-dev at openjdk.java.net>
> > > > > > > Date: 19/11/2018 18:12
> > > > > > > Subject: Re: RFR: JDK-8214063: OpenJDK will not build on AIX while
> > > > > > > using the xlc 13.1 compiler
> > > > > > >
> > > > > > > Hi Adam,
> > > > > > >
> > > > > > > could you please include link to the JBS issue and either
> > link to the
> > > > > > > patch/webrev or link to the webrev, or at the very least the patch
> > > > > > > verbatim?
> > > > > > >
> > > > > > > As for the issue itself: could you please elaborate why this
> > > > > fails with xlc13?
> > > > > > >
> > > > > > > Also, a real patch would be helpful instead here of yet another link
> > > > > > > to some J9 issue. We are really strapped for manpower and
> > the AIX port
> > > > > > > eats up enough time as it is.
> > > > > > >
> > > > > > > Thanks, Thomas
> > > > > > >
> > > > > > > On Mon, Nov 19, 2018 at 6:28 PM Adam Farley8
> > > > > <adam.farley at uk.ibm.com> wrote:
> > > > > > > >
> > > > > > > > Hi All
> > > > > > > >
> > > > > > > > Both the problem and the solution appear straight-forward enough.
> > > > > > > >
> > > > > > > > Details included in the bug description.
> > > > > > > >
> > > > > > > > Thoughts and opinions welcome.
> > > > > > > >
> > > > > > > > Best Regards
> > > > > > > >
> > > > > > > > Adam Farley
> > > > > > > > IBM Runtimes
> > > > > > > >
> > > > > > > > Unless stated otherwise above:
> > > > > > > > IBM United Kingdom Limited - Registered in England and
> > Wales with number
> > > > > > > > 741598.
> > > > > > > > Registered office: PO Box 41, North Harbour, Portsmouth,
> > > > > Hampshire PO6 3AU
> > > > > > >
> > > > > >
> > > > > > Unless stated otherwise above:
> > > > > > IBM United Kingdom Limited - Registered in England and Wales with
> > > > > number 741598.
> > > > > > Registered office: PO Box 41, North Harbour, Portsmouth,
> > Hampshire PO6 3AU
> > > > >
> > > >
> > > > Unless stated otherwise above:
> > > > IBM United Kingdom Limited - Registered in England and Wales
> > with number 741598.
> > > > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
> >
>
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU