RFR: JDK-8214063: OpenJDK will not build on AIX while using the xlc 13.1 compiler
Adam Farley8
adam.farley at uk.ibm.com
Mon Nov 26 13:57:05 UTC 2018
...Or we could simply remove the -qvisibility bit from the .mk file,
as Marcus mentioned.
https://bugs.openjdk.java.net/browse/JDK-8214063
Might be less technically correct, but it wouldn't make OpenJDK on
AIX more technically incorrect than it already is, with the added
bonus that it allows us to build on versions of xlC other than 12.1.
I propose we call that "Plan B".
Best Regards
Adam Farley
IBM Runtimes
Adam Farley8/UK/IBM wrote on 26/11/2018 13:16:18:
> From: Adam Farley8/UK/IBM
> To: Volker Simonis <volker.simonis at gmail.com>
> Cc: Java Core Libs <core-libs-dev at openjdk.java.net>, "Stuefe,
> Thomas" <thomas.stuefe at gmail.com>
> Date: 26/11/2018 13:16
> Subject: Re: RFR: JDK-8214063: OpenJDK will not build on AIX while
> using the xlc 13.1 compiler
>
> Hi Volker,
>
> Apologies for the delay.
>
> I ran the contents of the file as requested (neat tip, thanks!) and
> I discovered something:
>
> If jdk_internal_jimage_NativeImageBuffer.h contains this:
>
> -----
> extern "C" {
> __attribute__((visibility("default"))) jobject JNICALL
> Java_jdk_internal_jimage_NativeImageBuffer_getNativeMap
> (JNIEnv *, jclass, jstring);
> }
> -----
>
> It results in this error:
>
> -----
> blah blah "visibility" may be undeclared or ambiguous.
> -----
>
> But replacing that bit with this code:
>
> -----
> extern "C" __attribute__((visibility("default"))) jobject JNICALL
> Java_jdk_internal_jimage_NativeImageBuffer_getNativeMap
> (JNIEnv *, jclass, jstring);
> -----
>
> Results in no error.
>
> So it seems the difference between an "extern "C"" block and an
> inline "extern"C"" is what's causing the issue.
>
> I don't understand why, but there we have it.
>
> A shame the header file is generated. An ideal fix would either be to:
>
> 1) Change xlC to make this work inside an extern "C" block.
> or
> 2) Change the way the header file is generated to add that extern
> "C" bit onto the line itself.
>
> However both of those have the potential to break other stuff.
>
> I'm reaching out to the xlC developers now to figure out what's
> going on, but in the
> meantime it seems to me like the proposed fix is still the right way
> to go for a short-term
> resolution.
>
> What are your thoughts?
>
> Best Regards
>
> Adam Farley
> IBM Runtimes
>
> Volker Simonis <volker.simonis at gmail.com> wrote on 22/11/2018 14:25:04:
>
> > From: Volker Simonis <volker.simonis at gmail.com>
> > To: adam.farley at uk.ibm.com
> > Cc: Java Core Libs <core-libs-dev at openjdk.java.net>, "Stuefe,
> > Thomas" <thomas.stuefe at gmail.com>
> > Date: 22/11/2018 14:25
> > Subject: Re: RFR: JDK-8214063: OpenJDK will not build on AIX while
> > using the xlc 13.1 compiler
> >
> > On Thu, Nov 22, 2018 at 3:00 PM Adam Farley8 <adam.farley at uk.ibm.com>
wrote:
> > >
> > > Hi Volker,
> > >
> > > 1) Here is the "reasonable" code in the generated
> > jdk_internal_jimage_NativeImageBuffer.h
> > >
> > > ------------------------------------------------------
> > > /* DO NOT EDIT THIS FILE - it is machine generated */
> > > #include <jni.h>
> > > /* Header for class jdk_internal_jimage_NativeImageBuffer */
> > >
> > > #ifndef _Included_jdk_internal_jimage_NativeImageBuffer
> > > #define _Included_jdk_internal_jimage_NativeImageBuffer
> > > #ifdef __cplusplus
> > > extern "C" {
> > > #endif
> > > /*
> > > * Class: jdk_internal_jimage_NativeImageBuffer
> > > * Method: getNativeMap
> > > * Signature: (Ljava/lang/String;)Ljava/nio/ByteBuffer;
> > > */
> > > JNIEXPORT jobject JNICALL
> > Java_jdk_internal_jimage_NativeImageBuffer_getNativeMap
> > > (JNIEnv *, jclass, jstring);
> > >
> > > #ifdef __cplusplus
> > > }
> > > #endif
> > > #endif
> > > ------------------------------------------------------
> > >
> > >
> > > 2) I have not yet reported this as a bug to the xlc developers. I
> > will contact
> > > them now.
> > >
> > > 3) I did some experimenting, and it seems that the
> > NativeImageBuffer.cpp change
> > > is the only thing standing between us and a successful compilation
> > on aix using
> > > xlc 13.1 (assuming you're using source that compiles on aix withxlc
12.1).
> > >
> > > With that change (plus the jni_md change), the compilation
completes.
> > >
> > > Without that change (after you've added the jni_md change
> though),the build
> > > will fail with this error message:
> > >
> > > ------------------------------------------------------
> > > 12:19:58 "/workspace/build/aix-ppc64-normal-server-release/
> > support/headers/java.base/jdk_internal_jimage_NativeImageBuffer.h",
> > line 15.27: 1540-0040 (S) The text
> > "Java_jdk_internal_jimage_NativeImageBuffer_getNativeMap" is
> > unexpected. "visibility" may be undeclared or ambiguous.
> > > 12:19:59 CoreLibraries.gmk:192: recipe for target '/workspace/
> > build/aix-ppc64-normal-server-release/support/native/java.base/
> > libjimage/NativeImageBuffer.o' failed
> > > ------------------------------------------------------
> > >
> >
> > Can you please do the following:
> > - take the command line from
> > /workspace/build/aix-ppc64-normal-server-release/support/native/
> > java.base/libjimage/NativeImageBuffer.o.cmdline
> > - replace '-c' with '-E' to get the preprocessor output
> > - have a look at the offending line (e.g. have JNIEXPORT / JNICALL
> > been correctly expanded ?)
> >
> > Unfortunately I don't have a version of XLC 13 to test this.
> >
> > > Best Regards
> > >
> > > Adam Farley
> > > IBM Runtimes
> > >
> > > P.S. Tried making a small, stand-alone example and it failed to
> > reproduce the problem.
> > > Will keep trying, and I'll supply a further update in the event of
> > a) results,
> > > or b) a response from the xlc guys.
> > >
> > >
> > > Volker Simonis <volker.simonis at gmail.com> wrote on 21/11/2018
14:07:07:
> > >
> > > > From: Volker Simonis <volker.simonis at gmail.com>
> > > > To: adam.farley at uk.ibm.com
> > > > Cc: Java Core Libs <core-libs-dev at openjdk.java.net>, "Stuefe,
> > > > Thomas" <thomas.stuefe at gmail.com>
> > > > Date: 21/11/2018 14:07
> > > > Subject: Re: RFR: JDK-8214063: OpenJDK will not build on AIX while
> > > > using the xlc 13.1 compiler
> > > >
> > > > On Wed, Nov 21, 2018 at 1:46 PM Adam Farley8
> > <adam.farley at uk.ibm.com> wrote:
> > > > >
> > > > > Hi Volker,
> > > > >
> > > > > The NativeImageBuffer.cpp changes are best explained by the
> full text of
> > > > > the referenced GitHub Pull Request, copied here for simplicity:
> > > > >
> > > > > -----------------------------------------
> > > > > Define JNIEXPORT and JNIIMPORT for xlc version 13.1 or newer.
> > Without this,
> > > > > almost no symbols are exported from shared libraries due to use
of
> > > > > -qvisibility=hidden as specified in make/lib/LibCommon.gmk.
> The symptoms
> > > > > are reported in eclipse/openj9#2468.
> > > > >
> > > > > Unfortunately, this encounters a bug in xlc: it fails to
> parsewhat seems
> > > > > to be reasonable code.
> > > >
> > > > Sorry, but I don't see how this answers my question.
> > > >
> > > > 1. Which "reasonable code" does xlc fails to parse. A stand-alone
> > > > example would be nice.
> > > >
> > > > 2. Have you reported this as bug to the xlc developers? What
> didthey say?
> > > >
> > > > 3. "jdk_internal_jimage_NativeImageBuffer.h" doesn't seem to be
> > > > special. It's a plain, generated JNI header file as generated by
> > > > 'javah' or 'javac -h'. If XLC 13 has problems parsing it, there
should
> > > > be much more places which need fixing. So what's special about
> > > > "jdk_internal_jimage_NativeImageBuffer.h".
> > > >
> > > > In the referenced pull request
> > > > (INVALID URI REMOVED
> > > >
> >
u=https-3A__github.com_eclipse_openj9_issues_2468&d=DwIFaQ&c=jf_iaSHvJObTbx-
> > > > siA1ZOg&r=P5m8KWUXJf-
> > > >
> >
>
CeVJc0hDGD9AQ2LkcXDC0PMV9ntVw5Ho&m=sgfFd6mB1EYM4nOM89rgFFzUyX7B21XbckIY7L0kUNU&s=TJ-4nr8ikZKImwDygirRTxLybsnQWBN71nEZCwZ59NQ&e=
> > > > ) I can only see linker
> > > > errors (and no compiler errors). The linker errors are for both
> > > > libjsig and libjava. They are related to the symbol ".sigaction"
in
> > > > jsig.o and I don't see how this should be related to
> > > > NativeImageBuffer.cpp or
"jdk_internal_jimage_NativeImageBuffer.h".
> > > > NativeImageBuffer.cpp is only used to create libjimage and not
related
> > > > in any way to libjsig or libjava.
> > > >
> > > > It seems wired to do the change to NativeImageBuffer.cpp which
you've
> > > > proposed without understanding the real cause of the problem.
> > > >
> > > > Regards,
> > > > Volker
> > > >
> > > > > A workaround is required in just one place:
> > > > > src/java.base/share/native/libjimage/NativeImageBuffer.cpp.
> > > > > -----------------------------------------
> > > > >
> > > > > Best Regards
> > > > >
> > > > > Adam Farley
> > > > > IBM Runtimes
> > > > >
> > > > >
> > > > > Volker Simonis <volker.simonis at gmail.com> wrote on 20/11/
> 2018 17:50:41:
> > > > >
> > > > > > From: Volker Simonis <volker.simonis at gmail.com>
> > > > > > To: "Stuefe, Thomas" <thomas.stuefe at gmail.com>
> > > > > > Cc: adam.farley at uk.ibm.com, Java Core Libs <core-libs-
> > > > dev at openjdk.java.net>
> > > > > > Date: 20/11/2018 17:59
> > > > > > Subject: Re: RFR: JDK-8214063: OpenJDK will not build on AIX
while
> > > > > > using the xlc 13.1 compiler
> > > > > >
> > > > > > On Tue, Nov 20, 2018 at 6:15 PM Thomas Stüfe
> > > > <thomas.stuefe at gmail.com> wrote:
> > > > > > >
> > > > > > > On Tue, Nov 20, 2018 at 6:12 PM Adam Farley8
> > > > <adam.farley at uk.ibm.com> wrote:
> > > > > > > >
> > > > > > > > Heya Tom,
> > > > > > > >
> > > > > > > > "In JDK11 and JDK12, source files are compiled with -
> > > > qvisibility=hidden
> > > > > > > > when using xlc version other than 12.1. That doesn't
> > seem toplay well
> > > > > > > > with link option -bexpall. "
> > > > > > > >
> > > > > > > > Found that buried in one of the associated Git issues.
> > It appears that
> > > > > > > > it's OpenJDK's use of that option that's causing the
> > problem, though
> > > > > > > > I couldn't speculate as to why it was added in the first
place.
> > > > > > > >
> > > > > > > > I see this has also been noted in https://
> > > > > > urldefense.proofpoint.com/v2/url?
> > > > > >
> > > >
> >
>
u=https-3A__bugs.openjdk.java.net_browse_JDK-2D8204541&d=DwIFaQ&c=jf_iaSHvJObTbx-
> > > > > > siA1ZOg&r=P5m8KWUXJf-
> > > > > >
> > > >
> >
>
CeVJc0hDGD9AQ2LkcXDC0PMV9ntVw5Ho&m=SD6UdjysISJRBlWUm8pEzF5lRZ5opfbrKzEh_jrOras&s=5qDEdIfg8qZ-
> > > > > > vCglsZ9qNDTEPMnCkj-mVPVah6eEDLE&e=
> > > > > > > >
> > > > > > > > Does that answer your question?
> > > > > > > >
> > > > > > >
> > > > > > > Yes, Thank you. Odd. Will have to do archeology on that one.
> > > > > > >
> > > > > >
> > > > > > No I begin to understand the problem as well :)
> > > > > >
> > > > > > It was actually change "8202322: AIX: symbol visibility flags
not
> > > > > > support on xlc 12.1" [1] which introduced
"-qvisibility=hidden" for
> > > > > > XLC version not equal to 12.1. That's kind of a weak check and
I
> > > > > > suppose nobody has ever tested this change with an XLC version
other
> > > > > > than 12.1 (until you came along :). Maybe that check
> should be a more
> > > > > > precisly check for >= 13.1 (but I know such version checks
> are hard to
> > > > > > do in Makefile syntax)?
> > > > > >
> > > > > > The thing I don't understand about your patch (the changes in
> > > > > > "jni_md.h" look good although I haven't tested them) is why
you need
> > > > > > the extra changes in NativeImageBuffer.cpp?
> > > > > > "jdk_internal_jimage_NativeImageBuffer.h" is a plain,
generated JNI
> > > > > > header file. If XLC 13 has problems to parse it, there
> should be much
> > > > > > more places which need fixing. I think that part of your
> change needs
> > > > > > a closer evaluation.
> > > > > >
> > > > > > Thank you and best regards,
> > > > > > Volker
> > > > > >
> > > > > > [1] INVALID URI REMOVED
> > > > > >
> > > >
> >
>
u=https-3A__bugs.openjdk.java.net_browse_JDK-2D8202322&d=DwIFaQ&c=jf_iaSHvJObTbx-
> > > > > > siA1ZOg&r=P5m8KWUXJf-
> > > > > >
> > > >
> >
>
CeVJc0hDGD9AQ2LkcXDC0PMV9ntVw5Ho&m=SD6UdjysISJRBlWUm8pEzF5lRZ5opfbrKzEh_jrOras&s=JAEK6rePGMPinZzOquHBzj5oc7vA3kaFt9x0WIIUzvk&e=
> > > > > >
> > > > > > > ..Thomas
> > > > > > >
> > > > > > > > Best Regards
> > > > > > > >
> > > > > > > > Adam Farley
> > > > > > > > IBM Runtimes
> > > > > > > >
> > > > > > > >
> > > > > > > > "Thomas Stüfe" <thomas.stuefe at gmail.com> wrote on 20/11/
> > 201816:44:07:
> > > > > > > >
> > > > > > > > > From: "Thomas Stüfe" <thomas.stuefe at gmail.com>
> > > > > > > > > To: Adam Farley8 <adam.farley at uk.ibm.com>
> > > > > > > > > Cc: Java Core Libs <core-libs-dev at openjdk.java.net>
> > > > > > > > > Date: 20/11/2018 16:48
> > > > > > > > > Subject: Re: RFR: JDK-8214063: OpenJDK will not build
> > on AIX while
> > > > > > > > > using the xlc 13.1 compiler
> > > > > > > > >
> > > > > > > > > Hi Adam,
> > > > > > > > >
> > > > > > > > > On Tue, Nov 20, 2018 at 5:12 PM Adam Farley8
> > > > > > <adam.farley at uk.ibm.com> wrote:
> > > > > > > > > >
> > > > > > > > > > Hi Tom,
> > > > > > > > > >
> > > > > > > > > > Sounds reasonable. I've added a webex to the bug,
> and here's a
> > > > > > > > > link to the bug.
> > > > > > > > > >
> > > > > > > > > > INVALID URI REMOVED
> > > > > > > > >
> > > > > >
> > > >
> >
>
u=https-3A__bugs.openjdk.java.net_browse_JDK-2D8214063&d=DwIFaQ&c=jf_iaSHvJObTbx-
> > > > > > > > > siA1ZOg&r=P5m8KWUXJf-
> > > > > > > > >
> > > > > >
> > > >
> >
>
CeVJc0hDGD9AQ2LkcXDC0PMV9ntVw5Ho&m=z8YYwBXEfN7UtX1suPjpp9CZSHf8v0GrIMK3XGIC9VY&s=81TP9mIjhYD2Hmt8g7p2EHWRZXgiep21hxKLYRU7zIQ&e=
> > > > > > > > > >
> > > > > > > > > > This patch is required because otherwise, when
> building on AIX
> > > > > > > > > using xlc 3.1,
> > > > > > > > > > the build fails with this error:
> > > > > > > > > >
> > > > > > > > > > "Visibility is not allowed on a reference to an
> > imported symbol."
> > > > > > > > > >
> > > > > > > > > > We believe this is caused by JNIEXPORT and
> JNIIMPORTnot being
> > > > > > > > > defined. Without
> > > > > > > > > > this, almost no symbols are exported from shared
libraries
> > > > > > due to use of
> > > > > > > > > > -qvisibility=hidden as specified in
make/lib/LibCommon.gmk.
> > > > > > > > >
> > > > > > > > > Yes but what I try to understand is why does this
> > happen now with
> > > > > > > > > xlc13? Did xlc change the rules for -qvisibility from
> > v12 to v13 ?
> > > > > > > > > That would be quite a break in backward compatibility.
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > For convenience, here's a summary of the diffs:
> > > > > > > > > >
> > > > > > > > > > --------------------------------------
> > > > > > > > > > File 1 of 2) src/java.base/share/native/libjimage/
> > > > > > NativeImageBuffer.cpp
> > > > > > > > > >
> > > > > > > > > > #include "osSupport.hpp"
> > > > > > > > > >
> > > > > > > > > > +#if defined(__xlC__) && (__xlC__ >= 0x0d01)
> > > > > > > > > > +/*
> > > > > > > > > > + * Version 13.1.3 of xlc seems to have trouble
parsing the
> > > > > > `__attribute__`
> > > > > > > > > > + * annotation in the generated header file we're
about to
> > > > > > > > > include. Repeating
> > > > > > > > > > + * the forward declaration (without the braces) here
avoids
> > > > > > the diagnostic:
> > > > > > > > > > + * 1540-0040 (S) The text "void" is unexpected.
> > "visibility"
> > > > > > > > > may be undeclared or ambiguous.
> > > > > > > > > > + */
> > > > > > > > > > +extern "C" JNIEXPORT jobject JNICALL
> > > > > > > > >
> > Java_jdk_internal_jimage_NativeImageBuffer_getNativeMap(JNIEnv *,
> > > > > > > > > jclass, jstring);
> > > > > > > > > > +#endif
> > > > > > > > > > +
> > > > > > > > > > #include "jdk_internal_jimage_NativeImageBuffer.h"
> > > > > > > > > > --------------------------------------
> > > > > > > > > > File 2 of 2)
src/java.base/unix/native/include/jni_md.h
> > > > > > > > > >
> > > > > > > > > > #define JNIIMPORT __attribute__((visibility
> > ("default")))
> > > > > > > > > > #endif
> > > > > > > > > > +#elif defined(__xlC__) && (__xlC__ >= 0x0d01) /*
> > xlc version 13.1
> > > > > > > > > or better required */
> > > > > > > > > > + #define JNIEXPORT __attribute__((visibility
> > ("default")))
> > > > > > > > > > + #define JNIIMPORT __attribute__((visibility
> > ("default")))
> > > > > > > > > > #else
> > > > > > > > > > #define JNIEXPORT
> > > > > > > > > > --------------------------------------
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > Thank you.
> > > > > > > > >
> > > > > > > > > Cheers, Thomas
> > > > > > > > >
> > > > > > > > > > Best Regards
> > > > > > > > > >
> > > > > > > > > > Adam Farley
> > > > > > > > > > IBM Runtimes
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > "Thomas Stüfe" <thomas.stuefe at gmail.com> wrote on
19/11/
> > > > 201818:11:34:
> > > > > > > > > >
> > > > > > > > > > > From: "Thomas Stüfe" <thomas.stuefe at gmail.com>
> > > > > > > > > > > To: Adam Farley8 <adam.farley at uk.ibm.com>
> > > > > > > > > > > Cc: Java Core Libs <core-libs-dev at openjdk.java.net>
> > > > > > > > > > > Date: 19/11/2018 18:12
> > > > > > > > > > > Subject: Re: RFR: JDK-8214063: OpenJDK will not
build
> > > > on AIX while
> > > > > > > > > > > using the xlc 13.1 compiler
> > > > > > > > > > >
> > > > > > > > > > > Hi Adam,
> > > > > > > > > > >
> > > > > > > > > > > could you please include link to the JBS issue and
either
> > > > > > link to the
> > > > > > > > > > > patch/webrev or link to the webrev, or at the very
> > > > least the patch
> > > > > > > > > > > verbatim?
> > > > > > > > > > >
> > > > > > > > > > > As for the issue itself: could you please
> elaborate why this
> > > > > > > > > fails with xlc13?
> > > > > > > > > > >
> > > > > > > > > > > Also, a real patch would be helpful instead here of
> > > > yet another link
> > > > > > > > > > > to some J9 issue. We are really strapped for
manpower and
> > > > > > the AIX port
> > > > > > > > > > > eats up enough time as it is.
> > > > > > > > > > >
> > > > > > > > > > > Thanks, Thomas
> > > > > > > > > > >
> > > > > > > > > > > On Mon, Nov 19, 2018 at 6:28 PM Adam Farley8
> > > > > > > > > <adam.farley at uk.ibm.com> wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > Hi All
> > > > > > > > > > > >
> > > > > > > > > > > > Both the problem and the solution appear straight-
> > > > forward enough.
> > > > > > > > > > > >
> > > > > > > > > > > > Details included in the bug description.
> > > > > > > > > > > >
> > > > > > > > > > > > Thoughts and opinions welcome.
> > > > > > > > > > > >
> > > > > > > > > > > > Best Regards
> > > > > > > > > > > >
> > > > > > > > > > > > Adam Farley
> > > > > > > > > > > > IBM Runtimes
> > > > > > > > > > > >
> > > > > > > > > > > > Unless stated otherwise above:
> > > > > > > > > > > > IBM United Kingdom Limited - Registered in England
and
> > > > > > Wales with number
> > > > > > > > > > > > 741598.
> > > > > > > > > > > > Registered office: PO Box 41, North Harbour,
Portsmouth,
> > > > > > > > > Hampshire PO6 3AU
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Unless stated otherwise above:
> > > > > > > > > > IBM United Kingdom Limited - Registered in England
> > and Wales with
> > > > > > > > > number 741598.
> > > > > > > > > > Registered office: PO Box 41, North Harbour,
Portsmouth,
> > > > > > Hampshire PO6 3AU
> > > > > > > > >
> > > > > > > >
> > > > > > > > Unless stated otherwise above:
> > > > > > > > IBM United Kingdom Limited - Registered in England and
Wales
> > > > > > with number 741598.
> > > > > > > > Registered office: PO Box 41, North Harbour, Portsmouth,
> > > > Hampshire PO6 3AU
> > > > > >
> > > > >
> > > > > Unless stated otherwise above:
> > > > > IBM United Kingdom Limited - Registered in England and Wales
with
> > > > number 741598.
> > > > > Registered office: PO Box 41, North Harbour, Portsmouth,
> > Hampshire PO6 3AU
> > > >
> > >
> > > Unless stated otherwise above:
> > > IBM United Kingdom Limited - Registered in England and Wales with
> > number 741598.
> > > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire
PO6 3AU
> >
>
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with
> number 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
More information about the core-libs-dev
mailing list