RFR : 8204541 Correctly support AIX xlC 13.1 symbol visibility flags

Langer, Christoph christoph.langer at sap.com
Fri Jun 8 07:35:48 UTC 2018 

Hi Ichiroh,

Ok, so as per the output, via the include of osSupport.hpp, something must happen which undeclares "visibility" or makes it ambiguous. Looking at osSupport.hpp, I can't see anything special. It would just include pthread.h and declare some c++ classes.

You could try to get and analyze the preprocessed output of xlC by specifying the option -P or -E to the compile call. You will get the original xlC command line by calling 'cat support/native/java.base/libjimage/NativeImageBuffer.o.cmdline' in your build directory.

I think we should really understand what's happening there and fix it correctly instead of just excluding osSupport.hpp.

Best regards
Christoph

> -----Original Message-----
> From: Ichiroh Takiguchi [mailto:takiguc at linux.vnet.ibm.com]
> Sent: Donnerstag, 7. Juni 2018 18:29
> To: Langer, Christoph <christoph.langer at sap.com>
> Cc: build-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net; core-
> libs-dev at openjdk.java.net; Lindenmaier, Goetz
> <goetz.lindenmaier at sap.com>; Baesken, Matthias
> <matthias.baesken at sap.com>
> Subject: RE: RFR : 8204541 Correctly support AIX xlC 13.1 symbol visibility flags
> 
> Hello Christoph
> 
> According to build log, if <#include "osSupport.hpp"> was there:
> "/home/jdktest/sandbox/jdk/build/aix-ppc64-normal-server-
> release/support/headers/java.base/jdk_internal_jimage_NativeImageBuffe
> r.h",
> line 15.27: 1540-0040 (S) The text
>   "Java_jdk_internal_jimage_NativeImageBuffer_getNativeMap" is
> unexpected.  "visibility" may be undeclared or ambiguous.
> make[3]: ***
> [/home/jdktest/sandbox/jdk/build/aix-ppc64-normal-server-
> release/support/native/java.base/libjimage/NativeImageBuffer.o]
> Error 1
> make[3]: Leaving directory `/home/jdktest/sandbox/jdk/make'
> make[2]: *** [java.base-libs] Error 2
> make[2]: *** Waiting for unfinished jobs....
> 
> On 2018-06-07 22:06, Langer, Christoph wrote:
> > Hi Ichiroh,
> >
> > what's the exact error message if you  #include "osSupport.hpp"? (I
> > have no xlC 13 at hand to try myself...)
> >
> > Best regards
> > Christoph
> >
> >> -----Original Message-----
> >> From: Ichiroh Takiguchi [mailto:takiguc at linux.vnet.ibm.com]
> >> Sent: Donnerstag, 7. Juni 2018 14:53
> >> To: build-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net;
> >> core-
> >> libs-dev at openjdk.java.net
> >> Cc: Lindenmaier, Goetz <goetz.lindenmaier at sap.com>; Baesken,
> Matthias
> >> <matthias.baesken at sap.com>; Langer, Christoph
> >> <christoph.langer at sap.com>
> >> Subject: RFR : 8204541 Correctly support AIX xlC 13.1 symbol
> >> visibility flags
> >>
> >> Hello.
> >>
> >> Could you review it ?
> >> Bug:    https://bugs.openjdk.java.net/browse/JDK-8204541
> >> Change: http://cr.openjdk.java.net/~clanger/webrevs/8204541.0/
> >>
> >> Thanks,
> >> Ichiroh Takiguchi
> >> IBM Japan, Ltd.
> >>
> >> -------- Original Message --------
> >> Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support
> >> on
> >> xlc 12.1
> >> Date: 2018-06-07 20:43
> >>  From: "Langer, Christoph" <christoph.langer at sap.com>
> >> To: Ichiroh Takiguchi <takiguc at linux.vnet.ibm.com>
> >> Cc: "'build-dev at openjdk.java.net'" <build-dev at openjdk.java.net>,
> >> "ppc-aix-port-dev at openjdk.java.net" <ppc-aix-port-
> >> dev at openjdk.java.net>,
> >> "core-libs-dev at openjdk.java.net" <core-libs-dev at openjdk.java.net>,
> >> "Lindenmaier, Goetz" <goetz.lindenmaier at sap.com>, "Baesken,
> Matthias"
> >> <matthias.baesken at sap.com>
> >>
> >> Hi Ichiroh,
> >>
> >> your proposal seems to make sense. I have created a bug for this:
> >> https://bugs.openjdk.java.net/browse/JDK-8204541
> >>
> >> Can you please generate a webrev (referencing this bug, -c option of
> >> webrev.ksh) and mail it over to me. Then I'll upload it and you can
> >> post
> >> an official RFR mail.
> >>
> >> Best regards
> >> Christoph
> >>
> >> > -----Original Message-----
> >> > From: Ichiroh Takiguchi [mailto:takiguc at linux.vnet.ibm.com]
> >> > Sent: Dienstag, 5. Juni 2018 08:59
> >> > To: Baesken, Matthias <matthias.baesken at sap.com>
> >> > Cc: Langer, Christoph <christoph.langer at sap.com>; 'build-
> >> > dev at openjdk.java.net' <build-dev at openjdk.java.net>; ppc-aix-port-
> >> > dev at openjdk.java.net; core-libs-dev at openjdk.java.net; Lindenmaier,
> >> > Goetz <goetz.lindenmaier at sap.com>
> >> > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on
> >> > xlc 12.1
> >> >
> >> > Hello Matthias and Christoph.
> >> > Thank you for your explanations.
> >> >
> >> > I did not have enough knowledge about "visibility".
> >> >
> >> > I created following patches.
> >> >
> >> > ================================
> >> > diff -r 02934b0d661b
> >> > src/java.base/share/native/libjimage/NativeImageBuffer.cpp
> >> > --- a/src/java.base/share/native/libjimage/NativeImageBuffer.cpp
> 	Wed
> >> > May
> >> > 30 14:46:28 2018 +0200
> >> > +++ b/src/java.base/share/native/libjimage/NativeImageBuffer.cpp
> >> 	Tue
> >> > Jun
> >> > 05 12:10:41 2018 +0900
> >> > @@ -39,7 +39,9 @@
> >> >   #include "imageFile.hpp"
> >> >   #include "inttypes.hpp"
> >> >   #include "jimage.hpp"
> >> > +#if !defined(_AIX)
> >> >   #include "osSupport.hpp"
> >> > +#endif
> >> >
> >> >   #include "jdk_internal_jimage_NativeImageBuffer.h"
> >> >
> >> > diff -r 02934b0d661b src/java.base/unix/native/include/jni_md.h
> >> > --- a/src/java.base/unix/native/include/jni_md.h	Wed May 30
> 14:46:28
> >> > 2018 +0200
> >> > +++ b/src/java.base/unix/native/include/jni_md.h	Tue Jun 05
> 12:10:41
> >> > 2018 +0900
> >> > @@ -29,7 +29,8 @@
> >> >   #ifndef __has_attribute
> >> >     #define __has_attribute(x) 0
> >> >   #endif
> >> > -#if (defined(__GNUC__) && ((__GNUC__ > 4) || (__GNUC__ == 4)
> &&
> >> > (__GNUC_MINOR__ > 2))) || __has_attribute(visibility)
> >> > +#if (defined(__GNUC__) && ((__GNUC__ > 4) || (__GNUC__ == 4)
> &&
> >> > (__GNUC_MINOR__ > 2))) || __has_attribute(visibility) \
> >> > +    || (defined(_AIX) && (defined(__xlC__) && (__xlC__ >= 0xD01)))
> >> >     #ifdef ARM
> >> >       #define JNIEXPORT
> >> > __attribute__((externally_visible,visibility("default")))
> >> >       #define JNIIMPORT
> >> > __attribute__((externally_visible,visibility("default")))
> >> > ================================
> >> >
> >> > If "osSupport.hpp" was included, XLC++ compiler complained.
> >> > I could not understand, which part was invalid...
> >> > I'm not sure, "osSupport.hpp" is really required on
> >> > NativeImageBuffer.cpp or not...
> >> >
> >> > I added __xlC__ macro to determine XLC/C++'s version# into jni_md.h.
> >> > [1]
> >> > 0xD01 means 13.1 by hexadecimal.
> >> >
> >> > I checked symbol table by "dump -Tv -X64". [2]
> >> > It seemed it was fine, some symbols were hidden.
> >> >
> >> > Does someone review my code?
> >> >
> >> > [1]
> >> >
> >>
> https://www.ibm.com/support/knowledgecenter/en/SSGH2K_13.1.3/com.i
> >> > bm.xlc1313.aix.doc/compiler_ref/xlmacros.html
> >> > [2]
> >> > https://www.ibm.com/developerworks/aix/library/au-aix-symbol-
> >> > visibility/index.html
> >> >
> >> > On 2018-06-01 21:12, Baesken, Matthias wrote:
> >> > > Hi ,  my change  8202322   just  handled  the fact   that  the
> >> > > visibility - flags   are not supported  with  xlc 12.1  ,  so  setting
> >> > > them generated a TON of compile - time  warnings .
> >> > >
> >> > > The introduction of   the  "-qvisibility=hidden"    came with the
> >> > > mapfile removal changes :
> >> > >
> >> > > 8200358: Remove mapfiles for JDK executables
> >> > > http://hg.openjdk.java.net/jdk/jdk/rev/210cf224b690
> >> > >
> >> > > 8200178: Remove mapfiles for JDK native libraries
> >> > > http://hg.openjdk.java.net/jdk/jdk/rev/396ea30afbd5
> >> > >
> >> > > I guess it  might  need further  testing+adjustments  to make  the
> >> > > "visibility hiding" work nicely   with XLC13  ,  but currently  we
> >> > > build only with XLC12 .
> >> > >
> >> > > As a workaround  you might want to  remove  the  "-
> qvisibility=hidden"
> >> > >  setting for XLC 13 as well  ,  like I did  for XLC12 with  the change
> >> > >   8202322   .
> >> > >
> >> > >
> >> > > Best regards, Matthias
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >> -----Original Message-----
> >> > >> From: Langer, Christoph
> >> > >> Sent: Freitag, 1. Juni 2018 10:57
> >> > >> To: Ichiroh Takiguchi <takiguc at linux.vnet.ibm.com>
> >> > >> Cc: Baesken, Matthias <matthias.baesken at sap.com>; 'build-
> >> > >> dev at openjdk.java.net' <build-dev at openjdk.java.net>; ppc-aix-
> port-
> >> > >> dev at openjdk.java.net; core-libs-dev at openjdk.java.net;
> Lindenmaier,
> >> > >> Goetz <goetz.lindenmaier at sap.com>
> >> > >> Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support
> >> > >> on xlc 12.1
> >> > >>
> >> > >> Hi Ichiroh,
> >> > >>
> >> > >> we do not use the XLC 13 compiler on AIX yet here at SAP and I
> believe
> >> > >> nobody of my colleagues has played with it yet. So you are on a new
> >> > >> playground here 😊
> >> > >>
> >> > >> However, I believe the idea in OpenJDK with the abolition of map
> files
> >> > >> is that
> >> > >> symbols should be invisible externally unless they are declared
> >> > >> exported,
> >> > >> e.g. JNIEXPORT. So I would think "-qvisibility=hidden" should be the
> >> > >> correct
> >> > >> default and whatever JNIEXPORT expands to should contain the right
> >> > >> attributes to get that symbol visible.
> >> > >>
> >> > >> Can you check if either my assumption is completely wrong,
> JNIEXPORT
> >> > >> does
> >> > >> not expand to the right thing, XLC 13 has a bug or maybe just sume
> >> > >> specific
> >> > >> required symbols are not declared correctly?
> >> > >>
> >> > >> Best regards
> >> > >> Christoph
> >> > >>
> >> > >> > -----Original Message-----
> >> > >> > From: Ichiroh Takiguchi [mailto:takiguc at linux.vnet.ibm.com]
> >> > >> > Sent: Donnerstag, 31. Mai 2018 09:55
> >> > >> > To: Langer, Christoph <christoph.langer at sap.com>
> >> > >> > Cc: Baesken, Matthias <matthias.baesken at sap.com>; 'build-
> >> > >> > dev at openjdk.java.net' <build-dev at openjdk.java.net>; ppc-aix-
> port-
> >> > >> > dev at openjdk.java.net; core-libs-dev at openjdk.java.net;
> >> Lindenmaier,
> >> > >> > Goetz <goetz.lindenmaier at sap.com>
> >> > >> > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support
> on
> >> xlc
> >> > 12.1
> >> > >> >
> >> > >> > Hello.
> >> > >> > 8202322 was integrated into jdk-11+15.
> >> > >> > I'm using XLC 13.1.3 on AIX 7.1.4.
> >> > >> > Build was failed because of "-qvisibility=hidden" on
> >> > >> > make/lib/LibCommon.gmk.
> >> > >> > According to "XL C/C++ for AIX 13.1.3" documentation [1],
> >> > >> > "-qvisibility=hidden" cannot create shared libraries entry points.
> >> > >> > For example, libverify.so was there, but entry points were not
> >> resolved
> >> > >> > by "-lverify" option.
> >> > >> > I think it should be "-qvisibility=default" (I tried, it worked)
> >> > >> > or "-qvisibility=protected" (I had not tried) ?
> >> > >> > I'm not familiar with -qvisibility option, but I'd like to find out
> >> > >> > right way.
> >> > >> >
> >> > >> > [1]
> >> > >> >
> >> > >>
> >> >
> >>
> https://www.ibm.com/support/knowledgecenter/SSGH3R_13.1.3/com.ibm.
> >> > >> > xlcpp1313.aix.doc/compiler_ref/opt_visibility.html
> >> > >> >
> >> > >> > On 2018-05-16 16:08, Langer, Christoph wrote:
> >> > >> > > Hi Matthias,
> >> > >> > >
> >> > >> > > yes, reviewed.
> >> > >> > >
> >> > >> > > Best regards
> >> > >> > > Christoph
> >> > >> > >
> >> > >> > > From: Baesken, Matthias
> >> > >> > > Sent: Mittwoch, 16. Mai 2018 09:06
> >> > >> > > To: Langer, Christoph <christoph.langer at sap.com>;
> >> > >> > > 'build-dev at openjdk.java.net' <build-dev at openjdk.java.net>;
> >> > >> > > ppc-aix-port-dev at openjdk.java.net; core-libs-
> >> dev at openjdk.java.net
> >> > >> > > Cc: Lindenmaier, Goetz <goetz.lindenmaier at sap.com>
> >> > >> > > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support
> on
> >> > >> > > xlc 12.1
> >> > >> > >
> >> > >> > > Hi  Christoph can I add you as second reviewer  (other reviewer
> was
> >> > >> > > Erik Joelsson) can push the change ?
> >> > >> > >
> >> > >> > > Best regards, Matthias
> >> > >> > >
> >> > >> > >
> >> > >> > >
> >> > >> > > From: Langer, Christoph
> >> > >> > > Sent: Donnerstag, 26. April 2018 16:38
> >> > >> > > To: Baesken, Matthias
> >> > >> > >
> >> <matthias.baesken at sap.com<mailto:matthias.baesken at sap.com>>;
> >> > >> > > 'build-dev at openjdk.java.net'
> >> > >> > > <build-dev at openjdk.java.net<mailto:build-
> >> > dev at openjdk.java.net>>;
> >> > >> > > ppc-aix-port-dev at openjdk.java.net<mailto:ppc-aix-port-
> >> > >> > dev at openjdk.java.net>;
> >> > >> > > core-libs-dev at openjdk.java.net<mailto:core-libs-
> >> > >> dev at openjdk.java.net>
> >> > >> > > Cc: Simonis, Volker
> >> > >> > > <volker.simonis at sap.com<mailto:volker.simonis at sap.com>>
> >> > >> > > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support
> on
> >> > >> > > xlc 12.1
> >> > >> > >
> >> > >> > > Hi Matthias,
> >> > >> > >
> >> > >> > > to me the change in principal looks good.
> >> > >> > >
> >> > >> > > I'm wondering if it is possible to do a comparison like xlc < 13 (e.g.
> >> > >> > > extract major number before the first dot, then compare
> >> numerically)
> >> > -
> >> > >> > > but maybe it is too complicated and the current single version
> >> > compare
> >> > >> > > suits our needs ?
> >> > >> > >
> >> > >> > > Best regards
> >> > >> > > Christoph
> >> > >> > >
> >> > >> > > From: Baesken, Matthias
> >> > >> > > Sent: Donnerstag, 26. April 2018 16:14
> >> > >> > > To: 'build-dev at openjdk.java.net'
> >> > >> > > <build-dev at openjdk.java.net<mailto:build-
> >> > dev at openjdk.java.net>>;
> >> > >> > > ppc-aix-port-dev at openjdk.java.net<mailto:ppc-aix-port-
> >> > >> > dev at openjdk.java.net>;
> >> > >> > > core-libs-dev at openjdk.java.net<mailto:core-libs-
> >> > >> dev at openjdk.java.net>
> >> > >> > > Cc: Langer, Christoph
> >> > >> > >
> <christoph.langer at sap.com<mailto:christoph.langer at sap.com>>;
> >> > >> Simonis,
> >> > >> > > Volker
> >> <volker.simonis at sap.com<mailto:volker.simonis at sap.com>>
> >> > >> > > Subject: RFR : 8202322: AIX: symbol visibility flags not support on
> xlc
> >> > >> > > 12.1
> >> > >> > >
> >> > >> > > Hello  ,  could you please review this small adjustment to  the
> symbol
> >> > >> > > visibility compilation settings on AIX ?
> >> > >> > > Currently  we use  XLC 12.1  to compile  JDK on AIX .
> >> > >> > >
> >> > >> > > However XLC 12.1   does not support  the "-qvisibility=hidden"
> >> > >> > > setting currently set on AIX.
> >> > >> > > It was introduced with  XLC 13.1 . Christoph found some info
> about it
> >> > >> > > here :
> >> > >> > >
> >> > >> > > https://www.ibm.com/developerworks/aix/library/au-aix-
> symbol-
> >> > >> > visibility-part2/index.html
> >> > >> > >
> >> > >> > > Setting it  only generates  hundreds  of warnings  in the build log ,
> >> > >> > > warnings look like this :
> >> > >> > > XlC12.1
> >> > >> > >
> >> > >> > > bash-4.4$ xlC -qversion
> >> > >> > > IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72)
> >> > >> > > Version: 12.01.0000.0019
> >> > >> > >
> >> > >> > > bash-4.4$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc
> >> > >> > > 1506-173 (W) Option visibility=hidden is not valid. Enter xlC for list
> >> > >> > > of valid options.
> >> > >> > >
> >> > >> > > Compare to XLC13.1
> >> > >> > >
> >> > >> > > bash-3.00$ xlC -qversion
> >> > >> > > IBM XL C/C++ for AIX, V13.1 (5725-C72, 5765-J07)
> >> > >> > > Version: 13.01.0000.0008
> >> > >> > > bash-3.00$ xlC -qvisibility=default sizeof.c -o sizeof_aixxlc
> >> > >> > > bash-3.00$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc
> >> > >> > >
> >> > >> > >
> >> > >> > > So it is better to avoid  setting these flags when using xlc12.1   .
> >> > >> > > Please review :
> >> > >> > >
> >> > >> > > Bug :
> >> > >> > >
> >> > >> > > https://bugs.openjdk.java.net/browse/JDK-8202322
> >> > >> > >
> >> > >> > > Change :
> >> > >> > >
> >> > >> > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202322/
> >> > >> > >
> >> > >> > >
> >> > >> > > Best regards, Matthias

More information about the build-dev mailing list