From gromero at linux.vnet.ibm.com Mon Apr 2 11:54:44 2018 From: gromero at linux.vnet.ibm.com (Gustavo Romero) Date: Mon, 2 Apr 2018 08:54:44 -0300 Subject: [RFC] Re: POWER9: Is there a way to improve the random number generation on PPC64? In-Reply-To: References: <5A1887FF.1030406@linux.vnet.ibm.com> Message-ID: <8347ea90-1132-1dc0-b04c-cbcd081a68ef@linux.vnet.ibm.com> Hi Martin, Volker, Vladimir Sorry for the huge delay replaying on this... I hope Martin (and Lutz) are feeling better and fully recovered. On 11/24/2017 08:04 PM, Volker Simonis wrote: > Hi Gustavo, > > in one of my talks [1,2] I have an example on how to intrinsify > Random.nextInt() in C2 by using the Intel 'rdrandl' instruction. But > please notice that this is just a "toy" example - it is not production > ready. In fact I think the right way would be to create a new > SecureRandom provider where you may implement "engineNextBytes" by > using the new Power instruction (maybe by calling a native function). > This special implementation of "engineNextBytes" could then be > intrinsified as described in the talk. > > Also, before you start this, please contact the security mailing list > just to make sure you're not going into the wrong direction (I'm not a > security expert :) I've created a new JCA provider called 'P9TRNG' and implemented a darn intrinsic for Interpreter, C1, and C2 compiler and did a couple of tests using micro benches [1, 2] to check the latency and throughput to get a random number using generateSeed() and nextBytes() with darn in place. The 'P9TRNG' provider is basically a copy of 'NativePRNG' since it's necessary a software fallback in case darn instruction fails to return a valid random number after ten attempts (although it's very rare condition). On the other hand 'P9TRNG' uses the darn intrinsic when it's available. The maximum theoretical throughput on the machine I'm testing it (a POWER9 witherspoon) is 128Mbps and there is one RNG per socket, so only one RNG per CPU. With a simple C code it's possible to get very close to that value (please see C code [3] for code details and log [4] for the expected outputs). Unrolling the tight loop does not help and causes a performance degradation. On Hotspot, for Interpreter and C1 the throughput is ~3x higher than the version that does not use darn instruction (using micro benches [1, 2]): gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java next_bytes SHA1PRNG 1024 10 3.8759432E7 ns 2.113550 Mbps gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java next_bytes SHA1PRNG 1024 100000 2.65902244E10 ns 30.808313 Mbps gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java next_bytes NativePRNG 1024 100 7.1741008E7 ns 11.418853 Mbps gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java next_bytes NativePRNG 1024 100000 2.74547937E10 ns 29.838140 Mbps gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java -Xcomp -XX:TieredStopAtLevel=3 next_bytes NativePRNG 1024 100000 5.5632339E10 ns 14.725248 Mbps gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java -Xcomp -XX:-TieredCompilation next_bytes NativePRNG 1024 100000 2.78629519E10 ns 29.401051 Mbps [With darn disabled: performance like NativePRNG] gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java next_bytes P9TRNG 1024 100 7.0272888E7 ns 11.657412 Mbps gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java next_bytes P9TRNG 1024 100000 2.75566244E10 ns 29.727880 Mbps ... [With darn enabled] gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseDARN next_bytes P9TRNG 1024 100 8305029.0 ns 98.639030 Mbps gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseDARN next_bytes P9TRNG 1024 100000 6.442112E9 ns 127.163261 Mbps gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseDARN -Xcomp -XX:TieredStopAtLevel=3 next_bytes P9TRNG 1024 100000 1.57303337E10 ns 52.077728 Mbps gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseDARN -Xcomp -XX:-TieredCompilation next_bytes P9TRNG 1024 100000 6.46914E9 ns 126.631973 Mbps For C2 compiler using darn is better until it reaches ~128Mbps (maximum theoretical throughput), but on the other hand it never blocks, so, for instance, generateSeed() which normally uses /dev/random (blocking) is not affected by a lack of entropy in Linux entropy pool. @Vladimir, Volker mentioned that you already experimented with rand on Intel. Do you know if creating a new JCA provider as I did is a reasonable approach to exploit darn on POWER9? Also, in my implementation I had to create a VM intrinsic (_darn) in vmSymbols that is, let's say, arch dependent, and that seems to be the only case so far, but on the other hand a new JCA provider (with methods to be intrinsified) is necessary (I don't see another way to intrinsify the methods in NativePRNG/SHA1PRNG providers since I need a software fallback to darn). Do you have any recommendation about it? The patchset rebased on top of jdk11 (http://hg.openjdk.java.net/jdk/hs) is: http://cr.openjdk.java.net/~gromero/POWER9/darn/v1/0_PPC64_Add_JCA_provider_to_exploit_HW_RNG_on_POWER9.patch http://cr.openjdk.java.net/~gromero/POWER9/darn/v1/1_PPC64_Assembler_add_support_for_darn_Deliver_A_Random_Number_instruction.patch http://cr.openjdk.java.net/~gromero/POWER9/darn/v1/2_PPC64_Interpreter_add_template_to_exploit_darn_instruction.patch http://cr.openjdk.java.net/~gromero/POWER9/darn/v1/3_PPC64_C2_Compiler_add_new_node_to_exploit_darn_instruction.patch http://cr.openjdk.java.net/~gromero/POWER9/darn/v1/4_PPC64_C1_Compiler_add_intrinsic_to_exploit_darn_instruction.patch I intend to contribute that change as an experimental feature. Thank you. Best regards, Gustavo [1] https://github.com/gromero/darn/blob/master/next_bytes.java [2] https://github.com/gromero/darn/blob/master/generate_seed.java [3] https://github.com/gromero/darn/blob/master/C/darn.c [4] https://github.com/gromero/darn/blob/master/C/darn.log From martin.doerr at sap.com Tue Apr 3 12:50:29 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 3 Apr 2018 12:50:29 +0000 Subject: [RFC] Re: POWER9: Is there a way to improve the random number generation on PPC64? In-Reply-To: <8347ea90-1132-1dc0-b04c-cbcd081a68ef@linux.vnet.ibm.com> References: <5A1887FF.1030406@linux.vnet.ibm.com> <8347ea90-1132-1dc0-b04c-cbcd081a68ef@linux.vnet.ibm.com> Message-ID: <75501eb1672a45389db40ef0760142c3@sap.com> Hi Gustavo, thanks for posting your change. I think the Java and shared C++ code and should not use PPC64 specific names because it may get used for other platforms as well? Some people don't want to trust relying solely on the hardware number generator which cannot get reviewed publicly. So would it make sense to use the instruction mixed with something else? It would be good to have the complete change in one webrev for easier reviewing. Thanks and best regards, Martin -----Original Message----- From: Gustavo Romero [mailto:gromero at linux.vnet.ibm.com] Sent: Montag, 2. April 2018 13:55 To: Volker Simonis ; Doerr, Martin ; vladimir.kozlov at oracle.com Cc: ppc-aix-port-dev at openjdk.java.net Subject: [RFC] Re: POWER9: Is there a way to improve the random number generation on PPC64? Importance: High Hi Martin, Volker, Vladimir Sorry for the huge delay replaying on this... I hope Martin (and Lutz) are feeling better and fully recovered. On 11/24/2017 08:04 PM, Volker Simonis wrote: > Hi Gustavo, > > in one of my talks [1,2] I have an example on how to intrinsify > Random.nextInt() in C2 by using the Intel 'rdrandl' instruction. But > please notice that this is just a "toy" example - it is not production > ready. In fact I think the right way would be to create a new > SecureRandom provider where you may implement "engineNextBytes" by > using the new Power instruction (maybe by calling a native function). > This special implementation of "engineNextBytes" could then be > intrinsified as described in the talk. > > Also, before you start this, please contact the security mailing list > just to make sure you're not going into the wrong direction (I'm not a > security expert :) I've created a new JCA provider called 'P9TRNG' and implemented a darn intrinsic for Interpreter, C1, and C2 compiler and did a couple of tests using micro benches [1, 2] to check the latency and throughput to get a random number using generateSeed() and nextBytes() with darn in place. The 'P9TRNG' provider is basically a copy of 'NativePRNG' since it's necessary a software fallback in case darn instruction fails to return a valid random number after ten attempts (although it's very rare condition). On the other hand 'P9TRNG' uses the darn intrinsic when it's available. The maximum theoretical throughput on the machine I'm testing it (a POWER9 witherspoon) is 128Mbps and there is one RNG per socket, so only one RNG per CPU. With a simple C code it's possible to get very close to that value (please see C code [3] for code details and log [4] for the expected outputs). Unrolling the tight loop does not help and causes a performance degradation. On Hotspot, for Interpreter and C1 the throughput is ~3x higher than the version that does not use darn instruction (using micro benches [1, 2]): gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java next_bytes SHA1PRNG 1024 10 3.8759432E7 ns 2.113550 Mbps gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java next_bytes SHA1PRNG 1024 100000 2.65902244E10 ns 30.808313 Mbps gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java next_bytes NativePRNG 1024 100 7.1741008E7 ns 11.418853 Mbps gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java next_bytes NativePRNG 1024 100000 2.74547937E10 ns 29.838140 Mbps gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java -Xcomp -XX:TieredStopAtLevel=3 next_bytes NativePRNG 1024 100000 5.5632339E10 ns 14.725248 Mbps gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java -Xcomp -XX:-TieredCompilation next_bytes NativePRNG 1024 100000 2.78629519E10 ns 29.401051 Mbps [With darn disabled: performance like NativePRNG] gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java next_bytes P9TRNG 1024 100 7.0272888E7 ns 11.657412 Mbps gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java next_bytes P9TRNG 1024 100000 2.75566244E10 ns 29.727880 Mbps ... [With darn enabled] gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseDARN next_bytes P9TRNG 1024 100 8305029.0 ns 98.639030 Mbps gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseDARN next_bytes P9TRNG 1024 100000 6.442112E9 ns 127.163261 Mbps gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseDARN -Xcomp -XX:TieredStopAtLevel=3 next_bytes P9TRNG 1024 100000 1.57303337E10 ns 52.077728 Mbps gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseDARN -Xcomp -XX:-TieredCompilation next_bytes P9TRNG 1024 100000 6.46914E9 ns 126.631973 Mbps For C2 compiler using darn is better until it reaches ~128Mbps (maximum theoretical throughput), but on the other hand it never blocks, so, for instance, generateSeed() which normally uses /dev/random (blocking) is not affected by a lack of entropy in Linux entropy pool. @Vladimir, Volker mentioned that you already experimented with rand on Intel. Do you know if creating a new JCA provider as I did is a reasonable approach to exploit darn on POWER9? Also, in my implementation I had to create a VM intrinsic (_darn) in vmSymbols that is, let's say, arch dependent, and that seems to be the only case so far, but on the other hand a new JCA provider (with methods to be intrinsified) is necessary (I don't see another way to intrinsify the methods in NativePRNG/SHA1PRNG providers since I need a software fallback to darn). Do you have any recommendation about it? The patchset rebased on top of jdk11 (http://hg.openjdk.java.net/jdk/hs) is: http://cr.openjdk.java.net/~gromero/POWER9/darn/v1/0_PPC64_Add_JCA_provider_to_exploit_HW_RNG_on_POWER9.patch http://cr.openjdk.java.net/~gromero/POWER9/darn/v1/1_PPC64_Assembler_add_support_for_darn_Deliver_A_Random_Number_instruction.patch http://cr.openjdk.java.net/~gromero/POWER9/darn/v1/2_PPC64_Interpreter_add_template_to_exploit_darn_instruction.patch http://cr.openjdk.java.net/~gromero/POWER9/darn/v1/3_PPC64_C2_Compiler_add_new_node_to_exploit_darn_instruction.patch http://cr.openjdk.java.net/~gromero/POWER9/darn/v1/4_PPC64_C1_Compiler_add_intrinsic_to_exploit_darn_instruction.patch I intend to contribute that change as an experimental feature. Thank you. Best regards, Gustavo [1] https://github.com/gromero/darn/blob/master/next_bytes.java [2] https://github.com/gromero/darn/blob/master/generate_seed.java [3] https://github.com/gromero/darn/blob/master/C/darn.c [4] https://github.com/gromero/darn/blob/master/C/darn.log From gromero at linux.vnet.ibm.com Fri Apr 6 00:22:41 2018 From: gromero at linux.vnet.ibm.com (Gustavo Romero) Date: Thu, 5 Apr 2018 21:22:41 -0300 Subject: RFR(XS): 8201218: PPC64: Avoid use of yield instruction on spinlock Message-ID: Hi, Could the following change be reviewed please? bug : https://bugs.openjdk.java.net/browse/JDK-8201218 webrev: http://cr.openjdk.java.net/~gromero/8201218/v1/ It avoids the use of 'yield' ('or 27, 27, 27') supposed to yield up thread resources to another thread once, for instance, it's necessary to perform a spinlock as in the current code found in the following functions: MacroAssembler::rtm_retry_lock_on_abort() MacroAssembler::rtm_retry_lock_on_busy() because 'yield' has no effect on POWER processors, being treat as a 'nop' instruction and taking no action over the PPR (Program Priority Register). Instead of 'yield' 'or 1, 1, 1' instruction is used to set thread priority to low before entering the spin loop in rtm_retry_lock_on_busy and a proper 'or' instruction is used after leaving the spin loop in order to set thread priority back to its default value in userspace (to restore it). The default value in PPR's PRI field in userspace, unfortunately, is different on Linux and AIX: on Linux default is 0b011 (medium low) [1] whilst on AIX default is 0b100 (medium / aka normal) [2], so for each case a different 'or' is present immediately after leaving the loop. Before entering the loop, after discussing with the PPC Linux team, the recommendation is the following rule of thumb: - For known long delays (about u/mseconds) use 'very low'; - For shorter delays use just 'low' priority. So I selected 'low' priority for the spinlock in question. ISA in fact states that "if a program is waiting on a lock, it could set low priority", but it's not clear from the text if it means low in general sense or indeed 'low' (0b010), so I'm using 'low' before entering the loop based upon the rule above. Finally, 'yield' was removed from rtm_retry_lock_on_abort in order to avoid that the program execution leaves that function with PPR set as 'low' since it's possible to "escape" from that function without ever resetting the PPR back to it's default value in userspace. Thank you and best regards, Gustavo [1] https://github.com/torvalds/linux/blob/master/arch/powerpc/include/asm/processor.h#L32 [2] https://www.ibm.com/support/knowledgecenter/en/ssw_aix_71/com.ibm.aix.ktechrf1/thread_set_smt_priority.htm From martin.doerr at sap.com Fri Apr 6 08:23:13 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Fri, 6 Apr 2018 08:23:13 +0000 Subject: RFR(XS): 8201218: PPC64: Avoid use of yield instruction on spinlock In-Reply-To: References: Message-ID: <98056f62488a43028746dc8cae0642df@sap.com> Hi Gustavo, thanks for fixing this issue and for the explanations. Looks good. I've reviewed it and I can sponsor it after a 2nd review. Best regards, Martin -----Original Message----- From: Gustavo Romero [mailto:gromero at linux.vnet.ibm.com] Sent: Freitag, 6. April 2018 02:23 To: hotspot-compiler-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net Cc: Doerr, Martin Subject: RFR(XS): 8201218: PPC64: Avoid use of yield instruction on spinlock Hi, Could the following change be reviewed please? bug : https://bugs.openjdk.java.net/browse/JDK-8201218 webrev: http://cr.openjdk.java.net/~gromero/8201218/v1/ It avoids the use of 'yield' ('or 27, 27, 27') supposed to yield up thread resources to another thread once, for instance, it's necessary to perform a spinlock as in the current code found in the following functions: MacroAssembler::rtm_retry_lock_on_abort() MacroAssembler::rtm_retry_lock_on_busy() because 'yield' has no effect on POWER processors, being treat as a 'nop' instruction and taking no action over the PPR (Program Priority Register). Instead of 'yield' 'or 1, 1, 1' instruction is used to set thread priority to low before entering the spin loop in rtm_retry_lock_on_busy and a proper 'or' instruction is used after leaving the spin loop in order to set thread priority back to its default value in userspace (to restore it). The default value in PPR's PRI field in userspace, unfortunately, is different on Linux and AIX: on Linux default is 0b011 (medium low) [1] whilst on AIX default is 0b100 (medium / aka normal) [2], so for each case a different 'or' is present immediately after leaving the loop. Before entering the loop, after discussing with the PPC Linux team, the recommendation is the following rule of thumb: - For known long delays (about u/mseconds) use 'very low'; - For shorter delays use just 'low' priority. So I selected 'low' priority for the spinlock in question. ISA in fact states that "if a program is waiting on a lock, it could set low priority", but it's not clear from the text if it means low in general sense or indeed 'low' (0b010), so I'm using 'low' before entering the loop based upon the rule above. Finally, 'yield' was removed from rtm_retry_lock_on_abort in order to avoid that the program execution leaves that function with PPR set as 'low' since it's possible to "escape" from that function without ever resetting the PPR back to it's default value in userspace. Thank you and best regards, Gustavo [1] https://github.com/torvalds/linux/blob/master/arch/powerpc/include/asm/processor.h#L32 [2] https://www.ibm.com/support/knowledgecenter/en/ssw_aix_71/com.ibm.aix.ktechrf1/thread_set_smt_priority.htm From goetz.lindenmaier at sap.com Mon Apr 9 16:03:10 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 9 Apr 2018 16:03:10 +0000 Subject: RFR(XS): 8201218: PPC64: Avoid use of yield instruction on spinlock In-Reply-To: References: Message-ID: <81ea1232cbca4e17b968881f13ac09bb@sap.com> Hi Gustavo, the change looks good, thanks. Interesting to find that this was never implemented ... Will setting the priorities interfere with the thread's priorities that can be set in Java? In that case, re-setting the level might be not correct. Best regards, Goetz. > -----Original Message----- > From: ppc-aix-port-dev [mailto:ppc-aix-port-dev- > bounces at openjdk.java.net] On Behalf Of Gustavo Romero > Sent: Freitag, 6. April 2018 02:23 > To: hotspot-compiler-dev at openjdk.java.net; ppc-aix-port- > dev at openjdk.java.net > Subject: RFR(XS): 8201218: PPC64: Avoid use of yield instruction on spinlock > > Hi, > > Could the following change be reviewed please? > > bug : https://bugs.openjdk.java.net/browse/JDK-8201218 > webrev: http://cr.openjdk.java.net/~gromero/8201218/v1/ > > It avoids the use of 'yield' ('or 27, 27, 27') supposed to yield > up thread resources to another thread once, for instance, it's > necessary to perform a spinlock as in the current code found in > the following functions: > > MacroAssembler::rtm_retry_lock_on_abort() > MacroAssembler::rtm_retry_lock_on_busy() > > because 'yield' has no effect on POWER processors, being treat as > a 'nop' instruction and taking no action over the PPR (Program > Priority Register). > > Instead of 'yield' 'or 1, 1, 1' instruction is used to set thread > priority to low before entering the spin loop in > rtm_retry_lock_on_busy and a proper 'or' instruction is used after > leaving the spin loop in order to set thread priority back to its > default value in userspace (to restore it). > > The default value in PPR's PRI field in userspace, unfortunately, > is different on Linux and AIX: on Linux default is 0b011 > (medium low) [1] whilst on AIX default is 0b100 (medium / aka > normal) [2], so for each case a different 'or' is present > immediately after leaving the loop. > > Before entering the loop, after discussing with the PPC Linux > team, the recommendation is the following rule of thumb: > > - For known long delays (about u/mseconds) use 'very low'; > - For shorter delays use just 'low' priority. > > So I selected 'low' priority for the spinlock in question. > > ISA in fact states that "if a program is waiting on a lock, it > could set low priority", but it's not clear from the text if > it means low in general sense or indeed 'low' (0b010), so I'm > using 'low' before entering the loop based upon the rule above. > > Finally, 'yield' was removed from rtm_retry_lock_on_abort in > order to avoid that the program execution leaves that function > with PPR set as 'low' since it's possible to "escape" from that > function without ever resetting the PPR back to it's default > value in userspace. > > Thank you and best regards, > Gustavo > > [1] > https://github.com/torvalds/linux/blob/master/arch/powerpc/include/asm/ > processor.h#L32 > [2] > https://www.ibm.com/support/knowledgecenter/en/ssw_aix_71/com.ibm. > aix.ktechrf1/thread_set_smt_priority.htm From gromero at linux.vnet.ibm.com Tue Apr 10 04:02:41 2018 From: gromero at linux.vnet.ibm.com (Gustavo Romero) Date: Tue, 10 Apr 2018 01:02:41 -0300 Subject: RFR(XS): 8201218: PPC64: Avoid use of yield instruction on spinlock In-Reply-To: <81ea1232cbca4e17b968881f13ac09bb@sap.com> References: <81ea1232cbca4e17b968881f13ac09bb@sap.com> Message-ID: Hi Goetz, On 04/09/2018 01:03 PM, Lindenmaier, Goetz wrote: > the change looks good, thanks. Thanks a lot for reviewing it. > Interesting to find that this was never implemented ... Yes... they were removed from the text of ISA v3 btw. > Will setting the priorities interfere with the thread's > priorities that can be set in Java? In that case, re-setting > the level might be not correct. Underneath Java threads is pthreads and for pthreads PPR has not effect at all in the sense that it does not modify any kernel scheduling priority neither for the pthread / process that changed the PPR nor for the other pthreads / processes. Lowering the PPR priority just makes the waiter use less resources and release resources for the other threads (or CPUs) on the same core, including for the thread where the holder might be. On Linux, the PPR value will be saved/restored across the context switches. Just on a syscall the value will be reset back to its default (i.e. on Linux: medium low, 0b011). On AIX I'm not so sure, but I think that on the worst case AIX will reset back the PPR to its default value even on context switches. Regards, Gustavo > Best regards, > Goetz. > >> -----Original Message----- >> From: ppc-aix-port-dev [mailto:ppc-aix-port-dev- >> bounces at openjdk.java.net] On Behalf Of Gustavo Romero >> Sent: Freitag, 6. April 2018 02:23 >> To: hotspot-compiler-dev at openjdk.java.net; ppc-aix-port- >> dev at openjdk.java.net >> Subject: RFR(XS): 8201218: PPC64: Avoid use of yield instruction on spinlock >> >> Hi, >> >> Could the following change be reviewed please? >> >> bug : https://bugs.openjdk.java.net/browse/JDK-8201218 >> webrev: http://cr.openjdk.java.net/~gromero/8201218/v1/ >> >> It avoids the use of 'yield' ('or 27, 27, 27') supposed to yield >> up thread resources to another thread once, for instance, it's >> necessary to perform a spinlock as in the current code found in >> the following functions: >> >> MacroAssembler::rtm_retry_lock_on_abort() >> MacroAssembler::rtm_retry_lock_on_busy() >> >> because 'yield' has no effect on POWER processors, being treat as >> a 'nop' instruction and taking no action over the PPR (Program >> Priority Register). >> >> Instead of 'yield' 'or 1, 1, 1' instruction is used to set thread >> priority to low before entering the spin loop in >> rtm_retry_lock_on_busy and a proper 'or' instruction is used after >> leaving the spin loop in order to set thread priority back to its >> default value in userspace (to restore it). >> >> The default value in PPR's PRI field in userspace, unfortunately, >> is different on Linux and AIX: on Linux default is 0b011 >> (medium low) [1] whilst on AIX default is 0b100 (medium / aka >> normal) [2], so for each case a different 'or' is present >> immediately after leaving the loop. >> >> Before entering the loop, after discussing with the PPC Linux >> team, the recommendation is the following rule of thumb: >> >> - For known long delays (about u/mseconds) use 'very low'; >> - For shorter delays use just 'low' priority. >> >> So I selected 'low' priority for the spinlock in question. >> >> ISA in fact states that "if a program is waiting on a lock, it >> could set low priority", but it's not clear from the text if >> it means low in general sense or indeed 'low' (0b010), so I'm >> using 'low' before entering the loop based upon the rule above. >> >> Finally, 'yield' was removed from rtm_retry_lock_on_abort in >> order to avoid that the program execution leaves that function >> with PPR set as 'low' since it's possible to "escape" from that >> function without ever resetting the PPR back to it's default >> value in userspace. >> >> Thank you and best regards, >> Gustavo >> >> [1] >> https://github.com/torvalds/linux/blob/master/arch/powerpc/include/asm/ >> processor.h#L32 >> [2] >> https://www.ibm.com/support/knowledgecenter/en/ssw_aix_71/com.ibm. >> aix.ktechrf1/thread_set_smt_priority.htm > From goetz.lindenmaier at sap.com Tue Apr 10 07:30:07 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 10 Apr 2018 07:30:07 +0000 Subject: RFR(XS): 8201218: PPC64: Avoid use of yield instruction on spinlock In-Reply-To: References: <81ea1232cbca4e17b968881f13ac09bb@sap.com> Message-ID: Hi Gustavo, thanks for the explanation. Best regards, Goetz. > -----Original Message----- > From: Gustavo Romero [mailto:gromero at linux.vnet.ibm.com] > Sent: Dienstag, 10. April 2018 06:03 > To: Lindenmaier, Goetz ; hotspot-compiler- > dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net > Subject: Re: RFR(XS): 8201218: PPC64: Avoid use of yield instruction on > spinlock > > Hi Goetz, > > On 04/09/2018 01:03 PM, Lindenmaier, Goetz wrote: > > the change looks good, thanks. > > Thanks a lot for reviewing it. > > > > Interesting to find that this was never implemented ... > > Yes... they were removed from the text of ISA v3 btw. > > > > Will setting the priorities interfere with the thread's > > priorities that can be set in Java? In that case, re-setting > > the level might be not correct. > > Underneath Java threads is pthreads and for pthreads PPR has not effect at > all > in the sense that it does not modify any kernel scheduling priority neither for > the pthread / process that changed the PPR nor for the other pthreads / > processes. Lowering the PPR priority just makes the waiter use less > resources > and release resources for the other threads (or CPUs) on the same core, > including for the thread where the holder might be. > > On Linux, the PPR value will be saved/restored across the context switches. > Just > on a syscall the value will be reset back to its default (i.e. on Linux: medium > low, 0b011). On AIX I'm not so sure, but I think that on the worst case AIX > will reset back the PPR to its default value even on context switches. > > > Regards, > Gustavo > > > Best regards, > > Goetz. > > > >> -----Original Message----- > >> From: ppc-aix-port-dev [mailto:ppc-aix-port-dev- > >> bounces at openjdk.java.net] On Behalf Of Gustavo Romero > >> Sent: Freitag, 6. April 2018 02:23 > >> To: hotspot-compiler-dev at openjdk.java.net; ppc-aix-port- > >> dev at openjdk.java.net > >> Subject: RFR(XS): 8201218: PPC64: Avoid use of yield instruction on > spinlock > >> > >> Hi, > >> > >> Could the following change be reviewed please? > >> > >> bug : https://bugs.openjdk.java.net/browse/JDK-8201218 > >> webrev: http://cr.openjdk.java.net/~gromero/8201218/v1/ > >> > >> It avoids the use of 'yield' ('or 27, 27, 27') supposed to yield > >> up thread resources to another thread once, for instance, it's > >> necessary to perform a spinlock as in the current code found in > >> the following functions: > >> > >> MacroAssembler::rtm_retry_lock_on_abort() > >> MacroAssembler::rtm_retry_lock_on_busy() > >> > >> because 'yield' has no effect on POWER processors, being treat as > >> a 'nop' instruction and taking no action over the PPR (Program > >> Priority Register). > >> > >> Instead of 'yield' 'or 1, 1, 1' instruction is used to set thread > >> priority to low before entering the spin loop in > >> rtm_retry_lock_on_busy and a proper 'or' instruction is used after > >> leaving the spin loop in order to set thread priority back to its > >> default value in userspace (to restore it). > >> > >> The default value in PPR's PRI field in userspace, unfortunately, > >> is different on Linux and AIX: on Linux default is 0b011 > >> (medium low) [1] whilst on AIX default is 0b100 (medium / aka > >> normal) [2], so for each case a different 'or' is present > >> immediately after leaving the loop. > >> > >> Before entering the loop, after discussing with the PPC Linux > >> team, the recommendation is the following rule of thumb: > >> > >> - For known long delays (about u/mseconds) use 'very low'; > >> - For shorter delays use just 'low' priority. > >> > >> So I selected 'low' priority for the spinlock in question. > >> > >> ISA in fact states that "if a program is waiting on a lock, it > >> could set low priority", but it's not clear from the text if > >> it means low in general sense or indeed 'low' (0b010), so I'm > >> using 'low' before entering the loop based upon the rule above. > >> > >> Finally, 'yield' was removed from rtm_retry_lock_on_abort in > >> order to avoid that the program execution leaves that function > >> with PPR set as 'low' since it's possible to "escape" from that > >> function without ever resetting the PPR back to it's default > >> value in userspace. > >> > >> Thank you and best regards, > >> Gustavo > >> > >> [1] > >> > https://github.com/torvalds/linux/blob/master/arch/powerpc/include/asm/ > >> processor.h#L32 > >> [2] > >> > https://www.ibm.com/support/knowledgecenter/en/ssw_aix_71/com.ibm. > >> aix.ktechrf1/thread_set_smt_priority.htm > > From martin.doerr at sap.com Tue Apr 10 09:47:50 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 10 Apr 2018 09:47:50 +0000 Subject: RFR(XS): 8201218: PPC64: Avoid use of yield instruction on spinlock In-Reply-To: References: <81ea1232cbca4e17b968881f13ac09bb@sap.com> Message-ID: <24be893cdaf142debf800d0d95c81e80@sap.com> Hi Gustavo, thanks for the additional explanations. I've pushed it. Best regards, Martin -----Original Message----- From: hotspot-compiler-dev [mailto:hotspot-compiler-dev-bounces at openjdk.java.net] On Behalf Of Gustavo Romero Sent: Dienstag, 10. April 2018 06:03 To: Lindenmaier, Goetz ; hotspot-compiler-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net Subject: Re: RFR(XS): 8201218: PPC64: Avoid use of yield instruction on spinlock Hi Goetz, On 04/09/2018 01:03 PM, Lindenmaier, Goetz wrote: > the change looks good, thanks. Thanks a lot for reviewing it. > Interesting to find that this was never implemented ... Yes... they were removed from the text of ISA v3 btw. > Will setting the priorities interfere with the thread's > priorities that can be set in Java? In that case, re-setting > the level might be not correct. Underneath Java threads is pthreads and for pthreads PPR has not effect at all in the sense that it does not modify any kernel scheduling priority neither for the pthread / process that changed the PPR nor for the other pthreads / processes. Lowering the PPR priority just makes the waiter use less resources and release resources for the other threads (or CPUs) on the same core, including for the thread where the holder might be. On Linux, the PPR value will be saved/restored across the context switches. Just on a syscall the value will be reset back to its default (i.e. on Linux: medium low, 0b011). On AIX I'm not so sure, but I think that on the worst case AIX will reset back the PPR to its default value even on context switches. Regards, Gustavo > Best regards, > Goetz. > >> -----Original Message----- >> From: ppc-aix-port-dev [mailto:ppc-aix-port-dev- >> bounces at openjdk.java.net] On Behalf Of Gustavo Romero >> Sent: Freitag, 6. April 2018 02:23 >> To: hotspot-compiler-dev at openjdk.java.net; ppc-aix-port- >> dev at openjdk.java.net >> Subject: RFR(XS): 8201218: PPC64: Avoid use of yield instruction on spinlock >> >> Hi, >> >> Could the following change be reviewed please? >> >> bug : https://bugs.openjdk.java.net/browse/JDK-8201218 >> webrev: http://cr.openjdk.java.net/~gromero/8201218/v1/ >> >> It avoids the use of 'yield' ('or 27, 27, 27') supposed to yield >> up thread resources to another thread once, for instance, it's >> necessary to perform a spinlock as in the current code found in >> the following functions: >> >> MacroAssembler::rtm_retry_lock_on_abort() >> MacroAssembler::rtm_retry_lock_on_busy() >> >> because 'yield' has no effect on POWER processors, being treat as >> a 'nop' instruction and taking no action over the PPR (Program >> Priority Register). >> >> Instead of 'yield' 'or 1, 1, 1' instruction is used to set thread >> priority to low before entering the spin loop in >> rtm_retry_lock_on_busy and a proper 'or' instruction is used after >> leaving the spin loop in order to set thread priority back to its >> default value in userspace (to restore it). >> >> The default value in PPR's PRI field in userspace, unfortunately, >> is different on Linux and AIX: on Linux default is 0b011 >> (medium low) [1] whilst on AIX default is 0b100 (medium / aka >> normal) [2], so for each case a different 'or' is present >> immediately after leaving the loop. >> >> Before entering the loop, after discussing with the PPC Linux >> team, the recommendation is the following rule of thumb: >> >> - For known long delays (about u/mseconds) use 'very low'; >> - For shorter delays use just 'low' priority. >> >> So I selected 'low' priority for the spinlock in question. >> >> ISA in fact states that "if a program is waiting on a lock, it >> could set low priority", but it's not clear from the text if >> it means low in general sense or indeed 'low' (0b010), so I'm >> using 'low' before entering the loop based upon the rule above. >> >> Finally, 'yield' was removed from rtm_retry_lock_on_abort in >> order to avoid that the program execution leaves that function >> with PPR set as 'low' since it's possible to "escape" from that >> function without ever resetting the PPR back to it's default >> value in userspace. >> >> Thank you and best regards, >> Gustavo >> >> [1] >> https://github.com/torvalds/linux/blob/master/arch/powerpc/include/asm/ >> processor.h#L32 >> [2] >> https://www.ibm.com/support/knowledgecenter/en/ssw_aix_71/com.ibm. >> aix.ktechrf1/thread_set_smt_priority.htm > From gromero at linux.vnet.ibm.com Tue Apr 10 14:19:00 2018 From: gromero at linux.vnet.ibm.com (Gustavo Romero) Date: Tue, 10 Apr 2018 11:19:00 -0300 Subject: RFR(XS): 8201218: PPC64: Avoid use of yield instruction on spinlock In-Reply-To: <24be893cdaf142debf800d0d95c81e80@sap.com> References: <81ea1232cbca4e17b968881f13ac09bb@sap.com> <24be893cdaf142debf800d0d95c81e80@sap.com> Message-ID: <8325165a-5380-1c3c-a42e-8b5dbbb57204@linux.vnet.ibm.com> Thank you. On 04/10/2018 06:47 AM, Doerr, Martin wrote: > Hi Gustavo, > > thanks for the additional explanations. I've pushed it. > > Best regards, > Martin > > > -----Original Message----- > From: hotspot-compiler-dev [mailto:hotspot-compiler-dev-bounces at openjdk.java.net] On Behalf Of Gustavo Romero > Sent: Dienstag, 10. April 2018 06:03 > To: Lindenmaier, Goetz ; hotspot-compiler-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net > Subject: Re: RFR(XS): 8201218: PPC64: Avoid use of yield instruction on spinlock > > Hi Goetz, > > On 04/09/2018 01:03 PM, Lindenmaier, Goetz wrote: >> the change looks good, thanks. > > Thanks a lot for reviewing it. > > >> Interesting to find that this was never implemented ... > > Yes... they were removed from the text of ISA v3 btw. > > >> Will setting the priorities interfere with the thread's >> priorities that can be set in Java? In that case, re-setting >> the level might be not correct. > > Underneath Java threads is pthreads and for pthreads PPR has not effect at all > in the sense that it does not modify any kernel scheduling priority neither for > the pthread / process that changed the PPR nor for the other pthreads / > processes. Lowering the PPR priority just makes the waiter use less resources > and release resources for the other threads (or CPUs) on the same core, > including for the thread where the holder might be. > > On Linux, the PPR value will be saved/restored across the context switches. Just > on a syscall the value will be reset back to its default (i.e. on Linux: medium > low, 0b011). On AIX I'm not so sure, but I think that on the worst case AIX > will reset back the PPR to its default value even on context switches. > > > Regards, > Gustavo > >> Best regards, >> Goetz. >> >>> -----Original Message----- >>> From: ppc-aix-port-dev [mailto:ppc-aix-port-dev- >>> bounces at openjdk.java.net] On Behalf Of Gustavo Romero >>> Sent: Freitag, 6. April 2018 02:23 >>> To: hotspot-compiler-dev at openjdk.java.net; ppc-aix-port- >>> dev at openjdk.java.net >>> Subject: RFR(XS): 8201218: PPC64: Avoid use of yield instruction on spinlock >>> >>> Hi, >>> >>> Could the following change be reviewed please? >>> >>> bug : https://bugs.openjdk.java.net/browse/JDK-8201218 >>> webrev: http://cr.openjdk.java.net/~gromero/8201218/v1/ >>> >>> It avoids the use of 'yield' ('or 27, 27, 27') supposed to yield >>> up thread resources to another thread once, for instance, it's >>> necessary to perform a spinlock as in the current code found in >>> the following functions: >>> >>> MacroAssembler::rtm_retry_lock_on_abort() >>> MacroAssembler::rtm_retry_lock_on_busy() >>> >>> because 'yield' has no effect on POWER processors, being treat as >>> a 'nop' instruction and taking no action over the PPR (Program >>> Priority Register). >>> >>> Instead of 'yield' 'or 1, 1, 1' instruction is used to set thread >>> priority to low before entering the spin loop in >>> rtm_retry_lock_on_busy and a proper 'or' instruction is used after >>> leaving the spin loop in order to set thread priority back to its >>> default value in userspace (to restore it). >>> >>> The default value in PPR's PRI field in userspace, unfortunately, >>> is different on Linux and AIX: on Linux default is 0b011 >>> (medium low) [1] whilst on AIX default is 0b100 (medium / aka >>> normal) [2], so for each case a different 'or' is present >>> immediately after leaving the loop. >>> >>> Before entering the loop, after discussing with the PPC Linux >>> team, the recommendation is the following rule of thumb: >>> >>> - For known long delays (about u/mseconds) use 'very low'; >>> - For shorter delays use just 'low' priority. >>> >>> So I selected 'low' priority for the spinlock in question. >>> >>> ISA in fact states that "if a program is waiting on a lock, it >>> could set low priority", but it's not clear from the text if >>> it means low in general sense or indeed 'low' (0b010), so I'm >>> using 'low' before entering the loop based upon the rule above. >>> >>> Finally, 'yield' was removed from rtm_retry_lock_on_abort in >>> order to avoid that the program execution leaves that function >>> with PPR set as 'low' since it's possible to "escape" from that >>> function without ever resetting the PPR back to it's default >>> value in userspace. >>> >>> Thank you and best regards, >>> Gustavo >>> >>> [1] >>> https://github.com/torvalds/linux/blob/master/arch/powerpc/include/asm/ >>> processor.h#L32 >>> [2] >>> https://www.ibm.com/support/knowledgecenter/en/ssw_aix_71/com.ibm. >>> aix.ktechrf1/thread_set_smt_priority.htm >> > From volker.simonis at gmail.com Fri Apr 13 15:21:25 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Fri, 13 Apr 2018 17:21:25 +0200 Subject: Missing many locales support on AIX platform In-Reply-To: References: <59d3d31b-b596-0678-c0bf-69b8f951a40d@oracle.com> Message-ID: Hi Bhaktavatsal Reddy, thanks for addressing this long standing issue. I've opened "8201540: [AIX] Extend the set of supported charsets in java.base" [1] to track this issue. As I wrote in the bug report, this problem is the consequence of an emergency fix (JDK-8081332) I did back in 2015 to fix the build on AIX after the integration of the modularity support change (see discussion: [2]). At that time I only added the minimal set of charsets which were required to fix the build. It would be great if you can get in touch with your IBM colleagues to find out which are the default extended charsets supported by IBM J9 on AIX. I think we should try to use a similar set here in our OpenJDK port which is also used by the OpenJ9 for building OpenJ9 on AIX. Also, your IBM colleagues can help you to host a webrev which will make the further review of these changes much easier. I've but Tim from IBM on CC which should have an overview of all the IBM people involved in the OpenJDK project. Besides that I know at least the following people who might help you: Michihiro Horie: HORIE at jp.ibm.com Hiroshi H Horii: horii at jp.ibm.com Gustavo Romero: gromero at linux.vnet.ibm.com Matthew Brandyberyy: mbrandy at linux.vnet.ibm.com Thank you and best regards, Volker [1] https://bugs.openjdk.java.net/browse/JDK-8201540 [2] http://mail.openjdk.java.net/pipermail/2d-dev/2015-May/005431.html On Fri, Apr 13, 2018 at 2:42 PM, Bhaktavatsal R Maram wrote: > Hi Alan, > > Thank you for your response. I'm happy that my patch was attached. But, I don't see attachment. So, I inlined patch which contain diffs from 2 changesets in mail text. If a Jira bug is opened for this issue, probably I can attach complete and consolidated patch there. > > At high level, I'm adding following charsets to standard charset in java.base. For this, I have to change IBM943C and IBM942C from source to template to handle java package and aliases. It is also required to add codepage 932 as alias for IBM942C because both are one and same. > > Big5, Big5_HKSCS, GB18030, IBM942, IBM942C, IBM943, IBM943C, IBM950, IBM970, IBM1124, TIS_620 > > > These are default charsets for some of locales supported by Operating System (AIX). Since these are not available in standard charset, JDK can't be used in those locale even if they are available in jdk.charset module (java -version fails). > > I've followed some of the discussions around this in community and understand that default charset of a locale should be made available in java.charset module by using stdcs-* mechanism. On Linux, they were added to java.base in similar way. As it is missing for AIX, I've added them to enable JDK support for more locales. > > > Thanks, > Bhaktavatsal Reddy > > -----Alan Bateman wrote: ----- > To: Bhaktavatsal R Maram , core-libs-dev at openjdk.java.net > From: Alan Bateman > Date: 04/13/2018 03:52PM > Subject: Re: Missing many locales support on AIX platform > > On 13/04/2018 10:35, Bhaktavatsal R Maram wrote: >> Given that patch is big, I am sending patch as attachment again after changing some mail settings. Hopefully, it will make to community this time. > Your patch was attached. > > If I read it correctly, you've switched IBM943C to a template but there > aren't additional aliases so this part is effectively a no-op, is that > right? For IBM932C, you've moved it to be template and added several > aliases. > > The rest is AIX specific and I hope the SAP engineers that maintain the > AIX port can help you. It may be that you are testing with locales that > aren't supported configurations for the AIX port in OpenJDK. As a > general point, we try to keep as many of the exotic and multibyte > charsets out of java.base. They are of course still available to > applications via the API and the jdk.charsets service provider module. > > -Alan > > From bhamaram at in.ibm.com Fri Apr 13 15:31:05 2018 From: bhamaram at in.ibm.com (Bhaktavatsal R Maram) Date: Fri, 13 Apr 2018 15:31:05 +0000 Subject: Missing many locales support on AIX platform In-Reply-To: References: , <59d3d31b-b596-0678-c0bf-69b8f951a40d@oracle.com> Message-ID: Hi Volker, Thank you. I will take help of my colleagues and host webrev with all the default charsets that are missing for AIX. - Bhaktavatsal Reddy -----Volker Simonis wrote: ----- To: Bhaktavatsal R Maram From: Volker Simonis Date: 04/13/2018 08:51PM Cc: Alan Bateman , Java Core Libs , Tim Ellison , ppc-aix-port-dev at openjdk.java.net Subject: Re: Missing many locales support on AIX platform Hi Bhaktavatsal Reddy, thanks for addressing this long standing issue. I've opened "8201540: [AIX] Extend the set of supported charsets in java.base" [1] to track this issue. As I wrote in the bug report, this problem is the consequence of an emergency fix (JDK-8081332) I did back in 2015 to fix the build on AIX after the integration of the modularity support change (see discussion: [2]). At that time I only added the minimal set of charsets which were required to fix the build. It would be great if you can get in touch with your IBM colleagues to find out which are the default extended charsets supported by IBM J9 on AIX. I think we should try to use a similar set here in our OpenJDK port which is also used by the OpenJ9 for building OpenJ9 on AIX. Also, your IBM colleagues can help you to host a webrev which will make the further review of these changes much easier. I've but Tim from IBM on CC which should have an overview of all the IBM people involved in the OpenJDK project. Besides that I know at least the following people who might help you: Michihiro Horie: HORIE at jp.ibm.com Hiroshi H Horii: horii at jp.ibm.com Gustavo Romero: gromero at linux.vnet.ibm.com Matthew Brandyberyy: mbrandy at linux.vnet.ibm.com Thank you and best regards, Volker [1] https://urldefense.proofpoint.com/v2/url?u=https-3A__bugs.openjdk.java.net_browse_JDK-2D8201540&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=KUVGEwJiRVpNtQ9wUhGP6BKqzSTV1OWX31WWPdQMmqg&m=cieXGANE8bD3liEC2gJzl5hfZorR2qIfggL6U9t-Et8&s=hC4gcA6uomYgY26uR74VelobSIK1ReDsjRZGmI46UBY&e= [2] https://urldefense.proofpoint.com/v2/url?u=http-3A__mail.openjdk.java.net_pipermail_2d-2Ddev_2015-2DMay_005431.html&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=KUVGEwJiRVpNtQ9wUhGP6BKqzSTV1OWX31WWPdQMmqg&m=cieXGANE8bD3liEC2gJzl5hfZorR2qIfggL6U9t-Et8&s=qfYLaVVn7tapNDhv5bY6yE72nhngaWqte4wtRhQ3wB8&e= On Fri, Apr 13, 2018 at 2:42 PM, Bhaktavatsal R Maram wrote: > Hi Alan, > > Thank you for your response. I'm happy that my patch was attached. But, I don't see attachment. So, I inlined patch which contain diffs from 2 changesets in mail text. If a Jira bug is opened for this issue, probably I can attach complete and consolidated patch there. > > At high level, I'm adding following charsets to standard charset in java.base. For this, I have to change IBM943C and IBM942C from source to template to handle java package and aliases. It is also required to add codepage 932 as alias for IBM942C because both are one and same. > > Big5, Big5_HKSCS, GB18030, IBM942, IBM942C, IBM943, IBM943C, IBM950, IBM970, IBM1124, TIS_620 > > > These are default charsets for some of locales supported by Operating System (AIX). Since these are not available in standard charset, JDK can't be used in those locale even if they are available in jdk.charset module (java -version fails). > > I've followed some of the discussions around this in community and understand that default charset of a locale should be made available in java.charset module by using stdcs-* mechanism. On Linux, they were added to java.base in similar way. As it is missing for AIX, I've added them to enable JDK support for more locales. > > > Thanks, > Bhaktavatsal Reddy > > -----Alan Bateman wrote: ----- > To: Bhaktavatsal R Maram , core-libs-dev at openjdk.java.net > From: Alan Bateman > Date: 04/13/2018 03:52PM > Subject: Re: Missing many locales support on AIX platform > > On 13/04/2018 10:35, Bhaktavatsal R Maram wrote: >> Given that patch is big, I am sending patch as attachment again after changing some mail settings. Hopefully, it will make to community this time. > Your patch was attached. > > If I read it correctly, you've switched IBM943C to a template but there > aren't additional aliases so this part is effectively a no-op, is that > right? For IBM932C, you've moved it to be template and added several > aliases. > > The rest is AIX specific and I hope the SAP engineers that maintain the > AIX port can help you. It may be that you are testing with locales that > aren't supported configurations for the AIX port in OpenJDK. As a > general point, we try to keep as many of the exotic and multibyte > charsets out of java.base. They are of course still available to > applications via the API and the jdk.charsets service provider module. > > -Alan > > From gromero at linux.vnet.ibm.com Mon Apr 16 02:44:58 2018 From: gromero at linux.vnet.ibm.com (Gustavo Romero) Date: Sun, 15 Apr 2018 23:44:58 -0300 Subject: [RFC] Re: POWER9: Is there a way to improve the random number generation on PPC64? In-Reply-To: <75501eb1672a45389db40ef0760142c3@sap.com> References: <5A1887FF.1030406@linux.vnet.ibm.com> <8347ea90-1132-1dc0-b04c-cbcd081a68ef@linux.vnet.ibm.com> <75501eb1672a45389db40ef0760142c3@sap.com> Message-ID: <02141b62-2da0-d671-9bd7-6552b554731a@linux.vnet.ibm.com> Hi Martin, Thank you very much for your comments. On 04/03/2018 09:50 AM, Doerr, Martin wrote: > I think the Java and shared C++ code and should not use PPC64 specific names because it may get used for other platforms as well? I fixed all the names and changed the provider's name to HWTRNG (previously it was P9TRNG). I changed the names for the helpers to something more "neutral" and removed the snake_case from their names. Yes, it may get used for other platforms and can get used readily by other platforms by providing an intrinsic for the randomLong() method. I provided an unique value for serialVersionUID in the new JCA provider by using an increment of a preexisting one. Do you know if there is any other (formal) way to determine that value? > Some people don't want to trust relying solely on the hardware number generator which cannot get reviewed publicly. So would it make sense to use the instruction mixed with something else? Yes, I'm aware of the caveat... I the past Intel's 'rdrand' received a lot of criticism in that sense. I've talked to NX RNG designed and we've tried to find out a documentation about it on OpenPOWER foundation but it's not available yet. In any case, just like the use of 'rdrand' on OpenSSL that is disabled by default, wouldn't that be fine to use 'darn' on OpenJDK provided its use is optional and deliberated? Currently the user needs to (a) explicitly use the new provider by SecureRandom.getInstance("HWTRNG") and (b) unlock it using "-XX:+UnlockExperimentalVMOptions -XX:+UseRANDOMIntrinsics". In that sense, do you think it would be acceptable? > It would be good to have the complete change in one webrev for easier reviewing. Sure. Thanks for letting me know. Here is the new webrev: http://cr.openjdk.java.net/~gromero/POWER9/darn/v3/webrev/ Thanks a lot. Best regards, Gustavo > Thanks and best regards, > Martin > > > -----Original Message----- > From: Gustavo Romero [mailto:gromero at linux.vnet.ibm.com] > Sent: Montag, 2. April 2018 13:55 > To: Volker Simonis ; Doerr, Martin ; vladimir.kozlov at oracle.com > Cc: ppc-aix-port-dev at openjdk.java.net > Subject: [RFC] Re: POWER9: Is there a way to improve the random number generation on PPC64? > Importance: High > > Hi Martin, Volker, Vladimir > > Sorry for the huge delay replaying on this... > > I hope Martin (and Lutz) are feeling better and fully recovered. > > On 11/24/2017 08:04 PM, Volker Simonis wrote: >> Hi Gustavo, >> >> in one of my talks [1,2] I have an example on how to intrinsify >> Random.nextInt() in C2 by using the Intel 'rdrandl' instruction. But >> please notice that this is just a "toy" example - it is not production >> ready. In fact I think the right way would be to create a new >> SecureRandom provider where you may implement "engineNextBytes" by >> using the new Power instruction (maybe by calling a native function). >> This special implementation of "engineNextBytes" could then be >> intrinsified as described in the talk. >> >> Also, before you start this, please contact the security mailing list >> just to make sure you're not going into the wrong direction (I'm not a >> security expert :) > I've created a new JCA provider called 'P9TRNG' and implemented a darn > intrinsic for Interpreter, C1, and C2 compiler and did a couple of > tests using micro benches [1, 2] to check the latency and throughput > to get a random number using generateSeed() and nextBytes() with darn > in place. > > The 'P9TRNG' provider is basically a copy of 'NativePRNG' since it's > necessary a software fallback in case darn instruction fails to return > a valid random number after ten attempts (although it's very rare > condition). On the other hand 'P9TRNG' uses the darn intrinsic when > it's available. > > The maximum theoretical throughput on the machine I'm testing it (a > POWER9 witherspoon) is 128Mbps and there is one RNG per socket, so > only one RNG per CPU. With a simple C code it's possible to get very > close to that value (please see C code [3] for code details and > log [4] for the expected outputs). Unrolling the tight loop does not > help and causes a performance degradation. > > On Hotspot, for Interpreter and C1 the throughput is ~3x higher > than the version that does not use darn instruction (using micro > benches [1, 2]): > > gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java next_bytes SHA1PRNG 1024 10 > 3.8759432E7 ns > 2.113550 Mbps > gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java next_bytes SHA1PRNG 1024 100000 > 2.65902244E10 ns > 30.808313 Mbps > > gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java next_bytes NativePRNG 1024 100 > 7.1741008E7 ns > 11.418853 Mbps > gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java next_bytes NativePRNG 1024 100000 > 2.74547937E10 ns > 29.838140 Mbps > gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java -Xcomp -XX:TieredStopAtLevel=3 next_bytes NativePRNG 1024 100000 > 5.5632339E10 ns > 14.725248 Mbps > gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java -Xcomp -XX:-TieredCompilation next_bytes NativePRNG 1024 100000 > 2.78629519E10 ns > 29.401051 Mbps > > [With darn disabled: performance like NativePRNG] > gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java next_bytes P9TRNG 1024 100 > 7.0272888E7 ns > 11.657412 Mbps > gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java next_bytes P9TRNG 1024 100000 > 2.75566244E10 ns > 29.727880 Mbps > ... > > [With darn enabled] > gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseDARN next_bytes P9TRNG 1024 100 > 8305029.0 ns > 98.639030 Mbps > gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseDARN next_bytes P9TRNG 1024 100000 > 6.442112E9 ns > 127.163261 Mbps > gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseDARN -Xcomp -XX:TieredStopAtLevel=3 next_bytes P9TRNG 1024 100000 > 1.57303337E10 ns > 52.077728 Mbps > gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseDARN -Xcomp -XX:-TieredCompilation next_bytes P9TRNG 1024 100000 > 6.46914E9 ns > 126.631973 Mbps > > > For C2 compiler using darn is better until it reaches ~128Mbps > (maximum theoretical throughput), but on the other hand it never > blocks, so, for instance, generateSeed() which normally uses > /dev/random (blocking) is not affected by a lack of entropy in Linux > entropy pool. > > @Vladimir, Volker mentioned that you already experimented with rand on > Intel. Do you know if creating a new JCA provider as I did is a > reasonable approach to exploit darn on POWER9? Also, in my > implementation I had to create a VM intrinsic (_darn) in vmSymbols > that is, let's say, arch dependent, and that seems to be the only case > so far, but on the other hand a new JCA provider (with methods to be > intrinsified) is necessary (I don't see another way to intrinsify the > methods in NativePRNG/SHA1PRNG providers since I need a software > fallback to darn). Do you have any recommendation about it? > > The patchset rebased on top of > jdk11 (http://hg.openjdk.java.net/jdk/hs) is: > > http://cr.openjdk.java.net/~gromero/POWER9/darn/v1/0_PPC64_Add_JCA_provider_to_exploit_HW_RNG_on_POWER9.patch > http://cr.openjdk.java.net/~gromero/POWER9/darn/v1/1_PPC64_Assembler_add_support_for_darn_Deliver_A_Random_Number_instruction.patch > http://cr.openjdk.java.net/~gromero/POWER9/darn/v1/2_PPC64_Interpreter_add_template_to_exploit_darn_instruction.patch > http://cr.openjdk.java.net/~gromero/POWER9/darn/v1/3_PPC64_C2_Compiler_add_new_node_to_exploit_darn_instruction.patch > http://cr.openjdk.java.net/~gromero/POWER9/darn/v1/4_PPC64_C1_Compiler_add_intrinsic_to_exploit_darn_instruction.patch > > I intend to contribute that change as an experimental feature. > > Thank you. > > Best regards, > Gustavo > > [1] https://github.com/gromero/darn/blob/master/next_bytes.java > [2] https://github.com/gromero/darn/blob/master/generate_seed.java > [3] https://github.com/gromero/darn/blob/master/C/darn.c > [4] https://github.com/gromero/darn/blob/master/C/darn.log > From bhamaram at in.ibm.com Mon Apr 16 08:22:34 2018 From: bhamaram at in.ibm.com (Bhaktavatsal R Maram) Date: Mon, 16 Apr 2018 08:22:34 +0000 Subject: RFR(S): 8201540: [AIX] Extend the set of supported charsets in java.base In-Reply-To: References: , <59d3d31b-b596-0678-c0bf-69b8f951a40d@oracle.com> Message-ID: Hi All, Please review. Bug: https://bugs.openjdk.java.net/browse/JDK-8201540 webrev: http://cr.openjdk.java.net/~gromero/8201540/v1/webrev/ In this patch, 1. Default charsets Big5, Big5_Solaris, Big5_HKSCS, GB18030, IBM856, IBM921, IBM922, IBM942, IBM942C, IBM943, IBM943C, IBM950, IBM970, IBM1046, IBM1124, IBM1383, ISO_8859_6, ISO_8859_8, MS1252, TIS_620 for different locales supported on AIX are added to standard charsets in java.base module 2. More aliases are added to existing charsets. 3. Source files for IBM-942C and IBM-943C are changed to template to support #1 4. Modified file make/jdk/src/classes/build/tools/charsetmapping/SPI.java to increase the Hashtable capacity that holds aliases of standard charsets. As the no.of charsets we include in standard charsets are more in this patch, existing capacity of Hashtable used to hold aliases is not efficient. Without change to this file, JDK won't get built and following error is seen Exception in thread "main" java.lang.RuntimeException: Cannot find a suitable size within given constraints at build.tools.charsetmapping.Hasher.build(Hasher.java:122) at build.tools.charsetmapping.Hasher.genClass(Hasher.java:261) at build.tools.charsetmapping.SPI.genClass(SPI.java:130) at build.tools.charsetmapping.Main.main(Main.java:99) Please note that webrev/index.html is not showing below 2 files I deleted. However, webrev/jdk11.changeset file show them deleted properly. src/jdk.charsets/share/classes/sun/nio/cs/ext/IBM942C.java src/jdk.charsets/share/classes/sun/nio/cs/ext/IBM943C.java I've built the JDK with this patch on both Linux and Aix. Thanks, Bhaktavatsal Reddy -----Volker Simonis wrote: ----- To: Bhaktavatsal R Maram From: Volker Simonis Date: 04/13/2018 08:51PM Cc: Alan Bateman , Java Core Libs , Tim Ellison , ppc-aix-port-dev at openjdk.java.net Subject: Re: Missing many locales support on AIX platform Hi Bhaktavatsal Reddy, thanks for addressing this long standing issue. I've opened "8201540: [AIX] Extend the set of supported charsets in java.base" [1] to track this issue. As I wrote in the bug report, this problem is the consequence of an emergency fix (JDK-8081332) I did back in 2015 to fix the build on AIX after the integration of the modularity support change (see discussion: [2]). At that time I only added the minimal set of charsets which were required to fix the build. It would be great if you can get in touch with your IBM colleagues to find out which are the default extended charsets supported by IBM J9 on AIX. I think we should try to use a similar set here in our OpenJDK port which is also used by the OpenJ9 for building OpenJ9 on AIX. Also, your IBM colleagues can help you to host a webrev which will make the further review of these changes much easier. I've but Tim from IBM on CC which should have an overview of all the IBM people involved in the OpenJDK project. Besides that I know at least the following people who might help you: Michihiro Horie: HORIE at jp.ibm.com Hiroshi H Horii: horii at jp.ibm.com Gustavo Romero: gromero at linux.vnet.ibm.com Matthew Brandyberyy: mbrandy at linux.vnet.ibm.com Thank you and best regards, Volker [1] https://urldefense.proofpoint.com/v2/url?u=https-3A__bugs.openjdk.java.net_browse_JDK-2D8201540&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=KUVGEwJiRVpNtQ9wUhGP6BKqzSTV1OWX31WWPdQMmqg&m=cieXGANE8bD3liEC2gJzl5hfZorR2qIfggL6U9t-Et8&s=hC4gcA6uomYgY26uR74VelobSIK1ReDsjRZGmI46UBY&e= [2] https://urldefense.proofpoint.com/v2/url?u=http-3A__mail.openjdk.java.net_pipermail_2d-2Ddev_2015-2DMay_005431.html&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=KUVGEwJiRVpNtQ9wUhGP6BKqzSTV1OWX31WWPdQMmqg&m=cieXGANE8bD3liEC2gJzl5hfZorR2qIfggL6U9t-Et8&s=qfYLaVVn7tapNDhv5bY6yE72nhngaWqte4wtRhQ3wB8&e= On Fri, Apr 13, 2018 at 2:42 PM, Bhaktavatsal R Maram wrote: > Hi Alan, > > Thank you for your response. I'm happy that my patch was attached. But, I don't see attachment. So, I inlined patch which contain diffs from 2 changesets in mail text. If a Jira bug is opened for this issue, probably I can attach complete and consolidated patch there. > > At high level, I'm adding following charsets to standard charset in java.base. For this, I have to change IBM943C and IBM942C from source to template to handle java package and aliases. It is also required to add codepage 932 as alias for IBM942C because both are one and same. > > Big5, Big5_HKSCS, GB18030, IBM942, IBM942C, IBM943, IBM943C, IBM950, IBM970, IBM1124, TIS_620 > > > These are default charsets for some of locales supported by Operating System (AIX). Since these are not available in standard charset, JDK can't be used in those locale even if they are available in jdk.charset module (java -version fails). > > I've followed some of the discussions around this in community and understand that default charset of a locale should be made available in java.charset module by using stdcs-* mechanism. On Linux, they were added to java.base in similar way. As it is missing for AIX, I've added them to enable JDK support for more locales. > > > Thanks, > Bhaktavatsal Reddy > > -----Alan Bateman wrote: ----- > To: Bhaktavatsal R Maram , core-libs-dev at openjdk.java.net > From: Alan Bateman > Date: 04/13/2018 03:52PM > Subject: Re: Missing many locales support on AIX platform > > On 13/04/2018 10:35, Bhaktavatsal R Maram wrote: >> Given that patch is big, I am sending patch as attachment again after changing some mail settings. Hopefully, it will make to community this time. > Your patch was attached. > > If I read it correctly, you've switched IBM943C to a template but there > aren't additional aliases so this part is effectively a no-op, is that > right? For IBM932C, you've moved it to be template and added several > aliases. > > The rest is AIX specific and I hope the SAP engineers that maintain the > AIX port can help you. It may be that you are testing with locales that > aren't supported configurations for the AIX port in OpenJDK. As a > general point, we try to keep as many of the exotic and multibyte > charsets out of java.base. They are of course still available to > applications via the API and the jdk.charsets service provider module. > > -Alan > > From Alan.Bateman at oracle.com Mon Apr 16 08:46:39 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 16 Apr 2018 09:46:39 +0100 Subject: RFR(S): 8201540: [AIX] Extend the set of supported charsets in java.base In-Reply-To: References: <59d3d31b-b596-0678-c0bf-69b8f951a40d@oracle.com> Message-ID: <79b9aaac-aa33-1fc8-93a2-0b6b5bf7ea8d@oracle.com> On 16/04/2018 09:22, Bhaktavatsal R Maram wrote: > > 3. Source files for IBM-942C and IBM-943C are changed to template to support #1 > You might want to double check the webrev as it looks like you've added templates where as I assume you mean to use "hg rename" to rename IBM942C.java and IBM943C.java. -Alan From bhamaram at in.ibm.com Mon Apr 16 09:07:17 2018 From: bhamaram at in.ibm.com (Bhaktavatsal R Maram) Date: Mon, 16 Apr 2018 09:07:17 +0000 Subject: RFR(S): 8201540: [AIX] Extend the set of supported charsets in java.base In-Reply-To: <79b9aaac-aa33-1fc8-93a2-0b6b5bf7ea8d@oracle.com> References: <79b9aaac-aa33-1fc8-93a2-0b6b5bf7ea8d@oracle.com>, <59d3d31b-b596-0678-c0bf-69b8f951a40d@oracle.com> Message-ID: Hi Alan, I deleted IBM943C.java (using hg remove) and added new file IBM943C.java.template (using hg add). I now understand that using "hg rename" is giving more meaningful representation in webrev/index.html. I will re-generate webrev by renaming source files to templates using "hg rename" Thanks, Bhaktavatsal Reddy -----Alan Bateman wrote: ----- To: Bhaktavatsal R Maram , Volker Simonis From: Alan Bateman Date: 04/16/2018 02:16PM Cc: Java Core Libs , Tim Ellison , ppc-aix-port-dev at openjdk.java.net Subject: Re: RFR(S): 8201540: [AIX] Extend the set of supported charsets in java.base On 16/04/2018 09:22, Bhaktavatsal R Maram wrote: > > 3. Source files for IBM-942C and IBM-943C are changed to template to support #1 > You might want to double check the webrev as it looks like you've added templates where as I assume you mean to use "hg rename" to rename IBM942C.java and IBM943C.java. -Alan From bhamaram at in.ibm.com Mon Apr 16 11:10:24 2018 From: bhamaram at in.ibm.com (Bhaktavatsal R Maram) Date: Mon, 16 Apr 2018 11:10:24 +0000 Subject: RFR(S): 8201540: [AIX] Extend the set of supported charsets in java.base In-Reply-To: References: , <79b9aaac-aa33-1fc8-93a2-0b6b5bf7ea8d@oracle.com>, <59d3d31b-b596-0678-c0bf-69b8f951a40d@oracle.com> Message-ID: Hi All, I've regenerated webrev using "hg rename" to create template files. webrev looks much neat now.. Thanks Alan for suggestion. webrev - http://cr.openjdk.java.net/~gromero/8201540/v2/ Thanks, Bhaktavatsal Reddy -----"core-libs-dev" wrote: ----- To: Alan Bateman From: "Bhaktavatsal R Maram" Sent by: "core-libs-dev" Date: 04/16/2018 02:38PM Cc: Tim Ellison , ppc-aix-port-dev at openjdk.java.net, Java Core Libs Subject: Re: RFR(S): 8201540: [AIX] Extend the set of supported charsets in java.base Hi Alan, I deleted IBM943C.java (using hg remove) and added new file IBM943C.java.template (using hg add). I now understand that using "hg rename" is giving more meaningful representation in webrev/index.html. I will re-generate webrev by renaming source files to templates using "hg rename" Thanks, Bhaktavatsal Reddy -----Alan Bateman wrote: ----- To: Bhaktavatsal R Maram , Volker Simonis From: Alan Bateman Date: 04/16/2018 02:16PM Cc: Java Core Libs , Tim Ellison , ppc-aix-port-dev at openjdk.java.net Subject: Re: RFR(S): 8201540: [AIX] Extend the set of supported charsets in java.base On 16/04/2018 09:22, Bhaktavatsal R Maram wrote: > > 3. Source files for IBM-942C and IBM-943C are changed to template to support #1 > You might want to double check the webrev as it looks like you've added templates where as I assume you mean to use "hg rename" to rename IBM942C.java and IBM943C.java. -Alan From martin.doerr at sap.com Mon Apr 16 11:10:37 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Mon, 16 Apr 2018 11:10:37 +0000 Subject: [RFC] Re: POWER9: Is there a way to improve the random number generation on PPC64? In-Reply-To: <02141b62-2da0-d671-9bd7-6552b554731a@linux.vnet.ibm.com> References: <5A1887FF.1030406@linux.vnet.ibm.com> <8347ea90-1132-1dc0-b04c-cbcd081a68ef@linux.vnet.ibm.com> <75501eb1672a45389db40ef0760142c3@sap.com> <02141b62-2da0-d671-9bd7-6552b554731a@linux.vnet.ibm.com> Message-ID: <6a3b2f85cf4f452595414e657df877c4@sap.com> Hi Gustavo, thanks for providing the webrev. Please note that it needs to get reviewed on the official mailing lists hotspot-compiler-dev and security-dev (you should subscribe before posting). ppc.ad: I think the pipe classes are not implemented in a very useful way on PPC64 at the moment. Adding a new one just for this doesn't make any sense in my opinion. If you would like to improve the OptoScheduling, I suggest to do this separately. Will probably take quite some effort. templateInterpreterGenerator_ppc.cpp: I think the generator should return NULL for Power8 and below. templateInterpreterGenerator: Please move the generation below the basic entries. HWTRNG.java: The closing braces look weird at the end. A few declarations like generate_HWTRNG_randomLong_entry() or LIRGenerator::do_Random are in shared code, but only defined in PPC64 code. But it may make sense to fix that after you got a few reviews. > Do you know if there is any other (formal) way to determine that value? You can use the tool "serialver" from the jdk/bin. I think Eclipse can also generate it if you have a project for it. I'm ok with using darn directly in the initial version as long as it's not used by default. If it is supposed to get used by default, I think we should add something similar to linux' dev/random. I haven't checked how it's implemented on PPC64. Best regards, Martin -----Original Message----- From: Gustavo Romero [mailto:gromero at linux.vnet.ibm.com] Sent: Montag, 16. April 2018 04:45 To: Doerr, Martin ; Volker Simonis ; vladimir.kozlov at oracle.com Cc: ppc-aix-port-dev at openjdk.java.net Subject: Re: [RFC] Re: POWER9: Is there a way to improve the random number generation on PPC64? Hi Martin, Thank you very much for your comments. On 04/03/2018 09:50 AM, Doerr, Martin wrote: > I think the Java and shared C++ code and should not use PPC64 specific names because it may get used for other platforms as well? I fixed all the names and changed the provider's name to HWTRNG (previously it was P9TRNG). I changed the names for the helpers to something more "neutral" and removed the snake_case from their names. Yes, it may get used for other platforms and can get used readily by other platforms by providing an intrinsic for the randomLong() method. Do you know if there is any other (formal) way to determine that value? > Some people don't want to trust relying solely on the hardware number generator which cannot get reviewed publicly. So would it make sense to use the instruction mixed with something else? Yes, I'm aware of the caveat... I the past Intel's 'rdrand' received a lot of criticism in that sense. I've talked to NX RNG designed and we've tried to find out a documentation about it on OpenPOWER foundation but it's not available yet. In any case, just like the use of 'rdrand' on OpenSSL that is disabled by default, wouldn't that be fine to use 'darn' on OpenJDK provided its use is optional and deliberated? Currently the user needs to (a) explicitly use the new provider by SecureRandom.getInstance("HWTRNG") and (b) unlock it using "-XX:+UnlockExperimentalVMOptions -XX:+UseRANDOMIntrinsics". In that sense, do you think it would be acceptable? > It would be good to have the complete change in one webrev for easier reviewing. Sure. Thanks for letting me know. Here is the new webrev: http://cr.openjdk.java.net/~gromero/POWER9/darn/v3/webrev/ Thanks a lot. Best regards, Gustavo > Thanks and best regards, > Martin > > > -----Original Message----- > From: Gustavo Romero [mailto:gromero at linux.vnet.ibm.com] > Sent: Montag, 2. April 2018 13:55 > To: Volker Simonis ; Doerr, Martin ; vladimir.kozlov at oracle.com > Cc: ppc-aix-port-dev at openjdk.java.net > Subject: [RFC] Re: POWER9: Is there a way to improve the random number generation on PPC64? > Importance: High > > Hi Martin, Volker, Vladimir > > Sorry for the huge delay replaying on this... > > I hope Martin (and Lutz) are feeling better and fully recovered. > > On 11/24/2017 08:04 PM, Volker Simonis wrote: >> Hi Gustavo, >> >> in one of my talks [1,2] I have an example on how to intrinsify >> Random.nextInt() in C2 by using the Intel 'rdrandl' instruction. But >> please notice that this is just a "toy" example - it is not production >> ready. In fact I think the right way would be to create a new >> SecureRandom provider where you may implement "engineNextBytes" by >> using the new Power instruction (maybe by calling a native function). >> This special implementation of "engineNextBytes" could then be >> intrinsified as described in the talk. >> >> Also, before you start this, please contact the security mailing list >> just to make sure you're not going into the wrong direction (I'm not a >> security expert :) > I've created a new JCA provider called 'P9TRNG' and implemented a darn > intrinsic for Interpreter, C1, and C2 compiler and did a couple of > tests using micro benches [1, 2] to check the latency and throughput > to get a random number using generateSeed() and nextBytes() with darn > in place. > > The 'P9TRNG' provider is basically a copy of 'NativePRNG' since it's > necessary a software fallback in case darn instruction fails to return > a valid random number after ten attempts (although it's very rare > condition). On the other hand 'P9TRNG' uses the darn intrinsic when > it's available. > > The maximum theoretical throughput on the machine I'm testing it (a > POWER9 witherspoon) is 128Mbps and there is one RNG per socket, so > only one RNG per CPU. With a simple C code it's possible to get very > close to that value (please see C code [3] for code details and > log [4] for the expected outputs). Unrolling the tight loop does not > help and causes a performance degradation. > > On Hotspot, for Interpreter and C1 the throughput is ~3x higher > than the version that does not use darn instruction (using micro > benches [1, 2]): > > gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java next_bytes SHA1PRNG 1024 10 > 3.8759432E7 ns > 2.113550 Mbps > gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java next_bytes SHA1PRNG 1024 100000 > 2.65902244E10 ns > 30.808313 Mbps > > gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java next_bytes NativePRNG 1024 100 > 7.1741008E7 ns > 11.418853 Mbps > gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java next_bytes NativePRNG 1024 100000 > 2.74547937E10 ns > 29.838140 Mbps > gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java -Xcomp -XX:TieredStopAtLevel=3 next_bytes NativePRNG 1024 100000 > 5.5632339E10 ns > 14.725248 Mbps > gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java -Xcomp -XX:-TieredCompilation next_bytes NativePRNG 1024 100000 > 2.78629519E10 ns > 29.401051 Mbps > > [With darn disabled: performance like NativePRNG] > gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java next_bytes P9TRNG 1024 100 > 7.0272888E7 ns > 11.657412 Mbps > gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java next_bytes P9TRNG 1024 100000 > 2.75566244E10 ns > 29.727880 Mbps > ... > > [With darn enabled] > gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseDARN next_bytes P9TRNG 1024 100 > 8305029.0 ns > 98.639030 Mbps > gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseDARN next_bytes P9TRNG 1024 100000 > 6.442112E9 ns > 127.163261 Mbps > gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseDARN -Xcomp -XX:TieredStopAtLevel=3 next_bytes P9TRNG 1024 100000 > 1.57303337E10 ns > 52.077728 Mbps > gromero at gromero1:~/git/darn$ /tmp/jdk11/jvm/openjdk-11-internal/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseDARN -Xcomp -XX:-TieredCompilation next_bytes P9TRNG 1024 100000 > 6.46914E9 ns > 126.631973 Mbps > > > For C2 compiler using darn is better until it reaches ~128Mbps > (maximum theoretical throughput), but on the other hand it never > blocks, so, for instance, generateSeed() which normally uses > /dev/random (blocking) is not affected by a lack of entropy in Linux > entropy pool. > > @Vladimir, Volker mentioned that you already experimented with rand on > Intel. Do you know if creating a new JCA provider as I did is a > reasonable approach to exploit darn on POWER9? Also, in my > implementation I had to create a VM intrinsic (_darn) in vmSymbols > that is, let's say, arch dependent, and that seems to be the only case > so far, but on the other hand a new JCA provider (with methods to be > intrinsified) is necessary (I don't see another way to intrinsify the > methods in NativePRNG/SHA1PRNG providers since I need a software > fallback to darn). Do you have any recommendation about it? > > The patchset rebased on top of > jdk11 (http://hg.openjdk.java.net/jdk/hs) is: > > http://cr.openjdk.java.net/~gromero/POWER9/darn/v1/0_PPC64_Add_JCA_provider_to_exploit_HW_RNG_on_POWER9.patch > http://cr.openjdk.java.net/~gromero/POWER9/darn/v1/1_PPC64_Assembler_add_support_for_darn_Deliver_A_Random_Number_instruction.patch > http://cr.openjdk.java.net/~gromero/POWER9/darn/v1/2_PPC64_Interpreter_add_template_to_exploit_darn_instruction.patch > http://cr.openjdk.java.net/~gromero/POWER9/darn/v1/3_PPC64_C2_Compiler_add_new_node_to_exploit_darn_instruction.patch > http://cr.openjdk.java.net/~gromero/POWER9/darn/v1/4_PPC64_C1_Compiler_add_intrinsic_to_exploit_darn_instruction.patch > > I intend to contribute that change as an experimental feature. > > Thank you. > > Best regards, > Gustavo > > [1] https://github.com/gromero/darn/blob/master/next_bytes.java > [2] https://github.com/gromero/darn/blob/master/generate_seed.java > [3] https://github.com/gromero/darn/blob/master/C/darn.c > [4] https://github.com/gromero/darn/blob/master/C/darn.log > From takiguc at linux.vnet.ibm.com Mon Apr 16 17:41:51 2018 From: takiguc at linux.vnet.ibm.com (Ichiroh Takiguchi) Date: Tue, 17 Apr 2018 02:41:51 +0900 Subject: Proposal:AIX's IME support In-Reply-To: <428f37b3-0a64-b727-2426-9c96fd8155a9@oracle.com> References: <2ed52f9dc4e9487c7b3131b5af3666d9@linux.vnet.ibm.com> <428f37b3-0a64-b727-2426-9c96fd8155a9@oracle.com> Message-ID: <8062d9b71e62b0fc92ae505be9e08e85@linux.vnet.ibm.com> Hello. Sorry I'm late. Christoph Langer helped me to put webrev file into cr.openjdk.java.net. See, http://cr.openjdk.java.net/~clanger/webrevs/8201429.0/ Actually, he found *.patch files had unexpected entries, I fixed the issue. It seemed that webrev tool could not handle "hg copy" action. It was reported by, http://mail.openjdk.java.net/pipermail/webrev-dev/2018-April/000145.html So above webrev file should be fine. Please let me know if you have any question and suggestion about webrev file for AIX's IME support. Thanks, Ichiroh Takiguchi On 2018-04-10 07:39, Sergey Bylokhov wrote: > Hi, Ichiroh. > Your contribution is welcome. Can you send the change for review as a > webrev on cr.openjdk.java.net? > > On 09/04/2018 02:09, Ichiroh Takiguchi wrote: >> Hello, >> IBM would like to propose another Input Method Framework (IMF) >> implementation against IBM AIX platform. >> Same IMF implementation was used by IBM Java8 for AIX. >> IBM would like to contribute this IMF implementation to OpenJDK >> project. >> >> AIX's Input Method Editor (IME) working behavior is not same as >> Linux's and Solaris' one. >> On OpenJDK JDK9 for AIX, users cannot input any East Asian character >> with their locale by this IME. >> We think this situation is critical. >> Put modified files under src/java.desktop/aix/* directories in order >> not to affect the other unix platform. >> >> I'd like contribute following 4 files: >> M make/lib/Awt2dLibraries.gmk >> A src/java.desktop/aix/classes/sun/awt/X11InputMethod.java >> A src/java.desktop/aix/native/libawt_xawt/awt/awt_InputMethod.c >> A src/java.desktop/aix/native/libawt_xawt/xawt/XlibWrapper.c >> >> We need to apply too many changes including JNI, so we copied >> following 3 files from unix directory to aix directory: >> * src/java.desktop/unix/classes/sun/awt/X11InputMethod.java >> * src/java.desktop/unix/native/libawt_xawt/awt/awt_InputMethod.c >> * src/java.desktop/unix/native/libawt_xawt/xawt/XlibWrapper.c >> Then, we modified them. >> >> I'd appreciate any feedback please, and how I would go about obtaining >> a sponsor and contributor? >> >> Thanks, >> Ichiroh Takiguchi >> IBM Japan, Ltd. >> From volker.simonis at gmail.com Tue Apr 17 14:52:57 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 17 Apr 2018 16:52:57 +0200 Subject: RFR(S): 8201540: [AIX] Extend the set of supported charsets in java.base In-Reply-To: References: <79b9aaac-aa33-1fc8-93a2-0b6b5bf7ea8d@oracle.com> <59d3d31b-b596-0678-c0bf-69b8f951a40d@oracle.com> Message-ID: Hi Bhaktavatsal Reddy, you change looks good, although I can't really verify all the charset aliases. For example Wikipedia mentions that "ibm-932" is equivalent to "ibm-942" [1] but you made it an alias for "ibm-942C". What's actually the difference between "ibm-942C" and "ibm-942"? I can sponsor your change although I would appreciate if somebody else from IBM could have another look at your change. I tried to compare with "IBM Java 9" but it doesn't seem to exist. They only refer to AdoptOpenJDK and AdoptOpenJDK just uses a vanilla version of OpenJDK. Finally, I hope you won't mind if I update the copyright years on the files you changed before pushing (this is a convention in the OpenJDK project). Best regards, Volker [1] https://en.wikipedia.org/wiki/Code_page_932_(IBM) On Mon, Apr 16, 2018 at 1:10 PM, Bhaktavatsal R Maram wrote: > Hi All, > > I've regenerated webrev using "hg rename" to create template files. webrev looks much neat now.. Thanks Alan for suggestion. > > webrev - http://cr.openjdk.java.net/~gromero/8201540/v2/ > > Thanks, > Bhaktavatsal Reddy > > > -----"core-libs-dev" wrote: ----- > To: Alan Bateman > From: "Bhaktavatsal R Maram" > Sent by: "core-libs-dev" > Date: 04/16/2018 02:38PM > Cc: Tim Ellison , ppc-aix-port-dev at openjdk.java.net, Java Core Libs > Subject: Re: RFR(S): 8201540: [AIX] Extend the set of supported charsets in java.base > > Hi Alan, > > I deleted IBM943C.java (using hg remove) and added new file IBM943C.java.template (using hg add). I now understand that using "hg rename" is giving more meaningful representation in webrev/index.html. > > I will re-generate webrev by renaming source files to templates using "hg rename" > > Thanks, > Bhaktavatsal Reddy > > > > -----Alan Bateman wrote: ----- > To: Bhaktavatsal R Maram , Volker Simonis > From: Alan Bateman > Date: 04/16/2018 02:16PM > Cc: Java Core Libs , Tim Ellison , ppc-aix-port-dev at openjdk.java.net > Subject: Re: RFR(S): 8201540: [AIX] Extend the set of supported charsets in java.base > > > On 16/04/2018 09:22, Bhaktavatsal R Maram wrote: >> >> 3. Source files for IBM-942C and IBM-943C are changed to template to support #1 >> > You might want to double check the webrev as it looks like you've added > templates where as I assume you mean to use "hg rename" to rename > IBM942C.java and IBM943C.java. > > -Alan > > > From bhamaram at in.ibm.com Tue Apr 17 18:09:51 2018 From: bhamaram at in.ibm.com (Bhaktavatsal R Maram) Date: Tue, 17 Apr 2018 18:09:51 +0000 Subject: RFR(S): 8201540: [AIX] Extend the set of supported charsets in java.base In-Reply-To: References: , <79b9aaac-aa33-1fc8-93a2-0b6b5bf7ea8d@oracle.com> <59d3d31b-b596-0678-c0bf-69b8f951a40d@oracle.com> Message-ID: Hi Volker, Thank you for reviewing the patch. > you change looks good, although I can't really verify all the charset > aliases. For example Wikipedia mentions that "ibm-932" is equivalent > to "ibm-942" [1] but you made it an alias for "ibm-942C". What's > actually the difference between "ibm-942C" and "ibm-942"? IBM-942C is a customized version of IBM-942, in which following characters are replaced with ASCII thus making first 96 character mappings same as ASCII. 0x1A is mapped to 0x1C (in IBM-942) and to 0x1A (in IBM-942C) 0x1C is mapped to 0x7F (in IBM-942) and to 0x1C (in IBM-942C) 0x5C is mapped to 0xA5 (in IBM-942) and to 0x5C (in IBM-942C) 0x7E is mapped to 0x203E (in IBM-942) and to 0x7E (in IBM-942C) 0x7F is mapped to 0x1A (in IBM-942) and to 0x7F (in IBM-942C) Similarly, IBM-943C is a customization for IBM-943 in which character mappings for Yen(?) and overline(?) are replaced by their ASCII equivalents backslash (\) and tilde (~). So, we should be mapping OS code-page IBM-943 to code-page IBM-943C in Java. I am working on fixing these inconsistencies in another defect in-order not to confuse things (I hope it is alright). Current patch mainly address moving default codepage from extended codepage list to standard codepage list. Also, There are few codepages which are missing in OpenJDK. > I can sponsor your change although I would appreciate if somebody else > from IBM could have another look at your change. I tried to compare > with "IBM Java 9" but it doesn't seem to exist. They only refer to > AdoptOpenJDK and AdoptOpenJDK just uses a vanilla version of OpenJDK. Right! OpenJ9 version of JDK9 in AdoptOpenJDK is vanilla version of OpenJDK with OpenJ9. I've picked aliases for this patch from IBM JDK 8. > Finally, I hope you won't mind if I update the copyright years on the > files you changed before pushing (this is a convention in the OpenJDK > project). Sorry, I forgot to take care of copyright. Please change it this time before pushing. I will take care of it henceforth. Thanks, Bhaktavatsal Reddy -----Volker Simonis wrote: ----- To: Bhaktavatsal R Maram From: Volker Simonis Date: 04/17/2018 08:30PM Cc: Alan Bateman , Tim Ellison , ppc-aix-port-dev at openjdk.java.net, Java Core Libs Subject: Re: RFR(S): 8201540: [AIX] Extend the set of supported charsets in java.base Hi Bhaktavatsal Reddy, you change looks good, although I can't really verify all the charset aliases. For example Wikipedia mentions that "ibm-932" is equivalent to "ibm-942" [1] but you made it an alias for "ibm-942C". What's actually the difference between "ibm-942C" and "ibm-942"? I can sponsor your change although I would appreciate if somebody else from IBM could have another look at your change. I tried to compare with "IBM Java 9" but it doesn't seem to exist. They only refer to AdoptOpenJDK and AdoptOpenJDK just uses a vanilla version of OpenJDK. Finally, I hope you won't mind if I update the copyright years on the files you changed before pushing (this is a convention in the OpenJDK project). Best regards, Volker [1] https://urldefense.proofpoint.com/v2/url?u=https-3A__en.wikipedia.org_wiki_Code-5Fpage-5F932-5F-28IBM-29&d=DwIBaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=KUVGEwJiRVpNtQ9wUhGP6BKqzSTV1OWX31WWPdQMmqg&m=DencrOI40Trgt_TxNW4dYVWqYtpT7dPnHzaSOEsw_ZQ&s=xYfspcI7N7ZAbVMqyjM7YIb_kd-RsFPn6pINIFz_Oa4&e= On Mon, Apr 16, 2018 at 1:10 PM, Bhaktavatsal R Maram wrote: > Hi All, > > I've regenerated webrev using "hg rename" to create template files. webrev looks much neat now.. Thanks Alan for suggestion. > > webrev - https://urldefense.proofpoint.com/v2/url?u=http-3A__cr.openjdk.java.net_-7Egromero_8201540_v2_&d=DwIBaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=KUVGEwJiRVpNtQ9wUhGP6BKqzSTV1OWX31WWPdQMmqg&m=DencrOI40Trgt_TxNW4dYVWqYtpT7dPnHzaSOEsw_ZQ&s=mDikak1wXAwU-a0yd6dJml9X5N1DJg-GkQmgPl4v_5g&e= > > Thanks, > Bhaktavatsal Reddy > > > -----"core-libs-dev" wrote: ----- > To: Alan Bateman > From: "Bhaktavatsal R Maram" > Sent by: "core-libs-dev" > Date: 04/16/2018 02:38PM > Cc: Tim Ellison , ppc-aix-port-dev at openjdk.java.net, Java Core Libs > Subject: Re: RFR(S): 8201540: [AIX] Extend the set of supported charsets in java.base > > Hi Alan, > > I deleted IBM943C.java (using hg remove) and added new file IBM943C.java.template (using hg add). I now understand that using "hg rename" is giving more meaningful representation in webrev/index.html. > > I will re-generate webrev by renaming source files to templates using "hg rename" > > Thanks, > Bhaktavatsal Reddy > > > > -----Alan Bateman wrote: ----- > To: Bhaktavatsal R Maram , Volker Simonis > From: Alan Bateman > Date: 04/16/2018 02:16PM > Cc: Java Core Libs , Tim Ellison , ppc-aix-port-dev at openjdk.java.net > Subject: Re: RFR(S): 8201540: [AIX] Extend the set of supported charsets in java.base > > > On 16/04/2018 09:22, Bhaktavatsal R Maram wrote: >> >> 3. Source files for IBM-942C and IBM-943C are changed to template to support #1 >> > You might want to double check the webrev as it looks like you've added > templates where as I assume you mean to use "hg rename" to rename > IBM942C.java and IBM943C.java. > > -Alan > > > From takiguc at linux.vnet.ibm.com Wed Apr 18 05:01:05 2018 From: takiguc at linux.vnet.ibm.com (Ichiroh Takiguchi) Date: Wed, 18 Apr 2018 14:01:05 +0900 Subject: RFR(S): 8201540: [AIX] Extend the set of supported charsets in java.base In-Reply-To: References: <79b9aaac-aa33-1fc8-93a2-0b6b5bf7ea8d@oracle.com> <59d3d31b-b596-0678-c0bf-69b8f951a40d@oracle.com> Message-ID: <5909b598a1162cf353e928ef9b3ddff8@linux.vnet.ibm.com> Hello Volker. >> What's actually the difference between "ibm-942C" and "ibm-942"? They have two differences on single byte part: 1. Control Character rotation for 0x1A and 0x1C, 0x7F [2] 2. Character replacement for 0x5C and 0x7E (0xFE, 0xFF) For IBM-942 [3], 0x1A<=>U+001C,0x1C<=>U+007F,0x5C<=>U+00A5,0x7E<=>U+203E,0x7F<=>U+001A 0xFE<=>U+005C,0xFF<=>U+007E For IBM-942C, 0x1A<=>U+001A,0x1C<=>U+001C,0x5C<=>U+005C,0x7E<=>U+007E,0x7F<=>U+007F, 0x5C<=U+00A5,0x7E<=U+203E 0xFE=>U+005C,0xFF=>U+007E (It's ASCII compatible) IBM-942's single byte part is IBM-1041 [4]. IBM-932's single byte part is IBM-897 [5]. IBM-1041 is not same as IBM-897. 5 characters were added into IBM-1041 [4]. (0x80,0xA0,0xFD,0xFE,0xFF [2]) [2] https://www-01.ibm.com/software/globalization/cdra/appendix_g.html [3] https://en.wikipedia.org/wiki/Code_page_942 [4] http://www-01.ibm.com/software/globalization/cp/cp01041.html [5] http://www-01.ibm.com/software/globalization/cp/cp00897.html On 2018-04-17 23:52, Volker Simonis wrote: > Hi Bhaktavatsal Reddy, > > you change looks good, although I can't really verify all the charset > aliases. For example Wikipedia mentions that "ibm-932" is equivalent > to "ibm-942" [1] but you made it an alias for "ibm-942C". What's > actually the difference between "ibm-942C" and "ibm-942"? > > I can sponsor your change although I would appreciate if somebody else > from IBM could have another look at your change. I tried to compare > with "IBM Java 9" but it doesn't seem to exist. They only refer to > AdoptOpenJDK and AdoptOpenJDK just uses a vanilla version of OpenJDK. > > Finally, I hope you won't mind if I update the copyright years on the > files you changed before pushing (this is a convention in the OpenJDK > project). > > Best regards, > Volker > > [1] https://en.wikipedia.org/wiki/Code_page_932_(IBM) > > On Mon, Apr 16, 2018 at 1:10 PM, Bhaktavatsal R Maram > wrote: >> Hi All, >> >> I've regenerated webrev using "hg rename" to create template files. >> webrev looks much neat now.. Thanks Alan for suggestion. >> >> webrev - http://cr.openjdk.java.net/~gromero/8201540/v2/ >> >> Thanks, >> Bhaktavatsal Reddy >> >> >> -----"core-libs-dev" wrote: >> ----- >> To: Alan Bateman >> From: "Bhaktavatsal R Maram" >> Sent by: "core-libs-dev" >> Date: 04/16/2018 02:38PM >> Cc: Tim Ellison , >> ppc-aix-port-dev at openjdk.java.net, Java Core Libs >> >> Subject: Re: RFR(S): 8201540: [AIX] Extend the set of supported >> charsets in java.base >> >> Hi Alan, >> >> I deleted IBM943C.java (using hg remove) and added new file >> IBM943C.java.template (using hg add). I now understand that using "hg >> rename" is giving more meaningful representation in webrev/index.html. >> >> I will re-generate webrev by renaming source files to templates using >> "hg rename" >> >> Thanks, >> Bhaktavatsal Reddy >> >> >> >> -----Alan Bateman wrote: ----- >> To: Bhaktavatsal R Maram , Volker Simonis >> >> From: Alan Bateman >> Date: 04/16/2018 02:16PM >> Cc: Java Core Libs , Tim Ellison >> , ppc-aix-port-dev at openjdk.java.net >> Subject: Re: RFR(S): 8201540: [AIX] Extend the set of supported >> charsets in java.base >> >> >> On 16/04/2018 09:22, Bhaktavatsal R Maram wrote: >>> >>> 3. Source files for IBM-942C and IBM-943C are changed to template to >>> support #1 >>> >> You might want to double check the webrev as it looks like you've >> added >> templates where as I assume you mean to use "hg rename" to rename >> IBM942C.java and IBM943C.java. >> >> -Alan >> >> >> From christoph.langer at sap.com Wed Apr 18 07:08:43 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Wed, 18 Apr 2018 07:08:43 +0000 Subject: Proposal:AIX's IME support In-Reply-To: <8062d9b71e62b0fc92ae505be9e08e85@linux.vnet.ibm.com> References: <2ed52f9dc4e9487c7b3131b5af3666d9@linux.vnet.ibm.com> <428f37b3-0a64-b727-2426-9c96fd8155a9@oracle.com> <8062d9b71e62b0fc92ae505be9e08e85@linux.vnet.ibm.com> Message-ID: Hi Ichiroh, I just wanted to let you know that I'm currently working with your patch, e.g. test this on AIX. As I've written to you in direct mails, I don't really like that so much code is duplicated into the AIX subfolders. This can eventually lead to the fact that somebody does fixes in the common branch and AIX will miss it. I'll try to update your patch a bit to integrate better with the common codebase. Best regards Christoph > -----Original Message----- > From: awt-dev [mailto:awt-dev-bounces at openjdk.java.net] On Behalf Of > Ichiroh Takiguchi > Sent: Montag, 16. April 2018 19:42 > To: Sergey Bylokhov > Cc: awt-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net > Subject: Re: Proposal:AIX's IME support > > Hello. > > Sorry I'm late. > Christoph Langer helped me to put webrev file into cr.openjdk.java.net. > See, http://cr.openjdk.java.net/~clanger/webrevs/8201429.0/ > > Actually, he found *.patch files had unexpected entries, I fixed the > issue. > It seemed that webrev tool could not handle "hg copy" action. > It was reported by, > > http://mail.openjdk.java.net/pipermail/webrev-dev/2018-April/000145.html > So above webrev file should be fine. > > Please let me know if you have any question and suggestion about webrev > file for > AIX's IME support. > > Thanks, > Ichiroh Takiguchi > > On 2018-04-10 07:39, Sergey Bylokhov wrote: > > Hi, Ichiroh. > > Your contribution is welcome. Can you send the change for review as a > > webrev on cr.openjdk.java.net? > > > > On 09/04/2018 02:09, Ichiroh Takiguchi wrote: > >> Hello, > >> IBM would like to propose another Input Method Framework (IMF) > >> implementation against IBM AIX platform. > >> Same IMF implementation was used by IBM Java8 for AIX. > >> IBM would like to contribute this IMF implementation to OpenJDK > >> project. > >> > >> AIX's Input Method Editor (IME) working behavior is not same as > >> Linux's and Solaris' one. > >> On OpenJDK JDK9 for AIX, users cannot input any East Asian character > >> with their locale by this IME. > >> We think this situation is critical. > >> Put modified files under src/java.desktop/aix/* directories in order > >> not to affect the other unix platform. > >> > >> I'd like contribute following 4 files: > >> M make/lib/Awt2dLibraries.gmk > >> A src/java.desktop/aix/classes/sun/awt/X11InputMethod.java > >> A src/java.desktop/aix/native/libawt_xawt/awt/awt_InputMethod.c > >> A src/java.desktop/aix/native/libawt_xawt/xawt/XlibWrapper.c > >> > >> We need to apply too many changes including JNI, so we copied > >> following 3 files from unix directory to aix directory: > >> * src/java.desktop/unix/classes/sun/awt/X11InputMethod.java > >> * src/java.desktop/unix/native/libawt_xawt/awt/awt_InputMethod.c > >> * src/java.desktop/unix/native/libawt_xawt/xawt/XlibWrapper.c > >> Then, we modified them. > >> > >> I'd appreciate any feedback please, and how I would go about obtaining > >> a sponsor and contributor? > >> > >> Thanks, > >> Ichiroh Takiguchi > >> IBM Japan, Ltd. > >> From magnus.ihse.bursie at oracle.com Wed Apr 18 08:34:44 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Wed, 18 Apr 2018 10:34:44 +0200 Subject: Proposal:AIX's IME support In-Reply-To: References: <2ed52f9dc4e9487c7b3131b5af3666d9@linux.vnet.ibm.com> <428f37b3-0a64-b727-2426-9c96fd8155a9@oracle.com> <8062d9b71e62b0fc92ae505be9e08e85@linux.vnet.ibm.com> Message-ID: > 18 apr. 2018 kl. 09:08 skrev Langer, Christoph : > > Hi Ichiroh, > > I just wanted to let you know that I'm currently working with your patch, e.g. test this on AIX. > > As I've written to you in direct mails, I don't really like that so much code is duplicated into the AIX subfolders. This can eventually lead to the fact that somebody does fixes in the common branch and AIX will miss it. I agree, this is highly unfortunate. :-( We?ve been paying the price for a long time for duplicated code across platforms. Let?s not repeat the same mistake for AIX. While it seems ?safe? for the platform proponent at the moment (?I?m only touching my platform!?), it?s a loss for the OpenJDK project as a whole in the future. /Magnus > I'll try to update your patch a bit to integrate better with the common codebase. > > Best regards > Christoph > >> -----Original Message----- >> From: awt-dev [mailto:awt-dev-bounces at openjdk.java.net] On Behalf Of >> Ichiroh Takiguchi >> Sent: Montag, 16. April 2018 19:42 >> To: Sergey Bylokhov >> Cc: awt-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net >> Subject: Re: Proposal:AIX's IME support >> >> Hello. >> >> Sorry I'm late. >> Christoph Langer helped me to put webrev file into cr.openjdk.java.net. >> See, http://cr.openjdk.java.net/~clanger/webrevs/8201429.0/ >> >> Actually, he found *.patch files had unexpected entries, I fixed the >> issue. >> It seemed that webrev tool could not handle "hg copy" action. >> It was reported by, >> >> http://mail.openjdk.java.net/pipermail/webrev-dev/2018-April/000145.html >> So above webrev file should be fine. >> >> Please let me know if you have any question and suggestion about webrev >> file for >> AIX's IME support. >> >> Thanks, >> Ichiroh Takiguchi >> >> On 2018-04-10 07:39, Sergey Bylokhov wrote: >>> Hi, Ichiroh. >>> Your contribution is welcome. Can you send the change for review as a >>> webrev on cr.openjdk.java.net? >>> >>> On 09/04/2018 02:09, Ichiroh Takiguchi wrote: >>>> Hello, >>>> IBM would like to propose another Input Method Framework (IMF) >>>> implementation against IBM AIX platform. >>>> Same IMF implementation was used by IBM Java8 for AIX. >>>> IBM would like to contribute this IMF implementation to OpenJDK >>>> project. >>>> >>>> AIX's Input Method Editor (IME) working behavior is not same as >>>> Linux's and Solaris' one. >>>> On OpenJDK JDK9 for AIX, users cannot input any East Asian character >>>> with their locale by this IME. >>>> We think this situation is critical. >>>> Put modified files under src/java.desktop/aix/* directories in order >>>> not to affect the other unix platform. >>>> >>>> I'd like contribute following 4 files: >>>> M make/lib/Awt2dLibraries.gmk >>>> A src/java.desktop/aix/classes/sun/awt/X11InputMethod.java >>>> A src/java.desktop/aix/native/libawt_xawt/awt/awt_InputMethod.c >>>> A src/java.desktop/aix/native/libawt_xawt/xawt/XlibWrapper.c >>>> >>>> We need to apply too many changes including JNI, so we copied >>>> following 3 files from unix directory to aix directory: >>>> * src/java.desktop/unix/classes/sun/awt/X11InputMethod.java >>>> * src/java.desktop/unix/native/libawt_xawt/awt/awt_InputMethod.c >>>> * src/java.desktop/unix/native/libawt_xawt/xawt/XlibWrapper.c >>>> Then, we modified them. >>>> >>>> I'd appreciate any feedback please, and how I would go about obtaining >>>> a sponsor and contributor? >>>> >>>> Thanks, >>>> Ichiroh Takiguchi >>>> IBM Japan, Ltd. >>>> > From takiguc at linux.vnet.ibm.com Wed Apr 18 08:56:50 2018 From: takiguc at linux.vnet.ibm.com (Ichiroh Takiguchi) Date: Wed, 18 Apr 2018 17:56:50 +0900 Subject: RFR(S): 8201540: [AIX] Extend the set of supported charsets in java.base In-Reply-To: References: <79b9aaac-aa33-1fc8-93a2-0b6b5bf7ea8d@oracle.com> <59d3d31b-b596-0678-c0bf-69b8f951a40d@oracle.com> Message-ID: <5909b598a1162cf353e928ef9b3ddff8@linux.vnet.ibm.com> Hello Volker. (I'm sorry for duplicate posting, I did bad operation) >> What's actually the difference between "ibm-942C" and "ibm-942"? They have two differences on single byte part: 1. Control Character rotation for 0x1A and 0x1C, 0x7F [2] 2. Character replacement for 0x5C and 0x7E (0xFE, 0xFF) For IBM-942 [3], 0x1A<=>U+001C,0x1C<=>U+007F,0x5C<=>U+00A5,0x7E<=>U+203E,0x7F<=>U+001A 0xFE<=>U+005C,0xFF<=>U+007E For IBM-942C, 0x1A<=>U+001A,0x1C<=>U+001C,0x5C<=>U+005C,0x7E<=>U+007E,0x7F<=>U+007F, 0x5C<=U+00A5,0x7E<=U+203E 0xFE=>U+005C,0xFF=>U+007E (It's ASCII compatible) IBM-942's single byte part is IBM-1041 [4]. IBM-932's single byte part is IBM-897 [5]. IBM-1041 is not same as IBM-897. 5 characters were added into IBM-1041 [4]. (0x80,0xA0,0xFD,0xFE,0xFF [2]) [2] https://www-01.ibm.com/software/globalization/cdra/appendix_g.html [3] https://en.wikipedia.org/wiki/Code_page_942 [4] http://www-01.ibm.com/software/globalization/cp/cp01041.html [5] http://www-01.ibm.com/software/globalization/cp/cp00897.html On 2018-04-17 23:52, Volker Simonis wrote: > Hi Bhaktavatsal Reddy, > > you change looks good, although I can't really verify all the charset > aliases. For example Wikipedia mentions that "ibm-932" is equivalent > to "ibm-942" [1] but you made it an alias for "ibm-942C". What's > actually the difference between "ibm-942C" and "ibm-942"? > > I can sponsor your change although I would appreciate if somebody else > from IBM could have another look at your change. I tried to compare > with "IBM Java 9" but it doesn't seem to exist. They only refer to > AdoptOpenJDK and AdoptOpenJDK just uses a vanilla version of OpenJDK. > > Finally, I hope you won't mind if I update the copyright years on the > files you changed before pushing (this is a convention in the OpenJDK > project). > > Best regards, > Volker > > [1] https://en.wikipedia.org/wiki/Code_page_932_(IBM) > > On Mon, Apr 16, 2018 at 1:10 PM, Bhaktavatsal R Maram > wrote: >> Hi All, >> >> I've regenerated webrev using "hg rename" to create template files. >> webrev looks much neat now.. Thanks Alan for suggestion. >> >> webrev - http://cr.openjdk.java.net/~gromero/8201540/v2/ >> >> Thanks, >> Bhaktavatsal Reddy >> >> >> -----"core-libs-dev" wrote: >> ----- >> To: Alan Bateman >> From: "Bhaktavatsal R Maram" >> Sent by: "core-libs-dev" >> Date: 04/16/2018 02:38PM >> Cc: Tim Ellison , >> ppc-aix-port-dev at openjdk.java.net, Java Core Libs >> >> Subject: Re: RFR(S): 8201540: [AIX] Extend the set of supported >> charsets in java.base >> >> Hi Alan, >> >> I deleted IBM943C.java (using hg remove) and added new file >> IBM943C.java.template (using hg add). I now understand that using "hg >> rename" is giving more meaningful representation in webrev/index.html. >> >> I will re-generate webrev by renaming source files to templates using >> "hg rename" >> >> Thanks, >> Bhaktavatsal Reddy >> >> >> >> -----Alan Bateman wrote: ----- >> To: Bhaktavatsal R Maram , Volker Simonis >> >> From: Alan Bateman >> Date: 04/16/2018 02:16PM >> Cc: Java Core Libs , Tim Ellison >> , ppc-aix-port-dev at openjdk.java.net >> Subject: Re: RFR(S): 8201540: [AIX] Extend the set of supported >> charsets in java.base >> >> >> On 16/04/2018 09:22, Bhaktavatsal R Maram wrote: >>> >>> 3. Source files for IBM-942C and IBM-943C are changed to template to >>> support #1 >>> >> You might want to double check the webrev as it looks like you've >> added >> templates where as I assume you mean to use "hg rename" to rename >> IBM942C.java and IBM943C.java. >> >> -Alan >> >> >> From takiguc at linux.vnet.ibm.com Wed Apr 18 09:17:27 2018 From: takiguc at linux.vnet.ibm.com (Ichiroh Takiguchi) Date: Wed, 18 Apr 2018 18:17:27 +0900 Subject: Proposal:AIX's IME support In-Reply-To: References: <2ed52f9dc4e9487c7b3131b5af3666d9@linux.vnet.ibm.com> <428f37b3-0a64-b727-2426-9c96fd8155a9@oracle.com> <8062d9b71e62b0fc92ae505be9e08e85@linux.vnet.ibm.com> Message-ID: <8663d11893863e3e63d4671b02ff01f5@linux.vnet.ibm.com> Hello Magnus and Christoph. I think it's not easy to merge AIX IME support code without current code changes... On 2018-04-18 17:34, Magnus Ihse Bursie wrote: >> 18 apr. 2018 kl. 09:08 skrev Langer, Christoph >> : >> >> Hi Ichiroh, >> >> I just wanted to let you know that I'm currently working with your >> patch, e.g. test this on AIX. >> >> As I've written to you in direct mails, I don't really like that so >> much code is duplicated into the AIX subfolders. This can eventually >> lead to the fact that somebody does fixes in the common branch and AIX >> will miss it. > > I agree, this is highly unfortunate. :-( We?ve been paying the price > for a long time for duplicated code across platforms. Let?s not > repeat the same mistake for AIX. > > While it seems ?safe? for the platform proponent at the moment > (?I?m only touching my platform!?), it?s a loss for the > OpenJDK project as a whole in the future. > > /Magnus > >> I'll try to update your patch a bit to integrate better with the >> common codebase. > >> >> Best regards >> Christoph >> >>> -----Original Message----- >>> From: awt-dev [mailto:awt-dev-bounces at openjdk.java.net] On Behalf Of >>> Ichiroh Takiguchi >>> Sent: Montag, 16. April 2018 19:42 >>> To: Sergey Bylokhov >>> Cc: awt-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net >>> Subject: Re: Proposal:AIX's IME support >>> >>> Hello. >>> >>> Sorry I'm late. >>> Christoph Langer helped me to put webrev file into >>> cr.openjdk.java.net. >>> See, http://cr.openjdk.java.net/~clanger/webrevs/8201429.0/ >>> >>> Actually, he found *.patch files had unexpected entries, I fixed the >>> issue. >>> It seemed that webrev tool could not handle "hg copy" action. >>> It was reported by, >>> >>> http://mail.openjdk.java.net/pipermail/webrev-dev/2018-April/000145.html >>> So above webrev file should be fine. >>> >>> Please let me know if you have any question and suggestion about >>> webrev >>> file for >>> AIX's IME support. >>> >>> Thanks, >>> Ichiroh Takiguchi >>> >>> On 2018-04-10 07:39, Sergey Bylokhov wrote: >>>> Hi, Ichiroh. >>>> Your contribution is welcome. Can you send the change for review as >>>> a >>>> webrev on cr.openjdk.java.net? >>>> >>>> On 09/04/2018 02:09, Ichiroh Takiguchi wrote: >>>>> Hello, >>>>> IBM would like to propose another Input Method Framework (IMF) >>>>> implementation against IBM AIX platform. >>>>> Same IMF implementation was used by IBM Java8 for AIX. >>>>> IBM would like to contribute this IMF implementation to OpenJDK >>>>> project. >>>>> >>>>> AIX's Input Method Editor (IME) working behavior is not same as >>>>> Linux's and Solaris' one. >>>>> On OpenJDK JDK9 for AIX, users cannot input any East Asian >>>>> character >>>>> with their locale by this IME. >>>>> We think this situation is critical. >>>>> Put modified files under src/java.desktop/aix/* directories in >>>>> order >>>>> not to affect the other unix platform. >>>>> >>>>> I'd like contribute following 4 files: >>>>> M make/lib/Awt2dLibraries.gmk >>>>> A src/java.desktop/aix/classes/sun/awt/X11InputMethod.java >>>>> A src/java.desktop/aix/native/libawt_xawt/awt/awt_InputMethod.c >>>>> A src/java.desktop/aix/native/libawt_xawt/xawt/XlibWrapper.c >>>>> >>>>> We need to apply too many changes including JNI, so we copied >>>>> following 3 files from unix directory to aix directory: >>>>> * src/java.desktop/unix/classes/sun/awt/X11InputMethod.java >>>>> * src/java.desktop/unix/native/libawt_xawt/awt/awt_InputMethod.c >>>>> * src/java.desktop/unix/native/libawt_xawt/xawt/XlibWrapper.c >>>>> Then, we modified them. >>>>> >>>>> I'd appreciate any feedback please, and how I would go about >>>>> obtaining >>>>> a sponsor and contributor? >>>>> >>>>> Thanks, >>>>> Ichiroh Takiguchi >>>>> IBM Japan, Ltd. >>>>> >> From christoph.langer at sap.com Thu Apr 19 07:24:05 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 19 Apr 2018 07:24:05 +0000 Subject: RFR(XS): 8202000: AIX build broken after JDK-8195099 Message-ID: Hi, after JDK-8195099, the AIX build is broken. Please review this small fix which consists of renaming 2 macros that conflict with other macros coming in from a system header. Bug: https://bugs.openjdk.java.net/browse/JDK-8202000 Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8202000.0/ Thanks, Christoph -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.stuefe at gmail.com Thu Apr 19 07:26:45 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 19 Apr 2018 09:26:45 +0200 Subject: RFR(XS): 8202000: AIX build broken after JDK-8195099 In-Reply-To: References: Message-ID: Seems trivial and fine. Thanks for fixing. ..Thomas On Thu, Apr 19, 2018 at 9:24 AM, Langer, Christoph wrote: > Hi, > > > > after JDK-8195099, the AIX build is broken. > > > > Please review this small fix which consists of renaming 2 macros that > conflict with other macros coming in from a system header. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8202000 > > Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8202000.0/ > > > > Thanks, > > Christoph > > From goetz.lindenmaier at sap.com Thu Apr 19 07:27:05 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 19 Apr 2018 07:27:05 +0000 Subject: RFR(XS): 8202000: AIX build broken after JDK-8195099 In-Reply-To: References: Message-ID: <354fe465b55a4bcca46a3783ae247c9a@sap.com> Hi Christoph, Change looks good, thanks for fixing! Best regards, Goetz. > -----Original Message----- > From: ppc-aix-port-dev [mailto:ppc-aix-port-dev- > bounces at openjdk.java.net] On Behalf Of Langer, Christoph > Sent: Donnerstag, 19. April 2018 09:24 > To: hotspot-dev at openjdk.java.net > Cc: ppc-aix-port-dev at openjdk.java.net > Subject: [CAUTION] RFR(XS): 8202000: AIX build broken after JDK-8195099 > > Hi, > > > > after JDK-8195099, the AIX build is broken. > > > > Please review this small fix which consists of renaming 2 macros that conflict > with other macros coming in from a system header. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8202000 > > > Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8202000.0/ > > > > > Thanks, > > Christoph > > From matthias.baesken at sap.com Thu Apr 19 07:31:28 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 19 Apr 2018 07:31:28 +0000 Subject: RFR(XS): 8202000: AIX build broken after JDK-8195099 In-Reply-To: References: Message-ID: <80eb3c39d4a846e99faec1854c2590b0@sap.com> Hi Christoph, looks good (not a Reviewer however) . Best regards, Matthias From: ppc-aix-port-dev [mailto:ppc-aix-port-dev-bounces at openjdk.java.net] On Behalf Of Langer, Christoph Sent: Donnerstag, 19. April 2018 09:24 To: hotspot-dev at openjdk.java.net Cc: ppc-aix-port-dev at openjdk.java.net Subject: [CAUTION] RFR(XS): 8202000: AIX build broken after JDK-8195099 Hi, after JDK-8195099, the AIX build is broken. Please review this small fix which consists of renaming 2 macros that conflict with other macros coming in from a system header. Bug: https://bugs.openjdk.java.net/browse/JDK-8202000 Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8202000.0/ Thanks, Christoph -------------- next part -------------- An HTML attachment was scrubbed... URL: From christoph.langer at sap.com Thu Apr 19 07:40:43 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 19 Apr 2018 07:40:43 +0000 Subject: RFR(XS): 8202000: AIX build broken after JDK-8195099 In-Reply-To: <354fe465b55a4bcca46a3783ae247c9a@sap.com> References: <354fe465b55a4bcca46a3783ae247c9a@sap.com> Message-ID: Thanks for reviewing. I will run it through jdk-submit, then push it. > -----Original Message----- > From: Lindenmaier, Goetz > Sent: Donnerstag, 19. April 2018 09:27 > To: Langer, Christoph ; hotspot- > dev at openjdk.java.net > Cc: ppc-aix-port-dev at openjdk.java.net > Subject: RE: RFR(XS): 8202000: AIX build broken after JDK-8195099 > > Hi Christoph, > > Change looks good, thanks for fixing! > > Best regards, > Goetz. > > > -----Original Message----- > > From: ppc-aix-port-dev [mailto:ppc-aix-port-dev- > > bounces at openjdk.java.net] On Behalf Of Langer, Christoph > > Sent: Donnerstag, 19. April 2018 09:24 > > To: hotspot-dev at openjdk.java.net > > Cc: ppc-aix-port-dev at openjdk.java.net > > Subject: [CAUTION] RFR(XS): 8202000: AIX build broken after JDK-8195099 > > > > Hi, > > > > > > > > after JDK-8195099, the AIX build is broken. > > > > > > > > Please review this small fix which consists of renaming 2 macros that > conflict > > with other macros coming in from a system header. > > > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8202000 > > > > > > Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8202000.0/ > > > > > > > > > > Thanks, > > > > Christoph > > > > From HORIE at jp.ibm.com Mon Apr 23 10:33:59 2018 From: HORIE at jp.ibm.com (Michihiro Horie) Date: Mon, 23 Apr 2018 19:33:59 +0900 Subject: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64 Message-ID: Dear all, I would like to ask reviews on 8154736 ?enhancement of cmpxchg and copy_to_survivor?. The change adds options to avoid expensive syncs with compare-and-exchange. An experiment by using SPECjbb2015 showed 6% improvement in critical-jOPS. This change focuses on ppc64 but would be potentially beneficial for aarch64. Although discussions stopped at http://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/2016-October/002718.html , I would like to restart the review by taking over Hiroshi's work if the discussion is still open. Bug: https://bugs.openjdk.java.net/browse/JDK-8154736 Webrev: http://cr.openjdk.java.net/~mhorie/8154736/webrev.08/ Previous review had discussions on improper log output ( http://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/2016-September/002672.html ). Logs can be generated properly with this change, but I would like to ask if we should use ?if(log) OrderAccess:acquire()? as is in webrev or more general approach with a call to OrderAccess:consume() with empty implementation on all supported platforms. Also, there were discussions on the problem of unawareness of copied obj ( http://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/2016-October/002696.html ). This change adds ?release? in cmpxchg_pre_membar. This was discussed in http://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/2016-October/002698.html . I measured SPECjbb2015 with its multi JVMs mode on a POWER8 node (for JDK11 , I modified MANIFEST in specjbb2015.jar to specify locations of JAXB related libraries). As a result, critical-jOPS improved by 6% due to this change. Best regards, -- Michihiro, IBM Research - Tokyo -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin.doerr at sap.com Tue Apr 24 14:20:06 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 24 Apr 2018 14:20:06 +0000 Subject: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64 In-Reply-To: References: Message-ID: <63793e9f7b0e4989a9716cf87be3c7fe@sap.com> Hi Michihiro, thanks for posting the new webrev. I think this looks much better, now. I don't like the "if log_is_enabled() acquire()" code so much, but it looks correct to me. I'd prefer your OrderAccess:consume() proposal, but I can live with it if other reviewers prefer it. I couldn't find any store-load pattern which may miss ordering, but I'd highly appreciate if another reviewer double-checked that the barriers are right this time. If needed, we could use memory_order_acq_rel which I'm planning to add with JDK-8202080. I guess this wouldn't really impact performance. I think 6% performance gain is really worth doing this change. Best regards, Martin From: Michihiro Horie [mailto:HORIE at jp.ibm.com] Sent: Montag, 23. April 2018 12:34 To: ppc-aix-port-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net Cc: Hiroshi H Horii ; Gustavo Romero ; Kazunori Ogata ; shade at redhat.com; aph at redhat.com; Doerr, Martin Subject: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64 Dear all, I would like to ask reviews on 8154736 "enhancement of cmpxchg and copy_to_survivor". The change adds options to avoid expensive syncs with compare-and-exchange. An experiment by using SPECjbb2015 showed 6% improvement in critical-jOPS. This change focuses on ppc64 but would be potentially beneficial for aarch64. Although discussions stopped at http://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/2016-October/002718.html, I would like to restart the review by taking over Hiroshi's work if the discussion is still open. Bug: https://bugs.openjdk.java.net/browse/JDK-8154736 Webrev: http://cr.openjdk.java.net/~mhorie/8154736/webrev.08/ Previous review had discussions on improper log output (http://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/2016-September/002672.html). Logs can be generated properly with this change, but I would like to ask if we should use "if(log) OrderAccess:acquire()" as is in webrev or more general approach with a call to OrderAccess:consume() with empty implementation on all supported platforms. Also, there were discussions on the problem of unawareness of copied obj (http://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/2016-October/002696.html). This change adds "release" in cmpxchg_pre_membar. This was discussed in http://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/2016-October/002698.html. I measured SPECjbb2015 with its multi JVMs mode on a POWER8 node (for JDK11, I modified MANIFEST in specjbb2015.jar to specify locations of JAXB related libraries). As a result, critical-jOPS improved by 6% due to this change. Best regards, -- Michihiro, IBM Research - Tokyo -------------- next part -------------- An HTML attachment was scrubbed... URL: From glaubitz at physik.fu-berlin.de Tue Apr 24 14:42:26 2018 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Tue, 24 Apr 2018 16:42:26 +0200 Subject: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64 In-Reply-To: <63793e9f7b0e4989a9716cf87be3c7fe@sap.com> References: <63793e9f7b0e4989a9716cf87be3c7fe@sap.com> Message-ID: Hi! On 04/24/2018 04:20 PM, Doerr, Martin wrote: > I think 6% performance gain is really worth doing this change. Quick question: Does this pose any assumptions on the instruction set baseline on ppc64? I haven't build-tested this yet, but I want to avoid it doesn't break on POWER5, for example, which is used in Debian's big-endian ppc64 port. Thanks, Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From gromero at linux.vnet.ibm.com Tue Apr 24 15:54:29 2018 From: gromero at linux.vnet.ibm.com (Gustavo Romero) Date: Tue, 24 Apr 2018 12:54:29 -0300 Subject: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64 In-Reply-To: References: <63793e9f7b0e4989a9716cf87be3c7fe@sap.com> Message-ID: Hi John, On 04/24/2018 11:42 AM, John Paul Adrian Glaubitz wrote: > On 04/24/2018 04:20 PM, Doerr, Martin wrote: >> I think 6% performance gain is really worth doing this change. > > Quick question: Does this pose any assumptions on the instruction set baseline > on ppc64? I haven't build-tested this yet, but I want to avoid it doesn't break > on POWER5, for example, which is used in Debian's big-endian ppc64 port. Yes, it does. But POWER5 and above are ok. Lightweight sync (lwsync) was introduced with Power ISA v2.03 which, by its turn, maps to POWER5. Thanks for taking care of Power + Debian :-) Regards, Gustavo From glaubitz at physik.fu-berlin.de Tue Apr 24 18:42:15 2018 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Tue, 24 Apr 2018 20:42:15 +0200 Subject: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64 In-Reply-To: References: <63793e9f7b0e4989a9716cf87be3c7fe@sap.com> Message-ID: <95f13bd2-c173-fa2b-4454-cd101b79126d@physik.fu-berlin.de> Hi Gustavo! On 04/24/2018 05:54 PM, Gustavo Romero wrote: >> Quick question: Does this pose any assumptions on the instruction set baseline >> on ppc64? I haven't build-tested this yet, but I want to avoid it doesn't break >> on POWER5, for example, which is used in Debian's big-endian ppc64 port. > > Yes, it does. But POWER5 and above are ok. Lightweight sync (lwsync) was > introduced with Power ISA v2.03 which, by its turn, maps to POWER5. We are primarily interested in PPC970 and compatible which - according to Wikipedia - is compliant with POWER ISA 2.03: > https://en.wikipedia.org/wiki/Power_Architecture#Power_ISA_v.2.03 It's sometimes a bit confusing to differentiate all the different POWER ISAs because there are so many. That's why I usually ask in such cases. > Thanks for taking care of Power + Debian :-) You're welcome. Debian has both a big-endian (POWER5/970) and little-endian (POWER8) ppc64 port as well as powerpc (32-bit) and powerpcspe (e500v2) ports, so PowerPC is quite popular within Debian ;). Thanks, Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From david.holmes at oracle.com Wed Apr 25 12:45:15 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 25 Apr 2018 22:45:15 +1000 Subject: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64 In-Reply-To: References: Message-ID: <5625a595-1165-8d48-afbd-8229cdc4ac07@oracle.com> Hi Michihiro, On 23/04/2018 8:33 PM, Michihiro Horie wrote: > > > Dear all, > > I would like to ask reviews on 8154736 ?enhancement of cmpxchg and > copy_to_survivor?. The change adds options to avoid expensive syncs with > compare-and-exchange. An experiment by using SPECjbb2015 showed 6% > improvement in critical-jOPS. This change focuses on ppc64 but would be > potentially beneficial for aarch64. > > Although discussions stopped at > http://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/2016-October/002718.html > , I would like to restart the review by taking over Hiroshi's work if the > discussion is still open. So the very last comment there was about not implicitly assuming memory_order_consume, yet that has not been addressed in the proposal. Further the discussion on hotspot-runtime-dev through September and October was far more illuminating. I think my post here: http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2016-October/021617.html and the closely following one from Thomas Schatzl summed up the concerns about the proposed changes. AFAICS the restarted proposal addresses none of those concerns but simply takes up where the previous implementation suggestion left off. This is a proposal to change the memory ordering semantics of part of the shared GC code _not_ just the PPC64 implementation, but I have seen no analysis to demonstrate the correctness of such a proposal. David ----- > Bug: https://bugs.openjdk.java.net/browse/JDK-8154736 > Webrev: http://cr.openjdk.java.net/~mhorie/8154736/webrev.08/ > > Previous review had discussions on improper log output ( > http://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/2016-September/002672.html > ). Logs can be generated properly with this change, but I would like to ask > if we should use ?if(log) OrderAccess:acquire()? as is in webrev or more > general approach with a call to OrderAccess:consume() with empty > implementation on all supported platforms. > > Also, there were discussions on the problem of unawareness of copied obj ( > http://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/2016-October/002696.html > ). This change adds ?release? in cmpxchg_pre_membar. This was discussed in > http://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/2016-October/002698.html > . > > I measured SPECjbb2015 with its multi JVMs mode on a POWER8 node (for JDK11 > , I modified MANIFEST in specjbb2015.jar to specify locations of JAXB > related libraries). As a result, critical-jOPS improved by 6% due to this > change. > > Best regards, > -- > Michihiro, > IBM Research - Tokyo > From HORIE at jp.ibm.com Thu Apr 26 12:04:48 2018 From: HORIE at jp.ibm.com (Michihiro Horie) Date: Thu, 26 Apr 2018 21:04:48 +0900 Subject: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64 In-Reply-To: <5625a595-1165-8d48-afbd-8229cdc4ac07@oracle.com> References: <5625a595-1165-8d48-afbd-8229cdc4ac07@oracle.com> Message-ID: Hi David, >So the very last comment there was about not implicitly assuming >memory_order_consume, yet that has not been addressed in the proposal. > >Further the discussion on hotspot-runtime-dev through September and >October was far more illuminating. I think my post here: > https://urldefense.proofpoint.com/v2/url?u=http-3A__mail.openjdk.java.net_pipermail_hotspot-2Druntime-2Ddev_2016-2DOctober_021617.html&d=DwICJg&c=jf_iaSHvJObTbx-siA1ZOg&r=oecsIpYF-cifqq2i1JEH0Q&m=G-yYtdHSoT4SzbzpDm1wRbEXJKNJYc7V1jamUELVj_c&s=4DMMn8KoNrvfjd9HS_nXI5hLeagvDCcmeB6oFd-mcPk&e= >and the closely following one from Thomas Schatzl summed up the concerns >about the proposed changes. Thank you very much for pointing out the missing items I need to take into account. >This is a proposal to change the memory ordering semantics of part of >the shared GC code _not_ just the PPC64 implementation, but I have seen >no analysis to demonstrate the correctness of such a proposal. I do agree the necessity of demonstrating the correctness. I would try my best for this. Best regards, -- Michihiro, IBM Research - Tokyo From: David Holmes To: Michihiro Horie , ppc-aix-port-dev at openjdk.java.net, hotspot-dev at openjdk.java.net, hotspot-gc-dev at openjdk.java.net Cc: Hiroshi H Horii Date: 2018/04/25 21:45 Subject: Re: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64 Hi Michihiro, On 23/04/2018 8:33 PM, Michihiro Horie wrote: > > > Dear all, > > I would like to ask reviews on 8154736 ?enhancement of cmpxchg and > copy_to_survivor?. The change adds options to avoid expensive syncs with > compare-and-exchange. An experiment by using SPECjbb2015 showed 6% > improvement in critical-jOPS. This change focuses on ppc64 but would be > potentially beneficial for aarch64. > > Although discussions stopped at > https://urldefense.proofpoint.com/v2/url?u=http-3A__mail.openjdk.java.net_pipermail_ppc-2Daix-2Dport-2Ddev_2016-2DOctober_002718.html&d=DwICJg&c=jf_iaSHvJObTbx-siA1ZOg&r=oecsIpYF-cifqq2i1JEH0Q&m=G-yYtdHSoT4SzbzpDm1wRbEXJKNJYc7V1jamUELVj_c&s=C2p8l68JKbOLCTpy4Zu5280A84QXht0U4_3Ha7QBaRc&e= > , I would like to restart the review by taking over Hiroshi's work if the > discussion is still open. So the very last comment there was about not implicitly assuming memory_order_consume, yet that has not been addressed in the proposal. Further the discussion on hotspot-runtime-dev through September and October was far more illuminating. I think my post here: https://urldefense.proofpoint.com/v2/url?u=http-3A__mail.openjdk.java.net_pipermail_hotspot-2Druntime-2Ddev_2016-2DOctober_021617.html&d=DwICJg&c=jf_iaSHvJObTbx-siA1ZOg&r=oecsIpYF-cifqq2i1JEH0Q&m=G-yYtdHSoT4SzbzpDm1wRbEXJKNJYc7V1jamUELVj_c&s=4DMMn8KoNrvfjd9HS_nXI5hLeagvDCcmeB6oFd-mcPk&e= and the closely following one from Thomas Schatzl summed up the concerns about the proposed changes. AFAICS the restarted proposal addresses none of those concerns but simply takes up where the previous implementation suggestion left off. This is a proposal to change the memory ordering semantics of part of the shared GC code _not_ just the PPC64 implementation, but I have seen no analysis to demonstrate the correctness of such a proposal. David ----- > Bug: https://urldefense.proofpoint.com/v2/url?u=https-3A__bugs.openjdk.java.net_browse_JDK-2D8154736&d=DwICJg&c=jf_iaSHvJObTbx-siA1ZOg&r=oecsIpYF-cifqq2i1JEH0Q&m=G-yYtdHSoT4SzbzpDm1wRbEXJKNJYc7V1jamUELVj_c&s=4hpwVPathLPraOYinkDMNu4gAgivm62zURDtKyMKPe8&e= > Webrev: https://urldefense.proofpoint.com/v2/url?u=http-3A__cr.openjdk.java.net_-7Emhorie_8154736_webrev.08_&d=DwICJg&c=jf_iaSHvJObTbx-siA1ZOg&r=oecsIpYF-cifqq2i1JEH0Q&m=G-yYtdHSoT4SzbzpDm1wRbEXJKNJYc7V1jamUELVj_c&s=DMMm7IWjNJSRB69AbPfo-zPkhDNK8EbhxWEdT6z46k8&e= > > Previous review had discussions on improper log output ( > https://urldefense.proofpoint.com/v2/url?u=http-3A__mail.openjdk.java.net_pipermail_ppc-2Daix-2Dport-2Ddev_2016-2DSeptember_002672.html&d=DwICJg&c=jf_iaSHvJObTbx-siA1ZOg&r=oecsIpYF-cifqq2i1JEH0Q&m=G-yYtdHSoT4SzbzpDm1wRbEXJKNJYc7V1jamUELVj_c&s=s0YwAXbLaVr-SrmfU4DdH87Kd4baoN7bkGA1y-fBSrQ&e= > ). Logs can be generated properly with this change, but I would like to ask > if we should use ?if(log) OrderAccess:acquire()? as is in webrev or more > general approach with a call to OrderAccess:consume() with empty > implementation on all supported platforms. > > Also, there were discussions on the problem of unawareness of copied obj ( > https://urldefense.proofpoint.com/v2/url?u=http-3A__mail.openjdk.java.net_pipermail_ppc-2Daix-2Dport-2Ddev_2016-2DOctober_002696.html&d=DwICJg&c=jf_iaSHvJObTbx-siA1ZOg&r=oecsIpYF-cifqq2i1JEH0Q&m=G-yYtdHSoT4SzbzpDm1wRbEXJKNJYc7V1jamUELVj_c&s=HmMGF6U-OQ33yEAAUBjhhdBSVzw7m9ec0oiXn8y7eS8&e= > ). This change adds ?release? in cmpxchg_pre_membar. This was discussed in > https://urldefense.proofpoint.com/v2/url?u=http-3A__mail.openjdk.java.net_pipermail_ppc-2Daix-2Dport-2Ddev_2016-2DOctober_002698.html&d=DwICJg&c=jf_iaSHvJObTbx-siA1ZOg&r=oecsIpYF-cifqq2i1JEH0Q&m=G-yYtdHSoT4SzbzpDm1wRbEXJKNJYc7V1jamUELVj_c&s=mZWJvHc1jKeJlaDa1nu_vDz5FfB0WlCCnRmAOJYHLgA&e= > . > > I measured SPECjbb2015 with its multi JVMs mode on a POWER8 node (for JDK11 > , I modified MANIFEST in specjbb2015.jar to specify locations of JAXB > related libraries). As a result, critical-jOPS improved by 6% due to this > change. > > Best regards, > -- > Michihiro, > IBM Research - Tokyo > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: graycol.gif Type: image/gif Size: 105 bytes Desc: not available URL: From thomas.stuefe at gmail.com Thu Apr 26 14:03:52 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 26 Apr 2018 16:03:52 +0200 Subject: RFR(xs): 8202325: [aix] disable warnings-as-errors by default Message-ID: Hi all, may I have reviews please: https://bugs.openjdk.java.net/browse/JDK-8202325 http://cr.openjdk.java.net/~stuefe/webrevs/8202325-aix-disable-warnings-as-errors/webrev.00/webrev/ We decided to disable warnings as errors by default on AIX. This is a pragmatic decision - we will never be able to fix all the many xlC warnings and the build since long only worked when disabling warnings-as-errors. So, might as well make this the default. Best regards, Thomas From goetz.lindenmaier at sap.com Thu Apr 26 14:12:59 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 26 Apr 2018 14:12:59 +0000 Subject: RFR(xs): 8202325: [aix] disable warnings-as-errors by default In-Reply-To: References: Message-ID: <3e3b9ca8c23b484381190901ed6af52f@sap.com> Hi Thomas, this looks good and will simplify building on AIX. Thanks, Goetz. > -----Original Message----- > From: ppc-aix-port-dev [mailto:ppc-aix-port-dev- > bounces at openjdk.java.net] On Behalf Of Thomas St?fe > Sent: Donnerstag, 26. April 2018 16:04 > To: build-dev ; ppc-aix-port- > dev at openjdk.java.net > Subject: RFR(xs): 8202325: [aix] disable warnings-as-errors by default > > Hi all, > > may I have reviews please: > > https://bugs.openjdk.java.net/browse/JDK-8202325 > http://cr.openjdk.java.net/~stuefe/webrevs/8202325-aix-disable-warnings- > as-errors/webrev.00/webrev/ > > We decided to disable warnings as errors by default on AIX. This is a > pragmatic decision - we will never be able to fix all the many xlC > warnings and the build since long only worked when disabling > warnings-as-errors. > > So, might as well make this the default. > > Best regards, Thomas From matthias.baesken at sap.com Thu Apr 26 14:13:38 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 26 Apr 2018 14:13:38 +0000 Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 Message-ID: <8361024dba4f44309292e73ec9ad7b83@sap.com> Hello , could you please review this small adjustment to the symbol visibility compilation settings on AIX ? Currently we use XLC 12.1 to compile JDK on AIX . However XLC 12.1 does not support the "-qvisibility=hidden" setting currently set on AIX. It was introduced with XLC 13.1 . Christoph found some info about it here : https://www.ibm.com/developerworks/aix/library/au-aix-symbol-visibility-part2/index.html Setting it only generates hundreds of warnings in the build log , warnings look like this : XlC12.1 bash-4.4$ xlC -qversion IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72) Version: 12.01.0000.0019 bash-4.4$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc 1506-173 (W) Option visibility=hidden is not valid. Enter xlC for list of valid options. Compare to XLC13.1 bash-3.00$ xlC -qversion IBM XL C/C++ for AIX, V13.1 (5725-C72, 5765-J07) Version: 13.01.0000.0008 bash-3.00$ xlC -qvisibility=default sizeof.c -o sizeof_aixxlc bash-3.00$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc So it is better to avoid setting these flags when using xlc12.1 . Please review : Bug : https://bugs.openjdk.java.net/browse/JDK-8202322 Change : http://cr.openjdk.java.net/~mbaesken/webrevs/8202322/ Best regards, Matthias -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.stuefe at gmail.com Thu Apr 26 14:31:57 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 26 Apr 2018 16:31:57 +0200 Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 In-Reply-To: <8361024dba4f44309292e73ec9ad7b83@sap.com> References: <8361024dba4f44309292e73ec9ad7b83@sap.com> Message-ID: Hi Matthias, I would prefer a numerical comparison to version < 13. Otherwise the code looks like you want to just exclude 12.1 for some reason when in reality you want to omit the flag for any compiler older than xlc 13. Best Regards, Thomas On Thu, Apr 26, 2018 at 4:13 PM, Baesken, Matthias wrote: > Hello , could you please review this small adjustment to the symbol visibility compilation settings on AIX ? > Currently we use XLC 12.1 to compile JDK on AIX . > > However XLC 12.1 does not support the "-qvisibility=hidden" setting currently set on AIX. > It was introduced with XLC 13.1 . Christoph found some info about it here : > > https://www.ibm.com/developerworks/aix/library/au-aix-symbol-visibility-part2/index.html > > Setting it only generates hundreds of warnings in the build log , warnings look like this : > XlC12.1 > > bash-4.4$ xlC -qversion > IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72) > Version: 12.01.0000.0019 > > bash-4.4$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc > 1506-173 (W) Option visibility=hidden is not valid. Enter xlC for list of valid options. > > Compare to XLC13.1 > > bash-3.00$ xlC -qversion > IBM XL C/C++ for AIX, V13.1 (5725-C72, 5765-J07) > Version: 13.01.0000.0008 > bash-3.00$ xlC -qvisibility=default sizeof.c -o sizeof_aixxlc > bash-3.00$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc > > > So it is better to avoid setting these flags when using xlc12.1 . > Please review : > > Bug : > > https://bugs.openjdk.java.net/browse/JDK-8202322 > > Change : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202322/ > > > Best regards, Matthias > > From bhamaram at in.ibm.com Thu Apr 26 14:32:04 2018 From: bhamaram at in.ibm.com (Bhaktavatsal R Maram) Date: Thu, 26 Apr 2018 14:32:04 +0000 Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 In-Reply-To: <8361024dba4f44309292e73ec9ad7b83@sap.com> References: <8361024dba4f44309292e73ec9ad7b83@sap.com> Message-ID: Hi Matthias, Its right in time. I am facing "Undefined Symbol" errors with XLC 13.1 due to "-qvisibility=hidden" which hides all symbols by default. I workaround that by removing this flag -bash-4.4$ xlC -qversion IBM XL C/C++ for AIX, V13.1.3 (5725-C72, 5765-J07) Version: 13.01.0003.0000 Looks like there are many function declarations that require changes to make function visible. Thanks, Bhaktavatsal Reddy -----"core-libs-dev" wrote: ----- To: "'build-dev at openjdk.java.net'" , "ppc-aix-port-dev at openjdk.java.net" , "core-libs-dev at openjdk.java.net" From: "Baesken, Matthias" Sent by: "core-libs-dev" Date: 04/26/2018 07:45PM Cc: "Simonis, Volker" Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 Hello , could you please review this small adjustment to the symbol visibility compilation settings on AIX ? Currently we use XLC 12.1 to compile JDK on AIX . However XLC 12.1 does not support the "-qvisibility=hidden" setting currently set on AIX. It was introduced with XLC 13.1 . Christoph found some info about it here : https://www.ibm.com/developerworks/aix/library/au-aix-symbol-visibility-part2/index.html Setting it only generates hundreds of warnings in the build log , warnings look like this : XlC12.1 bash-4.4$ xlC -qversion IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72) Version: 12.01.0000.0019 bash-4.4$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc 1506-173 (W) Option visibility=hidden is not valid. Enter xlC for list of valid options. Compare to XLC13.1 bash-3.00$ xlC -qversion IBM XL C/C++ for AIX, V13.1 (5725-C72, 5765-J07) Version: 13.01.0000.0008 bash-3.00$ xlC -qvisibility=default sizeof.c -o sizeof_aixxlc bash-3.00$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc So it is better to avoid setting these flags when using xlc12.1 . Please review : Bug : https://urldefense.proofpoint.com/v2/url?u=https-3A__bugs.openjdk.java.net_browse_JDK-2D8202322&d=DwIFAg&c=jf_iaSHvJObTbx-siA1ZOg&r=KUVGEwJiRVpNtQ9wUhGP6BKqzSTV1OWX31WWPdQMmqg&m=F6_aATE7bsRWZG5lPZhJUESHNGqST5MNbkULllIH8l4&s=M4IJ-YdB3dfN7lj0DEWozQbipDXmgKtSu3pEyMcJF_E&e= Change : https://urldefense.proofpoint.com/v2/url?u=http-3A__cr.openjdk.java.net_-7Embaesken_webrevs_8202322_&d=DwIFAg&c=jf_iaSHvJObTbx-siA1ZOg&r=KUVGEwJiRVpNtQ9wUhGP6BKqzSTV1OWX31WWPdQMmqg&m=F6_aATE7bsRWZG5lPZhJUESHNGqST5MNbkULllIH8l4&s=gb0OP9fQTAWWJbBZDK6L_0gTKf1pwna8aXaYr_uVv8Q&e= Best regards, Matthias From matthias.baesken at sap.com Thu Apr 26 14:37:17 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 26 Apr 2018 14:37:17 +0000 Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 In-Reply-To: References: <8361024dba4f44309292e73ec9ad7b83@sap.com> Message-ID: <1ff5961701994081819d0bd7db3d3ec3@sap.com> > Its right in time. I am facing "Undefined Symbol" errors with XLC 13.1 due to > "-qvisibility=hidden" which hides all symbols by default. I workaround that by > removing this flag Hello, so it seems that I better remove the "-qvisibility=" completely for xlc (and not only for xlc12.1) ? (we currently compile only with XLC 12.1 , there might be more issues with 13.1 ) Best regards, Matthias > -----Original Message----- > From: Bhaktavatsal R Maram [mailto:bhamaram at in.ibm.com] > Sent: Donnerstag, 26. April 2018 16:32 > To: Baesken, Matthias > Cc: 'build-dev at openjdk.java.net' ; ppc-aix- > port-dev at openjdk.java.net; core-libs-dev at openjdk.java.net; Simonis, > Volker > Subject: Re: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 > > Hi Matthias, > > Its right in time. I am facing "Undefined Symbol" errors with XLC 13.1 due to > "-qvisibility=hidden" which hides all symbols by default. I workaround that by > removing this flag > > -bash-4.4$ xlC -qversion > IBM XL C/C++ for AIX, V13.1.3 (5725-C72, 5765-J07) > Version: 13.01.0003.0000 > > Looks like there are many function declarations that require changes to make > function visible. > > Thanks, > Bhaktavatsal Reddy > > > > -----"core-libs-dev" wrote: ----- > To: "'build-dev at openjdk.java.net'" , "ppc- > aix-port-dev at openjdk.java.net" , > "core-libs-dev at openjdk.java.net" > From: "Baesken, Matthias" > Sent by: "core-libs-dev" > Date: 04/26/2018 07:45PM > Cc: "Simonis, Volker" > Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 > > Hello , could you please review this small adjustment to the symbol visibility > compilation settings on AIX ? > Currently we use XLC 12.1 to compile JDK on AIX . > > However XLC 12.1 does not support the "-qvisibility=hidden" setting > currently set on AIX. > It was introduced with XLC 13.1 . Christoph found some info about it here : > > https://www.ibm.com/developerworks/aix/library/au-aix-symbol-visibility- > part2/index.html > > Setting it only generates hundreds of warnings in the build log , warnings > look like this : > XlC12.1 > > bash-4.4$ xlC -qversion > IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72) > Version: 12.01.0000.0019 > > bash-4.4$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc > 1506-173 (W) Option visibility=hidden is not valid. Enter xlC for list of valid > options. > > Compare to XLC13.1 > > bash-3.00$ xlC -qversion > IBM XL C/C++ for AIX, V13.1 (5725-C72, 5765-J07) > Version: 13.01.0000.0008 > bash-3.00$ xlC -qvisibility=default sizeof.c -o sizeof_aixxlc > bash-3.00$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc > > > So it is better to avoid setting these flags when using xlc12.1 . > Please review : > > Bug : > > https://urldefense.proofpoint.com/v2/url?u=https- > 3A__bugs.openjdk.java.net_browse_JDK- > 2D8202322&d=DwIFAg&c=jf_iaSHvJObTbx- > siA1ZOg&r=KUVGEwJiRVpNtQ9wUhGP6BKqzSTV1OWX31WWPdQMmqg&m > =F6_aATE7bsRWZG5lPZhJUESHNGqST5MNbkULllIH8l4&s=M4IJ- > YdB3dfN7lj0DEWozQbipDXmgKtSu3pEyMcJF_E&e= > > Change : > > https://urldefense.proofpoint.com/v2/url?u=http- > 3A__cr.openjdk.java.net_- > 7Embaesken_webrevs_8202322_&d=DwIFAg&c=jf_iaSHvJObTbx- > siA1ZOg&r=KUVGEwJiRVpNtQ9wUhGP6BKqzSTV1OWX31WWPdQMmqg&m > =F6_aATE7bsRWZG5lPZhJUESHNGqST5MNbkULllIH8l4&s=gb0OP9fQTAWWJb > BZDK6L_0gTKf1pwna8aXaYr_uVv8Q&e= > > > Best regards, Matthias > > From christoph.langer at sap.com Thu Apr 26 14:38:04 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 26 Apr 2018 14:38:04 +0000 Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 In-Reply-To: <8361024dba4f44309292e73ec9ad7b83@sap.com> References: <8361024dba4f44309292e73ec9ad7b83@sap.com> Message-ID: Hi Matthias, to me the change in principal looks good. I'm wondering if it is possible to do a comparison like xlc < 13 (e.g. extract major number before the first dot, then compare numerically) - but maybe it is too complicated and the current single version compare suits our needs ? Best regards Christoph From: Baesken, Matthias Sent: Donnerstag, 26. April 2018 16:14 To: 'build-dev at openjdk.java.net' ; ppc-aix-port-dev at openjdk.java.net; core-libs-dev at openjdk.java.net Cc: Langer, Christoph ; Simonis, Volker Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 Hello , could you please review this small adjustment to the symbol visibility compilation settings on AIX ? Currently we use XLC 12.1 to compile JDK on AIX . However XLC 12.1 does not support the "-qvisibility=hidden" setting currently set on AIX. It was introduced with XLC 13.1 . Christoph found some info about it here : https://www.ibm.com/developerworks/aix/library/au-aix-symbol-visibility-part2/index.html Setting it only generates hundreds of warnings in the build log , warnings look like this : XlC12.1 bash-4.4$ xlC -qversion IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72) Version: 12.01.0000.0019 bash-4.4$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc 1506-173 (W) Option visibility=hidden is not valid. Enter xlC for list of valid options. Compare to XLC13.1 bash-3.00$ xlC -qversion IBM XL C/C++ for AIX, V13.1 (5725-C72, 5765-J07) Version: 13.01.0000.0008 bash-3.00$ xlC -qvisibility=default sizeof.c -o sizeof_aixxlc bash-3.00$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc So it is better to avoid setting these flags when using xlc12.1 . Please review : Bug : https://bugs.openjdk.java.net/browse/JDK-8202322 Change : http://cr.openjdk.java.net/~mbaesken/webrevs/8202322/ Best regards, Matthias -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthias.baesken at sap.com Thu Apr 26 14:52:34 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 26 Apr 2018 14:52:34 +0000 Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 In-Reply-To: References: <8361024dba4f44309292e73ec9ad7b83@sap.com> Message-ID: <97c60ab29a6348769169b312f55832e7@sap.com> Hello Christoph, I think all XLC versions < 12.1 are unsupported (and probably not working anyway) for the OpenJDK build . I am only aware of XLC versions 12.1 and 13.1 which work / in case of 13.1 "might" work for OpenJDK compilation . And for 12.1 I want to remove the flags . ( waiting for the feedback of Bhaktavatsal Reddy , in case he prefers it I remove them for all xlC versions including 13.1 ) Best regards, Matthias From: Langer, Christoph Sent: Donnerstag, 26. April 2018 16:38 To: Baesken, Matthias ; 'build-dev at openjdk.java.net' ; ppc-aix-port-dev at openjdk.java.net; core-libs-dev at openjdk.java.net Cc: Simonis, Volker Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 Hi Matthias, to me the change in principal looks good. I'm wondering if it is possible to do a comparison like xlc < 13 (e.g. extract major number before the first dot, then compare numerically) - but maybe it is too complicated and the current single version compare suits our needs ? Best regards Christoph From: Baesken, Matthias Sent: Donnerstag, 26. April 2018 16:14 To: 'build-dev at openjdk.java.net' >; ppc-aix-port-dev at openjdk.java.net; core-libs-dev at openjdk.java.net Cc: Langer, Christoph >; Simonis, Volker > Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 Hello , could you please review this small adjustment to the symbol visibility compilation settings on AIX ? Currently we use XLC 12.1 to compile JDK on AIX . However XLC 12.1 does not support the "-qvisibility=hidden" setting currently set on AIX. It was introduced with XLC 13.1 . Christoph found some info about it here : https://www.ibm.com/developerworks/aix/library/au-aix-symbol-visibility-part2/index.html Setting it only generates hundreds of warnings in the build log , warnings look like this : XlC12.1 bash-4.4$ xlC -qversion IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72) Version: 12.01.0000.0019 bash-4.4$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc 1506-173 (W) Option visibility=hidden is not valid. Enter xlC for list of valid options. Compare to XLC13.1 bash-3.00$ xlC -qversion IBM XL C/C++ for AIX, V13.1 (5725-C72, 5765-J07) Version: 13.01.0000.0008 bash-3.00$ xlC -qvisibility=default sizeof.c -o sizeof_aixxlc bash-3.00$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc So it is better to avoid setting these flags when using xlc12.1 . Please review : Bug : https://bugs.openjdk.java.net/browse/JDK-8202322 Change : http://cr.openjdk.java.net/~mbaesken/webrevs/8202322/ Best regards, Matthias -------------- next part -------------- An HTML attachment was scrubbed... URL: From bhamaram at in.ibm.com Thu Apr 26 14:59:14 2018 From: bhamaram at in.ibm.com (Bhaktavatsal R Maram) Date: Thu, 26 Apr 2018 14:59:14 +0000 Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 In-Reply-To: <97c60ab29a6348769169b312f55832e7@sap.com> References: <97c60ab29a6348769169b312f55832e7@sap.com>, <8361024dba4f44309292e73ec9ad7b83@sap.com> Message-ID: Hi Matthias, At this point, I think we can remove the flag as you found that it is not supported in XLC < 13. And with XLC 13, it require more work to use this flag. Thanks, Bhaktavatsal Reddy -----"Baesken, Matthias" wrote: ----- To: "Langer, Christoph" , "'build-dev at openjdk.java.net'" , "ppc-aix-port-dev at openjdk.java.net" , "core-libs-dev at openjdk.java.net" From: "Baesken, Matthias" Date: 04/26/2018 08:23PM Cc: "Simonis, Volker" , Bhaktavatsal R Maram Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 Hello Christoph, I think all XLC versions < 12.1 are unsupported (and probably not working anyway) for the OpenJDK build . I am only aware of XLC versions 12.1 and 13.1 which work / in case of 13.1 ?might? work for OpenJDK compilation . And for 12.1 I want to remove the flags . ( waiting for the feedback of Bhaktavatsal Reddy , in case he prefers it I remove them for all xlC versions including 13.1 ) Best regards, Matthias From: Langer, Christoph Sent: Donnerstag, 26. April 2018 16:38 To: Baesken, Matthias ; 'build-dev at openjdk.java.net' ; ppc-aix-port-dev at openjdk.java.net; core-libs-dev at openjdk.java.net Cc: Simonis, Volker Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 Hi Matthias, to me the change in principal looks good. I?m wondering if it is possible to do a comparison like xlc < 13 (e.g. extract major number before the first dot, then compare numerically) ? but maybe it is too complicated and the current single version compare suits our needs ? Best regards Christoph From: Baesken, Matthias Sent: Donnerstag, 26. April 2018 16:14 To: 'build-dev at openjdk.java.net' ; ppc-aix-port-dev at openjdk.java.net; core-libs-dev at openjdk.java.net Cc: Langer, Christoph ; Simonis, Volker Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 Hello , could you please review this small adjustment to the symbol visibility compilation settings on AIX ? Currently we use XLC 12.1 to compile JDK on AIX . However XLC 12.1 does not support the ?-qvisibility=hidden? setting currently set on AIX. It was introduced with XLC 13.1 . Christoph found some info about it here : https://www.ibm.com/developerworks/aix/library/au-aix-symbol-visibility-part2/index.html Setting it only generates hundreds of warnings in the build log , warnings look like this : XlC12.1 bash-4.4$ xlC -qversion IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72) Version: 12.01.0000.0019 bash-4.4$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc 1506-173 (W) Option visibility=hidden is not valid. Enter xlC for list of valid options. Compare to XLC13.1 bash-3.00$ xlC -qversion IBM XL C/C++ for AIX, V13.1 (5725-C72, 5765-J07) Version: 13.01.0000.0008 bash-3.00$ xlC -qvisibility=default sizeof.c -o sizeof_aixxlc bash-3.00$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc So it is better to avoid setting these flags when using xlc12.1 . Please review : Bug : https://bugs.openjdk.java.net/browse/JDK-8202322 Change : http://cr.openjdk.java.net/~mbaesken/webrevs/8202322/ Best regards, Matthias From volker.simonis at gmail.com Thu Apr 26 15:16:45 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 26 Apr 2018 17:16:45 +0200 Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 In-Reply-To: References: <8361024dba4f44309292e73ec9ad7b83@sap.com> <97c60ab29a6348769169b312f55832e7@sap.com> Message-ID: Hi Matthias, after Bhaktavatsal Reddy's report about the problems with "-qvisibility" with xlC 13 and taking into account that we can't test this anyway because we don't currently have xlC 13 on our machines I think it would be best to completely remove this option for now on AIX. Once we get xlC 13 we can revisit the issue. Thanks, Volker On Thu, Apr 26, 2018 at 4:59 PM, Bhaktavatsal R Maram wrote: > Hi Matthias, > > At this point, I think we can remove the flag as you found that it is not supported in XLC < 13. And with XLC 13, it require more work to use this flag. > > Thanks, > Bhaktavatsal Reddy > > > > -----"Baesken, Matthias" wrote: ----- > To: "Langer, Christoph" , "'build-dev at openjdk.java.net'" , "ppc-aix-port-dev at openjdk.java.net" , "core-libs-dev at openjdk.java.net" > From: "Baesken, Matthias" > Date: 04/26/2018 08:23PM > Cc: "Simonis, Volker" , Bhaktavatsal R Maram > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 > > > Hello Christoph, I think all XLC versions < 12.1 are unsupported (and probably not working anyway) for the OpenJDK build . > I am only aware of XLC versions 12.1 and 13.1 which work / in case of 13.1 ?might? work for OpenJDK compilation . > And for 12.1 I want to remove the flags . > > ( waiting for the feedback of Bhaktavatsal Reddy , in case he prefers it I remove them for all xlC versions including 13.1 ) > > Best regards, Matthias > > > > > > > From: Langer, Christoph > Sent: Donnerstag, 26. April 2018 16:38 > To: Baesken, Matthias ; 'build-dev at openjdk.java.net' ; ppc-aix-port-dev at openjdk.java.net; core-libs-dev at openjdk.java.net > Cc: Simonis, Volker > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 > > Hi Matthias, > > to me the change in principal looks good. > > I?m wondering if it is possible to do a comparison like xlc < 13 (e.g. extract major number before the first dot, then compare numerically) ? but maybe it is too complicated and the current single version compare suits our needs ? > > Best regards > Christoph > > > > > From: Baesken, Matthias > Sent: Donnerstag, 26. April 2018 16:14 > To: 'build-dev at openjdk.java.net' ; ppc-aix-port-dev at openjdk.java.net; core-libs-dev at openjdk.java.net > Cc: Langer, Christoph ; Simonis, Volker > Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 > > Hello , could you please review this small adjustment to the symbol visibility compilation settings on AIX ? > Currently we use XLC 12.1 to compile JDK on AIX . > > However XLC 12.1 does not support the ?-qvisibility=hidden? setting currently set on AIX. > It was introduced with XLC 13.1 . Christoph found some info about it here : > > https://www.ibm.com/developerworks/aix/library/au-aix-symbol-visibility-part2/index.html > > Setting it only generates hundreds of warnings in the build log , warnings look like this : > XlC12.1 > > bash-4.4$ xlC -qversion > IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72) > Version: 12.01.0000.0019 > > bash-4.4$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc > 1506-173 (W) Option visibility=hidden is not valid. Enter xlC for list of valid options. > > Compare to XLC13.1 > > bash-3.00$ xlC -qversion > IBM XL C/C++ for AIX, V13.1 (5725-C72, 5765-J07) > Version: 13.01.0000.0008 > bash-3.00$ xlC -qvisibility=default sizeof.c -o sizeof_aixxlc > bash-3.00$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc > > > So it is better to avoid setting these flags when using xlc12.1 . > Please review : > > Bug : > > https://bugs.openjdk.java.net/browse/JDK-8202322 > > Change : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202322/ > > > Best regards, Matthias > > > > From erik.joelsson at oracle.com Thu Apr 26 16:09:36 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Thu, 26 Apr 2018 09:09:36 -0700 Subject: RFR(xs): 8202325: [aix] disable warnings-as-errors by default In-Reply-To: References: Message-ID: Looks good. /Erik On 2018-04-26 07:03, Thomas St?fe wrote: > Hi all, > > may I have reviews please: > > https://bugs.openjdk.java.net/browse/JDK-8202325 > http://cr.openjdk.java.net/~stuefe/webrevs/8202325-aix-disable-warnings-as-errors/webrev.00/webrev/ > > We decided to disable warnings as errors by default on AIX. This is a > pragmatic decision - we will never be able to fix all the many xlC > warnings and the build since long only worked when disabling > warnings-as-errors. > > So, might as well make this the default. > > Best regards, Thomas From erik.joelsson at oracle.com Thu Apr 26 16:11:28 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Thu, 26 Apr 2018 09:11:28 -0700 Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 In-Reply-To: <8361024dba4f44309292e73ec9ad7b83@sap.com> References: <8361024dba4f44309292e73ec9ad7b83@sap.com> Message-ID: <9637d378-1e49-df96-ccc7-b01efcbc9929@oracle.com> Looks ok. /Erik On 2018-04-26 07:13, Baesken, Matthias wrote: > Hello , could you please review this small adjustment to the symbol visibility compilation settings on AIX ? > Currently we use XLC 12.1 to compile JDK on AIX . > > However XLC 12.1 does not support the "-qvisibility=hidden" setting currently set on AIX. > It was introduced with XLC 13.1 . Christoph found some info about it here : > > https://www.ibm.com/developerworks/aix/library/au-aix-symbol-visibility-part2/index.html > > Setting it only generates hundreds of warnings in the build log , warnings look like this : > XlC12.1 > > bash-4.4$ xlC -qversion > IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72) > Version: 12.01.0000.0019 > > bash-4.4$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc > 1506-173 (W) Option visibility=hidden is not valid. Enter xlC for list of valid options. > > Compare to XLC13.1 > > bash-3.00$ xlC -qversion > IBM XL C/C++ for AIX, V13.1 (5725-C72, 5765-J07) > Version: 13.01.0000.0008 > bash-3.00$ xlC -qvisibility=default sizeof.c -o sizeof_aixxlc > bash-3.00$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc > > > So it is better to avoid setting these flags when using xlc12.1 . > Please review : > > Bug : > > https://bugs.openjdk.java.net/browse/JDK-8202322 > > Change : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202322/ > > > Best regards, Matthias > > From thomas.stuefe at gmail.com Thu Apr 26 17:02:55 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 26 Apr 2018 17:02:55 +0000 Subject: RFR(xs): 8202325: [aix] disable warnings-as-errors by default In-Reply-To: References: Message-ID: Thanks Erik! On Thu, Apr 26, 2018, 18:09 Erik Joelsson wrote: > Looks good. > > /Erik > > > On 2018-04-26 07:03, Thomas St?fe wrote: > > Hi all, > > > > may I have reviews please: > > > > https://bugs.openjdk.java.net/browse/JDK-8202325 > > > http://cr.openjdk.java.net/~stuefe/webrevs/8202325-aix-disable-warnings-as-errors/webrev.00/webrev/ > > > > We decided to disable warnings as errors by default on AIX. This is a > > pragmatic decision - we will never be able to fix all the many xlC > > warnings and the build since long only worked when disabling > > warnings-as-errors. > > > > So, might as well make this the default. > > > > Best regards, Thomas > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.stuefe at gmail.com Thu Apr 26 17:03:17 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 26 Apr 2018 17:03:17 +0000 Subject: RFR(xs): 8202325: [aix] disable warnings-as-errors by default In-Reply-To: <3e3b9ca8c23b484381190901ed6af52f@sap.com> References: <3e3b9ca8c23b484381190901ed6af52f@sap.com> Message-ID: Thank you Goetz! On Thu, Apr 26, 2018, 16:13 Lindenmaier, Goetz wrote: > Hi Thomas, > > this looks good and will simplify building on AIX. > > Thanks, > Goetz. > > > -----Original Message----- > > From: ppc-aix-port-dev [mailto:ppc-aix-port-dev- > > bounces at openjdk.java.net] On Behalf Of Thomas St?fe > > Sent: Donnerstag, 26. April 2018 16:04 > > To: build-dev ; ppc-aix-port- > > dev at openjdk.java.net > > Subject: RFR(xs): 8202325: [aix] disable warnings-as-errors by default > > > > Hi all, > > > > may I have reviews please: > > > > https://bugs.openjdk.java.net/browse/JDK-8202325 > > http://cr.openjdk.java.net/~stuefe/webrevs/8202325-aix-disable-warnings- > > as-errors/webrev.00/webrev/ > > > > We decided to disable warnings as errors by default on AIX. This is a > > pragmatic decision - we will never be able to fix all the many xlC > > warnings and the build since long only worked when disabling > > warnings-as-errors. > > > > So, might as well make this the default. > > > > Best regards, Thomas > -------------- next part -------------- An HTML attachment was scrubbed... URL: From magnus.ihse.bursie at oracle.com Thu Apr 26 17:26:59 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Thu, 26 Apr 2018 19:26:59 +0200 Subject: RFR(xs): 8202325: [aix] disable warnings-as-errors by default In-Reply-To: References: Message-ID: <6616C64E-7C1B-4614-929C-4D270F2FC724@oracle.com> Sounds reasonable. The code look good but the comment "Do not change default value." seems to be more of a comment on the removed code, so I think it just looks confusing without adding anything of real value in the resulting code. Maybe rephrase "default is already set" or so, or just delete? /Magnus > 26 apr. 2018 kl. 16:03 skrev Thomas St?fe : > > Hi all, > > may I have reviews please: > > https://bugs.openjdk.java.net/browse/JDK-8202325 > http://cr.openjdk.java.net/~stuefe/webrevs/8202325-aix-disable-warnings-as-errors/webrev.00/webrev/ > > We decided to disable warnings as errors by default on AIX. This is a > pragmatic decision - we will never be able to fix all the many xlC > warnings and the build since long only worked when disabling > warnings-as-errors. > > So, might as well make this the default. > > Best regards, Thomas From thomas.stuefe at gmail.com Thu Apr 26 18:51:36 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 26 Apr 2018 20:51:36 +0200 Subject: RFR(xs): 8202325: [aix] disable warnings-as-errors by default In-Reply-To: <6616C64E-7C1B-4614-929C-4D270F2FC724@oracle.com> References: <6616C64E-7C1B-4614-929C-4D270F2FC724@oracle.com> Message-ID: Hi Magnus, I think I'll just delete the comment. Thanks for the review! Thanks, Thomas On Thu, Apr 26, 2018 at 7:26 PM, Magnus Ihse Bursie wrote: > Sounds reasonable. > > The code look good but the comment "Do not change default value." seems to be more of a comment on the removed code, so I think it just looks confusing without adding anything of real value in the resulting code. Maybe rephrase "default is already set" or so, or just delete? > > /Magnus > >> 26 apr. 2018 kl. 16:03 skrev Thomas St?fe : >> >> Hi all, >> >> may I have reviews please: >> >> https://bugs.openjdk.java.net/browse/JDK-8202325 >> http://cr.openjdk.java.net/~stuefe/webrevs/8202325-aix-disable-warnings-as-errors/webrev.00/webrev/ >> >> We decided to disable warnings as errors by default on AIX. This is a >> pragmatic decision - we will never be able to fix all the many xlC >> warnings and the build since long only worked when disabling >> warnings-as-errors. >> >> So, might as well make this the default. >> >> Best regards, Thomas > From kim.barrett at oracle.com Thu Apr 26 21:03:16 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 26 Apr 2018 17:03:16 -0400 Subject: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64 In-Reply-To: <5625a595-1165-8d48-afbd-8229cdc4ac07@oracle.com> References: <5625a595-1165-8d48-afbd-8229cdc4ac07@oracle.com> Message-ID: > On Apr 25, 2018, at 8:45 AM, David Holmes wrote: > > Hi Michihiro, > > On 23/04/2018 8:33 PM, Michihiro Horie wrote: >> Dear all, >> I would like to ask reviews on 8154736 ?enhancement of cmpxchg and >> copy_to_survivor?. The change adds options to avoid expensive syncs with >> compare-and-exchange. An experiment by using SPECjbb2015 showed 6% >> improvement in critical-jOPS. This change focuses on ppc64 but would be >> potentially beneficial for aarch64. >> Although discussions stopped at >> http://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/2016-October/002718.html >> , I would like to restart the review by taking over Hiroshi's work if the >> discussion is still open. > > So the very last comment there was about not implicitly assuming memory_order_consume, yet that has not been addressed in the proposal. > > Further the discussion on hotspot-runtime-dev through September and October was far more illuminating. I think my post here: > > http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2016-October/021617.html > > and the closely following one from Thomas Schatzl summed up the concerns about the proposed changes. > > AFAICS the restarted proposal addresses none of those concerns but simply takes up where the previous implementation suggestion left off. > > This is a proposal to change the memory ordering semantics of part of the shared GC code _not_ just the PPC64 implementation, but I have seen no analysis to demonstrate the correctness of such a proposal. I agree with David here. So far we've seen no such analysis. All we have seen is a series of proposed changes and non-failing test results, all of which have then been shown to have holes. (Among other things, this suggests the set of tests being applied is inadequate.) Part of the author's job is to convince reviewers that the proposed change is correct. I'm not expecting a formal proof, but I am expecting a lot more than has been provided so far. In this latest proposal, the conditional acquire doesn't look right to me. If the logging is disabled so there is no acquire, the object is then returned to the callers, who might do what with it? Is that safe? For all callers? And is it stable through future maintenance? This is not to say that I think making those acquires unconditional is sufficient. From thomas.stuefe at gmail.com Fri Apr 27 04:55:25 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 27 Apr 2018 06:55:25 +0200 Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 In-Reply-To: References: <8361024dba4f44309292e73ec9ad7b83@sap.com> <97c60ab29a6348769169b312f55832e7@sap.com> Message-ID: Hi, This was added by "8200178: Remove mapfiles for JDK native libraries". But if the flag is not accepted, what is the default behavior? Do we now export everything? I'd like to understand this first before removing the flag to get rid of the warnings. Best Regards, Thomas On Thu, Apr 26, 2018 at 5:16 PM, Volker Simonis wrote: > Hi Matthias, > > after Bhaktavatsal Reddy's report about the problems with > "-qvisibility" with xlC 13 and taking into account that we can't test > this anyway because we don't currently have xlC 13 > on our machines I think it would be best to completely remove this > option for now on AIX. Once we get xlC 13 we can revisit the issue. > > Thanks, > Volker > > > On Thu, Apr 26, 2018 at 4:59 PM, Bhaktavatsal R Maram > wrote: >> Hi Matthias, >> >> At this point, I think we can remove the flag as you found that it is not supported in XLC < 13. And with XLC 13, it require more work to use this flag. >> >> Thanks, >> Bhaktavatsal Reddy >> >> >> >> -----"Baesken, Matthias" wrote: ----- >> To: "Langer, Christoph" , "'build-dev at openjdk.java.net'" , "ppc-aix-port-dev at openjdk.java.net" , "core-libs-dev at openjdk.java.net" >> From: "Baesken, Matthias" >> Date: 04/26/2018 08:23PM >> Cc: "Simonis, Volker" , Bhaktavatsal R Maram >> Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 >> >> >> Hello Christoph, I think all XLC versions < 12.1 are unsupported (and probably not working anyway) for the OpenJDK build . >> I am only aware of XLC versions 12.1 and 13.1 which work / in case of 13.1 ?might? work for OpenJDK compilation . >> And for 12.1 I want to remove the flags . >> >> ( waiting for the feedback of Bhaktavatsal Reddy , in case he prefers it I remove them for all xlC versions including 13.1 ) >> >> Best regards, Matthias >> >> >> >> >> >> >> From: Langer, Christoph >> Sent: Donnerstag, 26. April 2018 16:38 >> To: Baesken, Matthias ; 'build-dev at openjdk.java.net' ; ppc-aix-port-dev at openjdk.java.net; core-libs-dev at openjdk.java.net >> Cc: Simonis, Volker >> Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 >> >> Hi Matthias, >> >> to me the change in principal looks good. >> >> I?m wondering if it is possible to do a comparison like xlc < 13 (e.g. extract major number before the first dot, then compare numerically) ? but maybe it is too complicated and the current single version compare suits our needs ? >> >> Best regards >> Christoph >> >> >> >> >> From: Baesken, Matthias >> Sent: Donnerstag, 26. April 2018 16:14 >> To: 'build-dev at openjdk.java.net' ; ppc-aix-port-dev at openjdk.java.net; core-libs-dev at openjdk.java.net >> Cc: Langer, Christoph ; Simonis, Volker >> Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 >> >> Hello , could you please review this small adjustment to the symbol visibility compilation settings on AIX ? >> Currently we use XLC 12.1 to compile JDK on AIX . >> >> However XLC 12.1 does not support the ?-qvisibility=hidden? setting currently set on AIX. >> It was introduced with XLC 13.1 . Christoph found some info about it here : >> >> https://www.ibm.com/developerworks/aix/library/au-aix-symbol-visibility-part2/index.html >> >> Setting it only generates hundreds of warnings in the build log , warnings look like this : >> XlC12.1 >> >> bash-4.4$ xlC -qversion >> IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72) >> Version: 12.01.0000.0019 >> >> bash-4.4$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc >> 1506-173 (W) Option visibility=hidden is not valid. Enter xlC for list of valid options. >> >> Compare to XLC13.1 >> >> bash-3.00$ xlC -qversion >> IBM XL C/C++ for AIX, V13.1 (5725-C72, 5765-J07) >> Version: 13.01.0000.0008 >> bash-3.00$ xlC -qvisibility=default sizeof.c -o sizeof_aixxlc >> bash-3.00$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc >> >> >> So it is better to avoid setting these flags when using xlc12.1 . >> Please review : >> >> Bug : >> >> https://bugs.openjdk.java.net/browse/JDK-8202322 >> >> Change : >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8202322/ >> >> >> Best regards, Matthias >> >> >> >> From christoph.langer at sap.com Fri Apr 27 06:07:26 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Fri, 27 Apr 2018 06:07:26 +0000 Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 In-Reply-To: References: <8361024dba4f44309292e73ec9ad7b83@sap.com> <97c60ab29a6348769169b312f55832e7@sap.com> Message-ID: Hi Thomas, Look at this blog: https://www.ibm.com/developerworks/aix/library/au-aix-symbol-visibility-part2/index.html if I understand it correctly, the xlc 12 default behavior should be like what we'd expect from -qvisibility=hidden. Best regards Christoph > -----Original Message----- > From: build-dev [mailto:build-dev-bounces at openjdk.java.net] On Behalf Of > Thomas St?fe > Sent: Freitag, 27. April 2018 06:55 > To: Volker Simonis ; Baesken, Matthias > > Cc: Simonis, Volker ; ppc-aix-port- > dev at openjdk.java.net; core-libs-dev at openjdk.java.net; build- > dev at openjdk.java.net > Subject: Re: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 > > Hi, > > This was added by "8200178: Remove mapfiles for JDK native libraries". > But if the flag is not accepted, what is the default behavior? Do we > now export everything? > > I'd like to understand this first before removing the flag to get rid > of the warnings. > > Best Regards, Thomas > > On Thu, Apr 26, 2018 at 5:16 PM, Volker Simonis > wrote: > > Hi Matthias, > > > > after Bhaktavatsal Reddy's report about the problems with > > "-qvisibility" with xlC 13 and taking into account that we can't test > > this anyway because we don't currently have xlC 13 > > on our machines I think it would be best to completely remove this > > option for now on AIX. Once we get xlC 13 we can revisit the issue. > > > > Thanks, > > Volker > > > > > > On Thu, Apr 26, 2018 at 4:59 PM, Bhaktavatsal R Maram > > wrote: > >> Hi Matthias, > >> > >> At this point, I think we can remove the flag as you found that it is not > supported in XLC < 13. And with XLC 13, it require more work to use this flag. > >> > >> Thanks, > >> Bhaktavatsal Reddy > >> > >> > >> > >> -----"Baesken, Matthias" wrote: ----- > >> To: "Langer, Christoph" , "'build- > dev at openjdk.java.net'" , "ppc-aix-port- > dev at openjdk.java.net" , "core-libs- > dev at openjdk.java.net" > >> From: "Baesken, Matthias" > >> Date: 04/26/2018 08:23PM > >> Cc: "Simonis, Volker" , Bhaktavatsal R Maram > > >> Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on xlc > 12.1 > >> > >> > >> Hello Christoph, I think all XLC versions < 12.1 are unsupported (and > probably not working anyway) for the OpenJDK build . > >> I am only aware of XLC versions 12.1 and 13.1 which work / in case of > 13.1 ?might? work for OpenJDK compilation . > >> And for 12.1 I want to remove the flags . > >> > >> ( waiting for the feedback of Bhaktavatsal Reddy , in case he prefers it > I remove them for all xlC versions including 13.1 ) > >> > >> Best regards, Matthias > >> > >> > >> > >> > >> > >> > >> From: Langer, Christoph > >> Sent: Donnerstag, 26. April 2018 16:38 > >> To: Baesken, Matthias ; 'build- > dev at openjdk.java.net' ; ppc-aix-port- > dev at openjdk.java.net; core-libs-dev at openjdk.java.net > >> Cc: Simonis, Volker > >> Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on xlc > 12.1 > >> > >> Hi Matthias, > >> > >> to me the change in principal looks good. > >> > >> I?m wondering if it is possible to do a comparison like xlc < 13 (e.g. extract > major number before the first dot, then compare numerically) ? but maybe it > is too complicated and the current single version compare suits our needs ? > >> > >> Best regards > >> Christoph > >> > >> > >> > >> > >> From: Baesken, Matthias > >> Sent: Donnerstag, 26. April 2018 16:14 > >> To: 'build-dev at openjdk.java.net' ; ppc- > aix-port-dev at openjdk.java.net; core-libs-dev at openjdk.java.net > >> Cc: Langer, Christoph ; Simonis, Volker > > >> Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 > >> > >> Hello , could you please review this small adjustment to the symbol > visibility compilation settings on AIX ? > >> Currently we use XLC 12.1 to compile JDK on AIX . > >> > >> However XLC 12.1 does not support the ?-qvisibility=hidden? setting > currently set on AIX. > >> It was introduced with XLC 13.1 . Christoph found some info about it here > : > >> > >> https://www.ibm.com/developerworks/aix/library/au-aix-symbol- > visibility-part2/index.html > >> > >> Setting it only generates hundreds of warnings in the build log , > warnings look like this : > >> XlC12.1 > >> > >> bash-4.4$ xlC -qversion > >> IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72) > >> Version: 12.01.0000.0019 > >> > >> bash-4.4$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc > >> 1506-173 (W) Option visibility=hidden is not valid. Enter xlC for list of valid > options. > >> > >> Compare to XLC13.1 > >> > >> bash-3.00$ xlC -qversion > >> IBM XL C/C++ for AIX, V13.1 (5725-C72, 5765-J07) > >> Version: 13.01.0000.0008 > >> bash-3.00$ xlC -qvisibility=default sizeof.c -o sizeof_aixxlc > >> bash-3.00$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc > >> > >> > >> So it is better to avoid setting these flags when using xlc12.1 . > >> Please review : > >> > >> Bug : > >> > >> https://bugs.openjdk.java.net/browse/JDK-8202322 > >> > >> Change : > >> > >> http://cr.openjdk.java.net/~mbaesken/webrevs/8202322/ > >> > >> > >> Best regards, Matthias > >> > >> > >> > >> From thomas.stuefe at gmail.com Fri Apr 27 07:20:38 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 27 Apr 2018 09:20:38 +0200 Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 In-Reply-To: References: <8361024dba4f44309292e73ec9ad7b83@sap.com> <97c60ab29a6348769169b312f55832e7@sap.com> Message-ID: Hi Christoph On Fri, Apr 27, 2018 at 8:07 AM, Langer, Christoph wrote: > Hi Thomas, > > Look at this blog: https://www.ibm.com/developerworks/aix/library/au-aix-symbol-visibility-part2/index.html > > if I understand it correctly, the xlc 12 default behavior should be like what we'd expect from -qvisibility=hidden. > Where in this article do you read this? ..Thomas > Best regards > Christoph > >> -----Original Message----- >> From: build-dev [mailto:build-dev-bounces at openjdk.java.net] On Behalf Of >> Thomas St?fe >> Sent: Freitag, 27. April 2018 06:55 >> To: Volker Simonis ; Baesken, Matthias >> >> Cc: Simonis, Volker ; ppc-aix-port- >> dev at openjdk.java.net; core-libs-dev at openjdk.java.net; build- >> dev at openjdk.java.net >> Subject: Re: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 >> >> Hi, >> >> This was added by "8200178: Remove mapfiles for JDK native libraries". >> But if the flag is not accepted, what is the default behavior? Do we >> now export everything? >> >> I'd like to understand this first before removing the flag to get rid >> of the warnings. >> >> Best Regards, Thomas >> >> On Thu, Apr 26, 2018 at 5:16 PM, Volker Simonis >> wrote: >> > Hi Matthias, >> > >> > after Bhaktavatsal Reddy's report about the problems with >> > "-qvisibility" with xlC 13 and taking into account that we can't test >> > this anyway because we don't currently have xlC 13 >> > on our machines I think it would be best to completely remove this >> > option for now on AIX. Once we get xlC 13 we can revisit the issue. >> > >> > Thanks, >> > Volker >> > >> > >> > On Thu, Apr 26, 2018 at 4:59 PM, Bhaktavatsal R Maram >> > wrote: >> >> Hi Matthias, >> >> >> >> At this point, I think we can remove the flag as you found that it is not >> supported in XLC < 13. And with XLC 13, it require more work to use this flag. >> >> >> >> Thanks, >> >> Bhaktavatsal Reddy >> >> >> >> >> >> >> >> -----"Baesken, Matthias" wrote: ----- >> >> To: "Langer, Christoph" , "'build- >> dev at openjdk.java.net'" , "ppc-aix-port- >> dev at openjdk.java.net" , "core-libs- >> dev at openjdk.java.net" >> >> From: "Baesken, Matthias" >> >> Date: 04/26/2018 08:23PM >> >> Cc: "Simonis, Volker" , Bhaktavatsal R Maram >> >> >> Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on xlc >> 12.1 >> >> >> >> >> >> Hello Christoph, I think all XLC versions < 12.1 are unsupported (and >> probably not working anyway) for the OpenJDK build . >> >> I am only aware of XLC versions 12.1 and 13.1 which work / in case of >> 13.1 ?might? work for OpenJDK compilation . >> >> And for 12.1 I want to remove the flags . >> >> >> >> ( waiting for the feedback of Bhaktavatsal Reddy , in case he prefers it >> I remove them for all xlC versions including 13.1 ) >> >> >> >> Best regards, Matthias >> >> >> >> >> >> >> >> >> >> >> >> >> >> From: Langer, Christoph >> >> Sent: Donnerstag, 26. April 2018 16:38 >> >> To: Baesken, Matthias ; 'build- >> dev at openjdk.java.net' ; ppc-aix-port- >> dev at openjdk.java.net; core-libs-dev at openjdk.java.net >> >> Cc: Simonis, Volker >> >> Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on xlc >> 12.1 >> >> >> >> Hi Matthias, >> >> >> >> to me the change in principal looks good. >> >> >> >> I?m wondering if it is possible to do a comparison like xlc < 13 (e.g. extract >> major number before the first dot, then compare numerically) ? but maybe it >> is too complicated and the current single version compare suits our needs ? >> >> >> >> Best regards >> >> Christoph >> >> >> >> >> >> >> >> >> >> From: Baesken, Matthias >> >> Sent: Donnerstag, 26. April 2018 16:14 >> >> To: 'build-dev at openjdk.java.net' ; ppc- >> aix-port-dev at openjdk.java.net; core-libs-dev at openjdk.java.net >> >> Cc: Langer, Christoph ; Simonis, Volker >> >> >> Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 >> >> >> >> Hello , could you please review this small adjustment to the symbol >> visibility compilation settings on AIX ? >> >> Currently we use XLC 12.1 to compile JDK on AIX . >> >> >> >> However XLC 12.1 does not support the ?-qvisibility=hidden? setting >> currently set on AIX. >> >> It was introduced with XLC 13.1 . Christoph found some info about it here >> : >> >> >> >> https://www.ibm.com/developerworks/aix/library/au-aix-symbol- >> visibility-part2/index.html >> >> >> >> Setting it only generates hundreds of warnings in the build log , >> warnings look like this : >> >> XlC12.1 >> >> >> >> bash-4.4$ xlC -qversion >> >> IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72) >> >> Version: 12.01.0000.0019 >> >> >> >> bash-4.4$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc >> >> 1506-173 (W) Option visibility=hidden is not valid. Enter xlC for list of valid >> options. >> >> >> >> Compare to XLC13.1 >> >> >> >> bash-3.00$ xlC -qversion >> >> IBM XL C/C++ for AIX, V13.1 (5725-C72, 5765-J07) >> >> Version: 13.01.0000.0008 >> >> bash-3.00$ xlC -qvisibility=default sizeof.c -o sizeof_aixxlc >> >> bash-3.00$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc >> >> >> >> >> >> So it is better to avoid setting these flags when using xlc12.1 . >> >> Please review : >> >> >> >> Bug : >> >> >> >> https://bugs.openjdk.java.net/browse/JDK-8202322 >> >> >> >> Change : >> >> >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8202322/ >> >> >> >> >> >> Best regards, Matthias >> >> >> >> >> >> >> >> From christoph.langer at sap.com Fri Apr 27 07:27:45 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Fri, 27 Apr 2018 07:27:45 +0000 Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 In-Reply-To: References: <8361024dba4f44309292e73ec9ad7b83@sap.com> <97c60ab29a6348769169b312f55832e7@sap.com> Message-ID: <216dfcffaf2a4a1ea747eb4e63890611@sap.com> Hi Thomas, let me cite one section from the article: ---------------------------------------------- Visibility attribute and backward compatibility on AIX As we know from the previous article, on AIX, symbols are not visible by default unless we export them at the linking stage, either manually or with the help of CreateExportList. However, on Linux, symbols are, by default, with export (namely default) visibility. This brings a gap between the AIX visibility attribute and the GNU visibility attribute. To be backward compatible, on AIX, XL C/C++ would not set all the symbols to be exported like Linux. It might consider symbol without any visibility setting to be an unspecific visibility, which aligns with an old AIX implementation. For such symbols, AIX compiler, linker, and related tools would handle it as before. However, unspecific visibility does not mean that the symbol is internal or invisible at all. It is just a visibility that is specially designed to keep the compatibility. ... ---------------------------------------------- It says in the first sentence: " As we know from the previous article, on AIX, symbols are not visible by default unless we export them at the linking stage, either manually or with the help of CreateExportList". I guess that is why I was under the impression that with xlc12 symbols would not be visible... Best regards Christoph > -----Original Message----- > From: Thomas St?fe [mailto:thomas.stuefe at gmail.com] > Sent: Freitag, 27. April 2018 09:21 > To: Langer, Christoph > Cc: Volker Simonis ; Baesken, Matthias > ; Simonis, Volker ; > ppc-aix-port-dev at openjdk.java.net; core-libs-dev at openjdk.java.net; build- > dev at openjdk.java.net > Subject: Re: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 > > Hi Christoph > > On Fri, Apr 27, 2018 at 8:07 AM, Langer, Christoph > wrote: > > Hi Thomas, > > > > Look at this blog: https://www.ibm.com/developerworks/aix/library/au- > aix-symbol-visibility-part2/index.html > > > > if I understand it correctly, the xlc 12 default behavior should be like what > we'd expect from -qvisibility=hidden. > > > > Where in this article do you read this? > > ..Thomas > > > Best regards > > Christoph > > > >> -----Original Message----- > >> From: build-dev [mailto:build-dev-bounces at openjdk.java.net] On Behalf > Of > >> Thomas St?fe > >> Sent: Freitag, 27. April 2018 06:55 > >> To: Volker Simonis ; Baesken, Matthias > >> > >> Cc: Simonis, Volker ; ppc-aix-port- > >> dev at openjdk.java.net; core-libs-dev at openjdk.java.net; build- > >> dev at openjdk.java.net > >> Subject: Re: RFR : 8202322: AIX: symbol visibility flags not support on xlc > 12.1 > >> > >> Hi, > >> > >> This was added by "8200178: Remove mapfiles for JDK native libraries". > >> But if the flag is not accepted, what is the default behavior? Do we > >> now export everything? > >> > >> I'd like to understand this first before removing the flag to get rid > >> of the warnings. > >> > >> Best Regards, Thomas > >> > >> On Thu, Apr 26, 2018 at 5:16 PM, Volker Simonis > >> wrote: > >> > Hi Matthias, > >> > > >> > after Bhaktavatsal Reddy's report about the problems with > >> > "-qvisibility" with xlC 13 and taking into account that we can't test > >> > this anyway because we don't currently have xlC 13 > >> > on our machines I think it would be best to completely remove this > >> > option for now on AIX. Once we get xlC 13 we can revisit the issue. > >> > > >> > Thanks, > >> > Volker > >> > > >> > > >> > On Thu, Apr 26, 2018 at 4:59 PM, Bhaktavatsal R Maram > >> > wrote: > >> >> Hi Matthias, > >> >> > >> >> At this point, I think we can remove the flag as you found that it is not > >> supported in XLC < 13. And with XLC 13, it require more work to use this > flag. > >> >> > >> >> Thanks, > >> >> Bhaktavatsal Reddy > >> >> > >> >> > >> >> > >> >> -----"Baesken, Matthias" wrote: ----- > >> >> To: "Langer, Christoph" , "'build- > >> dev at openjdk.java.net'" , "ppc-aix-port- > >> dev at openjdk.java.net" , "core- > libs- > >> dev at openjdk.java.net" > >> >> From: "Baesken, Matthias" > >> >> Date: 04/26/2018 08:23PM > >> >> Cc: "Simonis, Volker" , Bhaktavatsal R > Maram > >> > >> >> Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on xlc > >> 12.1 > >> >> > >> >> > >> >> Hello Christoph, I think all XLC versions < 12.1 are unsupported > (and > >> probably not working anyway) for the OpenJDK build . > >> >> I am only aware of XLC versions 12.1 and 13.1 which work / in case > of > >> 13.1 ?might? work for OpenJDK compilation . > >> >> And for 12.1 I want to remove the flags . > >> >> > >> >> ( waiting for the feedback of Bhaktavatsal Reddy , in case he prefers > it > >> I remove them for all xlC versions including 13.1 ) > >> >> > >> >> Best regards, Matthias > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> From: Langer, Christoph > >> >> Sent: Donnerstag, 26. April 2018 16:38 > >> >> To: Baesken, Matthias ; 'build- > >> dev at openjdk.java.net' ; ppc-aix-port- > >> dev at openjdk.java.net; core-libs-dev at openjdk.java.net > >> >> Cc: Simonis, Volker > >> >> Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on > xlc > >> 12.1 > >> >> > >> >> Hi Matthias, > >> >> > >> >> to me the change in principal looks good. > >> >> > >> >> I?m wondering if it is possible to do a comparison like xlc < 13 (e.g. > extract > >> major number before the first dot, then compare numerically) ? but > maybe it > >> is too complicated and the current single version compare suits our needs > ? > >> >> > >> >> Best regards > >> >> Christoph > >> >> > >> >> > >> >> > >> >> > >> >> From: Baesken, Matthias > >> >> Sent: Donnerstag, 26. April 2018 16:14 > >> >> To: 'build-dev at openjdk.java.net' ; > ppc- > >> aix-port-dev at openjdk.java.net; core-libs-dev at openjdk.java.net > >> >> Cc: Langer, Christoph ; Simonis, Volker > >> > >> >> Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc > 12.1 > >> >> > >> >> Hello , could you please review this small adjustment to the symbol > >> visibility compilation settings on AIX ? > >> >> Currently we use XLC 12.1 to compile JDK on AIX . > >> >> > >> >> However XLC 12.1 does not support the ?-qvisibility=hidden? setting > >> currently set on AIX. > >> >> It was introduced with XLC 13.1 . Christoph found some info about it > here > >> : > >> >> > >> >> https://www.ibm.com/developerworks/aix/library/au-aix-symbol- > >> visibility-part2/index.html > >> >> > >> >> Setting it only generates hundreds of warnings in the build log , > >> warnings look like this : > >> >> XlC12.1 > >> >> > >> >> bash-4.4$ xlC -qversion > >> >> IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72) > >> >> Version: 12.01.0000.0019 > >> >> > >> >> bash-4.4$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc > >> >> 1506-173 (W) Option visibility=hidden is not valid. Enter xlC for list of > valid > >> options. > >> >> > >> >> Compare to XLC13.1 > >> >> > >> >> bash-3.00$ xlC -qversion > >> >> IBM XL C/C++ for AIX, V13.1 (5725-C72, 5765-J07) > >> >> Version: 13.01.0000.0008 > >> >> bash-3.00$ xlC -qvisibility=default sizeof.c -o sizeof_aixxlc > >> >> bash-3.00$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc > >> >> > >> >> > >> >> So it is better to avoid setting these flags when using xlc12.1 . > >> >> Please review : > >> >> > >> >> Bug : > >> >> > >> >> https://bugs.openjdk.java.net/browse/JDK-8202322 > >> >> > >> >> Change : > >> >> > >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8202322/ > >> >> > >> >> > >> >> Best regards, Matthias > >> >> > >> >> > >> >> > >> >> From thomas.stuefe at gmail.com Fri Apr 27 08:04:10 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 27 Apr 2018 10:04:10 +0200 Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 In-Reply-To: <216dfcffaf2a4a1ea747eb4e63890611@sap.com> References: <8361024dba4f44309292e73ec9ad7b83@sap.com> <97c60ab29a6348769169b312f55832e7@sap.com> <216dfcffaf2a4a1ea747eb4e63890611@sap.com> Message-ID: On Fri, Apr 27, 2018 at 9:27 AM, Langer, Christoph wrote: > Hi Thomas, > > let me cite one section from the article: > > ---------------------------------------------- > > Visibility attribute and backward compatibility on AIX > > As we know from the previous article, on AIX, symbols are not visible by default unless we export them at the linking stage, either manually or with the help of CreateExportList. However, on Linux, symbols are, by default, with export (namely default) visibility. This brings a gap between the AIX visibility attribute and the GNU visibility attribute. To be backward compatible, on AIX, XL C/C++ would not set all the symbols to be exported like Linux. It might consider symbol without any visibility setting to be an unspecific visibility, which aligns with an old AIX implementation. For such symbols, AIX compiler, linker, and related tools would handle it as before. However, unspecific visibility does not mean that the symbol is internal or invisible at all. It is just a visibility that is specially designed to keep the compatibility. > > ... > > ---------------------------------------------- > > It says in the first sentence: " As we know from the previous article, on AIX, symbols are not visible by default unless we export them at the linking stage, either manually or with the help of CreateExportList". I guess that is why I was under the impression that with xlc12 symbols would not be visible... > :) Thanks for that pointer. I did read: "Consequently, as we have mentioned at the beginning of this article, if the programmer does not explicitly specify the visibility attribute for a symbol, on Linux, the visibility of the symbol could be thedefault. But on AIX, the visibility would be unspecified." So I thought, default is "unspecified", which is not hidden. I just had a look at the libjvm.so from our nightly fastdebug build, using "nm -g". I see tons of exports listed, most of which do not have the static keyword. So, I would expect them to be exported from their compilation unit, but not from the linked shared library? Am I making a thinking error here? Anyway. I do not want to hold up this patch if you guys think it is okay to ignore the compiler warning, so it is okay by me. Best Regards, Thomas > Best regards > Christoph > >> -----Original Message----- >> From: Thomas St?fe [mailto:thomas.stuefe at gmail.com] >> Sent: Freitag, 27. April 2018 09:21 >> To: Langer, Christoph >> Cc: Volker Simonis ; Baesken, Matthias >> ; Simonis, Volker ; >> ppc-aix-port-dev at openjdk.java.net; core-libs-dev at openjdk.java.net; build- >> dev at openjdk.java.net >> Subject: Re: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 >> >> Hi Christoph >> >> On Fri, Apr 27, 2018 at 8:07 AM, Langer, Christoph >> wrote: >> > Hi Thomas, >> > >> > Look at this blog: https://www.ibm.com/developerworks/aix/library/au- >> aix-symbol-visibility-part2/index.html >> > >> > if I understand it correctly, the xlc 12 default behavior should be like what >> we'd expect from -qvisibility=hidden. >> > >> >> Where in this article do you read this? >> >> ..Thomas >> >> > Best regards >> > Christoph >> > >> >> -----Original Message----- >> >> From: build-dev [mailto:build-dev-bounces at openjdk.java.net] On Behalf >> Of >> >> Thomas St?fe >> >> Sent: Freitag, 27. April 2018 06:55 >> >> To: Volker Simonis ; Baesken, Matthias >> >> >> >> Cc: Simonis, Volker ; ppc-aix-port- >> >> dev at openjdk.java.net; core-libs-dev at openjdk.java.net; build- >> >> dev at openjdk.java.net >> >> Subject: Re: RFR : 8202322: AIX: symbol visibility flags not support on xlc >> 12.1 >> >> >> >> Hi, >> >> >> >> This was added by "8200178: Remove mapfiles for JDK native libraries". >> >> But if the flag is not accepted, what is the default behavior? Do we >> >> now export everything? >> >> >> >> I'd like to understand this first before removing the flag to get rid >> >> of the warnings. >> >> >> >> Best Regards, Thomas >> >> >> >> On Thu, Apr 26, 2018 at 5:16 PM, Volker Simonis >> >> wrote: >> >> > Hi Matthias, >> >> > >> >> > after Bhaktavatsal Reddy's report about the problems with >> >> > "-qvisibility" with xlC 13 and taking into account that we can't test >> >> > this anyway because we don't currently have xlC 13 >> >> > on our machines I think it would be best to completely remove this >> >> > option for now on AIX. Once we get xlC 13 we can revisit the issue. >> >> > >> >> > Thanks, >> >> > Volker >> >> > >> >> > >> >> > On Thu, Apr 26, 2018 at 4:59 PM, Bhaktavatsal R Maram >> >> > wrote: >> >> >> Hi Matthias, >> >> >> >> >> >> At this point, I think we can remove the flag as you found that it is not >> >> supported in XLC < 13. And with XLC 13, it require more work to use this >> flag. >> >> >> >> >> >> Thanks, >> >> >> Bhaktavatsal Reddy >> >> >> >> >> >> >> >> >> >> >> >> -----"Baesken, Matthias" wrote: ----- >> >> >> To: "Langer, Christoph" , "'build- >> >> dev at openjdk.java.net'" , "ppc-aix-port- >> >> dev at openjdk.java.net" , "core- >> libs- >> >> dev at openjdk.java.net" >> >> >> From: "Baesken, Matthias" >> >> >> Date: 04/26/2018 08:23PM >> >> >> Cc: "Simonis, Volker" , Bhaktavatsal R >> Maram >> >> >> >> >> Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on xlc >> >> 12.1 >> >> >> >> >> >> >> >> >> Hello Christoph, I think all XLC versions < 12.1 are unsupported >> (and >> >> probably not working anyway) for the OpenJDK build . >> >> >> I am only aware of XLC versions 12.1 and 13.1 which work / in case >> of >> >> 13.1 ?might? work for OpenJDK compilation . >> >> >> And for 12.1 I want to remove the flags . >> >> >> >> >> >> ( waiting for the feedback of Bhaktavatsal Reddy , in case he prefers >> it >> >> I remove them for all xlC versions including 13.1 ) >> >> >> >> >> >> Best regards, Matthias >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> From: Langer, Christoph >> >> >> Sent: Donnerstag, 26. April 2018 16:38 >> >> >> To: Baesken, Matthias ; 'build- >> >> dev at openjdk.java.net' ; ppc-aix-port- >> >> dev at openjdk.java.net; core-libs-dev at openjdk.java.net >> >> >> Cc: Simonis, Volker >> >> >> Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on >> xlc >> >> 12.1 >> >> >> >> >> >> Hi Matthias, >> >> >> >> >> >> to me the change in principal looks good. >> >> >> >> >> >> I?m wondering if it is possible to do a comparison like xlc < 13 (e.g. >> extract >> >> major number before the first dot, then compare numerically) ? but >> maybe it >> >> is too complicated and the current single version compare suits our needs >> ? >> >> >> >> >> >> Best regards >> >> >> Christoph >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> From: Baesken, Matthias >> >> >> Sent: Donnerstag, 26. April 2018 16:14 >> >> >> To: 'build-dev at openjdk.java.net' ; >> ppc- >> >> aix-port-dev at openjdk.java.net; core-libs-dev at openjdk.java.net >> >> >> Cc: Langer, Christoph ; Simonis, Volker >> >> >> >> >> Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc >> 12.1 >> >> >> >> >> >> Hello , could you please review this small adjustment to the symbol >> >> visibility compilation settings on AIX ? >> >> >> Currently we use XLC 12.1 to compile JDK on AIX . >> >> >> >> >> >> However XLC 12.1 does not support the ?-qvisibility=hidden? setting >> >> currently set on AIX. >> >> >> It was introduced with XLC 13.1 . Christoph found some info about it >> here >> >> : >> >> >> >> >> >> https://www.ibm.com/developerworks/aix/library/au-aix-symbol- >> >> visibility-part2/index.html >> >> >> >> >> >> Setting it only generates hundreds of warnings in the build log , >> >> warnings look like this : >> >> >> XlC12.1 >> >> >> >> >> >> bash-4.4$ xlC -qversion >> >> >> IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72) >> >> >> Version: 12.01.0000.0019 >> >> >> >> >> >> bash-4.4$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc >> >> >> 1506-173 (W) Option visibility=hidden is not valid. Enter xlC for list of >> valid >> >> options. >> >> >> >> >> >> Compare to XLC13.1 >> >> >> >> >> >> bash-3.00$ xlC -qversion >> >> >> IBM XL C/C++ for AIX, V13.1 (5725-C72, 5765-J07) >> >> >> Version: 13.01.0000.0008 >> >> >> bash-3.00$ xlC -qvisibility=default sizeof.c -o sizeof_aixxlc >> >> >> bash-3.00$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc >> >> >> >> >> >> >> >> >> So it is better to avoid setting these flags when using xlc12.1 . >> >> >> Please review : >> >> >> >> >> >> Bug : >> >> >> >> >> >> https://bugs.openjdk.java.net/browse/JDK-8202322 >> >> >> >> >> >> Change : >> >> >> >> >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8202322/ >> >> >> >> >> >> >> >> >> Best regards, Matthias >> >> >> >> >> >> >> >> >> >> >> >> From matthias.baesken at sap.com Fri Apr 27 15:01:32 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 27 Apr 2018 15:01:32 +0000 Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 In-Reply-To: References: <8361024dba4f44309292e73ec9ad7b83@sap.com> <97c60ab29a6348769169b312f55832e7@sap.com> <216dfcffaf2a4a1ea747eb4e63890611@sap.com> Message-ID: <6f8b83fe169c4b31a9bb459593e92d1a@sap.com> > I see tons of exports listed, most of which do not have the static > keyword. So, I would expect them to be exported from their compilation > unit, but not from the linked shared library? Am I making a thinking > error here? Hi Thomas , I see "a ton" of exported symbols as well when looking into it (e.g. libjvm.so) with dump . More than one would like to have . However for now I think we should progress with the suggested patch as it is. Once we switch to xlc 13 (or some follow up release of xlc) that supports the visibility attributes ( as decribed in https://www.ibm.com/developerworks/aix/library/au-aix-symbol-visibility/index.html ) we can open a follow up item with compiler flag adjustment and adjust src/java.base/unix/native/include/jni_md.h , I think this file misses a proper JNIEXPORT definition for AIX / xlc 13.1 ). Best regards, Matthias > -----Original Message----- > From: Thomas St?fe [mailto:thomas.stuefe at gmail.com] > Sent: Freitag, 27. April 2018 10:04 > To: Langer, Christoph > Cc: Volker Simonis ; Baesken, Matthias > ; Simonis, Volker ; > ppc-aix-port-dev at openjdk.java.net; core-libs-dev at openjdk.java.net; build- > dev at openjdk.java.net > Subject: Re: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 > > On Fri, Apr 27, 2018 at 9:27 AM, Langer, Christoph > wrote: > > Hi Thomas, > > > > let me cite one section from the article: > > > > ---------------------------------------------- > > > > Visibility attribute and backward compatibility on AIX > > > > As we know from the previous article, on AIX, symbols are not visible by > default unless we export them at the linking stage, either manually or with > the help of CreateExportList. However, on Linux, symbols are, by default, > with export (namely default) visibility. This brings a gap between the AIX > visibility attribute and the GNU visibility attribute. To be backward > compatible, on AIX, XL C/C++ would not set all the symbols to be exported > like Linux. It might consider symbol without any visibility setting to be an > unspecific visibility, which aligns with an old AIX implementation. For such > symbols, AIX compiler, linker, and related tools would handle it as before. > However, unspecific visibility does not mean that the symbol is internal or > invisible at all. It is just a visibility that is specially designed to keep the > compatibility. > > > > ... > > > > ---------------------------------------------- > > > > It says in the first sentence: " As we know from the previous article, on AIX, > symbols are not visible by default unless we export them at the linking stage, > either manually or with the help of CreateExportList". I guess that is why I > was under the impression that with xlc12 symbols would not be visible... > > > > :) Thanks for that pointer. > > I did read: > > "Consequently, as we have mentioned at the beginning of this article, > if the programmer does not explicitly specify the visibility attribute > for a symbol, on Linux, the visibility of the symbol could be > thedefault. But on AIX, the visibility would be unspecified." > > So I thought, default is "unspecified", which is not hidden. > > I just had a look at the libjvm.so from our nightly fastdebug build, > using "nm -g". > > I see tons of exports listed, most of which do not have the static > keyword. So, I would expect them to be exported from their compilation > unit, but not from the linked shared library? Am I making a thinking > error here? > > Anyway. I do not want to hold up this patch if you guys think it is > okay to ignore the compiler warning, so it is okay by me. > > Best Regards, Thomas > > > > Best regards > > Christoph > > > >> -----Original Message----- > >> From: Thomas St?fe [mailto:thomas.stuefe at gmail.com] > >> Sent: Freitag, 27. April 2018 09:21 > >> To: Langer, Christoph > >> Cc: Volker Simonis ; Baesken, Matthias > >> ; Simonis, Volker > ; > >> ppc-aix-port-dev at openjdk.java.net; core-libs-dev at openjdk.java.net; > build- > >> dev at openjdk.java.net > >> Subject: Re: RFR : 8202322: AIX: symbol visibility flags not support on xlc > 12.1 > >> > >> Hi Christoph > >> > >> On Fri, Apr 27, 2018 at 8:07 AM, Langer, Christoph > >> wrote: > >> > Hi Thomas, > >> > > >> > Look at this blog: > https://www.ibm.com/developerworks/aix/library/au- > >> aix-symbol-visibility-part2/index.html > >> > > >> > if I understand it correctly, the xlc 12 default behavior should be like > what > >> we'd expect from -qvisibility=hidden. > >> > > >> > >> Where in this article do you read this? > >> > >> ..Thomas > >> > >> > Best regards > >> > Christoph > >> > > >> >> -----Original Message----- > >> >> From: build-dev [mailto:build-dev-bounces at openjdk.java.net] On > Behalf > >> Of > >> >> Thomas St?fe > >> >> Sent: Freitag, 27. April 2018 06:55 > >> >> To: Volker Simonis ; Baesken, Matthias > >> >> > >> >> Cc: Simonis, Volker ; ppc-aix-port- > >> >> dev at openjdk.java.net; core-libs-dev at openjdk.java.net; build- > >> >> dev at openjdk.java.net > >> >> Subject: Re: RFR : 8202322: AIX: symbol visibility flags not support on xlc > >> 12.1 > >> >> > >> >> Hi, > >> >> > >> >> This was added by "8200178: Remove mapfiles for JDK native libraries". > >> >> But if the flag is not accepted, what is the default behavior? Do we > >> >> now export everything? > >> >> > >> >> I'd like to understand this first before removing the flag to get rid > >> >> of the warnings. > >> >> > >> >> Best Regards, Thomas > >> >> > >> >> On Thu, Apr 26, 2018 at 5:16 PM, Volker Simonis > >> >> wrote: > >> >> > Hi Matthias, > >> >> > > >> >> > after Bhaktavatsal Reddy's report about the problems with > >> >> > "-qvisibility" with xlC 13 and taking into account that we can't test > >> >> > this anyway because we don't currently have xlC 13 > >> >> > on our machines I think it would be best to completely remove this > >> >> > option for now on AIX. Once we get xlC 13 we can revisit the issue. > >> >> > > >> >> > Thanks, > >> >> > Volker > >> >> > > >> >> > > >> >> > On Thu, Apr 26, 2018 at 4:59 PM, Bhaktavatsal R Maram > >> >> > wrote: > >> >> >> Hi Matthias, > >> >> >> > >> >> >> At this point, I think we can remove the flag as you found that it is > not > >> >> supported in XLC < 13. And with XLC 13, it require more work to use > this > >> flag. > >> >> >> > >> >> >> Thanks, > >> >> >> Bhaktavatsal Reddy > >> >> >> > >> >> >> > >> >> >> > >> >> >> -----"Baesken, Matthias" wrote: ---- > - > >> >> >> To: "Langer, Christoph" , "'build- > >> >> dev at openjdk.java.net'" , "ppc-aix- > port- > >> >> dev at openjdk.java.net" , "core- > >> libs- > >> >> dev at openjdk.java.net" > >> >> >> From: "Baesken, Matthias" > >> >> >> Date: 04/26/2018 08:23PM > >> >> >> Cc: "Simonis, Volker" , Bhaktavatsal R > >> Maram > >> >> > >> >> >> Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on > xlc > >> >> 12.1 > >> >> >> > >> >> >> > >> >> >> Hello Christoph, I think all XLC versions < 12.1 are unsupported > >> (and > >> >> probably not working anyway) for the OpenJDK build . > >> >> >> I am only aware of XLC versions 12.1 and 13.1 which work / in > case > >> of > >> >> 13.1 ?might? work for OpenJDK compilation . > >> >> >> And for 12.1 I want to remove the flags . > >> >> >> > >> >> >> ( waiting for the feedback of Bhaktavatsal Reddy , in case he > prefers > >> it > >> >> I remove them for all xlC versions including 13.1 ) > >> >> >> > >> >> >> Best regards, Matthias > >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> From: Langer, Christoph > >> >> >> Sent: Donnerstag, 26. April 2018 16:38 > >> >> >> To: Baesken, Matthias ; 'build- > >> >> dev at openjdk.java.net' ; ppc-aix-port- > >> >> dev at openjdk.java.net; core-libs-dev at openjdk.java.net > >> >> >> Cc: Simonis, Volker > >> >> >> Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support > on > >> xlc > >> >> 12.1 > >> >> >> > >> >> >> Hi Matthias, > >> >> >> > >> >> >> to me the change in principal looks good. > >> >> >> > >> >> >> I?m wondering if it is possible to do a comparison like xlc < 13 (e.g. > >> extract > >> >> major number before the first dot, then compare numerically) ? but > >> maybe it > >> >> is too complicated and the current single version compare suits our > needs > >> ? > >> >> >> > >> >> >> Best regards > >> >> >> Christoph > >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> From: Baesken, Matthias > >> >> >> Sent: Donnerstag, 26. April 2018 16:14 > >> >> >> To: 'build-dev at openjdk.java.net' ; > >> ppc- > >> >> aix-port-dev at openjdk.java.net; core-libs-dev at openjdk.java.net > >> >> >> Cc: Langer, Christoph ; Simonis, > Volker > >> >> > >> >> >> Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc > >> 12.1 > >> >> >> > >> >> >> Hello , could you please review this small adjustment to the > symbol > >> >> visibility compilation settings on AIX ? > >> >> >> Currently we use XLC 12.1 to compile JDK on AIX . > >> >> >> > >> >> >> However XLC 12.1 does not support the ?-qvisibility=hidden? > setting > >> >> currently set on AIX. > >> >> >> It was introduced with XLC 13.1 . Christoph found some info about > it > >> here > >> >> : > >> >> >> > >> >> >> https://www.ibm.com/developerworks/aix/library/au-aix-symbol- > >> >> visibility-part2/index.html > >> >> >> > >> >> >> Setting it only generates hundreds of warnings in the build log , > >> >> warnings look like this : > >> >> >> XlC12.1 > >> >> >> > >> >> >> bash-4.4$ xlC -qversion > >> >> >> IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72) > >> >> >> Version: 12.01.0000.0019 > >> >> >> > >> >> >> bash-4.4$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc > >> >> >> 1506-173 (W) Option visibility=hidden is not valid. Enter xlC for list > of > >> valid > >> >> options. > >> >> >> > >> >> >> Compare to XLC13.1 > >> >> >> > >> >> >> bash-3.00$ xlC -qversion > >> >> >> IBM XL C/C++ for AIX, V13.1 (5725-C72, 5765-J07) > >> >> >> Version: 13.01.0000.0008 > >> >> >> bash-3.00$ xlC -qvisibility=default sizeof.c -o sizeof_aixxlc > >> >> >> bash-3.00$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc > >> >> >> > >> >> >> > >> >> >> So it is better to avoid setting these flags when using xlc12.1 . > >> >> >> Please review : > >> >> >> > >> >> >> Bug : > >> >> >> > >> >> >> https://bugs.openjdk.java.net/browse/JDK-8202322 > >> >> >> > >> >> >> Change : > >> >> >> > >> >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8202322/ > >> >> >> > >> >> >> > >> >> >> Best regards, Matthias > >> >> >> > >> >> >> > >> >> >> > >> >> >> From thomas.stuefe at gmail.com Fri Apr 27 15:58:53 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 27 Apr 2018 17:58:53 +0200 Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 In-Reply-To: <6f8b83fe169c4b31a9bb459593e92d1a@sap.com> References: <8361024dba4f44309292e73ec9ad7b83@sap.com> <97c60ab29a6348769169b312f55832e7@sap.com> <216dfcffaf2a4a1ea747eb4e63890611@sap.com> <6f8b83fe169c4b31a9bb459593e92d1a@sap.com> Message-ID: On Fri, Apr 27, 2018 at 5:01 PM, Baesken, Matthias wrote: >> I see tons of exports listed, most of which do not have the static >> keyword. So, I would expect them to be exported from their compilation >> unit, but not from the linked shared library? Am I making a thinking >> error here? > > Hi Thomas , I see "a ton" of exported symbols as well when looking into it (e.g. libjvm.so) with dump . > More than one would like to have . > Okay, thanks for confirming this! I was not sure if I was using nm correctly, or if it was lying to me. > However for now I think we should progress with the suggested patch as it is. > I agree, patch is fine. Thanks for fixing this. > Once we switch to xlc 13 (or some follow up release of xlc) that supports the visibility attributes > ( as decribed in https://www.ibm.com/developerworks/aix/library/au-aix-symbol-visibility/index.html ) > we can open a follow up item with compiler flag adjustment and adjust src/java.base/unix/native/include/jni_md.h , I think this file misses a proper JNIEXPORT definition for AIX / xlc 13.1 ). > Okay, fine with me. Best Regards, Thomas > > Best regards, Matthias > > > >> -----Original Message----- >> From: Thomas St?fe [mailto:thomas.stuefe at gmail.com] >> Sent: Freitag, 27. April 2018 10:04 >> To: Langer, Christoph >> Cc: Volker Simonis ; Baesken, Matthias >> ; Simonis, Volker ; >> ppc-aix-port-dev at openjdk.java.net; core-libs-dev at openjdk.java.net; build- >> dev at openjdk.java.net >> Subject: Re: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 >> >> On Fri, Apr 27, 2018 at 9:27 AM, Langer, Christoph >> wrote: >> > Hi Thomas, >> > >> > let me cite one section from the article: >> > >> > ---------------------------------------------- >> > >> > Visibility attribute and backward compatibility on AIX >> > >> > As we know from the previous article, on AIX, symbols are not visible by >> default unless we export them at the linking stage, either manually or with >> the help of CreateExportList. However, on Linux, symbols are, by default, >> with export (namely default) visibility. This brings a gap between the AIX >> visibility attribute and the GNU visibility attribute. To be backward >> compatible, on AIX, XL C/C++ would not set all the symbols to be exported >> like Linux. It might consider symbol without any visibility setting to be an >> unspecific visibility, which aligns with an old AIX implementation. For such >> symbols, AIX compiler, linker, and related tools would handle it as before. >> However, unspecific visibility does not mean that the symbol is internal or >> invisible at all. It is just a visibility that is specially designed to keep the >> compatibility. >> > >> > ... >> > >> > ---------------------------------------------- >> > >> > It says in the first sentence: " As we know from the previous article, on AIX, >> symbols are not visible by default unless we export them at the linking stage, >> either manually or with the help of CreateExportList". I guess that is why I >> was under the impression that with xlc12 symbols would not be visible... >> > >> >> :) Thanks for that pointer. >> >> I did read: >> >> "Consequently, as we have mentioned at the beginning of this article, >> if the programmer does not explicitly specify the visibility attribute >> for a symbol, on Linux, the visibility of the symbol could be >> thedefault. But on AIX, the visibility would be unspecified." >> >> So I thought, default is "unspecified", which is not hidden. >> >> I just had a look at the libjvm.so from our nightly fastdebug build, >> using "nm -g". >> >> I see tons of exports listed, most of which do not have the static >> keyword. So, I would expect them to be exported from their compilation >> unit, but not from the linked shared library? Am I making a thinking >> error here? >> >> Anyway. I do not want to hold up this patch if you guys think it is >> okay to ignore the compiler warning, so it is okay by me. >> >> Best Regards, Thomas >> >> >> > Best regards >> > Christoph >> > >> >> -----Original Message----- >> >> From: Thomas St?fe [mailto:thomas.stuefe at gmail.com] >> >> Sent: Freitag, 27. April 2018 09:21 >> >> To: Langer, Christoph >> >> Cc: Volker Simonis ; Baesken, Matthias >> >> ; Simonis, Volker >> ; >> >> ppc-aix-port-dev at openjdk.java.net; core-libs-dev at openjdk.java.net; >> build- >> >> dev at openjdk.java.net >> >> Subject: Re: RFR : 8202322: AIX: symbol visibility flags not support on xlc >> 12.1 >> >> >> >> Hi Christoph >> >> >> >> On Fri, Apr 27, 2018 at 8:07 AM, Langer, Christoph >> >> wrote: >> >> > Hi Thomas, >> >> > >> >> > Look at this blog: >> https://www.ibm.com/developerworks/aix/library/au- >> >> aix-symbol-visibility-part2/index.html >> >> > >> >> > if I understand it correctly, the xlc 12 default behavior should be like >> what >> >> we'd expect from -qvisibility=hidden. >> >> > >> >> >> >> Where in this article do you read this? >> >> >> >> ..Thomas >> >> >> >> > Best regards >> >> > Christoph >> >> > >> >> >> -----Original Message----- >> >> >> From: build-dev [mailto:build-dev-bounces at openjdk.java.net] On >> Behalf >> >> Of >> >> >> Thomas St?fe >> >> >> Sent: Freitag, 27. April 2018 06:55 >> >> >> To: Volker Simonis ; Baesken, Matthias >> >> >> >> >> >> Cc: Simonis, Volker ; ppc-aix-port- >> >> >> dev at openjdk.java.net; core-libs-dev at openjdk.java.net; build- >> >> >> dev at openjdk.java.net >> >> >> Subject: Re: RFR : 8202322: AIX: symbol visibility flags not support on xlc >> >> 12.1 >> >> >> >> >> >> Hi, >> >> >> >> >> >> This was added by "8200178: Remove mapfiles for JDK native libraries". >> >> >> But if the flag is not accepted, what is the default behavior? Do we >> >> >> now export everything? >> >> >> >> >> >> I'd like to understand this first before removing the flag to get rid >> >> >> of the warnings. >> >> >> >> >> >> Best Regards, Thomas >> >> >> >> >> >> On Thu, Apr 26, 2018 at 5:16 PM, Volker Simonis >> >> >> wrote: >> >> >> > Hi Matthias, >> >> >> > >> >> >> > after Bhaktavatsal Reddy's report about the problems with >> >> >> > "-qvisibility" with xlC 13 and taking into account that we can't test >> >> >> > this anyway because we don't currently have xlC 13 >> >> >> > on our machines I think it would be best to completely remove this >> >> >> > option for now on AIX. Once we get xlC 13 we can revisit the issue. >> >> >> > >> >> >> > Thanks, >> >> >> > Volker >> >> >> > >> >> >> > >> >> >> > On Thu, Apr 26, 2018 at 4:59 PM, Bhaktavatsal R Maram >> >> >> > wrote: >> >> >> >> Hi Matthias, >> >> >> >> >> >> >> >> At this point, I think we can remove the flag as you found that it is >> not >> >> >> supported in XLC < 13. And with XLC 13, it require more work to use >> this >> >> flag. >> >> >> >> >> >> >> >> Thanks, >> >> >> >> Bhaktavatsal Reddy >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> -----"Baesken, Matthias" wrote: ---- >> - >> >> >> >> To: "Langer, Christoph" , "'build- >> >> >> dev at openjdk.java.net'" , "ppc-aix- >> port- >> >> >> dev at openjdk.java.net" , "core- >> >> libs- >> >> >> dev at openjdk.java.net" >> >> >> >> From: "Baesken, Matthias" >> >> >> >> Date: 04/26/2018 08:23PM >> >> >> >> Cc: "Simonis, Volker" , Bhaktavatsal R >> >> Maram >> >> >> >> >> >> >> Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on >> xlc >> >> >> 12.1 >> >> >> >> >> >> >> >> >> >> >> >> Hello Christoph, I think all XLC versions < 12.1 are unsupported >> >> (and >> >> >> probably not working anyway) for the OpenJDK build . >> >> >> >> I am only aware of XLC versions 12.1 and 13.1 which work / in >> case >> >> of >> >> >> 13.1 ?might? work for OpenJDK compilation . >> >> >> >> And for 12.1 I want to remove the flags . >> >> >> >> >> >> >> >> ( waiting for the feedback of Bhaktavatsal Reddy , in case he >> prefers >> >> it >> >> >> I remove them for all xlC versions including 13.1 ) >> >> >> >> >> >> >> >> Best regards, Matthias >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> From: Langer, Christoph >> >> >> >> Sent: Donnerstag, 26. April 2018 16:38 >> >> >> >> To: Baesken, Matthias ; 'build- >> >> >> dev at openjdk.java.net' ; ppc-aix-port- >> >> >> dev at openjdk.java.net; core-libs-dev at openjdk.java.net >> >> >> >> Cc: Simonis, Volker >> >> >> >> Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support >> on >> >> xlc >> >> >> 12.1 >> >> >> >> >> >> >> >> Hi Matthias, >> >> >> >> >> >> >> >> to me the change in principal looks good. >> >> >> >> >> >> >> >> I?m wondering if it is possible to do a comparison like xlc < 13 (e.g. >> >> extract >> >> >> major number before the first dot, then compare numerically) ? but >> >> maybe it >> >> >> is too complicated and the current single version compare suits our >> needs >> >> ? >> >> >> >> >> >> >> >> Best regards >> >> >> >> Christoph >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> From: Baesken, Matthias >> >> >> >> Sent: Donnerstag, 26. April 2018 16:14 >> >> >> >> To: 'build-dev at openjdk.java.net' ; >> >> ppc- >> >> >> aix-port-dev at openjdk.java.net; core-libs-dev at openjdk.java.net >> >> >> >> Cc: Langer, Christoph ; Simonis, >> Volker >> >> >> >> >> >> >> Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc >> >> 12.1 >> >> >> >> >> >> >> >> Hello , could you please review this small adjustment to the >> symbol >> >> >> visibility compilation settings on AIX ? >> >> >> >> Currently we use XLC 12.1 to compile JDK on AIX . >> >> >> >> >> >> >> >> However XLC 12.1 does not support the ?-qvisibility=hidden? >> setting >> >> >> currently set on AIX. >> >> >> >> It was introduced with XLC 13.1 . Christoph found some info about >> it >> >> here >> >> >> : >> >> >> >> >> >> >> >> https://www.ibm.com/developerworks/aix/library/au-aix-symbol- >> >> >> visibility-part2/index.html >> >> >> >> >> >> >> >> Setting it only generates hundreds of warnings in the build log , >> >> >> warnings look like this : >> >> >> >> XlC12.1 >> >> >> >> >> >> >> >> bash-4.4$ xlC -qversion >> >> >> >> IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72) >> >> >> >> Version: 12.01.0000.0019 >> >> >> >> >> >> >> >> bash-4.4$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc >> >> >> >> 1506-173 (W) Option visibility=hidden is not valid. Enter xlC for list >> of >> >> valid >> >> >> options. >> >> >> >> >> >> >> >> Compare to XLC13.1 >> >> >> >> >> >> >> >> bash-3.00$ xlC -qversion >> >> >> >> IBM XL C/C++ for AIX, V13.1 (5725-C72, 5765-J07) >> >> >> >> Version: 13.01.0000.0008 >> >> >> >> bash-3.00$ xlC -qvisibility=default sizeof.c -o sizeof_aixxlc >> >> >> >> bash-3.00$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc >> >> >> >> >> >> >> >> >> >> >> >> So it is better to avoid setting these flags when using xlc12.1 . >> >> >> >> Please review : >> >> >> >> >> >> >> >> Bug : >> >> >> >> >> >> >> >> https://bugs.openjdk.java.net/browse/JDK-8202322 >> >> >> >> >> >> >> >> Change : >> >> >> >> >> >> >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8202322/ >> >> >> >> >> >> >> >> >> >> >> >> Best regards, Matthias >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >>