From yasu at ysfactory.dip.jp Sat Mar 1 06:07:06 2014 From: yasu at ysfactory.dip.jp (Yasumasa Suenaga) Date: Sat, 01 Mar 2014 23:07:06 +0900 Subject: JDK-8036003: Add variable not to separate debug information. In-Reply-To: <53112019.9050504@oracle.com> References: <20140228091835.58EB8C21F24@msgw007-05.ocn.ad.jp> <53106F4A.8030905@oracle.com> <53109772.9070808@oracle.com> <5310AD70.2070908@ysfactory.dip.jp> <53112019.9050504@oracle.com> Message-ID: <5311E98A.1020004@ysfactory.dip.jp> Hi David, > 1. Generating debug symbols in the binaries (via gcc -g or whatever) > 2. Generating debuginfo files (zipped or not) (FDS) > 3. Stripping debug symbols from the binaries (strip-policy) > > It may be that we don't have clean separation between them, and if so > that should be fixed, but I don't see the current proposal as the way > forward. Currently, 1. depends 2. If FDS set to disable, debug symbols (-g) are not generated. SEPARATED_DEBUGINFO_FILES which my patch provides can separate them. > Also there may well be differences between how things are handled on the > JDK side and hotspot side. libjvm, libjsig. libsaproc are built with each makefiles in hotspot/make . Other native library is built with make/common/NativeCompilation.gmk . Additionally strip command is executed in "image" target which is defined in jdk/make/Images.gmk . This "strip" ignores STRIP_POLICY . Thus ELF binaries which are contained in JDK/JRE images are stripped when we execute "images" or "all" target. So I change: 1. Separating to add "-g" option to compiler from executing objcopy. 2. jdk/make/Images.gmk regards STRIP_POLICY. As I mentioned earlier, this issue is related to RPM. So I hope to fix it before JDK8 GA is released. Thanks, Yasumasa On 2014/03/01 8:47, David Holmes wrote: > On 1/03/2014 1:38 AM, Yasumasa Suenaga wrote: >>> The proper way to fix this is to disable FDS. >> >> Does this mean I have to pass --disable-debug-symbols to configure ? >> I've added comment to JDK-8036003, I think if we pass --disable-debug-symbols >> to configure, gcc/g++ is not passed -g option. "-g" is needed to generate >> debuginfo. > > There are three pieces to all of this: > > 1. Generating debug symbols in the binaries (via gcc -g or whatever) > 2. Generating debuginfo files (zipped or not) (FDS) > 3. Stripping debug symbols from the binaries (strip-policy) > > It may be that we don't have clean separation between them, and if so > that should be fixed, but I don't see the current proposal as the way > forward. > > Also there may well be differences between how things are handled on the > JDK side and hotspot side. > > I will try to look closer if I get a chance but my time is limited at > the moment. > > David > >> >> Thanks, >> >> Yasumasa >> >> >> On 2014/02/28 23:04, Daniel D. Daugherty wrote: >>> The proper way to fix this is to disable FDS. We should not need >>> yet another option to control debug info. >>> >>> Dan >>> >>> >>> On 2/28/14 4:13 AM, David Holmes wrote: >>>> Hi, >>>> >>>> As I put in the bug report this seems way too complicated. Seems to me >>>> all you need to do to get what you want is not use FDS and not strip the >>>> symbols from the binary. The former is trivial. The latter is more >>>> awkward as the strip policy stuff does not work as I would want it to >>>> work, but still doable. >>>> >>>> David >>>> >>>> On 28/02/2014 7:18 PM, Yasumasa Suenaga wrote: >>>>> Hi all, >>>>> >>>>> >>>>> Currently, configure script can accept --disable-debug-symbols and >>>>> --disable-zip-debug-info as controlling to generate debug information. >>>>> However, current makefiles cannot build ELF binaries which is contained >>>>> debug information with "images" target. >>>>> >>>>> Some Linux distros use RPM as package manager. >>>>> RPM is built by rpmbuild command, it strips symbols and debug information >>>>> during to generate rpm packages. >>>>> https://fedoraproject.org/wiki/Packaging:Debuginfo >>>>> >>>>> For example, OpenJDK8 in Fedora20 ships libjvm.so and libjvm.debuginfo . >>>>> libjvm.debuginfo is generated in OpenJDK's makefiles, however it does not >>>>> contain debug information. Actual debug information is shipped by OpenJDK >>>>> debuginfo package. >>>>> This packaging is important when we have to debug JVM/JNI libraries. >>>>> If we want to use debugging tools (GDB, SystemTap, etc...), they may requires >>>>> debuginfo package. Debuggee (e.g. libjvm.so) points libjvm.so.debug which is >>>>> located at sub directory in /usr/lib/debug . It is defined in ELF section >>>>> (.note.gnu.build-id) of libjvm.so . However libjvm.so.debug does not contain >>>>> valid debug information. We need to access libjvm.debuginfo.debug at same location. >>>>> >>>>> >>>>> When we think to build OpenJDK rpm packages, we have to build ELF binaries >>>>> which contain all debug information. we should not to separate debug information. >>>>> >>>>> >>>>> Thus I want to add a variable "SEPARATED_DEBUGINFO_FILES" . >>>>> If we pass "SEPARATED_DEBUGINFO_FILES=false" to make command, "make" does not >>>>> execute objcopy command for generating debuginfo binaries. >>>>> >>>>> If we build rpm packages, we also have to add "STRIP_POLICY=no_strip" to arguments >>>>> of make command. Or ELF binaries will be stripped. >>>>> >>>>> >>>>> I've uploaded webrev for this enhancement. >>>>> This is separated 3-part: top of forest, hotspot, jdk >>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8036003/ >>>>> >>>>> Please review it and sponsoring! >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> P.S. >>>>> I tried to add SEPARATED_DEBUGINFO_FILES as a configure option like >>>>> "--disable-separated-debug-info" . >>>>> I ran configure which is located at jdk9/dev directory after I ran autoconf >>>>> and common/autoconf/autogen.sh. However it failed. >>>>> >>>>> I guess this is caused by changeset as below. >>>>> JDK-8035495: Improvements in autoconf integration >>>>> http://hg.openjdk.java.net/jdk9/dev/rev/6e29cd9ac2b4 >>>>> >>>>> This changes add "CHECKME" option to configure script. However, this changes >>>>> affects "configure" only. It should change "configure.ac" . >>>>> >>> >> > From christian.tornqvist at oracle.com Sat Mar 1 10:32:39 2014 From: christian.tornqvist at oracle.com (Christian Tornqvist) Date: Sat, 1 Mar 2014 13:32:39 -0500 Subject: CHA ignores default methods during analysis leading to incorrect code generation Message-ID: <000301cf357c$a1907710$e4b16530$@oracle.com> Hi Yann, Thanks for reporting this, the bug https://bugs.openjdk.java.net/browse/JDK-8036100 is tracking this issue now. Best regards, Christian -----Original Message----- From: hotspot-dev-bounces at openjdk.java.net [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Yann Le Tallec Sent: Friday, February 28, 2014 3:28 PM To: hotspot-dev at openjdk.java.net Subject: Re.: CHA ignores default methods during analysis leading to incorrect code generation Original message: http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-January/012252.html The test (DefaultAndConcreteMethodsCHA.java) passes as expected on b129. However a post on Stackoverflow reports a very similar bug that can still be reproduced on b129: http://stackoverflow.com/q/22096052/829571 Code copied below for reference - I get the erroneous output for i ~ 260, i.e. after approximately 25 seconds. Yann public class IsolatedBug { static int i = 0; private final Box box; private final List drawables; public IsolatedBug() { this.box = new Box(); this.drawables = new ArrayList<>(); drawables.add(box); drawables.forEach(drawable -> System.out.println(++i + " " + drawable + " C=" + drawable.isShadowCaster() + "/R=" + drawable.isShadowReceiver())); } private void init() throws InterruptedException { while (true) { drawables.forEach(drawable -> System.out.println(++i + " " + drawable + " C=" + drawable.isShadowCaster() + "/R=" + drawable.isShadowReceiver())); Thread.sleep(100); } } public static void main(String[] args) throws InterruptedException { new IsolatedBug().init(); } } abstract class Drawable implements DrawableInterface {} interface DrawableInterface { default public boolean isShadowReceiver() { return false; } default public boolean isShadowCaster() { return false; } } interface ShadowDrawable extends DrawableInterface { @Override default public boolean isShadowReceiver() { return true; } @Override default public boolean isShadowCaster() { return true; } } class Box extends Drawable implements ShadowDrawable {} From mike.duigou at oracle.com Sat Mar 1 15:08:56 2014 From: mike.duigou at oracle.com (Mike Duigou) Date: Sat, 1 Mar 2014 15:08:56 -0800 Subject: JDK-8036003: Add variable not to separate debug information. In-Reply-To: <5311E98A.1020004@ysfactory.dip.jp> References: <20140228091835.58EB8C21F24@msgw007-05.ocn.ad.jp> <53106F4A.8030905@oracle.com> <53109772.9070808@oracle.com> <5310AD70.2070908@ysfactory.dip.jp> <53112019.9050504@oracle.com> <5311E98A.1020004@ysfactory.dip.jp> Message-ID: On Mar 1 2014, at 06:07 , Yasumasa Suenaga wrote: > Hi David, > >> 1. Generating debug symbols in the binaries (via gcc -g or whatever) >> 2. Generating debuginfo files (zipped or not) (FDS) >> 3. Stripping debug symbols from the binaries (strip-policy) >> >> It may be that we don't have clean separation between them, and if so >> that should be fixed, but I don't see the current proposal as the way >> forward. > > Currently, 1. depends 2. If FDS set to disable, debug symbols (-g) are not > generated. > SEPARATED_DEBUGINFO_FILES which my patch provides can separate them. I think David's list (kudos) is very helpful as I haven't seen the requirements articulated this clearly anywhere else. Some comments: - Are there important tools that cannot deal with external debuginfo files? If we can put debuginfo into external files why would we ever want unstripped binaries? Is strip policy really necessary? Strip policy would seem to only be necessary if there are tools which can't handle external debuginfo. Assuming that everything can use external debuginfo then the policy would be to strip everything. Do I understand correctly that rpmbuild can only deal with unstripped binaries and generates the stripped rpm package and debuginfo package. It sounds kind of strange that find-debuginfo.sh wouldn't be able to use already generated debuginfo files. - I would expect that the flow is something like an extended version of https://blogs.oracle.com/dbx/entry/creating_separate_debug_info : 1. Compile source files with some form of "-g" 2. Create separate debug files for object files. 3. Strip object files. 4. Add gnu_debuglink to object files. 5. Link library or executable which should carry along links to debuginfo. 6a. Debugging, testing, profiling, etc. by developers. 6b. Packaging for program bits and debuginfo. 7. Testing and Use. Hopefully I didn't get any of those steps in the wrong order. What are the "you-cant-get-there-from-here" gaps and breakdowns from implementing this process? - Whether for the FDS debug symbols or RPM debuginfo packaging it seems that the default debug level isn't quite right. I believe that in both cases what's wanted is abbreviated debug info, mostly function names and line number tables, for building stack traces. This is not the debug info that is currently generated. There are four basic alternatives for levels of debug info: 1. (-g0) No debug info whatsoever. Only exported functions and globals will have names. 2. (-g1) Abbreviated debug info. All functions have names and line number information. This seems to be what is needed by both FDS and RPM debuginfo files to produce nice stack traces. 3. (-g) Default debugging info. Adds macro expansions and local variable names. Suitable for source level debugging. 4. (-g3 or -gdwarf-4 -fvar-tracking-assignments). Full debugging info. Most suitable for source level debugging including debugging of optimized code. It is not clear that anyone would want to build the entire product with this option. Compilations are currently done with a mix of -g, -g1, and -gstabs. It would be good to rationalize this somewhat and provide a mechanism for developers to tweak generated debugging output for files or components. - Eventual alignment with http://gcc.gnu.org/wiki/DebugFission on some platforms? > So I change: > > 1. Separating to add "-g" option to compiler from executing objcopy. > 2. jdk/make/Images.gmk regards STRIP_POLICY. > > > As I mentioned earlier, this issue is related to RPM. > So I hope to fix it before JDK8 GA is released. This won't happen (at least not for Java 8u0). Java 8 is already late in it's final candidate stage. It is too late for the initial Java 8 release to consider this magnitude of change. In any event, since the Java 8 rampdown began back in November, any change would first have to be applied to Java 9 and then approved for backport to a Java 8 or an update release (and it is also possibly too late for 8u20). Inability to include your suggested change in Java 8 or 8u20 is in no way a rejection of the ideas or contribution, it's merely a normal consequence of timelines and process. Mike From yasu at ysfactory.dip.jp Sun Mar 2 06:01:10 2014 From: yasu at ysfactory.dip.jp (Yasumasa Suenaga) Date: Sun, 02 Mar 2014 23:01:10 +0900 Subject: JDK-8036003: Add variable not to separate debug information. In-Reply-To: References: <20140228091835.58EB8C21F24@msgw007-05.ocn.ad.jp> <53106F4A.8030905@oracle.com> <53109772.9070808@oracle.com> <5310AD70.2070908@ysfactory.dip.jp> <53112019.9050504@oracle.com> <5311E98A.1020004@ysfactory.dip.jp> Message-ID: <531339A6.3050806@ysfactory.dip.jp> HI Mike, > - I would expect that the flow is something like an extended version of https://blogs.oracle.com/dbx/entry/creating_separate_debug_info : > 1. Compile source files with some form of "-g" > 2. Create separate debug files for object files. > 3. Strip object files. > 4. Add gnu_debuglink to object files. > 5. Link library or executable which should carry along links to debuginfo. > 6a. Debugging, testing, profiling, etc. by developers. > 6b. Packaging for program bits and debuginfo. > 7. Testing and Use. rpmbuild will execute 2 to 4 implicitly, Modern ELF binary has .note.gnu.build-id section. Debugging tools (e.g. GDB and SA in JDK) use this section for finding debuginfo files. If OpenJDK8 which is shipped by RPM is crashed and we want to analyze core image, we have to merge debuginfo and executable at first (e.g. eu-unstrip command). GDB and SA can find valid debuginfo through .note.gnu.debug-id or .gnu_debuglink section. But these tools can find /usr/lib/debug//.debug . They are cannot find *.debuginfo which are made by makefiles in OpenJDK tree. Yasumasa On 2014/03/02 8:08, Mike Duigou wrote: > > On Mar 1 2014, at 06:07 , Yasumasa Suenaga wrote: > >> Hi David, >> >>> 1. Generating debug symbols in the binaries (via gcc -g or whatever) >>> 2. Generating debuginfo files (zipped or not) (FDS) >>> 3. Stripping debug symbols from the binaries (strip-policy) >>> >>> It may be that we don't have clean separation between them, and if so >>> that should be fixed, but I don't see the current proposal as the way >>> forward. >> >> Currently, 1. depends 2. If FDS set to disable, debug symbols (-g) are not >> generated. >> SEPARATED_DEBUGINFO_FILES which my patch provides can separate them. > > I think David's list (kudos) is very helpful as I haven't seen the requirements articulated this clearly anywhere else. > > Some comments: > > - Are there important tools that cannot deal with external debuginfo files? If we can put debuginfo into external files why would we ever want unstripped binaries? Is strip policy really necessary? Strip policy would seem to only be necessary if there are tools which can't handle external debuginfo. Assuming that everything can use external debuginfo then the policy would be to strip everything. > > Do I understand correctly that rpmbuild can only deal with unstripped binaries and generates the stripped rpm package and debuginfo package. It sounds kind of strange that find-debuginfo.sh wouldn't be able to use already generated debuginfo files. > > - I would expect that the flow is something like an extended version of https://blogs.oracle.com/dbx/entry/creating_separate_debug_info : > 1. Compile source files with some form of "-g" > 2. Create separate debug files for object files. > 3. Strip object files. > 4. Add gnu_debuglink to object files. > 5. Link library or executable which should carry along links to debuginfo. > 6a. Debugging, testing, profiling, etc. by developers. > 6b. Packaging for program bits and debuginfo. > 7. Testing and Use. > > Hopefully I didn't get any of those steps in the wrong order. What are the "you-cant-get-there-from-here" gaps and breakdowns from implementing this process? > > - Whether for the FDS debug symbols or RPM debuginfo packaging it seems that the default debug level isn't quite right. I believe that in both cases what's wanted is abbreviated debug info, mostly function names and line number tables, for building stack traces. This is not the debug info that is currently generated. > > There are four basic alternatives for levels of debug info: > 1. (-g0) No debug info whatsoever. > Only exported functions and globals will have names. > 2. (-g1) Abbreviated debug info. > All functions have names and line number information. This seems to be what is needed by both FDS and RPM debuginfo files to produce nice stack traces. > 3. (-g) Default debugging info. > Adds macro expansions and local variable names. Suitable for source level debugging. > 4. (-g3 or -gdwarf-4 -fvar-tracking-assignments). Full debugging info. > Most suitable for source level debugging including debugging of optimized code. It is not clear that anyone would want to build the entire product with this option. > > Compilations are currently done with a mix of -g, -g1, and -gstabs. It would be good to rationalize this somewhat and provide a mechanism for developers to tweak generated debugging output for files or components. > > - Eventual alignment with http://gcc.gnu.org/wiki/DebugFission on some platforms? > >> So I change: >> >> 1. Separating to add "-g" option to compiler from executing objcopy. >> 2. jdk/make/Images.gmk regards STRIP_POLICY. >> >> >> As I mentioned earlier, this issue is related to RPM. >> So I hope to fix it before JDK8 GA is released. > > This won't happen (at least not for Java 8u0). Java 8 is already late in it's final candidate stage. It is too late for the initial Java 8 release to consider this magnitude of change. In any event, since the Java 8 rampdown began back in November, any change would first have to be applied to Java 9 and then approved for backport to a Java 8 or an update release (and it is also possibly too late for 8u20). > > Inability to include your suggested change in Java 8 or 8u20 is in no way a rejection of the ideas or contribution, it's merely a normal consequence of timelines and process. > > Mike From david.holmes at oracle.com Sun Mar 2 20:39:23 2014 From: david.holmes at oracle.com (David Holmes) Date: Mon, 03 Mar 2014 14:39:23 +1000 Subject: JDK-8036003: Add variable not to separate debug information. In-Reply-To: <5311E98A.1020004@ysfactory.dip.jp> References: <20140228091835.58EB8C21F24@msgw007-05.ocn.ad.jp> <53106F4A.8030905@oracle.com> <53109772.9070808@oracle.com> <5310AD70.2070908@ysfactory.dip.jp> <53112019.9050504@oracle.com> <5311E98A.1020004@ysfactory.dip.jp> Message-ID: <5314077B.9060908@oracle.com> Hi, On 2/03/2014 12:07 AM, Yasumasa Suenaga wrote: > Hi David, > >> 1. Generating debug symbols in the binaries (via gcc -g or whatever) >> 2. Generating debuginfo files (zipped or not) (FDS) >> 3. Stripping debug symbols from the binaries (strip-policy) >> >> It may be that we don't have clean separation between them, and if so >> that should be fixed, but I don't see the current proposal as the way >> forward. > > Currently, 1. depends 2. If FDS set to disable, debug symbols (-g) are not > generated. Okay. That would seem potentially inconvenient but not really critical. You simply enable FDS (the mere existence of the debuginfo files "shouldn't" cause a problem). Then you only need to address #3 - strip-policy. > SEPARATED_DEBUGINFO_FILES which my patch provides can separate them. > > >> Also there may well be differences between how things are handled on the >> JDK side and hotspot side. > > libjvm, libjsig. libsaproc are built with each makefiles in hotspot/make . > Other native library is built with make/common/NativeCompilation.gmk . > > Additionally strip command is executed in "image" target which is defined > in jdk/make/Images.gmk . This "strip" ignores STRIP_POLICY . Thus ELF binaries > which are contained in JDK/JRE images are stripped when we execute "images" > or "all" target. strip is also applied to the JVM libraries as part of the hotspot build. What occurs in the JDK "images" is additional to that.The hotspot build honors the STRIP_POLICY. It appears that STRIP_POLICY is not considered in the new build, though it was for the old. But perhaps I'm missing it somewhere in the autoconf settings? That aside it seems that you should be able to set POST_STRIP_CMD to "" avoid stripping. Moving forward I think we would need to expose the strip-policy via a configure option and have that pass STRIP_POLICY through to both hotspot and the Images target. > > So I change: > > 1. Separating to add "-g" option to compiler from executing objcopy. > 2. jdk/make/Images.gmk regards STRIP_POLICY. > > > As I mentioned earlier, this issue is related to RPM. > So I hope to fix it before JDK8 GA is released. As Mike said there is no chance of any changes to this getting into 8 GA. But if these are your own builds I'm not sure that is an issue anyway. That said, as per the above I think you can achieve what you want by enabling FDS but set STRIP_POLICY to none, and set POST_STRIP_CMD to empty. Thanks, David > > Thanks, > > Yasumasa > > > On 2014/03/01 8:47, David Holmes wrote: >> On 1/03/2014 1:38 AM, Yasumasa Suenaga wrote: >>>> The proper way to fix this is to disable FDS. >>> >>> Does this mean I have to pass --disable-debug-symbols to configure ? >>> I've added comment to JDK-8036003, I think if we pass --disable-debug-symbols >>> to configure, gcc/g++ is not passed -g option. "-g" is needed to generate >>> debuginfo. >> >> There are three pieces to all of this: >> >> 1. Generating debug symbols in the binaries (via gcc -g or whatever) >> 2. Generating debuginfo files (zipped or not) (FDS) >> 3. Stripping debug symbols from the binaries (strip-policy) >> >> It may be that we don't have clean separation between them, and if so >> that should be fixed, but I don't see the current proposal as the way >> forward. >> >> Also there may well be differences between how things are handled on the >> JDK side and hotspot side. >> >> I will try to look closer if I get a chance but my time is limited at >> the moment. >> >> David >> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2014/02/28 23:04, Daniel D. Daugherty wrote: >>>> The proper way to fix this is to disable FDS. We should not need >>>> yet another option to control debug info. >>>> >>>> Dan >>>> >>>> >>>> On 2/28/14 4:13 AM, David Holmes wrote: >>>>> Hi, >>>>> >>>>> As I put in the bug report this seems way too complicated. Seems to me >>>>> all you need to do to get what you want is not use FDS and not strip the >>>>> symbols from the binary. The former is trivial. The latter is more >>>>> awkward as the strip policy stuff does not work as I would want it to >>>>> work, but still doable. >>>>> >>>>> David >>>>> >>>>> On 28/02/2014 7:18 PM, Yasumasa Suenaga wrote: >>>>>> Hi all, >>>>>> >>>>>> >>>>>> Currently, configure script can accept --disable-debug-symbols and >>>>>> --disable-zip-debug-info as controlling to generate debug information. >>>>>> However, current makefiles cannot build ELF binaries which is contained >>>>>> debug information with "images" target. >>>>>> >>>>>> Some Linux distros use RPM as package manager. >>>>>> RPM is built by rpmbuild command, it strips symbols and debug information >>>>>> during to generate rpm packages. >>>>>> https://fedoraproject.org/wiki/Packaging:Debuginfo >>>>>> >>>>>> For example, OpenJDK8 in Fedora20 ships libjvm.so and libjvm.debuginfo . >>>>>> libjvm.debuginfo is generated in OpenJDK's makefiles, however it does not >>>>>> contain debug information. Actual debug information is shipped by OpenJDK >>>>>> debuginfo package. >>>>>> This packaging is important when we have to debug JVM/JNI libraries. >>>>>> If we want to use debugging tools (GDB, SystemTap, etc...), they may requires >>>>>> debuginfo package. Debuggee (e.g. libjvm.so) points libjvm.so.debug which is >>>>>> located at sub directory in /usr/lib/debug . It is defined in ELF section >>>>>> (.note.gnu.build-id) of libjvm.so . However libjvm.so.debug does not contain >>>>>> valid debug information. We need to access libjvm.debuginfo.debug at same location. >>>>>> >>>>>> >>>>>> When we think to build OpenJDK rpm packages, we have to build ELF binaries >>>>>> which contain all debug information. we should not to separate debug information. >>>>>> >>>>>> >>>>>> Thus I want to add a variable "SEPARATED_DEBUGINFO_FILES" . >>>>>> If we pass "SEPARATED_DEBUGINFO_FILES=false" to make command, "make" does not >>>>>> execute objcopy command for generating debuginfo binaries. >>>>>> >>>>>> If we build rpm packages, we also have to add "STRIP_POLICY=no_strip" to arguments >>>>>> of make command. Or ELF binaries will be stripped. >>>>>> >>>>>> >>>>>> I've uploaded webrev for this enhancement. >>>>>> This is separated 3-part: top of forest, hotspot, jdk >>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8036003/ >>>>>> >>>>>> Please review it and sponsoring! >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> P.S. >>>>>> I tried to add SEPARATED_DEBUGINFO_FILES as a configure option like >>>>>> "--disable-separated-debug-info" . >>>>>> I ran configure which is located at jdk9/dev directory after I ran autoconf >>>>>> and common/autoconf/autogen.sh. However it failed. >>>>>> >>>>>> I guess this is caused by changeset as below. >>>>>> JDK-8035495: Improvements in autoconf integration >>>>>> http://hg.openjdk.java.net/jdk9/dev/rev/6e29cd9ac2b4 >>>>>> >>>>>> This changes add "CHECKME" option to configure script. However, this changes >>>>>> affects "configure" only. It should change "configure.ac" . >>>>>> >>>> >>> >> > From albert.noll at oracle.com Sun Mar 2 22:16:42 2014 From: albert.noll at oracle.com (Albert) Date: Mon, 03 Mar 2014 07:16:42 +0100 Subject: [9] RFR(S): 8035946: Use ResourceHashtable for dependency checking In-Reply-To: <5310DF7C.7040309@oracle.com> References: <5310A433.3000807@oracle.com> <5310DF7C.7040309@oracle.com> Message-ID: <53141E4A.2030407@oracle.com> Coleen, Vladimir, thanks for the reviews. Best, Albert On 02/28/2014 08:11 PM, Coleen Phillimore wrote: > > Looks good! Thanks for doing this. > > Coleen > > On 2/28/14 9:58 AM, Albert wrote: >> Hi, >> >> I send this to hotspot-dev, since the change contains a small >> modification in >> resourceHash.hpp >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8035946 >> >> Problem: >> JDK-8034839 introduced a new hashtable for caching checked >> dependencies. However, a hashtable that already existed prior to >> JDK-8034839 (ResourceHashtable) could have been used instead of >> introducing the new one. >> >> Solution: >> Use ResourceHashtable and remove the new hashtable that was >> introduced with JDK-8034839. >> >> Testing: >> jprt, failing test >> >> webrev: >> http://cr.openjdk.java.net/~anoll/8035946/webrev.00/ >> >> Many thanks in advance, >> Albert > From aph at redhat.com Mon Mar 3 01:41:33 2014 From: aph at redhat.com (Andrew Haley) Date: Mon, 03 Mar 2014 09:41:33 +0000 Subject: JDK-8036003: Add variable not to separate debug information. In-Reply-To: <20140228091835.58EB8C21F24@msgw007-05.ocn.ad.jp> References: <20140228091835.58EB8C21F24@msgw007-05.ocn.ad.jp> Message-ID: <53144E4D.8080606@redhat.com> On 02/28/2014 09:18 AM, Yasumasa Suenaga wrote: > For example, OpenJDK8 in Fedora20 ships libjvm.so and libjvm.debuginfo . > libjvm.debuginfo is generated in OpenJDK's makefiles, however it does not > contain debug information. Actual debug information is shipped by OpenJDK > debuginfo package. That's a bug in Fedora's build. We should fix it. Andrew. From aph at redhat.com Mon Mar 3 01:46:38 2014 From: aph at redhat.com (Andrew Haley) Date: Mon, 03 Mar 2014 09:46:38 +0000 Subject: JDK-8036003: Add variable not to separate debug information. In-Reply-To: References: <20140228091835.58EB8C21F24@msgw007-05.ocn.ad.jp> <53106F4A.8030905@oracle.com> <53109772.9070808@oracle.com> <5310AD70.2070908@ysfactory.dip.jp> <53112019.9050504@oracle.com> <5311E98A.1020004@ysfactory.dip.jp> Message-ID: <53144F7E.5000105@redhat.com> On 03/01/2014 11:08 PM, Mike Duigou wrote: > Do I understand correctly that rpmbuild can only deal with > unstripped binaries and generates the stripped rpm package and > debuginfo package. Exactly. OpenJDK generating separate debuginfo files is very inconvenient for RPM builds. > It sounds kind of strange that find-debuginfo.sh wouldn't be able to > use already generated debuginfo files. The OpenJDK build can't have any idea of what debuginfo files the distro runtime requires, and indeed this may change. There is more than one mechanism for finding debuginfo. Separating debuginfo is not something that we want the OpenJDK build to do, and the most useful thing that we could have is a switch to turn all of OpenJDK's stripping and separate debuginfo off. I expect the same applies to all distros. Andrew. From yasu at ysfactory.dip.jp Mon Mar 3 04:04:51 2014 From: yasu at ysfactory.dip.jp (Yasumasa Suenaga) Date: Mon, 03 Mar 2014 21:04:51 +0900 Subject: JDK-8036003: Add variable not to separate debug information. In-Reply-To: <5314077B.9060908@oracle.com> References: <20140228091835.58EB8C21F24@msgw007-05.ocn.ad.jp> <53106F4A.8030905@oracle.com> <53109772.9070808@oracle.com> <5310AD70.2070908@ysfactory.dip.jp> <53112019.9050504@oracle.com> <5311E98A.1020004@ysfactory.dip.jp> <5314077B.9060908@oracle.com> Message-ID: <53146FE3.9040205@ysfactory.dip.jp> Hi David, > Moving forward I think we would need to expose the strip-policy via a > configure option and have that pass STRIP_POLICY through to both hotspot > and the Images target. I want you to do so :-) Additionally, >>> 1. Generating debug symbols in the binaries (via gcc -g or whatever) >>> 2. Generating debuginfo files (zipped or not) (FDS) >>> 3. Stripping debug symbols from the binaries (strip-policy) I think that FDS should be separated 1 from 2. That is more useful for building OpenJDK. > That said, as per the above I think you can achieve what you > want by enabling FDS but set STRIP_POLICY to none, and set > POST_STRIP_CMD to empty. I tried: $ make images STRIP_POLICY=no_strip POST_STRIP_CMD="" I was able to get binaries which is included debug information. Thanks, Yasumasa On 2014/03/03 13:39, David Holmes wrote: > Hi, > > On 2/03/2014 12:07 AM, Yasumasa Suenaga wrote: >> Hi David, >> >>> 1. Generating debug symbols in the binaries (via gcc -g or whatever) >>> 2. Generating debuginfo files (zipped or not) (FDS) >>> 3. Stripping debug symbols from the binaries (strip-policy) >>> >>> It may be that we don't have clean separation between them, and if so >>> that should be fixed, but I don't see the current proposal as the way >>> forward. >> >> Currently, 1. depends 2. If FDS set to disable, debug symbols (-g) are not >> generated. > > Okay. That would seem potentially inconvenient but not really critical. > You simply enable FDS (the mere existence of the debuginfo files > "shouldn't" cause a problem). Then you only need to address #3 - > strip-policy. > >> SEPARATED_DEBUGINFO_FILES which my patch provides can separate them. >> >> >>> Also there may well be differences between how things are handled on the >>> JDK side and hotspot side. >> >> libjvm, libjsig. libsaproc are built with each makefiles in hotspot/make . >> Other native library is built with make/common/NativeCompilation.gmk . >> >> Additionally strip command is executed in "image" target which is defined >> in jdk/make/Images.gmk . This "strip" ignores STRIP_POLICY . Thus ELF binaries >> which are contained in JDK/JRE images are stripped when we execute "images" >> or "all" target. > > strip is also applied to the JVM libraries as part of the hotspot build. > What occurs in the JDK "images" is additional to that.The hotspot build > honors the STRIP_POLICY. > > It appears that STRIP_POLICY is not considered in the new build, though > it was for the old. But perhaps I'm missing it somewhere in the autoconf > settings? That aside it seems that you should be able to set > POST_STRIP_CMD to "" avoid stripping. > > Moving forward I think we would need to expose the strip-policy via a > configure option and have that pass STRIP_POLICY through to both hotspot > and the Images target. > >> >> So I change: >> >> 1. Separating to add "-g" option to compiler from executing objcopy. >> 2. jdk/make/Images.gmk regards STRIP_POLICY. >> >> >> As I mentioned earlier, this issue is related to RPM. >> So I hope to fix it before JDK8 GA is released. > > As Mike said there is no chance of any changes to this getting into 8 > GA. But if these are your own builds I'm not sure that is an issue > anyway. That said, as per the above I think you can achieve what you > want by enabling FDS but set STRIP_POLICY to none, and set > POST_STRIP_CMD to empty. > > Thanks, > David > >> >> Thanks, >> >> Yasumasa >> >> >> On 2014/03/01 8:47, David Holmes wrote: >>> On 1/03/2014 1:38 AM, Yasumasa Suenaga wrote: >>>>> The proper way to fix this is to disable FDS. >>>> >>>> Does this mean I have to pass --disable-debug-symbols to configure ? >>>> I've added comment to JDK-8036003, I think if we pass --disable-debug-symbols >>>> to configure, gcc/g++ is not passed -g option. "-g" is needed to generate >>>> debuginfo. >>> >>> There are three pieces to all of this: >>> >>> 1. Generating debug symbols in the binaries (via gcc -g or whatever) >>> 2. Generating debuginfo files (zipped or not) (FDS) >>> 3. Stripping debug symbols from the binaries (strip-policy) >>> >>> It may be that we don't have clean separation between them, and if so >>> that should be fixed, but I don't see the current proposal as the way >>> forward. >>> >>> Also there may well be differences between how things are handled on the >>> JDK side and hotspot side. >>> >>> I will try to look closer if I get a chance but my time is limited at >>> the moment. >>> >>> David >>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2014/02/28 23:04, Daniel D. Daugherty wrote: >>>>> The proper way to fix this is to disable FDS. We should not need >>>>> yet another option to control debug info. >>>>> >>>>> Dan >>>>> >>>>> >>>>> On 2/28/14 4:13 AM, David Holmes wrote: >>>>>> Hi, >>>>>> >>>>>> As I put in the bug report this seems way too complicated. Seems to me >>>>>> all you need to do to get what you want is not use FDS and not strip the >>>>>> symbols from the binary. The former is trivial. The latter is more >>>>>> awkward as the strip policy stuff does not work as I would want it to >>>>>> work, but still doable. >>>>>> >>>>>> David >>>>>> >>>>>> On 28/02/2014 7:18 PM, Yasumasa Suenaga wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> >>>>>>> Currently, configure script can accept --disable-debug-symbols and >>>>>>> --disable-zip-debug-info as controlling to generate debug information. >>>>>>> However, current makefiles cannot build ELF binaries which is contained >>>>>>> debug information with "images" target. >>>>>>> >>>>>>> Some Linux distros use RPM as package manager. >>>>>>> RPM is built by rpmbuild command, it strips symbols and debug information >>>>>>> during to generate rpm packages. >>>>>>> https://fedoraproject.org/wiki/Packaging:Debuginfo >>>>>>> >>>>>>> For example, OpenJDK8 in Fedora20 ships libjvm.so and libjvm.debuginfo . >>>>>>> libjvm.debuginfo is generated in OpenJDK's makefiles, however it does not >>>>>>> contain debug information. Actual debug information is shipped by OpenJDK >>>>>>> debuginfo package. >>>>>>> This packaging is important when we have to debug JVM/JNI libraries. >>>>>>> If we want to use debugging tools (GDB, SystemTap, etc...), they may requires >>>>>>> debuginfo package. Debuggee (e.g. libjvm.so) points libjvm.so.debug which is >>>>>>> located at sub directory in /usr/lib/debug . It is defined in ELF section >>>>>>> (.note.gnu.build-id) of libjvm.so . However libjvm.so.debug does not contain >>>>>>> valid debug information. We need to access libjvm.debuginfo.debug at same location. >>>>>>> >>>>>>> >>>>>>> When we think to build OpenJDK rpm packages, we have to build ELF binaries >>>>>>> which contain all debug information. we should not to separate debug information. >>>>>>> >>>>>>> >>>>>>> Thus I want to add a variable "SEPARATED_DEBUGINFO_FILES" . >>>>>>> If we pass "SEPARATED_DEBUGINFO_FILES=false" to make command, "make" does not >>>>>>> execute objcopy command for generating debuginfo binaries. >>>>>>> >>>>>>> If we build rpm packages, we also have to add "STRIP_POLICY=no_strip" to arguments >>>>>>> of make command. Or ELF binaries will be stripped. >>>>>>> >>>>>>> >>>>>>> I've uploaded webrev for this enhancement. >>>>>>> This is separated 3-part: top of forest, hotspot, jdk >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8036003/ >>>>>>> >>>>>>> Please review it and sponsoring! >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> P.S. >>>>>>> I tried to add SEPARATED_DEBUGINFO_FILES as a configure option like >>>>>>> "--disable-separated-debug-info" . >>>>>>> I ran configure which is located at jdk9/dev directory after I ran autoconf >>>>>>> and common/autoconf/autogen.sh. However it failed. >>>>>>> >>>>>>> I guess this is caused by changeset as below. >>>>>>> JDK-8035495: Improvements in autoconf integration >>>>>>> http://hg.openjdk.java.net/jdk9/dev/rev/6e29cd9ac2b4 >>>>>>> >>>>>>> This changes add "CHECKME" option to configure script. However, this changes >>>>>>> affects "configure" only. It should change "configure.ac" . >>>>>>> >>>>> >>>> >>> >> > From yasu at ysfactory.dip.jp Mon Mar 3 04:04:59 2014 From: yasu at ysfactory.dip.jp (Yasumasa Suenaga) Date: Mon, 03 Mar 2014 21:04:59 +0900 Subject: JDK-8036003: Add variable not to separate debug information. In-Reply-To: <53144F7E.5000105@redhat.com> References: <20140228091835.58EB8C21F24@msgw007-05.ocn.ad.jp> <53106F4A.8030905@oracle.com> <53109772.9070808@oracle.com> <5310AD70.2070908@ysfactory.dip.jp> <53112019.9050504@oracle.com> <5311E98A.1020004@ysfactory.dip.jp> <53144F7E.5000105@redhat.com> Message-ID: <53146FEB.8050300@ysfactory.dip.jp> Hi Andrew, > Separating debuginfo is not > something that we want the OpenJDK build to do, and the most useful > thing that we could have is a switch to turn all of OpenJDK's > stripping and separate debuginfo off. I expect the same applies to > all distros. My patch provides "SEPARATED_DEBUGINFO_FILES" as a switch of separating debuginfo files, and affects "STRIP_POILCY" in HotSpot and other ELF binaries. > I expect the same applies to all distros. We need to pass ' STRIP_POLICY=no_strip POST_STRIP_CMD="" ' to make command as David said. And we need to pick up binaries which are NOT named ".debuginfo" or ".diz" suffix. Thanks, Yasumasa On 2014/03/03 18:46, Andrew Haley wrote: > On 03/01/2014 11:08 PM, Mike Duigou wrote: > >> Do I understand correctly that rpmbuild can only deal with >> unstripped binaries and generates the stripped rpm package and >> debuginfo package. > > Exactly. OpenJDK generating separate debuginfo files is very > inconvenient for RPM builds. > >> It sounds kind of strange that find-debuginfo.sh wouldn't be able to >> use already generated debuginfo files. > > The OpenJDK build can't have any idea of what debuginfo files the > distro runtime requires, and indeed this may change. There is more > than one mechanism for finding debuginfo. Separating debuginfo is not > something that we want the OpenJDK build to do, and the most useful > thing that we could have is a switch to turn all of OpenJDK's > stripping and separate debuginfo off. I expect the same applies to > all distros. > > Andrew. > From goetz.lindenmaier at sap.com Mon Mar 3 05:22:05 2014 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 3 Mar 2014 13:22:05 +0000 Subject: RFR(S): 8036122: Fix warning 'format not a string literal' Message-ID: <4295855A5C1DE049A61835A1887419CC2CEA5701@DEWDFEMB12A.global.corp.sap> Hi, Since "8030350: Enable additional compiler warnings for GCC", the hotspot build issues warnings: src/share/vm/compiler/compilerOracle.cpp: In function 'bool scan_line(const char*, char*, MethodMatcher::Mode*, char*, MethodMatcher::Mode*, int*, const char*&)': src/share/vm/compiler/compilerOracle.cpp:425: warning: format not a string literal, argument types not checked src/os/linux/vm/os_linux.cpp: In function 'jlong slow_thread_cpu_time(Thread*, bool)': src/os/linux/vm/os_linux.cpp:5316: warning: format not a string literal, argument types not checked Also, older gccs (4.2.2 and before) can not build hotspot any more. They don't know the option introduced in that change: cc1plus: error: unrecognized command line option "-Wno-error=format-nonliteral" This change fixes both by - Fixing the warning in the hotspot code - removing the flag from the makefiles. Thus, the build with new gcc does no more issue warnings, and the build with older ones works again. Please review and test this change - and I please need a sponsor. http://cr.openjdk.java.net/~goetz/webrevs/8036122-1-fmtWrn/ Best regards, Goetz. From neugens at redhat.com Mon Mar 3 05:28:33 2014 From: neugens at redhat.com (Mario Torre) Date: Mon, 03 Mar 2014 14:28:33 +0100 Subject: Bring Zero to llvm 3.4 In-Reply-To: <1393256374.11941.3.camel@galactica.localdomain> References: <1393239625.3873.13.camel@galactica.localdomain> <1489929.lba08dEc0C@mgerdin03> <1393250766.3873.16.camel@galactica.localdomain> <1445297.1Zhykx3zvx@mgerdin03> <1393256374.11941.3.camel@galactica.localdomain> Message-ID: <1393853313.3834.6.camel@galactica.localdomain> On Mon, 2014-02-24 at 16:39 +0100, Mario Torre wrote: > On Mon, 2014-02-24 at 15:50 +0100, Mikael Gerdin wrote: > > > > Interesting... > > > > Indeed, what version of webrev are you using? > > The .changeset file created by recent versions of webrev has the advantage > > that it contains HG meta-information about which change the patch was created > > from. > > I used the one that comes from within the JDK8 repository (cloned > sometime on Friday). I run webrev from a mercurial patch queue with the > applied patch, which should be visible as a committed changeset I think. > > Cheers, > Mario Hello Mikael, The patch applied without modification on JDK9, here's the actual webrev: http://cr.openjdk.java.net/~neugens/zero-2014-02-24/webrev.02/ If the patch is OK, can you please push it for me? Cheers, Mario From per.liden at oracle.com Mon Mar 3 05:33:08 2014 From: per.liden at oracle.com (Per Liden) Date: Mon, 03 Mar 2014 14:33:08 +0100 Subject: RFR(L): 8029075 - String deduplication in G1 Message-ID: <53148494.60806@oracle.com> Hi, Could I please have this patch reviewed. Summary ------- This patch implements JEP 192 - String deduplication in G1. The goal of string deduplication is to reduce the Java heap live-data set by enhancing the G1 garbage collector so that duplicate instances of String are automatically and continuously deduplicated. I'd like to refer to the JEP for a more detailed description of this feature, the motivation for it, the expected benefit, how it's implemented, etc. This patch is mainly G1-related, but also touches a few runtime files. Webrev: http://cr.openjdk.java.net/~pliden/8029075/webrev.0/ JEP: http://openjdk.java.net/jeps/192 RFE: https://bugs.openjdk.java.net/browse/JDK-8029075 Testing ------- * JTreg - The patch includes the following 8 new tests: - TestStringDeduplicationYoungGC.java: Tests deduplication during Young GC. - TestStringDeduplicationFullGC.java: Tests deduplication during Full GC. - TestStringDeduplicationAgeThreshold.java: Tests both valid and invalid age threshold settings. - TestStringDeduplicationInterned.java: Tests that interned strings are deduplication before being interned. - TestStringDeduplicationTableRehash.java: Stresses the hashtables ability to rehash all entries. - TestStringDeduplicationTableResize.java: Stresses the hashtables ability to resize itself. - TestStringDeduplicationMemoryUsage.java: Tests heap reduction when string deduplication is enabled. - TestStringDeduplicationPrintOptions.java: Tests command line options. * Stress testing: - Kitchensink - GCBasher * Regression testing: - JCK - vmTestbase - Bigapps * Large scale benchmarks to test heap reduction and performance impact: - FA CRM Sales Op. Flow - FA DSS * The following benchmarks have been executed to verify that this feature doesn't impact performance when disabled (even when disabled there are still some "if (UseStringDeduplication)" executed in some hot paths). - SPECjbb2005 - SPECjbb2013 - SPECjvm2008-XML * Various ad-hoc tests and microbenchmarks were also written and executed during the course of the development. Thanks! /Per From mikael.gerdin at oracle.com Mon Mar 3 05:44:00 2014 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Mon, 03 Mar 2014 14:44 +0100 Subject: Bring Zero to llvm 3.4 In-Reply-To: <1393853313.3834.6.camel@galactica.localdomain> References: <1393239625.3873.13.camel@galactica.localdomain> <1393256374.11941.3.camel@galactica.localdomain> <1393853313.3834.6.camel@galactica.localdomain> Message-ID: <1581851.jFBPTQiW9h@mgerdin03> Hi Mario On Monday 03 March 2014 14.28.33 Mario Torre wrote: > On Mon, 2014-02-24 at 16:39 +0100, Mario Torre wrote: > > On Mon, 2014-02-24 at 15:50 +0100, Mikael Gerdin wrote: > > > > Interesting... > > > > > > Indeed, what version of webrev are you using? > > > The .changeset file created by recent versions of webrev has the > > > advantage > > > that it contains HG meta-information about which change the patch was > > > created from. > > > > I used the one that comes from within the JDK8 repository (cloned > > sometime on Friday). I run webrev from a mercurial patch queue with the > > applied patch, which should be visible as a committed changeset I think. > > > > Cheers, > > Mario > > Hello Mikael, > > The patch applied without modification on JDK9, here's the actual > webrev: > > http://cr.openjdk.java.net/~neugens/zero-2014-02-24/webrev.02/ > > If the patch is OK, can you please push it for me? Unfortunately I'm not too familiar with the Zero/Shark code or the makefiles so I wouldn't feel comfortable putting my name as a reviewer on these changes. I'm sure that whoever reviews this change can push it for you as well. /Mikael > > Cheers, > Mario From neugens at redhat.com Mon Mar 3 05:54:21 2014 From: neugens at redhat.com (Mario Torre) Date: Mon, 03 Mar 2014 14:54:21 +0100 Subject: Bring Zero to llvm 3.4 In-Reply-To: <1581851.jFBPTQiW9h@mgerdin03> References: <1393239625.3873.13.camel@galactica.localdomain> <1393256374.11941.3.camel@galactica.localdomain> <1393853313.3834.6.camel@galactica.localdomain> <1581851.jFBPTQiW9h@mgerdin03> Message-ID: <1393854861.3834.8.camel@galactica.localdomain> On Mon, 2014-03-03 at 14:44 +0100, Mikael Gerdin wrote: > Hi Mario > > On Monday 03 March 2014 14.28.33 Mario Torre wrote: > > On Mon, 2014-02-24 at 16:39 +0100, Mario Torre wrote: > > > On Mon, 2014-02-24 at 15:50 +0100, Mikael Gerdin wrote: > > > > > Interesting... > > > > > > > > Indeed, what version of webrev are you using? > > > > The .changeset file created by recent versions of webrev has the > > > > advantage > > > > that it contains HG meta-information about which change the patch was > > > > created from. > > > > > > I used the one that comes from within the JDK8 repository (cloned > > > sometime on Friday). I run webrev from a mercurial patch queue with the > > > applied patch, which should be visible as a committed changeset I think. > > > > > > Cheers, > > > Mario > > > > Hello Mikael, > > > > The patch applied without modification on JDK9, here's the actual > > webrev: > > > > http://cr.openjdk.java.net/~neugens/zero-2014-02-24/webrev.02/ > > > > If the patch is OK, can you please push it for me? > > Unfortunately I'm not too familiar with the Zero/Shark code or the makefiles > so I wouldn't feel comfortable putting my name as a reviewer on these changes. > > I'm sure that whoever reviews this change can push it for you as well. > > /Mikael Hi Mikael, Thanks. Anyone that can review it? Christian maybe? Cheers, Mario From daniel.daugherty at oracle.com Mon Mar 3 06:56:38 2014 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 03 Mar 2014 07:56:38 -0700 Subject: JDK-8036003: Add variable not to separate debug information. In-Reply-To: References: <20140228091835.58EB8C21F24@msgw007-05.ocn.ad.jp> <53106F4A.8030905@oracle.com> <53109772.9070808@oracle.com> <5310AD70.2070908@ysfactory.dip.jp> <53112019.9050504@oracle.com> <5311E98A.1020004@ysfactory.dip.jp> Message-ID: <53149826.7030106@oracle.com> I think we're having a problem with not all replies making it to all the aliases. Yasumasa's reply to David below that Mike is replying to did not arrive on any of the aliases that I'm on... Folks need to remember to reply to all of the aliases... On 3/1/14 4:08 PM, Mike Duigou wrote: > On Mar 1 2014, at 06:07 , Yasumasa Suenaga wrote: > >> Hi David, >> >>> 1. Generating debug symbols in the binaries (via gcc -g or whatever) >>> 2. Generating debuginfo files (zipped or not) (FDS) >>> 3. Stripping debug symbols from the binaries (strip-policy) >>> >>> It may be that we don't have clean separation between them, and if so >>> that should be fixed, but I don't see the current proposal as the way >>> forward. >> Currently, 1. depends 2. If FDS set to disable, debug symbols (-g) are not >> generated. Bit of history here. When FDS is disabled, the build system is supposed to go back to what was there prior to the FDS project. On Linux, one of the debug configs didn't actually have any debugging flags enabled (debug or fastdebug, I don't remember which). So when I implemented FDS, disabling FDS meant no debug info. Bug in the original? You betcha. >> SEPARATED_DEBUGINFO_FILES which my patch provides can separate them. > I think David's list (kudos) is very helpful as I haven't seen the requirements articulated this clearly anywhere else. > > Some comments: > > - Are there important tools that cannot deal with external debuginfo files? Not that I'm aware of. There have been bugs in the way gobjcopy is extracting the debuginfo into the separate files, but no complete failures as far as I know... > If we can put debuginfo into external files why would we ever want unstripped binaries? We deliver minimal stripped binaries so that can have stack traces with some amount of information in them when the debuginfo files are not available. We have not yet figured out how to implement minimal stripping on Windows so Windows hs_err_pid files have no symbols in them when the debuginfo bundles are not present. > Is strip policy really necessary? Yes. STRIP_POLICY allows folks to configure what they want for the build that they are doing. It is also meant to map to the Oracle policy where we were granted a waiver to deliver minimal symbols in FDS project. > Strip policy would seem to only be necessary if there are tools which can't handle external debuginfo. Assuming that everything can use external debuginfo then the policy would be to strip everything. No you don't want to strip everything. See above aboutminimal info in stack traces. > Do I understand correctly that rpmbuild can only deal with unstripped binaries and generates the stripped rpm package and debuginfo package. It sounds kind of strange that find-debuginfo.sh wouldn't be able to use already generated debuginfo files. > > - I would expect that the flow is something like an extended version of https://blogs.oracle.com/dbx/entry/creating_separate_debug_info : > 1. Compile source files with some form of "-g" > 2. Create separate debug files for object files. > 3. Strip object files. > 4. Add gnu_debuglink to object files. > 5. Link library or executable which should carry along links to debuginfo. > 6a. Debugging, testing, profiling, etc. by developers. > 6b. Packaging for program bits and debuginfo. > 7. Testing and Use. > > Hopefully I didn't get any of those steps in the wrong order. What are the "you-cant-get-there-from-here" gaps and breakdowns from implementing this process? > > - Whether for the FDS debug symbols or RPM debuginfo packaging it seems that the default debug level isn't quite right. I believe that in both cases what's wanted is abbreviated debug info, mostly function names and line number tables, for building stack traces. This is not the debug info that is currently generated. It's STRIP_POLICY=min_strip that leaves the minimal debug info attached to the binaries and the more complete debug info in left in the debuginfo file(s). And I'll disagree about only wanting abbreviated debug info in the separate debuginfo files. > There are four basic alternatives for levels of debug info: > 1. (-g0) No debug info whatsoever. > Only exported functions and globals will have names. > 2. (-g1) Abbreviated debug info. > All functions have names and line number information. This seems to be what is needed by both FDS and RPM debuginfo files to produce nice stack traces. > 3. (-g) Default debugging info. > Adds macro expansions and local variable names. Suitable for source level debugging. > 4. (-g3 or -gdwarf-4 -fvar-tracking-assignments). Full debugging info. > Most suitable for source level debugging including debugging of optimized code. It is not clear that anyone would want to build the entire product with this option. > > Compilations are currently done with a mix of -g, -g1, and -gstabs. It would be good to rationalize this somewhat and provide a mechanism for developers to tweak generated debugging output for files or components. There is a whole boatload of history behind using STABS or DWARF or -g1 and usually (at least in HotSpot) there are comments in the Makefiles explaining the sometimes arcane reasons why these choices were made... On Linux there is a special option DEBUG_BINARIES that is used to specify '-g' for everything. I believe the magic incantation for making RPM happy is something like: DEBUG_BINARIES=true FULL_DEBUG_SYMBOLS=0 STRIP_POLICY=none ALT_OBJCOPY=/does_not_exist The combination of FULL_DEBUG_SYMBOLS=0 and ALT_OBJCOPY=/does_not_exist disables FDS for all build configs and reverts to pre-FDS make logic. STRIP_POLICY=none says don't do any stripping. DEBUG_BINARIES=true says ignore all the other logic about which debug options and just do '-g'. I've accidentally broken DEBUG_BINARIES at least once, but I usually get pinged by the Red Hat folks when I do... Dan > - Eventual alignment with http://gcc.gnu.org/wiki/DebugFission on some platforms? > >> So I change: >> >> 1. Separating to add "-g" option to compiler from executing objcopy. >> 2. jdk/make/Images.gmk regards STRIP_POLICY. >> >> >> As I mentioned earlier, this issue is related to RPM. >> So I hope to fix it before JDK8 GA is released. > This won't happen (at least not for Java 8u0). Java 8 is already late in it's final candidate stage. It is too late for the initial Java 8 release to consider this magnitude of change. In any event, since the Java 8 rampdown began back in November, any change would first have to be applied to Java 9 and then approved for backport to a Java 8 or an update release (and it is also possibly too late for 8u20). > > Inability to include your suggested change in Java 8 or 8u20 is in no way a rejection of the ideas or contribution, it's merely a normal consequence of timelines and process. > > Mike From goetz.lindenmaier at sap.com Mon Mar 3 08:03:43 2014 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 3 Mar 2014 16:03:43 +0000 Subject: RFR(S): 8036122: Fix warning 'format not a string literal' Message-ID: <4295855A5C1DE049A61835A1887419CC2CEA576C@DEWDFEMB12A.global.corp.sap> Hi, I found a way to simplify my change considerably, and thus updated the webrev: http://cr.openjdk.java.net/~goetz/webrevs/8036122-1-fmtWrn/ In compilerOracle.cpp was a loop checking three format strings with sscanf. Before, I had unrolled the loop. Now I figured, that before this loop is reached, the command lines are normalized to format exclude java/lang/String indexOf so that only one of the three format strings used will ever apply. So I need to check only this one of the three formats anyways. This makes this already small change much nicer. Best regards, Goetz. From: Lindenmaier, Goetz Sent: Montag, 3. M?rz 2014 14:22 To: hotspot-dev Source Developers; 'Mike Duigou' Subject: RFR(S): 8036122: Fix warning 'format not a string literal' Hi, Since "8030350: Enable additional compiler warnings for GCC", the hotspot build issues warnings: src/share/vm/compiler/compilerOracle.cpp: In function 'bool scan_line(const char*, char*, MethodMatcher::Mode*, char*, MethodMatcher::Mode*, int*, const char*&)': src/share/vm/compiler/compilerOracle.cpp:425: warning: format not a string literal, argument types not checked src/os/linux/vm/os_linux.cpp: In function 'jlong slow_thread_cpu_time(Thread*, bool)': src/os/linux/vm/os_linux.cpp:5316: warning: format not a string literal, argument types not checked Also, older gccs (4.2.2 and before) can not build hotspot any more. They don't know the option introduced in that change: cc1plus: error: unrecognized command line option "-Wno-error=format-nonliteral" This change fixes both by - Fixing the warning in the hotspot code - removing the flag from the makefiles. Thus, the build with new gcc does no more issue warnings, and the build with older ones works again. Please review and test this change - and I please need a sponsor. http://cr.openjdk.java.net/~goetz/webrevs/8036122-1-fmtWrn/ Best regards, Goetz. From mike.duigou at oracle.com Mon Mar 3 09:06:39 2014 From: mike.duigou at oracle.com (Mike Duigou) Date: Mon, 3 Mar 2014 09:06:39 -0800 Subject: RFR(S): 8036122: Fix warning 'format not a string literal' In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CEA576C@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CEA576C@DEWDFEMB12A.global.corp.sap> Message-ID: <502467DC-5857-4F18-BAD2-02321615A6E0@oracle.com> This looks fine. Minor comments: - I removed the --no-error=format-string-literal from solaris/gcc.make as well. - The initial snprintf in os_linux.cpp could be the other branch of the "if (proc_task_unchecked && os::Linux::is_NPTL())" conditional. I am testing the patch now though another reviewer is needed before this can be pushed. Mike On Mar 3 2014, at 08:03 , Lindenmaier, Goetz wrote: > Hi, > > I found a way to simplify my change considerably, and thus updated the webrev: > http://cr.openjdk.java.net/~goetz/webrevs/8036122-1-fmtWrn/ > > In compilerOracle.cpp was a loop checking three format strings with sscanf. > Before, I had unrolled the loop. > Now I figured, that before this loop is reached, the command lines are normalized > to format > exclude java/lang/String indexOf > so that only one of the three format strings used will ever apply. So I need to > check only this one of the three formats anyways. > > This makes this already small change much nicer. > > Best regards, > Goetz. > > > > From: Lindenmaier, Goetz > Sent: Montag, 3. M?rz 2014 14:22 > To: hotspot-dev Source Developers; 'Mike Duigou' > Subject: RFR(S): 8036122: Fix warning 'format not a string literal' > > Hi, > Since "8030350: Enable additional compiler warnings for GCC", the hotspot build issues warnings: > > src/share/vm/compiler/compilerOracle.cpp: In function 'bool scan_line(const char*, char*, MethodMatcher::Mode*, char*, MethodMatcher::Mode*, int*, const char*&)': > src/share/vm/compiler/compilerOracle.cpp:425: warning: format not a string literal, argument types not checked > src/os/linux/vm/os_linux.cpp: In function 'jlong slow_thread_cpu_time(Thread*, bool)': > src/os/linux/vm/os_linux.cpp:5316: warning: format not a string literal, argument types not checked > > Also, older gccs (4.2.2 and before) can not build hotspot any more. They don't know the option introduced in that change: > > cc1plus: error: unrecognized command line option "-Wno-error=format-nonliteral" > > This change fixes both by > > - Fixing the warning in the hotspot code > > - removing the flag from the makefiles. > > Thus, the build with new gcc does no more issue warnings, and the build with older ones works again. > > Please review and test this change ? and I please need a sponsor. > > http://cr.openjdk.java.net/~goetz/webrevs/8036122-1-fmtWrn/ > > Best regards, > > Goetz. > From omajid at redhat.com Mon Mar 3 11:01:03 2014 From: omajid at redhat.com (Omair Majid) Date: Mon, 3 Mar 2014 14:01:03 -0500 Subject: JDK-8036003: Add variable not to separate debug information. In-Reply-To: <53144E4D.8080606@redhat.com> References: <20140228091835.58EB8C21F24@msgw007-05.ocn.ad.jp> <53144E4D.8080606@redhat.com> Message-ID: <20140303190102.GE2045@redhat.com> * Andrew Haley [2014-03-03 04:43]: > On 02/28/2014 09:18 AM, Yasumasa Suenaga wrote: > > For example, OpenJDK8 in Fedora20 ships libjvm.so and libjvm.debuginfo . > > libjvm.debuginfo is generated in OpenJDK's makefiles, however it does not > > contain debug information. Actual debug information is shipped by OpenJDK > > debuginfo package. > > That's a bug in Fedora's build. We should fix it. This should fix it: http://pkgs.fedoraproject.org/cgit/java-1.8.0-openjdk.git/commit/?id=1734315551d634b7467f28788d90595b467ea5eb Thanks, Omair -- PGP Key: 66484681 (http://pgp.mit.edu/) Fingerprint = F072 555B 0A17 3957 4E95 0056 F286 F14F 6648 4681 From omajid at redhat.com Mon Mar 3 11:11:18 2014 From: omajid at redhat.com (Omair Majid) Date: Mon, 3 Mar 2014 14:11:18 -0500 Subject: JDK-8036003: Add variable not to separate debug information. In-Reply-To: <53149826.7030106@oracle.com> References: <20140228091835.58EB8C21F24@msgw007-05.ocn.ad.jp> <53106F4A.8030905@oracle.com> <53109772.9070808@oracle.com> <5310AD70.2070908@ysfactory.dip.jp> <53112019.9050504@oracle.com> <5311E98A.1020004@ysfactory.dip.jp> <53149826.7030106@oracle.com> Message-ID: <20140303191118.GF2045@redhat.com> Hi, * Daniel D. Daugherty [2014-03-03 09:57]: > On 3/1/14 4:08 PM, Mike Duigou wrote: > > If we can put debuginfo into external files why would we ever want unstripped binaries? > > We deliver minimal stripped binaries so that can have stack traces > with some amount of information in them when the debuginfo files > are not available. We have not yet figured out how to implement > minimal stripping on Windows so Windows hs_err_pid files have no > symbols in them when the debuginfo bundles are not present. Am I reading it right that when tools (like RPM) strip the debug information from these binaries, hotspot will break completely when it tries to generate a stack trace on crash? Thanks, Omair -- PGP Key: 66484681 (http://pgp.mit.edu/) Fingerprint = F072 555B 0A17 3957 4E95 0056 F286 F14F 6648 4681 From daniel.daugherty at oracle.com Mon Mar 3 11:20:38 2014 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 03 Mar 2014 12:20:38 -0700 Subject: JDK-8036003: Add variable not to separate debug information. In-Reply-To: <20140303191118.GF2045@redhat.com> References: <20140228091835.58EB8C21F24@msgw007-05.ocn.ad.jp> <53106F4A.8030905@oracle.com> <53109772.9070808@oracle.com> <5310AD70.2070908@ysfactory.dip.jp> <53112019.9050504@oracle.com> <5311E98A.1020004@ysfactory.dip.jp> <53149826.7030106@oracle.com> <20140303191118.GF2045@redhat.com> Message-ID: <5314D606.70804@oracle.com> On 3/3/14 12:11 PM, Omair Majid wrote: > Hi, > > * Daniel D. Daugherty [2014-03-03 09:57]: >> On 3/1/14 4:08 PM, Mike Duigou wrote: >>> If we can put debuginfo into external files why would we ever want unstripped binaries? >> We deliver minimal stripped binaries so that can have stack traces >> with some amount of information in them when the debuginfo files >> are not available. We have not yet figured out how to implement >> minimal stripping on Windows so Windows hs_err_pid files have no >> symbols in them when the debuginfo bundles are not present. > Am I reading it right that when tools (like RPM) strip the debug > information from these binaries, hotspot will break completely when it > tries to generate a stack trace on crash? Not quite. If you follow this process: - use gobjcopy to copy debug info from libjvm.so -> libjvm.debuginfo - use gobjcopy to tell libjvm.so that debug info is found in libjvm.debuginfo - strip libjvm.so (I don't remember the strip everything option) then you can still get symbols in your crashes as long as you have BOTH libjvm.so and libjvm.debuginfo. However, if you are in a situation where you don't have libjvm.debuginfo, then you do not get symbols in your crashes. When I did the Full Debug Symbols (FDS) project, I copied complete debug info from libjvm.so -> libjvm.debuginfo and left minimal debug info in libjvm.so so that you could still get symbols on crashes on Linux and Solaris when the libjvm.debuginfo files were not present. Dan > > Thanks, > Omair > From omajid at redhat.com Mon Mar 3 13:49:24 2014 From: omajid at redhat.com (Omair Majid) Date: Mon, 3 Mar 2014 16:49:24 -0500 Subject: JDK-8036003: Add variable not to separate debug information. In-Reply-To: <53112019.9050504@oracle.com> References: <20140228091835.58EB8C21F24@msgw007-05.ocn.ad.jp> <53106F4A.8030905@oracle.com> <53109772.9070808@oracle.com> <5310AD70.2070908@ysfactory.dip.jp> <53112019.9050504@oracle.com> Message-ID: <20140303214923.GA26825@redhat.com> * David Holmes [2014-02-28 18:48]: > There are three pieces to all of this: > > 1. Generating debug symbols in the binaries (via gcc -g or whatever) > 2. Generating debuginfo files (zipped or not) (FDS) > 3. Stripping debug symbols from the binaries (strip-policy) > > It may be that we don't have clean separation between them, and if so > that should be fixed, but I don't see the current proposal as the way > forward. Any chance we could clean up the names too? It's not obvious to me why FDS means 'generating debuginfo files'. What do folks think of modifying the OPENJDK build to use this as the settings for the options above by default. 1. Yes 2. No 3. No > Also there may well be differences between how things are handled on the > JDK side and hotspot side. Is there any chance of this becoming consistent between hotspot and JDK? It would be much nicer if concepts like strip-policy were consistent between hotspot and the rest of the JDK. Thanks, Omair -- PGP Key: 66484681 (http://pgp.mit.edu/) Fingerprint = F072 555B 0A17 3957 4E95 0056 F286 F14F 6648 4681 From vladimir.kozlov at oracle.com Mon Mar 3 14:08:19 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 03 Mar 2014 14:08:19 -0800 Subject: RFR(S): 8036122: Fix warning 'format not a string literal' In-Reply-To: <502467DC-5857-4F18-BAD2-02321615A6E0@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CEA576C@DEWDFEMB12A.global.corp.sap> <502467DC-5857-4F18-BAD2-02321615A6E0@oracle.com> Message-ID: <5314FD53.4090806@oracle.com> Goetz, What is difference between 8036120 and 8036122 you filed? Please, close 8036120 if it is dup of this one we are reviewing. Changes look fine to me. I assume you tested all variations of command line processed by compilerOracle. Thanks, Vladimir On 3/3/14 9:06 AM, Mike Duigou wrote: > This looks fine. > > Minor comments: > > - I removed the --no-error=format-string-literal from solaris/gcc.make as well. > > - The initial snprintf in os_linux.cpp could be the other branch of the "if (proc_task_unchecked && os::Linux::is_NPTL())" conditional. > > I am testing the patch now though another reviewer is needed before this can be pushed. > > Mike > > On Mar 3 2014, at 08:03 , Lindenmaier, Goetz wrote: > >> Hi, >> >> I found a way to simplify my change considerably, and thus updated the webrev: >> http://cr.openjdk.java.net/~goetz/webrevs/8036122-1-fmtWrn/ >> >> In compilerOracle.cpp was a loop checking three format strings with sscanf. >> Before, I had unrolled the loop. >> Now I figured, that before this loop is reached, the command lines are normalized >> to format >> exclude java/lang/String indexOf >> so that only one of the three format strings used will ever apply. So I need to >> check only this one of the three formats anyways. >> >> This makes this already small change much nicer. >> >> Best regards, >> Goetz. >> >> >> >> From: Lindenmaier, Goetz >> Sent: Montag, 3. M?rz 2014 14:22 >> To: hotspot-dev Source Developers; 'Mike Duigou' >> Subject: RFR(S): 8036122: Fix warning 'format not a string literal' >> >> Hi, >> Since "8030350: Enable additional compiler warnings for GCC", the hotspot build issues warnings: >> >> src/share/vm/compiler/compilerOracle.cpp: In function 'bool scan_line(const char*, char*, MethodMatcher::Mode*, char*, MethodMatcher::Mode*, int*, const char*&)': >> src/share/vm/compiler/compilerOracle.cpp:425: warning: format not a string literal, argument types not checked >> src/os/linux/vm/os_linux.cpp: In function 'jlong slow_thread_cpu_time(Thread*, bool)': >> src/os/linux/vm/os_linux.cpp:5316: warning: format not a string literal, argument types not checked >> >> Also, older gccs (4.2.2 and before) can not build hotspot any more. They don't know the option introduced in that change: >> >> cc1plus: error: unrecognized command line option "-Wno-error=format-nonliteral" >> >> This change fixes both by >> >> - Fixing the warning in the hotspot code >> >> - removing the flag from the makefiles. >> >> Thus, the build with new gcc does no more issue warnings, and the build with older ones works again. >> >> Please review and test this change ? and I please need a sponsor. >> >> http://cr.openjdk.java.net/~goetz/webrevs/8036122-1-fmtWrn/ >> >> Best regards, >> >> Goetz. >> > From goetz.lindenmaier at sap.com Mon Mar 3 14:33:12 2014 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 3 Mar 2014 22:33:12 +0000 Subject: RFR(S): 8036122: Fix warning 'format not a string literal' In-Reply-To: <502467DC-5857-4F18-BAD2-02321615A6E0@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CEA576C@DEWDFEMB12A.global.corp.sap> <502467DC-5857-4F18-BAD2-02321615A6E0@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CEA586D@DEWDFEMB12A.global.corp.sap> Hi Mike, great, I missed the other ones! There is a third of the flags in the bsd gcc.make. I updated the webrev to also remove the one in bsd and as you probably did on solaris. I also added some output of the compiler oracle test I did. Thanks for your help, Goetz. From: Mike Duigou [mailto:mike.duigou at oracle.com] Sent: Monday, March 03, 2014 6:07 PM To: Lindenmaier, Goetz Cc: hotspot-dev Source Developers Subject: Re: RFR(S): 8036122: Fix warning 'format not a string literal' This looks fine. Minor comments: - I removed the --no-error=format-string-literal from solaris/gcc.make as well. - The initial snprintf in os_linux.cpp could be the other branch of the "if (proc_task_unchecked && os::Linux::is_NPTL())" conditional. I am testing the patch now though another reviewer is needed before this can be pushed. Mike On Mar 3 2014, at 08:03 , Lindenmaier, Goetz > wrote: Hi, I found a way to simplify my change considerably, and thus updated the webrev: http://cr.openjdk.java.net/~goetz/webrevs/8036122-1-fmtWrn/ In compilerOracle.cpp was a loop checking three format strings with sscanf. Before, I had unrolled the loop. Now I figured, that before this loop is reached, the command lines are normalized to format exclude java/lang/String indexOf so that only one of the three format strings used will ever apply. So I need to check only this one of the three formats anyways. This makes this already small change much nicer. Best regards, Goetz. From: Lindenmaier, Goetz Sent: Montag, 3. M?rz 2014 14:22 To: hotspot-dev Source Developers; 'Mike Duigou' Subject: RFR(S): 8036122: Fix warning 'format not a string literal' Hi, Since "8030350: Enable additional compiler warnings for GCC", the hotspot build issues warnings: src/share/vm/compiler/compilerOracle.cpp: In function 'bool scan_line(const char*, char*, MethodMatcher::Mode*, char*, MethodMatcher::Mode*, int*, const char*&)': src/share/vm/compiler/compilerOracle.cpp:425: warning: format not a string literal, argument types not checked src/os/linux/vm/os_linux.cpp: In function 'jlong slow_thread_cpu_time(Thread*, bool)': src/os/linux/vm/os_linux.cpp:5316: warning: format not a string literal, argument types not checked Also, older gccs (4.2.2 and before) can not build hotspot any more. They don't know the option introduced in that change: cc1plus: error: unrecognized command line option "-Wno-error=format-nonliteral" This change fixes both by - Fixing the warning in the hotspot code - removing the flag from the makefiles. Thus, the build with new gcc does no more issue warnings, and the build with older ones works again. Please review and test this change - and I please need a sponsor. http://cr.openjdk.java.net/~goetz/webrevs/8036122-1-fmtWrn/ Best regards, Goetz. From goetz.lindenmaier at sap.com Mon Mar 3 14:41:56 2014 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 3 Mar 2014 22:41:56 +0000 Subject: RFR(S): 8036122: Fix warning 'format not a string literal' In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CEA576C@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CEA576C@DEWDFEMB12A.global.corp.sap> Message-ID: <4295855A5C1DE049A61835A1887419CC2CEA588B@DEWDFEMB12A.global.corp.sap> Hi Christian, >From your comment in the bug: > Did the runtime team fix the one in: > src/os/linux/vm/os_linux.cpp: In function 'jlong slow_thread_cpu_time(Thread*, bool)': > src/os/linux/vm/os_linux.cpp:5316: warning: format not a string literal, argument types not checked Currently, there is no fix for the warning in the repo jdk9/hs-rt. Also, I asked before whether somebody would fix it, but got no answer saying somebody would do so. My change fixes it, though. Unfortunately I have to close the bug you commented on, as I generated two of the same, and sent the webrev with the other bugid. Should I move the comment into the other bug? Best regards, Goetz. -----Original Message----- From: hotspot-dev-bounces at openjdk.java.net [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Lindenmaier, Goetz Sent: Monday, March 03, 2014 5:04 PM To: hotspot-dev Source Developers; Mike Duigou Subject: RE: RFR(S): 8036122: Fix warning 'format not a string literal' Hi, I found a way to simplify my change considerably, and thus updated the webrev: http://cr.openjdk.java.net/~goetz/webrevs/8036122-1-fmtWrn/ In compilerOracle.cpp was a loop checking three format strings with sscanf. Before, I had unrolled the loop. Now I figured, that before this loop is reached, the command lines are normalized to format exclude java/lang/String indexOf so that only one of the three format strings used will ever apply. So I need to check only this one of the three formats anyways. This makes this already small change much nicer. Best regards, Goetz. From: Lindenmaier, Goetz Sent: Montag, 3. M?rz 2014 14:22 To: hotspot-dev Source Developers; 'Mike Duigou' Subject: RFR(S): 8036122: Fix warning 'format not a string literal' Hi, Since "8030350: Enable additional compiler warnings for GCC", the hotspot build issues warnings: src/share/vm/compiler/compilerOracle.cpp: In function 'bool scan_line(const char*, char*, MethodMatcher::Mode*, char*, MethodMatcher::Mode*, int*, const char*&)': src/share/vm/compiler/compilerOracle.cpp:425: warning: format not a string literal, argument types not checked src/os/linux/vm/os_linux.cpp: In function 'jlong slow_thread_cpu_time(Thread*, bool)': src/os/linux/vm/os_linux.cpp:5316: warning: format not a string literal, argument types not checked Also, older gccs (4.2.2 and before) can not build hotspot any more. They don't know the option introduced in that change: cc1plus: error: unrecognized command line option "-Wno-error=format-nonliteral" This change fixes both by - Fixing the warning in the hotspot code - removing the flag from the makefiles. Thus, the build with new gcc does no more issue warnings, and the build with older ones works again. Please review and test this change - and I please need a sponsor. http://cr.openjdk.java.net/~goetz/webrevs/8036122-1-fmtWrn/ Best regards, Goetz. From christian.thalinger at oracle.com Mon Mar 3 17:14:09 2014 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Mon, 3 Mar 2014 17:14:09 -0800 Subject: RFR (XS): 8035983: Fix "Native frames:" in crash report (hs_err file) In-Reply-To: <53111AFC.6@oracle.com> References: <530FD966.7030306@oracle.com> <72C0DBF6-6BFE-4560-B2C8-950D39E84C8B@oracle.com> <530FEFA4.1040105@oracle.com> <48C6875B-3BB8-4D47-A798-3610EFF33C73@oracle.com> <530FF6DB.50301@oracle.com> <2FBF907F-6AD8-4D8A-9266-01D7AD8C0EEB@oracle.com> <53111AFC.6@oracle.com> Message-ID: On Feb 28, 2014, at 3:25 PM, Vladimir Kozlov wrote: > On 2/27/14 7:59 PM, Christian Thalinger wrote: >> >> On Feb 27, 2014, at 6:39 PM, Vladimir Kozlov wrote: >> >>> http://cr.openjdk.java.net/~kvn/8035983/webrev.01/ >>> >>> New check is used only for JavaThread where it is definitely initialized and where we have problem with call stack dump. >> >> Looks good. Just to be sure; +1 would also work here: >> >> + if (!_thread->on_local_stack((address)(fr.sp() + 4))) >> >> Right? > > No, for Stack: [0xffff80ffa69f8000,0xffff80ffa6af8000] > > I see for first frame "_lwp_start+0x0" > > frame::_sp = 0xffff80ffa6af7ff0 > > I just tested it on SPARC and it needs more than +4 (+11), the first frame has: > > frame::_sp = 0xffffffff6befff50 > > It seems using fr.sender_sp() in the check work on x86 and sparc. > On x86 it return stack_base value on sparc it returns STACK_BIAS. > > if (!_thread->on_local_stack((address)(fr.sender_sp() + 1))) > break; > > http://cr.openjdk.java.net/~kvn/8035983/webrev.02/ That looks good and supposedly also works on PPC, ARM and all the others. > > Thanks, > Vladimir > >> >>> >>> thanks, >>> Vladimir >>> >>> On 2/27/14 6:31 PM, Christian Thalinger wrote: >>>> >>>> On Feb 27, 2014, at 6:08 PM, Vladimir Kozlov wrote: >>>> >>>>> >>>>> >>>>> On 2/27/14 5:38 PM, Christian Thalinger wrote: >>>>>> >>>>>> On Feb 27, 2014, at 4:33 PM, Vladimir Kozlov wrote: >>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8035983 >>>>>>> http://cr.openjdk.java.net/~kvn/8035983/webrev/ >>>>>>> >>>>>>> We don't print whole stack if native frames intermix with compiled java frames (on x86 fp is used by compiled code). >>>>>>> Instead of using os::is_first_C_frame() which produces incorrect result for compiled java frames I am suggesting to look on frame's stack pointer relative to stack base. >>>>>> >>>>>> I have two questions: >>>>>> >>>>>> 1) Why does this work: >>>>>> >>>>>> + // Catch very first native frame by using stack address. >>>>>> + if (!_thread->on_local_stack((address)(fr.sp() + 4))) break; >>>>>> >>>>>> Because we set the stack base to the current value when we start the thread? >>>>> >>>>> You are right not all threads have stack base set (all VM's thread do). So I need to add check for stack_size() != 0 and use is_first_C_frame() if it is 0 (uninitialized). >>>> >>>> Maybe add a comment why the on_local_stack() check works. >>>> >>>>> >>>>>> >>>>>> 2) Why does this not work? I have seen many correct stack traces in the past. >>>>> >>>>> Look on os::is_first_C_frame(). It does some stupid simple checks which nothing to do with real value. The test will pass if a java compiled code has a pointer in EBP register. >>>> >>>> Ah, that?s what I was missing. Makes sense then. Yeah, the code in os::is_first_C_frame() is weird :-) >>>> >>>>> >>>>> Thanks, >>>>> Vladimir >>>>> >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Vladimir >>>>>>> >>>>>>> Output before the fix: >>>>>>> >>>>>>> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) >>>>>>> V [libjvm.so+0x1e28428] void VMError::report(outputStream*)+0x1478 >>>>>>> V [libjvm.so+0x1e29dd4] void VMError::report_and_die()+0x6b4 >>>>>>> V [libjvm.so+0x14ad9ba] void report_vm_error(const char*,int,const char*,const char*)+0x9a >>>>>>> V [libjvm.so+0x1b6ccf5] void ObjectMonitor::exit(bool,Thread*)+0x125 >>>>>>> V [libjvm.so+0x1d41cda] void ObjectSynchronizer::fast_exit(oopDesc*,BasicLock*,Thread*)+0x38a >>>>>>> V [libjvm.so+0x1d41fba] void ObjectSynchronizer::slow_exit(oopDesc*,BasicLock*,Thread*)+0x2a >>>>>>> V [libjvm.so+0x1caa13f] void SharedRuntime::complete_monitor_unlocking_C(oopDesc*,BasicLock*)+0x27f >>>>>>> >>>>>>> >>>>>>> After the fix: >>>>>>> >>>>>>> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) >>>>>>> V [libjvm.so+0x1e28428] void VMError::report(outputStream*)+0x1478 >>>>>>> V [libjvm.so+0x1e29dd4] void VMError::report_and_die()+0x6b4 >>>>>>> V [libjvm.so+0x14ad9ba] void report_vm_error(const char*,int,const char*,const char*)+0x9a >>>>>>> V [libjvm.so+0x1b6ccf5] void ObjectMonitor::exit(bool,Thread*)+0x125 >>>>>>> V [libjvm.so+0x1d41cda] void ObjectSynchronizer::fast_exit(oopDesc*,BasicLock*,Thread*)+0x38a >>>>>>> V [libjvm.so+0x1d41fba] void ObjectSynchronizer::slow_exit(oopDesc*,BasicLock*,Thread*)+0x2a >>>>>>> V [libjvm.so+0x1caa13f] void SharedRuntime::complete_monitor_unlocking_C(oopDesc*,BasicLock*)+0x27f >>>>>>> J 13 C2 java.util.Hashtable.get(Ljava/lang/Object;)Ljava/lang/Object; (69 bytes) @ 0xffff80ffb979da84 [0xffff80ffb979d700+0x384] >>>>>>> J 10% C2 Test.Worker()V (381 bytes) @ 0xffff80ffb97a1c8c [0xffff80ffb97a1840+0x44c] >>>>>>> j Test$1.run()V+0 >>>>>>> v ~StubRoutines::call_stub >>>>>>> V [libjvm.so+0x177d196] void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*)+0x886 >>>>>>> V [libjvm.so+0x1b9a6c8] void os::os_exception_wrapper(void(*)(JavaValue*,methodHandle*,JavaCallArguments*,Thread*),JavaValue*,methodHandle*,JavaCallArguments*,Thread*)+0x38 >>>>>>> V [libjvm.so+0x177c90a] void JavaCalls::call(JavaValue*,methodHandle,JavaCallArguments*,Thread*)+0x9a >>>>>>> V [libjvm.so+0x177be70] void JavaCalls::call_virtual(JavaValue*,KlassHandle,Symbol*,Symbol*,JavaCallArguments*,Thread*)+0x250 >>>>>>> V [libjvm.so+0x177bf56] void JavaCalls::call_virtual(JavaValue*,Handle,KlassHandle,Symbol*,Symbol*,Thread*)+0x66 >>>>>>> V [libjvm.so+0x1896897] void thread_entry(JavaThread*,Thread*)+0xc7 >>>>>>> V [libjvm.so+0x1d8f23e] void JavaThread::thread_main_inner()+0x18e >>>>>>> V [libjvm.so+0x1d8f09a] void JavaThread::run()+0x1fa >>>>>>> V [libjvm.so+0x1b8f3c8] java_start+0x248 >>>>>>> C [libc.so.1+0x122105] _thrp_setup+0xa5 >>>>>>> C [libc.so.1+0x1223b0] _lwp_start+0x0 >>>> >> From vladimir.kozlov at oracle.com Mon Mar 3 17:27:00 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 03 Mar 2014 17:27:00 -0800 Subject: RFR (XS): 8035983: Fix "Native frames:" in crash report (hs_err file) In-Reply-To: References: <530FD966.7030306@oracle.com> <72C0DBF6-6BFE-4560-B2C8-950D39E84C8B@oracle.com> <530FEFA4.1040105@oracle.com> <48C6875B-3BB8-4D47-A798-3610EFF33C73@oracle.com> <530FF6DB.50301@oracle.com> <2FBF907F-6AD8-4D8A-9266-01D7AD8C0EEB@oracle.com> <53111AFC.6@oracle.com> Message-ID: <53152BE4.1080003@oracle.com> Than you, Christian I am trying to test on "others". Vladimir On 3/3/14 5:14 PM, Christian Thalinger wrote: > > On Feb 28, 2014, at 3:25 PM, Vladimir Kozlov wrote: > >> On 2/27/14 7:59 PM, Christian Thalinger wrote: >>> >>> On Feb 27, 2014, at 6:39 PM, Vladimir Kozlov wrote: >>> >>>> http://cr.openjdk.java.net/~kvn/8035983/webrev.01/ >>>> >>>> New check is used only for JavaThread where it is definitely initialized and where we have problem with call stack dump. >>> >>> Looks good. Just to be sure; +1 would also work here: >>> >>> + if (!_thread->on_local_stack((address)(fr.sp() + 4))) >>> >>> Right? >> >> No, for Stack: [0xffff80ffa69f8000,0xffff80ffa6af8000] >> >> I see for first frame "_lwp_start+0x0" >> >> frame::_sp = 0xffff80ffa6af7ff0 >> >> I just tested it on SPARC and it needs more than +4 (+11), the first frame has: >> >> frame::_sp = 0xffffffff6befff50 >> >> It seems using fr.sender_sp() in the check work on x86 and sparc. >> On x86 it return stack_base value on sparc it returns STACK_BIAS. >> >> if (!_thread->on_local_stack((address)(fr.sender_sp() + 1))) >> break; >> >> http://cr.openjdk.java.net/~kvn/8035983/webrev.02/ > > That looks good and supposedly also works on PPC, ARM and all the others. > >> >> Thanks, >> Vladimir >> >>> >>>> >>>> thanks, >>>> Vladimir >>>> >>>> On 2/27/14 6:31 PM, Christian Thalinger wrote: >>>>> >>>>> On Feb 27, 2014, at 6:08 PM, Vladimir Kozlov wrote: >>>>> >>>>>> >>>>>> >>>>>> On 2/27/14 5:38 PM, Christian Thalinger wrote: >>>>>>> >>>>>>> On Feb 27, 2014, at 4:33 PM, Vladimir Kozlov wrote: >>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8035983 >>>>>>>> http://cr.openjdk.java.net/~kvn/8035983/webrev/ >>>>>>>> >>>>>>>> We don't print whole stack if native frames intermix with compiled java frames (on x86 fp is used by compiled code). >>>>>>>> Instead of using os::is_first_C_frame() which produces incorrect result for compiled java frames I am suggesting to look on frame's stack pointer relative to stack base. >>>>>>> >>>>>>> I have two questions: >>>>>>> >>>>>>> 1) Why does this work: >>>>>>> >>>>>>> + // Catch very first native frame by using stack address. >>>>>>> + if (!_thread->on_local_stack((address)(fr.sp() + 4))) break; >>>>>>> >>>>>>> Because we set the stack base to the current value when we start the thread? >>>>>> >>>>>> You are right not all threads have stack base set (all VM's thread do). So I need to add check for stack_size() != 0 and use is_first_C_frame() if it is 0 (uninitialized). >>>>> >>>>> Maybe add a comment why the on_local_stack() check works. >>>>> >>>>>> >>>>>>> >>>>>>> 2) Why does this not work? I have seen many correct stack traces in the past. >>>>>> >>>>>> Look on os::is_first_C_frame(). It does some stupid simple checks which nothing to do with real value. The test will pass if a java compiled code has a pointer in EBP register. >>>>> >>>>> Ah, that?s what I was missing. Makes sense then. Yeah, the code in os::is_first_C_frame() is weird :-) >>>>> >>>>>> >>>>>> Thanks, >>>>>> Vladimir >>>>>> >>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Vladimir >>>>>>>> >>>>>>>> Output before the fix: >>>>>>>> >>>>>>>> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) >>>>>>>> V [libjvm.so+0x1e28428] void VMError::report(outputStream*)+0x1478 >>>>>>>> V [libjvm.so+0x1e29dd4] void VMError::report_and_die()+0x6b4 >>>>>>>> V [libjvm.so+0x14ad9ba] void report_vm_error(const char*,int,const char*,const char*)+0x9a >>>>>>>> V [libjvm.so+0x1b6ccf5] void ObjectMonitor::exit(bool,Thread*)+0x125 >>>>>>>> V [libjvm.so+0x1d41cda] void ObjectSynchronizer::fast_exit(oopDesc*,BasicLock*,Thread*)+0x38a >>>>>>>> V [libjvm.so+0x1d41fba] void ObjectSynchronizer::slow_exit(oopDesc*,BasicLock*,Thread*)+0x2a >>>>>>>> V [libjvm.so+0x1caa13f] void SharedRuntime::complete_monitor_unlocking_C(oopDesc*,BasicLock*)+0x27f >>>>>>>> >>>>>>>> >>>>>>>> After the fix: >>>>>>>> >>>>>>>> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) >>>>>>>> V [libjvm.so+0x1e28428] void VMError::report(outputStream*)+0x1478 >>>>>>>> V [libjvm.so+0x1e29dd4] void VMError::report_and_die()+0x6b4 >>>>>>>> V [libjvm.so+0x14ad9ba] void report_vm_error(const char*,int,const char*,const char*)+0x9a >>>>>>>> V [libjvm.so+0x1b6ccf5] void ObjectMonitor::exit(bool,Thread*)+0x125 >>>>>>>> V [libjvm.so+0x1d41cda] void ObjectSynchronizer::fast_exit(oopDesc*,BasicLock*,Thread*)+0x38a >>>>>>>> V [libjvm.so+0x1d41fba] void ObjectSynchronizer::slow_exit(oopDesc*,BasicLock*,Thread*)+0x2a >>>>>>>> V [libjvm.so+0x1caa13f] void SharedRuntime::complete_monitor_unlocking_C(oopDesc*,BasicLock*)+0x27f >>>>>>>> J 13 C2 java.util.Hashtable.get(Ljava/lang/Object;)Ljava/lang/Object; (69 bytes) @ 0xffff80ffb979da84 [0xffff80ffb979d700+0x384] >>>>>>>> J 10% C2 Test.Worker()V (381 bytes) @ 0xffff80ffb97a1c8c [0xffff80ffb97a1840+0x44c] >>>>>>>> j Test$1.run()V+0 >>>>>>>> v ~StubRoutines::call_stub >>>>>>>> V [libjvm.so+0x177d196] void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*)+0x886 >>>>>>>> V [libjvm.so+0x1b9a6c8] void os::os_exception_wrapper(void(*)(JavaValue*,methodHandle*,JavaCallArguments*,Thread*),JavaValue*,methodHandle*,JavaCallArguments*,Thread*)+0x38 >>>>>>>> V [libjvm.so+0x177c90a] void JavaCalls::call(JavaValue*,methodHandle,JavaCallArguments*,Thread*)+0x9a >>>>>>>> V [libjvm.so+0x177be70] void JavaCalls::call_virtual(JavaValue*,KlassHandle,Symbol*,Symbol*,JavaCallArguments*,Thread*)+0x250 >>>>>>>> V [libjvm.so+0x177bf56] void JavaCalls::call_virtual(JavaValue*,Handle,KlassHandle,Symbol*,Symbol*,Thread*)+0x66 >>>>>>>> V [libjvm.so+0x1896897] void thread_entry(JavaThread*,Thread*)+0xc7 >>>>>>>> V [libjvm.so+0x1d8f23e] void JavaThread::thread_main_inner()+0x18e >>>>>>>> V [libjvm.so+0x1d8f09a] void JavaThread::run()+0x1fa >>>>>>>> V [libjvm.so+0x1b8f3c8] java_start+0x248 >>>>>>>> C [libc.so.1+0x122105] _thrp_setup+0xa5 >>>>>>>> C [libc.so.1+0x1223b0] _lwp_start+0x0 >>>>> >>> > From vladimir.x.ivanov at oracle.com Mon Mar 3 17:42:23 2014 From: vladimir.x.ivanov at oracle.com (vladimir.x.ivanov at oracle.com) Date: Tue, 04 Mar 2014 01:42:23 +0000 Subject: hg: hsx/hotspot-main/hotspot: 8036100: Default method returns true for a while, and then returns false Message-ID: <20140304014230.4FA2E62480@hg.openjdk.java.net> Changeset: e46f2ee62e78 Author: vlivanov Date: 2014-03-03 16:10 -0800 URL: http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/e46f2ee62e78 8036100: Default method returns true for a while, and then returns false Reviewed-by: kvn, jrose ! src/share/vm/ci/ciMethod.cpp + test/compiler/inlining/InlineDefaultMethod1.java From christian.thalinger at oracle.com Mon Mar 3 17:48:14 2014 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Mon, 3 Mar 2014 17:48:14 -0800 Subject: RFR(L): 8029075 - String deduplication in G1 In-Reply-To: <53148494.60806@oracle.com> References: <53148494.60806@oracle.com> Message-ID: <3A26A868-51A8-4169-B147-9AEB8B35EC5E@oracle.com> Just a general question: Have you thought about writing the deduplication thread in Java? Maybe also the deduplication table? There is clearly the problem of enqueuing. One easy solution would be to have the deduplication thread poll into the VM to get the next entries but there might be a better away to do it. On Mar 3, 2014, at 5:33 AM, Per Liden wrote: > Hi, > > Could I please have this patch reviewed. > > > Summary > ------- > This patch implements JEP 192 - String deduplication in G1. The goal of string deduplication is to reduce the Java heap live-data set by enhancing the G1 garbage collector so that duplicate instances of String are automatically and continuously deduplicated. > > I'd like to refer to the JEP for a more detailed description of this feature, the motivation for it, the expected benefit, how it's implemented, etc. > > This patch is mainly G1-related, but also touches a few runtime files. > > Webrev: http://cr.openjdk.java.net/~pliden/8029075/webrev.0/ > > JEP: http://openjdk.java.net/jeps/192 > > RFE: https://bugs.openjdk.java.net/browse/JDK-8029075 > > > Testing > ------- > * JTreg - The patch includes the following 8 new tests: > - TestStringDeduplicationYoungGC.java: Tests deduplication during Young GC. > - TestStringDeduplicationFullGC.java: Tests deduplication during Full GC. > - TestStringDeduplicationAgeThreshold.java: Tests both valid and invalid age threshold settings. > - TestStringDeduplicationInterned.java: Tests that interned strings are deduplication before being interned. > - TestStringDeduplicationTableRehash.java: Stresses the hashtables ability to rehash all entries. > - TestStringDeduplicationTableResize.java: Stresses the hashtables ability to resize itself. > - TestStringDeduplicationMemoryUsage.java: Tests heap reduction when string deduplication is enabled. > - TestStringDeduplicationPrintOptions.java: Tests command line options. > > * Stress testing: > - Kitchensink > - GCBasher > > * Regression testing: > - JCK > - vmTestbase > - Bigapps > > * Large scale benchmarks to test heap reduction and performance impact: > - FA CRM Sales Op. Flow > - FA DSS > > * The following benchmarks have been executed to verify that this feature doesn't impact performance when disabled (even when disabled there are still some "if (UseStringDeduplication)" executed in some hot paths). > - SPECjbb2005 > - SPECjbb2013 > - SPECjvm2008-XML > > * Various ad-hoc tests and microbenchmarks were also written and executed during the course of the development. > > > Thanks! > /Per From christian.thalinger at oracle.com Mon Mar 3 17:49:51 2014 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Mon, 3 Mar 2014 17:49:51 -0800 Subject: Bring Zero to llvm 3.4 In-Reply-To: <1393854861.3834.8.camel@galactica.localdomain> References: <1393239625.3873.13.camel@galactica.localdomain> <1393256374.11941.3.camel@galactica.localdomain> <1393853313.3834.6.camel@galactica.localdomain> <1581851.jFBPTQiW9h@mgerdin03> <1393854861.3834.8.camel@galactica.localdomain> Message-ID: <38D22382-B630-4EF3-9B20-02D380484155@oracle.com> On Mar 3, 2014, at 5:54 AM, Mario Torre wrote: > On Mon, 2014-03-03 at 14:44 +0100, Mikael Gerdin wrote: >> Hi Mario >> >> On Monday 03 March 2014 14.28.33 Mario Torre wrote: >>> On Mon, 2014-02-24 at 16:39 +0100, Mario Torre wrote: >>>> On Mon, 2014-02-24 at 15:50 +0100, Mikael Gerdin wrote: >>>>>> Interesting... >>>>> >>>>> Indeed, what version of webrev are you using? >>>>> The .changeset file created by recent versions of webrev has the >>>>> advantage >>>>> that it contains HG meta-information about which change the patch was >>>>> created from. >>>> >>>> I used the one that comes from within the JDK8 repository (cloned >>>> sometime on Friday). I run webrev from a mercurial patch queue with the >>>> applied patch, which should be visible as a committed changeset I think. >>>> >>>> Cheers, >>>> Mario >>> >>> Hello Mikael, >>> >>> The patch applied without modification on JDK9, here's the actual >>> webrev: >>> >>> http://cr.openjdk.java.net/~neugens/zero-2014-02-24/webrev.02/ >>> >>> If the patch is OK, can you please push it for me? >> >> Unfortunately I'm not too familiar with the Zero/Shark code or the makefiles >> so I wouldn't feel comfortable putting my name as a reviewer on these changes. >> >> I'm sure that whoever reviews this change can push it for you as well. >> >> /Mikael > > Hi Mikael, > > Thanks. > > Anyone that can review it? Christian maybe? Sure. Looks good to me. > > Cheers, > Mario From daniel.daugherty at oracle.com Mon Mar 3 19:02:34 2014 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 03 Mar 2014 20:02:34 -0700 Subject: JDK-8036003: Add variable not to separate debug information. In-Reply-To: <20140303214923.GA26825@redhat.com> References: <20140228091835.58EB8C21F24@msgw007-05.ocn.ad.jp> <53106F4A.8030905@oracle.com> <53109772.9070808@oracle.com> <5310AD70.2070908@ysfactory.dip.jp> <53112019.9050504@oracle.com> <20140303214923.GA26825@redhat.com> Message-ID: <5315424A.4080201@oracle.com> On 3/3/14 2:49 PM, Omair Majid wrote: > * David Holmes [2014-02-28 18:48]: >> There are three pieces to all of this: >> >> 1. Generating debug symbols in the binaries (via gcc -g or whatever) >> 2. Generating debuginfo files (zipped or not) (FDS) >> 3. Stripping debug symbols from the binaries (strip-policy) >> >> It may be that we don't have clean separation between them, and if so >> that should be fixed, but I don't see the current proposal as the way >> forward. > Any chance we could clean up the names too? It's not obvious to me why > FDS means 'generating debuginfo files'. FDS stands for Full Debug Symbols and is defined that way in quite a few Makefiles... We just call it FDS for short... > What do folks think of modifying the OPENJDK build to use this as the > settings for the options above by default. > > 1. Yes > 2. No > 3. No Whatever we do here has be architecturallyconsistent across the various OpenJDK platforms. The above settings would not work well on non-Linux platforms so I'm not in favor of doing that. However, I don't have a problem with providing support for config'ing a build to work with the downstream Linux repos... >> Also there may well be differences between how things are handled on the >> JDK side and hotspot side. > Is there any chance of this becoming consistent between hotspot and JDK? > It would be much nicer if concepts like strip-policy were consistent > between hotspot and the rest of the JDK. When I did the Full Debug Symbols (FDS) project, I made it architecturally consistent between the HotSpot and JDK sides. Since the new build system has arrived, something of the consistency has been broken and I haven't been back on the JDK side to see what happened... Dan > > Thanks, > Omair > From alejandro.murillo at oracle.com Mon Mar 3 19:13:53 2014 From: alejandro.murillo at oracle.com (alejandro.murillo at oracle.com) Date: Tue, 04 Mar 2014 03:13:53 +0000 Subject: hg: hsx/hsx25/hotspot: 11 new changesets Message-ID: <20140304031420.239726248F@hg.openjdk.java.net> Changeset: ce0320cdb075 Author: jeff Date: 2014-01-28 20:09 +0000 URL: http://hg.openjdk.java.net/hsx/hsx25/hotspot/rev/ce0320cdb075 8032816: THIRDPARTYREADME LittleCMS preamble missing JRE 8 & JDK 8 Reviewed-by: lana ! THIRD_PARTY_README Changeset: 35038da7bb9d Author: lana Date: 2014-01-29 11:11 -0800 URL: http://hg.openjdk.java.net/hsx/hsx25/hotspot/rev/35038da7bb9d Merge Changeset: 874c0b4a946c Author: katleman Date: 2014-01-30 12:16 -0800 URL: http://hg.openjdk.java.net/hsx/hsx25/hotspot/rev/874c0b4a946c Added tag jdk8-b127 for changeset 35038da7bb9d ! .hgtags Changeset: cb39165c4a65 Author: katleman Date: 2014-02-01 18:21 -0800 URL: http://hg.openjdk.java.net/hsx/hsx25/hotspot/rev/cb39165c4a65 Added tag jdk8-b128 for changeset 874c0b4a946c ! .hgtags Changeset: 1dbaf664a611 Author: katleman Date: 2014-02-06 17:34 -0800 URL: http://hg.openjdk.java.net/hsx/hsx25/hotspot/rev/1dbaf664a611 Added tag jdk8-b129 for changeset cb39165c4a65 ! .hgtags Changeset: b5e7ebfe185c Author: katleman Date: 2014-02-28 10:06 -0800 URL: http://hg.openjdk.java.net/hsx/hsx25/hotspot/rev/b5e7ebfe185c Added tag jdk8-b130 for changeset 1dbaf664a611 ! .hgtags Changeset: 5380dc5d007e Author: katleman Date: 2014-02-28 13:36 -0800 URL: http://hg.openjdk.java.net/hsx/hsx25/hotspot/rev/5380dc5d007e Added tag jdk8-b131 for changeset b5e7ebfe185c ! .hgtags Changeset: 54f0c207dc35 Author: amurillo Date: 2014-01-28 15:11 -0800 URL: http://hg.openjdk.java.net/hsx/hsx25/hotspot/rev/54f0c207dc35 8032984: new hotspot build - hs25-b70 Reviewed-by: jcoomes ! make/hotspot_version Changeset: e46f2ee62e78 Author: vlivanov Date: 2014-03-03 16:10 -0800 URL: http://hg.openjdk.java.net/hsx/hsx25/hotspot/rev/e46f2ee62e78 8036100: Default method returns true for a while, and then returns false Reviewed-by: kvn, jrose ! src/share/vm/ci/ciMethod.cpp + test/compiler/inlining/InlineDefaultMethod1.java Changeset: 9f9179e8f0cf Author: amurillo Date: 2014-03-03 17:48 -0800 URL: http://hg.openjdk.java.net/hsx/hsx25/hotspot/rev/9f9179e8f0cf Merge Changeset: 0c94c41dcd70 Author: amurillo Date: 2014-03-03 17:48 -0800 URL: http://hg.openjdk.java.net/hsx/hsx25/hotspot/rev/0c94c41dcd70 Added tag hs25-b70 for changeset 9f9179e8f0cf ! .hgtags From junjie.qian at yahoo.com Mon Mar 3 19:48:28 2014 From: junjie.qian at yahoo.com (Junjie Qian) Date: Mon, 3 Mar 2014 19:48:28 -0800 (PST) Subject: Profile the lock usage in openjdk Message-ID: <1393904908.69184.YahooMailNeo@web124901.mail.ne1.yahoo.com> Dear all, I am one graduate student working on Java VM, and would like to study the multithread in Java benchmarks. The information I want to get is the lock contentions in the benchmark, such as fatlock/thinLock/SpinLock number, or the time spent on waiting for these lock. Could you give me some advices on this, as how to insert code in openjdk to get such info? Thanks! Best Junjie? From david.holmes at oracle.com Mon Mar 3 20:14:13 2014 From: david.holmes at oracle.com (David Holmes) Date: Tue, 04 Mar 2014 14:14:13 +1000 Subject: Profile the lock usage in openjdk In-Reply-To: <1393904908.69184.YahooMailNeo@web124901.mail.ne1.yahoo.com> References: <1393904908.69184.YahooMailNeo@web124901.mail.ne1.yahoo.com> Message-ID: <53155315.2080209@oracle.com> On 4/03/2014 1:48 PM, Junjie Qian wrote: > Dear all, > > I am one graduate student working on Java VM, and would like to study the multithread in Java benchmarks. The information I want to get is the lock contentions in the benchmark, such as fatlock/thinLock/SpinLock number, or the time spent on waiting for these lock. > > Could you give me some advices on this, as how to insert code in openjdk to get such info? You will find that there is already a lot of information gathered within the VM, but some probably only for non-product builds and most only for the contention case where the monitor is inflated. The problem with overly instrumenting the lock code to "measure" contention is that the instrumentation code completely changes the contention patterns. Adding additional instrumentation is non-trivial as you need to handle both interpreted and JIT'd code. David > Thanks! > Best > Junjie > From alejandro.murillo at oracle.com Mon Mar 3 20:37:39 2014 From: alejandro.murillo at oracle.com (alejandro.murillo at oracle.com) Date: Tue, 04 Mar 2014 04:37:39 +0000 Subject: hg: hsx/hotspot-main/hotspot: 9 new changesets Message-ID: <20140304043757.AB8C862492@hg.openjdk.java.net> Changeset: ce0320cdb075 Author: jeff Date: 2014-01-28 20:09 +0000 URL: http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/ce0320cdb075 8032816: THIRDPARTYREADME LittleCMS preamble missing JRE 8 & JDK 8 Reviewed-by: lana ! THIRD_PARTY_README Changeset: 35038da7bb9d Author: lana Date: 2014-01-29 11:11 -0800 URL: http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/35038da7bb9d Merge Changeset: 874c0b4a946c Author: katleman Date: 2014-01-30 12:16 -0800 URL: http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/874c0b4a946c Added tag jdk8-b127 for changeset 35038da7bb9d ! .hgtags Changeset: cb39165c4a65 Author: katleman Date: 2014-02-01 18:21 -0800 URL: http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/cb39165c4a65 Added tag jdk8-b128 for changeset 874c0b4a946c ! .hgtags Changeset: 1dbaf664a611 Author: katleman Date: 2014-02-06 17:34 -0800 URL: http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/1dbaf664a611 Added tag jdk8-b129 for changeset cb39165c4a65 ! .hgtags Changeset: b5e7ebfe185c Author: katleman Date: 2014-02-28 10:06 -0800 URL: http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/b5e7ebfe185c Added tag jdk8-b130 for changeset 1dbaf664a611 ! .hgtags Changeset: 5380dc5d007e Author: katleman Date: 2014-02-28 13:36 -0800 URL: http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/5380dc5d007e Added tag jdk8-b131 for changeset b5e7ebfe185c ! .hgtags Changeset: 9f9179e8f0cf Author: amurillo Date: 2014-03-03 17:48 -0800 URL: http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/9f9179e8f0cf Merge Changeset: 0c94c41dcd70 Author: amurillo Date: 2014-03-03 17:48 -0800 URL: http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/0c94c41dcd70 Added tag hs25-b70 for changeset 9f9179e8f0cf ! .hgtags From jiangli.zhou at oracle.com Mon Mar 3 21:55:09 2014 From: jiangli.zhou at oracle.com (jiangli.zhou at oracle.com) Date: Tue, 04 Mar 2014 05:55:09 +0000 Subject: hg: hsx/jdk7u/hotspot: 8006023: Embedded Builds fail management test because of requirement for UsePerfData being enabled. Message-ID: <20140304055524.27D9D62496@hg.openjdk.java.net> Changeset: 1b9722b5134a Author: jiangli Date: 2013-07-01 19:44 -0400 URL: http://hg.openjdk.java.net/hsx/jdk7u/hotspot/rev/1b9722b5134a 8006023: Embedded Builds fail management test because of requirement for UsePerfData being enabled. Summary: Added -XX:+UsePerfData to Test7196045.java. Reviewed-by: dholmes, collins ! test/runtime/7196045/Test7196045.java From jeremymanson at google.com Mon Mar 3 21:55:42 2014 From: jeremymanson at google.com (Jeremy Manson) Date: Mon, 3 Mar 2014 21:55:42 -0800 Subject: Profile the lock usage in openjdk In-Reply-To: <53155315.2080209@oracle.com> References: <1393904908.69184.YahooMailNeo@web124901.mail.ne1.yahoo.com> <53155315.2080209@oracle.com> Message-ID: Of course, if all you want is a coarse measurement, you can use JVMTI. Just have a timer that starts at the first one of these, and stops at the second: http://docs.oracle.com/javase/7/docs/platform/jvmti/jvmti.html#MonitorContendedEnter http://docs.oracle.com/javase/7/docs/platform/jvmti/jvmti.html#MonitorContendedEntered We instrument these on a sampling basis across large numbers of invocations. It doesn't perturb the application behavior too much. Jeremy On Mon, Mar 3, 2014 at 8:14 PM, David Holmes wrote: > On 4/03/2014 1:48 PM, Junjie Qian wrote: > >> Dear all, >> >> I am one graduate student working on Java VM, and would like to study the >> multithread in Java benchmarks. The information I want to get is the lock >> contentions in the benchmark, such as fatlock/thinLock/SpinLock number, or >> the time spent on waiting for these lock. >> >> Could you give me some advices on this, as how to insert code in openjdk >> to get such info? >> > > You will find that there is already a lot of information gathered within > the VM, but some probably only for non-product builds and most only for the > contention case where the monitor is inflated. The problem with overly > instrumenting the lock code to "measure" contention is that the > instrumentation code completely changes the contention patterns. Adding > additional instrumentation is non-trivial as you need to handle both > interpreted and JIT'd code. > > David > > > Thanks! >> Best >> Junjie >> >> From jeremymanson at google.com Mon Mar 3 22:02:11 2014 From: jeremymanson at google.com (Jeremy Manson) Date: Mon, 3 Mar 2014 22:02:11 -0800 Subject: Profile the lock usage in openjdk In-Reply-To: References: <1393904908.69184.YahooMailNeo@web124901.mail.ne1.yahoo.com> <53155315.2080209@oracle.com> Message-ID: Oh, and you can get an even more coarse measurement of contention from ThreadMXBean: http://docs.oracle.com/javase/7/docs/api/java/lang/management/ThreadMXBean.html Jeremy On Mon, Mar 3, 2014 at 9:55 PM, Jeremy Manson wrote: > Of course, if all you want is a coarse measurement, you can use JVMTI. > Just have a timer that starts at the first one of these, and stops at the > second: > > > http://docs.oracle.com/javase/7/docs/platform/jvmti/jvmti.html#MonitorContendedEnter > > http://docs.oracle.com/javase/7/docs/platform/jvmti/jvmti.html#MonitorContendedEntered > > We instrument these on a sampling basis across large numbers of > invocations. It doesn't perturb the application behavior too much. > > Jeremy > > > On Mon, Mar 3, 2014 at 8:14 PM, David Holmes wrote: > >> On 4/03/2014 1:48 PM, Junjie Qian wrote: >> >>> Dear all, >>> >>> I am one graduate student working on Java VM, and would like to study >>> the multithread in Java benchmarks. The information I want to get is the >>> lock contentions in the benchmark, such as fatlock/thinLock/SpinLock >>> number, or the time spent on waiting for these lock. >>> >>> Could you give me some advices on this, as how to insert code in openjdk >>> to get such info? >>> >> >> You will find that there is already a lot of information gathered within >> the VM, but some probably only for non-product builds and most only for the >> contention case where the monitor is inflated. The problem with overly >> instrumenting the lock code to "measure" contention is that the >> instrumentation code completely changes the contention patterns. Adding >> additional instrumentation is non-trivial as you need to handle both >> interpreted and JIT'd code. >> >> David >> >> >> Thanks! >>> Best >>> Junjie >>> >>> > From david.holmes at oracle.com Mon Mar 3 23:31:57 2014 From: david.holmes at oracle.com (David Holmes) Date: Tue, 04 Mar 2014 17:31:57 +1000 Subject: Memory ordering in C2 In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CEA4CB2@DEWDFEMB12A.global.corp.sap> References: <530CA489.5060805@redhat.com> <4295855A5C1DE049A61835A1887419CC2CEA469C@DEWDFEMB12A.global.corp.sap> <530CB709.1040205@redhat.com> <4295855A5C1DE049A61835A1887419CC2CEA46C6@DEWDFEMB12A.global.corp.sap> <530D6E70.20206@oracle.com> <530DB953.3060703@redhat.com> <4295855A5C1DE049A61835A1887419CC2CEA4CB2@DEWDFEMB12A.global.corp.sap> Message-ID: <5315816D.8040303@oracle.com> On 27/02/2014 8:13 AM, Lindenmaier, Goetz wrote: > Hi, > > We have a load that does the Acquire itself. So that we don't get a > useless lwsync instruction after the load, the match rule for > MemBarAcquire is empty, i.e., does not emit an instruction. I'm not a compiler person so grant me some slack here :) but it sounds to me like you want to be able to express the requirements as distinct nodes eg: load -> membarAcquire -> and then fold the membar back into the load if the platform has a loadWithAcquire capability. David ----- > Best regards, > Goetz. > > > > -----Original Message----- > From: Andrew Haley [mailto:aph at redhat.com] > Sent: Wednesday, February 26, 2014 10:52 AM > To: David Holmes; Lindenmaier, Goetz; hotspot-dev Source Developers > Subject: Re: Memory ordering in C2 > > On 02/26/2014 04:32 AM, David Holmes wrote: >> On 26/02/2014 1:41 AM, Lindenmaier, Goetz wrote: >>> Hi, >>> >>>> From what I've seen, C2 is enthusiastically emitting barrier nodes >>>> around CompareAndSwap, so I'm not quite sure what this means. >>> Yes, you are right ... that's again because we implement MemBArAcquire empty, >>> but after the CompareAndSwap we need one, so we had to add it in the node ;) >>> So same issue, distinguishing nodes would help! The bad thing is that MemBarAcquire >>> is implemented empty, that spoils the other uses. If there is a way to avoid MemBarAcquire >>> where it needs to be empty (after volatile load), then it can be implemented >>> properly ... >> >> I'm unclear what you mean by "needs to be empty" ?? > > "Does not need to be emitted," I think. > > Andrew. > From neugens at redhat.com Tue Mar 4 00:06:51 2014 From: neugens at redhat.com (Mario Torre) Date: Tue, 04 Mar 2014 09:06:51 +0100 Subject: Bring Zero to llvm 3.4 In-Reply-To: <38D22382-B630-4EF3-9B20-02D380484155@oracle.com> References: <1393239625.3873.13.camel@galactica.localdomain> <1393256374.11941.3.camel@galactica.localdomain> <1393853313.3834.6.camel@galactica.localdomain> <1581851.jFBPTQiW9h@mgerdin03> <1393854861.3834.8.camel@galactica.localdomain> <38D22382-B630-4EF3-9B20-02D380484155@oracle.com> Message-ID: <1393920411.3834.12.camel@galactica.localdomain> On Mon, 2014-03-03 at 17:49 -0800, Christian Thalinger wrote: > On Mar 3, 2014, at 5:54 AM, Mario Torre wrote: > > > On Mon, 2014-03-03 at 14:44 +0100, Mikael Gerdin wrote: > >> Hi Mario > >> > >> On Monday 03 March 2014 14.28.33 Mario Torre wrote: > >>> On Mon, 2014-02-24 at 16:39 +0100, Mario Torre wrote: > >>>> On Mon, 2014-02-24 at 15:50 +0100, Mikael Gerdin wrote: > >>>>>> Interesting... > >>>>> > >>>>> Indeed, what version of webrev are you using? > >>>>> The .changeset file created by recent versions of webrev has the > >>>>> advantage > >>>>> that it contains HG meta-information about which change the patch was > >>>>> created from. > >>>> > >>>> I used the one that comes from within the JDK8 repository (cloned > >>>> sometime on Friday). I run webrev from a mercurial patch queue with the > >>>> applied patch, which should be visible as a committed changeset I think. > >>>> > >>>> Cheers, > >>>> Mario > >>> > >>> Hello Mikael, > >>> > >>> The patch applied without modification on JDK9, here's the actual > >>> webrev: > >>> > >>> http://cr.openjdk.java.net/~neugens/zero-2014-02-24/webrev.02/ > >>> > >>> If the patch is OK, can you please push it for me? > >> > >> Unfortunately I'm not too familiar with the Zero/Shark code or the makefiles > >> so I wouldn't feel comfortable putting my name as a reviewer on these changes. > >> > >> I'm sure that whoever reviews this change can push it for you as well. > >> > >> /Mikael > > > > Hi Mikael, > > > > Thanks. > > > > Anyone that can review it? Christian maybe? > > Sure. Looks good to me. > Thanks a ton! I guess I can't push it myself, right? :) Can you please push the patch on 9 on my behalf? I also would like to push it on 8u, but I think I need to ask a backport on the 8u mailing list fist, right? Cheers, Mario From per.liden at oracle.com Tue Mar 4 01:34:02 2014 From: per.liden at oracle.com (Per Liden) Date: Tue, 04 Mar 2014 10:34:02 +0100 Subject: RFR(L): 8029075 - String deduplication in G1 In-Reply-To: <3A26A868-51A8-4169-B147-9AEB8B35EC5E@oracle.com> References: <53148494.60806@oracle.com> <3A26A868-51A8-4169-B147-9AEB8B35EC5E@oracle.com> Message-ID: <53159E0A.60403@oracle.com> Hi Christian, That was something Mikael Vidstedt and I briefly touched on when we first started to think about deduplication. However, we didn't go down that path, partly because of the problem you mention, but also because of the general overhead of WeakReferences (in terms of memory and work added to the reference processor and Reference Handler). /Per On 03/04/2014 02:48 AM, Christian Thalinger wrote: > Just a general question: Have you thought about writing the deduplication thread in Java? Maybe also the deduplication table? > > There is clearly the problem of enqueuing. One easy solution would be to have the deduplication thread poll into the VM to get the next entries but there might be a better away to do it. > > On Mar 3, 2014, at 5:33 AM, Per Liden wrote: > >> Hi, >> >> Could I please have this patch reviewed. >> >> >> Summary >> ------- >> This patch implements JEP 192 - String deduplication in G1. The goal of string deduplication is to reduce the Java heap live-data set by enhancing the G1 garbage collector so that duplicate instances of String are automatically and continuously deduplicated. >> >> I'd like to refer to the JEP for a more detailed description of this feature, the motivation for it, the expected benefit, how it's implemented, etc. >> >> This patch is mainly G1-related, but also touches a few runtime files. >> >> Webrev: http://cr.openjdk.java.net/~pliden/8029075/webrev.0/ >> >> JEP: http://openjdk.java.net/jeps/192 >> >> RFE: https://bugs.openjdk.java.net/browse/JDK-8029075 >> >> >> Testing >> ------- >> * JTreg - The patch includes the following 8 new tests: >> - TestStringDeduplicationYoungGC.java: Tests deduplication during Young GC. >> - TestStringDeduplicationFullGC.java: Tests deduplication during Full GC. >> - TestStringDeduplicationAgeThreshold.java: Tests both valid and invalid age threshold settings. >> - TestStringDeduplicationInterned.java: Tests that interned strings are deduplication before being interned. >> - TestStringDeduplicationTableRehash.java: Stresses the hashtables ability to rehash all entries. >> - TestStringDeduplicationTableResize.java: Stresses the hashtables ability to resize itself. >> - TestStringDeduplicationMemoryUsage.java: Tests heap reduction when string deduplication is enabled. >> - TestStringDeduplicationPrintOptions.java: Tests command line options. >> >> * Stress testing: >> - Kitchensink >> - GCBasher >> >> * Regression testing: >> - JCK >> - vmTestbase >> - Bigapps >> >> * Large scale benchmarks to test heap reduction and performance impact: >> - FA CRM Sales Op. Flow >> - FA DSS >> >> * The following benchmarks have been executed to verify that this feature doesn't impact performance when disabled (even when disabled there are still some "if (UseStringDeduplication)" executed in some hot paths). >> - SPECjbb2005 >> - SPECjbb2013 >> - SPECjvm2008-XML >> >> * Various ad-hoc tests and microbenchmarks were also written and executed during the course of the development. >> >> >> Thanks! >> /Per > From junjie.qian at yahoo.com Tue Mar 4 08:04:45 2014 From: junjie.qian at yahoo.com (Junjie Qian) Date: Tue, 4 Mar 2014 08:04:45 -0800 (PST) Subject: Profile the lock usage in openjdk In-Reply-To: References: <1393904908.69184.YahooMailNeo@web124901.mail.ne1.yahoo.com> <53155315.2080209@oracle.com> Message-ID: <1393949085.45661.YahooMailNeo@web124901.mail.ne1.yahoo.com> Dear all, First thank you so much for your suggestions! I need to spent some time on these documents, and try to work with them first. I will let you know if I have any updates or problems on it. Thanks! Best Junjie On Tuesday, March 4, 2014 12:02 AM, Jeremy Manson wrote: Oh, and you can get an even more coarse measurement of contention from ThreadMXBean: http://docs.oracle.com/javase/7/docs/api/java/lang/management/ThreadMXBean.html Jeremy On Mon, Mar 3, 2014 at 9:55 PM, Jeremy Manson wrote: Of course, if all you want is a coarse measurement, you can use JVMTI. ?Just have a timer that starts at the first one of these, and stops at the second: > > >http://docs.oracle.com/javase/7/docs/platform/jvmti/jvmti.html#MonitorContendedEnter > >http://docs.oracle.com/javase/7/docs/platform/jvmti/jvmti.html#MonitorContendedEntered > > > >We instrument these on a sampling basis across large numbers of invocations. ?It doesn't perturb the application behavior too much. > > >Jeremy > > > >On Mon, Mar 3, 2014 at 8:14 PM, David Holmes wrote: > >On 4/03/2014 1:48 PM, Junjie Qian wrote: >> >>Dear all, >>> >>>I am one graduate student working on Java VM, and would like to study the multithread in Java benchmarks. The information I want to get is the lock contentions in the benchmark, such as fatlock/thinLock/SpinLock number, or the time spent on waiting for these lock. >>> >>>Could you give me some advices on this, as how to insert code in openjdk to get such info? >>> >> You will find that there is already a lot of information gathered within the VM, but some probably only for non-product builds and most only for the contention case where the monitor is inflated. The problem with overly instrumenting the lock code to "measure" contention is that the instrumentation code completely changes the contention patterns. Adding additional instrumentation is non-trivial as you need to handle both interpreted and JIT'd code. >> >>David >> >> >> >>Thanks! >>>Best >>>Junjie >>> >>> >? From volker.simonis at gmail.com Tue Mar 4 08:27:50 2014 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 4 Mar 2014 17:27:50 +0100 Subject: RFR[9](XS): 8036614: AIX: fix adjust-mflags.sh to build with GNU Make 4.0 (adapt 8028407 for AIX) Message-ID: Hi, can somebody please review and sponsor the following little change which is actually just a "copy and paste" of "8028407 adjust-mflags.sh failed build with GNU Make 4.0 with -I" for the AIX version of adjust-mflags.sh: http://cr.openjdk.java.net/~simonis/webrevs/8036614 https://bugs.openjdk.java.net/browse/JDK-8036614 Thank you and best regards, Volker From christian.thalinger at oracle.com Tue Mar 4 08:55:33 2014 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Tue, 4 Mar 2014 08:55:33 -0800 Subject: RFR(L): 8029075 - String deduplication in G1 In-Reply-To: <53159E0A.60403@oracle.com> References: <53148494.60806@oracle.com> <3A26A868-51A8-4169-B147-9AEB8B35EC5E@oracle.com> <53159E0A.60403@oracle.com> Message-ID: <5A84BB3B-9763-47C3-9EEF-0D07EB763576@oracle.com> That?s unfortunate. It clearly would have been easier to do all that work in Java. On Mar 4, 2014, at 1:34 AM, Per Liden wrote: > Hi Christian, > > That was something Mikael Vidstedt and I briefly touched on when we first started to think about deduplication. However, we didn't go down that path, partly because of the problem you mention, but also because of the general overhead of WeakReferences (in terms of memory and work added to the reference processor and Reference Handler). > > /Per > > On 03/04/2014 02:48 AM, Christian Thalinger wrote: >> Just a general question: Have you thought about writing the deduplication thread in Java? Maybe also the deduplication table? >> >> There is clearly the problem of enqueuing. One easy solution would be to have the deduplication thread poll into the VM to get the next entries but there might be a better away to do it. >> >> On Mar 3, 2014, at 5:33 AM, Per Liden wrote: >> >>> Hi, >>> >>> Could I please have this patch reviewed. >>> >>> >>> Summary >>> ------- >>> This patch implements JEP 192 - String deduplication in G1. The goal of string deduplication is to reduce the Java heap live-data set by enhancing the G1 garbage collector so that duplicate instances of String are automatically and continuously deduplicated. >>> >>> I'd like to refer to the JEP for a more detailed description of this feature, the motivation for it, the expected benefit, how it's implemented, etc. >>> >>> This patch is mainly G1-related, but also touches a few runtime files. >>> >>> Webrev: http://cr.openjdk.java.net/~pliden/8029075/webrev.0/ >>> >>> JEP: http://openjdk.java.net/jeps/192 >>> >>> RFE: https://bugs.openjdk.java.net/browse/JDK-8029075 >>> >>> >>> Testing >>> ------- >>> * JTreg - The patch includes the following 8 new tests: >>> - TestStringDeduplicationYoungGC.java: Tests deduplication during Young GC. >>> - TestStringDeduplicationFullGC.java: Tests deduplication during Full GC. >>> - TestStringDeduplicationAgeThreshold.java: Tests both valid and invalid age threshold settings. >>> - TestStringDeduplicationInterned.java: Tests that interned strings are deduplication before being interned. >>> - TestStringDeduplicationTableRehash.java: Stresses the hashtables ability to rehash all entries. >>> - TestStringDeduplicationTableResize.java: Stresses the hashtables ability to resize itself. >>> - TestStringDeduplicationMemoryUsage.java: Tests heap reduction when string deduplication is enabled. >>> - TestStringDeduplicationPrintOptions.java: Tests command line options. >>> >>> * Stress testing: >>> - Kitchensink >>> - GCBasher >>> >>> * Regression testing: >>> - JCK >>> - vmTestbase >>> - Bigapps >>> >>> * Large scale benchmarks to test heap reduction and performance impact: >>> - FA CRM Sales Op. Flow >>> - FA DSS >>> >>> * The following benchmarks have been executed to verify that this feature doesn't impact performance when disabled (even when disabled there are still some "if (UseStringDeduplication)" executed in some hot paths). >>> - SPECjbb2005 >>> - SPECjbb2013 >>> - SPECjvm2008-XML >>> >>> * Various ad-hoc tests and microbenchmarks were also written and executed during the course of the development. >>> >>> >>> Thanks! >>> /Per >> > From christian.thalinger at oracle.com Tue Mar 4 09:25:25 2014 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Tue, 4 Mar 2014 09:25:25 -0800 Subject: Bring Zero to llvm 3.4 In-Reply-To: <1393920411.3834.12.camel@galactica.localdomain> References: <1393239625.3873.13.camel@galactica.localdomain> <1393256374.11941.3.camel@galactica.localdomain> <1393853313.3834.6.camel@galactica.localdomain> <1581851.jFBPTQiW9h@mgerdin03> <1393854861.3834.8.camel@galactica.localdomain> <38D22382-B630-4EF3-9B20-02D380484155@oracle.com> <1393920411.3834.12.camel@galactica.localdomain> Message-ID: On Mar 4, 2014, at 12:06 AM, Mario Torre wrote: > On Mon, 2014-03-03 at 17:49 -0800, Christian Thalinger wrote: >> On Mar 3, 2014, at 5:54 AM, Mario Torre wrote: >> >>> On Mon, 2014-03-03 at 14:44 +0100, Mikael Gerdin wrote: >>>> Hi Mario >>>> >>>> On Monday 03 March 2014 14.28.33 Mario Torre wrote: >>>>> On Mon, 2014-02-24 at 16:39 +0100, Mario Torre wrote: >>>>>> On Mon, 2014-02-24 at 15:50 +0100, Mikael Gerdin wrote: >>>>>>>> Interesting... >>>>>>> >>>>>>> Indeed, what version of webrev are you using? >>>>>>> The .changeset file created by recent versions of webrev has the >>>>>>> advantage >>>>>>> that it contains HG meta-information about which change the patch was >>>>>>> created from. >>>>>> >>>>>> I used the one that comes from within the JDK8 repository (cloned >>>>>> sometime on Friday). I run webrev from a mercurial patch queue with the >>>>>> applied patch, which should be visible as a committed changeset I think. >>>>>> >>>>>> Cheers, >>>>>> Mario >>>>> >>>>> Hello Mikael, >>>>> >>>>> The patch applied without modification on JDK9, here's the actual >>>>> webrev: >>>>> >>>>> http://cr.openjdk.java.net/~neugens/zero-2014-02-24/webrev.02/ >>>>> >>>>> If the patch is OK, can you please push it for me? >>>> >>>> Unfortunately I'm not too familiar with the Zero/Shark code or the makefiles >>>> so I wouldn't feel comfortable putting my name as a reviewer on these changes. >>>> >>>> I'm sure that whoever reviews this change can push it for you as well. >>>> >>>> /Mikael >>> >>> Hi Mikael, >>> >>> Thanks. >>> >>> Anyone that can review it? Christian maybe? >> >> Sure. Looks good to me. >> > > Thanks a ton! > > I guess I can't push it myself, right? :) Nope :-) > > Can you please push the patch on 9 on my behalf? Yes, I will take care of it. > > I also would like to push it on 8u, but I think I need to ask a backport > on the 8u mailing list fist, right? Correct. > > Cheers, > Mario From neugens at redhat.com Tue Mar 4 09:29:20 2014 From: neugens at redhat.com (Mario Torre) Date: Tue, 04 Mar 2014 18:29:20 +0100 Subject: Bring Zero to llvm 3.4 In-Reply-To: References: <1393239625.3873.13.camel@galactica.localdomain> <1393256374.11941.3.camel@galactica.localdomain> <1393853313.3834.6.camel@galactica.localdomain> <1581851.jFBPTQiW9h@mgerdin03> <1393854861.3834.8.camel@galactica.localdomain> <38D22382-B630-4EF3-9B20-02D380484155@oracle.com> <1393920411.3834.12.camel@galactica.localdomain> Message-ID: <1393954160.3834.27.camel@galactica.localdomain> On Tue, 2014-03-04 at 09:25 -0800, Christian Thalinger wrote: > > I guess I can't push it myself, right? :) > > Nope :-) :) One day we'll be able to change that I hope! > > > > Can you please push the patch on 9 on my behalf? > > Yes, I will take care of it. Thanks! > > > > I also would like to push it on 8u, but I think I need to ask a backport > > on the 8u mailing list fist, right? > > Correct. Great. Please, let me know if you need further assistance, I'll ping the 8u mailing list once the fix lands into the repository. Cheers, Mario From christian.thalinger at oracle.com Tue Mar 4 09:33:00 2014 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Tue, 4 Mar 2014 09:33:00 -0800 Subject: Bring Zero to llvm 3.4 In-Reply-To: <1393954160.3834.27.camel@galactica.localdomain> References: <1393239625.3873.13.camel@galactica.localdomain> <1393256374.11941.3.camel@galactica.localdomain> <1393853313.3834.6.camel@galactica.localdomain> <1581851.jFBPTQiW9h@mgerdin03> <1393854861.3834.8.camel@galactica.localdomain> <38D22382-B630-4EF3-9B20-02D380484155@oracle.com> <1393920411.3834.12.camel@galactica.localdomain> <1393954160.3834.27.camel@galactica.localdomain> Message-ID: Filed: [#JDK-8036619] Shark: add LLVM 3.4 support - Java Bug System I?m going to add these changes I have as well: diff -r 194e8b7fe9ca src/share/vm/shark/sharkCompiler.cpp --- a/src/share/vm/shark/sharkCompiler.cpp Thu Jan 30 14:30:01 2014 +0100 +++ b/src/share/vm/shark/sharkCompiler.cpp Tue Mar 04 09:30:55 2014 -0800 @@ -364,3 +364,7 @@ const char* SharkCompiler::methodname(co *(dst++) = '\0'; return buf; } + +void SharkCompiler::print_timers() { + // do nothing +} diff -r 194e8b7fe9ca src/share/vm/shark/sharkCompiler.hpp --- a/src/share/vm/shark/sharkCompiler.hpp Thu Jan 30 14:30:01 2014 +0100 +++ b/src/share/vm/shark/sharkCompiler.hpp Tue Mar 04 09:30:55 2014 -0800 @@ -56,6 +56,9 @@ class SharkCompiler : public AbstractCom // Compile a normal (bytecode) method and install it in the VM void compile_method(ciEnv* env, ciMethod* target, int entry_bci); + // Print compilation timers and statistics + void print_timers(); + // Generate a wrapper for a native (JNI) method nmethod* generate_native_wrapper(MacroAssembler* masm, methodHandle target, diff -r 194e8b7fe9ca src/share/vm/shark/sharkInliner.cpp --- a/src/share/vm/shark/sharkInliner.cpp Thu Jan 30 14:30:01 2014 +0100 +++ b/src/share/vm/shark/sharkInliner.cpp Tue Mar 04 09:30:55 2014 -0800 @@ -744,6 +744,10 @@ bool SharkInlinerHelper::do_field_access } bool SharkInliner::attempt_inline(ciMethod *target, SharkState *state) { + if (!Inline) { + return false; + } + if (SharkIntrinsics::is_intrinsic(target)) { SharkIntrinsics::inline_intrinsic(target, state); return true; On Mar 4, 2014, at 9:29 AM, Mario Torre wrote: > On Tue, 2014-03-04 at 09:25 -0800, Christian Thalinger wrote: > >>> I guess I can't push it myself, right? :) >> >> Nope :-) > > :) One day we'll be able to change that I hope! > >>> >>> Can you please push the patch on 9 on my behalf? >> >> Yes, I will take care of it. > > Thanks! > >>> >>> I also would like to push it on 8u, but I think I need to ask a backport >>> on the 8u mailing list fist, right? >> >> Correct. > > Great. Please, let me know if you need further assistance, I'll ping the > 8u mailing list once the fix lands into the repository. > > Cheers, > Mario > > From vladimir.kozlov at oracle.com Tue Mar 4 09:32:34 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 04 Mar 2014 09:32:34 -0800 Subject: RFR[9](XS): 8036614: AIX: fix adjust-mflags.sh to build with GNU Make 4.0 (adapt 8028407 for AIX) In-Reply-To: References: Message-ID: <53160E32.8030406@oracle.com> Looks good. I will push it into hs-comp repo, the job is in JPRT. Thanks, Vlaidmir On 3/4/14 8:27 AM, Volker Simonis wrote: > Hi, > > can somebody please review and sponsor the following little change > which is actually just a "copy and paste" of "8028407 adjust-mflags.sh > failed build with GNU Make 4.0 with -I" for the AIX > version of adjust-mflags.sh: > > http://cr.openjdk.java.net/~simonis/webrevs/8036614 > https://bugs.openjdk.java.net/browse/JDK-8036614 > > Thank you and best regards, > Volker > From neugens at redhat.com Tue Mar 4 09:42:16 2014 From: neugens at redhat.com (Mario Torre) Date: Tue, 04 Mar 2014 18:42:16 +0100 Subject: Bring Zero to llvm 3.4 In-Reply-To: References: <1393239625.3873.13.camel@galactica.localdomain> <1393256374.11941.3.camel@galactica.localdomain> <1393853313.3834.6.camel@galactica.localdomain> <1581851.jFBPTQiW9h@mgerdin03> <1393854861.3834.8.camel@galactica.localdomain> <38D22382-B630-4EF3-9B20-02D380484155@oracle.com> <1393920411.3834.12.camel@galactica.localdomain> <1393954160.3834.27.camel@galactica.localdomain> Message-ID: <1393954936.3834.30.camel@galactica.localdomain> On Tue, 2014-03-04 at 09:33 -0800, Christian Thalinger wrote: > Filed: > > [#JDK-8036619] Shark: add LLVM 3.4 support - Java Bug System > > I?m going to add these changes I have as well: Ok, thanks a lot again. Cheers, Mario From christian.thalinger at oracle.com Tue Mar 4 14:00:00 2014 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Tue, 4 Mar 2014 14:00:00 -0800 Subject: Bring Zero to llvm 3.4 In-Reply-To: <1393954936.3834.30.camel@galactica.localdomain> References: <1393239625.3873.13.camel@galactica.localdomain> <1393256374.11941.3.camel@galactica.localdomain> <1393853313.3834.6.camel@galactica.localdomain> <1581851.jFBPTQiW9h@mgerdin03> <1393854861.3834.8.camel@galactica.localdomain> <38D22382-B630-4EF3-9B20-02D380484155@oracle.com> <1393920411.3834.12.camel@galactica.localdomain> <1393954160.3834.27.camel@galactica.localdomain> <1393954936.3834.30.camel@galactica.localdomain> Message-ID: Hmm. This is what I get with LLVM 3.4: $ java -version Error: dl failure on line 864 Error: failed /export/twisti/build/hotspot/jdk-linux/jre/lib/amd64/server/libjvm.so, because /export/twisti/build/hotspot/jdk-linux/jre/lib/amd64/server/libjvm.so: undefined symbol: setupterm Any ideas? On Mar 4, 2014, at 9:42 AM, Mario Torre wrote: > On Tue, 2014-03-04 at 09:33 -0800, Christian Thalinger wrote: >> Filed: >> >> [#JDK-8036619] Shark: add LLVM 3.4 support - Java Bug System >> >> I?m going to add these changes I have as well: > > Ok, thanks a lot again. > > Cheers, > Mario > > From per.liden at oracle.com Wed Mar 5 01:56:34 2014 From: per.liden at oracle.com (Per Liden) Date: Wed, 05 Mar 2014 10:56:34 +0100 Subject: RFR(L): 8029075 - String deduplication in G1 In-Reply-To: <5A84BB3B-9763-47C3-9EEF-0D07EB763576@oracle.com> References: <53148494.60806@oracle.com> <3A26A868-51A8-4169-B147-9AEB8B35EC5E@oracle.com> <53159E0A.60403@oracle.com> <5A84BB3B-9763-47C3-9EEF-0D07EB763576@oracle.com> Message-ID: <5316F4D2.7010600@oracle.com> Yep, a more efficient mechanism for reference processing without the current overhead per reference would open up for some interesting possibilities. I have some loose/untested ideas on how we could address some of the efficiency problems, but the memory overhead is trickier to solve. /Per On 03/04/2014 05:55 PM, Christian Thalinger wrote: > That?s unfortunate. It clearly would have been easier to do all that work in Java. > > On Mar 4, 2014, at 1:34 AM, Per Liden wrote: > >> Hi Christian, >> >> That was something Mikael Vidstedt and I briefly touched on when we first started to think about deduplication. However, we didn't go down that path, partly because of the problem you mention, but also because of the general overhead of WeakReferences (in terms of memory and work added to the reference processor and Reference Handler). >> >> /Per >> >> On 03/04/2014 02:48 AM, Christian Thalinger wrote: >>> Just a general question: Have you thought about writing the deduplication thread in Java? Maybe also the deduplication table? >>> >>> There is clearly the problem of enqueuing. One easy solution would be to have the deduplication thread poll into the VM to get the next entries but there might be a better away to do it. >>> >>> On Mar 3, 2014, at 5:33 AM, Per Liden wrote: >>> >>>> Hi, >>>> >>>> Could I please have this patch reviewed. >>>> >>>> >>>> Summary >>>> ------- >>>> This patch implements JEP 192 - String deduplication in G1. The goal of string deduplication is to reduce the Java heap live-data set by enhancing the G1 garbage collector so that duplicate instances of String are automatically and continuously deduplicated. >>>> >>>> I'd like to refer to the JEP for a more detailed description of this feature, the motivation for it, the expected benefit, how it's implemented, etc. >>>> >>>> This patch is mainly G1-related, but also touches a few runtime files. >>>> >>>> Webrev: http://cr.openjdk.java.net/~pliden/8029075/webrev.0/ >>>> >>>> JEP: http://openjdk.java.net/jeps/192 >>>> >>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8029075 >>>> >>>> >>>> Testing >>>> ------- >>>> * JTreg - The patch includes the following 8 new tests: >>>> - TestStringDeduplicationYoungGC.java: Tests deduplication during Young GC. >>>> - TestStringDeduplicationFullGC.java: Tests deduplication during Full GC. >>>> - TestStringDeduplicationAgeThreshold.java: Tests both valid and invalid age threshold settings. >>>> - TestStringDeduplicationInterned.java: Tests that interned strings are deduplication before being interned. >>>> - TestStringDeduplicationTableRehash.java: Stresses the hashtables ability to rehash all entries. >>>> - TestStringDeduplicationTableResize.java: Stresses the hashtables ability to resize itself. >>>> - TestStringDeduplicationMemoryUsage.java: Tests heap reduction when string deduplication is enabled. >>>> - TestStringDeduplicationPrintOptions.java: Tests command line options. >>>> >>>> * Stress testing: >>>> - Kitchensink >>>> - GCBasher >>>> >>>> * Regression testing: >>>> - JCK >>>> - vmTestbase >>>> - Bigapps >>>> >>>> * Large scale benchmarks to test heap reduction and performance impact: >>>> - FA CRM Sales Op. Flow >>>> - FA DSS >>>> >>>> * The following benchmarks have been executed to verify that this feature doesn't impact performance when disabled (even when disabled there are still some "if (UseStringDeduplication)" executed in some hot paths). >>>> - SPECjbb2005 >>>> - SPECjbb2013 >>>> - SPECjvm2008-XML >>>> >>>> * Various ad-hoc tests and microbenchmarks were also written and executed during the course of the development. >>>> >>>> >>>> Thanks! >>>> /Per >>> >> > From bengt.rutisson at oracle.com Wed Mar 5 02:00:17 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 05 Mar 2014 11:00:17 +0100 Subject: RFR(L): 8029075 - String deduplication in G1 In-Reply-To: <53148494.60806@oracle.com> References: <53148494.60806@oracle.com> Message-ID: <5316F5B1.60006@oracle.com> Hi Per and everyone else, Overall this looks good to me. Me and Per went through this patch together and here are the notes from this code walk through. There are two small bug fixes included in the change. It would be good to separate these out to separate changes. We can review them together with the main change, but having them as separate changesets will make it easier to backport them if necessary. stringDedup.hpp The comments are not that helpful to me. I would suggest to either remove them or make more descriptive. UseG1GC && UseStringDeduplication Only allow UseStringDeduplication to be true if UseG1GC is true instead of checking both? G1CollectedHeap::tear_down_region_sets() and G1CollectedHeap::rebuild_region_sets() Please add an RFE to clean these up as the naming and implementations don't match anymore. StringDedupTable::trim() rename to trim_cache? stringDedup.cpp Two occurrences of: assert(UseG1GC, "String deduplication not available with G1"); The "not" should be a "only". Dummy statistics from interned stings is ignored. Please add an RFE to try to include this statistics and possibly reduce the "known" count. Add comments explaining why resize and rehash is not allowed for these code paths: void StringDedup::unlink(BoolObjectClosure* is_alive) { unlink_or_oops_do(is_alive, NULL, false /* allow_resize_and_rehash */); } void StringDedup::oops_do(OopClosure* keep_alive) { unlink_or_oops_do(NULL, keep_alive, false /* allow_resize_and_rehash */); } TestStringDeduplicationTools.java verifyStrings put the sleep at the end to possibly speed up the test a little. InternedTest Comments to explain what it does. testYoungGC - check that no full GCs happen testFullGC - check that no young GCs happen testMemoryUsage - add "memoryUsageWithDedup > memoryUsageWithDedupExpected" to the exception message Thanks, Bengt On 3/3/14 2:33 PM, Per Liden wrote: > Hi, > > Could I please have this patch reviewed. > > > Summary > ------- > This patch implements JEP 192 - String deduplication in G1. The goal > of string deduplication is to reduce the Java heap live-data set by > enhancing the G1 garbage collector so that duplicate instances of > String are automatically and continuously deduplicated. > > I'd like to refer to the JEP for a more detailed description of this > feature, the motivation for it, the expected benefit, how it's > implemented, etc. > > This patch is mainly G1-related, but also touches a few runtime files. > > Webrev: http://cr.openjdk.java.net/~pliden/8029075/webrev.0/ > > JEP: http://openjdk.java.net/jeps/192 > > RFE: https://bugs.openjdk.java.net/browse/JDK-8029075 > > > Testing > ------- > * JTreg - The patch includes the following 8 new tests: > - TestStringDeduplicationYoungGC.java: Tests deduplication during > Young GC. > - TestStringDeduplicationFullGC.java: Tests deduplication during > Full GC. > - TestStringDeduplicationAgeThreshold.java: Tests both valid and > invalid age threshold settings. > - TestStringDeduplicationInterned.java: Tests that interned strings > are deduplication before being interned. > - TestStringDeduplicationTableRehash.java: Stresses the hashtables > ability to rehash all entries. > - TestStringDeduplicationTableResize.java: Stresses the hashtables > ability to resize itself. > - TestStringDeduplicationMemoryUsage.java: Tests heap reduction when > string deduplication is enabled. > - TestStringDeduplicationPrintOptions.java: Tests command line options. > > * Stress testing: > - Kitchensink > - GCBasher > > * Regression testing: > - JCK > - vmTestbase > - Bigapps > > * Large scale benchmarks to test heap reduction and performance impact: > - FA CRM Sales Op. Flow > - FA DSS > > * The following benchmarks have been executed to verify that this > feature doesn't impact performance when disabled (even when disabled > there are still some "if (UseStringDeduplication)" executed in some > hot paths). > - SPECjbb2005 > - SPECjbb2013 > - SPECjvm2008-XML > > * Various ad-hoc tests and microbenchmarks were also written and > executed during the course of the development. > > > Thanks! > /Per From neugens at redhat.com Wed Mar 5 02:22:35 2014 From: neugens at redhat.com (Mario Torre) Date: Wed, 05 Mar 2014 11:22:35 +0100 Subject: Bring Zero to llvm 3.4 In-Reply-To: References: <1393239625.3873.13.camel@galactica.localdomain> <1393256374.11941.3.camel@galactica.localdomain> <1393853313.3834.6.camel@galactica.localdomain> <1581851.jFBPTQiW9h@mgerdin03> <1393854861.3834.8.camel@galactica.localdomain> <38D22382-B630-4EF3-9B20-02D380484155@oracle.com> <1393920411.3834.12.camel@galactica.localdomain> <1393954160.3834.27.camel@galactica.localdomain> <1393954936.3834.30.camel@galactica.localdomain> Message-ID: <1394014955.3834.33.camel@galactica.localdomain> On Tue, 2014-03-04 at 14:00 -0800, Christian Thalinger wrote: > Hmm. This is what I get with LLVM 3.4: > > $ java -version > Error: dl failure on line 864 > Error: failed /export/twisti/build/hotspot/jdk-linux/jre/lib/amd64/server/libjvm.so, because /export/twisti/build/hotspot/jdk-linux/jre/lib/amd64/server/libjvm.so: undefined symbol: setupterm > > Any ideas? Hi Christian, Yes, when building llvm, you need to configure it without libterm support: --enable-terminfo=no --enable-curses=no I think something changed recently in llvm, because I didn't see this error before either, maybe now those are defaults and before not, not sure. Maybe I should add this information to some of our docs? Cheers, Mario From thomas.schatzl at oracle.com Wed Mar 5 02:56:45 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 05 Mar 2014 11:56:45 +0100 Subject: RFR(L): 8029075 - String deduplication in G1 In-Reply-To: <53148494.60806@oracle.com> References: <53148494.60806@oracle.com> Message-ID: <1394017005.2674.11.camel@cirrus> Hi Per, On Mon, 2014-03-03 at 14:33 +0100, Per Liden wrote: > Hi, > > Could I please have this patch reviewed. this is not a complete review (still working on it), but some comments about measuring the time taken. When printing thread processing times, is it possible to provide per-thread times for the specific parts (queue processing, table processing) and the total? E.g. instead of only the total time for weak root fixup, provide also string table processing time. This would help finding issues with particular phases of the weak root fixup, and in particular string dedup. Per thread times (at maximum logging level) will help finding issues. Potentially this code is trying to distribute little work on hundreds of threads after all and this would help identifying the problem quickly. Something like: [Weak Root Fixup ....] [String Dedup ...] (Additionally, probably better in a separate CR, add timing for processing the JNI weak root handles). Thanks, Thomas From thomas.schatzl at oracle.com Wed Mar 5 03:27:09 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 05 Mar 2014 12:27:09 +0100 Subject: RFR(L): 8029075 - String deduplication in G1 In-Reply-To: <1394017005.2674.11.camel@cirrus> References: <53148494.60806@oracle.com> <1394017005.2674.11.camel@cirrus> Message-ID: <1394018829.2674.13.camel@cirrus> Hi again, some comment below... On Wed, 2014-03-05 at 11:56 +0100, Thomas Schatzl wrote: > Hi Per, > > On Mon, 2014-03-03 at 14:33 +0100, Per Liden wrote: > > Hi, > > > > Could I please have this patch reviewed. > > this is not a complete review (still working on it), but some comments > about measuring the time taken. > > When printing thread processing times, is it possible to provide > per-thread times for the specific parts (queue processing, table > processing) and the total? > > E.g. instead of only the total time for weak root fixup, provide also > string table processing time. > > This would help finding issues with particular phases of the weak root > fixup, and in particular string dedup. Per thread times (at maximum > logging level) will help finding issues. > > Potentially this code is trying to distribute little work on hundreds of > threads after all and this would help identifying the problem quickly. > > Something like: > > [Weak Root Fixup ....] > [String Dedup ...] Probably it's better that this change only adds the String Dedup entry. > > (Additionally, probably better in a separate CR, add timing for > processing the JNI weak root handles). That separate CR should probably add both "Weak Root Fixup" and "JNI Handle Fixup" entries. Thanks, Thomas From per.liden at oracle.com Wed Mar 5 04:02:25 2014 From: per.liden at oracle.com (Per Liden) Date: Wed, 05 Mar 2014 13:02:25 +0100 Subject: RFR(L): 8029075 - String deduplication in G1 In-Reply-To: <1394018829.2674.13.camel@cirrus> References: <53148494.60806@oracle.com> <1394017005.2674.11.camel@cirrus> <1394018829.2674.13.camel@cirrus> Message-ID: <53171251.1050606@oracle.com> Hi Thomas, On 03/05/2014 12:27 PM, Thomas Schatzl wrote: > Hi again, > > some comment below... > > On Wed, 2014-03-05 at 11:56 +0100, Thomas Schatzl wrote: >> Hi Per, >> >> On Mon, 2014-03-03 at 14:33 +0100, Per Liden wrote: >>> Hi, >>> >>> Could I please have this patch reviewed. >> >> this is not a complete review (still working on it), but some comments >> about measuring the time taken. >> >> When printing thread processing times, is it possible to provide >> per-thread times for the specific parts (queue processing, table >> processing) and the total? >> >> E.g. instead of only the total time for weak root fixup, provide also >> string table processing time. >> >> This would help finding issues with particular phases of the weak root >> fixup, and in particular string dedup. Per thread times (at maximum >> logging level) will help finding issues. >> >> Potentially this code is trying to distribute little work on hundreds of >> threads after all and this would help identifying the problem quickly. >> >> Something like: >> >> [Weak Root Fixup ....] >> [String Dedup ...] > > Probably it's better that this change only adds the String Dedup entry. Sure, I can exclude the JNI time for now. > >> >> (Additionally, probably better in a separate CR, add timing for >> processing the JNI weak root handles). > > That separate CR should probably add both "Weak Root Fixup" and "JNI > Handle Fixup" entries. Yep, I'll create a separate CR for that. /Per > > Thanks, > Thomas > > From asmundak at google.com Wed Mar 5 13:42:44 2014 From: asmundak at google.com (Alexander Smundak) Date: Wed, 5 Mar 2014 13:42:44 -0800 Subject: RFR (M): 8035647 : PPC64: Support for elf v2 abi. Message-ID: This patch adds support for the recently introduced ELFv2 ABI for the PowerPC64. The patch was generated for the jdk9/hs-rt/hotspot repository (the same patch has been already committed to the ppc-aix-port/jdk7u). It should for to jdk8u via ppc-aix-port/stage. This patch is ppc-only. ELFv2 ABI has been introduced recently and is used on by Linux running on the little endian PowerPC64. There is no official ELF ABI supplement for it, but the differences with "old" ABI for PowerPC64 are explained in the comments for the patches to the GCC compiler: http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01144.html http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01141.html http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01140.html http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01145.html http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01147.html http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01149.html http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01146.html http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01150.html The change that affects JVM most is the elimination of the function descriptors (which unfortunately requires introducing a number of #ifdef's into the code), and dropping two words from the base frame (which required mostly renaming of the structs from `abi_48' to a more neutral `abi_minframe', etc.) There are additional patches needed to fully support building JDK on the little endian PowerPC64; RFRs for them will follow. The development was done on Ubuntu 14.04 running on QEMU. With all the patches applied, I can build working JDK7 and JDK9 there. Please review and test this change. I need a sponsor. The patch is at: http://cr.openjdk.java.net/~rasbold/8035647/webrev.00/ Sasha From vladimir.kozlov at oracle.com Wed Mar 5 18:24:43 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 05 Mar 2014 18:24:43 -0800 Subject: RFR (M): 8035647 : PPC64: Support for elf v2 abi. In-Reply-To: References: Message-ID: <5317DC6B.7050801@oracle.com> Looks fine to me. If ppc64 community is okay with this I can push it. I saw discussion on ppc-aix alias but I need their comments on hotspot-dev. For Hotspot-dev, Alexander can contribute a code into open jdk because he is under Google's umbrella 'Oracle Contribution Agreement'. Thanks, Vladimir On 3/5/14 1:42 PM, Alexander Smundak wrote: > This patch adds support for the recently introduced ELFv2 ABI > for the PowerPC64. > The patch was generated for the jdk9/hs-rt/hotspot repository > (the same patch has been already committed to the ppc-aix-port/jdk7u). > It should for to jdk8u via ppc-aix-port/stage. > > This patch is ppc-only. > > ELFv2 ABI has been introduced recently and is used on by Linux running on > the little endian PowerPC64. There is no official ELF ABI supplement for it, > but the differences with "old" ABI for PowerPC64 are explained in the > comments for the patches to the GCC compiler: > http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01144.html > http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01141.html > http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01140.html > http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01145.html > http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01147.html > http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01149.html > http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01146.html > http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01150.html > > The change that affects JVM most is the elimination of the function > descriptors (which unfortunately requires introducing a number of #ifdef's > into the code), and dropping two words from the base frame (which > required mostly renaming of the structs from `abi_48' to a more neutral > `abi_minframe', etc.) > > There are additional patches needed to fully support building > JDK on the little endian PowerPC64; RFRs for them will follow. > > The development was done on Ubuntu 14.04 running on QEMU. > With all the patches applied, I can build working JDK7 and JDK9 there. > > Please review and test this change. I need a sponsor. > > The patch is at: > http://cr.openjdk.java.net/~rasbold/8035647/webrev.00/ > > Sasha > From erik.helin at oracle.com Thu Mar 6 00:34:46 2014 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 06 Mar 2014 09:34:46 +0100 Subject: RFR: 8036702: Refactor os::safe_for_sender to work with non-Java threads Message-ID: <53183326.10902@oracle.com> Hi all, this patch changes the type of the parameter to the function os::safe_for_sender from JavaThread* to Thread*. There is no code in os::safe_for_sender that uses members that are specific to JavaThread. This change is in preparation for JDK-8036699, which will do some stack walking on non-Java thread and makes use of os::safe_for_sender for non-Java threads. Issue: https://bugs.openjdk.java.net/browse/JDK-8036702 Webrev: http://cr.openjdk.java.net/~ehelin/8036702/webrev.00/ Testing: - JPRT Thanks, Erik From erik.helin at oracle.com Thu Mar 6 01:00:21 2014 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 06 Mar 2014 10:00:21 +0100 Subject: RFR: 8036699: Add trace event when a metaspace allocation fails Message-ID: <53183925.4020001@oracle.com> Hi all, this patch adds the new trace event vm/gc/metaspace/allocation_failure. The event will be sent when we get an allocation failure in metaspace. The event will contain the following information: - classLoader - the class loader doing the allocation - anonymousClassLoader - if the classLoader is anonymous - nativeStackTrace - a VM stack trace (see more below) - metadataType - the kind of metadata (class or non-class) - metaspaceObjectType - the kind of metaspace object The field nativeStackTrace will contain a string representation of the Threads C frames (up until the first Java frame). This is very useful for debugging metaspace allocation failures coming from VM internals such as the interpreter or GC code. I would especially appreciate if someone could look over the code in metaspaceTracer.cpp performing the stack walking. Webrev: http://cr.openjdk.java.net/~ehelin/8036699/webrev.00/ Enhancement: https://bugs.openjdk.java.net/browse/JDK-8036699 Testing: - JFR JTREG tests - JPRT - Manual stress testing of the stack walking code by writing small test programs that causes metaspace allocation failures from both Java threads and non-Java threads. Thanks, Erik From erik.helin at oracle.com Thu Mar 6 01:19:56 2014 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 06 Mar 2014 10:19:56 +0100 Subject: RFR: 8036701: Add trace event when a metaspace throws out of memory error Message-ID: <53183DBC.1050707@oracle.com> Hi all, this patch adds a new trace event for tracking metaspace out of memory exceptions - vm/gc/metaspace/out_of_memory. This patch builds upon the patch for "JDK-8036699: Add trace event when a metaspace allocation fails". The information in the allocation failure event and the OOM event are the same, see the review request for the allocation failure event for more information. Webrev: http://cr.openjdk.java.net/~ehelin/8036701/webrev.00/ Enhancement: https://bugs.openjdk.java.net/browse/JDK-8036701 Testing: - JFR JTREG tests - JPRT Thanks, Erik From stefan.karlsson at oracle.com Thu Mar 6 01:26:26 2014 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 06 Mar 2014 10:26:26 +0100 Subject: RFR: 8036699: Add trace event when a metaspace allocation fails In-Reply-To: <53183925.4020001@oracle.com> References: <53183925.4020001@oracle.com> Message-ID: <53183F42.3010908@oracle.com> On 2014-03-06 10:00, Erik Helin wrote: > Hi all, > > this patch adds the new trace event > vm/gc/metaspace/allocation_failure. The event will be sent when we get > an allocation failure in metaspace. The event will contain the > following information: > - classLoader - the class loader doing the allocation > - anonymousClassLoader - if the classLoader is anonymous > - nativeStackTrace - a VM stack trace (see more below) > - metadataType - the kind of metadata (class or non-class) > - metaspaceObjectType - the kind of metaspace object > > The field nativeStackTrace will contain a string representation of the > Threads C frames (up until the first Java frame). This is very useful > for debugging metaspace allocation failures coming from VM internals > such as the interpreter or GC code. > > I would especially appreciate if someone could look over the code in > metaspaceTracer.cpp performing the stack walking. > > Webrev: > http://cr.openjdk.java.net/~ehelin/8036699/webrev.00/ I'm not going to review the stack walking code, but the rest looks good. Some thoughts: 1) I wonder if the reported field "classLoader" should be reported as classLoaderClass instead? http://cr.openjdk.java.net/~ehelin/8036699/webrev.00/src/share/vm/trace/trace.xml.udiff.html + 2) Should we report the Host Class Loader Klass for anonymous classes, instead of reporting NULL? thanks, StefanK > > Enhancement: > https://bugs.openjdk.java.net/browse/JDK-8036699 > > Testing: > - JFR JTREG tests > - JPRT > - Manual stress testing of the stack walking code by writing small test > programs that causes metaspace allocation failures from both Java > threads and non-Java threads. > > Thanks, > Erik From erik.helin at oracle.com Thu Mar 6 01:35:00 2014 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 06 Mar 2014 10:35:00 +0100 Subject: RFR: 8036703: Add trace event with statistics for the metaspace chunk free lists Message-ID: <53184144.3060905@oracle.com> Hi all, this patch adds a new trace event - vm/gc/metaspace/chunk_free_list_summary for getting statistics for metaspace's chunk free lists. The event contains the numbers of chunks in the free list and the size of the chunks in the free lists. This information is provided for all different kinds of chunks - special, small, medium and humongous. This information will be useful when diagnosing fragmentation issues in metaspace. Webrev: http://cr.openjdk.java.net/~ehelin/8036703/webrev.00/ Enhancement: https://bugs.openjdk.java.net/browse/JDK-8036703 Testing: - JFR JTREG tests - JPRT - Manually verifying that the numbers look correct by looking at recordings from small test cases. Thanks, Erik From stefan.karlsson at oracle.com Thu Mar 6 02:02:25 2014 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 06 Mar 2014 11:02:25 +0100 Subject: RFR: 8036703: Add trace event with statistics for the metaspace chunk free lists In-Reply-To: <53184144.3060905@oracle.com> References: <53184144.3060905@oracle.com> Message-ID: <531847B1.6080709@oracle.com> On 2014-03-06 10:35, Erik Helin wrote: > Hi all, > > this patch adds a new trace event - > vm/gc/metaspace/chunk_free_list_summary for getting statistics for > metaspace's chunk free lists. The event contains the numbers of chunks > in the free list and the size of the chunks in the free lists. This > information is provided for all different kinds of chunks - special, > small, medium and humongous. > > This information will be useful when diagnosing fragmentation issues > in metaspace. > > Webrev: > http://cr.openjdk.java.net/~ehelin/8036703/webrev.00/ http://cr.openjdk.java.net/~ehelin/8036703/webrev.00/src/share/vm/gc_interface/collectedHeap.cpp.udiff.html This isn't used: + MetaspaceChunkFreeListSummary chunk_free_list_summary; http://cr.openjdk.java.net/~ehelin/8036703/webrev.00/src/share/vm/memory/metaspace.cpp.udiff.html + void index_bounds_check(ChunkIndex index) const { Should be static. Or maybe a #define if you want to get the line where the assert happens. err_msg("Bad index: " INT32_FORMAT, (int) index)); Should be %d instead of INT32_FORMAT. 2682 return MetaspaceChunkFreeListSummary(cm->num_free_chunks(SpecializedIndex), 2683 cm->num_free_chunks(SmallIndex), 2684 cm->num_free_chunks(MediumIndex), 2685 cm->num_free_chunks(HumongousIndex), 2686 cm->size_free_chunks_in_bytes(SpecializedIndex), 2687 cm->size_free_chunks_in_bytes(SmallIndex), 2688 cm->size_free_chunks_in_bytes(MediumIndex), 2689 cm->size_free_chunks_in_bytes(HumongousIndex)); I think it's better to keep the setup of MetaspaceChunkFreeListSummary in ChunkManager. Otherwise, this looks good. thanks, StefanK > > Enhancement: > https://bugs.openjdk.java.net/browse/JDK-8036703 > > Testing: > - JFR JTREG tests > - JPRT > - Manually verifying that the numbers look correct by looking at > recordings from small test cases. > > Thanks, > Erik From goetz.lindenmaier at sap.com Thu Mar 6 02:27:13 2014 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 6 Mar 2014 10:27:13 +0000 Subject: RFR (M): 8035647 : PPC64: Support for elf v2 abi. In-Reply-To: <5317DC6B.7050801@oracle.com> References: <5317DC6B.7050801@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CEA6040@DEWDFEMB12A.global.corp.sap> Hi, I looked at the change and tested it on our linux ppc64 and on aix. It's good. I also tested it with the stage repo, that works fine, too. Vladimir, can you push it to both, jdk9 and stage? Sasha, I added parts of your Mail to the bug description: https://bugs.openjdk.java.net/browse/JDK-8035647 Best regards, Goetz. -----Original Message----- From: ppc-aix-port-dev-bounces at openjdk.java.net [mailto:ppc-aix-port-dev-bounces at openjdk.java.net] On Behalf Of Vladimir Kozlov Sent: Donnerstag, 6. M?rz 2014 03:25 To: Alexander Smundak; hotspot-dev at openjdk.java.net Cc: ppc-aix-port-dev at openjdk.java.net Subject: Re: RFR (M): 8035647 : PPC64: Support for elf v2 abi. Looks fine to me. If ppc64 community is okay with this I can push it. I saw discussion on ppc-aix alias but I need their comments on hotspot-dev. For Hotspot-dev, Alexander can contribute a code into open jdk because he is under Google's umbrella 'Oracle Contribution Agreement'. Thanks, Vladimir On 3/5/14 1:42 PM, Alexander Smundak wrote: > This patch adds support for the recently introduced ELFv2 ABI > for the PowerPC64. > The patch was generated for the jdk9/hs-rt/hotspot repository > (the same patch has been already committed to the ppc-aix-port/jdk7u). > It should for to jdk8u via ppc-aix-port/stage. > > This patch is ppc-only. > > ELFv2 ABI has been introduced recently and is used on by Linux running on > the little endian PowerPC64. There is no official ELF ABI supplement for it, > but the differences with "old" ABI for PowerPC64 are explained in the > comments for the patches to the GCC compiler: > http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01144.html > http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01141.html > http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01140.html > http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01145.html > http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01147.html > http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01149.html > http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01146.html > http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01150.html > > The change that affects JVM most is the elimination of the function > descriptors (which unfortunately requires introducing a number of #ifdef's > into the code), and dropping two words from the base frame (which > required mostly renaming of the structs from `abi_48' to a more neutral > `abi_minframe', etc.) > > There are additional patches needed to fully support building > JDK on the little endian PowerPC64; RFRs for them will follow. > > The development was done on Ubuntu 14.04 running on QEMU. > With all the patches applied, I can build working JDK7 and JDK9 there. > > Please review and test this change. I need a sponsor. > > The patch is at: > http://cr.openjdk.java.net/~rasbold/8035647/webrev.00/ > > Sasha > From goetz.lindenmaier at sap.com Thu Mar 6 02:40:48 2014 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 6 Mar 2014 10:40:48 +0000 Subject: 8036702: Refactor os::safe_for_sender to work with non-Java threads In-Reply-To: <53183326.10902@oracle.com> References: <53183326.10902@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CEA605C@DEWDFEMB12A.global.corp.sap> Hi Erik, I tested your change with the ppc64 port. Could you please add the patch below to it, so that it does not break the ppc64 build? Thanks, Goetz. diff -r 5157d22ca82d src/cpu/ppc/vm/frame_ppc.cpp --- a/src/cpu/ppc/vm/frame_ppc.cpp Thu Mar 06 11:32:05 2014 +0100 +++ b/src/cpu/ppc/vm/frame_ppc.cpp Thu Mar 06 11:39:43 2014 +0100 @@ -51,7 +51,7 @@ } #endif // ASSERT -bool frame::safe_for_sender(JavaThread *thread) { +bool frame::safe_for_sender(Thread *thread) { bool safe = false; address cursp = (address)sp(); address curfp = (address)fp(); @@ -165,7 +165,7 @@ } } -bool frame::is_interpreted_frame_valid(JavaThread* thread) const { +bool frame::is_interpreted_frame_valid(Thread* thread) const { // Is there anything to do? assert(is_interpreted_frame(), "Not an interpreted frame"); return true; -----Original Message----- From: hotspot-dev-bounces at openjdk.java.net [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Erik Helin Sent: Donnerstag, 6. M?rz 2014 09:35 To: hotspot-dev at openjdk.java.net Subject: RFR: 8036702: Refactor os::safe_for_sender to work with non-Java threads Hi all, this patch changes the type of the parameter to the function os::safe_for_sender from JavaThread* to Thread*. There is no code in os::safe_for_sender that uses members that are specific to JavaThread. This change is in preparation for JDK-8036699, which will do some stack walking on non-Java thread and makes use of os::safe_for_sender for non-Java threads. Issue: https://bugs.openjdk.java.net/browse/JDK-8036702 Webrev: http://cr.openjdk.java.net/~ehelin/8036702/webrev.00/ Testing: - JPRT Thanks, Erik From erik.helin at oracle.com Thu Mar 6 02:56:14 2014 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 06 Mar 2014 11:56:14 +0100 Subject: RFR: 8036703: Add trace event with statistics for the metaspace chunk free lists In-Reply-To: <53184144.3060905@oracle.com> References: <53184144.3060905@oracle.com> Message-ID: <5318544E.7050006@oracle.com> A small bug had found its way into this patch: metaspace.cpp: +bool MetaspaceAux::has_chunk_free_list(Metaspace::MetadataType mdtype) { + return Metaspace::get_chunk_manager(mdtype) == NULL; +} should of course be: +bool MetaspaceAux::has_chunk_free_list(Metaspace::MetadataType mdtype) { + return Metaspace::get_chunk_manager(mdtype) != NULL; +} Please see new webrev at: http://cr.openjdk.java.net/~ehelin/8036703/webrev.01/ Thanks, Erik On 2014-03-06 10:35, Erik Helin wrote: > Hi all, > > this patch adds a new trace event - > vm/gc/metaspace/chunk_free_list_summary for getting statistics for > metaspace's chunk free lists. The event contains the numbers of chunks > in the free list and the size of the chunks in the free lists. This > information is provided for all different kinds of chunks - special, > small, medium and humongous. > > This information will be useful when diagnosing fragmentation issues in > metaspace. > > Webrev: > http://cr.openjdk.java.net/~ehelin/8036703/webrev.00/ > > Enhancement: > https://bugs.openjdk.java.net/browse/JDK-8036703 > > Testing: > - JFR JTREG tests > - JPRT > - Manually verifying that the numbers look correct by looking at > recordings from small test cases. > > Thanks, > Erik From erik.helin at oracle.com Thu Mar 6 05:31:35 2014 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 06 Mar 2014 14:31:35 +0100 Subject: 8036702: Refactor os::safe_for_sender to work with non-Java threads In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CEA605C@DEWDFEMB12A.global.corp.sap> References: <53183326.10902@oracle.com> <4295855A5C1DE049A61835A1887419CC2CEA605C@DEWDFEMB12A.global.corp.sap> Message-ID: <531878B7.3010704@oracle.com> Goetz, On 2014-03-06 11:40, Lindenmaier, Goetz wrote: > Hi Erik, > > I tested your change with the ppc64 port. > > Could you please add the patch below to it, so that it does not break > the ppc64 build? thanks for catching this! Please see new webrev at: http://cr.openjdk.java.net/~ehelin/8036702/webrev.01/ Thanks, Erik > Thanks, > Goetz. > > diff -r 5157d22ca82d src/cpu/ppc/vm/frame_ppc.cpp > --- a/src/cpu/ppc/vm/frame_ppc.cpp Thu Mar 06 11:32:05 2014 +0100 > +++ b/src/cpu/ppc/vm/frame_ppc.cpp Thu Mar 06 11:39:43 2014 +0100 > @@ -51,7 +51,7 @@ > } > #endif // ASSERT > > -bool frame::safe_for_sender(JavaThread *thread) { > +bool frame::safe_for_sender(Thread *thread) { > bool safe = false; > address cursp = (address)sp(); > address curfp = (address)fp(); > @@ -165,7 +165,7 @@ > } > } > > -bool frame::is_interpreted_frame_valid(JavaThread* thread) const { > +bool frame::is_interpreted_frame_valid(Thread* thread) const { > // Is there anything to do? > assert(is_interpreted_frame(), "Not an interpreted frame"); > return true; > > > > > > > -----Original Message----- > From: hotspot-dev-bounces at openjdk.java.net [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Erik Helin > Sent: Donnerstag, 6. M?rz 2014 09:35 > To: hotspot-dev at openjdk.java.net > Subject: RFR: 8036702: Refactor os::safe_for_sender to work with non-Java threads > > Hi all, > > this patch changes the type of the parameter to the function > os::safe_for_sender from JavaThread* to Thread*. There is no code in > os::safe_for_sender that uses members that are specific to JavaThread. > > This change is in preparation for JDK-8036699, which will do some stack > walking on non-Java thread and makes use of os::safe_for_sender for > non-Java threads. > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8036702 > > Webrev: > http://cr.openjdk.java.net/~ehelin/8036702/webrev.00/ > > Testing: > - JPRT > > Thanks, > Erik > From volker.simonis at gmail.com Thu Mar 6 07:09:51 2014 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 6 Mar 2014 16:09:51 +0100 Subject: RFR[9](S): 8036777: AIX: Adapt implementation after '6546236: Thread interrupt() of Thread.sleep() can be lost on Solaris..' Message-ID: Hi, could you please review and push the following little AIX-only change which adapts the refactorings of '6546236: Thread interrupt() of Thread.sleep() can be lost on Solaris..' to AIX: http://cr.openjdk.java.net/~simonis/webrevs/8036777/ https://bugs.openjdk.java.net/browse/JDK-8036777 The change removes the implementations of os::sleep(), os::interrupt() and os::is_interrupted() from os_aix.cpp such that AIX now uses the implementations from os_posix.cpp like the other Unix platfroms and defines os::supports_monotonic_clock() in os_aix.inline.hpp. Thank you and best regards, Volker From alejandro.murillo at oracle.com Thu Mar 6 07:21:30 2014 From: alejandro.murillo at oracle.com (alejandro.murillo at oracle.com) Date: Thu, 06 Mar 2014 15:21:30 +0000 Subject: hg: hsx/jdk7u/hotspot: Added tag hs24.80-b04 for changeset 1b9722b5134a Message-ID: <20140306152142.8FF0D62579@hg.openjdk.java.net> Changeset: b33ee95e17c6 Author: amurillo Date: 2014-03-06 05:53 -0800 URL: http://hg.openjdk.java.net/hsx/jdk7u/hotspot/rev/b33ee95e17c6 Added tag hs24.80-b04 for changeset 1b9722b5134a ! .hgtags From jan.siersch at uni-ulm.de Thu Mar 6 08:15:32 2014 From: jan.siersch at uni-ulm.de (Jan Siersch) Date: Thu, 06 Mar 2014 17:15:32 +0100 Subject: RFE: 4508774: Implementation for GetOperandStack In-Reply-To: <52FA739F.9070204@oracle.com> References: <52F90A98.4050808@uni-ulm.de> <52F9629A.9080300@oracle.com> <52F9DEFE.7080100@oracle.com> <52FA4AAF.40503@uni-ulm.de> <52FA739F.9070204@oracle.com> Message-ID: <53189F24.3050101@uni-ulm.de> Hi Serguei, I have received confirmation that my OCA signing request has been processed. You should now be able to use the code I provided for RFE: 4508774. Regards, Jan On 02/11/2014 08:01 PM, serguei.spitsyn at oracle.com wrote: > Hi Jan, > > Great. > Please, let me know when it is completed. > > Thanks, > Serguei > > On 2/11/14 8:07 AM, Jan Siersch wrote: >> Hi Serguei, >> >> I have just signed the OCA and mailed it to oracle. Hope you can use >> the code once they have processed my request. >> >> Regards, >> Jan >> >> On 02/11/2014 09:27 AM, serguei.spitsyn at oracle.com wrote: >>> Hi Jan, >>> >>> I was told that you are not allowed to contribute until you sign an >>> OSA. >>> This is a link to follow: >>> http://openjdk.java.net/contribute/ >>> >>> I'm deleting the attachment from the bug report at the moment. >>> Sorry. :( >>> >>> Thanks, >>> Serguei >>> >>> On 2/10/14 3:36 PM, serguei.spitsyn at oracle.com wrote: >>>> Hi Jan, >>>> >>>> Thank you for contributing a suggested fix for this enhancement! >>>> I've attached your diff into the bug report file getOpStack.patch. >>>> Of course, we will take care and make sure the final fix is type safe. >>>> >>>> In case the attachment was removed from the mailing list. >>>> Here is an external link to the diff: >>>> http://pastebin.com/fbDd7mU3 >>>> >>>> The above link is helpful as your original diff was removed from >>>> the email. >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 2/10/14 9:21 AM, Jan Siersch wrote: >>>>> Hi all, >>>>> >>>>> this post is about a feature request for JVMTI to allow access to >>>>> the operand stack: >>>>> https://bugs.openjdk.java.net/browse/JDK-4508774 >>>>> >>>>> As part of a project I'm working on, I have implemented a set of >>>>> functions, namely GetOperand, which provide the requested >>>>> functionality, and I would like to provide this implementation to >>>>> the OpenJDK project. As I am not a regular contributor to the >>>>> OpenJDK project, I do not have an account for the bug tracker and >>>>> cannot post my code there. >>>>> >>>>> I have attached a diff with the modified hotspot source code to >>>>> this e-mail. This implementation adds a new function category >>>>> "Operand Stack" to JVMTI that contains 5 functions >>>>> (GetOperandObject, GetOperandInt, GetOperandLong, GetOperandFloat, >>>>> GetOperandDouble) which can be used the same way as the >>>>> corresponding GetLocal functions. >>>>> >>>>> All changes to the source files are purely additive. No existing >>>>> functions are modified. One new VM Operation "GetOperand" is added >>>>> and implemented. The changes are based on the "JDK 7 Updates >>>>> Master" branch. I have compiled OpenJDK locally with these changes >>>>> and had no problems with them. >>>>> >>>>> However this implementation still has one drawback: the functions >>>>> are not type-safe, as I was unable to find the necessary >>>>> information within hotspot to do proper type-checking of operands. >>>>> For me this is not a problem, because in my project I have a way >>>>> of inferring operand types externally, but for proper production >>>>> code type-checking should be added (there is a FIXME in the added >>>>> source for this purpose). >>>>> >>>>> I hope someone reading this list has the time to look at this code >>>>> and if they find it suitable, to integrate it into the OpenJDK >>>>> project. About licensing: you may use this code in whatever way, >>>>> shape or form you see fit. >>>>> >>>>> Regards, >>>>> Jan >>>> >>> > From erik.helin at oracle.com Thu Mar 6 08:16:54 2014 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 06 Mar 2014 17:16:54 +0100 Subject: RFR: 8036703: Add trace event with statistics for the metaspace chunk free lists In-Reply-To: <531847B1.6080709@oracle.com> References: <53184144.3060905@oracle.com> <531847B1.6080709@oracle.com> Message-ID: <53189F76.80503@oracle.com> Hi Stefan, thanks for reviewing! Please see comments inline and the new webrev at: http://cr.openjdk.java.net/~ehelin/8036703/webrev.02/ On 2014-03-06 11:02, Stefan Karlsson wrote: > > On 2014-03-06 10:35, Erik Helin wrote: >> Hi all, >> >> this patch adds a new trace event - >> vm/gc/metaspace/chunk_free_list_summary for getting statistics for >> metaspace's chunk free lists. The event contains the numbers of chunks >> in the free list and the size of the chunks in the free lists. This >> information is provided for all different kinds of chunks - special, >> small, medium and humongous. >> >> This information will be useful when diagnosing fragmentation issues >> in metaspace. >> >> Webrev: >> http://cr.openjdk.java.net/~ehelin/8036703/webrev.00/ > > http://cr.openjdk.java.net/~ehelin/8036703/webrev.00/src/share/vm/gc_interface/collectedHeap.cpp.udiff.html > > > This isn't used: > > + MetaspaceChunkFreeListSummary chunk_free_list_summary; Fixed. On 2014-03-06 11:02, Stefan Karlsson wrote: > http://cr.openjdk.java.net/~ehelin/8036703/webrev.00/src/share/vm/memory/metaspace.cpp.udiff.html > > > + void index_bounds_check(ChunkIndex index) const { > > Should be static. Or maybe a #define if you want to get the line where > the assert happens. Agree, I turned it into a macro to get the correct line numbers in an eventual hs_err file (good if the build is lacking debug symbols). On 2014-03-06 11:02, Stefan Karlsson wrote: > err_msg("Bad index: " INT32_FORMAT, (int) index)); > > Should be %d instead of INT32_FORMAT. Fixed. On 2014-03-06 11:02, Stefan Karlsson wrote: > 2682 return > MetaspaceChunkFreeListSummary(cm->num_free_chunks(SpecializedIndex), > 2683 > cm->num_free_chunks(SmallIndex), > 2684 > cm->num_free_chunks(MediumIndex), > 2685 > cm->num_free_chunks(HumongousIndex), > 2686 > cm->size_free_chunks_in_bytes(SpecializedIndex), > 2687 > cm->size_free_chunks_in_bytes(SmallIndex), > 2688 > cm->size_free_chunks_in_bytes(MediumIndex), > 2689 > cm->size_free_chunks_in_bytes(HumongousIndex)); > > I think it's better to keep the setup of MetaspaceChunkFreeListSummary > in ChunkManager. Agree, I've added the method ChunkManager::chunk_free_list_summary. On 2014-03-06 11:02, Stefan Karlsson wrote: > Otherwise, this looks good. Thanks! Erik > thanks, > StefanK > >> >> Enhancement: >> https://bugs.openjdk.java.net/browse/JDK-8036703 >> >> Testing: >> - JFR JTREG tests >> - JPRT >> - Manually verifying that the numbers look correct by looking at >> recordings from small test cases. >> >> Thanks, >> Erik > From vladimir.kozlov at oracle.com Thu Mar 6 08:57:50 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 06 Mar 2014 08:57:50 -0800 Subject: RFR (M): 8035647 : PPC64: Support for elf v2 abi. In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CEA6040@DEWDFEMB12A.global.corp.sap> References: <5317DC6B.7050801@oracle.com> <4295855A5C1DE049A61835A1887419CC2CEA6040@DEWDFEMB12A.global.corp.sap> Message-ID: <5318A90E.8050203@oracle.com> Good, I will push it today. Thanks, Vladimir On 3/6/14 2:27 AM, Lindenmaier, Goetz wrote: > Hi, > > I looked at the change and tested it on our linux ppc64 and on aix. > > It's good. > > I also tested it with the stage repo, that works fine, too. > Vladimir, can you push it to both, jdk9 and stage? > > Sasha, I added parts of your Mail to the bug description: > https://bugs.openjdk.java.net/browse/JDK-8035647 > > Best regards, > Goetz. > > > > -----Original Message----- > From: ppc-aix-port-dev-bounces at openjdk.java.net [mailto:ppc-aix-port-dev-bounces at openjdk.java.net] On Behalf Of Vladimir Kozlov > Sent: Donnerstag, 6. M?rz 2014 03:25 > To: Alexander Smundak; hotspot-dev at openjdk.java.net > Cc: ppc-aix-port-dev at openjdk.java.net > Subject: Re: RFR (M): 8035647 : PPC64: Support for elf v2 abi. > > Looks fine to me. > > If ppc64 community is okay with this I can push it. > I saw discussion on ppc-aix alias but I need their comments on hotspot-dev. > > For Hotspot-dev, Alexander can contribute a code into open jdk because > he is under Google's umbrella 'Oracle Contribution Agreement'. > > Thanks, > Vladimir > > On 3/5/14 1:42 PM, Alexander Smundak wrote: >> This patch adds support for the recently introduced ELFv2 ABI >> for the PowerPC64. >> The patch was generated for the jdk9/hs-rt/hotspot repository >> (the same patch has been already committed to the ppc-aix-port/jdk7u). >> It should for to jdk8u via ppc-aix-port/stage. >> >> This patch is ppc-only. >> >> ELFv2 ABI has been introduced recently and is used on by Linux running on >> the little endian PowerPC64. There is no official ELF ABI supplement for it, >> but the differences with "old" ABI for PowerPC64 are explained in the >> comments for the patches to the GCC compiler: >> http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01144.html >> http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01141.html >> http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01140.html >> http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01145.html >> http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01147.html >> http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01149.html >> http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01146.html >> http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01150.html >> >> The change that affects JVM most is the elimination of the function >> descriptors (which unfortunately requires introducing a number of #ifdef's >> into the code), and dropping two words from the base frame (which >> required mostly renaming of the structs from `abi_48' to a more neutral >> `abi_minframe', etc.) >> >> There are additional patches needed to fully support building >> JDK on the little endian PowerPC64; RFRs for them will follow. >> >> The development was done on Ubuntu 14.04 running on QEMU. >> With all the patches applied, I can build working JDK7 and JDK9 there. >> >> Please review and test this change. I need a sponsor. >> >> The patch is at: >> http://cr.openjdk.java.net/~rasbold/8035647/webrev.00/ >> >> Sasha >> From stefan.johansson at oracle.com Thu Mar 6 10:35:16 2014 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Thu, 06 Mar 2014 19:35:16 +0100 Subject: RFR: 8028498: runtime/SharedArchiveFile/CdsDifferentObjectAlignment.java asserts in RT_Baseline In-Reply-To: <5301DD2D.3040204@oracle.com> References: <52E90809.3000900@oracle.com> <52F91199.6030308@oracle.com> <52FA3DD9.20004@oracle.com> <52FBAB90.1020608@oracle.com> <5301DD2D.3040204@oracle.com> Message-ID: <5318BFE4.8010807@oracle.com> As Per noticed the _gc_locked_out_count isn't updated anymore when removing the GC locker lock/unlock methods. I've looked through the code to see what would be the proper action to take in clear_unhandled_oops(). The first observation was that the check for is_gc_locked_out() only had any effect during a short period of time during startup (the only place we used GC locker lock/unlock). This leads to the question, do we need to avoid clearing the unhandled oops when the GC lockers other lock mechanism is used, and I've look this through together with Per and it seems like it's not needed. One can argue that if we've locked out the GC using the GC locker, it should be fine to have unhandled oops, but since we don't make use of that today I see no reason for enabling it. If I haven't missed anything this should allow us to remove the check from clear_unhandled_oops() and the _gc_locked_out_count with accessors all together. Please let me know if you have any fears or thoughts around this. I've run both JPRT tests and some bigapps testing with +CheckUnhandledOops without any problem as sanity testing. Regarding the message I updated it to use the methods suggested by Per. Here's the updated webrev: http://cr.openjdk.java.net/~sjohanss/8028498/webrev.03/ Thanks, Stefan On 2014-02-17 10:58, Per Liden wrote: > Stefan, > > I think this is a nice fix for the problem and it also cleans up the > gclocker a bit. But I have a few minor comments: > > vmGCOperations.cpp: > 110 err_msg("GC triggered before VM initialization completed. > Try increasing " > 111 "NewSize, current value " UINTX_FORMAT "K.", > NewSize / K)); > > I'd suggest you use > byte_size_in_proper_unit()/proper_unit_for_byte_size() instead of a > "K", or maybe just not print the size at all. > > > With the removal of GC_locker::lock()/unlock() there's no longer > anyone how modifies Thread::current()->_gc_locked_out_count. So we > should be able to remove it. However, > UnhandledOops::clear_unhandled_oops() still uses is_gc_locked_out() so > that needs to be changed to check for something else, > is_init_completed maybe? > > cheers, > /Per > > On 02/12/2014 06:12 PM, Stefan Johansson wrote: >> Hi, >> >> Stefan K pointed out to me that in the Metaspace code we use >> is_init_completed() to verify that the VM is fully initialized. I >> started looking at where we do set_init_completed() and realized that it >> is right where we are ready to handle a GC. It makes sense to try to >> minimize the different ways we check if the VM is fully initialized. >> Using is_init_completed() instead of the GC locker to prevent GCs during >> startup has another nice effect too; we can remove one of the two >> different lock mechanisms in the GC locker. >> >> The new proposal removes the use of GC_locker::lock()/unlock() which is >> only used to prevent GCs during startup. Instead there is a check in the >> GC operations prologue, to ensure that the VM is initialized when the GC >> starts. >> >> Please review the new the change: >> http://cr.openjdk.java.net/~sjohanss/8028498/webrev.02/ >> >> Testing: >> * JPRT >> * GC locker test through UTE >> >> Thanks, >> Stefan >> >> On 2014-02-11 16:12, Stefan Johansson wrote: >>> Hi again =) >>> >>> Don't spend time on the current webrev (02). I've gotten some more >>> comments on the proposed fix and will look at a solution that allow us >>> to remove the GC locker usage during initialization. >>> >>> Cheers, >>> Stefan >>> >>> On 2014-02-10 18:51, Stefan Johansson wrote: >>>> Hi again, >>>> >>>> After a couple of offline discussions with different people I've come >>>> to the conclusion to split out the sizing of the young generation >>>> from the fix for this issue. I've created a new bug (JDK-8033426) for >>>> that, which is currently out for review. >>>> >>>> The discussions also lead to a different approach for fixing this >>>> issue. The new proposal is to use the GC locker to prevent GCs until >>>> we are ready to handle them and otherwise fail the VM initialization. >>>> The gc locker has two different mechanisms for locking out the GC, >>>> one which is used during initialization and one used to stall GCs >>>> when doing critical JNI work. The part of the initialization that >>>> currently is locked by the GC locker is to narrow and there is no >>>> guarantee that a GC can be handled after the GC locker is unlocked. >>>> This fix will extend the part that is locked and also add a check >>>> that if a GC is initiated while locked during startup the VM exits. >>>> >>>> New webrev: >>>> http://cr.openjdk.java.net/~sjohanss/8028498/webrev.01/ >>>> >>>> Thanks, >>>> Stefan >>>> >>>> On 2014-01-29 14:54, Stefan Johansson wrote: >>>>> Hi, >>>>> >>>>> Please review this fix for: >>>>> https://bugs.openjdk.java.net/browse/JDK-8028498 >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~sjohanss/8028498/webrev.00/ >>>>> >>>>> Summary: >>>>> The initial young generation size has been fairly small by default >>>>> (1.5M) and if setting ObjectAlignmentInBytes to something larger >>>>> than the default it can causes CDS to trigger a GC when dumping the >>>>> archive. At this point the VM is not ready to handle a GC and we >>>>> will hit an assertion. Making sure we can handle a GC at this point >>>>> is not trivial and the proposed solution is to alter the default >>>>> sizing of the young generation as well as adding a safety check when >>>>> dumping the archive to exit the VM if the young generation is too >>>>> small. >>>>> >>>>> Testing: >>>>> * JPRT build and test passes >>>>> * Failing test now passes on all platforms (tested throught JPRT) >>>>> * Verified that the GC and Runtime jtreg tests still passes >>>>> >>>>> Cheers, >>>>> Stefan >>>> >>> >> > From vladimir.kozlov at oracle.com Thu Mar 6 10:50:05 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 06 Mar 2014 10:50:05 -0800 Subject: RFR (M): 8035647 : PPC64: Support for elf v2 abi. In-Reply-To: <5318A90E.8050203@oracle.com> References: <5317DC6B.7050801@oracle.com> <4295855A5C1DE049A61835A1887419CC2CEA6040@DEWDFEMB12A.global.corp.sap> <5318A90E.8050203@oracle.com> Message-ID: <5318C35D.3010506@oracle.com> Alexander, Please, run jcheck on your changes. src/cpu/ppc/vm/cppInterpreter_ppc.cpp:1302: Tab character src/cpu/ppc/vm/macroAssembler_ppc.hpp:322: Trailing whitespace src/cpu/ppc/vm/stubGenerator_ppc.cpp:1708: Trailing whitespace I will do clean up this time. But next time I will not accept such changes. Thanks, Vladimir On 3/6/14 8:57 AM, Vladimir Kozlov wrote: > Good, I will push it today. > > Thanks, > Vladimir > > On 3/6/14 2:27 AM, Lindenmaier, Goetz wrote: >> Hi, >> >> I looked at the change and tested it on our linux ppc64 and on aix. >> >> It's good. >> >> I also tested it with the stage repo, that works fine, too. >> Vladimir, can you push it to both, jdk9 and stage? >> >> Sasha, I added parts of your Mail to the bug description: >> https://bugs.openjdk.java.net/browse/JDK-8035647 >> >> Best regards, >> Goetz. >> >> >> >> -----Original Message----- >> From: ppc-aix-port-dev-bounces at openjdk.java.net >> [mailto:ppc-aix-port-dev-bounces at openjdk.java.net] On Behalf Of >> Vladimir Kozlov >> Sent: Donnerstag, 6. M?rz 2014 03:25 >> To: Alexander Smundak; hotspot-dev at openjdk.java.net >> Cc: ppc-aix-port-dev at openjdk.java.net >> Subject: Re: RFR (M): 8035647 : PPC64: Support for elf v2 abi. >> >> Looks fine to me. >> >> If ppc64 community is okay with this I can push it. >> I saw discussion on ppc-aix alias but I need their comments on >> hotspot-dev. >> >> For Hotspot-dev, Alexander can contribute a code into open jdk because >> he is under Google's umbrella 'Oracle Contribution Agreement'. >> >> Thanks, >> Vladimir >> >> On 3/5/14 1:42 PM, Alexander Smundak wrote: >>> This patch adds support for the recently introduced ELFv2 ABI >>> for the PowerPC64. >>> The patch was generated for the jdk9/hs-rt/hotspot repository >>> (the same patch has been already committed to the ppc-aix-port/jdk7u). >>> It should for to jdk8u via ppc-aix-port/stage. >>> >>> This patch is ppc-only. >>> >>> ELFv2 ABI has been introduced recently and is used on by Linux >>> running on >>> the little endian PowerPC64. There is no official ELF ABI supplement >>> for it, >>> but the differences with "old" ABI for PowerPC64 are explained in the >>> comments for the patches to the GCC compiler: >>> http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01144.html >>> http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01141.html >>> http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01140.html >>> http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01145.html >>> http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01147.html >>> http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01149.html >>> http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01146.html >>> http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01150.html >>> >>> The change that affects JVM most is the elimination of the function >>> descriptors (which unfortunately requires introducing a number of >>> #ifdef's >>> into the code), and dropping two words from the base frame (which >>> required mostly renaming of the structs from `abi_48' to a more neutral >>> `abi_minframe', etc.) >>> >>> There are additional patches needed to fully support building >>> JDK on the little endian PowerPC64; RFRs for them will follow. >>> >>> The development was done on Ubuntu 14.04 running on QEMU. >>> With all the patches applied, I can build working JDK7 and JDK9 there. >>> >>> Please review and test this change. I need a sponsor. >>> >>> The patch is at: >>> http://cr.openjdk.java.net/~rasbold/8035647/webrev.00/ >>> >>> Sasha >>> From christian.thalinger at oracle.com Thu Mar 6 13:02:49 2014 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Thu, 6 Mar 2014 13:02:49 -0800 Subject: Bring Zero to llvm 3.4 In-Reply-To: <1394014955.3834.33.camel@galactica.localdomain> References: <1393239625.3873.13.camel@galactica.localdomain> <1393256374.11941.3.camel@galactica.localdomain> <1393853313.3834.6.camel@galactica.localdomain> <1581851.jFBPTQiW9h@mgerdin03> <1393854861.3834.8.camel@galactica.localdomain> <38D22382-B630-4EF3-9B20-02D380484155@oracle.com> <1393920411.3834.12.camel@galactica.localdomain> <1393954160.3834.27.camel@galactica.localdomain> <1393954936.3834.30.camel@galactica.localdomain> <1394014955.3834.33.camel@galactica.localdomain> Message-ID: <6319B2C7-E920-4A82-A947-77F692C3ECC1@oracle.com> On Mar 5, 2014, at 2:22 AM, Mario Torre wrote: > On Tue, 2014-03-04 at 14:00 -0800, Christian Thalinger wrote: >> Hmm. This is what I get with LLVM 3.4: >> >> $ java -version >> Error: dl failure on line 864 >> Error: failed /export/twisti/build/hotspot/jdk-linux/jre/lib/amd64/server/libjvm.so, because /export/twisti/build/hotspot/jdk-linux/jre/lib/amd64/server/libjvm.so: undefined symbol: setupterm >> >> Any ideas? > > Hi Christian, > > Yes, when building llvm, you need to configure it without libterm > support: > > --enable-terminfo=no --enable-curses=no Right. I?ve figured that out after a while and got it working. Thanks. > > I think something changed recently in llvm, because I didn't see this > error before either, maybe now those are defaults and before not, not > sure. > > Maybe I should add this information to some of our docs? That would be good. > > Cheers, > Mario > > From neugens at redhat.com Thu Mar 6 13:04:24 2014 From: neugens at redhat.com (Mario Torre) Date: Thu, 06 Mar 2014 22:04:24 +0100 Subject: Bring Zero to llvm 3.4 In-Reply-To: <6319B2C7-E920-4A82-A947-77F692C3ECC1@oracle.com> References: <1393239625.3873.13.camel@galactica.localdomain> <1393256374.11941.3.camel@galactica.localdomain> <1393853313.3834.6.camel@galactica.localdomain> <1581851.jFBPTQiW9h@mgerdin03> <1393854861.3834.8.camel@galactica.localdomain> <38D22382-B630-4EF3-9B20-02D380484155@oracle.com> <1393920411.3834.12.camel@galactica.localdomain> <1393954160.3834.27.camel@galactica.localdomain> <1393954936.3834.30.camel@galactica.localdomain> <1394014955.3834.33.camel@galactica.localdomain> <6319B2C7-E920-4A82-A947-77F692C3ECC1@oracle.com> Message-ID: <1394139864.3834.141.camel@galactica.localdomain> On Thu, 2014-03-06 at 13:02 -0800, Christian Thalinger wrote: > On Mar 5, 2014, at 2:22 AM, Mario Torre wrote: > > > On Tue, 2014-03-04 at 14:00 -0800, Christian Thalinger wrote: > >> Hmm. This is what I get with LLVM 3.4: > >> > >> $ java -version > >> Error: dl failure on line 864 > >> Error: failed /export/twisti/build/hotspot/jdk-linux/jre/lib/amd64/server/libjvm.so, because /export/twisti/build/hotspot/jdk-linux/jre/lib/amd64/server/libjvm.so: undefined symbol: setupterm > >> > >> Any ideas? > > > > Hi Christian, > > > > Yes, when building llvm, you need to configure it without libterm > > support: > > > > --enable-terminfo=no --enable-curses=no > > Right. I?ve figured that out after a while and got it working. Thanks. > > > > > I think something changed recently in llvm, because I didn't see this > > error before either, maybe now those are defaults and before not, not > > sure. > > > > Maybe I should add this information to some of our docs? > > That would be good. I'll prepare a patch asap then. Cheers, Mario From christian.thalinger at oracle.com Thu Mar 6 13:04:45 2014 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Thu, 6 Mar 2014 13:04:45 -0800 Subject: RFR(L): 8029075 - String deduplication in G1 In-Reply-To: <5316F4D2.7010600@oracle.com> References: <53148494.60806@oracle.com> <3A26A868-51A8-4169-B147-9AEB8B35EC5E@oracle.com> <53159E0A.60403@oracle.com> <5A84BB3B-9763-47C3-9EEF-0D07EB763576@oracle.com> <5316F4D2.7010600@oracle.com> Message-ID: On Mar 5, 2014, at 1:56 AM, Per Liden wrote: > Yep, a more efficient mechanism for reference processing without the current overhead per reference would open up for some interesting possibilities. I have some loose/untested ideas on how we could address some of the efficiency problems, but the memory overhead is trickier to solve. I talked to John Rose about this and he has some ideas. One way to solve this is to use Value Types but that is still far out. Another way would be to use some kind of annotation to mark references and teach the GCs about that. > > /Per > > On 03/04/2014 05:55 PM, Christian Thalinger wrote: >> That?s unfortunate. It clearly would have been easier to do all that work in Java. >> >> On Mar 4, 2014, at 1:34 AM, Per Liden wrote: >> >>> Hi Christian, >>> >>> That was something Mikael Vidstedt and I briefly touched on when we first started to think about deduplication. However, we didn't go down that path, partly because of the problem you mention, but also because of the general overhead of WeakReferences (in terms of memory and work added to the reference processor and Reference Handler). >>> >>> /Per >>> >>> On 03/04/2014 02:48 AM, Christian Thalinger wrote: >>>> Just a general question: Have you thought about writing the deduplication thread in Java? Maybe also the deduplication table? >>>> >>>> There is clearly the problem of enqueuing. One easy solution would be to have the deduplication thread poll into the VM to get the next entries but there might be a better away to do it. >>>> >>>> On Mar 3, 2014, at 5:33 AM, Per Liden wrote: >>>> >>>>> Hi, >>>>> >>>>> Could I please have this patch reviewed. >>>>> >>>>> >>>>> Summary >>>>> ------- >>>>> This patch implements JEP 192 - String deduplication in G1. The goal of string deduplication is to reduce the Java heap live-data set by enhancing the G1 garbage collector so that duplicate instances of String are automatically and continuously deduplicated. >>>>> >>>>> I'd like to refer to the JEP for a more detailed description of this feature, the motivation for it, the expected benefit, how it's implemented, etc. >>>>> >>>>> This patch is mainly G1-related, but also touches a few runtime files. >>>>> >>>>> Webrev: http://cr.openjdk.java.net/~pliden/8029075/webrev.0/ >>>>> >>>>> JEP: http://openjdk.java.net/jeps/192 >>>>> >>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8029075 >>>>> >>>>> >>>>> Testing >>>>> ------- >>>>> * JTreg - The patch includes the following 8 new tests: >>>>> - TestStringDeduplicationYoungGC.java: Tests deduplication during Young GC. >>>>> - TestStringDeduplicationFullGC.java: Tests deduplication during Full GC. >>>>> - TestStringDeduplicationAgeThreshold.java: Tests both valid and invalid age threshold settings. >>>>> - TestStringDeduplicationInterned.java: Tests that interned strings are deduplication before being interned. >>>>> - TestStringDeduplicationTableRehash.java: Stresses the hashtables ability to rehash all entries. >>>>> - TestStringDeduplicationTableResize.java: Stresses the hashtables ability to resize itself. >>>>> - TestStringDeduplicationMemoryUsage.java: Tests heap reduction when string deduplication is enabled. >>>>> - TestStringDeduplicationPrintOptions.java: Tests command line options. >>>>> >>>>> * Stress testing: >>>>> - Kitchensink >>>>> - GCBasher >>>>> >>>>> * Regression testing: >>>>> - JCK >>>>> - vmTestbase >>>>> - Bigapps >>>>> >>>>> * Large scale benchmarks to test heap reduction and performance impact: >>>>> - FA CRM Sales Op. Flow >>>>> - FA DSS >>>>> >>>>> * The following benchmarks have been executed to verify that this feature doesn't impact performance when disabled (even when disabled there are still some "if (UseStringDeduplication)" executed in some hot paths). >>>>> - SPECjbb2005 >>>>> - SPECjbb2013 >>>>> - SPECjvm2008-XML >>>>> >>>>> * Various ad-hoc tests and microbenchmarks were also written and executed during the course of the development. >>>>> >>>>> >>>>> Thanks! >>>>> /Per >>>> >>> >> > From vladimir.kozlov at oracle.com Thu Mar 6 14:08:23 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 06 Mar 2014 14:08:23 -0800 Subject: RFR[9](S): 8036777: AIX: Adapt implementation after '6546236: Thread interrupt() of Thread.sleep() can be lost on Solaris..' In-Reply-To: References: Message-ID: <5318F1D7.8010909@oracle.com> Reviewed and the push in JPRT queue. Thanks, Vladimir K On 3/6/14 7:09 AM, Volker Simonis wrote: > Hi, > > could you please review and push the following little AIX-only change > which adapts the refactorings of '6546236: Thread interrupt() of > Thread.sleep() can be lost on Solaris..' to AIX: > > http://cr.openjdk.java.net/~simonis/webrevs/8036777/ > https://bugs.openjdk.java.net/browse/JDK-8036777 > > The change removes the implementations of os::sleep(), os::interrupt() > and os::is_interrupted() from os_aix.cpp such that AIX now uses the > implementations from os_posix.cpp like the other Unix platfroms and > defines os::supports_monotonic_clock() in os_aix.inline.hpp. > > Thank you and best regards, > Volker > From christian.thalinger at oracle.com Thu Mar 6 14:31:10 2014 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Thu, 6 Mar 2014 14:31:10 -0800 Subject: RFR(L): 8029075 - String deduplication in G1 In-Reply-To: References: <53148494.60806@oracle.com> <3A26A868-51A8-4169-B147-9AEB8B35EC5E@oracle.com> <53159E0A.60403@oracle.com> <5A84BB3B-9763-47C3-9EEF-0D07EB763576@oracle.com> <5316F4D2.7010600@oracle.com> Message-ID: <2A9E63DB-F0DE-4E82-A6D1-C116C4113C2B@oracle.com> [Putting John in the loop.] On Mar 6, 2014, at 1:04 PM, Christian Thalinger wrote: > > On Mar 5, 2014, at 1:56 AM, Per Liden wrote: > >> Yep, a more efficient mechanism for reference processing without the current overhead per reference would open up for some interesting possibilities. I have some loose/untested ideas on how we could address some of the efficiency problems, but the memory overhead is trickier to solve. > > I talked to John Rose about this and he has some ideas. One way to solve this is to use Value Types but that is still far out. Another way would be to use some kind of annotation to mark references and teach the GCs about that. > >> >> /Per >> >> On 03/04/2014 05:55 PM, Christian Thalinger wrote: >>> That?s unfortunate. It clearly would have been easier to do all that work in Java. >>> >>> On Mar 4, 2014, at 1:34 AM, Per Liden wrote: >>> >>>> Hi Christian, >>>> >>>> That was something Mikael Vidstedt and I briefly touched on when we first started to think about deduplication. However, we didn't go down that path, partly because of the problem you mention, but also because of the general overhead of WeakReferences (in terms of memory and work added to the reference processor and Reference Handler). >>>> >>>> /Per >>>> >>>> On 03/04/2014 02:48 AM, Christian Thalinger wrote: >>>>> Just a general question: Have you thought about writing the deduplication thread in Java? Maybe also the deduplication table? >>>>> >>>>> There is clearly the problem of enqueuing. One easy solution would be to have the deduplication thread poll into the VM to get the next entries but there might be a better away to do it. >>>>> >>>>> On Mar 3, 2014, at 5:33 AM, Per Liden wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> Could I please have this patch reviewed. >>>>>> >>>>>> >>>>>> Summary >>>>>> ------- >>>>>> This patch implements JEP 192 - String deduplication in G1. The goal of string deduplication is to reduce the Java heap live-data set by enhancing the G1 garbage collector so that duplicate instances of String are automatically and continuously deduplicated. >>>>>> >>>>>> I'd like to refer to the JEP for a more detailed description of this feature, the motivation for it, the expected benefit, how it's implemented, etc. >>>>>> >>>>>> This patch is mainly G1-related, but also touches a few runtime files. >>>>>> >>>>>> Webrev: http://cr.openjdk.java.net/~pliden/8029075/webrev.0/ >>>>>> >>>>>> JEP: http://openjdk.java.net/jeps/192 >>>>>> >>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8029075 >>>>>> >>>>>> >>>>>> Testing >>>>>> ------- >>>>>> * JTreg - The patch includes the following 8 new tests: >>>>>> - TestStringDeduplicationYoungGC.java: Tests deduplication during Young GC. >>>>>> - TestStringDeduplicationFullGC.java: Tests deduplication during Full GC. >>>>>> - TestStringDeduplicationAgeThreshold.java: Tests both valid and invalid age threshold settings. >>>>>> - TestStringDeduplicationInterned.java: Tests that interned strings are deduplication before being interned. >>>>>> - TestStringDeduplicationTableRehash.java: Stresses the hashtables ability to rehash all entries. >>>>>> - TestStringDeduplicationTableResize.java: Stresses the hashtables ability to resize itself. >>>>>> - TestStringDeduplicationMemoryUsage.java: Tests heap reduction when string deduplication is enabled. >>>>>> - TestStringDeduplicationPrintOptions.java: Tests command line options. >>>>>> >>>>>> * Stress testing: >>>>>> - Kitchensink >>>>>> - GCBasher >>>>>> >>>>>> * Regression testing: >>>>>> - JCK >>>>>> - vmTestbase >>>>>> - Bigapps >>>>>> >>>>>> * Large scale benchmarks to test heap reduction and performance impact: >>>>>> - FA CRM Sales Op. Flow >>>>>> - FA DSS >>>>>> >>>>>> * The following benchmarks have been executed to verify that this feature doesn't impact performance when disabled (even when disabled there are still some "if (UseStringDeduplication)" executed in some hot paths). >>>>>> - SPECjbb2005 >>>>>> - SPECjbb2013 >>>>>> - SPECjvm2008-XML >>>>>> >>>>>> * Various ad-hoc tests and microbenchmarks were also written and executed during the course of the development. >>>>>> >>>>>> >>>>>> Thanks! >>>>>> /Per >>>>> >>>> >>> >> > From alejandro.murillo at oracle.com Thu Mar 6 14:23:31 2014 From: alejandro.murillo at oracle.com (alejandro.murillo at oracle.com) Date: Thu, 06 Mar 2014 22:23:31 +0000 Subject: hg: hsx/jdk7u/hotspot: 8036778: new hotspot build - hs24.80-b05 Message-ID: <20140306222346.092CC625BA@hg.openjdk.java.net> Changeset: b41876fe0247 Author: amurillo Date: 2014-03-06 06:06 -0800 URL: http://hg.openjdk.java.net/hsx/jdk7u/hotspot/rev/b41876fe0247 8036778: new hotspot build - hs24.80-b05 Reviewed-by: jcoomes ! make/hotspot_version From john.r.rose at oracle.com Thu Mar 6 16:40:00 2014 From: john.r.rose at oracle.com (John Rose) Date: Thu, 6 Mar 2014 16:40:00 -0800 Subject: RFR(L): 8029075 - String deduplication in G1 In-Reply-To: References: <53148494.60806@oracle.com> <3A26A868-51A8-4169-B147-9AEB8B35EC5E@oracle.com> <53159E0A.60403@oracle.com> <5A84BB3B-9763-47C3-9EEF-0D07EB763576@oracle.com> <5316F4D2.7010600@oracle.com> Message-ID: <7AD2B58F-1268-4A9C-9749-C6C7DAF5AB25@oracle.com> On Mar 6, 2014, at 1:04 PM, Christian Thalinger wrote: > Another way would be to use some kind of annotation to mark references and teach the GCs about that. Here's how the story looks to me: Define a new annotation @WeakReferenceField. Take Alexey's code for @Contended and duplicate with adjustments for @WeakReferenceField. Define another field bit and store it in metadata. Looks for the bit in the JVM wherever it currently asks about java_lang_ref_Reference::referent_offset. (This is the hard part, especially for G1, but it is traded against adding more GC-sensitive C++ code into the JVM, for a benefit of narrow scope.) Don't bother about queueing; it's optional for WeakReference. The goal is that the following classes work about the same, except for heap pressure: > class A1 { > private WeakReference ref1 = new WeakReference<>(something); > private WeakReference ref2 = new WeakReference<>(somethingElse); > } > class A2 { > private @WeakReferenceField String ref1 = something; > private @WeakReferenceField String ref2 = somethingElse; > } Put the annotation in sun.misc or some other internal place. Use it only for private JVM-coupled things like string dedup (no JCK or general testing). Put whatever practical limits you want on the implementation. Let the next user (there will be a next user) refine it further. Enjoy coding and debugging multi-thread GC-safe code in Java. I hope this helps! ? John P.S. Connection with value types: Eventually, we could add value types like java.lang.ref.WeakReferenceValue which would provide a standard form for this. Or we could standardize the annotation, perhaps. P.P.S. For a little more on value types, see https://blogs.oracle.com/jrose/entry/value_types_and_struct_tearing From david.holmes at oracle.com Thu Mar 6 20:14:06 2014 From: david.holmes at oracle.com (David Holmes) Date: Fri, 07 Mar 2014 14:14:06 +1000 Subject: RFR[9](S): 8036777: AIX: Adapt implementation after '6546236: Thread interrupt() of Thread.sleep() can be lost on Solaris..' In-Reply-To: References: Message-ID: <5319478E.2030705@oracle.com> Hi Volker, I see Vladimir is already pushing this. It looks okay to me too. Aside: I was surprised to see this: 889 bool os::create_thread(Thread* thread, ThreadType thr_type, size_t stack_size) { 890 891 // We want the whole function to be synchronized. 892 ThreadCritical cs; Why do you need to serialize thread creation? David ----- On 7/03/2014 1:09 AM, Volker Simonis wrote: > Hi, > > could you please review and push the following little AIX-only change > which adapts the refactorings of '6546236: Thread interrupt() of > Thread.sleep() can be lost on Solaris..' to AIX: > > http://cr.openjdk.java.net/~simonis/webrevs/8036777/ > https://bugs.openjdk.java.net/browse/JDK-8036777 > > The change removes the implementations of os::sleep(), os::interrupt() > and os::is_interrupted() from os_aix.cpp such that AIX now uses the > implementations from os_posix.cpp like the other Unix platfroms and > defines os::supports_monotonic_clock() in os_aix.inline.hpp. > > Thank you and best regards, > Volker > From vladimir.kempik at oracle.com Fri Mar 7 03:19:45 2014 From: vladimir.kempik at oracle.com (Vladimir Kempik) Date: Fri, 07 Mar 2014 15:19:45 +0400 Subject: RFR: 8036863: Update jdk7 testlibrary to match jdk8 in hotspot Message-ID: <5319AB51.7030704@oracle.com> Please review this change to update hotspot part of testlibrary in jdk7. It updates testlibrary with new apis from jdk8's testlibrary. Every test in open part of hotspot that uses testlibrary and was working fine before update still works after update. Webrev: http://cr.openjdk.java.net/~vkempik/8036863/webrev.00/ Thanks, Vladimir. From per.liden at oracle.com Fri Mar 7 06:13:56 2014 From: per.liden at oracle.com (Per Liden) Date: Fri, 07 Mar 2014 15:13:56 +0100 Subject: RFR(L): 8029075 - String deduplication in G1 In-Reply-To: <5316F5B1.60006@oracle.com> References: <53148494.60806@oracle.com> <5316F5B1.60006@oracle.com> Message-ID: <5319D424.90306@oracle.com> Hi, Here's an updated webrev based on the feedback from Bengt and Thomas. http://cr.openjdk.java.net/~pliden/8029075/webrev.1/ Diffs against previous webrev available here: http://cr.openjdk.java.net/~pliden/8029075/webrev.1-bengt_review_fixes/ http://cr.openjdk.java.net/~pliden/8029075/webrev.1-thomas_review_fixes/ And as suggested, two changes in the initial webrev were broken out into separate CRs, available here: Bug: https://bugs.openjdk.java.net/browse/JDK-8036672 Webrev: http://cr.openjdk.java.net/~pliden/8036672/webrev.0/ Bug: https://bugs.openjdk.java.net/browse/JDK-8036673 Webrev: http://cr.openjdk.java.net/~pliden/8036673/webrev.0/ /Per On 03/05/2014 11:00 AM, Bengt Rutisson wrote: > > Hi Per and everyone else, > > Overall this looks good to me. Me and Per went through this patch > together and here are the notes from this code walk through. > > There are two small bug fixes included in the change. It would be good > to separate these out to separate changes. We can review them together > with the main change, but having them as separate changesets will make > it easier to backport them if necessary. > > > stringDedup.hpp > The comments are not that helpful to me. I would suggest to either > remove them or make more descriptive. > > UseG1GC && UseStringDeduplication > Only allow UseStringDeduplication to be true if UseG1GC is true instead > of checking both? > > G1CollectedHeap::tear_down_region_sets() and > G1CollectedHeap::rebuild_region_sets() > Please add an RFE to clean these up as the naming and implementations > don't match anymore. > > StringDedupTable::trim() > rename to trim_cache? > > stringDedup.cpp > Two occurrences of: > assert(UseG1GC, "String deduplication not available with G1"); > The "not" should be a "only". > > Dummy statistics from interned stings is ignored. Please add an RFE to > try to include this statistics and possibly reduce the "known" count. > > > Add comments explaining why resize and rehash is not allowed for these > code paths: > > void StringDedup::unlink(BoolObjectClosure* is_alive) { > unlink_or_oops_do(is_alive, NULL, false /* allow_resize_and_rehash */); > } > > void StringDedup::oops_do(OopClosure* keep_alive) { > unlink_or_oops_do(NULL, keep_alive, false /* allow_resize_and_rehash > */); > } > > TestStringDeduplicationTools.java > verifyStrings put the sleep at the end to possibly speed up the test a > little. > > InternedTest > Comments to explain what it does. > > testYoungGC - check that no full GCs happen > testFullGC - check that no young GCs happen > > testMemoryUsage - add "memoryUsageWithDedup > > memoryUsageWithDedupExpected" to the exception message > > > Thanks, > Bengt > > > On 3/3/14 2:33 PM, Per Liden wrote: >> Hi, >> >> Could I please have this patch reviewed. >> >> >> Summary >> ------- >> This patch implements JEP 192 - String deduplication in G1. The goal >> of string deduplication is to reduce the Java heap live-data set by >> enhancing the G1 garbage collector so that duplicate instances of >> String are automatically and continuously deduplicated. >> >> I'd like to refer to the JEP for a more detailed description of this >> feature, the motivation for it, the expected benefit, how it's >> implemented, etc. >> >> This patch is mainly G1-related, but also touches a few runtime files. >> >> Webrev: http://cr.openjdk.java.net/~pliden/8029075/webrev.0/ >> >> JEP: http://openjdk.java.net/jeps/192 >> >> RFE: https://bugs.openjdk.java.net/browse/JDK-8029075 >> >> >> Testing >> ------- >> * JTreg - The patch includes the following 8 new tests: >> - TestStringDeduplicationYoungGC.java: Tests deduplication during >> Young GC. >> - TestStringDeduplicationFullGC.java: Tests deduplication during >> Full GC. >> - TestStringDeduplicationAgeThreshold.java: Tests both valid and >> invalid age threshold settings. >> - TestStringDeduplicationInterned.java: Tests that interned strings >> are deduplication before being interned. >> - TestStringDeduplicationTableRehash.java: Stresses the hashtables >> ability to rehash all entries. >> - TestStringDeduplicationTableResize.java: Stresses the hashtables >> ability to resize itself. >> - TestStringDeduplicationMemoryUsage.java: Tests heap reduction when >> string deduplication is enabled. >> - TestStringDeduplicationPrintOptions.java: Tests command line options. >> >> * Stress testing: >> - Kitchensink >> - GCBasher >> >> * Regression testing: >> - JCK >> - vmTestbase >> - Bigapps >> >> * Large scale benchmarks to test heap reduction and performance impact: >> - FA CRM Sales Op. Flow >> - FA DSS >> >> * The following benchmarks have been executed to verify that this >> feature doesn't impact performance when disabled (even when disabled >> there are still some "if (UseStringDeduplication)" executed in some >> hot paths). >> - SPECjbb2005 >> - SPECjbb2013 >> - SPECjvm2008-XML >> >> * Various ad-hoc tests and microbenchmarks were also written and >> executed during the course of the development. >> >> >> Thanks! >> /Per > From volker.simonis at gmail.com Fri Mar 7 06:33:48 2014 From: volker.simonis at gmail.com (Volker Simonis) Date: Fri, 7 Mar 2014 15:33:48 +0100 Subject: RFR[9](S): 8036777: AIX: Adapt implementation after '6546236: Thread interrupt() of Thread.sleep() can be lost on Solaris..' In-Reply-To: <5319478E.2030705@oracle.com> References: <5319478E.2030705@oracle.com> Message-ID: Hi David, that's a good question! I dug in our repos and asked around and it seems that this is in the port port since the very first days for now unknown reasons. I removed it in our internal version and if we don't see any regressions I'll remove it from the OpenJDK port as well. Thanks for pointing this out, Volker On Fri, Mar 7, 2014 at 5:14 AM, David Holmes wrote: > Hi Volker, > > I see Vladimir is already pushing this. It looks okay to me too. > > Aside: I was surprised to see this: > > 889 bool os::create_thread(Thread* thread, ThreadType thr_type, size_t > stack_size) { > 890 > 891 // We want the whole function to be synchronized. > 892 ThreadCritical cs; > > Why do you need to serialize thread creation? > > David > ----- > > > On 7/03/2014 1:09 AM, Volker Simonis wrote: >> >> Hi, >> >> could you please review and push the following little AIX-only change >> which adapts the refactorings of '6546236: Thread interrupt() of >> Thread.sleep() can be lost on Solaris..' to AIX: >> >> http://cr.openjdk.java.net/~simonis/webrevs/8036777/ >> https://bugs.openjdk.java.net/browse/JDK-8036777 >> >> The change removes the implementations of os::sleep(), os::interrupt() >> and os::is_interrupted() from os_aix.cpp such that AIX now uses the >> implementations from os_posix.cpp like the other Unix platfroms and >> defines os::supports_monotonic_clock() in os_aix.inline.hpp. >> >> Thank you and best regards, >> Volker >> > From gnu.andrew at redhat.com Sun Mar 9 22:30:52 2014 From: gnu.andrew at redhat.com (Andrew Hughes) Date: Sun, 9 Mar 2014 18:30:52 -0400 (EDT) Subject: RR(S): 8009062 poor performance of JNI AttachCurrentThread after fix for 7017193 In-Reply-To: <5213B5C1.3000705@oracle.com> References: <5213B5C1.3000705@oracle.com> Message-ID: <826130727.1094256.1394404252778.JavaMail.zimbra@redhat.com> ----- Original Message ----- > Hi Everybody, > > Here is webrev of changes I'm about to integrate: > > http://cr.openjdk.java.net/~dsamersoff/8009062/webrev.13/ > > The problem: > > Under Linux stack of main thread is growable, so we have to make sure > that address we plan to put a guard pages to and below is not mapped. > > Historically we find bounds of the stack of main thread by seeking > /proc/self/maps for "[stack]" and parsing this line. > > But after intensive testing we figured out that it's not required to > re-read stack bounds after create_attached_thread() because we are > manually grow stack up to the bounds in create_attached_thread() > > > Solution: > > I use mincore(2) to check whether > the page is mapped or not. Typically mincore(2) is used to check whether > the page is resides in physical memory or not, but this function returns > -1 and set errno to ENOMEM if a region we are asking about contains not > mapped memory. > > Testing: > > Passed jprt, jck and and couple of ute tests. Wrote special regression test. > > -Dmitry > > -- > Dmitry Samersoff > Oracle Java development team, Saint Petersburg, Russia > * I would love to change the world, but they won't give me the sources. > I noticed this fix still isn't in 7u. Are there plans to backport it? 7017193, which causes a major performance regression, is present in 7u. Thanks, -- Andrew :) Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) PGP Key: 248BDC07 (https://keys.indymedia.org/) Fingerprint = EC5A 1F5E C0AD 1D15 8F1F 8F91 3B96 A578 248B DC07 From stefan.karlsson at oracle.com Mon Mar 10 09:04:37 2014 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 10 Mar 2014 10:04:37 +0100 Subject: RFR: 8036703: Add trace event with statistics for the metaspace chunk free lists In-Reply-To: <53189F76.80503@oracle.com> References: <53184144.3060905@oracle.com> <531847B1.6080709@oracle.com> <53189F76.80503@oracle.com> Message-ID: <531D8025.1050604@oracle.com> On 2014-03-06 17:16, Erik Helin wrote: > Hi Stefan, > > thanks for reviewing! Please see comments inline and the new webrev at: > http://cr.openjdk.java.net/~ehelin/8036703/webrev.02/ The updates look good. StefanK > > On 2014-03-06 11:02, Stefan Karlsson wrote: >> >> On 2014-03-06 10:35, Erik Helin wrote: >>> Hi all, >>> >>> this patch adds a new trace event - >>> vm/gc/metaspace/chunk_free_list_summary for getting statistics for >>> metaspace's chunk free lists. The event contains the numbers of chunks >>> in the free list and the size of the chunks in the free lists. This >>> information is provided for all different kinds of chunks - special, >>> small, medium and humongous. >>> >>> This information will be useful when diagnosing fragmentation issues >>> in metaspace. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~ehelin/8036703/webrev.00/ >> >> http://cr.openjdk.java.net/~ehelin/8036703/webrev.00/src/share/vm/gc_interface/collectedHeap.cpp.udiff.html >> >> >> >> This isn't used: >> >> + MetaspaceChunkFreeListSummary chunk_free_list_summary; > > Fixed. > > On 2014-03-06 11:02, Stefan Karlsson wrote: >> http://cr.openjdk.java.net/~ehelin/8036703/webrev.00/src/share/vm/memory/metaspace.cpp.udiff.html >> >> >> >> + void index_bounds_check(ChunkIndex index) const { >> >> Should be static. Or maybe a #define if you want to get the line where >> the assert happens. > > Agree, I turned it into a macro to get the correct line numbers in an > eventual hs_err file (good if the build is lacking debug symbols). > > On 2014-03-06 11:02, Stefan Karlsson wrote: >> err_msg("Bad index: " INT32_FORMAT, (int) index)); >> >> Should be %d instead of INT32_FORMAT. > > Fixed. > > On 2014-03-06 11:02, Stefan Karlsson wrote: >> 2682 return >> MetaspaceChunkFreeListSummary(cm->num_free_chunks(SpecializedIndex), >> 2683 >> cm->num_free_chunks(SmallIndex), >> 2684 >> cm->num_free_chunks(MediumIndex), >> 2685 >> cm->num_free_chunks(HumongousIndex), >> 2686 >> cm->size_free_chunks_in_bytes(SpecializedIndex), >> 2687 >> cm->size_free_chunks_in_bytes(SmallIndex), >> 2688 >> cm->size_free_chunks_in_bytes(MediumIndex), >> 2689 >> cm->size_free_chunks_in_bytes(HumongousIndex)); >> >> I think it's better to keep the setup of MetaspaceChunkFreeListSummary >> in ChunkManager. > > Agree, I've added the method ChunkManager::chunk_free_list_summary. > > On 2014-03-06 11:02, Stefan Karlsson wrote: >> Otherwise, this looks good. > > Thanks! > Erik > >> thanks, >> StefanK >> >>> >>> Enhancement: >>> https://bugs.openjdk.java.net/browse/JDK-8036703 >>> >>> Testing: >>> - JFR JTREG tests >>> - JPRT >>> - Manually verifying that the numbers look correct by looking at >>> recordings from small test cases. >>> >>> Thanks, >>> Erik >> From per.liden at oracle.com Mon Mar 10 12:36:13 2014 From: per.liden at oracle.com (Per Liden) Date: Mon, 10 Mar 2014 13:36:13 +0100 Subject: RFR(L): 8029075 - String deduplication in G1 In-Reply-To: <7AD2B58F-1268-4A9C-9749-C6C7DAF5AB25@oracle.com> References: <53148494.60806@oracle.com> <3A26A868-51A8-4169-B147-9AEB8B35EC5E@oracle.com> <53159E0A.60403@oracle.com> <5A84BB3B-9763-47C3-9EEF-0D07EB763576@oracle.com> <5316F4D2.7010600@oracle.com> <7AD2B58F-1268-4A9C-9749-C6C7DAF5AB25@oracle.com> Message-ID: <531DB1BD.4050404@oracle.com> Hi John & Christian, On 03/07/2014 01:40 AM, John Rose wrote: > On Mar 6, 2014, at 1:04 PM, Christian Thalinger > > > wrote: > >> Another way would be to use some kind of annotation to mark references >> and teach the GCs about that. > > Here's how the story looks to me: > > Define a new annotation @WeakReferenceField. > > Take Alexey's code for @Contended and duplicate with adjustments for > @WeakReferenceField. Define another field bit and store it in metadata. > > Looks for the bit in the JVM wherever it currently asks about > java_lang_ref_Reference::referent_offset. > (This is the hard part, especially for G1, but it is traded against > adding more GC-sensitive C++ code into the JVM, for a benefit of narrow > scope.) > Interesting idea. And the equivalent of the Reference.discovered field would be injected after each weak-annotated field? > Don't bother about queueing; it's optional for WeakReference. In cases where we keep lots of weak references in things like hashmaps I can easily imagine that we'd want some kind of notification when references become stale so that the map can be pruned. Injecting another field after the weak-field to keep track of the queue would of course be an option in that case. > > The goal is that the following classes work about the same, except for > heap pressure: > >> class A1 { >> private WeakReference ref1 = new WeakReference<>(something); >> private WeakReference ref2 = new WeakReference<>(somethingElse); >> } >> class A2 { >> private @WeakReferenceField String ref1 = something; >> private @WeakReferenceField String ref2 = somethingElse; >> } > > Put the annotation in sun.misc or some other internal place. Use it > only for private JVM-coupled things like string dedup (no JCK or general > testing). > > Put whatever practical limits you want on the implementation. Let the > next user (there will be a next user) refine it further. Even if the memory overhead isn't completely removed here it's probably starting to approach a level that's good enough even for heavy users or weak refs. > > Enjoy coding and debugging multi-thread GC-safe code in Java. > > I hope this helps! The other part of the story, orthogonal to the memory overhead, is making reference processing more efficient once the referent becomes unreachable. I've seen many cases where heavy use of weak references (registered or not) overloads the reference handler thread. To improve that situation, I'd like to see the reference handler thread removed completely, and instead let the GC enqueue reference directly onto the reference queues. /Per > > ? John > > P.S. Connection with value types: Eventually, we could add value types > like java.lang.ref.WeakReferenceValue which would provide a standard > form for this. Or we could standardize the annotation, perhaps. > > P.P.S. For a little more on value types, see > https://blogs.oracle.com/jrose/entry/value_types_and_struct_tearing From asmundak at google.com Mon Mar 10 17:44:32 2014 From: asmundak at google.com (Alexander Smundak) Date: Mon, 10 Mar 2014 10:44:32 -0700 Subject: RFR (M): 8036767 PPC64: Support for little endian execution model Message-ID: This patch contains the changes to the build files to build JDK on the little endian PowerPC64 running Linux, and the changes to the source files to support little endian mode. The preceding related change: https://bugs.openjdk.java.net/browse/JDK-8036767 added support for the ELFv2 ABi used on the little endian PowerPC64. Please review and test this change. I need a sponsor. The patch is at: http://cr.openjdk.java.net/~martin/asmundak/8036767/webrev.00/ Sasha From volker.simonis at gmail.com Mon Mar 10 17:58:17 2014 From: volker.simonis at gmail.com (Volker Simonis) Date: Mon, 10 Mar 2014 18:58:17 +0100 Subject: RFR (M): 8036767 PPC64: Support for little endian execution model In-Reply-To: References: Message-ID: The first review happened on the ppc-aix-port list (see this thread http://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/2014-March/001775.html) The previous, related change mentioned by Sasha, which added support for the ELFv2 ABi was https://bugs.openjdk.java.net/browse/JDK-8035647 It was already pushed to jdk9/hs-comp @Sasha: you can remove the "LFLAGS_VM" settings from ppc64le.make Regards, Volker On Mon, Mar 10, 2014 at 6:44 PM, Alexander Smundak wrote: > This patch contains the changes to the build files to build JDK on the > little endian PowerPC64 running Linux, and the changes to the source > files to support little endian mode. > The preceding related change: > https://bugs.openjdk.java.net/browse/JDK-8036767 > added support for the ELFv2 ABi used on the little endian PowerPC64. > > Please review and test this change. I need a sponsor. > > The patch is at: > http://cr.openjdk.java.net/~martin/asmundak/8036767/webrev.00/ > > Sasha From christian.thalinger at oracle.com Mon Mar 10 18:18:19 2014 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Mon, 10 Mar 2014 11:18:19 -0700 Subject: RFR(L): 8029075 - String deduplication in G1 In-Reply-To: <531DB1BD.4050404@oracle.com> References: <53148494.60806@oracle.com> <3A26A868-51A8-4169-B147-9AEB8B35EC5E@oracle.com> <53159E0A.60403@oracle.com> <5A84BB3B-9763-47C3-9EEF-0D07EB763576@oracle.com> <5316F4D2.7010600@oracle.com> <7AD2B58F-1268-4A9C-9749-C6C7DAF5AB25@oracle.com> <531DB1BD.4050404@oracle.com> Message-ID: <545F2E7F-AF3D-4646-86EC-7EB2EFD0F427@oracle.com> On Mar 10, 2014, at 5:36 AM, Per Liden wrote: > Hi John & Christian, > > On 03/07/2014 01:40 AM, John Rose wrote: >> On Mar 6, 2014, at 1:04 PM, Christian Thalinger >> > >> wrote: >> >>> Another way would be to use some kind of annotation to mark references >>> and teach the GCs about that. >> >> Here's how the story looks to me: >> >> Define a new annotation @WeakReferenceField. >> >> Take Alexey's code for @Contended and duplicate with adjustments for >> @WeakReferenceField. Define another field bit and store it in metadata. >> >> Looks for the bit in the JVM wherever it currently asks about >> java_lang_ref_Reference::referent_offset. >> (This is the hard part, especially for G1, but it is traded against >> adding more GC-sensitive C++ code into the JVM, for a benefit of narrow >> scope.) >> > > Interesting idea. And the equivalent of the Reference.discovered field would be injected after each weak-annotated field? > >> Don't bother about queueing; it's optional for WeakReference. > > In cases where we keep lots of weak references in things like hashmaps I can easily imagine that we'd want some kind of notification when references become stale so that the map can be pruned. Injecting another field after the weak-field to keep track of the queue would of course be an option in that case. > >> >> The goal is that the following classes work about the same, except for >> heap pressure: >> >>> class A1 { >>> private WeakReference ref1 = new WeakReference<>(something); >>> private WeakReference ref2 = new WeakReference<>(somethingElse); >>> } >>> class A2 { >>> private @WeakReferenceField String ref1 = something; >>> private @WeakReferenceField String ref2 = somethingElse; >>> } >> >> Put the annotation in sun.misc or some other internal place. Use it >> only for private JVM-coupled things like string dedup (no JCK or general >> testing). >> >> Put whatever practical limits you want on the implementation. Let the >> next user (there will be a next user) refine it further. > > Even if the memory overhead isn't completely removed here it's probably starting to approach a level that's good enough even for heavy users or weak refs. > >> >> Enjoy coding and debugging multi-thread GC-safe code in Java. >> >> I hope this helps! > > The other part of the story, orthogonal to the memory overhead, is making reference processing more efficient once the referent becomes unreachable. I've seen many cases where heavy use of weak references (registered or not) overloads the reference handler thread. Interesting. Is that because it?s just one thread? How about using something like ThreadPoolExecutor for the reference handler thread(s)? > To improve that situation, I'd like to see the reference handler thread removed completely, and instead let the GC enqueue reference directly onto the reference queues. > > /Per > >> >> ? John >> >> P.S. Connection with value types: Eventually, we could add value types >> like java.lang.ref.WeakReferenceValue which would provide a standard >> form for this. Or we could standardize the annotation, perhaps. >> >> P.P.S. For a little more on value types, see >> https://blogs.oracle.com/jrose/entry/value_types_and_struct_tearing From asmundak at google.com Fri Mar 7 03:37:44 2014 From: asmundak at google.com (Alexander Smundak) Date: Thu, 6 Mar 2014 19:37:44 -0800 Subject: RFR (M): 8036767 PPC64: Support for little endian execution model Message-ID: This patch contains the changes to the build files to build JDK on the little endian PowerPC64 running Linux, and the changes to the source files to support little endian mode. This preceding related change https://bugs.openjdk.java.net/browse/JDK-8036767 added support for the ELFv2 ABi used on the little endian PowerPC64. Please review and test this change. I need a sponsor. The patch is at: http://cr.openjdk.java.net/~martin/asmundak/8036767/webrev.00/ Sasha From vladimir.kozlov at oracle.com Mon Mar 10 22:49:44 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 10 Mar 2014 15:49:44 -0700 Subject: RFR (M): 8036767 PPC64: Support for little endian execution model In-Reply-To: References: Message-ID: <531E4188.60906@oracle.com> Changes looks fine to me. They passed JPRT testing which means they do not affect our closed code. I will wait LFLAGS_VM change and may be someone else have comments. I will push it after that tomorrow. Thanks, Vladimir On 3/10/14 10:58 AM, Volker Simonis wrote: > The first review happened on the ppc-aix-port list (see this thread > http://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/2014-March/001775.html) > > The previous, related change mentioned by Sasha, which added support > for the ELFv2 ABi was https://bugs.openjdk.java.net/browse/JDK-8035647 > It was already pushed to jdk9/hs-comp > > @Sasha: you can remove the "LFLAGS_VM" settings from ppc64le.make > > Regards, > Volker > > On Mon, Mar 10, 2014 at 6:44 PM, Alexander Smundak wrote: >> This patch contains the changes to the build files to build JDK on the >> little endian PowerPC64 running Linux, and the changes to the source >> files to support little endian mode. >> The preceding related change: >> https://bugs.openjdk.java.net/browse/JDK-8036767 >> added support for the ELFv2 ABi used on the little endian PowerPC64. >> >> Please review and test this change. I need a sponsor. >> >> The patch is at: >> http://cr.openjdk.java.net/~martin/asmundak/8036767/webrev.00/ >> >> Sasha From asmundak at google.com Tue Mar 11 00:33:14 2014 From: asmundak at google.com (Alexander Smundak) Date: Mon, 10 Mar 2014 17:33:14 -0700 Subject: RFR (M): 8036767 PPC64: Support for little endian execution model In-Reply-To: References: Message-ID: On Mon, Mar 10, 2014 at 10:58 AM, Volker Simonis wrote: > @Sasha: you can remove the "LFLAGS_VM" settings from ppc64le.make Done. I have also removed -D_LP64=1, it is enabled by default in the preprocessor: ~ gcc -E -Wp,-dM - https://bugs.openjdk.java.net/browse/JDK-8037043 http://cr.openjdk.java.net/~twisti/8037043 8037043: put Method flag bits in predictable positions Reviewed-by: Currently Method flag bits are managed as a bit field which is C++ implementation specific. Sometimes we need to know the exact position of flag bits in its enclosing data type. For this reason other flag bits in HotSpot are implemented with enum shift values so the exact bit position can be easily identified. From vladimir.x.ivanov at oracle.com Tue Mar 11 00:49:02 2014 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 11 Mar 2014 04:49:02 +0400 Subject: RFR (XS): 8037043: put Method flag bits in predictable positions In-Reply-To: References: Message-ID: <531E5D7E.3020805@oracle.com> Looks good (not a Reviewer). Best regards, Vladimir Ivanov On 3/11/14 4:44 AM, Christian Thalinger wrote: > https://bugs.openjdk.java.net/browse/JDK-8037043 > http://cr.openjdk.java.net/~twisti/8037043 > > 8037043: put Method flag bits in predictable positions > Reviewed-by: > > Currently Method flag bits are managed as a bit field which is C++ implementation specific. Sometimes we need to know the exact position of flag bits in its enclosing data type. For this reason other flag bits in HotSpot are implemented with enum shift values so the exact bit position can be easily identified. > From vladimir.kozlov at oracle.com Tue Mar 11 00:59:58 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 10 Mar 2014 17:59:58 -0700 Subject: RFR (XS): 8037043: put Method flag bits in predictable positions In-Reply-To: References: Message-ID: <531E600E.1090707@oracle.com> Looks fine. Vladimir On 3/10/14 5:44 PM, Christian Thalinger wrote: > https://bugs.openjdk.java.net/browse/JDK-8037043 > http://cr.openjdk.java.net/~twisti/8037043 > > 8037043: put Method flag bits in predictable positions > Reviewed-by: > > Currently Method flag bits are managed as a bit field which is C++ implementation specific. Sometimes we need to know the exact position of flag bits in its enclosing data type. For this reason other flag bits in HotSpot are implemented with enum shift values so the exact bit position can be easily identified. > From david.holmes at oracle.com Tue Mar 11 03:29:13 2014 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Mar 2014 13:29:13 +1000 Subject: RFR (M): 8036767 PPC64: Support for little endian execution model In-Reply-To: References: Message-ID: <531E8309.2060206@oracle.com> I don't see anything that actually controls what is built. Is it the case that the new arch value will be reported by uname on the ppc64le system? (I cringe seeing a new top-level arch introduced for this.) Thanks, David On 7/03/2014 1:37 PM, Alexander Smundak wrote: > This patch contains the changes to the build files to build JDK on the > little endian PowerPC64 running Linux, and the changes to the source > files to support little endian mode. > This preceding related change > https://bugs.openjdk.java.net/browse/JDK-8036767 added support for the > ELFv2 ABi used on the little endian PowerPC64. > > Please review and test this change. I need a sponsor. > > The patch is at: > http://cr.openjdk.java.net/~martin/asmundak/8036767/webrev.00/ > > Sasha > From david.holmes at oracle.com Tue Mar 11 03:44:01 2014 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Mar 2014 13:44:01 +1000 Subject: RFR(L): 8029075 - String deduplication in G1 In-Reply-To: <545F2E7F-AF3D-4646-86EC-7EB2EFD0F427@oracle.com> References: <53148494.60806@oracle.com> <3A26A868-51A8-4169-B147-9AEB8B35EC5E@oracle.com> <53159E0A.60403@oracle.com> <5A84BB3B-9763-47C3-9EEF-0D07EB763576@oracle.com> <5316F4D2.7010600@oracle.com> <7AD2B58F-1268-4A9C-9749-C6C7DAF5AB25@oracle.com> <531DB1BD.4050404@oracle.com> <545F2E7F-AF3D-4646-86EC-7EB2EFD0F427@oracle.com> Message-ID: <531E8681.5030809@oracle.com> On 11/03/2014 4:18 AM, Christian Thalinger wrote: > > On Mar 10, 2014, at 5:36 AM, Per Liden wrote: >> The other part of the story, orthogonal to the memory overhead, is making reference processing more efficient once the referent becomes unreachable. I've seen many cases where heavy use of weak references (registered or not) overloads the reference handler thread. > > Interesting. Is that because it?s just one thread? How about using something like ThreadPoolExecutor for the reference handler thread(s)? The synchronization that would be needed across multiple reference-handler threads might negate any benefit from having more than one. And a full blown ThreadPoolExecutor may be overkill here. >> To improve that situation, I'd like to see the reference handler thread removed completely, and instead let the GC enqueue reference directly onto the reference queues. That would require that the GC threads be able to execute Java code. It would also mean that a GC thread would experience synchronization (and possible contention) with Java threads. David ----- >> >> /Per >> >>> >>> ? John >>> >>> P.S. Connection with value types: Eventually, we could add value types >>> like java.lang.ref.WeakReferenceValue which would provide a standard >>> form for this. Or we could standardize the annotation, perhaps. >>> >>> P.P.S. For a little more on value types, see >>> https://blogs.oracle.com/jrose/entry/value_types_and_struct_tearing > From christian.thalinger at oracle.com Tue Mar 11 03:47:15 2014 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Mon, 10 Mar 2014 20:47:15 -0700 Subject: RFR(L): 8029075 - String deduplication in G1 In-Reply-To: <531E8681.5030809@oracle.com> References: <53148494.60806@oracle.com> <3A26A868-51A8-4169-B147-9AEB8B35EC5E@oracle.com> <53159E0A.60403@oracle.com> <5A84BB3B-9763-47C3-9EEF-0D07EB763576@oracle.com> <5316F4D2.7010600@oracle.com> <7AD2B58F-1268-4A9C-9749-C6C7DAF5AB25@oracle.com> <531DB1BD.4050404@oracle.com> <545F2E7F-AF3D-4646-86EC-7EB2EFD0F427@oracle.com> <531E8681.5030809@oracle.com> Message-ID: <6A0DA260-FB05-4BE8-A4D8-C4D69E90AE43@oracle.com> On Mar 10, 2014, at 8:44 PM, David Holmes wrote: > > > On 11/03/2014 4:18 AM, Christian Thalinger wrote: >> >> On Mar 10, 2014, at 5:36 AM, Per Liden wrote: >>> The other part of the story, orthogonal to the memory overhead, is making reference processing more efficient once the referent becomes unreachable. I've seen many cases where heavy use of weak references (registered or not) overloads the reference handler thread. >> >> Interesting. Is that because it?s just one thread? How about using something like ThreadPoolExecutor for the reference handler thread(s)? > > The synchronization that would be needed across multiple reference-handler threads might negate any benefit from having more than one. And a full blown ThreadPoolExecutor may be overkill here. That?s my feeling too but it would be an interesting experiment. > >>> To improve that situation, I'd like to see the reference handler thread removed completely, and instead let the GC enqueue reference directly onto the reference queues. > > That would require that the GC threads be able to execute Java code. It would also mean that a GC thread would experience synchronization (and possible contention) with Java threads. > > David > ----- > >>> >>> /Per >>> >>>> >>>> ? John >>>> >>>> P.S. Connection with value types: Eventually, we could add value types >>>> like java.lang.ref.WeakReferenceValue which would provide a standard >>>> form for this. Or we could standardize the annotation, perhaps. >>>> >>>> P.P.S. For a little more on value types, see >>>> https://blogs.oracle.com/jrose/entry/value_types_and_struct_tearing >> From volker.simonis at gmail.com Tue Mar 11 08:36:02 2014 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 11 Mar 2014 09:36:02 +0100 Subject: RFR (M): 8036767 PPC64: Support for little endian execution model In-Reply-To: <531E8309.2060206@oracle.com> References: <531E8309.2060206@oracle.com> Message-ID: On Tue, Mar 11, 2014 at 4:29 AM, David Holmes wrote: > I don't see anything that actually controls what is built. Is it the case > that the new arch value will be reported by uname on the ppc64le system? > > (I cringe seeing a new top-level arch introduced for this.) > I didn't liked this too much as well in my first review and I think we can get along without a new top-level arch (see http://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/2014-March/001790.html). But in the end I can live with both approaches (and having two top-level archs probably makes it slightly simpler to support them both in parallel) so I think finally it is up to you which approach we take. Regards, Volker > Thanks, > David > > > On 7/03/2014 1:37 PM, Alexander Smundak wrote: >> >> This patch contains the changes to the build files to build JDK on the >> little endian PowerPC64 running Linux, and the changes to the source >> files to support little endian mode. >> This preceding related change >> https://bugs.openjdk.java.net/browse/JDK-8036767 added support for the >> ELFv2 ABi used on the little endian PowerPC64. >> >> Please review and test this change. I need a sponsor. >> >> The patch is at: >> http://cr.openjdk.java.net/~martin/asmundak/8036767/webrev.00/ >> >> Sasha >> > From david.holmes at oracle.com Tue Mar 11 08:59:27 2014 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Mar 2014 18:59:27 +1000 Subject: RFR (M): 8036767 PPC64: Support for little endian execution model In-Reply-To: References: <531E8309.2060206@oracle.com> Message-ID: <531ED06F.5040905@oracle.com> On 11/03/2014 6:36 PM, Volker Simonis wrote: > On Tue, Mar 11, 2014 at 4:29 AM, David Holmes wrote: >> I don't see anything that actually controls what is built. Is it the case >> that the new arch value will be reported by uname on the ppc64le system? >> >> (I cringe seeing a new top-level arch introduced for this.) >> > > I didn't liked this too much as well in my first review and I think we > can get along without a new top-level arch (see > http://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/2014-March/001790.html). > > But in the end I can live with both approaches (and having two > top-level archs probably makes it slightly simpler to support them > both in parallel) so I think finally it is up to you which approach we > take. I agree with your initial review. I don't see a LE build as a distinct architecture but a variant of ppc64. Lets see what anyone else has to say. Thanks, David > Regards, > Volker > >> Thanks, >> David >> >> >> On 7/03/2014 1:37 PM, Alexander Smundak wrote: >>> >>> This patch contains the changes to the build files to build JDK on the >>> little endian PowerPC64 running Linux, and the changes to the source >>> files to support little endian mode. >>> This preceding related change >>> https://bugs.openjdk.java.net/browse/JDK-8036767 added support for the >>> ELFv2 ABi used on the little endian PowerPC64. >>> >>> Please review and test this change. I need a sponsor. >>> >>> The patch is at: >>> http://cr.openjdk.java.net/~martin/asmundak/8036767/webrev.00/ >>> >>> Sasha >>> >> From goetz.lindenmaier at sap.com Tue Mar 11 09:13:04 2014 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 11 Mar 2014 09:13:04 +0000 Subject: RFR(L): 8036976: PPC64: implement the template interpreter Message-ID: <4295855A5C1DE049A61835A1887419CC2CEA69D7@DEWDFEMB12A.global.corp.sap> Hi, This time I have a really nice one: the template interpreter for ppc64! Great thanks to my colleagues, especially Axel and Martin, who ported this. http://cr.openjdk.java.net/~goetz/webrevs/8036976-0-ti/ With this change, both interpreters can be used on linux_ppc64 and aix_ppc64. This change almost only touches ppc platform files. I shared files, I needed to add a special case for PPC64 in make/linux/Makefile, as I found no way to set FORCE_TIERED=0 in any ppc file that's included above the use of FORCE_TIERED. As we don't have C1, the current scheme does not work for ppc64. Actually, one should have lines as supports_cppInterpreter = 1 supports_c2 = 1 supports_c1 = 0 in platform_. But that's out of scope of this change. Further I add a #include in src/share/vm/interpreter/templateTable.hpp. That's all for shared changes. Please review and test this change. Best regards, Goetz. From goetz.lindenmaier at sap.com Tue Mar 11 09:34:23 2014 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 11 Mar 2014 09:34:23 +0000 Subject: RFR (M): 8036767 PPC64: Support for little endian execution model In-Reply-To: <531ED06F.5040905@oracle.com> References: <531E8309.2060206@oracle.com> <531ED06F.5040905@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CEA6A09@DEWDFEMB12A.global.corp.sap> Hi, basically this comes down to the setting of BUILDARCH in the defs.make file. I think it is useful the le-port is built into a directory linux_ppc64le_server (Infix le!!). It?s the purpose of this path naming to keep incompatible builds apart. Unfortunately BUILDARCH is also used to find the submakefiles, so these two extra ones are needed. But the ppc64le port needs different compiler and linker settings, so there is a good reason to have an extra ppc64le.make. Only the platform_ppc64le file is completely redundant, but that's acceptable I would say. David, this is the complete change to hostpot for this port - there will not be new arch subdirectories in the C-code. Please watch out, there are two threads of this because of the mail server problems on the weekend. Best regards, Goetz. -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of David Holmes Sent: Dienstag, 11. M?rz 2014 09:59 To: Volker Simonis Cc: HotSpot Open Source Developers; Alexander Smundak Subject: Re: RFR (M): 8036767 PPC64: Support for little endian execution model On 11/03/2014 6:36 PM, Volker Simonis wrote: > On Tue, Mar 11, 2014 at 4:29 AM, David Holmes wrote: >> I don't see anything that actually controls what is built. Is it the case >> that the new arch value will be reported by uname on the ppc64le system? >> >> (I cringe seeing a new top-level arch introduced for this.) >> > > I didn't liked this too much as well in my first review and I think we > can get along without a new top-level arch (see > http://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/2014-March/001790.html). > > But in the end I can live with both approaches (and having two > top-level archs probably makes it slightly simpler to support them > both in parallel) so I think finally it is up to you which approach we > take. I agree with your initial review. I don't see a LE build as a distinct architecture but a variant of ppc64. Lets see what anyone else has to say. Thanks, David > Regards, > Volker > >> Thanks, >> David >> >> >> On 7/03/2014 1:37 PM, Alexander Smundak wrote: >>> >>> This patch contains the changes to the build files to build JDK on the >>> little endian PowerPC64 running Linux, and the changes to the source >>> files to support little endian mode. >>> This preceding related change >>> https://bugs.openjdk.java.net/browse/JDK-8036767 added support for the >>> ELFv2 ABi used on the little endian PowerPC64. >>> >>> Please review and test this change. I need a sponsor. >>> >>> The patch is at: >>> http://cr.openjdk.java.net/~martin/asmundak/8036767/webrev.00/ >>> >>> Sasha >>> >> From bengt.rutisson at oracle.com Tue Mar 11 11:04:28 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Tue, 11 Mar 2014 12:04:28 +0100 Subject: RFR(L): 8029075 - String deduplication in G1 In-Reply-To: <5319D424.90306@oracle.com> References: <53148494.60806@oracle.com> <5316F5B1.60006@oracle.com> <5319D424.90306@oracle.com> Message-ID: <531EEDBC.9070500@oracle.com> Hi Per, On 2014-03-07 15:13, Per Liden wrote: > Hi, > > Here's an updated webrev based on the feedback from Bengt and Thomas. > > http://cr.openjdk.java.net/~pliden/8029075/webrev.1/ > > Diffs against previous webrev available here: > http://cr.openjdk.java.net/~pliden/8029075/webrev.1-bengt_review_fixes/ > http://cr.openjdk.java.net/~pliden/8029075/webrev.1-thomas_review_fixes/ Thanks for providing these diffs. Made it easy to review. Looks good to me. > > And as suggested, two changes in the initial webrev were broken out > into separate CRs, available here: Great! Thanks for splitting these out. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8036672 > Webrev: http://cr.openjdk.java.net/~pliden/8036672/webrev.0/ Looks good. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8036673 > Webrev: http://cr.openjdk.java.net/~pliden/8036673/webrev.0/ Looks good. Bengt > > /Per > > On 03/05/2014 11:00 AM, Bengt Rutisson wrote: >> >> Hi Per and everyone else, >> >> Overall this looks good to me. Me and Per went through this patch >> together and here are the notes from this code walk through. >> >> There are two small bug fixes included in the change. It would be good >> to separate these out to separate changes. We can review them together >> with the main change, but having them as separate changesets will make >> it easier to backport them if necessary. >> >> >> stringDedup.hpp >> The comments are not that helpful to me. I would suggest to either >> remove them or make more descriptive. >> >> UseG1GC && UseStringDeduplication >> Only allow UseStringDeduplication to be true if UseG1GC is true instead >> of checking both? >> >> G1CollectedHeap::tear_down_region_sets() and >> G1CollectedHeap::rebuild_region_sets() >> Please add an RFE to clean these up as the naming and implementations >> don't match anymore. >> >> StringDedupTable::trim() >> rename to trim_cache? >> >> stringDedup.cpp >> Two occurrences of: >> assert(UseG1GC, "String deduplication not available with G1"); >> The "not" should be a "only". >> >> Dummy statistics from interned stings is ignored. Please add an RFE to >> try to include this statistics and possibly reduce the "known" count. >> >> >> Add comments explaining why resize and rehash is not allowed for these >> code paths: >> >> void StringDedup::unlink(BoolObjectClosure* is_alive) { >> unlink_or_oops_do(is_alive, NULL, false /* allow_resize_and_rehash >> */); >> } >> >> void StringDedup::oops_do(OopClosure* keep_alive) { >> unlink_or_oops_do(NULL, keep_alive, false /* allow_resize_and_rehash >> */); >> } >> >> TestStringDeduplicationTools.java >> verifyStrings put the sleep at the end to possibly speed up the test a >> little. >> >> InternedTest >> Comments to explain what it does. >> >> testYoungGC - check that no full GCs happen >> testFullGC - check that no young GCs happen >> >> testMemoryUsage - add "memoryUsageWithDedup > >> memoryUsageWithDedupExpected" to the exception message >> >> >> Thanks, >> Bengt >> >> >> On 3/3/14 2:33 PM, Per Liden wrote: >>> Hi, >>> >>> Could I please have this patch reviewed. >>> >>> >>> Summary >>> ------- >>> This patch implements JEP 192 - String deduplication in G1. The goal >>> of string deduplication is to reduce the Java heap live-data set by >>> enhancing the G1 garbage collector so that duplicate instances of >>> String are automatically and continuously deduplicated. >>> >>> I'd like to refer to the JEP for a more detailed description of this >>> feature, the motivation for it, the expected benefit, how it's >>> implemented, etc. >>> >>> This patch is mainly G1-related, but also touches a few runtime files. >>> >>> Webrev: http://cr.openjdk.java.net/~pliden/8029075/webrev.0/ >>> >>> JEP: http://openjdk.java.net/jeps/192 >>> >>> RFE: https://bugs.openjdk.java.net/browse/JDK-8029075 >>> >>> >>> Testing >>> ------- >>> * JTreg - The patch includes the following 8 new tests: >>> - TestStringDeduplicationYoungGC.java: Tests deduplication during >>> Young GC. >>> - TestStringDeduplicationFullGC.java: Tests deduplication during >>> Full GC. >>> - TestStringDeduplicationAgeThreshold.java: Tests both valid and >>> invalid age threshold settings. >>> - TestStringDeduplicationInterned.java: Tests that interned strings >>> are deduplication before being interned. >>> - TestStringDeduplicationTableRehash.java: Stresses the hashtables >>> ability to rehash all entries. >>> - TestStringDeduplicationTableResize.java: Stresses the hashtables >>> ability to resize itself. >>> - TestStringDeduplicationMemoryUsage.java: Tests heap reduction when >>> string deduplication is enabled. >>> - TestStringDeduplicationPrintOptions.java: Tests command line >>> options. >>> >>> * Stress testing: >>> - Kitchensink >>> - GCBasher >>> >>> * Regression testing: >>> - JCK >>> - vmTestbase >>> - Bigapps >>> >>> * Large scale benchmarks to test heap reduction and performance impact: >>> - FA CRM Sales Op. Flow >>> - FA DSS >>> >>> * The following benchmarks have been executed to verify that this >>> feature doesn't impact performance when disabled (even when disabled >>> there are still some "if (UseStringDeduplication)" executed in some >>> hot paths). >>> - SPECjbb2005 >>> - SPECjbb2013 >>> - SPECjvm2008-XML >>> >>> * Various ad-hoc tests and microbenchmarks were also written and >>> executed during the course of the development. >>> >>> >>> Thanks! >>> /Per >> > From per.liden at oracle.com Tue Mar 11 11:24:22 2014 From: per.liden at oracle.com (Per Liden) Date: Tue, 11 Mar 2014 12:24:22 +0100 Subject: RFR(L): 8029075 - String deduplication in G1 In-Reply-To: <531E8681.5030809@oracle.com> References: <53148494.60806@oracle.com> <3A26A868-51A8-4169-B147-9AEB8B35EC5E@oracle.com> <53159E0A.60403@oracle.com> <5A84BB3B-9763-47C3-9EEF-0D07EB763576@oracle.com> <5316F4D2.7010600@oracle.com> <7AD2B58F-1268-4A9C-9749-C6C7DAF5AB25@oracle.com> <531DB1BD.4050404@oracle.com> <545F2E7F-AF3D-4646-86EC-7EB2EFD0F427@oracle.com> <531E8681.5030809@oracle.com> Message-ID: <531EF266.3050403@oracle.com> On 2014-03-11 04:44, David Holmes wrote: > > > On 11/03/2014 4:18 AM, Christian Thalinger wrote: >> >> On Mar 10, 2014, at 5:36 AM, Per Liden wrote: >>> The other part of the story, orthogonal to the memory overhead, is >>> making reference processing more efficient once the referent becomes >>> unreachable. I've seen many cases where heavy use of weak references >>> (registered or not) overloads the reference handler thread. >> >> Interesting. Is that because it?s just one thread? How about using >> something like ThreadPoolExecutor for the reference handler thread(s)? > > The synchronization that would be needed across multiple > reference-handler threads might negate any benefit from having more > than one. And a full blown ThreadPoolExecutor may be overkill here. Gut feeling also tells me that the synchronization would kill it, but might be worth a try. > >>> To improve that situation, I'd like to see the reference handler >>> thread removed completely, and instead let the GC enqueue reference >>> directly onto the reference queues. > > That would require that the GC threads be able to execute Java code. > It would also mean that a GC thread would experience synchronization > (and possible contention) with Java threads. I was more like thinking of modifying the underlying queue in ReferenceQueue so that the GC could enqueue entries from a native context (somewhat similar to how the pending list works today). Admittedly, I haven't thought about all the details here, but I'm thinking that should be doable with a CAS in the fast path for both enqueue and dequeue, and maybe allow a down call from ReferenceQueue.remove() in the slow path in case the queue is empty and the caller needs to block (say, on a ParkEvent maybe). Does that sound completely unreasonable? /Per > > David > ----- > >>> >>> /Per >>> >>>> >>>> ? John >>>> >>>> P.S. Connection with value types: Eventually, we could add value >>>> types >>>> like java.lang.ref.WeakReferenceValue which would provide a standard >>>> form for this. Or we could standardize the annotation, perhaps. >>>> >>>> P.P.S. For a little more on value types, see >>>> https://blogs.oracle.com/jrose/entry/value_types_and_struct_tearing >> From david.holmes at oracle.com Tue Mar 11 12:04:27 2014 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Mar 2014 22:04:27 +1000 Subject: RFR(L): 8029075 - String deduplication in G1 In-Reply-To: <531EF266.3050403@oracle.com> References: <53148494.60806@oracle.com> <3A26A868-51A8-4169-B147-9AEB8B35EC5E@oracle.com> <53159E0A.60403@oracle.com> <5A84BB3B-9763-47C3-9EEF-0D07EB763576@oracle.com> <5316F4D2.7010600@oracle.com> <7AD2B58F-1268-4A9C-9749-C6C7DAF5AB25@oracle.com> <531DB1BD.4050404@oracle.com> <545F2E7F-AF3D-4646-86EC-7EB2EFD0F427@oracle.com> <531E8681.5030809@oracle.com> <531EF266.3050403@oracle.com> Message-ID: <531EFBCB.6090204@oracle.com> On 11/03/2014 9:24 PM, Per Liden wrote: > On 2014-03-11 04:44, David Holmes wrote: >> >> >> On 11/03/2014 4:18 AM, Christian Thalinger wrote: >>> >>> On Mar 10, 2014, at 5:36 AM, Per Liden wrote: >>>> The other part of the story, orthogonal to the memory overhead, is >>>> making reference processing more efficient once the referent becomes >>>> unreachable. I've seen many cases where heavy use of weak references >>>> (registered or not) overloads the reference handler thread. >>> >>> Interesting. Is that because it?s just one thread? How about using >>> something like ThreadPoolExecutor for the reference handler thread(s)? >> >> The synchronization that would be needed across multiple >> reference-handler threads might negate any benefit from having more >> than one. And a full blown ThreadPoolExecutor may be overkill here. > > Gut feeling also tells me that the synchronization would kill it, but > might be worth a try. > >> >>>> To improve that situation, I'd like to see the reference handler >>>> thread removed completely, and instead let the GC enqueue reference >>>> directly onto the reference queues. >> >> That would require that the GC threads be able to execute Java code. >> It would also mean that a GC thread would experience synchronization >> (and possible contention) with Java threads. > > I was more like thinking of modifying the underlying queue in > ReferenceQueue so that the GC could enqueue entries from a native > context (somewhat similar to how the pending list works today). > Admittedly, I haven't thought about all the details here, but I'm > thinking that should be doable with a CAS in the fast path for both > enqueue and dequeue, and maybe allow a down call from > ReferenceQueue.remove() in the slow path in case the queue is empty and > the caller needs to block (say, on a ParkEvent maybe). Does that sound > completely unreasonable? The ReferenceHandler thread does more than just enqueue the references, and enqueuing requires waking up blocked Java threads. Simple atomic ops to add elements to the queue would not in themselves be sufficient. The details need to be fleshed out here. David > /Per > >> >> David >> ----- >> >>>> >>>> /Per >>>> >>>>> >>>>> ? John >>>>> >>>>> P.S. Connection with value types: Eventually, we could add value >>>>> types >>>>> like java.lang.ref.WeakReferenceValue which would provide a standard >>>>> form for this. Or we could standardize the annotation, perhaps. >>>>> >>>>> P.P.S. For a little more on value types, see >>>>> https://blogs.oracle.com/jrose/entry/value_types_and_struct_tearing >>> > From thomas.schatzl at oracle.com Tue Mar 11 12:39:51 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 11 Mar 2014 13:39:51 +0100 Subject: RFR(L): 8029075 - String deduplication in G1 In-Reply-To: <5319D424.90306@oracle.com> References: <53148494.60806@oracle.com> <5316F5B1.60006@oracle.com> <5319D424.90306@oracle.com> Message-ID: <1394541591.2696.20.camel@cirrus> Hi Per, On Fri, 2014-03-07 at 15:13 +0100, Per Liden wrote: > Hi, > > Here's an updated webrev based on the feedback from Bengt and Thomas. > > http://cr.openjdk.java.net/~pliden/8029075/webrev.1/ > See further below. > > And as suggested, two changes in the initial webrev were broken out into > separate CRs, available here: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8036672 > Webrev: http://cr.openjdk.java.net/~pliden/8036672/webrev.0/ Looks good. Did you check the impact on something like specjbb2005 which does nothing but object copying? That code is somewhat performance sensitive. > Bug: https://bugs.openjdk.java.net/browse/JDK-8036673 > Webrev: http://cr.openjdk.java.net/~pliden/8036673/webrev.0/ Looks good. Here are my comments: there are a few minor issues, and a single major: basically the documentation for this feature is lacking imo. I listed a few particular issues of mine with it. First the minor issues: - could you check on copyright dates on the modified files? They have not been updated anywhere afaics. - not sure if good, but maybe make PrintStringDeduplicationStatistics diagnostic? It seems to really be some diagnosis functions. - maybe make the new *ALot flags notproduct. These seem to be debugging helpers similar to other *ALot flags. - StringDeduplicationAgeThreshold: maybe also needs to mention that if the object is promoted, that it is always considered as candidate. marksweep.inline.hpp - MarkSweep::mark_object(): did you measure impact on other collectors using this code? It is enabled unconditionally. Also I would prefer that !UseG1GC automatically disables UseStringDeduplication (or use a global variable for that) to avoid the two checks. (The latter option is probably better). symbolTable.cpp: - again both UseG1GC and UseStringDedupliation are paired. Could this be put in a method/global variable somewhere that is set up correctly at startup? g1CollectedHeap.cpp: - StringDedup::initialize() is run unconditionally (the check for UseStringDeduplication is inside the method) while it's outside for everything else. Just a note, keep it if wanted. TestStringDeduplicationTools.java: - maybe put retrieval of the unsafe object into the test library. Browsing through the code shows quite a lot copy+paste here already. - "qeueue" -> "queue" all tests: - missing bug ids >From here it is mostly about comments/documentation: stringdedup.?pp: - is it possible to put an overview about how the string deduplication works here? There is absolutely no place in the code where e.g. the string deduplication cycle is explained. There is no comprehensive information about this feature anywhere, just bits about corner cases or particular here and there, but nothing that can be used as entry point to understand what this is actually doing. To some extent, the comments seem to be repetitions of what the code already shows. E.g. "some_constant = (1 << 10); // 1024 (must be power of 2)" or "// Store hash code in cache" followed by "java_lang_String::set_hash(java_string, hash);". There is also no reference to the JEP (I am not sure if it is customary to put references to outside documents - I would prefer some); however the JEP does not replace documentation in the source as it is not detailed enough, and the JEP cannot be changed any more. The JEP is interesting for the alternatives considered though. Ideally, to reasonably understand the feature I would like to not need to spend hours looking through the implementation. - how many threads does this feature add, and are supported by this code (for which parts)? Synchronization between them? - the comments "For XY" in the StringDedup header file do not give any hint about what they are doing (not sure if it is interesting as it is easy to find out the only callers). It seems that this information has been put at the place where these methods are invoked (or just somewhere in the cpp file). Imo the methods and members (except trivial) should be documented in the hpp file in the class body, i.e. in the interface. - there are a few methods (also in the StringDedupTable class) in StringDedup where the results of the methods do not seem obvious and should be commented. - I really dislike the location of the "Candidate selection" explanation within the cpp file. I would not have found it if I were not looking for some documentation ;) It would probably fit very well in the above mentioned overview paragraphs. - please add the static helper functions that implement the actual policies (is_candidate_from_mark, is_candidate_from_evacuation) to the StringDedup class (as private static methods) so that they can be found easily. Static non-class helper functions in cpp files should only be used for stuff that is really unimportant and not relevant to the code imo. However these seem to be _the_ central policy methods for string dedup. So it would be nice to have them mentioned in the class/hpp file and documented there. - facts about the code are repeated, e.g. that the table size must be a power of two. This is nice, but has a few drawbacks: the compiler does not care about comments at all, so it might be useful to put asserts in these places instead, and second, it's just duplication of text with the usual problem of getting out of date. - I am not completely sure why the class has not been called G1StringDedup. It won't work without G1 anyway and calls G1 internals all the time. There does not seem to be a way to make it more generic without considerable refactoring. It is fine to keep the name though, just asking. I also know that this feature does not want to be a generic solution. - is it possible to separate out the statistics variables of StringDedupTable into either a separate class instead of using globals or separate them a little from the other unrelated members? (And describe them) This would improve readability. I *think* this is everything from StringDedupTable::_entries_added to _rehash_count? I have no idea without searching for every use and deducing from that. - when implementing the hash table for the StringDedupTable, did you consider trying to reuse one of the existing hashmap implementations? And potentially extend them. Imo we already have way too many ad-hoc implementations of often used data structures (particularly) in G1 and GC code. - please document the properties of the StringDedupTable (type, hashing, automatic resizing behavior, thread safety, ...). - please explain what StringDedupTable/Cache are and what they are used for in context of the feature. - stringdedup.cpp, line 1309: candicate -> candidate Thanks, Thomas From magnus.ihse.bursie at oracle.com Tue Mar 11 12:47:15 2014 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Tue, 11 Mar 2014 13:47:15 +0100 Subject: RFR : JDK-8032045 : (m/l) Enable compiler and linker safety switches for debug builds In-Reply-To: References: Message-ID: <531F05D3.6020008@oracle.com> On 2014-03-11 00:49, Mike Duigou wrote: > I have updated the patch to respond to Magnus's feedback and to accommodate intervening changes to the configure and hotspot make files. > > https://bugs.openjdk.java.net/browse/JDK-8032045 > http://cr.openjdk.java.net/~mduigou/JDK-8032045/3/webrev/ > > This version is, hopefully, almost ready to be pushed. I have only glanced at the hotspot build changes and can't really say anything about them. The hotspot team still owns these; I'm cc:ing them now. The top-level build changes looks fine. Thank you Mike for cleaning things up! /Magnus > > Mike > > On Feb 20 2014, at 15:43 , Mike Duigou wrote: > >> Hello all; >> >> This issue is a followon to JDK-8030350 (https://bugs.openjdk.java.net/browse/JDK-8030350) which enhanced the compiler warnings used for compiling native code. The proposed changes principally impact the Linux platform. >> >> While 8030350 was focused on compiler warnings which did not impact code generation, this changeset will, for some configurations, change the native code generated and likely change performance. These proposed option changes prevent specific types of relocation table, stack and heap memory corruption in native code. Preventing these types of memory corruption may be useful for finding certain kinds of bugs though and do provide some minimal additional protections against malicious attacks. They aren't, by any means, a substitute for following appropriate secure coding guidelines. >> >> The rationale behind the implementation is as follows. For release builds during the initial phase of JDK 9 I would like to enable only compile time checks. This ends up being similar to the warnings in JDK-8030350. These options have no runtime impact on footprint or performance and very minimal additional compile time cost while providing value. **Release builds are not expected to see any performance or footprint change as a result of this changeset** >> >> For fast debug builds we can enable linker protections (relro) and static compile time bounds checks (FORTIFY_SOURCE=1). FORTIFY_SOURCE=1 might be moved to the production builds as well because it has no runtime cost or executable size cost. >> >> For slow debug builds we can enable full linker protection (at a potential cost in startup time), runtime bounds checks and stack protection (FORTIFY_SOURCE=2 -fprotect-stack-all). We will likely enable -fprotect-stack-strong when available in GCC 4.9 >> >> The basis for enabling the additional protections in debug builds is that it will help us find bugs in our native code and we aren't as concerned in debug builds with footprint and performance. Since many developers already do their personal builds in fastdebug or slowdebug mode for testing this will provide good opportunity to shake out any problems with the options while not impacting release builds. Should we find that any of the options provide significant value for their cost we can move them to fastdebug or release. If any of the options prove too costly they can be demoted or removed entirely. >> >> https://bugs.openjdk.java.net/browse/JDK-8032045 >> http://cr.openjdk.java.net/~mduigou/JDK-8032045/2/webrev/ >> >> Additional to enabling the various compiler options I attempted to rationalize some of the skew between the various hotspot/make/{platform}/makefiles/gcc.make files while avoiding changing existing behaviour. I have also introduced the new -Og "optimize for debugging" option and there are now an explicit C{XX}_O_FLAG_DEBUG definitions to complement the C{XX}_O_FLAG_{DEBUG|NORM|HI|HIGHEST|NONE} optimization options. >> >> Thanks, >> >> Mike From per.liden at oracle.com Tue Mar 11 13:08:18 2014 From: per.liden at oracle.com (Per Liden) Date: Tue, 11 Mar 2014 14:08:18 +0100 Subject: RFR(L): 8029075 - String deduplication in G1 In-Reply-To: <531EFBCB.6090204@oracle.com> References: <53148494.60806@oracle.com> <3A26A868-51A8-4169-B147-9AEB8B35EC5E@oracle.com> <53159E0A.60403@oracle.com> <5A84BB3B-9763-47C3-9EEF-0D07EB763576@oracle.com> <5316F4D2.7010600@oracle.com> <7AD2B58F-1268-4A9C-9749-C6C7DAF5AB25@oracle.com> <531DB1BD.4050404@oracle.com> <545F2E7F-AF3D-4646-86EC-7EB2EFD0F427@oracle.com> <531E8681.5030809@oracle.com> <531EF266.3050403@oracle.com> <531EFBCB.6090204@oracle.com> Message-ID: <531F0AC2.4020701@oracle.com> Inline below... On 2014-03-11 13:04, David Holmes wrote: > On 11/03/2014 9:24 PM, Per Liden wrote: >> On 2014-03-11 04:44, David Holmes wrote: >>> >>> >>> On 11/03/2014 4:18 AM, Christian Thalinger wrote: >>>> >>>> On Mar 10, 2014, at 5:36 AM, Per Liden wrote: >>>>> The other part of the story, orthogonal to the memory overhead, is >>>>> making reference processing more efficient once the referent becomes >>>>> unreachable. I've seen many cases where heavy use of weak references >>>>> (registered or not) overloads the reference handler thread. >>>> >>>> Interesting. Is that because it?s just one thread? How about using >>>> something like ThreadPoolExecutor for the reference handler thread(s)? >>> >>> The synchronization that would be needed across multiple >>> reference-handler threads might negate any benefit from having more >>> than one. And a full blown ThreadPoolExecutor may be overkill here. >> >> Gut feeling also tells me that the synchronization would kill it, but >> might be worth a try. >> >>> >>>>> To improve that situation, I'd like to see the reference handler >>>>> thread removed completely, and instead let the GC enqueue reference >>>>> directly onto the reference queues. >>> >>> That would require that the GC threads be able to execute Java code. >>> It would also mean that a GC thread would experience synchronization >>> (and possible contention) with Java threads. >> >> I was more like thinking of modifying the underlying queue in >> ReferenceQueue so that the GC could enqueue entries from a native >> context (somewhat similar to how the pending list works today). >> Admittedly, I haven't thought about all the details here, but I'm >> thinking that should be doable with a CAS in the fast path for both >> enqueue and dequeue, and maybe allow a down call from >> ReferenceQueue.remove() in the slow path in case the queue is empty and >> the caller needs to block (say, on a ParkEvent maybe). Does that sound >> completely unreasonable? > > The ReferenceHandler thread does more than just enqueue the references, What else are you thinking of? sun.misc.Cleaner would obviously have to be redesigned or removed, but other than that? > and enqueuing requires waking up blocked Java threads. Simple atomic > ops to add elements to the queue would not in themselves be sufficient. Yes, a CAS would have to be followed by a potential slow path to wakeup any waiter if it's the first element that goes on the queue, etc. > The details need to be fleshed out here. Indeed. /Per > > David > >> /Per >> >>> >>> David >>> ----- >>> >>>>> >>>>> /Per >>>>> >>>>>> >>>>>> ? John >>>>>> >>>>>> P.S. Connection with value types: Eventually, we could add value >>>>>> types >>>>>> like java.lang.ref.WeakReferenceValue which would provide a standard >>>>>> form for this. Or we could standardize the annotation, perhaps. >>>>>> >>>>>> P.P.S. For a little more on value types, see >>>>>> https://blogs.oracle.com/jrose/entry/value_types_and_struct_tearing >>>> >> From coleen.phillimore at oracle.com Tue Mar 11 14:37:40 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 11 Mar 2014 10:37:40 -0400 Subject: RFR (XS): 8037043: put Method flag bits in predictable positions In-Reply-To: References: Message-ID: <531F1FB4.2070507@oracle.com> No wonder you don't like C++ with code like that. Can you put () around the ?: expressions so I don't have to dig out my C++ manual to figure out order of operations in the getting and setting functions. Thanks, Coleen On 3/10/14 8:44 PM, Christian Thalinger wrote: > https://bugs.openjdk.java.net/browse/JDK-8037043 > http://cr.openjdk.java.net/~twisti/8037043 > > 8037043: put Method flag bits in predictable positions > Reviewed-by: > > Currently Method flag bits are managed as a bit field which is C++ implementation specific. Sometimes we need to know the exact position of flag bits in its enclosing data type. For this reason other flag bits in HotSpot are implemented with enum shift values so the exact bit position can be easily identified. > From coleen.phillimore at oracle.com Tue Mar 11 14:42:10 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 11 Mar 2014 10:42:10 -0400 Subject: RFR (XS) 8025550 valgrind: Conditional jump depends on uninitialised value in Arena::set_size_in_bytes() In-Reply-To: References: Message-ID: <531F20C2.1010207@oracle.com> This looks good. Sorry for the delay, especially since the change is simple. Coleen On 2/25/14 12:43 PM, Zhengyu Gu wrote: > This is a simple fix to NMT, which can mis-count arena size due to uninitialized value. In rare case, if uninitialized size happens to be the same value as arena's initial size, NMT will mis-count the size, since NMT only updates size when the new size does not equal to the old one. > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8025550 > Webrev: http://cr.openjdk.java.net/~zgu/8025550/webrev.00/ > > Thanks, > > -Zhengyu From zhengyu.gu at oracle.com Tue Mar 11 14:24:49 2014 From: zhengyu.gu at oracle.com (Zhengyu Gu) Date: Tue, 11 Mar 2014 10:24:49 -0400 Subject: RFR (XS) 8025550 valgrind: Conditional jump depends on uninitialised value in Arena::set_size_in_bytes() In-Reply-To: <531F20C2.1010207@oracle.com> References: <531F20C2.1010207@oracle.com> Message-ID: Thanks for the review. -Zhengyu On Mar 11, 2014, at 10:42 AM, Coleen Phillimore wrote: > > This looks good. Sorry for the delay, especially since the change is simple. > Coleen > > On 2/25/14 12:43 PM, Zhengyu Gu wrote: >> This is a simple fix to NMT, which can mis-count arena size due to uninitialized value. In rare case, if uninitialized size happens to be the same value as arena's initial size, NMT will mis-count the size, since NMT only updates size when the new size does not equal to the old one. >> >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8025550 >> Webrev: http://cr.openjdk.java.net/~zgu/8025550/webrev.00/ >> >> Thanks, >> >> -Zhengyu > From coleen.phillimore at oracle.com Tue Mar 11 14:55:09 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 11 Mar 2014 10:55:09 -0400 Subject: RFR(L): 8036976: PPC64: implement the template interpreter In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CEA69D7@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CEA69D7@DEWDFEMB12A.global.corp.sap> Message-ID: <531F23CD.1040703@oracle.com> Hi, This is a lot of code to review but I have a couple of comments anyway. It doesn't seem like the functions are in the same place as they are for the other platforms, eg. generate_accessor_entry is in interpreter_ppc.cpp and not in templateInterpreter_ppc.cpp. Also, it appears that you ported the code for UseFastAccessorEntries and UseFastEmptyMethods. We have that turned off and it's been bit rotting (maybe not currently rotten but nobody knows) for years because it made performance worse rather than better. We had hoped to remove this code, so that there was less assembly code. Did you find any performance benefits to this code? https://bugs.openjdk.java.net/browse/JDK-8003426 Thanks, Coleen On 3/11/14 5:13 AM, Lindenmaier, Goetz wrote: > Hi, > > This time I have a really nice one: the template interpreter for ppc64! > Great thanks to my colleagues, especially Axel and Martin, who ported > this. > http://cr.openjdk.java.net/~goetz/webrevs/8036976-0-ti/ > > With this change, both interpreters can be used on linux_ppc64 and aix_ppc64. > > This change almost only touches ppc platform files. > > I shared files, I needed to add a special case for PPC64 in make/linux/Makefile, as I found no way to set FORCE_TIERED=0 in any ppc file that's included above the use of FORCE_TIERED. As we don't have C1, the current scheme does not work for ppc64. > > Actually, one should have lines as > supports_cppInterpreter = 1 > supports_c2 = 1 > supports_c1 = 0 > in platform_. But that's out of scope of this change. > > Further I add a #include in src/share/vm/interpreter/templateTable.hpp. That's all for shared changes. > Please review and test this change. > > Best regards, > Goetz. From per.liden at oracle.com Tue Mar 11 15:00:16 2014 From: per.liden at oracle.com (Per Liden) Date: Tue, 11 Mar 2014 16:00:16 +0100 Subject: RFR(L): 8029075 - String deduplication in G1 In-Reply-To: <531F0AC2.4020701@oracle.com> References: <53148494.60806@oracle.com> <3A26A868-51A8-4169-B147-9AEB8B35EC5E@oracle.com> <53159E0A.60403@oracle.com> <5A84BB3B-9763-47C3-9EEF-0D07EB763576@oracle.com> <5316F4D2.7010600@oracle.com> <7AD2B58F-1268-4A9C-9749-C6C7DAF5AB25@oracle.com> <531DB1BD.4050404@oracle.com> <545F2E7F-AF3D-4646-86EC-7EB2EFD0F427@oracle.com> <531E8681.5030809@oracle.com> <531EF266.3050403@oracle.com> <531EFBCB.6090204@oracle.com> <531F0AC2.4020701@oracle.com> Message-ID: <531F2500.4000600@oracle.com> Just an additional comment below. On 2014-03-11 14:08, Per Liden wrote: > Inline below... > > On 2014-03-11 13:04, David Holmes wrote: >> On 11/03/2014 9:24 PM, Per Liden wrote: >>> On 2014-03-11 04:44, David Holmes wrote: >>>> >>>> >>>> On 11/03/2014 4:18 AM, Christian Thalinger wrote: >>>>> >>>>> On Mar 10, 2014, at 5:36 AM, Per Liden wrote: >>>>>> The other part of the story, orthogonal to the memory overhead, is >>>>>> making reference processing more efficient once the referent becomes >>>>>> unreachable. I've seen many cases where heavy use of weak references >>>>>> (registered or not) overloads the reference handler thread. >>>>> >>>>> Interesting. Is that because it?s just one thread? How about using >>>>> something like ThreadPoolExecutor for the reference handler >>>>> thread(s)? >>>> >>>> The synchronization that would be needed across multiple >>>> reference-handler threads might negate any benefit from having more >>>> than one. And a full blown ThreadPoolExecutor may be overkill here. >>> >>> Gut feeling also tells me that the synchronization would kill it, but >>> might be worth a try. >>> >>>> >>>>>> To improve that situation, I'd like to see the reference handler >>>>>> thread removed completely, and instead let the GC enqueue reference >>>>>> directly onto the reference queues. >>>> >>>> That would require that the GC threads be able to execute Java code. >>>> It would also mean that a GC thread would experience synchronization >>>> (and possible contention) with Java threads. >>> >>> I was more like thinking of modifying the underlying queue in >>> ReferenceQueue so that the GC could enqueue entries from a native >>> context (somewhat similar to how the pending list works today). >>> Admittedly, I haven't thought about all the details here, but I'm >>> thinking that should be doable with a CAS in the fast path for both >>> enqueue and dequeue, and maybe allow a down call from >>> ReferenceQueue.remove() in the slow path in case the queue is empty and >>> the caller needs to block (say, on a ParkEvent maybe). Does that sound >>> completely unreasonable? >> >> The ReferenceHandler thread does more than just enqueue the references, > > What else are you thinking of? sun.misc.Cleaner would obviously have > to be redesigned or removed, but other than that? > >> and enqueuing requires waking up blocked Java threads. Simple atomic >> ops to add elements to the queue would not in themselves be sufficient. > > Yes, a CAS would have to be followed by a potential slow path to > wakeup any waiter if it's the first element that goes on the queue, etc. > >> The details need to be fleshed out here. > > Indeed. Just to be super clear here. There are indeed many details to hash out before anything like this would be implemented. I'm just brainstorming here. This isn't something I (or anyone else in the GC team) is currently working on, but if we believe some of these ideas can improve reference handling it might be worth considering as a future project. cheers, /Per > > /Per > >> >> David >> >>> /Per >>> >>>> >>>> David >>>> ----- >>>> >>>>>> >>>>>> /Per >>>>>> >>>>>>> >>>>>>> ? John >>>>>>> >>>>>>> P.S. Connection with value types: Eventually, we could add value >>>>>>> types >>>>>>> like java.lang.ref.WeakReferenceValue which would provide a >>>>>>> standard >>>>>>> form for this. Or we could standardize the annotation, perhaps. >>>>>>> >>>>>>> P.P.S. For a little more on value types, see >>>>>>> https://blogs.oracle.com/jrose/entry/value_types_and_struct_tearing >>>>> >>> > From goetz.lindenmaier at sap.com Tue Mar 11 15:47:39 2014 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 11 Mar 2014 15:47:39 +0000 Subject: RFR(L): 8036976: PPC64: implement the template interpreter In-Reply-To: <531F23CD.1040703@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CEA69D7@DEWDFEMB12A.global.corp.sap> <531F23CD.1040703@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CEA6AFF@DEWDFEMB12A.global.corp.sap> Hi Coleen, thanks for looking at this! It's really not that small ;) We put functions to interpreter_ppc.cpp if we can use the same implementation for both interpreters. I didn't check all of them, but at least for generate_accessor_entry this is the case. We implemented the UseFast* support for experiments, but we didn't switch it on, either, as it didn't show sufficient benefit. We also figured then that methods are miscounted and other insufficiencies. Best regards, Goetz. -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Coleen Phillimore Sent: Dienstag, 11. M?rz 2014 15:55 To: hotspot-dev at openjdk.java.net Subject: Re: RFR(L): 8036976: PPC64: implement the template interpreter Hi, This is a lot of code to review but I have a couple of comments anyway. It doesn't seem like the functions are in the same place as they are for the other platforms, eg. generate_accessor_entry is in interpreter_ppc.cpp and not in templateInterpreter_ppc.cpp. Also, it appears that you ported the code for UseFastAccessorEntries and UseFastEmptyMethods. We have that turned off and it's been bit rotting (maybe not currently rotten but nobody knows) for years because it made performance worse rather than better. We had hoped to remove this code, so that there was less assembly code. Did you find any performance benefits to this code? https://bugs.openjdk.java.net/browse/JDK-8003426 Thanks, Coleen On 3/11/14 5:13 AM, Lindenmaier, Goetz wrote: > Hi, > > This time I have a really nice one: the template interpreter for ppc64! > Great thanks to my colleagues, especially Axel and Martin, who ported > this. > http://cr.openjdk.java.net/~goetz/webrevs/8036976-0-ti/ > > With this change, both interpreters can be used on linux_ppc64 and aix_ppc64. > > This change almost only touches ppc platform files. > > I shared files, I needed to add a special case for PPC64 in make/linux/Makefile, as I found no way to set FORCE_TIERED=0 in any ppc file that's included above the use of FORCE_TIERED. As we don't have C1, the current scheme does not work for ppc64. > > Actually, one should have lines as > supports_cppInterpreter = 1 > supports_c2 = 1 > supports_c1 = 0 > in platform_. But that's out of scope of this change. > > Further I add a #include in src/share/vm/interpreter/templateTable.hpp. That's all for shared changes. > Please review and test this change. > > Best regards, > Goetz. From asmundak at google.com Tue Mar 11 16:29:23 2014 From: asmundak at google.com (Alexander Smundak) Date: Tue, 11 Mar 2014 09:29:23 -0700 Subject: RFR (M): 8036767 PPC64: Support for little endian execution model In-Reply-To: <531E8309.2060206@oracle.com> References: <531E8309.2060206@oracle.com> Message-ID: On Mon, Mar 10, 2014 at 8:29 PM, David Holmes wrote: > I don't see anything that actually controls what is built. Is it the case > that the new arch value will be reported by uname on the ppc64le system? Yes, $(uname -m) is 'ppc64le' on the little endian, and 'ppc64' on the big endian. And this distinction propagates to gcc, binutils, etc. From coleen.phillimore at oracle.com Tue Mar 11 18:03:03 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 11 Mar 2014 14:03:03 -0400 Subject: RFR(L): 8036976: PPC64: implement the template interpreter In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CEA6AFF@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CEA69D7@DEWDFEMB12A.global.corp.sap> <531F23CD.1040703@oracle.com> <4295855A5C1DE049A61835A1887419CC2CEA6AFF@DEWDFEMB12A.global.corp.sap> Message-ID: <531F4FD7.1060506@oracle.com> On 3/11/14 11:47 AM, Lindenmaier, Goetz wrote: > Hi Coleen, > > thanks for looking at this! It's really not that small ;) > > We put functions to interpreter_ppc.cpp if we can use the same > implementation for both interpreters. I didn't check all of them, > but at least for generate_accessor_entry this is the case. > > We implemented the UseFast* support for experiments, but we didn't > switch it on, either, as it didn't show sufficient benefit. We also figured > then that methods are miscounted and other insufficiencies. Yes, I think that's exactly what happens. They aren't considered run and not inlined, iirc. So you won't mind if these options go away then? Should we preserve the PPC code when they do? I'm optimistic that we'll get to this code (since it's essentially dead code for our port) pretty soon. Thanks, Coleen > > Best regards, > Goetz. > > > > > > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Coleen Phillimore > Sent: Dienstag, 11. M?rz 2014 15:55 > To: hotspot-dev at openjdk.java.net > Subject: Re: RFR(L): 8036976: PPC64: implement the template interpreter > > > Hi, > This is a lot of code to review but I have a couple of comments anyway. > It doesn't seem like the functions are in the same place as they are for > the other platforms, eg. generate_accessor_entry is in > interpreter_ppc.cpp and not in templateInterpreter_ppc.cpp. > > Also, it appears that you ported the code for UseFastAccessorEntries and > UseFastEmptyMethods. We have that turned off and it's been bit rotting > (maybe not currently rotten but nobody knows) for years because it made > performance worse rather than better. We had hoped to remove this code, > so that there was less assembly code. Did you find any performance > benefits to this code? > > https://bugs.openjdk.java.net/browse/JDK-8003426 > > Thanks, > Coleen > > > On 3/11/14 5:13 AM, Lindenmaier, Goetz wrote: >> Hi, >> >> This time I have a really nice one: the template interpreter for ppc64! >> Great thanks to my colleagues, especially Axel and Martin, who ported >> this. >> http://cr.openjdk.java.net/~goetz/webrevs/8036976-0-ti/ >> >> With this change, both interpreters can be used on linux_ppc64 and aix_ppc64. >> >> This change almost only touches ppc platform files. >> >> I shared files, I needed to add a special case for PPC64 in make/linux/Makefile, as I found no way to set FORCE_TIERED=0 in any ppc file that's included above the use of FORCE_TIERED. As we don't have C1, the current scheme does not work for ppc64. >> >> Actually, one should have lines as >> supports_cppInterpreter = 1 >> supports_c2 = 1 >> supports_c1 = 0 >> in platform_. But that's out of scope of this change. >> >> Further I add a #include in src/share/vm/interpreter/templateTable.hpp. That's all for shared changes. >> Please review and test this change. >> >> Best regards, >> Goetz. From christian.thalinger at oracle.com Tue Mar 11 18:14:38 2014 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Tue, 11 Mar 2014 11:14:38 -0700 Subject: RFR (XS): 8037043: put Method flag bits in predictable positions In-Reply-To: <531F1FB4.2070507@oracle.com> References: <531F1FB4.2070507@oracle.com> Message-ID: <06929753-6878-4BF2-90DB-77049162ED09@oracle.com> On Mar 11, 2014, at 7:37 AM, Coleen Phillimore wrote: > > No wonder you don't like C++ with code like that. Can you put () around the ?: expressions so I don't have to dig out my C++ manual to figure out order of operations in the getting and setting functions. Sure: http://cr.openjdk.java.net/~twisti/8037043/webrev.01/ > > Thanks, > Coleen > > On 3/10/14 8:44 PM, Christian Thalinger wrote: >> https://bugs.openjdk.java.net/browse/JDK-8037043 >> http://cr.openjdk.java.net/~twisti/8037043 >> >> 8037043: put Method flag bits in predictable positions >> Reviewed-by: >> >> Currently Method flag bits are managed as a bit field which is C++ implementation specific. Sometimes we need to know the exact position of flag bits in its enclosing data type. For this reason other flag bits in HotSpot are implemented with enum shift values so the exact bit position can be easily identified. >> > From vladimir.kozlov at oracle.com Tue Mar 11 19:02:51 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 11 Mar 2014 12:02:51 -0700 Subject: [8u20] Request for approval: 8025644: java/util/stream/test/org/openjdk/tests/java/util/stream/ToArrayOpTest.java fails with TestData$OfRef): failure java.lang.AssertionError: expected [true] but found [false] In-Reply-To: <32FC335E-CBAF-402B-85AC-63F999D89400@oracle.com> References: <32FC335E-CBAF-402B-85AC-63F999D89400@oracle.com> Message-ID: <531F5DDB.5090607@oracle.com> Igor, You don't need formal approval for Hotspot changes. Alejandro does it for all hotspot changes. You only need to send normal RFR to hotspot-dev and push it into hotspot staging repo from where Alejandor push it into 8u20: http://hg.openjdk.java.net/jdk8u/hs-dev/hotspot Changes are good. Thanks, Vladimri On 3/11/14 11:54 AM, Igor Veresov wrote: > The bug: https://bugs.openjdk.java.net/browse/JDK-8025644 > The JDK9 changeset: http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/5e2306b00977 > Review discussion: http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2014-January/013247.html > > Thanks! > igor > From igor.veresov at oracle.com Tue Mar 11 19:12:47 2014 From: igor.veresov at oracle.com (Igor Veresov) Date: Tue, 11 Mar 2014 12:12:47 -0700 Subject: [8u20] Request for approval: 8025644: java/util/stream/test/org/openjdk/tests/java/util/stream/ToArrayOpTest.java fails with TestData$OfRef): failure java.lang.AssertionError: expected [true] but found [false] In-Reply-To: <531F5DDB.5090607@oracle.com> References: <32FC335E-CBAF-402B-85AC-63F999D89400@oracle.com> <531F5DDB.5090607@oracle.com> Message-ID: <81A871B9-8766-4FE2-9E70-71937FB32EC9@oracle.com> Ugh, alright, thanks! igor On Mar 11, 2014, at 12:02 PM, Vladimir Kozlov wrote: > Igor, > > You don't need formal approval for Hotspot changes. Alejandro does it for all hotspot changes. > You only need to send normal RFR to hotspot-dev and push it into hotspot staging repo from where Alejandor push it into 8u20: > > http://hg.openjdk.java.net/jdk8u/hs-dev/hotspot > > Changes are good. > > Thanks, > Vladimri > > On 3/11/14 11:54 AM, Igor Veresov wrote: >> The bug: https://bugs.openjdk.java.net/browse/JDK-8025644 >> The JDK9 changeset: http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/5e2306b00977 >> Review discussion: http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2014-January/013247.html >> >> Thanks! >> igor >> From goetz.lindenmaier at sap.com Tue Mar 11 19:25:23 2014 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 11 Mar 2014 19:25:23 +0000 Subject: RFR(L): 8036976: PPC64: implement the template interpreter In-Reply-To: <531F4FD7.1060506@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CEA69D7@DEWDFEMB12A.global.corp.sap> <531F23CD.1040703@oracle.com> <4295855A5C1DE049A61835A1887419CC2CEA6AFF@DEWDFEMB12A.global.corp.sap> <531F4FD7.1060506@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CEA6BAA@DEWDFEMB12A.global.corp.sap> Hi Coleen, no, I think it's no problem if you remove it. Best regards, Goetz. -----Original Message----- From: Coleen Phillimore [mailto:coleen.phillimore at oracle.com] Sent: Tuesday, March 11, 2014 7:03 PM To: Lindenmaier, Goetz; hotspot-dev at openjdk.java.net Subject: Re: RFR(L): 8036976: PPC64: implement the template interpreter On 3/11/14 11:47 AM, Lindenmaier, Goetz wrote: > Hi Coleen, > > thanks for looking at this! It's really not that small ;) > > We put functions to interpreter_ppc.cpp if we can use the same > implementation for both interpreters. I didn't check all of them, > but at least for generate_accessor_entry this is the case. > > We implemented the UseFast* support for experiments, but we didn't > switch it on, either, as it didn't show sufficient benefit. We also figured > then that methods are miscounted and other insufficiencies. Yes, I think that's exactly what happens. They aren't considered run and not inlined, iirc. So you won't mind if these options go away then? Should we preserve the PPC code when they do? I'm optimistic that we'll get to this code (since it's essentially dead code for our port) pretty soon. Thanks, Coleen > > Best regards, > Goetz. > > > > > > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Coleen Phillimore > Sent: Dienstag, 11. M?rz 2014 15:55 > To: hotspot-dev at openjdk.java.net > Subject: Re: RFR(L): 8036976: PPC64: implement the template interpreter > > > Hi, > This is a lot of code to review but I have a couple of comments anyway. > It doesn't seem like the functions are in the same place as they are for > the other platforms, eg. generate_accessor_entry is in > interpreter_ppc.cpp and not in templateInterpreter_ppc.cpp. > > Also, it appears that you ported the code for UseFastAccessorEntries and > UseFastEmptyMethods. We have that turned off and it's been bit rotting > (maybe not currently rotten but nobody knows) for years because it made > performance worse rather than better. We had hoped to remove this code, > so that there was less assembly code. Did you find any performance > benefits to this code? > > https://bugs.openjdk.java.net/browse/JDK-8003426 > > Thanks, > Coleen > > > On 3/11/14 5:13 AM, Lindenmaier, Goetz wrote: >> Hi, >> >> This time I have a really nice one: the template interpreter for ppc64! >> Great thanks to my colleagues, especially Axel and Martin, who ported >> this. >> http://cr.openjdk.java.net/~goetz/webrevs/8036976-0-ti/ >> >> With this change, both interpreters can be used on linux_ppc64 and aix_ppc64. >> >> This change almost only touches ppc platform files. >> >> I shared files, I needed to add a special case for PPC64 in make/linux/Makefile, as I found no way to set FORCE_TIERED=0 in any ppc file that's included above the use of FORCE_TIERED. As we don't have C1, the current scheme does not work for ppc64. >> >> Actually, one should have lines as >> supports_cppInterpreter = 1 >> supports_c2 = 1 >> supports_c1 = 0 >> in platform_. But that's out of scope of this change. >> >> Further I add a #include in src/share/vm/interpreter/templateTable.hpp. That's all for shared changes. >> Please review and test this change. >> >> Best regards, >> Goetz. From coleen.phillimore at oracle.com Tue Mar 11 20:25:58 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 11 Mar 2014 16:25:58 -0400 Subject: RFR (XS): 8037043: put Method flag bits in predictable positions In-Reply-To: <06929753-6878-4BF2-90DB-77049162ED09@oracle.com> References: <531F1FB4.2070507@oracle.com> <06929753-6878-4BF2-90DB-77049162ED09@oracle.com> Message-ID: <531F7156.2010609@oracle.com> On 3/11/14 2:14 PM, Christian Thalinger wrote: > On Mar 11, 2014, at 7:37 AM, Coleen Phillimore wrote: > >> No wonder you don't like C++ with code like that. Can you put () around the ?: expressions so I don't have to dig out my C++ manual to figure out order of operations in the getting and setting functions. > Sure: > > http://cr.openjdk.java.net/~twisti/8037043/webrev.01/ It looks less like line noise. Reviewed. Coleen > >> Thanks, >> Coleen >> >> On 3/10/14 8:44 PM, Christian Thalinger wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8037043 >>> http://cr.openjdk.java.net/~twisti/8037043 >>> >>> 8037043: put Method flag bits in predictable positions >>> Reviewed-by: >>> >>> Currently Method flag bits are managed as a bit field which is C++ implementation specific. Sometimes we need to know the exact position of flag bits in its enclosing data type. For this reason other flag bits in HotSpot are implemented with enum shift values so the exact bit position can be easily identified. >>> From coleen.phillimore at oracle.com Tue Mar 11 20:49:19 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 11 Mar 2014 16:49:19 -0400 Subject: RFR (XS): 8037043: put Method flag bits in predictable positions In-Reply-To: <06929753-6878-4BF2-90DB-77049162ED09@oracle.com> References: <531F1FB4.2070507@oracle.com> <06929753-6878-4BF2-90DB-77049162ED09@oracle.com> Message-ID: <531F76CF.8000108@oracle.com> Ignore my other reply. This looks good. Thank you for adding the ()s. Coleen On 3/11/14 2:14 PM, Christian Thalinger wrote: > On Mar 11, 2014, at 7:37 AM, Coleen Phillimore wrote: > >> No wonder you don't like C++ with code like that. Can you put () around the ?: expressions so I don't have to dig out my C++ manual to figure out order of operations in the getting and setting functions. > Sure: > > http://cr.openjdk.java.net/~twisti/8037043/webrev.01/ > >> Thanks, >> Coleen >> >> On 3/10/14 8:44 PM, Christian Thalinger wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8037043 >>> http://cr.openjdk.java.net/~twisti/8037043 >>> >>> 8037043: put Method flag bits in predictable positions >>> Reviewed-by: >>> >>> Currently Method flag bits are managed as a bit field which is C++ implementation specific. Sometimes we need to know the exact position of flag bits in its enclosing data type. For this reason other flag bits in HotSpot are implemented with enum shift values so the exact bit position can be easily identified. >>> From vladimir.kozlov at oracle.com Tue Mar 11 21:12:37 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 11 Mar 2014 14:12:37 -0700 Subject: RFR : JDK-8032045 : (m/l) Enable compiler and linker safety switches for debug builds In-Reply-To: <531F05D3.6020008@oracle.com> References: <531F05D3.6020008@oracle.com> Message-ID: <531F7C45.30607@oracle.com> Mike, About hotspot makefiles changes. There is no DBG variant in hotspot: ifeq ($(VARIANT), DBG) We have specialized makefiles debug.make and fastdebug.make you can used for such settings. Note, normal Hotspot build (at least now) does not invoke top level make files. I would suggest to run JPRT hotspot test job (-stree . from hotspot repo). And also JDK control build with -buildenv SKIP_BOOT_CYCLE=false. Thanks, Vladimir On 3/11/14 5:47 AM, Magnus Ihse Bursie wrote: > On 2014-03-11 00:49, Mike Duigou wrote: >> I have updated the patch to respond to Magnus's feedback and to >> accommodate intervening changes to the configure and hotspot make files. >> >> https://bugs.openjdk.java.net/browse/JDK-8032045 >> http://cr.openjdk.java.net/~mduigou/JDK-8032045/3/webrev/ >> >> This version is, hopefully, almost ready to be pushed. > > I have only glanced at the hotspot build changes and can't really say > anything about them. The hotspot team still owns these; I'm cc:ing them > now. > > The top-level build changes looks fine. Thank you Mike for cleaning > things up! > > /Magnus > >> >> Mike >> >> On Feb 20 2014, at 15:43 , Mike Duigou wrote: >> >>> Hello all; >>> >>> This issue is a followon to JDK-8030350 >>> (https://bugs.openjdk.java.net/browse/JDK-8030350) which enhanced the >>> compiler warnings used for compiling native code. The proposed >>> changes principally impact the Linux platform. >>> >>> While 8030350 was focused on compiler warnings which did not impact >>> code generation, this changeset will, for some configurations, change >>> the native code generated and likely change performance. These >>> proposed option changes prevent specific types of relocation table, >>> stack and heap memory corruption in native code. Preventing these >>> types of memory corruption may be useful for finding certain kinds of >>> bugs though and do provide some minimal additional protections >>> against malicious attacks. They aren't, by any means, a substitute >>> for following appropriate secure coding guidelines. >>> >>> The rationale behind the implementation is as follows. For release >>> builds during the initial phase of JDK 9 I would like to enable only >>> compile time checks. This ends up being similar to the warnings in >>> JDK-8030350. These options have no runtime impact on footprint or >>> performance and very minimal additional compile time cost while >>> providing value. **Release builds are not expected to see any >>> performance or footprint change as a result of this changeset** >>> >>> For fast debug builds we can enable linker protections (relro) and >>> static compile time bounds checks (FORTIFY_SOURCE=1). >>> FORTIFY_SOURCE=1 might be moved to the production builds as well >>> because it has no runtime cost or executable size cost. >>> >>> For slow debug builds we can enable full linker protection (at a >>> potential cost in startup time), runtime bounds checks and stack >>> protection (FORTIFY_SOURCE=2 -fprotect-stack-all). We will likely >>> enable -fprotect-stack-strong when available in GCC 4.9 >>> >>> The basis for enabling the additional protections in debug builds is >>> that it will help us find bugs in our native code and we aren't as >>> concerned in debug builds with footprint and performance. Since many >>> developers already do their personal builds in fastdebug or slowdebug >>> mode for testing this will provide good opportunity to shake out any >>> problems with the options while not impacting release builds. Should >>> we find that any of the options provide significant value for their >>> cost we can move them to fastdebug or release. If any of the options >>> prove too costly they can be demoted or removed entirely. >>> >>> https://bugs.openjdk.java.net/browse/JDK-8032045 >>> http://cr.openjdk.java.net/~mduigou/JDK-8032045/2/webrev/ >>> >>> Additional to enabling the various compiler options I attempted to >>> rationalize some of the skew between the various >>> hotspot/make/{platform}/makefiles/gcc.make files while avoiding >>> changing existing behaviour. I have also introduced the new -Og >>> "optimize for debugging" option and there are now an explicit >>> C{XX}_O_FLAG_DEBUG definitions to complement the >>> C{XX}_O_FLAG_{DEBUG|NORM|HI|HIGHEST|NONE} optimization options. >>> >>> Thanks, >>> >>> Mike > From christian.thalinger at oracle.com Tue Mar 11 21:23:37 2014 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Tue, 11 Mar 2014 14:23:37 -0700 Subject: RFR (XS): 8037043: put Method flag bits in predictable positions In-Reply-To: <531F76CF.8000108@oracle.com> References: <531F1FB4.2070507@oracle.com> <06929753-6878-4BF2-90DB-77049162ED09@oracle.com> <531F76CF.8000108@oracle.com> Message-ID: <85903DF7-A86C-41B1-8B67-1C8CAE7EA89F@oracle.com> Thanks for the review, Coleen. On Mar 11, 2014, at 1:49 PM, Coleen Phillimore wrote: > > Ignore my other reply. This looks good. Thank you for adding the ()s. > Coleen > > On 3/11/14 2:14 PM, Christian Thalinger wrote: >> On Mar 11, 2014, at 7:37 AM, Coleen Phillimore wrote: >> >>> No wonder you don't like C++ with code like that. Can you put () around the ?: expressions so I don't have to dig out my C++ manual to figure out order of operations in the getting and setting functions. >> Sure: >> >> http://cr.openjdk.java.net/~twisti/8037043/webrev.01/ >> >>> Thanks, >>> Coleen >>> >>> On 3/10/14 8:44 PM, Christian Thalinger wrote: >>>> https://bugs.openjdk.java.net/browse/JDK-8037043 >>>> http://cr.openjdk.java.net/~twisti/8037043 >>>> >>>> 8037043: put Method flag bits in predictable positions >>>> Reviewed-by: >>>> >>>> Currently Method flag bits are managed as a bit field which is C++ implementation specific. Sometimes we need to know the exact position of flag bits in its enclosing data type. For this reason other flag bits in HotSpot are implemented with enum shift values so the exact bit position can be easily identified. >>>> > From goetz.lindenmaier at sap.com Tue Mar 11 21:37:28 2014 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 11 Mar 2014 21:37:28 +0000 Subject: RFR(L): 8036976: PPC64: implement the template interpreter In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CEA69E2@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CEA69E2@DEWDFEMB12A.global.corp.sap> Message-ID: <4295855A5C1DE049A61835A1887419CC2CEA6C1E@DEWDFEMB12A.global.corp.sap> Hi, I had missed all the new files in the webrev. I updated it with the files in it. Sorry for this trouble, Goetz. From: Lindenmaier, Goetz Sent: Tuesday, March 11, 2014 10:13 AM To: hotspot-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net Subject: RFR(L): 8036976: PPC64: implement the template interpreter Hi, This time I have a really nice one: the template interpreter for ppc64! Great thanks to my colleagues, especially Axel and Martin, who ported this. http://cr.openjdk.java.net/~goetz/webrevs/8036976-0-ti/ With this change, both interpreters can be used on linux_ppc64 and aix_ppc64. This change almost only touches ppc platform files. I shared files, I needed to add a special case for PPC64 in make/linux/Makefile, as I found no way to set FORCE_TIERED=0 in any ppc file that's included above the use of FORCE_TIERED. As we don't have C1, the current scheme does not work for ppc64. Actually, one should have lines as supports_cppInterpreter = 1 supports_c2 = 1 supports_c1 = 0 in platform_. But that's out of scope of this change. Further I add a #include in src/share/vm/interpreter/templateTable.hpp. That's all for shared changes. Please review and test this change. Best regards, Goetz. From john.r.rose at oracle.com Tue Mar 11 22:09:06 2014 From: john.r.rose at oracle.com (John Rose) Date: Tue, 11 Mar 2014 15:09:06 -0700 Subject: RFR (XS): 8037043: put Method flag bits in predictable positions In-Reply-To: <06929753-6878-4BF2-90DB-77049162ED09@oracle.com> References: <531F1FB4.2070507@oracle.com> <06929753-6878-4BF2-90DB-77049162ED09@oracle.com> Message-ID: On Mar 11, 2014, at 11:14 AM, Christian Thalinger wrote: >> Can you put () around the ?: expressions so I don't have to dig out my C++ manual to figure out order of operations in the getting and setting functions. FTR, adding the parens is a good application of this bullet in the style guide: ? Use extra parentheses in expressions whenever operator precedence seems doubtful. Always use parentheses in shift/mask expressions (<<, &, |, ~). Ref: https://wiki.openjdk.java.net/display/HotSpot/StyleGuide#StyleGuide-Whitespace (The unary "~" probably doesn't belong in the list.) ? John From vladimir.kozlov at oracle.com Tue Mar 11 22:51:28 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 11 Mar 2014 15:51:28 -0700 Subject: RFR (M): 8036767 PPC64: Support for little endian execution model In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CEA6A09@DEWDFEMB12A.global.corp.sap> References: <531E8309.2060206@oracle.com> <531ED06F.5040905@oracle.com> <4295855A5C1DE049A61835A1887419CC2CEA6A09@DEWDFEMB12A.global.corp.sap> Message-ID: <531F9370.4020305@oracle.com> On 3/11/14 2:34 AM, Lindenmaier, Goetz wrote: > Hi, > > basically this comes down to the setting of BUILDARCH in the > defs.make file. > I think it is useful the le-port is built into a directory linux_ppc64le_server (Infix le!!). > It?s the purpose of this path naming to keep incompatible builds apart. It would only help if you could do cross compilation to have both build variants at the same place. Currently you can only build le variant on ppc64le machine and vice versa. That is why, I think, David asked if we can control what variant to build. I would like to see the changes based on Volker suggestion. We can compare them and decide which way to go. Thanks, Vladimir > Unfortunately BUILDARCH is also used to find the submakefiles, so these > two extra ones are needed. > > But the ppc64le port needs different compiler and linker settings, so there > is a good reason to have an extra ppc64le.make. > > Only the platform_ppc64le file is completely redundant, but that's acceptable I > would say. > > David, this is the complete change to hostpot for this port - there will not be new > arch subdirectories in the C-code. > > > Please watch out, there are two threads of this because of the mail server > problems on the weekend. > > Best regards, > Goetz. > > > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of David Holmes > Sent: Dienstag, 11. M?rz 2014 09:59 > To: Volker Simonis > Cc: HotSpot Open Source Developers; Alexander Smundak > Subject: Re: RFR (M): 8036767 PPC64: Support for little endian execution model > > On 11/03/2014 6:36 PM, Volker Simonis wrote: >> On Tue, Mar 11, 2014 at 4:29 AM, David Holmes wrote: >>> I don't see anything that actually controls what is built. Is it the case >>> that the new arch value will be reported by uname on the ppc64le system? >>> >>> (I cringe seeing a new top-level arch introduced for this.) >>> >> >> I didn't liked this too much as well in my first review and I think we >> can get along without a new top-level arch (see >> http://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/2014-March/001790.html). >> >> But in the end I can live with both approaches (and having two >> top-level archs probably makes it slightly simpler to support them >> both in parallel) so I think finally it is up to you which approach we >> take. > > I agree with your initial review. I don't see a LE build as a distinct > architecture but a variant of ppc64. > > Lets see what anyone else has to say. > > Thanks, > David > > > > > >> Regards, >> Volker >> >>> Thanks, >>> David >>> >>> >>> On 7/03/2014 1:37 PM, Alexander Smundak wrote: >>>> >>>> This patch contains the changes to the build files to build JDK on the >>>> little endian PowerPC64 running Linux, and the changes to the source >>>> files to support little endian mode. >>>> This preceding related change >>>> https://bugs.openjdk.java.net/browse/JDK-8036767 added support for the >>>> ELFv2 ABi used on the little endian PowerPC64. >>>> >>>> Please review and test this change. I need a sponsor. >>>> >>>> The patch is at: >>>> http://cr.openjdk.java.net/~martin/asmundak/8036767/webrev.00/ >>>> >>>> Sasha >>>> >>> From vladimir.kozlov at oracle.com Tue Mar 11 23:12:10 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 11 Mar 2014 16:12:10 -0700 Subject: RFR(L): 8036976: PPC64: implement the template interpreter In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CEA6C1E@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CEA69E2@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC2CEA6C1E@DEWDFEMB12A.global.corp.sap> Message-ID: <531F984A.7090805@oracle.com> Hi Goetz, Should you also check for ppc64le?: ifeq ($(ARCH), ppc64) FORCE_TIERED=0 Please, update year in Oracle copyright line in new files: Copyright (c) 1997, 2010, Oracle to Copyright (c) 2014, Oracle I know that you copied and modified from our existing files but HG history does not show it anyway and they all looks new. We should have done it for all ppc files but it is different issue. These changes passed JPRT build with our closed sources. Thanks, Vladimir On 3/11/14 2:37 PM, Lindenmaier, Goetz wrote: > Hi, > > I had missed all the new files in the webrev. I updated it with the > files in it. > > Sorry for this trouble, > Goetz. > > From: Lindenmaier, Goetz > Sent: Tuesday, March 11, 2014 10:13 AM > To: hotspot-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net > Subject: RFR(L): 8036976: PPC64: implement the template interpreter > > Hi, > > This time I have a really nice one: the template interpreter for ppc64! > Great thanks to my colleagues, especially Axel and Martin, who ported > this. > http://cr.openjdk.java.net/~goetz/webrevs/8036976-0-ti/ > > With this change, both interpreters can be used on linux_ppc64 and aix_ppc64. > > This change almost only touches ppc platform files. > > I shared files, I needed to add a special case for PPC64 in make/linux/Makefile, as I found no way to set FORCE_TIERED=0 in any ppc file that's included above the use of FORCE_TIERED. As we don't have C1, the current scheme does not work for ppc64. > > Actually, one should have lines as > supports_cppInterpreter = 1 > supports_c2 = 1 > supports_c1 = 0 > in platform_. But that's out of scope of this change. > > Further I add a #include in src/share/vm/interpreter/templateTable.hpp. That's all for shared changes. > Please review and test this change. > > Best regards, > Goetz. > From asmundak at google.com Tue Mar 11 23:19:52 2014 From: asmundak at google.com (Alexander Smundak) Date: Tue, 11 Mar 2014 16:19:52 -0700 Subject: RFR (M): 8036767 PPC64: Support for little endian execution model In-Reply-To: <531F9370.4020305@oracle.com> References: <531E8309.2060206@oracle.com> <531ED06F.5040905@oracle.com> <4295855A5C1DE049A61835A1887419CC2CEA6A09@DEWDFEMB12A.global.corp.sap> <531F9370.4020305@oracle.com> Message-ID: On Tue, Mar 11, 2014 at 3:51 PM, Vladimir Kozlov wrote: > It would only help if you could do cross compilation to have both build > variants at the same place. Currently you can only build le variant on > ppc64le machine and vice versa. That is why, I think, David asked if we can > control what variant to build. Just to clarify the situation a bit: ppc64le is not a variant of ppc64. That is, an application compiled for the little-endian PowerPC64 does not "just run" on the big-endian PowerPC64 (albeit OS can have such feature, similar to the ability of the Linux running on x86_64 CPU to run 32-bit x86 applications). So ppc64le is a different architecture from ppc64. > I would like to see the changes based on Volker suggestion. We can compare > them and decide which way to go. Volker has the detailed suggestion here: http://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/2014-March/001790.html and it involves additional Make variable and if statements in the platform makefile where they are not supposed to be present. From goetz.lindenmaier at sap.com Wed Mar 12 07:47:27 2014 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 12 Mar 2014 07:47:27 +0000 Subject: RFR (M): 8036767 PPC64: Support for little endian execution model In-Reply-To: <531F9370.4020305@oracle.com> References: <531E8309.2060206@oracle.com> <531ED06F.5040905@oracle.com> <4295855A5C1DE049A61835A1887419CC2CEA6A09@DEWDFEMB12A.global.corp.sap> <531F9370.4020305@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CEA6E16@DEWDFEMB12A.global.corp.sap> Hi, > Currently you can only build le variant on ppc64le machine and vice versa. If you have a repo on a central storage, you can build into the same directory from several different machines. That's what I do all the time (except for ppc64le, because I don't have such a machine). Else, we can remove os and cpu from the path, as it's even less likely that you do a windows cross build on a linux machine. Best regards, Goetz. -----Original Message----- From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] Sent: Dienstag, 11. M?rz 2014 23:51 To: Lindenmaier, Goetz; David Holmes; Volker Simonis Cc: HotSpot Open Source Developers; Alexander Smundak Subject: Re: RFR (M): 8036767 PPC64: Support for little endian execution model On 3/11/14 2:34 AM, Lindenmaier, Goetz wrote: > Hi, > > basically this comes down to the setting of BUILDARCH in the > defs.make file. > I think it is useful the le-port is built into a directory linux_ppc64le_server (Infix le!!). > It?s the purpose of this path naming to keep incompatible builds apart. It would only help if you could do cross compilation to have both build variants at the same place. Currently you can only build le variant on ppc64le machine and vice versa. That is why, I think, David asked if we can control what variant to build. I would like to see the changes based on Volker suggestion. We can compare them and decide which way to go. Thanks, Vladimir > Unfortunately BUILDARCH is also used to find the submakefiles, so these > two extra ones are needed. > > But the ppc64le port needs different compiler and linker settings, so there > is a good reason to have an extra ppc64le.make. > > Only the platform_ppc64le file is completely redundant, but that's acceptable I > would say. > > David, this is the complete change to hostpot for this port - there will not be new > arch subdirectories in the C-code. > > > Please watch out, there are two threads of this because of the mail server > problems on the weekend. > > Best regards, > Goetz. > > > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of David Holmes > Sent: Dienstag, 11. M?rz 2014 09:59 > To: Volker Simonis > Cc: HotSpot Open Source Developers; Alexander Smundak > Subject: Re: RFR (M): 8036767 PPC64: Support for little endian execution model > > On 11/03/2014 6:36 PM, Volker Simonis wrote: >> On Tue, Mar 11, 2014 at 4:29 AM, David Holmes wrote: >>> I don't see anything that actually controls what is built. Is it the case >>> that the new arch value will be reported by uname on the ppc64le system? >>> >>> (I cringe seeing a new top-level arch introduced for this.) >>> >> >> I didn't liked this too much as well in my first review and I think we >> can get along without a new top-level arch (see >> http://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/2014-March/001790.html). >> >> But in the end I can live with both approaches (and having two >> top-level archs probably makes it slightly simpler to support them >> both in parallel) so I think finally it is up to you which approach we >> take. > > I agree with your initial review. I don't see a LE build as a distinct > architecture but a variant of ppc64. > > Lets see what anyone else has to say. > > Thanks, > David > > > > > >> Regards, >> Volker >> >>> Thanks, >>> David >>> >>> >>> On 7/03/2014 1:37 PM, Alexander Smundak wrote: >>>> >>>> This patch contains the changes to the build files to build JDK on the >>>> little endian PowerPC64 running Linux, and the changes to the source >>>> files to support little endian mode. >>>> This preceding related change >>>> https://bugs.openjdk.java.net/browse/JDK-8036767 added support for the >>>> ELFv2 ABi used on the little endian PowerPC64. >>>> >>>> Please review and test this change. I need a sponsor. >>>> >>>> The patch is at: >>>> http://cr.openjdk.java.net/~martin/asmundak/8036767/webrev.00/ >>>> >>>> Sasha >>>> >>> From david.holmes at oracle.com Wed Mar 12 07:20:20 2014 From: david.holmes at oracle.com (David Holmes) Date: Wed, 12 Mar 2014 17:20:20 +1000 Subject: RFR(L): 8029075 - String deduplication in G1 In-Reply-To: <531F0AC2.4020701@oracle.com> References: <53148494.60806@oracle.com> <3A26A868-51A8-4169-B147-9AEB8B35EC5E@oracle.com> <53159E0A.60403@oracle.com> <5A84BB3B-9763-47C3-9EEF-0D07EB763576@oracle.com> <5316F4D2.7010600@oracle.com> <7AD2B58F-1268-4A9C-9749-C6C7DAF5AB25@oracle.com> <531DB1BD.4050404@oracle.com> <545F2E7F-AF3D-4646-86EC-7EB2EFD0F427@oracle.com> <531E8681.5030809@oracle.com> <531EF266.3050403@oracle.com> <531EFBCB.6090204@oracle.com> <531F0AC2.4020701@oracle.com> Message-ID: <53200AB4.90704@oracle.com> On 11/03/2014 11:08 PM, Per Liden wrote: > Inline below... > > On 2014-03-11 13:04, David Holmes wrote: >> On 11/03/2014 9:24 PM, Per Liden wrote: >>> On 2014-03-11 04:44, David Holmes wrote: >>>> >>>> >>>> On 11/03/2014 4:18 AM, Christian Thalinger wrote: >>>>> >>>>> On Mar 10, 2014, at 5:36 AM, Per Liden wrote: >>>>>> The other part of the story, orthogonal to the memory overhead, is >>>>>> making reference processing more efficient once the referent becomes >>>>>> unreachable. I've seen many cases where heavy use of weak references >>>>>> (registered or not) overloads the reference handler thread. >>>>> >>>>> Interesting. Is that because it?s just one thread? How about using >>>>> something like ThreadPoolExecutor for the reference handler thread(s)? >>>> >>>> The synchronization that would be needed across multiple >>>> reference-handler threads might negate any benefit from having more >>>> than one. And a full blown ThreadPoolExecutor may be overkill here. >>> >>> Gut feeling also tells me that the synchronization would kill it, but >>> might be worth a try. >>> >>>> >>>>>> To improve that situation, I'd like to see the reference handler >>>>>> thread removed completely, and instead let the GC enqueue reference >>>>>> directly onto the reference queues. >>>> >>>> That would require that the GC threads be able to execute Java code. >>>> It would also mean that a GC thread would experience synchronization >>>> (and possible contention) with Java threads. >>> >>> I was more like thinking of modifying the underlying queue in >>> ReferenceQueue so that the GC could enqueue entries from a native >>> context (somewhat similar to how the pending list works today). >>> Admittedly, I haven't thought about all the details here, but I'm >>> thinking that should be doable with a CAS in the fast path for both >>> enqueue and dequeue, and maybe allow a down call from >>> ReferenceQueue.remove() in the slow path in case the queue is empty and >>> the caller needs to block (say, on a ParkEvent maybe). Does that sound >>> completely unreasonable? >> >> The ReferenceHandler thread does more than just enqueue the references, > > What else are you thinking of? sun.misc.Cleaner would obviously have to > be redesigned or removed, but other than that? Yes the Cleaner. >> and enqueuing requires waking up blocked Java threads. Simple atomic >> ops to add elements to the queue would not in themselves be sufficient. > > Yes, a CAS would have to be followed by a potential slow path to wakeup > any waiter if it's the first element that goes on the queue, etc. Again this is not so simple in that it involves synchronization with the Java threads - to do a notifyAll you need the monitor and the monitor may be locked by a thread now paused at a safepoint because of the current GC action. Cheers, David >> The details need to be fleshed out here. > > Indeed. > > /Per > >> >> David >> >>> /Per >>> >>>> >>>> David >>>> ----- >>>> >>>>>> >>>>>> /Per >>>>>> >>>>>>> >>>>>>> ? John >>>>>>> >>>>>>> P.S. Connection with value types: Eventually, we could add value >>>>>>> types >>>>>>> like java.lang.ref.WeakReferenceValue which would provide a standard >>>>>>> form for this. Or we could standardize the annotation, perhaps. >>>>>>> >>>>>>> P.P.S. For a little more on value types, see >>>>>>> https://blogs.oracle.com/jrose/entry/value_types_and_struct_tearing >>>>> >>> > From david.holmes at oracle.com Wed Mar 12 09:57:28 2014 From: david.holmes at oracle.com (David Holmes) Date: Wed, 12 Mar 2014 19:57:28 +1000 Subject: RFR : JDK-8032045 : (m/l) Enable compiler and linker safety switches for debug builds In-Reply-To: <531F7C45.30607@oracle.com> References: <531F05D3.6020008@oracle.com> <531F7C45.30607@oracle.com> Message-ID: <53202F88.4030402@oracle.com> On 12/03/2014 7:12 AM, Vladimir Kozlov wrote: > Mike, > > About hotspot makefiles changes. There is no DBG variant in hotspot: > > ifeq ($(VARIANT), DBG) > > We have specialized makefiles debug.make and fastdebug.make you can used > for such settings. Though we are very inconsistent about whether things go in those files or in the compiler specific ones. You either have to put compiler conditionals in the debug/fastdebug files, or else debug/fastdebug conditionals in the compiler .make files. > Note, normal Hotspot build (at least now) does not invoke top level make > files. What are you referring to? A "normal" hotspot build starts with make/Makefile. A developer on platform X may kick of a build using make/X/Makefile (though we need to wean them off that). A top-level configure-based build will also use make/Makefile. David > I would suggest to run JPRT hotspot test job (-stree . from hotspot > repo). And also JDK control build with -buildenv SKIP_BOOT_CYCLE=false. > > Thanks, > Vladimir > > On 3/11/14 5:47 AM, Magnus Ihse Bursie wrote: >> On 2014-03-11 00:49, Mike Duigou wrote: >>> I have updated the patch to respond to Magnus's feedback and to >>> accommodate intervening changes to the configure and hotspot make files. >>> >>> https://bugs.openjdk.java.net/browse/JDK-8032045 >>> http://cr.openjdk.java.net/~mduigou/JDK-8032045/3/webrev/ >>> >>> This version is, hopefully, almost ready to be pushed. >> >> I have only glanced at the hotspot build changes and can't really say >> anything about them. The hotspot team still owns these; I'm cc:ing them >> now. >> >> The top-level build changes looks fine. Thank you Mike for cleaning >> things up! >> >> /Magnus >> >>> >>> Mike >>> >>> On Feb 20 2014, at 15:43 , Mike Duigou wrote: >>> >>>> Hello all; >>>> >>>> This issue is a followon to JDK-8030350 >>>> (https://bugs.openjdk.java.net/browse/JDK-8030350) which enhanced the >>>> compiler warnings used for compiling native code. The proposed >>>> changes principally impact the Linux platform. >>>> >>>> While 8030350 was focused on compiler warnings which did not impact >>>> code generation, this changeset will, for some configurations, change >>>> the native code generated and likely change performance. These >>>> proposed option changes prevent specific types of relocation table, >>>> stack and heap memory corruption in native code. Preventing these >>>> types of memory corruption may be useful for finding certain kinds of >>>> bugs though and do provide some minimal additional protections >>>> against malicious attacks. They aren't, by any means, a substitute >>>> for following appropriate secure coding guidelines. >>>> >>>> The rationale behind the implementation is as follows. For release >>>> builds during the initial phase of JDK 9 I would like to enable only >>>> compile time checks. This ends up being similar to the warnings in >>>> JDK-8030350. These options have no runtime impact on footprint or >>>> performance and very minimal additional compile time cost while >>>> providing value. **Release builds are not expected to see any >>>> performance or footprint change as a result of this changeset** >>>> >>>> For fast debug builds we can enable linker protections (relro) and >>>> static compile time bounds checks (FORTIFY_SOURCE=1). >>>> FORTIFY_SOURCE=1 might be moved to the production builds as well >>>> because it has no runtime cost or executable size cost. >>>> >>>> For slow debug builds we can enable full linker protection (at a >>>> potential cost in startup time), runtime bounds checks and stack >>>> protection (FORTIFY_SOURCE=2 -fprotect-stack-all). We will likely >>>> enable -fprotect-stack-strong when available in GCC 4.9 >>>> >>>> The basis for enabling the additional protections in debug builds is >>>> that it will help us find bugs in our native code and we aren't as >>>> concerned in debug builds with footprint and performance. Since many >>>> developers already do their personal builds in fastdebug or slowdebug >>>> mode for testing this will provide good opportunity to shake out any >>>> problems with the options while not impacting release builds. Should >>>> we find that any of the options provide significant value for their >>>> cost we can move them to fastdebug or release. If any of the options >>>> prove too costly they can be demoted or removed entirely. >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8032045 >>>> http://cr.openjdk.java.net/~mduigou/JDK-8032045/2/webrev/ >>>> >>>> Additional to enabling the various compiler options I attempted to >>>> rationalize some of the skew between the various >>>> hotspot/make/{platform}/makefiles/gcc.make files while avoiding >>>> changing existing behaviour. I have also introduced the new -Og >>>> "optimize for debugging" option and there are now an explicit >>>> C{XX}_O_FLAG_DEBUG definitions to complement the >>>> C{XX}_O_FLAG_{DEBUG|NORM|HI|HIGHEST|NONE} optimization options. >>>> >>>> Thanks, >>>> >>>> Mike >> From goetz.lindenmaier at sap.com Wed Mar 12 11:04:46 2014 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 12 Mar 2014 11:04:46 +0000 Subject: RFR(L): 8036976: PPC64: implement the template interpreter In-Reply-To: <531F984A.7090805@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CEA69E2@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC2CEA6C1E@DEWDFEMB12A.global.corp.sap> <531F984A.7090805@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CEA744D@DEWDFEMB12A.global.corp.sap> Hi Vladimir, thanks for the hint with the makefile -- that would have popped up soon after I guess. I fixed that and the Copyright, and updated the webrev. http://cr.openjdk.java.net/~goetz/webrevs/8036976-0-ti/ Best regards, Goetz. -----Original Message----- From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] Sent: Mittwoch, 12. M?rz 2014 00:12 To: Lindenmaier, Goetz; 'hotspot-dev at openjdk.java.net'; 'ppc-aix-port-dev at openjdk.java.net' Subject: Re: RFR(L): 8036976: PPC64: implement the template interpreter Hi Goetz, Should you also check for ppc64le?: ifeq ($(ARCH), ppc64) FORCE_TIERED=0 Please, update year in Oracle copyright line in new files: Copyright (c) 1997, 2010, Oracle to Copyright (c) 2014, Oracle I know that you copied and modified from our existing files but HG history does not show it anyway and they all looks new. We should have done it for all ppc files but it is different issue. These changes passed JPRT build with our closed sources. Thanks, Vladimir On 3/11/14 2:37 PM, Lindenmaier, Goetz wrote: > Hi, > > I had missed all the new files in the webrev. I updated it with the > files in it. > > Sorry for this trouble, > Goetz. > > From: Lindenmaier, Goetz > Sent: Tuesday, March 11, 2014 10:13 AM > To: hotspot-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net > Subject: RFR(L): 8036976: PPC64: implement the template interpreter > > Hi, > > This time I have a really nice one: the template interpreter for ppc64! > Great thanks to my colleagues, especially Axel and Martin, who ported > this. > http://cr.openjdk.java.net/~goetz/webrevs/8036976-0-ti/ > > With this change, both interpreters can be used on linux_ppc64 and aix_ppc64. > > This change almost only touches ppc platform files. > > I shared files, I needed to add a special case for PPC64 in make/linux/Makefile, as I found no way to set FORCE_TIERED=0 in any ppc file that's included above the use of FORCE_TIERED. As we don't have C1, the current scheme does not work for ppc64. > > Actually, one should have lines as > supports_cppInterpreter = 1 > supports_c2 = 1 > supports_c1 = 0 > in platform_. But that's out of scope of this change. > > Further I add a #include in src/share/vm/interpreter/templateTable.hpp. That's all for shared changes. > Please review and test this change. > > Best regards, > Goetz. > From per.liden at oracle.com Wed Mar 12 13:34:38 2014 From: per.liden at oracle.com (Per Liden) Date: Wed, 12 Mar 2014 14:34:38 +0100 Subject: RFR(L): 8029075 - String deduplication in G1 In-Reply-To: <1394541591.2696.20.camel@cirrus> References: <53148494.60806@oracle.com> <5316F5B1.60006@oracle.com> <5319D424.90306@oracle.com> <1394541591.2696.20.camel@cirrus> Message-ID: <5320626E.7020102@oracle.com> Hi Thomas, Thanks for reviewing. Comments inline. On 03/11/2014 01:39 PM, Thomas Schatzl wrote: > Hi Per, > > > On Fri, 2014-03-07 at 15:13 +0100, Per Liden wrote: >> Hi, >> >> Here's an updated webrev based on the feedback from Bengt and Thomas. >> >> http://cr.openjdk.java.net/~pliden/8029075/webrev.1/ >> > > See further below. > >> >> And as suggested, two changes in the initial webrev were broken out into >> separate CRs, available here: >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8036672 >> Webrev: http://cr.openjdk.java.net/~pliden/8036672/webrev.0/ > > Looks good. > > Did you check the impact on something like specjbb2005 which does > nothing but object copying? That code is somewhat performance sensitive. Yes, I did Aurora runs with jbb2005 and jbb2013, didn't see any difference. > > >> Bug: https://bugs.openjdk.java.net/browse/JDK-8036673 >> Webrev: http://cr.openjdk.java.net/~pliden/8036673/webrev.0/ > > Looks good. > > Here are my comments: there are a few minor issues, and a single major: > basically the documentation for this feature is lacking imo. I listed a > few particular issues of mine with it. > > First the minor issues: > > - could you check on copyright dates on the modified files? They have > not been updated anywhere afaics. > > - not sure if good, but maybe make PrintStringDeduplicationStatistics > diagnostic? It seems to really be some diagnosis functions. > > - maybe make the new *ALot flags notproduct. These seem to be debugging > helpers similar to other *ALot flags. I had a conversion with Bengt about this some time ago and I think the options are categorized correctly. The ALot flags needs be available in product builds because they are used by tests. Most Print*Statistics are product therefore it makes sense to follow that pattern for PrintStringDeduplicationStatistics. > > - StringDeduplicationAgeThreshold: maybe also needs to mention that if > the object is promoted, that it is always considered as candidate. Check. > > marksweep.inline.hpp > > - MarkSweep::mark_object(): did you measure impact on other collectors > using this code? It is enabled unconditionally. Also I would prefer > that !UseG1GC automatically disables UseStringDeduplication (or use a > global variable for that) to avoid the two checks. (The latter option is > probably better). Yes, ran jbb2005/2013. > > symbolTable.cpp: > > - again both UseG1GC and UseStringDedupliation are paired. Could this be > put in a method/global variable somewhere that is set up correctly at > startup? I think using the options is much more readable. Adding another state which tells us if G1 & Dedup is enabled means we need a pre-init stage to set up that state, which I don't think is very nice. > > g1CollectedHeap.cpp: > > - StringDedup::initialize() is run unconditionally (the check for > UseStringDeduplication is inside the method) while it's outside for > everything else. Just a note, keep it if wanted. They are hidden inside dedup everywhere, except where the performance overhead of a call might be an issue (like mark and copy). > > TestStringDeduplicationTools.java: > > - maybe put retrieval of the unsafe object into the test library. > Browsing through the code shows quite a lot copy+paste here already. > > - "qeueue" -> "queue" Check. > > all tests: > > - missing bug ids Check. > > From here it is mostly about comments/documentation: I agree that adding a feature overview and some additional comments is probably a good idea. However, I also know that Bengt and StefanK aren't too crazy about large comment blobs so I tried to kept the noise down. I also disagree with some of the other comments below which tend to be about personal taste rather than code quality or breaking some coding convention we have, e.g. the static helpers, readability, naming, etc. I'll add some comments and make some other changes and send out an updated webrev shortly. cheers, /Per > > stringdedup.?pp: > > - is it possible to put an overview about how the string deduplication > works here? There is absolutely no place in the code where e.g. the > string deduplication cycle is explained. There is no comprehensive > information about this feature anywhere, just bits about corner cases or > particular here and there, but nothing that can be used as entry point > to understand what this is actually doing. > > To some extent, the comments seem to be repetitions of what the code > already shows. E.g. "some_constant = (1 << 10); // 1024 (must be power > of 2)" or "// Store hash code in cache" followed by > "java_lang_String::set_hash(java_string, hash);". > > There is also no reference to the JEP (I am not sure if it is customary > to put references to outside documents - I would prefer some); however > the JEP does not replace documentation in the source as it is not > detailed enough, and the JEP cannot be changed any more. The JEP is > interesting for the alternatives considered though. > > Ideally, to reasonably understand the feature I would like to not need > to spend hours looking through the implementation. > > - how many threads does this feature add, and are supported by this code > (for which parts)? Synchronization between them? > > - the comments "For XY" in the StringDedup header file do not give any > hint about what they are doing (not sure if it is interesting as it is > easy to find out the only callers). It seems that this information has > been put at the place where these methods are invoked (or just somewhere > in the cpp file). Imo the methods and members (except trivial) should be > documented in the hpp file in the class body, i.e. in the interface. > > - there are a few methods (also in the StringDedupTable class) in > StringDedup where the results of the methods do not seem obvious and > should be commented. > > - I really dislike the location of the "Candidate selection" explanation > within the cpp file. I would not have found it if I were not looking for > some documentation ;) It would probably fit very well in the above > mentioned overview paragraphs. > > - please add the static helper functions that implement the actual > policies (is_candidate_from_mark, is_candidate_from_evacuation) to the > StringDedup class (as private static methods) so that they can be found > easily. Static non-class helper functions in cpp files should only be > used for stuff that is really unimportant and not relevant to the code > imo. However these seem to be _the_ central policy methods for string > dedup. > So it would be nice to have them mentioned in the class/hpp file and > documented there. > > - facts about the code are repeated, e.g. that the table size must be a > power of two. This is nice, but has a few drawbacks: the compiler does > not care about comments at all, so it might be useful to put asserts in > these places instead, and second, it's just duplication of text with the > usual problem of getting out of date. > > - I am not completely sure why the class has not been called > G1StringDedup. It won't work without G1 anyway and calls G1 internals > all the time. There does not seem to be a way to make it more generic > without considerable refactoring. It is fine to keep the name though, > just asking. I also know that this feature does not want to be a generic > solution. > > - is it possible to separate out the statistics variables of > StringDedupTable into either a separate class instead of using globals > or separate them a little from the other unrelated members? (And > describe them) This would improve readability. > > I *think* this is everything from StringDedupTable::_entries_added to > _rehash_count? I have no idea without searching for every use and > deducing from that. > > - when implementing the hash table for the StringDedupTable, did you > consider trying to reuse one of the existing hashmap implementations? > And potentially extend them. > > Imo we already have way too many ad-hoc implementations of often used > data structures (particularly) in G1 and GC code. > > - please document the properties of the StringDedupTable (type, hashing, > automatic resizing behavior, thread safety, ...). > > - please explain what StringDedupTable/Cache are and what they are used > for in context of the feature. > > - stringdedup.cpp, line 1309: candicate -> candidate > > Thanks, > Thomas > > From erik.helin at oracle.com Wed Mar 12 16:04:26 2014 From: erik.helin at oracle.com (Erik Helin) Date: Wed, 12 Mar 2014 17:04:26 +0100 Subject: RFR: 8036703: Add trace event with statistics for the metaspace chunk free lists In-Reply-To: <5318544E.7050006@oracle.com> References: <53184144.3060905@oracle.com> <5318544E.7050006@oracle.com> Message-ID: <5320858A.8030302@oracle.com> Based on feedback from Mikael and StefanK, I've reworked the patch to put the ChunkFreeListSummaries in the class MetaspaceSummary. Please see the new version at: http://cr.openjdk.java.net/~ehelin/8036703/webrev.03/ Thanks, Erik On 2014-03-06 11:56, Erik Helin wrote: > A small bug had found its way into this patch: > > metaspace.cpp: > +bool MetaspaceAux::has_chunk_free_list(Metaspace::MetadataType mdtype) { > + return Metaspace::get_chunk_manager(mdtype) == NULL; > +} > > should of course be: > > +bool MetaspaceAux::has_chunk_free_list(Metaspace::MetadataType mdtype) { > + return Metaspace::get_chunk_manager(mdtype) != NULL; > +} > > Please see new webrev at: > http://cr.openjdk.java.net/~ehelin/8036703/webrev.01/ > > Thanks, > Erik > > On 2014-03-06 10:35, Erik Helin wrote: >> Hi all, >> >> this patch adds a new trace event - >> vm/gc/metaspace/chunk_free_list_summary for getting statistics for >> metaspace's chunk free lists. The event contains the numbers of chunks >> in the free list and the size of the chunks in the free lists. This >> information is provided for all different kinds of chunks - special, >> small, medium and humongous. >> >> This information will be useful when diagnosing fragmentation issues in >> metaspace. >> >> Webrev: >> http://cr.openjdk.java.net/~ehelin/8036703/webrev.00/ >> >> Enhancement: >> https://bugs.openjdk.java.net/browse/JDK-8036703 >> >> Testing: >> - JFR JTREG tests >> - JPRT >> - Manually verifying that the numbers look correct by looking at >> recordings from small test cases. >> >> Thanks, >> Erik From aph at redhat.com Wed Mar 12 16:11:46 2014 From: aph at redhat.com (Andrew Haley) Date: Wed, 12 Mar 2014 16:11:46 +0000 Subject: AArch64: We have a release Message-ID: <53208742.3070309@redhat.com> We at the AArch64 Port Project are pleased to announce the first release of OpenJDK on the Linux/AArch64 platform. It is the first implementation of the Java platform to be made available for this processor architecture. For those who haven't heard: AArch64 is the latest architecture from ARM. It is an entirely new instruction set, not compatible with the earlier generation of 32-bit ARM processors, so we need a new OpenJDK port for it. This is a complete port, with template interpreter and client and server JIT compilers. We have JDK8 and JDK7 versions available for download now. It will shortly be included in Fedora for AArch64. It is, of course, entirely free software, available under the GPL. Source bundles are available from http://icedtea.wildebeest.org/download/source/aarch64-port-jdk7u60-aarch64-831-b04.tar.xz http://icedtea.wildebeest.org/download/source/aarch64-port-jdk8-aarch64-992-b128.tar.xz We will now to make plans to integrate this port with the main OpenJDK project. Credit where credit is due: We started this work at Red Hat almost two years ago with two Red Hat engineers, Andrew Haley and Andrew Dinn. It became an OpenJDK project and we were joined by Linaro, who have been extremely helpful with testing, bug fixing, and keeping up with OpenJDK's continuous updates. Many people have helped, but in particular I'd like to thank Edward Nevill for his support and tireless work. Of course the work will go on: there is performance tuning to be done on real hardware and we need to do a lot more real-world testing. In particular, we haven't yet done work to optimize HotSpot for the Advanced SIMD and Cryptography units. If you are interested in working on this exciting project or if you just want to chat about it, come over to aarch64-port-dev at openjdk.java.net . Andrew Haley OpenJDK AArch64 port Project Lead From erik.gahlin at oracle.com Wed Mar 12 17:01:09 2014 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Wed, 12 Mar 2014 18:01:09 +0100 Subject: RFR: 8036703: Add trace event with statistics for the metaspace chunk free lists In-Reply-To: <53184144.3060905@oracle.com> References: <53184144.3060905@oracle.com> Message-ID: <532092D5.1040902@oracle.com> Metadata looks good Erik rik Helin skrev 2014-03-06 10:35: > Hi all, > > this patch adds a new trace event - > vm/gc/metaspace/chunk_free_list_summary for getting statistics for > metaspace's chunk free lists. The event contains the numbers of chunks > in the free list and the size of the chunks in the free lists. This > information is provided for all different kinds of chunks - special, > small, medium and humongous. > > This information will be useful when diagnosing fragmentation issues > in metaspace. > > Webrev: > http://cr.openjdk.java.net/~ehelin/8036703/webrev.00/ > > Enhancement: > https://bugs.openjdk.java.net/browse/JDK-8036703 > > Testing: > - JFR JTREG tests > - JPRT > - Manually verifying that the numbers look correct by looking at > recordings from small test cases. > > Thanks, > Erik From volker.simonis at gmail.com Wed Mar 12 17:33:44 2014 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 12 Mar 2014 18:33:44 +0100 Subject: AArch64: We have a release In-Reply-To: <53208742.3070309@redhat.com> References: <53208742.3070309@redhat.com> Message-ID: Congratulations!!! Did you pass the Java 7 and Java 8 TCK? Regards, Volker On Wed, Mar 12, 2014 at 5:11 PM, Andrew Haley wrote: > We at the AArch64 Port Project are pleased to announce the first > release of OpenJDK on the Linux/AArch64 platform. It is the first > implementation of the Java platform to be made available for this > processor architecture. > > For those who haven't heard: AArch64 is the latest architecture from > ARM. It is an entirely new instruction set, not compatible with the > earlier generation of 32-bit ARM processors, so we need a new OpenJDK > port for it. > > This is a complete port, with template interpreter and client and > server JIT compilers. We have JDK8 and JDK7 versions available for > download now. It will shortly be included in Fedora for AArch64. > It is, of course, entirely free software, available under the GPL. > > Source bundles are available from > http://icedtea.wildebeest.org/download/source/aarch64-port-jdk7u60-aarch64-831-b04.tar.xz > http://icedtea.wildebeest.org/download/source/aarch64-port-jdk8-aarch64-992-b128.tar.xz > > We will now to make plans to integrate this port with the main OpenJDK > project. > > Credit where credit is due: > > We started this work at Red Hat almost two years ago with two Red Hat > engineers, Andrew Haley and Andrew Dinn. It became an OpenJDK project > and we were joined by Linaro, who have been extremely helpful with > testing, bug fixing, and keeping up with OpenJDK's continuous updates. > Many people have helped, but in particular I'd like to thank Edward > Nevill for his support and tireless work. > > Of course the work will go on: there is performance tuning to be done > on real hardware and we need to do a lot more real-world testing. In > particular, we haven't yet done work to optimize HotSpot for the > Advanced SIMD and Cryptography units. If you are interested in > working on this exciting project or if you just want to chat about it, > come over to aarch64-port-dev at openjdk.java.net . > > Andrew Haley > OpenJDK AArch64 port Project Lead From aph at redhat.com Wed Mar 12 17:37:51 2014 From: aph at redhat.com (Andrew Haley) Date: Wed, 12 Mar 2014 17:37:51 +0000 Subject: AArch64: We have a release In-Reply-To: References: <53208742.3070309@redhat.com> Message-ID: <53209B6F.60009@redhat.com> On 03/12/2014 05:33 PM, Volker Simonis wrote: > Did you pass the Java 7 and Java 8 TCK? Hmmm, good question. Java 7, certainly. Java 8, Ill have to check. Andrew. From vladimir.kozlov at oracle.com Wed Mar 12 19:22:35 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 12 Mar 2014 12:22:35 -0700 Subject: RFR : JDK-8032045 : (m/l) Enable compiler and linker safety switches for debug builds In-Reply-To: <53202F88.4030402@oracle.com> References: <531F05D3.6020008@oracle.com> <531F7C45.30607@oracle.com> <53202F88.4030402@oracle.com> Message-ID: <5320B3FB.4080208@oracle.com> On 3/12/14 2:57 AM, David Holmes wrote: > On 12/03/2014 7:12 AM, Vladimir Kozlov wrote: >> Mike, >> >> About hotspot makefiles changes. There is no DBG variant in hotspot: >> >> ifeq ($(VARIANT), DBG) >> >> We have specialized makefiles debug.make and fastdebug.make you can used >> for such settings. > > Though we are very inconsistent about whether things go in those files > or in the compiler specific ones. You either have to put compiler > conditionals in the debug/fastdebug files, or else debug/fastdebug > conditionals in the compiler .make files. > >> Note, normal Hotspot build (at least now) does not invoke top level make >> files. > > What are you referring to? A "normal" hotspot build starts with > make/Makefile. A developer on platform X may kick of a build using > make/X/Makefile (though we need to wean them off that). A top-level > configure-based build will also use make/Makefile. DBG is defined in JDK make files only: common//autoconf/generated-configure.sh: VARIANT="DBG" common//autoconf/generated-configure.sh: VARIANT="DBG" common//autoconf/hotspot-spec.gmk.in:# Legacy setting: OPT or DBG common//autoconf/jdk-options.m4: VARIANT="DBG" common//autoconf/jdk-options.m4: VARIANT="DBG" But when we build Hotspot standalone we don't use or even have those files. So we should not check DBG in Hotspot's makefiles. Thanks, Vladimir > > David > >> I would suggest to run JPRT hotspot test job (-stree . from hotspot >> repo). And also JDK control build with -buildenv SKIP_BOOT_CYCLE=false. >> >> Thanks, >> Vladimir >> >> On 3/11/14 5:47 AM, Magnus Ihse Bursie wrote: >>> On 2014-03-11 00:49, Mike Duigou wrote: >>>> I have updated the patch to respond to Magnus's feedback and to >>>> accommodate intervening changes to the configure and hotspot make >>>> files. >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8032045 >>>> http://cr.openjdk.java.net/~mduigou/JDK-8032045/3/webrev/ >>>> >>>> This version is, hopefully, almost ready to be pushed. >>> >>> I have only glanced at the hotspot build changes and can't really say >>> anything about them. The hotspot team still owns these; I'm cc:ing them >>> now. >>> >>> The top-level build changes looks fine. Thank you Mike for cleaning >>> things up! >>> >>> /Magnus >>> >>>> >>>> Mike >>>> >>>> On Feb 20 2014, at 15:43 , Mike Duigou wrote: >>>> >>>>> Hello all; >>>>> >>>>> This issue is a followon to JDK-8030350 >>>>> (https://bugs.openjdk.java.net/browse/JDK-8030350) which enhanced the >>>>> compiler warnings used for compiling native code. The proposed >>>>> changes principally impact the Linux platform. >>>>> >>>>> While 8030350 was focused on compiler warnings which did not impact >>>>> code generation, this changeset will, for some configurations, change >>>>> the native code generated and likely change performance. These >>>>> proposed option changes prevent specific types of relocation table, >>>>> stack and heap memory corruption in native code. Preventing these >>>>> types of memory corruption may be useful for finding certain kinds of >>>>> bugs though and do provide some minimal additional protections >>>>> against malicious attacks. They aren't, by any means, a substitute >>>>> for following appropriate secure coding guidelines. >>>>> >>>>> The rationale behind the implementation is as follows. For release >>>>> builds during the initial phase of JDK 9 I would like to enable only >>>>> compile time checks. This ends up being similar to the warnings in >>>>> JDK-8030350. These options have no runtime impact on footprint or >>>>> performance and very minimal additional compile time cost while >>>>> providing value. **Release builds are not expected to see any >>>>> performance or footprint change as a result of this changeset** >>>>> >>>>> For fast debug builds we can enable linker protections (relro) and >>>>> static compile time bounds checks (FORTIFY_SOURCE=1). >>>>> FORTIFY_SOURCE=1 might be moved to the production builds as well >>>>> because it has no runtime cost or executable size cost. >>>>> >>>>> For slow debug builds we can enable full linker protection (at a >>>>> potential cost in startup time), runtime bounds checks and stack >>>>> protection (FORTIFY_SOURCE=2 -fprotect-stack-all). We will likely >>>>> enable -fprotect-stack-strong when available in GCC 4.9 >>>>> >>>>> The basis for enabling the additional protections in debug builds is >>>>> that it will help us find bugs in our native code and we aren't as >>>>> concerned in debug builds with footprint and performance. Since many >>>>> developers already do their personal builds in fastdebug or slowdebug >>>>> mode for testing this will provide good opportunity to shake out any >>>>> problems with the options while not impacting release builds. Should >>>>> we find that any of the options provide significant value for their >>>>> cost we can move them to fastdebug or release. If any of the options >>>>> prove too costly they can be demoted or removed entirely. >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8032045 >>>>> http://cr.openjdk.java.net/~mduigou/JDK-8032045/2/webrev/ >>>>> >>>>> Additional to enabling the various compiler options I attempted to >>>>> rationalize some of the skew between the various >>>>> hotspot/make/{platform}/makefiles/gcc.make files while avoiding >>>>> changing existing behaviour. I have also introduced the new -Og >>>>> "optimize for debugging" option and there are now an explicit >>>>> C{XX}_O_FLAG_DEBUG definitions to complement the >>>>> C{XX}_O_FLAG_{DEBUG|NORM|HI|HIGHEST|NONE} optimization options. >>>>> >>>>> Thanks, >>>>> >>>>> Mike >>> From vladimir.kozlov at oracle.com Wed Mar 12 19:25:05 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 12 Mar 2014 12:25:05 -0700 Subject: RFR(L): 8036976: PPC64: implement the template interpreter In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CEA744D@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CEA69E2@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC2CEA6C1E@DEWDFEMB12A.global.corp.sap> <531F984A.7090805@oracle.com> <4295855A5C1DE049A61835A1887419CC2CEA744D@DEWDFEMB12A.global.corp.sap> Message-ID: <5320B491.3010103@oracle.com> Looks fine to me. I will push it wiht me and Coleen as reviewers. I assume I need to push it into ppc-aix 8u stage repo. Right? Thanks, Vladimir On 3/12/14 4:04 AM, Lindenmaier, Goetz wrote: > Hi Vladimir, > > thanks for the hint with the makefile -- that would have popped up > soon after I guess. > > I fixed that and the Copyright, and updated the webrev. > http://cr.openjdk.java.net/~goetz/webrevs/8036976-0-ti/ > > Best regards, > Goetz. > > > > -----Original Message----- > From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] > Sent: Mittwoch, 12. M?rz 2014 00:12 > To: Lindenmaier, Goetz; 'hotspot-dev at openjdk.java.net'; 'ppc-aix-port-dev at openjdk.java.net' > Subject: Re: RFR(L): 8036976: PPC64: implement the template interpreter > > Hi Goetz, > > Should you also check for ppc64le?: > > ifeq ($(ARCH), ppc64) > FORCE_TIERED=0 > > Please, update year in Oracle copyright line in new files: > > Copyright (c) 1997, 2010, Oracle > > to > > Copyright (c) 2014, Oracle > > I know that you copied and modified from our existing files but HG > history does not show it anyway and they all looks new. We should have > done it for all ppc files but it is different issue. > > These changes passed JPRT build with our closed sources. > > Thanks, > Vladimir > > On 3/11/14 2:37 PM, Lindenmaier, Goetz wrote: >> Hi, >> >> I had missed all the new files in the webrev. I updated it with the >> files in it. >> >> Sorry for this trouble, >> Goetz. >> >> From: Lindenmaier, Goetz >> Sent: Tuesday, March 11, 2014 10:13 AM >> To: hotspot-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net >> Subject: RFR(L): 8036976: PPC64: implement the template interpreter >> >> Hi, >> >> This time I have a really nice one: the template interpreter for ppc64! >> Great thanks to my colleagues, especially Axel and Martin, who ported >> this. >> http://cr.openjdk.java.net/~goetz/webrevs/8036976-0-ti/ >> >> With this change, both interpreters can be used on linux_ppc64 and aix_ppc64. >> >> This change almost only touches ppc platform files. >> >> I shared files, I needed to add a special case for PPC64 in make/linux/Makefile, as I found no way to set FORCE_TIERED=0 in any ppc file that's included above the use of FORCE_TIERED. As we don't have C1, the current scheme does not work for ppc64. >> >> Actually, one should have lines as >> supports_cppInterpreter = 1 >> supports_c2 = 1 >> supports_c1 = 0 >> in platform_. But that's out of scope of this change. >> >> Further I add a #include in src/share/vm/interpreter/templateTable.hpp. That's all for shared changes. >> Please review and test this change. >> >> Best regards, >> Goetz. >> From coleen.phillimore at oracle.com Wed Mar 12 19:36:07 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Wed, 12 Mar 2014 15:36:07 -0400 Subject: RFR(L): 8036976: PPC64: implement the template interpreter In-Reply-To: <5320B491.3010103@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CEA69E2@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC2CEA6C1E@DEWDFEMB12A.global.corp.sap> <531F984A.7090805@oracle.com> <4295855A5C1DE049A61835A1887419CC2CEA744D@DEWDFEMB12A.global.corp.sap> <5320B491.3010103@oracle.com> Message-ID: <5320B727.1090109@oracle.com> Yes, you can count me as a reviewer. Coleen On 3/12/14 3:25 PM, Vladimir Kozlov wrote: > Looks fine to me. I will push it wiht me and Coleen as reviewers. > I assume I need to push it into ppc-aix 8u stage repo. Right? > > Thanks, > Vladimir > > On 3/12/14 4:04 AM, Lindenmaier, Goetz wrote: >> Hi Vladimir, >> >> thanks for the hint with the makefile -- that would have popped up >> soon after I guess. >> >> I fixed that and the Copyright, and updated the webrev. >> http://cr.openjdk.java.net/~goetz/webrevs/8036976-0-ti/ >> >> Best regards, >> Goetz. >> >> >> >> -----Original Message----- >> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] >> Sent: Mittwoch, 12. M?rz 2014 00:12 >> To: Lindenmaier, Goetz; 'hotspot-dev at openjdk.java.net'; >> 'ppc-aix-port-dev at openjdk.java.net' >> Subject: Re: RFR(L): 8036976: PPC64: implement the template interpreter >> >> Hi Goetz, >> >> Should you also check for ppc64le?: >> >> ifeq ($(ARCH), ppc64) >> FORCE_TIERED=0 >> >> Please, update year in Oracle copyright line in new files: >> >> Copyright (c) 1997, 2010, Oracle >> >> to >> >> Copyright (c) 2014, Oracle >> >> I know that you copied and modified from our existing files but HG >> history does not show it anyway and they all looks new. We should have >> done it for all ppc files but it is different issue. >> >> These changes passed JPRT build with our closed sources. >> >> Thanks, >> Vladimir >> >> On 3/11/14 2:37 PM, Lindenmaier, Goetz wrote: >>> Hi, >>> >>> I had missed all the new files in the webrev. I updated it with the >>> files in it. >>> >>> Sorry for this trouble, >>> Goetz. >>> >>> From: Lindenmaier, Goetz >>> Sent: Tuesday, March 11, 2014 10:13 AM >>> To: hotspot-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net >>> Subject: RFR(L): 8036976: PPC64: implement the template interpreter >>> >>> Hi, >>> >>> This time I have a really nice one: the template interpreter for ppc64! >>> Great thanks to my colleagues, especially Axel and Martin, who ported >>> this. >>> http://cr.openjdk.java.net/~goetz/webrevs/8036976-0-ti/ >>> >>> With this change, both interpreters can be used on linux_ppc64 and >>> aix_ppc64. >>> >>> This change almost only touches ppc platform files. >>> >>> I shared files, I needed to add a special case for PPC64 in >>> make/linux/Makefile, as I found no way to set FORCE_TIERED=0 in any >>> ppc file that's included above the use of FORCE_TIERED. As we don't >>> have C1, the current scheme does not work for ppc64. >>> >>> Actually, one should have lines as >>> supports_cppInterpreter = 1 >>> supports_c2 = 1 >>> supports_c1 = 0 >>> in platform_. But that's out of scope of this change. >>> >>> Further I add a #include in >>> src/share/vm/interpreter/templateTable.hpp. That's all for shared >>> changes. >>> Please review and test this change. >>> >>> Best regards, >>> Goetz. >>> From mike.duigou at oracle.com Wed Mar 12 20:40:22 2014 From: mike.duigou at oracle.com (Mike Duigou) Date: Wed, 12 Mar 2014 13:40:22 -0700 Subject: RFR : JDK-8032045 : (m/l) Enable compiler and linker safety switches for debug builds In-Reply-To: <5320B3FB.4080208@oracle.com> References: <531F05D3.6020008@oracle.com> <531F7C45.30607@oracle.com> <53202F88.4030402@oracle.com> <5320B3FB.4080208@oracle.com> Message-ID: <6EE1D39F-6883-4C61-AB31-38E06051F471@oracle.com> On Mar 12 2014, at 12:22 , Vladimir Kozlov wrote: >> What are you referring to? A "normal" hotspot build starts with >> make/Makefile. A developer on platform X may kick of a build using >> make/X/Makefile (though we need to wean them off that). A top-level >> configure-based build will also use make/Makefile. > > DBG is defined in JDK make files only: > > common//autoconf/generated-configure.sh: VARIANT="DBG" > common//autoconf/generated-configure.sh: VARIANT="DBG" > common//autoconf/hotspot-spec.gmk.in:# Legacy setting: OPT or DBG > common//autoconf/jdk-options.m4: VARIANT="DBG" > common//autoconf/jdk-options.m4: VARIANT="DBG" > > But when we build Hotspot standalone we don't use or even have those files. So we should not check DBG in Hotspot's makefiles. Do you mean that there are still builds of hotspot done without including hotspot-spec.gmk? I'd certainly prefer to not have to accommodate that build configuration. It appears that the spec.gmk variable DEBUG_LEVEL is more appropriate to consult than VARIANT and it is presumably included through hotspot-spec.gmk (which presumes hotspot-spec.gmk is included at all) Mike From vladimir.kozlov at oracle.com Wed Mar 12 20:51:57 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 12 Mar 2014 13:51:57 -0700 Subject: RFR : JDK-8032045 : (m/l) Enable compiler and linker safety switches for debug builds In-Reply-To: <6EE1D39F-6883-4C61-AB31-38E06051F471@oracle.com> References: <531F05D3.6020008@oracle.com> <531F7C45.30607@oracle.com> <53202F88.4030402@oracle.com> <5320B3FB.4080208@oracle.com> <6EE1D39F-6883-4C61-AB31-38E06051F471@oracle.com> Message-ID: <5320C8ED.4040209@oracle.com> Currently all Hotspot JPRT jobs build only hotspot part of forest (it clone only jdk/hotspot subforest). It was always this way to keep time small for all platforms we build on. There is work in progress to include whole forest build but it is still far away. Vladimir On 3/12/14 1:40 PM, Mike Duigou wrote: > > On Mar 12 2014, at 12:22 , Vladimir Kozlov wrote: > >>> What are you referring to? A "normal" hotspot build starts with >>> make/Makefile. A developer on platform X may kick of a build using >>> make/X/Makefile (though we need to wean them off that). A top-level >>> configure-based build will also use make/Makefile. >> >> DBG is defined in JDK make files only: >> >> common//autoconf/generated-configure.sh: VARIANT="DBG" >> common//autoconf/generated-configure.sh: VARIANT="DBG" >> common//autoconf/hotspot-spec.gmk.in:# Legacy setting: OPT or DBG >> common//autoconf/jdk-options.m4: VARIANT="DBG" >> common//autoconf/jdk-options.m4: VARIANT="DBG" >> >> But when we build Hotspot standalone we don't use or even have those files. So we should not check DBG in Hotspot's makefiles. > > Do you mean that there are still builds of hotspot done without including hotspot-spec.gmk? > > I'd certainly prefer to not have to accommodate that build configuration. > > It appears that the spec.gmk variable DEBUG_LEVEL is more appropriate to consult than VARIANT and it is presumably included through hotspot-spec.gmk (which presumes hotspot-spec.gmk is included at all) > > Mike > > From coleen.phillimore at oracle.com Wed Mar 12 21:10:11 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Wed, 12 Mar 2014 17:10:11 -0400 Subject: RFR: 8036703: Add trace event with statistics for the metaspace chunk free lists In-Reply-To: <5320858A.8030302@oracle.com> References: <53184144.3060905@oracle.com> <5318544E.7050006@oracle.com> <5320858A.8030302@oracle.com> Message-ID: <5320CD33.2070603@oracle.com> Wow, that looks good. The only comment I had was that this macro for index index_bounds_check in metaspace.cpp could be an inlined static function, then you could avoid the odd while loop. Coleen On 3/12/14 12:04 PM, Erik Helin wrote: > Based on feedback from Mikael and StefanK, I've reworked the patch to > put the ChunkFreeListSummaries in the class MetaspaceSummary. > > Please see the new version at: > http://cr.openjdk.java.net/~ehelin/8036703/webrev.03/ > > Thanks, > Erik > > On 2014-03-06 11:56, Erik Helin wrote: >> A small bug had found its way into this patch: >> >> metaspace.cpp: >> +bool MetaspaceAux::has_chunk_free_list(Metaspace::MetadataType >> mdtype) { >> + return Metaspace::get_chunk_manager(mdtype) == NULL; >> +} >> >> should of course be: >> >> +bool MetaspaceAux::has_chunk_free_list(Metaspace::MetadataType >> mdtype) { >> + return Metaspace::get_chunk_manager(mdtype) != NULL; >> +} >> >> Please see new webrev at: >> http://cr.openjdk.java.net/~ehelin/8036703/webrev.01/ >> >> Thanks, >> Erik >> >> On 2014-03-06 10:35, Erik Helin wrote: >>> Hi all, >>> >>> this patch adds a new trace event - >>> vm/gc/metaspace/chunk_free_list_summary for getting statistics for >>> metaspace's chunk free lists. The event contains the numbers of chunks >>> in the free list and the size of the chunks in the free lists. This >>> information is provided for all different kinds of chunks - special, >>> small, medium and humongous. >>> >>> This information will be useful when diagnosing fragmentation issues in >>> metaspace. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~ehelin/8036703/webrev.00/ >>> >>> Enhancement: >>> https://bugs.openjdk.java.net/browse/JDK-8036703 >>> >>> Testing: >>> - JFR JTREG tests >>> - JPRT >>> - Manually verifying that the numbers look correct by looking at >>> recordings from small test cases. >>> >>> Thanks, >>> Erik From vladimir.kozlov at oracle.com Wed Mar 12 21:33:13 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 12 Mar 2014 14:33:13 -0700 Subject: [8u20] RFR (XS) 8035983: Fix "Native frames:" in crash report (hs_err file) Message-ID: <5320D299.1030002@oracle.com> Please, review backport. Changes were tested for a week in jdk9 sources. http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/e0365e7b7454 https://bugs.openjdk.java.net/browse/JDK-8035983 thanks, Vladimir From vladimir.kozlov at oracle.com Wed Mar 12 21:35:46 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 12 Mar 2014 14:35:46 -0700 Subject: [8u20] RFR (XS): 8033805: Move Fast_Lock/Fast_Unlock code from .ad files to macroassembler Message-ID: <5320D332.2010105@oracle.com> Please, review backport to 8u20. Changes were tested for two weeks in jdk9 sources. https://bugs.openjdk.java.net/browse/JDK-8033805 http://cr.openjdk.java.net/~kvn/8033805/webrev/ http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/5292439ef895 Thanks, Vladimir From coleen.phillimore at oracle.com Wed Mar 12 22:10:19 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Wed, 12 Mar 2014 18:10:19 -0400 Subject: RFR(L): 8029075 - String deduplication in G1 In-Reply-To: <5320626E.7020102@oracle.com> References: <53148494.60806@oracle.com> <5316F5B1.60006@oracle.com> <5319D424.90306@oracle.com> <1394541591.2696.20.camel@cirrus> <5320626E.7020102@oracle.com> Message-ID: <5320DB4B.2020801@oracle.com> Hi Per, Why not add the call for StringDedup::unlink() inside the function unlink_string_and_symbol_table rather than just above in two places, since they go together. + // Delete dead entries in string deduplication queue/table + StringDedup::unlink(&GenMarkSweep::is_alive); + // Delete entries for dead interned string and clean up unreferenced symbols in symbol table. G1CollectedHeap::heap()->unlink_string_and_symbol_table(&GenMarkSweep::is_alive); I didn't see anything else specific to change. Some general comments which are similar to Thomas's. There seems to be a lack of comments in the deduplication file. I hate huge block comments too but some comments about what it does in general would be nice. Like 3 sentences per class. The file is overwhelming (ie, I completely lose interest) with the statistics printing at the front. Maybe you could put the most interesting classes at the front of the file? Maybe statistics printing could be a separate file. One of the conventions that we have is that accessors for classes are on one line, so you could make the file smaller by doing that reformatting. It is also a general convention to have comments above the declaration of a class to say generally what it's for, when it's used, etc. It's not easy to tell by the name or we could guess wrong. What is StringDedupQueue for, eg? That would make reviewing the code easier. There seems to be a lot of duplication of code with the hashtable in hashtable.hpp, only different. We have lots of hashtables in hotspot. Did you try to reinstantiate this hashtable and why didn't that not work? There are interesting memory allocation functions for the entries in this normal hashtable that might help with fragmentation, for instance. Lastly, why is the hash code in the string written to and used? The StringTable uses the same hash algorithm until it has to change the hash code, but the new hash code isn't stored in the string because it would be incompatible. I would like to see this be a few different files. It seems like this file does three things conceptually, interacts with G1 somehow to determine which strings are candidates for deduplication, stores these things in a rehashable, resizable hash table and gathers statistics. I'm sorry to have so many comments so late and I haven't really tried to read the code for correctness. I think some reformatting and reorganization would make the reviewers job easier. It's a big change and worthwhile. Thanks, Coleen On 3/12/14 9:34 AM, Per Liden wrote: > Hi Thomas, > > Thanks for reviewing. Comments inline. > > On 03/11/2014 01:39 PM, Thomas Schatzl wrote: >> Hi Per, >> >> >> On Fri, 2014-03-07 at 15:13 +0100, Per Liden wrote: >>> Hi, >>> >>> Here's an updated webrev based on the feedback from Bengt and Thomas. >>> >>> http://cr.openjdk.java.net/~pliden/8029075/webrev.1/ >>> >> >> See further below. >> >>> >>> And as suggested, two changes in the initial webrev were broken out >>> into >>> separate CRs, available here: >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8036672 >>> Webrev: http://cr.openjdk.java.net/~pliden/8036672/webrev.0/ >> >> Looks good. >> >> Did you check the impact on something like specjbb2005 which does >> nothing but object copying? That code is somewhat performance sensitive. > > Yes, I did Aurora runs with jbb2005 and jbb2013, didn't see any > difference. > >> >> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8036673 >>> Webrev: http://cr.openjdk.java.net/~pliden/8036673/webrev.0/ >> >> Looks good. >> >> Here are my comments: there are a few minor issues, and a single major: >> basically the documentation for this feature is lacking imo. I listed a >> few particular issues of mine with it. >> >> First the minor issues: >> >> - could you check on copyright dates on the modified files? They have >> not been updated anywhere afaics. >> >> - not sure if good, but maybe make PrintStringDeduplicationStatistics >> diagnostic? It seems to really be some diagnosis functions. >> >> - maybe make the new *ALot flags notproduct. These seem to be debugging >> helpers similar to other *ALot flags. > > I had a conversion with Bengt about this some time ago and I think the > options are categorized correctly. The ALot flags needs be available > in product builds because they are used by tests. Most > Print*Statistics are product therefore it makes sense to follow that > pattern for PrintStringDeduplicationStatistics. > >> >> - StringDeduplicationAgeThreshold: maybe also needs to mention that if >> the object is promoted, that it is always considered as candidate. > > Check. > >> >> marksweep.inline.hpp >> >> - MarkSweep::mark_object(): did you measure impact on other collectors >> using this code? It is enabled unconditionally. Also I would prefer >> that !UseG1GC automatically disables UseStringDeduplication (or use a >> global variable for that) to avoid the two checks. (The latter option is >> probably better). > > Yes, ran jbb2005/2013. > >> >> symbolTable.cpp: >> >> - again both UseG1GC and UseStringDedupliation are paired. Could this be >> put in a method/global variable somewhere that is set up correctly at >> startup? > > I think using the options is much more readable. Adding another state > which tells us if G1 & Dedup is enabled means we need a pre-init stage > to set up that state, which I don't think is very nice. > >> >> g1CollectedHeap.cpp: >> >> - StringDedup::initialize() is run unconditionally (the check for >> UseStringDeduplication is inside the method) while it's outside for >> everything else. Just a note, keep it if wanted. > > They are hidden inside dedup everywhere, except where the performance > overhead of a call might be an issue (like mark and copy). > >> >> TestStringDeduplicationTools.java: >> >> - maybe put retrieval of the unsafe object into the test library. >> Browsing through the code shows quite a lot copy+paste here already. >> >> - "qeueue" -> "queue" > > Check. > >> >> all tests: >> >> - missing bug ids > > Check. > >> >> From here it is mostly about comments/documentation: > > I agree that adding a feature overview and some additional comments is > probably a good idea. However, I also know that Bengt and StefanK > aren't too crazy about large comment blobs so I tried to kept the > noise down. I also disagree with some of the other comments below > which tend to be about personal taste rather than code quality or > breaking some coding convention we have, e.g. the static helpers, > readability, naming, etc. > > I'll add some comments and make some other changes and send out an > updated webrev shortly. > > cheers, > /Per > >> >> stringdedup.?pp: >> >> - is it possible to put an overview about how the string deduplication >> works here? There is absolutely no place in the code where e.g. the >> string deduplication cycle is explained. There is no comprehensive >> information about this feature anywhere, just bits about corner cases or >> particular here and there, but nothing that can be used as entry point >> to understand what this is actually doing. >> >> To some extent, the comments seem to be repetitions of what the code >> already shows. E.g. "some_constant = (1 << 10); // 1024 (must be power >> of 2)" or "// Store hash code in cache" followed by >> "java_lang_String::set_hash(java_string, hash);". >> >> There is also no reference to the JEP (I am not sure if it is customary >> to put references to outside documents - I would prefer some); however >> the JEP does not replace documentation in the source as it is not >> detailed enough, and the JEP cannot be changed any more. The JEP is >> interesting for the alternatives considered though. >> >> Ideally, to reasonably understand the feature I would like to not need >> to spend hours looking through the implementation. >> >> - how many threads does this feature add, and are supported by this code >> (for which parts)? Synchronization between them? >> >> - the comments "For XY" in the StringDedup header file do not give any >> hint about what they are doing (not sure if it is interesting as it is >> easy to find out the only callers). It seems that this information has >> been put at the place where these methods are invoked (or just somewhere >> in the cpp file). Imo the methods and members (except trivial) should be >> documented in the hpp file in the class body, i.e. in the interface. >> >> - there are a few methods (also in the StringDedupTable class) in >> StringDedup where the results of the methods do not seem obvious and >> should be commented. >> >> - I really dislike the location of the "Candidate selection" explanation >> within the cpp file. I would not have found it if I were not looking for >> some documentation ;) It would probably fit very well in the above >> mentioned overview paragraphs. >> >> - please add the static helper functions that implement the actual >> policies (is_candidate_from_mark, is_candidate_from_evacuation) to the >> StringDedup class (as private static methods) so that they can be found >> easily. Static non-class helper functions in cpp files should only be >> used for stuff that is really unimportant and not relevant to the code >> imo. However these seem to be _the_ central policy methods for string >> dedup. >> So it would be nice to have them mentioned in the class/hpp file and >> documented there. >> >> - facts about the code are repeated, e.g. that the table size must be a >> power of two. This is nice, but has a few drawbacks: the compiler does >> not care about comments at all, so it might be useful to put asserts in >> these places instead, and second, it's just duplication of text with the >> usual problem of getting out of date. >> >> - I am not completely sure why the class has not been called >> G1StringDedup. It won't work without G1 anyway and calls G1 internals >> all the time. There does not seem to be a way to make it more generic >> without considerable refactoring. It is fine to keep the name though, >> just asking. I also know that this feature does not want to be a generic >> solution. >> >> - is it possible to separate out the statistics variables of >> StringDedupTable into either a separate class instead of using globals >> or separate them a little from the other unrelated members? (And >> describe them) This would improve readability. >> >> I *think* this is everything from StringDedupTable::_entries_added to >> _rehash_count? I have no idea without searching for every use and >> deducing from that. >> >> - when implementing the hash table for the StringDedupTable, did you >> consider trying to reuse one of the existing hashmap implementations? >> And potentially extend them. >> >> Imo we already have way too many ad-hoc implementations of often used >> data structures (particularly) in G1 and GC code. >> >> - please document the properties of the StringDedupTable (type, hashing, >> automatic resizing behavior, thread safety, ...). >> >> - please explain what StringDedupTable/Cache are and what they are used >> for in context of the feature. >> >> - stringdedup.cpp, line 1309: candicate -> candidate >> >> Thanks, >> Thomas >> >> > From christian.thalinger at oracle.com Wed Mar 12 22:43:16 2014 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Wed, 12 Mar 2014 15:43:16 -0700 Subject: [8u20] RFR (XS) 8035983: Fix "Native frames:" in crash report (hs_err file) In-Reply-To: <5320D299.1030002@oracle.com> References: <5320D299.1030002@oracle.com> Message-ID: <964501D4-767E-4245-8EAF-FDF2DE50335B@oracle.com> Looks good. On Mar 12, 2014, at 2:33 PM, Vladimir Kozlov wrote: > Please, review backport. Changes were tested for a week in jdk9 sources. > > http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/e0365e7b7454 > https://bugs.openjdk.java.net/browse/JDK-8035983 > > thanks, > Vladimir From christian.thalinger at oracle.com Wed Mar 12 22:43:35 2014 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Wed, 12 Mar 2014 15:43:35 -0700 Subject: [8u20] RFR (XS): 8033805: Move Fast_Lock/Fast_Unlock code from .ad files to macroassembler In-Reply-To: <5320D332.2010105@oracle.com> References: <5320D332.2010105@oracle.com> Message-ID: Looks good. On Mar 12, 2014, at 2:35 PM, Vladimir Kozlov wrote: > Please, review backport to 8u20. Changes were tested for two weeks in jdk9 sources. > > https://bugs.openjdk.java.net/browse/JDK-8033805 > http://cr.openjdk.java.net/~kvn/8033805/webrev/ > http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/5292439ef895 > > Thanks, > Vladimir > From vladimir.kozlov at oracle.com Wed Mar 12 22:44:34 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 12 Mar 2014 15:44:34 -0700 Subject: [8u20] RFR (XS) 8035983: Fix "Native frames:" in crash report (hs_err file) In-Reply-To: <964501D4-767E-4245-8EAF-FDF2DE50335B@oracle.com> References: <5320D299.1030002@oracle.com> <964501D4-767E-4245-8EAF-FDF2DE50335B@oracle.com> Message-ID: <5320E352.8040600@oracle.com> Thanks, Christian Vladimir On 3/12/14 3:43 PM, Christian Thalinger wrote: > Looks good. > > On Mar 12, 2014, at 2:33 PM, Vladimir Kozlov wrote: > >> Please, review backport. Changes were tested for a week in jdk9 sources. >> >> http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/e0365e7b7454 >> https://bugs.openjdk.java.net/browse/JDK-8035983 >> >> thanks, >> Vladimir > From vladimir.kozlov at oracle.com Wed Mar 12 22:45:03 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 12 Mar 2014 15:45:03 -0700 Subject: [8u20] RFR (XS): 8033805: Move Fast_Lock/Fast_Unlock code from .ad files to macroassembler In-Reply-To: References: <5320D332.2010105@oracle.com> Message-ID: <5320E36F.807@oracle.com> Thanks, Christian Vladimir On 3/12/14 3:43 PM, Christian Thalinger wrote: > Looks good. > > On Mar 12, 2014, at 2:35 PM, Vladimir Kozlov wrote: > >> Please, review backport to 8u20. Changes were tested for two weeks in jdk9 sources. >> >> https://bugs.openjdk.java.net/browse/JDK-8033805 >> http://cr.openjdk.java.net/~kvn/8033805/webrev/ >> http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/5292439ef895 >> >> Thanks, >> Vladimir >> > From david.holmes at oracle.com Wed Mar 12 22:55:52 2014 From: david.holmes at oracle.com (David Holmes) Date: Thu, 13 Mar 2014 08:55:52 +1000 Subject: RFR (M): 8036767 PPC64: Support for little endian execution model In-Reply-To: References: <531E8309.2060206@oracle.com> <531ED06F.5040905@oracle.com> <4295855A5C1DE049A61835A1887419CC2CEA6A09@DEWDFEMB12A.global.corp.sap> <531F9370.4020305@oracle.com> Message-ID: <5320E5F8.3020500@oracle.com> On 12/03/2014 9:19 AM, Alexander Smundak wrote: > On Tue, Mar 11, 2014 at 3:51 PM, Vladimir Kozlov > wrote: >> It would only help if you could do cross compilation to have both build >> variants at the same place. Currently you can only build le variant on >> ppc64le machine and vice versa. That is why, I think, David asked if we can >> control what variant to build. > Just to clarify the situation a bit: ppc64le is not a variant of ppc64. That is, > an application compiled for the little-endian PowerPC64 does not "just run" on > the big-endian PowerPC64 (albeit OS can have such feature, similar to the > ability of the Linux running on x86_64 CPU to run 32-bit x86 applications). > So ppc64le is a different architecture from ppc64. I disagree with that classification for "architecture" and it seems at odds with the literature which describes the endian-ness selection as a "mode". David ----- >> I would like to see the changes based on Volker suggestion. We can compare >> them and decide which way to go. > Volker has the detailed suggestion here: > http://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/2014-March/001790.html > and it involves additional Make variable and if statements in the > platform makefile > where they are not supposed to be present. > From asmundak at google.com Wed Mar 12 23:59:03 2014 From: asmundak at google.com (Alexander Smundak) Date: Wed, 12 Mar 2014 16:59:03 -0700 Subject: RFR (M): 8036767 PPC64: Support for little endian execution model In-Reply-To: <5320E5F8.3020500@oracle.com> References: <531E8309.2060206@oracle.com> <531ED06F.5040905@oracle.com> <4295855A5C1DE049A61835A1887419CC2CEA6A09@DEWDFEMB12A.global.corp.sap> <531F9370.4020305@oracle.com> <5320E5F8.3020500@oracle.com> Message-ID: I was concerned by the term 'variant', which might suggest that the applications built for PPC64 and PPC64LE are binary compatible. They are not. On Wed, Mar 12, 2014 at 3:55 PM, David Holmes wrote: > On 12/03/2014 9:19 AM, Alexander Smundak wrote: >> >> On Tue, Mar 11, 2014 at 3:51 PM, Vladimir Kozlov >> wrote: >>> >>> It would only help if you could do cross compilation to have both build >>> variants at the same place. Currently you can only build le variant on >>> ppc64le machine and vice versa. That is why, I think, David asked if we >>> can >>> control what variant to build. >> >> Just to clarify the situation a bit: ppc64le is not a variant of ppc64. >> That is, >> an application compiled for the little-endian PowerPC64 does not "just >> run" on >> the big-endian PowerPC64 (albeit OS can have such feature, similar to the >> ability of the Linux running on x86_64 CPU to run 32-bit x86 >> applications). >> So ppc64le is a different architecture from ppc64. > > > I disagree with that classification for "architecture" and it seems at odds > with the literature which describes the endian-ness selection as a "mode". > > David > ----- > > >>> I would like to see the changes based on Volker suggestion. We can >>> compare >>> them and decide which way to go. >> >> Volker has the detailed suggestion here: >> >> http://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/2014-March/001790.html >> and it involves additional Make variable and if statements in the >> platform makefile >> where they are not supposed to be present. >> > From coleen.phillimore at oracle.com Thu Mar 13 00:31:04 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Wed, 12 Mar 2014 20:31:04 -0400 Subject: [8u20] RFR (XS) 8035983: Fix "Native frames:" in crash report (hs_err file) In-Reply-To: <5320D299.1030002@oracle.com> References: <5320D299.1030002@oracle.com> Message-ID: <5320FC48.9080802@oracle.com> Looks good. Coleen On 3/12/14 5:33 PM, Vladimir Kozlov wrote: > Please, review backport. Changes were tested for a week in jdk9 sources. > > http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/e0365e7b7454 > https://bugs.openjdk.java.net/browse/JDK-8035983 > > thanks, > Vladimir From vladimir.kozlov at oracle.com Thu Mar 13 00:35:11 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 12 Mar 2014 17:35:11 -0700 Subject: [8u20] RFR (XS) 8035983: Fix "Native frames:" in crash report (hs_err file) In-Reply-To: <5320FC48.9080802@oracle.com> References: <5320D299.1030002@oracle.com> <5320FC48.9080802@oracle.com> Message-ID: <5320FD3F.5010208@oracle.com> Thanks, Coleen Vladimir On 3/12/14 5:31 PM, Coleen Phillimore wrote: > > Looks good. > Coleen > > On 3/12/14 5:33 PM, Vladimir Kozlov wrote: >> Please, review backport. Changes were tested for a week in jdk9 sources. >> >> http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/e0365e7b7454 >> https://bugs.openjdk.java.net/browse/JDK-8035983 >> >> thanks, >> Vladimir > From david.holmes at oracle.com Thu Mar 13 05:04:11 2014 From: david.holmes at oracle.com (David Holmes) Date: Thu, 13 Mar 2014 15:04:11 +1000 Subject: RFR(L): 8036976: PPC64: implement the template interpreter In-Reply-To: <5320B491.3010103@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CEA69E2@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC2CEA6C1E@DEWDFEMB12A.global.corp.sap> <531F984A.7090805@oracle.com> <4295855A5C1DE049A61835A1887419CC2CEA744D@DEWDFEMB12A.global.corp.sap> <5320B491.3010103@oracle.com> Message-ID: <53213C4B.4090006@oracle.com> So we're assuming the ppc64le stuff will go in as a distinct architecture after all ??? In my opinion this fix should not have the ppc64le change - it should be added if/when the ppc64le stuff gets accepted and pushed. David On 13/03/2014 5:25 AM, Vladimir Kozlov wrote: > Looks fine to me. I will push it wiht me and Coleen as reviewers. > I assume I need to push it into ppc-aix 8u stage repo. Right? > > Thanks, > Vladimir > > On 3/12/14 4:04 AM, Lindenmaier, Goetz wrote: >> Hi Vladimir, >> >> thanks for the hint with the makefile -- that would have popped up >> soon after I guess. >> >> I fixed that and the Copyright, and updated the webrev. >> http://cr.openjdk.java.net/~goetz/webrevs/8036976-0-ti/ >> >> Best regards, >> Goetz. >> >> >> >> -----Original Message----- >> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] >> Sent: Mittwoch, 12. M?rz 2014 00:12 >> To: Lindenmaier, Goetz; 'hotspot-dev at openjdk.java.net'; >> 'ppc-aix-port-dev at openjdk.java.net' >> Subject: Re: RFR(L): 8036976: PPC64: implement the template interpreter >> >> Hi Goetz, >> >> Should you also check for ppc64le?: >> >> ifeq ($(ARCH), ppc64) >> FORCE_TIERED=0 >> >> Please, update year in Oracle copyright line in new files: >> >> Copyright (c) 1997, 2010, Oracle >> >> to >> >> Copyright (c) 2014, Oracle >> >> I know that you copied and modified from our existing files but HG >> history does not show it anyway and they all looks new. We should have >> done it for all ppc files but it is different issue. >> >> These changes passed JPRT build with our closed sources. >> >> Thanks, >> Vladimir >> >> On 3/11/14 2:37 PM, Lindenmaier, Goetz wrote: >>> Hi, >>> >>> I had missed all the new files in the webrev. I updated it with the >>> files in it. >>> >>> Sorry for this trouble, >>> Goetz. >>> >>> From: Lindenmaier, Goetz >>> Sent: Tuesday, March 11, 2014 10:13 AM >>> To: hotspot-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net >>> Subject: RFR(L): 8036976: PPC64: implement the template interpreter >>> >>> Hi, >>> >>> This time I have a really nice one: the template interpreter for ppc64! >>> Great thanks to my colleagues, especially Axel and Martin, who ported >>> this. >>> http://cr.openjdk.java.net/~goetz/webrevs/8036976-0-ti/ >>> >>> With this change, both interpreters can be used on linux_ppc64 and >>> aix_ppc64. >>> >>> This change almost only touches ppc platform files. >>> >>> I shared files, I needed to add a special case for PPC64 in >>> make/linux/Makefile, as I found no way to set FORCE_TIERED=0 in any >>> ppc file that's included above the use of FORCE_TIERED. As we don't >>> have C1, the current scheme does not work for ppc64. >>> >>> Actually, one should have lines as >>> supports_cppInterpreter = 1 >>> supports_c2 = 1 >>> supports_c1 = 0 >>> in platform_. But that's out of scope of this change. >>> >>> Further I add a #include in >>> src/share/vm/interpreter/templateTable.hpp. That's all for shared >>> changes. >>> Please review and test this change. >>> >>> Best regards, >>> Goetz. >>> From mikael.gerdin at oracle.com Thu Mar 13 08:57:09 2014 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Thu, 13 Mar 2014 09:57:09 +0100 Subject: RFR: 8036703: Add trace event with statistics for the metaspace chunk free lists In-Reply-To: <5320858A.8030302@oracle.com> References: <53184144.3060905@oracle.com> <5318544E.7050006@oracle.com> <5320858A.8030302@oracle.com> Message-ID: <1629610.a2STeJrlTr@mgerdin03> Hi Erik, On Wednesday 12 March 2014 17.04.26 Erik Helin wrote: > Based on feedback from Mikael and StefanK, I've reworked the patch to > put the ChunkFreeListSummaries in the class MetaspaceSummary. > > Please see the new version at: > http://cr.openjdk.java.net/~ehelin/8036703/webrev.03/ I think this version is much better! Just a small issue: collectedHeap.cpp 99 100 const MetaspaceChunkFreeListSummary& ms_chunk_free_list_summary = 101 MetaspaceAux::chunk_free_list_summary(Metaspace::ClassType); 102 const MetaspaceChunkFreeListSummary& class_chunk_free_list_summary = 103 MetaspaceAux::chunk_free_list_summary(Metaspace::NonClassType); 104 It looks like you pass the wrong MetadataType to chunk_free_list_summary here, ClassType and NonClassType should be swapped. /Mikael > > Thanks, > Erik > > On 2014-03-06 11:56, Erik Helin wrote: > > A small bug had found its way into this patch: > > > > metaspace.cpp: > > +bool MetaspaceAux::has_chunk_free_list(Metaspace::MetadataType mdtype) { > > + return Metaspace::get_chunk_manager(mdtype) == NULL; > > +} > > > > should of course be: > > > > +bool MetaspaceAux::has_chunk_free_list(Metaspace::MetadataType mdtype) { > > + return Metaspace::get_chunk_manager(mdtype) != NULL; > > +} > > > > Please see new webrev at: > > http://cr.openjdk.java.net/~ehelin/8036703/webrev.01/ > > > > Thanks, > > Erik > > > > On 2014-03-06 10:35, Erik Helin wrote: > >> Hi all, > >> > >> this patch adds a new trace event - > >> vm/gc/metaspace/chunk_free_list_summary for getting statistics for > >> metaspace's chunk free lists. The event contains the numbers of chunks > >> in the free list and the size of the chunks in the free lists. This > >> information is provided for all different kinds of chunks - special, > >> small, medium and humongous. > >> > >> This information will be useful when diagnosing fragmentation issues in > >> metaspace. > >> > >> Webrev: > >> http://cr.openjdk.java.net/~ehelin/8036703/webrev.00/ > >> > >> Enhancement: > >> https://bugs.openjdk.java.net/browse/JDK-8036703 > >> > >> Testing: > >> - JFR JTREG tests > >> - JPRT > >> - Manually verifying that the numbers look correct by looking at > >> > >> recordings from small test cases. > >> > >> Thanks, > >> Erik From stefan.karlsson at oracle.com Thu Mar 13 09:40:25 2014 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 13 Mar 2014 10:40:25 +0100 Subject: RFR: 8036703: Add trace event with statistics for the metaspace chunk free lists In-Reply-To: <1629610.a2STeJrlTr@mgerdin03> References: <53184144.3060905@oracle.com> <5318544E.7050006@oracle.com> <5320858A.8030302@oracle.com> <1629610.a2STeJrlTr@mgerdin03> Message-ID: <53217D09.2010009@oracle.com> On 2014-03-13 09:57, Mikael Gerdin wrote: > Hi Erik, > > On Wednesday 12 March 2014 17.04.26 Erik Helin wrote: >> Based on feedback from Mikael and StefanK, I've reworked the patch to >> put the ChunkFreeListSummaries in the class MetaspaceSummary. >> >> Please see the new version at: >> http://cr.openjdk.java.net/~ehelin/8036703/webrev.03/ > I think this version is much better! > Just a small issue: > > collectedHeap.cpp > 99 > 100 const MetaspaceChunkFreeListSummary& ms_chunk_free_list_summary = > 101 MetaspaceAux::chunk_free_list_summary(Metaspace::ClassType); > 102 const MetaspaceChunkFreeListSummary& class_chunk_free_list_summary = > 103 MetaspaceAux::chunk_free_list_summary(Metaspace::NonClassType); > 104 > It looks like you pass the wrong MetadataType to chunk_free_list_summary here, > ClassType and NonClassType should be swapped. Except for this bug, this looks good. thanks, StefanK > > /Mikael > >> Thanks, >> Erik >> >> On 2014-03-06 11:56, Erik Helin wrote: >>> A small bug had found its way into this patch: >>> >>> metaspace.cpp: >>> +bool MetaspaceAux::has_chunk_free_list(Metaspace::MetadataType mdtype) { >>> + return Metaspace::get_chunk_manager(mdtype) == NULL; >>> +} >>> >>> should of course be: >>> >>> +bool MetaspaceAux::has_chunk_free_list(Metaspace::MetadataType mdtype) { >>> + return Metaspace::get_chunk_manager(mdtype) != NULL; >>> +} >>> >>> Please see new webrev at: >>> http://cr.openjdk.java.net/~ehelin/8036703/webrev.01/ >>> >>> Thanks, >>> Erik >>> >>> On 2014-03-06 10:35, Erik Helin wrote: >>>> Hi all, >>>> >>>> this patch adds a new trace event - >>>> vm/gc/metaspace/chunk_free_list_summary for getting statistics for >>>> metaspace's chunk free lists. The event contains the numbers of chunks >>>> in the free list and the size of the chunks in the free lists. This >>>> information is provided for all different kinds of chunks - special, >>>> small, medium and humongous. >>>> >>>> This information will be useful when diagnosing fragmentation issues in >>>> metaspace. >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~ehelin/8036703/webrev.00/ >>>> >>>> Enhancement: >>>> https://bugs.openjdk.java.net/browse/JDK-8036703 >>>> >>>> Testing: >>>> - JFR JTREG tests >>>> - JPRT >>>> - Manually verifying that the numbers look correct by looking at >>>> >>>> recordings from small test cases. >>>> >>>> Thanks, >>>> Erik From roland.westrelin at oracle.com Thu Mar 13 10:17:02 2014 From: roland.westrelin at oracle.com (Roland Westrelin) Date: Thu, 13 Mar 2014 11:17:02 +0100 Subject: [8u20] RFR (XS): 8033805: Move Fast_Lock/Fast_Unlock code from .ad files to macroassembler In-Reply-To: <5320D332.2010105@oracle.com> References: <5320D332.2010105@oracle.com> Message-ID: <609656F4-D911-4147-81CE-B25A00F9BFE3@oracle.com> Looks good. Roland. From martijnverburg at gmail.com Thu Mar 13 10:36:41 2014 From: martijnverburg at gmail.com (Martijn Verburg) Date: Thu, 13 Mar 2014 10:36:41 +0000 Subject: AArch64: We have a release In-Reply-To: <53209B6F.60009@redhat.com> References: <53208742.3070309@redhat.com> <53209B6F.60009@redhat.com> Message-ID: Many congratulations - that's two/three large ports released over the last couple of years - great to see the JVM getting first class support on new platforms! Cheers, Martijn On 12 March 2014 17:37, Andrew Haley wrote: > On 03/12/2014 05:33 PM, Volker Simonis wrote: > > Did you pass the Java 7 and Java 8 TCK? > > Hmmm, good question. Java 7, certainly. Java 8, Ill have to check. > > Andrew. > > From erik.helin at oracle.com Thu Mar 13 13:00:33 2014 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 13 Mar 2014 14:00:33 +0100 Subject: RFR: 8036703: Add trace event with statistics for the metaspace chunk free lists In-Reply-To: <53217D09.2010009@oracle.com> References: <53184144.3060905@oracle.com> <5318544E.7050006@oracle.com> <5320858A.8030302@oracle.com> <1629610.a2STeJrlTr@mgerdin03> <53217D09.2010009@oracle.com> Message-ID: <5321ABF1.9030403@oracle.com> Hi Mikael and Stefan, thanks for having a look and special thanks to Mikael for finding the bug! Please see new webrev at: http://cr.openjdk.java.net/~ehelin/8036703/webrev.04/ Please see the incremental webrev at: http://cr.openjdk.java.net/~ehelin/8036703/webrev.03-04/ Based on some internal feedback, I also corrected a few nits (see the incremental webrev for details). Thanks, Erik On 2014-03-13 10:40, Stefan Karlsson wrote: > > On 2014-03-13 09:57, Mikael Gerdin wrote: >> Hi Erik, >> >> On Wednesday 12 March 2014 17.04.26 Erik Helin wrote: >>> Based on feedback from Mikael and StefanK, I've reworked the patch to >>> put the ChunkFreeListSummaries in the class MetaspaceSummary. >>> >>> Please see the new version at: >>> http://cr.openjdk.java.net/~ehelin/8036703/webrev.03/ >> I think this version is much better! >> Just a small issue: >> >> collectedHeap.cpp >> 99 >> 100 const MetaspaceChunkFreeListSummary& ms_chunk_free_list_summary = >> 101 MetaspaceAux::chunk_free_list_summary(Metaspace::ClassType); >> 102 const MetaspaceChunkFreeListSummary& >> class_chunk_free_list_summary = >> 103 MetaspaceAux::chunk_free_list_summary(Metaspace::NonClassType); >> 104 >> It looks like you pass the wrong MetadataType to >> chunk_free_list_summary here, >> ClassType and NonClassType should be swapped. > > Except for this bug, this looks good. > > thanks, > StefanK > >> >> /Mikael >> >>> Thanks, >>> Erik >>> >>> On 2014-03-06 11:56, Erik Helin wrote: >>>> A small bug had found its way into this patch: >>>> >>>> metaspace.cpp: >>>> +bool MetaspaceAux::has_chunk_free_list(Metaspace::MetadataType >>>> mdtype) { >>>> + return Metaspace::get_chunk_manager(mdtype) == NULL; >>>> +} >>>> >>>> should of course be: >>>> >>>> +bool MetaspaceAux::has_chunk_free_list(Metaspace::MetadataType >>>> mdtype) { >>>> + return Metaspace::get_chunk_manager(mdtype) != NULL; >>>> +} >>>> >>>> Please see new webrev at: >>>> http://cr.openjdk.java.net/~ehelin/8036703/webrev.01/ >>>> >>>> Thanks, >>>> Erik >>>> >>>> On 2014-03-06 10:35, Erik Helin wrote: >>>>> Hi all, >>>>> >>>>> this patch adds a new trace event - >>>>> vm/gc/metaspace/chunk_free_list_summary for getting statistics for >>>>> metaspace's chunk free lists. The event contains the numbers of chunks >>>>> in the free list and the size of the chunks in the free lists. This >>>>> information is provided for all different kinds of chunks - special, >>>>> small, medium and humongous. >>>>> >>>>> This information will be useful when diagnosing fragmentation >>>>> issues in >>>>> metaspace. >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~ehelin/8036703/webrev.00/ >>>>> >>>>> Enhancement: >>>>> https://bugs.openjdk.java.net/browse/JDK-8036703 >>>>> >>>>> Testing: >>>>> - JFR JTREG tests >>>>> - JPRT >>>>> - Manually verifying that the numbers look correct by looking at >>>>> >>>>> recordings from small test cases. >>>>> >>>>> Thanks, >>>>> Erik > From erik.helin at oracle.com Thu Mar 13 13:06:27 2014 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 13 Mar 2014 14:06:27 +0100 Subject: RFR: 8036703: Add trace event with statistics for the metaspace chunk free lists In-Reply-To: <5320CD33.2070603@oracle.com> References: <53184144.3060905@oracle.com> <5318544E.7050006@oracle.com> <5320858A.8030302@oracle.com> <5320CD33.2070603@oracle.com> Message-ID: <5321AD53.805@oracle.com> Hi Coleen, thanks for having a look! On 2014-03-12 22:10, Coleen Phillimore wrote: > Wow, that looks good. Thanks for your kind words :) On 2014-03-12 22:10, Coleen Phillimore wrote: > The only comment I had was that this macro for > index index_bounds_check in metaspace.cpp could be an inlined static > function, then you could avoid the odd while loop. I use a macro because I want to get the correct line in the assertion failure in an eventual hs_err file. Using a static inline function would only give me the line numbers of the static inline function. Since we don't always get a stack trace in the hs_err file, this would make it hard to determine if the assertion failure comes from size_free_chunks_in_bytes or num_free_chunks. You can see the updated macro in the incremental webrev (see metaspace.cpp): http://cr.openjdk.java.net/~ehelin/8036703/webrev.03-04/ What do you think? Thanks, Erik > Coleen > > On 3/12/14 12:04 PM, Erik Helin wrote: >> Based on feedback from Mikael and StefanK, I've reworked the patch to >> put the ChunkFreeListSummaries in the class MetaspaceSummary. >> >> Please see the new version at: >> http://cr.openjdk.java.net/~ehelin/8036703/webrev.03/ >> >> Thanks, >> Erik >> >> On 2014-03-06 11:56, Erik Helin wrote: >>> A small bug had found its way into this patch: >>> >>> metaspace.cpp: >>> +bool MetaspaceAux::has_chunk_free_list(Metaspace::MetadataType >>> mdtype) { >>> + return Metaspace::get_chunk_manager(mdtype) == NULL; >>> +} >>> >>> should of course be: >>> >>> +bool MetaspaceAux::has_chunk_free_list(Metaspace::MetadataType >>> mdtype) { >>> + return Metaspace::get_chunk_manager(mdtype) != NULL; >>> +} >>> >>> Please see new webrev at: >>> http://cr.openjdk.java.net/~ehelin/8036703/webrev.01/ >>> >>> Thanks, >>> Erik >>> >>> On 2014-03-06 10:35, Erik Helin wrote: >>>> Hi all, >>>> >>>> this patch adds a new trace event - >>>> vm/gc/metaspace/chunk_free_list_summary for getting statistics for >>>> metaspace's chunk free lists. The event contains the numbers of chunks >>>> in the free list and the size of the chunks in the free lists. This >>>> information is provided for all different kinds of chunks - special, >>>> small, medium and humongous. >>>> >>>> This information will be useful when diagnosing fragmentation issues in >>>> metaspace. >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~ehelin/8036703/webrev.00/ >>>> >>>> Enhancement: >>>> https://bugs.openjdk.java.net/browse/JDK-8036703 >>>> >>>> Testing: >>>> - JFR JTREG tests >>>> - JPRT >>>> - Manually verifying that the numbers look correct by looking at >>>> recordings from small test cases. >>>> >>>> Thanks, >>>> Erik > From mattis.castegren at oracle.com Thu Mar 13 14:02:19 2014 From: mattis.castegren at oracle.com (Mattis Castegren) Date: Thu, 13 Mar 2014 07:02:19 -0700 (PDT) Subject: RFR: 8036863: Update jdk7 testlibrary to match jdk8 in hotspot In-Reply-To: <5319AB51.7030704@oracle.com> References: <5319AB51.7030704@oracle.com> Message-ID: <2f0413d1-5037-4876-bc19-084b4e90885d@default> Hi Could we have someone review this change. This is just a backport from JDK8, but there were some small changes so we want to get a proper review here. We have several other backports waiting for this change (fixes using the new test libraries), so it would be good to get this going. Kind Regards /Mattis -----Original Message----- From: Vladimir Kempik Sent: den 7 mars 2014 12:20 To: hotspot-dev at openjdk.java.net Cc: Mattis Castegren Subject: RFR: 8036863: Update jdk7 testlibrary to match jdk8 in hotspot Please review this change to update hotspot part of testlibrary in jdk7. It updates testlibrary with new apis from jdk8's testlibrary. Every test in open part of hotspot that uses testlibrary and was working fine before update still works after update. Webrev: http://cr.openjdk.java.net/~vkempik/8036863/webrev.00/ Thanks, Vladimir. From christian.tornqvist at oracle.com Thu Mar 13 14:30:11 2014 From: christian.tornqvist at oracle.com (Christian Tornqvist) Date: Thu, 13 Mar 2014 10:30:11 -0400 Subject: 8036863: Update jdk7 testlibrary to match jdk8 in hotspot In-Reply-To: <5319AB51.7030704@oracle.com> References: <5319AB51.7030704@oracle.com> Message-ID: <014f01cf3ec8$c0528ad0$40f7a070$@oracle.com> Hi Vladimir, You might want to include the tests in \test\testlibrary_tests as well. Otherwise it looks good. Thanks, Christian -----Original Message----- From: hotspot-dev-bounces at openjdk.java.net [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Vladimir Kempik Sent: Friday, March 7, 2014 6:20 AM To: hotspot-dev at openjdk.java.net Subject: RFR: 8036863: Update jdk7 testlibrary to match jdk8 in hotspot Please review this change to update hotspot part of testlibrary in jdk7. It updates testlibrary with new apis from jdk8's testlibrary. Every test in open part of hotspot that uses testlibrary and was working fine before update still works after update. Webrev: http://cr.openjdk.java.net/~vkempik/8036863/webrev.00/ Thanks, Vladimir. From thomas.stuefe at gmail.com Thu Mar 13 14:45:50 2014 From: thomas.stuefe at gmail.com (=?ISO-8859-1?Q?Thomas_St=FCfe?=) Date: Thu, 13 Mar 2014 15:45:50 +0100 Subject: NMT shutdown in VMError::report_and_die() Message-ID: Greetings, I am with the SAP JVM; right now I am trying to consolidate our error handling code a bit (bringing our coding closer to OpenJDK). I have a small question about shutting down NMT in VMError::report_and_die(): VMError::report_and_die() is called recursivly in case of secondary crashes. NMT shutdown is done at the beginning of the function. If NMT shutdown itself crashes, we will have infinite recursion, resulting in a stack overflow and no hs-err file. Would it not be better to do MemTracker::shutdown() only for the first recursion level? By moving it down some lines into the "if (first_error == NULL && Atomic::cmpxchg_ptr(this, &first_error, NULL) == NULL)" condition? Kind regards, Thomas From coleen.phillimore at oracle.com Thu Mar 13 15:00:42 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Thu, 13 Mar 2014 11:00:42 -0400 Subject: RFR: 8036703: Add trace event with statistics for the metaspace chunk free lists In-Reply-To: <5321AD53.805@oracle.com> References: <53184144.3060905@oracle.com> <5318544E.7050006@oracle.com> <5320858A.8030302@oracle.com> <5320CD33.2070603@oracle.com> <5321AD53.805@oracle.com> Message-ID: <5321C81A.7080700@oracle.com> Erik, On 3/13/14 9:06 AM, Erik Helin wrote: > Hi Coleen, > > thanks for having a look! > > On 2014-03-12 22:10, Coleen Phillimore wrote: >> Wow, that looks good. > > Thanks for your kind words :) > > On 2014-03-12 22:10, Coleen Phillimore wrote: >> The only comment I had was that this macro for >> index index_bounds_check in metaspace.cpp could be an inlined static >> function, then you could avoid the odd while loop. > > I use a macro because I want to get the correct line in the assertion > failure in an eventual hs_err file. Using a static inline function > would only give me the line numbers of the static inline function. I see. That makes sense. The revised macro looks fine. Coleen > > Since we don't always get a stack trace in the hs_err file, this would > make it hard to determine if the assertion failure comes from > size_free_chunks_in_bytes or num_free_chunks. > > You can see the updated macro in the incremental webrev (see > metaspace.cpp): > http://cr.openjdk.java.net/~ehelin/8036703/webrev.03-04/ > > What do you think? > > Thanks, > Erik > >> Coleen >> >> On 3/12/14 12:04 PM, Erik Helin wrote: >>> Based on feedback from Mikael and StefanK, I've reworked the patch to >>> put the ChunkFreeListSummaries in the class MetaspaceSummary. >>> >>> Please see the new version at: >>> http://cr.openjdk.java.net/~ehelin/8036703/webrev.03/ >>> >>> Thanks, >>> Erik >>> >>> On 2014-03-06 11:56, Erik Helin wrote: >>>> A small bug had found its way into this patch: >>>> >>>> metaspace.cpp: >>>> +bool MetaspaceAux::has_chunk_free_list(Metaspace::MetadataType >>>> mdtype) { >>>> + return Metaspace::get_chunk_manager(mdtype) == NULL; >>>> +} >>>> >>>> should of course be: >>>> >>>> +bool MetaspaceAux::has_chunk_free_list(Metaspace::MetadataType >>>> mdtype) { >>>> + return Metaspace::get_chunk_manager(mdtype) != NULL; >>>> +} >>>> >>>> Please see new webrev at: >>>> http://cr.openjdk.java.net/~ehelin/8036703/webrev.01/ >>>> >>>> Thanks, >>>> Erik >>>> >>>> On 2014-03-06 10:35, Erik Helin wrote: >>>>> Hi all, >>>>> >>>>> this patch adds a new trace event - >>>>> vm/gc/metaspace/chunk_free_list_summary for getting statistics for >>>>> metaspace's chunk free lists. The event contains the numbers of >>>>> chunks >>>>> in the free list and the size of the chunks in the free lists. This >>>>> information is provided for all different kinds of chunks - special, >>>>> small, medium and humongous. >>>>> >>>>> This information will be useful when diagnosing fragmentation >>>>> issues in >>>>> metaspace. >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~ehelin/8036703/webrev.00/ >>>>> >>>>> Enhancement: >>>>> https://bugs.openjdk.java.net/browse/JDK-8036703 >>>>> >>>>> Testing: >>>>> - JFR JTREG tests >>>>> - JPRT >>>>> - Manually verifying that the numbers look correct by looking at >>>>> recordings from small test cases. >>>>> >>>>> Thanks, >>>>> Erik >> From per.liden at oracle.com Thu Mar 13 15:02:59 2014 From: per.liden at oracle.com (Per Liden) Date: Thu, 13 Mar 2014 16:02:59 +0100 Subject: RFR(L): 8029075 - String deduplication in G1 In-Reply-To: <5320DB4B.2020801@oracle.com> References: <53148494.60806@oracle.com> <5316F5B1.60006@oracle.com> <5319D424.90306@oracle.com> <1394541591.2696.20.camel@cirrus> <5320626E.7020102@oracle.com> <5320DB4B.2020801@oracle.com> Message-ID: <5321C8A3.70206@oracle.com> Hi Coleen, Thanks for looking at the patch! Comment below. On 2014-03-12 23:10, Coleen Phillimore wrote: > > Hi Per, > > Why not add the call for StringDedup::unlink() inside the function > unlink_string_and_symbol_table rather than just above in two places, > since they go together. > > + // Delete dead entries in string deduplication queue/table > + StringDedup::unlink(&GenMarkSweep::is_alive); > + > // Delete entries for dead interned string and clean up > unreferenced symbols in symbol table. > G1CollectedHeap::heap()->unlink_string_and_symbol_table(&GenMarkSweep::is_alive); Yep, I'll move that call into unlink_string_and_symbol_table(). The unlink_string_and_symbol_table() is a fairly new addition, there used to be StringTable/SymbolTable calls there when I added the StrDedup call, that's why it looks like that. > > > I didn't see anything else specific to change. Some general comments > which are similar to Thomas's. There seems to be a lack of comments > in the deduplication file. I hate huge block comments too but some > comments about what it does in general would be nice. Like 3 sentences > per class. The file is overwhelming (ie, I completely lose interest) > with the statistics printing at the front. Maybe you could put the > most interesting classes at the front of the file? Maybe statistics > printing could be a separate file. > > One of the conventions that we have is that accessors for classes are > on one line, so you could make the file smaller by doing that > reformatting. > > It is also a general convention to have comments above the declaration > of a class to say generally what it's for, when it's used, etc. It's > not easy to tell by the name or we could guess wrong. What is > StringDedupQueue for, eg? That would make reviewing the code easier. Agree, I'll be adding comments to each class and a general description of what dedup does. > > There seems to be a lot of duplication of code with the hashtable in > hashtable.hpp, only different. We have lots of hashtables in > hotspot. Did you try to reinstantiate this hashtable and why didn't > that not work? There are interesting memory allocation functions for > the entries in this normal hashtable that might help with > fragmentation, for instance. Very early on I actually did use an instance of Hashtable and even played with the idea of having a combined StringTable/DedupTable. I decided not to use that though, mostly because I felt I would need to make too many modifications to non-dedup related code and because Hashtable<> carries concepts, like shared entries, which didn't quite fit with dedup. Using the existing StringTable would mean it would have to have logic to manage entries which point to either Strings or char[], which again seemed like an too intrusive change. The Hashtable allocation strategy (with allocating blocks of entries) seems nice from a performance perspective, but freeing those becomes a bit of a challenge, which I guess is why it never frees any entries. Since the dedup table is there to in the end save memory, I felt I needed better control over the table's memory usage. At one point I actually started writing a WeakOopsHashTable<>, which I hoped could serve as the base for both StringTable and DedupTable, but again I felt I was losing focus a bit as it would mean modifying large parts of non-GC and non-dedeup related stuff. The potential connection to CDS here also made me a bit scared of going in that direction, as I don't have a complete understanding of the implications of that. > > Lastly, why is the hash code in the string written to and used? The > StringTable uses the same hash algorithm until it has to change the > hash code, but the new hash code isn't stored in the string because it > would be incompatible. I think StringTable and the dedup table is doing the same thing here. The hash code in the String object is only used/updated in the case we're using the standard/compatible hash function. As soon as the table is rehashed and switches to another hash function it no longer updates the hash code in the String. use_java_hash() tells use if we're using the standard/compatible hash and it's used here: if (use_java_hash()) { // Get hash code from cache hash = java_lang_String::hash(java_string); } and later here: if (use_java_hash() && hash != 0) { // Store hash code in cache java_lang_String::set_hash(java_string, hash); } Concerns? > > I would like to see this be a few different files. It seems like > this file does three things conceptually, interacts with G1 somehow to > determine which strings are candidates for deduplication, stores these > things in a rehashable, resizable hash table and gathers statistics. > > I'm sorry to have so many comments so late and I haven't really tried > to read the code for correctness. I think some reformatting and > reorganization would make the reviewers job easier. It's a big change > and worthwhile. Great, I've no problems splitting this up. I'm more of a "one class per file" person myself, but I had the (apparently incorrect) impression that wasn't the hotspot way. I'm thinking I'll split this up into the following files (while I'm at it I'll also add the G1 prefix as Thomas suggested): g1StringDedup - Main interface for the GC, would contain entry point and things like the task and closure classes g1StringDedupTable - The table, cache and entry classes g1StringDedupQueue - The queue g1StringDedupStat - The statistics sutff g1StringDedupThread - The dedup thread Sounds good? Will send out and updated webrev. Thanks, /Per > > Thanks, > Coleen > > > On 3/12/14 9:34 AM, Per Liden wrote: >> Hi Thomas, >> >> Thanks for reviewing. Comments inline. >> >> On 03/11/2014 01:39 PM, Thomas Schatzl wrote: >>> Hi Per, >>> >>> >>> On Fri, 2014-03-07 at 15:13 +0100, Per Liden wrote: >>>> Hi, >>>> >>>> Here's an updated webrev based on the feedback from Bengt and Thomas. >>>> >>>> http://cr.openjdk.java.net/~pliden/8029075/webrev.1/ >>>> >>> >>> See further below. >>> >>>> >>>> And as suggested, two changes in the initial webrev were broken out >>>> into >>>> separate CRs, available here: >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8036672 >>>> Webrev: http://cr.openjdk.java.net/~pliden/8036672/webrev.0/ >>> >>> Looks good. >>> >>> Did you check the impact on something like specjbb2005 which does >>> nothing but object copying? That code is somewhat performance >>> sensitive. >> >> Yes, I did Aurora runs with jbb2005 and jbb2013, didn't see any >> difference. >> >>> >>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8036673 >>>> Webrev: http://cr.openjdk.java.net/~pliden/8036673/webrev.0/ >>> >>> Looks good. >>> >>> Here are my comments: there are a few minor issues, and a single major: >>> basically the documentation for this feature is lacking imo. I listed a >>> few particular issues of mine with it. >>> >>> First the minor issues: >>> >>> - could you check on copyright dates on the modified files? They have >>> not been updated anywhere afaics. >>> >>> - not sure if good, but maybe make PrintStringDeduplicationStatistics >>> diagnostic? It seems to really be some diagnosis functions. >>> >>> - maybe make the new *ALot flags notproduct. These seem to be debugging >>> helpers similar to other *ALot flags. >> >> I had a conversion with Bengt about this some time ago and I think >> the options are categorized correctly. The ALot flags needs be >> available in product builds because they are used by tests. Most >> Print*Statistics are product therefore it makes sense to follow that >> pattern for PrintStringDeduplicationStatistics. >> >>> >>> - StringDeduplicationAgeThreshold: maybe also needs to mention that if >>> the object is promoted, that it is always considered as candidate. >> >> Check. >> >>> >>> marksweep.inline.hpp >>> >>> - MarkSweep::mark_object(): did you measure impact on other collectors >>> using this code? It is enabled unconditionally. Also I would prefer >>> that !UseG1GC automatically disables UseStringDeduplication (or use a >>> global variable for that) to avoid the two checks. (The latter >>> option is >>> probably better). >> >> Yes, ran jbb2005/2013. >> >>> >>> symbolTable.cpp: >>> >>> - again both UseG1GC and UseStringDedupliation are paired. Could >>> this be >>> put in a method/global variable somewhere that is set up correctly at >>> startup? >> >> I think using the options is much more readable. Adding another state >> which tells us if G1 & Dedup is enabled means we need a pre-init >> stage to set up that state, which I don't think is very nice. >> >>> >>> g1CollectedHeap.cpp: >>> >>> - StringDedup::initialize() is run unconditionally (the check for >>> UseStringDeduplication is inside the method) while it's outside for >>> everything else. Just a note, keep it if wanted. >> >> They are hidden inside dedup everywhere, except where the performance >> overhead of a call might be an issue (like mark and copy). >> >>> >>> TestStringDeduplicationTools.java: >>> >>> - maybe put retrieval of the unsafe object into the test library. >>> Browsing through the code shows quite a lot copy+paste here already. >>> >>> - "qeueue" -> "queue" >> >> Check. >> >>> >>> all tests: >>> >>> - missing bug ids >> >> Check. >> >>> >>> From here it is mostly about comments/documentation: >> >> I agree that adding a feature overview and some additional comments >> is probably a good idea. However, I also know that Bengt and StefanK >> aren't too crazy about large comment blobs so I tried to kept the >> noise down. I also disagree with some of the other comments below >> which tend to be about personal taste rather than code quality or >> breaking some coding convention we have, e.g. the static helpers, >> readability, naming, etc. >> >> I'll add some comments and make some other changes and send out an >> updated webrev shortly. >> >> cheers, >> /Per >> >>> >>> stringdedup.?pp: >>> >>> - is it possible to put an overview about how the string deduplication >>> works here? There is absolutely no place in the code where e.g. the >>> string deduplication cycle is explained. There is no comprehensive >>> information about this feature anywhere, just bits about corner >>> cases or >>> particular here and there, but nothing that can be used as entry point >>> to understand what this is actually doing. >>> >>> To some extent, the comments seem to be repetitions of what the code >>> already shows. E.g. "some_constant = (1 << 10); // 1024 (must be power >>> of 2)" or "// Store hash code in cache" followed by >>> "java_lang_String::set_hash(java_string, hash);". >>> >>> There is also no reference to the JEP (I am not sure if it is customary >>> to put references to outside documents - I would prefer some); however >>> the JEP does not replace documentation in the source as it is not >>> detailed enough, and the JEP cannot be changed any more. The JEP is >>> interesting for the alternatives considered though. >>> >>> Ideally, to reasonably understand the feature I would like to not need >>> to spend hours looking through the implementation. >>> >>> - how many threads does this feature add, and are supported by this >>> code >>> (for which parts)? Synchronization between them? >>> >>> - the comments "For XY" in the StringDedup header file do not give any >>> hint about what they are doing (not sure if it is interesting as it is >>> easy to find out the only callers). It seems that this information has >>> been put at the place where these methods are invoked (or just >>> somewhere >>> in the cpp file). Imo the methods and members (except trivial) >>> should be >>> documented in the hpp file in the class body, i.e. in the interface. >>> >>> - there are a few methods (also in the StringDedupTable class) in >>> StringDedup where the results of the methods do not seem obvious and >>> should be commented. >>> >>> - I really dislike the location of the "Candidate selection" >>> explanation >>> within the cpp file. I would not have found it if I were not looking >>> for >>> some documentation ;) It would probably fit very well in the above >>> mentioned overview paragraphs. >>> >>> - please add the static helper functions that implement the actual >>> policies (is_candidate_from_mark, is_candidate_from_evacuation) to the >>> StringDedup class (as private static methods) so that they can be found >>> easily. Static non-class helper functions in cpp files should only be >>> used for stuff that is really unimportant and not relevant to the code >>> imo. However these seem to be _the_ central policy methods for string >>> dedup. >>> So it would be nice to have them mentioned in the class/hpp file and >>> documented there. >>> >>> - facts about the code are repeated, e.g. that the table size must be a >>> power of two. This is nice, but has a few drawbacks: the compiler does >>> not care about comments at all, so it might be useful to put asserts in >>> these places instead, and second, it's just duplication of text with >>> the >>> usual problem of getting out of date. >>> >>> - I am not completely sure why the class has not been called >>> G1StringDedup. It won't work without G1 anyway and calls G1 internals >>> all the time. There does not seem to be a way to make it more generic >>> without considerable refactoring. It is fine to keep the name though, >>> just asking. I also know that this feature does not want to be a >>> generic >>> solution. >>> >>> - is it possible to separate out the statistics variables of >>> StringDedupTable into either a separate class instead of using globals >>> or separate them a little from the other unrelated members? (And >>> describe them) This would improve readability. >>> >>> I *think* this is everything from StringDedupTable::_entries_added to >>> _rehash_count? I have no idea without searching for every use and >>> deducing from that. >>> >>> - when implementing the hash table for the StringDedupTable, did you >>> consider trying to reuse one of the existing hashmap implementations? >>> And potentially extend them. >>> >>> Imo we already have way too many ad-hoc implementations of often used >>> data structures (particularly) in G1 and GC code. >>> >>> - please document the properties of the StringDedupTable (type, >>> hashing, >>> automatic resizing behavior, thread safety, ...). >>> >>> - please explain what StringDedupTable/Cache are and what they are used >>> for in context of the feature. >>> >>> - stringdedup.cpp, line 1309: candicate -> candidate >>> >>> Thanks, >>> Thomas >>> >>> >> > From volker.simonis at gmail.com Thu Mar 13 15:04:33 2014 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 13 Mar 2014 16:04:33 +0100 Subject: RFR[9](XS): 8037298: Export HotSpots 'optimized' (i.e. not-product) configuration in the top-level configure/makefile Message-ID: Hi, could somebody please review this small change which exports HotSpots 'optimized' (i.e. not-product) configuration in the top-level configure/makefile: http://cr.openjdk.java.net/~simonis/webrevs/8037298/ https://bugs.openjdk.java.net/browse/JDK-8037298 The HotSpot offers a so called 'optimized' configuration which is actually the same as a product build, but without setting '-DPRODUCT'. That means that an 'optimized' HotSpot VM offers additional command line options and functionality which is not present in an actual product build. While the 'product' configuration is exported in the top-level configure/make system as "--with-debug-level=release" the , this was not the case for the 'optimized' configuration. I think I need a sponsor for this change because it requires the regeneration of common/autoconf/generated-configure.sh as well as of the closed generated configure files. Thank you and best regards, Volker From zhengyu.gu at oracle.com Thu Mar 13 15:47:15 2014 From: zhengyu.gu at oracle.com (Zhengyu Gu) Date: Thu, 13 Mar 2014 11:47:15 -0400 Subject: NMT shutdown in VMError::report_and_die() In-Reply-To: References: Message-ID: <5321D303.2070203@oracle.com> Hi Thomas, Your suggestion does make sense and it is better. We are addressing several NMT issues, one of them is to eliminate this shutdown call, so NMT can report memory statistics as part of error reporting. Thanks, -Zhengyu On 3/13/2014 10:45 AM, Thomas St?fe wrote: > Greetings, > > I am with the SAP JVM; right now I am trying to consolidate our error > handling code a bit (bringing our coding closer to OpenJDK). > > I have a small question about shutting down NMT in > VMError::report_and_die(): > > VMError::report_and_die() is called recursivly in case of secondary > crashes. NMT shutdown is done at the beginning of the function. If NMT > shutdown itself crashes, we will have infinite recursion, resulting in a > stack overflow and no hs-err file. > > Would it not be better to do MemTracker::shutdown() only for the first > recursion level? By moving it down some lines into the "if (first_error == > NULL && Atomic::cmpxchg_ptr(this, &first_error, NULL) == NULL)" condition? > > Kind regards, > > Thomas From erik.helin at oracle.com Thu Mar 13 16:03:29 2014 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 13 Mar 2014 17:03:29 +0100 Subject: RFR: 8036699: Add trace event when a metaspace allocation fails In-Reply-To: <53183925.4020001@oracle.com> References: <53183925.4020001@oracle.com> Message-ID: <5321D6D1.8060503@oracle.com> Hi all, due to lack of reviewers for the native stack trace code, I've decided to split this patch into two patches: - adding the event - adding the native stack trace information the event (will be sent out later) This review request is now only for adding the event. Please see a new webrev at: http://cr.openjdk.java.net/~ehelin/8036699/webrev.01/ For an incremental webrev (that only removes the native stack walking code) please see: http://cr.openjdk.java.net/~ehelin/8036699/webrev.00-01/ Thanks, Erik On 2014-03-06 10:00, Erik Helin wrote: > Hi all, > > this patch adds the new trace event vm/gc/metaspace/allocation_failure. > The event will be sent when we get an allocation failure in metaspace. > The event will contain the following information: > - classLoader - the class loader doing the allocation > - anonymousClassLoader - if the classLoader is anonymous > - nativeStackTrace - a VM stack trace (see more below) > - metadataType - the kind of metadata (class or non-class) > - metaspaceObjectType - the kind of metaspace object > > The field nativeStackTrace will contain a string representation of the > Threads C frames (up until the first Java frame). This is very useful > for debugging metaspace allocation failures coming from VM internals > such as the interpreter or GC code. > > I would especially appreciate if someone could look over the code in > metaspaceTracer.cpp performing the stack walking. > > Webrev: > http://cr.openjdk.java.net/~ehelin/8036699/webrev.00/ > > Enhancement: > https://bugs.openjdk.java.net/browse/JDK-8036699 > > Testing: > - JFR JTREG tests > - JPRT > - Manual stress testing of the stack walking code by writing small test > programs that causes metaspace allocation failures from both Java > threads and non-Java threads. > > Thanks, > Erik From vladimir.kozlov at oracle.com Thu Mar 13 16:09:53 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 13 Mar 2014 09:09:53 -0700 Subject: [8u20] RFR (XS): 8033805: Move Fast_Lock/Fast_Unlock code from .ad files to macroassembler In-Reply-To: <609656F4-D911-4147-81CE-B25A00F9BFE3@oracle.com> References: <5320D332.2010105@oracle.com> <609656F4-D911-4147-81CE-B25A00F9BFE3@oracle.com> Message-ID: <5321D851.2030108@oracle.com> Thank you, Roland Vladimir On 3/13/14 3:17 AM, Roland Westrelin wrote: > Looks good. > > Roland. > > From thomas.stuefe at gmail.com Thu Mar 13 16:39:48 2014 From: thomas.stuefe at gmail.com (=?ISO-8859-1?Q?Thomas_St=FCfe?=) Date: Thu, 13 Mar 2014 17:39:48 +0100 Subject: NMT shutdown in VMError::report_and_die() In-Reply-To: <5321D303.2070203@oracle.com> References: <5321D303.2070203@oracle.com> Message-ID: Thanks :-) Kind Regards, Thomas On 13 March 2014 16:47, Zhengyu Gu wrote: > Hi Thomas, > > Your suggestion does make sense and it is better. We are addressing > several NMT issues, one of them is to eliminate this shutdown call, so NMT > can report memory statistics as part of error reporting. > > Thanks, > > -Zhengyu > > > > On 3/13/2014 10:45 AM, Thomas St?fe wrote: > >> Greetings, >> >> I am with the SAP JVM; right now I am trying to consolidate our error >> handling code a bit (bringing our coding closer to OpenJDK). >> >> I have a small question about shutting down NMT in >> VMError::report_and_die(): >> >> VMError::report_and_die() is called recursivly in case of secondary >> crashes. NMT shutdown is done at the beginning of the function. If NMT >> shutdown itself crashes, we will have infinite recursion, resulting in a >> stack overflow and no hs-err file. >> >> Would it not be better to do MemTracker::shutdown() only for the first >> recursion level? By moving it down some lines into the "if (first_error == >> NULL && Atomic::cmpxchg_ptr(this, &first_error, NULL) == NULL)" condition? >> >> Kind regards, >> >> Thomas >> > > From erik.helin at oracle.com Thu Mar 13 17:04:19 2014 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 13 Mar 2014 18:04:19 +0100 Subject: RFR: 8036701: Add trace event when a metaspace throws out of memory error In-Reply-To: <53183DBC.1050707@oracle.com> References: <53183DBC.1050707@oracle.com> Message-ID: <5321E513.60005@oracle.com> Hi all, since I decided to do the native stack allocation as a separate feature for the metaspace allocation failure event, this patch required a very minor update. I just removed the field "nativeStackTrace" from the event. New webrev: http://cr.openjdk.java.net/~ehelin/8036701/webrev.01/ Incremental webrev: http://cr.openjdk.java.net/~ehelin/8036701/webrev.00-01/ Thanks, Erik On 2014-03-06 10:19, Erik Helin wrote: > Hi all, > > this patch adds a new trace event for tracking metaspace out of memory > exceptions - vm/gc/metaspace/out_of_memory. This patch builds upon the > patch for "JDK-8036699: Add trace event when a metaspace allocation > fails". The information in the allocation failure event and the OOM > event are the same, see the review request for the allocation failure > event for more information. > > Webrev: > http://cr.openjdk.java.net/~ehelin/8036701/webrev.00/ > > Enhancement: > https://bugs.openjdk.java.net/browse/JDK-8036701 > > Testing: > - JFR JTREG tests > - JPRT > > Thanks, > Erik From vladimir.kozlov at oracle.com Thu Mar 13 17:17:27 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 13 Mar 2014 10:17:27 -0700 Subject: RFR[9](XS): 8037298: Export HotSpots 'optimized' (i.e. not-product) configuration in the top-level configure/makefile In-Reply-To: References: Message-ID: <5321E827.2070204@oracle.com> Looks good to me. Vladimir On 3/13/14 8:04 AM, Volker Simonis wrote: > Hi, > > could somebody please review this small change which exports HotSpots > 'optimized' (i.e. not-product) configuration in the top-level > configure/makefile: > > http://cr.openjdk.java.net/~simonis/webrevs/8037298/ > https://bugs.openjdk.java.net/browse/JDK-8037298 > > The HotSpot offers a so called 'optimized' configuration which is > actually the same as a product build, but without setting '-DPRODUCT'. > That means that an 'optimized' HotSpot VM offers additional command > line options and functionality which is not present in an actual > product build. > > While the 'product' configuration is exported in the top-level > configure/make system as "--with-debug-level=release" the , this was > not the case for the 'optimized' configuration. > > I think I need a sponsor for this change because it requires the > regeneration of common/autoconf/generated-configure.sh as well as of > the closed generated configure files. > > Thank you and best regards, > Volker > From vladimir.kozlov at oracle.com Thu Mar 13 19:32:55 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 13 Mar 2014 12:32:55 -0700 Subject: RFR(L): 8036976: PPC64: implement the template interpreter In-Reply-To: <53213C4B.4090006@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CEA69E2@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC2CEA6C1E@DEWDFEMB12A.global.corp.sap> <531F984A.7090805@oracle.com> <4295855A5C1DE049A61835A1887419CC2CEA744D@DEWDFEMB12A.global.corp.sap> <5320B491.3010103@oracle.com> <53213C4B.4090006@oracle.com> Message-ID: <532207E7.3080809@oracle.com> On 3/12/14 10:04 PM, David Holmes wrote: > So we're assuming the ppc64le stuff will go in as a distinct > architecture after all ??? > > In my opinion this fix should not have the ppc64le change - it should be > added if/when the ppc64le stuff gets accepted and pushed. I suggested it. From one side, yes ARCH should reflect architecture. But on other side, in reality, in our makefiles ARCH is a string returned by 'uname': # ARCH - uname output ARCH := $(shell uname -m) And it is the only way to see if it is pp64le. We can rename ARCH to UNAME_OUTPUT to avoid confusion but it is a different issue. Thanks, Vladimir > > David > > On 13/03/2014 5:25 AM, Vladimir Kozlov wrote: >> Looks fine to me. I will push it wiht me and Coleen as reviewers. >> I assume I need to push it into ppc-aix 8u stage repo. Right? >> >> Thanks, >> Vladimir >> >> On 3/12/14 4:04 AM, Lindenmaier, Goetz wrote: >>> Hi Vladimir, >>> >>> thanks for the hint with the makefile -- that would have popped up >>> soon after I guess. >>> >>> I fixed that and the Copyright, and updated the webrev. >>> http://cr.openjdk.java.net/~goetz/webrevs/8036976-0-ti/ >>> >>> Best regards, >>> Goetz. >>> >>> >>> >>> -----Original Message----- >>> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] >>> Sent: Mittwoch, 12. M?rz 2014 00:12 >>> To: Lindenmaier, Goetz; 'hotspot-dev at openjdk.java.net'; >>> 'ppc-aix-port-dev at openjdk.java.net' >>> Subject: Re: RFR(L): 8036976: PPC64: implement the template interpreter >>> >>> Hi Goetz, >>> >>> Should you also check for ppc64le?: >>> >>> ifeq ($(ARCH), ppc64) >>> FORCE_TIERED=0 >>> >>> Please, update year in Oracle copyright line in new files: >>> >>> Copyright (c) 1997, 2010, Oracle >>> >>> to >>> >>> Copyright (c) 2014, Oracle >>> >>> I know that you copied and modified from our existing files but HG >>> history does not show it anyway and they all looks new. We should have >>> done it for all ppc files but it is different issue. >>> >>> These changes passed JPRT build with our closed sources. >>> >>> Thanks, >>> Vladimir >>> >>> On 3/11/14 2:37 PM, Lindenmaier, Goetz wrote: >>>> Hi, >>>> >>>> I had missed all the new files in the webrev. I updated it with the >>>> files in it. >>>> >>>> Sorry for this trouble, >>>> Goetz. >>>> >>>> From: Lindenmaier, Goetz >>>> Sent: Tuesday, March 11, 2014 10:13 AM >>>> To: hotspot-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net >>>> Subject: RFR(L): 8036976: PPC64: implement the template interpreter >>>> >>>> Hi, >>>> >>>> This time I have a really nice one: the template interpreter for ppc64! >>>> Great thanks to my colleagues, especially Axel and Martin, who ported >>>> this. >>>> http://cr.openjdk.java.net/~goetz/webrevs/8036976-0-ti/ >>>> >>>> With this change, both interpreters can be used on linux_ppc64 and >>>> aix_ppc64. >>>> >>>> This change almost only touches ppc platform files. >>>> >>>> I shared files, I needed to add a special case for PPC64 in >>>> make/linux/Makefile, as I found no way to set FORCE_TIERED=0 in any >>>> ppc file that's included above the use of FORCE_TIERED. As we don't >>>> have C1, the current scheme does not work for ppc64. >>>> >>>> Actually, one should have lines as >>>> supports_cppInterpreter = 1 >>>> supports_c2 = 1 >>>> supports_c1 = 0 >>>> in platform_. But that's out of scope of this change. >>>> >>>> Further I add a #include in >>>> src/share/vm/interpreter/templateTable.hpp. That's all for shared >>>> changes. >>>> Please review and test this change. >>>> >>>> Best regards, >>>> Goetz. >>>> From vladimir.kozlov at oracle.com Thu Mar 13 19:56:01 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 13 Mar 2014 12:56:01 -0700 Subject: RFR(L): 8036976: PPC64: implement the template interpreter In-Reply-To: <532207E7.3080809@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CEA69E2@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC2CEA6C1E@DEWDFEMB12A.global.corp.sap> <531F984A.7090805@oracle.com> <4295855A5C1DE049A61835A1887419CC2CEA744D@DEWDFEMB12A.global.corp.sap> <5320B491.3010103@oracle.com> <53213C4B.4090006@oracle.com> <532207E7.3080809@oracle.com> Message-ID: <53220D51.9060508@oracle.com> And we can change this check to something else in ppc64le changes if we want. I was concern that we will forgot about that place. Vladimir On 3/13/14 12:32 PM, Vladimir Kozlov wrote: > On 3/12/14 10:04 PM, David Holmes wrote: >> So we're assuming the ppc64le stuff will go in as a distinct >> architecture after all ??? >> >> In my opinion this fix should not have the ppc64le change - it should be >> added if/when the ppc64le stuff gets accepted and pushed. > > I suggested it. From one side, yes ARCH should reflect architecture. But > on other side, in reality, in our makefiles ARCH is a string returned by > 'uname': > > # ARCH - uname output > ARCH := $(shell uname -m) > > And it is the only way to see if it is pp64le. We can rename ARCH to > UNAME_OUTPUT to avoid confusion but it is a different issue. > > Thanks, > Vladimir > >> >> David >> >> On 13/03/2014 5:25 AM, Vladimir Kozlov wrote: >>> Looks fine to me. I will push it wiht me and Coleen as reviewers. >>> I assume I need to push it into ppc-aix 8u stage repo. Right? >>> >>> Thanks, >>> Vladimir >>> >>> On 3/12/14 4:04 AM, Lindenmaier, Goetz wrote: >>>> Hi Vladimir, >>>> >>>> thanks for the hint with the makefile -- that would have popped up >>>> soon after I guess. >>>> >>>> I fixed that and the Copyright, and updated the webrev. >>>> http://cr.openjdk.java.net/~goetz/webrevs/8036976-0-ti/ >>>> >>>> Best regards, >>>> Goetz. >>>> >>>> >>>> >>>> -----Original Message----- >>>> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] >>>> Sent: Mittwoch, 12. M?rz 2014 00:12 >>>> To: Lindenmaier, Goetz; 'hotspot-dev at openjdk.java.net'; >>>> 'ppc-aix-port-dev at openjdk.java.net' >>>> Subject: Re: RFR(L): 8036976: PPC64: implement the template interpreter >>>> >>>> Hi Goetz, >>>> >>>> Should you also check for ppc64le?: >>>> >>>> ifeq ($(ARCH), ppc64) >>>> FORCE_TIERED=0 >>>> >>>> Please, update year in Oracle copyright line in new files: >>>> >>>> Copyright (c) 1997, 2010, Oracle >>>> >>>> to >>>> >>>> Copyright (c) 2014, Oracle >>>> >>>> I know that you copied and modified from our existing files but HG >>>> history does not show it anyway and they all looks new. We should have >>>> done it for all ppc files but it is different issue. >>>> >>>> These changes passed JPRT build with our closed sources. >>>> >>>> Thanks, >>>> Vladimir >>>> >>>> On 3/11/14 2:37 PM, Lindenmaier, Goetz wrote: >>>>> Hi, >>>>> >>>>> I had missed all the new files in the webrev. I updated it with the >>>>> files in it. >>>>> >>>>> Sorry for this trouble, >>>>> Goetz. >>>>> >>>>> From: Lindenmaier, Goetz >>>>> Sent: Tuesday, March 11, 2014 10:13 AM >>>>> To: hotspot-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net >>>>> Subject: RFR(L): 8036976: PPC64: implement the template interpreter >>>>> >>>>> Hi, >>>>> >>>>> This time I have a really nice one: the template interpreter for >>>>> ppc64! >>>>> Great thanks to my colleagues, especially Axel and Martin, who ported >>>>> this. >>>>> http://cr.openjdk.java.net/~goetz/webrevs/8036976-0-ti/ >>>>> >>>>> With this change, both interpreters can be used on linux_ppc64 and >>>>> aix_ppc64. >>>>> >>>>> This change almost only touches ppc platform files. >>>>> >>>>> I shared files, I needed to add a special case for PPC64 in >>>>> make/linux/Makefile, as I found no way to set FORCE_TIERED=0 in any >>>>> ppc file that's included above the use of FORCE_TIERED. As we don't >>>>> have C1, the current scheme does not work for ppc64. >>>>> >>>>> Actually, one should have lines as >>>>> supports_cppInterpreter = 1 >>>>> supports_c2 = 1 >>>>> supports_c1 = 0 >>>>> in platform_. But that's out of scope of this change. >>>>> >>>>> Further I add a #include in >>>>> src/share/vm/interpreter/templateTable.hpp. That's all for shared >>>>> changes. >>>>> Please review and test this change. >>>>> >>>>> Best regards, >>>>> Goetz. >>>>> From serguei.spitsyn at oracle.com Thu Mar 13 21:24:24 2014 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 13 Mar 2014 14:24:24 -0700 Subject: RFR (XS) 6976636: JVM/TI test ex03t001 fails assertion Message-ID: <53222208.9010009@oracle.com> Please, review the fix for: https://bugs.openjdk.java.net/browse/JDK-6976636 Open webrev: http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/6976636-JVMTI-unload.1 Summary: This is a Nightly Stabilization issue. The class unload event post in the JvmtiExport.cpp fails at the assert assuming the proxy thread causing the class unload must be a JavaThread. It is not the case when the proxy thread is a CMS ConcurrentGCThread. This fix is to relax this check allowing this case. The downside of this approach is that the jthread parameter passed to the even handler callback is NULL for the CMS thread. This must be Ok as it would indicates that the proxy thread is a CMS thread. In fact, I have a doubt this parameter is needed at all. So that an alternative approach could be to remove it from the event callback. My preference is to leave the jthread parameter as it is as it does not impact anything. The class unload event is implemented as a source debug extension for the sake of testing the extension functionality. The JDWP agent does not use it. The class unload events are not expected to be used by any external tool. As a consequence of this fix the failing nsk.jvmti test ex03t001 must be tweeked as well. If review is positive then I'll file a bug and proceed with the test fix. Testing: Running the failing test: nsk/jvmti/scenarios/extension/EX03/ex03t001 Thanks, Serguei From vladimir.kozlov at oracle.com Fri Mar 14 06:11:03 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 13 Mar 2014 23:11:03 -0700 Subject: RFR (S) 8037226: compiler/7196199/Test7196199.java fails on 32-bit linux with MaxVectorSize > 16 Message-ID: <53229D77.6030405@oracle.com> WARNING: it is ugly but I don't know an other way to fix the problem and keep wide AVX vectors. Some operating systems (or VMs) does not restore upper 128 bit of AVX registers YMM after signal processing. About year ago Mac OS had this problem (8003878). And now we found 32-bit Ubuntu (Linux 3.8.0-29) which has the same problem. Instead of reducing vectorization to 16 bytes (XMM registers) I added code which verifies YMM state after signal processing and set limit on vector's size. The code is part of cpuid info code which JVM executes during startup. https://bugs.openjdk.java.net/browse/JDK-8037226 http://cr.openjdk.java.net/~kvn/8037226/webrev/ Tested on failing machines. thanks, Vladimir From rickard.backman at oracle.com Fri Mar 14 07:40:19 2014 From: rickard.backman at oracle.com (Rickard =?iso-8859-1?Q?B=E4ckman?=) Date: Fri, 14 Mar 2014 08:40:19 +0100 Subject: RFR(L) [8u-backport] 8027754: Enable loop optimizations for loops with MathExact inside Message-ID: <20140314074019.GS9323@rbackman> Hi, can I have this backport reviewed? Patches applies cleanly to 8u-dev. Original commits to jdk9 done on 25th of Feb so the changes had some time to bake. Webrev: http://cr.openjdk.java.net/~rbackman/8027754.4/ Changeset: http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/cd5d10655495 Bug: https://bugs.openjdk.java.net/browse/JDK-8027754 Thanks /R From vladimir.kozlov at oracle.com Fri Mar 14 07:54:31 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 14 Mar 2014 00:54:31 -0700 Subject: RFR(L) [8u-backport] 8027754: Enable loop optimizations for loops with MathExact inside In-Reply-To: <20140314074019.GS9323@rbackman> References: <20140314074019.GS9323@rbackman> Message-ID: <5322B5B7.6040105@oracle.com> Good. Thanks, Vladimir On 3/14/14 12:40 AM, Rickard B?ckman wrote: > Hi, > > can I have this backport reviewed? Patches applies cleanly to 8u-dev. > Original commits to jdk9 done on 25th of Feb so the changes had some > time to bake. > > Webrev: http://cr.openjdk.java.net/~rbackman/8027754.4/ > Changeset: http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/cd5d10655495 > Bug: https://bugs.openjdk.java.net/browse/JDK-8027754 > > Thanks > /R > From neugens at redhat.com Fri Mar 14 08:05:17 2014 From: neugens at redhat.com (Mario Torre) Date: Fri, 14 Mar 2014 09:05:17 +0100 Subject: Request backport JDK-8036619 Message-ID: <1394784317.4008.19.camel@galactica.localdomain> Hello! I would like to ask the backport of the changes made for the following bug report into JDK 8: https://bugs.openjdk.java.net/browse/JDK-8036619 The changes apply cleanly without modification: http://cr.openjdk.java.net/~neugens/zero-2014-02-24/webrev.01/ Somehow the webrev was a bit broken, here is the actual patch: http://cr.openjdk.java.net/~neugens/zero-2014-02-24/webrev.01/shark-patches.patch The related changes for JDK 9 are here: http://cr.openjdk.java.net/~neugens/zero-2014-02-24/webrev.02/ The discussion here: http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-March/012814.html and here: http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-February/012679.html I can't push myself, so if OK, someone would need to do for me. Thanks! Cheers, Mario From magnus.ihse.bursie at oracle.com Fri Mar 14 09:44:55 2014 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Fri, 14 Mar 2014 10:44:55 +0100 Subject: RFR[9](XS): 8037298: Export HotSpots 'optimized' (i.e. not-product) configuration in the top-level configure/makefile In-Reply-To: References: Message-ID: <5322CF97.6080301@oracle.com> On 2014-03-13 16:04, Volker Simonis wrote: > Hi, > > could somebody please review this small change which exports HotSpots > 'optimized' (i.e. not-product) configuration in the top-level > configure/makefile: > > http://cr.openjdk.java.net/~simonis/webrevs/8037298/ > https://bugs.openjdk.java.net/browse/JDK-8037298 > > The HotSpot offers a so called 'optimized' configuration which is > actually the same as a product build, but without setting '-DPRODUCT'. > That means that an 'optimized' HotSpot VM offers additional command > line options and functionality which is not present in an actual > product build. > > While the 'product' configuration is exported in the top-level > configure/make system as "--with-debug-level=release" the , this was > not the case for the 'optimized' configuration. > > I think I need a sponsor for this change because it requires the > regeneration of common/autoconf/generated-configure.sh as well as of > the closed generated configure files. Hi Volker, Unfortunately, this is more of a mine field than what first looks will tell. :( The problem here is that the DEBUG_LEVEL variable that is set by --with-debug-level is used for setting *both* the Hotspot debug level and the JDK debug level. In JDK, the only real difference is between "debug" and "release", so the existing "fastdebug" and "slowdebug" are generally considered to be "debug", in contrast to "release". Maybe it was incorrect to introduce a single debug level for the whole product -- maybe we should have kept a separate hotspot-debug-level and a jdk-debug-level. In a way, that makes sense. On the other hand, it also makes sense to try to unify the product as a whole. So I'm a bit ambivalent here. But here we are now; while it is possible to change this now I'd rather avoid it. If you introduce "optimized" as a new level, this will affect all the existing code as well that checks the DEBUG_LEVEL. So you will need to determine how this should be interpreted by the JDK part of the build. With your patch as it looks today, the effects will be a bit random. Most of the time, we check if debug level is release or not, where the "not" case (currently fastdebug and slowdebug) is bunched together and treated as a general debug case. Your "optimized" level would end up there. Probably correct, but that depends on what you want to achieve. In a few cases we explicitely look for slowdebug or fastdebug, and consider the alternative as release; there your optimized level will be treated as release. It might very well be that this treatment of optimized as a mix between debug and release is what's needed here, but the decisions should be made explicitely. You should at least check the uses of DEBUG_LEVEL in flags.m4 and NativeCompilation.gmk. Also, how would you feel about naming the level "optdebug" instead of "optimized"? I think that would make things clearer in the rest of the build, since then you'd have three debug levels: "[slow|fast|opt]debug" in contrast to "release", and anything that is not explicitely a release build is some kind of debug build -- not intended for shipping. It would make it possible to keep the abilitiy in the make files to check DEBUG_LEVEL to see if it contains "debug". And I believe it would make it more clear to future jdk make file developers to understand that this is some kind of debug (i.e. non-release) build. /Magnus I'm not convinced this is the best way to achieve what you want. The "debug levels" in the configure system is not really the same as the "configurations" in hotspot. The optimized configuration in hotspot is really, from the configuration point of view, a release build (DEBUG_LEVEL=release), but at the same time, it is telling hotspot not to disable non-product functionality. From coleen.phillimore at oracle.com Fri Mar 14 12:19:31 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 14 Mar 2014 08:19:31 -0400 Subject: RFR(L): 8029075 - String deduplication in G1 In-Reply-To: <5321C8A3.70206@oracle.com> References: <53148494.60806@oracle.com> <5316F5B1.60006@oracle.com> <5319D424.90306@oracle.com> <1394541591.2696.20.camel@cirrus> <5320626E.7020102@oracle.com> <5320DB4B.2020801@oracle.com> <5321C8A3.70206@oracle.com> Message-ID: <5322F3D3.6060303@oracle.com> Per, 3 quick comments below. Thank you for the reorganization and additional comments. On 3/13/14 11:02 AM, Per Liden wrote: > Hi Coleen, > > Thanks for looking at the patch! Comment below. > > On 2014-03-12 23:10, Coleen Phillimore wrote: >> >> Hi Per, >> >> Why not add the call for StringDedup::unlink() inside the function >> unlink_string_and_symbol_table rather than just above in two places, >> since they go together. >> >> + // Delete dead entries in string deduplication queue/table >> + StringDedup::unlink(&GenMarkSweep::is_alive); >> + >> // Delete entries for dead interned string and clean up >> unreferenced symbols in symbol table. >> G1CollectedHeap::heap()->unlink_string_and_symbol_table(&GenMarkSweep::is_alive); >> > > Yep, I'll move that call into unlink_string_and_symbol_table(). The > unlink_string_and_symbol_table() is a fairly new addition, there used > to be StringTable/SymbolTable calls there when I added the StrDedup > call, that's why it looks like that. > >> >> >> I didn't see anything else specific to change. Some general comments >> which are similar to Thomas's. There seems to be a lack of comments >> in the deduplication file. I hate huge block comments too but some >> comments about what it does in general would be nice. Like 3 >> sentences per class. The file is overwhelming (ie, I completely lose >> interest) with the statistics printing at the front. Maybe you >> could put the most interesting classes at the front of the file? >> Maybe statistics printing could be a separate file. >> >> One of the conventions that we have is that accessors for classes are >> on one line, so you could make the file smaller by doing that >> reformatting. >> >> It is also a general convention to have comments above the >> declaration of a class to say generally what it's for, when it's >> used, etc. It's not easy to tell by the name or we could guess >> wrong. What is StringDedupQueue for, eg? That would make reviewing >> the code easier. > > Agree, I'll be adding comments to each class and a general description > of what dedup does. > >> >> There seems to be a lot of duplication of code with the hashtable in >> hashtable.hpp, only different. We have lots of hashtables in >> hotspot. Did you try to reinstantiate this hashtable and why didn't >> that not work? There are interesting memory allocation functions for >> the entries in this normal hashtable that might help with >> fragmentation, for instance. > > Very early on I actually did use an instance of Hashtable and even > played with the idea of having a combined StringTable/DedupTable. I > decided not to use that though, mostly because I felt I would need to > make too many modifications to non-dedup related code and because > Hashtable<> carries concepts, like shared entries, which didn't quite > fit with dedup. Using the existing StringTable would mean it would > have to have logic to manage entries which point to either Strings or > char[], which again seemed like an too intrusive change. The Hashtable > allocation strategy (with allocating blocks of entries) seems nice > from a performance perspective, but freeing those becomes a bit of a > challenge, which I guess is why it never frees any entries. Since the > dedup table is there to in the end save memory, I felt I needed better > control over the table's memory usage. At one point I actually started > writing a WeakOopsHashTable<>, which I hoped could serve as the base > for both StringTable and DedupTable, but again I felt I was losing > focus a bit as it would mean modifying large parts of non-GC and > non-dedeup related stuff. The potential connection to CDS here also > made me a bit scared of going in that direction, as I don't have a > complete understanding of the implications of that. Ok, I'd figured they didn't map that well. There's a lot of duplication already in symbol and string tables also. > >> >> Lastly, why is the hash code in the string written to and used? The >> StringTable uses the same hash algorithm until it has to change the >> hash code, but the new hash code isn't stored in the string because >> it would be incompatible. > > I think StringTable and the dedup table is doing the same thing here. > The hash code in the String object is only used/updated in the case > we're using the standard/compatible hash function. As soon as the > table is rehashed and switches to another hash function it no longer > updates the hash code in the String. use_java_hash() tells use if > we're using the standard/compatible hash and it's used here: > > if (use_java_hash()) { > // Get hash code from cache > hash = java_lang_String::hash(java_string); > } > > and later here: > > if (use_java_hash() && hash != 0) { > // Store hash code in cache > java_lang_String::set_hash(java_string, hash); > } > > Concerns? I have to look again but wanted to answer quickly (forgot to last night). This is good, but why do we have to update the hash code in the java/lang/String instance at all? > >> >> I would like to see this be a few different files. It seems like >> this file does three things conceptually, interacts with G1 somehow >> to determine which strings are candidates for deduplication, stores >> these things in a rehashable, resizable hash table and gathers >> statistics. >> >> I'm sorry to have so many comments so late and I haven't really tried >> to read the code for correctness. I think some reformatting and >> reorganization would make the reviewers job easier. It's a big >> change and worthwhile. > > Great, I've no problems splitting this up. I'm more of a "one class > per file" person myself, but I had the (apparently incorrect) > impression that wasn't the hotspot way. I'm thinking I'll split this > up into the following files (while I'm at it I'll also add the G1 > prefix as Thomas suggested): > > g1StringDedup - Main interface for the GC, would contain entry point > and things like the task and closure classes > g1StringDedupTable - The table, cache and entry classes > g1StringDedupQueue - The queue > g1StringDedupStat - The statistics sutff > g1StringDedupThread - The dedup thread Yes, this would be nice. symbolTable.cpp should be split up too if that is what gave you the impression. I will file an RFE to do that, it's been at the back of my mind to do so. Thanks, Coleen > > Sounds good? > > Will send out and updated webrev. > > Thanks, > /Per > >> >> Thanks, >> Coleen >> >> >> On 3/12/14 9:34 AM, Per Liden wrote: >>> Hi Thomas, >>> >>> Thanks for reviewing. Comments inline. >>> >>> On 03/11/2014 01:39 PM, Thomas Schatzl wrote: >>>> Hi Per, >>>> >>>> >>>> On Fri, 2014-03-07 at 15:13 +0100, Per Liden wrote: >>>>> Hi, >>>>> >>>>> Here's an updated webrev based on the feedback from Bengt and Thomas. >>>>> >>>>> http://cr.openjdk.java.net/~pliden/8029075/webrev.1/ >>>>> >>>> >>>> See further below. >>>> >>>>> >>>>> And as suggested, two changes in the initial webrev were broken >>>>> out into >>>>> separate CRs, available here: >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8036672 >>>>> Webrev: http://cr.openjdk.java.net/~pliden/8036672/webrev.0/ >>>> >>>> Looks good. >>>> >>>> Did you check the impact on something like specjbb2005 which does >>>> nothing but object copying? That code is somewhat performance >>>> sensitive. >>> >>> Yes, I did Aurora runs with jbb2005 and jbb2013, didn't see any >>> difference. >>> >>>> >>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8036673 >>>>> Webrev: http://cr.openjdk.java.net/~pliden/8036673/webrev.0/ >>>> >>>> Looks good. >>>> >>>> Here are my comments: there are a few minor issues, and a single >>>> major: >>>> basically the documentation for this feature is lacking imo. I >>>> listed a >>>> few particular issues of mine with it. >>>> >>>> First the minor issues: >>>> >>>> - could you check on copyright dates on the modified files? They have >>>> not been updated anywhere afaics. >>>> >>>> - not sure if good, but maybe make PrintStringDeduplicationStatistics >>>> diagnostic? It seems to really be some diagnosis functions. >>>> >>>> - maybe make the new *ALot flags notproduct. These seem to be >>>> debugging >>>> helpers similar to other *ALot flags. >>> >>> I had a conversion with Bengt about this some time ago and I think >>> the options are categorized correctly. The ALot flags needs be >>> available in product builds because they are used by tests. Most >>> Print*Statistics are product therefore it makes sense to follow that >>> pattern for PrintStringDeduplicationStatistics. >>> >>>> >>>> - StringDeduplicationAgeThreshold: maybe also needs to mention that if >>>> the object is promoted, that it is always considered as candidate. >>> >>> Check. >>> >>>> >>>> marksweep.inline.hpp >>>> >>>> - MarkSweep::mark_object(): did you measure impact on other collectors >>>> using this code? It is enabled unconditionally. Also I would prefer >>>> that !UseG1GC automatically disables UseStringDeduplication (or use a >>>> global variable for that) to avoid the two checks. (The latter >>>> option is >>>> probably better). >>> >>> Yes, ran jbb2005/2013. >>> >>>> >>>> symbolTable.cpp: >>>> >>>> - again both UseG1GC and UseStringDedupliation are paired. Could >>>> this be >>>> put in a method/global variable somewhere that is set up correctly at >>>> startup? >>> >>> I think using the options is much more readable. Adding another >>> state which tells us if G1 & Dedup is enabled means we need a >>> pre-init stage to set up that state, which I don't think is very nice. >>> >>>> >>>> g1CollectedHeap.cpp: >>>> >>>> - StringDedup::initialize() is run unconditionally (the check for >>>> UseStringDeduplication is inside the method) while it's outside for >>>> everything else. Just a note, keep it if wanted. >>> >>> They are hidden inside dedup everywhere, except where the >>> performance overhead of a call might be an issue (like mark and copy). >>> >>>> >>>> TestStringDeduplicationTools.java: >>>> >>>> - maybe put retrieval of the unsafe object into the test library. >>>> Browsing through the code shows quite a lot copy+paste here already. >>>> >>>> - "qeueue" -> "queue" >>> >>> Check. >>> >>>> >>>> all tests: >>>> >>>> - missing bug ids >>> >>> Check. >>> >>>> >>>> From here it is mostly about comments/documentation: >>> >>> I agree that adding a feature overview and some additional comments >>> is probably a good idea. However, I also know that Bengt and StefanK >>> aren't too crazy about large comment blobs so I tried to kept the >>> noise down. I also disagree with some of the other comments below >>> which tend to be about personal taste rather than code quality or >>> breaking some coding convention we have, e.g. the static helpers, >>> readability, naming, etc. >>> >>> I'll add some comments and make some other changes and send out an >>> updated webrev shortly. >>> >>> cheers, >>> /Per >>> >>>> >>>> stringdedup.?pp: >>>> >>>> - is it possible to put an overview about how the string deduplication >>>> works here? There is absolutely no place in the code where e.g. the >>>> string deduplication cycle is explained. There is no comprehensive >>>> information about this feature anywhere, just bits about corner >>>> cases or >>>> particular here and there, but nothing that can be used as entry point >>>> to understand what this is actually doing. >>>> >>>> To some extent, the comments seem to be repetitions of what the code >>>> already shows. E.g. "some_constant = (1 << 10); // 1024 (must be power >>>> of 2)" or "// Store hash code in cache" followed by >>>> "java_lang_String::set_hash(java_string, hash);". >>>> >>>> There is also no reference to the JEP (I am not sure if it is >>>> customary >>>> to put references to outside documents - I would prefer some); however >>>> the JEP does not replace documentation in the source as it is not >>>> detailed enough, and the JEP cannot be changed any more. The JEP is >>>> interesting for the alternatives considered though. >>>> >>>> Ideally, to reasonably understand the feature I would like to not need >>>> to spend hours looking through the implementation. >>>> >>>> - how many threads does this feature add, and are supported by this >>>> code >>>> (for which parts)? Synchronization between them? >>>> >>>> - the comments "For XY" in the StringDedup header file do not give any >>>> hint about what they are doing (not sure if it is interesting as it is >>>> easy to find out the only callers). It seems that this information has >>>> been put at the place where these methods are invoked (or just >>>> somewhere >>>> in the cpp file). Imo the methods and members (except trivial) >>>> should be >>>> documented in the hpp file in the class body, i.e. in the interface. >>>> >>>> - there are a few methods (also in the StringDedupTable class) in >>>> StringDedup where the results of the methods do not seem obvious and >>>> should be commented. >>>> >>>> - I really dislike the location of the "Candidate selection" >>>> explanation >>>> within the cpp file. I would not have found it if I were not >>>> looking for >>>> some documentation ;) It would probably fit very well in the above >>>> mentioned overview paragraphs. >>>> >>>> - please add the static helper functions that implement the actual >>>> policies (is_candidate_from_mark, is_candidate_from_evacuation) to the >>>> StringDedup class (as private static methods) so that they can be >>>> found >>>> easily. Static non-class helper functions in cpp files should only be >>>> used for stuff that is really unimportant and not relevant to the code >>>> imo. However these seem to be _the_ central policy methods for string >>>> dedup. >>>> So it would be nice to have them mentioned in the class/hpp file and >>>> documented there. >>>> >>>> - facts about the code are repeated, e.g. that the table size must >>>> be a >>>> power of two. This is nice, but has a few drawbacks: the compiler does >>>> not care about comments at all, so it might be useful to put >>>> asserts in >>>> these places instead, and second, it's just duplication of text >>>> with the >>>> usual problem of getting out of date. >>>> >>>> - I am not completely sure why the class has not been called >>>> G1StringDedup. It won't work without G1 anyway and calls G1 internals >>>> all the time. There does not seem to be a way to make it more generic >>>> without considerable refactoring. It is fine to keep the name though, >>>> just asking. I also know that this feature does not want to be a >>>> generic >>>> solution. >>>> >>>> - is it possible to separate out the statistics variables of >>>> StringDedupTable into either a separate class instead of using globals >>>> or separate them a little from the other unrelated members? (And >>>> describe them) This would improve readability. >>>> >>>> I *think* this is everything from StringDedupTable::_entries_added to >>>> _rehash_count? I have no idea without searching for every use and >>>> deducing from that. >>>> >>>> - when implementing the hash table for the StringDedupTable, did you >>>> consider trying to reuse one of the existing hashmap implementations? >>>> And potentially extend them. >>>> >>>> Imo we already have way too many ad-hoc implementations of often used >>>> data structures (particularly) in G1 and GC code. >>>> >>>> - please document the properties of the StringDedupTable (type, >>>> hashing, >>>> automatic resizing behavior, thread safety, ...). >>>> >>>> - please explain what StringDedupTable/Cache are and what they are >>>> used >>>> for in context of the feature. >>>> >>>> - stringdedup.cpp, line 1309: candicate -> candidate >>>> >>>> Thanks, >>>> Thomas >>>> >>>> >>> >> > From per.liden at oracle.com Fri Mar 14 13:19:09 2014 From: per.liden at oracle.com (Per Liden) Date: Fri, 14 Mar 2014 14:19:09 +0100 Subject: RFR(L): 8029075 - String deduplication in G1 In-Reply-To: <5322F3D3.6060303@oracle.com> References: <53148494.60806@oracle.com> <5316F5B1.60006@oracle.com> <5319D424.90306@oracle.com> <1394541591.2696.20.camel@cirrus> <5320626E.7020102@oracle.com> <5320DB4B.2020801@oracle.com> <5321C8A3.70206@oracle.com> <5322F3D3.6060303@oracle.com> Message-ID: <532301CD.7020709@oracle.com> Hi Coleen, Inline below... On 2014-03-14 13:19, Coleen Phillimore wrote: > > Per, 3 quick comments below. Thank you for the reorganization and > additional comments. > > On 3/13/14 11:02 AM, Per Liden wrote: >> Hi Coleen, >> >> Thanks for looking at the patch! Comment below. >> >> On 2014-03-12 23:10, Coleen Phillimore wrote: >>> >>> Hi Per, >>> >>> Why not add the call for StringDedup::unlink() inside the function >>> unlink_string_and_symbol_table rather than just above in two places, >>> since they go together. >>> >>> + // Delete dead entries in string deduplication queue/table >>> + StringDedup::unlink(&GenMarkSweep::is_alive); >>> + >>> // Delete entries for dead interned string and clean up >>> unreferenced symbols in symbol table. >>> G1CollectedHeap::heap()->unlink_string_and_symbol_table(&GenMarkSweep::is_alive); >>> >> >> Yep, I'll move that call into unlink_string_and_symbol_table(). The >> unlink_string_and_symbol_table() is a fairly new addition, there used >> to be StringTable/SymbolTable calls there when I added the StrDedup >> call, that's why it looks like that. >> >>> >>> >>> I didn't see anything else specific to change. Some general >>> comments which are similar to Thomas's. There seems to be a lack of >>> comments in the deduplication file. I hate huge block comments too >>> but some comments about what it does in general would be nice. Like >>> 3 sentences per class. The file is overwhelming (ie, I completely >>> lose interest) with the statistics printing at the front. Maybe >>> you could put the most interesting classes at the front of the >>> file? Maybe statistics printing could be a separate file. >>> >>> One of the conventions that we have is that accessors for classes >>> are on one line, so you could make the file smaller by doing that >>> reformatting. >>> >>> It is also a general convention to have comments above the >>> declaration of a class to say generally what it's for, when it's >>> used, etc. It's not easy to tell by the name or we could guess >>> wrong. What is StringDedupQueue for, eg? That would make >>> reviewing the code easier. >> >> Agree, I'll be adding comments to each class and a general >> description of what dedup does. >> >>> >>> There seems to be a lot of duplication of code with the hashtable in >>> hashtable.hpp, only different. We have lots of hashtables in >>> hotspot. Did you try to reinstantiate this hashtable and why didn't >>> that not work? There are interesting memory allocation functions >>> for the entries in this normal hashtable that might help with >>> fragmentation, for instance. >> >> Very early on I actually did use an instance of Hashtable and even >> played with the idea of having a combined StringTable/DedupTable. I >> decided not to use that though, mostly because I felt I would need to >> make too many modifications to non-dedup related code and because >> Hashtable<> carries concepts, like shared entries, which didn't quite >> fit with dedup. Using the existing StringTable would mean it would >> have to have logic to manage entries which point to either Strings or >> char[], which again seemed like an too intrusive change. The >> Hashtable allocation strategy (with allocating blocks of entries) >> seems nice from a performance perspective, but freeing those becomes >> a bit of a challenge, which I guess is why it never frees any >> entries. Since the dedup table is there to in the end save memory, I >> felt I needed better control over the table's memory usage. At one >> point I actually started writing a WeakOopsHashTable<>, which I hoped >> could serve as the base for both StringTable and DedupTable, but >> again I felt I was losing focus a bit as it would mean modifying >> large parts of non-GC and non-dedeup related stuff. The potential >> connection to CDS here also made me a bit scared of going in that >> direction, as I don't have a complete understanding of the >> implications of that. > > Ok, I'd figured they didn't map that well. There's a lot of > duplication already in symbol and string tables also. >> >>> >>> Lastly, why is the hash code in the string written to and used? The >>> StringTable uses the same hash algorithm until it has to change the >>> hash code, but the new hash code isn't stored in the string because >>> it would be incompatible. >> >> I think StringTable and the dedup table is doing the same thing here. >> The hash code in the String object is only used/updated in the case >> we're using the standard/compatible hash function. As soon as the >> table is rehashed and switches to another hash function it no longer >> updates the hash code in the String. use_java_hash() tells use if >> we're using the standard/compatible hash and it's used here: >> >> if (use_java_hash()) { >> // Get hash code from cache >> hash = java_lang_String::hash(java_string); >> } >> >> and later here: >> >> if (use_java_hash() && hash != 0) { >> // Store hash code in cache >> java_lang_String::set_hash(java_string, hash); >> } >> >> Concerns? > > I have to look again but wanted to answer quickly (forgot to last > night). This is good, but why do we have to update the hash code in > the java/lang/String instance at all? We don't strictly have to, but it mimics the behavior of String.hashCode() and it is seems like the natural thing to do to given that the hash field in String is a cache for this purpose so that future users of String.hashCode() don't have to do the same calculation. Do you have a reason in mind why we wouldn't want to write it back? cheers, /Per > >> >>> >>> I would like to see this be a few different files. It seems like >>> this file does three things conceptually, interacts with G1 somehow >>> to determine which strings are candidates for deduplication, stores >>> these things in a rehashable, resizable hash table and gathers >>> statistics. >>> >>> I'm sorry to have so many comments so late and I haven't really >>> tried to read the code for correctness. I think some reformatting >>> and reorganization would make the reviewers job easier. It's a big >>> change and worthwhile. >> >> Great, I've no problems splitting this up. I'm more of a "one class >> per file" person myself, but I had the (apparently incorrect) >> impression that wasn't the hotspot way. I'm thinking I'll split this >> up into the following files (while I'm at it I'll also add the G1 >> prefix as Thomas suggested): >> >> g1StringDedup - Main interface for the GC, would contain entry point >> and things like the task and closure classes >> g1StringDedupTable - The table, cache and entry classes >> g1StringDedupQueue - The queue >> g1StringDedupStat - The statistics sutff >> g1StringDedupThread - The dedup thread > > Yes, this would be nice. symbolTable.cpp should be split up too if > that is what gave you the impression. I will file an RFE to do that, > it's been at the back of my mind to do so. > > Thanks, > Coleen >> >> Sounds good? >> >> Will send out and updated webrev. >> >> Thanks, >> /Per >> >>> >>> Thanks, >>> Coleen >>> >>> >>> On 3/12/14 9:34 AM, Per Liden wrote: >>>> Hi Thomas, >>>> >>>> Thanks for reviewing. Comments inline. >>>> >>>> On 03/11/2014 01:39 PM, Thomas Schatzl wrote: >>>>> Hi Per, >>>>> >>>>> >>>>> On Fri, 2014-03-07 at 15:13 +0100, Per Liden wrote: >>>>>> Hi, >>>>>> >>>>>> Here's an updated webrev based on the feedback from Bengt and >>>>>> Thomas. >>>>>> >>>>>> http://cr.openjdk.java.net/~pliden/8029075/webrev.1/ >>>>>> >>>>> >>>>> See further below. >>>>> >>>>>> >>>>>> And as suggested, two changes in the initial webrev were broken >>>>>> out into >>>>>> separate CRs, available here: >>>>>> >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8036672 >>>>>> Webrev: http://cr.openjdk.java.net/~pliden/8036672/webrev.0/ >>>>> >>>>> Looks good. >>>>> >>>>> Did you check the impact on something like specjbb2005 which does >>>>> nothing but object copying? That code is somewhat performance >>>>> sensitive. >>>> >>>> Yes, I did Aurora runs with jbb2005 and jbb2013, didn't see any >>>> difference. >>>> >>>>> >>>>> >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8036673 >>>>>> Webrev: http://cr.openjdk.java.net/~pliden/8036673/webrev.0/ >>>>> >>>>> Looks good. >>>>> >>>>> Here are my comments: there are a few minor issues, and a single >>>>> major: >>>>> basically the documentation for this feature is lacking imo. I >>>>> listed a >>>>> few particular issues of mine with it. >>>>> >>>>> First the minor issues: >>>>> >>>>> - could you check on copyright dates on the modified files? They have >>>>> not been updated anywhere afaics. >>>>> >>>>> - not sure if good, but maybe make PrintStringDeduplicationStatistics >>>>> diagnostic? It seems to really be some diagnosis functions. >>>>> >>>>> - maybe make the new *ALot flags notproduct. These seem to be >>>>> debugging >>>>> helpers similar to other *ALot flags. >>>> >>>> I had a conversion with Bengt about this some time ago and I think >>>> the options are categorized correctly. The ALot flags needs be >>>> available in product builds because they are used by tests. Most >>>> Print*Statistics are product therefore it makes sense to follow >>>> that pattern for PrintStringDeduplicationStatistics. >>>> >>>>> >>>>> - StringDeduplicationAgeThreshold: maybe also needs to mention >>>>> that if >>>>> the object is promoted, that it is always considered as candidate. >>>> >>>> Check. >>>> >>>>> >>>>> marksweep.inline.hpp >>>>> >>>>> - MarkSweep::mark_object(): did you measure impact on other >>>>> collectors >>>>> using this code? It is enabled unconditionally. Also I would prefer >>>>> that !UseG1GC automatically disables UseStringDeduplication (or use a >>>>> global variable for that) to avoid the two checks. (The latter >>>>> option is >>>>> probably better). >>>> >>>> Yes, ran jbb2005/2013. >>>> >>>>> >>>>> symbolTable.cpp: >>>>> >>>>> - again both UseG1GC and UseStringDedupliation are paired. Could >>>>> this be >>>>> put in a method/global variable somewhere that is set up correctly at >>>>> startup? >>>> >>>> I think using the options is much more readable. Adding another >>>> state which tells us if G1 & Dedup is enabled means we need a >>>> pre-init stage to set up that state, which I don't think is very nice. >>>> >>>>> >>>>> g1CollectedHeap.cpp: >>>>> >>>>> - StringDedup::initialize() is run unconditionally (the check for >>>>> UseStringDeduplication is inside the method) while it's outside for >>>>> everything else. Just a note, keep it if wanted. >>>> >>>> They are hidden inside dedup everywhere, except where the >>>> performance overhead of a call might be an issue (like mark and copy). >>>> >>>>> >>>>> TestStringDeduplicationTools.java: >>>>> >>>>> - maybe put retrieval of the unsafe object into the test library. >>>>> Browsing through the code shows quite a lot copy+paste here already. >>>>> >>>>> - "qeueue" -> "queue" >>>> >>>> Check. >>>> >>>>> >>>>> all tests: >>>>> >>>>> - missing bug ids >>>> >>>> Check. >>>> >>>>> >>>>> From here it is mostly about comments/documentation: >>>> >>>> I agree that adding a feature overview and some additional comments >>>> is probably a good idea. However, I also know that Bengt and >>>> StefanK aren't too crazy about large comment blobs so I tried to >>>> kept the noise down. I also disagree with some of the other >>>> comments below which tend to be about personal taste rather than >>>> code quality or breaking some coding convention we have, e.g. the >>>> static helpers, readability, naming, etc. >>>> >>>> I'll add some comments and make some other changes and send out an >>>> updated webrev shortly. >>>> >>>> cheers, >>>> /Per >>>> >>>>> >>>>> stringdedup.?pp: >>>>> >>>>> - is it possible to put an overview about how the string >>>>> deduplication >>>>> works here? There is absolutely no place in the code where e.g. the >>>>> string deduplication cycle is explained. There is no comprehensive >>>>> information about this feature anywhere, just bits about corner >>>>> cases or >>>>> particular here and there, but nothing that can be used as entry >>>>> point >>>>> to understand what this is actually doing. >>>>> >>>>> To some extent, the comments seem to be repetitions of what the code >>>>> already shows. E.g. "some_constant = (1 << 10); // 1024 (must be >>>>> power >>>>> of 2)" or "// Store hash code in cache" followed by >>>>> "java_lang_String::set_hash(java_string, hash);". >>>>> >>>>> There is also no reference to the JEP (I am not sure if it is >>>>> customary >>>>> to put references to outside documents - I would prefer some); >>>>> however >>>>> the JEP does not replace documentation in the source as it is not >>>>> detailed enough, and the JEP cannot be changed any more. The JEP is >>>>> interesting for the alternatives considered though. >>>>> >>>>> Ideally, to reasonably understand the feature I would like to not >>>>> need >>>>> to spend hours looking through the implementation. >>>>> >>>>> - how many threads does this feature add, and are supported by >>>>> this code >>>>> (for which parts)? Synchronization between them? >>>>> >>>>> - the comments "For XY" in the StringDedup header file do not give >>>>> any >>>>> hint about what they are doing (not sure if it is interesting as >>>>> it is >>>>> easy to find out the only callers). It seems that this information >>>>> has >>>>> been put at the place where these methods are invoked (or just >>>>> somewhere >>>>> in the cpp file). Imo the methods and members (except trivial) >>>>> should be >>>>> documented in the hpp file in the class body, i.e. in the interface. >>>>> >>>>> - there are a few methods (also in the StringDedupTable class) in >>>>> StringDedup where the results of the methods do not seem obvious and >>>>> should be commented. >>>>> >>>>> - I really dislike the location of the "Candidate selection" >>>>> explanation >>>>> within the cpp file. I would not have found it if I were not >>>>> looking for >>>>> some documentation ;) It would probably fit very well in the above >>>>> mentioned overview paragraphs. >>>>> >>>>> - please add the static helper functions that implement the actual >>>>> policies (is_candidate_from_mark, is_candidate_from_evacuation) to >>>>> the >>>>> StringDedup class (as private static methods) so that they can be >>>>> found >>>>> easily. Static non-class helper functions in cpp files should only be >>>>> used for stuff that is really unimportant and not relevant to the >>>>> code >>>>> imo. However these seem to be _the_ central policy methods for string >>>>> dedup. >>>>> So it would be nice to have them mentioned in the class/hpp file and >>>>> documented there. >>>>> >>>>> - facts about the code are repeated, e.g. that the table size must >>>>> be a >>>>> power of two. This is nice, but has a few drawbacks: the compiler >>>>> does >>>>> not care about comments at all, so it might be useful to put >>>>> asserts in >>>>> these places instead, and second, it's just duplication of text >>>>> with the >>>>> usual problem of getting out of date. >>>>> >>>>> - I am not completely sure why the class has not been called >>>>> G1StringDedup. It won't work without G1 anyway and calls G1 internals >>>>> all the time. There does not seem to be a way to make it more generic >>>>> without considerable refactoring. It is fine to keep the name though, >>>>> just asking. I also know that this feature does not want to be a >>>>> generic >>>>> solution. >>>>> >>>>> - is it possible to separate out the statistics variables of >>>>> StringDedupTable into either a separate class instead of using >>>>> globals >>>>> or separate them a little from the other unrelated members? (And >>>>> describe them) This would improve readability. >>>>> >>>>> I *think* this is everything from StringDedupTable::_entries_added to >>>>> _rehash_count? I have no idea without searching for every use and >>>>> deducing from that. >>>>> >>>>> - when implementing the hash table for the StringDedupTable, did you >>>>> consider trying to reuse one of the existing hashmap implementations? >>>>> And potentially extend them. >>>>> >>>>> Imo we already have way too many ad-hoc implementations of often used >>>>> data structures (particularly) in G1 and GC code. >>>>> >>>>> - please document the properties of the StringDedupTable (type, >>>>> hashing, >>>>> automatic resizing behavior, thread safety, ...). >>>>> >>>>> - please explain what StringDedupTable/Cache are and what they are >>>>> used >>>>> for in context of the feature. >>>>> >>>>> - stringdedup.cpp, line 1309: candicate -> candidate >>>>> >>>>> Thanks, >>>>> Thomas >>>>> >>>>> >>>> >>> >> > From mikael.gerdin at oracle.com Fri Mar 14 15:28:09 2014 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Fri, 14 Mar 2014 16:28:09 +0100 Subject: RFR: 8036701: Add trace event when a metaspace throws out of memory error In-Reply-To: <5321E513.60005@oracle.com> References: <53183DBC.1050707@oracle.com> <5321E513.60005@oracle.com> Message-ID: <6362861.Lsl7LcVfB0@mgerdin03> Erik, On Thursday 13 March 2014 18.04.19 Erik Helin wrote: > Hi all, > > since I decided to do the native stack allocation as a separate feature > for the metaspace allocation failure event, this patch required a very > minor update. I just removed the field "nativeStackTrace" from the event. > > New webrev: > http://cr.openjdk.java.net/~ehelin/8036701/webrev.01/ Change looks good. /Mikael > > Incremental webrev: > http://cr.openjdk.java.net/~ehelin/8036701/webrev.00-01/ > > Thanks, > Erik > > On 2014-03-06 10:19, Erik Helin wrote: > > Hi all, > > > > this patch adds a new trace event for tracking metaspace out of memory > > exceptions - vm/gc/metaspace/out_of_memory. This patch builds upon the > > patch for "JDK-8036699: Add trace event when a metaspace allocation > > fails". The information in the allocation failure event and the OOM > > event are the same, see the review request for the allocation failure > > event for more information. > > > > Webrev: > > http://cr.openjdk.java.net/~ehelin/8036701/webrev.00/ > > > > Enhancement: > > https://bugs.openjdk.java.net/browse/JDK-8036701 > > > > Testing: > > - JFR JTREG tests > > - JPRT > > > > Thanks, > > Erik From erik.helin at oracle.com Fri Mar 14 15:50:38 2014 From: erik.helin at oracle.com (Erik Helin) Date: Fri, 14 Mar 2014 16:50:38 +0100 Subject: RFR: 8036701: Add trace event when a metaspace throws out of memory error In-Reply-To: <6362861.Lsl7LcVfB0@mgerdin03> References: <53183DBC.1050707@oracle.com> <5321E513.60005@oracle.com> <6362861.Lsl7LcVfB0@mgerdin03> Message-ID: <5323254E.6080808@oracle.com> Hi Mikael, thanks for reviewing! Erik On 2014-03-14 16:28, Mikael Gerdin wrote: > Erik, > > On Thursday 13 March 2014 18.04.19 Erik Helin wrote: >> Hi all, >> >> since I decided to do the native stack allocation as a separate feature >> for the metaspace allocation failure event, this patch required a very >> minor update. I just removed the field "nativeStackTrace" from the event. >> >> New webrev: >> http://cr.openjdk.java.net/~ehelin/8036701/webrev.01/ > > Change looks good. > /Mikael > >> >> Incremental webrev: >> http://cr.openjdk.java.net/~ehelin/8036701/webrev.00-01/ >> >> Thanks, >> Erik >> >> On 2014-03-06 10:19, Erik Helin wrote: >>> Hi all, >>> >>> this patch adds a new trace event for tracking metaspace out of memory >>> exceptions - vm/gc/metaspace/out_of_memory. This patch builds upon the >>> patch for "JDK-8036699: Add trace event when a metaspace allocation >>> fails". The information in the allocation failure event and the OOM >>> event are the same, see the review request for the allocation failure >>> event for more information. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~ehelin/8036701/webrev.00/ >>> >>> Enhancement: >>> https://bugs.openjdk.java.net/browse/JDK-8036701 >>> >>> Testing: >>> - JFR JTREG tests >>> - JPRT >>> >>> Thanks, >>> Erik > From erik.helin at oracle.com Fri Mar 14 15:55:04 2014 From: erik.helin at oracle.com (Erik Helin) Date: Fri, 14 Mar 2014 16:55:04 +0100 Subject: RFR: 8036701: Add trace event when a metaspace throws out of memory error In-Reply-To: <5321E513.60005@oracle.com> References: <53183DBC.1050707@oracle.com> <5321E513.60005@oracle.com> Message-ID: <53232658.3090202@oracle.com> Based on a discussion with Jon Masamitsu and StefanK, I've decided to change metadata_type_name to return a slightly more user-friendly value. Please see new webrev at: http://cr.openjdk.java.net/~ehelin/8036699/webrev.02/ Incremental webrev: http://cr.openjdk.java.net/~ehelin/8036699/webrev.01-02/ Thanks, Erik On 2014-03-13 18:04, Erik Helin wrote: > Hi all, > > since I decided to do the native stack allocation as a separate feature > for the metaspace allocation failure event, this patch required a very > minor update. I just removed the field "nativeStackTrace" from the event. > > New webrev: > http://cr.openjdk.java.net/~ehelin/8036701/webrev.01/ > > Incremental webrev: > http://cr.openjdk.java.net/~ehelin/8036701/webrev.00-01/ > > Thanks, > Erik > > On 2014-03-06 10:19, Erik Helin wrote: >> Hi all, >> >> this patch adds a new trace event for tracking metaspace out of memory >> exceptions - vm/gc/metaspace/out_of_memory. This patch builds upon the >> patch for "JDK-8036699: Add trace event when a metaspace allocation >> fails". The information in the allocation failure event and the OOM >> event are the same, see the review request for the allocation failure >> event for more information. >> >> Webrev: >> http://cr.openjdk.java.net/~ehelin/8036701/webrev.00/ >> >> Enhancement: >> https://bugs.openjdk.java.net/browse/JDK-8036701 >> >> Testing: >> - JFR JTREG tests >> - JPRT >> >> Thanks, >> Erik From alejandro.murillo at oracle.com Fri Mar 14 16:26:34 2014 From: alejandro.murillo at oracle.com (alejandro.murillo at oracle.com) Date: Fri, 14 Mar 2014 16:26:34 +0000 Subject: hg: hsx/jdk7u/jdk: 17 new changesets Message-ID: <201403141627.s2EGRmUs015012@aojmv0008> Changeset: db7a06b2b7ee Author: khazra Date: 2013-06-13 17:48 -0700 URL: http://hg.openjdk.java.net/hsx/jdk7u/jdk/rev/db7a06b2b7ee 7169142: CookieHandler does not work with localhost Summary: Add .local to derived effective hostnames without dot Reviewed-by: chegar ! src/share/classes/java/net/CookieManager.java + test/java/net/CookieHandler/LocalHostCookie.java Changeset: 1fa79e009882 Author: khazra Date: 2013-06-13 11:23 -0700 URL: http://hg.openjdk.java.net/hsx/jdk7u/jdk/rev/1fa79e009882 8015421: NegativeArraySizeException occurs in ChunkedOutputStream() with Integer.MAX_VALUE Summary: Ensure integer overflow does not occur Reviewed-by: chegar ! src/share/classes/sun/net/www/http/ChunkedOutputStream.java Changeset: 8e2b4808a068 Author: anashaty Date: 2014-02-07 16:00 +0400 URL: http://hg.openjdk.java.net/hsx/jdk7u/jdk/rev/8e2b4808a068 8023990: Regression: postscript size increase from 6u18 Reviewed-by: prr, bae ! src/share/classes/sun/awt/FontConfiguration.java ! src/solaris/classes/sun/font/FcFontConfiguration.java Changeset: 13473046fdfe Author: robm Date: 2014-02-11 16:06 +0000 URL: http://hg.openjdk.java.net/hsx/jdk7u/jdk/rev/13473046fdfe 7084032: test/java/net/Inet6Address/B6558853.java fails on Windows XP/2003 if IPv6 Reviewed-by: chegar ! src/windows/native/java/net/TwoStacksPlainSocketImpl.c Changeset: 3edb49b50486 Author: twisti Date: 2013-07-03 11:35 -0700 URL: http://hg.openjdk.java.net/hsx/jdk7u/jdk/rev/3edb49b50486 8019184: MethodHandles.catchException() fails when methods have 8 args + varargs Reviewed-by: jrose ! src/share/classes/java/lang/invoke/MethodHandleImpl.java + test/java/lang/invoke/TestCatchExceptionWithVarargs.java Changeset: c5d7cbccfcd7 Author: robm Date: 2014-02-13 21:35 +0000 URL: http://hg.openjdk.java.net/hsx/jdk7u/jdk/rev/c5d7cbccfcd7 7152892: some jtreg tests fail with permission denied Reviewed-by: coffeys ! test/java/lang/ClassLoader/Assert.sh ! test/java/rmi/registry/readTest/readTest.sh ! test/java/util/zip/ZipFile/ReadZip.java ! test/sun/net/www/protocol/jar/jarbug/run.sh Changeset: e1f33d3b7054 Author: kshefov Date: 2014-02-18 16:00 +0400 URL: http://hg.openjdk.java.net/hsx/jdk7u/jdk/rev/e1f33d3b7054 8017456: [TEST_BUG] java/awt/DataFlavor/MissedHtmlAndRtfBug/MissedHtmlAndRtfBug.html: test frames remain after test execution Reviewed-by: serb, alexsch ! test/java/awt/DataFlavor/MissedHtmlAndRtfBug/InterprocessMessages.java ! test/java/awt/DataFlavor/MissedHtmlAndRtfBug/MissedHtmlAndRtfBug.html ! test/java/awt/DataFlavor/MissedHtmlAndRtfBug/MissedHtmlAndRtfBug.java Changeset: 544e82c5a52d Author: weijun Date: 2013-10-23 08:32 +0800 URL: http://hg.openjdk.java.net/hsx/jdk7u/jdk/rev/544e82c5a52d 8027026: Change keytool -genkeypair to use -keyalg RSA Reviewed-by: alanb, chegar, mullan ! test/ProblemList.txt ! test/java/util/TimeZone/TimeZoneDatePermissionCheck.sh ! test/java/util/jar/JarInputStream/ExtraFileInMetaInf.java ! test/sun/security/pkcs12/PKCS12SameKeyId.java ! test/sun/security/tools/jarsigner/TimestampCheck.java ! test/sun/security/tools/jarsigner/checkusage.sh ! test/sun/security/tools/jarsigner/crl.sh ! test/sun/security/tools/jarsigner/jvindex.sh ! test/sun/security/tools/jarsigner/newsize7.sh ! test/sun/security/tools/jarsigner/onlymanifest.sh ! test/sun/security/tools/jarsigner/passtype.sh ! test/sun/security/tools/jarsigner/samename.sh ! test/sun/security/tools/jarsigner/ts.sh ! test/sun/security/tools/keytool/CloseFile.java ! test/sun/security/tools/keytool/ListKeychainStore.sh ! test/sun/security/tools/keytool/StartDateTest.java ! test/sun/security/tools/keytool/emptysubject.sh ! test/sun/security/tools/keytool/importreadall.sh ! test/sun/security/tools/keytool/readjar.sh ! test/sun/security/tools/keytool/selfissued.sh ! test/sun/security/tools/keytool/trystore.sh ! test/sun/security/validator/samedn.sh Changeset: 827bc20e8f61 Author: dsamersoff Date: 2014-02-21 19:26 +0400 URL: http://hg.openjdk.java.net/hsx/jdk7u/jdk/rev/827bc20e8f61 8019800: Test sun/tools/jcmd/jcmd-f.sh failing after JDK-8017011 Summary: removed extra colon Reviewed-by: sla, coffeys ! test/ProblemList.txt ! test/sun/tools/jcmd/help_help.out Changeset: efb51c945698 Author: igerasim Date: 2014-02-20 16:01 +0400 URL: http://hg.openjdk.java.net/hsx/jdk7u/jdk/rev/efb51c945698 8027212: java/nio/channels/Selector/SelectAfterRead.java fails intermittently Reviewed-by: chegar, ewang ! test/java/nio/channels/Selector/ByteServer.java ! test/java/nio/channels/Selector/ReadAfterConnect.java ! test/java/nio/channels/Selector/SelectAfterRead.java ! test/java/nio/channels/Selector/SelectWrite.java Changeset: e03ef8c99170 Author: dmarkov Date: 2014-02-24 11:59 +0400 URL: http://hg.openjdk.java.net/hsx/jdk7u/jdk/rev/e03ef8c99170 8032669: Mouse release not being delivered to Swing component in 7u45 Reviewed-by: alexsch, pchelko ! src/macosx/classes/sun/lwawt/LWWindowPeer.java ! src/macosx/classes/sun/lwawt/PlatformWindow.java ! src/macosx/classes/sun/lwawt/macosx/CPlatformEmbeddedFrame.java ! src/macosx/classes/sun/lwawt/macosx/CPlatformWindow.java ! src/macosx/classes/sun/lwawt/macosx/CViewPlatformEmbeddedFrame.java ! src/macosx/native/sun/awt/AWTView.h ! src/macosx/native/sun/awt/AWTView.m ! src/macosx/native/sun/awt/AWTWindow.m + test/java/awt/Mouse/EnterExitEvents/DragWindowTest.java Changeset: 97de64f172be Author: alitvinov Date: 2014-02-24 19:17 +0400 URL: http://hg.openjdk.java.net/hsx/jdk7u/jdk/rev/97de64f172be 8032078: [macosx] CPlatformWindow.setWindowState throws RuntimeException, if windowState=ICONIFIED|MAXIMIZED_BOTH Reviewed-by: pchelko, serb ! src/macosx/classes/sun/lwawt/macosx/CPlatformWindow.java + test/java/awt/Frame/ExceptionOnSetExtendedStateTest/ExceptionOnSetExtendedStateTest.java Changeset: b387fbb83529 Author: msheppar Date: 2014-02-17 16:15 +0000 URL: http://hg.openjdk.java.net/hsx/jdk7u/jdk/rev/b387fbb83529 8021372: NetworkInterface.getNetworkInterfaces() returns duplicate hardware address Summary: amended src/windows/native/java/net/NetworkInterface_winXP.c to "properly" handle Ipv6IfIndex Reviewed-by: chegar, dsamersoff ! src/windows/native/java/net/NetworkInterface_winXP.c + test/java/net/NetworkInterface/UniqueMacAddressesTest.java Changeset: 0ad0b65578da Author: msheppar Date: 2013-09-13 12:20 +0100 URL: http://hg.openjdk.java.net/hsx/jdk7u/jdk/rev/0ad0b65578da 8024675: java/net/NetworkInterface/UniqueMacAddressesTest.java fails on Windows Summary: amended test to add active, i.e. isUp(), NetworkInterfaces to test list Reviewed-by: alanb, chegar ! test/java/net/NetworkInterface/UniqueMacAddressesTest.java Changeset: 0da092934c9d Author: coffeys Date: 2014-02-25 20:29 +0000 URL: http://hg.openjdk.java.net/hsx/jdk7u/jdk/rev/0da092934c9d Merge Changeset: 4963f233f46a Author: coffeys Date: 2014-02-26 22:44 +0000 URL: http://hg.openjdk.java.net/hsx/jdk7u/jdk/rev/4963f233f46a 7076487: (sctp) SCTP API classes does not exist in JDK for Mac Reviewed-by: michaelm, chegar ! make/com/sun/nio/Makefile ! make/com/sun/nio/sctp/Exportedfiles.gmk ! make/com/sun/nio/sctp/FILES_c.gmk ! make/com/sun/nio/sctp/FILES_java.gmk ! make/com/sun/nio/sctp/Makefile + src/macosx/classes/sun/nio/ch/SctpChannelImpl.java + src/macosx/classes/sun/nio/ch/SctpMultiChannelImpl.java + src/macosx/classes/sun/nio/ch/SctpServerChannelImpl.java Changeset: eeb571eea68d Author: lana Date: 2014-02-27 10:36 -0800 URL: http://hg.openjdk.java.net/hsx/jdk7u/jdk/rev/eeb571eea68d Merge From alejandro.murillo at oracle.com Fri Mar 14 16:29:37 2014 From: alejandro.murillo at oracle.com (alejandro.murillo at oracle.com) Date: Fri, 14 Mar 2014 16:29:37 +0000 Subject: hg: hsx/jdk7u/langtools: 6 new changesets Message-ID: <201403141629.s2EGThbO015331@aojmv0008> Changeset: 00eaa9e78c47 Author: igerasim Date: 2014-02-09 20:41 +0400 URL: http://hg.openjdk.java.net/hsx/jdk7u/langtools/rev/00eaa9e78c47 7052170: javadoc -charset option generates wrong meta tag Reviewed-by: jjg ! src/share/classes/com/sun/tools/doclets/formats/html/HtmlDocletWriter.java ! src/share/classes/com/sun/tools/doclets/formats/html/markup/HtmlAttr.java ! src/share/classes/com/sun/tools/doclets/formats/html/markup/HtmlDocWriter.java ! src/share/classes/com/sun/tools/doclets/formats/html/markup/HtmlTree.java + test/com/sun/javadoc/testCharset/TestCharset.java + test/com/sun/javadoc/testCharset/pkg/Foo.java Changeset: 420dc86b1716 Author: vromero Date: 2014-02-11 21:01 +0000 URL: http://hg.openjdk.java.net/hsx/jdk7u/langtools/rev/420dc86b1716 8033294: javac, spurious warning for instanceof operator Reviewed-by: jjg, jlahoda ! src/share/classes/com/sun/tools/javac/code/Type.java ! src/share/classes/com/sun/tools/javac/code/Types.java + test/tools/javac/T8033294/RedundantWarningInIntersectionTest.java Changeset: 0774840070f5 Author: robm Date: 2014-02-13 21:39 +0000 URL: http://hg.openjdk.java.net/hsx/jdk7u/langtools/rev/0774840070f5 7152892: some jtreg tests fail with permission denied Reviewed-by: coffeys ! test/tools/javac/ClassPathTest/ClassPathTest.sh Changeset: 71b361cf81bc Author: bpatel Date: 2014-02-13 15:15 -0800 URL: http://hg.openjdk.java.net/hsx/jdk7u/langtools/rev/71b361cf81bc 8016328: Regression : Javadoc i18n regression caused by fix for 8012375 Reviewed-by: jjg ! src/share/classes/com/sun/tools/doclets/formats/html/markup/HtmlDocWriter.java ! src/share/classes/com/sun/tools/doclets/formats/html/markup/HtmlTree.java ! src/share/classes/com/sun/tools/doclets/formats/html/markup/HtmlWriter.java ! test/com/sun/javadoc/testHref/TestHref.java ! test/com/sun/javadoc/testJavascript/TestJavascript.java ! test/com/sun/javadoc/testLinkTaglet/TestLinkTaglet.java ! test/com/sun/javadoc/testPrivateClasses/TestPrivateClasses.java Changeset: 618b253436e4 Author: kizune Date: 2014-02-26 17:51 +0400 URL: http://hg.openjdk.java.net/hsx/jdk7u/langtools/rev/618b253436e4 7186887: Test T6567415.java can fail on a slow machine Reviewed-by: jjg, ksrini ! test/tools/javac/6567415/T6567415.java Changeset: 5c9759e0d341 Author: lana Date: 2014-02-27 10:36 -0800 URL: http://hg.openjdk.java.net/hsx/jdk7u/langtools/rev/5c9759e0d341 Merge From roland.westrelin at oracle.com Fri Mar 14 16:54:36 2014 From: roland.westrelin at oracle.com (Roland Westrelin) Date: Fri, 14 Mar 2014 17:54:36 +0100 Subject: [8u20] RFR(M): 8031752: Failed speculative optimization should be reattempted when root of compilation is different Message-ID: Please, review backport to 8u20. The patch applies cleanly. It?s been in jdk9 for more than 2 weeks. 8032633 that enabled type speculation and stresses this change was pushed a week ago. http://cr.openjdk.java.net/~roland/8031752/webrev.03/ https://bugs.openjdk.java.net/browse/JDK-8031752 http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/fdad2932c73f Roland. From volker.simonis at gmail.com Fri Mar 14 16:55:10 2014 From: volker.simonis at gmail.com (Volker Simonis) Date: Fri, 14 Mar 2014 17:55:10 +0100 Subject: RFR[9](XS): 8037298: Export HotSpots 'optimized' (i.e. not-product) configuration in the top-level configure/makefile In-Reply-To: <5322CF97.6080301@oracle.com> References: <5322CF97.6080301@oracle.com> Message-ID: Hi Magnus, thanks for your detailed comments. I generally agree with your analysis of the problem. However, as you correctly noticed, the 'optimized' configuration is actually a "HotSpot only" concept. It is used to enable extra options/functionality which should not be shipped to the customer. Everything flagged with "#ifndef PRODUCT" in HotSpot is enabled by default in the "slowdebug" and "fastdebug" configuration - so no problem. But the Hotspot has this extra configuration called 'optimized' which is exactly like a release build, but with the extra, "#ifndef PRODUCT" stuff enabled. This may be useful for example for testing/benchmarking new functionality (i.e. you don't want to benchmark with a debug build). There's no general 'optimized' concept for the jdk and the other repositories as far as I can see. The "PRODUCT" macro is only used in src/share/native/com/sun/java/util/jar/pack/* within the 'jdk/' repository (and it is used there with a DEBUG/ASSERT semantics). I'd therefore propose to slightly modify my change as follows: optimized ) DEBUG_LEVEL="release" VARIANT="OPT" FASTDEBUG="false" DEBUG_CLASSFILES="false" BUILD_VARIANT_RELEASE="-optimized" HOTSPOT_DEBUG_LEVEL="optimized" HOTSPOT_EXPORT="optimized" ;; Notice that 'DEBUG_LEVEL' is now reset to "release" for the 'optimized' configuration. And this is actually exactly what we want: an optimized JDK which behaves exactly like a release version but with the extra "NOT_PRODUCT" features of HotSpot.Thanks for catching this. I've slightly changed the description of the 'optimized' configuration to make it clear that it only applies to the HotSpot build. And I now also print the exact HotSpot debug level in the configuration summary to make it clear what we are actually building. Here's the new webrev: http://cr.openjdk.java.net/~simonis/webrevs/8037298_2/ Altogether, I think this change will be mostly helpful to Hotspot engineers and won't bother others too much:) Regards, Volker On Fri, Mar 14, 2014 at 10:44 AM, Magnus Ihse Bursie wrote: > On 2014-03-13 16:04, Volker Simonis wrote: >> >> Hi, >> >> could somebody please review this small change which exports HotSpots >> 'optimized' (i.e. not-product) configuration in the top-level >> configure/makefile: >> >> http://cr.openjdk.java.net/~simonis/webrevs/8037298/ >> https://bugs.openjdk.java.net/browse/JDK-8037298 >> >> The HotSpot offers a so called 'optimized' configuration which is >> actually the same as a product build, but without setting '-DPRODUCT'. >> That means that an 'optimized' HotSpot VM offers additional command >> line options and functionality which is not present in an actual >> product build. >> >> While the 'product' configuration is exported in the top-level >> configure/make system as "--with-debug-level=release" the , this was >> not the case for the 'optimized' configuration. >> >> I think I need a sponsor for this change because it requires the >> regeneration of common/autoconf/generated-configure.sh as well as of >> the closed generated configure files. > > > Hi Volker, > > Unfortunately, this is more of a mine field than what first looks will tell. > :( > > The problem here is that the DEBUG_LEVEL variable that is set by > --with-debug-level is used for setting *both* the Hotspot debug level and > the JDK debug level. > > In JDK, the only real difference is between "debug" and "release", so the > existing "fastdebug" and "slowdebug" are generally considered to be "debug", > in contrast to "release". > > Maybe it was incorrect to introduce a single debug level for the whole > product -- maybe we should have kept a separate hotspot-debug-level and a > jdk-debug-level. In a way, that makes sense. On the other hand, it also > makes sense to try to unify the product as a whole. So I'm a bit ambivalent > here. But here we are now; while it is possible to change this now I'd > rather avoid it. > > If you introduce "optimized" as a new level, this will affect all the > existing code as well that checks the DEBUG_LEVEL. So you will need to > determine how this should be interpreted by the JDK part of the build. With > your patch as it looks today, the effects will be a bit random. Most of the > time, we check if debug level is release or not, where the "not" case > (currently fastdebug and slowdebug) is bunched together and treated as a > general debug case. Your "optimized" level would end up there. Probably > correct, but that depends on what you want to achieve. In a few cases we > explicitely look for slowdebug or fastdebug, and consider the alternative as > release; there your optimized level will be treated as release. > > It might very well be that this treatment of optimized as a mix between > debug and release is what's needed here, but the decisions should be made > explicitely. You should at least check the uses of DEBUG_LEVEL in flags.m4 > and NativeCompilation.gmk. > > Also, how would you feel about naming the level "optdebug" instead of > "optimized"? I think that would make things clearer in the rest of the > build, since then you'd have three debug levels: "[slow|fast|opt]debug" in > contrast to "release", and anything that is not explicitely a release build > is some kind of debug build -- not intended for shipping. It would make it > possible to keep the abilitiy in the make files to check DEBUG_LEVEL to see > if it contains "debug". And I believe it would make it more clear to future > jdk make file developers to understand that this is some kind of debug (i.e. > non-release) build. > > /Magnus > > > > > > > > I'm not convinced this is the best way to achieve what you want. > > The "debug levels" in the configure system is not really the same as the > "configurations" in hotspot. The optimized configuration in hotspot is > really, from the configuration point of view, a release build > (DEBUG_LEVEL=release), but at the same time, it is telling hotspot not to > disable non-product functionality. From vladimir.kozlov at oracle.com Fri Mar 14 17:49:00 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 14 Mar 2014 10:49:00 -0700 Subject: [8u20] RFR(M): 8031752: Failed speculative optimization should be reattempted when root of compilation is different In-Reply-To: References: Message-ID: <5323410C.2020206@oracle.com> Looks good. Vladimir On 3/14/14 9:54 AM, Roland Westrelin wrote: > Please, review backport to 8u20. The patch applies cleanly. It?s been in jdk9 for more than 2 weeks. 8032633 that enabled type speculation and stresses this change was pushed a week ago. > > http://cr.openjdk.java.net/~roland/8031752/webrev.03/ > https://bugs.openjdk.java.net/browse/JDK-8031752 > http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/fdad2932c73f > > Roland. > From igor.veresov at oracle.com Fri Mar 14 17:54:02 2014 From: igor.veresov at oracle.com (Igor Veresov) Date: Fri, 14 Mar 2014 10:54:02 -0700 Subject: [8u20] RFR 8031321: Support Intel bit manipulation instructions Message-ID: <6D90A9CC-663F-4505-8026-4BE092BA9287@oracle.com> No additional changes for 8u are necessary. Nightlies seem to be clean. Webrev: http://cr.openjdk.java.net/~iveresov/8031321/webrev.04/ JBS: https://bugs.openjdk.java.net/browse/JDK-8031321 JDK9 changeset: http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/9e9af3aa4278 Thanks! igor From igor.veresov at oracle.com Fri Mar 14 17:54:28 2014 From: igor.veresov at oracle.com (Igor Veresov) Date: Fri, 14 Mar 2014 10:54:28 -0700 Subject: RFR(L) [8u-backport] 8027754: Enable loop optimizations for loops with MathExact inside In-Reply-To: <20140314074019.GS9323@rbackman> References: <20140314074019.GS9323@rbackman> Message-ID: Looks fine. igor On Mar 14, 2014, at 12:40 AM, Rickard B?ckman wrote: > Hi, > > can I have this backport reviewed? Patches applies cleanly to 8u-dev. > Original commits to jdk9 done on 25th of Feb so the changes had some > time to bake. > > Webrev: http://cr.openjdk.java.net/~rbackman/8027754.4/ > Changeset: http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/cd5d10655495 > Bug: https://bugs.openjdk.java.net/browse/JDK-8027754 > > Thanks > /R From vladimir.kozlov at oracle.com Fri Mar 14 18:06:52 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 14 Mar 2014 11:06:52 -0700 Subject: [8u20] RFR 8031321: Support Intel bit manipulation instructions In-Reply-To: <6D90A9CC-663F-4505-8026-4BE092BA9287@oracle.com> References: <6D90A9CC-663F-4505-8026-4BE092BA9287@oracle.com> Message-ID: <5323453C.6070206@oracle.com> Good. Thanks, Vladimir K On 3/14/14 10:54 AM, Igor Veresov wrote: > No additional changes for 8u are necessary. Nightlies seem to be clean. > > Webrev: http://cr.openjdk.java.net/~iveresov/8031321/webrev.04/ > JBS: https://bugs.openjdk.java.net/browse/JDK-8031321 > JDK9 changeset: http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/9e9af3aa4278 > > Thanks! > igor > From roland.westrelin at oracle.com Fri Mar 14 18:50:18 2014 From: roland.westrelin at oracle.com (Roland Westrelin) Date: Fri, 14 Mar 2014 19:50:18 +0100 Subject: [8u20] RFR(M): 8031754: Type speculation should favor profile data from outermost inlined method Message-ID: Please, review backport to 8u20. The patch applies cleanly except for c2_globals.hpp. jdk8u doesn?t have doesn?t have: product_pd(bool, TrapBasedRangeChecks, \ "Generate code for range checks that uses a cmp and trap " \ "instruction raising SIGTRAP. Used on PPC64.") \ from: 8028767: PPC64: (part 121): smaller shared changes needed to build C2 that I don?t think we want to backport. It?s been in jdk9 for 2 weeks. 8032633 that enabled type speculation and stresses this change was pushed a week ago. http://cr.openjdk.java.net/~roland/8031754/webrev.02/ https://bugs.openjdk.java.net/browse/JDK-8031754 http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/484a359ff649 Roland. From per.liden at oracle.com Fri Mar 14 19:07:31 2014 From: per.liden at oracle.com (Per Liden) Date: Fri, 14 Mar 2014 20:07:31 +0100 Subject: RFR(L): 8029075 - String deduplication in G1 In-Reply-To: <532301CD.7020709@oracle.com> References: <53148494.60806@oracle.com> <5316F5B1.60006@oracle.com> <5319D424.90306@oracle.com> <1394541591.2696.20.camel@cirrus> <5320626E.7020102@oracle.com> <5320DB4B.2020801@oracle.com> <5321C8A3.70206@oracle.com> <5322F3D3.6060303@oracle.com> <532301CD.7020709@oracle.com> Message-ID: <53235373.8080901@oracle.com> Hi, Here's an updated webrev, based on feedback from primarily Thomas and Coleen (Thanks!). http://cr.openjdk.java.net/~pliden/8029075/webrev.2/ I hope to have addressed all issues that were raised. The main theme of course is the addition of documentation/comments and the splitting of some classes into separate files. I think I've addressed most, if not all, of the other minor issues Thomas mentioned. The only thing I can think of that wasn't changed was the categorization of the VM flags, which I motivated in a different mail. I didn't think it made sense to upload a diff against the previous webrev as that diff turned out to be a lot bigger than real webrev itself. Sorry if this makes reviewing harder. cheers, /Per On 2014-03-14 14:19, Per Liden wrote: > Hi Coleen, > > Inline below... > > On 2014-03-14 13:19, Coleen Phillimore wrote: >> >> Per, 3 quick comments below. Thank you for the reorganization and >> additional comments. >> >> On 3/13/14 11:02 AM, Per Liden wrote: >>> Hi Coleen, >>> >>> Thanks for looking at the patch! Comment below. >>> >>> On 2014-03-12 23:10, Coleen Phillimore wrote: >>>> >>>> Hi Per, >>>> >>>> Why not add the call for StringDedup::unlink() inside the function >>>> unlink_string_and_symbol_table rather than just above in two >>>> places, since they go together. >>>> >>>> + // Delete dead entries in string deduplication queue/table >>>> + StringDedup::unlink(&GenMarkSweep::is_alive); >>>> + >>>> // Delete entries for dead interned string and clean up >>>> unreferenced symbols in symbol table. >>>> G1CollectedHeap::heap()->unlink_string_and_symbol_table(&GenMarkSweep::is_alive); >>>> >>> >>> Yep, I'll move that call into unlink_string_and_symbol_table(). The >>> unlink_string_and_symbol_table() is a fairly new addition, there >>> used to be StringTable/SymbolTable calls there when I added the >>> StrDedup call, that's why it looks like that. >>> >>>> >>>> >>>> I didn't see anything else specific to change. Some general >>>> comments which are similar to Thomas's. There seems to be a lack >>>> of comments in the deduplication file. I hate huge block comments >>>> too but some comments about what it does in general would be nice. >>>> Like 3 sentences per class. The file is overwhelming (ie, I >>>> completely lose interest) with the statistics printing at the >>>> front. Maybe you could put the most interesting classes at the >>>> front of the file? Maybe statistics printing could be a separate >>>> file. >>>> >>>> One of the conventions that we have is that accessors for classes >>>> are on one line, so you could make the file smaller by doing that >>>> reformatting. >>>> >>>> It is also a general convention to have comments above the >>>> declaration of a class to say generally what it's for, when it's >>>> used, etc. It's not easy to tell by the name or we could guess >>>> wrong. What is StringDedupQueue for, eg? That would make >>>> reviewing the code easier. >>> >>> Agree, I'll be adding comments to each class and a general >>> description of what dedup does. >>> >>>> >>>> There seems to be a lot of duplication of code with the hashtable >>>> in hashtable.hpp, only different. We have lots of hashtables in >>>> hotspot. Did you try to reinstantiate this hashtable and why >>>> didn't that not work? There are interesting memory allocation >>>> functions for the entries in this normal hashtable that might help >>>> with fragmentation, for instance. >>> >>> Very early on I actually did use an instance of Hashtable and even >>> played with the idea of having a combined StringTable/DedupTable. I >>> decided not to use that though, mostly because I felt I would need >>> to make too many modifications to non-dedup related code and because >>> Hashtable<> carries concepts, like shared entries, which didn't >>> quite fit with dedup. Using the existing StringTable would mean it >>> would have to have logic to manage entries which point to either >>> Strings or char[], which again seemed like an too intrusive change. >>> The Hashtable allocation strategy (with allocating blocks of >>> entries) seems nice from a performance perspective, but freeing >>> those becomes a bit of a challenge, which I guess is why it never >>> frees any entries. Since the dedup table is there to in the end save >>> memory, I felt I needed better control over the table's memory >>> usage. At one point I actually started writing a >>> WeakOopsHashTable<>, which I hoped could serve as the base for both >>> StringTable and DedupTable, but again I felt I was losing focus a >>> bit as it would mean modifying large parts of non-GC and non-dedeup >>> related stuff. The potential connection to CDS here also made me a >>> bit scared of going in that direction, as I don't have a complete >>> understanding of the implications of that. >> >> Ok, I'd figured they didn't map that well. There's a lot of >> duplication already in symbol and string tables also. >>> >>>> >>>> Lastly, why is the hash code in the string written to and used? The >>>> StringTable uses the same hash algorithm until it has to change the >>>> hash code, but the new hash code isn't stored in the string because >>>> it would be incompatible. >>> >>> I think StringTable and the dedup table is doing the same thing >>> here. The hash code in the String object is only used/updated in the >>> case we're using the standard/compatible hash function. As soon as >>> the table is rehashed and switches to another hash function it no >>> longer updates the hash code in the String. use_java_hash() tells >>> use if we're using the standard/compatible hash and it's used here: >>> >>> if (use_java_hash()) { >>> // Get hash code from cache >>> hash = java_lang_String::hash(java_string); >>> } >>> >>> and later here: >>> >>> if (use_java_hash() && hash != 0) { >>> // Store hash code in cache >>> java_lang_String::set_hash(java_string, hash); >>> } >>> >>> Concerns? >> >> I have to look again but wanted to answer quickly (forgot to last >> night). This is good, but why do we have to update the hash code in >> the java/lang/String instance at all? > > We don't strictly have to, but it mimics the behavior of > String.hashCode() and it is seems like the natural thing to do to > given that the hash field in String is a cache for this purpose so > that future users of String.hashCode() don't have to do the same > calculation. Do you have a reason in mind why we wouldn't want to > write it back? > > cheers, > /Per > >> >>> >>>> >>>> I would like to see this be a few different files. It seems like >>>> this file does three things conceptually, interacts with G1 somehow >>>> to determine which strings are candidates for deduplication, stores >>>> these things in a rehashable, resizable hash table and gathers >>>> statistics. >>>> >>>> I'm sorry to have so many comments so late and I haven't really >>>> tried to read the code for correctness. I think some reformatting >>>> and reorganization would make the reviewers job easier. It's a big >>>> change and worthwhile. >>> >>> Great, I've no problems splitting this up. I'm more of a "one class >>> per file" person myself, but I had the (apparently incorrect) >>> impression that wasn't the hotspot way. I'm thinking I'll split this >>> up into the following files (while I'm at it I'll also add the G1 >>> prefix as Thomas suggested): >>> >>> g1StringDedup - Main interface for the GC, would contain entry point >>> and things like the task and closure classes >>> g1StringDedupTable - The table, cache and entry classes >>> g1StringDedupQueue - The queue >>> g1StringDedupStat - The statistics sutff >>> g1StringDedupThread - The dedup thread >> >> Yes, this would be nice. symbolTable.cpp should be split up too if >> that is what gave you the impression. I will file an RFE to do >> that, it's been at the back of my mind to do so. >> >> Thanks, >> Coleen >>> >>> Sounds good? >>> >>> Will send out and updated webrev. >>> >>> Thanks, >>> /Per >>> >>>> >>>> Thanks, >>>> Coleen >>>> >>>> >>>> On 3/12/14 9:34 AM, Per Liden wrote: >>>>> Hi Thomas, >>>>> >>>>> Thanks for reviewing. Comments inline. >>>>> >>>>> On 03/11/2014 01:39 PM, Thomas Schatzl wrote: >>>>>> Hi Per, >>>>>> >>>>>> >>>>>> On Fri, 2014-03-07 at 15:13 +0100, Per Liden wrote: >>>>>>> Hi, >>>>>>> >>>>>>> Here's an updated webrev based on the feedback from Bengt and >>>>>>> Thomas. >>>>>>> >>>>>>> http://cr.openjdk.java.net/~pliden/8029075/webrev.1/ >>>>>>> >>>>>> >>>>>> See further below. >>>>>> >>>>>>> >>>>>>> And as suggested, two changes in the initial webrev were broken >>>>>>> out into >>>>>>> separate CRs, available here: >>>>>>> >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8036672 >>>>>>> Webrev: http://cr.openjdk.java.net/~pliden/8036672/webrev.0/ >>>>>> >>>>>> Looks good. >>>>>> >>>>>> Did you check the impact on something like specjbb2005 which does >>>>>> nothing but object copying? That code is somewhat performance >>>>>> sensitive. >>>>> >>>>> Yes, I did Aurora runs with jbb2005 and jbb2013, didn't see any >>>>> difference. >>>>> >>>>>> >>>>>> >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8036673 >>>>>>> Webrev: http://cr.openjdk.java.net/~pliden/8036673/webrev.0/ >>>>>> >>>>>> Looks good. >>>>>> >>>>>> Here are my comments: there are a few minor issues, and a single >>>>>> major: >>>>>> basically the documentation for this feature is lacking imo. I >>>>>> listed a >>>>>> few particular issues of mine with it. >>>>>> >>>>>> First the minor issues: >>>>>> >>>>>> - could you check on copyright dates on the modified files? They >>>>>> have >>>>>> not been updated anywhere afaics. >>>>>> >>>>>> - not sure if good, but maybe make >>>>>> PrintStringDeduplicationStatistics >>>>>> diagnostic? It seems to really be some diagnosis functions. >>>>>> >>>>>> - maybe make the new *ALot flags notproduct. These seem to be >>>>>> debugging >>>>>> helpers similar to other *ALot flags. >>>>> >>>>> I had a conversion with Bengt about this some time ago and I think >>>>> the options are categorized correctly. The ALot flags needs be >>>>> available in product builds because they are used by tests. Most >>>>> Print*Statistics are product therefore it makes sense to follow >>>>> that pattern for PrintStringDeduplicationStatistics. >>>>> >>>>>> >>>>>> - StringDeduplicationAgeThreshold: maybe also needs to mention >>>>>> that if >>>>>> the object is promoted, that it is always considered as candidate. >>>>> >>>>> Check. >>>>> >>>>>> >>>>>> marksweep.inline.hpp >>>>>> >>>>>> - MarkSweep::mark_object(): did you measure impact on other >>>>>> collectors >>>>>> using this code? It is enabled unconditionally. Also I would prefer >>>>>> that !UseG1GC automatically disables UseStringDeduplication (or >>>>>> use a >>>>>> global variable for that) to avoid the two checks. (The latter >>>>>> option is >>>>>> probably better). >>>>> >>>>> Yes, ran jbb2005/2013. >>>>> >>>>>> >>>>>> symbolTable.cpp: >>>>>> >>>>>> - again both UseG1GC and UseStringDedupliation are paired. Could >>>>>> this be >>>>>> put in a method/global variable somewhere that is set up >>>>>> correctly at >>>>>> startup? >>>>> >>>>> I think using the options is much more readable. Adding another >>>>> state which tells us if G1 & Dedup is enabled means we need a >>>>> pre-init stage to set up that state, which I don't think is very >>>>> nice. >>>>> >>>>>> >>>>>> g1CollectedHeap.cpp: >>>>>> >>>>>> - StringDedup::initialize() is run unconditionally (the check for >>>>>> UseStringDeduplication is inside the method) while it's outside for >>>>>> everything else. Just a note, keep it if wanted. >>>>> >>>>> They are hidden inside dedup everywhere, except where the >>>>> performance overhead of a call might be an issue (like mark and >>>>> copy). >>>>> >>>>>> >>>>>> TestStringDeduplicationTools.java: >>>>>> >>>>>> - maybe put retrieval of the unsafe object into the test library. >>>>>> Browsing through the code shows quite a lot copy+paste here already. >>>>>> >>>>>> - "qeueue" -> "queue" >>>>> >>>>> Check. >>>>> >>>>>> >>>>>> all tests: >>>>>> >>>>>> - missing bug ids >>>>> >>>>> Check. >>>>> >>>>>> >>>>>> From here it is mostly about comments/documentation: >>>>> >>>>> I agree that adding a feature overview and some additional >>>>> comments is probably a good idea. However, I also know that Bengt >>>>> and StefanK aren't too crazy about large comment blobs so I tried >>>>> to kept the noise down. I also disagree with some of the other >>>>> comments below which tend to be about personal taste rather than >>>>> code quality or breaking some coding convention we have, e.g. the >>>>> static helpers, readability, naming, etc. >>>>> >>>>> I'll add some comments and make some other changes and send out an >>>>> updated webrev shortly. >>>>> >>>>> cheers, >>>>> /Per >>>>> >>>>>> >>>>>> stringdedup.?pp: >>>>>> >>>>>> - is it possible to put an overview about how the string >>>>>> deduplication >>>>>> works here? There is absolutely no place in the code where e.g. the >>>>>> string deduplication cycle is explained. There is no comprehensive >>>>>> information about this feature anywhere, just bits about corner >>>>>> cases or >>>>>> particular here and there, but nothing that can be used as entry >>>>>> point >>>>>> to understand what this is actually doing. >>>>>> >>>>>> To some extent, the comments seem to be repetitions of what the code >>>>>> already shows. E.g. "some_constant = (1 << 10); // 1024 (must be >>>>>> power >>>>>> of 2)" or "// Store hash code in cache" followed by >>>>>> "java_lang_String::set_hash(java_string, hash);". >>>>>> >>>>>> There is also no reference to the JEP (I am not sure if it is >>>>>> customary >>>>>> to put references to outside documents - I would prefer some); >>>>>> however >>>>>> the JEP does not replace documentation in the source as it is not >>>>>> detailed enough, and the JEP cannot be changed any more. The JEP is >>>>>> interesting for the alternatives considered though. >>>>>> >>>>>> Ideally, to reasonably understand the feature I would like to not >>>>>> need >>>>>> to spend hours looking through the implementation. >>>>>> >>>>>> - how many threads does this feature add, and are supported by >>>>>> this code >>>>>> (for which parts)? Synchronization between them? >>>>>> >>>>>> - the comments "For XY" in the StringDedup header file do not >>>>>> give any >>>>>> hint about what they are doing (not sure if it is interesting as >>>>>> it is >>>>>> easy to find out the only callers). It seems that this >>>>>> information has >>>>>> been put at the place where these methods are invoked (or just >>>>>> somewhere >>>>>> in the cpp file). Imo the methods and members (except trivial) >>>>>> should be >>>>>> documented in the hpp file in the class body, i.e. in the interface. >>>>>> >>>>>> - there are a few methods (also in the StringDedupTable class) in >>>>>> StringDedup where the results of the methods do not seem obvious and >>>>>> should be commented. >>>>>> >>>>>> - I really dislike the location of the "Candidate selection" >>>>>> explanation >>>>>> within the cpp file. I would not have found it if I were not >>>>>> looking for >>>>>> some documentation ;) It would probably fit very well in the above >>>>>> mentioned overview paragraphs. >>>>>> >>>>>> - please add the static helper functions that implement the actual >>>>>> policies (is_candidate_from_mark, is_candidate_from_evacuation) >>>>>> to the >>>>>> StringDedup class (as private static methods) so that they can be >>>>>> found >>>>>> easily. Static non-class helper functions in cpp files should >>>>>> only be >>>>>> used for stuff that is really unimportant and not relevant to the >>>>>> code >>>>>> imo. However these seem to be _the_ central policy methods for >>>>>> string >>>>>> dedup. >>>>>> So it would be nice to have them mentioned in the class/hpp file and >>>>>> documented there. >>>>>> >>>>>> - facts about the code are repeated, e.g. that the table size >>>>>> must be a >>>>>> power of two. This is nice, but has a few drawbacks: the compiler >>>>>> does >>>>>> not care about comments at all, so it might be useful to put >>>>>> asserts in >>>>>> these places instead, and second, it's just duplication of text >>>>>> with the >>>>>> usual problem of getting out of date. >>>>>> >>>>>> - I am not completely sure why the class has not been called >>>>>> G1StringDedup. It won't work without G1 anyway and calls G1 >>>>>> internals >>>>>> all the time. There does not seem to be a way to make it more >>>>>> generic >>>>>> without considerable refactoring. It is fine to keep the name >>>>>> though, >>>>>> just asking. I also know that this feature does not want to be a >>>>>> generic >>>>>> solution. >>>>>> >>>>>> - is it possible to separate out the statistics variables of >>>>>> StringDedupTable into either a separate class instead of using >>>>>> globals >>>>>> or separate them a little from the other unrelated members? (And >>>>>> describe them) This would improve readability. >>>>>> >>>>>> I *think* this is everything from >>>>>> StringDedupTable::_entries_added to >>>>>> _rehash_count? I have no idea without searching for every use and >>>>>> deducing from that. >>>>>> >>>>>> - when implementing the hash table for the StringDedupTable, did you >>>>>> consider trying to reuse one of the existing hashmap >>>>>> implementations? >>>>>> And potentially extend them. >>>>>> >>>>>> Imo we already have way too many ad-hoc implementations of often >>>>>> used >>>>>> data structures (particularly) in G1 and GC code. >>>>>> >>>>>> - please document the properties of the StringDedupTable (type, >>>>>> hashing, >>>>>> automatic resizing behavior, thread safety, ...). >>>>>> >>>>>> - please explain what StringDedupTable/Cache are and what they >>>>>> are used >>>>>> for in context of the feature. >>>>>> >>>>>> - stringdedup.cpp, line 1309: candicate -> candidate >>>>>> >>>>>> Thanks, >>>>>> Thomas >>>>>> >>>>>> >>>>> >>>> >>> >> > From vladimir.kozlov at oracle.com Fri Mar 14 19:14:37 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 14 Mar 2014 12:14:37 -0700 Subject: [8u20] RFR(M): 8031754: Type speculation should favor profile data from outermost inlined method In-Reply-To: References: Message-ID: <5323551D.3030508@oracle.com> Yes, until we merge ppc64 port into 8u20 we will have these problems. You changes are good. Thanks, Vladimir On 3/14/14 11:50 AM, Roland Westrelin wrote: > Please, review backport to 8u20. The patch applies cleanly except for c2_globals.hpp. jdk8u doesn?t have doesn?t have: > > product_pd(bool, TrapBasedRangeChecks, \ > "Generate code for range checks that uses a cmp and trap " \ > "instruction raising SIGTRAP. Used on PPC64.") \ > > from: > > 8028767: PPC64: (part 121): smaller shared changes needed to build C2 > > that I don?t think we want to backport. > > > It?s been in jdk9 for 2 weeks. 8032633 that enabled type speculation and stresses this change was pushed a week ago. > > http://cr.openjdk.java.net/~roland/8031754/webrev.02/ > https://bugs.openjdk.java.net/browse/JDK-8031754 > http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/484a359ff649 > > Roland. > From daniel.daugherty at oracle.com Fri Mar 14 20:52:24 2014 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 14 Mar 2014 14:52:24 -0600 Subject: RFR (XS) 6976636: JVM/TI test ex03t001 fails assertion In-Reply-To: <53222208.9010009@oracle.com> References: <53222208.9010009@oracle.com> Message-ID: <53236C08.9090506@oracle.com> On 3/13/14 3:24 PM, serguei.spitsyn at oracle.com wrote: > Please, review the fix for: > https://bugs.openjdk.java.net/browse/JDK-6976636 > > > Open webrev: > http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/6976636-JVMTI-unload.1 > Thumbs up. src/share/vm/prims/jvmtiExport.cpp No comments. As for the extension call back stuff, I think the idea is to pass a little more info about which thread motivated the class unload. An agent might be interested in such a statistic. > The class unload event is implemented as a source debug extension Not "source debug extension", but "JVM/TI extension event". The source debug extension is a different thing (see JDI). > As a consequence of this fix the failing nsk.jvmti test ex03t001 > must be tweeked as well. Since you are changing an assert here, I'm curious why the test needs to change. Does the test presume the jthread refers to a real JavaThread? Dan > > Summary: > > This is a Nightly Stabilization issue. > The class unload event post in the JvmtiExport.cpp fails at the assert > assuming the proxy thread causing the class unload must be a > JavaThread. > It is not the case when the proxy thread is a CMS ConcurrentGCThread. > > This fix is to relax this check allowing this case. > The downside of this approach is that the jthread parameter passed > to the > even handler callback is NULL for the CMS thread. > This must be Ok as it would indicates that the proxy thread is a CMS > thread. > > In fact, I have a doubt this parameter is needed at all. > So that an alternative approach could be to remove it from the event > callback. > My preference is to leave the jthread parameter as it is as it does > not impact anything. > The class unload event is implemented as a source debug extension > for the > sake of testing the extension functionality. The JDWP agent does not > use it. > The class unload events are not expected to be used by any external > tool. > > As a consequence of this fix the failing nsk.jvmti test ex03t001 > must be tweeked as well. > If review is positive then I'll file a bug and proceed with the test > fix. > > Testing: > Running the failing test: nsk/jvmti/scenarios/extension/EX03/ex03t001 > > > Thanks, > Serguei From asmundak at google.com Fri Mar 14 21:11:17 2014 From: asmundak at google.com (Alexander Smundak) Date: Fri, 14 Mar 2014 14:11:17 -0700 Subject: RFR (M): 8036767 PPC64: Support for little endian execution model In-Reply-To: References: <531E8309.2060206@oracle.com> <531ED06F.5040905@oracle.com> <4295855A5C1DE049A61835A1887419CC2CEA6A09@DEWDFEMB12A.global.corp.sap> <531F9370.4020305@oracle.com> <5320E5F8.3020500@oracle.com> Message-ID: Ping. On Wed, Mar 12, 2014 at 4:59 PM, Alexander Smundak wrote: > I was concerned by the term 'variant', which might suggest that the applications > built for PPC64 and PPC64LE are binary compatible. They are not. > > On Wed, Mar 12, 2014 at 3:55 PM, David Holmes wrote: >> On 12/03/2014 9:19 AM, Alexander Smundak wrote: >>> >>> On Tue, Mar 11, 2014 at 3:51 PM, Vladimir Kozlov >>> wrote: >>>> >>>> It would only help if you could do cross compilation to have both build >>>> variants at the same place. Currently you can only build le variant on >>>> ppc64le machine and vice versa. That is why, I think, David asked if we >>>> can >>>> control what variant to build. >>> >>> Just to clarify the situation a bit: ppc64le is not a variant of ppc64. >>> That is, >>> an application compiled for the little-endian PowerPC64 does not "just >>> run" on >>> the big-endian PowerPC64 (albeit OS can have such feature, similar to the >>> ability of the Linux running on x86_64 CPU to run 32-bit x86 >>> applications). >>> So ppc64le is a different architecture from ppc64. >> >> >> I disagree with that classification for "architecture" and it seems at odds >> with the literature which describes the endian-ness selection as a "mode". >> >> David >> ----- >> >> >>>> I would like to see the changes based on Volker suggestion. We can >>>> compare >>>> them and decide which way to go. >>> >>> Volker has the detailed suggestion here: >>> >>> http://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/2014-March/001790.html >>> and it involves additional Make variable and if statements in the >>> platform makefile >>> where they are not supposed to be present. >>> >> From serguei.spitsyn at oracle.com Fri Mar 14 21:29:16 2014 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 14 Mar 2014 14:29:16 -0700 Subject: RFR (XS) 6976636: JVM/TI test ex03t001 fails assertion In-Reply-To: <53236C08.9090506@oracle.com> References: <53222208.9010009@oracle.com> <53236C08.9090506@oracle.com> Message-ID: <532374AC.10309@oracle.com> On 3/14/14 1:52 PM, Daniel D. Daugherty wrote: > On 3/13/14 3:24 PM, serguei.spitsyn at oracle.com wrote: >> Please, review the fix for: >> https://bugs.openjdk.java.net/browse/JDK-6976636 >> >> >> Open webrev: >> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/6976636-JVMTI-unload.1 >> > > Thumbs up. Thank you a lot for the code review! The fix is simple but the issue is ugly and so, generates questions. :) > > src/share/vm/prims/jvmtiExport.cpp > No comments. > > As for the extension call back stuff, I think the idea > is to pass a little more info about which thread motivated > the class unload. An agent might be interested in such a > statistic. Right. It is why I prefer to keep this callback parameter. BTW, the jdwp agent is posting synthetic class unload events. If I remember correctly, they have no jthread parameter. > > > The class unload event is implemented as a source debug extension > > Not "source debug extension", but "JVM/TI extension event". The > source debug extension is a different thing (see JDI). Right. My head is broken, I keep using incorrect terms. :) > > > As a consequence of this fix the failing nsk.jvmti test ex03t001 > > must be tweeked as well. > > Since you are changing an assert here, I'm curious why the test > needs to change. Does the test presume the jthread refers to > a real JavaThread? The test checks and fails if the NULL is returned. I'll file a bug with a necessary DKFL comment to recognize this failure mode until the SQE testbase build with the fix gets released. Thanks, Serguei > > > Dan > > > >> >> Summary: >> >> This is a Nightly Stabilization issue. >> The class unload event post in the JvmtiExport.cpp fails at the assert >> assuming the proxy thread causing the class unload must be a >> JavaThread. >> It is not the case when the proxy thread is a CMS ConcurrentGCThread. >> >> This fix is to relax this check allowing this case. >> The downside of this approach is that the jthread parameter passed >> to the >> even handler callback is NULL for the CMS thread. >> This must be Ok as it would indicates that the proxy thread is a >> CMS thread. >> >> In fact, I have a doubt this parameter is needed at all. >> So that an alternative approach could be to remove it from the >> event callback. >> My preference is to leave the jthread parameter as it is as it does >> not impact anything. >> The class unload event is implemented as a source debug extension >> for the >> sake of testing the extension functionality. The JDWP agent does >> not use it. >> The class unload events are not expected to be used by any external >> tool. >> >> As a consequence of this fix the failing nsk.jvmti test ex03t001 >> must be tweeked as well. >> If review is positive then I'll file a bug and proceed with the >> test fix. >> >> Testing: >> Running the failing test: nsk/jvmti/scenarios/extension/EX03/ex03t001 >> >> >> Thanks, >> Serguei > From volker.simonis at gmail.com Fri Mar 14 21:33:08 2014 From: volker.simonis at gmail.com (Volker Simonis) Date: Fri, 14 Mar 2014 22:33:08 +0100 Subject: RFR[9](M): 8037476: Tag MetaSpace objects with size and type to make them traceable at runtime Message-ID: Hi, I'm looking for some comments and feedback on the following change which enhances MetaSpace objects with tags containng their size and type to make them traceable at runtime: http://cr.openjdk.java.net/~simonis/webrevs/8037476/ https://bugs.openjdk.java.net/browse/JDK-8037476 The HotSpot VM contains some debugging helper functions which can be called during a live debugging session. One of these functions ("find(intptr_t x)" in debug.cpp) calls os::print_location() which tries very hard to print some useful information for a given pointer (e.g. if the pointer points into the Java heap, the function will call "print_on()" for the corresponding oop which provides a lot of useful meta-information for that object). This functionality is not only restricted to live debugging. It is for example also used in the hs_err reporting in the "Register to memory mapping" section. In the past (i.e. before the PermGen removal in HS25) os::print_location() worked equally well for true Java objects and meta objects (like Symbols, Klass mirrors, etc.) because these meta objects were stores as plain oops in the permanent generation of the Java heap. However, after the permgen removal, his metadata is now stored in the so called "Metaspace" which currently does not allow for a reverse mapping of a native pointer back to the owning meta object. The consequence is that os::print_location() currently only prints "is pointing into metadata" for all native pointers pointing into the metaspace. This change transparently adds a tag in front of every MetaspaceObj which contains the object's size and type. Together with a dispatching method which is necessary because some of the classes in the MetaspaceObj type hierarchy don't have a virtual function table it becomes now possible to walk and print all the MetaspaceObj objects in a typesafe way. Of course, tagging each MetaspaceObj adds some space overhead but in my opinion, this overhead is not very big. I've done some measuring (with -XX:+PrintGCDetails -XX:+Verbose) and found that on 64-bit platforms the tagging consumes about 4% additional 'used' metaspace, but only about 3% additional committed metaspace memory (that's probably because we waste some space anyway because of chunking). On 32-bit, the overhead is a little bigger - currently about 6%. However that's because of the fact that although a Metaword is only 32 bit on a 32-bit platform, the memory allocated for MetaspaceObjs is kept 64-bit aligned. In order to keep the prototype simple, the current implementation just uses two Metawords for the tag (although one would be enough to hold the tag information). Changing this is possible and could theoretically lead to an even smaller overhead on 32-bit platforms compared to 64-bit ones (because we could use the otherwise wasted Metaword in about 50% of the allocations). Unfortunately, the current Metaspace implementation does the padding to 64-bit before the actual allocation which makes it a little tricky to implement the tagging. The solution would be to not pad the MetaspaceObj's size to 64-bit but instead align their starting address during allocation. The current implementation is protected with "NOT_PRODUCT" and switchable at startup with -XX:+UseMetaspaceTags. If metaspace tagging is switched on, we can now call "find()" in the debugger for a metaspace address and get again an output as attached at the end of this mail. Also the output in the "Register to memory mapping" section of the hs_err file can now contain more useful information (i.e. see the line for "R13" which was simple "is pointing into metadata" before): Register to memory mapping: RAX=0x0000000854bf6698 is an unknown value ... R11=0x00007fbb412312c0 is at entry_point+32 in (nmethod*)0x00007fbb41231150 R12=0x0000000000000000 is an unknown value R13=0x00007fbb552ed9e7 is pointing into metadata (start=0x00007fbb552ed9b0, size=2, tag=9, type=ConstMethod) {constMethod} - method: 0x00007fbb552eda00 {method} {0x00007fbb552eda00} '' '()V' in 'sun/font/FontManagerNativeLibrary' R14=0x00007fbb19345d08 is pointing into the stack for thread: 0x00007fbae400b800 As a future enhancement and if we decide to make this permanent feature (i.e. not switachable at startup) this could also be implemented more elegantly by simply storing the actual size and type of every object directly in the 'MetaspaceObj'. Another possibility would be to store just the object size in 'MetaspaceObj' and add virtual print() function, although that would probably require even more overhead. I've tested this implementation so far on Linux/x86/x86_64/ppc64 and did some basic testing on Solaris/Sparcv9. The only thing I haven't looked at until now is support for class data sharing. Thank you and best regards, Volker Below you can see the output of the debugger helper function "find()" with the new metaspace tagging implementation when called during a live debugging session: (gdb) call find(0x00007fffec41d4c0) "Executing find" 0x00007fffec41d4c0 is pointing into metadata (start=0x00007fffec41d4c0, size=5, tag=8, type=Method) {method} - this oop: 0x00007fffec41d4c0 - method holder: synchronized 'IntLoopWithGC' - constants: 0x00007fffec41d098 constant pool [78] {0x00007fffec41d098} for synchronized 'IntLoopWithGC' cache=0x00007fffec41d698 - access: 0xc0000008 static - name: 'loop' - signature: '(I)V' ... (gdb) call find(0xafdda210) "Executing find" 0xafdda210 is pointing into metadata (start=0xafdda210, size=2, tag=5, type=TypeArrayU4) 0: 0xafdda300 1: 0xafdda260 2: 0xafdda3f8 (gdb) call find(0xafdda2a0) "Executing find" 0xafdda2a0 is pointing into metadata (start=0xafdda2a0, size=20, tag=9, type=ConstMethod) {constMethod} - method: 0xafdda300 {method} {0xafdda300} 'loop' '(I)V' in synchronized 'IntLoopWithGC' - stackmap data: Array(0xafdda340) (gdb) call findpc(0x00007fffec41d300) "Executing findpc" 0x00007fffec41d300 is pointing into metadata (start=0x00007fffec41d098, size=82, tag=11, type=ConstantPool) {constant pool} - holder: 0x0000000100060038 - cache: 0x00007fffec41d698 - resolved_references: 0x00000000d6bd8000 - reference_map: 0x00007fffec41d870 - 1 : Method : klass_index=23 name_and_type_index=39 - 2 : Integer : 1000000 - 3 : Field : klass_index=22 name_and_type_index=40 ... From daniel.daugherty at oracle.com Fri Mar 14 21:45:11 2014 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 14 Mar 2014 15:45:11 -0600 Subject: RFR (XS) 6976636: JVM/TI test ex03t001 fails assertion In-Reply-To: <532374AC.10309@oracle.com> References: <53222208.9010009@oracle.com> <53236C08.9090506@oracle.com> <532374AC.10309@oracle.com> Message-ID: <53237867.3090206@oracle.com> On 3/14/14 3:29 PM, serguei.spitsyn at oracle.com wrote: > On 3/14/14 1:52 PM, Daniel D. Daugherty wrote: >> On 3/13/14 3:24 PM, serguei.spitsyn at oracle.com wrote: >>> Please, review the fix for: >>> https://bugs.openjdk.java.net/browse/JDK-6976636 >>> >>> >>> Open webrev: >>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/6976636-JVMTI-unload.1 >>> >> >> Thumbs up. > > Thank you a lot for the code review! > The fix is simple but the issue is ugly and so, generates questions. :) > >> >> src/share/vm/prims/jvmtiExport.cpp >> No comments. >> >> As for the extension call back stuff, I think the idea >> is to pass a little more info about which thread motivated >> the class unload. An agent might be interested in such a >> statistic. > > Right. > It is why I prefer to keep this callback parameter. > BTW, the jdwp agent is posting synthetic class unload events. > If I remember correctly, they have no jthread parameter. There are several disagreements about what is in a JVM/TI event versus what is in a JDI event. It can really mess up your mind... Dan > >> >> > The class unload event is implemented as a source debug extension >> >> Not "source debug extension", but "JVM/TI extension event". The >> source debug extension is a different thing (see JDI). > > Right. > My head is broken, I keep using incorrect terms. :) > >> >> > As a consequence of this fix the failing nsk.jvmti test ex03t001 >> > must be tweeked as well. >> >> Since you are changing an assert here, I'm curious why the test >> needs to change. Does the test presume the jthread refers to >> a real JavaThread? > > The test checks and fails if the NULL is returned. > I'll file a bug with a necessary DKFL comment to recognize this > failure mode until the SQE testbase build with the fix gets released. > > > Thanks, > Serguei > >> >> >> Dan >> >> >> >>> >>> Summary: >>> >>> This is a Nightly Stabilization issue. >>> The class unload event post in the JvmtiExport.cpp fails at the >>> assert >>> assuming the proxy thread causing the class unload must be a >>> JavaThread. >>> It is not the case when the proxy thread is a CMS ConcurrentGCThread. >>> >>> This fix is to relax this check allowing this case. >>> The downside of this approach is that the jthread parameter passed >>> to the >>> even handler callback is NULL for the CMS thread. >>> This must be Ok as it would indicates that the proxy thread is a >>> CMS thread. >>> >>> In fact, I have a doubt this parameter is needed at all. >>> So that an alternative approach could be to remove it from the >>> event callback. >>> My preference is to leave the jthread parameter as it is as it >>> does not impact anything. >>> The class unload event is implemented as a source debug extension >>> for the >>> sake of testing the extension functionality. The JDWP agent does >>> not use it. >>> The class unload events are not expected to be used by any >>> external tool. >>> >>> As a consequence of this fix the failing nsk.jvmti test ex03t001 >>> must be tweeked as well. >>> If review is positive then I'll file a bug and proceed with the >>> test fix. >>> >>> Testing: >>> Running the failing test: nsk/jvmti/scenarios/extension/EX03/ex03t001 >>> >>> >>> Thanks, >>> Serguei >> > From roland.westrelin at oracle.com Fri Mar 14 22:13:38 2014 From: roland.westrelin at oracle.com (Roland Westrelin) Date: Fri, 14 Mar 2014 23:13:38 +0100 Subject: [8u20] RFR(S): 8035841: assert(dp_src->tag() == dp_dst->tag()) failed: should be same tags 1 != 0 at ciMethodData.cpp:90 Message-ID: <28364BF7-38E1-44FA-BD38-D31F14A78C39@oracle.com> Please, review backport to 8u20. The patch applies cleanly except. It?s been in jdk9 for more than 1 week and fixes an important issue with 8031752 that is also backported. http://cr.openjdk.java.net/~roland/8035841/webrev.02/ https://bugs.openjdk.java.net/browse/JDK-8035841 http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/53ed0f89f44e Roland. From vladimir.kozlov at oracle.com Fri Mar 14 22:20:18 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 14 Mar 2014 15:20:18 -0700 Subject: [8u20] RFR(S): 8035841: assert(dp_src->tag() == dp_dst->tag()) failed: should be same tags 1 != 0 at ciMethodData.cpp:90 In-Reply-To: <28364BF7-38E1-44FA-BD38-D31F14A78C39@oracle.com> References: <28364BF7-38E1-44FA-BD38-D31F14A78C39@oracle.com> Message-ID: <532380A2.7020603@oracle.com> Looks good. Thanks, Vladimir On 3/14/14 3:13 PM, Roland Westrelin wrote: > Please, review backport to 8u20. The patch applies cleanly except. It?s been in jdk9 for more than 1 week and fixes an important issue with 8031752 that is also backported. > > http://cr.openjdk.java.net/~roland/8035841/webrev.02/ > https://bugs.openjdk.java.net/browse/JDK-8035841 > http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/53ed0f89f44e > > Roland. > From roland.westrelin at oracle.com Fri Mar 14 22:22:47 2014 From: roland.westrelin at oracle.com (Roland Westrelin) Date: Fri, 14 Mar 2014 23:22:47 +0100 Subject: 8u20] RFR(XXS): 8036092: [TESTBUG] compiler/uncommontrap/TestSpecTrapClassUnloading.java fails with: Unrecognized VM option 'UseTypeSpeculation' Message-ID: Please, review backport to 8u20. The patch applies cleanly. Albert pushed it to jdk9 a bit more than 1 week ago. It fixes the regression test for 8031752. http://cr.openjdk.java.net/~roland/8036092/webrev.00/ https://bugs.openjdk.java.net/browse/JDK-8036092 http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/27689a7550a8 Roland. From roland.westrelin at oracle.com Fri Mar 14 22:29:29 2014 From: roland.westrelin at oracle.com (Roland Westrelin) Date: Fri, 14 Mar 2014 23:29:29 +0100 Subject: [8u20] RFR(XXS): 8032633: Enable type speculation by default Message-ID: Please, review backport to 8u20. The patch applies cleanly. This was pushed to jdk9 a bit more than 1 week ago. http://cr.openjdk.java.net/~roland/8032633/webrev.01/ https://bugs.openjdk.java.net/browse/JDK-8032633 http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/45b0159f30f2 Roland. From vladimir.kozlov at oracle.com Fri Mar 14 22:57:49 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 14 Mar 2014 15:57:49 -0700 Subject: 8u20] RFR(XXS): 8036092: [TESTBUG] compiler/uncommontrap/TestSpecTrapClassUnloading.java fails with: Unrecognized VM option 'UseTypeSpeculation' In-Reply-To: References: Message-ID: <5323896D.40002@oracle.com> Good. Thanks, Vladimir On 3/14/14 3:22 PM, Roland Westrelin wrote: > Please, review backport to 8u20. The patch applies cleanly. Albert pushed it to jdk9 a bit more than 1 week ago. It fixes the regression test for > 8031752. > > http://cr.openjdk.java.net/~roland/8036092/webrev.00/ > https://bugs.openjdk.java.net/browse/JDK-8036092 > http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/27689a7550a8 > > Roland. > From vladimir.kozlov at oracle.com Fri Mar 14 22:58:40 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 14 Mar 2014 15:58:40 -0700 Subject: [8u20] RFR(XXS): 8032633: Enable type speculation by default In-Reply-To: References: Message-ID: <532389A0.2020009@oracle.com> Okay. Vladimir On 3/14/14 3:29 PM, Roland Westrelin wrote: > Please, review backport to 8u20. The patch applies cleanly. This was pushed to jdk9 a bit more than 1 week ago. > > http://cr.openjdk.java.net/~roland/8032633/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8032633 > http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/45b0159f30f2 > > Roland. > > From vladimir.kozlov at oracle.com Fri Mar 14 23:01:40 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 14 Mar 2014 16:01:40 -0700 Subject: RFR (S) 8037226: compiler/7196199/Test7196199.java fails on 32-bit linux with MaxVectorSize > 16 In-Reply-To: <53229D77.6030405@oracle.com> References: <53229D77.6030405@oracle.com> Message-ID: <53238A54.3070408@oracle.com> I updated changes to include more ymm registers to test and print them in debug VM. http://cr.openjdk.java.net/~kvn/8037226/webrev.01/ Thanks, Vladimir On 3/13/14 11:11 PM, Vladimir Kozlov wrote: > WARNING: it is ugly but I don't know an other way to fix the problem and > keep wide AVX vectors. > > Some operating systems (or VMs) does not restore upper 128 bit of AVX > registers YMM after signal processing. About year ago Mac OS had this > problem (8003878). And now we found 32-bit Ubuntu (Linux 3.8.0-29) which > has the same problem. > > Instead of reducing vectorization to 16 bytes (XMM registers) I added > code which verifies YMM state after signal processing and set limit on > vector's size. The code is part of cpuid info code which JVM executes > during startup. > > https://bugs.openjdk.java.net/browse/JDK-8037226 > http://cr.openjdk.java.net/~kvn/8037226/webrev/ > > Tested on failing machines. > > thanks, > Vladimir From roland.westrelin at oracle.com Fri Mar 14 23:06:31 2014 From: roland.westrelin at oracle.com (Roland Westrelin) Date: Sat, 15 Mar 2014 00:06:31 +0100 Subject: [8u20] RFR(XXS): 8032633: Enable type speculation by default In-Reply-To: <532389A0.2020009@oracle.com> References: <532389A0.2020009@oracle.com> Message-ID: <0C0ED1E2-D7BA-49AF-947E-57929913DE8A@oracle.com> Thanks for all the backport reviews, Vladimir. Roland. From igor.veresov at oracle.com Fri Mar 14 23:23:54 2014 From: igor.veresov at oracle.com (Igor Veresov) Date: Fri, 14 Mar 2014 16:23:54 -0700 Subject: RFR (S) 8037226: compiler/7196199/Test7196199.java fails on 32-bit linux with MaxVectorSize > 16 In-Reply-To: <53238A54.3070408@oracle.com> References: <53229D77.6030405@oracle.com> <53238A54.3070408@oracle.com> Message-ID: <9EF2DA69-9C5E-43E5-8C0C-E38CD91C41CF@oracle.com> Look good, except for the typo in vm_version_x86.cpp: 55 // Address of instruction which causes SEGV 56 address VM_Version::_cpuinfo_cont_addr = 0; The comment probably has to say: "Address of the instruction after the one that causes SEGV" ? igor On Mar 14, 2014, at 4:01 PM, Vladimir Kozlov wrote: > I updated changes to include more ymm registers to test and print them in debug VM. > > http://cr.openjdk.java.net/~kvn/8037226/webrev.01/ > > Thanks, > Vladimir > > On 3/13/14 11:11 PM, Vladimir Kozlov wrote: >> WARNING: it is ugly but I don't know an other way to fix the problem and >> keep wide AVX vectors. >> >> Some operating systems (or VMs) does not restore upper 128 bit of AVX >> registers YMM after signal processing. About year ago Mac OS had this >> problem (8003878). And now we found 32-bit Ubuntu (Linux 3.8.0-29) which >> has the same problem. >> >> Instead of reducing vectorization to 16 bytes (XMM registers) I added >> code which verifies YMM state after signal processing and set limit on >> vector's size. The code is part of cpuid info code which JVM executes >> during startup. >> >> https://bugs.openjdk.java.net/browse/JDK-8037226 >> http://cr.openjdk.java.net/~kvn/8037226/webrev/ >> >> Tested on failing machines. >> >> thanks, >> Vladimir From vladimir.kozlov at oracle.com Fri Mar 14 23:30:56 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 14 Mar 2014 16:30:56 -0700 Subject: RFR (S) 8037226: compiler/7196199/Test7196199.java fails on 32-bit linux with MaxVectorSize > 16 In-Reply-To: <9EF2DA69-9C5E-43E5-8C0C-E38CD91C41CF@oracle.com> References: <53229D77.6030405@oracle.com> <53238A54.3070408@oracle.com> <9EF2DA69-9C5E-43E5-8C0C-E38CD91C41CF@oracle.com> Message-ID: <53239130.7090706@oracle.com> Thank you, Igor I fixed the comment as you suggested. Thanks, Vladimir On 3/14/14 4:23 PM, Igor Veresov wrote: > Look good, except for the typo in vm_version_x86.cpp: > 55 // Address of instruction which causes SEGV > 56 address VM_Version::_cpuinfo_cont_addr = 0; > The comment probably has to say: "Address of the instruction after the one that causes SEGV" ? > > igor > > On Mar 14, 2014, at 4:01 PM, Vladimir Kozlov wrote: > >> I updated changes to include more ymm registers to test and print them in debug VM. >> >> http://cr.openjdk.java.net/~kvn/8037226/webrev.01/ >> >> Thanks, >> Vladimir >> >> On 3/13/14 11:11 PM, Vladimir Kozlov wrote: >>> WARNING: it is ugly but I don't know an other way to fix the problem and >>> keep wide AVX vectors. >>> >>> Some operating systems (or VMs) does not restore upper 128 bit of AVX >>> registers YMM after signal processing. About year ago Mac OS had this >>> problem (8003878). And now we found 32-bit Ubuntu (Linux 3.8.0-29) which >>> has the same problem. >>> >>> Instead of reducing vectorization to 16 bytes (XMM registers) I added >>> code which verifies YMM state after signal processing and set limit on >>> vector's size. The code is part of cpuid info code which JVM executes >>> during startup. >>> >>> https://bugs.openjdk.java.net/browse/JDK-8037226 >>> http://cr.openjdk.java.net/~kvn/8037226/webrev/ >>> >>> Tested on failing machines. >>> >>> thanks, >>> Vladimir > From christian.thalinger at oracle.com Fri Mar 14 23:48:21 2014 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Fri, 14 Mar 2014 16:48:21 -0700 Subject: RFR (S) 8037226: compiler/7196199/Test7196199.java fails on 32-bit linux with MaxVectorSize > 16 In-Reply-To: <53238A54.3070408@oracle.com> References: <53229D77.6030405@oracle.com> <53238A54.3070408@oracle.com> Message-ID: <0B8AB723-4BFE-4719-886A-30835F7A5E87@oracle.com> + static bool os_supports_avx_vectors() { + if (!supports_avx()) { + return false; + } A small comment here would be nice. Maybe: + // Verify that OS save/restore AVX registers. + int nreg = 2 LP64_ONLY(+2); + for (int i = 0; i < 8 * nreg; i++) { // 32 bytes 4 ymm registers + if (_cpuid_info.ymm_save[i] != ymm_test_value()) { + return false; + } + } + return true; + } Otherwise this looks good. On Mar 14, 2014, at 4:01 PM, Vladimir Kozlov wrote: > I updated changes to include more ymm registers to test and print them in debug VM. > > http://cr.openjdk.java.net/~kvn/8037226/webrev.01/ > > Thanks, > Vladimir > > On 3/13/14 11:11 PM, Vladimir Kozlov wrote: >> WARNING: it is ugly but I don't know an other way to fix the problem and >> keep wide AVX vectors. >> >> Some operating systems (or VMs) does not restore upper 128 bit of AVX >> registers YMM after signal processing. About year ago Mac OS had this >> problem (8003878). And now we found 32-bit Ubuntu (Linux 3.8.0-29) which >> has the same problem. >> >> Instead of reducing vectorization to 16 bytes (XMM registers) I added >> code which verifies YMM state after signal processing and set limit on >> vector's size. The code is part of cpuid info code which JVM executes >> during startup. >> >> https://bugs.openjdk.java.net/browse/JDK-8037226 >> http://cr.openjdk.java.net/~kvn/8037226/webrev/ >> >> Tested on failing machines. >> >> thanks, >> Vladimir From vladimir.kozlov at oracle.com Sat Mar 15 00:21:02 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 14 Mar 2014 17:21:02 -0700 Subject: RFR (S) 8037226: compiler/7196199/Test7196199.java fails on 32-bit linux with MaxVectorSize > 16 In-Reply-To: <0B8AB723-4BFE-4719-886A-30835F7A5E87@oracle.com> References: <53229D77.6030405@oracle.com> <53238A54.3070408@oracle.com> <0B8AB723-4BFE-4719-886A-30835F7A5E87@oracle.com> Message-ID: <53239CEE.4030306@oracle.com> Thank you, Christian I added: // Verify that OS save/restore all bits of AVX registers // during signal processing. Thanks, Vladimir On 3/14/14 4:48 PM, Christian Thalinger wrote: > + static bool os_supports_avx_vectors() { > + if (!supports_avx()) { > + return false; > + } > > A small comment here would be nice. Maybe: > > + // Verify that OS save/restore AVX registers. > > + int nreg = 2 LP64_ONLY(+2); > + for (int i = 0; i < 8 * nreg; i++) { // 32 bytes 4 ymm registers > + if (_cpuid_info.ymm_save[i] != ymm_test_value()) { > + return false; > + } > + } > + return true; > + } > > Otherwise this looks good. > > On Mar 14, 2014, at 4:01 PM, Vladimir Kozlov wrote: > >> I updated changes to include more ymm registers to test and print them in debug VM. >> >> http://cr.openjdk.java.net/~kvn/8037226/webrev.01/ >> >> Thanks, >> Vladimir >> >> On 3/13/14 11:11 PM, Vladimir Kozlov wrote: >>> WARNING: it is ugly but I don't know an other way to fix the problem and >>> keep wide AVX vectors. >>> >>> Some operating systems (or VMs) does not restore upper 128 bit of AVX >>> registers YMM after signal processing. About year ago Mac OS had this >>> problem (8003878). And now we found 32-bit Ubuntu (Linux 3.8.0-29) which >>> has the same problem. >>> >>> Instead of reducing vectorization to 16 bytes (XMM registers) I added >>> code which verifies YMM state after signal processing and set limit on >>> vector's size. The code is part of cpuid info code which JVM executes >>> during startup. >>> >>> https://bugs.openjdk.java.net/browse/JDK-8037226 >>> http://cr.openjdk.java.net/~kvn/8037226/webrev/ >>> >>> Tested on failing machines. >>> >>> thanks, >>> Vladimir > From david.holmes at oracle.com Mon Mar 17 04:13:35 2014 From: david.holmes at oracle.com (David Holmes) Date: Mon, 17 Mar 2014 14:13:35 +1000 Subject: RFR (M): 8036767 PPC64: Support for little endian execution model In-Reply-To: References: <531E8309.2060206@oracle.com> <531ED06F.5040905@oracle.com> <4295855A5C1DE049A61835A1887419CC2CEA6A09@DEWDFEMB12A.global.corp.sap> <531F9370.4020305@oracle.com> <5320E5F8.3020500@oracle.com> Message-ID: <5326766F.80104@oracle.com> On 15/03/2014 7:11 AM, Alexander Smundak wrote: > Ping. My position hasn't changed. I don't think this needs to be, or should be, a distinct architecture. I've added build-dev to cc list to see what our build experts think. David ----- > On Wed, Mar 12, 2014 at 4:59 PM, Alexander Smundak wrote: >> I was concerned by the term 'variant', which might suggest that the applications >> built for PPC64 and PPC64LE are binary compatible. They are not. >> >> On Wed, Mar 12, 2014 at 3:55 PM, David Holmes wrote: >>> On 12/03/2014 9:19 AM, Alexander Smundak wrote: >>>> >>>> On Tue, Mar 11, 2014 at 3:51 PM, Vladimir Kozlov >>>> wrote: >>>>> >>>>> It would only help if you could do cross compilation to have both build >>>>> variants at the same place. Currently you can only build le variant on >>>>> ppc64le machine and vice versa. That is why, I think, David asked if we >>>>> can >>>>> control what variant to build. >>>> >>>> Just to clarify the situation a bit: ppc64le is not a variant of ppc64. >>>> That is, >>>> an application compiled for the little-endian PowerPC64 does not "just >>>> run" on >>>> the big-endian PowerPC64 (albeit OS can have such feature, similar to the >>>> ability of the Linux running on x86_64 CPU to run 32-bit x86 >>>> applications). >>>> So ppc64le is a different architecture from ppc64. >>> >>> >>> I disagree with that classification for "architecture" and it seems at odds >>> with the literature which describes the endian-ness selection as a "mode". >>> >>> David >>> ----- >>> >>> >>>>> I would like to see the changes based on Volker suggestion. We can >>>>> compare >>>>> them and decide which way to go. >>>> >>>> Volker has the detailed suggestion here: >>>> >>>> http://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/2014-March/001790.html >>>> and it involves additional Make variable and if statements in the >>>> platform makefile >>>> where they are not supposed to be present. >>>> >>> From goetz.lindenmaier at sap.com Mon Mar 17 08:18:50 2014 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 17 Mar 2014 08:18:50 +0000 Subject: RFR (M): 8036767 PPC64: Support for little endian execution model In-Reply-To: <5326766F.80104@oracle.com> References: <531E8309.2060206@oracle.com> <531ED06F.5040905@oracle.com> <4295855A5C1DE049A61835A1887419CC2CEA6A09@DEWDFEMB12A.global.corp.sap> <531F9370.4020305@oracle.com> <5320E5F8.3020500@oracle.com> <5326766F.80104@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CEA843F@DEWDFEMB12A.global.corp.sap> Hi, having said what I wanted to say about the build output directory path, I'm fine with both solutions. In the end it affects only a few line in shared make files. Best regards, Goetz -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of David Holmes Sent: Montag, 17. M?rz 2014 05:14 To: Alexander Smundak; HotSpot Open Source Developers; build-dev Subject: Re: RFR (M): 8036767 PPC64: Support for little endian execution model On 15/03/2014 7:11 AM, Alexander Smundak wrote: > Ping. My position hasn't changed. I don't think this needs to be, or should be, a distinct architecture. I've added build-dev to cc list to see what our build experts think. David ----- > On Wed, Mar 12, 2014 at 4:59 PM, Alexander Smundak wrote: >> I was concerned by the term 'variant', which might suggest that the applications >> built for PPC64 and PPC64LE are binary compatible. They are not. >> >> On Wed, Mar 12, 2014 at 3:55 PM, David Holmes wrote: >>> On 12/03/2014 9:19 AM, Alexander Smundak wrote: >>>> >>>> On Tue, Mar 11, 2014 at 3:51 PM, Vladimir Kozlov >>>> wrote: >>>>> >>>>> It would only help if you could do cross compilation to have both build >>>>> variants at the same place. Currently you can only build le variant on >>>>> ppc64le machine and vice versa. That is why, I think, David asked if we >>>>> can >>>>> control what variant to build. >>>> >>>> Just to clarify the situation a bit: ppc64le is not a variant of ppc64. >>>> That is, >>>> an application compiled for the little-endian PowerPC64 does not "just >>>> run" on >>>> the big-endian PowerPC64 (albeit OS can have such feature, similar to the >>>> ability of the Linux running on x86_64 CPU to run 32-bit x86 >>>> applications). >>>> So ppc64le is a different architecture from ppc64. >>> >>> >>> I disagree with that classification for "architecture" and it seems at odds >>> with the literature which describes the endian-ness selection as a "mode". >>> >>> David >>> ----- >>> >>> >>>>> I would like to see the changes based on Volker suggestion. We can >>>>> compare >>>>> them and decide which way to go. >>>> >>>> Volker has the detailed suggestion here: >>>> >>>> http://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/2014-March/001790.html >>>> and it involves additional Make variable and if statements in the >>>> platform makefile >>>> where they are not supposed to be present. >>>> >>> From bengt.rutisson at oracle.com Mon Mar 17 08:15:40 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Mon, 17 Mar 2014 09:15:40 +0100 Subject: RFR (XXS): JDK-8037495: Remove platform specific specification of SurvivorRatio for BSD Message-ID: <5326AF2C.9030809@oracle.com> Hi everyone, Can I have a couple of reviews for this very small cleanup? http://cr.openjdk.java.net/~brutisso/8037495/webrev.00/ From the bug report: The SurvivorRatio flag used to be a platform dependent flag. It was changed to a platform independent flag with this change from October 2009: http://hg.openjdk.java.net/jdk9/hs/hotspot/rev/473cce303f13 /src/share/vm/runtime/globals.hpp - product_pd(intx, SurvivorRatio, \ + product(intx, SurvivorRatio, 8, \ However, when the BSD port was integrated in 2011 the platform specific definition for BSD came in along with the port: http://hg.openjdk.java.net/jdk9/hs/hotspot/rev/f08d439fab8c /src/os_cpu/bsd_x86/vm/globals_bsd_x86.hpp +define_pd_global(intx, SurvivorRatio, 8); Both definitions are using the same default value, so there should be no need to have this platform dependent. Thanks, Bengt From mikael.gerdin at oracle.com Mon Mar 17 08:34:15 2014 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Mon, 17 Mar 2014 09:34:15 +0100 Subject: RFR: 8036703: Add trace event with statistics for the metaspace chunk free lists In-Reply-To: <5321ABF1.9030403@oracle.com> References: <53184144.3060905@oracle.com> <53217D09.2010009@oracle.com> <5321ABF1.9030403@oracle.com> Message-ID: <2258461.Rnco8YdIIz@mgerdin03> Erik, On Thursday 13 March 2014 14.00.33 Erik Helin wrote: > Hi Mikael and Stefan, > > thanks for having a look and special thanks to Mikael for finding the bug! > > Please see new webrev at: > http://cr.openjdk.java.net/~ehelin/8036703/webrev.04/ > > Please see the incremental webrev at: > http://cr.openjdk.java.net/~ehelin/8036703/webrev.03-04/ Looks good. /Mikael > > Based on some internal feedback, I also corrected a few nits (see the > incremental webrev for details). > > Thanks, > Erik > > On 2014-03-13 10:40, Stefan Karlsson wrote: > > On 2014-03-13 09:57, Mikael Gerdin wrote: > >> Hi Erik, > >> > >> On Wednesday 12 March 2014 17.04.26 Erik Helin wrote: > >>> Based on feedback from Mikael and StefanK, I've reworked the patch to > >>> put the ChunkFreeListSummaries in the class MetaspaceSummary. > >>> > >>> Please see the new version at: > >>> http://cr.openjdk.java.net/~ehelin/8036703/webrev.03/ > >> > >> I think this version is much better! > >> Just a small issue: > >> > >> collectedHeap.cpp > >> > >> 99 > >> > >> 100 const MetaspaceChunkFreeListSummary& ms_chunk_free_list_summary = > >> 101 MetaspaceAux::chunk_free_list_summary(Metaspace::ClassType); > >> 102 const MetaspaceChunkFreeListSummary& > >> > >> class_chunk_free_list_summary = > >> > >> 103 MetaspaceAux::chunk_free_list_summary(Metaspace::NonClassType); > >> 104 > >> > >> It looks like you pass the wrong MetadataType to > >> chunk_free_list_summary here, > >> ClassType and NonClassType should be swapped. > > > > Except for this bug, this looks good. > > > > thanks, > > StefanK > > > >> /Mikael > >> > >>> Thanks, > >>> Erik > >>> > >>> On 2014-03-06 11:56, Erik Helin wrote: > >>>> A small bug had found its way into this patch: > >>>> > >>>> metaspace.cpp: > >>>> +bool MetaspaceAux::has_chunk_free_list(Metaspace::MetadataType > >>>> mdtype) { > >>>> + return Metaspace::get_chunk_manager(mdtype) == NULL; > >>>> +} > >>>> > >>>> should of course be: > >>>> > >>>> +bool MetaspaceAux::has_chunk_free_list(Metaspace::MetadataType > >>>> mdtype) { > >>>> + return Metaspace::get_chunk_manager(mdtype) != NULL; > >>>> +} > >>>> > >>>> Please see new webrev at: > >>>> http://cr.openjdk.java.net/~ehelin/8036703/webrev.01/ > >>>> > >>>> Thanks, > >>>> Erik > >>>> > >>>> On 2014-03-06 10:35, Erik Helin wrote: > >>>>> Hi all, > >>>>> > >>>>> this patch adds a new trace event - > >>>>> vm/gc/metaspace/chunk_free_list_summary for getting statistics for > >>>>> metaspace's chunk free lists. The event contains the numbers of chunks > >>>>> in the free list and the size of the chunks in the free lists. This > >>>>> information is provided for all different kinds of chunks - special, > >>>>> small, medium and humongous. > >>>>> > >>>>> This information will be useful when diagnosing fragmentation > >>>>> issues in > >>>>> metaspace. > >>>>> > >>>>> Webrev: > >>>>> http://cr.openjdk.java.net/~ehelin/8036703/webrev.00/ > >>>>> > >>>>> Enhancement: > >>>>> https://bugs.openjdk.java.net/browse/JDK-8036703 > >>>>> > >>>>> Testing: > >>>>> - JFR JTREG tests > >>>>> - JPRT > >>>>> - Manually verifying that the numbers look correct by looking at > >>>>> > >>>>> recordings from small test cases. > >>>>> > >>>>> Thanks, > >>>>> Erik From mikael.gerdin at oracle.com Mon Mar 17 09:00:19 2014 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Mon, 17 Mar 2014 10:00:19 +0100 Subject: RFR (XXS): JDK-8037495: Remove platform specific specification of SurvivorRatio for BSD In-Reply-To: <5326AF2C.9030809@oracle.com> References: <5326AF2C.9030809@oracle.com> Message-ID: <24497317.dHKKLduU6G@mgerdin03> Bengt, On Monday 17 March 2014 09.15.40 Bengt Rutisson wrote: > Hi everyone, > > Can I have a couple of reviews for this very small cleanup? > > http://cr.openjdk.java.net/~brutisso/8037495/webrev.00/ Looks good, ship it! /Mikael > > > From the bug report: > > The SurvivorRatio flag used to be a platform dependent flag. It was > changed to a platform independent flag with this change from October 2009: > > http://hg.openjdk.java.net/jdk9/hs/hotspot/rev/473cce303f13 > > /src/share/vm/runtime/globals.hpp > > - product_pd(intx, SurvivorRatio, \ > + product(intx, SurvivorRatio, 8, \ > > However, when the BSD port was integrated in 2011 the platform specific > definition for BSD came in along with the port: > > http://hg.openjdk.java.net/jdk9/hs/hotspot/rev/f08d439fab8c > > /src/os_cpu/bsd_x86/vm/globals_bsd_x86.hpp > > +define_pd_global(intx, SurvivorRatio, 8); > > Both definitions are using the same default value, so there should be no > need to have this platform dependent. > > Thanks, > Bengt From bengt.rutisson at oracle.com Mon Mar 17 08:58:53 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Mon, 17 Mar 2014 09:58:53 +0100 Subject: RFR (XXS): JDK-8037495: Remove platform specific specification of SurvivorRatio for BSD In-Reply-To: <24497317.dHKKLduU6G@mgerdin03> References: <5326AF2C.9030809@oracle.com> <24497317.dHKKLduU6G@mgerdin03> Message-ID: <5326B94D.4080509@oracle.com> Thanks, Mikael! Bengt On 2014-03-17 10:00, Mikael Gerdin wrote: > Bengt, > > On Monday 17 March 2014 09.15.40 Bengt Rutisson wrote: >> Hi everyone, >> >> Can I have a couple of reviews for this very small cleanup? >> >> http://cr.openjdk.java.net/~brutisso/8037495/webrev.00/ > Looks good, ship it! > /Mikael > >> >> From the bug report: >> >> The SurvivorRatio flag used to be a platform dependent flag. It was >> changed to a platform independent flag with this change from October 2009: >> >> http://hg.openjdk.java.net/jdk9/hs/hotspot/rev/473cce303f13 >> >> /src/share/vm/runtime/globals.hpp >> >> - product_pd(intx, SurvivorRatio, \ >> + product(intx, SurvivorRatio, 8, \ >> >> However, when the BSD port was integrated in 2011 the platform specific >> definition for BSD came in along with the port: >> >> http://hg.openjdk.java.net/jdk9/hs/hotspot/rev/f08d439fab8c >> >> /src/os_cpu/bsd_x86/vm/globals_bsd_x86.hpp >> >> +define_pd_global(intx, SurvivorRatio, 8); >> >> Both definitions are using the same default value, so there should be no >> need to have this platform dependent. >> >> Thanks, >> Bengt From stefan.karlsson at oracle.com Mon Mar 17 09:03:07 2014 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 17 Mar 2014 10:03:07 +0100 Subject: RFR: 8036699: Add trace event when a metaspace allocation fails In-Reply-To: <5321D6D1.8060503@oracle.com> References: <53183925.4020001@oracle.com> <5321D6D1.8060503@oracle.com> Message-ID: <5326BA4B.7000506@oracle.com> On 2014-03-13 17:03, Erik Helin wrote: > Hi all, > > due to lack of reviewers for the native stack trace code, I've decided > to split this patch into two patches: > - adding the event > - adding the native stack trace information the event > (will be sent out later) > > This review request is now only for adding the event. Please see a new > webrev at: > http://cr.openjdk.java.net/~ehelin/8036699/webrev.01/ > > For an incremental webrev (that only removes the native stack walking > code) please see: > http://cr.openjdk.java.net/~ehelin/8036699/webrev.00-01/ This diff looks good. StefanK > > Thanks, > Erik > > On 2014-03-06 10:00, Erik Helin wrote: >> Hi all, >> >> this patch adds the new trace event vm/gc/metaspace/allocation_failure. >> The event will be sent when we get an allocation failure in metaspace. >> The event will contain the following information: >> - classLoader - the class loader doing the allocation >> - anonymousClassLoader - if the classLoader is anonymous >> - nativeStackTrace - a VM stack trace (see more below) >> - metadataType - the kind of metadata (class or non-class) >> - metaspaceObjectType - the kind of metaspace object >> >> The field nativeStackTrace will contain a string representation of the >> Threads C frames (up until the first Java frame). This is very useful >> for debugging metaspace allocation failures coming from VM internals >> such as the interpreter or GC code. >> >> I would especially appreciate if someone could look over the code in >> metaspaceTracer.cpp performing the stack walking. >> >> Webrev: >> http://cr.openjdk.java.net/~ehelin/8036699/webrev.00/ >> >> Enhancement: >> https://bugs.openjdk.java.net/browse/JDK-8036699 >> >> Testing: >> - JFR JTREG tests >> - JPRT >> - Manual stress testing of the stack walking code by writing small test >> programs that causes metaspace allocation failures from both Java >> threads and non-Java threads. >> >> Thanks, >> Erik From stefan.karlsson at oracle.com Mon Mar 17 09:06:00 2014 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 17 Mar 2014 10:06:00 +0100 Subject: RFR: 8036701: Add trace event when a metaspace throws out of memory error In-Reply-To: <5321E513.60005@oracle.com> References: <53183DBC.1050707@oracle.com> <5321E513.60005@oracle.com> Message-ID: <5326BAF8.4020100@oracle.com> On 2014-03-13 18:04, Erik Helin wrote: > Hi all, > > since I decided to do the native stack allocation as a separate > feature for the metaspace allocation failure event, this patch > required a very minor update. I just removed the field > "nativeStackTrace" from the event. > > New webrev: > http://cr.openjdk.java.net/~ehelin/8036701/webrev.01/ > > Incremental webrev: > http://cr.openjdk.java.net/~ehelin/8036701/webrev.00-01/ The diff looks good. StefanK > > Thanks, > Erik > > On 2014-03-06 10:19, Erik Helin wrote: >> Hi all, >> >> this patch adds a new trace event for tracking metaspace out of memory >> exceptions - vm/gc/metaspace/out_of_memory. This patch builds upon the >> patch for "JDK-8036699: Add trace event when a metaspace allocation >> fails". The information in the allocation failure event and the OOM >> event are the same, see the review request for the allocation failure >> event for more information. >> >> Webrev: >> http://cr.openjdk.java.net/~ehelin/8036701/webrev.00/ >> >> Enhancement: >> https://bugs.openjdk.java.net/browse/JDK-8036701 >> >> Testing: >> - JFR JTREG tests >> - JPRT >> >> Thanks, >> Erik From stefan.karlsson at oracle.com Mon Mar 17 09:07:39 2014 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 17 Mar 2014 10:07:39 +0100 Subject: RFR (XXS): JDK-8037495: Remove platform specific specification of SurvivorRatio for BSD In-Reply-To: <5326AF2C.9030809@oracle.com> References: <5326AF2C.9030809@oracle.com> Message-ID: <5326BB5B.1050301@oracle.com> On 2014-03-17 09:15, Bengt Rutisson wrote: > > Hi everyone, > > Can I have a couple of reviews for this very small cleanup? > > http://cr.openjdk.java.net/~brutisso/8037495/webrev.00/ Looks good. StefanK > > > From the bug report: > > The SurvivorRatio flag used to be a platform dependent flag. It was > changed to a platform independent flag with this change from October > 2009: > > http://hg.openjdk.java.net/jdk9/hs/hotspot/rev/473cce303f13 > > /src/share/vm/runtime/globals.hpp > > - product_pd(intx, SurvivorRatio, \ > + product(intx, SurvivorRatio, 8, \ > > However, when the BSD port was integrated in 2011 the platform > specific definition for BSD came in along with the port: > > http://hg.openjdk.java.net/jdk9/hs/hotspot/rev/f08d439fab8c > > /src/os_cpu/bsd_x86/vm/globals_bsd_x86.hpp > > +define_pd_global(intx, SurvivorRatio, 8); > > Both definitions are using the same default value, so there should be > no need to have this platform dependent. > > Thanks, > Bengt From bengt.rutisson at oracle.com Mon Mar 17 09:51:32 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Mon, 17 Mar 2014 10:51:32 +0100 Subject: RFR(L): 8029075 - String deduplication in G1 In-Reply-To: <53235373.8080901@oracle.com> References: <53148494.60806@oracle.com> <5316F5B1.60006@oracle.com> <5319D424.90306@oracle.com> <1394541591.2696.20.camel@cirrus> <5320626E.7020102@oracle.com> <5320DB4B.2020801@oracle.com> <5321C8A3.70206@oracle.com> <5322F3D3.6060303@oracle.com> <532301CD.7020709@oracle.com> <53235373.8080901@oracle.com> Message-ID: <5326C5A4.1020809@oracle.com> Hi Per, On 2014-03-14 20:07, Per Liden wrote: > Hi, > > Here's an updated webrev, based on feedback from primarily Thomas and > Coleen (Thanks!). > > http://cr.openjdk.java.net/~pliden/8029075/webrev.2/ Looks good to me. Bengt > > I hope to have addressed all issues that were raised. The main theme > of course is the addition of documentation/comments and the splitting > of some classes into separate files. I think I've addressed most, if > not all, of the other minor issues Thomas mentioned. The only thing I > can think of that wasn't changed was the categorization of the VM > flags, which I motivated in a different mail. > > I didn't think it made sense to upload a diff against the previous > webrev as that diff turned out to be a lot bigger than real webrev > itself. Sorry if this makes reviewing harder. > > cheers, > /Per > > On 2014-03-14 14:19, Per Liden wrote: >> Hi Coleen, >> >> Inline below... >> >> On 2014-03-14 13:19, Coleen Phillimore wrote: >>> >>> Per, 3 quick comments below. Thank you for the reorganization and >>> additional comments. >>> >>> On 3/13/14 11:02 AM, Per Liden wrote: >>>> Hi Coleen, >>>> >>>> Thanks for looking at the patch! Comment below. >>>> >>>> On 2014-03-12 23:10, Coleen Phillimore wrote: >>>>> >>>>> Hi Per, >>>>> >>>>> Why not add the call for StringDedup::unlink() inside the function >>>>> unlink_string_and_symbol_table rather than just above in two >>>>> places, since they go together. >>>>> >>>>> + // Delete dead entries in string deduplication queue/table >>>>> + StringDedup::unlink(&GenMarkSweep::is_alive); >>>>> + >>>>> // Delete entries for dead interned string and clean up >>>>> unreferenced symbols in symbol table. >>>>> G1CollectedHeap::heap()->unlink_string_and_symbol_table(&GenMarkSweep::is_alive); >>>>> >>>> >>>> Yep, I'll move that call into unlink_string_and_symbol_table(). The >>>> unlink_string_and_symbol_table() is a fairly new addition, there >>>> used to be StringTable/SymbolTable calls there when I added the >>>> StrDedup call, that's why it looks like that. >>>> >>>>> >>>>> >>>>> I didn't see anything else specific to change. Some general >>>>> comments which are similar to Thomas's. There seems to be a lack >>>>> of comments in the deduplication file. I hate huge block >>>>> comments too but some comments about what it does in general would >>>>> be nice. Like 3 sentences per class. The file is overwhelming >>>>> (ie, I completely lose interest) with the statistics printing at >>>>> the front. Maybe you could put the most interesting classes at >>>>> the front of the file? Maybe statistics printing could be a >>>>> separate file. >>>>> >>>>> One of the conventions that we have is that accessors for classes >>>>> are on one line, so you could make the file smaller by doing that >>>>> reformatting. >>>>> >>>>> It is also a general convention to have comments above the >>>>> declaration of a class to say generally what it's for, when it's >>>>> used, etc. It's not easy to tell by the name or we could guess >>>>> wrong. What is StringDedupQueue for, eg? That would make >>>>> reviewing the code easier. >>>> >>>> Agree, I'll be adding comments to each class and a general >>>> description of what dedup does. >>>> >>>>> >>>>> There seems to be a lot of duplication of code with the hashtable >>>>> in hashtable.hpp, only different. We have lots of hashtables in >>>>> hotspot. Did you try to reinstantiate this hashtable and why >>>>> didn't that not work? There are interesting memory allocation >>>>> functions for the entries in this normal hashtable that might help >>>>> with fragmentation, for instance. >>>> >>>> Very early on I actually did use an instance of Hashtable and even >>>> played with the idea of having a combined StringTable/DedupTable. I >>>> decided not to use that though, mostly because I felt I would need >>>> to make too many modifications to non-dedup related code and >>>> because Hashtable<> carries concepts, like shared entries, which >>>> didn't quite fit with dedup. Using the existing StringTable would >>>> mean it would have to have logic to manage entries which point to >>>> either Strings or char[], which again seemed like an too intrusive >>>> change. The Hashtable allocation strategy (with allocating blocks >>>> of entries) seems nice from a performance perspective, but freeing >>>> those becomes a bit of a challenge, which I guess is why it never >>>> frees any entries. Since the dedup table is there to in the end >>>> save memory, I felt I needed better control over the table's memory >>>> usage. At one point I actually started writing a >>>> WeakOopsHashTable<>, which I hoped could serve as the base for both >>>> StringTable and DedupTable, but again I felt I was losing focus a >>>> bit as it would mean modifying large parts of non-GC and non-dedeup >>>> related stuff. The potential connection to CDS here also made me a >>>> bit scared of going in that direction, as I don't have a complete >>>> understanding of the implications of that. >>> >>> Ok, I'd figured they didn't map that well. There's a lot of >>> duplication already in symbol and string tables also. >>>> >>>>> >>>>> Lastly, why is the hash code in the string written to and used? >>>>> The StringTable uses the same hash algorithm until it has to >>>>> change the hash code, but the new hash code isn't stored in the >>>>> string because it would be incompatible. >>>> >>>> I think StringTable and the dedup table is doing the same thing >>>> here. The hash code in the String object is only used/updated in >>>> the case we're using the standard/compatible hash function. As soon >>>> as the table is rehashed and switches to another hash function it >>>> no longer updates the hash code in the String. use_java_hash() >>>> tells use if we're using the standard/compatible hash and it's used >>>> here: >>>> >>>> if (use_java_hash()) { >>>> // Get hash code from cache >>>> hash = java_lang_String::hash(java_string); >>>> } >>>> >>>> and later here: >>>> >>>> if (use_java_hash() && hash != 0) { >>>> // Store hash code in cache >>>> java_lang_String::set_hash(java_string, hash); >>>> } >>>> >>>> Concerns? >>> >>> I have to look again but wanted to answer quickly (forgot to last >>> night). This is good, but why do we have to update the hash code in >>> the java/lang/String instance at all? >> >> We don't strictly have to, but it mimics the behavior of >> String.hashCode() and it is seems like the natural thing to do to >> given that the hash field in String is a cache for this purpose so >> that future users of String.hashCode() don't have to do the same >> calculation. Do you have a reason in mind why we wouldn't want to >> write it back? >> >> cheers, >> /Per >> >>> >>>> >>>>> >>>>> I would like to see this be a few different files. It seems like >>>>> this file does three things conceptually, interacts with G1 >>>>> somehow to determine which strings are candidates for >>>>> deduplication, stores these things in a rehashable, resizable hash >>>>> table and gathers statistics. >>>>> >>>>> I'm sorry to have so many comments so late and I haven't really >>>>> tried to read the code for correctness. I think some >>>>> reformatting and reorganization would make the reviewers job >>>>> easier. It's a big change and worthwhile. >>>> >>>> Great, I've no problems splitting this up. I'm more of a "one class >>>> per file" person myself, but I had the (apparently incorrect) >>>> impression that wasn't the hotspot way. I'm thinking I'll split >>>> this up into the following files (while I'm at it I'll also add the >>>> G1 prefix as Thomas suggested): >>>> >>>> g1StringDedup - Main interface for the GC, would contain entry >>>> point and things like the task and closure classes >>>> g1StringDedupTable - The table, cache and entry classes >>>> g1StringDedupQueue - The queue >>>> g1StringDedupStat - The statistics sutff >>>> g1StringDedupThread - The dedup thread >>> >>> Yes, this would be nice. symbolTable.cpp should be split up too if >>> that is what gave you the impression. I will file an RFE to do >>> that, it's been at the back of my mind to do so. >>> >>> Thanks, >>> Coleen >>>> >>>> Sounds good? >>>> >>>> Will send out and updated webrev. >>>> >>>> Thanks, >>>> /Per >>>> >>>>> >>>>> Thanks, >>>>> Coleen >>>>> >>>>> >>>>> On 3/12/14 9:34 AM, Per Liden wrote: >>>>>> Hi Thomas, >>>>>> >>>>>> Thanks for reviewing. Comments inline. >>>>>> >>>>>> On 03/11/2014 01:39 PM, Thomas Schatzl wrote: >>>>>>> Hi Per, >>>>>>> >>>>>>> >>>>>>> On Fri, 2014-03-07 at 15:13 +0100, Per Liden wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> Here's an updated webrev based on the feedback from Bengt and >>>>>>>> Thomas. >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~pliden/8029075/webrev.1/ >>>>>>>> >>>>>>> >>>>>>> See further below. >>>>>>> >>>>>>>> >>>>>>>> And as suggested, two changes in the initial webrev were broken >>>>>>>> out into >>>>>>>> separate CRs, available here: >>>>>>>> >>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8036672 >>>>>>>> Webrev: http://cr.openjdk.java.net/~pliden/8036672/webrev.0/ >>>>>>> >>>>>>> Looks good. >>>>>>> >>>>>>> Did you check the impact on something like specjbb2005 which does >>>>>>> nothing but object copying? That code is somewhat performance >>>>>>> sensitive. >>>>>> >>>>>> Yes, I did Aurora runs with jbb2005 and jbb2013, didn't see any >>>>>> difference. >>>>>> >>>>>>> >>>>>>> >>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8036673 >>>>>>>> Webrev: http://cr.openjdk.java.net/~pliden/8036673/webrev.0/ >>>>>>> >>>>>>> Looks good. >>>>>>> >>>>>>> Here are my comments: there are a few minor issues, and a single >>>>>>> major: >>>>>>> basically the documentation for this feature is lacking imo. I >>>>>>> listed a >>>>>>> few particular issues of mine with it. >>>>>>> >>>>>>> First the minor issues: >>>>>>> >>>>>>> - could you check on copyright dates on the modified files? They >>>>>>> have >>>>>>> not been updated anywhere afaics. >>>>>>> >>>>>>> - not sure if good, but maybe make >>>>>>> PrintStringDeduplicationStatistics >>>>>>> diagnostic? It seems to really be some diagnosis functions. >>>>>>> >>>>>>> - maybe make the new *ALot flags notproduct. These seem to be >>>>>>> debugging >>>>>>> helpers similar to other *ALot flags. >>>>>> >>>>>> I had a conversion with Bengt about this some time ago and I >>>>>> think the options are categorized correctly. The ALot flags needs >>>>>> be available in product builds because they are used by tests. >>>>>> Most Print*Statistics are product therefore it makes sense to >>>>>> follow that pattern for PrintStringDeduplicationStatistics. >>>>>> >>>>>>> >>>>>>> - StringDeduplicationAgeThreshold: maybe also needs to mention >>>>>>> that if >>>>>>> the object is promoted, that it is always considered as candidate. >>>>>> >>>>>> Check. >>>>>> >>>>>>> >>>>>>> marksweep.inline.hpp >>>>>>> >>>>>>> - MarkSweep::mark_object(): did you measure impact on other >>>>>>> collectors >>>>>>> using this code? It is enabled unconditionally. Also I would prefer >>>>>>> that !UseG1GC automatically disables UseStringDeduplication (or >>>>>>> use a >>>>>>> global variable for that) to avoid the two checks. (The latter >>>>>>> option is >>>>>>> probably better). >>>>>> >>>>>> Yes, ran jbb2005/2013. >>>>>> >>>>>>> >>>>>>> symbolTable.cpp: >>>>>>> >>>>>>> - again both UseG1GC and UseStringDedupliation are paired. Could >>>>>>> this be >>>>>>> put in a method/global variable somewhere that is set up >>>>>>> correctly at >>>>>>> startup? >>>>>> >>>>>> I think using the options is much more readable. Adding another >>>>>> state which tells us if G1 & Dedup is enabled means we need a >>>>>> pre-init stage to set up that state, which I don't think is very >>>>>> nice. >>>>>> >>>>>>> >>>>>>> g1CollectedHeap.cpp: >>>>>>> >>>>>>> - StringDedup::initialize() is run unconditionally (the check for >>>>>>> UseStringDeduplication is inside the method) while it's outside for >>>>>>> everything else. Just a note, keep it if wanted. >>>>>> >>>>>> They are hidden inside dedup everywhere, except where the >>>>>> performance overhead of a call might be an issue (like mark and >>>>>> copy). >>>>>> >>>>>>> >>>>>>> TestStringDeduplicationTools.java: >>>>>>> >>>>>>> - maybe put retrieval of the unsafe object into the test library. >>>>>>> Browsing through the code shows quite a lot copy+paste here >>>>>>> already. >>>>>>> >>>>>>> - "qeueue" -> "queue" >>>>>> >>>>>> Check. >>>>>> >>>>>>> >>>>>>> all tests: >>>>>>> >>>>>>> - missing bug ids >>>>>> >>>>>> Check. >>>>>> >>>>>>> >>>>>>> From here it is mostly about comments/documentation: >>>>>> >>>>>> I agree that adding a feature overview and some additional >>>>>> comments is probably a good idea. However, I also know that Bengt >>>>>> and StefanK aren't too crazy about large comment blobs so I tried >>>>>> to kept the noise down. I also disagree with some of the other >>>>>> comments below which tend to be about personal taste rather than >>>>>> code quality or breaking some coding convention we have, e.g. the >>>>>> static helpers, readability, naming, etc. >>>>>> >>>>>> I'll add some comments and make some other changes and send out >>>>>> an updated webrev shortly. >>>>>> >>>>>> cheers, >>>>>> /Per >>>>>> >>>>>>> >>>>>>> stringdedup.?pp: >>>>>>> >>>>>>> - is it possible to put an overview about how the string >>>>>>> deduplication >>>>>>> works here? There is absolutely no place in the code where e.g. the >>>>>>> string deduplication cycle is explained. There is no comprehensive >>>>>>> information about this feature anywhere, just bits about corner >>>>>>> cases or >>>>>>> particular here and there, but nothing that can be used as entry >>>>>>> point >>>>>>> to understand what this is actually doing. >>>>>>> >>>>>>> To some extent, the comments seem to be repetitions of what the >>>>>>> code >>>>>>> already shows. E.g. "some_constant = (1 << 10); // 1024 (must be >>>>>>> power >>>>>>> of 2)" or "// Store hash code in cache" followed by >>>>>>> "java_lang_String::set_hash(java_string, hash);". >>>>>>> >>>>>>> There is also no reference to the JEP (I am not sure if it is >>>>>>> customary >>>>>>> to put references to outside documents - I would prefer some); >>>>>>> however >>>>>>> the JEP does not replace documentation in the source as it is not >>>>>>> detailed enough, and the JEP cannot be changed any more. The JEP is >>>>>>> interesting for the alternatives considered though. >>>>>>> >>>>>>> Ideally, to reasonably understand the feature I would like to >>>>>>> not need >>>>>>> to spend hours looking through the implementation. >>>>>>> >>>>>>> - how many threads does this feature add, and are supported by >>>>>>> this code >>>>>>> (for which parts)? Synchronization between them? >>>>>>> >>>>>>> - the comments "For XY" in the StringDedup header file do not >>>>>>> give any >>>>>>> hint about what they are doing (not sure if it is interesting as >>>>>>> it is >>>>>>> easy to find out the only callers). It seems that this >>>>>>> information has >>>>>>> been put at the place where these methods are invoked (or just >>>>>>> somewhere >>>>>>> in the cpp file). Imo the methods and members (except trivial) >>>>>>> should be >>>>>>> documented in the hpp file in the class body, i.e. in the >>>>>>> interface. >>>>>>> >>>>>>> - there are a few methods (also in the StringDedupTable class) in >>>>>>> StringDedup where the results of the methods do not seem obvious >>>>>>> and >>>>>>> should be commented. >>>>>>> >>>>>>> - I really dislike the location of the "Candidate selection" >>>>>>> explanation >>>>>>> within the cpp file. I would not have found it if I were not >>>>>>> looking for >>>>>>> some documentation ;) It would probably fit very well in the above >>>>>>> mentioned overview paragraphs. >>>>>>> >>>>>>> - please add the static helper functions that implement the actual >>>>>>> policies (is_candidate_from_mark, is_candidate_from_evacuation) >>>>>>> to the >>>>>>> StringDedup class (as private static methods) so that they can >>>>>>> be found >>>>>>> easily. Static non-class helper functions in cpp files should >>>>>>> only be >>>>>>> used for stuff that is really unimportant and not relevant to >>>>>>> the code >>>>>>> imo. However these seem to be _the_ central policy methods for >>>>>>> string >>>>>>> dedup. >>>>>>> So it would be nice to have them mentioned in the class/hpp file >>>>>>> and >>>>>>> documented there. >>>>>>> >>>>>>> - facts about the code are repeated, e.g. that the table size >>>>>>> must be a >>>>>>> power of two. This is nice, but has a few drawbacks: the >>>>>>> compiler does >>>>>>> not care about comments at all, so it might be useful to put >>>>>>> asserts in >>>>>>> these places instead, and second, it's just duplication of text >>>>>>> with the >>>>>>> usual problem of getting out of date. >>>>>>> >>>>>>> - I am not completely sure why the class has not been called >>>>>>> G1StringDedup. It won't work without G1 anyway and calls G1 >>>>>>> internals >>>>>>> all the time. There does not seem to be a way to make it more >>>>>>> generic >>>>>>> without considerable refactoring. It is fine to keep the name >>>>>>> though, >>>>>>> just asking. I also know that this feature does not want to be a >>>>>>> generic >>>>>>> solution. >>>>>>> >>>>>>> - is it possible to separate out the statistics variables of >>>>>>> StringDedupTable into either a separate class instead of using >>>>>>> globals >>>>>>> or separate them a little from the other unrelated members? (And >>>>>>> describe them) This would improve readability. >>>>>>> >>>>>>> I *think* this is everything from >>>>>>> StringDedupTable::_entries_added to >>>>>>> _rehash_count? I have no idea without searching for every use and >>>>>>> deducing from that. >>>>>>> >>>>>>> - when implementing the hash table for the StringDedupTable, did >>>>>>> you >>>>>>> consider trying to reuse one of the existing hashmap >>>>>>> implementations? >>>>>>> And potentially extend them. >>>>>>> >>>>>>> Imo we already have way too many ad-hoc implementations of often >>>>>>> used >>>>>>> data structures (particularly) in G1 and GC code. >>>>>>> >>>>>>> - please document the properties of the StringDedupTable (type, >>>>>>> hashing, >>>>>>> automatic resizing behavior, thread safety, ...). >>>>>>> >>>>>>> - please explain what StringDedupTable/Cache are and what they >>>>>>> are used >>>>>>> for in context of the feature. >>>>>>> >>>>>>> - stringdedup.cpp, line 1309: candicate -> candidate >>>>>>> >>>>>>> Thanks, >>>>>>> Thomas >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From per.liden at oracle.com Mon Mar 17 11:32:09 2014 From: per.liden at oracle.com (Per Liden) Date: Mon, 17 Mar 2014 12:32:09 +0100 Subject: RFR: 8028498: runtime/SharedArchiveFile/CdsDifferentObjectAlignment.java asserts in RT_Baseline In-Reply-To: <5318BFE4.8010807@oracle.com> References: <52E90809.3000900@oracle.com> <52F91199.6030308@oracle.com> <52FA3DD9.20004@oracle.com> <52FBAB90.1020608@oracle.com> <5301DD2D.3040204@oracle.com> <5318BFE4.8010807@oracle.com> Message-ID: <5326DD39.7000101@oracle.com> Hi Stefan, On 03/06/2014 07:35 PM, Stefan Johansson wrote: > As Per noticed the _gc_locked_out_count isn't updated anymore when > removing the GC locker lock/unlock methods. I've looked through the code > to see what would be the proper action to take in > clear_unhandled_oops(). The first observation was that the check for > is_gc_locked_out() only had any effect during a short period of time > during startup (the only place we used GC locker lock/unlock). This > leads to the question, do we need to avoid clearing the unhandled oops > when the GC lockers other lock mechanism is used, and I've look this > through together with Per and it seems like it's not needed. > > One can argue that if we've locked out the GC using the GC locker, it > should be fine to have unhandled oops, but since we don't make use of > that today I see no reason for enabling it. If I haven't missed anything > this should allow us to remove the check from clear_unhandled_oops() and > the _gc_locked_out_count with accessors all together. > > Please let me know if you have any fears or thoughts around this. I've > run both JPRT tests and some bigapps testing with +CheckUnhandledOops > without any problem as sanity testing. > > Regarding the message I updated it to use the methods suggested by Per. > > Here's the updated webrev: > http://cr.openjdk.java.net/~sjohanss/8028498/webrev.03/ Looks good to me. /Per > > Thanks, > Stefan > > On 2014-02-17 10:58, Per Liden wrote: >> Stefan, >> >> I think this is a nice fix for the problem and it also cleans up the >> gclocker a bit. But I have a few minor comments: >> >> vmGCOperations.cpp: >> 110 err_msg("GC triggered before VM initialization completed. >> Try increasing " >> 111 "NewSize, current value " UINTX_FORMAT "K.", >> NewSize / K)); >> >> I'd suggest you use >> byte_size_in_proper_unit()/proper_unit_for_byte_size() instead of a >> "K", or maybe just not print the size at all. >> >> >> With the removal of GC_locker::lock()/unlock() there's no longer >> anyone how modifies Thread::current()->_gc_locked_out_count. So we >> should be able to remove it. However, >> UnhandledOops::clear_unhandled_oops() still uses is_gc_locked_out() so >> that needs to be changed to check for something else, >> is_init_completed maybe? >> >> cheers, >> /Per >> >> On 02/12/2014 06:12 PM, Stefan Johansson wrote: >>> Hi, >>> >>> Stefan K pointed out to me that in the Metaspace code we use >>> is_init_completed() to verify that the VM is fully initialized. I >>> started looking at where we do set_init_completed() and realized that it >>> is right where we are ready to handle a GC. It makes sense to try to >>> minimize the different ways we check if the VM is fully initialized. >>> Using is_init_completed() instead of the GC locker to prevent GCs during >>> startup has another nice effect too; we can remove one of the two >>> different lock mechanisms in the GC locker. >>> >>> The new proposal removes the use of GC_locker::lock()/unlock() which is >>> only used to prevent GCs during startup. Instead there is a check in the >>> GC operations prologue, to ensure that the VM is initialized when the GC >>> starts. >>> >>> Please review the new the change: >>> http://cr.openjdk.java.net/~sjohanss/8028498/webrev.02/ >>> >>> Testing: >>> * JPRT >>> * GC locker test through UTE >>> >>> Thanks, >>> Stefan >>> >>> On 2014-02-11 16:12, Stefan Johansson wrote: >>>> Hi again =) >>>> >>>> Don't spend time on the current webrev (02). I've gotten some more >>>> comments on the proposed fix and will look at a solution that allow us >>>> to remove the GC locker usage during initialization. >>>> >>>> Cheers, >>>> Stefan >>>> >>>> On 2014-02-10 18:51, Stefan Johansson wrote: >>>>> Hi again, >>>>> >>>>> After a couple of offline discussions with different people I've come >>>>> to the conclusion to split out the sizing of the young generation >>>>> from the fix for this issue. I've created a new bug (JDK-8033426) for >>>>> that, which is currently out for review. >>>>> >>>>> The discussions also lead to a different approach for fixing this >>>>> issue. The new proposal is to use the GC locker to prevent GCs until >>>>> we are ready to handle them and otherwise fail the VM initialization. >>>>> The gc locker has two different mechanisms for locking out the GC, >>>>> one which is used during initialization and one used to stall GCs >>>>> when doing critical JNI work. The part of the initialization that >>>>> currently is locked by the GC locker is to narrow and there is no >>>>> guarantee that a GC can be handled after the GC locker is unlocked. >>>>> This fix will extend the part that is locked and also add a check >>>>> that if a GC is initiated while locked during startup the VM exits. >>>>> >>>>> New webrev: >>>>> http://cr.openjdk.java.net/~sjohanss/8028498/webrev.01/ >>>>> >>>>> Thanks, >>>>> Stefan >>>>> >>>>> On 2014-01-29 14:54, Stefan Johansson wrote: >>>>>> Hi, >>>>>> >>>>>> Please review this fix for: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8028498 >>>>>> >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~sjohanss/8028498/webrev.00/ >>>>>> >>>>>> Summary: >>>>>> The initial young generation size has been fairly small by default >>>>>> (1.5M) and if setting ObjectAlignmentInBytes to something larger >>>>>> than the default it can causes CDS to trigger a GC when dumping the >>>>>> archive. At this point the VM is not ready to handle a GC and we >>>>>> will hit an assertion. Making sure we can handle a GC at this point >>>>>> is not trivial and the proposed solution is to alter the default >>>>>> sizing of the young generation as well as adding a safety check when >>>>>> dumping the archive to exit the VM if the young generation is too >>>>>> small. >>>>>> >>>>>> Testing: >>>>>> * JPRT build and test passes >>>>>> * Failing test now passes on all platforms (tested throught JPRT) >>>>>> * Verified that the GC and Runtime jtreg tests still passes >>>>>> >>>>>> Cheers, >>>>>> Stefan >>>>> >>>> >>> >> > From magnus.ihse.bursie at oracle.com Mon Mar 17 11:36:15 2014 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Mon, 17 Mar 2014 12:36:15 +0100 Subject: RFR (M): 8036767 PPC64: Support for little endian execution model In-Reply-To: <5326766F.80104@oracle.com> References: <531E8309.2060206@oracle.com> <531ED06F.5040905@oracle.com> <4295855A5C1DE049A61835A1887419CC2CEA6A09@DEWDFEMB12A.global.corp.sap> <531F9370.4020305@oracle.com> <5320E5F8.3020500@oracle.com> <5326766F.80104@oracle.com> Message-ID: <5326DE2F.6000107@oracle.com> On 2014-03-17 05:13, David Holmes wrote: > On 15/03/2014 7:11 AM, Alexander Smundak wrote: >> Ping. > > My position hasn't changed. I don't think this needs to be, or should > be, a distinct architecture. > > I've added build-dev to cc list to see what our build experts think. I'm not sure I've completely understood the discussion (it has jumped around a bit between mailing lists), but I think I agree with David here. Architecture is a term that implies general common features, not specific differences. While hotspot and the rest of the jdk build is not currently in sync with this terminology, in the common configure and makefiles we use the terminology "target cpu" (which can be for instance "x86_64" or "sparcv9" and "target cpu arch" (which can be for instance "x86" or "sparc"). Regarding ppc, we currently define the target cpu arch "ppc" and two target cpus, "ppc" and "ppc64". Perhaps it would be more approriate to consider this a separate target cpu ("ppc64le"), but not a separate target cpu architecture? But that would be on the top-level -- I'm not sure how that would map properly to the current hotspot build system. /Magnus From erik.helin at oracle.com Mon Mar 17 12:40:19 2014 From: erik.helin at oracle.com (Erik Helin) Date: Mon, 17 Mar 2014 13:40:19 +0100 Subject: RFR: 8036699: Add trace event when a metaspace allocation fails In-Reply-To: <5321D6D1.8060503@oracle.com> References: <53183925.4020001@oracle.com> <5321D6D1.8060503@oracle.com> Message-ID: <5326ED33.7090107@oracle.com> (this reply went to the wrong RFR, reposting to get it back on to the right thread, sorry for the noise) Based on a discussion with Jon Masamitsu and StefanK, I've decided to change metadata_type_name to return a slightly more user-friendly value. Please see new webrev at: http://cr.openjdk.java.net/~ehelin/8036699/webrev.02/ Incremental webrev: http://cr.openjdk.java.net/~ehelin/8036699/webrev.01-02/ Thanks, Erik On 2014-03-13 17:03, Erik Helin wrote: > Hi all, > > due to lack of reviewers for the native stack trace code, I've decided > to split this patch into two patches: > - adding the event > - adding the native stack trace information the event > (will be sent out later) > > This review request is now only for adding the event. Please see a new > webrev at: > http://cr.openjdk.java.net/~ehelin/8036699/webrev.01/ > > For an incremental webrev (that only removes the native stack walking > code) please see: > http://cr.openjdk.java.net/~ehelin/8036699/webrev.00-01/ > > Thanks, > Erik > > On 2014-03-06 10:00, Erik Helin wrote: >> Hi all, >> >> this patch adds the new trace event vm/gc/metaspace/allocation_failure. >> The event will be sent when we get an allocation failure in metaspace. >> The event will contain the following information: >> - classLoader - the class loader doing the allocation >> - anonymousClassLoader - if the classLoader is anonymous >> - nativeStackTrace - a VM stack trace (see more below) >> - metadataType - the kind of metadata (class or non-class) >> - metaspaceObjectType - the kind of metaspace object >> >> The field nativeStackTrace will contain a string representation of the >> Threads C frames (up until the first Java frame). This is very useful >> for debugging metaspace allocation failures coming from VM internals >> such as the interpreter or GC code. >> >> I would especially appreciate if someone could look over the code in >> metaspaceTracer.cpp performing the stack walking. >> >> Webrev: >> http://cr.openjdk.java.net/~ehelin/8036699/webrev.00/ >> >> Enhancement: >> https://bugs.openjdk.java.net/browse/JDK-8036699 >> >> Testing: >> - JFR JTREG tests >> - JPRT >> - Manual stress testing of the stack walking code by writing small test >> programs that causes metaspace allocation failures from both Java >> threads and non-Java threads. >> >> Thanks, >> Erik From stefan.karlsson at oracle.com Mon Mar 17 12:39:40 2014 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 17 Mar 2014 13:39:40 +0100 Subject: RFR: 8036703: Add trace event with statistics for the metaspace chunk free lists In-Reply-To: <5321ABF1.9030403@oracle.com> References: <53184144.3060905@oracle.com> <5318544E.7050006@oracle.com> <5320858A.8030302@oracle.com> <1629610.a2STeJrlTr@mgerdin03> <53217D09.2010009@oracle.com> <5321ABF1.9030403@oracle.com> Message-ID: <5326ED0C.1070705@oracle.com> On 2014-03-13 14:00, Erik Helin wrote: > Hi Mikael and Stefan, > > thanks for having a look and special thanks to Mikael for finding the > bug! > > Please see new webrev at: > http://cr.openjdk.java.net/~ehelin/8036703/webrev.04/ > > Please see the incremental webrev at: > http://cr.openjdk.java.net/~ehelin/8036703/webrev.03-04/ Looks good. StefanK > > Based on some internal feedback, I also corrected a few nits (see the > incremental webrev for details). > > Thanks, > Erik > > On 2014-03-13 10:40, Stefan Karlsson wrote: >> >> On 2014-03-13 09:57, Mikael Gerdin wrote: >>> Hi Erik, >>> >>> On Wednesday 12 March 2014 17.04.26 Erik Helin wrote: >>>> Based on feedback from Mikael and StefanK, I've reworked the patch to >>>> put the ChunkFreeListSummaries in the class MetaspaceSummary. >>>> >>>> Please see the new version at: >>>> http://cr.openjdk.java.net/~ehelin/8036703/webrev.03/ >>> I think this version is much better! >>> Just a small issue: >>> >>> collectedHeap.cpp >>> 99 >>> 100 const MetaspaceChunkFreeListSummary& >>> ms_chunk_free_list_summary = >>> 101 MetaspaceAux::chunk_free_list_summary(Metaspace::ClassType); >>> 102 const MetaspaceChunkFreeListSummary& >>> class_chunk_free_list_summary = >>> 103 MetaspaceAux::chunk_free_list_summary(Metaspace::NonClassType); >>> 104 >>> It looks like you pass the wrong MetadataType to >>> chunk_free_list_summary here, >>> ClassType and NonClassType should be swapped. >> >> Except for this bug, this looks good. >> >> thanks, >> StefanK >> >>> >>> /Mikael >>> >>>> Thanks, >>>> Erik >>>> >>>> On 2014-03-06 11:56, Erik Helin wrote: >>>>> A small bug had found its way into this patch: >>>>> >>>>> metaspace.cpp: >>>>> +bool MetaspaceAux::has_chunk_free_list(Metaspace::MetadataType >>>>> mdtype) { >>>>> + return Metaspace::get_chunk_manager(mdtype) == NULL; >>>>> +} >>>>> >>>>> should of course be: >>>>> >>>>> +bool MetaspaceAux::has_chunk_free_list(Metaspace::MetadataType >>>>> mdtype) { >>>>> + return Metaspace::get_chunk_manager(mdtype) != NULL; >>>>> +} >>>>> >>>>> Please see new webrev at: >>>>> http://cr.openjdk.java.net/~ehelin/8036703/webrev.01/ >>>>> >>>>> Thanks, >>>>> Erik >>>>> >>>>> On 2014-03-06 10:35, Erik Helin wrote: >>>>>> Hi all, >>>>>> >>>>>> this patch adds a new trace event - >>>>>> vm/gc/metaspace/chunk_free_list_summary for getting statistics for >>>>>> metaspace's chunk free lists. The event contains the numbers of >>>>>> chunks >>>>>> in the free list and the size of the chunks in the free lists. This >>>>>> information is provided for all different kinds of chunks - special, >>>>>> small, medium and humongous. >>>>>> >>>>>> This information will be useful when diagnosing fragmentation >>>>>> issues in >>>>>> metaspace. >>>>>> >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~ehelin/8036703/webrev.00/ >>>>>> >>>>>> Enhancement: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8036703 >>>>>> >>>>>> Testing: >>>>>> - JFR JTREG tests >>>>>> - JPRT >>>>>> - Manually verifying that the numbers look correct by looking at >>>>>> >>>>>> recordings from small test cases. >>>>>> >>>>>> Thanks, >>>>>> Erik >> From stefan.karlsson at oracle.com Mon Mar 17 12:41:47 2014 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 17 Mar 2014 13:41:47 +0100 Subject: RFR: 8036699: Add trace event when a metaspace allocation fails In-Reply-To: <5326ED33.7090107@oracle.com> References: <53183925.4020001@oracle.com> <5321D6D1.8060503@oracle.com> <5326ED33.7090107@oracle.com> Message-ID: <5326ED8B.5010400@oracle.com> On 2014-03-17 13:40, Erik Helin wrote: > (this reply went to the wrong RFR, reposting to get it back on to the > right thread, sorry for the noise) > > Based on a discussion with Jon Masamitsu and StefanK, I've decided to > change metadata_type_name to return a slightly more user-friendly value. > > Please see new webrev at: > http://cr.openjdk.java.net/~ehelin/8036699/webrev.02/ > > Incremental webrev: > http://cr.openjdk.java.net/~ehelin/8036699/webrev.01-02/ The diff looks good. StefanK > > Thanks, > Erik > > On 2014-03-13 17:03, Erik Helin wrote: >> Hi all, >> >> due to lack of reviewers for the native stack trace code, I've decided >> to split this patch into two patches: >> - adding the event >> - adding the native stack trace information the event >> (will be sent out later) >> >> This review request is now only for adding the event. Please see a new >> webrev at: >> http://cr.openjdk.java.net/~ehelin/8036699/webrev.01/ >> >> For an incremental webrev (that only removes the native stack walking >> code) please see: >> http://cr.openjdk.java.net/~ehelin/8036699/webrev.00-01/ >> >> Thanks, >> Erik >> >> On 2014-03-06 10:00, Erik Helin wrote: >>> Hi all, >>> >>> this patch adds the new trace event vm/gc/metaspace/allocation_failure. >>> The event will be sent when we get an allocation failure in metaspace. >>> The event will contain the following information: >>> - classLoader - the class loader doing the allocation >>> - anonymousClassLoader - if the classLoader is anonymous >>> - nativeStackTrace - a VM stack trace (see more below) >>> - metadataType - the kind of metadata (class or non-class) >>> - metaspaceObjectType - the kind of metaspace object >>> >>> The field nativeStackTrace will contain a string representation of the >>> Threads C frames (up until the first Java frame). This is very useful >>> for debugging metaspace allocation failures coming from VM internals >>> such as the interpreter or GC code. >>> >>> I would especially appreciate if someone could look over the code in >>> metaspaceTracer.cpp performing the stack walking. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~ehelin/8036699/webrev.00/ >>> >>> Enhancement: >>> https://bugs.openjdk.java.net/browse/JDK-8036699 >>> >>> Testing: >>> - JFR JTREG tests >>> - JPRT >>> - Manual stress testing of the stack walking code by writing small test >>> programs that causes metaspace allocation failures from both Java >>> threads and non-Java threads. >>> >>> Thanks, >>> Erik From neugens at redhat.com Mon Mar 17 13:15:46 2014 From: neugens at redhat.com (Mario Torre) Date: Mon, 17 Mar 2014 14:15:46 +0100 Subject: [8u20] RFR (XS): 8036619: Shark: add LLVM 3.4 support Message-ID: <1395062146.3881.4.camel@galactica.localdomain> Resending with correct webrev and subject :) I would like to ask the backport of the changes made for the following bug report into JDK 8 (jdk8u20): https://bugs.openjdk.java.net/browse/JDK-8036619 The changes apply cleanly without modification: http://cr.openjdk.java.net/~neugens/zero-2014-02-24/webrev.03/ The related changes for JDK 9 are here: http://cr.openjdk.java.net/~neugens/zero-2014-02-24/webrev.02/ The discussion here: http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-March/012814.html and here: http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-February/012679.html I can't push myself, so if OK, someone would need to do for me. Cheers, Mario From goetz.lindenmaier at sap.com Mon Mar 17 13:36:07 2014 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 17 Mar 2014 13:36:07 +0000 Subject: RFR (M): 8036767 PPC64: Support for little endian execution model In-Reply-To: <5326DE2F.6000107@oracle.com> References: <531E8309.2060206@oracle.com> <531ED06F.5040905@oracle.com> <4295855A5C1DE049A61835A1887419CC2CEA6A09@DEWDFEMB12A.global.corp.sap> <531F9370.4020305@oracle.com> <5320E5F8.3020500@oracle.com> <5326766F.80104@oracle.com> <5326DE2F.6000107@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CEA859D@DEWDFEMB12A.global.corp.sap> Hi Magnus, The change keeps target cpu and target cpu arch as is. The C-code uses -DLITTLE_ENDIAN to distinguish implementation differences. As I understand, we discuss the setting of BUILDARCH in the makefiles. Indirectly it is set from uname, which returns ppc64le on these machines. This affects the following namings: 1. the build path: linux_ppc64le-compiler2 2. which makefiles used: platform_ppc64le and ppc64le.make 1.) would allow to build ppc64 and ppc64le from the same repo without setting ALT_OUTPUTDIR 2.) BUILDARCH=ppc64le means two new files, where platform_ppc64 and platform_ppc64le are the same. BUILDARCH=ppc64 means no new files, but lots of platform checks in ppc64.make, which is uncommon. So far these files are just settings. I don't think setting LIBARCH=ppc64le ==> jre/lib/ppc64le/server/libjvm.so and the settings in os_linux.cpp are questioned. Best regards, Goetz. http://cr.openjdk.java.net/~martin/asmundak/8036767/webrev.00/ -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Magnus Ihse Bursie Sent: Montag, 17. M?rz 2014 12:36 To: David Holmes Cc: build-dev; HotSpot Open Source Developers; Alexander Smundak Subject: Re: RFR (M): 8036767 PPC64: Support for little endian execution model On 2014-03-17 05:13, David Holmes wrote: > On 15/03/2014 7:11 AM, Alexander Smundak wrote: >> Ping. > > My position hasn't changed. I don't think this needs to be, or should > be, a distinct architecture. > > I've added build-dev to cc list to see what our build experts think. I'm not sure I've completely understood the discussion (it has jumped around a bit between mailing lists), but I think I agree with David here. Architecture is a term that implies general common features, not specific differences. While hotspot and the rest of the jdk build is not currently in sync with this terminology, in the common configure and makefiles we use the terminology "target cpu" (which can be for instance "x86_64" or "sparcv9" and "target cpu arch" (which can be for instance "x86" or "sparc"). Regarding ppc, we currently define the target cpu arch "ppc" and two target cpus, "ppc" and "ppc64". Perhaps it would be more approriate to consider this a separate target cpu ("ppc64le"), but not a separate target cpu architecture? But that would be on the top-level -- I'm not sure how that would map properly to the current hotspot build system. /Magnus From aph at redhat.com Mon Mar 17 15:33:53 2014 From: aph at redhat.com (Andrew Haley) Date: Mon, 17 Mar 2014 15:33:53 +0000 Subject: HotSpot update for 7u? Message-ID: <532715E1.4040305@redhat.com> We have a fairly simple test case that crashes C2 in jdk7u: http://icedtea.classpath.org/bugzilla/show_bug.cgi?id=1710 It is fixed in HS25. I just built jdk7u60 and can confirm that it crashes that too. I've looked at the HotSpot logs, and it appears that the register allocator was reorganized shortly after the HS25 split. It looks like this chunk of HotSpot has been substantially rewritten. So, the question is this: should we update HotSpot in jdk7u? Or is that too much of a risk? Andrew. From calvin.cheung at oracle.com Mon Mar 17 17:07:08 2014 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Mon, 17 Mar 2014 10:07:08 -0700 Subject: RFR (XS) 8025550 valgrind: Conditional jump depends on uninitialised value in Arena::set_size_in_bytes() In-Reply-To: References: Message-ID: <53272BBC.8090405@oracle.com> Looks good. Calvin On 2/25/2014 9:43 AM, Zhengyu Gu wrote: > This is a simple fix to NMT, which can mis-count arena size due to uninitialized value. In rare case, if uninitialized size happens to be the same value as arena's initial size, NMT will mis-count the size, since NMT only updates size when the new size does not equal to the old one. > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8025550 > Webrev: http://cr.openjdk.java.net/~zgu/8025550/webrev.00/ > > Thanks, > > -Zhengyu From zhengyu.gu at oracle.com Mon Mar 17 17:43:09 2014 From: zhengyu.gu at oracle.com (Zhengyu Gu) Date: Mon, 17 Mar 2014 13:43:09 -0400 Subject: RFR (XS) 8025550 valgrind: Conditional jump depends on uninitialised value in Arena::set_size_in_bytes() In-Reply-To: <53272BBC.8090405@oracle.com> References: <53272BBC.8090405@oracle.com> Message-ID: <0698A1EA-E97C-42D6-86C7-B4FCD158479A@oracle.com> Thanks Calvin. -Zhengyu On Mar 17, 2014, at 1:07 PM, Calvin Cheung wrote: > Looks good. > > Calvin > > On 2/25/2014 9:43 AM, Zhengyu Gu wrote: >> This is a simple fix to NMT, which can mis-count arena size due to uninitialized value. In rare case, if uninitialized size happens to be the same value as arena's initial size, NMT will mis-count the size, since NMT only updates size when the new size does not equal to the old one. >> >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8025550 >> Webrev: http://cr.openjdk.java.net/~zgu/8025550/webrev.00/ >> >> Thanks, >> >> -Zhengyu > From calvin.cheung at oracle.com Mon Mar 17 18:13:34 2014 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Mon, 17 Mar 2014 11:13:34 -0700 Subject: RFR (S): 8034034: JNI exc. pending in hotspot/agent/src/os/bsd/MacosxDebuggerLocal.m Message-ID: <53273B4E.9040500@oracle.com> Please review this small fix for adding JNI exception check in MacosxDebuggerLocal.m. This fix addresses the following warnings from our internal code analysis tool: 1) Java exception thrown by 'GetFieldID(env, cls, "symbolicator", "J")' pending at call to 'GetFieldID' at line 132 of jdk8-b128/src/hotspot/agent/src/os/bsd/MacosxDebuggerLocal.m in function Java_sun_jvm_hotspot_debugger_bsd_BsdDebuggerLocal_init0 JNI function 'GetFieldID' may throw an exception at line 131 JNI function 'GetFieldID' is unsafe for a pending exception at line 132 2) Java exception thrown by 'NewStringUTF(env, sym)' pending at call to 'CallObjectMethod' at line 245 of jdk8-b128/src/hotspot/agent/src/os/bsd/MacosxDebuggerLocal.m in function Java_sun_jvm_hotspot_debugger_bsd_BsdDebuggerLocal_lookupByAddress0 JNI function 'NewStringUTF' may throw an exception at line 245 JNI function 'CallObjectMethod' is unsafe for a pending exception at line 245 3) Java exception thrown by 'NewStringUTF(env, name)' pending at call to 'CallObjectMethod' at line 756 of jdk8-b128/src/hotspot/agent/src/os/bsd/MacosxDebuggerLocal.m in function fillLoadObjects JNI function 'NewStringUTF' may throw an exception at line 756 JNI function 'CallObjectMethod' is unsafe for a pending exception at line 756 4) Java exception thrown by 'FindClass(env, "sun/jvm/hotspot/debugger/DebuggerException")' pending at call to 'ThrowNew' at line 98 of jdk8-b128/src/hotspot/agent/src/os/bsd/MacosxDebuggerLocal.m in function throw_new_debugger_exception JNI function 'FindClass' may throw an exception at line 98 JNI function 'ThrowNew' is unsafe for a pending exception at line 98 bug: https://bugs.openjdk.java.net/browse/JDK-8034034 (sorry, internal bug) webrev: http://cr.openjdk.java.net/~ccheung/8034034/webrev/ test: JPRT thanks, Calvin From staffan.larsen at oracle.com Mon Mar 17 18:25:44 2014 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Mon, 17 Mar 2014 19:25:44 +0100 Subject: RFR (S): 8034034: JNI exc. pending in hotspot/agent/src/os/bsd/MacosxDebuggerLocal.m In-Reply-To: <53273B4E.9040500@oracle.com> References: <53273B4E.9040500@oracle.com> Message-ID: Looks good! Thanks, /Staffan On 17 mar 2014, at 19:13, Calvin Cheung wrote: > Please review this small fix for adding JNI exception check in MacosxDebuggerLocal.m. > > This fix addresses the following warnings from our internal code analysis tool: > > 1) > Java exception thrown by 'GetFieldID(env, cls, "symbolicator", "J")' pending > at call to 'GetFieldID' at line 132 of > jdk8-b128/src/hotspot/agent/src/os/bsd/MacosxDebuggerLocal.m in function > Java_sun_jvm_hotspot_debugger_bsd_BsdDebuggerLocal_init0 > JNI function 'GetFieldID' may throw an exception at line 131 > JNI function 'GetFieldID' is unsafe for a pending exception at line 132 > > 2) > Java exception thrown by 'NewStringUTF(env, sym)' pending at call to > 'CallObjectMethod' at line 245 of > jdk8-b128/src/hotspot/agent/src/os/bsd/MacosxDebuggerLocal.m in function > Java_sun_jvm_hotspot_debugger_bsd_BsdDebuggerLocal_lookupByAddress0 > JNI function 'NewStringUTF' may throw an exception at line 245 > JNI function 'CallObjectMethod' is unsafe for a pending exception at line 245 > > 3) > Java exception thrown by 'NewStringUTF(env, name)' pending at call to > 'CallObjectMethod' at line 756 of > jdk8-b128/src/hotspot/agent/src/os/bsd/MacosxDebuggerLocal.m in function > fillLoadObjects > JNI function 'NewStringUTF' may throw an exception at line 756 > JNI function 'CallObjectMethod' is unsafe for a pending exception at line 756 > > 4) > Java exception thrown by 'FindClass(env, > "sun/jvm/hotspot/debugger/DebuggerException")' pending at call to 'ThrowNew' > at line 98 of jdk8-b128/src/hotspot/agent/src/os/bsd/MacosxDebuggerLocal.m in > function throw_new_debugger_exception > JNI function 'FindClass' may throw an exception at line 98 > JNI function 'ThrowNew' is unsafe for a pending exception at line 98 > > > bug: https://bugs.openjdk.java.net/browse/JDK-8034034 (sorry, internal bug) > > webrev: http://cr.openjdk.java.net/~ccheung/8034034/webrev/ > > test: JPRT > > thanks, > Calvin > From calvin.cheung at oracle.com Mon Mar 17 18:43:00 2014 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Mon, 17 Mar 2014 11:43:00 -0700 Subject: RFR (S): 8034034: JNI exc. pending in hotspot/agent/src/os/bsd/MacosxDebuggerLocal.m In-Reply-To: References: <53273B4E.9040500@oracle.com> Message-ID: <53274234.5080206@oracle.com> Thanks for your quick review, Staffan. Calvin On 3/17/2014 11:25 AM, Staffan Larsen wrote: > Looks good! > > Thanks, > /Staffan > > On 17 mar 2014, at 19:13, Calvin Cheung wrote: > >> Please review this small fix for adding JNI exception check in MacosxDebuggerLocal.m. >> >> This fix addresses the following warnings from our internal code analysis tool: >> >> 1) >> Java exception thrown by 'GetFieldID(env, cls, "symbolicator", "J")' pending >> at call to 'GetFieldID' at line 132 of >> jdk8-b128/src/hotspot/agent/src/os/bsd/MacosxDebuggerLocal.m in function >> Java_sun_jvm_hotspot_debugger_bsd_BsdDebuggerLocal_init0 >> JNI function 'GetFieldID' may throw an exception at line 131 >> JNI function 'GetFieldID' is unsafe for a pending exception at line 132 >> >> 2) >> Java exception thrown by 'NewStringUTF(env, sym)' pending at call to >> 'CallObjectMethod' at line 245 of >> jdk8-b128/src/hotspot/agent/src/os/bsd/MacosxDebuggerLocal.m in function >> Java_sun_jvm_hotspot_debugger_bsd_BsdDebuggerLocal_lookupByAddress0 >> JNI function 'NewStringUTF' may throw an exception at line 245 >> JNI function 'CallObjectMethod' is unsafe for a pending exception at line 245 >> >> 3) >> Java exception thrown by 'NewStringUTF(env, name)' pending at call to >> 'CallObjectMethod' at line 756 of >> jdk8-b128/src/hotspot/agent/src/os/bsd/MacosxDebuggerLocal.m in function >> fillLoadObjects >> JNI function 'NewStringUTF' may throw an exception at line 756 >> JNI function 'CallObjectMethod' is unsafe for a pending exception at line 756 >> >> 4) >> Java exception thrown by 'FindClass(env, >> "sun/jvm/hotspot/debugger/DebuggerException")' pending at call to 'ThrowNew' >> at line 98 of jdk8-b128/src/hotspot/agent/src/os/bsd/MacosxDebuggerLocal.m in >> function throw_new_debugger_exception >> JNI function 'FindClass' may throw an exception at line 98 >> JNI function 'ThrowNew' is unsafe for a pending exception at line 98 >> >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8034034 (sorry, internal bug) >> >> webrev: http://cr.openjdk.java.net/~ccheung/8034034/webrev/ >> >> test: JPRT >> >> thanks, >> Calvin >> From vladimir.kozlov at oracle.com Mon Mar 17 19:11:43 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 17 Mar 2014 12:11:43 -0700 Subject: RFR (XL) 8031320: Use Intel RTM instructions for locks Message-ID: <532748EF.6080200@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8031320 http://cr.openjdk.java.net/~kvn/8031320_9/webrev/ The Intel architectures codenamed Haswell has support for RTM (Restricted Transactional Memory) instructions xbegin, xabort, xend and xtest as part of Intel Transactional Synchronization Extension (TSX). The xbegin and xend instructions enclose a set of instructions to be executed as a transaction. If no conflict found during execution of the transaction, the memory and register modifications are committed together at xend. xabort instruction can be used for explicit abort of transaction and xtest to check if we are in transaction. RTM is useful for highly contended locks with low conflict in the critical region. The highly contended locks don't scale well otherwise but with RTM they show good scaling. RTM allows using coarse grain locking for applications. Also for lightly contended locks which are used by different threads RTM can reduce cache line ping pong and thereby show performance improvement too. Implementation: Generate RTM locking code for all inflated locks when "UseRTMLocking" option is on with normal locking as fall back mechanism. On abort or lock busy the lock will be retried a fixed number of times as specified by "RTMRetryCount" option. The locks which abort too often can be auto tuned or manually tuned. Auto-tuning can be done using "UseRTMDeopt" flag which will add an abort ratio calculation code for each lock. The abort ratio will be calculated after "RTMAbortThreshold" aborts are encountered. With "UseRTMDeopt" if the aborts ratio reaches "RTMAbortRatio" the nmethod containing the lock will be deoptimized and recompiled with all locks as normal (stack) locks. If the abort ratio continues to remain low after "RTMLockingThreshold" attempted locks, then the method will be deoptimized and recompiled with all locks as RTM locks without abort ratio calculation code. The abort ratio calculation can be delayed by specifying -XX:RTMLockingCalculationDelay= flag. Deoptimization of nmethod is done by adding an uncommon trap at the beginning of the code which checks rtm state field in MDO which is modified by the abort calculation code. For manual tuning the abort statistics for each lock could be provided to a user using "PrintPreciseRTMLockingStatistics" diagnostic flag. Based on the abort statistics users can create a .hotspot_compiler file or use -XX:CompileCommand=