From david.holmes at oracle.com  Mon Jan  2 21:41:20 2012
From: david.holmes at oracle.com (David Holmes)
Date: Tue, 03 Jan 2012 15:41:20 +1000
Subject: RFR (S): 7125793: MAC: test_gamma should always work
In-Reply-To: <4EFF3BC6.3060407@oracle.com>
References: <4EFF3BC6.3060407@oracle.com>
Message-ID: <4F029500.4080407@oracle.com>

Hi Jim,

I'm getting increasingly concerned about platform specific code being 
in, or being added to, what is nominally shared code, so I'd like to see 
if we can reduce some of that:

src/os/posix/launcher/java_md.c

Instead of the ifdef APPLE can we not factor out a JRE_LIB_PATH (or 
something like that) that is set to jre/lib or jre/lib/<arch> as 
appropriate by the platform specific build files? This would also allow 
the new case

/* Is the JRE universal, i.e. no arch dir? */

to be handled by the existing code.

---

src/os/bsd/vm/os_bsd.cpp

Similar comment - use JRE_LIB_PATH instead of jre/lib/%s etc

---

David
-----



On 1/01/2012 2:43 AM, James Melvin wrote:
> Hi,
>
> This change fixes the 'gamma' simple launcher for HotSpot on Mac OS X.
> There were 3 primary changes required to re-enable gamma...
>
> 1) Statically link with CoreFoundation framework to resolve symbols
>
> The gamma launcher dlopen()s libjava.dylib from $JAVA_HOME/jre/lib.
> Because Mac OS X files are case-insensitive by default, we collide on
> $FRAMEWORK/libJPEG.dylib and ${JAVA_HOME}/jre/lib/libjpeg.dylib. This
> resulted in unresolved symbols in the Mac OS X framework libraries. The
> solution for gamma was to statically link with CoreFoundation framework
> to properly resolve framework symbols and allow gamma to successfully
> dlopen() libjava.dylib.
>
> 2) Adjust various paths to reflect no arch subdirs
>
> On Mac OS X, there are no arch subdirs, e.g jre/lib vs jre/lib/<arch>.
> Instead, one can use universal binaries to package multiple
> architectures in a single binary. At the moment though, we are only
> building 64-bit non-universal binaries. Note, the test_gamma script
> assumes an Oracle JDK layout for JAVA_HOME, derived from ALT_BOOTDIR.
> Using an Apple JDK for ALT_BOOTDIR will fail the test_gamma script
> gracefully, as libjava.dylib is in a different, unexpected place.
>
> 3) Modify test_gamma script to set library path only for gamma launch
>
> Setting DYLD_LIBRARY_PATH adversely affects the real java launcher(s).
> Instead, set this later in the script only for the gamma launcher test
> run. While in there, I took the liberty of decrypting the script to make
> it more maintainable and more easily merged whenever we reconcile the
> unix ports into a single codebase. There is no change to the script output.
>
> Feedback welcome...
>
> WEBREV:
> http://cr.openjdk.java.net/~jmelvin/7125793/webrev.00
>
> TESTS RUN:
> JPRT 2011-12-31-061123.jmelvin.7125793
> local Mac OS X builds/tests
>
>
> Thanks and Happy New Year!
>
> Jim

From daniel.daugherty at oracle.com  Tue Jan  3 10:39:13 2012
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Tue, 03 Jan 2012 11:39:13 -0700
Subject: RFR (S): 7125793: MAC: test_gamma should always work
In-Reply-To: <4EFECA5D.6010905@oracle.com>
References: <4EFECA5D.6010905@oracle.com>
Message-ID: <4F034B51.3070609@oracle.com>

> http://cr.openjdk.java.net/~jmelvin/7125793/webrev.00 

Jim,

Thanks for diving in and improving the MacOS X port!
Comments below.

Dan


make/bsd/makefiles/buildtree.make

     line 422: The new 'java -fullversion' invocation does not include
         the $(JAVA_FLAG) option like the old code did. Any particular
         reason for the change?

         Looks like that means the '-d32' or '-d64' options won't be
         specified as they were before.

     line 447: Why not just echo FULL_VERSION? Why pipe to awk?

     line 465: The 'jre/lib/libjava.dylib' part of the new check is
         MacOS X specific. Other BSDs don't necessarily use the
         '.dylib' extension (instead of .so) and I don't think that
         other BSDs have dropped the "arch" subdir.

     line 484: The DYLD_LIBRARY_PATH part is MacOS X specific. Will
         still need to set LD_LIBRARY_PATH for other BSDs.

     line 492: You switched from $(TESTFLAGS) to literal flag values,
         but you left the TESTFLAGS variable around. Any reason for
         the switch?


make/bsd/makefiles/launcher.make
     Please add a comment explaining why '-framework CoreFoundation'
     is needed. Your explanatory block below is a really good start.


make/bsd/makefiles/vm.make
     No comments.


src/os/bsd/vm/os_bsd.cpp
     line 2585: Uses a suffix of ".so". That shouldn't work on MacOS X
         since MacOS X uses '.dylib'. That's OK for other BSDs, but not
         MacOS X. Also the comments that mention '.so' should be updated
         to include '.dylib' (not caused by your changes).

     To David H. - Yes, this change added another '#fdef __APPLE__'. It
         is not the first and it likely won't be the last since we're
         not done yet with the MacOS X port. There are a number of
         things that need to be cleaned up and we're tracking them.
         However, as you know, we don't have enough folks to handle all
         of the work so we'll just have to live with the warts for now.

src/os/posix/launcher/java_md.c
     No comments.


On 12/31/11 1:39 AM, James Melvin wrote:
> Hi,
>
> This change fixes the 'gamma' simple launcher for HotSpot on Mac OS X.
> There were 3 primary changes required to re-enable gamma...
>
> 1) Statically link with CoreFoundation framework to resolve symbols
>
> The gamma launcher dlopen()s libjava.dylib from $JAVA_HOME/jre/lib.
> Because Mac OS X files are case-insensitive by default, we collide on
> $FRAMEWORK/libJPEG.dylib and ${JAVA_HOME}/jre/lib/libjpeg.dylib. This
> resulted in unresolved symbols in the Mac OS X framework libraries. The
> solution for gamma was to statically link with CoreFoundation framework
> to properly resolve framework symbols and allow gamma to successfully
> dlopen() libjava.dylib.
>
> 2) Adjust various paths to reflect no arch subdirs
>
> On Mac OS X, there are no arch subdirs, e.g jre/lib vs jre/lib/<arch>.
> Instead, one can use universal binaries to package multiple
> architectures in a single binary. At the moment though, we are only
> building 64-bit non-universal binaries. Note, the test_gamma script
> assumes an Oracle JDK layout for JAVA_HOME, derived from ALT_BOOTDIR.
> Using an Apple JDK for ALT_BOOTDIR will fail the test_gamma script
> gracefully, as libjava.dylib is in a different, unexpected place.
>
> 3) Modify test_gamma script to set library path only for gamma launch
>
> Setting DYLD_LIBRARY_PATH adversely affects the real java launcher(s).
> Instead, set this later in the script only for the gamma launcher test
> run. While in there, I took the liberty of decrypting the script to make
> it more maintainable and more easily merged whenever we reconcile the
> unix ports into a single codebase. There is no change to the script 
> output.
>
> Feedback welcome...
>
> WEBREV:
>   http://cr.openjdk.java.net/~jmelvin/7125793/webrev.00
>
> TESTS RUN:
>   JPRT  2011-12-31-061123.jmelvin.7125793
>   local Mac OS X builds/tests
>
>
> Thanks and Happy New Year!
>
> Jim

From kelly.ohair at oracle.com  Tue Jan  3 11:23:26 2012
From: kelly.ohair at oracle.com (Kelly O'Hair)
Date: Tue, 3 Jan 2012 11:23:26 -0800
Subject: RFR (S): 7125793: MAC: test_gamma should always work
In-Reply-To: <4F029500.4080407@oracle.com>
References: <4EFF3BC6.3060407@oracle.com> <4F029500.4080407@oracle.com>
Message-ID: <462344E2-8F4D-46B2-8BF2-473DBCBC2C1A@oracle.com>

I was having similar concerns, but wasn't able to come up with a clean alternative at the time.

Having a ARCH_LIB_PATH_APPEND and a ARCH_BIN_PATH_APPEND might be better,
(I think bin and lib might use different names sometimes? Or they did once upon a time? bin/v9 lib/sparcv9?)

Anyway, I'd like to see the proprietary names left out. Or minimized. Or isolated to the Makefiles?

-kto

On Jan 2, 2012, at 9:41 PM, David Holmes wrote:

> Hi Jim,
> 
> I'm getting increasingly concerned about platform specific code being in, or being added to, what is nominally shared code, so I'd like to see if we can reduce some of that:
> 
> src/os/posix/launcher/java_md.c
> 
> Instead of the ifdef APPLE can we not factor out a JRE_LIB_PATH (or something like that) that is set to jre/lib or jre/lib/<arch> as appropriate by the platform specific build files? This would also allow the new case
> 
> /* Is the JRE universal, i.e. no arch dir? */
> 
> to be handled by the existing code.
> 
> ---
> 
> src/os/bsd/vm/os_bsd.cpp
> 
> Similar comment - use JRE_LIB_PATH instead of jre/lib/%s etc
> 
> ---
> 
> David
> -----
> 
> 
> 
> On 1/01/2012 2:43 AM, James Melvin wrote:
>> Hi,
>> 
>> This change fixes the 'gamma' simple launcher for HotSpot on Mac OS X.
>> There were 3 primary changes required to re-enable gamma...
>> 
>> 1) Statically link with CoreFoundation framework to resolve symbols
>> 
>> The gamma launcher dlopen()s libjava.dylib from $JAVA_HOME/jre/lib.
>> Because Mac OS X files are case-insensitive by default, we collide on
>> $FRAMEWORK/libJPEG.dylib and ${JAVA_HOME}/jre/lib/libjpeg.dylib. This
>> resulted in unresolved symbols in the Mac OS X framework libraries. The
>> solution for gamma was to statically link with CoreFoundation framework
>> to properly resolve framework symbols and allow gamma to successfully
>> dlopen() libjava.dylib.
>> 
>> 2) Adjust various paths to reflect no arch subdirs
>> 
>> On Mac OS X, there are no arch subdirs, e.g jre/lib vs jre/lib/<arch>.
>> Instead, one can use universal binaries to package multiple
>> architectures in a single binary. At the moment though, we are only
>> building 64-bit non-universal binaries. Note, the test_gamma script
>> assumes an Oracle JDK layout for JAVA_HOME, derived from ALT_BOOTDIR.
>> Using an Apple JDK for ALT_BOOTDIR will fail the test_gamma script
>> gracefully, as libjava.dylib is in a different, unexpected place.
>> 
>> 3) Modify test_gamma script to set library path only for gamma launch
>> 
>> Setting DYLD_LIBRARY_PATH adversely affects the real java launcher(s).
>> Instead, set this later in the script only for the gamma launcher test
>> run. While in there, I took the liberty of decrypting the script to make
>> it more maintainable and more easily merged whenever we reconcile the
>> unix ports into a single codebase. There is no change to the script output.
>> 
>> Feedback welcome...
>> 
>> WEBREV:
>> http://cr.openjdk.java.net/~jmelvin/7125793/webrev.00
>> 
>> TESTS RUN:
>> JPRT 2011-12-31-061123.jmelvin.7125793
>> local Mac OS X builds/tests
>> 
>> 
>> Thanks and Happy New Year!
>> 
>> Jim


From john.coomes at oracle.com  Tue Jan  3 12:46:09 2012
From: john.coomes at oracle.com (john.coomes at oracle.com)
Date: Tue, 03 Jan 2012 20:46:09 +0000
Subject: hg: hsx/hotspot-main/langtools: 12 new changesets
Message-ID: <20120103204637.CC12547861@hg.openjdk.java.net>

Changeset: 4822dfe0922b
Author:    ohair
Date:      2011-12-12 08:15 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/langtools/rev/4822dfe0922b

7119829: Adjust default jprt testing configuration
Reviewed-by: alanb

! make/jprt.properties

Changeset: 3809292620c9
Author:    jjg
Date:      2011-12-13 11:21 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/langtools/rev/3809292620c9

7120736: refactor javac option handling
Reviewed-by: mcimadamore

! src/share/classes/com/sun/tools/javac/api/JavacTool.java
! src/share/classes/com/sun/tools/javac/code/Source.java
! src/share/classes/com/sun/tools/javac/comp/Check.java
! src/share/classes/com/sun/tools/javac/comp/Enter.java
! src/share/classes/com/sun/tools/javac/comp/Lower.java
! src/share/classes/com/sun/tools/javac/file/Locations.java
! src/share/classes/com/sun/tools/javac/jvm/ClassReader.java
! src/share/classes/com/sun/tools/javac/jvm/ClassWriter.java
! src/share/classes/com/sun/tools/javac/jvm/Gen.java
! src/share/classes/com/sun/tools/javac/jvm/Target.java
! src/share/classes/com/sun/tools/javac/main/JavaCompiler.java
! src/share/classes/com/sun/tools/javac/main/Main.java
! src/share/classes/com/sun/tools/javac/nio/JavacPathFileManager.java
! src/share/classes/com/sun/tools/javac/processing/JavacProcessingEnvironment.java
! src/share/classes/com/sun/tools/javac/util/BaseFileManager.java
! src/share/classes/com/sun/tools/javac/util/Log.java
! src/share/classes/com/sun/tools/javac/util/Options.java
! test/tools/javac/diags/examples/UnsupportedEncoding.java

Changeset: 4e4fed1d02f9
Author:    jjg
Date:      2011-12-13 14:33 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/langtools/rev/4e4fed1d02f9

7121164: renamed files not committed
Reviewed-by: ksrini

- src/share/classes/com/sun/tools/javac/main/JavacOption.java
+ src/share/classes/com/sun/tools/javac/main/Option.java
+ src/share/classes/com/sun/tools/javac/main/OptionHelper.java
- src/share/classes/com/sun/tools/javac/main/OptionName.java
- src/share/classes/com/sun/tools/javac/main/RecognizedOptions.java

Changeset: 4261dc8af622
Author:    jjg
Date:      2011-12-14 16:16 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/langtools/rev/4261dc8af622

7111022: javac no long prints last round of processing
7121323: Sqe tests using -Xstdout option fail with an invalid flag error message
Reviewed-by: darcy

! src/share/classes/com/sun/tools/javac/main/Option.java
! src/share/classes/com/sun/tools/javac/processing/JavacProcessingEnvironment.java
! src/share/classes/com/sun/tools/javac/util/Log.java
! test/tools/javac/4846262/Test.sh
+ test/tools/javac/processing/options/testPrintProcessorInfo/TestWithXstdout.java
! test/tools/javac/util/T6597678.java

Changeset: 281eeedf9755
Author:    jjg
Date:      2011-12-14 17:52 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/langtools/rev/281eeedf9755

7121681: compiler message file broken for javac -fullversion
Reviewed-by: jjh

! src/share/classes/com/sun/tools/javac/main/Option.java

Changeset: 42ffceeceeca
Author:    jjg
Date:      2011-12-14 21:52 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/langtools/rev/42ffceeceeca

7121682: remove obsolete import
Reviewed-by: jjh

! test/tools/javac/api/T6838467.java

Changeset: ab2a880cc23b
Author:    lana
Date:      2011-12-15 19:53 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/langtools/rev/ab2a880cc23b

Merge


Changeset: 6b773fdeb633
Author:    jjg
Date:      2011-12-16 13:49 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/langtools/rev/6b773fdeb633

7121961: javadoc is missing a resource property
Reviewed-by: bpatel

! src/share/classes/com/sun/tools/doclets/formats/html/resources/standard.properties

Changeset: a7a2720c7897
Author:    jjh
Date:      2011-12-16 16:41 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/langtools/rev/a7a2720c7897

7122342: testPrintProcessorInfo/TestWithXstdout.java failed for JDK8 nightly build at 12/16/2011
Summary: Do not pass empty args to javac
Reviewed-by: jjg

! test/tools/javac/processing/options/testPrintProcessorInfo/TestWithXstdout.java

Changeset: 1ae5988e201b
Author:    mcimadamore
Date:      2011-12-19 12:07 +0000
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/langtools/rev/1ae5988e201b

7120463: Fix method reference parser support in order to avoid ambiguities
Summary: Add lookahead routine to disambiguate between method reference in method context and binary expression
Reviewed-by: jjg, dlsmith

! src/share/classes/com/sun/tools/javac/parser/JavacParser.java
! test/tools/javac/lambda/MethodReferenceParserTest.java

Changeset: 77b2c066084c
Author:    lana
Date:      2011-12-23 16:39 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/langtools/rev/77b2c066084c

Merge

- src/share/classes/com/sun/tools/javac/main/JavacOption.java
- src/share/classes/com/sun/tools/javac/main/OptionName.java
- src/share/classes/com/sun/tools/javac/main/RecognizedOptions.java

Changeset: ffd294128a48
Author:    katleman
Date:      2011-12-29 15:14 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/langtools/rev/ffd294128a48

Added tag jdk8-b19 for changeset 77b2c066084c

! .hgtags


From jon.masamitsu at oracle.com  Tue Jan  3 12:47:30 2012
From: jon.masamitsu at oracle.com (jon.masamitsu at oracle.com)
Date: Tue, 03 Jan 2012 20:47:30 +0000
Subject: hg: hsx/hotspot-main/hotspot: 3 new changesets
Message-ID: <20120103204741.A0F8E47862@hg.openjdk.java.net>

Changeset: 20bfb6d15a94
Author:    iveresov
Date:      2011-12-27 16:43 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/20bfb6d15a94

7124829: NUMA: memory leak on Linux with large pages
Summary: In os::free_memory() use mmap with the same attributes as for the heap space
Reviewed-by: kvn
Contributed-by: Aleksey Ignatenko <aleksey.v.ignatenko at intel.com>

! src/os/bsd/vm/os_bsd.cpp
! src/os/linux/vm/os_linux.cpp
! src/os/solaris/vm/os_solaris.cpp
! src/os/windows/vm/os_windows.cpp
! src/share/vm/gc_implementation/shared/mutableNUMASpace.cpp
! src/share/vm/gc_implementation/shared/mutableSpace.cpp
! src/share/vm/runtime/os.hpp

Changeset: 776173fc2df9
Author:    stefank
Date:      2011-12-29 07:37 +0100
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/776173fc2df9

7125516: G1: ~ConcurrentMark() frees incorrectly
Summary: Replaced the code with a ShouldNotReachHere
Reviewed-by: tonyp, jmasa

! src/share/vm/gc_implementation/g1/concurrentMark.cpp

Changeset: 5ee33ff9b1c4
Author:    jmasa
Date:      2012-01-03 10:22 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/5ee33ff9b1c4

Merge



From John.Coomes at oracle.com  Tue Jan  3 15:17:58 2012
From: John.Coomes at oracle.com (John Coomes)
Date: Tue, 3 Jan 2012 15:17:58 -0800
Subject: RFR (S): 7125793: MAC: test_gamma should always work
In-Reply-To: <4EFF3BC6.3060407@oracle.com>
References: <4EFF3BC6.3060407@oracle.com>
Message-ID: <20227.36006.993739.512423@oracle.com>

James Melvin (james.melvin at oracle.com) wrote:
> Hi,
> 
> This change fixes the 'gamma' simple launcher for HotSpot on Mac OS X.
> There were 3 primary changes required to re-enable gamma...
> 
> 1) Statically link with CoreFoundation framework to resolve symbols
> 
> The gamma launcher dlopen()s libjava.dylib from $JAVA_HOME/jre/lib.
> Because Mac OS X files are case-insensitive by default, we collide on
> $FRAMEWORK/libJPEG.dylib and ${JAVA_HOME}/jre/lib/libjpeg.dylib. This
> resulted in unresolved symbols in the Mac OS X framework libraries. The
> solution for gamma was to statically link with CoreFoundation framework
> to properly resolve framework symbols and allow gamma to successfully
> dlopen() libjava.dylib.
> 
> 2) Adjust various paths to reflect no arch subdirs
> 
> On Mac OS X, there are no arch subdirs, e.g jre/lib vs jre/lib/<arch>.
> Instead, one can use universal binaries to package multiple
> architectures in a single binary. At the moment though, we are only
> building 64-bit non-universal binaries. Note, the test_gamma script
> assumes an Oracle JDK layout for JAVA_HOME, derived from ALT_BOOTDIR.
> Using an Apple JDK for ALT_BOOTDIR will fail the test_gamma script
> gracefully, as libjava.dylib is in a different, unexpected place.

Hi Jim,

As others have said, avoid #ifdef __APPLE__ in the shared sources if
at all possible.

> 3) Modify test_gamma script to set library path only for gamma launch
> 
> Setting DYLD_LIBRARY_PATH adversely affects the real java launcher(s).
> Instead, set this later in the script only for the gamma launcher test
> run. While in there, I took the liberty of decrypting the script to make
> it more maintainable and more easily merged whenever we reconcile the
> unix ports into a single codebase. There is no change to the script output.

Better not to take that liberty, as it will make reconciling things
more difficult.  With your change, the osx version can't be easily
compared with the other ports; any semantic differences are lost in
the noise.  Also, brevity is a virtue--what once fit on a few lines
has bloated to multiple screenfulls.

-John

From rednaxelafx at gmail.com  Wed Jan  4 03:10:53 2012
From: rednaxelafx at gmail.com (Krystal Mok)
Date: Wed, 4 Jan 2012 19:10:53 +0800
Subject: Fwd: Request for review (XS): SA should cope with partially loaded
	ConstantPool
In-Reply-To: <CA+cQ+tRSf-iDrmgK7QDTz4aKrksK-_Yu0ypFEyKkYwO9V0-+bQ@mail.gmail.com>
References: <CA+cQ+tRSf-iDrmgK7QDTz4aKrksK-_Yu0ypFEyKkYwO9V0-+bQ@mail.gmail.com>
Message-ID: <CA+cQ+tTasAsjU+5i25UemSxfErtH4mtAGhsgThVW4a8ECQfyUg@mail.gmail.com>

cc'ing hotspot-dev

---------- Forwarded message ----------
From: Krystal Mok <rednaxelafx at gmail.com>
Date: Fri, Dec 30, 2011 at 8:36 PM
Subject: Request for review (XS): SA should cope with partially loaded
ConstantPool
To: serviceability-dev at openjdk.java.net


Hi all,

I was using CLHSDB to dump the contents of PermGen the other day, and ran
into a ClassCastException, as shown in [1].

It turns out that there was a partially loaded constantPoolOopDesc instance
in the PermGen, which is actually dead already, but not collected yet
(because no GC has happened yet). The way it's marked to be "partially
loaded" is setting a pointer to this constantPoolOopDesc object itself to
its _pool_holder field, which caused the exception in the Serviceability
Agent.

There's no problem with VM, but I think SA should cope with this behavior.
So here's a patch to fix SA:

diff -r fe2c87649981
agent/src/share/classes/sun/jvm/hotspot/oops/ConstantPool.java
--- a/agent/src/share/classes/sun/jvm/hotspot/oops/ConstantPool.java    Thu
Dec 29 15:14:33 2011 -0800
+++ b/agent/src/share/classes/sun/jvm/hotspot/oops/ConstantPool.java    Fri
Dec 30 20:15:10 2011 +0800
@@ -648,7 +648,12 @@
   }

   public void printValueOn(PrintStream tty) {
-    tty.print("ConstantPool for " + getPoolHolder().getName().asString());
+    Oop holder = poolHolder.getValue(this);
+    if (holder instanceof Klass) {
+      tty.print("ConstantPool for " + ((Klass)
holder).getName().asString());
+    } else {
+      tty.print("ConstantPool for partially loaded class");
+    }
   }

   public long getObjectSize() {


By the way, there's another bug in current tip version of SA.
In 6990754 [2], Symbols were moved into native memory, and SA was updated
accordingly. But it missed a case in ConstantPool.iterateFields(OopVisitor
visitor, boolean doVMFields).

A quick-n-dirty fix would be:

diff -r fe2c87649981
agent/src/share/classes/sun/jvm/hotspot/oops/ConstantPool.java
--- a/agent/src/share/classes/sun/jvm/hotspot/oops/ConstantPool.java    Thu
Dec 29 15:14:33 2011 -0800
+++ b/agent/src/share/classes/sun/jvm/hotspot/oops/ConstantPool.java    Fri
Dec 30 20:15:10 2011 +0800
@@ -454,7 +454,7 @@
         case JVM_CONSTANT_Class:
         case JVM_CONSTANT_UnresolvedString:
         case JVM_CONSTANT_Utf8:
-          visitor.doOop(new OopField(new
NamedFieldIdentifier(nameForTag(ctag)), indexOffset(index), true), true);
+          visitor.doInt(new IntField(new
NamedFieldIdentifier(nameForTag(ctag)), indexOffset(index), true), true);
           break;

         case JVM_CONSTANT_Fieldref:

But fixing it like this would make it hard to see the connection between a
ConstantPool and the Symbols it's referencing.
I'm not so sure about what the best fix would look like. Tried adding a
"SymbolField" type, but it felt too heavy. Any suggestions?

Regards,
Kris Mok

[1]: https://gist.github.com/1526668#file_clhsdb_session
[2]: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6990754
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120104/ca0fadc2/attachment.html 

From bob.vandette at oracle.com  Wed Jan  4 16:56:22 2012
From: bob.vandette at oracle.com (bob.vandette at oracle.com)
Date: Thu, 05 Jan 2012 00:56:22 +0000
Subject: hg: hsx/hotspot-main/hotspot: 9 new changesets
Message-ID: <20120105005643.961564788C@hg.openjdk.java.net>

Changeset: 75c0a73eee98
Author:    coleenp
Date:      2011-11-17 12:53 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/75c0a73eee98

7102776: Pack instanceKlass boolean fields into single u1 field
Summary: Reduce class runtime memory usage by packing 4 instanceKlass boolean fields into single u1 field. Save 4-byte for each loaded class.
Reviewed-by: dholmes, bobv, phh, twisti, never, coleenp
Contributed-by: Jiangli Zhou <jiangli.zhou at oracle.com>

! agent/src/share/classes/sun/jvm/hotspot/oops/InstanceKlass.java
! src/share/vm/code/dependencies.cpp
! src/share/vm/oops/instanceKlass.hpp
! src/share/vm/oops/instanceKlassKlass.cpp
! src/share/vm/runtime/vmStructs.cpp

Changeset: da4dd142ea01
Author:    bobv
Date:      2011-11-29 14:44 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/da4dd142ea01

Merge

! src/share/vm/code/dependencies.cpp

Changeset: 52b5d32fbfaf
Author:    coleenp
Date:      2011-12-06 18:28 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/52b5d32fbfaf

7117052: instanceKlass::_init_state can be u1 type
Summary: Change instanceKlass::_init_state field to u1 type.
Reviewed-by: bdelsart, coleenp, dholmes, phh, never
Contributed-by: Jiangli Zhou <jiangli.zhou at oracle.com>

! src/cpu/sparc/vm/c1_LIRAssembler_sparc.cpp
! src/cpu/sparc/vm/c1_Runtime1_sparc.cpp
! src/cpu/sparc/vm/templateTable_sparc.cpp
! src/cpu/x86/vm/c1_LIRAssembler_x86.cpp
! src/cpu/x86/vm/c1_Runtime1_x86.cpp
! src/cpu/x86/vm/templateTable_x86_32.cpp
! src/cpu/x86/vm/templateTable_x86_64.cpp
! src/share/vm/ci/ciInstanceKlass.cpp
! src/share/vm/memory/dump.cpp
! src/share/vm/oops/instanceKlass.cpp
! src/share/vm/oops/instanceKlass.hpp
! src/share/vm/opto/library_call.cpp
! src/share/vm/opto/parseHelper.cpp
! src/share/vm/runtime/vmStructs.cpp

Changeset: eccc4b1f8945
Author:    vladidan
Date:      2011-12-07 16:47 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/eccc4b1f8945

7050298: ARM: SIGSEGV in JNIHandleBlock::allocate_handle
Summary: missing release barrier in Monitor::IUnlock
Reviewed-by: dholmes, dice

! src/share/vm/runtime/mutex.cpp

Changeset: 2685ea97b89f
Author:    jiangli
Date:      2011-12-09 11:29 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/2685ea97b89f

Merge

! src/cpu/sparc/vm/c1_LIRAssembler_sparc.cpp

Changeset: 8fdf463085e1
Author:    jiangli
Date:      2011-12-16 17:33 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/8fdf463085e1

Merge


Changeset: dca455dea3a7
Author:    bdelsart
Date:      2011-12-20 12:33 +0100
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/dca455dea3a7

7116216: StackOverflow GC crash
Summary: GC crash for explicit stack overflow checks after a C2I transition.
Reviewed-by: coleenp, never
Contributed-by: yang02.wang at sap.com, bertrand.delsart at oracle.com

! src/cpu/sparc/vm/stubGenerator_sparc.cpp
! src/cpu/sparc/vm/templateInterpreter_sparc.cpp
! src/cpu/x86/vm/stubGenerator_x86_32.cpp
! src/cpu/x86/vm/stubGenerator_x86_64.cpp
! src/cpu/x86/vm/templateInterpreter_x86_32.cpp
! src/cpu/x86/vm/templateInterpreter_x86_64.cpp
+ test/compiler/7116216/LargeFrame.java
+ test/compiler/7116216/StackOverflow.java

Changeset: cd5d8cafcc84
Author:    jiangli
Date:      2011-12-28 12:15 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/cd5d8cafcc84

7123315: instanceKlass::_static_oop_field_count and instanceKlass::_java_fields_count should be u2 type.
Summary: Change instanceKlass::_static_oop_field_count and instanceKlass::_java_fields_count to u2 type.
Reviewed-by: never, bdelsart, dholmes
Contributed-by: Jiangli Zhou <jiangli.zhou at oracle.com>

! src/share/vm/classfile/classFileParser.cpp
! src/share/vm/classfile/classFileParser.hpp
! src/share/vm/oops/instanceKlass.hpp
! src/share/vm/runtime/vmStructs.cpp

Changeset: 05de27e852c4
Author:    jiangli
Date:      2012-01-04 12:36 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/05de27e852c4

Merge

! src/share/vm/classfile/classFileParser.cpp


From tony.printezis at oracle.com  Thu Jan  5 08:12:02 2012
From: tony.printezis at oracle.com (Tony Printezis)
Date: Thu, 05 Jan 2012 11:12:02 -0500
Subject: JEP 132: More-prompt finalization
In-Reply-To: <4899B594-66EA-444B-9224-9EA660B4E346@kodewerk.com>
References: <20111222230542.48A451084@eggemoggin.niobe.net>
	<4EF3E44F.1060409@oracle.com> <4EF4A8A4.1060502@oracle.com>
	<4EF5C695.4060206@oracle.com> <4EF9F709.4030101@oracle.com>
	<4EFA2869.6070105@oracle.com>
	<712FF280-2C59-47A0-99DE-ECEAD5FCFB15@kodewerk.com>
	<4EFAC2C7.7060508@oracle.com>
	<4E9BFF43-DAC8-480A-AF46-3AADD3FE2ED7@kodewerk.com>
	<4EFB37E0.1090509@oracle.com>
	<4899B594-66EA-444B-9224-9EA660B4E346@kodewerk.com>
Message-ID: <4F05CBD2.6080406@oracle.com>

Kirk,

Trying to clean my inbox post-holidays.... I hope your temperature is 
now back no normal. :-)

On 12/28/2011 11:21 AM, Kirk Pepperdine wrote:
> Ok, I'm reading this with a 38 degree temp so maybe that's why I'm not 
> getting it, my brain is slow?. I've looked at the link Jon provided.. 
> very nice but still leaves me puzzled. Wouldn't simply implementing 
> Unreferenced  be enough to trigger the clean up? I would imagine a 
> broken pipe or some other fault should cause the distributed objects 
> to be dereferenced (i.e. become collectable). At the end of the day, 
> this seems like calling System.gc() in Servlet.destroy().

I'm also not sure I understand your point above about simply 
implementing Unreferenced. Consider the following:

Host 1: has object A
Host 2: has object B that has a remote reference to A

Host 1 does not know anything about what's happening in Host 2. The only 
thing it has been told is that there's a remote reference to object A. 
When Host 2 discovers that B is dead it has to somehow tell Host 1 that 
the remote reference to A does not exist any more. This will allow Host 
1 to collect A as long as it's not otherwise unreachable. If Host 2 
crashes, that message will never be sent. Not sure what happens in that 
case, I assume hosts have to frequently refresh the remote references so 
the ref to A will not be refreshed and eventually be considered dead?

Tony

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120105/dddac8a2/attachment.html 

From kirk at kodewerk.com  Thu Jan  5 08:29:52 2012
From: kirk at kodewerk.com (Kirk Pepperdine)
Date: Thu, 5 Jan 2012 17:29:52 +0100
Subject: JEP 132: More-prompt finalization
In-Reply-To: <4F05CBD2.6080406@oracle.com>
References: <20111222230542.48A451084@eggemoggin.niobe.net>
	<4EF3E44F.1060409@oracle.com> <4EF4A8A4.1060502@oracle.com>
	<4EF5C695.4060206@oracle.com> <4EF9F709.4030101@oracle.com>
	<4EFA2869.6070105@oracle.com>
	<712FF280-2C59-47A0-99DE-ECEAD5FCFB15@kodewerk.com>
	<4EFAC2C7.7060508@oracle.com>
	<4E9BFF43-DAC8-480A-AF46-3AADD3FE2ED7@kodewerk.com>
	<4EFB37E0.1090509@oracle.com>
	<4899B594-66EA-444B-9224-9EA660B4E346@kodewerk.com>
	<4F05CBD2.6080406@oracle.com>
Message-ID: <4DD6120C-4B9B-49F4-A793-80628081F79E@kodewerk.com>

Hi Tony,

Temp is back to normal and I'm much better.

On 2012-01-05, at 5:12 PM, Tony Printezis wrote:

> Kirk,
> 
> Trying to clean my inbox post-holidays.... I hope your temperature is now back no normal. :-)
> 
> On 12/28/2011 11:21 AM, Kirk Pepperdine wrote:
>> 
>> Ok, I'm reading this with a 38 degree temp so maybe that's why I'm not getting it, my brain is slow?. I've looked at the link Jon provided.. very nice but still leaves me puzzled. Wouldn't simply implementing Unreferenced  be enough to trigger the clean up? I would imagine a broken pipe or some other fault should cause the distributed objects to be       dereferenced (i.e. become collectable). At the end of the day, this seems like calling System.gc() in Servlet.destroy().
> 
> I'm also not sure I understand your point above about simply implementing Unreferenced. Consider the following:
> 
> Host 1: has object A
> Host 2: has object B that has a remote reference to A
> 
> Host 1 does not know anything about what's happening in Host 2. The only thing it has been told is that there's a remote reference to object A. When Host 2 discovers that B is dead it has to somehow tell Host 1 that the remote reference to A does not exist any more. This will allow Host 1 to collect A as long as it's not otherwise unreachable. If Host 2 crashes, that message will never be sent. Not sure what happens in that case, I assume hosts have to frequently refresh the remote references so the ref to A will not be refreshed and eventually be considered dead?

If host2 dies, I would assume that the socket connection it had opened with host1 would break. But that is an exceptional case. In the functional case, B will dereference A which should pass a single along to the B proxy running in host 1 that A should be dereferenced.  If B is collected in Host2, B proxy should be released and collected in Host1 via normal dereferencing. I don't see a need to call System.gc().

Kirk

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120105/44f31bf2/attachment.html 

From tony.printezis at oracle.com  Thu Jan  5 08:45:20 2012
From: tony.printezis at oracle.com (Tony Printezis)
Date: Thu, 05 Jan 2012 11:45:20 -0500
Subject: JEP 132: More-prompt finalization
In-Reply-To: <4EFB5583.5000005@oracle.com>
References: <20111222230542.48A451084@eggemoggin.niobe.net>
	<4EF3E44F.1060409@oracle.com> <4EF4A8A4.1060502@oracle.com>
	<4EF5C695.4060206@oracle.com> <4EF9F709.4030101@oracle.com>
	<4EFA2869.6070105@oracle.com>
	<712FF280-2C59-47A0-99DE-ECEAD5FCFB15@kodewerk.com>
	<4EFAC2C7.7060508@oracle.com>
	<32CBF5EE-6AB8-4888-895E-37E495ACCD74@kodewerk.com>
	<4EFB5583.5000005@oracle.com>
Message-ID: <4F05D3A0.1050503@oracle.com>

Dmitry,

On 12/28/2011 12:44 PM, Dmitry Samersoff wrote:
>>> Each of them has it's own workaround (e.g. connection pool manager with refcounting or separate checker thread)
>> I'm not sure that I'd call these work-arounds as they all serve a multitude of purposes.. but, beyond the scope
> Nowdays we have plenty of memory so we can delay socket (an other
> resources) reclamation but save some CPU power.

Well, having lots of memory can allow us to have lots of "room" in the 
heap to postpone GC. However, there are native resources that are 
reclaimed by finalization that are scarce (typically there's a fixed 
number of them, or limited amount of memory we can dedicate to them, 
etc.) so extra memory is just not going to help: there are likely to run 
out before the heap is full enough to cause a GC. Increasing their max 
number is a short-term fix and will only postpone the inevitable.

> It's especially valuable
> if an application have clear visible pick and spare hours.

>   I agree with you - there is no reason to have an API to trigger GC or
> finalization explicitly.

I totally agree with this ....but also see below.

>   I dream about a time when JVM would be able to
> detect low load time and start GC/finalization automatically.

I can't see how this is going to help:

- If you detect that the machine load is low it doesn't also mean that 
there are garbage objects in the heap that need to be reclaimed or 
finalized. So, triggering GC "opportunistically" will be, I'd guess, 
unproductive most of the time.

- In fact, if the machine load is low it means that the application is 
not doing much, therefore maybe there are not many objects to be 
reclaimed / finalized. Which means that this is probably the worst time 
to trigger GC.

- It's not always desirable to do work that might be unproductive when 
the machine load is low. Consider battery powered mobile devices: doing 
potentially unproductive work could drain battery unnecessarily.

>    But today there are a cu's cases that can't be solved without such API.

I agree with you that giving users an API to trigger GCs / finalization 
is not optimal given that they will most likely mis-use it (and they 
do). However, giving an API to library writers to inform the GC how much 
of a native resource is currently being consumed (say: 43 files are open 
out of a max of 100) and let the GC decide what to do is probably a 
better approach (IMHO).

Tony


From tony.printezis at oracle.com  Thu Jan  5 08:52:24 2012
From: tony.printezis at oracle.com (Tony Printezis)
Date: Thu, 05 Jan 2012 11:52:24 -0500
Subject: JEP 132: More-prompt finalization
In-Reply-To: <4DD6120C-4B9B-49F4-A793-80628081F79E@kodewerk.com>
References: <20111222230542.48A451084@eggemoggin.niobe.net>
	<4EF3E44F.1060409@oracle.com> <4EF4A8A4.1060502@oracle.com>
	<4EF5C695.4060206@oracle.com> <4EF9F709.4030101@oracle.com>
	<4EFA2869.6070105@oracle.com>
	<712FF280-2C59-47A0-99DE-ECEAD5FCFB15@kodewerk.com>
	<4EFAC2C7.7060508@oracle.com>
	<4E9BFF43-DAC8-480A-AF46-3AADD3FE2ED7@kodewerk.com>
	<4EFB37E0.1090509@oracle.com>
	<4899B594-66EA-444B-9224-9EA660B4E346@kodewerk.com>
	<4F05CBD2.6080406@oracle.com>
	<4DD6120C-4B9B-49F4-A793-80628081F79E@kodewerk.com>
Message-ID: <4F05D548.6060606@oracle.com>

Kirk,

Consider the case where B becomes garbage in Host 2 but Host 2 is not 
doing much so the next Full GC in Host 2 happens 2 days later. During 
that time A cannot be collected as Host 1 still thinks there's a remote 
reference to A and Host 2 knows that it holds a remote reference from B 
to A but does not know that B is dead.

Tony

On 01/05/2012 11:29 AM, Kirk Pepperdine wrote:
>
>> Host 1: has object A
>> Host 2: has object B that has a remote reference to A
>>
>> Host 1 does not know anything about what's happening in Host 2. The 
>> only thing it has been told is that there's a remote reference to 
>> object A. When Host 2 discovers that B is dead it has to somehow tell 
>> Host 1 that the remote reference to A does not exist any more. This 
>> will allow Host 1 to collect A as long as it's not otherwise 
>> unreachable. If Host 2 crashes, that message will never be sent. Not 
>> sure what happens in that case, I assume hosts have to frequently 
>> refresh the remote references so the ref to A will not be refreshed 
>> and eventually be considered dead?
>
> If host2 dies, I would assume that the socket connection it had opened 
> with host1 would break. But that is an exceptional case. In the 
> functional case, B will dereference A which should pass a single along 
> to the B proxy running in host 1 that A should be dereferenced.  If B 
> is collected in Host2, B proxy should be released and collected in 
> Host1 via normal dereferencing. I don't see a need to call System.gc().
>
> Kirk
>
>

From kirk at kodewerk.com  Thu Jan  5 09:37:41 2012
From: kirk at kodewerk.com (Kirk Pepperdine)
Date: Thu, 5 Jan 2012 18:37:41 +0100
Subject: JEP 132: More-prompt finalization
In-Reply-To: <4F05D3A0.1050503@oracle.com>
References: <20111222230542.48A451084@eggemoggin.niobe.net>
	<4EF3E44F.1060409@oracle.com> <4EF4A8A4.1060502@oracle.com>
	<4EF5C695.4060206@oracle.com> <4EF9F709.4030101@oracle.com>
	<4EFA2869.6070105@oracle.com>
	<712FF280-2C59-47A0-99DE-ECEAD5FCFB15@kodewerk.com>
	<4EFAC2C7.7060508@oracle.com>
	<32CBF5EE-6AB8-4888-895E-37E495ACCD74@kodewerk.com>
	<4EFB5583.5000005@oracle.com> <4F05D3A0.1050503@oracle.com>
Message-ID: <08196B9F-49AC-4DBF-B0C7-76903539C217@kodewerk.com>


On 2012-01-05, at 5:45 PM, Tony Printezis wrote:

> Dmitry,
> 
> On 12/28/2011 12:44 PM, Dmitry Samersoff wrote:
>>>> Each of them has it's own workaround (e.g. connection pool manager with refcounting or separate checker thread)
>>> I'm not sure that I'd call these work-arounds as they all serve a multitude of purposes.. but, beyond the scope
>> Nowdays we have plenty of memory so we can delay socket (an other
>> resources) reclamation but save some CPU power.
> 
> Well, having lots of memory can allow us to have lots of "room" in the heap to postpone GC. However, there are native resources that are reclaimed by finalization that are scarce (typically there's a fixed number of them, or limited amount of memory we can dedicate to them, etc.) so extra memory is just not going to help: there are likely to run out before the heap is full enough to cause a GC. Increasing their max number is a short-term fix and will only postpone the inevitable.

Right, so you're relying on finalization when you should be using a normal close mechanism.
> 
>> It's especially valuable
>> if an application have clear visible pick and spare hours.
> 
>>  I agree with you - there is no reason to have an API to trigger GC or
>> finalization explicitly.
> 
> I totally agree with this ....but also see below.
> 
>>  I dream about a time when JVM would be able to
>> detect low load time and start GC/finalization automatically.
> 
> I can't see how this is going to help:
> 
> - If you detect that the machine load is low it doesn't also mean that there are garbage objects in the heap that need to be reclaimed or finalized. So, triggering GC "opportunistically" will be, I'd guess, unproductive most of the time.
> 
> - In fact, if the machine load is low it means that the application is not doing much, therefore maybe there are not many objects to be reclaimed / finalized. Which means that this is probably the worst time to trigger GC.
> 
> - It's not always desirable to do work that might be unproductive when the machine load is low. Consider battery powered mobile devices: doing potentially unproductive work could drain battery unnecessarily.

Right, this assumes that the app is creating objects when it's busy. We both know that this isn't necessarily true in the low-latency case. That said, triggering a GC when the machine appears to be idle in a low latency application is unlikely to do much more than burn CPU cycles (and battery).

> 
>>   But today there are a cu's cases that can't be solved without such API.
> 
> I agree with you that giving users an API to trigger GCs / finalization is not optimal given that they will most likely mis-use it (and they do). However, giving an API to library writers to inform the GC how much of a native resource is currently being consumed (say: 43 files are open out of a max of 100) and let the GC decide what to do is probably a better approach (IMHO).

I get the feeling that what people are looking for is a destructor.. and in Java the destructor is close(). One has to consider finalization of any resource to be the mechanism of last resort or when all else fails, finalization will catch it (assuming it has a chance to run). Eg, it's the application's responsibility to call close(). If you know enough to call System.gc() (or any other API), you should know enough to call close.

Regards,
Kirk


From vitalyd at gmail.com  Thu Jan  5 09:48:10 2012
From: vitalyd at gmail.com (Vitaly Davidovich)
Date: Thu, 5 Jan 2012 12:48:10 -0500
Subject: JEP 132: More-prompt finalization
In-Reply-To: <4F05D3A0.1050503@oracle.com>
References: <20111222230542.48A451084@eggemoggin.niobe.net>
	<4EF3E44F.1060409@oracle.com> <4EF4A8A4.1060502@oracle.com>
	<4EF5C695.4060206@oracle.com> <4EF9F709.4030101@oracle.com>
	<4EFA2869.6070105@oracle.com>
	<712FF280-2C59-47A0-99DE-ECEAD5FCFB15@kodewerk.com>
	<4EFAC2C7.7060508@oracle.com>
	<32CBF5EE-6AB8-4888-895E-37E495ACCD74@kodewerk.com>
	<4EFB5583.5000005@oracle.com> <4F05D3A0.1050503@oracle.com>
Message-ID: <CAHjP37EQKQNfDg5s2y0RFFSxtdSqgiRnXVfq2QJpP9NyPaMxaw@mail.gmail.com>

Hi Tony,

One case I can see where doing a GC during low activity would help is so
that when activity resumes the GC spaces are as clean (and compacted, if
necessary) as can be and thus throughput and latency can be reduced when
activity resumes.  Presumably you'd run just one such GC per low activity
period.
On Jan 5, 2012 11:46 AM, "Tony Printezis" <tony.printezis at oracle.com> wrote:

> Dmitry,
>
> On 12/28/2011 12:44 PM, Dmitry Samersoff wrote:
>
>> Each of them has it's own workaround (e.g. connection pool manager with
>>>> refcounting or separate checker thread)
>>>>
>>> I'm not sure that I'd call these work-arounds as they all serve a
>>> multitude of purposes.. but, beyond the scope
>>>
>> Nowdays we have plenty of memory so we can delay socket (an other
>> resources) reclamation but save some CPU power.
>>
>
> Well, having lots of memory can allow us to have lots of "room" in the
> heap to postpone GC. However, there are native resources that are reclaimed
> by finalization that are scarce (typically there's a fixed number of them,
> or limited amount of memory we can dedicate to them, etc.) so extra memory
> is just not going to help: there are likely to run out before the heap is
> full enough to cause a GC. Increasing their max number is a short-term fix
> and will only postpone the inevitable.
>
>  It's especially valuable
>> if an application have clear visible pick and spare hours.
>>
>
>   I agree with you - there is no reason to have an API to trigger GC or
>> finalization explicitly.
>>
>
> I totally agree with this ....but also see below.
>
>   I dream about a time when JVM would be able to
>> detect low load time and start GC/finalization automatically.
>>
>
> I can't see how this is going to help:
>
> - If you detect that the machine load is low it doesn't also mean that
> there are garbage objects in the heap that need to be reclaimed or
> finalized. So, triggering GC "opportunistically" will be, I'd guess,
> unproductive most of the time.
>
> - In fact, if the machine load is low it means that the application is not
> doing much, therefore maybe there are not many objects to be reclaimed /
> finalized. Which means that this is probably the worst time to trigger GC.
>
> - It's not always desirable to do work that might be unproductive when the
> machine load is low. Consider battery powered mobile devices: doing
> potentially unproductive work could drain battery unnecessarily.
>
>    But today there are a cu's cases that can't be solved without such API.
>>
>
> I agree with you that giving users an API to trigger GCs / finalization is
> not optimal given that they will most likely mis-use it (and they do).
> However, giving an API to library writers to inform the GC how much of a
> native resource is currently being consumed (say: 43 files are open out of
> a max of 100) and let the GC decide what to do is probably a better
> approach (IMHO).
>
> Tony
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120105/1334e3df/attachment.html 

From david.holmes at oracle.com  Thu Jan  5 18:41:33 2012
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 06 Jan 2012 12:41:33 +1000
Subject: JEP 132: More-prompt finalization
In-Reply-To: <08196B9F-49AC-4DBF-B0C7-76903539C217@kodewerk.com>
References: <20111222230542.48A451084@eggemoggin.niobe.net>
	<4EF3E44F.1060409@oracle.com> <4EF4A8A4.1060502@oracle.com>
	<4EF5C695.4060206@oracle.com> <4EF9F709.4030101@oracle.com>
	<4EFA2869.6070105@oracle.com>
	<712FF280-2C59-47A0-99DE-ECEAD5FCFB15@kodewerk.com>
	<4EFAC2C7.7060508@oracle.com>
	<32CBF5EE-6AB8-4888-895E-37E495ACCD74@kodewerk.com>
	<4EFB5583.5000005@oracle.com> <4F05D3A0.1050503@oracle.com>
	<08196B9F-49AC-4DBF-B0C7-76903539C217@kodewerk.com>
Message-ID: <4F065F5D.6040605@oracle.com>

On 6/01/2012 3:37 AM, Kirk Pepperdine wrote:
> I get the feeling that what people are looking for is a destructor..
> and in Java the destructor is close(). One has to consider finalization
> of any resource to be the mechanism of last resort or when all else
> fails, finalization will catch it (assuming it has a chance to run). Eg,
> it's the application's responsibility to call close(). If you know
> enough to call System.gc() (or any other API), you should know enough to
> call close.

Not to defend finalization in any way but the key difference is that any 
part of the code can call System.gc() or System.runFinalization() 
without needing to know what exactly needs to be finalized. Afterall the 
key thing about GC is it relieves the programmer from having to manage 
object lifetimes, so if you don't know when the object is no longer used 
you don't know when to call close.

David

From john.coomes at oracle.com  Thu Jan  5 20:32:10 2012
From: john.coomes at oracle.com (john.coomes at oracle.com)
Date: Fri, 06 Jan 2012 04:32:10 +0000
Subject: hg: hsx/hotspot-main: Added tag jdk8-b20 for changeset 5a5eaf6374bc
Message-ID: <20120106043210.38CD0478AD@hg.openjdk.java.net>

Changeset: cc771d92284f
Author:    katleman
Date:      2012-01-05 08:42 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/rev/cc771d92284f

Added tag jdk8-b20 for changeset 5a5eaf6374bc

! .hgtags


From john.coomes at oracle.com  Thu Jan  5 20:32:16 2012
From: john.coomes at oracle.com (john.coomes at oracle.com)
Date: Fri, 06 Jan 2012 04:32:16 +0000
Subject: hg: hsx/hotspot-main/corba: Added tag jdk8-b20 for changeset
	51d8b6cb18c0
Message-ID: <20120106043220.3E9DF478AE@hg.openjdk.java.net>

Changeset: f157fc2a71a3
Author:    katleman
Date:      2012-01-05 08:42 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/corba/rev/f157fc2a71a3

Added tag jdk8-b20 for changeset 51d8b6cb18c0

! .hgtags


From john.coomes at oracle.com  Thu Jan  5 20:32:27 2012
From: john.coomes at oracle.com (john.coomes at oracle.com)
Date: Fri, 06 Jan 2012 04:32:27 +0000
Subject: hg: hsx/hotspot-main/jaxp: Added tag jdk8-b20 for changeset
	f052abb8f374
Message-ID: <20120106043227.31275478AF@hg.openjdk.java.net>

Changeset: d41eeadf5c13
Author:    katleman
Date:      2012-01-05 08:42 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jaxp/rev/d41eeadf5c13

Added tag jdk8-b20 for changeset f052abb8f374

! .hgtags


From john.coomes at oracle.com  Thu Jan  5 20:32:34 2012
From: john.coomes at oracle.com (john.coomes at oracle.com)
Date: Fri, 06 Jan 2012 04:32:34 +0000
Subject: hg: hsx/hotspot-main/jaxws: Added tag jdk8-b20 for changeset
	2b2818e3386f
Message-ID: <20120106043234.31631478B0@hg.openjdk.java.net>

Changeset: dc2ee8b87884
Author:    katleman
Date:      2012-01-05 08:42 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jaxws/rev/dc2ee8b87884

Added tag jdk8-b20 for changeset 2b2818e3386f

! .hgtags


From john.coomes at oracle.com  Thu Jan  5 20:33:27 2012
From: john.coomes at oracle.com (john.coomes at oracle.com)
Date: Fri, 06 Jan 2012 04:33:27 +0000
Subject: hg: hsx/hotspot-main/jdk: 9 new changesets
Message-ID: <20120106043526.1C178478B1@hg.openjdk.java.net>

Changeset: 172d70c50c65
Author:    cgruszka
Date:      2011-09-15 13:59 -0400
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/172d70c50c65

7066713: Separate demos from the bundles on Solaris and Linux
Summary: add new license files to demos and samples, new directory for bundling
Reviewed-by: katleman, ohair, billyh

! make/common/Release.gmk
! make/common/shared/Defs-control.gmk

Changeset: eaf967fd25c5
Author:    cgruszka
Date:      2011-10-18 14:21 -0400
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/eaf967fd25c5

7099017: jdk7u2-dev does not build
Summary: changes to skip demo/DEMOS_LICENSE and sample/SAMPLES_LICENSE when building OPENJDK
Reviewed-by: ohair, billyh

! make/common/Release.gmk

Changeset: 39b7f01203c9
Author:    cgruszka
Date:      2011-11-17 16:57 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/39b7f01203c9

Merge


Changeset: b64e7263b4fd
Author:    cgruszka
Date:      2011-11-18 01:03 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/b64e7263b4fd

Merge


Changeset: e98869ff9f1e
Author:    cgruszka
Date:      2011-12-05 12:41 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/e98869ff9f1e

Merge

- test/java/io/FileDescriptor/FileChannelFDTest.java
- test/java/io/etc/FileDescriptorSharing.java

Changeset: ffa36a6a46f5
Author:    cgruszka
Date:      2011-12-16 15:01 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/ffa36a6a46f5

Merge

- make/sun/motif12/reorder-i586
- make/sun/motif12/reorder-sparc
- make/sun/motif12/reorder-sparcv9
- src/share/native/java/util/zip/zlib-1.2.3/ChangeLog
- src/share/native/java/util/zip/zlib-1.2.3/README
- src/share/native/java/util/zip/zlib-1.2.3/compress.c
- src/share/native/java/util/zip/zlib-1.2.3/crc32.h
- src/share/native/java/util/zip/zlib-1.2.3/deflate.c
- src/share/native/java/util/zip/zlib-1.2.3/deflate.h
- src/share/native/java/util/zip/zlib-1.2.3/gzio.c
- src/share/native/java/util/zip/zlib-1.2.3/infback.c
- src/share/native/java/util/zip/zlib-1.2.3/inffast.c
- src/share/native/java/util/zip/zlib-1.2.3/inffast.h
- src/share/native/java/util/zip/zlib-1.2.3/inffixed.h
- src/share/native/java/util/zip/zlib-1.2.3/inflate.c
- src/share/native/java/util/zip/zlib-1.2.3/inflate.h
- src/share/native/java/util/zip/zlib-1.2.3/inftrees.c
- src/share/native/java/util/zip/zlib-1.2.3/inftrees.h
- src/share/native/java/util/zip/zlib-1.2.3/patches/ChangeLog_java
- src/share/native/java/util/zip/zlib-1.2.3/patches/crc32.c.diff
- src/share/native/java/util/zip/zlib-1.2.3/patches/inflate.c.diff
- src/share/native/java/util/zip/zlib-1.2.3/patches/zconf.h.diff
- src/share/native/java/util/zip/zlib-1.2.3/patches/zlib.h.diff
- src/share/native/java/util/zip/zlib-1.2.3/trees.c
- src/share/native/java/util/zip/zlib-1.2.3/trees.h
- src/share/native/java/util/zip/zlib-1.2.3/uncompr.c
- src/share/native/java/util/zip/zlib-1.2.3/zadler32.c
- src/share/native/java/util/zip/zlib-1.2.3/zconf.h
- src/share/native/java/util/zip/zlib-1.2.3/zcrc32.c
- src/share/native/java/util/zip/zlib-1.2.3/zlib.h
- src/share/native/java/util/zip/zlib-1.2.3/zutil.c
- src/share/native/java/util/zip/zlib-1.2.3/zutil.h
- src/solaris/classes/sun/awt/motif/AWTLockAccess.java
- src/solaris/classes/sun/awt/motif/MFontPeer.java
- src/solaris/classes/sun/awt/motif/MToolkit.java
- src/solaris/classes/sun/awt/motif/MToolkitThreadBlockedHandler.java
- src/solaris/classes/sun/awt/motif/MWindowAttributes.java
- src/solaris/classes/sun/awt/motif/X11FontMetrics.java
- src/solaris/native/sun/awt/MouseInfo.c
- src/solaris/native/sun/awt/XDrawingArea.c
- src/solaris/native/sun/awt/XDrawingArea.h
- src/solaris/native/sun/awt/XDrawingAreaP.h
- src/solaris/native/sun/awt/awt_Cursor.h
- src/solaris/native/sun/awt/awt_KeyboardFocusManager.h
- src/solaris/native/sun/awt/awt_MToolkit.c
- src/solaris/native/sun/awt/awt_MToolkit.h
- src/solaris/native/sun/awt/awt_MenuItem.h
- src/solaris/native/sun/awt/awt_PopupMenu.h
- src/solaris/native/sun/awt/awt_TopLevel.h
- src/solaris/native/sun/awt/awt_Window.h
- src/solaris/native/sun/awt/awt_mgrsel.c
- src/solaris/native/sun/awt/awt_mgrsel.h
- src/solaris/native/sun/awt/awt_motif.h
- src/solaris/native/sun/awt/awt_wm.c
- src/solaris/native/sun/awt/awt_wm.h
- src/solaris/native/sun/awt/awt_xembed.h
- src/solaris/native/sun/awt/awt_xembed_server.c
- src/solaris/native/sun/awt/awt_xembed_server.h
- test/java/util/ResourceBundle/Control/ExpirationTest.java
- test/java/util/ResourceBundle/Control/ExpirationTest.sh

Changeset: 5fe1525e6e2c
Author:    cgruszka
Date:      2011-12-23 10:43 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/5fe1525e6e2c

Merge


Changeset: 39e938cd1b82
Author:    cgruszka
Date:      2012-01-03 14:34 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/39e938cd1b82

Merge


Changeset: fc050750f8a0
Author:    katleman
Date:      2012-01-05 08:42 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/fc050750f8a0

Added tag jdk8-b20 for changeset 39e938cd1b82

! .hgtags


From john.coomes at oracle.com  Thu Jan  5 20:36:49 2012
From: john.coomes at oracle.com (john.coomes at oracle.com)
Date: Fri, 06 Jan 2012 04:36:49 +0000
Subject: hg: hsx/hotspot-main/langtools: Added tag jdk8-b20 for changeset
	ffd294128a48
Message-ID: <20120106043656.C11D7478B2@hg.openjdk.java.net>

Changeset: 020819eb56d2
Author:    katleman
Date:      2012-01-05 08:42 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/langtools/rev/020819eb56d2

Added tag jdk8-b20 for changeset ffd294128a48

! .hgtags


From kirk at kodewerk.com  Thu Jan  5 22:24:18 2012
From: kirk at kodewerk.com (Kirk Pepperdine)
Date: Fri, 6 Jan 2012 07:24:18 +0100
Subject: JEP 132: More-prompt finalization
In-Reply-To: <4F065F5D.6040605@oracle.com>
References: <20111222230542.48A451084@eggemoggin.niobe.net>
	<4EF3E44F.1060409@oracle.com> <4EF4A8A4.1060502@oracle.com>
	<4EF5C695.4060206@oracle.com> <4EF9F709.4030101@oracle.com>
	<4EFA2869.6070105@oracle.com>
	<712FF280-2C59-47A0-99DE-ECEAD5FCFB15@kodewerk.com>
	<4EFAC2C7.7060508@oracle.com>
	<32CBF5EE-6AB8-4888-895E-37E495ACCD74@kodewerk.com>
	<4EFB5583.5000005@oracle.com> <4F05D3A0.1050503@oracle.com>
	<08196B9F-49AC-4DBF-B0C7-76903539C217@kodewerk.com>
	<4F065F5D.6040605@oracle.com>
Message-ID: <F2F8A459-BBE5-483C-AC8D-F154BD2E4ADA@kodewerk.com>


On 2012-01-06, at 3:41 AM, David Holmes wrote:

> On 6/01/2012 3:37 AM, Kirk Pepperdine wrote:
>> I get the feeling that what people are looking for is a destructor..
>> and in Java the destructor is close(). One has to consider finalization
>> of any resource to be the mechanism of last resort or when all else
>> fails, finalization will catch it (assuming it has a chance to run). Eg,
>> it's the application's responsibility to call close(). If you know
>> enough to call System.gc() (or any other API), you should know enough to
>> call close.
> 
> Not to defend finalization in any way but the key difference is that any part of the code can call System.gc() or System.runFinalization() without needing to know what exactly needs to be finalized. Afterall the key thing about GC is it relieves the programmer from having to manage object lifetimes, so if you don't know when the object is no longer used you don't know when to call close.

I think this is my point, band aiding over not knowing when to call close with a call to the system saying do something expensive that will most likely have little value isn't a call that I'd like to see. More over, if you dig deeper into these types of problems it seems like there are safer, more viable solutions.

Regards,
Kirk


From tony.printezis at oracle.com  Fri Jan  6 02:26:22 2012
From: tony.printezis at oracle.com (Tony Printezis)
Date: Fri, 06 Jan 2012 05:26:22 -0500
Subject: JEP 132: More-prompt finalization
In-Reply-To: <08196B9F-49AC-4DBF-B0C7-76903539C217@kodewerk.com>
References: <20111222230542.48A451084@eggemoggin.niobe.net>
	<4EF3E44F.1060409@oracle.com> <4EF4A8A4.1060502@oracle.com>
	<4EF5C695.4060206@oracle.com> <4EF9F709.4030101@oracle.com>
	<4EFA2869.6070105@oracle.com>
	<712FF280-2C59-47A0-99DE-ECEAD5FCFB15@kodewerk.com>
	<4EFAC2C7.7060508@oracle.com>
	<32CBF5EE-6AB8-4888-895E-37E495ACCD74@kodewerk.com>
	<4EFB5583.5000005@oracle.com> <4F05D3A0.1050503@oracle.com>
	<08196B9F-49AC-4DBF-B0C7-76903539C217@kodewerk.com>
Message-ID: <4F06CC4E.7020302@oracle.com>

Kirk,

On 01/05/2012 12:37 PM, Kirk Pepperdine wrote:
>> Well, having lots of memory can allow us to have lots of "room" in the heap to postpone GC. However, there are native resources that are reclaimed by finalization that are scarce (typically there's a fixed number of them, or limited amount of memory we can dedicate to them, etc.) so extra memory is just not going to help: there are likely to run out before the heap is full enough to cause a GC. Increasing their max number is a short-term fix and will only postpone the inevitable.
> Right, so you're relying on finalization when you should be using a normal close mechanism.

See below.

>
>>>   I dream about a time when JVM would be able to
>>> detect low load time and start GC/finalization automatically.
>> I can't see how this is going to help:
>>
>> - If you detect that the machine load is low it doesn't also mean that there are garbage objects in the heap that need to be reclaimed or finalized. So, triggering GC "opportunistically" will be, I'd guess, unproductive most of the time.
>>
>> - In fact, if the machine load is low it means that the application is not doing much, therefore maybe there are not many objects to be reclaimed / finalized. Which means that this is probably the worst time to trigger GC.
>>
>> - It's not always desirable to do work that might be unproductive when the machine load is low. Consider battery powered mobile devices: doing potentially unproductive work could drain battery unnecessarily.
> Right, this assumes that the app is creating objects when it's busy. We both know that this isn't necessarily true in the low-latency case.

Sure, but that's the exception. I'd bet that the above is true 95+% of 
the time.

>   That said, triggering a GC when the machine appears to be idle in a low latency application is unlikely to do much more than burn CPU cycles (and battery).

Bingo.

>>>    But today there are a cu's cases that can't be solved without such API.
>> I agree with you that giving users an API to trigger GCs / finalization is not optimal given that they will most likely mis-use it (and they do). However, giving an API to library writers to inform the GC how much of a native resource is currently being consumed (say: 43 files are open out of a max of 100) and let the GC decide what to do is probably a better approach (IMHO).
> I get the feeling that what people are looking for is a destructor.. and in Java the destructor is close().

....inside a finally { } clause. :-)

> One has to consider finalization of any resource to be the mechanism of last resort or when all else fails, finalization will catch it (assuming it has a chance to run). Eg, it's the application's responsibility to call close().

I agree. And I'm sure I can dig up some of my slides from past talks 
that make this exact point: if you know you're done with an object, 
please call close() on it. However, as it's already been pointed out in 
an earlier reply to this thread, it's not always possible to rely on 
close(): reclaiming DirectByteBuffers that are not guaranteed to be 
unreachable could be a security issue.

Tony

> If you know enough to call System.gc() (or any other API), you should know enough to call close.
>
> Regards,
> Kirk
>

From tony.printezis at oracle.com  Fri Jan  6 02:34:08 2012
From: tony.printezis at oracle.com (Tony Printezis)
Date: Fri, 06 Jan 2012 05:34:08 -0500
Subject: JEP 132: More-prompt finalization
In-Reply-To: <4F065F5D.6040605@oracle.com>
References: <20111222230542.48A451084@eggemoggin.niobe.net>
	<4EF3E44F.1060409@oracle.com> <4EF4A8A4.1060502@oracle.com>
	<4EF5C695.4060206@oracle.com> <4EF9F709.4030101@oracle.com>
	<4EFA2869.6070105@oracle.com>
	<712FF280-2C59-47A0-99DE-ECEAD5FCFB15@kodewerk.com>
	<4EFAC2C7.7060508@oracle.com>
	<32CBF5EE-6AB8-4888-895E-37E495ACCD74@kodewerk.com>
	<4EFB5583.5000005@oracle.com> <4F05D3A0.1050503@oracle.com>
	<08196B9F-49AC-4DBF-B0C7-76903539C217@kodewerk.com>
	<4F065F5D.6040605@oracle.com>
Message-ID: <4F06CE20.10809@oracle.com>



On 01/05/2012 09:41 PM, David Holmes wrote:
> On 6/01/2012 3:37 AM, Kirk Pepperdine wrote:
>> I get the feeling that what people are looking for is a destructor..
>> and in Java the destructor is close(). One has to consider finalization
>> of any resource to be the mechanism of last resort or when all else
>> fails, finalization will catch it (assuming it has a chance to run). Eg,
>> it's the application's responsibility to call close(). If you know
>> enough to call System.gc() (or any other API), you should know enough to
>> call close.
>
> Not to defend finalization 

Good. :-)

> in any way but the key difference is that any part of the code can 
> call System.gc() or System.runFinalization() without needing to know 
> what exactly needs to be finalized. 

If each library could indicate to the GC whether the resource it manages 
is running low or not, using the API I mentioned, the GC could do the 
above automatically, behind the scenes, without the user having to do 
anything else.

Tony

> Afterall the key thing about GC is it relieves the programmer from 
> having to manage object lifetimes, so if you don't know when the 
> object is no longer used you don't know when to call close.
>
> David

From tony.printezis at oracle.com  Fri Jan  6 02:34:34 2012
From: tony.printezis at oracle.com (Tony Printezis)
Date: Fri, 06 Jan 2012 05:34:34 -0500
Subject: JEP 132: More-prompt finalization
In-Reply-To: <CAHjP37EQKQNfDg5s2y0RFFSxtdSqgiRnXVfq2QJpP9NyPaMxaw@mail.gmail.com>
References: <20111222230542.48A451084@eggemoggin.niobe.net>
	<4EF3E44F.1060409@oracle.com> <4EF4A8A4.1060502@oracle.com>
	<4EF5C695.4060206@oracle.com> <4EF9F709.4030101@oracle.com>
	<4EFA2869.6070105@oracle.com>
	<712FF280-2C59-47A0-99DE-ECEAD5FCFB15@kodewerk.com>
	<4EFAC2C7.7060508@oracle.com>
	<32CBF5EE-6AB8-4888-895E-37E495ACCD74@kodewerk.com>
	<4EFB5583.5000005@oracle.com> <4F05D3A0.1050503@oracle.com>
	<CAHjP37EQKQNfDg5s2y0RFFSxtdSqgiRnXVfq2QJpP9NyPaMxaw@mail.gmail.com>
Message-ID: <4F06CE3A.2040101@oracle.com>

Vitaly,

Sure, but if the GC detects that the load is low it doesn't know whether 
the load will remain low for 5 ms or 5 hours (and it's impossible to 
know, maybe not even the application knows). I can already imagine the 
bug reports: a spike suddenly happened in the market and the JVM was 
"locked up" for several seconds!!!

Tony

On 01/05/2012 12:48 PM, Vitaly Davidovich wrote:
>
> Hi Tony,
>
> One case I can see where doing a GC during low activity would help is 
> so that when activity resumes the GC spaces are as clean (and 
> compacted, if necessary) as can be and thus throughput and latency can 
> be reduced when activity resumes.  Presumably you'd run just one such 
> GC per low activity period.
>
> On Jan 5, 2012 11:46 AM, "Tony Printezis" <tony.printezis at oracle.com 
> <mailto:tony.printezis at oracle.com>> wrote:
>
>     Dmitry,
>
>     On 12/28/2011 12:44 PM, Dmitry Samersoff wrote:
>
>                 Each of them has it's own workaround (e.g. connection
>                 pool manager with refcounting or separate checker thread)
>
>             I'm not sure that I'd call these work-arounds as they all
>             serve a multitude of purposes.. but, beyond the scope
>
>         Nowdays we have plenty of memory so we can delay socket (an other
>         resources) reclamation but save some CPU power.
>
>
>     Well, having lots of memory can allow us to have lots of "room" in
>     the heap to postpone GC. However, there are native resources that
>     are reclaimed by finalization that are scarce (typically there's a
>     fixed number of them, or limited amount of memory we can dedicate
>     to them, etc.) so extra memory is just not going to help: there
>     are likely to run out before the heap is full enough to cause a
>     GC. Increasing their max number is a short-term fix and will only
>     postpone the inevitable.
>
>         It's especially valuable
>         if an application have clear visible pick and spare hours.
>
>
>          I agree with you - there is no reason to have an API to
>         trigger GC or
>         finalization explicitly.
>
>
>     I totally agree with this ....but also see below.
>
>          I dream about a time when JVM would be able to
>         detect low load time and start GC/finalization automatically.
>
>
>     I can't see how this is going to help:
>
>     - If you detect that the machine load is low it doesn't also mean
>     that there are garbage objects in the heap that need to be
>     reclaimed or finalized. So, triggering GC "opportunistically" will
>     be, I'd guess, unproductive most of the time.
>
>     - In fact, if the machine load is low it means that the
>     application is not doing much, therefore maybe there are not many
>     objects to be reclaimed / finalized. Which means that this is
>     probably the worst time to trigger GC.
>
>     - It's not always desirable to do work that might be unproductive
>     when the machine load is low. Consider battery powered mobile
>     devices: doing potentially unproductive work could drain battery
>     unnecessarily.
>
>           But today there are a cu's cases that can't be solved
>         without such API.
>
>
>     I agree with you that giving users an API to trigger GCs /
>     finalization is not optimal given that they will most likely
>     mis-use it (and they do). However, giving an API to library
>     writers to inform the GC how much of a native resource is
>     currently being consumed (say: 43 files are open out of a max of
>     100) and let the GC decide what to do is probably a better
>     approach (IMHO).
>
>     Tony
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120106/a43c670e/attachment.html 

From david.holmes at oracle.com  Fri Jan  6 03:14:47 2012
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 06 Jan 2012 21:14:47 +1000
Subject: JEP 132: More-prompt finalization
In-Reply-To: <4F06CE20.10809@oracle.com>
References: <20111222230542.48A451084@eggemoggin.niobe.net>
	<4EF3E44F.1060409@oracle.com> <4EF4A8A4.1060502@oracle.com>
	<4EF5C695.4060206@oracle.com> <4EF9F709.4030101@oracle.com>
	<4EFA2869.6070105@oracle.com>
	<712FF280-2C59-47A0-99DE-ECEAD5FCFB15@kodewerk.com>
	<4EFAC2C7.7060508@oracle.com>
	<32CBF5EE-6AB8-4888-895E-37E495ACCD74@kodewerk.com>
	<4EFB5583.5000005@oracle.com> <4F05D3A0.1050503@oracle.com>
	<08196B9F-49AC-4DBF-B0C7-76903539C217@kodewerk.com>
	<4F065F5D.6040605@oracle.com> <4F06CE20.10809@oracle.com>
Message-ID: <4F06D7A7.6010603@oracle.com>

Hi Tony,

On 6/01/2012 8:34 PM, Tony Printezis wrote:
> If each library could indicate to the GC whether the resource it manages
> is running low or not, using the API I mentioned, the GC could do the
> above automatically, behind the scenes, without the user having to do
> anything else.

I'm not sure a library writer has the necessary information to do this. 
Seems to me that how an application uses a particular type determines 
the scarcity of the resource. I can imagine something like:

void setFinalizationLimit(Class<?> cls, int limit)

so that GC runs finalization once a "reference count" reaches "limit". 
That limits the frequency of finalization, but the actual finalization 
cost may still be unacceptably high.

Cheers,
David

> Tony
>
>> Afterall the key thing about GC is it relieves the programmer from
>> having to manage object lifetimes, so if you don't know when the
>> object is no longer used you don't know when to call close.
>>
>> David

From kirk at kodewerk.com  Fri Jan  6 05:57:51 2012
From: kirk at kodewerk.com (Kirk Pepperdine)
Date: Fri, 6 Jan 2012 14:57:51 +0100
Subject: JEP 132: More-prompt finalization
In-Reply-To: <4F06D7A7.6010603@oracle.com>
References: <20111222230542.48A451084@eggemoggin.niobe.net>
	<4EF3E44F.1060409@oracle.com> <4EF4A8A4.1060502@oracle.com>
	<4EF5C695.4060206@oracle.com> <4EF9F709.4030101@oracle.com>
	<4EFA2869.6070105@oracle.com>
	<712FF280-2C59-47A0-99DE-ECEAD5FCFB15@kodewerk.com>
	<4EFAC2C7.7060508@oracle.com>
	<32CBF5EE-6AB8-4888-895E-37E495ACCD74@kodewerk.com>
	<4EFB5583.5000005@oracle.com> <4F05D3A0.1050503@oracle.com>
	<08196B9F-49AC-4DBF-B0C7-76903539C217@kodewerk.com>
	<4F065F5D.6040605@oracle.com> <4F06CE20.10809@oracle.com>
	<4F06D7A7.6010603@oracle.com>
Message-ID: <3EF8333A-5812-41DD-8180-4AE70F581161@kodewerk.com>

Tony, David,

I really feel that this problem is being solved at the wrong level. The JVM lacks application semantics to know when to do the "right" thing. So, IMHO,  this should be managed by the application. For the same reason, there are other things that the application shouldn't touch like telling the JVM it's time to run a collection.

That said, one thing the JVM might know about is how many file descriptors it's allowed and it could trigger an attempt to recover them (i.e. run finalization) once they start running low.. just as the JVM manages memory by recovering it when it runs low. That said, I did run a bench where we opened 1,000,000,000 sockets on a single VM. That is just about as many sockets as can be opened on a machine without recompiling the kernel to configure a bigger file descriptor bitmap. I'm quite happy that the JVM wasn't fighting me as I was trying to open all these sockets. While this bench was a bit extreme, it's clear that WebSocket gateways *will* stress file descriptor.

Not sure about direct buffers though I do feel they should implement finalization as a fallback position.

Cheers,
Kirk

On 2012-01-06, at 12:14 PM, David Holmes wrote:

> Hi Tony,
> 
> On 6/01/2012 8:34 PM, Tony Printezis wrote:
>> If each library could indicate to the GC whether the resource it manages
>> is running low or not, using the API I mentioned, the GC could do the
>> above automatically, behind the scenes, without the user having to do
>> anything else.
> 
> I'm not sure a library writer has the necessary information to do this. Seems to me that how an application uses a particular type determines the scarcity of the resource. I can imagine something like:
> 
> void setFinalizationLimit(Class<?> cls, int limit)
> 
> so that GC runs finalization once a "reference count" reaches "limit". That limits the frequency of finalization, but the actual finalization cost may still be unacceptably high.
> 
> Cheers,
> David
> 
>> Tony
>> 
>>> Afterall the key thing about GC is it relieves the programmer from
>>> having to manage object lifetimes, so if you don't know when the
>>> object is no longer used you don't know when to call close.
>>> 
>>> David


From Alan.Bateman at oracle.com  Fri Jan  6 06:58:27 2012
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Fri, 06 Jan 2012 14:58:27 +0000
Subject: JEP 132: More-prompt finalization
In-Reply-To: <4F06CC4E.7020302@oracle.com>
References: <20111222230542.48A451084@eggemoggin.niobe.net>
	<4EF3E44F.1060409@oracle.com> <4EF4A8A4.1060502@oracle.com>
	<4EF5C695.4060206@oracle.com> <4EF9F709.4030101@oracle.com>
	<4EFA2869.6070105@oracle.com>
	<712FF280-2C59-47A0-99DE-ECEAD5FCFB15@kodewerk.com>
	<4EFAC2C7.7060508@oracle.com>
	<32CBF5EE-6AB8-4888-895E-37E495ACCD74@kodewerk.com>
	<4EFB5583.5000005@oracle.com> <4F05D3A0.1050503@oracle.com>
	<08196B9F-49AC-4DBF-B0C7-76903539C217@kodewerk.com>
	<4F06CC4E.7020302@oracle.com>
Message-ID: <4F070C13.3090407@oracle.com>

On 06/01/2012 10:26, Tony Printezis wrote:
>
> I agree. And I'm sure I can dig up some of my slides from past talks 
> that make this exact point: if you know you're done with an object, 
> please call close() on it. However, as it's already been pointed out 
> in an earlier reply to this thread, it's not always possible to rely 
> on close(): reclaiming DirectByteBuffers that are not guaranteed to be 
> unreachable could be a security issue.
Just to mention that in Java SE then all APIs for sockets and files 
implement Closeable so they have a close method, can be used with the 
try-with-resources construct where appropriate. While some of the older 
APIs (FileInputStream, FileOutputStream, the default SocketImpls) have 
finalizers, the newer APIs do not and so the resources must be 
explicitly closed.

As you mention, direct and mapped buffers can't be explicitly reclaimed. 
We don't have a good solution to that problem which is why System.gc is 
invoked when limits are reached. It would be nice to re-visit this some 
day. Off-hand I think this is the only other place in the JDK, aside 
from RMI DGC, where System.gc is invoked explicitly.

-Alan.


From tony.printezis at oracle.com  Fri Jan  6 08:07:39 2012
From: tony.printezis at oracle.com (Tony Printezis)
Date: Fri, 06 Jan 2012 11:07:39 -0500
Subject: JEP 132: More-prompt finalization
In-Reply-To: <4F06D7A7.6010603@oracle.com>
References: <20111222230542.48A451084@eggemoggin.niobe.net>
	<4EF3E44F.1060409@oracle.com> <4EF4A8A4.1060502@oracle.com>
	<4EF5C695.4060206@oracle.com> <4EF9F709.4030101@oracle.com>
	<4EFA2869.6070105@oracle.com>
	<712FF280-2C59-47A0-99DE-ECEAD5FCFB15@kodewerk.com>
	<4EFAC2C7.7060508@oracle.com>
	<32CBF5EE-6AB8-4888-895E-37E495ACCD74@kodewerk.com>
	<4EFB5583.5000005@oracle.com> <4F05D3A0.1050503@oracle.com>
	<08196B9F-49AC-4DBF-B0C7-76903539C217@kodewerk.com>
	<4F065F5D.6040605@oracle.com> <4F06CE20.10809@oracle.com>
	<4F06D7A7.6010603@oracle.com>
Message-ID: <4F071C4B.8010002@oracle.com>

David,

On 01/06/2012 06:14 AM, David Holmes wrote:
> Hi Tony,
>
> On 6/01/2012 8:34 PM, Tony Printezis wrote:
>> If each library could indicate to the GC whether the resource it manages
>> is running low or not, using the API I mentioned, the GC could do the
>> above automatically, behind the scenes, without the user having to do
>> anything else.
>
> I'm not sure a library writer has the necessary information to do 
> this. Seems to me that how an application uses a particular type 
> determines the scarcity of the resource. 

I will be very surprised if the majority of application developers will 
be willing to measure at what rate the application consumes certain 
resources and update said measure when the application changes, load 
increases, OS changes, hardware changes, etc. And, sure, I think there 
will be many cases where the library writer has a good idea / can find 
out the max number of instances of a particular resource (file 
descriptors, sockets, etc.).

Tony

> I can imagine something like:
>
> void setFinalizationLimit(Class<?> cls, int limit)
>
> so that GC runs finalization once a "reference count" reaches "limit". 
> That limits the frequency of finalization, but the actual finalization 
> cost may still be unacceptably high.
>
> Cheers,
> David
>
>> Tony
>>
>>> Afterall the key thing about GC is it relieves the programmer from
>>> having to manage object lifetimes, so if you don't know when the
>>> object is no longer used you don't know when to call close.
>>>
>>> David

From tony.printezis at oracle.com  Fri Jan  6 08:16:21 2012
From: tony.printezis at oracle.com (Tony Printezis)
Date: Fri, 06 Jan 2012 11:16:21 -0500
Subject: JEP 132: More-prompt finalization
In-Reply-To: <3EF8333A-5812-41DD-8180-4AE70F581161@kodewerk.com>
References: <20111222230542.48A451084@eggemoggin.niobe.net>
	<4EF3E44F.1060409@oracle.com> <4EF4A8A4.1060502@oracle.com>
	<4EF5C695.4060206@oracle.com> <4EF9F709.4030101@oracle.com>
	<4EFA2869.6070105@oracle.com>
	<712FF280-2C59-47A0-99DE-ECEAD5FCFB15@kodewerk.com>
	<4EFAC2C7.7060508@oracle.com>
	<32CBF5EE-6AB8-4888-895E-37E495ACCD74@kodewerk.com>
	<4EFB5583.5000005@oracle.com> <4F05D3A0.1050503@oracle.com>
	<08196B9F-49AC-4DBF-B0C7-76903539C217@kodewerk.com>
	<4F065F5D.6040605@oracle.com> <4F06CE20.10809@oracle.com>
	<4F06D7A7.6010603@oracle.com>
	<3EF8333A-5812-41DD-8180-4AE70F581161@kodewerk.com>
Message-ID: <4F071E55.9050203@oracle.com>

Kirk,

Inline.

On 01/06/2012 08:57 AM, Kirk Pepperdine wrote:
> Tony, David,
>
> I really feel that this problem is being solved at the wrong level. The JVM lacks application semantics to know when to do the "right" thing.

I absolutely agree.

> So, IMHO,  this should be managed by the application.

I absolutely disagree. :-)

First, let's define some terminology to make sure we're all on the same 
page:

JVM : HotSpot
library : the Java code that manages certain native resource (e.g., 
classes in the java.io package) - I do not consider this part of the JVM
application : what the user writes which runs on top of HotSpot and uses 
java.io.File's.

> For the same reason, there are other things that the application shouldn't touch like telling the JVM it's time to run a collection.

Amen to that. :-)

> That said, one thing the JVM might know about is how many file descriptors it's allowed and it could trigger an attempt to recover them (i.e. run finalization) once they start running low..

Indeed. But the JVM (as defined above) does not know anything about file 
descriptors. It's the library, in this case classes in java.io, that 
does. However, I don't think said library should also be calling 
System.gc() either. It should be providing information to the GC, via 
the API I have been suggesting, on how much of a certain resource we 
have and the GC should be making informed decisions on whether it when 
it should trigger a cycle.

Tony

>   just as the JVM manages memory by recovering it when it runs low. That said, I did run a bench where we opened 1,000,000,000 sockets on a single VM. That is just about as many sockets as can be opened on a machine without recompiling the kernel to configure a bigger file descriptor bitmap. I'm quite happy that the JVM wasn't fighting me as I was trying to open all these sockets. While this bench was a bit extreme, it's clear that WebSocket gateways *will* stress file descriptor.
>
> Not sure about direct buffers though I do feel they should implement finalization as a fallback position.
>
> Cheers,
> Kirk
>
> On 2012-01-06, at 12:14 PM, David Holmes wrote:
>
>> Hi Tony,
>>
>> On 6/01/2012 8:34 PM, Tony Printezis wrote:
>>> If each library could indicate to the GC whether the resource it manages
>>> is running low or not, using the API I mentioned, the GC could do the
>>> above automatically, behind the scenes, without the user having to do
>>> anything else.
>> I'm not sure a library writer has the necessary information to do this. Seems to me that how an application uses a particular type determines the scarcity of the resource. I can imagine something like:
>>
>> void setFinalizationLimit(Class<?>  cls, int limit)
>>
>> so that GC runs finalization once a "reference count" reaches "limit". That limits the frequency of finalization, but the actual finalization cost may still be unacceptably high.
>>
>> Cheers,
>> David
>>
>>> Tony
>>>
>>>> Afterall the key thing about GC is it relieves the programmer from
>>>> having to manage object lifetimes, so if you don't know when the
>>>> object is no longer used you don't know when to call close.
>>>>
>>>> David

From vitalyd at gmail.com  Fri Jan  6 08:32:07 2012
From: vitalyd at gmail.com (Vitaly Davidovich)
Date: Fri, 6 Jan 2012 11:32:07 -0500
Subject: JEP 132: More-prompt finalization
In-Reply-To: <4F071E55.9050203@oracle.com>
References: <20111222230542.48A451084@eggemoggin.niobe.net>
	<4EF3E44F.1060409@oracle.com> <4EF4A8A4.1060502@oracle.com>
	<4EF5C695.4060206@oracle.com> <4EF9F709.4030101@oracle.com>
	<4EFA2869.6070105@oracle.com>
	<712FF280-2C59-47A0-99DE-ECEAD5FCFB15@kodewerk.com>
	<4EFAC2C7.7060508@oracle.com>
	<32CBF5EE-6AB8-4888-895E-37E495ACCD74@kodewerk.com>
	<4EFB5583.5000005@oracle.com> <4F05D3A0.1050503@oracle.com>
	<08196B9F-49AC-4DBF-B0C7-76903539C217@kodewerk.com>
	<4F065F5D.6040605@oracle.com> <4F06CE20.10809@oracle.com>
	<4F06D7A7.6010603@oracle.com>
	<3EF8333A-5812-41DD-8180-4AE70F581161@kodewerk.com>
	<4F071E55.9050203@oracle.com>
Message-ID: <CAHjP37GVPWNZAdtM4xPAnt_L-y1R55x-yLZY3B1YFNGkYnf-rg@mail.gmail.com>

Tony,

Interestingly the .NET CLR has an API  to allow apps to tell the runtime(gc
specifically) to be more aggressive -
System.GC.AddMemoryPressure/RemoveMemoryPressure.  The idea being that
you'd call this when a managed object allocates native memory and thus adds
overall pressure.  This hint is then used by the gc.  What are your
thoughts on that? Granted this is more amenable for mem management and not
other scarce resources.
On Jan 6, 2012 11:17 AM, "Tony Printezis" <tony.printezis at oracle.com> wrote:

> Kirk,
>
> Inline.
>
> On 01/06/2012 08:57 AM, Kirk Pepperdine wrote:
>
>> Tony, David,
>>
>> I really feel that this problem is being solved at the wrong level. The
>> JVM lacks application semantics to know when to do the "right" thing.
>>
>
> I absolutely agree.
>
>  So, IMHO,  this should be managed by the application.
>>
>
> I absolutely disagree. :-)
>
> First, let's define some terminology to make sure we're all on the same
> page:
>
> JVM : HotSpot
> library : the Java code that manages certain native resource (e.g.,
> classes in the java.io package) - I do not consider this part of the JVM
> application : what the user writes which runs on top of HotSpot and uses
> java.io.File's.
>
>  For the same reason, there are other things that the application
>> shouldn't touch like telling the JVM it's time to run a collection.
>>
>
> Amen to that. :-)
>
>  That said, one thing the JVM might know about is how many file
>> descriptors it's allowed and it could trigger an attempt to recover them
>> (i.e. run finalization) once they start running low..
>>
>
> Indeed. But the JVM (as defined above) does not know anything about file
> descriptors. It's the library, in this case classes in java.io, that
> does. However, I don't think said library should also be calling
> System.gc() either. It should be providing information to the GC, via the
> API I have been suggesting, on how much of a certain resource we have and
> the GC should be making informed decisions on whether it when it should
> trigger a cycle.
>
> Tony
>
>   just as the JVM manages memory by recovering it when it runs low. That
>> said, I did run a bench where we opened 1,000,000,000 sockets on a single
>> VM. That is just about as many sockets as can be opened on a machine
>> without recompiling the kernel to configure a bigger file descriptor
>> bitmap. I'm quite happy that the JVM wasn't fighting me as I was trying to
>> open all these sockets. While this bench was a bit extreme, it's clear that
>> WebSocket gateways *will* stress file descriptor.
>>
>> Not sure about direct buffers though I do feel they should implement
>> finalization as a fallback position.
>>
>> Cheers,
>> Kirk
>>
>> On 2012-01-06, at 12:14 PM, David Holmes wrote:
>>
>>  Hi Tony,
>>>
>>> On 6/01/2012 8:34 PM, Tony Printezis wrote:
>>>
>>>> If each library could indicate to the GC whether the resource it manages
>>>> is running low or not, using the API I mentioned, the GC could do the
>>>> above automatically, behind the scenes, without the user having to do
>>>> anything else.
>>>>
>>> I'm not sure a library writer has the necessary information to do this.
>>> Seems to me that how an application uses a particular type determines the
>>> scarcity of the resource. I can imagine something like:
>>>
>>> void setFinalizationLimit(Class<?>  cls, int limit)
>>>
>>> so that GC runs finalization once a "reference count" reaches "limit".
>>> That limits the frequency of finalization, but the actual finalization cost
>>> may still be unacceptably high.
>>>
>>> Cheers,
>>> David
>>>
>>>  Tony
>>>>
>>>>  Afterall the key thing about GC is it relieves the programmer from
>>>>> having to manage object lifetimes, so if you don't know when the
>>>>> object is no longer used you don't know when to call close.
>>>>>
>>>>> David
>>>>>
>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120106/57cc47ed/attachment.html 

From tony.printezis at oracle.com  Fri Jan  6 08:37:14 2012
From: tony.printezis at oracle.com (Tony Printezis)
Date: Fri, 06 Jan 2012 11:37:14 -0500
Subject: JEP 132: More-prompt finalization
In-Reply-To: <CAHjP37GVPWNZAdtM4xPAnt_L-y1R55x-yLZY3B1YFNGkYnf-rg@mail.gmail.com>
References: <20111222230542.48A451084@eggemoggin.niobe.net>
	<4EF3E44F.1060409@oracle.com> <4EF4A8A4.1060502@oracle.com>
	<4EF5C695.4060206@oracle.com> <4EF9F709.4030101@oracle.com>
	<4EFA2869.6070105@oracle.com>
	<712FF280-2C59-47A0-99DE-ECEAD5FCFB15@kodewerk.com>
	<4EFAC2C7.7060508@oracle.com>
	<32CBF5EE-6AB8-4888-895E-37E495ACCD74@kodewerk.com>
	<4EFB5583.5000005@oracle.com> <4F05D3A0.1050503@oracle.com>
	<08196B9F-49AC-4DBF-B0C7-76903539C217@kodewerk.com>
	<4F065F5D.6040605@oracle.com> <4F06CE20.10809@oracle.com>
	<4F06D7A7.6010603@oracle.com>
	<3EF8333A-5812-41DD-8180-4AE70F581161@kodewerk.com>
	<4F071E55.9050203@oracle.com>
	<CAHjP37GVPWNZAdtM4xPAnt_L-y1R55x-yLZY3B1YFNGkYnf-rg@mail.gmail.com>
Message-ID: <4F07233A.5010001@oracle.com>

Does AddMemoryPressure() take any parameters? Creating a new socket (out 
of a max of say 1,000,000) should have different weight to, say, 
creating a new file (out of a of say 50).

Tony

On 01/06/2012 11:32 AM, Vitaly Davidovich wrote:
>
> Tony,
>
> Interestingly the .NET CLR has an API  to allow apps to tell the 
> runtime(gc specifically) to be more aggressive - 
> System.GC.AddMemoryPressure/RemoveMemoryPressure.  The idea being that 
> you'd call this when a managed object allocates native memory and thus 
> adds overall pressure.  This hint is then used by the gc.  What are 
> your thoughts on that? Granted this is more amenable for mem 
> management and not other scarce resources.
>
> On Jan 6, 2012 11:17 AM, "Tony Printezis" <tony.printezis at oracle.com 
> <mailto:tony.printezis at oracle.com>> wrote:
>
>     Kirk,
>
>     Inline.
>
>     On 01/06/2012 08:57 AM, Kirk Pepperdine wrote:
>
>         Tony, David,
>
>         I really feel that this problem is being solved at the wrong
>         level. The JVM lacks application semantics to know when to do
>         the "right" thing.
>
>
>     I absolutely agree.
>
>         So, IMHO,  this should be managed by the application.
>
>
>     I absolutely disagree. :-)
>
>     First, let's define some terminology to make sure we're all on the
>     same page:
>
>     JVM : HotSpot
>     library : the Java code that manages certain native resource
>     (e.g., classes in the java.io <http://java.io> package) - I do not
>     consider this part of the JVM
>     application : what the user writes which runs on top of HotSpot
>     and uses java.io.File's.
>
>         For the same reason, there are other things that the
>         application shouldn't touch like telling the JVM it's time to
>         run a collection.
>
>
>     Amen to that. :-)
>
>         That said, one thing the JVM might know about is how many file
>         descriptors it's allowed and it could trigger an attempt to
>         recover them (i.e. run finalization) once they start running low..
>
>
>     Indeed. But the JVM (as defined above) does not know anything
>     about file descriptors. It's the library, in this case classes in
>     java.io <http://java.io>, that does. However, I don't think said
>     library should also be calling System.gc() either. It should be
>     providing information to the GC, via the API I have been
>     suggesting, on how much of a certain resource we have and the GC
>     should be making informed decisions on whether it when it should
>     trigger a cycle.
>
>     Tony
>
>          just as the JVM manages memory by recovering it when it runs
>         low. That said, I did run a bench where we opened
>         1,000,000,000 sockets on a single VM. That is just about as
>         many sockets as can be opened on a machine without recompiling
>         the kernel to configure a bigger file descriptor bitmap. I'm
>         quite happy that the JVM wasn't fighting me as I was trying to
>         open all these sockets. While this bench was a bit extreme,
>         it's clear that WebSocket gateways *will* stress file descriptor.
>
>         Not sure about direct buffers though I do feel they should
>         implement finalization as a fallback position.
>
>         Cheers,
>         Kirk
>
>         On 2012-01-06, at 12:14 PM, David Holmes wrote:
>
>             Hi Tony,
>
>             On 6/01/2012 8:34 PM, Tony Printezis wrote:
>
>                 If each library could indicate to the GC whether the
>                 resource it manages
>                 is running low or not, using the API I mentioned, the
>                 GC could do the
>                 above automatically, behind the scenes, without the
>                 user having to do
>                 anything else.
>
>             I'm not sure a library writer has the necessary
>             information to do this. Seems to me that how an
>             application uses a particular type determines the scarcity
>             of the resource. I can imagine something like:
>
>             void setFinalizationLimit(Class<?>  cls, int limit)
>
>             so that GC runs finalization once a "reference count"
>             reaches "limit". That limits the frequency of
>             finalization, but the actual finalization cost may still
>             be unacceptably high.
>
>             Cheers,
>             David
>
>                 Tony
>
>                     Afterall the key thing about GC is it relieves the
>                     programmer from
>                     having to manage object lifetimes, so if you don't
>                     know when the
>                     object is no longer used you don't know when to
>                     call close.
>
>                     David
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120106/5111f34c/attachment-0001.html 

From kirk at kodewerk.com  Fri Jan  6 08:54:37 2012
From: kirk at kodewerk.com (Kirk Pepperdine)
Date: Fri, 6 Jan 2012 17:54:37 +0100
Subject: JEP 132: More-prompt finalization
In-Reply-To: <CAHjP37GVPWNZAdtM4xPAnt_L-y1R55x-yLZY3B1YFNGkYnf-rg@mail.gmail.com>
References: <20111222230542.48A451084@eggemoggin.niobe.net>
	<4EF3E44F.1060409@oracle.com> <4EF4A8A4.1060502@oracle.com>
	<4EF5C695.4060206@oracle.com> <4EF9F709.4030101@oracle.com>
	<4EFA2869.6070105@oracle.com>
	<712FF280-2C59-47A0-99DE-ECEAD5FCFB15@kodewerk.com>
	<4EFAC2C7.7060508@oracle.com>
	<32CBF5EE-6AB8-4888-895E-37E495ACCD74@kodewerk.com>
	<4EFB5583.5000005@oracle.com> <4F05D3A0.1050503@oracle.com>
	<08196B9F-49AC-4DBF-B0C7-76903539C217@kodewerk.com>
	<4F065F5D.6040605@oracle.com> <4F06CE20.10809@oracle.com>
	<4F06D7A7.6010603@oracle.com>
	<3EF8333A-5812-41DD-8180-4AE70F581161@kodewerk.com>
	<4F071E55.9050203@oracle.com>
	<CAHjP37GVPWNZAdtM4xPAnt_L-y1R55x-yLZY3B1YFNGkYnf-rg@mail.gmail.com>
Message-ID: <A5D45A8F-D4A6-41FD-A23A-B4B01BD332EC@kodewerk.com>

when it comes to performance, I'm not sure that the CLR is a good role model ;-)

Regards,
Kirk

On 2012-01-06, at 5:32 PM, Vitaly Davidovich wrote:

> Tony,
> 
> Interestingly the .NET CLR has an API  to allow apps to tell the runtime(gc specifically) to be more aggressive - System.GC.AddMemoryPressure/RemoveMemoryPressure.  The idea being that you'd call this when a managed object allocates native memory and thus adds overall pressure.  This hint is then used by the gc.  What are your thoughts on that? Granted this is more amenable for mem management and not other scarce resources.
> 
> On Jan 6, 2012 11:17 AM, "Tony Printezis" <tony.printezis at oracle.com> wrote:
> Kirk,
> 
> Inline.
> 
> On 01/06/2012 08:57 AM, Kirk Pepperdine wrote:
> Tony, David,
> 
> I really feel that this problem is being solved at the wrong level. The JVM lacks application semantics to know when to do the "right" thing.
> 
> I absolutely agree.
> 
> So, IMHO,  this should be managed by the application.
> 
> I absolutely disagree. :-)
> 
> First, let's define some terminology to make sure we're all on the same page:
> 
> JVM : HotSpot
> library : the Java code that manages certain native resource (e.g., classes in the java.io package) - I do not consider this part of the JVM
> application : what the user writes which runs on top of HotSpot and uses java.io.File's.
> 
> For the same reason, there are other things that the application shouldn't touch like telling the JVM it's time to run a collection.
> 
> Amen to that. :-)
> 
> That said, one thing the JVM might know about is how many file descriptors it's allowed and it could trigger an attempt to recover them (i.e. run finalization) once they start running low..
> 
> Indeed. But the JVM (as defined above) does not know anything about file descriptors. It's the library, in this case classes in java.io, that does. However, I don't think said library should also be calling System.gc() either. It should be providing information to the GC, via the API I have been suggesting, on how much of a certain resource we have and the GC should be making informed decisions on whether it when it should trigger a cycle.
> 
> Tony
> 
>  just as the JVM manages memory by recovering it when it runs low. That said, I did run a bench where we opened 1,000,000,000 sockets on a single VM. That is just about as many sockets as can be opened on a machine without recompiling the kernel to configure a bigger file descriptor bitmap. I'm quite happy that the JVM wasn't fighting me as I was trying to open all these sockets. While this bench was a bit extreme, it's clear that WebSocket gateways *will* stress file descriptor.
> 
> Not sure about direct buffers though I do feel they should implement finalization as a fallback position.
> 
> Cheers,
> Kirk
> 
> On 2012-01-06, at 12:14 PM, David Holmes wrote:
> 
> Hi Tony,
> 
> On 6/01/2012 8:34 PM, Tony Printezis wrote:
> If each library could indicate to the GC whether the resource it manages
> is running low or not, using the API I mentioned, the GC could do the
> above automatically, behind the scenes, without the user having to do
> anything else.
> I'm not sure a library writer has the necessary information to do this. Seems to me that how an application uses a particular type determines the scarcity of the resource. I can imagine something like:
> 
> void setFinalizationLimit(Class<?>  cls, int limit)
> 
> so that GC runs finalization once a "reference count" reaches "limit". That limits the frequency of finalization, but the actual finalization cost may still be unacceptably high.
> 
> Cheers,
> David
> 
> Tony
> 
> Afterall the key thing about GC is it relieves the programmer from
> having to manage object lifetimes, so if you don't know when the
> object is no longer used you don't know when to call close.
> 
> David

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120106/b26e0c83/attachment.html 

From vitalyd at gmail.com  Fri Jan  6 08:57:05 2012
From: vitalyd at gmail.com (Vitaly Davidovich)
Date: Fri, 6 Jan 2012 11:57:05 -0500
Subject: JEP 132: More-prompt finalization
In-Reply-To: <4F07233A.5010001@oracle.com>
References: <20111222230542.48A451084@eggemoggin.niobe.net>
	<4EF3E44F.1060409@oracle.com> <4EF4A8A4.1060502@oracle.com>
	<4EF5C695.4060206@oracle.com> <4EF9F709.4030101@oracle.com>
	<4EFA2869.6070105@oracle.com>
	<712FF280-2C59-47A0-99DE-ECEAD5FCFB15@kodewerk.com>
	<4EFAC2C7.7060508@oracle.com>
	<32CBF5EE-6AB8-4888-895E-37E495ACCD74@kodewerk.com>
	<4EFB5583.5000005@oracle.com> <4F05D3A0.1050503@oracle.com>
	<08196B9F-49AC-4DBF-B0C7-76903539C217@kodewerk.com>
	<4F065F5D.6040605@oracle.com> <4F06CE20.10809@oracle.com>
	<4F06D7A7.6010603@oracle.com>
	<3EF8333A-5812-41DD-8180-4AE70F581161@kodewerk.com>
	<4F071E55.9050203@oracle.com>
	<CAHjP37GVPWNZAdtM4xPAnt_L-y1R55x-yLZY3B1YFNGkYnf-rg@mail.gmail.com>
	<4F07233A.5010001@oracle.com>
Message-ID: <CAHjP37FE0+gzk_KrhGKEkQyZAU_cgF5-9GoCffYm5CGH+aWXoQ@mail.gmail.com>

Yes it takes an int64.
On Jan 6, 2012 11:37 AM, "Tony Printezis" <tony.printezis at oracle.com> wrote:

>  Does AddMemoryPressure() take any parameters? Creating a new socket (out
> of a max of say 1,000,000) should have different weight to, say, creating a
> new file (out of a of say 50).
>
> Tony
>
> On 01/06/2012 11:32 AM, Vitaly Davidovich wrote:
>
> Tony,
>
> Interestingly the .NET CLR has an API  to allow apps to tell the
> runtime(gc specifically) to be more aggressive -
> System.GC.AddMemoryPressure/RemoveMemoryPressure.  The idea being that
> you'd call this when a managed object allocates native memory and thus adds
> overall pressure.  This hint is then used by the gc.  What are your
> thoughts on that? Granted this is more amenable for mem management and not
> other scarce resources.
> On Jan 6, 2012 11:17 AM, "Tony Printezis" <tony.printezis at oracle.com>
> wrote:
>
>> Kirk,
>>
>> Inline.
>>
>> On 01/06/2012 08:57 AM, Kirk Pepperdine wrote:
>>
>>> Tony, David,
>>>
>>> I really feel that this problem is being solved at the wrong level. The
>>> JVM lacks application semantics to know when to do the "right" thing.
>>>
>>
>> I absolutely agree.
>>
>>  So, IMHO,  this should be managed by the application.
>>>
>>
>> I absolutely disagree. :-)
>>
>> First, let's define some terminology to make sure we're all on the same
>> page:
>>
>> JVM : HotSpot
>> library : the Java code that manages certain native resource (e.g.,
>> classes in the java.io package) - I do not consider this part of the JVM
>> application : what the user writes which runs on top of HotSpot and uses
>> java.io.File's.
>>
>>  For the same reason, there are other things that the application
>>> shouldn't touch like telling the JVM it's time to run a collection.
>>>
>>
>> Amen to that. :-)
>>
>>  That said, one thing the JVM might know about is how many file
>>> descriptors it's allowed and it could trigger an attempt to recover them
>>> (i.e. run finalization) once they start running low..
>>>
>>
>> Indeed. But the JVM (as defined above) does not know anything about file
>> descriptors. It's the library, in this case classes in java.io, that
>> does. However, I don't think said library should also be calling
>> System.gc() either. It should be providing information to the GC, via the
>> API I have been suggesting, on how much of a certain resource we have and
>> the GC should be making informed decisions on whether it when it should
>> trigger a cycle.
>>
>> Tony
>>
>>   just as the JVM manages memory by recovering it when it runs low. That
>>> said, I did run a bench where we opened 1,000,000,000 sockets on a single
>>> VM. That is just about as many sockets as can be opened on a machine
>>> without recompiling the kernel to configure a bigger file descriptor
>>> bitmap. I'm quite happy that the JVM wasn't fighting me as I was trying to
>>> open all these sockets. While this bench was a bit extreme, it's clear that
>>> WebSocket gateways *will* stress file descriptor.
>>>
>>> Not sure about direct buffers though I do feel they should implement
>>> finalization as a fallback position.
>>>
>>> Cheers,
>>> Kirk
>>>
>>> On 2012-01-06, at 12:14 PM, David Holmes wrote:
>>>
>>>  Hi Tony,
>>>>
>>>> On 6/01/2012 8:34 PM, Tony Printezis wrote:
>>>>
>>>>> If each library could indicate to the GC whether the resource it
>>>>> manages
>>>>> is running low or not, using the API I mentioned, the GC could do the
>>>>> above automatically, behind the scenes, without the user having to do
>>>>> anything else.
>>>>>
>>>> I'm not sure a library writer has the necessary information to do this.
>>>> Seems to me that how an application uses a particular type determines the
>>>> scarcity of the resource. I can imagine something like:
>>>>
>>>> void setFinalizationLimit(Class<?>  cls, int limit)
>>>>
>>>> so that GC runs finalization once a "reference count" reaches "limit".
>>>> That limits the frequency of finalization, but the actual finalization cost
>>>> may still be unacceptably high.
>>>>
>>>> Cheers,
>>>> David
>>>>
>>>>  Tony
>>>>>
>>>>>  Afterall the key thing about GC is it relieves the programmer from
>>>>>> having to manage object lifetimes, so if you don't know when the
>>>>>> object is no longer used you don't know when to call close.
>>>>>>
>>>>>> David
>>>>>>
>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120106/f4cbd233/attachment.html 

From vitalyd at gmail.com  Fri Jan  6 08:59:04 2012
From: vitalyd at gmail.com (Vitaly Davidovich)
Date: Fri, 6 Jan 2012 11:59:04 -0500
Subject: JEP 132: More-prompt finalization
In-Reply-To: <A5D45A8F-D4A6-41FD-A23A-B4B01BD332EC@kodewerk.com>
References: <20111222230542.48A451084@eggemoggin.niobe.net>
	<4EF3E44F.1060409@oracle.com> <4EF4A8A4.1060502@oracle.com>
	<4EF5C695.4060206@oracle.com> <4EF9F709.4030101@oracle.com>
	<4EFA2869.6070105@oracle.com>
	<712FF280-2C59-47A0-99DE-ECEAD5FCFB15@kodewerk.com>
	<4EFAC2C7.7060508@oracle.com>
	<32CBF5EE-6AB8-4888-895E-37E495ACCD74@kodewerk.com>
	<4EFB5583.5000005@oracle.com> <4F05D3A0.1050503@oracle.com>
	<08196B9F-49AC-4DBF-B0C7-76903539C217@kodewerk.com>
	<4F065F5D.6040605@oracle.com> <4F06CE20.10809@oracle.com>
	<4F06D7A7.6010603@oracle.com>
	<3EF8333A-5812-41DD-8180-4AE70F581161@kodewerk.com>
	<4F071E55.9050203@oracle.com>
	<CAHjP37GVPWNZAdtM4xPAnt_L-y1R55x-yLZY3B1YFNGkYnf-rg@mail.gmail.com>
	<A5D45A8F-D4A6-41FD-A23A-B4B01BD332EC@kodewerk.com>
Message-ID: <CAHjP37HgNHo06-UkSZ+B-w_EdJEo_d4jpqdyEfWTAqp8G0zF9Q@mail.gmail.com>

Curious - why do you say this? I don't want to diverge this thread but
that's a bold statement :).
On Jan 6, 2012 11:54 AM, "Kirk Pepperdine" <kirk at kodewerk.com> wrote:

> when it comes to performance, I'm not sure that the CLR is a good role
> model ;-)
>
> Regards,
> Kirk
>
> On 2012-01-06, at 5:32 PM, Vitaly Davidovich wrote:
>
> Tony,
>
> Interestingly the .NET CLR has an API  to allow apps to tell the
> runtime(gc specifically) to be more aggressive -
> System.GC.AddMemoryPressure/RemoveMemoryPressure.  The idea being that
> you'd call this when a managed object allocates native memory and thus adds
> overall pressure.  This hint is then used by the gc.  What are your
> thoughts on that? Granted this is more amenable for mem management and not
> other scarce resources.
> On Jan 6, 2012 11:17 AM, "Tony Printezis" <tony.printezis at oracle.com>
> wrote:
>
>> Kirk,
>>
>> Inline.
>>
>> On 01/06/2012 08:57 AM, Kirk Pepperdine wrote:
>>
>>> Tony, David,
>>>
>>> I really feel that this problem is being solved at the wrong level. The
>>> JVM lacks application semantics to know when to do the "right" thing.
>>>
>>
>> I absolutely agree.
>>
>>  So, IMHO,  this should be managed by the application.
>>>
>>
>> I absolutely disagree. :-)
>>
>> First, let's define some terminology to make sure we're all on the same
>> page:
>>
>> JVM : HotSpot
>> library : the Java code that manages certain native resource (e.g.,
>> classes in the java.io package) - I do not consider this part of the JVM
>> application : what the user writes which runs on top of HotSpot and uses
>> java.io.File's.
>>
>>  For the same reason, there are other things that the application
>>> shouldn't touch like telling the JVM it's time to run a collection.
>>>
>>
>> Amen to that. :-)
>>
>>  That said, one thing the JVM might know about is how many file
>>> descriptors it's allowed and it could trigger an attempt to recover them
>>> (i.e. run finalization) once they start running low..
>>>
>>
>> Indeed. But the JVM (as defined above) does not know anything about file
>> descriptors. It's the library, in this case classes in java.io, that
>> does. However, I don't think said library should also be calling
>> System.gc() either. It should be providing information to the GC, via the
>> API I have been suggesting, on how much of a certain resource we have and
>> the GC should be making informed decisions on whether it when it should
>> trigger a cycle.
>>
>> Tony
>>
>>   just as the JVM manages memory by recovering it when it runs low. That
>>> said, I did run a bench where we opened 1,000,000,000 sockets on a single
>>> VM. That is just about as many sockets as can be opened on a machine
>>> without recompiling the kernel to configure a bigger file descriptor
>>> bitmap. I'm quite happy that the JVM wasn't fighting me as I was trying to
>>> open all these sockets. While this bench was a bit extreme, it's clear that
>>> WebSocket gateways *will* stress file descriptor.
>>>
>>> Not sure about direct buffers though I do feel they should implement
>>> finalization as a fallback position.
>>>
>>> Cheers,
>>> Kirk
>>>
>>> On 2012-01-06, at 12:14 PM, David Holmes wrote:
>>>
>>>  Hi Tony,
>>>>
>>>> On 6/01/2012 8:34 PM, Tony Printezis wrote:
>>>>
>>>>> If each library could indicate to the GC whether the resource it
>>>>> manages
>>>>> is running low or not, using the API I mentioned, the GC could do the
>>>>> above automatically, behind the scenes, without the user having to do
>>>>> anything else.
>>>>>
>>>> I'm not sure a library writer has the necessary information to do this.
>>>> Seems to me that how an application uses a particular type determines the
>>>> scarcity of the resource. I can imagine something like:
>>>>
>>>> void setFinalizationLimit(Class<?>  cls, int limit)
>>>>
>>>> so that GC runs finalization once a "reference count" reaches "limit".
>>>> That limits the frequency of finalization, but the actual finalization cost
>>>> may still be unacceptably high.
>>>>
>>>> Cheers,
>>>> David
>>>>
>>>>  Tony
>>>>>
>>>>>  Afterall the key thing about GC is it relieves the programmer from
>>>>>> having to manage object lifetimes, so if you don't know when the
>>>>>> object is no longer used you don't know when to call close.
>>>>>>
>>>>>> David
>>>>>>
>>>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120106/7b55777e/attachment-0001.html 

From kirk at kodewerk.com  Fri Jan  6 09:02:14 2012
From: kirk at kodewerk.com (Kirk Pepperdine)
Date: Fri, 6 Jan 2012 18:02:14 +0100
Subject: JEP 132: More-prompt finalization
In-Reply-To: <4F071E55.9050203@oracle.com>
References: <20111222230542.48A451084@eggemoggin.niobe.net>
	<4EF3E44F.1060409@oracle.com> <4EF4A8A4.1060502@oracle.com>
	<4EF5C695.4060206@oracle.com> <4EF9F709.4030101@oracle.com>
	<4EFA2869.6070105@oracle.com>
	<712FF280-2C59-47A0-99DE-ECEAD5FCFB15@kodewerk.com>
	<4EFAC2C7.7060508@oracle.com>
	<32CBF5EE-6AB8-4888-895E-37E495ACCD74@kodewerk.com>
	<4EFB5583.5000005@oracle.com> <4F05D3A0.1050503@oracle.com>
	<08196B9F-49AC-4DBF-B0C7-76903539C217@kodewerk.com>
	<4F065F5D.6040605@oracle.com> <4F06CE20.10809@oracle.com>
	<4F06D7A7.6010603@oracle.com>
	<3EF8333A-5812-41DD-8180-4AE70F581161@kodewerk.com>
	<4F071E55.9050203@oracle.com>
Message-ID: <A1107F16-438D-447C-B38E-A77B34435057@kodewerk.com>


On 2012-01-06, at 5:16 PM, Tony Printezis wrote:

> Kirk,
> 
> Inline.
> 
> On 01/06/2012 08:57 AM, Kirk Pepperdine wrote:
>> Tony, David,
>> 
>> I really feel that this problem is being solved at the wrong level. The JVM lacks application semantics to know when to do the "right" thing.
> 
> I absolutely agree.
> 
>> So, IMHO,  this should be managed by the application.
> 
> I absolutely disagree. :-)
> 
> First, let's define some terminology to make sure we're all on the same page:
> 
> JVM : HotSpot
> library : the Java code that manages certain native resource (e.g., classes in the java.io package) - I do not consider this part of the JVM
> application : what the user writes which runs on top of HotSpot and uses java.io.File's.

Ok, so with these definitions I'd say that anything but the JVM should manage those things to which it doesn't have semantics knowledge. That would be the application or library (I bundled these together when I shouldn't have).

> 
>> For the same reason, there are other things that the application shouldn't touch like telling the JVM it's time to run a collection.
> 
> Amen to that. :-)
> 
>> That said, one thing the JVM might know about is how many file descriptors it's allowed and it could trigger an attempt to recover them (i.e. run finalization) once they start running low..
> 
> Indeed. But the JVM (as defined above) does not know anything about file descriptors. It's the library, in this case classes in java.io, that does. However, I don't think said library should also be calling System.gc() either. It should be providing information to the GC, via the API I have been suggesting, on how much of a certain resource we have and the GC should be making informed decisions on whether it when it should trigger a cycle.

Ok, again, my confused terminology? basically I don't see how the collector could make an informed decision on anything other than file descriptors. Would it have to track rates of consumption and desposal to build an understanding of when it should take action?

Regards,
Kirk


From jon.masamitsu at oracle.com  Fri Jan  6 10:16:37 2012
From: jon.masamitsu at oracle.com (jon.masamitsu at oracle.com)
Date: Fri, 06 Jan 2012 18:16:37 +0000
Subject: hg: hsx/hotspot-main/hotspot: 3 new changesets
Message-ID: <20120106181649.EF4B1478D1@hg.openjdk.java.net>

Changeset: b6a04c79ccbc
Author:    stefank
Date:      2012-01-02 10:01 +0100
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/b6a04c79ccbc

7125503: Compiling collectedHeap.cpp fails with -Werror=int-to-pointer-cast with g++ 4.6.1
Summary: Used uintptr_t and void* for all the casts and checks in test_is_in.
Reviewed-by: tonyp, jmasa

! src/share/vm/gc_interface/collectedHeap.cpp

Changeset: 4753e3dda3c8
Author:    jmasa
Date:      2012-01-04 07:56 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/4753e3dda3c8

Merge


Changeset: 2ee4167627a3
Author:    jmasa
Date:      2012-01-05 21:02 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/2ee4167627a3

Merge



From daniel.daugherty at oracle.com  Fri Jan  6 19:47:45 2012
From: daniel.daugherty at oracle.com (daniel.daugherty at oracle.com)
Date: Sat, 07 Jan 2012 03:47:45 +0000
Subject: hg: hsx/hotspot-main/hotspot: 7 new changesets
Message-ID: <20120107034759.8E1ED478DD@hg.openjdk.java.net>

Changeset: 7ab5f6318694
Author:    phh
Date:      2012-01-01 11:17 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/7ab5f6318694

7125934: Add a fast unordered timestamp capability to Hotspot on x86/x64
Summary: Add rdtsc detection and inline generation.
Reviewed-by: kamg, dholmes
Contributed-by: karen.kinnear at oracle.com

! src/cpu/x86/vm/vm_version_x86.cpp
! src/cpu/x86/vm/vm_version_x86.hpp
! src/os_cpu/bsd_x86/vm/os_bsd_x86.hpp
+ src/os_cpu/bsd_x86/vm/os_bsd_x86.inline.hpp
! src/os_cpu/linux_x86/vm/os_linux_x86.hpp
+ src/os_cpu/linux_x86/vm/os_linux_x86.inline.hpp
! src/os_cpu/solaris_x86/vm/os_solaris_x86.hpp
+ src/os_cpu/solaris_x86/vm/os_solaris_x86.inline.hpp
! src/os_cpu/solaris_x86/vm/solaris_x86_32.il
! src/os_cpu/solaris_x86/vm/solaris_x86_64.il
! src/os_cpu/windows_x86/vm/os_windows_x86.hpp
+ src/os_cpu/windows_x86/vm/os_windows_x86.inline.hpp
! src/share/vm/runtime/init.cpp
! src/share/vm/runtime/os.cpp
! src/share/vm/runtime/os.hpp
+ src/share/vm/runtime/os_ext.hpp

Changeset: b16494a69d3d
Author:    phh
Date:      2012-01-03 15:11 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/b16494a69d3d

7126185: Clean up lasterror handling, add os::get_last_error()
Summary: Add os::get_last_error(), replace getLastErrorString() by os::lasterror() in os_windows.cpp.
Reviewed-by: kamg, dholmes
Contributed-by: erik.gahlin at oracle.com

! src/os/posix/vm/os_posix.cpp
! src/os/windows/vm/os_windows.cpp
! src/share/vm/runtime/os.hpp

Changeset: 5b58979183f9
Author:    dcubed
Date:      2012-01-05 06:24 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/5b58979183f9

7127032: fix for 7122253 adds a JvmtiThreadState earlier than necessary
Summary: Use JavaThread::jvmti_thread_state() instead of JvmtiThreadState::state_for().
Reviewed-by: coleenp, poonam, acorn

! src/share/vm/classfile/classFileParser.cpp

Changeset: 8a63c6323842
Author:    fparain
Date:      2012-01-05 07:26 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/8a63c6323842

7125594: C-heap growth issue in ThreadService::find_deadlocks_at_safepoint
Reviewed-by: sspitsyn, dcubed, mchung, dholmes

! src/share/vm/services/threadService.cpp

Changeset: 2e0ef19fc891
Author:    phh
Date:      2012-01-05 17:14 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/2e0ef19fc891

7126480: Make JVM start time in milliseconds since the Java epoch available
Summary: Expose existing Management::_begin_vm_creation_time via new accessor Management::begin_vm_creation_time().
Reviewed-by: acorn, dcubed

! src/share/vm/services/management.hpp

Changeset: 66259eca2bf7
Author:    phh
Date:      2012-01-05 17:16 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/66259eca2bf7

Merge


Changeset: 2b3acb34791f
Author:    dcubed
Date:      2012-01-06 16:18 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/2b3acb34791f

Merge

! src/os/windows/vm/os_windows.cpp
! src/share/vm/classfile/classFileParser.cpp
! src/share/vm/runtime/os.hpp


From vladimir.kozlov at oracle.com  Fri Jan  6 22:35:41 2012
From: vladimir.kozlov at oracle.com (vladimir.kozlov at oracle.com)
Date: Sat, 07 Jan 2012 06:35:41 +0000
Subject: hg: hsx/hotspot-main/hotspot: 16 new changesets
Message-ID: <20120107063612.AD879478DE@hg.openjdk.java.net>

Changeset: abcceac2f7cd
Author:    iveresov
Date:      2011-12-12 12:44 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/abcceac2f7cd

7119730: Tiered: SIGSEGV in AdvancedThresholdPolicy::is_method_profiled(methodOop)
Summary: Added handles for references to methods in select_task()
Reviewed-by: twisti, kvn

! src/share/vm/runtime/advancedThresholdPolicy.cpp

Changeset: 7bca37d28f32
Author:    roland
Date:      2011-12-13 10:54 +0100
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/7bca37d28f32

7114106: C1: assert(goto_state->is_same(sux_state)) failed: states must match now
Summary: fix C1's CEE to take inlining into account when the stacks in states are compared.
Reviewed-by: iveresov, never

! src/share/vm/c1/c1_Optimizer.cpp

Changeset: d725f0affb1a
Author:    iveresov
Date:      2011-12-13 17:10 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/d725f0affb1a

7121111: -server -Xcomp -XX:+TieredCompilation does not invoke C2 compiler
Summary: Exercise C2 more in tiered mode with Xcomp
Reviewed-by: kvn, never

! src/share/vm/runtime/arguments.cpp

Changeset: 127b3692c168
Author:    kvn
Date:      2011-12-14 14:54 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/127b3692c168

7116452: Add support for AVX instructions
Summary: Added support for AVX extension to the x86 instruction set.
Reviewed-by: never

! src/cpu/x86/vm/assembler_x86.cpp
! src/cpu/x86/vm/assembler_x86.hpp
! src/cpu/x86/vm/assembler_x86.inline.hpp
! src/cpu/x86/vm/nativeInst_x86.cpp
! src/cpu/x86/vm/nativeInst_x86.hpp
! src/cpu/x86/vm/register_definitions_x86.cpp
! src/cpu/x86/vm/vm_version_x86.cpp
! src/cpu/x86/vm/vm_version_x86.hpp
! src/cpu/x86/vm/x86_32.ad
! src/cpu/x86/vm/x86_64.ad
! src/share/vm/runtime/globals.hpp

Changeset: 669f6a7d5b70
Author:    never
Date:      2011-12-19 14:16 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/669f6a7d5b70

7121073: secondary_super_cache memory slice has incorrect bounds in flatten_alias_type
Reviewed-by: kvn

! src/share/vm/opto/compile.cpp

Changeset: 65149e74c706
Author:    kvn
Date:      2011-12-20 00:55 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/65149e74c706

7121648: Use 3-operands SIMD instructions on x86 with AVX
Summary: Use 3-operands SIMD instructions in C2 generated code for machines with AVX.
Reviewed-by: never

! make/bsd/makefiles/adlc.make
! make/linux/makefiles/adlc.make
! make/solaris/makefiles/adlc.make
! make/windows/makefiles/adlc.make
! src/cpu/x86/vm/assembler_x86.cpp
! src/cpu/x86/vm/assembler_x86.hpp
+ src/cpu/x86/vm/x86.ad
! src/cpu/x86/vm/x86_32.ad
! src/cpu/x86/vm/x86_64.ad
! src/share/vm/opto/matcher.cpp

Changeset: 069ab3f976d3
Author:    stefank
Date:      2011-12-07 11:35 +0100
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/069ab3f976d3

7118863: Move sizeof(klassOopDesc) into the *Klass::*_offset_in_bytes() functions
Summary: Moved sizeof(klassOopDesc), changed the return type to ByteSize and removed the _in_bytes suffix.
Reviewed-by: never, bdelsart, coleenp, jrose

! src/cpu/sparc/vm/assembler_sparc.cpp
! src/cpu/sparc/vm/c1_CodeStubs_sparc.cpp
! src/cpu/sparc/vm/c1_LIRAssembler_sparc.cpp
! src/cpu/sparc/vm/c1_MacroAssembler_sparc.cpp
! src/cpu/sparc/vm/c1_Runtime1_sparc.cpp
! src/cpu/sparc/vm/cppInterpreter_sparc.cpp
! src/cpu/sparc/vm/methodHandles_sparc.cpp
! src/cpu/sparc/vm/stubGenerator_sparc.cpp
! src/cpu/sparc/vm/templateInterpreter_sparc.cpp
! src/cpu/sparc/vm/templateTable_sparc.cpp
! src/cpu/x86/vm/assembler_x86.cpp
! src/cpu/x86/vm/c1_CodeStubs_x86.cpp
! src/cpu/x86/vm/c1_LIRAssembler_x86.cpp
! src/cpu/x86/vm/c1_MacroAssembler_x86.cpp
! src/cpu/x86/vm/c1_Runtime1_x86.cpp
! src/cpu/x86/vm/cppInterpreter_x86.cpp
! src/cpu/x86/vm/methodHandles_x86.cpp
! src/cpu/x86/vm/stubGenerator_x86_32.cpp
! src/cpu/x86/vm/stubGenerator_x86_64.cpp
! src/cpu/x86/vm/templateInterpreter_x86_32.cpp
! src/cpu/x86/vm/templateInterpreter_x86_64.cpp
! src/cpu/x86/vm/templateTable_x86_32.cpp
! src/cpu/x86/vm/templateTable_x86_64.cpp
! src/cpu/x86/vm/x86_64.ad
! src/share/vm/c1/c1_LIRGenerator.cpp
! src/share/vm/oops/arrayKlass.hpp
! src/share/vm/oops/instanceKlass.hpp
! src/share/vm/oops/klass.cpp
! src/share/vm/oops/klass.hpp
! src/share/vm/oops/klassOop.hpp
! src/share/vm/oops/objArrayKlass.hpp
! src/share/vm/opto/compile.cpp
! src/share/vm/opto/graphKit.cpp
! src/share/vm/opto/library_call.cpp
! src/share/vm/opto/macro.cpp
! src/share/vm/opto/memnode.cpp
! src/share/vm/opto/parse1.cpp
! src/share/vm/opto/parseHelper.cpp
! src/share/vm/shark/sharkIntrinsics.cpp
! src/share/vm/shark/sharkTopLevelBlock.cpp

Changeset: 1dc233a8c7fe
Author:    roland
Date:      2011-12-20 16:56 +0100
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/1dc233a8c7fe

7121140: Allocation paths require explicit memory synchronization operations for RMO systems
Summary: adds store store barrier after initialization of header and body of objects.
Reviewed-by: never, kvn

! src/cpu/sparc/vm/sparc.ad
! src/cpu/x86/vm/x86_32.ad
! src/cpu/x86/vm/x86_64.ad
! src/share/vm/adlc/formssel.cpp
! src/share/vm/opto/callnode.hpp
! src/share/vm/opto/classes.hpp
! src/share/vm/opto/escape.cpp
! src/share/vm/opto/graphKit.cpp
! src/share/vm/opto/library_call.cpp
! src/share/vm/opto/macro.cpp
! src/share/vm/opto/memnode.cpp
! src/share/vm/opto/memnode.hpp
! src/share/vm/opto/node.hpp

Changeset: e5ac210043cd
Author:    roland
Date:      2011-12-22 10:55 +0100
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/e5ac210043cd

7123108: C1: assert(if_state != NULL) failed: states do not match up
Summary: In CEE, ensure if and common successor state are at the same inline level
Reviewed-by: never

! src/share/vm/c1/c1_Optimizer.cpp
+ test/compiler/7123108/Test7123108.java

Changeset: b642b49f9738
Author:    roland
Date:      2011-12-23 09:36 +0100
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/b642b49f9738

7123253: C1: in store check code, usage of registers may be incorrect
Summary: fix usage of input register in assembly code for store check.
Reviewed-by: never

! src/share/vm/c1/c1_LIR.cpp

Changeset: 40c2484c09e1
Author:    kvn
Date:      2011-12-23 15:24 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/40c2484c09e1

7110832: ctw/.../org_apache_avalon_composition_util_StringHelper crashes the VM
Summary: Distance is too large for one short branch in string_indexofC8().
Reviewed-by: iveresov

! src/cpu/x86/vm/assembler_x86.cpp
! src/share/vm/asm/assembler.cpp
! src/share/vm/asm/assembler.hpp

Changeset: d12a66fa3820
Author:    kvn
Date:      2011-12-27 15:08 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/d12a66fa3820

7123954: Some CTW test crash with SIGSEGV
Summary: Correct Allocate expansion code to preserve i_o when only slow call is generated.
Reviewed-by: iveresov

! src/share/vm/opto/compile.cpp
! src/share/vm/opto/macro.cpp

Changeset: 8940fd98d540
Author:    kvn
Date:      2011-12-29 11:37 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/8940fd98d540

Merge

! src/cpu/x86/vm/assembler_x86.cpp
! src/share/vm/runtime/globals.hpp

Changeset: 9c87bcb3b4dd
Author:    kvn
Date:      2011-12-30 11:43 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/9c87bcb3b4dd

7125879: assert(proj != NULL) failed: must be found
Summary: Leave i_o attached to slow allocation call when there are no i_o users after the call.
Reviewed-by: iveresov, twisti

! src/share/vm/opto/macro.cpp
+ test/compiler/7125879/Test7125879.java

Changeset: 1cb50d7a9d95
Author:    iveresov
Date:      2012-01-05 17:25 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/1cb50d7a9d95

7119294: Two command line options cause JVM to crash
Summary: Setup thread register in MacroAssembler::incr_allocated_bytes() on x64
Reviewed-by: kvn

! src/cpu/x86/vm/assembler_x86.cpp

Changeset: 22cee0ee8927
Author:    kvn
Date:      2012-01-06 20:09 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/22cee0ee8927

Merge

! src/cpu/sparc/vm/c1_LIRAssembler_sparc.cpp
! src/cpu/sparc/vm/c1_Runtime1_sparc.cpp
! src/cpu/sparc/vm/stubGenerator_sparc.cpp
! src/cpu/sparc/vm/templateInterpreter_sparc.cpp
! src/cpu/sparc/vm/templateTable_sparc.cpp
! src/cpu/x86/vm/c1_LIRAssembler_x86.cpp
! src/cpu/x86/vm/c1_Runtime1_x86.cpp
! src/cpu/x86/vm/stubGenerator_x86_32.cpp
! src/cpu/x86/vm/stubGenerator_x86_64.cpp
! src/cpu/x86/vm/templateInterpreter_x86_32.cpp
! src/cpu/x86/vm/templateInterpreter_x86_64.cpp
! src/cpu/x86/vm/templateTable_x86_32.cpp
! src/cpu/x86/vm/templateTable_x86_64.cpp
! src/cpu/x86/vm/vm_version_x86.cpp
! src/cpu/x86/vm/vm_version_x86.hpp
! src/share/vm/oops/instanceKlass.hpp
! src/share/vm/opto/library_call.cpp
! src/share/vm/opto/parseHelper.cpp


From james.melvin at oracle.com  Sat Jan  7 08:38:46 2012
From: james.melvin at oracle.com (James Melvin)
Date: Sat, 07 Jan 2012 11:38:46 -0500
Subject: RFR (S): 7125793: MAC: test_gamma should always work
In-Reply-To: <4F034B51.3070609@oracle.com>
References: <4EFECA5D.6010905@oracle.com> <4F034B51.3070609@oracle.com>
Message-ID: <4F087516.40505@oracle.com>

Hi Dan,

Finally getting back on the trail to fix the gamma launcher. Sorry for
the delayed response. Thanks for the review, Dan and David. Replies
inline...


On 1/3/12 1:39 PM, Daniel D. Daugherty wrote:
>> http://cr.openjdk.java.net/~jmelvin/7125793/webrev.00
>
> Jim,
>
> Thanks for diving in and improving the MacOS X port!
> Comments below.
>
> Dan
>
>
> make/bsd/makefiles/buildtree.make
>
> line 422: The new 'java -fullversion' invocation does not include
> the $(JAVA_FLAG) option like the old code did. Any particular
> reason for the change?
>
> Looks like that means the '-d32' or '-d64' options won't be
> specified as they were before.

Originally, this no longer made sense as both -d32 and -d64 were mapped
to 64-bit. After further review, I'm going to readd this option in case
we ever change our minds and decide to support both 32 and 64-bit JVMs
on Mac OS X.


>
> line 447: Why not just echo FULL_VERSION? Why pipe to awk?

To preserve the original script output, I needed to trim the extra
newline... 1 from $FULLVERSION and 1 from echo.

>
> line 465: The 'jre/lib/libjava.dylib' part of the new check is
> MacOS X specific. Other BSDs don't necessarily use the
> '.dylib' extension (instead of .so) and I don't think that
> other BSDs have dropped the "arch" subdir.

To be more friendly to other BSDs, I've added a $(LIBARCH) subdir check
and $(LIBRARY_SUFFIX) instead of hardcoded .dylib. However, I really
don't have a way of testing this for those other BSDs.

>
> line 484: The DYLD_LIBRARY_PATH part is MacOS X specific. Will
> still need to set LD_LIBRARY_PATH for other BSDs.

Also, a good point. I've re-added LD_LIBRARY_PATH with it's original
setting.

>
> line 492: You switched from $(TESTFLAGS) to literal flag values,
> but you left the TESTFLAGS variable around. Any reason for
> the switch?

Nice find.  Cut-paste overwrite.  Fixed by restoring $(TESTFLAGS).

>
>
> make/bsd/makefiles/launcher.make
> Please add a comment explaining why '-framework CoreFoundation'
> is needed. Your explanatory block below is a really good start.

Done.

>
>
> make/bsd/makefiles/vm.make
> No comments.
>
>
> src/os/bsd/vm/os_bsd.cpp
> line 2585: Uses a suffix of ".so". That shouldn't work on MacOS X
> since MacOS X uses '.dylib'. That's OK for other BSDs, but not
> MacOS X. Also the comments that mention '.so' should be updated
> to include '.dylib' (not caused by your changes).

I've replaced .so with $JNI_LIB_SUFFIX defined earlier in the source.
In the area comments, I've just dropped .so extension altogether to
cheaply ambiguate.

>
> To David H. - Yes, this change added another '#fdef __APPLE__'. It
> is not the first and it likely won't be the last since we're
> not done yet with the MacOS X port. There are a number of
> things that need to be cleaned up and we're tracking them.
> However, as you know, we don't have enough folks to handle all
> of the work so we'll just have to live with the warts for now.

For this particular change to fix gamma, I've managed to resolve David's
concerns by adding support for no-arch paths in the code rather than
using #ifdefs. However, ifdefs are sprinkled everywhere and this will
need to be resolved whenever we reconcile the unix platforms into a more
common codebase.


>
> src/os/posix/launcher/java_md.c
> No comments.
>

Thanks for the review comments. I've also added a 1 line change in
make/bsd/makefiles/defs.make to fix a build warning around duplicate
targets for Xusage.txt due to a variable expansion. This has already
been resolved for other platforms.


Changes included in new webrev.  More feedback welcome.

WEBREV:
   http://cr.openjdk.java.net/~jmelvin/7125793/webrev.01

TESTS RUN:
   JPRT 2012-01-07-064433.jmelvin.7125793
   local Mac OS X builds/tests


- Jim


>
> On 12/31/11 1:39 AM, James Melvin wrote:
>> Hi,
>>
>> This change fixes the 'gamma' simple launcher for HotSpot on Mac OS X.
>> There were 3 primary changes required to re-enable gamma...
>>
>> 1) Statically link with CoreFoundation framework to resolve symbols
>>
>> The gamma launcher dlopen()s libjava.dylib from $JAVA_HOME/jre/lib.
>> Because Mac OS X files are case-insensitive by default, we collide on
>> $FRAMEWORK/libJPEG.dylib and ${JAVA_HOME}/jre/lib/libjpeg.dylib. This
>> resulted in unresolved symbols in the Mac OS X framework libraries. The
>> solution for gamma was to statically link with CoreFoundation framework
>> to properly resolve framework symbols and allow gamma to successfully
>> dlopen() libjava.dylib.
>>
>> 2) Adjust various paths to reflect no arch subdirs
>>
>> On Mac OS X, there are no arch subdirs, e.g jre/lib vs jre/lib/<arch>.
>> Instead, one can use universal binaries to package multiple
>> architectures in a single binary. At the moment though, we are only
>> building 64-bit non-universal binaries. Note, the test_gamma script
>> assumes an Oracle JDK layout for JAVA_HOME, derived from ALT_BOOTDIR.
>> Using an Apple JDK for ALT_BOOTDIR will fail the test_gamma script
>> gracefully, as libjava.dylib is in a different, unexpected place.
>>
>> 3) Modify test_gamma script to set library path only for gamma launch
>>
>> Setting DYLD_LIBRARY_PATH adversely affects the real java launcher(s).
>> Instead, set this later in the script only for the gamma launcher test
>> run. While in there, I took the liberty of decrypting the script to make
>> it more maintainable and more easily merged whenever we reconcile the
>> unix ports into a single codebase. There is no change to the script
>> output.
>>
>> Feedback welcome...
>>
>> WEBREV:
>> http://cr.openjdk.java.net/~jmelvin/7125793/webrev.00
>>
>> TESTS RUN:
>> JPRT 2011-12-31-061123.jmelvin.7125793
>> local Mac OS X builds/tests
>>
>>
>> Thanks and Happy New Year!
>>
>> Jim

From david.holmes at oracle.com  Sun Jan  8 15:23:38 2012
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 09 Jan 2012 09:23:38 +1000
Subject: JEP 132: More-prompt finalization
In-Reply-To: <4F071C4B.8010002@oracle.com>
References: <20111222230542.48A451084@eggemoggin.niobe.net>
	<4EF3E44F.1060409@oracle.com> <4EF4A8A4.1060502@oracle.com>
	<4EF5C695.4060206@oracle.com> <4EF9F709.4030101@oracle.com>
	<4EFA2869.6070105@oracle.com>
	<712FF280-2C59-47A0-99DE-ECEAD5FCFB15@kodewerk.com>
	<4EFAC2C7.7060508@oracle.com>
	<32CBF5EE-6AB8-4888-895E-37E495ACCD74@kodewerk.com>
	<4EFB5583.5000005@oracle.com> <4F05D3A0.1050503@oracle.com>
	<08196B9F-49AC-4DBF-B0C7-76903539C217@kodewerk.com>
	<4F065F5D.6040605@oracle.com> <4F06CE20.10809@oracle.com>
	<4F06D7A7.6010603@oracle.com> <4F071C4B.8010002@oracle.com>
Message-ID: <4F0A257A.50104@oracle.com>

On 7/01/2012 2:07 AM, Tony Printezis wrote:
> David,
>
> On 01/06/2012 06:14 AM, David Holmes wrote:
>> Hi Tony,
>>
>> On 6/01/2012 8:34 PM, Tony Printezis wrote:
>>> If each library could indicate to the GC whether the resource it manages
>>> is running low or not, using the API I mentioned, the GC could do the
>>> above automatically, behind the scenes, without the user having to do
>>> anything else.
>>
>> I'm not sure a library writer has the necessary information to do
>> this. Seems to me that how an application uses a particular type
>> determines the scarcity of the resource.
>
> I will be very surprised if the majority of application developers will
> be willing to measure at what rate the application consumes certain
> resources and update said measure when the application changes, load
> increases, OS changes, hardware changes, etc. And, sure, I think there
> will be many cases where the library writer has a good idea / can find
> out the max number of instances of a particular resource (file
> descriptors, sockets, etc.).

Perhaps, but the library writer may only have partial knowledge. The 
library can use reference counts to track how much of a resource it has 
handed out, and what has been handed back. It might know what the 
absolute limit for a resource is (via getrlimit etc). But can it know 
the absolute usage rate of a given resource? Can you query how many 
available file descriptors a process has left? Some resources will be 
used by native code outside the libs (including the VM).

So at best this is a heuristic, so the library tracks the resource and 
at some threshold it invoke System.runFinalization. The GC itself 
doesn't make an informed decision because, as you said your self, the VM 
(and hence GC) knows nothing about the resource being tracked.

But would we want to burden all users of these classes with the overhead 
of resource tracking?

Cheers,
David

> Tony
>
>> I can imagine something like:
>>
>> void setFinalizationLimit(Class<?> cls, int limit)
>>
>> so that GC runs finalization once a "reference count" reaches "limit".
>> That limits the frequency of finalization, but the actual finalization
>> cost may still be unacceptably high.
>>
>> Cheers,
>> David
>>
>>> Tony
>>>
>>>> Afterall the key thing about GC is it relieves the programmer from
>>>> having to manage object lifetimes, so if you don't know when the
>>>> object is no longer used you don't know when to call close.
>>>>
>>>> David

From david.holmes at oracle.com  Sun Jan  8 15:51:19 2012
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 09 Jan 2012 09:51:19 +1000
Subject: RFR (S): 7125793: MAC: test_gamma should always work
In-Reply-To: <4F087516.40505@oracle.com>
References: <4EFECA5D.6010905@oracle.com> <4F034B51.3070609@oracle.com>
	<4F087516.40505@oracle.com>
Message-ID: <4F0A2BF7.8060903@oracle.com>

Thanks Jim, I'm much happier now.

David
-----

On 8/01/2012 2:38 AM, James Melvin wrote:
> Hi Dan,
>
> Finally getting back on the trail to fix the gamma launcher. Sorry for
> the delayed response. Thanks for the review, Dan and David. Replies
> inline...
>
>
> On 1/3/12 1:39 PM, Daniel D. Daugherty wrote:
>>> http://cr.openjdk.java.net/~jmelvin/7125793/webrev.00
>>
>> Jim,
>>
>> Thanks for diving in and improving the MacOS X port!
>> Comments below.
>>
>> Dan
>>
>>
>> make/bsd/makefiles/buildtree.make
>>
>> line 422: The new 'java -fullversion' invocation does not include
>> the $(JAVA_FLAG) option like the old code did. Any particular
>> reason for the change?
>>
>> Looks like that means the '-d32' or '-d64' options won't be
>> specified as they were before.
>
> Originally, this no longer made sense as both -d32 and -d64 were mapped
> to 64-bit. After further review, I'm going to readd this option in case
> we ever change our minds and decide to support both 32 and 64-bit JVMs
> on Mac OS X.
>
>
>>
>> line 447: Why not just echo FULL_VERSION? Why pipe to awk?
>
> To preserve the original script output, I needed to trim the extra
> newline... 1 from $FULLVERSION and 1 from echo.
>
>>
>> line 465: The 'jre/lib/libjava.dylib' part of the new check is
>> MacOS X specific. Other BSDs don't necessarily use the
>> '.dylib' extension (instead of .so) and I don't think that
>> other BSDs have dropped the "arch" subdir.
>
> To be more friendly to other BSDs, I've added a $(LIBARCH) subdir check
> and $(LIBRARY_SUFFIX) instead of hardcoded .dylib. However, I really
> don't have a way of testing this for those other BSDs.
>
>>
>> line 484: The DYLD_LIBRARY_PATH part is MacOS X specific. Will
>> still need to set LD_LIBRARY_PATH for other BSDs.
>
> Also, a good point. I've re-added LD_LIBRARY_PATH with it's original
> setting.
>
>>
>> line 492: You switched from $(TESTFLAGS) to literal flag values,
>> but you left the TESTFLAGS variable around. Any reason for
>> the switch?
>
> Nice find. Cut-paste overwrite. Fixed by restoring $(TESTFLAGS).
>
>>
>>
>> make/bsd/makefiles/launcher.make
>> Please add a comment explaining why '-framework CoreFoundation'
>> is needed. Your explanatory block below is a really good start.
>
> Done.
>
>>
>>
>> make/bsd/makefiles/vm.make
>> No comments.
>>
>>
>> src/os/bsd/vm/os_bsd.cpp
>> line 2585: Uses a suffix of ".so". That shouldn't work on MacOS X
>> since MacOS X uses '.dylib'. That's OK for other BSDs, but not
>> MacOS X. Also the comments that mention '.so' should be updated
>> to include '.dylib' (not caused by your changes).
>
> I've replaced .so with $JNI_LIB_SUFFIX defined earlier in the source.
> In the area comments, I've just dropped .so extension altogether to
> cheaply ambiguate.
>
>>
>> To David H. - Yes, this change added another '#fdef __APPLE__'. It
>> is not the first and it likely won't be the last since we're
>> not done yet with the MacOS X port. There are a number of
>> things that need to be cleaned up and we're tracking them.
>> However, as you know, we don't have enough folks to handle all
>> of the work so we'll just have to live with the warts for now.
>
> For this particular change to fix gamma, I've managed to resolve David's
> concerns by adding support for no-arch paths in the code rather than
> using #ifdefs. However, ifdefs are sprinkled everywhere and this will
> need to be resolved whenever we reconcile the unix platforms into a more
> common codebase.
>
>
>>
>> src/os/posix/launcher/java_md.c
>> No comments.
>>
>
> Thanks for the review comments. I've also added a 1 line change in
> make/bsd/makefiles/defs.make to fix a build warning around duplicate
> targets for Xusage.txt due to a variable expansion. This has already
> been resolved for other platforms.
>
>
> Changes included in new webrev. More feedback welcome.
>
> WEBREV:
> http://cr.openjdk.java.net/~jmelvin/7125793/webrev.01
>
> TESTS RUN:
> JPRT 2012-01-07-064433.jmelvin.7125793
> local Mac OS X builds/tests
>
>
> - Jim
>
>
>>
>> On 12/31/11 1:39 AM, James Melvin wrote:
>>> Hi,
>>>
>>> This change fixes the 'gamma' simple launcher for HotSpot on Mac OS X.
>>> There were 3 primary changes required to re-enable gamma...
>>>
>>> 1) Statically link with CoreFoundation framework to resolve symbols
>>>
>>> The gamma launcher dlopen()s libjava.dylib from $JAVA_HOME/jre/lib.
>>> Because Mac OS X files are case-insensitive by default, we collide on
>>> $FRAMEWORK/libJPEG.dylib and ${JAVA_HOME}/jre/lib/libjpeg.dylib. This
>>> resulted in unresolved symbols in the Mac OS X framework libraries. The
>>> solution for gamma was to statically link with CoreFoundation framework
>>> to properly resolve framework symbols and allow gamma to successfully
>>> dlopen() libjava.dylib.
>>>
>>> 2) Adjust various paths to reflect no arch subdirs
>>>
>>> On Mac OS X, there are no arch subdirs, e.g jre/lib vs jre/lib/<arch>.
>>> Instead, one can use universal binaries to package multiple
>>> architectures in a single binary. At the moment though, we are only
>>> building 64-bit non-universal binaries. Note, the test_gamma script
>>> assumes an Oracle JDK layout for JAVA_HOME, derived from ALT_BOOTDIR.
>>> Using an Apple JDK for ALT_BOOTDIR will fail the test_gamma script
>>> gracefully, as libjava.dylib is in a different, unexpected place.
>>>
>>> 3) Modify test_gamma script to set library path only for gamma launch
>>>
>>> Setting DYLD_LIBRARY_PATH adversely affects the real java launcher(s).
>>> Instead, set this later in the script only for the gamma launcher test
>>> run. While in there, I took the liberty of decrypting the script to make
>>> it more maintainable and more easily merged whenever we reconcile the
>>> unix ports into a single codebase. There is no change to the script
>>> output.
>>>
>>> Feedback welcome...
>>>
>>> WEBREV:
>>> http://cr.openjdk.java.net/~jmelvin/7125793/webrev.00
>>>
>>> TESTS RUN:
>>> JPRT 2011-12-31-061123.jmelvin.7125793
>>> local Mac OS X builds/tests
>>>
>>>
>>> Thanks and Happy New Year!
>>>
>>> Jim

From ysr1729 at gmail.com  Sun Jan  8 16:08:14 2012
From: ysr1729 at gmail.com (Srinivas Ramakrishna)
Date: Sun, 8 Jan 2012 16:08:14 -0800
Subject: JEP 132: More-prompt finalization
In-Reply-To: <4F0A257A.50104@oracle.com>
References: <20111222230542.48A451084@eggemoggin.niobe.net>
	<4EF3E44F.1060409@oracle.com> <4EF4A8A4.1060502@oracle.com>
	<4EF5C695.4060206@oracle.com> <4EF9F709.4030101@oracle.com>
	<4EFA2869.6070105@oracle.com>
	<712FF280-2C59-47A0-99DE-ECEAD5FCFB15@kodewerk.com>
	<4EFAC2C7.7060508@oracle.com>
	<32CBF5EE-6AB8-4888-895E-37E495ACCD74@kodewerk.com>
	<4EFB5583.5000005@oracle.com> <4F05D3A0.1050503@oracle.com>
	<08196B9F-49AC-4DBF-B0C7-76903539C217@kodewerk.com>
	<4F065F5D.6040605@oracle.com> <4F06CE20.10809@oracle.com>
	<4F06D7A7.6010603@oracle.com> <4F071C4B.8010002@oracle.com>
	<4F0A257A.50104@oracle.com>
Message-ID: <CABzyjynZs8pPJM-18VXiZS6QWf7e3UXo7dYk=53qthcG=Pffyg@mail.gmail.com>

Hi David --


On Sun, Jan 8, 2012 at 3:23 PM, David Holmes <david.holmes at oracle.com>wrote:

> On 7/01/2012 2:07 AM, Tony Printezis wrote:
>
>> David,
>>
>> On 01/06/2012 06:14 AM, David Holmes wrote:
>>
>>> Hi Tony,
>>>
>>> On 6/01/2012 8:34 PM, Tony Printezis wrote:
>>>
>>>> If each library could indicate to the GC whether the resource it manages
>>>> is running low or not, using the API I mentioned, the GC could do the
>>>> above automatically, behind the scenes, without the user having to do
>>>> anything else.
>>>>
>>>
>>> I'm not sure a library writer has the necessary information to do
>>> this. Seems to me that how an application uses a particular type
>>> determines the scarcity of the resource.
>>>
>>
>> I will be very surprised if the majority of application developers will
>> be willing to measure at what rate the application consumes certain
>> resources and update said measure when the application changes, load
>> increases, OS changes, hardware changes, etc. And, sure, I think there
>> will be many cases where the library writer has a good idea / can find
>> out the max number of instances of a particular resource (file
>> descriptors, sockets, etc.).
>>
>
> Perhaps, but the library writer may only have partial knowledge. The
> library can use reference counts to track how much of a resource it has
> handed out, and what has been handed back. It might know what the absolute
> limit for a resource is (via getrlimit etc). But can it know the absolute
> usage rate of a given resource? Can you query how many available file
> descriptors a process has left? Some resources will be used by native code
> outside the libs (including the VM).
>

The VM can probably keep track of its own descriptors, but you are right
that pure native code may make the final calculation noisy unless the
underlying
platform provides suitable query API's. Rates are easy (modulo the noise in
numbers) given that the VM/libs can keep track of the numbers it
understands. Don't think of it as (this is what the libraries cab do abd
this is what the VM can do, but rather as, this is what the VM+libs can
provide to the (pure) Java application.


>
> So at best this is a heuristic, so the library tracks the resource and at
> some threshold it invoke System.runFinalization. The GC itself doesn't make
> an informed decision because, as you said your self, the VM (and hence GC)
> knows nothing about the resource being tracked.
>

Yes, it would be a heuristic-driven policy (or set of policy choices), but
it's better than each application rolling
its own policy and infrastructure from scratch.

>
> But would we want to burden all users of these classes with the overhead
> of resource tracking?
>

The objective is to do the resource tracking from within the libraries
(+JVM), so the user wouldn't have to bother.
For example, by providing suitable high level API's at the library level
which would be wrappers
around specific native resources that would be subject to such tracking.

-- ramki


>
> Cheers,
> David
>
>
>  Tony
>>
>>  I can imagine something like:
>>>
>>> void setFinalizationLimit(Class<?> cls, int limit)
>>>
>>> so that GC runs finalization once a "reference count" reaches "limit".
>>> That limits the frequency of finalization, but the actual finalization
>>> cost may still be unacceptably high.
>>>
>>> Cheers,
>>> David
>>>
>>>  Tony
>>>>
>>>>  Afterall the key thing about GC is it relieves the programmer from
>>>>> having to manage object lifetimes, so if you don't know when the
>>>>> object is no longer used you don't know when to call close.
>>>>>
>>>>> David
>>>>>
>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120108/cbbc302d/attachment.html 

From david.holmes at oracle.com  Sun Jan  8 16:14:51 2012
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 09 Jan 2012 10:14:51 +1000
Subject: JEP 132: More-prompt finalization
In-Reply-To: <CABzyjynZs8pPJM-18VXiZS6QWf7e3UXo7dYk=53qthcG=Pffyg@mail.gmail.com>
References: <20111222230542.48A451084@eggemoggin.niobe.net>
	<4EF3E44F.1060409@oracle.com> <4EF4A8A4.1060502@oracle.com>
	<4EF5C695.4060206@oracle.com> <4EF9F709.4030101@oracle.com>
	<4EFA2869.6070105@oracle.com>
	<712FF280-2C59-47A0-99DE-ECEAD5FCFB15@kodewerk.com>
	<4EFAC2C7.7060508@oracle.com>
	<32CBF5EE-6AB8-4888-895E-37E495ACCD74@kodewerk.com>
	<4EFB5583.5000005@oracle.com> <4F05D3A0.1050503@oracle.com>
	<08196B9F-49AC-4DBF-B0C7-76903539C217@kodewerk.com>
	<4F065F5D.6040605@oracle.com> <4F06CE20.10809@oracle.com>
	<4F06D7A7.6010603@oracle.com> <4F071C4B.8010002@oracle.com>
	<4F0A257A.50104@oracle.com>
	<CABzyjynZs8pPJM-18VXiZS6QWf7e3UXo7dYk=53qthcG=Pffyg@mail.gmail.com>
Message-ID: <4F0A317B.6000702@oracle.com>

Hi Ramki,

On 9/01/2012 10:08 AM, Srinivas Ramakrishna wrote:
<snip>

>     So at best this is a heuristic, so the library tracks the resource
>     and at some threshold it invoke System.runFinalization. The GC
>     itself doesn't make an informed decision because, as you said your
>     self, the VM (and hence GC) knows nothing about the resource being
>     tracked.
>
>
> Yes, it would be a heuristic-driven policy (or set of policy choices),
> but it's better than each application rolling its own policy and infrastructure from scratch.

If it can be provided as an additional feature that applications only 
pay for if they need it.

>     But would we want to burden all users of these classes with the
>     overhead of resource tracking?
>
>
> The objective is to do the resource tracking from within the libraries
> (+JVM), so the user wouldn't have to bother.
> For example, by providing suitable high level API's at the library level
> which would be wrappers
> around specific native resources that would be subject to such tracking.

That's exactly the overhead I'm referring to. If these resources are 
always managed then all applications pay the price for the few that need it.

David
-----

> -- ramki
>
>
>     Cheers,
>     David
>
>
>         Tony
>
>             I can imagine something like:
>
>             void setFinalizationLimit(Class<?> cls, int limit)
>
>             so that GC runs finalization once a "reference count"
>             reaches "limit".
>             That limits the frequency of finalization, but the actual
>             finalization
>             cost may still be unacceptably high.
>
>             Cheers,
>             David
>
>                 Tony
>
>                     Afterall the key thing about GC is it relieves the
>                     programmer from
>                     having to manage object lifetimes, so if you don't
>                     know when the
>                     object is no longer used you don't know when to call
>                     close.
>
>                     David
>
>

From kirk at kodewerk.com  Sun Jan  8 22:31:11 2012
From: kirk at kodewerk.com (Kirk Pepperdine)
Date: Mon, 9 Jan 2012 07:31:11 +0100
Subject: JEP 132: More-prompt finalization
In-Reply-To: <4F0A257A.50104@oracle.com>
References: <20111222230542.48A451084@eggemoggin.niobe.net>
	<4EF3E44F.1060409@oracle.com> <4EF4A8A4.1060502@oracle.com>
	<4EF5C695.4060206@oracle.com> <4EF9F709.4030101@oracle.com>
	<4EFA2869.6070105@oracle.com>
	<712FF280-2C59-47A0-99DE-ECEAD5FCFB15@kodewerk.com>
	<4EFAC2C7.7060508@oracle.com>
	<32CBF5EE-6AB8-4888-895E-37E495ACCD74@kodewerk.com>
	<4EFB5583.5000005@oracle.com> <4F05D3A0.1050503@oracle.com>
	<08196B9F-49AC-4DBF-B0C7-76903539C217@kodewerk.com>
	<4F065F5D.6040605@oracle.com> <4F06CE20.10809@oracle.com>
	<4F06D7A7.6010603@oracle.com> <4F071C4B.8010002@oracle.com>
	<4F0A257A.50104@oracle.com>
Message-ID: <F3474FC3-A4EE-4A00-B09A-AD026B50D84D@kodewerk.com>

>> 
> 
> Perhaps, but the library writer may only have partial knowledge. The library can use reference counts to track how much of a resource it has handed out, and what has been handed back. It might know what the absolute limit for a resource is (via getrlimit etc). But can it know the absolute usage rate of a given resource? Can you query how many available file descriptors a process has left? Some resources will be used by native code outside the libs (including the VM).
> 
> So at best this is a heuristic, so the library tracks the resource and at some threshold it invoke System.runFinalization. The GC itself doesn't make an informed decision because, as you said your self, the VM (and hence GC) knows nothing about the resource being tracked.
> 
> But would we want to burden all users of these classes with the overhead of resource tracking?

Isn't this what a pool is for? I don't know but IMHO, this as going off very quickly. My feeling we're trying to hack in a work-around for an API to do have GC do something that it was never intended for it to do. Further more, if reference objects need to be treated specially then maybe the should be tracked specially so that a different mechanism can be triggered, one that doesn't interfere with GC. Crazy idea, what if reference objects were loosely connected to a special GC root so one could trace them on their own and if you could quickly determine that they weren't connected to anything else,....

Regards,
Kirk


From david.holmes at oracle.com  Sun Jan  8 22:56:56 2012
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 09 Jan 2012 16:56:56 +1000
Subject: JEP 132: More-prompt finalization
In-Reply-To: <F3474FC3-A4EE-4A00-B09A-AD026B50D84D@kodewerk.com>
References: <20111222230542.48A451084@eggemoggin.niobe.net>
	<4EF3E44F.1060409@oracle.com> <4EF4A8A4.1060502@oracle.com>
	<4EF5C695.4060206@oracle.com> <4EF9F709.4030101@oracle.com>
	<4EFA2869.6070105@oracle.com>
	<712FF280-2C59-47A0-99DE-ECEAD5FCFB15@kodewerk.com>
	<4EFAC2C7.7060508@oracle.com>
	<32CBF5EE-6AB8-4888-895E-37E495ACCD74@kodewerk.com>
	<4EFB5583.5000005@oracle.com> <4F05D3A0.1050503@oracle.com>
	<08196B9F-49AC-4DBF-B0C7-76903539C217@kodewerk.com>
	<4F065F5D.6040605@oracle.com> <4F06CE20.10809@oracle.com>
	<4F06D7A7.6010603@oracle.com> <4F071C4B.8010002@oracle.com>
	<4F0A257A.50104@oracle.com>
	<F3474FC3-A4EE-4A00-B09A-AD026B50D84D@kodewerk.com>
Message-ID: <4F0A8FB8.7080600@oracle.com>

On 9/01/2012 4:31 PM, Kirk Pepperdine wrote:
>>>
>>
>> Perhaps, but the library writer may only have partial knowledge. The library can use reference counts to track how much of a resource it has handed out, and what has been handed back. It might know what the absolute limit for a resource is (via getrlimit etc). But can it know the absolute usage rate of a given resource? Can you query how many available file descriptors a process has left? Some resources will be used by native code outside the libs (including the VM).
>>
>> So at best this is a heuristic, so the library tracks the resource and at some threshold it invoke System.runFinalization. The GC itself doesn't make an informed decision because, as you said your self, the VM (and hence GC) knows nothing about the resource being tracked.
>>
>> But would we want to burden all users of these classes with the overhead of resource tracking?
>
> Isn't this what a pool is for? I don't know but IMHO, this as going off very quickly. My feeling we're trying to hack in a work-around for an API to do have GC do something that it was never intended for it to do. Further more, if reference objects need to be treated specially then maybe the should be tracked specially so that a different mechanism can be triggered, one that doesn't interfere with GC. Crazy idea, what if reference objects were loosely connected to a special GC root so one could trace them on their own and if you could quickly determine that they weren't connected to anything else,....

I'm not quite sure what you are arguing for or against :) but isn't what 
you describe called "reference counting"?

You can imagine a class that uses a "finalizable" resource tracking 
construction and close() itself and also initiating some kind of cleanup 
mechanism. The cleanup has to involve the GC though because unless you 
have reference-counted classes (ala C++ auto_ptr - which Java can not 
support) there is no way to know when a resource is reclaimable.

I also think the there will be an overhead associated with such resource 
tracking, and I don't think everyone should have to pay the price for 
that if they don't need it.

Anyway I'm just repeating myself so I'll stop :)

Cheers,
David

> Regards,
> Kirk
>

From kirk at kodewerk.com  Sun Jan  8 23:17:00 2012
From: kirk at kodewerk.com (Kirk Pepperdine)
Date: Mon, 9 Jan 2012 08:17:00 +0100
Subject: JEP 132: More-prompt finalization
In-Reply-To: <4F0A8FB8.7080600@oracle.com>
References: <20111222230542.48A451084@eggemoggin.niobe.net>
	<4EF3E44F.1060409@oracle.com> <4EF4A8A4.1060502@oracle.com>
	<4EF5C695.4060206@oracle.com> <4EF9F709.4030101@oracle.com>
	<4EFA2869.6070105@oracle.com>
	<712FF280-2C59-47A0-99DE-ECEAD5FCFB15@kodewerk.com>
	<4EFAC2C7.7060508@oracle.com>
	<32CBF5EE-6AB8-4888-895E-37E495ACCD74@kodewerk.com>
	<4EFB5583.5000005@oracle.com> <4F05D3A0.1050503@oracle.com>
	<08196B9F-49AC-4DBF-B0C7-76903539C217@kodewerk.com>
	<4F065F5D.6040605@oracle.com> <4F06CE20.10809@oracle.com>
	<4F06D7A7.6010603@oracle.com> <4F071C4B.8010002@oracle.com>
	<4F0A257A.50104@oracle.com>
	<F3474FC3-A4EE-4A00-B09A-AD026B50D84D@kodewerk.com>
	<4F0A8FB8.7080600@oracle.com>
Message-ID: <8EC11CA6-B90E-466D-86F7-E37070A1EF88@kodewerk.com>


On 2012-01-09, at 7:56 AM, David Holmes wrote:

> On 9/01/2012 4:31 PM, Kirk Pepperdine wrote:
>>>> 
>>> 
>>> Perhaps, but the library writer may only have partial knowledge. The library can use reference counts to track how much of a resource it has handed out, and what has been handed back. It might know what the absolute limit for a resource is (via getrlimit etc). But can it know the absolute usage rate of a given resource? Can you query how many available file descriptors a process has left? Some resources will be used by native code outside the libs (including the VM).
>>> 
>>> So at best this is a heuristic, so the library tracks the resource and at some threshold it invoke System.runFinalization. The GC itself doesn't make an informed decision because, as you said your self, the VM (and hence GC) knows nothing about the resource being tracked.
>>> 
>>> But would we want to burden all users of these classes with the overhead of resource tracking?
>> 
>> Isn't this what a pool is for? I don't know but IMHO, this as going off very quickly. My feeling we're trying to hack in a work-around for an API to do have GC do something that it was never intended for it to do. Further more, if reference objects need to be treated specially then maybe the should be tracked specially so that a different mechanism can be triggered, one that doesn't interfere with GC. Crazy idea, what if reference objects were loosely connected to a special GC root so one could trace them on their own and if you could quickly determine that they weren't connected to anything else,....
> 
> I'm not quite sure what you are arguing for or against :) but isn't what you describe called "reference counting"?

no no no, definitely not reference counting, I was thinking partial mark sweep focused on the finalizable objects with no guarantees that it would find objects ready to be finalized. One that would find the reference objects that were very obviously collectable.

> 
> I also think the there will be an overhead associated with such resource tracking, and I don't think everyone should have to pay the price for that if they don't need it.

Well, I think the whole idea is insane which is the *only* reason I've engaged in this conversation. I much prefer just being an observer and cherry picking the diagnostic/troubleshooting postings.

Regards,
Kirk


From ysr1729 at gmail.com  Sun Jan  8 23:27:54 2012
From: ysr1729 at gmail.com (Srinivas Ramakrishna)
Date: Sun, 8 Jan 2012 23:27:54 -0800
Subject: JEP 132: More-prompt finalization
In-Reply-To: <4F0A317B.6000702@oracle.com>
References: <20111222230542.48A451084@eggemoggin.niobe.net>
	<4EF3E44F.1060409@oracle.com> <4EF4A8A4.1060502@oracle.com>
	<4EF5C695.4060206@oracle.com> <4EF9F709.4030101@oracle.com>
	<4EFA2869.6070105@oracle.com>
	<712FF280-2C59-47A0-99DE-ECEAD5FCFB15@kodewerk.com>
	<4EFAC2C7.7060508@oracle.com>
	<32CBF5EE-6AB8-4888-895E-37E495ACCD74@kodewerk.com>
	<4EFB5583.5000005@oracle.com> <4F05D3A0.1050503@oracle.com>
	<08196B9F-49AC-4DBF-B0C7-76903539C217@kodewerk.com>
	<4F065F5D.6040605@oracle.com> <4F06CE20.10809@oracle.com>
	<4F06D7A7.6010603@oracle.com> <4F071C4B.8010002@oracle.com>
	<4F0A257A.50104@oracle.com>
	<CABzyjynZs8pPJM-18VXiZS6QWf7e3UXo7dYk=53qthcG=Pffyg@mail.gmail.com>
	<4F0A317B.6000702@oracle.com>
Message-ID: <CABzyjykfwxuOs1S4NJ5ncXpj+eWALS4+MC0=6r8mQscjTZR8Pw@mail.gmail.com>

Ah, I see what you meant. Yes, there'd probably be some (theoretical)
overhead from such tracking,
if there are multiple applications within a single JVM some of which wanted
to use these APIs
and some did not, then the ones that do not care for such tracking or
policies would pay
the price (and besides one would have to resolve composition issues in such
cases). But
if one confined oneself for the moment to the simpler case of a single
application per JVM,
then if that application chose not to use such tracking, then there would
be little to no overhead
for it.

Even in the case of multiple applications, my guess is that the overhead
for other applications
could be managed. I suspect that, compared to the JNI transition overheads,
resource tracking
overheads per se (and not the implementation of specific finalization
triggering policies thereof)
could, i think, be engineered to be very small.

If, on the other hand, you are referring to the overhead of the work done
for prompt finalization
related to specific resources that one of a set of applications running on
a JVM cares about
and that others do not, that is a legitimate concern, but seems to me to be
no different from these disparate
applications sharing the same JVM heap which is subject to a global
collection which stops
the threads of each application (or of concurrent collection which imposes
an overhead on
all the applications sharing that JVM, or indeed the hardware), if you know
what i am getting at...

-- ramki

On Sun, Jan 8, 2012 at 4:14 PM, David Holmes <david.holmes at oracle.com>wrote:

> Hi Ramki,
>
> On 9/01/2012 10:08 AM, Srinivas Ramakrishna wrote:
> <snip>
>
>
>     So at best this is a heuristic, so the library tracks the resource
>>    and at some threshold it invoke System.runFinalization. The GC
>>    itself doesn't make an informed decision because, as you said your
>>    self, the VM (and hence GC) knows nothing about the resource being
>>    tracked.
>>
>>
>> Yes, it would be a heuristic-driven policy (or set of policy choices),
>> but it's better than each application rolling its own policy and
>> infrastructure from scratch.
>>
>
> If it can be provided as an additional feature that applications only pay
> for if they need it.
>
>
>     But would we want to burden all users of these classes with the
>>    overhead of resource tracking?
>>
>>
>> The objective is to do the resource tracking from within the libraries
>> (+JVM), so the user wouldn't have to bother.
>> For example, by providing suitable high level API's at the library level
>> which would be wrappers
>> around specific native resources that would be subject to such tracking.
>>
>
> That's exactly the overhead I'm referring to. If these resources are
> always managed then all applications pay the price for the few that need it.
>
> David
> -----
>
>
>  -- ramki
>>
>>
>>    Cheers,
>>    David
>>
>>
>>        Tony
>>
>>            I can imagine something like:
>>
>>            void setFinalizationLimit(Class<?> cls, int limit)
>>
>>            so that GC runs finalization once a "reference count"
>>            reaches "limit".
>>            That limits the frequency of finalization, but the actual
>>            finalization
>>            cost may still be unacceptably high.
>>
>>            Cheers,
>>            David
>>
>>                Tony
>>
>>                    Afterall the key thing about GC is it relieves the
>>                    programmer from
>>                    having to manage object lifetimes, so if you don't
>>                    know when the
>>                    object is no longer used you don't know when to call
>>                    close.
>>
>>                    David
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120108/eb15576d/attachment.html 

From david.holmes at oracle.com  Sun Jan  8 23:44:35 2012
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 09 Jan 2012 17:44:35 +1000
Subject: JEP 132: More-prompt finalization
In-Reply-To: <CABzyjykfwxuOs1S4NJ5ncXpj+eWALS4+MC0=6r8mQscjTZR8Pw@mail.gmail.com>
References: <20111222230542.48A451084@eggemoggin.niobe.net>
	<4EF3E44F.1060409@oracle.com> <4EF4A8A4.1060502@oracle.com>
	<4EF5C695.4060206@oracle.com> <4EF9F709.4030101@oracle.com>
	<4EFA2869.6070105@oracle.com>
	<712FF280-2C59-47A0-99DE-ECEAD5FCFB15@kodewerk.com>
	<4EFAC2C7.7060508@oracle.com>
	<32CBF5EE-6AB8-4888-895E-37E495ACCD74@kodewerk.com>
	<4EFB5583.5000005@oracle.com> <4F05D3A0.1050503@oracle.com>
	<08196B9F-49AC-4DBF-B0C7-76903539C217@kodewerk.com>
	<4F065F5D.6040605@oracle.com> <4F06CE20.10809@oracle.com>
	<4F06D7A7.6010603@oracle.com> <4F071C4B.8010002@oracle.com>
	<4F0A257A.50104@oracle.com>
	<CABzyjynZs8pPJM-18VXiZS6QWf7e3UXo7dYk=53qthcG=Pffyg@mail.gmail.com>
	<4F0A317B.6000702@oracle.com>
	<CABzyjykfwxuOs1S4NJ5ncXpj+eWALS4+MC0=6r8mQscjTZR8Pw@mail.gmail.com>
Message-ID: <4F0A9AE3.7000602@oracle.com>

On 9/01/2012 5:27 PM, Srinivas Ramakrishna wrote:
> Ah, I see what you meant. Yes, there'd probably be some (theoretical)
> overhead from such tracking,

I'm not considering multiple apps per VM I'm just thinking of a 
programmer using java.io.* or java.net.* classes which have been 
"enhanced" to support resource tracking.

If the proposal involves new wrapper classes that an application has to 
opt-in to use then that is fine.

But I think this has gone off-track somewhat - finding alternatives to 
finalization. The JEP is more about speeding up finalization for those 
apps that (for better or worse) rely on it. I think there are two main 
aspects of this:
- adding more threads to process finalizers and/or reference objects
- adding an API to request and run finalization etc (more efficiently 
that doing: System.gc(); System.gc(); System.runFinalization() )

Although we "all" know "finalizers are evil", they exist and are 
(mis-)used by some apps and cause some issues.

Cheers,
David

> if there are multiple applications within a single JVM some of which
> wanted to use these APIs
> and some did not, then the ones that do not care for such tracking or
> policies would pay
> the price (and besides one would have to resolve composition issues in
> such cases). But
> if one confined oneself for the moment to the simpler case of a single
> application per JVM,
> then if that application chose not to use such tracking, then there
> would be little to no overhead
> for it.
>
> Even in the case of multiple applications, my guess is that the overhead
> for other applications
> could be managed. I suspect that, compared to the JNI transition
> overheads, resource tracking
> overheads per se (and not the implementation of specific finalization
> triggering policies thereof)
> could, i think, be engineered to be very small.
>
> If, on the other hand, you are referring to the overhead of the work
> done for prompt finalization
> related to specific resources that one of a set of applications running
> on a JVM cares about
> and that others do not, that is a legitimate concern, but seems to me to
> be no different from these disparate
> applications sharing the same JVM heap which is subject to a global
> collection which stops
> the threads of each application (or of concurrent collection which
> imposes an overhead on
> all the applications sharing that JVM, or indeed the hardware), if you
> know what i am getting at...
>
> -- ramki
>
> On Sun, Jan 8, 2012 at 4:14 PM, David Holmes <david.holmes at oracle.com
> <mailto:david.holmes at oracle.com>> wrote:
>
>     Hi Ramki,
>
>     On 9/01/2012 10:08 AM, Srinivas Ramakrishna wrote:
>     <snip>
>
>
>             So at best this is a heuristic, so the library tracks the
>         resource
>             and at some threshold it invoke System.runFinalization. The GC
>             itself doesn't make an informed decision because, as you
>         said your
>             self, the VM (and hence GC) knows nothing about the resource
>         being
>             tracked.
>
>
>         Yes, it would be a heuristic-driven policy (or set of policy
>         choices),
>         but it's better than each application rolling its own policy and
>         infrastructure from scratch.
>
>
>     If it can be provided as an additional feature that applications
>     only pay for if they need it.
>
>
>             But would we want to burden all users of these classes with the
>             overhead of resource tracking?
>
>
>         The objective is to do the resource tracking from within the
>         libraries
>         (+JVM), so the user wouldn't have to bother.
>         For example, by providing suitable high level API's at the
>         library level
>         which would be wrappers
>         around specific native resources that would be subject to such
>         tracking.
>
>
>     That's exactly the overhead I'm referring to. If these resources are
>     always managed then all applications pay the price for the few that
>     need it.
>
>     David
>     -----
>
>
>         -- ramki
>
>
>             Cheers,
>             David
>
>
>                 Tony
>
>                     I can imagine something like:
>
>                     void setFinalizationLimit(Class<?> cls, int limit)
>
>                     so that GC runs finalization once a "reference count"
>                     reaches "limit".
>                     That limits the frequency of finalization, but the
>         actual
>                     finalization
>                     cost may still be unacceptably high.
>
>                     Cheers,
>                     David
>
>                         Tony
>
>                             Afterall the key thing about GC is it
>         relieves the
>                             programmer from
>                             having to manage object lifetimes, so if you
>         don't
>                             know when the
>                             object is no longer used you don't know when
>         to call
>                             close.
>
>                             David
>
>
>

From kirk at kodewerk.com  Mon Jan  9 04:21:49 2012
From: kirk at kodewerk.com (Kirk Pepperdine)
Date: Mon, 9 Jan 2012 13:21:49 +0100
Subject: Very long young gc pause (ParNew with CMS)
In-Reply-To: <4F0AD41E.6000109@java4.info>
References: <4F0ACAAC.8020103@java4.info>
	<997C208B-7920-446E-8A90-A0D6B752996F@kodewerk.com>
	<4F0AD41E.6000109@java4.info>
Message-ID: <5C4AFCB8-764B-4BA1-B1BA-075416FE1A58@kodewerk.com>

There is so much stuff in this gc log it's hard to see the real problem... As as much data as there is in the flags, we're missing the trigger for the CMS cycle

That said, these settings will cause premature promotion in most applications. This will put more stress on CMS

>>> -XX:SurvivorRatio=8 \
>>> -XX:TargetSurvivorRatio=90 \


The initiating occupancy fraction is very high IMHO and that will cause more difficulties. In most cases, it's better to have a lower threshold as this will give more head room to sort out fragmentation. Also, new size pinned to 256m is also causing a lot of premature promotion. And -Xmx==-Xms.... you can read one of my blog postings on that subject. I do find it interesting that you have a 28G heap with new size set to 256m.

IMHO, too  much work has been done setting flags. I'd be included to turn most of them back to default settings. I'd turn off most of the data being collected in the GC log as I think it's mostly for debugging GC and it's not very helpful for sizing analysis.

Last point,  the benefit of CMS is shorter pause times and if you're not seeing that, you might be better off with the parallel collector.


Note for the guys I have having the RMI/full gc discussion with...
>>> -Dsun.rmi.dgc.server.gcInterval=9223372036854775807 \
>>> -Dsun.rmi.dgc.client.gcInterval=9223372036854775807 \

You've gotta love these settings... ;And I'll have to say they are quite typical.  8^)

Regards,
Kirk


On 2012-01-09, at 12:48 PM, Florian Binder wrote:

> Hi Kirk,
> 
> I have attached the log since 8:45 am today to this mail.
> Every full hour the application is creating a lot of objects which are kept in memory for a long time. This process may take a few minutes (should be done in less than 10 min) and may explain the premature promotion. But not all young gcs, which are taking so long, are during this process.
> 
> Regards,
> Flo
> 
> 
> Am 09.01.2012 12:23, schrieb Kirk Pepperdine:
>> Can you post the complete log? I see a premature promotion event but you'd need the entire log to see if this is pathological case or just an anomaly.
>> 
>> Regards,
>> Kirk
>> 
>> On 2012-01-09, at 12:08 PM, Florian Binder wrote:
>> 
>>> Hi everybody,
>>> 
>>> I am using CMS (with ParNew) gc and have very long (>  6 seconds) young
>>> gc pauses.
>>> As you can see in the log below the old-gen-heap consists of one large
>>> block, the new Size has 256m, it uses 13 worker threads and it has to
>>> copy 27505761 words (~210mb) directly from eden to old gen.
>>> I have seen that this problem occurs only after about one week of
>>> uptime. Even thought we make a full (compacting) gc every night.
>>> Since real-time>  user-time I assume it might be a synchronization
>>> problem. Can this be true?
>>> 
>>> Do you have any Ideas how I can speed up this gcs?
>>> 
>>> Please let me know, if you need more informations.
>>> 
>>> Thank you,
>>> Flo
>>> 
>>> 
>>> ##### java -version #####
>>> java version "1.6.0_29"
>>> Java(TM) SE Runtime Environment (build 1.6.0_29-b11)
>>> Java HotSpot(TM) 64-Bit Server VM (build 20.4-b02, mixed mode)
>>> 
>>> ##### The startup parameters: #####
>>> -Xms28G -Xmx28G
>>> -XX:+UseConcMarkSweepGC \
>>> -XX:CMSMaxAbortablePrecleanTime=10000 \
>>> -XX:SurvivorRatio=8 \
>>> -XX:TargetSurvivorRatio=90 \
>>> -XX:MaxTenuringThreshold=31 \
>>> -XX:CMSInitiatingOccupancyFraction=80 \
>>> -XX:NewSize=256M \
>>> 
>>> -verbose:gc \
>>> -XX:+PrintFlagsFinal \
>>> -XX:PrintFLSStatistics=1 \
>>> -XX:+PrintGCDetails \
>>> -XX:+PrintGCDateStamps \
>>> -XX:-TraceClassUnloading \
>>> -XX:+PrintGCApplicationConcurrentTime \
>>> -XX:+PrintGCApplicationStoppedTime \
>>> -XX:+PrintTenuringDistribution \
>>> -XX:+CMSClassUnloadingEnabled \
>>> -Dsun.rmi.dgc.server.gcInterval=9223372036854775807 \
>>> -Dsun.rmi.dgc.client.gcInterval=9223372036854775807 \
>>> 
>>> -Djava.awt.headless=true
>>> 
>>> ##### From the out-file (as of +PrintFlagsFinal): #####
>>> ParallelGCThreads                         = 13
>>> 
>>> ##### The gc.log-excerpt: #####
>>> Application time: 20,0617700 seconds
>>> 2011-12-22T12:02:03.289+0100: [GC Before GC:
>>> Statistics for BinaryTreeDictionary:
>>> ------------------------------------
>>> Total Free Space: 1183290963
>>> Max   Chunk Size: 1183290963
>>> Number of Blocks: 1
>>> Av.  Block  Size: 1183290963
>>> Tree      Height: 1
>>> Before GC:
>>> Statistics for BinaryTreeDictionary:
>>> ------------------------------------
>>> Total Free Space: 0
>>> Max   Chunk Size: 0
>>> Number of Blocks: 0
>>> Tree      Height: 0
>>> [ParNew
>>> Desired survivor size 25480392 bytes, new threshold 1 (max 31)
>>> - age   1:   28260160 bytes,   28260160 total
>>> : 249216K->27648K(249216K), 6,1808130 secs]
>>> 20061765K->20056210K(29332480K)After GC:
>>> Statistics for BinaryTreeDictionary:
>>> ------------------------------------
>>> Total Free Space: 1155785202
>>> Max   Chunk Size: 1155785202
>>> Number of Blocks: 1
>>> Av.  Block  Size: 1155785202
>>> Tree      Height: 1
>>> After GC:
>>> Statistics for BinaryTreeDictionary:
>>> ------------------------------------
>>> Total Free Space: 0
>>> Max   Chunk Size: 0
>>> Number of Blocks: 0
>>> Tree      Height: 0
>>> , 6,1809440 secs] [Times: user=3,08 sys=0,51, real=6,18 secs]
>>> Total time for which application threads were stopped: 6,1818730 seconds
>>> _______________________________________________
>>> hotspot-gc-use mailing list
>>> hotspot-gc-use at openjdk.java.net
>>> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
> 
> <gc.log>


From daniel.daugherty at oracle.com  Mon Jan  9 15:37:24 2012
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Mon, 09 Jan 2012 16:37:24 -0700
Subject: RFR (S): 7125793: MAC: test_gamma should always work
In-Reply-To: <4F087516.40505@oracle.com>
References: <4EFECA5D.6010905@oracle.com> <4F034B51.3070609@oracle.com>
	<4F087516.40505@oracle.com>
Message-ID: <4F0B7A34.5070406@oracle.com>

On 1/7/12 9:38 AM, James Melvin wrote:
> WEBREV:
>   http://cr.openjdk.java.net/~jmelvin/7125793/webrev.01

make/bsd/Makefile
     No comments.

make/bsd/makefiles/buildtree.make
     No comments.

make/bsd/makefiles/defs.make
     Thanks for fixing this one for BSD platforms.

make/bsd/makefiles/launcher.make
     line 60: typo: 'inadvertenly' -> 'inadvertently'

     Sorry I missed this in my first review, but the addition
     of '-framework CoreFoundation' to LFLAGS_LAUNCHER is
     probably MacOS X specific. I think:

         ifeq ($(OS_VENDOR), Darwin)
         else
         endif

     will work in launcher.make also.

make/bsd/makefiles/vm.make
     No comments.

src/os/bsd/vm/os_bsd.cpp
     line 2544: typo: 'overriden' -> 'overridden'
     line 2588: typo: 'overriden' -> 'overridden'

     Looks like old code line 2576 depended on the 'hotspot'
     symlink to refer to either 'client' or 'server' or whatever
     JVM you wanted to run. I'm fairly certain that the 'hotspot'
     symlink was retired; I'm just not sure when.

src/os/posix/launcher/java_md.c
     No comments.

Dan



From mikael.gerdin at oracle.com  Tue Jan 10 11:27:06 2012
From: mikael.gerdin at oracle.com (Mikael Gerdin)
Date: Tue, 10 Jan 2012 20:27:06 +0100
Subject: Review request: White box testing API for HotSpot
In-Reply-To: <4ED50287.3070102@oracle.com>
References: <4ED50287.3070102@oracle.com>
Message-ID: <4F0C910A.5070205@oracle.com>

Hi all

Back from vacations now with an updated version of the webrev based on 
the feedback received in this thread.
Changes include:
* removed install target from makefiles
* renamed flag form EnableWhiteBoxAPI to remove redundant Enable
* installs wb.jar into jre/lib and made -XX:+WhiteBoxAPI add wb.jar to 
the boot class path from inside the VM.

http://cr.openjdk.java.net/~mgerdin/wbapi/webrev.2/

Thanks
Mikael Gerdin

On 2011-11-29 17:04, Mikael Gerdin wrote:
> Hi
>
> I've been working on a white box testing API for HotSpot in order to
> allow for improved precision in vm testing.
>
> The basic idea is to open up the possibility for tests written in Java
> to call native methods which query or poke the vm in some way.
>
> The API is accessible by using the class sun/hotspot/WhiteBox which is
> not intended to be available in public builds.
> In order to allow the WhiteBox class access to the VM the
> registerNatives function is linked to JVM_RegisterWhiteBoxMethods. That
> function then links all the implementation functions using normal JNI
> RegisterNatives.
>
> The API is not meant to be used by end users for any intent or purpose
> and as such it is both guarded by "-XX:+UnlockDiagnosticVMOptions
> -XX:+EnableWhiteboxAPI" and the fact that the class files will not be
> present in an end user build of a JDK.
> If the VM crashes after this API has been accessed a note will be
> written in the hs_err file to signal that the API has been used.
>
> Webrev:
> http://cr.openjdk.java.net/~stefank/mgerdin/wbapi.0/webrev/
> (thanks to stefank for hosting my webrev :)
>
> CR:
> I'll file a CR tomorrow.
>
> Change comments:
>
> make/jprt.properties
>
> Add a test target to make sure that the API is available on all
> supported platforms
>
> make/**
>
> Makefile changes to build the class sun/hotspot/WhiteBox, put it in a
> JAR file and copy it to the jre/lib/endorsed directory in the export
> targets.
> The BSD makefile changes are not tested since I don't have access to any
> BSD/OSX machine to test them on.
>
> src/share/vm/prims/nativeLookup.cpp
>
> Special-case the method sun/hotspot/WhiteBox/registerNatives and link it
> to JVM_RegisterWhiteBoxMethods
>
> src/share/vm/prims/whitebox.*
>
> The implementation of the white box API. The actual API functions are
> only examples of what we want to be able to do using the API.
>
> src/share/vm/runtime/globals.hpp
>
> Add the command line flag
>
> src/share/vm/utilities/vmError.cpp
>
> Print a message in hs_err files when white box API has been used.
>
> test/Makefile
>
> Add a makefile test target for the white box API test
>
> test/sanity/wbapi.java
>
> JTreg test to ensure that the API works.
>
>
> Thanks
> /Mikael Gerdin


From james.melvin at oracle.com  Tue Jan 10 13:05:13 2012
From: james.melvin at oracle.com (James Melvin)
Date: Tue, 10 Jan 2012 16:05:13 -0500
Subject: RFR (S): 7125793: MAC: test_gamma should always work
In-Reply-To: <4F0B7A34.5070406@oracle.com>
References: <4EFECA5D.6010905@oracle.com> <4F034B51.3070609@oracle.com>
	<4F087516.40505@oracle.com> <4F0B7A34.5070406@oracle.com>
Message-ID: <4F0CA809.7020001@oracle.com>

Hi Dan,

Final webrev to reflect your comments...

http://cr.openjdk.java.net/~jmelvin/7125793/webrev.02

Minor changes this round:

   make/bsd/makefiles/buildtree.make   # Fail gracefully on Apple BOOTDIR
   make/bsd/makefiles/launcher.make    # Link with framework only on Mac
   src/os/bsd/vm/os_bsd.cpp            # Just spelling fix

Lastly, I wanted to reply to John Coomes comments earlier about the
test_gamma script simplification. Although I also value economy of
expression, in this case I think the use of more advanced shell
constructs increases the time for fresh eyes to decipher. Given
performance and such is not an issue, I'd prefer to keep the simpler
version I'm proposing with this change on Mac OS X, to make it easier on
future maintenance. This should be a model for the other platforms when
we reconcile. I've attached the before and after copies should there be
further comments on the simplified short script.

Thanks,

Jim


On 1/9/12 6:37 PM, Daniel D. Daugherty wrote:
> On 1/7/12 9:38 AM, James Melvin wrote:
>> WEBREV:
>> http://cr.openjdk.java.net/~jmelvin/7125793/webrev.01
>
> make/bsd/Makefile
> No comments.
>
> make/bsd/makefiles/buildtree.make
> No comments.
>
> make/bsd/makefiles/defs.make
> Thanks for fixing this one for BSD platforms.
>
> make/bsd/makefiles/launcher.make
> line 60: typo: 'inadvertenly' -> 'inadvertently'
>
> Sorry I missed this in my first review, but the addition
> of '-framework CoreFoundation' to LFLAGS_LAUNCHER is
> probably MacOS X specific. I think:
>
> ifeq ($(OS_VENDOR), Darwin)
> else
> endif
>
> will work in launcher.make also.
>
> make/bsd/makefiles/vm.make
> No comments.
>
> src/os/bsd/vm/os_bsd.cpp
> line 2544: typo: 'overriden' -> 'overridden'
> line 2588: typo: 'overriden' -> 'overridden'
>
> Looks like old code line 2576 depended on the 'hotspot'
> symlink to refer to either 'client' or 'server' or whatever
> JVM you wanted to run. I'm fairly certain that the 'hotspot'
> symlink was retired; I'm just not sure when.
>
> src/os/posix/launcher/java_md.c
> No comments.
>
> Dan
>
>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: test_gamma.before
Url: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120110/6a912938/test_gamma.before 
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: test_gamma.after
Url: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120110/6a912938/test_gamma.after 

From daniel.daugherty at oracle.com  Tue Jan 10 13:27:17 2012
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Tue, 10 Jan 2012 14:27:17 -0700
Subject: RFR (S): 7125793: MAC: test_gamma should always work
In-Reply-To: <4F0CA809.7020001@oracle.com>
References: <4EFECA5D.6010905@oracle.com> <4F034B51.3070609@oracle.com>
	<4F087516.40505@oracle.com> <4F0B7A34.5070406@oracle.com>
	<4F0CA809.7020001@oracle.com>
Message-ID: <4F0CAD35.40809@oracle.com>

On 1/10/12 2:05 PM, James Melvin wrote:
> Hi Dan,
>
> Final webrev to reflect your comments...
>
> http://cr.openjdk.java.net/~jmelvin/7125793/webrev.02
>
> Minor changes this round:
>
>   make/bsd/makefiles/buildtree.make   # Fail gracefully on Apple BOOTDIR
>   make/bsd/makefiles/launcher.make    # Link with framework only on Mac
>   src/os/bsd/vm/os_bsd.cpp            # Just spelling fix

Thumbs up on the current version.

To close the loop on one of my earlier comments:

$ ls -l binaries/solsparc/jre/lib/sparc/hotspot
lrwxrwxrwx   1 nobody   nobody         6 Apr  1  2009 
binaries/solsparc/jre/lib/sparc/hotspot -> client

This symlink exists in JDK1.3.1, but I didn't find it in JDK1.4.0.

> Lastly, I wanted to reply to John Coomes comments earlier about the
> test_gamma script simplification. Although I also value economy of
> expression, in this case I think the use of more advanced shell
> constructs increases the time for fresh eyes to decipher. Given
> performance and such is not an issue, I'd prefer to keep the simpler
> version I'm proposing with this change on Mac OS X, to make it easier on
> future maintenance. This should be a model for the other platforms when
> we reconcile. I've attached the before and after copies should there be
> further comments on the simplified short script.

The attachments didn't come through because your e-mail went
through the OpenJDK list servers.

Just to be clear: I vote for the newer version. It is more
straight forward and has comments.

Dan


>
> Thanks,
>
> Jim
>
>
> On 1/9/12 6:37 PM, Daniel D. Daugherty wrote:
>> On 1/7/12 9:38 AM, James Melvin wrote:
>>> WEBREV:
>>> http://cr.openjdk.java.net/~jmelvin/7125793/webrev.01
>>
>> make/bsd/Makefile
>> No comments.
>>
>> make/bsd/makefiles/buildtree.make
>> No comments.
>>
>> make/bsd/makefiles/defs.make
>> Thanks for fixing this one for BSD platforms.
>>
>> make/bsd/makefiles/launcher.make
>> line 60: typo: 'inadvertenly' -> 'inadvertently'
>>
>> Sorry I missed this in my first review, but the addition
>> of '-framework CoreFoundation' to LFLAGS_LAUNCHER is
>> probably MacOS X specific. I think:
>>
>> ifeq ($(OS_VENDOR), Darwin)
>> else
>> endif
>>
>> will work in launcher.make also.
>>
>> make/bsd/makefiles/vm.make
>> No comments.
>>
>> src/os/bsd/vm/os_bsd.cpp
>> line 2544: typo: 'overriden' -> 'overridden'
>> line 2588: typo: 'overriden' -> 'overridden'
>>
>> Looks like old code line 2576 depended on the 'hotspot'
>> symlink to refer to either 'client' or 'server' or whatever
>> JVM you wanted to run. I'm fairly certain that the 'hotspot'
>> symlink was retired; I'm just not sure when.
>>
>> src/os/posix/launcher/java_md.c
>> No comments.
>>
>> Dan
>>
>>

From John.Coomes at oracle.com  Wed Jan 11 12:01:04 2012
From: John.Coomes at oracle.com (John Coomes)
Date: Wed, 11 Jan 2012 12:01:04 -0800
Subject: RFR (S): 7125793: MAC: test_gamma should always work
In-Reply-To: <4F0CA809.7020001@oracle.com>
References: <4EFECA5D.6010905@oracle.com> <4F034B51.3070609@oracle.com>
	<4F087516.40505@oracle.com> <4F0B7A34.5070406@oracle.com>
	<4F0CA809.7020001@oracle.com>
Message-ID: <20237.60032.473316.761490@oracle.com>

James Melvin (james.melvin at oracle.com) wrote:
> Hi Dan,
> 
> Final webrev to reflect your comments...
> 
> http://cr.openjdk.java.net/~jmelvin/7125793/webrev.02
> 
> Minor changes this round:
> 
>    make/bsd/makefiles/buildtree.make   # Fail gracefully on Apple BOOTDIR
>    make/bsd/makefiles/launcher.make    # Link with framework only on Mac
>    src/os/bsd/vm/os_bsd.cpp            # Just spelling fix
> 
> Lastly, I wanted to reply to John Coomes comments earlier about the
> test_gamma script simplification. Although I also value economy of
> expression, in this case I think the use of more advanced shell
> constructs increases the time for fresh eyes to decipher. Given
> performance and such is not an issue, I'd prefer to keep the simpler
> version I'm proposing with this change on Mac OS X, to make it easier on
> future maintenance. This should be a model for the other platforms when
> we reconcile. I've attached the before and after copies should there be
> further comments on the simplified short script.

As mentioned before, I have problems with you're proposed change
beyond just brevity.  First, it makes reconciling changes with the
other platforms more difficult, so do all (if you get agreement) or do
none.  Second, any semantic changes are lost in the reformatting
noise.  Do the reformatting separately from the semantic change.

And the expanded version is waaaay to bloated.  I'm open to some code
expansion, but the whitespace decorators around comment blocks for
each trivial statement is too much.  It reminds me of beginning cobol.

-John

> On 1/9/12 6:37 PM, Daniel D. Daugherty wrote:
> > On 1/7/12 9:38 AM, James Melvin wrote:
> >> WEBREV:
> >> http://cr.openjdk.java.net/~jmelvin/7125793/webrev.01
> >
> > make/bsd/Makefile
> > No comments.
> >
> > make/bsd/makefiles/buildtree.make
> > No comments.
> >
> > make/bsd/makefiles/defs.make
> > Thanks for fixing this one for BSD platforms.
> >
> > make/bsd/makefiles/launcher.make
> > line 60: typo: 'inadvertenly' -> 'inadvertently'
> >
> > Sorry I missed this in my first review, but the addition
> > of '-framework CoreFoundation' to LFLAGS_LAUNCHER is
> > probably MacOS X specific. I think:
> >
> > ifeq ($(OS_VENDOR), Darwin)
> > else
> > endif
> >
> > will work in launcher.make also.
> >
> > make/bsd/makefiles/vm.make
> > No comments.
> >
> > src/os/bsd/vm/os_bsd.cpp
> > line 2544: typo: 'overriden' -> 'overridden'
> > line 2588: typo: 'overriden' -> 'overridden'
> >
> > Looks like old code line 2576 depended on the 'hotspot'
> > symlink to refer to either 'client' or 'server' or whatever
> > JVM you wanted to run. I'm fairly certain that the 'hotspot'
> > symlink was retired; I'm just not sure when.
> >
> > src/os/posix/launcher/java_md.c
> > No comments.
> >
> > Dan
> >
> >
> 
> ----------------------------------------------------------------------
> #!/bin/sh
> # Generated by /Users/jmelvin/dev/testing/make/bsd/makefiles/buildtree.make
> . ./env.sh
> if [ "" != "" ]; then { echo Cross compiling for ARCH , skipping gamma run.; exit 0; }; fi
> if [ -z $JAVA_HOME ]; then { echo JAVA_HOME must be set to run this test.; exit 0; }; fi
> if ! ${JAVA_HOME}/bin/java -d32 -fullversion 2>&1 > /dev/null
> then
>   echo JAVA_HOME must point to 32bit JDK.; exit 0;
> fi
> rm -f Queens.class
> ${JAVA_HOME}/bin/javac -d . /Users/jmelvin/dev/testing/make/test/Queens.java
> [ -f gamma_g ] && { gamma=gamma_g; }
> ./${gamma:-gamma} -Xbatch -showversion Queens < /dev/null
> exit 0
> 
> ----------------------------------------------------------------------
> #!/bin/sh
> 
> # Generated by /Users/jmelvin/dev/7125793/make/bsd/makefiles/buildtree.make
> 
> #
> # Include environment settings for gamma run
> #
> 
> . ./env.sh
> 
> 
> #
> # Do not run gamma test for cross compiles
> #
> 
> if [ -n "" ]; then 
>   echo Cross compiling for ARCH , skipping gamma run.
>   exit 0
> fi
> 
> 
> #
> # Make sure JAVA_HOME is set as it is required for gamma
> #
> 
> if [ -z "${JAVA_HOME}" ]; then 
>   echo JAVA_HOME must be set to run this test.
>   exit 0
> fi
> 
> 
> #
> # Report JAVA_HOME version to be used for the test
> #
> 
> FULL_VERSION="`${JAVA_HOME}/bin/java -d64 -fullversion`"
> if [ $? -ne 0 ]; then 
>   echo JAVA_HOME must point to a 64-bit OpenJDK.
>   exit 0
> fi
> echo "${FULL_VERSION}" | awk '{printf("%s",$0);}'
> 
> 
> #
> # Use gamma_g if it exists
> #
> 
> GAMMA_PROG=gamma
> if [ -f gamma_g ]; then 
>   GAMMA_PROG=gamma_g
> fi
> 
> 
> #
> # Ensure architecture for gamma and JAVA_HOME is the same.
> # NOTE: gamma assumes the OpenJDK directory layout.
> #
> 
> GAMMA_ARCH="`file ${GAMMA_PROG} | awk '{print $NF}'`"
> JVM_LIB="${JAVA_HOME}/jre/lib/libjava.dylib"
> if [ ! -f ${JVM_LIB} ]; then
>   JVM_LIB="${JAVA_HOME}/jre/lib/amd64/libjava.dylib"
> fi
> if [ ! -f ${JVM_LIB} ] || [ -z "`file ${JVM_LIB} | grep ${GAMMA_ARCH}`" ]; then 
>   echo JAVA_HOME must point to a 64-bit OpenJDK.
>   exit 0
> fi
> 
> 
> #
> # Compile Queens program for test
> #
> 
> rm -f Queens.class
> ${JAVA_HOME}/bin/javac -d . /Users/jmelvin/dev/7125793/make/test/Queens.java
> 
> 
> #
> # Set library path solely for gamma launcher test run
> #
> 
> LD_LIBRARY_PATH=.:${JAVA_HOME}/jre/lib/amd64/native_threads:${JAVA_HOME}/jre/lib/amd64:
> DYLD_LIBRARY_PATH=.:${JAVA_HOME}/jre/lib/native_threads:${JAVA_HOME}/jre/lib:${JAVA_HOME}/jre/lib/amd64/native_threads:${JAVA_HOME}/jre/lib/amd64:
> export DYLD_LIBRARY_PATH
> 
> 
> #
> # Use the gamma launcher and JAVA_HOME to run the test
> #
> 
> ./${GAMMA_PROG} -Xbatch -showversion Queens < /dev/null

From david.holmes at oracle.com  Wed Jan 11 23:26:08 2012
From: david.holmes at oracle.com (David Holmes)
Date: Thu, 12 Jan 2012 17:26:08 +1000
Subject: Review request: White box testing API for HotSpot
In-Reply-To: <4F0C910A.5070205@oracle.com>
References: <4ED50287.3070102@oracle.com> <4F0C910A.5070205@oracle.com>
Message-ID: <4F0E8B10.3030600@oracle.com>

Hi Mikael,

This seems to address my concerns with the previous implementation of 
this. Some further minor comments:

In whitebox.cpp:

  181         if (result != 0) {
  182           WhiteBox::set_used();

shouldn't the above be testing for == 0 ?

---

wbapi.java: normal Java naming style is to use camel-case for class 
names. Though as WB is itself an acronym I'd be okay with WBApi. In fact 
I'd be happy with anything other than initial lower-case :)

---

test/Makefile: does wbapitest need to be added to the phoney list?

---

Cheers,
David
-----


On 11/01/2012 5:27 AM, Mikael Gerdin wrote:
> Hi all
>
> Back from vacations now with an updated version of the webrev based on
> the feedback received in this thread.
> Changes include:
> * removed install target from makefiles
> * renamed flag form EnableWhiteBoxAPI to remove redundant Enable
> * installs wb.jar into jre/lib and made -XX:+WhiteBoxAPI add wb.jar to
> the boot class path from inside the VM.
>
> http://cr.openjdk.java.net/~mgerdin/wbapi/webrev.2/
>
> Thanks
> Mikael Gerdin
>
> On 2011-11-29 17:04, Mikael Gerdin wrote:
>> Hi
>>
>> I've been working on a white box testing API for HotSpot in order to
>> allow for improved precision in vm testing.
>>
>> The basic idea is to open up the possibility for tests written in Java
>> to call native methods which query or poke the vm in some way.
>>
>> The API is accessible by using the class sun/hotspot/WhiteBox which is
>> not intended to be available in public builds.
>> In order to allow the WhiteBox class access to the VM the
>> registerNatives function is linked to JVM_RegisterWhiteBoxMethods. That
>> function then links all the implementation functions using normal JNI
>> RegisterNatives.
>>
>> The API is not meant to be used by end users for any intent or purpose
>> and as such it is both guarded by "-XX:+UnlockDiagnosticVMOptions
>> -XX:+EnableWhiteboxAPI" and the fact that the class files will not be
>> present in an end user build of a JDK.
>> If the VM crashes after this API has been accessed a note will be
>> written in the hs_err file to signal that the API has been used.
>>
>> Webrev:
>> http://cr.openjdk.java.net/~stefank/mgerdin/wbapi.0/webrev/
>> (thanks to stefank for hosting my webrev :)
>>
>> CR:
>> I'll file a CR tomorrow.
>>
>> Change comments:
>>
>> make/jprt.properties
>>
>> Add a test target to make sure that the API is available on all
>> supported platforms
>>
>> make/**
>>
>> Makefile changes to build the class sun/hotspot/WhiteBox, put it in a
>> JAR file and copy it to the jre/lib/endorsed directory in the export
>> targets.
>> The BSD makefile changes are not tested since I don't have access to any
>> BSD/OSX machine to test them on.
>>
>> src/share/vm/prims/nativeLookup.cpp
>>
>> Special-case the method sun/hotspot/WhiteBox/registerNatives and link it
>> to JVM_RegisterWhiteBoxMethods
>>
>> src/share/vm/prims/whitebox.*
>>
>> The implementation of the white box API. The actual API functions are
>> only examples of what we want to be able to do using the API.
>>
>> src/share/vm/runtime/globals.hpp
>>
>> Add the command line flag
>>
>> src/share/vm/utilities/vmError.cpp
>>
>> Print a message in hs_err files when white box API has been used.
>>
>> test/Makefile
>>
>> Add a makefile test target for the white box API test
>>
>> test/sanity/wbapi.java
>>
>> JTreg test to ensure that the API works.
>>
>>
>> Thanks
>> /Mikael Gerdin
>

From aph at redhat.com  Thu Jan 12 03:53:39 2012
From: aph at redhat.com (Andrew Haley)
Date: Thu, 12 Jan 2012 11:53:39 +0000
Subject: Question re safepoints and monitors
Message-ID: <4F0EC9C3.7000909@redhat.com>

This is re HS20 on Zero, but it might apply to any HotSpot port AFAIK.

We invoke Thread.stop on a thread.  To do this, we need to wait for
the thread to reach a safepoint.  So, thread->set_pending_exception()
is called, and SafepointSynchronize::_state is set to
SafepointSynchronize::_synchronizing.

The thread needs to acquire a lock, so it enters
InterpreterRuntime::monitorenter().  This does the safepoint check.
monitorenter() is marked IRT_ENTRY_NO_ASYNC, so it does not check for
pending async exceptions.  Control returns from monitorenter and the
thread continues to execute Java.  The async exception is not
processed.

As far as I can see, there is still a pending exception for this
thread but it won't be processed until something else causes the
thread to move to a safepoint.  Is that right?

Thanks,
Andrew.

From david.holmes at oracle.com  Thu Jan 12 04:39:35 2012
From: david.holmes at oracle.com (David Holmes)
Date: Thu, 12 Jan 2012 22:39:35 +1000
Subject: Question re safepoints and monitors
In-Reply-To: <4F0EC9C3.7000909@redhat.com>
References: <4F0EC9C3.7000909@redhat.com>
Message-ID: <4F0ED487.8060504@oracle.com>

Andrew,

On 12/01/2012 9:53 PM, Andrew Haley wrote:
> This is re HS20 on Zero, but it might apply to any HotSpot port AFAIK.
>
> We invoke Thread.stop on a thread.  To do this, we need to wait for
> the thread to reach a safepoint.  So, thread->set_pending_exception()
> is called, and SafepointSynchronize::_state is set to
> SafepointSynchronize::_synchronizing.
>
> The thread needs to acquire a lock, so it enters
> InterpreterRuntime::monitorenter().  This does the safepoint check.
> monitorenter() is marked IRT_ENTRY_NO_ASYNC, so it does not check for
> pending async exceptions.  Control returns from monitorenter and the
> thread continues to execute Java.  The async exception is not
> processed.

Yes it is - the interpreter executes the monitorenter() call inside a 
CALL_VM macro which checks for the pending exception.

David
-----

> As far as I can see, there is still a pending exception for this
> thread but it won't be processed until something else causes the
> thread to move to a safepoint.  Is that right?
>
> Thanks,
> Andrew.

From aph at redhat.com  Thu Jan 12 05:01:00 2012
From: aph at redhat.com (Andrew Haley)
Date: Thu, 12 Jan 2012 13:01:00 +0000
Subject: Question re safepoints and monitors
In-Reply-To: <4F0ED487.8060504@oracle.com>
References: <4F0EC9C3.7000909@redhat.com> <4F0ED487.8060504@oracle.com>
Message-ID: <4F0ED98C.4060007@redhat.com>

On 01/12/2012 12:39 PM, David Holmes wrote:
> Yes it is - the interpreter executes the monitorenter() call inside a 
> CALL_VM macro which checks for the pending exception.

Thanks, but I still don't quite get it.

The interrupting thread calls send_thread_stop(), which sets
thread._pending_async_exception:

  void set_pending_async_exception(oop e) {
    _pending_async_exception = e;
    _special_runtime_exit_condition = _async_exception;
    set_has_async_exception();
  }

As far as I can see the code that actually reads
thread._pending_async_exception and sets thread._pending_exception is
JavaThread::check_and_handle_async_exceptions():

  // Check for pending async. exception
  if (_pending_async_exception != NULL) {
    // Only overwrite an already pending exception, if it is not a threadDeath.
    if (!has_pending_exception() || !pending_exception()->is_a(SystemDictionary::ThreadDeath_klass())) {

      // We cannot call Exceptions::_throw(...) here because we cannot block
      set_pending_exception(_pending_async_exception, __FILE__, __LINE__);

but check_and_handle_async_exceptions() is not called in the case of a
monitor operation.  We need something to copy _pending_async_exception
to _pending_exception, but I can't see when that would ever be called.

Andrew.

From daniel.daugherty at oracle.com  Thu Jan 12 05:17:46 2012
From: daniel.daugherty at oracle.com (daniel.daugherty at oracle.com)
Date: Thu, 12 Jan 2012 13:17:46 +0000
Subject: hg: hsx/hotspot-main/hotspot: 7129240: backout fix for 7102776 until
	7128770 is resolved
Message-ID: <20120112131754.38E694793A@hg.openjdk.java.net>

Changeset: 8f8b94305aff
Author:    dcubed
Date:      2012-01-11 19:54 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/8f8b94305aff

7129240: backout fix for 7102776 until 7128770 is resolved
Reviewed-by: phh, bobv, coleenp, dcubed
Contributed-by: Jiangli Zhou <jiangli.zhou at oracle.com>

! agent/src/share/classes/sun/jvm/hotspot/oops/InstanceKlass.java
! src/share/vm/code/dependencies.cpp
! src/share/vm/oops/instanceKlass.hpp
! src/share/vm/oops/instanceKlassKlass.cpp
! src/share/vm/runtime/vmStructs.cpp


From david.holmes at oracle.com  Thu Jan 12 05:21:27 2012
From: david.holmes at oracle.com (David Holmes)
Date: Thu, 12 Jan 2012 23:21:27 +1000
Subject: Question re safepoints and monitors
In-Reply-To: <4F0ED98C.4060007@redhat.com>
References: <4F0EC9C3.7000909@redhat.com> <4F0ED487.8060504@oracle.com>
	<4F0ED98C.4060007@redhat.com>
Message-ID: <4F0EDE57.1010307@oracle.com>

On 12/01/2012 11:01 PM, Andrew Haley wrote:
> On 01/12/2012 12:39 PM, David Holmes wrote:
>> Yes it is - the interpreter executes the monitorenter() call inside a
>> CALL_VM macro which checks for the pending exception.
>
> Thanks, but I still don't quite get it.
>
> The interrupting thread calls send_thread_stop(), which sets
> thread._pending_async_exception:
>
>    void set_pending_async_exception(oop e) {
>      _pending_async_exception = e;
>      _special_runtime_exit_condition = _async_exception;
>      set_has_async_exception();
>    }
>
> As far as I can see the code that actually reads
> thread._pending_async_exception and sets thread._pending_exception is
> JavaThread::check_and_handle_async_exceptions():
>
>    // Check for pending async. exception
>    if (_pending_async_exception != NULL) {
>      // Only overwrite an already pending exception, if it is not a threadDeath.
>      if (!has_pending_exception() || !pending_exception()->is_a(SystemDictionary::ThreadDeath_klass())) {
>
>        // We cannot call Exceptions::_throw(...) here because we cannot block
>        set_pending_exception(_pending_async_exception, __FILE__, __LINE__);
>
> but check_and_handle_async_exceptions() is not called in the case of a
> monitor operation.  We need something to copy _pending_async_exception
> to _pending_exception, but I can't see when that would ever be called.

Yes I think you are correct. The exception will remain as a pending 
async exception until the next thread state transition that causes 
handle_special_runtime_exit_condition to be called (or there may be some 
other path that forces an async check - like the next safepoint).

There is a lot of history to the way Thread.stop operates ...

David
-----


> Andrew.

From aph at redhat.com  Thu Jan 12 07:26:34 2012
From: aph at redhat.com (Andrew Haley)
Date: Thu, 12 Jan 2012 15:26:34 +0000
Subject: Question re safepoints and monitors
In-Reply-To: <4F0EDE57.1010307@oracle.com>
References: <4F0EC9C3.7000909@redhat.com> <4F0ED487.8060504@oracle.com>
	<4F0ED98C.4060007@redhat.com> <4F0EDE57.1010307@oracle.com>
Message-ID: <4F0EFBAA.2000304@redhat.com>

On 01/12/2012 01:21 PM, David Holmes wrote:
> Yes I think you are correct. The exception will remain as a pending 
> async exception until the next thread state transition that causes 
> handle_special_runtime_exit_condition to be called (or there may be some 
> other path that forces an async check - like the next safepoint).
> 
> There is a lot of history to the way Thread.stop operates ...

I can well imagine!

Thanks very much,
Andrew.



From mikael.gerdin at oracle.com  Thu Jan 12 08:20:32 2012
From: mikael.gerdin at oracle.com (Mikael Gerdin)
Date: Thu, 12 Jan 2012 17:20:32 +0100
Subject: Review request: White box testing API for HotSpot
In-Reply-To: <4F0E8B10.3030600@oracle.com>
References: <4ED50287.3070102@oracle.com> <4F0C910A.5070205@oracle.com>
	<4F0E8B10.3030600@oracle.com>
Message-ID: <4F0F0850.8050402@oracle.com>

Hi David,


On 2012-01-12 08:26, David Holmes wrote:
> Hi Mikael,
>
> This seems to address my concerns with the previous implementation of
> this. Some further minor comments:
>
> In whitebox.cpp:
>
> 181 if (result != 0) {
> 182 WhiteBox::set_used();
>
> shouldn't the above be testing for == 0 ?

Yes it should, fixed.

>
> ---
>
> wbapi.java: normal Java naming style is to use camel-case for class
> names. Though as WB is itself an acronym I'd be okay with WBApi. In fact
> I'd be happy with anything other than initial lower-case :)

Many of our existing tests have lower-case names so I guess I thought 
that was some sort of convention, it does not really matter to me.
WBApi it is then.

>
> ---
>
> test/Makefile: does wbapitest need to be added to the phoney list?

Yes, fixed.

New webrev at:
http://cr.openjdk.java.net/~mgerdin/wbapi/webrev.3/
Incremental at:
http://cr.openjdk.java.net/~mgerdin/wbapi/webrev.2-to-3/webrev/

/Mikael

>
> ---
>
> Cheers,
> David
> -----
>
>
> On 11/01/2012 5:27 AM, Mikael Gerdin wrote:
>> Hi all
>>
>> Back from vacations now with an updated version of the webrev based on
>> the feedback received in this thread.
>> Changes include:
>> * removed install target from makefiles
>> * renamed flag form EnableWhiteBoxAPI to remove redundant Enable
>> * installs wb.jar into jre/lib and made -XX:+WhiteBoxAPI add wb.jar to
>> the boot class path from inside the VM.
>>
>> http://cr.openjdk.java.net/~mgerdin/wbapi/webrev.2/
>>
>> Thanks
>> Mikael Gerdin
>>
>> On 2011-11-29 17:04, Mikael Gerdin wrote:
>>> Hi
>>>
>>> I've been working on a white box testing API for HotSpot in order to
>>> allow for improved precision in vm testing.
>>>
>>> The basic idea is to open up the possibility for tests written in Java
>>> to call native methods which query or poke the vm in some way.
>>>
>>> The API is accessible by using the class sun/hotspot/WhiteBox which is
>>> not intended to be available in public builds.
>>> In order to allow the WhiteBox class access to the VM the
>>> registerNatives function is linked to JVM_RegisterWhiteBoxMethods. That
>>> function then links all the implementation functions using normal JNI
>>> RegisterNatives.
>>>
>>> The API is not meant to be used by end users for any intent or purpose
>>> and as such it is both guarded by "-XX:+UnlockDiagnosticVMOptions
>>> -XX:+EnableWhiteboxAPI" and the fact that the class files will not be
>>> present in an end user build of a JDK.
>>> If the VM crashes after this API has been accessed a note will be
>>> written in the hs_err file to signal that the API has been used.
>>>
>>> Webrev:
>>> http://cr.openjdk.java.net/~stefank/mgerdin/wbapi.0/webrev/
>>> (thanks to stefank for hosting my webrev :)
>>>
>>> CR:
>>> I'll file a CR tomorrow.
>>>
>>> Change comments:
>>>
>>> make/jprt.properties
>>>
>>> Add a test target to make sure that the API is available on all
>>> supported platforms
>>>
>>> make/**
>>>
>>> Makefile changes to build the class sun/hotspot/WhiteBox, put it in a
>>> JAR file and copy it to the jre/lib/endorsed directory in the export
>>> targets.
>>> The BSD makefile changes are not tested since I don't have access to any
>>> BSD/OSX machine to test them on.
>>>
>>> src/share/vm/prims/nativeLookup.cpp
>>>
>>> Special-case the method sun/hotspot/WhiteBox/registerNatives and link it
>>> to JVM_RegisterWhiteBoxMethods
>>>
>>> src/share/vm/prims/whitebox.*
>>>
>>> The implementation of the white box API. The actual API functions are
>>> only examples of what we want to be able to do using the API.
>>>
>>> src/share/vm/runtime/globals.hpp
>>>
>>> Add the command line flag
>>>
>>> src/share/vm/utilities/vmError.cpp
>>>
>>> Print a message in hs_err files when white box API has been used.
>>>
>>> test/Makefile
>>>
>>> Add a makefile test target for the white box API test
>>>
>>> test/sanity/wbapi.java
>>>
>>> JTreg test to ensure that the API works.
>>>
>>>
>>> Thanks
>>> /Mikael Gerdin
>>

From david.holmes at oracle.com  Thu Jan 12 15:00:21 2012
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 13 Jan 2012 09:00:21 +1000
Subject: Review request: White box testing API for HotSpot
In-Reply-To: <4F0F0850.8050402@oracle.com>
References: <4ED50287.3070102@oracle.com> <4F0C910A.5070205@oracle.com>
	<4F0E8B10.3030600@oracle.com> <4F0F0850.8050402@oracle.com>
Message-ID: <4F0F6605.5080900@oracle.com>

Hi Mikael,

On 13/01/2012 2:20 AM, Mikael Gerdin wrote:
>> wbapi.java: normal Java naming style is to use camel-case for class
>> names. Though as WB is itself an acronym I'd be okay with WBApi. In fact
>> I'd be happy with anything other than initial lower-case :)
>
> Many of our existing tests have lower-case names so I guess I thought
> that was some sort of convention, it does not really matter to me.

I think those tests must have been written by C programers ;-)

> WBApi it is then.

Thanks.There is a slight typo in that the file is WBapi.java not WBApi.java

David
-----

>
>>
>> ---
>>
>> test/Makefile: does wbapitest need to be added to the phoney list?
>
> Yes, fixed.
>
> New webrev at:
> http://cr.openjdk.java.net/~mgerdin/wbapi/webrev.3/
> Incremental at:
> http://cr.openjdk.java.net/~mgerdin/wbapi/webrev.2-to-3/webrev/
>
> /Mikael
>
>>
>> ---
>>
>> Cheers,
>> David
>> -----
>>
>>
>> On 11/01/2012 5:27 AM, Mikael Gerdin wrote:
>>> Hi all
>>>
>>> Back from vacations now with an updated version of the webrev based on
>>> the feedback received in this thread.
>>> Changes include:
>>> * removed install target from makefiles
>>> * renamed flag form EnableWhiteBoxAPI to remove redundant Enable
>>> * installs wb.jar into jre/lib and made -XX:+WhiteBoxAPI add wb.jar to
>>> the boot class path from inside the VM.
>>>
>>> http://cr.openjdk.java.net/~mgerdin/wbapi/webrev.2/
>>>
>>> Thanks
>>> Mikael Gerdin
>>>
>>> On 2011-11-29 17:04, Mikael Gerdin wrote:
>>>> Hi
>>>>
>>>> I've been working on a white box testing API for HotSpot in order to
>>>> allow for improved precision in vm testing.
>>>>
>>>> The basic idea is to open up the possibility for tests written in Java
>>>> to call native methods which query or poke the vm in some way.
>>>>
>>>> The API is accessible by using the class sun/hotspot/WhiteBox which is
>>>> not intended to be available in public builds.
>>>> In order to allow the WhiteBox class access to the VM the
>>>> registerNatives function is linked to JVM_RegisterWhiteBoxMethods. That
>>>> function then links all the implementation functions using normal JNI
>>>> RegisterNatives.
>>>>
>>>> The API is not meant to be used by end users for any intent or purpose
>>>> and as such it is both guarded by "-XX:+UnlockDiagnosticVMOptions
>>>> -XX:+EnableWhiteboxAPI" and the fact that the class files will not be
>>>> present in an end user build of a JDK.
>>>> If the VM crashes after this API has been accessed a note will be
>>>> written in the hs_err file to signal that the API has been used.
>>>>
>>>> Webrev:
>>>> http://cr.openjdk.java.net/~stefank/mgerdin/wbapi.0/webrev/
>>>> (thanks to stefank for hosting my webrev :)
>>>>
>>>> CR:
>>>> I'll file a CR tomorrow.
>>>>
>>>> Change comments:
>>>>
>>>> make/jprt.properties
>>>>
>>>> Add a test target to make sure that the API is available on all
>>>> supported platforms
>>>>
>>>> make/**
>>>>
>>>> Makefile changes to build the class sun/hotspot/WhiteBox, put it in a
>>>> JAR file and copy it to the jre/lib/endorsed directory in the export
>>>> targets.
>>>> The BSD makefile changes are not tested since I don't have access to
>>>> any
>>>> BSD/OSX machine to test them on.
>>>>
>>>> src/share/vm/prims/nativeLookup.cpp
>>>>
>>>> Special-case the method sun/hotspot/WhiteBox/registerNatives and
>>>> link it
>>>> to JVM_RegisterWhiteBoxMethods
>>>>
>>>> src/share/vm/prims/whitebox.*
>>>>
>>>> The implementation of the white box API. The actual API functions are
>>>> only examples of what we want to be able to do using the API.
>>>>
>>>> src/share/vm/runtime/globals.hpp
>>>>
>>>> Add the command line flag
>>>>
>>>> src/share/vm/utilities/vmError.cpp
>>>>
>>>> Print a message in hs_err files when white box API has been used.
>>>>
>>>> test/Makefile
>>>>
>>>> Add a makefile test target for the white box API test
>>>>
>>>> test/sanity/wbapi.java
>>>>
>>>> JTreg test to ensure that the API works.
>>>>
>>>>
>>>> Thanks
>>>> /Mikael Gerdin
>>>

From keith.mcguigan at oracle.com  Thu Jan 12 18:38:57 2012
From: keith.mcguigan at oracle.com (keith.mcguigan at oracle.com)
Date: Fri, 13 Jan 2012 02:38:57 +0000
Subject: hg: hsx/hotspot-main/hotspot: 3 new changesets
Message-ID: <20120113023907.D45364794E@hg.openjdk.java.net>

Changeset: 4f25538b54c9
Author:    fparain
Date:      2012-01-09 10:27 +0100
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/4f25538b54c9

7120511: Add diagnostic commands
Reviewed-by: acorn, phh, dcubed, sspitsyn

! src/share/vm/classfile/vmSymbols.hpp
! src/share/vm/runtime/arguments.cpp
! src/share/vm/runtime/globals.cpp
! src/share/vm/runtime/globals.hpp
! src/share/vm/runtime/init.cpp
! src/share/vm/services/attachListener.cpp
! src/share/vm/services/diagnosticCommand.cpp
! src/share/vm/services/diagnosticCommand.hpp
! src/share/vm/services/diagnosticFramework.cpp
! src/share/vm/services/diagnosticFramework.hpp
! src/share/vm/services/management.cpp

Changeset: 865e0817f32b
Author:    kamg
Date:      2012-01-10 15:47 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/865e0817f32b

Merge

! src/share/vm/runtime/arguments.cpp
! src/share/vm/runtime/globals.hpp

Changeset: efdf6985a3a2
Author:    kamg
Date:      2012-01-12 09:59 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/efdf6985a3a2

Merge



From john.r.rose at oracle.com  Fri Jan 13 03:30:19 2012
From: john.r.rose at oracle.com (john.r.rose at oracle.com)
Date: Fri, 13 Jan 2012 11:30:19 +0000
Subject: hg: hsx/hotspot-main/hotspot: 5 new changesets
Message-ID: <20120113113032.725D34795C@hg.openjdk.java.net>

Changeset: 5da7201222d5
Author:    kvn
Date:      2012-01-07 10:39 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/5da7201222d5

7110824: ctw/jarfiles/GUI3rdParty_jar/ob_mask_DateField crashes VM
Summary: Change yank_if_dead() to recursive method to remove all dead inputs.
Reviewed-by: never

! src/cpu/sparc/vm/sparc.ad
! src/share/vm/opto/chaitin.hpp
! src/share/vm/opto/postaloc.cpp

Changeset: e9a5e0a812c8
Author:    kvn
Date:      2012-01-07 13:26 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/e9a5e0a812c8

7125896: Eliminate nested locks
Summary: Nested locks elimination done before lock nodes expansion by looking for outer locks of the same object.
Reviewed-by: never, twisti

! src/cpu/sparc/vm/sparc.ad
! src/cpu/x86/vm/x86_32.ad
! src/cpu/x86/vm/x86_64.ad
! src/share/vm/ci/ciTypeFlow.cpp
! src/share/vm/ci/ciTypeFlow.hpp
! src/share/vm/opto/c2_globals.hpp
! src/share/vm/opto/callnode.cpp
! src/share/vm/opto/callnode.hpp
! src/share/vm/opto/escape.cpp
! src/share/vm/opto/locknode.cpp
! src/share/vm/opto/locknode.hpp
! src/share/vm/opto/macro.cpp
! src/share/vm/opto/macro.hpp
! src/share/vm/opto/output.cpp
! src/share/vm/opto/parse1.cpp
! src/share/vm/runtime/arguments.cpp
! src/share/vm/runtime/deoptimization.cpp

Changeset: 35acf8f0a2e4
Author:    kvn
Date:      2012-01-10 18:05 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/35acf8f0a2e4

7128352: assert(obj_node == obj) failed
Summary: Compare uncasted object nodes.
Reviewed-by: never

! src/share/vm/opto/callnode.cpp
! src/share/vm/opto/cfgnode.cpp
! src/share/vm/opto/library_call.cpp
! src/share/vm/opto/locknode.cpp
! src/share/vm/opto/macro.cpp
! src/share/vm/opto/memnode.cpp
! src/share/vm/opto/node.cpp
! src/share/vm/opto/node.hpp
! src/share/vm/opto/phaseX.hpp
! src/share/vm/opto/subnode.cpp
! test/compiler/7116216/StackOverflow.java

Changeset: c8d8e124380c
Author:    kvn
Date:      2012-01-12 12:28 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/c8d8e124380c

7064302: JDK7 build 147 crashed after testing my java 6-compiled web app
Summary: Don't split CMove node if it's control edge is different from split region.
Reviewed-by: never

! src/share/vm/opto/loopnode.cpp
! src/share/vm/opto/loopnode.hpp
! src/share/vm/opto/loopopts.cpp

Changeset: 31a5b9aad4bc
Author:    jrose
Date:      2012-01-13 00:27 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/31a5b9aad4bc

Merge

! src/share/vm/runtime/arguments.cpp


From Dmitry.Samersoff at oracle.com  Fri Jan 13 06:34:57 2012
From: Dmitry.Samersoff at oracle.com (Dmitry Samersoff)
Date: Fri, 13 Jan 2012 18:34:57 +0400
Subject: JEP 132: More-prompt finalization
In-Reply-To: <4F06CE3A.2040101@oracle.com>
References: <20111222230542.48A451084@eggemoggin.niobe.net>
	<4EF3E44F.1060409@oracle.com> <4EF4A8A4.1060502@oracle.com>
	<4EF5C695.4060206@oracle.com> <4EF9F709.4030101@oracle.com>
	<4EFA2869.6070105@oracle.com>
	<712FF280-2C59-47A0-99DE-ECEAD5FCFB15@kodewerk.com>
	<4EFAC2C7.7060508@oracle.com>
	<32CBF5EE-6AB8-4888-895E-37E495ACCD74@kodewerk.com>
	<4EFB5583.5000005@oracle.com> <4F05D3A0.1050503@oracle.com>
	<CAHjP37EQKQNfDg5s2y0RFFSxtdSqgiRnXVfq2QJpP9NyPaMxaw@mail.gmail.com>
	<4F06CE3A.2040101@oracle.com>
Message-ID: <4F104111.5090809@oracle.com>

Tony,


On 2012-01-06 14:34, Tony Printezis wrote:
> Vitaly,
> 
> Sure, but if the GC detects that the load is low it doesn't know whether
> the load will remain low for 5 ms or 5 hours (and it's impossible to
> know, maybe not even the application knows). I can already imagine the
> bug reports: a spike suddenly happened in the market and the JVM was
> "locked up" for several seconds!!!

To be more practical, lets consider a vital case - a shop selling
something to particular region. (e.g city's best pizza)

It has clear visible pick hours (evening) and clear visible spare time
(early morning)

So cu would like to have as much free resources as possible at a pick
hours.  Solution widely used today - create a cluster, then restart
nodes one by one at a spare time.

What we(java) can do for such case:

   1 (simple, doable today). Give a way to setup ergonomic profile e.g.
something like -XX:PreferredGCTime. During this time VM sets GC
parameters to collect as much as possible at the cost of higher
load by GC thread.

  2. (advanced) Collect heap usage statistics and calculate a
probability of low load time.

-Dmitry

-- 
Dmitry Samersoff
Java Hotspot development team, SPB04
* There will come soft rains ...

From tony.printezis at oracle.com  Fri Jan 13 08:47:27 2012
From: tony.printezis at oracle.com (Tony Printezis)
Date: Fri, 13 Jan 2012 11:47:27 -0500
Subject: JEP 132: More-prompt finalization
In-Reply-To: <4F104111.5090809@oracle.com>
References: <20111222230542.48A451084@eggemoggin.niobe.net>
	<4EF3E44F.1060409@oracle.com> <4EF4A8A4.1060502@oracle.com>
	<4EF5C695.4060206@oracle.com> <4EF9F709.4030101@oracle.com>
	<4EFA2869.6070105@oracle.com>
	<712FF280-2C59-47A0-99DE-ECEAD5FCFB15@kodewerk.com>
	<4EFAC2C7.7060508@oracle.com>
	<32CBF5EE-6AB8-4888-895E-37E495ACCD74@kodewerk.com>
	<4EFB5583.5000005@oracle.com> <4F05D3A0.1050503@oracle.com>
	<CAHjP37EQKQNfDg5s2y0RFFSxtdSqgiRnXVfq2QJpP9NyPaMxaw@mail.gmail.com>
	<4F06CE3A.2040101@oracle.com> <4F104111.5090809@oracle.com>
Message-ID: <4F10601F.9010208@oracle.com>

Dmitry,

I totally appreciate that a lot of apps have non-trivial periods of idle 
time. However, you assume that there will be enough resources available 
to handle the peak time load so that we can wait to reclaim them when 
the load drops. I just don't think this is always a reasonable 
assumption. Depending what it's doing, an app can consume a lot of 
resources during peak time, If peak time lasts for a few hours (as it's 
probably the case in your pizza delivery example below).

FWIW,

Tony

On 01/13/2012 09:34 AM, Dmitry Samersoff wrote:
> Tony,
>
>
> On 2012-01-06 14:34, Tony Printezis wrote:
>> Vitaly,
>>
>> Sure, but if the GC detects that the load is low it doesn't know whether
>> the load will remain low for 5 ms or 5 hours (and it's impossible to
>> know, maybe not even the application knows). I can already imagine the
>> bug reports: a spike suddenly happened in the market and the JVM was
>> "locked up" for several seconds!!!
> To be more practical, lets consider a vital case - a shop selling
> something to particular region. (e.g city's best pizza)
>
> It has clear visible pick hours (evening) and clear visible spare time
> (early morning)
>
> So cu would like to have as much free resources as possible at a pick
> hours.  Solution widely used today - create a cluster, then restart
> nodes one by one at a spare time.
>
> What we(java) can do for such case:
>
>     1 (simple, doable today). Give a way to setup ergonomic profile e.g.
> something like -XX:PreferredGCTime. During this time VM sets GC
> parameters to collect as much as possible at the cost of higher
> load by GC thread.
>
>    2. (advanced) Collect heap usage statistics and calculate a
> probability of low load time.
>
> -Dmitry
>

From bengt.rutisson at oracle.com  Fri Jan 13 09:02:16 2012
From: bengt.rutisson at oracle.com (bengt.rutisson at oracle.com)
Date: Fri, 13 Jan 2012 17:02:16 +0000
Subject: hg: hsx/hotspot-main/hotspot: 9 new changesets
Message-ID: <20120113170236.16C4847962@hg.openjdk.java.net>

Changeset: bacb651cf5bf
Author:    tonyp
Date:      2012-01-05 05:54 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/bacb651cf5bf

7113006: G1: excessive ergo output when an evac failure happens
Summary: Introduce a flag that is set when a heap expansion attempt during a GC fails so that we do not consantly attempt to expand the heap when it's going to fail anyway. This not only prevents the excessive ergo output (which is generated when a region allocation fails) but also avoids excessive and ultimately unsuccessful expansion attempts.
Reviewed-by: jmasa, johnc

! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp
! src/share/vm/gc_implementation/g1/g1CollectedHeap.hpp

Changeset: 5fd354a959c5
Author:    jmasa
Date:      2012-01-05 21:21 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/5fd354a959c5

Merge


Changeset: 023652e49ac0
Author:    johnc
Date:      2011-12-23 11:14 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/023652e49ac0

7121496: G1: do the per-region evacuation failure handling work in parallel
Summary: Parallelize the removal of self forwarding pointers etc. by wrapping in a HeapRegion closure, which is then wrapped inside an AbstractGangTask.
Reviewed-by: tonyp, iveresov

! src/share/vm/gc_implementation/g1/concurrentMark.cpp
! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp
! src/share/vm/gc_implementation/g1/g1CollectedHeap.hpp
+ src/share/vm/gc_implementation/g1/g1EvacFailure.hpp
! src/share/vm/gc_implementation/g1/heapRegion.hpp

Changeset: 02838862dec8
Author:    tonyp
Date:      2012-01-07 00:43 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/02838862dec8

7121623: G1: always be able to reliably calculate the length of a forwarded chunked array
Summary: Store the "next chunk start index" in the length field of the to-space object, instead of the from-space object, so that we can always reliably read the size of all from-space objects.
Reviewed-by: johnc, ysr, jmasa

! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp

Changeset: 97c00e21fecb
Author:    tonyp
Date:      2012-01-09 23:50 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/97c00e21fecb

7125281: G1: heap expansion code is replicated
Reviewed-by: brutisso, johnc

! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp

Changeset: 1d6185f732aa
Author:    brutisso
Date:      2012-01-10 20:02 +0100
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/1d6185f732aa

7128532: G1: Change default value of G1DefaultMaxNewGenPercent to 80
Reviewed-by: tonyp, jmasa

! src/share/vm/gc_implementation/g1/g1_globals.hpp

Changeset: 2ace1c4ee8da
Author:    tonyp
Date:      2012-01-10 18:58 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/2ace1c4ee8da

6888336: G1: avoid explicitly marking and pushing objects in survivor spaces
Summary: This change simplifies the interaction between GC and concurrent marking. By disabling survivor spaces during the initial-mark pause we don't need to propagate marks of objects we copy during each GC (since we never need to copy an explicitly marked object).
Reviewed-by: johnc, brutisso

! src/share/vm/gc_implementation/g1/concurrentMark.cpp
! src/share/vm/gc_implementation/g1/concurrentMark.hpp
! src/share/vm/gc_implementation/g1/concurrentMark.inline.hpp
! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp
! src/share/vm/gc_implementation/g1/g1CollectedHeap.hpp
! src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp
! src/share/vm/gc_implementation/g1/g1CollectorPolicy.hpp
! src/share/vm/gc_implementation/g1/g1EvacFailure.hpp
! src/share/vm/gc_implementation/g1/g1OopClosures.hpp
! src/share/vm/gc_implementation/g1/heapRegion.cpp
! src/share/vm/gc_implementation/g1/heapRegion.hpp
! src/share/vm/gc_implementation/g1/heapRegion.inline.hpp
! src/share/vm/gc_implementation/g1/ptrQueue.hpp
! src/share/vm/gc_implementation/g1/satbQueue.cpp
! src/share/vm/gc_implementation/g1/satbQueue.hpp

Changeset: 9d4f4a1825e4
Author:    brutisso
Date:      2012-01-13 01:55 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/9d4f4a1825e4

Merge


Changeset: 5acd82522540
Author:    brutisso
Date:      2012-01-13 06:18 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/5acd82522540

Merge



From ysr1729 at gmail.com  Fri Jan 13 09:15:44 2012
From: ysr1729 at gmail.com (Srinivas Ramakrishna)
Date: Fri, 13 Jan 2012 09:15:44 -0800
Subject: JEP 132: More-prompt finalization
In-Reply-To: <4F0A9AE3.7000602@oracle.com>
References: <20111222230542.48A451084@eggemoggin.niobe.net>
	<4EF3E44F.1060409@oracle.com> <4EF4A8A4.1060502@oracle.com>
	<4EF5C695.4060206@oracle.com> <4EF9F709.4030101@oracle.com>
	<4EFA2869.6070105@oracle.com>
	<712FF280-2C59-47A0-99DE-ECEAD5FCFB15@kodewerk.com>
	<4EFAC2C7.7060508@oracle.com>
	<32CBF5EE-6AB8-4888-895E-37E495ACCD74@kodewerk.com>
	<4EFB5583.5000005@oracle.com> <4F05D3A0.1050503@oracle.com>
	<08196B9F-49AC-4DBF-B0C7-76903539C217@kodewerk.com>
	<4F065F5D.6040605@oracle.com> <4F06CE20.10809@oracle.com>
	<4F06D7A7.6010603@oracle.com> <4F071C4B.8010002@oracle.com>
	<4F0A257A.50104@oracle.com>
	<CABzyjynZs8pPJM-18VXiZS6QWf7e3UXo7dYk=53qthcG=Pffyg@mail.gmail.com>
	<4F0A317B.6000702@oracle.com>
	<CABzyjykfwxuOs1S4NJ5ncXpj+eWALS4+MC0=6r8mQscjTZR8Pw@mail.gmail.com>
	<4F0A9AE3.7000602@oracle.com>
Message-ID: <CABzyjyn5RbbQMiibTb=2KySgs1wQMjYX1nyRJeyEeoaqvG+Rrw@mail.gmail.com>

On Sun, Jan 8, 2012 at 11:44 PM, David Holmes <david.holmes at oracle.com>wrote:

> On 9/01/2012 5:27 PM, Srinivas Ramakrishna wrote:
>
>> Ah, I see what you meant. Yes, there'd probably be some (theoretical)
>> overhead from such tracking,
>>
>
> I'm not considering multiple apps per VM I'm just thinking of a programmer
> using java.io.* or java.net.* classes which have been "enhanced" to support
> resource tracking.
>

I don't think the overhead would be high. The tracking we are considering
is of a very simple nature, with the libraries doing
some (atomic) incrementing and decrementing of a counter per allocation or
finalization respectively (we could avoid
contention by deferring updates to a global counter except at specific
points although that may make rate calculations noisy),
and GC polling using these  counters to decide if a GC cycle should be run
because the resource may be close to exhaustion
in a little while. (One could alternatively have the allocation itself
detecting that fact and nudging the GC subsystem to check;
one can engineer this in a manner such that the cost itself can be made
fairly small.)

I suspect that the inline cost of these actions on native resource
allocation will likely be quite small, even if not
negligible, compared with the total cost of such a native resource
allocation, i am guessing.


> If the proposal involves new wrapper classes that an application has to
> opt-in to use then that is fine.
>

Certainly, any such API should allow for opt-in (perhaps on a
per-native-resource basis), albeit likely on a JVM-wide basis (per-resource)
 to make the resource-tracking effective.


>
> But I think this has gone off-track somewhat - finding alternatives to
> finalization. The JEP is more about speeding up finalization for those apps
> that (for better or worse) rely on it. I think there are two main aspects
> of this:
> - adding more threads to process finalizers and/or reference objects
> - adding an API to request and run finalization etc (more efficiently that
> doing: System.gc(); System.gc(); System.runFinalization() )
>

Certainly that is the first step and who knows, it may allow us to mostly
kick the can further down the road for a bit longer.


> Although we "all" know "finalizers are evil", they exist and are
> (mis-)used by some apps and cause some issues.
>

Fully agree.
-- ramki


>
> Cheers,
> David
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120113/3054bcbf/attachment-0001.html 

From vladimir.kozlov at oracle.com  Fri Jan 13 19:02:50 2012
From: vladimir.kozlov at oracle.com (vladimir.kozlov at oracle.com)
Date: Sat, 14 Jan 2012 03:02:50 +0000
Subject: hg: hsx/hotspot-main/hotspot: 4 new changesets
Message-ID: <20120114030303.0F4D347973@hg.openjdk.java.net>

Changeset: b0ff910edfc9
Author:    kvn
Date:      2012-01-12 14:45 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/b0ff910edfc9

7128355: assert(!nocreate) failed: Cannot build a phi for a block already parsed
Summary: Do not common BoxLock nodes and avoid creating phis of boxes.
Reviewed-by: never

! src/share/vm/opto/callnode.cpp
! src/share/vm/opto/locknode.cpp
! src/share/vm/opto/locknode.hpp
! src/share/vm/opto/macro.cpp
! src/share/vm/opto/parse1.cpp

Changeset: f4d8930a45b9
Author:    jrose
Date:      2012-01-13 00:51 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/f4d8930a45b9

Merge


Changeset: 89d0a5d40008
Author:    kvn
Date:      2012-01-13 12:58 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/89d0a5d40008

7129618: assert(obj_node->eqv_uncast(obj),"");
Summary: Relax verification and locks elimination checks for new implementation (EliminateNestedLocks).
Reviewed-by: iveresov

! src/share/vm/opto/locknode.cpp
! src/share/vm/opto/macro.cpp

Changeset: e504fd26c073
Author:    kvn
Date:      2012-01-13 14:21 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/e504fd26c073

Merge



From john.coomes at oracle.com  Sat Jan 14 03:42:36 2012
From: john.coomes at oracle.com (john.coomes at oracle.com)
Date: Sat, 14 Jan 2012 11:42:36 +0000
Subject: hg: hsx/hsx23/hotspot: 75 new changesets
Message-ID: <20120114114512.BED8E47978@hg.openjdk.java.net>

Changeset: fe2c87649981
Author:    katleman
Date:      2011-12-29 15:14 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/fe2c87649981

Added tag jdk8-b19 for changeset 9232e0ecbc2c

! .hgtags

Changeset: 9952d1c439d6
Author:    katleman
Date:      2012-01-05 08:42 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/9952d1c439d6

Added tag jdk8-b20 for changeset fe2c87649981

! .hgtags

Changeset: ed621d125d02
Author:    katleman
Date:      2012-01-13 10:05 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/ed621d125d02

Added tag jdk8-b21 for changeset 9952d1c439d6

! .hgtags

Changeset: 0841c0ec2ed6
Author:    amurillo
Date:      2011-12-23 15:29 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/0841c0ec2ed6

7123810: new hotspot build - hs23-b10
Reviewed-by: jcoomes

! make/hotspot_version

Changeset: 3b2b58fb1425
Author:    tonyp
Date:      2011-12-20 12:59 -0500
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/3b2b58fb1425

7123165: G1: output during parallel verification can get messed up
Summary: Serialize the worker threads that are generating output during parallel heap verification to make sure the output is consistent.
Reviewed-by: brutisso, johnc, jmasa

! src/share/vm/gc_implementation/g1/heapRegion.cpp

Changeset: d15b458c4225
Author:    jmasa
Date:      2011-12-20 20:29 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/d15b458c4225

Merge


Changeset: 67fdcb391461
Author:    tonyp
Date:      2011-12-21 07:53 -0500
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/67fdcb391461

7119027: G1: use atomics to update RS length / predict time of inc CSet
Summary: Make sure that the updates to the RS length and inc CSet predicted time are updated in an MT-safe way.
Reviewed-by: brutisso, iveresov

! src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp
! src/share/vm/gc_implementation/g1/g1CollectorPolicy.hpp

Changeset: 441e946dc1af
Author:    jmasa
Date:      2011-12-14 13:34 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/441e946dc1af

7121618: Change type of number of GC workers to unsigned int.
Summary: Change variables representing the number of GC workers to uint from int and size_t.  Change the parameter in work(int i) to work(uint worker_id).
Reviewed-by: brutisso, tonyp

! src/share/vm/gc_implementation/concurrentMarkSweep/compactibleFreeListSpace.cpp
! src/share/vm/gc_implementation/concurrentMarkSweep/compactibleFreeListSpace.hpp
! src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.cpp
! src/share/vm/gc_implementation/g1/collectionSetChooser.cpp
! src/share/vm/gc_implementation/g1/concurrentMark.cpp
! src/share/vm/gc_implementation/g1/concurrentMark.hpp
! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp
! src/share/vm/gc_implementation/g1/g1CollectedHeap.hpp
! src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp
! src/share/vm/gc_implementation/g1/g1RemSet.cpp
! src/share/vm/gc_implementation/g1/g1RemSet.hpp
! src/share/vm/gc_implementation/g1/g1RemSet.inline.hpp
! src/share/vm/gc_implementation/parNew/parCardTableModRefBS.cpp
! src/share/vm/gc_implementation/parNew/parNewGeneration.cpp
! src/share/vm/gc_implementation/parNew/parNewGeneration.hpp
! src/share/vm/gc_interface/collectedHeap.hpp
! src/share/vm/memory/genCollectedHeap.cpp
! src/share/vm/memory/genCollectedHeap.hpp
! src/share/vm/memory/referenceProcessor.cpp
! src/share/vm/memory/referenceProcessor.hpp
! src/share/vm/memory/sharedHeap.cpp
! src/share/vm/memory/sharedHeap.hpp
! src/share/vm/runtime/globals.hpp
! src/share/vm/utilities/workgroup.cpp
! src/share/vm/utilities/workgroup.hpp
! src/share/vm/utilities/yieldingWorkgroup.cpp
! src/share/vm/utilities/yieldingWorkgroup.hpp

Changeset: 1cbe7978b021
Author:    brutisso
Date:      2011-12-21 22:13 +0100
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/1cbe7978b021

7113021: G1: automatically enable young gen size auto-tuning when -Xms==-Xmx
Summary: Use a percentage of -Xms as min and another percentage of -Xmx as max for the young gen size
Reviewed-by: tonyp, johnc

! src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp
! src/share/vm/gc_implementation/g1/g1CollectorPolicy.hpp
! src/share/vm/gc_implementation/g1/g1_globals.hpp

Changeset: 7faca6dfa2ed
Author:    jmasa
Date:      2011-12-27 12:38 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/7faca6dfa2ed

Merge

! src/share/vm/runtime/globals.hpp

Changeset: 4ceaf61479fc
Author:    dcubed
Date:      2011-12-22 12:50 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/4ceaf61479fc

7122253: Instrumentation.retransformClasses() leaks class bytes
Summary: Change ClassFileParser::parseClassFile() to use the instanceKlass:_cached_class_file_bytes field to avoid leaking the cache.
Reviewed-by: coleenp, acorn, poonam

! src/share/vm/classfile/classFileParser.cpp
! src/share/vm/prims/jvmtiEnv.cpp
! src/share/vm/prims/jvmtiExport.cpp
! src/share/vm/prims/jvmtiRedefineClasses.cpp

Changeset: 4ec93d767458
Author:    vladidan
Date:      2011-12-26 20:36 -0500
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/4ec93d767458

Merge


Changeset: 3db6ea5ce021
Author:    vladidan
Date:      2011-12-29 20:09 -0500
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/3db6ea5ce021

Merge


Changeset: 20bfb6d15a94
Author:    iveresov
Date:      2011-12-27 16:43 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/20bfb6d15a94

7124829: NUMA: memory leak on Linux with large pages
Summary: In os::free_memory() use mmap with the same attributes as for the heap space
Reviewed-by: kvn
Contributed-by: Aleksey Ignatenko <aleksey.v.ignatenko at intel.com>

! src/os/bsd/vm/os_bsd.cpp
! src/os/linux/vm/os_linux.cpp
! src/os/solaris/vm/os_solaris.cpp
! src/os/windows/vm/os_windows.cpp
! src/share/vm/gc_implementation/shared/mutableNUMASpace.cpp
! src/share/vm/gc_implementation/shared/mutableSpace.cpp
! src/share/vm/runtime/os.hpp

Changeset: 776173fc2df9
Author:    stefank
Date:      2011-12-29 07:37 +0100
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/776173fc2df9

7125516: G1: ~ConcurrentMark() frees incorrectly
Summary: Replaced the code with a ShouldNotReachHere
Reviewed-by: tonyp, jmasa

! src/share/vm/gc_implementation/g1/concurrentMark.cpp

Changeset: 5ee33ff9b1c4
Author:    jmasa
Date:      2012-01-03 10:22 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/5ee33ff9b1c4

Merge


Changeset: 75c0a73eee98
Author:    coleenp
Date:      2011-11-17 12:53 -0500
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/75c0a73eee98

7102776: Pack instanceKlass boolean fields into single u1 field
Summary: Reduce class runtime memory usage by packing 4 instanceKlass boolean fields into single u1 field. Save 4-byte for each loaded class.
Reviewed-by: dholmes, bobv, phh, twisti, never, coleenp
Contributed-by: Jiangli Zhou <jiangli.zhou at oracle.com>

! agent/src/share/classes/sun/jvm/hotspot/oops/InstanceKlass.java
! src/share/vm/code/dependencies.cpp
! src/share/vm/oops/instanceKlass.hpp
! src/share/vm/oops/instanceKlassKlass.cpp
! src/share/vm/runtime/vmStructs.cpp

Changeset: da4dd142ea01
Author:    bobv
Date:      2011-11-29 14:44 -0500
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/da4dd142ea01

Merge

! src/share/vm/code/dependencies.cpp

Changeset: 52b5d32fbfaf
Author:    coleenp
Date:      2011-12-06 18:28 -0500
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/52b5d32fbfaf

7117052: instanceKlass::_init_state can be u1 type
Summary: Change instanceKlass::_init_state field to u1 type.
Reviewed-by: bdelsart, coleenp, dholmes, phh, never
Contributed-by: Jiangli Zhou <jiangli.zhou at oracle.com>

! src/cpu/sparc/vm/c1_LIRAssembler_sparc.cpp
! src/cpu/sparc/vm/c1_Runtime1_sparc.cpp
! src/cpu/sparc/vm/templateTable_sparc.cpp
! src/cpu/x86/vm/c1_LIRAssembler_x86.cpp
! src/cpu/x86/vm/c1_Runtime1_x86.cpp
! src/cpu/x86/vm/templateTable_x86_32.cpp
! src/cpu/x86/vm/templateTable_x86_64.cpp
! src/share/vm/ci/ciInstanceKlass.cpp
! src/share/vm/memory/dump.cpp
! src/share/vm/oops/instanceKlass.cpp
! src/share/vm/oops/instanceKlass.hpp
! src/share/vm/opto/library_call.cpp
! src/share/vm/opto/parseHelper.cpp
! src/share/vm/runtime/vmStructs.cpp

Changeset: eccc4b1f8945
Author:    vladidan
Date:      2011-12-07 16:47 -0500
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/eccc4b1f8945

7050298: ARM: SIGSEGV in JNIHandleBlock::allocate_handle
Summary: missing release barrier in Monitor::IUnlock
Reviewed-by: dholmes, dice

! src/share/vm/runtime/mutex.cpp

Changeset: 2685ea97b89f
Author:    jiangli
Date:      2011-12-09 11:29 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/2685ea97b89f

Merge

! src/cpu/sparc/vm/c1_LIRAssembler_sparc.cpp

Changeset: 8fdf463085e1
Author:    jiangli
Date:      2011-12-16 17:33 -0500
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/8fdf463085e1

Merge


Changeset: dca455dea3a7
Author:    bdelsart
Date:      2011-12-20 12:33 +0100
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/dca455dea3a7

7116216: StackOverflow GC crash
Summary: GC crash for explicit stack overflow checks after a C2I transition.
Reviewed-by: coleenp, never
Contributed-by: yang02.wang at sap.com, bertrand.delsart at oracle.com

! src/cpu/sparc/vm/stubGenerator_sparc.cpp
! src/cpu/sparc/vm/templateInterpreter_sparc.cpp
! src/cpu/x86/vm/stubGenerator_x86_32.cpp
! src/cpu/x86/vm/stubGenerator_x86_64.cpp
! src/cpu/x86/vm/templateInterpreter_x86_32.cpp
! src/cpu/x86/vm/templateInterpreter_x86_64.cpp
+ test/compiler/7116216/LargeFrame.java
+ test/compiler/7116216/StackOverflow.java

Changeset: cd5d8cafcc84
Author:    jiangli
Date:      2011-12-28 12:15 -0500
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/cd5d8cafcc84

7123315: instanceKlass::_static_oop_field_count and instanceKlass::_java_fields_count should be u2 type.
Summary: Change instanceKlass::_static_oop_field_count and instanceKlass::_java_fields_count to u2 type.
Reviewed-by: never, bdelsart, dholmes
Contributed-by: Jiangli Zhou <jiangli.zhou at oracle.com>

! src/share/vm/classfile/classFileParser.cpp
! src/share/vm/classfile/classFileParser.hpp
! src/share/vm/oops/instanceKlass.hpp
! src/share/vm/runtime/vmStructs.cpp

Changeset: 05de27e852c4
Author:    jiangli
Date:      2012-01-04 12:36 -0500
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/05de27e852c4

Merge

! src/share/vm/classfile/classFileParser.cpp

Changeset: b6a04c79ccbc
Author:    stefank
Date:      2012-01-02 10:01 +0100
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/b6a04c79ccbc

7125503: Compiling collectedHeap.cpp fails with -Werror=int-to-pointer-cast with g++ 4.6.1
Summary: Used uintptr_t and void* for all the casts and checks in test_is_in.
Reviewed-by: tonyp, jmasa

! src/share/vm/gc_interface/collectedHeap.cpp

Changeset: 4753e3dda3c8
Author:    jmasa
Date:      2012-01-04 07:56 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/4753e3dda3c8

Merge


Changeset: 2ee4167627a3
Author:    jmasa
Date:      2012-01-05 21:02 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/2ee4167627a3

Merge


Changeset: 7ab5f6318694
Author:    phh
Date:      2012-01-01 11:17 -0500
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/7ab5f6318694

7125934: Add a fast unordered timestamp capability to Hotspot on x86/x64
Summary: Add rdtsc detection and inline generation.
Reviewed-by: kamg, dholmes
Contributed-by: karen.kinnear at oracle.com

! src/cpu/x86/vm/vm_version_x86.cpp
! src/cpu/x86/vm/vm_version_x86.hpp
! src/os_cpu/bsd_x86/vm/os_bsd_x86.hpp
+ src/os_cpu/bsd_x86/vm/os_bsd_x86.inline.hpp
! src/os_cpu/linux_x86/vm/os_linux_x86.hpp
+ src/os_cpu/linux_x86/vm/os_linux_x86.inline.hpp
! src/os_cpu/solaris_x86/vm/os_solaris_x86.hpp
+ src/os_cpu/solaris_x86/vm/os_solaris_x86.inline.hpp
! src/os_cpu/solaris_x86/vm/solaris_x86_32.il
! src/os_cpu/solaris_x86/vm/solaris_x86_64.il
! src/os_cpu/windows_x86/vm/os_windows_x86.hpp
+ src/os_cpu/windows_x86/vm/os_windows_x86.inline.hpp
! src/share/vm/runtime/init.cpp
! src/share/vm/runtime/os.cpp
! src/share/vm/runtime/os.hpp
+ src/share/vm/runtime/os_ext.hpp

Changeset: b16494a69d3d
Author:    phh
Date:      2012-01-03 15:11 -0500
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/b16494a69d3d

7126185: Clean up lasterror handling, add os::get_last_error()
Summary: Add os::get_last_error(), replace getLastErrorString() by os::lasterror() in os_windows.cpp.
Reviewed-by: kamg, dholmes
Contributed-by: erik.gahlin at oracle.com

! src/os/posix/vm/os_posix.cpp
! src/os/windows/vm/os_windows.cpp
! src/share/vm/runtime/os.hpp

Changeset: 5b58979183f9
Author:    dcubed
Date:      2012-01-05 06:24 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/5b58979183f9

7127032: fix for 7122253 adds a JvmtiThreadState earlier than necessary
Summary: Use JavaThread::jvmti_thread_state() instead of JvmtiThreadState::state_for().
Reviewed-by: coleenp, poonam, acorn

! src/share/vm/classfile/classFileParser.cpp

Changeset: 8a63c6323842
Author:    fparain
Date:      2012-01-05 07:26 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/8a63c6323842

7125594: C-heap growth issue in ThreadService::find_deadlocks_at_safepoint
Reviewed-by: sspitsyn, dcubed, mchung, dholmes

! src/share/vm/services/threadService.cpp

Changeset: 2e0ef19fc891
Author:    phh
Date:      2012-01-05 17:14 -0500
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/2e0ef19fc891

7126480: Make JVM start time in milliseconds since the Java epoch available
Summary: Expose existing Management::_begin_vm_creation_time via new accessor Management::begin_vm_creation_time().
Reviewed-by: acorn, dcubed

! src/share/vm/services/management.hpp

Changeset: 66259eca2bf7
Author:    phh
Date:      2012-01-05 17:16 -0500
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/66259eca2bf7

Merge


Changeset: 2b3acb34791f
Author:    dcubed
Date:      2012-01-06 16:18 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/2b3acb34791f

Merge

! src/os/windows/vm/os_windows.cpp
! src/share/vm/classfile/classFileParser.cpp
! src/share/vm/runtime/os.hpp

Changeset: abcceac2f7cd
Author:    iveresov
Date:      2011-12-12 12:44 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/abcceac2f7cd

7119730: Tiered: SIGSEGV in AdvancedThresholdPolicy::is_method_profiled(methodOop)
Summary: Added handles for references to methods in select_task()
Reviewed-by: twisti, kvn

! src/share/vm/runtime/advancedThresholdPolicy.cpp

Changeset: 7bca37d28f32
Author:    roland
Date:      2011-12-13 10:54 +0100
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/7bca37d28f32

7114106: C1: assert(goto_state->is_same(sux_state)) failed: states must match now
Summary: fix C1's CEE to take inlining into account when the stacks in states are compared.
Reviewed-by: iveresov, never

! src/share/vm/c1/c1_Optimizer.cpp

Changeset: d725f0affb1a
Author:    iveresov
Date:      2011-12-13 17:10 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/d725f0affb1a

7121111: -server -Xcomp -XX:+TieredCompilation does not invoke C2 compiler
Summary: Exercise C2 more in tiered mode with Xcomp
Reviewed-by: kvn, never

! src/share/vm/runtime/arguments.cpp

Changeset: 127b3692c168
Author:    kvn
Date:      2011-12-14 14:54 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/127b3692c168

7116452: Add support for AVX instructions
Summary: Added support for AVX extension to the x86 instruction set.
Reviewed-by: never

! src/cpu/x86/vm/assembler_x86.cpp
! src/cpu/x86/vm/assembler_x86.hpp
! src/cpu/x86/vm/assembler_x86.inline.hpp
! src/cpu/x86/vm/nativeInst_x86.cpp
! src/cpu/x86/vm/nativeInst_x86.hpp
! src/cpu/x86/vm/register_definitions_x86.cpp
! src/cpu/x86/vm/vm_version_x86.cpp
! src/cpu/x86/vm/vm_version_x86.hpp
! src/cpu/x86/vm/x86_32.ad
! src/cpu/x86/vm/x86_64.ad
! src/share/vm/runtime/globals.hpp

Changeset: 669f6a7d5b70
Author:    never
Date:      2011-12-19 14:16 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/669f6a7d5b70

7121073: secondary_super_cache memory slice has incorrect bounds in flatten_alias_type
Reviewed-by: kvn

! src/share/vm/opto/compile.cpp

Changeset: 65149e74c706
Author:    kvn
Date:      2011-12-20 00:55 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/65149e74c706

7121648: Use 3-operands SIMD instructions on x86 with AVX
Summary: Use 3-operands SIMD instructions in C2 generated code for machines with AVX.
Reviewed-by: never

! make/bsd/makefiles/adlc.make
! make/linux/makefiles/adlc.make
! make/solaris/makefiles/adlc.make
! make/windows/makefiles/adlc.make
! src/cpu/x86/vm/assembler_x86.cpp
! src/cpu/x86/vm/assembler_x86.hpp
+ src/cpu/x86/vm/x86.ad
! src/cpu/x86/vm/x86_32.ad
! src/cpu/x86/vm/x86_64.ad
! src/share/vm/opto/matcher.cpp

Changeset: 069ab3f976d3
Author:    stefank
Date:      2011-12-07 11:35 +0100
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/069ab3f976d3

7118863: Move sizeof(klassOopDesc) into the *Klass::*_offset_in_bytes() functions
Summary: Moved sizeof(klassOopDesc), changed the return type to ByteSize and removed the _in_bytes suffix.
Reviewed-by: never, bdelsart, coleenp, jrose

! src/cpu/sparc/vm/assembler_sparc.cpp
! src/cpu/sparc/vm/c1_CodeStubs_sparc.cpp
! src/cpu/sparc/vm/c1_LIRAssembler_sparc.cpp
! src/cpu/sparc/vm/c1_MacroAssembler_sparc.cpp
! src/cpu/sparc/vm/c1_Runtime1_sparc.cpp
! src/cpu/sparc/vm/cppInterpreter_sparc.cpp
! src/cpu/sparc/vm/methodHandles_sparc.cpp
! src/cpu/sparc/vm/stubGenerator_sparc.cpp
! src/cpu/sparc/vm/templateInterpreter_sparc.cpp
! src/cpu/sparc/vm/templateTable_sparc.cpp
! src/cpu/x86/vm/assembler_x86.cpp
! src/cpu/x86/vm/c1_CodeStubs_x86.cpp
! src/cpu/x86/vm/c1_LIRAssembler_x86.cpp
! src/cpu/x86/vm/c1_MacroAssembler_x86.cpp
! src/cpu/x86/vm/c1_Runtime1_x86.cpp
! src/cpu/x86/vm/cppInterpreter_x86.cpp
! src/cpu/x86/vm/methodHandles_x86.cpp
! src/cpu/x86/vm/stubGenerator_x86_32.cpp
! src/cpu/x86/vm/stubGenerator_x86_64.cpp
! src/cpu/x86/vm/templateInterpreter_x86_32.cpp
! src/cpu/x86/vm/templateInterpreter_x86_64.cpp
! src/cpu/x86/vm/templateTable_x86_32.cpp
! src/cpu/x86/vm/templateTable_x86_64.cpp
! src/cpu/x86/vm/x86_64.ad
! src/share/vm/c1/c1_LIRGenerator.cpp
! src/share/vm/oops/arrayKlass.hpp
! src/share/vm/oops/instanceKlass.hpp
! src/share/vm/oops/klass.cpp
! src/share/vm/oops/klass.hpp
! src/share/vm/oops/klassOop.hpp
! src/share/vm/oops/objArrayKlass.hpp
! src/share/vm/opto/compile.cpp
! src/share/vm/opto/graphKit.cpp
! src/share/vm/opto/library_call.cpp
! src/share/vm/opto/macro.cpp
! src/share/vm/opto/memnode.cpp
! src/share/vm/opto/parse1.cpp
! src/share/vm/opto/parseHelper.cpp
! src/share/vm/shark/sharkIntrinsics.cpp
! src/share/vm/shark/sharkTopLevelBlock.cpp

Changeset: 1dc233a8c7fe
Author:    roland
Date:      2011-12-20 16:56 +0100
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/1dc233a8c7fe

7121140: Allocation paths require explicit memory synchronization operations for RMO systems
Summary: adds store store barrier after initialization of header and body of objects.
Reviewed-by: never, kvn

! src/cpu/sparc/vm/sparc.ad
! src/cpu/x86/vm/x86_32.ad
! src/cpu/x86/vm/x86_64.ad
! src/share/vm/adlc/formssel.cpp
! src/share/vm/opto/callnode.hpp
! src/share/vm/opto/classes.hpp
! src/share/vm/opto/escape.cpp
! src/share/vm/opto/graphKit.cpp
! src/share/vm/opto/library_call.cpp
! src/share/vm/opto/macro.cpp
! src/share/vm/opto/memnode.cpp
! src/share/vm/opto/memnode.hpp
! src/share/vm/opto/node.hpp

Changeset: e5ac210043cd
Author:    roland
Date:      2011-12-22 10:55 +0100
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/e5ac210043cd

7123108: C1: assert(if_state != NULL) failed: states do not match up
Summary: In CEE, ensure if and common successor state are at the same inline level
Reviewed-by: never

! src/share/vm/c1/c1_Optimizer.cpp
+ test/compiler/7123108/Test7123108.java

Changeset: b642b49f9738
Author:    roland
Date:      2011-12-23 09:36 +0100
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/b642b49f9738

7123253: C1: in store check code, usage of registers may be incorrect
Summary: fix usage of input register in assembly code for store check.
Reviewed-by: never

! src/share/vm/c1/c1_LIR.cpp

Changeset: 40c2484c09e1
Author:    kvn
Date:      2011-12-23 15:24 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/40c2484c09e1

7110832: ctw/.../org_apache_avalon_composition_util_StringHelper crashes the VM
Summary: Distance is too large for one short branch in string_indexofC8().
Reviewed-by: iveresov

! src/cpu/x86/vm/assembler_x86.cpp
! src/share/vm/asm/assembler.cpp
! src/share/vm/asm/assembler.hpp

Changeset: d12a66fa3820
Author:    kvn
Date:      2011-12-27 15:08 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/d12a66fa3820

7123954: Some CTW test crash with SIGSEGV
Summary: Correct Allocate expansion code to preserve i_o when only slow call is generated.
Reviewed-by: iveresov

! src/share/vm/opto/compile.cpp
! src/share/vm/opto/macro.cpp

Changeset: 8940fd98d540
Author:    kvn
Date:      2011-12-29 11:37 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/8940fd98d540

Merge

! src/cpu/x86/vm/assembler_x86.cpp
! src/share/vm/runtime/globals.hpp

Changeset: 9c87bcb3b4dd
Author:    kvn
Date:      2011-12-30 11:43 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/9c87bcb3b4dd

7125879: assert(proj != NULL) failed: must be found
Summary: Leave i_o attached to slow allocation call when there are no i_o users after the call.
Reviewed-by: iveresov, twisti

! src/share/vm/opto/macro.cpp
+ test/compiler/7125879/Test7125879.java

Changeset: 1cb50d7a9d95
Author:    iveresov
Date:      2012-01-05 17:25 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/1cb50d7a9d95

7119294: Two command line options cause JVM to crash
Summary: Setup thread register in MacroAssembler::incr_allocated_bytes() on x64
Reviewed-by: kvn

! src/cpu/x86/vm/assembler_x86.cpp

Changeset: 22cee0ee8927
Author:    kvn
Date:      2012-01-06 20:09 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/22cee0ee8927

Merge

! src/cpu/sparc/vm/c1_LIRAssembler_sparc.cpp
! src/cpu/sparc/vm/c1_Runtime1_sparc.cpp
! src/cpu/sparc/vm/stubGenerator_sparc.cpp
! src/cpu/sparc/vm/templateInterpreter_sparc.cpp
! src/cpu/sparc/vm/templateTable_sparc.cpp
! src/cpu/x86/vm/c1_LIRAssembler_x86.cpp
! src/cpu/x86/vm/c1_Runtime1_x86.cpp
! src/cpu/x86/vm/stubGenerator_x86_32.cpp
! src/cpu/x86/vm/stubGenerator_x86_64.cpp
! src/cpu/x86/vm/templateInterpreter_x86_32.cpp
! src/cpu/x86/vm/templateInterpreter_x86_64.cpp
! src/cpu/x86/vm/templateTable_x86_32.cpp
! src/cpu/x86/vm/templateTable_x86_64.cpp
! src/cpu/x86/vm/vm_version_x86.cpp
! src/cpu/x86/vm/vm_version_x86.hpp
! src/share/vm/oops/instanceKlass.hpp
! src/share/vm/opto/library_call.cpp
! src/share/vm/opto/parseHelper.cpp

Changeset: 8f8b94305aff
Author:    dcubed
Date:      2012-01-11 19:54 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/8f8b94305aff

7129240: backout fix for 7102776 until 7128770 is resolved
Reviewed-by: phh, bobv, coleenp, dcubed
Contributed-by: Jiangli Zhou <jiangli.zhou at oracle.com>

! agent/src/share/classes/sun/jvm/hotspot/oops/InstanceKlass.java
! src/share/vm/code/dependencies.cpp
! src/share/vm/oops/instanceKlass.hpp
! src/share/vm/oops/instanceKlassKlass.cpp
! src/share/vm/runtime/vmStructs.cpp

Changeset: 4f25538b54c9
Author:    fparain
Date:      2012-01-09 10:27 +0100
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/4f25538b54c9

7120511: Add diagnostic commands
Reviewed-by: acorn, phh, dcubed, sspitsyn

! src/share/vm/classfile/vmSymbols.hpp
! src/share/vm/runtime/arguments.cpp
! src/share/vm/runtime/globals.cpp
! src/share/vm/runtime/globals.hpp
! src/share/vm/runtime/init.cpp
! src/share/vm/services/attachListener.cpp
! src/share/vm/services/diagnosticCommand.cpp
! src/share/vm/services/diagnosticCommand.hpp
! src/share/vm/services/diagnosticFramework.cpp
! src/share/vm/services/diagnosticFramework.hpp
! src/share/vm/services/management.cpp

Changeset: 865e0817f32b
Author:    kamg
Date:      2012-01-10 15:47 -0500
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/865e0817f32b

Merge

! src/share/vm/runtime/arguments.cpp
! src/share/vm/runtime/globals.hpp

Changeset: efdf6985a3a2
Author:    kamg
Date:      2012-01-12 09:59 -0500
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/efdf6985a3a2

Merge


Changeset: 5da7201222d5
Author:    kvn
Date:      2012-01-07 10:39 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/5da7201222d5

7110824: ctw/jarfiles/GUI3rdParty_jar/ob_mask_DateField crashes VM
Summary: Change yank_if_dead() to recursive method to remove all dead inputs.
Reviewed-by: never

! src/cpu/sparc/vm/sparc.ad
! src/share/vm/opto/chaitin.hpp
! src/share/vm/opto/postaloc.cpp

Changeset: e9a5e0a812c8
Author:    kvn
Date:      2012-01-07 13:26 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/e9a5e0a812c8

7125896: Eliminate nested locks
Summary: Nested locks elimination done before lock nodes expansion by looking for outer locks of the same object.
Reviewed-by: never, twisti

! src/cpu/sparc/vm/sparc.ad
! src/cpu/x86/vm/x86_32.ad
! src/cpu/x86/vm/x86_64.ad
! src/share/vm/ci/ciTypeFlow.cpp
! src/share/vm/ci/ciTypeFlow.hpp
! src/share/vm/opto/c2_globals.hpp
! src/share/vm/opto/callnode.cpp
! src/share/vm/opto/callnode.hpp
! src/share/vm/opto/escape.cpp
! src/share/vm/opto/locknode.cpp
! src/share/vm/opto/locknode.hpp
! src/share/vm/opto/macro.cpp
! src/share/vm/opto/macro.hpp
! src/share/vm/opto/output.cpp
! src/share/vm/opto/parse1.cpp
! src/share/vm/runtime/arguments.cpp
! src/share/vm/runtime/deoptimization.cpp

Changeset: 35acf8f0a2e4
Author:    kvn
Date:      2012-01-10 18:05 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/35acf8f0a2e4

7128352: assert(obj_node == obj) failed
Summary: Compare uncasted object nodes.
Reviewed-by: never

! src/share/vm/opto/callnode.cpp
! src/share/vm/opto/cfgnode.cpp
! src/share/vm/opto/library_call.cpp
! src/share/vm/opto/locknode.cpp
! src/share/vm/opto/macro.cpp
! src/share/vm/opto/memnode.cpp
! src/share/vm/opto/node.cpp
! src/share/vm/opto/node.hpp
! src/share/vm/opto/phaseX.hpp
! src/share/vm/opto/subnode.cpp
! test/compiler/7116216/StackOverflow.java

Changeset: c8d8e124380c
Author:    kvn
Date:      2012-01-12 12:28 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/c8d8e124380c

7064302: JDK7 build 147 crashed after testing my java 6-compiled web app
Summary: Don't split CMove node if it's control edge is different from split region.
Reviewed-by: never

! src/share/vm/opto/loopnode.cpp
! src/share/vm/opto/loopnode.hpp
! src/share/vm/opto/loopopts.cpp

Changeset: 31a5b9aad4bc
Author:    jrose
Date:      2012-01-13 00:27 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/31a5b9aad4bc

Merge

! src/share/vm/runtime/arguments.cpp

Changeset: bacb651cf5bf
Author:    tonyp
Date:      2012-01-05 05:54 -0500
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/bacb651cf5bf

7113006: G1: excessive ergo output when an evac failure happens
Summary: Introduce a flag that is set when a heap expansion attempt during a GC fails so that we do not consantly attempt to expand the heap when it's going to fail anyway. This not only prevents the excessive ergo output (which is generated when a region allocation fails) but also avoids excessive and ultimately unsuccessful expansion attempts.
Reviewed-by: jmasa, johnc

! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp
! src/share/vm/gc_implementation/g1/g1CollectedHeap.hpp

Changeset: 5fd354a959c5
Author:    jmasa
Date:      2012-01-05 21:21 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/5fd354a959c5

Merge


Changeset: 023652e49ac0
Author:    johnc
Date:      2011-12-23 11:14 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/023652e49ac0

7121496: G1: do the per-region evacuation failure handling work in parallel
Summary: Parallelize the removal of self forwarding pointers etc. by wrapping in a HeapRegion closure, which is then wrapped inside an AbstractGangTask.
Reviewed-by: tonyp, iveresov

! src/share/vm/gc_implementation/g1/concurrentMark.cpp
! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp
! src/share/vm/gc_implementation/g1/g1CollectedHeap.hpp
+ src/share/vm/gc_implementation/g1/g1EvacFailure.hpp
! src/share/vm/gc_implementation/g1/heapRegion.hpp

Changeset: 02838862dec8
Author:    tonyp
Date:      2012-01-07 00:43 -0500
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/02838862dec8

7121623: G1: always be able to reliably calculate the length of a forwarded chunked array
Summary: Store the "next chunk start index" in the length field of the to-space object, instead of the from-space object, so that we can always reliably read the size of all from-space objects.
Reviewed-by: johnc, ysr, jmasa

! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp

Changeset: 97c00e21fecb
Author:    tonyp
Date:      2012-01-09 23:50 -0500
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/97c00e21fecb

7125281: G1: heap expansion code is replicated
Reviewed-by: brutisso, johnc

! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp

Changeset: 1d6185f732aa
Author:    brutisso
Date:      2012-01-10 20:02 +0100
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/1d6185f732aa

7128532: G1: Change default value of G1DefaultMaxNewGenPercent to 80
Reviewed-by: tonyp, jmasa

! src/share/vm/gc_implementation/g1/g1_globals.hpp

Changeset: 2ace1c4ee8da
Author:    tonyp
Date:      2012-01-10 18:58 -0500
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/2ace1c4ee8da

6888336: G1: avoid explicitly marking and pushing objects in survivor spaces
Summary: This change simplifies the interaction between GC and concurrent marking. By disabling survivor spaces during the initial-mark pause we don't need to propagate marks of objects we copy during each GC (since we never need to copy an explicitly marked object).
Reviewed-by: johnc, brutisso

! src/share/vm/gc_implementation/g1/concurrentMark.cpp
! src/share/vm/gc_implementation/g1/concurrentMark.hpp
! src/share/vm/gc_implementation/g1/concurrentMark.inline.hpp
! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp
! src/share/vm/gc_implementation/g1/g1CollectedHeap.hpp
! src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp
! src/share/vm/gc_implementation/g1/g1CollectorPolicy.hpp
! src/share/vm/gc_implementation/g1/g1EvacFailure.hpp
! src/share/vm/gc_implementation/g1/g1OopClosures.hpp
! src/share/vm/gc_implementation/g1/heapRegion.cpp
! src/share/vm/gc_implementation/g1/heapRegion.hpp
! src/share/vm/gc_implementation/g1/heapRegion.inline.hpp
! src/share/vm/gc_implementation/g1/ptrQueue.hpp
! src/share/vm/gc_implementation/g1/satbQueue.cpp
! src/share/vm/gc_implementation/g1/satbQueue.hpp

Changeset: 9d4f4a1825e4
Author:    brutisso
Date:      2012-01-13 01:55 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/9d4f4a1825e4

Merge


Changeset: 5acd82522540
Author:    brutisso
Date:      2012-01-13 06:18 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/5acd82522540

Merge


Changeset: b0ff910edfc9
Author:    kvn
Date:      2012-01-12 14:45 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/b0ff910edfc9

7128355: assert(!nocreate) failed: Cannot build a phi for a block already parsed
Summary: Do not common BoxLock nodes and avoid creating phis of boxes.
Reviewed-by: never

! src/share/vm/opto/callnode.cpp
! src/share/vm/opto/locknode.cpp
! src/share/vm/opto/locknode.hpp
! src/share/vm/opto/macro.cpp
! src/share/vm/opto/parse1.cpp

Changeset: f4d8930a45b9
Author:    jrose
Date:      2012-01-13 00:51 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/f4d8930a45b9

Merge


Changeset: 89d0a5d40008
Author:    kvn
Date:      2012-01-13 12:58 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/89d0a5d40008

7129618: assert(obj_node->eqv_uncast(obj),"");
Summary: Relax verification and locks elimination checks for new implementation (EliminateNestedLocks).
Reviewed-by: iveresov

! src/share/vm/opto/locknode.cpp
! src/share/vm/opto/macro.cpp

Changeset: e504fd26c073
Author:    kvn
Date:      2012-01-13 14:21 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/e504fd26c073

Merge


Changeset: 513351373923
Author:    amurillo
Date:      2012-01-14 00:47 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/513351373923

Merge


Changeset: 24727fb37561
Author:    amurillo
Date:      2012-01-14 00:47 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/24727fb37561

Added tag hs23-b10 for changeset 513351373923

! .hgtags


From james.melvin at oracle.com  Sat Jan 14 09:02:47 2012
From: james.melvin at oracle.com (James Melvin)
Date: Sat, 14 Jan 2012 12:02:47 -0500
Subject: RFR (XS): 7126732: MAC: Require Mac OS X builds/tests for JPRT
	integrate jobs for HotSpot
Message-ID: <4F11B537.2000504@oracle.com>

Greetings,

We're ready to require HotSpot builds on Mac OS X for JPRT integrate
jobs. There are 3 mac-minis in each queue. Build/Test times are short
relative to other platforms. Uses the stable Linux testlist for now.

http://cr.openjdk.java.net/~jmelvin/7126732/webrev.00

Tested with *several* JPRT submissions for other bugfixes. I'd like to
integrate this change right after the current snapshot window.

Feedback welcome.

Thanks,

Jim

From john.coomes at oracle.com  Sat Jan 14 09:15:38 2012
From: john.coomes at oracle.com (john.coomes at oracle.com)
Date: Sat, 14 Jan 2012 17:15:38 +0000
Subject: hg: hsx/hotspot-main/hotspot: 6 new changesets
Message-ID: <20120114171557.8674747979@hg.openjdk.java.net>

Changeset: fe2c87649981
Author:    katleman
Date:      2011-12-29 15:14 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/fe2c87649981

Added tag jdk8-b19 for changeset 9232e0ecbc2c

! .hgtags

Changeset: 9952d1c439d6
Author:    katleman
Date:      2012-01-05 08:42 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/9952d1c439d6

Added tag jdk8-b20 for changeset fe2c87649981

! .hgtags

Changeset: ed621d125d02
Author:    katleman
Date:      2012-01-13 10:05 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/ed621d125d02

Added tag jdk8-b21 for changeset 9952d1c439d6

! .hgtags

Changeset: 513351373923
Author:    amurillo
Date:      2012-01-14 00:47 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/513351373923

Merge


Changeset: 24727fb37561
Author:    amurillo
Date:      2012-01-14 00:47 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/24727fb37561

Added tag hs23-b10 for changeset 513351373923

! .hgtags

Changeset: 4e80db53c323
Author:    amurillo
Date:      2012-01-14 00:52 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/4e80db53c323

7129512: new hotspot build - hs23-b11
Reviewed-by: jcoomes

! make/hotspot_version


From manojo10386 at gmail.com  Sun Jan 15 09:59:26 2012
From: manojo10386 at gmail.com (Manohar Jonnalagedda)
Date: Sun, 15 Jan 2012 18:59:26 +0100
Subject: Detecting range check elimination with PrintAssembly
Message-ID: <CABhSbLyAwFLFwhmTzPE=0DiXK8e8kbmvr3tr-0AQ=KyFz1Mjsw@mail.gmail.com>

Hello,

following this reference on Range Check Elimination done by the Hotspot
compiler [1], I was keen in knowing how I can detect whether range checks
are taking place in loops by inspecting output using the PrintAssembly
flag; with the old PrintOptoAssembly flag, I have seen output such as the
following, which I assume to be range checks :

B11: #  B73 B12 &lt;- B10  Freq: 1.21365
139     movq    RAX, [rsp + #24]  # spill
13e     movl    RSI, [RAX + #12 (8-bit)]  # range
141     NullCheck RAX

What is the equivalent with the new PrintAssembly flag (using hsdis)?

Moreover, as stated on the wiki page [1], loops are optimized if the stride
is a compile-time constant. I performed a few tests on a kmeans program,
with 3 nested loops, having the following (high-level) structure:

===
void method1(){
//loop 1
for(int i = 0; i< rows1; i++){
 //...
  for(int j = 0; j< rows2; j++){
  //...
   for(int k = 0; j < cols; k++){ array[j * cols + k] = //...}
  }
}
}

void method2(){
//loop 2
for(int i =0; i < rows1; i++){
  for(int j=0 ; i< rows2; j++){
    for(int k=0 ; k< cols; k++){
      array[i*cols+k] = //...
    }
  }
}
}

void main(){

 do{
   method1(); method2();
 }while(!converged)

}
====

In the first test, cols is an int whose value is determined at runtime (by
reading a file), in the second test, it is given as a compile-time
constant(3). In the second test, there is a **significant** speed-up
(around 40%). However, when studying the diff of the output of
PrintOptoAssembly for both method1 and method2, there is no difference
(apart from slight value changes in frequency). Would you have any hints as
to where I could look for differences?

Thanks a lot,
Manohar

[1] https://wikis.oracle.com/display/HotSpotInternals/RangeCheckElimination
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120115/819ae1fc/attachment.html 

From rednaxelafx at gmail.com  Sun Jan 15 12:22:59 2012
From: rednaxelafx at gmail.com (Krystal Mok)
Date: Mon, 16 Jan 2012 04:22:59 +0800
Subject: Detecting range check elimination with PrintAssembly
In-Reply-To: <CABhSbLyAwFLFwhmTzPE=0DiXK8e8kbmvr3tr-0AQ=KyFz1Mjsw@mail.gmail.com>
References: <CABhSbLyAwFLFwhmTzPE=0DiXK8e8kbmvr3tr-0AQ=KyFz1Mjsw@mail.gmail.com>
Message-ID: <CA+cQ+tTd_yhyu1=06JSgskU-kJ_qh=cXmnYdE6a7TgCR7s=uxg@mail.gmail.com>

Hi,

In your PrintOptoAssembly output snippet, the instruction at 0x13e is a
LoadRange, which loads the range from the header of an array:

(from x86_64.ad)
// Load Range
instruct loadRange(rRegI dst, memory mem)
%{
  match(Set dst (LoadRange mem));

  ins_cost(125); // XXX
  format %{ "movl    $dst, $mem\t# range" %}
  opcode(0x8B);
  ins_encode(REX_reg_mem(dst, mem), OpcP, reg_mem(dst, mem));
  ins_pipe(ialu_reg_mem);
%}

That's not a range check just yet; the real check, if any, should come
after the null check, in the form of comparing something else with RSI. But
you didn't show what's after the null check, how RSI is used, so it's hard
to say what you're seeing in your example.

As for the two test examples, could you paste the entire source code, with
the PrintOptoAssembly output of method1() and method2() ? The first example
looks weird, maybe it's a typo but you're using "j < cols" as the loop
condition for the inner loop.

I'd guess it's the difference in locality that made the difference in
performance in your two tests.

- Kris

On Mon, Jan 16, 2012 at 1:59 AM, Manohar Jonnalagedda <manojo10386 at gmail.com
> wrote:

> Hello,
>
> following this reference on Range Check Elimination done by the Hotspot
> compiler [1], I was keen in knowing how I can detect whether range checks
> are taking place in loops by inspecting output using the PrintAssembly
> flag; with the old PrintOptoAssembly flag, I have seen output such as the
> following, which I assume to be range checks :
>
> B11: #  B73 B12 &lt;- B10  Freq: 1.21365
> 139     movq    RAX, [rsp + #24]  # spill
> 13e     movl    RSI, [RAX + #12 (8-bit)]  # range
> 141     NullCheck RAX
>
> What is the equivalent with the new PrintAssembly flag (using hsdis)?
>
> Moreover, as stated on the wiki page [1], loops are optimized if the
> stride is a compile-time constant. I performed a few tests on a kmeans
> program, with 3 nested loops, having the following (high-level) structure:
>
> ===
> void method1(){
> //loop 1
> for(int i = 0; i< rows1; i++){
>  //...
>   for(int j = 0; j< rows2; j++){
>   //...
>    for(int k = 0; j < cols; k++){ array[j * cols + k] = //...}
>   }
> }
> }
>
> void method2(){
> //loop 2
> for(int i =0; i < rows1; i++){
>   for(int j=0 ; i< rows2; j++){
>     for(int k=0 ; k< cols; k++){
>       array[i*cols+k] = //...
>     }
>   }
> }
> }
>
> void main(){
>
>  do{
>    method1(); method2();
>  }while(!converged)
>
> }
> ====
>
> In the first test, cols is an int whose value is determined at runtime
> (by reading a file), in the second test, it is given as a compile-time
> constant(3). In the second test, there is a **significant** speed-up
> (around 40%). However, when studying the diff of the output of
> PrintOptoAssembly for both method1 and method2, there is no difference
> (apart from slight value changes in frequency). Would you have any hints as
> to where I could look for differences?
>
> Thanks a lot,
> Manohar
>
> [1]
> https://wikis.oracle.com/display/HotSpotInternals/RangeCheckElimination
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120116/691adbe4/attachment.html 

From david.holmes at oracle.com  Sun Jan 15 13:47:09 2012
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 16 Jan 2012 07:47:09 +1000
Subject: RFR (XS): 7126732: MAC: Require Mac OS X builds/tests for JPRT
	integrate jobs for HotSpot
In-Reply-To: <4F11B537.2000504@oracle.com>
References: <4F11B537.2000504@oracle.com>
Message-ID: <4F13495D.9040302@oracle.com>

Hi Jim,

On 15/01/2012 3:02 AM, James Melvin wrote:
> Greetings,
>
> We're ready to require HotSpot builds on Mac OS X for JPRT integrate
> jobs. There are 3 mac-minis in each queue. Build/Test times are short
> relative to other platforms. Uses the stable Linux testlist for now.

Maybe I'm missing something but isn't the OSX code currently primarily 
in the jdk7u-macosx repo? Or is all the hotspot code already in mainline?

David
-----

> http://cr.openjdk.java.net/~jmelvin/7126732/webrev.00
>
> Tested with *several* JPRT submissions for other bugfixes. I'd like to
> integrate this change right after the current snapshot window.
>
> Feedback welcome.
>
> Thanks,
>
> Jim

From vladimir.kozlov at oracle.com  Sun Jan 15 14:30:12 2012
From: vladimir.kozlov at oracle.com (Vladimir Kozlov)
Date: Sun, 15 Jan 2012 14:30:12 -0800
Subject: Detecting range check elimination with PrintAssembly
In-Reply-To: <CA+cQ+tTd_yhyu1=06JSgskU-kJ_qh=cXmnYdE6a7TgCR7s=uxg@mail.gmail.com>
References: <CABhSbLyAwFLFwhmTzPE=0DiXK8e8kbmvr3tr-0AQ=KyFz1Mjsw@mail.gmail.com>
	<CA+cQ+tTd_yhyu1=06JSgskU-kJ_qh=cXmnYdE6a7TgCR7s=uxg@mail.gmail.com>
Message-ID: <4F135374.8020004@oracle.com>

Manohar,

As Kris pointed you need to fix your example:

void method1(){
...
    for(int k = 0; j < cols; k++){
                   ^ 'k' ?

void method2(){
...
   for(int j=0 ; i< rows2; j++){
                 ^ 'j' ?

Second, your two test methods are different so you can't directly compare them. method1() iterates over rows using 
middle loop index 'j' and method2() uses external loop index 'i'. Unless they are typos again.

Third, PrintOptoAssembly flag is not 'old'. It can be only used in debug VM to print 'pseudo' assembler by JIT compiler 
itself and not by hsdis disassembler. PrintAssembly flag can be used with product VM but it needs hsdis library.

If you are using jdk7 there are few flags you can use to print loop optimizations information. They need debug version 
of VM but it is not problem for you, I think, since you can use debug PrintOptoAssembly flag.

-XX:+TraceLoopOpts prints all happened loop optimizations and loop tree after each round of loop opts,
-XX:+TraceLoopPredicate prints RC information when it is moved from a loop,
-XX:+TraceRangeLimitCheck prints additional information for RC elimination optimization.

Fourth, range check expression in your example is not what you think. RC expression should be next:
(i*stride+offset) where 'i' is loop variable, 'stride' is constant and 'offset' is loop invariant.

In your example 'offset' is (j * cols) since it is loop invariant, 'k' is loop variable and stride is '1' (one).
In both your methods RC will be moved out of inner loop so the code for it will be the same. The only difference in 
these methods will be where and how (j * cols) and (i * cols) expressions are calculated.

Regards,
Vladimir

On 1/15/12 12:22 PM, Krystal Mok wrote:
> Hi,
>
> In your PrintOptoAssembly output snippet, the instruction at 0x13e is a LoadRange, which loads the range from the header
> of an array:
>
> (from x86_64.ad <http://x86_64.ad>)
> // Load Range
> instruct loadRange(rRegI dst, memory mem)
> %{
>    match(Set dst (LoadRange mem));
>
>    ins_cost(125); // XXX
>    format %{ "movl    $dst, $mem\t# range" %}
>    opcode(0x8B);
>    ins_encode(REX_reg_mem(dst, mem), OpcP, reg_mem(dst, mem));
>    ins_pipe(ialu_reg_mem);
> %}
>
> That's not a range check just yet; the real check, if any, should come after the null check, in the form of comparing
> something else with RSI. But you didn't show what's after the null check, how RSI is used, so it's hard to say what
> you're seeing in your example.
>
> As for the two test examples, could you paste the entire source code, with the PrintOptoAssembly output of method1() and
> method2() ? The first example looks weird, maybe it's a typo but you're using "j < cols" as the loop condition for the
> inner loop.
>
> I'd guess it's the difference in locality that made the difference in performance in your two tests.
>
> - Kris
>
> On Mon, Jan 16, 2012 at 1:59 AM, Manohar Jonnalagedda <manojo10386 at gmail.com <mailto:manojo10386 at gmail.com>> wrote:
>
>     Hello,
>
>     following this reference on Range Check Elimination done by the Hotspot compiler [1], I was keen in knowing how I
>     can detect whether range checks are taking place in loops by inspecting output using the PrintAssembly flag; with
>     the old PrintOptoAssembly flag, I have seen output such as the following, which I assume to be range checks :
>
>     B11: #  B73 B12 &lt;- B10  Freq: 1.21365
>     139     movq    RAX, [rsp + #24]  # spill
>     13e     movl    RSI, [RAX + #12 (8-bit)]  # range
>     141     NullCheck RAX
>
>     What is the equivalent with the new PrintAssembly flag (using hsdis)?
>
>     Moreover, as stated on the wiki page [1], loops are optimized if the stride is a compile-time constant. I performed
>     a few tests on a kmeans program, with 3 nested loops, having the following (high-level) structure:
>
>     ===
>     void method1(){
>     //loop 1
>     for(int i = 0; i< rows1; i++){
>       //...
>        for(int j = 0; j< rows2; j++){
>        //...
>         for(int k = 0; j < cols; k++){ array[j * cols + k] = //...}
>        }
>     }
>     }
>
>     void method2(){
>     //loop 2
>     for(int i =0; i < rows1; i++){
>        for(int j=0 ; i< rows2; j++){
>          for(int k=0 ; k< cols; k++){
>            array[i*cols+k] = //...
>          }
>        }
>     }
>     }
>
>     void main(){
>
>       do{
>         method1(); method2();
>       }while(!converged)
>
>     }
>     ====
>
>     In the first test, cols is an int whose value is determined at runtime (by reading a file), in the second test, it
>     is given as a compile-time constant(3). In the second test, there is a */significant*/ speed-up (around 40%).
>     However, when studying the diff of the output of PrintOptoAssembly for both method1 and method2, there is no
>     difference (apart from slight value changes in frequency). Would you have any hints as to where I could look for
>     differences?
>
>     Thanks a lot,
>     Manohar
>
>     [1] https://wikis.oracle.com/display/HotSpotInternals/RangeCheckElimination
>
>

From daniel.daugherty at oracle.com  Sun Jan 15 15:35:31 2012
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Sun, 15 Jan 2012 16:35:31 -0700
Subject: RFR (XS): 7126732: MAC: Require Mac OS X builds/tests for JPRT
	integrate jobs for HotSpot
In-Reply-To: <4F13495D.9040302@oracle.com>
References: <4F11B537.2000504@oracle.com> <4F13495D.9040302@oracle.com>
Message-ID: <4F1362C3.4060600@oracle.com>

The hotspot part of the MacOS X port was pushed to Main_Baseline on
2011.10.21. I've attached the changeset notification.

The rest of the MacOS X port is primarily in jdk7u-osx forest...

Item of interest: the second reviewer that got back to me on those
hotspot bits was this 'dholmes' guy... :-)

Dan


On 1/15/12 2:47 PM, David Holmes wrote:
> Hi Jim,
>
> On 15/01/2012 3:02 AM, James Melvin wrote:
>> Greetings,
>>
>> We're ready to require HotSpot builds on Mac OS X for JPRT integrate
>> jobs. There are 3 mac-minis in each queue. Build/Test times are short
>> relative to other platforms. Uses the stable Linux testlist for now.
>
> Maybe I'm missing something but isn't the OSX code currently primarily 
> in the jdk7u-macosx repo? Or is all the hotspot code already in mainline?
>
> David
> -----
>
>> http://cr.openjdk.java.net/~jmelvin/7126732/webrev.00
>>
>> Tested with *several* JPRT submissions for other bugfixes. I'd like to
>> integrate this change right after the current snapshot window.
>>
>> Feedback welcome.
>>
>> Thanks,
>>
>> Jim
-------------- next part --------------
An embedded message was scrubbed...
From: daniel.daugherty at oracle.com
Subject: hg: hsx/hotspot-main/hotspot: 3 new changesets
Date: Fri, 21 Oct 2011 07:29:37 +0000
Size: 6613
Url: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120115/97113485/AttachedMessage-0001.nws 

From james.melvin at oracle.com  Sun Jan 15 18:25:30 2012
From: james.melvin at oracle.com (James Melvin)
Date: Sun, 15 Jan 2012 21:25:30 -0500
Subject: RFR (XS): 7126732: MAC: Require Mac OS X builds/tests for JPRT
	integrate jobs for HotSpot
In-Reply-To: <4F1362C3.4060600@oracle.com>
References: <4F11B537.2000504@oracle.com> <4F13495D.9040302@oracle.com>
	<4F1362C3.4060600@oracle.com>
Message-ID: <4F138A9A.8030102@oracle.com>

 > The hotspot part of the MacOS X port was pushed to Main_Baseline on
 > 2011.10.21. I've attached the changeset notification.

Yes, the current HotSpot source includes the port to Mac OS X. The
change I am proposing will simply add Mac OS X to the default set of
platforms in JPRT which must pass for integrate jobs. If the job does
not pass, the changeset will not be integrated. This required smoke test
ensures HotSpot is beta quality or better at any point in time. The only
risk is if future changesets break the Mac OS X build or test. Such
breakage would need to be resolved before the change can be integrated.

- Jim


On 1/15/12 6:35 PM, Daniel D. Daugherty wrote:
> The hotspot part of the MacOS X port was pushed to Main_Baseline on
> 2011.10.21. I've attached the changeset notification.
>
> The rest of the MacOS X port is primarily in jdk7u-osx forest...
>
> Item of interest: the second reviewer that got back to me on those
> hotspot bits was this 'dholmes' guy... :-)
>
> Dan
>
>
> On 1/15/12 2:47 PM, David Holmes wrote:
>> Hi Jim,
>>
>> On 15/01/2012 3:02 AM, James Melvin wrote:
>>> Greetings,
>>>
>>> We're ready to require HotSpot builds on Mac OS X for JPRT integrate
>>> jobs. There are 3 mac-minis in each queue. Build/Test times are short
>>> relative to other platforms. Uses the stable Linux testlist for now.
>>
>> Maybe I'm missing something but isn't the OSX code currently primarily
>> in the jdk7u-macosx repo? Or is all the hotspot code already in mainline?
>>
>> David
>> -----
>>
>>> http://cr.openjdk.java.net/~jmelvin/7126732/webrev.00
>>>
>>> Tested with *several* JPRT submissions for other bugfixes. I'd like to
>>> integrate this change right after the current snapshot window.
>>>
>>> Feedback welcome.
>>>
>>> Thanks,
>>>
>>> Jim

From david.holmes at oracle.com  Sun Jan 15 18:42:49 2012
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 16 Jan 2012 12:42:49 +1000
Subject: RFR (XS): 7126732: MAC: Require Mac OS X builds/tests for JPRT
	integrate jobs for HotSpot
In-Reply-To: <4F1362C3.4060600@oracle.com>
References: <4F11B537.2000504@oracle.com> <4F13495D.9040302@oracle.com>
	<4F1362C3.4060600@oracle.com>
Message-ID: <4F138EA9.2050203@oracle.com>


On 16/01/2012 9:35 AM, Daniel D. Daugherty wrote:
> The hotspot part of the MacOS X port was pushed to Main_Baseline on
> 2011.10.21. I've attached the changeset notification.
>
> The rest of the MacOS X port is primarily in jdk7u-osx forest...
>
> Item of interest: the second reviewer that got back to me on those
> hotspot bits was this 'dholmes' guy... :-)

Yes but since then I've been watching the 7u-osx updates and seeing 
bunches of bugs filed, and so was left wondering exactly what state the 
OSX "port" is in on the hotspot side. Or for that matter on the JDK side 
as the hotspot build has to be placed into an OSX JDK.

As long as the code builds and passes the tests for 7 and 8 then that 
should be fine - and Jim indicates that is the case.

Cheers,
David

> Dan
>
>
> On 1/15/12 2:47 PM, David Holmes wrote:
>> Hi Jim,
>>
>> On 15/01/2012 3:02 AM, James Melvin wrote:
>>> Greetings,
>>>
>>> We're ready to require HotSpot builds on Mac OS X for JPRT integrate
>>> jobs. There are 3 mac-minis in each queue. Build/Test times are short
>>> relative to other platforms. Uses the stable Linux testlist for now.
>>
>> Maybe I'm missing something but isn't the OSX code currently primarily
>> in the jdk7u-macosx repo? Or is all the hotspot code already in mainline?
>>
>> David
>> -----
>>
>>> http://cr.openjdk.java.net/~jmelvin/7126732/webrev.00
>>>
>>> Tested with *several* JPRT submissions for other bugfixes. I'd like to
>>> integrate this change right after the current snapshot window.
>>>
>>> Feedback welcome.
>>>
>>> Thanks,
>>>
>>> Jim

From manojo10386 at gmail.com  Mon Jan 16 01:39:49 2012
From: manojo10386 at gmail.com (Manohar Jonnalagedda)
Date: Mon, 16 Jan 2012 10:39:49 +0100
Subject: Detecting range check elimination with PrintAssembly
In-Reply-To: <4F135374.8020004@oracle.com>
References: <CABhSbLyAwFLFwhmTzPE=0DiXK8e8kbmvr3tr-0AQ=KyFz1Mjsw@mail.gmail.com>
	<CA+cQ+tTd_yhyu1=06JSgskU-kJ_qh=cXmnYdE6a7TgCR7s=uxg@mail.gmail.com>
	<4F135374.8020004@oracle.com>
Message-ID: <CABhSbLy0gd3BFmBRX-aUgEt7h2HBHer_AQENMGWYTRN4p5-wEg@mail.gmail.com>

Hi Kris, Vladimir,

thanks for both your responses.

Second, your two test methods are different so you can't directly compare
> them. method1() iterates over rows using middle loop index 'j' and
> method2() uses external loop index 'i'. Unless they are typos again.
>

You are right, these are indeed typos. As Kris suggested, I have the code
printed here: http://pastebin.com/xRFD1Nt1. The methods corresponding to
method1, and method2 are constructNearestClusterVector and
computeNewCentroids. Their PrintOptoAssembly outputs are respectively at
http://pastebin.com/1evN8b3K and http://pastebin.com/FxkVWTD5

Also, it seems I have not explained myself correctly. I am not trying to
compare the performance of method1 with respect to that of method2: method1
and method2 both run in the same program. What I am trying to compare is
their performance in two cases:
- when cols is a compile-time constant (much faster)
- when cols is a value determined at run-time



> If you are using jdk7 there are few flags you can use to print loop
> optimizations information. They need debug version of VM but it is not
> problem for you, I think, since you can use debug PrintOptoAssembly flag.
>
> -XX:+TraceLoopOpts prints all happened loop optimizations and loop tree
> after each round of loop opts,
> -XX:+TraceLoopPredicate prints RC information when it is moved from a loop,
> -XX:+TraceRangeLimitCheck prints additional information for RC elimination
> optimization.
>

Thanks for these, I will have a look at what they output.

Fourth, range check expression in your example is not what you think. RC
> expression should be next:
> (i*stride+offset) where 'i' is loop variable, 'stride' is constant and
> 'offset' is loop invariant.
>
> In your example 'offset' is (j * cols) since it is loop invariant, 'k' is
> loop variable and stride is '1' (one).
> In both your methods RC will be moved out of inner loop so the code for it
> will be the same. The only difference in these methods will be where and
> how (j * cols) and (i * cols) expressions are calculated.
>

I'd guess it's the difference in locality that made the difference in
>> performance in your two tests.
>>
>
 Thanks for the explanation. I understand from the above that the assembly
output in both cases mentioned above may not be different, because the
expressions are similar. The difference in runtime (due to cols being a
compile-time constant) will be visible elsewhere. Is that right? If so,
where would I be able to detect this?

Cheers,
Manohar


> In your PrintOptoAssembly output snippet, the instruction at 0x13e is a
>> LoadRange, which loads the range from the header
>> of an array:
>>
>> (from x86_64.ad <http://x86_64.ad>)
>>
>> // Load Range
>> instruct loadRange(rRegI dst, memory mem)
>> %{
>>   match(Set dst (LoadRange mem));
>>
>>   ins_cost(125); // XXX
>>   format %{ "movl    $dst, $mem\t# range" %}
>>   opcode(0x8B);
>>   ins_encode(REX_reg_mem(dst, mem), OpcP, reg_mem(dst, mem));
>>   ins_pipe(ialu_reg_mem);
>> %}
>>
>> That's not a range check just yet; the real check, if any, should come
>> after the null check, in the form of comparing
>> something else with RSI. But you didn't show what's after the null check,
>> how RSI is used, so it's hard to say what
>> you're seeing in your example.
>>
>> As for the two test examples, could you paste the entire source code,
>> with the PrintOptoAssembly output of method1() and
>> method2() ? The first example looks weird, maybe it's a typo but you're
>> using "j < cols" as the loop condition for the
>> inner loop.
>>
>>
>
>> - Kris
>>
>> On Mon, Jan 16, 2012 at 1:59 AM, Manohar Jonnalagedda <
>> manojo10386 at gmail.com <mailto:manojo10386 at gmail.com>> wrote:
>>
>>    Hello,
>>
>>    following this reference on Range Check Elimination done by the
>> Hotspot compiler [1], I was keen in knowing how I
>>    can detect whether range checks are taking place in loops by
>> inspecting output using the PrintAssembly flag; with
>>    the old PrintOptoAssembly flag, I have seen output such as the
>> following, which I assume to be range checks :
>>
>>    B11: #  B73 B12 &lt;- B10  Freq: 1.21365
>>    139     movq    RAX, [rsp + #24]  # spill
>>    13e     movl    RSI, [RAX + #12 (8-bit)]  # range
>>    141     NullCheck RAX
>>
>>    What is the equivalent with the new PrintAssembly flag (using hsdis)?
>>
>>    Moreover, as stated on the wiki page [1], loops are optimized if the
>> stride is a compile-time constant. I performed
>>    a few tests on a kmeans program, with 3 nested loops, having the
>> following (high-level) structure:
>>
>>    ===
>>    void method1(){
>>    //loop 1
>>    for(int i = 0; i< rows1; i++){
>>      //...
>>       for(int j = 0; j< rows2; j++){
>>       //...
>>        for(int k = 0; j < cols; k++){ array[j * cols + k] = //...}
>>       }
>>    }
>>    }
>>
>>    void method2(){
>>    //loop 2
>>    for(int i =0; i < rows1; i++){
>>       for(int j=0 ; i< rows2; j++){
>>         for(int k=0 ; k< cols; k++){
>>           array[i*cols+k] = //...
>>         }
>>       }
>>    }
>>    }
>>
>>    void main(){
>>
>>      do{
>>        method1(); method2();
>>      }while(!converged)
>>
>>    }
>>    ====
>>
>>    In the first test, cols is an int whose value is determined at runtime
>> (by reading a file), in the second test, it
>>    is given as a compile-time constant(3). In the second test, there is a
>> */significant*/ speed-up (around 40%).
>>
>>    However, when studying the diff of the output of PrintOptoAssembly for
>> both method1 and method2, there is no
>>    difference (apart from slight value changes in frequency). Would you
>> have any hints as to where I could look for
>>    differences?
>>
>>    Thanks a lot,
>>    Manohar
>>
>>    [1]
>> https://wikis.oracle.com/display/HotSpotInternals/RangeCheckElimination
>>
>>
>>
> As Kris pointed you need to fix your example:
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120116/65204a22/attachment.html 

From mikael.gerdin at oracle.com  Mon Jan 16 01:44:46 2012
From: mikael.gerdin at oracle.com (Mikael Gerdin)
Date: Mon, 16 Jan 2012 10:44:46 +0100
Subject: RFR (XS): 7126732: MAC: Require Mac OS X builds/tests for JPRT
	integrate jobs for HotSpot
In-Reply-To: <4F11B537.2000504@oracle.com>
References: <4F11B537.2000504@oracle.com>
Message-ID: <4F13F18E.1080300@oracle.com>

Hi Jim

on line 575, is there any particular reason that you don't add macosx to 
jprt.make.rule.test.targets.standard.internalvmtests?

/Mikael

On 2012-01-14 18:02, James Melvin wrote:
> Greetings,
>
> We're ready to require HotSpot builds on Mac OS X for JPRT integrate
> jobs. There are 3 mac-minis in each queue. Build/Test times are short
> relative to other platforms. Uses the stable Linux testlist for now.
>
> http://cr.openjdk.java.net/~jmelvin/7126732/webrev.00
>
> Tested with *several* JPRT submissions for other bugfixes. I'd like to
> integrate this change right after the current snapshot window.
>
> Feedback welcome.
>
> Thanks,
>
> Jim

From vitalyd at gmail.com  Mon Jan 16 10:55:11 2012
From: vitalyd at gmail.com (Vitaly Davidovich)
Date: Mon, 16 Jan 2012 13:55:11 -0500
Subject: Detecting range check elimination with PrintAssembly
In-Reply-To: <CABhSbLy0gd3BFmBRX-aUgEt7h2HBHer_AQENMGWYTRN4p5-wEg@mail.gmail.com>
References: <CABhSbLyAwFLFwhmTzPE=0DiXK8e8kbmvr3tr-0AQ=KyFz1Mjsw@mail.gmail.com>
	<CA+cQ+tTd_yhyu1=06JSgskU-kJ_qh=cXmnYdE6a7TgCR7s=uxg@mail.gmail.com>
	<4F135374.8020004@oracle.com>
	<CABhSbLy0gd3BFmBRX-aUgEt7h2HBHer_AQENMGWYTRN4p5-wEg@mail.gmail.com>
Message-ID: <CAHjP37HLTt_=WuEgJjk1rNmLbmohx8vHdOS_3ohnyJ1PxeBK1Q@mail.gmail.com>

Hi Manohar,

Are you repeatedly seeing ~40% speedup with a compile-time constant? In the
two assembly dumps you posted, the computeNewCentroids seems to have some
loop unrolling + non-temporal prefetch instructions that I don't see (at a
cursory glance, albeit) in the 2nd method.  How big is the input array for
these functions? If it's larger than your highest level cache, can you try
running the same tests (constant vs non-constant) with a size that would
fit into L2/L3?

What cpu are you running these tests on?

Vitaly

On Mon, Jan 16, 2012 at 4:39 AM, Manohar Jonnalagedda <manojo10386 at gmail.com
> wrote:

> Hi Kris, Vladimir,
>
> thanks for both your responses.
>
> Second, your two test methods are different so you can't directly compare
>> them. method1() iterates over rows using middle loop index 'j' and
>> method2() uses external loop index 'i'. Unless they are typos again.
>>
>
> You are right, these are indeed typos. As Kris suggested, I have the code
> printed here: http://pastebin.com/xRFD1Nt1. The methods corresponding to
> method1, and method2 are constructNearestClusterVector and
> computeNewCentroids. Their PrintOptoAssembly outputs are respectively at
> http://pastebin.com/1evN8b3K and http://pastebin.com/FxkVWTD5
>
> Also, it seems I have not explained myself correctly. I am not trying to
> compare the performance of method1 with respect to that of method2: method1
> and method2 both run in the same program. What I am trying to compare is
> their performance in two cases:
> - when cols is a compile-time constant (much faster)
> - when cols is a value determined at run-time
>
>
>
>> If you are using jdk7 there are few flags you can use to print loop
>> optimizations information. They need debug version of VM but it is not
>> problem for you, I think, since you can use debug PrintOptoAssembly flag.
>>
>> -XX:+TraceLoopOpts prints all happened loop optimizations and loop tree
>> after each round of loop opts,
>> -XX:+TraceLoopPredicate prints RC information when it is moved from a
>> loop,
>> -XX:+TraceRangeLimitCheck prints additional information for RC
>> elimination optimization.
>>
>
> Thanks for these, I will have a look at what they output.
>
> Fourth, range check expression in your example is not what you think. RC
>> expression should be next:
>> (i*stride+offset) where 'i' is loop variable, 'stride' is constant and
>> 'offset' is loop invariant.
>>
>> In your example 'offset' is (j * cols) since it is loop invariant, 'k' is
>> loop variable and stride is '1' (one).
>> In both your methods RC will be moved out of inner loop so the code for
>> it will be the same. The only difference in these methods will be where and
>> how (j * cols) and (i * cols) expressions are calculated.
>>
>
>  I'd guess it's the difference in locality that made the difference in
>>> performance in your two tests.
>>>
>>
>  Thanks for the explanation. I understand from the above that the assembly
> output in both cases mentioned above may not be different, because the
> expressions are similar. The difference in runtime (due to cols being a
> compile-time constant) will be visible elsewhere. Is that right? If so,
> where would I be able to detect this?
>
> Cheers,
> Manohar
>
>
>>  In your PrintOptoAssembly output snippet, the instruction at 0x13e is a
>>> LoadRange, which loads the range from the header
>>> of an array:
>>>
>>> (from x86_64.ad <http://x86_64.ad>)
>>>
>>> // Load Range
>>> instruct loadRange(rRegI dst, memory mem)
>>> %{
>>>   match(Set dst (LoadRange mem));
>>>
>>>   ins_cost(125); // XXX
>>>   format %{ "movl    $dst, $mem\t# range" %}
>>>   opcode(0x8B);
>>>   ins_encode(REX_reg_mem(dst, mem), OpcP, reg_mem(dst, mem));
>>>   ins_pipe(ialu_reg_mem);
>>> %}
>>>
>>> That's not a range check just yet; the real check, if any, should come
>>> after the null check, in the form of comparing
>>> something else with RSI. But you didn't show what's after the null
>>> check, how RSI is used, so it's hard to say what
>>> you're seeing in your example.
>>>
>>> As for the two test examples, could you paste the entire source code,
>>> with the PrintOptoAssembly output of method1() and
>>> method2() ? The first example looks weird, maybe it's a typo but you're
>>> using "j < cols" as the loop condition for the
>>> inner loop.
>>>
>>>
>>
>>> - Kris
>>>
>>> On Mon, Jan 16, 2012 at 1:59 AM, Manohar Jonnalagedda <
>>> manojo10386 at gmail.com <mailto:manojo10386 at gmail.com>> wrote:
>>>
>>>    Hello,
>>>
>>>    following this reference on Range Check Elimination done by the
>>> Hotspot compiler [1], I was keen in knowing how I
>>>    can detect whether range checks are taking place in loops by
>>> inspecting output using the PrintAssembly flag; with
>>>    the old PrintOptoAssembly flag, I have seen output such as the
>>> following, which I assume to be range checks :
>>>
>>>    B11: #  B73 B12 &lt;- B10  Freq: 1.21365
>>>    139     movq    RAX, [rsp + #24]  # spill
>>>    13e     movl    RSI, [RAX + #12 (8-bit)]  # range
>>>    141     NullCheck RAX
>>>
>>>    What is the equivalent with the new PrintAssembly flag (using hsdis)?
>>>
>>>    Moreover, as stated on the wiki page [1], loops are optimized if the
>>> stride is a compile-time constant. I performed
>>>    a few tests on a kmeans program, with 3 nested loops, having the
>>> following (high-level) structure:
>>>
>>>    ===
>>>    void method1(){
>>>    //loop 1
>>>    for(int i = 0; i< rows1; i++){
>>>      //...
>>>       for(int j = 0; j< rows2; j++){
>>>       //...
>>>        for(int k = 0; j < cols; k++){ array[j * cols + k] = //...}
>>>       }
>>>    }
>>>    }
>>>
>>>    void method2(){
>>>    //loop 2
>>>    for(int i =0; i < rows1; i++){
>>>       for(int j=0 ; i< rows2; j++){
>>>         for(int k=0 ; k< cols; k++){
>>>           array[i*cols+k] = //...
>>>         }
>>>       }
>>>    }
>>>    }
>>>
>>>    void main(){
>>>
>>>      do{
>>>        method1(); method2();
>>>      }while(!converged)
>>>
>>>    }
>>>    ====
>>>
>>>    In the first test, cols is an int whose value is determined at
>>> runtime (by reading a file), in the second test, it
>>>    is given as a compile-time constant(3). In the second test, there is
>>> a */significant*/ speed-up (around 40%).
>>>
>>>    However, when studying the diff of the output of PrintOptoAssembly
>>> for both method1 and method2, there is no
>>>    difference (apart from slight value changes in frequency). Would you
>>> have any hints as to where I could look for
>>>    differences?
>>>
>>>    Thanks a lot,
>>>    Manohar
>>>
>>>    [1]
>>> https://wikis.oracle.com/display/HotSpotInternals/RangeCheckElimination
>>>
>>>
>>>
>> As Kris pointed you need to fix your example:
>
>


-- 
Vitaly
617-548-7007 (mobile)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120116/59687f71/attachment-0001.html 

From james.melvin at oracle.com  Mon Jan 16 13:46:34 2012
From: james.melvin at oracle.com (James Melvin)
Date: Mon, 16 Jan 2012 16:46:34 -0500
Subject: RFR (XS): 7126732: MAC: Require Mac OS X builds/tests for JPRT
	integrate jobs for HotSpot
In-Reply-To: <4F13F18E.1080300@oracle.com>
References: <4F11B537.2000504@oracle.com> <4F13F18E.1080300@oracle.com>
Message-ID: <4F149ABA.3000708@oracle.com>

Hi Mikael,

Nice find! I've added the missing line to add internalvmtests to each
JPRT run on Mac OS X. New webrev posted and JPRT test job complete.

WEBREV: http://cr.openjdk.java.net/~jmelvin/7126732/webrev.01
JPRT: 
http://prt-web.us.oracle.com//archive/2012/01/2012-01-16-172504.jmelvin.7126732

Thanks!

Jim



On 1/16/12 4:44 AM, Mikael Gerdin wrote:
> Hi Jim
>
> on line 575, is there any particular reason that you don't add macosx to
> jprt.make.rule.test.targets.standard.internalvmtests?
>
> /Mikael
>
> On 2012-01-14 18:02, James Melvin wrote:
>> Greetings,
>>
>> We're ready to require HotSpot builds on Mac OS X for JPRT integrate
>> jobs. There are 3 mac-minis in each queue. Build/Test times are short
>> relative to other platforms. Uses the stable Linux testlist for now.
>>
>> http://cr.openjdk.java.net/~jmelvin/7126732/webrev.00
>>
>> Tested with *several* JPRT submissions for other bugfixes. I'd like to
>> integrate this change right after the current snapshot window.
>>
>> Feedback welcome.
>>
>> Thanks,
>>
>> Jim

From vladimir.kozlov at oracle.com  Mon Jan 16 13:58:32 2012
From: vladimir.kozlov at oracle.com (Vladimir Kozlov)
Date: Mon, 16 Jan 2012 13:58:32 -0800
Subject: Detecting range check elimination with PrintAssembly
In-Reply-To: <CABhSbLy0gd3BFmBRX-aUgEt7h2HBHer_AQENMGWYTRN4p5-wEg@mail.gmail.com>
References: <CABhSbLyAwFLFwhmTzPE=0DiXK8e8kbmvr3tr-0AQ=KyFz1Mjsw@mail.gmail.com>	<CA+cQ+tTd_yhyu1=06JSgskU-kJ_qh=cXmnYdE6a7TgCR7s=uxg@mail.gmail.com>	<4F135374.8020004@oracle.com>
	<CABhSbLy0gd3BFmBRX-aUgEt7h2HBHer_AQENMGWYTRN4p5-wEg@mail.gmail.com>
Message-ID: <4F149D88.2060305@oracle.com>

 > be different, because the expressions are similar. The difference in runtime (due to cols being a compile-time
 > constant) will be visible elsewhere. Is that right? If so, where would I be able to detect this?

In such situations we usually use some visual tools to see difference between log outputs. At least you can use 'diff'. 
You may need to replace instructions addresses in outputs (number at the beginning of lines) with the same value to 
match. There are few tricks you may use to get similar PrintOptoAssembly output. Use next flags to avoid mixing output 
from program output and from 2 compiler threads (flags stop program until a method is compiled and run only one compiler 
thread):

-Xbatch -XX:CICompilerCount=1

Also add -XX:+PrintCompilation -XX:+PrintInlining to see what method is compiled and inlined. Note that you may see 
similar output for individual methods but could be big difference in compiled caller (computeAll()) method where 2 loop 
methods could be inlined. So you need to compare all compiled methods.

In general, to have constant as loop limit is always win because some checks in generated code could be avoided and more 
optimizations could be done for such loops. Use -XX:+TraceLoopOpts to see what loop optimizations are done in both cases.

For example, in your code you set 'x_col = 3', as result the next loop in constructNearestClusterVector() will be fully 
unrolled when this method is inlined into computeAll() and x_col is replaced with '3':

         for(k = 0; k < x_col; k++) {
           double tmp = x[i*x_col + k] - mu[j* mu_col + k];
           dist += tmp * tmp;
         }

Vladimir

On 1/16/12 1:39 AM, Manohar Jonnalagedda wrote:
> Hi Kris, Vladimir,
>
> thanks for both your responses.
>
>     Second, your two test methods are different so you can't directly compare them. method1() iterates over rows using
>     middle loop index 'j' and method2() uses external loop index 'i'. Unless they are typos again.
>
>
> You are right, these are indeed typos. As Kris suggested, I have the code printed here: http://pastebin.com/xRFD1Nt1.
> The methods corresponding to method1, and method2 are constructNearestClusterVector and computeNewCentroids. Their
> PrintOptoAssembly outputs are respectively at http://pastebin.com/1evN8b3K and http://pastebin.com/FxkVWTD5
>
> Also, it seems I have not explained myself correctly. I am not trying to compare the performance of method1 with respect
> to that of method2: method1 and method2 both run in the same program. What I am trying to compare is their performance
> in two cases:
> - when cols is a compile-time constant (much faster)
> - when cols is a value determined at run-time
>
>     If you are using jdk7 there are few flags you can use to print loop optimizations information. They need debug
>     version of VM but it is not problem for you, I think, since you can use debug PrintOptoAssembly flag.
>
>     -XX:+TraceLoopOpts prints all happened loop optimizations and loop tree after each round of loop opts,
>     -XX:+TraceLoopPredicate prints RC information when it is moved from a loop,
>     -XX:+TraceRangeLimitCheck prints additional information for RC elimination optimization.
>
>
> Thanks for these, I will have a look at what they output.
>
>     Fourth, range check expression in your example is not what you think. RC expression should be next:
>     (i*stride+offset) where 'i' is loop variable, 'stride' is constant and 'offset' is loop invariant.
>
>     In your example 'offset' is (j * cols) since it is loop invariant, 'k' is loop variable and stride is '1' (one).
>     In both your methods RC will be moved out of inner loop so the code for it will be the same. The only difference in
>     these methods will be where and how (j * cols) and (i * cols) expressions are calculated.
>
>
>         I'd guess it's the difference in locality that made the difference in performance in your two tests.
>
>   Thanks for the explanation. I understand from the above that the assembly output in both cases mentioned above may not
> be different, because the expressions are similar. The difference in runtime (due to cols being a compile-time constant)
> will be visible elsewhere. Is that right? If so, where would I be able to detect this?
>
> Cheers,
> Manohar
>
>         In your PrintOptoAssembly output snippet, the instruction at 0x13e is a LoadRange, which loads the range from
>         the header
>         of an array:
>
>         (from x86_64.ad <http://x86_64.ad> <http://x86_64.ad>)
>
>         // Load Range
>         instruct loadRange(rRegI dst, memory mem)
>         %{
>            match(Set dst (LoadRange mem));
>
>            ins_cost(125); // XXX
>            format %{ "movl    $dst, $mem\t# range" %}
>            opcode(0x8B);
>            ins_encode(REX_reg_mem(dst, mem), OpcP, reg_mem(dst, mem));
>            ins_pipe(ialu_reg_mem);
>         %}
>
>         That's not a range check just yet; the real check, if any, should come after the null check, in the form of
>         comparing
>         something else with RSI. But you didn't show what's after the null check, how RSI is used, so it's hard to say what
>         you're seeing in your example.
>
>         As for the two test examples, could you paste the entire source code, with the PrintOptoAssembly output of
>         method1() and
>         method2() ? The first example looks weird, maybe it's a typo but you're using "j < cols" as the loop condition
>         for the
>         inner loop.
>
>
>         - Kris
>
>         On Mon, Jan 16, 2012 at 1:59 AM, Manohar Jonnalagedda <manojo10386 at gmail.com <mailto:manojo10386 at gmail.com>
>         <mailto:manojo10386 at gmail.com <mailto:manojo10386 at gmail.com>>> wrote:
>
>             Hello,
>
>             following this reference on Range Check Elimination done by the Hotspot compiler [1], I was keen in knowing
>         how I
>             can detect whether range checks are taking place in loops by inspecting output using the PrintAssembly flag;
>         with
>             the old PrintOptoAssembly flag, I have seen output such as the following, which I assume to be range checks :
>
>             B11: #  B73 B12 &lt;- B10  Freq: 1.21365
>             139     movq    RAX, [rsp + #24]  # spill
>             13e     movl    RSI, [RAX + #12 (8-bit)]  # range
>             141     NullCheck RAX
>
>             What is the equivalent with the new PrintAssembly flag (using hsdis)?
>
>             Moreover, as stated on the wiki page [1], loops are optimized if the stride is a compile-time constant. I
>         performed
>             a few tests on a kmeans program, with 3 nested loops, having the following (high-level) structure:
>
>             ===
>             void method1(){
>             //loop 1
>             for(int i = 0; i< rows1; i++){
>               //...
>                for(int j = 0; j< rows2; j++){
>                //...
>                 for(int k = 0; j < cols; k++){ array[j * cols + k] = //...}
>                }
>             }
>             }
>
>             void method2(){
>             //loop 2
>             for(int i =0; i < rows1; i++){
>                for(int j=0 ; i< rows2; j++){
>                  for(int k=0 ; k< cols; k++){
>                    array[i*cols+k] = //...
>                  }
>                }
>             }
>             }
>
>             void main(){
>
>               do{
>                 method1(); method2();
>               }while(!converged)
>
>             }
>             ====
>
>             In the first test, cols is an int whose value is determined at runtime (by reading a file), in the second
>         test, it
>             is given as a compile-time constant(3). In the second test, there is a */significant*/ speed-up (around 40%).
>
>             However, when studying the diff of the output of PrintOptoAssembly for both method1 and method2, there is no
>             difference (apart from slight value changes in frequency). Would you have any hints as to where I could look for
>             differences?
>
>             Thanks a lot,
>             Manohar
>
>             [1] https://wikis.oracle.com/display/HotSpotInternals/RangeCheckElimination
>
>
>
> As Kris pointed you need to fix your example:
>

From vitalyd at gmail.com  Mon Jan 16 14:37:42 2012
From: vitalyd at gmail.com (Vitaly Davidovich)
Date: Mon, 16 Jan 2012 17:37:42 -0500
Subject: Detecting range check elimination with PrintAssembly
In-Reply-To: <4F149D88.2060305@oracle.com>
References: <CABhSbLyAwFLFwhmTzPE=0DiXK8e8kbmvr3tr-0AQ=KyFz1Mjsw@mail.gmail.com>
	<CA+cQ+tTd_yhyu1=06JSgskU-kJ_qh=cXmnYdE6a7TgCR7s=uxg@mail.gmail.com>
	<4F135374.8020004@oracle.com>
	<CABhSbLy0gd3BFmBRX-aUgEt7h2HBHer_AQENMGWYTRN4p5-wEg@mail.gmail.com>
	<4F149D88.2060305@oracle.com>
Message-ID: <CAHjP37F+tF=66sCWNOGzH+4v3YeEQ6x+2NE_MPFZUa2B7uV-0g@mail.gmail.com>

Hi Vladimir,

If x_col is always seen to be same value in the profile shouldn't the loop
be unrolled as well with some deopt guard? Or does this not participate in
profiling?

Thanks
On Jan 16, 2012 4:57 PM, "Vladimir Kozlov" <vladimir.kozlov at oracle.com>
wrote:

> > be different, because the expressions are similar. The difference in
> runtime (due to cols being a compile-time
> > constant) will be visible elsewhere. Is that right? If so, where would I
> be able to detect this?
>
> In such situations we usually use some visual tools to see difference
> between log outputs. At least you can use 'diff'. You may need to replace
> instructions addresses in outputs (number at the beginning of lines) with
> the same value to match. There are few tricks you may use to get similar
> PrintOptoAssembly output. Use next flags to avoid mixing output from
> program output and from 2 compiler threads (flags stop program until a
> method is compiled and run only one compiler thread):
>
> -Xbatch -XX:CICompilerCount=1
>
> Also add -XX:+PrintCompilation -XX:+PrintInlining to see what method is
> compiled and inlined. Note that you may see similar output for individual
> methods but could be big difference in compiled caller (computeAll())
> method where 2 loop methods could be inlined. So you need to compare all
> compiled methods.
>
> In general, to have constant as loop limit is always win because some
> checks in generated code could be avoided and more optimizations could be
> done for such loops. Use -XX:+TraceLoopOpts to see what loop optimizations
> are done in both cases.
>
> For example, in your code you set 'x_col = 3', as result the next loop in
> constructNearestClusterVector(**) will be fully unrolled when this method
> is inlined into computeAll() and x_col is replaced with '3':
>
>        for(k = 0; k < x_col; k++) {
>          double tmp = x[i*x_col + k] - mu[j* mu_col + k];
>          dist += tmp * tmp;
>        }
>
> Vladimir
>
> On 1/16/12 1:39 AM, Manohar Jonnalagedda wrote:
>
>> Hi Kris, Vladimir,
>>
>> thanks for both your responses.
>>
>>    Second, your two test methods are different so you can't directly
>> compare them. method1() iterates over rows using
>>    middle loop index 'j' and method2() uses external loop index 'i'.
>> Unless they are typos again.
>>
>>
>> You are right, these are indeed typos. As Kris suggested, I have the code
>> printed here: http://pastebin.com/xRFD1Nt1.
>> The methods corresponding to method1, and method2 are
>> constructNearestClusterVector and computeNewCentroids. Their
>> PrintOptoAssembly outputs are respectively at
>> http://pastebin.com/1evN8b3K and http://pastebin.com/FxkVWTD5
>>
>> Also, it seems I have not explained myself correctly. I am not trying to
>> compare the performance of method1 with respect
>> to that of method2: method1 and method2 both run in the same program.
>> What I am trying to compare is their performance
>> in two cases:
>> - when cols is a compile-time constant (much faster)
>> - when cols is a value determined at run-time
>>
>>    If you are using jdk7 there are few flags you can use to print loop
>> optimizations information. They need debug
>>    version of VM but it is not problem for you, I think, since you can
>> use debug PrintOptoAssembly flag.
>>
>>    -XX:+TraceLoopOpts prints all happened loop optimizations and loop
>> tree after each round of loop opts,
>>    -XX:+TraceLoopPredicate prints RC information when it is moved from a
>> loop,
>>    -XX:+TraceRangeLimitCheck prints additional information for RC
>> elimination optimization.
>>
>>
>> Thanks for these, I will have a look at what they output.
>>
>>    Fourth, range check expression in your example is not what you think.
>> RC expression should be next:
>>    (i*stride+offset) where 'i' is loop variable, 'stride' is constant and
>> 'offset' is loop invariant.
>>
>>    In your example 'offset' is (j * cols) since it is loop invariant, 'k'
>> is loop variable and stride is '1' (one).
>>    In both your methods RC will be moved out of inner loop so the code
>> for it will be the same. The only difference in
>>    these methods will be where and how (j * cols) and (i * cols)
>> expressions are calculated.
>>
>>
>>        I'd guess it's the difference in locality that made the difference
>> in performance in your two tests.
>>
>>  Thanks for the explanation. I understand from the above that the
>> assembly output in both cases mentioned above may not
>> be different, because the expressions are similar. The difference in
>> runtime (due to cols being a compile-time constant)
>> will be visible elsewhere. Is that right? If so, where would I be able to
>> detect this?
>>
>> Cheers,
>> Manohar
>>
>>        In your PrintOptoAssembly output snippet, the instruction at 0x13e
>> is a LoadRange, which loads the range from
>>        the header
>>        of an array:
>>
>>        (from x86_64.ad <http://x86_64.ad> <http://x86_64.ad>)
>>
>>        // Load Range
>>        instruct loadRange(rRegI dst, memory mem)
>>        %{
>>           match(Set dst (LoadRange mem));
>>
>>           ins_cost(125); // XXX
>>           format %{ "movl    $dst, $mem\t# range" %}
>>           opcode(0x8B);
>>           ins_encode(REX_reg_mem(dst, mem), OpcP, reg_mem(dst, mem));
>>           ins_pipe(ialu_reg_mem);
>>        %}
>>
>>        That's not a range check just yet; the real check, if any, should
>> come after the null check, in the form of
>>        comparing
>>        something else with RSI. But you didn't show what's after the null
>> check, how RSI is used, so it's hard to say what
>>        you're seeing in your example.
>>
>>        As for the two test examples, could you paste the entire source
>> code, with the PrintOptoAssembly output of
>>        method1() and
>>        method2() ? The first example looks weird, maybe it's a typo but
>> you're using "j < cols" as the loop condition
>>        for the
>>        inner loop.
>>
>>
>>        - Kris
>>
>>        On Mon, Jan 16, 2012 at 1:59 AM, Manohar Jonnalagedda <
>> manojo10386 at gmail.com <mailto:manojo10386 at gmail.com>
>>        <mailto:manojo10386 at gmail.com <mailto:manojo10386 at gmail.com>**>>
>> wrote:
>>
>>            Hello,
>>
>>            following this reference on Range Check Elimination done by
>> the Hotspot compiler [1], I was keen in knowing
>>        how I
>>            can detect whether range checks are taking place in loops by
>> inspecting output using the PrintAssembly flag;
>>        with
>>            the old PrintOptoAssembly flag, I have seen output such as the
>> following, which I assume to be range checks :
>>
>>            B11: #  B73 B12 &lt;- B10  Freq: 1.21365
>>            139     movq    RAX, [rsp + #24]  # spill
>>            13e     movl    RSI, [RAX + #12 (8-bit)]  # range
>>            141     NullCheck RAX
>>
>>            What is the equivalent with the new PrintAssembly flag (using
>> hsdis)?
>>
>>            Moreover, as stated on the wiki page [1], loops are optimized
>> if the stride is a compile-time constant. I
>>        performed
>>            a few tests on a kmeans program, with 3 nested loops, having
>> the following (high-level) structure:
>>
>>            ===
>>            void method1(){
>>            //loop 1
>>            for(int i = 0; i< rows1; i++){
>>              //...
>>               for(int j = 0; j< rows2; j++){
>>               //...
>>                for(int k = 0; j < cols; k++){ array[j * cols + k] = //...}
>>               }
>>            }
>>            }
>>
>>            void method2(){
>>            //loop 2
>>            for(int i =0; i < rows1; i++){
>>               for(int j=0 ; i< rows2; j++){
>>                 for(int k=0 ; k< cols; k++){
>>                   array[i*cols+k] = //...
>>                 }
>>               }
>>            }
>>            }
>>
>>            void main(){
>>
>>              do{
>>                method1(); method2();
>>              }while(!converged)
>>
>>            }
>>            ====
>>
>>            In the first test, cols is an int whose value is determined at
>> runtime (by reading a file), in the second
>>        test, it
>>            is given as a compile-time constant(3). In the second test,
>> there is a */significant*/ speed-up (around 40%).
>>
>>            However, when studying the diff of the output of
>> PrintOptoAssembly for both method1 and method2, there is no
>>            difference (apart from slight value changes in frequency).
>> Would you have any hints as to where I could look for
>>            differences?
>>
>>            Thanks a lot,
>>            Manohar
>>
>>            [1] https://wikis.oracle.com/**display/HotSpotInternals/**
>> RangeCheckElimination<https://wikis.oracle.com/display/HotSpotInternals/RangeCheckElimination>
>>
>>
>>
>> As Kris pointed you need to fix your example:
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120116/ef3cf0f0/attachment-0001.html 

From vladimir.kozlov at oracle.com  Mon Jan 16 15:45:14 2012
From: vladimir.kozlov at oracle.com (Vladimir Kozlov)
Date: Mon, 16 Jan 2012 15:45:14 -0800
Subject: Detecting range check elimination with PrintAssembly
In-Reply-To: <CAHjP37F+tF=66sCWNOGzH+4v3YeEQ6x+2NE_MPFZUa2B7uV-0g@mail.gmail.com>
References: <CABhSbLyAwFLFwhmTzPE=0DiXK8e8kbmvr3tr-0AQ=KyFz1Mjsw@mail.gmail.com>	<CA+cQ+tTd_yhyu1=06JSgskU-kJ_qh=cXmnYdE6a7TgCR7s=uxg@mail.gmail.com>	<4F135374.8020004@oracle.com>	<CABhSbLy0gd3BFmBRX-aUgEt7h2HBHer_AQENMGWYTRN4p5-wEg@mail.gmail.com>	<4F149D88.2060305@oracle.com>
	<CAHjP37F+tF=66sCWNOGzH+4v3YeEQ6x+2NE_MPFZUa2B7uV-0g@mail.gmail.com>
Message-ID: <4F14B68A.6020808@oracle.com>

Vitaly,

We do use profile_trip_cnt during loop unroll calculation but not during fully unroll because we can't trust it 100% 
since program's phase and number of iterations could change after method is compiled. See policy_unroll() and 
policy_maximally_unroll():

http://hg.openjdk.java.net/hsx/hotspot-comp/hotspot/file/89d0a5d40008/src/share/vm/opto/loopTransform.cpp

We could use deopt as you suggested but deoptimization is double-edge sword, when method recompiled after deoptimization 
some aggressive optimizations will not be executed for it so the new generated code could be slower.

Regards,
Vladimir

On 1/16/12 2:37 PM, Vitaly Davidovich wrote:
> Hi Vladimir,
>
> If x_col is always seen to be same value in the profile shouldn't the loop be unrolled as well with some deopt guard? Or
> does this not participate in profiling?
>
> Thanks
>
> On Jan 16, 2012 4:57 PM, "Vladimir Kozlov" <vladimir.kozlov at oracle.com <mailto:vladimir.kozlov at oracle.com>> wrote:
>
>      > be different, because the expressions are similar. The difference in runtime (due to cols being a compile-time
>      > constant) will be visible elsewhere. Is that right? If so, where would I be able to detect this?
>
>     In such situations we usually use some visual tools to see difference between log outputs. At least you can use
>     'diff'. You may need to replace instructions addresses in outputs (number at the beginning of lines) with the same
>     value to match. There are few tricks you may use to get similar PrintOptoAssembly output. Use next flags to avoid
>     mixing output from program output and from 2 compiler threads (flags stop program until a method is compiled and run
>     only one compiler thread):
>
>     -Xbatch -XX:CICompilerCount=1
>
>     Also add -XX:+PrintCompilation -XX:+PrintInlining to see what method is compiled and inlined. Note that you may see
>     similar output for individual methods but could be big difference in compiled caller (computeAll()) method where 2
>     loop methods could be inlined. So you need to compare all compiled methods.
>
>     In general, to have constant as loop limit is always win because some checks in generated code could be avoided and
>     more optimizations could be done for such loops. Use -XX:+TraceLoopOpts to see what loop optimizations are done in
>     both cases.
>
>     For example, in your code you set 'x_col = 3', as result the next loop in constructNearestClusterVector(__) will be
>     fully unrolled when this method is inlined into computeAll() and x_col is replaced with '3':
>
>             for(k = 0; k < x_col; k++) {
>               double tmp = x[i*x_col + k] - mu[j* mu_col + k];
>               dist += tmp * tmp;
>             }
>
>     Vladimir
>
>     On 1/16/12 1:39 AM, Manohar Jonnalagedda wrote:
>
>         Hi Kris, Vladimir,
>
>         thanks for both your responses.
>
>             Second, your two test methods are different so you can't directly compare them. method1() iterates over rows
>         using
>             middle loop index 'j' and method2() uses external loop index 'i'. Unless they are typos again.
>
>
>         You are right, these are indeed typos. As Kris suggested, I have the code printed here:
>         http://pastebin.com/xRFD1Nt1.
>         The methods corresponding to method1, and method2 are constructNearestClusterVector and computeNewCentroids. Their
>         PrintOptoAssembly outputs are respectively at http://pastebin.com/1evN8b3K and http://pastebin.com/FxkVWTD5
>
>         Also, it seems I have not explained myself correctly. I am not trying to compare the performance of method1 with
>         respect
>         to that of method2: method1 and method2 both run in the same program. What I am trying to compare is their
>         performance
>         in two cases:
>         - when cols is a compile-time constant (much faster)
>         - when cols is a value determined at run-time
>
>             If you are using jdk7 there are few flags you can use to print loop optimizations information. They need debug
>             version of VM but it is not problem for you, I think, since you can use debug PrintOptoAssembly flag.
>
>             -XX:+TraceLoopOpts prints all happened loop optimizations and loop tree after each round of loop opts,
>             -XX:+TraceLoopPredicate prints RC information when it is moved from a loop,
>             -XX:+TraceRangeLimitCheck prints additional information for RC elimination optimization.
>
>
>         Thanks for these, I will have a look at what they output.
>
>             Fourth, range check expression in your example is not what you think. RC expression should be next:
>             (i*stride+offset) where 'i' is loop variable, 'stride' is constant and 'offset' is loop invariant.
>
>             In your example 'offset' is (j * cols) since it is loop invariant, 'k' is loop variable and stride is '1' (one).
>             In both your methods RC will be moved out of inner loop so the code for it will be the same. The only
>         difference in
>             these methods will be where and how (j * cols) and (i * cols) expressions are calculated.
>
>
>                 I'd guess it's the difference in locality that made the difference in performance in your two tests.
>
>           Thanks for the explanation. I understand from the above that the assembly output in both cases mentioned above
>         may not
>         be different, because the expressions are similar. The difference in runtime (due to cols being a compile-time
>         constant)
>         will be visible elsewhere. Is that right? If so, where would I be able to detect this?
>
>         Cheers,
>         Manohar
>
>                 In your PrintOptoAssembly output snippet, the instruction at 0x13e is a LoadRange, which loads the range
>         from
>                 the header
>                 of an array:
>
>                 (from x86_64.ad <http://x86_64.ad> <http://x86_64.ad> <http://x86_64.ad>)
>
>                 // Load Range
>                 instruct loadRange(rRegI dst, memory mem)
>                 %{
>                    match(Set dst (LoadRange mem));
>
>                    ins_cost(125); // XXX
>                    format %{ "movl    $dst, $mem\t# range" %}
>                    opcode(0x8B);
>                    ins_encode(REX_reg_mem(dst, mem), OpcP, reg_mem(dst, mem));
>                    ins_pipe(ialu_reg_mem);
>                 %}
>
>                 That's not a range check just yet; the real check, if any, should come after the null check, in the form of
>                 comparing
>                 something else with RSI. But you didn't show what's after the null check, how RSI is used, so it's hard
>         to say what
>                 you're seeing in your example.
>
>                 As for the two test examples, could you paste the entire source code, with the PrintOptoAssembly output of
>                 method1() and
>                 method2() ? The first example looks weird, maybe it's a typo but you're using "j < cols" as the loop
>         condition
>                 for the
>                 inner loop.
>
>
>                 - Kris
>
>                 On Mon, Jan 16, 2012 at 1:59 AM, Manohar Jonnalagedda <manojo10386 at gmail.com
>         <mailto:manojo10386 at gmail.com> <mailto:manojo10386 at gmail.com <mailto:manojo10386 at gmail.com>>
>         <mailto:manojo10386 at gmail.com <mailto:manojo10386 at gmail.com> <mailto:manojo10386 at gmail.com
>         <mailto:manojo10386 at gmail.com>>__>> wrote:
>
>                     Hello,
>
>                     following this reference on Range Check Elimination done by the Hotspot compiler [1], I was keen in
>         knowing
>                 how I
>                     can detect whether range checks are taking place in loops by inspecting output using the
>         PrintAssembly flag;
>                 with
>                     the old PrintOptoAssembly flag, I have seen output such as the following, which I assume to be range
>         checks :
>
>                     B11: #  B73 B12 &lt;- B10  Freq: 1.21365
>                     139     movq    RAX, [rsp + #24]  # spill
>                     13e     movl    RSI, [RAX + #12 (8-bit)]  # range
>                     141     NullCheck RAX
>
>                     What is the equivalent with the new PrintAssembly flag (using hsdis)?
>
>                     Moreover, as stated on the wiki page [1], loops are optimized if the stride is a compile-time
>         constant. I
>                 performed
>                     a few tests on a kmeans program, with 3 nested loops, having the following (high-level) structure:
>
>                     ===
>                     void method1(){
>                     //loop 1
>                     for(int i = 0; i< rows1; i++){
>                       //...
>                        for(int j = 0; j< rows2; j++){
>                        //...
>                         for(int k = 0; j < cols; k++){ array[j * cols + k] = //...}
>                        }
>                     }
>                     }
>
>                     void method2(){
>                     //loop 2
>                     for(int i =0; i < rows1; i++){
>                        for(int j=0 ; i< rows2; j++){
>                          for(int k=0 ; k< cols; k++){
>                            array[i*cols+k] = //...
>                          }
>                        }
>                     }
>                     }
>
>                     void main(){
>
>                       do{
>                         method1(); method2();
>                       }while(!converged)
>
>                     }
>                     ====
>
>                     In the first test, cols is an int whose value is determined at runtime (by reading a file), in the
>         second
>                 test, it
>                     is given as a compile-time constant(3). In the second test, there is a */significant*/ speed-up
>         (around 40%).
>
>                     However, when studying the diff of the output of PrintOptoAssembly for both method1 and method2,
>         there is no
>                     difference (apart from slight value changes in frequency). Would you have any hints as to where I
>         could look for
>                     differences?
>
>                     Thanks a lot,
>                     Manohar
>
>                     [1] https://wikis.oracle.com/__display/HotSpotInternals/__RangeCheckElimination
>         <https://wikis.oracle.com/display/HotSpotInternals/RangeCheckElimination>
>
>
>
>         As Kris pointed you need to fix your example:
>

From vitalyd at gmail.com  Mon Jan 16 16:15:18 2012
From: vitalyd at gmail.com (Vitaly Davidovich)
Date: Mon, 16 Jan 2012 19:15:18 -0500
Subject: Detecting range check elimination with PrintAssembly
In-Reply-To: <4F14B68A.6020808@oracle.com>
References: <CABhSbLyAwFLFwhmTzPE=0DiXK8e8kbmvr3tr-0AQ=KyFz1Mjsw@mail.gmail.com>
	<CA+cQ+tTd_yhyu1=06JSgskU-kJ_qh=cXmnYdE6a7TgCR7s=uxg@mail.gmail.com>
	<4F135374.8020004@oracle.com>
	<CABhSbLy0gd3BFmBRX-aUgEt7h2HBHer_AQENMGWYTRN4p5-wEg@mail.gmail.com>
	<4F149D88.2060305@oracle.com>
	<CAHjP37F+tF=66sCWNOGzH+4v3YeEQ6x+2NE_MPFZUa2B7uV-0g@mail.gmail.com>
	<4F14B68A.6020808@oracle.com>
Message-ID: <CAHjP37GDEs-8CnNcCjQVEKCfqQdxWHn=pNFaB_XRTt1fORdwPQ@mail.gmail.com>

Vladimir, thanks for the explanation and the code pointer.

Intuitively, it would seem like a good idea to trust the profile 100% if it
reports the same value used 100% of the time (I can see how anything less
than 100%, even a very high probability of same value, is not trustworthy)
given sufficient trips through the loop.  Although I can see how an app may
have phases where same value is seen for a while before it's switched, but
that's where I thought deopt would help.  There must be a good chunk of
code out there that doesn't know at static compilation time the loop count
(so can't use compile-time constant), but at runtime the actual value
doesn't change for many many trips through the loop; I know I have code
like that in various places.

What's the reason a compilation after deopt would not be as aggressive as
the 1st time? Is it because the profile information may be "weaker" (i.e.
more uncertainty in it)? I thought the profile is completely reset after
deopt, so I would think if the loop is now executed with a different
"constant" value (e.g. in our example, instead of 3 it's now 4), then the
same optimizations will be applied (of course if unrolling the loop is no
longer advantageous due to a much different value, I can see how different
optimizations will be applied).

Thanks

On Mon, Jan 16, 2012 at 6:45 PM, Vladimir Kozlov <vladimir.kozlov at oracle.com
> wrote:

> Vitaly,
>
> We do use profile_trip_cnt during loop unroll calculation but not during
> fully unroll because we can't trust it 100% since program's phase and
> number of iterations could change after method is compiled. See
> policy_unroll() and policy_maximally_unroll():
>
> http://hg.openjdk.java.net/**hsx/hotspot-comp/hotspot/file/**
> 89d0a5d40008/src/share/vm/**opto/loopTransform.cpp<http://hg.openjdk.java.net/hsx/hotspot-comp/hotspot/file/89d0a5d40008/src/share/vm/opto/loopTransform.cpp>
>
> We could use deopt as you suggested but deoptimization is double-edge
> sword, when method recompiled after deoptimization some aggressive
> optimizations will not be executed for it so the new generated code could
> be slower.
>
> Regards,
> Vladimir
>
>
> On 1/16/12 2:37 PM, Vitaly Davidovich wrote:
>
>> Hi Vladimir,
>>
>> If x_col is always seen to be same value in the profile shouldn't the
>> loop be unrolled as well with some deopt guard? Or
>> does this not participate in profiling?
>>
>> Thanks
>>
>> On Jan 16, 2012 4:57 PM, "Vladimir Kozlov" <vladimir.kozlov at oracle.com<mailto:
>> vladimir.kozlov@**oracle.com <vladimir.kozlov at oracle.com>>> wrote:
>>
>>     > be different, because the expressions are similar. The difference
>> in runtime (due to cols being a compile-time
>>     > constant) will be visible elsewhere. Is that right? If so, where
>> would I be able to detect this?
>>
>>    In such situations we usually use some visual tools to see difference
>> between log outputs. At least you can use
>>    'diff'. You may need to replace instructions addresses in outputs
>> (number at the beginning of lines) with the same
>>    value to match. There are few tricks you may use to get similar
>> PrintOptoAssembly output. Use next flags to avoid
>>    mixing output from program output and from 2 compiler threads (flags
>> stop program until a method is compiled and run
>>    only one compiler thread):
>>
>>    -Xbatch -XX:CICompilerCount=1
>>
>>    Also add -XX:+PrintCompilation -XX:+PrintInlining to see what method
>> is compiled and inlined. Note that you may see
>>    similar output for individual methods but could be big difference in
>> compiled caller (computeAll()) method where 2
>>    loop methods could be inlined. So you need to compare all compiled
>> methods.
>>
>>    In general, to have constant as loop limit is always win because some
>> checks in generated code could be avoided and
>>    more optimizations could be done for such loops. Use
>> -XX:+TraceLoopOpts to see what loop optimizations are done in
>>    both cases.
>>
>>    For example, in your code you set 'x_col = 3', as result the next loop
>> in constructNearestClusterVector(**__) will be
>>
>>    fully unrolled when this method is inlined into computeAll() and x_col
>> is replaced with '3':
>>
>>            for(k = 0; k < x_col; k++) {
>>              double tmp = x[i*x_col + k] - mu[j* mu_col + k];
>>              dist += tmp * tmp;
>>            }
>>
>>    Vladimir
>>
>>    On 1/16/12 1:39 AM, Manohar Jonnalagedda wrote:
>>
>>        Hi Kris, Vladimir,
>>
>>        thanks for both your responses.
>>
>>            Second, your two test methods are different so you can't
>> directly compare them. method1() iterates over rows
>>        using
>>            middle loop index 'j' and method2() uses external loop index
>> 'i'. Unless they are typos again.
>>
>>
>>        You are right, these are indeed typos. As Kris suggested, I have
>> the code printed here:
>>        http://pastebin.com/xRFD1Nt1.
>>        The methods corresponding to method1, and method2 are
>> constructNearestClusterVector and computeNewCentroids. Their
>>        PrintOptoAssembly outputs are respectively at
>> http://pastebin.com/1evN8b3K and http://pastebin.com/FxkVWTD5
>>
>>        Also, it seems I have not explained myself correctly. I am not
>> trying to compare the performance of method1 with
>>        respect
>>        to that of method2: method1 and method2 both run in the same
>> program. What I am trying to compare is their
>>        performance
>>        in two cases:
>>        - when cols is a compile-time constant (much faster)
>>        - when cols is a value determined at run-time
>>
>>            If you are using jdk7 there are few flags you can use to print
>> loop optimizations information. They need debug
>>            version of VM but it is not problem for you, I think, since
>> you can use debug PrintOptoAssembly flag.
>>
>>            -XX:+TraceLoopOpts prints all happened loop optimizations and
>> loop tree after each round of loop opts,
>>            -XX:+TraceLoopPredicate prints RC information when it is moved
>> from a loop,
>>            -XX:+TraceRangeLimitCheck prints additional information for RC
>> elimination optimization.
>>
>>
>>        Thanks for these, I will have a look at what they output.
>>
>>            Fourth, range check expression in your example is not what you
>> think. RC expression should be next:
>>            (i*stride+offset) where 'i' is loop variable, 'stride' is
>> constant and 'offset' is loop invariant.
>>
>>            In your example 'offset' is (j * cols) since it is loop
>> invariant, 'k' is loop variable and stride is '1' (one).
>>            In both your methods RC will be moved out of inner loop so the
>> code for it will be the same. The only
>>        difference in
>>            these methods will be where and how (j * cols) and (i * cols)
>> expressions are calculated.
>>
>>
>>                I'd guess it's the difference in locality that made the
>> difference in performance in your two tests.
>>
>>          Thanks for the explanation. I understand from the above that the
>> assembly output in both cases mentioned above
>>        may not
>>        be different, because the expressions are similar. The difference
>> in runtime (due to cols being a compile-time
>>        constant)
>>        will be visible elsewhere. Is that right? If so, where would I be
>> able to detect this?
>>
>>        Cheers,
>>        Manohar
>>
>>                In your PrintOptoAssembly output snippet, the instruction
>> at 0x13e is a LoadRange, which loads the range
>>        from
>>                the header
>>                of an array:
>>
>>                (from x86_64.ad <http://x86_64.ad> <http://x86_64.ad> <
>> http://x86_64.ad>)
>>
>>
>>                // Load Range
>>                instruct loadRange(rRegI dst, memory mem)
>>                %{
>>                   match(Set dst (LoadRange mem));
>>
>>                   ins_cost(125); // XXX
>>                   format %{ "movl    $dst, $mem\t# range" %}
>>                   opcode(0x8B);
>>                   ins_encode(REX_reg_mem(dst, mem), OpcP, reg_mem(dst,
>> mem));
>>                   ins_pipe(ialu_reg_mem);
>>                %}
>>
>>                That's not a range check just yet; the real check, if any,
>> should come after the null check, in the form of
>>                comparing
>>                something else with RSI. But you didn't show what's after
>> the null check, how RSI is used, so it's hard
>>        to say what
>>                you're seeing in your example.
>>
>>                As for the two test examples, could you paste the entire
>> source code, with the PrintOptoAssembly output of
>>                method1() and
>>                method2() ? The first example looks weird, maybe it's a
>> typo but you're using "j < cols" as the loop
>>        condition
>>                for the
>>                inner loop.
>>
>>
>>                - Kris
>>
>>                On Mon, Jan 16, 2012 at 1:59 AM, Manohar Jonnalagedda <
>> manojo10386 at gmail.com
>>        <mailto:manojo10386 at gmail.com> <mailto:manojo10386 at gmail.com<mailto:
>> manojo10386 at gmail.com>**>
>>        <mailto:manojo10386 at gmail.com <mailto:manojo10386 at gmail.com>
>> <mailto:manojo10386 at gmail.com
>>        <mailto:manojo10386 at gmail.com>**>__>> wrote:
>>
>>                    Hello,
>>
>>                    following this reference on Range Check Elimination
>> done by the Hotspot compiler [1], I was keen in
>>        knowing
>>                how I
>>                    can detect whether range checks are taking place in
>> loops by inspecting output using the
>>        PrintAssembly flag;
>>                with
>>                    the old PrintOptoAssembly flag, I have seen output
>> such as the following, which I assume to be range
>>        checks :
>>
>>                    B11: #  B73 B12 &lt;- B10  Freq: 1.21365
>>                    139     movq    RAX, [rsp + #24]  # spill
>>                    13e     movl    RSI, [RAX + #12 (8-bit)]  # range
>>                    141     NullCheck RAX
>>
>>                    What is the equivalent with the new PrintAssembly flag
>> (using hsdis)?
>>
>>                    Moreover, as stated on the wiki page [1], loops are
>> optimized if the stride is a compile-time
>>        constant. I
>>                performed
>>                    a few tests on a kmeans program, with 3 nested loops,
>> having the following (high-level) structure:
>>
>>                    ===
>>                    void method1(){
>>                    //loop 1
>>                    for(int i = 0; i< rows1; i++){
>>                      //...
>>                       for(int j = 0; j< rows2; j++){
>>                       //...
>>                        for(int k = 0; j < cols; k++){ array[j * cols + k]
>> = //...}
>>                       }
>>                    }
>>                    }
>>
>>                    void method2(){
>>                    //loop 2
>>                    for(int i =0; i < rows1; i++){
>>                       for(int j=0 ; i< rows2; j++){
>>                         for(int k=0 ; k< cols; k++){
>>                           array[i*cols+k] = //...
>>                         }
>>                       }
>>                    }
>>                    }
>>
>>                    void main(){
>>
>>                      do{
>>                        method1(); method2();
>>                      }while(!converged)
>>
>>                    }
>>                    ====
>>
>>                    In the first test, cols is an int whose value is
>> determined at runtime (by reading a file), in the
>>        second
>>                test, it
>>                    is given as a compile-time constant(3). In the second
>> test, there is a */significant*/ speed-up
>>        (around 40%).
>>
>>                    However, when studying the diff of the output of
>> PrintOptoAssembly for both method1 and method2,
>>        there is no
>>                    difference (apart from slight value changes in
>> frequency). Would you have any hints as to where I
>>        could look for
>>                    differences?
>>
>>                    Thanks a lot,
>>                    Manohar
>>
>>                    [1] https://wikis.oracle.com/__**
>> display/HotSpotInternals/__**RangeCheckElimination<https://wikis.oracle.com/__display/HotSpotInternals/__RangeCheckElimination>
>>
>>        <https://wikis.oracle.com/**display/HotSpotInternals/**
>> RangeCheckElimination<https://wikis.oracle.com/display/HotSpotInternals/RangeCheckElimination>
>> >
>>
>>
>>
>>        As Kris pointed you need to fix your example:
>>
>>


-- 
Vitaly
617-548-7007 (mobile)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120116/cb2bce5a/attachment-0001.html 

From robert.ottenhag at oracle.com  Mon Jan 16 18:04:18 2012
From: robert.ottenhag at oracle.com (Robert Ottenhag)
Date: Mon, 16 Jan 2012 18:04:18 -0800 (PST)
Subject: RFR (S): 7130391: Add a framework for vendor-specific validation
	control of setting command-line switches in Hotspot
Message-ID: <5a181ad1-ef64-4b00-88ce-69829f2ce80b@default>

Hi,

Webrev: http://cr.openjdk.java.net/~rottenha/7130391/webrev.00

This fix adds optional validation control to the setting of command-line switches in Hotspot, and allows it to have vendor-specific extensions if necessary.

The design follows the previously added framework for vendor-specific command-line switch extensions in CR7117389.

The validation control is handled by new boolean methods Flag::is_valid_<type>(value,origin) that are called at the beginning of each call to CommandLineFlags[Ex]::<type>AtPut() to verify that the new value and origin are valid replacements for the current value and origin for this flag.

When parsing the command line options, a failed validation will typically result in an error message and exit with "Unrecognized VM option '<flag-name>'".  When used dynamically using the attach API or management API the resulting operation will fail, leaving it up to the caller to handle it as appropriate.

A simple use case for validation is a manageable flag whose current value can not be less than the previous value, while a more complex example may base the validation on multiple other flags, etc.

Thanks,

/Robert

-- 
Oracle
Robert Ottenhag | Senior Member of Technical Staff
Oracle Oracle Java VM | ORACLE Sweden | Stockholm

From david.holmes at oracle.com  Mon Jan 16 19:09:21 2012
From: david.holmes at oracle.com (David Holmes)
Date: Tue, 17 Jan 2012 13:09:21 +1000
Subject: RFR (S): 7130391: Add a framework for vendor-specific validation
	control of setting command-line switches in Hotspot
In-Reply-To: <5a181ad1-ef64-4b00-88ce-69829f2ce80b@default>
References: <5a181ad1-ef64-4b00-88ce-69829f2ce80b@default>
Message-ID: <4F14E661.5080405@oracle.com>

Hi Robert,

I've added serviceability to the cc list.

On 17/01/2012 12:04 PM, Robert Ottenhag wrote:
> Webrev: http://cr.openjdk.java.net/~rottenha/7130391/webrev.00
>
> This fix adds optional validation control to the setting of command-line switches in Hotspot, and allows it to have vendor-specific extensions if necessary.

Does this imply that the Java management APIs (eg 
com.sun.management.VMOption) need to be changed to reflect these 
restrictions? Presently VMOptions are either writeable or not, but this 
makes them conditionally-writeable.

> The design follows the previously added framework for vendor-specific command-line switch extensions in CR7117389.
>
> The validation control is handled by new boolean methods Flag::is_valid_<type>(value,origin) that are called at the beginning of each call to CommandLineFlags[Ex]::<type>AtPut() to verify that the new value and origin are valid replacements for the current value and origin for this flag.
>
> When parsing the command line options, a failed validation will typically result in an error message and exit with "Unrecognized VM option '<flag-name>'".  When used dynamically using the attach API or management API the resulting operation will fail, leaving it up to the caller to handle it as appropriate.

The error message doesn't really seem appropriate - it may well be a 
recognized option, you just can't set it to that value in that way. 
Ideally there would be a way for the validation logic to supply a 
meaningful error message. In its absence the top-level message should 
reflect the new type of error.

Also some of the failures lead to crashes - which seems wrong to me - 
see below.

----

src/share/vm/services/management.cpp:

1821   if (!succeed) {
1822     THROW_MSG(vmSymbols::java_lang_IllegalArgumentException(),
1823               "This flag is not writeable with this value or origin.");

That's a rather cryptic error message. How about:

"Flag can not be set to the requested value using this API"

?

----

src/share/vm/runtime/globals_ext.hpp

With all the

inline bool Flag::is_valid_ext_T(T value, FlagValueOrigin origin)

functions, is it necessary to include the type T in the function name?

-----

src/share/vm/runtime/globals.cpp

The use of the guarantees seems wrong as it means an invalid option will 
trigger a VM crash rather than a clean exit during initialization. It 
seems to me that none of the code in arguments.cpp that uses the 
FLAG_SET_* macros (which in turn use the CommandLineFlagsEx functions 
you added the guarantees to) anticipates any possibility for failure. I 
think if you are going this path then you have no choice but to change 
the CommandLineFlagsEx methods to return bool and update the FLAG_SET 
macros to try and perform appropriate error handling.

David
-----


> A simple use case for validation is a manageable flag whose current value can not be less than the previous value, while a more complex example may base the validation on multiple other flags, etc.
>
> Thanks,
>
> /Robert
>

From vladimir.kozlov at oracle.com  Mon Jan 16 20:52:49 2012
From: vladimir.kozlov at oracle.com (Vladimir Kozlov)
Date: Mon, 16 Jan 2012 20:52:49 -0800
Subject: Detecting range check elimination with PrintAssembly
In-Reply-To: <CAHjP37GDEs-8CnNcCjQVEKCfqQdxWHn=pNFaB_XRTt1fORdwPQ@mail.gmail.com>
References: <CABhSbLyAwFLFwhmTzPE=0DiXK8e8kbmvr3tr-0AQ=KyFz1Mjsw@mail.gmail.com>	<CA+cQ+tTd_yhyu1=06JSgskU-kJ_qh=cXmnYdE6a7TgCR7s=uxg@mail.gmail.com>	<4F135374.8020004@oracle.com>	<CABhSbLy0gd3BFmBRX-aUgEt7h2HBHer_AQENMGWYTRN4p5-wEg@mail.gmail.com>	<4F149D88.2060305@oracle.com>	<CAHjP37F+tF=66sCWNOGzH+4v3YeEQ6x+2NE_MPFZUa2B7uV-0g@mail.gmail.com>	<4F14B68A.6020808@oracle.com>
	<CAHjP37GDEs-8CnNcCjQVEKCfqQdxWHn=pNFaB_XRTt1fORdwPQ@mail.gmail.com>
Message-ID: <4F14FEA1.6020704@oracle.com>

I need to be more clear. We can't get 100% correct profiling result for counters. First, profiling structure 
(MethodData) are created only after method is executed for some time (for Server compiler it is 3000 invocations). It is 
done to avoid profiling short programs since profiling will slowdown Interpreter by about 20%. This delayed profiling 
introduce first discrepancy between counters. MD could be also created when method has a hot loop (a lot of iterations) 
and OSR compilation is requested. As result counters in the loop will not correlate with counters before the loop. An 
other reason for not precise profiling counters is execution of method by multiple threads. We have only one MD 
structure per method which is updated (not atomically for speed) by all such threads. And when method is prepared for 
compilation compiler thread creates snapshot of MD which could happened in the middle of method execution by java threads.

We don't reset MD during deoptimization because we still need previous values (in addition to counters we collect type 
information). So counters are accumulated. And profiling is resumed from deoptimization point which again adds discrepancy.

I hope it explains why we can't trust 100% to profiling counters.

Regards,
Vladimir

On 1/16/12 4:15 PM, Vitaly Davidovich wrote:
> Vladimir, thanks for the explanation and the code pointer.
>
> Intuitively, it would seem like a good idea to trust the profile 100% if it reports the same value used 100% of the time
> (I can see how anything less than 100%, even a very high probability of same value, is not trustworthy) given sufficient
> trips through the loop.  Although I can see how an app may have phases where same value is seen for a while before it's
> switched, but that's where I thought deopt would help.  There must be a good chunk of code out there that doesn't know
> at static compilation time the loop count (so can't use compile-time constant), but at runtime the actual value doesn't
> change for many many trips through the loop; I know I have code like that in various places.
>
> What's the reason a compilation after deopt would not be as aggressive as the 1st time? Is it because the profile
> information may be "weaker" (i.e. more uncertainty in it)? I thought the profile is completely reset after deopt, so I
> would think if the loop is now executed with a different "constant" value (e.g. in our example, instead of 3 it's now
> 4), then the same optimizations will be applied (of course if unrolling the loop is no longer advantageous due to a much
> different value, I can see how different optimizations will be applied).
>
> Thanks
>
> On Mon, Jan 16, 2012 at 6:45 PM, Vladimir Kozlov <vladimir.kozlov at oracle.com <mailto:vladimir.kozlov at oracle.com>> wrote:
>
>     Vitaly,
>
>     We do use profile_trip_cnt during loop unroll calculation but not during fully unroll because we can't trust it 100%
>     since program's phase and number of iterations could change after method is compiled. See policy_unroll() and
>     policy_maximally_unroll():
>
>     http://hg.openjdk.java.net/__hsx/hotspot-comp/hotspot/file/__89d0a5d40008/src/share/vm/__opto/loopTransform.cpp
>     <http://hg.openjdk.java.net/hsx/hotspot-comp/hotspot/file/89d0a5d40008/src/share/vm/opto/loopTransform.cpp>
>
>     We could use deopt as you suggested but deoptimization is double-edge sword, when method recompiled after
>     deoptimization some aggressive optimizations will not be executed for it so the new generated code could be slower.
>
>     Regards,
>     Vladimir
>
>
>     On 1/16/12 2:37 PM, Vitaly Davidovich wrote:
>
>         Hi Vladimir,
>
>         If x_col is always seen to be same value in the profile shouldn't the loop be unrolled as well with some deopt
>         guard? Or
>         does this not participate in profiling?
>
>         Thanks
>
>         On Jan 16, 2012 4:57 PM, "Vladimir Kozlov" <vladimir.kozlov at oracle.com <mailto:vladimir.kozlov at oracle.com>
>         <mailto:vladimir.kozlov at __oracle.com <mailto:vladimir.kozlov at oracle.com>>> wrote:
>
>          > be different, because the expressions are similar. The difference in runtime (due to cols being a compile-time
>          > constant) will be visible elsewhere. Is that right? If so, where would I be able to detect this?
>
>             In such situations we usually use some visual tools to see difference between log outputs. At least you can use
>         'diff'. You may need to replace instructions addresses in outputs (number at the beginning of lines) with the same
>             value to match. There are few tricks you may use to get similar PrintOptoAssembly output. Use next flags to
>         avoid
>             mixing output from program output and from 2 compiler threads (flags stop program until a method is compiled
>         and run
>             only one compiler thread):
>
>             -Xbatch -XX:CICompilerCount=1
>
>             Also add -XX:+PrintCompilation -XX:+PrintInlining to see what method is compiled and inlined. Note that you
>         may see
>             similar output for individual methods but could be big difference in compiled caller (computeAll()) method
>         where 2
>             loop methods could be inlined. So you need to compare all compiled methods.
>
>             In general, to have constant as loop limit is always win because some checks in generated code could be
>         avoided and
>             more optimizations could be done for such loops. Use -XX:+TraceLoopOpts to see what loop optimizations are
>         done in
>             both cases.
>
>             For example, in your code you set 'x_col = 3', as result the next loop in
>         constructNearestClusterVector(____) will be
>
>             fully unrolled when this method is inlined into computeAll() and x_col is replaced with '3':
>
>                     for(k = 0; k < x_col; k++) {
>                       double tmp = x[i*x_col + k] - mu[j* mu_col + k];
>                       dist += tmp * tmp;
>                     }
>
>             Vladimir
>
>             On 1/16/12 1:39 AM, Manohar Jonnalagedda wrote:
>
>                 Hi Kris, Vladimir,
>
>                 thanks for both your responses.
>
>                     Second, your two test methods are different so you can't directly compare them. method1() iterates
>         over rows
>                 using
>                     middle loop index 'j' and method2() uses external loop index 'i'. Unless they are typos again.
>
>
>                 You are right, these are indeed typos. As Kris suggested, I have the code printed here:
>         http://pastebin.com/xRFD1Nt1.
>                 The methods corresponding to method1, and method2 are constructNearestClusterVector and
>         computeNewCentroids. Their
>                 PrintOptoAssembly outputs are respectively at http://pastebin.com/1evN8b3K and http://pastebin.com/FxkVWTD5
>
>                 Also, it seems I have not explained myself correctly. I am not trying to compare the performance of
>         method1 with
>                 respect
>                 to that of method2: method1 and method2 both run in the same program. What I am trying to compare is their
>                 performance
>                 in two cases:
>                 - when cols is a compile-time constant (much faster)
>                 - when cols is a value determined at run-time
>
>                     If you are using jdk7 there are few flags you can use to print loop optimizations information. They
>         need debug
>                     version of VM but it is not problem for you, I think, since you can use debug PrintOptoAssembly flag.
>
>                     -XX:+TraceLoopOpts prints all happened loop optimizations and loop tree after each round of loop opts,
>                     -XX:+TraceLoopPredicate prints RC information when it is moved from a loop,
>                     -XX:+TraceRangeLimitCheck prints additional information for RC elimination optimization.
>
>
>                 Thanks for these, I will have a look at what they output.
>
>                     Fourth, range check expression in your example is not what you think. RC expression should be next:
>                     (i*stride+offset) where 'i' is loop variable, 'stride' is constant and 'offset' is loop invariant.
>
>                     In your example 'offset' is (j * cols) since it is loop invariant, 'k' is loop variable and stride
>         is '1' (one).
>                     In both your methods RC will be moved out of inner loop so the code for it will be the same. The only
>                 difference in
>                     these methods will be where and how (j * cols) and (i * cols) expressions are calculated.
>
>
>                         I'd guess it's the difference in locality that made the difference in performance in your two tests.
>
>                   Thanks for the explanation. I understand from the above that the assembly output in both cases
>         mentioned above
>                 may not
>                 be different, because the expressions are similar. The difference in runtime (due to cols being a
>         compile-time
>                 constant)
>                 will be visible elsewhere. Is that right? If so, where would I be able to detect this?
>
>                 Cheers,
>                 Manohar
>
>                         In your PrintOptoAssembly output snippet, the instruction at 0x13e is a LoadRange, which loads
>         the range
>                 from
>                         the header
>                         of an array:
>
>                         (from x86_64.ad <http://x86_64.ad> <http://x86_64.ad> <http://x86_64.ad> <http://x86_64.ad>)
>
>
>                         // Load Range
>                         instruct loadRange(rRegI dst, memory mem)
>                         %{
>                            match(Set dst (LoadRange mem));
>
>                            ins_cost(125); // XXX
>                            format %{ "movl    $dst, $mem\t# range" %}
>                            opcode(0x8B);
>                            ins_encode(REX_reg_mem(dst, mem), OpcP, reg_mem(dst, mem));
>                            ins_pipe(ialu_reg_mem);
>                         %}
>
>                         That's not a range check just yet; the real check, if any, should come after the null check, in
>         the form of
>                         comparing
>                         something else with RSI. But you didn't show what's after the null check, how RSI is used, so
>         it's hard
>                 to say what
>                         you're seeing in your example.
>
>                         As for the two test examples, could you paste the entire source code, with the PrintOptoAssembly
>         output of
>                         method1() and
>                         method2() ? The first example looks weird, maybe it's a typo but you're using "j < cols" as the loop
>                 condition
>                         for the
>                         inner loop.
>
>
>                         - Kris
>
>                         On Mon, Jan 16, 2012 at 1:59 AM, Manohar Jonnalagedda <manojo10386 at gmail.com
>         <mailto:manojo10386 at gmail.com>
>         <mailto:manojo10386 at gmail.com <mailto:manojo10386 at gmail.com>> <mailto:manojo10386 at gmail.com
>         <mailto:manojo10386 at gmail.com> <mailto:manojo10386 at gmail.com <mailto:manojo10386 at gmail.com>>__>
>         <mailto:manojo10386 at gmail.com <mailto:manojo10386 at gmail.com> <mailto:manojo10386 at gmail.com
>         <mailto:manojo10386 at gmail.com>> <mailto:manojo10386 at gmail.com <mailto:manojo10386 at gmail.com>
>         <mailto:manojo10386 at gmail.com <mailto:manojo10386 at gmail.com>>__>__>> wrote:
>
>                             Hello,
>
>                             following this reference on Range Check Elimination done by the Hotspot compiler [1], I was
>         keen in
>                 knowing
>                         how I
>                             can detect whether range checks are taking place in loops by inspecting output using the
>                 PrintAssembly flag;
>                         with
>                             the old PrintOptoAssembly flag, I have seen output such as the following, which I assume to
>         be range
>                 checks :
>
>                             B11: #  B73 B12 &lt;- B10  Freq: 1.21365
>                             139     movq    RAX, [rsp + #24]  # spill
>                             13e     movl    RSI, [RAX + #12 (8-bit)]  # range
>                             141     NullCheck RAX
>
>                             What is the equivalent with the new PrintAssembly flag (using hsdis)?
>
>                             Moreover, as stated on the wiki page [1], loops are optimized if the stride is a compile-time
>                 constant. I
>                         performed
>                             a few tests on a kmeans program, with 3 nested loops, having the following (high-level)
>         structure:
>
>                             ===
>                             void method1(){
>                             //loop 1
>                             for(int i = 0; i< rows1; i++){
>                               //...
>                                for(int j = 0; j< rows2; j++){
>                                //...
>                                 for(int k = 0; j < cols; k++){ array[j * cols + k] = //...}
>                                }
>                             }
>                             }
>
>                             void method2(){
>                             //loop 2
>                             for(int i =0; i < rows1; i++){
>                                for(int j=0 ; i< rows2; j++){
>                                  for(int k=0 ; k< cols; k++){
>                                    array[i*cols+k] = //...
>                                  }
>                                }
>                             }
>                             }
>
>                             void main(){
>
>                               do{
>                                 method1(); method2();
>                               }while(!converged)
>
>                             }
>                             ====
>
>                             In the first test, cols is an int whose value is determined at runtime (by reading a file),
>         in the
>                 second
>                         test, it
>                             is given as a compile-time constant(3). In the second test, there is a */significant*/ speed-up
>                 (around 40%).
>
>                             However, when studying the diff of the output of PrintOptoAssembly for both method1 and method2,
>                 there is no
>                             difference (apart from slight value changes in frequency). Would you have any hints as to
>         where I
>                 could look for
>                             differences?
>
>                             Thanks a lot,
>                             Manohar
>
>                             [1] https://wikis.oracle.com/____display/HotSpotInternals/____RangeCheckElimination
>         <https://wikis.oracle.com/__display/HotSpotInternals/__RangeCheckElimination>
>
>         <https://wikis.oracle.com/__display/HotSpotInternals/__RangeCheckElimination
>         <https://wikis.oracle.com/display/HotSpotInternals/RangeCheckElimination>>
>
>
>
>                 As Kris pointed you need to fix your example:
>
>
>
>
> --
> Vitaly
> 617-548-7007 (mobile)

From vitalyd at gmail.com  Mon Jan 16 21:23:34 2012
From: vitalyd at gmail.com (Vitaly Davidovich)
Date: Tue, 17 Jan 2012 00:23:34 -0500
Subject: Detecting range check elimination with PrintAssembly
In-Reply-To: <4F14FEA1.6020704@oracle.com>
References: <CABhSbLyAwFLFwhmTzPE=0DiXK8e8kbmvr3tr-0AQ=KyFz1Mjsw@mail.gmail.com>
	<CA+cQ+tTd_yhyu1=06JSgskU-kJ_qh=cXmnYdE6a7TgCR7s=uxg@mail.gmail.com>
	<4F135374.8020004@oracle.com>
	<CABhSbLy0gd3BFmBRX-aUgEt7h2HBHer_AQENMGWYTRN4p5-wEg@mail.gmail.com>
	<4F149D88.2060305@oracle.com>
	<CAHjP37F+tF=66sCWNOGzH+4v3YeEQ6x+2NE_MPFZUa2B7uV-0g@mail.gmail.com>
	<4F14B68A.6020808@oracle.com>
	<CAHjP37GDEs-8CnNcCjQVEKCfqQdxWHn=pNFaB_XRTt1fORdwPQ@mail.gmail.com>
	<4F14FEA1.6020704@oracle.com>
Message-ID: <CAHjP37H_xCp8D4sV+e=hE3jF2_g6CJb9TJ4dgxhp+U8aGV86tQ@mail.gmail.com>

Vladimir, thanks for that -- very useful and interesting to know.  So I
take it a compilation after initial deopt may be less aggressive than 1st
time around simply because the accumulated MD might be in a state such that
it guides optimization in a different direction, is that right?

I guess how does one reason about the generated assembly when it's so
(possibly) dynamic? :) I've looked at the assembly in a product hotspot via
hsdis, but then you realize that you may be looking at just one compilation
of it, and it may be different at some point later.  Also, for something
like Intel's VTune (or Amplifier) that support showing JIT'd assembly, I
guess it's the same issue? This is quite different, of course, with .NET
CLR's JIT because there's no profiling there -- 1st time method is hit, it
gets JIT'd and there's no code pitching (makes looking at the assembly seem
a bit more "reassuring").

In general, would you say that the C2 compiler favors "over aggressiveness"
with deopt guards? Or does it try to avoid deopts and only perform
aggressive optimizations where the profile is quite conclusive on type
and/or counters?

Thanks

On Mon, Jan 16, 2012 at 11:52 PM, Vladimir Kozlov <
vladimir.kozlov at oracle.com> wrote:

> I need to be more clear. We can't get 100% correct profiling result for
> counters. First, profiling structure (MethodData) are created only after
> method is executed for some time (for Server compiler it is 3000
> invocations). It is done to avoid profiling short programs since profiling
> will slowdown Interpreter by about 20%. This delayed profiling introduce
> first discrepancy between counters. MD could be also created when method
> has a hot loop (a lot of iterations) and OSR compilation is requested. As
> result counters in the loop will not correlate with counters before the
> loop. An other reason for not precise profiling counters is execution of
> method by multiple threads. We have only one MD structure per method which
> is updated (not atomically for speed) by all such threads. And when method
> is prepared for compilation compiler thread creates snapshot of MD which
> could happened in the middle of method execution by java threads.
>
> We don't reset MD during deoptimization because we still need previous
> values (in addition to counters we collect type information). So counters
> are accumulated. And profiling is resumed from deoptimization point which
> again adds discrepancy.
>
> I hope it explains why we can't trust 100% to profiling counters.
>
> Regards,
> Vladimir
>
>
> On 1/16/12 4:15 PM, Vitaly Davidovich wrote:
>
>> Vladimir, thanks for the explanation and the code pointer.
>>
>> Intuitively, it would seem like a good idea to trust the profile 100% if
>> it reports the same value used 100% of the time
>> (I can see how anything less than 100%, even a very high probability of
>> same value, is not trustworthy) given sufficient
>> trips through the loop.  Although I can see how an app may have phases
>> where same value is seen for a while before it's
>> switched, but that's where I thought deopt would help.  There must be a
>> good chunk of code out there that doesn't know
>> at static compilation time the loop count (so can't use compile-time
>> constant), but at runtime the actual value doesn't
>> change for many many trips through the loop; I know I have code like that
>> in various places.
>>
>> What's the reason a compilation after deopt would not be as aggressive as
>> the 1st time? Is it because the profile
>> information may be "weaker" (i.e. more uncertainty in it)? I thought the
>> profile is completely reset after deopt, so I
>> would think if the loop is now executed with a different "constant" value
>> (e.g. in our example, instead of 3 it's now
>> 4), then the same optimizations will be applied (of course if unrolling
>> the loop is no longer advantageous due to a much
>> different value, I can see how different optimizations will be applied).
>>
>> Thanks
>>
>> On Mon, Jan 16, 2012 at 6:45 PM, Vladimir Kozlov <
>> vladimir.kozlov at oracle.com <mailto:vladimir.kozlov@**oracle.com<vladimir.kozlov at oracle.com>>>
>> wrote:
>>
>>    Vitaly,
>>
>>    We do use profile_trip_cnt during loop unroll calculation but not
>> during fully unroll because we can't trust it 100%
>>    since program's phase and number of iterations could change after
>> method is compiled. See policy_unroll() and
>>    policy_maximally_unroll():
>>
>>    http://hg.openjdk.java.net/__**hsx/hotspot-comp/hotspot/file/**
>> __89d0a5d40008/src/share/vm/__**opto/loopTransform.cpp<http://hg.openjdk.java.net/__hsx/hotspot-comp/hotspot/file/__89d0a5d40008/src/share/vm/__opto/loopTransform.cpp>
>>
>>    <http://hg.openjdk.java.net/**hsx/hotspot-comp/hotspot/file/**
>> 89d0a5d40008/src/share/vm/**opto/loopTransform.cpp<http://hg.openjdk.java.net/hsx/hotspot-comp/hotspot/file/89d0a5d40008/src/share/vm/opto/loopTransform.cpp>
>> >
>>
>>    We could use deopt as you suggested but deoptimization is double-edge
>> sword, when method recompiled after
>>    deoptimization some aggressive optimizations will not be executed for
>> it so the new generated code could be slower.
>>
>>    Regards,
>>    Vladimir
>>
>>
>>    On 1/16/12 2:37 PM, Vitaly Davidovich wrote:
>>
>>        Hi Vladimir,
>>
>>        If x_col is always seen to be same value in the profile shouldn't
>> the loop be unrolled as well with some deopt
>>        guard? Or
>>        does this not participate in profiling?
>>
>>        Thanks
>>
>>        On Jan 16, 2012 4:57 PM, "Vladimir Kozlov" <
>> vladimir.kozlov at oracle.com <mailto:vladimir.kozlov@**oracle.com<vladimir.kozlov at oracle.com>
>> >
>>        <mailto:vladimir.kozlov at __orac**le.com <http://oracle.com><mailto:
>> vladimir.kozlov@**oracle.com <vladimir.kozlov at oracle.com>>>> wrote:
>>
>>         > be different, because the expressions are similar. The
>> difference in runtime (due to cols being a compile-time
>>         > constant) will be visible elsewhere. Is that right? If so,
>> where would I be able to detect this?
>>
>>            In such situations we usually use some visual tools to see
>> difference between log outputs. At least you can use
>>        'diff'. You may need to replace instructions addresses in outputs
>> (number at the beginning of lines) with the same
>>            value to match. There are few tricks you may use to get
>> similar PrintOptoAssembly output. Use next flags to
>>        avoid
>>            mixing output from program output and from 2 compiler threads
>> (flags stop program until a method is compiled
>>        and run
>>            only one compiler thread):
>>
>>            -Xbatch -XX:CICompilerCount=1
>>
>>            Also add -XX:+PrintCompilation -XX:+PrintInlining to see what
>> method is compiled and inlined. Note that you
>>        may see
>>            similar output for individual methods but could be big
>> difference in compiled caller (computeAll()) method
>>        where 2
>>            loop methods could be inlined. So you need to compare all
>> compiled methods.
>>
>>            In general, to have constant as loop limit is always win
>> because some checks in generated code could be
>>        avoided and
>>            more optimizations could be done for such loops. Use
>> -XX:+TraceLoopOpts to see what loop optimizations are
>>        done in
>>            both cases.
>>
>>            For example, in your code you set 'x_col = 3', as result the
>> next loop in
>>        constructNearestClusterVector(**____) will be
>>
>>
>>            fully unrolled when this method is inlined into computeAll()
>> and x_col is replaced with '3':
>>
>>                    for(k = 0; k < x_col; k++) {
>>                      double tmp = x[i*x_col + k] - mu[j* mu_col + k];
>>                      dist += tmp * tmp;
>>                    }
>>
>>            Vladimir
>>
>>            On 1/16/12 1:39 AM, Manohar Jonnalagedda wrote:
>>
>>                Hi Kris, Vladimir,
>>
>>                thanks for both your responses.
>>
>>                    Second, your two test methods are different so you
>> can't directly compare them. method1() iterates
>>        over rows
>>                using
>>                    middle loop index 'j' and method2() uses external loop
>> index 'i'. Unless they are typos again.
>>
>>
>>                You are right, these are indeed typos. As Kris suggested,
>> I have the code printed here:
>>        http://pastebin.com/xRFD1Nt1.
>>                The methods corresponding to method1, and method2 are
>> constructNearestClusterVector and
>>        computeNewCentroids. Their
>>                PrintOptoAssembly outputs are respectively at
>> http://pastebin.com/1evN8b3K and http://pastebin.com/FxkVWTD5
>>
>>                Also, it seems I have not explained myself correctly. I am
>> not trying to compare the performance of
>>        method1 with
>>                respect
>>                to that of method2: method1 and method2 both run in the
>> same program. What I am trying to compare is their
>>                performance
>>                in two cases:
>>                - when cols is a compile-time constant (much faster)
>>                - when cols is a value determined at run-time
>>
>>                    If you are using jdk7 there are few flags you can use
>> to print loop optimizations information. They
>>        need debug
>>                    version of VM but it is not problem for you, I think,
>> since you can use debug PrintOptoAssembly flag.
>>
>>                    -XX:+TraceLoopOpts prints all happened loop
>> optimizations and loop tree after each round of loop opts,
>>                    -XX:+TraceLoopPredicate prints RC information when it
>> is moved from a loop,
>>                    -XX:+TraceRangeLimitCheck prints additional
>> information for RC elimination optimization.
>>
>>
>>                Thanks for these, I will have a look at what they output.
>>
>>                    Fourth, range check expression in your example is not
>> what you think. RC expression should be next:
>>                    (i*stride+offset) where 'i' is loop variable, 'stride'
>> is constant and 'offset' is loop invariant.
>>
>>                    In your example 'offset' is (j * cols) since it is
>> loop invariant, 'k' is loop variable and stride
>>        is '1' (one).
>>                    In both your methods RC will be moved out of inner
>> loop so the code for it will be the same. The only
>>                difference in
>>                    these methods will be where and how (j * cols) and (i
>> * cols) expressions are calculated.
>>
>>
>>                        I'd guess it's the difference in locality that
>> made the difference in performance in your two tests.
>>
>>                  Thanks for the explanation. I understand from the above
>> that the assembly output in both cases
>>        mentioned above
>>                may not
>>                be different, because the expressions are similar. The
>> difference in runtime (due to cols being a
>>        compile-time
>>                constant)
>>                will be visible elsewhere. Is that right? If so, where
>> would I be able to detect this?
>>
>>                Cheers,
>>                Manohar
>>
>>                        In your PrintOptoAssembly output snippet, the
>> instruction at 0x13e is a LoadRange, which loads
>>        the range
>>                from
>>                        the header
>>                        of an array:
>>
>>                        (from x86_64.ad <http://x86_64.ad> <
>> http://x86_64.ad> <http://x86_64.ad> <http://x86_64.ad>)
>>
>>
>>
>>                        // Load Range
>>                        instruct loadRange(rRegI dst, memory mem)
>>                        %{
>>                           match(Set dst (LoadRange mem));
>>
>>                           ins_cost(125); // XXX
>>                           format %{ "movl    $dst, $mem\t# range" %}
>>                           opcode(0x8B);
>>                           ins_encode(REX_reg_mem(dst, mem), OpcP,
>> reg_mem(dst, mem));
>>                           ins_pipe(ialu_reg_mem);
>>                        %}
>>
>>                        That's not a range check just yet; the real check,
>> if any, should come after the null check, in
>>        the form of
>>                        comparing
>>                        something else with RSI. But you didn't show
>> what's after the null check, how RSI is used, so
>>        it's hard
>>                to say what
>>                        you're seeing in your example.
>>
>>                        As for the two test examples, could you paste the
>> entire source code, with the PrintOptoAssembly
>>        output of
>>                        method1() and
>>                        method2() ? The first example looks weird, maybe
>> it's a typo but you're using "j < cols" as the loop
>>                condition
>>                        for the
>>                        inner loop.
>>
>>
>>                        - Kris
>>
>>                        On Mon, Jan 16, 2012 at 1:59 AM, Manohar
>> Jonnalagedda <manojo10386 at gmail.com
>>        <mailto:manojo10386 at gmail.com>
>>        <mailto:manojo10386 at gmail.com <mailto:manojo10386 at gmail.com>**>
>> <mailto:manojo10386 at gmail.com
>>        <mailto:manojo10386 at gmail.com> <mailto:manojo10386 at gmail.com<mailto:
>> manojo10386 at gmail.com>**>__>
>>        <mailto:manojo10386 at gmail.com <mailto:manojo10386 at gmail.com>
>> <mailto:manojo10386 at gmail.com
>>        <mailto:manojo10386 at gmail.com>**> <mailto:manojo10386 at gmail.com<mailto:
>> manojo10386 at gmail.com>
>>        <mailto:manojo10386 at gmail.com <mailto:manojo10386 at gmail.com>**>__>__>>
>> wrote:
>>
>>                            Hello,
>>
>>                            following this reference on Range Check
>> Elimination done by the Hotspot compiler [1], I was
>>        keen in
>>                knowing
>>                        how I
>>                            can detect whether range checks are taking
>> place in loops by inspecting output using the
>>                PrintAssembly flag;
>>                        with
>>                            the old PrintOptoAssembly flag, I have seen
>> output such as the following, which I assume to
>>        be range
>>                checks :
>>
>>                            B11: #  B73 B12 &lt;- B10  Freq: 1.21365
>>                            139     movq    RAX, [rsp + #24]  # spill
>>                            13e     movl    RSI, [RAX + #12 (8-bit)]  #
>> range
>>                            141     NullCheck RAX
>>
>>                            What is the equivalent with the new
>> PrintAssembly flag (using hsdis)?
>>
>>                            Moreover, as stated on the wiki page [1],
>> loops are optimized if the stride is a compile-time
>>                constant. I
>>                        performed
>>                            a few tests on a kmeans program, with 3 nested
>> loops, having the following (high-level)
>>        structure:
>>
>>                            ===
>>                            void method1(){
>>                            //loop 1
>>                            for(int i = 0; i< rows1; i++){
>>                              //...
>>                               for(int j = 0; j< rows2; j++){
>>                               //...
>>                                for(int k = 0; j < cols; k++){ array[j *
>> cols + k] = //...}
>>                               }
>>                            }
>>                            }
>>
>>                            void method2(){
>>                            //loop 2
>>                            for(int i =0; i < rows1; i++){
>>                               for(int j=0 ; i< rows2; j++){
>>                                 for(int k=0 ; k< cols; k++){
>>                                   array[i*cols+k] = //...
>>                                 }
>>                               }
>>                            }
>>                            }
>>
>>                            void main(){
>>
>>                              do{
>>                                method1(); method2();
>>                              }while(!converged)
>>
>>                            }
>>                            ====
>>
>>                            In the first test, cols is an int whose value
>> is determined at runtime (by reading a file),
>>        in the
>>                second
>>                        test, it
>>                            is given as a compile-time constant(3). In the
>> second test, there is a */significant*/ speed-up
>>                (around 40%).
>>
>>                            However, when studying the diff of the output
>> of PrintOptoAssembly for both method1 and method2,
>>                there is no
>>                            difference (apart from slight value changes in
>> frequency). Would you have any hints as to
>>        where I
>>                could look for
>>                            differences?
>>
>>                            Thanks a lot,
>>                            Manohar
>>
>>                            [1] https://wikis.oracle.com/____**
>> display/HotSpotInternals/____**RangeCheckElimination<https://wikis.oracle.com/____display/HotSpotInternals/____RangeCheckElimination>
>>        <https://wikis.oracle.com/__**display/HotSpotInternals/__**
>> RangeCheckElimination<https://wikis.oracle.com/__display/HotSpotInternals/__RangeCheckElimination>
>> >
>>
>>
>>        <https://wikis.oracle.com/__**display/HotSpotInternals/__**
>> RangeCheckElimination<https://wikis.oracle.com/__display/HotSpotInternals/__RangeCheckElimination>
>>        <https://wikis.oracle.com/**display/HotSpotInternals/**
>> RangeCheckElimination<https://wikis.oracle.com/display/HotSpotInternals/RangeCheckElimination>
>> >>
>>
>>
>>
>>                As Kris pointed you need to fix your example:
>>
>>
>>
>>
>> --
>> Vitaly
>> 617-548-7007 (mobile)
>>
>


-- 
Vitaly
617-548-7007 (mobile)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120117/784a0a21/attachment-0001.html 

From robert.ottenhag at oracle.com  Tue Jan 17 04:07:50 2012
From: robert.ottenhag at oracle.com (Robert Ottenhag)
Date: Tue, 17 Jan 2012 13:07:50 +0100
Subject: RFR (S): 7130391: Add a framework for vendor-specific validation
	control of setting command-line switches in Hotspot
In-Reply-To: <4F14E661.5080405@oracle.com>
References: <5a181ad1-ef64-4b00-88ce-69829f2ce80b@default>
	<4F14E661.5080405@oracle.com>
Message-ID: <4F156496.3060206@oracle.com>

David,

Thanks for the review.

On 01/17/2012 04:09 AM, David Holmes wrote:
> Hi Robert,
>
> I've added serviceability to the cc list.

Good, will try to remember that ;-)

>
> On 17/01/2012 12:04 PM, Robert Ottenhag wrote:
>> Webrev: http://cr.openjdk.java.net/~rottenha/7130391/webrev.00
>>
>> This fix adds optional validation control to the setting of 
>> command-line switches in Hotspot, and allows it to have 
>> vendor-specific extensions if necessary.
>
> Does this imply that the Java management APIs (eg 
> com.sun.management.VMOption) need to be changed to reflect these 
> restrictions? Presently VMOptions are either writeable or not, but 
> this makes them conditionally-writeable.

No, since the Java management  APIs already cares for conditional 
writes. According to 
com.sun.management.HotSpotDiagnosticMXBean.setVMOption() it will throw 
IllegalArgumentException if the new value is invalid.

>
>> The design follows the previously added framework for vendor-specific 
>> command-line switch extensions in CR7117389.
>>
>> The validation control is handled by new boolean methods 
>> Flag::is_valid_<type>(value,origin) that are called at the beginning 
>> of each call to CommandLineFlags[Ex]::<type>AtPut() to verify that 
>> the new value and origin are valid replacements for the current value 
>> and origin for this flag.
>>
>> When parsing the command line options, a failed validation will 
>> typically result in an error message and exit with "Unrecognized VM 
>> option '<flag-name>'".  When used dynamically using the attach API or 
>> management API the resulting operation will fail, leaving it up to 
>> the caller to handle it as appropriate.
>
> The error message doesn't really seem appropriate - it may well be a 
> recognized option, you just can't set it to that value in that way. 
> Ideally there would be a way for the validation logic to supply a 
> meaningful error message. In its absence the top-level message should 
> reflect the new type of error.

You are absolutely right, but the current fix is in line with the 
existing bad error messages where any kind of malformatted command line 
flags results in Unrecognized VM option, whether the reason is an 
unknown name, bad type semantics (using +- for bool semantics on an 
integer flag), or if the flag is locked.

I will target meaningful error messages for command line parsing in a 
direct follow up bug to this fix.

>
> Also some of the failures lead to crashes - which seems wrong to me - 
> see below.
>
> ----
>
> src/share/vm/services/management.cpp:
>
> 1821   if (!succeed) {
> 1822     THROW_MSG(vmSymbols::java_lang_IllegalArgumentException(),
> 1823               "This flag is not writeable with this value or 
> origin.");
>
> That's a rather cryptic error message. How about:
>
> "Flag can not be set to the requested value using this API"
>
> ?

Yes, "origin" does not make sense to the upper Java layer. I will use 
your suggestion.

>
> ----
>
> src/share/vm/runtime/globals_ext.hpp
>
> With all the
>
> inline bool Flag::is_valid_ext_T(T value, FlagValueOrigin origin)
>
> functions, is it necessary to include the type T in the function name?

It is necessary if using type safe variants with T value as argument 
since overloading does not differ between different typedef names that 
resolves to the same native types, e.g. uintx and uint64_t are both 
unsigned long int.

I am considering a condensed variant that replaces T by void* instead, 
and do the type casting based on the targeted flag, reducing the number 
of functions.

>
>
> -----
>
> src/share/vm/runtime/globals.cpp
>
> The use of the guarantees seems wrong as it means an invalid option 
> will trigger a VM crash rather than a clean exit during 
> initialization. It seems to me that none of the code in arguments.cpp 
> that uses the FLAG_SET_* macros (which in turn use the 
> CommandLineFlagsEx functions you added the guarantees to) anticipates 
> any possibility for failure. I think if you are going this path then 
> you have no choice but to change the CommandLineFlagsEx methods to 
> return bool and update the FLAG_SET macros to try and perform 
> appropriate error handling.

I see your point, and in theory such as VM crash could occur anytime 
later in a JVM session if rarely running code would make use of 
FLAG_SET_* to change the value of a VM flag to an invalid value or origin.

Seems as if the options are either to
a) ignore validation tests for the FLAG_SET_* macros, and trust that 
they always set valid values. This can be partly verified by static code 
inspection by looking for any variables that actually have validation 
logic associated to them (since the variable name is defined at compile 
time), assuming one has access to all code, but it is not perfect in 
case code for changing a variable with validation logic exists.
b) contain the error handling within the FLAG_SET_* macros, like using 
guarantee(), but maybe exception logic can help?
c) require usage of the FLAG_SET_* macros to handle result codes and 
pass it up the call chain.

Also, the current macro FLAG_SET_DEFAULT does a direct write to the flag 
value without going through <type>AtPut(). This macro must be rewritten 
to have validation control to close the holes. The current call format 
will require all call sites to include type name as with 
FLAG_SET_{CMDLINE,ERGO} has, or to use slower lookup by variable name.

/Robert

>
> David
> -----
>
>
>> A simple use case for validation is a manageable flag whose current 
>> value can not be less than the previous value, while a more complex 
>> example may base the validation on multiple other flags, etc.
>>
>> Thanks,
>>
>> /Robert
>>


-- 
Oracle
Robert Ottenhag | Senior Member of Technical Staff
Phone: +46850630961 | Fax: +46850630911 | Mobile: +46707106161
Oracle Java HotSpot Virtual Machine
ORACLE Sweden | Folkungagatan 122 | SE-116 30 Stockholm

Oracle Svenska AB, Kronborgsgr?nd 17, S-164 28 KISTA, reg.no. 556254-6746

Green Oracle

Oracle is committed to developing practices and products that help protect the environment
--


From Dmitry.Samersoff at oracle.com  Tue Jan 17 04:22:11 2012
From: Dmitry.Samersoff at oracle.com (Dmitry Samersoff)
Date: Tue, 17 Jan 2012 16:22:11 +0400
Subject: RFR (S): 7130391: Add a framework for vendor-specific validation
	control of setting command-line switches in Hotspot
In-Reply-To: <5a181ad1-ef64-4b00-88ce-69829f2ce80b@default>
References: <5a181ad1-ef64-4b00-88ce-69829f2ce80b@default>
Message-ID: <4F1567F3.70106@oracle.com>

Robert,

I'm second to David,a message "wrong flag value or origin" looks very
cryptic.

Besides it, looks good for me.

-Dmitry


On 2012-01-17 06:04, Robert Ottenhag wrote:
> Hi,
> 
> Webrev: http://cr.openjdk.java.net/~rottenha/7130391/webrev.00
> 
> This fix adds optional validation control to the setting of command-line switches in Hotspot, and allows it to have vendor-specific extensions if necessary.
> 
> The design follows the previously added framework for vendor-specific command-line switch extensions in CR7117389.
> 
> The validation control is handled by new boolean methods Flag::is_valid_<type>(value,origin) that are called at the beginning of each call to CommandLineFlags[Ex]::<type>AtPut() to verify that the new value and origin are valid replacements for the current value and origin for this flag.
> 
> When parsing the command line options, a failed validation will typically result in an error message and exit with "Unrecognized VM option '<flag-name>'".  When used dynamically using the attach API or management API the resulting operation will fail, leaving it up to the caller to handle it as appropriate.
> 
> A simple use case for validation is a manageable flag whose current value can not be less than the previous value, while a more complex example may base the validation on multiple other flags, etc.
> 
> Thanks,
> 
> /Robert
> 


-- 
Dmitry Samersoff
Java Hotspot development team, SPB04
* There will come soft rains ...

From robert.ottenhag at oracle.com  Tue Jan 17 05:33:01 2012
From: robert.ottenhag at oracle.com (Robert Ottenhag)
Date: Tue, 17 Jan 2012 14:33:01 +0100
Subject: RFR (S): 7130391: Add a framework for vendor-specific validation
	control of setting command-line switches in Hotspot
In-Reply-To: <4F1567F3.70106@oracle.com>
References: <5a181ad1-ef64-4b00-88ce-69829f2ce80b@default>
	<4F1567F3.70106@oracle.com>
Message-ID: <4F15788D.7020004@oracle.com>

Dmitry,

Thanks for the review.

/Robert

On 01/17/2012 01:22 PM, Dmitry Samersoff wrote:
> Robert,
>
> I'm second to David,a message "wrong flag value or origin" looks very
> cryptic.
>
> Besides it, looks good for me.
>
> -Dmitry
>
>
> On 2012-01-17 06:04, Robert Ottenhag wrote:
>> Hi,
>>
>> Webrev: http://cr.openjdk.java.net/~rottenha/7130391/webrev.00
>>
>> This fix adds optional validation control to the setting of command-line switches in Hotspot, and allows it to have vendor-specific extensions if necessary.
>>
>> The design follows the previously added framework for vendor-specific command-line switch extensions in CR7117389.
>>
>> The validation control is handled by new boolean methods Flag::is_valid_<type>(value,origin) that are called at the beginning of each call to CommandLineFlags[Ex]::<type>AtPut() to verify that the new value and origin are valid replacements for the current value and origin for this flag.
>>
>> When parsing the command line options, a failed validation will typically result in an error message and exit with "Unrecognized VM option '<flag-name>'".  When used dynamically using the attach API or management API the resulting operation will fail, leaving it up to the caller to handle it as appropriate.
>>
>> A simple use case for validation is a manageable flag whose current value can not be less than the previous value, while a more complex example may base the validation on multiple other flags, etc.
>>
>> Thanks,
>>
>> /Robert
>>
>


-- 
Oracle
Robert Ottenhag | Senior Member of Technical Staff
Phone: +46850630961 | Fax: +46850630911 | Mobile: +46707106161
Oracle Java HotSpot Virtual Machine
ORACLE Sweden | Folkungagatan 122 | SE-116 30 Stockholm

Oracle Svenska AB, Kronborgsgr?nd 17, S-164 28 KISTA, reg.no. 556254-6746

Green Oracle

Oracle is committed to developing practices and products that help protect the environment
--


From robert.ottenhag at oracle.com  Tue Jan 17 05:41:37 2012
From: robert.ottenhag at oracle.com (Robert Ottenhag)
Date: Tue, 17 Jan 2012 14:41:37 +0100
Subject: RFR (S): 7130391: Add a framework for vendor-specific validation
	control of setting command-line switches in Hotspot
In-Reply-To: <4F156496.3060206@oracle.com>
References: <5a181ad1-ef64-4b00-88ce-69829f2ce80b@default>
	<4F14E661.5080405@oracle.com> <4F156496.3060206@oracle.com>
Message-ID: <4F157A91.8080907@oracle.com>

David,

Regarding the FLAG_SET_* macros, I am thinking that we can leave them to 
a follow up bug instead.

The reason is that it can be verified by code inspection (of 
preprocessed sources) if any FLAG_SET_* macro writes to a variable known 
to have validation control.

Also, fixing that hole would require any access to the variables to 
occur through interface get/set functions, preventing direct read and 
write access (wrapping the variable in a class to prevent direct 
writes), a change too intrusive for now.

Will come back with an updated and cleaned up patch.

/Robert

On 01/17/2012 01:07 PM, Robert Ottenhag wrote:
> David,
>
> Thanks for the review.
>
> On 01/17/2012 04:09 AM, David Holmes wrote:
>> Hi Robert,
>>
>> I've added serviceability to the cc list.
>
> Good, will try to remember that ;-)
>
>>
>> On 17/01/2012 12:04 PM, Robert Ottenhag wrote:
>>> Webrev: http://cr.openjdk.java.net/~rottenha/7130391/webrev.00
>>>
>>> This fix adds optional validation control to the setting of 
>>> command-line switches in Hotspot, and allows it to have 
>>> vendor-specific extensions if necessary.
>>
>> Does this imply that the Java management APIs (eg 
>> com.sun.management.VMOption) need to be changed to reflect these 
>> restrictions? Presently VMOptions are either writeable or not, but 
>> this makes them conditionally-writeable.
>
> No, since the Java management  APIs already cares for conditional 
> writes. According to 
> com.sun.management.HotSpotDiagnosticMXBean.setVMOption() it will throw 
> IllegalArgumentException if the new value is invalid.
>
>>
>>> The design follows the previously added framework for 
>>> vendor-specific command-line switch extensions in CR7117389.
>>>
>>> The validation control is handled by new boolean methods 
>>> Flag::is_valid_<type>(value,origin) that are called at the beginning 
>>> of each call to CommandLineFlags[Ex]::<type>AtPut() to verify that 
>>> the new value and origin are valid replacements for the current 
>>> value and origin for this flag.
>>>
>>> When parsing the command line options, a failed validation will 
>>> typically result in an error message and exit with "Unrecognized VM 
>>> option '<flag-name>'".  When used dynamically using the attach API 
>>> or management API the resulting operation will fail, leaving it up 
>>> to the caller to handle it as appropriate.
>>
>> The error message doesn't really seem appropriate - it may well be a 
>> recognized option, you just can't set it to that value in that way. 
>> Ideally there would be a way for the validation logic to supply a 
>> meaningful error message. In its absence the top-level message should 
>> reflect the new type of error.
>
> You are absolutely right, but the current fix is in line with the 
> existing bad error messages where any kind of malformatted command 
> line flags results in Unrecognized VM option, whether the reason is an 
> unknown name, bad type semantics (using +- for bool semantics on an 
> integer flag), or if the flag is locked.
>
> I will target meaningful error messages for command line parsing in a 
> direct follow up bug to this fix.
>
>>
>> Also some of the failures lead to crashes - which seems wrong to me - 
>> see below.
>>
>> ----
>>
>> src/share/vm/services/management.cpp:
>>
>> 1821   if (!succeed) {
>> 1822     THROW_MSG(vmSymbols::java_lang_IllegalArgumentException(),
>> 1823               "This flag is not writeable with this value or 
>> origin.");
>>
>> That's a rather cryptic error message. How about:
>>
>> "Flag can not be set to the requested value using this API"
>>
>> ?
>
> Yes, "origin" does not make sense to the upper Java layer. I will use 
> your suggestion.
>
>>
>> ----
>>
>> src/share/vm/runtime/globals_ext.hpp
>>
>> With all the
>>
>> inline bool Flag::is_valid_ext_T(T value, FlagValueOrigin origin)
>>
>> functions, is it necessary to include the type T in the function name?
>
> It is necessary if using type safe variants with T value as argument 
> since overloading does not differ between different typedef names that 
> resolves to the same native types, e.g. uintx and uint64_t are both 
> unsigned long int.
>
> I am considering a condensed variant that replaces T by void* instead, 
> and do the type casting based on the targeted flag, reducing the 
> number of functions.
>
>>
>>
>> -----
>>
>> src/share/vm/runtime/globals.cpp
>>
>> The use of the guarantees seems wrong as it means an invalid option 
>> will trigger a VM crash rather than a clean exit during 
>> initialization. It seems to me that none of the code in arguments.cpp 
>> that uses the FLAG_SET_* macros (which in turn use the 
>> CommandLineFlagsEx functions you added the guarantees to) anticipates 
>> any possibility for failure. I think if you are going this path then 
>> you have no choice but to change the CommandLineFlagsEx methods to 
>> return bool and update the FLAG_SET macros to try and perform 
>> appropriate error handling.
>
> I see your point, and in theory such as VM crash could occur anytime 
> later in a JVM session if rarely running code would make use of 
> FLAG_SET_* to change the value of a VM flag to an invalid value or 
> origin.
>
> Seems as if the options are either to
> a) ignore validation tests for the FLAG_SET_* macros, and trust that 
> they always set valid values. This can be partly verified by static 
> code inspection by looking for any variables that actually have 
> validation logic associated to them (since the variable name is 
> defined at compile time), assuming one has access to all code, but it 
> is not perfect in case code for changing a variable with validation 
> logic exists.
> b) contain the error handling within the FLAG_SET_* macros, like using 
> guarantee(), but maybe exception logic can help?
> c) require usage of the FLAG_SET_* macros to handle result codes and 
> pass it up the call chain.
>
> Also, the current macro FLAG_SET_DEFAULT does a direct write to the 
> flag value without going through <type>AtPut(). This macro must be 
> rewritten to have validation control to close the holes. The current 
> call format will require all call sites to include type name as with 
> FLAG_SET_{CMDLINE,ERGO} has, or to use slower lookup by variable name.
>
> /Robert
>
>>
>> David
>> -----
>>
>>
>>> A simple use case for validation is a manageable flag whose current 
>>> value can not be less than the previous value, while a more complex 
>>> example may base the validation on multiple other flags, etc.
>>>
>>> Thanks,
>>>
>>> /Robert
>>>
>
>


-- 
Oracle
Robert Ottenhag | Senior Member of Technical Staff
Phone: +46850630961 | Fax: +46850630911 | Mobile: +46707106161
Oracle Java HotSpot Virtual Machine
ORACLE Sweden | Folkungagatan 122 | SE-116 30 Stockholm

Oracle Svenska AB, Kronborgsgr?nd 17, S-164 28 KISTA, reg.no. 556254-6746

Green Oracle

Oracle is committed to developing practices and products that help protect the environment
--


From mikael.gerdin at oracle.com  Tue Jan 17 07:34:53 2012
From: mikael.gerdin at oracle.com (Mikael Gerdin)
Date: Tue, 17 Jan 2012 16:34:53 +0100
Subject: Review request: White box testing API for HotSpot
In-Reply-To: <4F0F6605.5080900@oracle.com>
References: <4ED50287.3070102@oracle.com> <4F0C910A.5070205@oracle.com>
	<4F0E8B10.3030600@oracle.com> <4F0F0850.8050402@oracle.com>
	<4F0F6605.5080900@oracle.com>
Message-ID: <4F15951D.7080508@oracle.com>

David,

On 2012-01-13 00:00, David Holmes wrote:
> Hi Mikael,
>
> On 13/01/2012 2:20 AM, Mikael Gerdin wrote:
>>> wbapi.java: normal Java naming style is to use camel-case for class
>>> names. Though as WB is itself an acronym I'd be okay with WBApi. In fact
>>> I'd be happy with anything other than initial lower-case :)
>>
>> Many of our existing tests have lower-case names so I guess I thought
>> that was some sort of convention, it does not really matter to me.
>
> I think those tests must have been written by C programers ;-)
>
>> WBApi it is then.
>
> Thanks.There is a slight typo in that the file is WBapi.java not WBApi.java

Fixed.

I also re-ran JPRT to verify that it still builds on all platforms and 
found that the size of a region in G1 had changed to size_t, so I added 
a cast to jint (region sizes of >2G seems to be unreasonable).

I also tried with Jim Melvin's patch to run OS X and verified that 
"wbapitest" works on OS X as well.

/Mikael

>
> David
> -----
>
>>
>>>
>>> ---
>>>
>>> test/Makefile: does wbapitest need to be added to the phoney list?
>>
>> Yes, fixed.
>>
>> New webrev at:
>> http://cr.openjdk.java.net/~mgerdin/wbapi/webrev.3/
>> Incremental at:
>> http://cr.openjdk.java.net/~mgerdin/wbapi/webrev.2-to-3/webrev/
>>
>> /Mikael
>>
>>>
>>> ---
>>>
>>> Cheers,
>>> David
>>> -----
>>>
>>>
>>> On 11/01/2012 5:27 AM, Mikael Gerdin wrote:
>>>> Hi all
>>>>
>>>> Back from vacations now with an updated version of the webrev based on
>>>> the feedback received in this thread.
>>>> Changes include:
>>>> * removed install target from makefiles
>>>> * renamed flag form EnableWhiteBoxAPI to remove redundant Enable
>>>> * installs wb.jar into jre/lib and made -XX:+WhiteBoxAPI add wb.jar to
>>>> the boot class path from inside the VM.
>>>>
>>>> http://cr.openjdk.java.net/~mgerdin/wbapi/webrev.2/
>>>>
>>>> Thanks
>>>> Mikael Gerdin
>>>>
>>>> On 2011-11-29 17:04, Mikael Gerdin wrote:
>>>>> Hi
>>>>>
>>>>> I've been working on a white box testing API for HotSpot in order to
>>>>> allow for improved precision in vm testing.
>>>>>
>>>>> The basic idea is to open up the possibility for tests written in Java
>>>>> to call native methods which query or poke the vm in some way.
>>>>>
>>>>> The API is accessible by using the class sun/hotspot/WhiteBox which is
>>>>> not intended to be available in public builds.
>>>>> In order to allow the WhiteBox class access to the VM the
>>>>> registerNatives function is linked to JVM_RegisterWhiteBoxMethods.
>>>>> That
>>>>> function then links all the implementation functions using normal JNI
>>>>> RegisterNatives.
>>>>>
>>>>> The API is not meant to be used by end users for any intent or purpose
>>>>> and as such it is both guarded by "-XX:+UnlockDiagnosticVMOptions
>>>>> -XX:+EnableWhiteboxAPI" and the fact that the class files will not be
>>>>> present in an end user build of a JDK.
>>>>> If the VM crashes after this API has been accessed a note will be
>>>>> written in the hs_err file to signal that the API has been used.
>>>>>
>>>>> Webrev:
>>>>> http://cr.openjdk.java.net/~stefank/mgerdin/wbapi.0/webrev/
>>>>> (thanks to stefank for hosting my webrev :)
>>>>>
>>>>> CR:
>>>>> I'll file a CR tomorrow.
>>>>>
>>>>> Change comments:
>>>>>
>>>>> make/jprt.properties
>>>>>
>>>>> Add a test target to make sure that the API is available on all
>>>>> supported platforms
>>>>>
>>>>> make/**
>>>>>
>>>>> Makefile changes to build the class sun/hotspot/WhiteBox, put it in a
>>>>> JAR file and copy it to the jre/lib/endorsed directory in the export
>>>>> targets.
>>>>> The BSD makefile changes are not tested since I don't have access to
>>>>> any
>>>>> BSD/OSX machine to test them on.
>>>>>
>>>>> src/share/vm/prims/nativeLookup.cpp
>>>>>
>>>>> Special-case the method sun/hotspot/WhiteBox/registerNatives and
>>>>> link it
>>>>> to JVM_RegisterWhiteBoxMethods
>>>>>
>>>>> src/share/vm/prims/whitebox.*
>>>>>
>>>>> The implementation of the white box API. The actual API functions are
>>>>> only examples of what we want to be able to do using the API.
>>>>>
>>>>> src/share/vm/runtime/globals.hpp
>>>>>
>>>>> Add the command line flag
>>>>>
>>>>> src/share/vm/utilities/vmError.cpp
>>>>>
>>>>> Print a message in hs_err files when white box API has been used.
>>>>>
>>>>> test/Makefile
>>>>>
>>>>> Add a makefile test target for the white box API test
>>>>>
>>>>> test/sanity/wbapi.java
>>>>>
>>>>> JTreg test to ensure that the API works.
>>>>>
>>>>>
>>>>> Thanks
>>>>> /Mikael Gerdin
>>>>

From mikael.gerdin at oracle.com  Tue Jan 17 07:36:10 2012
From: mikael.gerdin at oracle.com (Mikael Gerdin)
Date: Tue, 17 Jan 2012 16:36:10 +0100
Subject: Review request: White box testing API for HotSpot
In-Reply-To: <4F15951D.7080508@oracle.com>
References: <4ED50287.3070102@oracle.com> <4F0C910A.5070205@oracle.com>
	<4F0E8B10.3030600@oracle.com> <4F0F0850.8050402@oracle.com>
	<4F0F6605.5080900@oracle.com> <4F15951D.7080508@oracle.com>
Message-ID: <4F15956A.2000300@oracle.com>

..
and the new webrev is at http://cr.openjdk.java.net/~mgerdin/wbapi/webrev.4/


On 2012-01-17 16:34, Mikael Gerdin wrote:
> David,
>
> On 2012-01-13 00:00, David Holmes wrote:
>> Hi Mikael,
>>
>> On 13/01/2012 2:20 AM, Mikael Gerdin wrote:
>>>> wbapi.java: normal Java naming style is to use camel-case for class
>>>> names. Though as WB is itself an acronym I'd be okay with WBApi. In
>>>> fact
>>>> I'd be happy with anything other than initial lower-case :)
>>>
>>> Many of our existing tests have lower-case names so I guess I thought
>>> that was some sort of convention, it does not really matter to me.
>>
>> I think those tests must have been written by C programers ;-)
>>
>>> WBApi it is then.
>>
>> Thanks.There is a slight typo in that the file is WBapi.java not
>> WBApi.java
>
> Fixed.
>
> I also re-ran JPRT to verify that it still builds on all platforms and
> found that the size of a region in G1 had changed to size_t, so I added
> a cast to jint (region sizes of >2G seems to be unreasonable).
>
> I also tried with Jim Melvin's patch to run OS X and verified that
> "wbapitest" works on OS X as well.
>
> /Mikael
>
>>
>> David
>> -----
>>
>>>
>>>>
>>>> ---
>>>>
>>>> test/Makefile: does wbapitest need to be added to the phoney list?
>>>
>>> Yes, fixed.
>>>
>>> New webrev at:
>>> http://cr.openjdk.java.net/~mgerdin/wbapi/webrev.3/
>>> Incremental at:
>>> http://cr.openjdk.java.net/~mgerdin/wbapi/webrev.2-to-3/webrev/
>>>
>>> /Mikael
>>>
>>>>
>>>> ---
>>>>
>>>> Cheers,
>>>> David
>>>> -----
>>>>
>>>>
>>>> On 11/01/2012 5:27 AM, Mikael Gerdin wrote:
>>>>> Hi all
>>>>>
>>>>> Back from vacations now with an updated version of the webrev based on
>>>>> the feedback received in this thread.
>>>>> Changes include:
>>>>> * removed install target from makefiles
>>>>> * renamed flag form EnableWhiteBoxAPI to remove redundant Enable
>>>>> * installs wb.jar into jre/lib and made -XX:+WhiteBoxAPI add wb.jar to
>>>>> the boot class path from inside the VM.
>>>>>
>>>>> http://cr.openjdk.java.net/~mgerdin/wbapi/webrev.2/
>>>>>
>>>>> Thanks
>>>>> Mikael Gerdin
>>>>>
>>>>> On 2011-11-29 17:04, Mikael Gerdin wrote:
>>>>>> Hi
>>>>>>
>>>>>> I've been working on a white box testing API for HotSpot in order to
>>>>>> allow for improved precision in vm testing.
>>>>>>
>>>>>> The basic idea is to open up the possibility for tests written in
>>>>>> Java
>>>>>> to call native methods which query or poke the vm in some way.
>>>>>>
>>>>>> The API is accessible by using the class sun/hotspot/WhiteBox
>>>>>> which is
>>>>>> not intended to be available in public builds.
>>>>>> In order to allow the WhiteBox class access to the VM the
>>>>>> registerNatives function is linked to JVM_RegisterWhiteBoxMethods.
>>>>>> That
>>>>>> function then links all the implementation functions using normal JNI
>>>>>> RegisterNatives.
>>>>>>
>>>>>> The API is not meant to be used by end users for any intent or
>>>>>> purpose
>>>>>> and as such it is both guarded by "-XX:+UnlockDiagnosticVMOptions
>>>>>> -XX:+EnableWhiteboxAPI" and the fact that the class files will not be
>>>>>> present in an end user build of a JDK.
>>>>>> If the VM crashes after this API has been accessed a note will be
>>>>>> written in the hs_err file to signal that the API has been used.
>>>>>>
>>>>>> Webrev:
>>>>>> http://cr.openjdk.java.net/~stefank/mgerdin/wbapi.0/webrev/
>>>>>> (thanks to stefank for hosting my webrev :)
>>>>>>
>>>>>> CR:
>>>>>> I'll file a CR tomorrow.
>>>>>>
>>>>>> Change comments:
>>>>>>
>>>>>> make/jprt.properties
>>>>>>
>>>>>> Add a test target to make sure that the API is available on all
>>>>>> supported platforms
>>>>>>
>>>>>> make/**
>>>>>>
>>>>>> Makefile changes to build the class sun/hotspot/WhiteBox, put it in a
>>>>>> JAR file and copy it to the jre/lib/endorsed directory in the export
>>>>>> targets.
>>>>>> The BSD makefile changes are not tested since I don't have access to
>>>>>> any
>>>>>> BSD/OSX machine to test them on.
>>>>>>
>>>>>> src/share/vm/prims/nativeLookup.cpp
>>>>>>
>>>>>> Special-case the method sun/hotspot/WhiteBox/registerNatives and
>>>>>> link it
>>>>>> to JVM_RegisterWhiteBoxMethods
>>>>>>
>>>>>> src/share/vm/prims/whitebox.*
>>>>>>
>>>>>> The implementation of the white box API. The actual API functions are
>>>>>> only examples of what we want to be able to do using the API.
>>>>>>
>>>>>> src/share/vm/runtime/globals.hpp
>>>>>>
>>>>>> Add the command line flag
>>>>>>
>>>>>> src/share/vm/utilities/vmError.cpp
>>>>>>
>>>>>> Print a message in hs_err files when white box API has been used.
>>>>>>
>>>>>> test/Makefile
>>>>>>
>>>>>> Add a makefile test target for the white box API test
>>>>>>
>>>>>> test/sanity/wbapi.java
>>>>>>
>>>>>> JTreg test to ensure that the API works.
>>>>>>
>>>>>>
>>>>>> Thanks
>>>>>> /Mikael Gerdin
>>>>>

From daniel.daugherty at oracle.com  Tue Jan 17 09:47:37 2012
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Tue, 17 Jan 2012 10:47:37 -0700
Subject: RFR (XS): 7126732: MAC: Require Mac OS X builds/tests for JPRT
	integrate jobs for HotSpot
In-Reply-To: <4F149ABA.3000708@oracle.com>
References: <4F11B537.2000504@oracle.com> <4F13F18E.1080300@oracle.com>
	<4F149ABA.3000708@oracle.com>
Message-ID: <4F15B439.1030604@oracle.com>

Thumbs up!

Dan


On 1/16/12 2:46 PM, James Melvin wrote:
> Hi Mikael,
>
> Nice find! I've added the missing line to add internalvmtests to each
> JPRT run on Mac OS X. New webrev posted and JPRT test job complete.
>
> WEBREV: http://cr.openjdk.java.net/~jmelvin/7126732/webrev.01
> JPRT: 
> http://prt-web.us.oracle.com//archive/2012/01/2012-01-16-172504.jmelvin.7126732
>
> Thanks!
>
> Jim
>
>
>
> On 1/16/12 4:44 AM, Mikael Gerdin wrote:
>> Hi Jim
>>
>> on line 575, is there any particular reason that you don't add macosx to
>> jprt.make.rule.test.targets.standard.internalvmtests?
>>
>> /Mikael
>>
>> On 2012-01-14 18:02, James Melvin wrote:
>>> Greetings,
>>>
>>> We're ready to require HotSpot builds on Mac OS X for JPRT integrate
>>> jobs. There are 3 mac-minis in each queue. Build/Test times are short
>>> relative to other platforms. Uses the stable Linux testlist for now.
>>>
>>> http://cr.openjdk.java.net/~jmelvin/7126732/webrev.00
>>>
>>> Tested with *several* JPRT submissions for other bugfixes. I'd like to
>>> integrate this change right after the current snapshot window.
>>>
>>> Feedback welcome.
>>>
>>> Thanks,
>>>
>>> Jim

From vladimir.kozlov at oracle.com  Tue Jan 17 10:02:19 2012
From: vladimir.kozlov at oracle.com (Vladimir Kozlov)
Date: Tue, 17 Jan 2012 10:02:19 -0800
Subject: Detecting range check elimination with PrintAssembly
In-Reply-To: <CAHjP37H_xCp8D4sV+e=hE3jF2_g6CJb9TJ4dgxhp+U8aGV86tQ@mail.gmail.com>
References: <CABhSbLyAwFLFwhmTzPE=0DiXK8e8kbmvr3tr-0AQ=KyFz1Mjsw@mail.gmail.com>	<CA+cQ+tTd_yhyu1=06JSgskU-kJ_qh=cXmnYdE6a7TgCR7s=uxg@mail.gmail.com>	<4F135374.8020004@oracle.com>	<CABhSbLy0gd3BFmBRX-aUgEt7h2HBHer_AQENMGWYTRN4p5-wEg@mail.gmail.com>	<4F149D88.2060305@oracle.com>	<CAHjP37F+tF=66sCWNOGzH+4v3YeEQ6x+2NE_MPFZUa2B7uV-0g@mail.gmail.com>	<4F14B68A.6020808@oracle.com>	<CAHjP37GDEs-8CnNcCjQVEKCfqQdxWHn=pNFaB_XRTt1fORdwPQ@mail.gmail.com>	<4F14FEA1.6020704@oracle.com>
	<CAHjP37H_xCp8D4sV+e=hE3jF2_g6CJb9TJ4dgxhp+U8aGV86tQ@mail.gmail.com>
Message-ID: <4F15B7AB.2050107@oracle.com>

Vitaly Davidovich wrote:
> Vladimir, thanks for that -- very useful and interesting to know.  So I 
> take it a compilation after initial deopt may be less aggressive than 
> 1st time around simply because the accumulated MD might be in a state 
> such that it guides optimization in a different direction, is that right?

Yes, it is right, especially for inlining dynamic calls based on target 
profiling when after some time an other method is called. We also have limit how 
many times method could be recompiled after which the method will be only 
interpreted. Also we have limit on how many uncommon traps (deoptimizations) 
happened in method after which we stop generating some optimizations which use 
uncommon traps. Usually it means generating slow paths in compiled code which we 
thought (based on original profiling) should not be executed (so we generated 
uncommon trap). As result compiled code size will be changed and inlining also. 
Note, it happens not immediately but only after significant number of 
deoptimizations: PerMethodTrapLimit is 100 and PerMethodRecompilationCutoff is 400.

> 
> I guess how does one reason about the generated assembly when it's so 
> (possibly) dynamic? :) I've looked at the assembly in a product hotspot 
> via hsdis, but then you realize that you may be looking at just one 
> compilation of it, and it may be different at some point later.  Also, 
> for something like Intel's VTune (or Amplifier) that support showing 
> JIT'd assembly, I guess it's the same issue? This is quite different, of 
> course, with .NET CLR's JIT because there's no profiling there -- 1st 
> time method is hit, it gets JIT'd and there's no code pitching (makes 
> looking at the assembly seem a bit more "reassuring").

VTune shows several compiled methods. We have our profiling tool on Solaris 
(Analizer) which do the same. Note that deoptimization and recompilation usually 
happen only during startup. So only latest compiled version used in long run.

> 
> In general, would you say that the C2 compiler favors "over 
> aggressiveness" with deopt guards? Or does it try to avoid deopts and 
> only perform aggressive optimizations where the profile is quite 
> conclusive on type and/or counters?

C2 does compilations as aggressive as possible. And it will be less aggressive 
only during later recompilations.

Regards,
Vladimir

> 
> Thanks
> 
> On Mon, Jan 16, 2012 at 11:52 PM, Vladimir Kozlov 
> <vladimir.kozlov at oracle.com <mailto:vladimir.kozlov at oracle.com>> wrote:
> 
>     I need to be more clear. We can't get 100% correct profiling result
>     for counters. First, profiling structure (MethodData) are created
>     only after method is executed for some time (for Server compiler it
>     is 3000 invocations). It is done to avoid profiling short programs
>     since profiling will slowdown Interpreter by about 20%. This delayed
>     profiling introduce first discrepancy between counters. MD could be
>     also created when method has a hot loop (a lot of iterations) and
>     OSR compilation is requested. As result counters in the loop will
>     not correlate with counters before the loop. An other reason for not
>     precise profiling counters is execution of method by multiple
>     threads. We have only one MD structure per method which is updated
>     (not atomically for speed) by all such threads. And when method is
>     prepared for compilation compiler thread creates snapshot of MD
>     which could happened in the middle of method execution by java threads.
> 
>     We don't reset MD during deoptimization because we still need
>     previous values (in addition to counters we collect type
>     information). So counters are accumulated. And profiling is resumed
>     from deoptimization point which again adds discrepancy.
> 
>     I hope it explains why we can't trust 100% to profiling counters.
> 
>     Regards,
>     Vladimir
> 
> 
>     On 1/16/12 4:15 PM, Vitaly Davidovich wrote:
> 
>         Vladimir, thanks for the explanation and the code pointer.
> 
>         Intuitively, it would seem like a good idea to trust the profile
>         100% if it reports the same value used 100% of the time
>         (I can see how anything less than 100%, even a very high
>         probability of same value, is not trustworthy) given sufficient
>         trips through the loop.  Although I can see how an app may have
>         phases where same value is seen for a while before it's
>         switched, but that's where I thought deopt would help.  There
>         must be a good chunk of code out there that doesn't know
>         at static compilation time the loop count (so can't use
>         compile-time constant), but at runtime the actual value doesn't
>         change for many many trips through the loop; I know I have code
>         like that in various places.
> 
>         What's the reason a compilation after deopt would not be as
>         aggressive as the 1st time? Is it because the profile
>         information may be "weaker" (i.e. more uncertainty in it)? I
>         thought the profile is completely reset after deopt, so I
>         would think if the loop is now executed with a different
>         "constant" value (e.g. in our example, instead of 3 it's now
>         4), then the same optimizations will be applied (of course if
>         unrolling the loop is no longer advantageous due to a much
>         different value, I can see how different optimizations will be
>         applied).
> 
>         Thanks
> 
>         On Mon, Jan 16, 2012 at 6:45 PM, Vladimir Kozlov
>         <vladimir.kozlov at oracle.com <mailto:vladimir.kozlov at oracle.com>
>         <mailto:vladimir.kozlov at __oracle.com
>         <mailto:vladimir.kozlov at oracle.com>>> wrote:
> 
>            Vitaly,
> 
>            We do use profile_trip_cnt during loop unroll calculation but
>         not during fully unroll because we can't trust it 100%
>            since program's phase and number of iterations could change
>         after method is compiled. See policy_unroll() and
>            policy_maximally_unroll():
> 
>          
>          http://hg.openjdk.java.net/____hsx/hotspot-comp/hotspot/file/____89d0a5d40008/src/share/vm/____opto/loopTransform.cpp
>         <http://hg.openjdk.java.net/__hsx/hotspot-comp/hotspot/file/__89d0a5d40008/src/share/vm/__opto/loopTransform.cpp>
> 
>          
>          <http://hg.openjdk.java.net/__hsx/hotspot-comp/hotspot/file/__89d0a5d40008/src/share/vm/__opto/loopTransform.cpp
>         <http://hg.openjdk.java.net/hsx/hotspot-comp/hotspot/file/89d0a5d40008/src/share/vm/opto/loopTransform.cpp>>
> 
>            We could use deopt as you suggested but deoptimization is
>         double-edge sword, when method recompiled after
>            deoptimization some aggressive optimizations will not be
>         executed for it so the new generated code could be slower.
> 
>            Regards,
>            Vladimir
> 
> 
>            On 1/16/12 2:37 PM, Vitaly Davidovich wrote:
> 
>                Hi Vladimir,
> 
>                If x_col is always seen to be same value in the profile
>         shouldn't the loop be unrolled as well with some deopt
>                guard? Or
>                does this not participate in profiling?
> 
>                Thanks
> 
>                On Jan 16, 2012 4:57 PM, "Vladimir Kozlov"
>         <vladimir.kozlov at oracle.com <mailto:vladimir.kozlov at oracle.com>
>         <mailto:vladimir.kozlov at __oracle.com
>         <mailto:vladimir.kozlov at oracle.com>>
>                <mailto:vladimir.kozlov@
>         <mailto:vladimir.kozlov@>__orac__le.com <http://oracle.com>
>         <mailto:vladimir.kozlov at __oracle.com
>         <mailto:vladimir.kozlov at oracle.com>>>> wrote:
> 
>                 > be different, because the expressions are similar. The
>         difference in runtime (due to cols being a compile-time
>                 > constant) will be visible elsewhere. Is that right? If
>         so, where would I be able to detect this?
> 
>                    In such situations we usually use some visual tools
>         to see difference between log outputs. At least you can use
>                'diff'. You may need to replace instructions addresses in
>         outputs (number at the beginning of lines) with the same
>                    value to match. There are few tricks you may use to
>         get similar PrintOptoAssembly output. Use next flags to
>                avoid
>                    mixing output from program output and from 2 compiler
>         threads (flags stop program until a method is compiled
>                and run
>                    only one compiler thread):
> 
>                    -Xbatch -XX:CICompilerCount=1
> 
>                    Also add -XX:+PrintCompilation -XX:+PrintInlining to
>         see what method is compiled and inlined. Note that you
>                may see
>                    similar output for individual methods but could be
>         big difference in compiled caller (computeAll()) method
>                where 2
>                    loop methods could be inlined. So you need to compare
>         all compiled methods.
> 
>                    In general, to have constant as loop limit is always
>         win because some checks in generated code could be
>                avoided and
>                    more optimizations could be done for such loops. Use
>         -XX:+TraceLoopOpts to see what loop optimizations are
>                done in
>                    both cases.
> 
>                    For example, in your code you set 'x_col = 3', as
>         result the next loop in
>                constructNearestClusterVector(______) will be
> 
> 
>                    fully unrolled when this method is inlined into
>         computeAll() and x_col is replaced with '3':
> 
>                            for(k = 0; k < x_col; k++) {
>                              double tmp = x[i*x_col + k] - mu[j* mu_col
>         + k];
>                              dist += tmp * tmp;
>                            }
> 
>                    Vladimir
> 
>                    On 1/16/12 1:39 AM, Manohar Jonnalagedda wrote:
> 
>                        Hi Kris, Vladimir,
> 
>                        thanks for both your responses.
> 
>                            Second, your two test methods are different
>         so you can't directly compare them. method1() iterates
>                over rows
>                        using
>                            middle loop index 'j' and method2() uses
>         external loop index 'i'. Unless they are typos again.
> 
> 
>                        You are right, these are indeed typos. As Kris
>         suggested, I have the code printed here:
>                http://pastebin.com/xRFD1Nt1.
>                        The methods corresponding to method1, and method2
>         are constructNearestClusterVector and
>                computeNewCentroids. Their
>                        PrintOptoAssembly outputs are respectively at
>         http://pastebin.com/1evN8b3K and http://pastebin.com/FxkVWTD5
> 
>                        Also, it seems I have not explained myself
>         correctly. I am not trying to compare the performance of
>                method1 with
>                        respect
>                        to that of method2: method1 and method2 both run
>         in the same program. What I am trying to compare is their
>                        performance
>                        in two cases:
>                        - when cols is a compile-time constant (much faster)
>                        - when cols is a value determined at run-time
> 
>                            If you are using jdk7 there are few flags you
>         can use to print loop optimizations information. They
>                need debug
>                            version of VM but it is not problem for you,
>         I think, since you can use debug PrintOptoAssembly flag.
> 
>                            -XX:+TraceLoopOpts prints all happened loop
>         optimizations and loop tree after each round of loop opts,
>                            -XX:+TraceLoopPredicate prints RC information
>         when it is moved from a loop,
>                            -XX:+TraceRangeLimitCheck prints additional
>         information for RC elimination optimization.
> 
> 
>                        Thanks for these, I will have a look at what they
>         output.
> 
>                            Fourth, range check expression in your
>         example is not what you think. RC expression should be next:
>                            (i*stride+offset) where 'i' is loop variable,
>         'stride' is constant and 'offset' is loop invariant.
> 
>                            In your example 'offset' is (j * cols) since
>         it is loop invariant, 'k' is loop variable and stride
>                is '1' (one).
>                            In both your methods RC will be moved out of
>         inner loop so the code for it will be the same. The only
>                        difference in
>                            these methods will be where and how (j *
>         cols) and (i * cols) expressions are calculated.
> 
> 
>                                I'd guess it's the difference in locality
>         that made the difference in performance in your two tests.
> 
>                          Thanks for the explanation. I understand from
>         the above that the assembly output in both cases
>                mentioned above
>                        may not
>                        be different, because the expressions are
>         similar. The difference in runtime (due to cols being a
>                compile-time
>                        constant)
>                        will be visible elsewhere. Is that right? If so,
>         where would I be able to detect this?
> 
>                        Cheers,
>                        Manohar
> 
>                                In your PrintOptoAssembly output snippet,
>         the instruction at 0x13e is a LoadRange, which loads
>                the range
>                        from
>                                the header
>                                of an array:
> 
>                                (from x86_64.ad <http://x86_64.ad>
>         <http://x86_64.ad> <http://x86_64.ad> <http://x86_64.ad>
>         <http://x86_64.ad>)
> 
> 
> 
>                                // Load Range
>                                instruct loadRange(rRegI dst, memory mem)
>                                %{
>                                   match(Set dst (LoadRange mem));
> 
>                                   ins_cost(125); // XXX
>                                   format %{ "movl    $dst, $mem\t# range" %}
>                                   opcode(0x8B);
>                                   ins_encode(REX_reg_mem(dst, mem),
>         OpcP, reg_mem(dst, mem));
>                                   ins_pipe(ialu_reg_mem);
>                                %}
> 
>                                That's not a range check just yet; the
>         real check, if any, should come after the null check, in
>                the form of
>                                comparing
>                                something else with RSI. But you didn't
>         show what's after the null check, how RSI is used, so
>                it's hard
>                        to say what
>                                you're seeing in your example.
> 
>                                As for the two test examples, could you
>         paste the entire source code, with the PrintOptoAssembly
>                output of
>                                method1() and
>                                method2() ? The first example looks
>         weird, maybe it's a typo but you're using "j < cols" as the loop
>                        condition
>                                for the
>                                inner loop.
> 
> 
>                                - Kris
> 
>                                On Mon, Jan 16, 2012 at 1:59 AM, Manohar
>         Jonnalagedda <manojo10386 at gmail.com <mailto:manojo10386 at gmail.com>
>                <mailto:manojo10386 at gmail.com <mailto:manojo10386 at gmail.com>>
>                <mailto:manojo10386 at gmail.com
>         <mailto:manojo10386 at gmail.com> <mailto:manojo10386 at gmail.com
>         <mailto:manojo10386 at gmail.com>>__> <mailto:manojo10386 at gmail.com
>         <mailto:manojo10386 at gmail.com>
>                <mailto:manojo10386 at gmail.com
>         <mailto:manojo10386 at gmail.com>> <mailto:manojo10386 at gmail.com
>         <mailto:manojo10386 at gmail.com> <mailto:manojo10386 at gmail.com
>         <mailto:manojo10386 at gmail.com>>__>__>
>                <mailto:manojo10386 at gmail.com
>         <mailto:manojo10386 at gmail.com> <mailto:manojo10386 at gmail.com
>         <mailto:manojo10386 at gmail.com>> <mailto:manojo10386 at gmail.com
>         <mailto:manojo10386 at gmail.com>
>                <mailto:manojo10386 at gmail.com
>         <mailto:manojo10386 at gmail.com>>__> <mailto:manojo10386 at gmail.com
>         <mailto:manojo10386 at gmail.com> <mailto:manojo10386 at gmail.com
>         <mailto:manojo10386 at gmail.com>>
>                <mailto:manojo10386 at gmail.com
>         <mailto:manojo10386 at gmail.com> <mailto:manojo10386 at gmail.com
>         <mailto:manojo10386 at gmail.com>>__>__>__>> wrote:
> 
>                                    Hello,
> 
>                                    following this reference on Range
>         Check Elimination done by the Hotspot compiler [1], I was
>                keen in
>                        knowing
>                                how I
>                                    can detect whether range checks are
>         taking place in loops by inspecting output using the
>                        PrintAssembly flag;
>                                with
>                                    the old PrintOptoAssembly flag, I
>         have seen output such as the following, which I assume to
>                be range
>                        checks :
> 
>                                    B11: #  B73 B12 &lt;- B10  Freq: 1.21365
>                                    139     movq    RAX, [rsp + #24]  # spill
>                                    13e     movl    RSI, [RAX + #12
>         (8-bit)]  # range
>                                    141     NullCheck RAX
> 
>                                    What is the equivalent with the new
>         PrintAssembly flag (using hsdis)?
> 
>                                    Moreover, as stated on the wiki page
>         [1], loops are optimized if the stride is a compile-time
>                        constant. I
>                                performed
>                                    a few tests on a kmeans program, with
>         3 nested loops, having the following (high-level)
>                structure:
> 
>                                    ===
>                                    void method1(){
>                                    //loop 1
>                                    for(int i = 0; i< rows1; i++){
>                                      //...
>                                       for(int j = 0; j< rows2; j++){
>                                       //...
>                                        for(int k = 0; j < cols; k++){
>         array[j * cols + k] = //...}
>                                       }
>                                    }
>                                    }
> 
>                                    void method2(){
>                                    //loop 2
>                                    for(int i =0; i < rows1; i++){
>                                       for(int j=0 ; i< rows2; j++){
>                                         for(int k=0 ; k< cols; k++){
>                                           array[i*cols+k] = //...
>                                         }
>                                       }
>                                    }
>                                    }
> 
>                                    void main(){
> 
>                                      do{
>                                        method1(); method2();
>                                      }while(!converged)
> 
>                                    }
>                                    ====
> 
>                                    In the first test, cols is an int
>         whose value is determined at runtime (by reading a file),
>                in the
>                        second
>                                test, it
>                                    is given as a compile-time
>         constant(3). In the second test, there is a */significant*/ speed-up
>                        (around 40%).
> 
>                                    However, when studying the diff of
>         the output of PrintOptoAssembly for both method1 and method2,
>                        there is no
>                                    difference (apart from slight value
>         changes in frequency). Would you have any hints as to
>                where I
>                        could look for
>                                    differences?
> 
>                                    Thanks a lot,
>                                    Manohar
> 
>                                    [1]
>         https://wikis.oracle.com/______display/HotSpotInternals/______RangeCheckElimination
>         <https://wikis.oracle.com/____display/HotSpotInternals/____RangeCheckElimination>
>              
>          <https://wikis.oracle.com/____display/HotSpotInternals/____RangeCheckElimination
>         <https://wikis.oracle.com/__display/HotSpotInternals/__RangeCheckElimination>>
> 
> 
>              
>          <https://wikis.oracle.com/____display/HotSpotInternals/____RangeCheckElimination
>         <https://wikis.oracle.com/__display/HotSpotInternals/__RangeCheckElimination>
>              
>          <https://wikis.oracle.com/__display/HotSpotInternals/__RangeCheckElimination
>         <https://wikis.oracle.com/display/HotSpotInternals/RangeCheckElimination>>>
> 
> 
> 
>                        As Kris pointed you need to fix your example:
> 
> 
> 
> 
>         --
>         Vitaly
>         617-548-7007 <tel:617-548-7007> (mobile)
> 
> 
> 
> 
> -- 
> Vitaly
> 617-548-7007 (mobile)

From vitalyd at gmail.com  Tue Jan 17 10:10:05 2012
From: vitalyd at gmail.com (Vitaly Davidovich)
Date: Tue, 17 Jan 2012 13:10:05 -0500
Subject: Detecting range check elimination with PrintAssembly
In-Reply-To: <4F15B7AB.2050107@oracle.com>
References: <CABhSbLyAwFLFwhmTzPE=0DiXK8e8kbmvr3tr-0AQ=KyFz1Mjsw@mail.gmail.com>
	<CA+cQ+tTd_yhyu1=06JSgskU-kJ_qh=cXmnYdE6a7TgCR7s=uxg@mail.gmail.com>
	<4F135374.8020004@oracle.com>
	<CABhSbLy0gd3BFmBRX-aUgEt7h2HBHer_AQENMGWYTRN4p5-wEg@mail.gmail.com>
	<4F149D88.2060305@oracle.com>
	<CAHjP37F+tF=66sCWNOGzH+4v3YeEQ6x+2NE_MPFZUa2B7uV-0g@mail.gmail.com>
	<4F14B68A.6020808@oracle.com>
	<CAHjP37GDEs-8CnNcCjQVEKCfqQdxWHn=pNFaB_XRTt1fORdwPQ@mail.gmail.com>
	<4F14FEA1.6020704@oracle.com>
	<CAHjP37H_xCp8D4sV+e=hE3jF2_g6CJb9TJ4dgxhp+U8aGV86tQ@mail.gmail.com>
	<4F15B7AB.2050107@oracle.com>
Message-ID: <CAHjP37HWNqScHKKkdZ9nQhkj_BWq6PQ79jSDANLyyLkLRjD0PA@mail.gmail.com>

Awesome info - thanks very much for taking the time to answer.

Sent from my phone
On Jan 17, 2012 1:02 PM, "Vladimir Kozlov" <vladimir.kozlov at oracle.com>
wrote:

> Vitaly Davidovich wrote:
>
>> Vladimir, thanks for that -- very useful and interesting to know.  So I
>> take it a compilation after initial deopt may be less aggressive than 1st
>> time around simply because the accumulated MD might be in a state such that
>> it guides optimization in a different direction, is that right?
>>
>
> Yes, it is right, especially for inlining dynamic calls based on target
> profiling when after some time an other method is called. We also have
> limit how many times method could be recompiled after which the method will
> be only interpreted. Also we have limit on how many uncommon traps
> (deoptimizations) happened in method after which we stop generating some
> optimizations which use uncommon traps. Usually it means generating slow
> paths in compiled code which we thought (based on original profiling)
> should not be executed (so we generated uncommon trap). As result compiled
> code size will be changed and inlining also. Note, it happens not
> immediately but only after significant number of deoptimizations:
> PerMethodTrapLimit is 100 and PerMethodRecompilationCutoff is 400.
>
>
>> I guess how does one reason about the generated assembly when it's so
>> (possibly) dynamic? :) I've looked at the assembly in a product hotspot via
>> hsdis, but then you realize that you may be looking at just one compilation
>> of it, and it may be different at some point later.  Also, for something
>> like Intel's VTune (or Amplifier) that support showing JIT'd assembly, I
>> guess it's the same issue? This is quite different, of course, with .NET
>> CLR's JIT because there's no profiling there -- 1st time method is hit, it
>> gets JIT'd and there's no code pitching (makes looking at the assembly seem
>> a bit more "reassuring").
>>
>
> VTune shows several compiled methods. We have our profiling tool on
> Solaris (Analizer) which do the same. Note that deoptimization and
> recompilation usually happen only during startup. So only latest compiled
> version used in long run.
>
>
>> In general, would you say that the C2 compiler favors "over
>> aggressiveness" with deopt guards? Or does it try to avoid deopts and only
>> perform aggressive optimizations where the profile is quite conclusive on
>> type and/or counters?
>>
>
> C2 does compilations as aggressive as possible. And it will be less
> aggressive only during later recompilations.
>
> Regards,
> Vladimir
>
>
>> Thanks
>>
>> On Mon, Jan 16, 2012 at 11:52 PM, Vladimir Kozlov <
>> vladimir.kozlov at oracle.com <mailto:vladimir.kozlov@**oracle.com<vladimir.kozlov at oracle.com>>>
>> wrote:
>>
>>    I need to be more clear. We can't get 100% correct profiling result
>>    for counters. First, profiling structure (MethodData) are created
>>    only after method is executed for some time (for Server compiler it
>>    is 3000 invocations). It is done to avoid profiling short programs
>>    since profiling will slowdown Interpreter by about 20%. This delayed
>>    profiling introduce first discrepancy between counters. MD could be
>>    also created when method has a hot loop (a lot of iterations) and
>>    OSR compilation is requested. As result counters in the loop will
>>    not correlate with counters before the loop. An other reason for not
>>    precise profiling counters is execution of method by multiple
>>    threads. We have only one MD structure per method which is updated
>>    (not atomically for speed) by all such threads. And when method is
>>    prepared for compilation compiler thread creates snapshot of MD
>>    which could happened in the middle of method execution by java threads.
>>
>>    We don't reset MD during deoptimization because we still need
>>    previous values (in addition to counters we collect type
>>    information). So counters are accumulated. And profiling is resumed
>>    from deoptimization point which again adds discrepancy.
>>
>>    I hope it explains why we can't trust 100% to profiling counters.
>>
>>    Regards,
>>    Vladimir
>>
>>
>>    On 1/16/12 4:15 PM, Vitaly Davidovich wrote:
>>
>>        Vladimir, thanks for the explanation and the code pointer.
>>
>>        Intuitively, it would seem like a good idea to trust the profile
>>        100% if it reports the same value used 100% of the time
>>        (I can see how anything less than 100%, even a very high
>>        probability of same value, is not trustworthy) given sufficient
>>        trips through the loop.  Although I can see how an app may have
>>        phases where same value is seen for a while before it's
>>        switched, but that's where I thought deopt would help.  There
>>        must be a good chunk of code out there that doesn't know
>>        at static compilation time the loop count (so can't use
>>        compile-time constant), but at runtime the actual value doesn't
>>        change for many many trips through the loop; I know I have code
>>        like that in various places.
>>
>>        What's the reason a compilation after deopt would not be as
>>        aggressive as the 1st time? Is it because the profile
>>        information may be "weaker" (i.e. more uncertainty in it)? I
>>        thought the profile is completely reset after deopt, so I
>>        would think if the loop is now executed with a different
>>        "constant" value (e.g. in our example, instead of 3 it's now
>>        4), then the same optimizations will be applied (of course if
>>        unrolling the loop is no longer advantageous due to a much
>>        different value, I can see how different optimizations will be
>>        applied).
>>
>>        Thanks
>>
>>        On Mon, Jan 16, 2012 at 6:45 PM, Vladimir Kozlov
>>        <vladimir.kozlov at oracle.com <mailto:vladimir.kozlov@**oracle.com<vladimir.kozlov at oracle.com>
>> >
>>        <mailto:vladimir.kozlov at __orac**le.com <http://oracle.com>
>>        <mailto:vladimir.kozlov@**oracle.com <vladimir.kozlov at oracle.com>>>>
>> wrote:
>>
>>           Vitaly,
>>
>>           We do use profile_trip_cnt during loop unroll calculation but
>>        not during fully unroll because we can't trust it 100%
>>           since program's phase and number of iterations could change
>>        after method is compiled. See policy_unroll() and
>>           policy_maximally_unroll():
>>
>>                  http://hg.openjdk.java.net/___**
>> _hsx/hotspot-comp/hotspot/**file/____89d0a5d40008/src/**
>> share/vm/____opto/**loopTransform.cpp<http://hg.openjdk.java.net/____hsx/hotspot-comp/hotspot/file/____89d0a5d40008/src/share/vm/____opto/loopTransform.cpp>
>>        <http://hg.openjdk.java.net/__**hsx/hotspot-comp/hotspot/file/**
>> __89d0a5d40008/src/share/vm/__**opto/loopTransform.cpp<http://hg.openjdk.java.net/__hsx/hotspot-comp/hotspot/file/__89d0a5d40008/src/share/vm/__opto/loopTransform.cpp>
>> >
>>
>>                  <http://hg.openjdk.java.net/__**
>> hsx/hotspot-comp/hotspot/file/**__89d0a5d40008/src/share/vm/__**
>> opto/loopTransform.cpp<http://hg.openjdk.java.net/__hsx/hotspot-comp/hotspot/file/__89d0a5d40008/src/share/vm/__opto/loopTransform.cpp>
>>        <http://hg.openjdk.java.net/**hsx/hotspot-comp/hotspot/file/**
>> 89d0a5d40008/src/share/vm/**opto/loopTransform.cpp<http://hg.openjdk.java.net/hsx/hotspot-comp/hotspot/file/89d0a5d40008/src/share/vm/opto/loopTransform.cpp>
>> >>
>>
>>           We could use deopt as you suggested but deoptimization is
>>        double-edge sword, when method recompiled after
>>           deoptimization some aggressive optimizations will not be
>>        executed for it so the new generated code could be slower.
>>
>>           Regards,
>>           Vladimir
>>
>>
>>           On 1/16/12 2:37 PM, Vitaly Davidovich wrote:
>>
>>               Hi Vladimir,
>>
>>               If x_col is always seen to be same value in the profile
>>        shouldn't the loop be unrolled as well with some deopt
>>               guard? Or
>>               does this not participate in profiling?
>>
>>               Thanks
>>
>>               On Jan 16, 2012 4:57 PM, "Vladimir Kozlov"
>>        <vladimir.kozlov at oracle.com <mailto:vladimir.kozlov@**oracle.com<vladimir.kozlov at oracle.com>
>> >
>>        <mailto:vladimir.kozlov at __orac**le.com <http://oracle.com>
>>        <mailto:vladimir.kozlov@**oracle.com <vladimir.kozlov at oracle.com>
>> >>
>>               <mailto:vladimir.kozlov@
>>        <mailto:vladimir.kozlov@>__ora**c__le.com <http://orac__le.com> <
>> http://oracle.com>
>>        <mailto:vladimir.kozlov at __orac**le.com <http://oracle.com>
>>        <mailto:vladimir.kozlov@**oracle.com <vladimir.kozlov at oracle.com>>>>>
>> wrote:
>>
>>                > be different, because the expressions are similar. The
>>        difference in runtime (due to cols being a compile-time
>>                > constant) will be visible elsewhere. Is that right? If
>>        so, where would I be able to detect this?
>>
>>                   In such situations we usually use some visual tools
>>        to see difference between log outputs. At least you can use
>>               'diff'. You may need to replace instructions addresses in
>>        outputs (number at the beginning of lines) with the same
>>                   value to match. There are few tricks you may use to
>>        get similar PrintOptoAssembly output. Use next flags to
>>               avoid
>>                   mixing output from program output and from 2 compiler
>>        threads (flags stop program until a method is compiled
>>               and run
>>                   only one compiler thread):
>>
>>                   -Xbatch -XX:CICompilerCount=1
>>
>>                   Also add -XX:+PrintCompilation -XX:+PrintInlining to
>>        see what method is compiled and inlined. Note that you
>>               may see
>>                   similar output for individual methods but could be
>>        big difference in compiled caller (computeAll()) method
>>               where 2
>>                   loop methods could be inlined. So you need to compare
>>        all compiled methods.
>>
>>                   In general, to have constant as loop limit is always
>>        win because some checks in generated code could be
>>               avoided and
>>                   more optimizations could be done for such loops. Use
>>        -XX:+TraceLoopOpts to see what loop optimizations are
>>               done in
>>                   both cases.
>>
>>                   For example, in your code you set 'x_col = 3', as
>>        result the next loop in
>>               constructNearestClusterVector(**______) will be
>>
>>
>>                   fully unrolled when this method is inlined into
>>        computeAll() and x_col is replaced with '3':
>>
>>                           for(k = 0; k < x_col; k++) {
>>                             double tmp = x[i*x_col + k] - mu[j* mu_col
>>        + k];
>>                             dist += tmp * tmp;
>>                           }
>>
>>                   Vladimir
>>
>>                   On 1/16/12 1:39 AM, Manohar Jonnalagedda wrote:
>>
>>                       Hi Kris, Vladimir,
>>
>>                       thanks for both your responses.
>>
>>                           Second, your two test methods are different
>>        so you can't directly compare them. method1() iterates
>>               over rows
>>                       using
>>                           middle loop index 'j' and method2() uses
>>        external loop index 'i'. Unless they are typos again.
>>
>>
>>                       You are right, these are indeed typos. As Kris
>>        suggested, I have the code printed here:
>>               http://pastebin.com/xRFD1Nt1.
>>                       The methods corresponding to method1, and method2
>>        are constructNearestClusterVector and
>>               computeNewCentroids. Their
>>                       PrintOptoAssembly outputs are respectively at
>>        http://pastebin.com/1evN8b3K and http://pastebin.com/FxkVWTD5
>>
>>                       Also, it seems I have not explained myself
>>        correctly. I am not trying to compare the performance of
>>               method1 with
>>                       respect
>>                       to that of method2: method1 and method2 both run
>>        in the same program. What I am trying to compare is their
>>                       performance
>>                       in two cases:
>>                       - when cols is a compile-time constant (much faster)
>>                       - when cols is a value determined at run-time
>>
>>                           If you are using jdk7 there are few flags you
>>        can use to print loop optimizations information. They
>>               need debug
>>                           version of VM but it is not problem for you,
>>        I think, since you can use debug PrintOptoAssembly flag.
>>
>>                           -XX:+TraceLoopOpts prints all happened loop
>>        optimizations and loop tree after each round of loop opts,
>>                           -XX:+TraceLoopPredicate prints RC information
>>        when it is moved from a loop,
>>                           -XX:+TraceRangeLimitCheck prints additional
>>        information for RC elimination optimization.
>>
>>
>>                       Thanks for these, I will have a look at what they
>>        output.
>>
>>                           Fourth, range check expression in your
>>        example is not what you think. RC expression should be next:
>>                           (i*stride+offset) where 'i' is loop variable,
>>        'stride' is constant and 'offset' is loop invariant.
>>
>>                           In your example 'offset' is (j * cols) since
>>        it is loop invariant, 'k' is loop variable and stride
>>               is '1' (one).
>>                           In both your methods RC will be moved out of
>>        inner loop so the code for it will be the same. The only
>>                       difference in
>>                           these methods will be where and how (j *
>>        cols) and (i * cols) expressions are calculated.
>>
>>
>>                               I'd guess it's the difference in locality
>>        that made the difference in performance in your two tests.
>>
>>                         Thanks for the explanation. I understand from
>>        the above that the assembly output in both cases
>>               mentioned above
>>                       may not
>>                       be different, because the expressions are
>>        similar. The difference in runtime (due to cols being a
>>               compile-time
>>                       constant)
>>                       will be visible elsewhere. Is that right? If so,
>>        where would I be able to detect this?
>>
>>                       Cheers,
>>                       Manohar
>>
>>                               In your PrintOptoAssembly output snippet,
>>        the instruction at 0x13e is a LoadRange, which loads
>>               the range
>>                       from
>>                               the header
>>                               of an array:
>>
>>                               (from x86_64.ad <http://x86_64.ad>
>>        <http://x86_64.ad> <http://x86_64.ad> <http://x86_64.ad>
>>        <http://x86_64.ad>)
>>
>>
>>
>>                               // Load Range
>>                               instruct loadRange(rRegI dst, memory mem)
>>                               %{
>>                                  match(Set dst (LoadRange mem));
>>
>>                                  ins_cost(125); // XXX
>>                                  format %{ "movl    $dst, $mem\t# range"
>> %}
>>                                  opcode(0x8B);
>>                                  ins_encode(REX_reg_mem(dst, mem),
>>        OpcP, reg_mem(dst, mem));
>>                                  ins_pipe(ialu_reg_mem);
>>                               %}
>>
>>                               That's not a range check just yet; the
>>        real check, if any, should come after the null check, in
>>               the form of
>>                               comparing
>>                               something else with RSI. But you didn't
>>        show what's after the null check, how RSI is used, so
>>               it's hard
>>                       to say what
>>                               you're seeing in your example.
>>
>>                               As for the two test examples, could you
>>        paste the entire source code, with the PrintOptoAssembly
>>               output of
>>                               method1() and
>>                               method2() ? The first example looks
>>        weird, maybe it's a typo but you're using "j < cols" as the loop
>>                       condition
>>                               for the
>>                               inner loop.
>>
>>
>>                               - Kris
>>
>>                               On Mon, Jan 16, 2012 at 1:59 AM, Manohar
>>        Jonnalagedda <manojo10386 at gmail.com <mailto:manojo10386 at gmail.com>
>>               <mailto:manojo10386 at gmail.com <mailto:manojo10386 at gmail.com
>> >**>
>>               <mailto:manojo10386 at gmail.com
>>        <mailto:manojo10386 at gmail.com> <mailto:manojo10386 at gmail.com
>>        <mailto:manojo10386 at gmail.com>**>__> <mailto:manojo10386 at gmail.com
>>        <mailto:manojo10386 at gmail.com>
>>               <mailto:manojo10386 at gmail.com
>>        <mailto:manojo10386 at gmail.com>**> <mailto:manojo10386 at gmail.com
>>        <mailto:manojo10386 at gmail.com> <mailto:manojo10386 at gmail.com
>>        <mailto:manojo10386 at gmail.com>**>__>__>
>>               <mailto:manojo10386 at gmail.com
>>        <mailto:manojo10386 at gmail.com> <mailto:manojo10386 at gmail.com
>>        <mailto:manojo10386 at gmail.com>**> <mailto:manojo10386 at gmail.com
>>        <mailto:manojo10386 at gmail.com>
>>               <mailto:manojo10386 at gmail.com
>>        <mailto:manojo10386 at gmail.com>**>__> <mailto:manojo10386 at gmail.com
>>        <mailto:manojo10386 at gmail.com> <mailto:manojo10386 at gmail.com
>>        <mailto:manojo10386 at gmail.com>**>
>>               <mailto:manojo10386 at gmail.com
>>        <mailto:manojo10386 at gmail.com> <mailto:manojo10386 at gmail.com
>>        <mailto:manojo10386 at gmail.com>**>__>__>__>> wrote:
>>
>>                                   Hello,
>>
>>                                   following this reference on Range
>>        Check Elimination done by the Hotspot compiler [1], I was
>>               keen in
>>                       knowing
>>                               how I
>>                                   can detect whether range checks are
>>        taking place in loops by inspecting output using the
>>                       PrintAssembly flag;
>>                               with
>>                                   the old PrintOptoAssembly flag, I
>>        have seen output such as the following, which I assume to
>>               be range
>>                       checks :
>>
>>                                   B11: #  B73 B12 &lt;- B10  Freq: 1.21365
>>                                   139     movq    RAX, [rsp + #24]  #
>> spill
>>                                   13e     movl    RSI, [RAX + #12
>>        (8-bit)]  # range
>>                                   141     NullCheck RAX
>>
>>                                   What is the equivalent with the new
>>        PrintAssembly flag (using hsdis)?
>>
>>                                   Moreover, as stated on the wiki page
>>        [1], loops are optimized if the stride is a compile-time
>>                       constant. I
>>                               performed
>>                                   a few tests on a kmeans program, with
>>        3 nested loops, having the following (high-level)
>>               structure:
>>
>>                                   ===
>>                                   void method1(){
>>                                   //loop 1
>>                                   for(int i = 0; i< rows1; i++){
>>                                     //...
>>                                      for(int j = 0; j< rows2; j++){
>>                                      //...
>>                                       for(int k = 0; j < cols; k++){
>>        array[j * cols + k] = //...}
>>                                      }
>>                                   }
>>                                   }
>>
>>                                   void method2(){
>>                                   //loop 2
>>                                   for(int i =0; i < rows1; i++){
>>                                      for(int j=0 ; i< rows2; j++){
>>                                        for(int k=0 ; k< cols; k++){
>>                                          array[i*cols+k] = //...
>>                                        }
>>                                      }
>>                                   }
>>                                   }
>>
>>                                   void main(){
>>
>>                                     do{
>>                                       method1(); method2();
>>                                     }while(!converged)
>>
>>                                   }
>>                                   ====
>>
>>                                   In the first test, cols is an int
>>        whose value is determined at runtime (by reading a file),
>>               in the
>>                       second
>>                               test, it
>>                                   is given as a compile-time
>>        constant(3). In the second test, there is a */significant*/
>> speed-up
>>                       (around 40%).
>>
>>                                   However, when studying the diff of
>>        the output of PrintOptoAssembly for both method1 and method2,
>>                       there is no
>>                                   difference (apart from slight value
>>        changes in frequency). Would you have any hints as to
>>               where I
>>                       could look for
>>                                   differences?
>>
>>                                   Thanks a lot,
>>                                   Manohar
>>
>>                                   [1]
>>        https://wikis.oracle.com/_____**_display/HotSpotInternals/____**
>> __RangeCheckElimination<https://wikis.oracle.com/______display/HotSpotInternals/______RangeCheckElimination>
>>        <https://wikis.oracle.com/____**display/HotSpotInternals/____**
>> RangeCheckElimination<https://wikis.oracle.com/____display/HotSpotInternals/____RangeCheckElimination>
>> >
>>                      <https://wikis.oracle.com/____**
>> display/HotSpotInternals/____**RangeCheckElimination<https://wikis.oracle.com/____display/HotSpotInternals/____RangeCheckElimination>
>>        <https://wikis.oracle.com/__**display/HotSpotInternals/__**
>> RangeCheckElimination<https://wikis.oracle.com/__display/HotSpotInternals/__RangeCheckElimination>
>> >>
>>
>>
>>                      <https://wikis.oracle.com/____**
>> display/HotSpotInternals/____**RangeCheckElimination<https://wikis.oracle.com/____display/HotSpotInternals/____RangeCheckElimination>
>>        <https://wikis.oracle.com/__**display/HotSpotInternals/__**
>> RangeCheckElimination<https://wikis.oracle.com/__display/HotSpotInternals/__RangeCheckElimination>
>> >
>>                      <https://wikis.oracle.com/__**
>> display/HotSpotInternals/__**RangeCheckElimination<https://wikis.oracle.com/__display/HotSpotInternals/__RangeCheckElimination>
>>        <https://wikis.oracle.com/**display/HotSpotInternals/**
>> RangeCheckElimination<https://wikis.oracle.com/display/HotSpotInternals/RangeCheckElimination>
>> >>>
>>
>>
>>
>>                       As Kris pointed you need to fix your example:
>>
>>
>>
>>
>>        --
>>        Vitaly
>>        617-548-7007 <tel:617-548-7007> (mobile)
>>
>>
>>
>>
>> --
>> Vitaly
>> 617-548-7007 (mobile)
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120117/033303e0/attachment-0001.html 

From kelly.ohair at oracle.com  Tue Jan 17 11:44:39 2012
From: kelly.ohair at oracle.com (Kelly O'Hair)
Date: Tue, 17 Jan 2012 11:44:39 -0800
Subject: RFR (XS): 7126732: MAC: Require Mac OS X builds/tests for JPRT
	integrate jobs for HotSpot
In-Reply-To: <4F11B537.2000504@oracle.com>
References: <4F11B537.2000504@oracle.com>
Message-ID: <4814BBC3-95A0-49D0-B3F0-157B0525E898@oracle.com>


It seems to me that with this change, only jdk7 or jdk8 builds will work properly.
Which is fine with me, but then it argues that all these lines that mention:
 jdk7b107
 jdk7temp
 jdk6
 jdk6perf
 jdk6u10
 jdk6u14
 jdk6u18
 jdk6u20
 ejdk7
 ejdk6
should all just be deleted. I doubt that a 'jprt submit -release XXX' where XXX is any of the
above releases will even work.

-kto

On Jan 14, 2012, at 9:02 AM, James Melvin wrote:

> Greetings,
> 
> We're ready to require HotSpot builds on Mac OS X for JPRT integrate
> jobs. There are 3 mac-minis in each queue. Build/Test times are short
> relative to other platforms. Uses the stable Linux testlist for now.
> 
> http://cr.openjdk.java.net/~jmelvin/7126732/webrev.00
> 
> Tested with *several* JPRT submissions for other bugfixes. I'd like to
> integrate this change right after the current snapshot window.
> 
> Feedback welcome.
> 
> Thanks,
> 
> Jim


From dalibor.topic at oracle.com  Tue Jan 17 16:11:55 2012
From: dalibor.topic at oracle.com (Dalibor Topic)
Date: Wed, 18 Jan 2012 01:11:55 +0100
Subject: Fwd: OpenJDK wikis have moved
In-Reply-To: <4F160D49.3090107@oracle.com>
References: <4F160D49.3090107@oracle.com>
Message-ID: <4F160E4B.40409@oracle.com>

-------- Original Message --------
Subject: OpenJDK wikis have moved
Date: Wed, 18 Jan 2012 01:07:37 +0100
From: Dalibor Topic <dalibor.topic at oracle.com>
To: web-discuss at openjdk.java.net

See http://robilad.livejournal.com/111187.html .

cheers,
dalibor topic

-- 
Oracle <http://www.oracle.com>
Dalibor Topic | Java F/OSS Ambassador
Phone: +494023646738 <tel:+494023646738> | Mobile: +491772664192 <tel:+491772664192>
Oracle Java Platform Group

ORACLE Deutschland B.V. & Co. KG | Nagelsweg 55 | 20097 Hamburg

ORACLE Deutschland B.V. & Co. KG
Hauptverwaltung: Riesstr. 25, D-80992 M?nchen
Registergericht: Amtsgericht M?nchen, HRA 95603
Gesch?ftsf?hrer: J?rgen Kunz

Komplement?rin: ORACLE Deutschland Verwaltung B.V.
Hertogswetering 163/167, 3543 AS Utrecht, Niederlande
Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697
Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher

Green Oracle <http://www.oracle.com/commitment> Oracle is committed to developing practices and products that help protect the environment

From david.holmes at oracle.com  Tue Jan 17 19:06:55 2012
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 18 Jan 2012 13:06:55 +1000
Subject: RFR (XS): 7126732: MAC: Require Mac OS X builds/tests for JPRT
	integrate jobs for HotSpot
In-Reply-To: <4814BBC3-95A0-49D0-B3F0-157B0525E898@oracle.com>
References: <4F11B537.2000504@oracle.com>
	<4814BBC3-95A0-49D0-B3F0-157B0525E898@oracle.com>
Message-ID: <4F16374F.7050709@oracle.com>

On 18/01/2012 5:44 AM, Kelly O'Hair wrote:
> It seems to me that with this change, only jdk7 or jdk8 builds will work properly.
> Which is fine with me, but then it argues that all these lines that mention:
>   jdk7b107
>   jdk7temp
>   jdk6
>   jdk6perf
>   jdk6u10
>   jdk6u14
>   jdk6u18
>   jdk6u20
>   ejdk7
>   ejdk6
> should all just be deleted. I doubt that a 'jprt submit -release XXX' where XXX is any of the
> above releases will even work.

Oops! Yes I missed that too. When we added the embedded targets we had 
to split things into two groups (standard and all) and only include the 
embedded targets in "all". Then we only used "all" for releases that 
supported embedded.

This argues for removing macosx from the set of standard targets, and 
adding it to the set of all targets via another grouping.

David
-----

> -kto
>
> On Jan 14, 2012, at 9:02 AM, James Melvin wrote:
>
>> Greetings,
>>
>> We're ready to require HotSpot builds on Mac OS X for JPRT integrate
>> jobs. There are 3 mac-minis in each queue. Build/Test times are short
>> relative to other platforms. Uses the stable Linux testlist for now.
>>
>> http://cr.openjdk.java.net/~jmelvin/7126732/webrev.00
>>
>> Tested with *several* JPRT submissions for other bugfixes. I'd like to
>> integrate this change right after the current snapshot window.
>>
>> Feedback welcome.
>>
>> Thanks,
>>
>> Jim
>

From david.holmes at oracle.com  Tue Jan 17 19:36:55 2012
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 18 Jan 2012 13:36:55 +1000
Subject: RFR (S): 7130391: Add a framework for vendor-specific validation
	control of setting command-line switches in Hotspot
In-Reply-To: <4F156496.3060206@oracle.com>
References: <5a181ad1-ef64-4b00-88ce-69829f2ce80b@default>
	<4F14E661.5080405@oracle.com> <4F156496.3060206@oracle.com>
Message-ID: <4F163E57.5010401@oracle.com>

Hi Robert,

Comments inline ...

On 17/01/2012 10:07 PM, Robert Ottenhag wrote:
>> On 17/01/2012 12:04 PM, Robert Ottenhag wrote:
>>> Webrev: http://cr.openjdk.java.net/~rottenha/7130391/webrev.00
>>>
>>> This fix adds optional validation control to the setting of
>>> command-line switches in Hotspot, and allows it to have
>>> vendor-specific extensions if necessary.
>>
>> Does this imply that the Java management APIs (eg
>> com.sun.management.VMOption) need to be changed to reflect these
>> restrictions? Presently VMOptions are either writeable or not, but
>> this makes them conditionally-writeable.
>
> No, since the Java management APIs already cares for conditional writes.
> According to com.sun.management.HotSpotDiagnosticMXBean.setVMOption() it
> will throw IllegalArgumentException if the new value is invalid.

I think there is a significant difference between trying to set an 
invalid value and setting to a valid value at a time that it is not 
permitted.

<snip>

>> src/share/vm/runtime/globals_ext.hpp
>>
>> With all the
>>
>> inline bool Flag::is_valid_ext_T(T value, FlagValueOrigin origin)
>>
>> functions, is it necessary to include the type T in the function name?
>
> It is necessary if using type safe variants with T value as argument
> since overloading does not differ between different typedef names that
> resolves to the same native types, e.g. uintx and uint64_t are both
> unsigned long int.
>
> I am considering a condensed variant that replaces T by void* instead,
> and do the type casting based on the targeted flag, reducing the number
> of functions.

Ok. I thought there might be some conflict with the integral types.

>> src/share/vm/runtime/globals.cpp
>>
>> The use of the guarantees seems wrong as it means an invalid option
>> will trigger a VM crash rather than a clean exit during
>> initialization. It seems to me that none of the code in arguments.cpp
>> that uses the FLAG_SET_* macros (which in turn use the
>> CommandLineFlagsEx functions you added the guarantees to) anticipates
>> any possibility for failure. I think if you are going this path then
>> you have no choice but to change the CommandLineFlagsEx methods to
>> return bool and update the FLAG_SET macros to try and perform
>> appropriate error handling.
>
> I see your point, and in theory such as VM crash could occur anytime
> later in a JVM session if rarely running code would make use of
> FLAG_SET_* to change the value of a VM flag to an invalid value or origin.
>
> Seems as if the options are either to
> a) ignore validation tests for the FLAG_SET_* macros, and trust that
> they always set valid values. This can be partly verified by static code
> inspection by looking for any variables that actually have validation
> logic associated to them (since the variable name is defined at compile
> time), assuming one has access to all code, but it is not perfect in
> case code for changing a variable with validation logic exists.
> b) contain the error handling within the FLAG_SET_* macros, like using
> guarantee(), but maybe exception logic can help?
> c) require usage of the FLAG_SET_* macros to handle result codes and
> pass it up the call chain.
>
> Also, the current macro FLAG_SET_DEFAULT does a direct write to the flag
> value without going through <type>AtPut(). This macro must be rewritten
> to have validation control to close the holes. The current call format
> will require all call sites to include type name as with
> FLAG_SET_{CMDLINE,ERGO} has, or to use slower lookup by variable name.

I think you touched on the real problem in your later email - really 
these flags/options and the ways you can interact with them should be 
encapsulated in objects. Each different flag can then define its valid 
values, whether it is "locked", "writeable" etc. But that means every 
use of those flags in the VM would need changing - which is indeed a 
very intrusive change.

But I can't help but feel that we are going to far in what we are trying 
to do with these flags when they are in fact simple variables.

Also I think we may be overcomplicating this. I don't see why we can't 
consider the uses of the flags at initialization time and runtime to be 
distinct use-cases and use different APIs to interact with them. For 
initialization we have the FLAGS_SET_* macros, and the end result is 
that we have a set of flags that are either at their default values or 
have been set to a valid value. I don't think we need to consider (as I 
believe the current proposal does) multiple settings of a given flag at 
initialization time ie:

java -XX:+UseFoo ... -XX:-UseFoo ... -XX:+UseFoo

should simply result in UseFoo==true. Even if we have stated that once 
UseFoo is turned on it can't be turned off again. To me that should only 
relate to true "dynamic" runtime setting of the flags. In which case 
only the management APIs need to be augmented to support this and we may 
be able to create "shadow" objects for flags we need to handle specially 
at runtime.

David
-----

> /Robert
>
>>
>> David
>> -----
>>
>>
>>> A simple use case for validation is a manageable flag whose current
>>> value can not be less than the previous value, while a more complex
>>> example may base the validation on multiple other flags, etc.
>>>
>>> Thanks,
>>>
>>> /Robert
>>>
>
>

From david.holmes at oracle.com  Tue Jan 17 21:21:09 2012
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 18 Jan 2012 15:21:09 +1000
Subject: RFR (XS): 7126732: MAC: Require Mac OS X builds/tests for JPRT
	integrate jobs for HotSpot
In-Reply-To: <4F16374F.7050709@oracle.com>
References: <4F11B537.2000504@oracle.com>
	<4814BBC3-95A0-49D0-B3F0-157B0525E898@oracle.com>
	<4F16374F.7050709@oracle.com>
Message-ID: <4F1656C5.8070308@oracle.com>

On 18/01/2012 1:06 PM, David Holmes wrote:
> On 18/01/2012 5:44 AM, Kelly O'Hair wrote:
>> It seems to me that with this change, only jdk7 or jdk8 builds will
>> work properly.
>> Which is fine with me, but then it argues that all these lines that
>> mention:
>> jdk7b107
>> jdk7temp
>> jdk6
>> jdk6perf
>> jdk6u10
>> jdk6u14
>> jdk6u18
>> jdk6u20
>> ejdk7
>> ejdk6
>> should all just be deleted. I doubt that a 'jprt submit -release XXX'
>> where XXX is any of the
>> above releases will even work.
>
> Oops! Yes I missed that too. When we added the embedded targets we had
> to split things into two groups (standard and all) and only include the
> embedded targets in "all". Then we only used "all" for releases that
> supported embedded.
>
> This argues for removing macosx from the set of standard targets, and
> adding it to the set of all targets via another grouping.

On further reflection it argues for cleaning up the obsolete release 
definitions. Is there any reason someone with the latest hotspot 
workspace would be trying to do a JDK6 build in JPRT? Doesn't seem 
reasonable to me given we are not using latest hotspot in JDK6. And the 
two jdk7 releases mentioned were only temporary fixes for something or 
other anyway.

The ejdk releases are obsolete as well.

David

> David
> -----
>
>> -kto
>>
>> On Jan 14, 2012, at 9:02 AM, James Melvin wrote:
>>
>>> Greetings,
>>>
>>> We're ready to require HotSpot builds on Mac OS X for JPRT integrate
>>> jobs. There are 3 mac-minis in each queue. Build/Test times are short
>>> relative to other platforms. Uses the stable Linux testlist for now.
>>>
>>> http://cr.openjdk.java.net/~jmelvin/7126732/webrev.00
>>>
>>> Tested with *several* JPRT submissions for other bugfixes. I'd like to
>>> integrate this change right after the current snapshot window.
>>>
>>> Feedback welcome.
>>>
>>> Thanks,
>>>
>>> Jim
>>

From robert.ottenhag at oracle.com  Wed Jan 18 06:33:13 2012
From: robert.ottenhag at oracle.com (Robert Ottenhag)
Date: Wed, 18 Jan 2012 15:33:13 +0100
Subject: RFR (S): 7130391: Add a framework for vendor-specific validation
	control of setting command-line switches in Hotspot
In-Reply-To: <4F163E57.5010401@oracle.com>
References: <5a181ad1-ef64-4b00-88ce-69829f2ce80b@default>
	<4F14E661.5080405@oracle.com> <4F156496.3060206@oracle.com>
	<4F163E57.5010401@oracle.com>
Message-ID: <4F16D829.7090201@oracle.com>

Hi David,

More comments inline...

On 01/18/2012 04:36 AM, David Holmes wrote:
> Hi Robert,
>
> Comments inline ...
>
> On 17/01/2012 10:07 PM, Robert Ottenhag wrote:
>>> On 17/01/2012 12:04 PM, Robert Ottenhag wrote:
>>>> Webrev: http://cr.openjdk.java.net/~rottenha/7130391/webrev.00
>>>>
>>>> This fix adds optional validation control to the setting of
>>>> command-line switches in Hotspot, and allows it to have
>>>> vendor-specific extensions if necessary.
>>>
>>> Does this imply that the Java management APIs (eg
>>> com.sun.management.VMOption) need to be changed to reflect these
>>> restrictions? Presently VMOptions are either writeable or not, but
>>> this makes them conditionally-writeable.
>>
>> No, since the Java management APIs already cares for conditional writes.
>> According to com.sun.management.HotSpotDiagnosticMXBean.setVMOption() it
>> will throw IllegalArgumentException if the new value is invalid.
>
> I think there is a significant difference between trying to set an 
> invalid value and setting to a valid value at a time that it is not 
> permitted.

Agreed. There is a semantic difference here that can be confusing.

<snip>
>
>>> src/share/vm/runtime/globals.cpp
>>>
>>> The use of the guarantees seems wrong as it means an invalid option
>>> will trigger a VM crash rather than a clean exit during
>>> initialization. It seems to me that none of the code in arguments.cpp
>>> that uses the FLAG_SET_* macros (which in turn use the
>>> CommandLineFlagsEx functions you added the guarantees to) anticipates
>>> any possibility for failure. I think if you are going this path then
>>> you have no choice but to change the CommandLineFlagsEx methods to
>>> return bool and update the FLAG_SET macros to try and perform
>>> appropriate error handling.
>>
>> I see your point, and in theory such as VM crash could occur anytime
>> later in a JVM session if rarely running code would make use of
>> FLAG_SET_* to change the value of a VM flag to an invalid value or 
>> origin.
>>
>> Seems as if the options are either to
>> a) ignore validation tests for the FLAG_SET_* macros, and trust that
>> they always set valid values. This can be partly verified by static code
>> inspection by looking for any variables that actually have validation
>> logic associated to them (since the variable name is defined at compile
>> time), assuming one has access to all code, but it is not perfect in
>> case code for changing a variable with validation logic exists.
>> b) contain the error handling within the FLAG_SET_* macros, like using
>> guarantee(), but maybe exception logic can help?
>> c) require usage of the FLAG_SET_* macros to handle result codes and
>> pass it up the call chain.
>>
>> Also, the current macro FLAG_SET_DEFAULT does a direct write to the flag
>> value without going through <type>AtPut(). This macro must be rewritten
>> to have validation control to close the holes. The current call format
>> will require all call sites to include type name as with
>> FLAG_SET_{CMDLINE,ERGO} has, or to use slower lookup by variable name.
>
> I think you touched on the real problem in your later email - really 
> these flags/options and the ways you can interact with them should be 
> encapsulated in objects. Each different flag can then define its valid 
> values, whether it is "locked", "writeable" etc. But that means every 
> use of those flags in the VM would need changing - which is indeed a 
> very intrusive change.

We might be able to keep existing usage by encapsulating it within a 
class that overrides the assignment and type conversion operators, but 
as you say it is a little more work than expected right now.

>
> But I can't help but feel that we are going to far in what we are 
> trying to do with these flags when they are in fact simple variables.
>
> Also I think we may be overcomplicating this. I don't see why we can't 
> consider the uses of the flags at initialization time and runtime to 
> be distinct use-cases and use different APIs to interact with them. 
> For initialization we have the FLAGS_SET_* macros, and the end result 
> is that we have a set of flags that are either at their default values 
> or have been set to a valid value. I don't think we need to consider 
> (as I believe the current proposal does) multiple settings of a given 
> flag at initialization time ie:
>
> java -XX:+UseFoo ... -XX:-UseFoo ... -XX:+UseFoo
>
> should simply result in UseFoo==true. Even if we have stated that once 
> UseFoo is turned on it can't be turned off again. To me that should 
> only relate to true "dynamic" runtime setting of the flags. In which 
> case only the management APIs need to be augmented to support this and 
> we may be able to create "shadow" objects for flags we need to handle 
> specially at runtime.

The problem with the FLAG_SET_* macros is that they can also be used 
after initialization, or for that matter direct variable assignment can 
be also done, to bypass any validation logic that is observed by the 
dynamic interfaces.

However, at this point I am also leaning towards a design that only 
focuses on the dynamic setting, which will not change existing behavior 
of command line flags parsing, i.e. any variable is writable with any 
value during the initialization phase.

>
> David
> -----
>
>> /Robert
>>
>>>
>>> David
>>> -----
>>>
>>>
>>>> A simple use case for validation is a manageable flag whose current
>>>> value can not be less than the previous value, while a more complex
>>>> example may base the validation on multiple other flags, etc.
>>>>
>>>> Thanks,
>>>>
>>>> /Robert
>>>>
>>
>>


-- 
Oracle
Robert Ottenhag | Senior Member of Technical Staff
Phone: +46850630961 | Fax: +46850630911 | Mobile: +46707106161
Oracle Java HotSpot Virtual Machine
ORACLE Sweden | Folkungagatan 122 | SE-116 30 Stockholm

Oracle Svenska AB, Kronborgsgr?nd 17, S-164 28 KISTA, reg.no. 556254-6746

Green Oracle

Oracle is committed to developing practices and products that help protect the environment
--


From robert.ottenhag at oracle.com  Wed Jan 18 07:00:30 2012
From: robert.ottenhag at oracle.com (Robert Ottenhag)
Date: Wed, 18 Jan 2012 16:00:30 +0100
Subject: RFR (S): 7130391: Add a framework for vendor-specific validation
	control of setting command-line switches in Hotspot
In-Reply-To: <4F157A91.8080907@oracle.com>
References: <5a181ad1-ef64-4b00-88ce-69829f2ce80b@default>
	<4F14E661.5080405@oracle.com> <4F156496.3060206@oracle.com>
	<4F157A91.8080907@oracle.com>
Message-ID: <4F16DE8E.1090609@oracle.com>

Updated webrev: http://cr.openjdk.java.net/~rottenha/7130391/webrev.01

Changes to the previous version are:
   * src/share/vm/runtime/globals.cpp:
Remove validation control from <type>AtPut(CommandLineFlagsWithType, 
...), that is only used by FLAG_SET_* macros in globals_extension.hpp.

   * src/share/vm/runtime/{globals.hpp, globals.cpp, globals_ext.hpp}:
Replace multiple public type safe functions 
Flag::is_valid[_ext]_<type>(<type> value, ...) by single protected type 
generic functions CommandLineFlags::is_valid[_ext (const Flag*, const 
void*, ...),  then do internal type casts on the values based on the 
type of the targeted flag (and assert on type correctness).

   * src/share/vm/services/management.cpp:
Use a better error message (David Holmes).

/Robert


On 01/17/2012 02:41 PM, Robert Ottenhag wrote:
> David,
>
> Regarding the FLAG_SET_* macros, I am thinking that we can leave them 
> to a follow up bug instead.
>
> The reason is that it can be verified by code inspection (of 
> preprocessed sources) if any FLAG_SET_* macro writes to a variable 
> known to have validation control.
>
> Also, fixing that hole would require any access to the variables to 
> occur through interface get/set functions, preventing direct read and 
> write access (wrapping the variable in a class to prevent direct 
> writes), a change too intrusive for now.
>
> Will come back with an updated and cleaned up patch.
>
> /Robert
>
> On 01/17/2012 01:07 PM, Robert Ottenhag wrote:
>> David,
>>
>> Thanks for the review.
>>
>> On 01/17/2012 04:09 AM, David Holmes wrote:
>>> Hi Robert,
>>>
>>> I've added serviceability to the cc list.
>>
>> Good, will try to remember that ;-)
>>
>>>
>>> On 17/01/2012 12:04 PM, Robert Ottenhag wrote:
>>>> Webrev: http://cr.openjdk.java.net/~rottenha/7130391/webrev.00
>>>>
>>>> This fix adds optional validation control to the setting of 
>>>> command-line switches in Hotspot, and allows it to have 
>>>> vendor-specific extensions if necessary.
>>>
>>> Does this imply that the Java management APIs (eg 
>>> com.sun.management.VMOption) need to be changed to reflect these 
>>> restrictions? Presently VMOptions are either writeable or not, but 
>>> this makes them conditionally-writeable.
>>
>> No, since the Java management  APIs already cares for conditional 
>> writes. According to 
>> com.sun.management.HotSpotDiagnosticMXBean.setVMOption() it will 
>> throw IllegalArgumentException if the new value is invalid.
>>
>>>
>>>> The design follows the previously added framework for 
>>>> vendor-specific command-line switch extensions in CR7117389.
>>>>
>>>> The validation control is handled by new boolean methods 
>>>> Flag::is_valid_<type>(value,origin) that are called at the 
>>>> beginning of each call to CommandLineFlags[Ex]::<type>AtPut() to 
>>>> verify that the new value and origin are valid replacements for the 
>>>> current value and origin for this flag.
>>>>
>>>> When parsing the command line options, a failed validation will 
>>>> typically result in an error message and exit with "Unrecognized VM 
>>>> option '<flag-name>'".  When used dynamically using the attach API 
>>>> or management API the resulting operation will fail, leaving it up 
>>>> to the caller to handle it as appropriate.
>>>
>>> The error message doesn't really seem appropriate - it may well be a 
>>> recognized option, you just can't set it to that value in that way. 
>>> Ideally there would be a way for the validation logic to supply a 
>>> meaningful error message. In its absence the top-level message 
>>> should reflect the new type of error.
>>
>> You are absolutely right, but the current fix is in line with the 
>> existing bad error messages where any kind of malformatted command 
>> line flags results in Unrecognized VM option, whether the reason is 
>> an unknown name, bad type semantics (using +- for bool semantics on 
>> an integer flag), or if the flag is locked.
>>
>> I will target meaningful error messages for command line parsing in a 
>> direct follow up bug to this fix.
>>
>>>
>>> Also some of the failures lead to crashes - which seems wrong to me 
>>> - see below.
>>>
>>> ----
>>>
>>> src/share/vm/services/management.cpp:
>>>
>>> 1821   if (!succeed) {
>>> 1822     THROW_MSG(vmSymbols::java_lang_IllegalArgumentException(),
>>> 1823               "This flag is not writeable with this value or 
>>> origin.");
>>>
>>> That's a rather cryptic error message. How about:
>>>
>>> "Flag can not be set to the requested value using this API"
>>>
>>> ?
>>
>> Yes, "origin" does not make sense to the upper Java layer. I will use 
>> your suggestion.
>>
>>>
>>> ----
>>>
>>> src/share/vm/runtime/globals_ext.hpp
>>>
>>> With all the
>>>
>>> inline bool Flag::is_valid_ext_T(T value, FlagValueOrigin origin)
>>>
>>> functions, is it necessary to include the type T in the function name?
>>
>> It is necessary if using type safe variants with T value as argument 
>> since overloading does not differ between different typedef names 
>> that resolves to the same native types, e.g. uintx and uint64_t are 
>> both unsigned long int.
>>
>> I am considering a condensed variant that replaces T by void* 
>> instead, and do the type casting based on the targeted flag, reducing 
>> the number of functions.
>>
>>>
>>>
>>> -----
>>>
>>> src/share/vm/runtime/globals.cpp
>>>
>>> The use of the guarantees seems wrong as it means an invalid option 
>>> will trigger a VM crash rather than a clean exit during 
>>> initialization. It seems to me that none of the code in 
>>> arguments.cpp that uses the FLAG_SET_* macros (which in turn use the 
>>> CommandLineFlagsEx functions you added the guarantees to) 
>>> anticipates any possibility for failure. I think if you are going 
>>> this path then you have no choice but to change the 
>>> CommandLineFlagsEx methods to return bool and update the FLAG_SET 
>>> macros to try and perform appropriate error handling.
>>
>> I see your point, and in theory such as VM crash could occur anytime 
>> later in a JVM session if rarely running code would make use of 
>> FLAG_SET_* to change the value of a VM flag to an invalid value or 
>> origin.
>>
>> Seems as if the options are either to
>> a) ignore validation tests for the FLAG_SET_* macros, and trust that 
>> they always set valid values. This can be partly verified by static 
>> code inspection by looking for any variables that actually have 
>> validation logic associated to them (since the variable name is 
>> defined at compile time), assuming one has access to all code, but it 
>> is not perfect in case code for changing a variable with validation 
>> logic exists.
>> b) contain the error handling within the FLAG_SET_* macros, like 
>> using guarantee(), but maybe exception logic can help?
>> c) require usage of the FLAG_SET_* macros to handle result codes and 
>> pass it up the call chain.
>>
>> Also, the current macro FLAG_SET_DEFAULT does a direct write to the 
>> flag value without going through <type>AtPut(). This macro must be 
>> rewritten to have validation control to close the holes. The current 
>> call format will require all call sites to include type name as with 
>> FLAG_SET_{CMDLINE,ERGO} has, or to use slower lookup by variable name.
>>
>> /Robert
>>
>>>
>>> David
>>> -----
>>>
>>>
>>>> A simple use case for validation is a manageable flag whose current 
>>>> value can not be less than the previous value, while a more complex 
>>>> example may base the validation on multiple other flags, etc.
>>>>
>>>> Thanks,
>>>>
>>>> /Robert
>>>>
>>
>>
>
>


-- 
Oracle
Robert Ottenhag | Senior Member of Technical Staff
Phone: +46850630961 | Fax: +46850630911 | Mobile: +46707106161
Oracle Java HotSpot Virtual Machine
ORACLE Sweden | Folkungagatan 122 | SE-116 30 Stockholm

Oracle Svenska AB, Kronborgsgr?nd 17, S-164 28 KISTA, reg.no. 556254-6746

Green Oracle

Oracle is committed to developing practices and products that help protect the environment
--


From keith.mcguigan at oracle.com  Wed Jan 18 14:14:24 2012
From: keith.mcguigan at oracle.com (keith.mcguigan at oracle.com)
Date: Wed, 18 Jan 2012 22:14:24 +0000
Subject: hg: hsx/hotspot-main/hotspot: 6 new changesets
Message-ID: <20120118221444.EB9F3479D8@hg.openjdk.java.net>

Changeset: 94ec88ca68e2
Author:    phh
Date:      2012-01-11 17:34 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/94ec88ca68e2

7115199: Add event tracing hooks and Java Flight Recorder infrastructure
Summary: Added a nop tracing infrastructure, JFR makefile changes and other infrastructure used only by JFR.
Reviewed-by: acorn, sspitsyn
Contributed-by: markus.gronlund at oracle.com

! make/Makefile
! make/bsd/makefiles/vm.make
! make/defs.make
! make/linux/makefiles/vm.make
! make/solaris/makefiles/vm.make
! make/windows/build.bat
! make/windows/create_obj_files.sh
! make/windows/makefiles/projectcreator.make
! make/windows/makefiles/vm.make
! src/share/vm/classfile/symbolTable.cpp
! src/share/vm/classfile/symbolTable.hpp
! src/share/vm/classfile/systemDictionary.cpp
! src/share/vm/oops/klass.cpp
! src/share/vm/oops/klass.hpp
! src/share/vm/oops/methodKlass.cpp
! src/share/vm/oops/methodOop.hpp
! src/share/vm/prims/jni.cpp
+ src/share/vm/prims/jniExport.hpp
! src/share/vm/runtime/java.cpp
! src/share/vm/runtime/mutexLocker.cpp
! src/share/vm/runtime/mutexLocker.hpp
! src/share/vm/runtime/os.cpp
! src/share/vm/runtime/thread.cpp
! src/share/vm/runtime/thread.hpp
! src/share/vm/runtime/vm_operations.hpp
+ src/share/vm/trace/traceEventTypes.hpp
+ src/share/vm/trace/traceMacros.hpp
+ src/share/vm/trace/tracing.hpp
! src/share/vm/utilities/globalDefinitions.hpp

Changeset: 4f3ce9284781
Author:    phh
Date:      2012-01-11 17:58 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/4f3ce9284781

Merge

! src/share/vm/oops/klass.cpp
! src/share/vm/oops/klass.hpp

Changeset: f1cd52d6ce02
Author:    kamg
Date:      2012-01-17 10:16 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/f1cd52d6ce02

Merge


Changeset: d7e3846464d0
Author:    zgu
Date:      2012-01-17 13:08 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/d7e3846464d0

7071311: Decoder enhancement
Summary: Made decoder thread-safe
Reviewed-by: coleenp, kamg

- src/os/bsd/vm/decoder_bsd.cpp
+ src/os/bsd/vm/decoder_machO.cpp
+ src/os/bsd/vm/decoder_machO.hpp
! src/os/linux/vm/decoder_linux.cpp
! src/os/linux/vm/os_linux.cpp
! src/os/solaris/vm/decoder_solaris.cpp
! src/os/solaris/vm/os_solaris.cpp
! src/os/windows/vm/decoder_windows.cpp
+ src/os/windows/vm/decoder_windows.hpp
! src/os/windows/vm/os_windows.cpp
! src/share/vm/utilities/decoder.cpp
! src/share/vm/utilities/decoder.hpp
+ src/share/vm/utilities/decoder_elf.cpp
+ src/share/vm/utilities/decoder_elf.hpp
! src/share/vm/utilities/elfFile.cpp
! src/share/vm/utilities/elfFile.hpp
! src/share/vm/utilities/elfStringTable.cpp
! src/share/vm/utilities/elfStringTable.hpp
! src/share/vm/utilities/elfSymbolTable.cpp
! src/share/vm/utilities/elfSymbolTable.hpp
! src/share/vm/utilities/vmError.cpp

Changeset: 6520f9861937
Author:    kamg
Date:      2012-01-17 21:25 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/6520f9861937

Merge


Changeset: db18ca98d237
Author:    zgu
Date:      2012-01-18 11:45 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/db18ca98d237

7131050: fix for "7071311 Decoder enhancement" does not build on MacOS X
Summary: Decoder API changes did not reflect in os_bsd
Reviewed-by: kamg, dcubed

! src/os/bsd/vm/os_bsd.cpp


From manojo10386 at gmail.com  Thu Jan 19 03:18:09 2012
From: manojo10386 at gmail.com (Manohar Jonnalagedda)
Date: Thu, 19 Jan 2012 12:18:09 +0100
Subject: Detecting range check elimination with PrintAssembly
In-Reply-To: <4F149D88.2060305@oracle.com>
References: <CABhSbLyAwFLFwhmTzPE=0DiXK8e8kbmvr3tr-0AQ=KyFz1Mjsw@mail.gmail.com>
	<CA+cQ+tTd_yhyu1=06JSgskU-kJ_qh=cXmnYdE6a7TgCR7s=uxg@mail.gmail.com>
	<4F135374.8020004@oracle.com>
	<CABhSbLy0gd3BFmBRX-aUgEt7h2HBHer_AQENMGWYTRN4p5-wEg@mail.gmail.com>
	<4F149D88.2060305@oracle.com>
Message-ID: <CABhSbLyzsAssScg+6oCDJPfzac5SdyoZMAMAxr6Vv-WjG8amqg@mail.gmail.com>

Hi Vladimir, Vitaly,

In such situations we usually use some visual tools to see difference
> between log outputs.


Would you recommend any? I have had troubles using IdealGraphVisualizer:
either the overhead of using it makes the methods not get JIT compiled, or,
if I decrese the CompileThreshold, the VM crashes with an out of memory
error.


> At least you can use 'diff'. You may need to replace instructions
> addresses in outputs (number at the beginning of lines) with the same value
> to match. There are few tricks you may use to get similar PrintOptoAssembly
> output. Use next flags to avoid mixing output from program output and from
> 2 compiler threads (flags stop program until a method is compiled and run
> only one compiler thread):
>
> -Xbatch -XX:CICompilerCount=1
>
> Also add -XX:+PrintCompilation -XX:+PrintInlining to see what method is
> compiled and inlined. Note that you may see similar output for individual
> methods but could be big difference in compiled caller (computeAll())
> method where 2 loop methods could be inlined. So you need to compare all
> compiled methods.
>

PrintInlining tells me that constructNearestClusterVector is inlined into
ComputeAll, and from what I understand, computeNewCentroids is not:

@ 109   Kmeans::computeNewCentroids (163 bytes)   already compiled into a
big method

I have this message 3 times, do these belong to the Pre-Main-Post of the
loop in the computeAll method?


> In general, to have constant as loop limit is always win because some
> checks in generated code could be avoided and more optimizations could be
> done for such loops. Use -XX:+TraceLoopOpts to see what loop optimizations
> are done in both cases.
>



I am new to this flag: should I use it in conjunction with the
PrintOptoAssembly output for identifying loop numbers with the code they
represent? Would there be any other way to interpret these?


Thanks,
Manohar


>
> For example, in your code you set 'x_col = 3', as result the next loop in
> constructNearestClusterVector() will be fully unrolled when this method is
> inlined into computeAll() and x_col is replaced with '3':
>
>        for(k = 0; k < x_col; k++) {
>          double tmp = x[i*x_col + k] - mu[j* mu_col + k];
>          dist += tmp * tmp;
>        }
>
> Vladimir
>
>
> On 1/16/12 1:39 AM, Manohar Jonnalagedda wrote:
>
>> Hi Kris, Vladimir,
>>
>> thanks for both your responses.
>>
>>    Second, your two test methods are different so you can't directly
>> compare them. method1() iterates over rows using
>>    middle loop index 'j' and method2() uses external loop index 'i'.
>> Unless they are typos again.
>>
>>
>> You are right, these are indeed typos. As Kris suggested, I have the code
>> printed here: http://pastebin.com/xRFD1Nt1.
>> The methods corresponding to method1, and method2 are
>> constructNearestClusterVector and computeNewCentroids. Their
>> PrintOptoAssembly outputs are respectively at
>> http://pastebin.com/1evN8b3K and http://pastebin.com/FxkVWTD5
>>
>> Also, it seems I have not explained myself correctly. I am not trying to
>> compare the performance of method1 with respect
>> to that of method2: method1 and method2 both run in the same program.
>> What I am trying to compare is their performance
>> in two cases:
>> - when cols is a compile-time constant (much faster)
>> - when cols is a value determined at run-time
>>
>>    If you are using jdk7 there are few flags you can use to print loop
>> optimizations information. They need debug
>>    version of VM but it is not problem for you, I think, since you can
>> use debug PrintOptoAssembly flag.
>>
>>    -XX:+TraceLoopOpts prints all happened loop optimizations and loop
>> tree after each round of loop opts,
>>    -XX:+TraceLoopPredicate prints RC information when it is moved from a
>> loop,
>>    -XX:+TraceRangeLimitCheck prints additional information for RC
>> elimination optimization.
>>
>>
>> Thanks for these, I will have a look at what they output.
>>
>>    Fourth, range check expression in your example is not what you think.
>> RC expression should be next:
>>    (i*stride+offset) where 'i' is loop variable, 'stride' is constant and
>> 'offset' is loop invariant.
>>
>>    In your example 'offset' is (j * cols) since it is loop invariant, 'k'
>> is loop variable and stride is '1' (one).
>>    In both your methods RC will be moved out of inner loop so the code
>> for it will be the same. The only difference in
>>    these methods will be where and how (j * cols) and (i * cols)
>> expressions are calculated.
>>
>>
>>        I'd guess it's the difference in locality that made the difference
>> in performance in your two tests.
>>
>>  Thanks for the explanation. I understand from the above that the
>> assembly output in both cases mentioned above may not
>> be different, because the expressions are similar. The difference in
>> runtime (due to cols being a compile-time constant)
>> will be visible elsewhere. Is that right? If so, where would I be able to
>> detect this?
>>
>> Cheers,
>> Manohar
>>
>>        In your PrintOptoAssembly output snippet, the instruction at 0x13e
>> is a LoadRange, which loads the range from
>>        the header
>>        of an array:
>>
>>        (from x86_64.ad <http://x86_64.ad> <http://x86_64.ad>)
>>
>>
>>        // Load Range
>>        instruct loadRange(rRegI dst, memory mem)
>>        %{
>>           match(Set dst (LoadRange mem));
>>
>>           ins_cost(125); // XXX
>>           format %{ "movl    $dst, $mem\t# range" %}
>>           opcode(0x8B);
>>           ins_encode(REX_reg_mem(dst, mem), OpcP, reg_mem(dst, mem));
>>           ins_pipe(ialu_reg_mem);
>>        %}
>>
>>        That's not a range check just yet; the real check, if any, should
>> come after the null check, in the form of
>>        comparing
>>        something else with RSI. But you didn't show what's after the null
>> check, how RSI is used, so it's hard to say what
>>        you're seeing in your example.
>>
>>        As for the two test examples, could you paste the entire source
>> code, with the PrintOptoAssembly output of
>>        method1() and
>>        method2() ? The first example looks weird, maybe it's a typo but
>> you're using "j < cols" as the loop condition
>>        for the
>>        inner loop.
>>
>>
>>        - Kris
>>
>>        On Mon, Jan 16, 2012 at 1:59 AM, Manohar Jonnalagedda <
>> manojo10386 at gmail.com <mailto:manojo10386 at gmail.com>
>>        <mailto:manojo10386 at gmail.com <mailto:manojo10386 at gmail.com>>>
>> wrote:
>>
>>            Hello,
>>
>>            following this reference on Range Check Elimination done by
>> the Hotspot compiler [1], I was keen in knowing
>>        how I
>>            can detect whether range checks are taking place in loops by
>> inspecting output using the PrintAssembly flag;
>>        with
>>            the old PrintOptoAssembly flag, I have seen output such as the
>> following, which I assume to be range checks :
>>
>>            B11: #  B73 B12 &lt;- B10  Freq: 1.21365
>>            139     movq    RAX, [rsp + #24]  # spill
>>            13e     movl    RSI, [RAX + #12 (8-bit)]  # range
>>            141     NullCheck RAX
>>
>>            What is the equivalent with the new PrintAssembly flag (using
>> hsdis)?
>>
>>            Moreover, as stated on the wiki page [1], loops are optimized
>> if the stride is a compile-time constant. I
>>        performed
>>            a few tests on a kmeans program, with 3 nested loops, having
>> the following (high-level) structure:
>>
>>            ===
>>            void method1(){
>>            //loop 1
>>            for(int i = 0; i< rows1; i++){
>>              //...
>>               for(int j = 0; j< rows2; j++){
>>               //...
>>                for(int k = 0; j < cols; k++){ array[j * cols + k] = //...}
>>               }
>>            }
>>            }
>>
>>            void method2(){
>>            //loop 2
>>            for(int i =0; i < rows1; i++){
>>               for(int j=0 ; i< rows2; j++){
>>                 for(int k=0 ; k< cols; k++){
>>                   array[i*cols+k] = //...
>>                 }
>>               }
>>            }
>>            }
>>
>>            void main(){
>>
>>              do{
>>                method1(); method2();
>>              }while(!converged)
>>
>>            }
>>            ====
>>
>>            In the first test, cols is an int whose value is determined at
>> runtime (by reading a file), in the second
>>        test, it
>>            is given as a compile-time constant(3). In the second test,
>> there is a */significant*/ speed-up (around 40%).
>>
>>            However, when studying the diff of the output of
>> PrintOptoAssembly for both method1 and method2, there is no
>>            difference (apart from slight value changes in frequency).
>> Would you have any hints as to where I could look for
>>            differences?
>>
>>            Thanks a lot,
>>            Manohar
>>
>>            [1]
>> https://wikis.oracle.com/display/HotSpotInternals/RangeCheckElimination
>>
>>
>>
>> As Kris pointed you need to fix your example:
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120119/2d6a78c8/attachment.html 

From manojo10386 at gmail.com  Thu Jan 19 03:22:08 2012
From: manojo10386 at gmail.com (Manohar Jonnalagedda)
Date: Thu, 19 Jan 2012 12:22:08 +0100
Subject: Detecting range check elimination with PrintAssembly
In-Reply-To: <CAHjP37HLTt_=WuEgJjk1rNmLbmohx8vHdOS_3ohnyJ1PxeBK1Q@mail.gmail.com>
References: <CABhSbLyAwFLFwhmTzPE=0DiXK8e8kbmvr3tr-0AQ=KyFz1Mjsw@mail.gmail.com>
	<CA+cQ+tTd_yhyu1=06JSgskU-kJ_qh=cXmnYdE6a7TgCR7s=uxg@mail.gmail.com>
	<4F135374.8020004@oracle.com>
	<CABhSbLy0gd3BFmBRX-aUgEt7h2HBHer_AQENMGWYTRN4p5-wEg@mail.gmail.com>
	<CAHjP37HLTt_=WuEgJjk1rNmLbmohx8vHdOS_3ohnyJ1PxeBK1Q@mail.gmail.com>
Message-ID: <CABhSbLycXV=Pzjo=M2aYWBS_ijBia_ruUjuQcuGWMaV-QqtXww@mail.gmail.com>

Hi Vitaly,

Are you repeatedly seeing ~40% speedup with a compile-time constant?
>

Yes I am indeed. I have a running time of around 7.8 s drop down to around
5.3 when constants are used.


> In the two assembly dumps you posted, the computeNewCentroids seems to
> have some loop unrolling + non-temporal prefetch instructions that I don't
> see (at a cursory glance, albeit) in the 2nd method.  How big is the input
> array for these functions? If it's larger than your highest level cache,
> can you try running the same tests (constant vs non-constant) with a size
> that would fit into L2/L3?
>

The input array is of size 262'144 * 3 ints, which is around 3 mbytes. So
it is definitely already in the L2/L3 range


> What cpu are you running these tests on?
>

I am running the tests on an Intel Xeon X550 :
http://ark.intel.com/products/37106/Intel-Xeon-Processor-X5550-%288M-Cache-2_66-GHz-6_40-GTs-Intel-QPI%29

Thanks,
Manohar


On Mon, Jan 16, 2012 at 4:39 AM, Manohar Jonnalagedda <manojo10386 at gmail.com
> wrote:

> Hi Kris, Vladimir,
>
> thanks for both your responses.
>
> Second, your two test methods are different so you can't directly compare
>> them. method1() iterates over rows using middle loop index 'j' and
>> method2() uses external loop index 'i'. Unless they are typos again.
>>
>
> You are right, these are indeed typos. As Kris suggested, I have the code
> printed here: http://pastebin.com/xRFD1Nt1. The methods corresponding to
> method1, and method2 are constructNearestClusterVector and
> computeNewCentroids. Their PrintOptoAssembly outputs are respectively at
> http://pastebin.com/1evN8b3K and http://pastebin.com/FxkVWTD5
>
> Also, it seems I have not explained myself correctly. I am not trying to
> compare the performance of method1 with respect to that of method2: method1
> and method2 both run in the same program. What I am trying to compare is
> their performance in two cases:
> - when cols is a compile-time constant (much faster)
> - when cols is a value determined at run-time
>
>
>
>> If you are using jdk7 there are few flags you can use to print loop
>> optimizations information. They need debug version of VM but it is not
>> problem for you, I think, since you can use debug PrintOptoAssembly flag.
>>
>> -XX:+TraceLoopOpts prints all happened loop optimizations and loop tree
>> after each round of loop opts,
>> -XX:+TraceLoopPredicate prints RC information when it is moved from a
>> loop,
>> -XX:+TraceRangeLimitCheck prints additional information for RC
>> elimination optimization.
>>
>
> Thanks for these, I will have a look at what they output.
>
> Fourth, range check expression in your example is not what you think. RC
>> expression should be next:
>> (i*stride+offset) where 'i' is loop variable, 'stride' is constant and
>> 'offset' is loop invariant.
>>
>> In your example 'offset' is (j * cols) since it is loop invariant, 'k' is
>> loop variable and stride is '1' (one).
>> In both your methods RC will be moved out of inner loop so the code for
>> it will be the same. The only difference in these methods will be where and
>> how (j * cols) and (i * cols) expressions are calculated.
>>
>
>  I'd guess it's the difference in locality that made the difference in
>>> performance in your two tests.
>>>
>>
>  Thanks for the explanation. I understand from the above that the assembly
> output in both cases mentioned above may not be different, because the
> expressions are similar. The difference in runtime (due to cols being a
> compile-time constant) will be visible elsewhere. Is that right? If so,
> where would I be able to detect this?
>
> Cheers,
> Manohar
>
>
>>  In your PrintOptoAssembly output snippet, the instruction at 0x13e is a
>>> LoadRange, which loads the range from the header
>>> of an array:
>>>
>>> (from x86_64.ad <http://x86_64.ad>)
>>>
>>> // Load Range
>>> instruct loadRange(rRegI dst, memory mem)
>>> %{
>>>   match(Set dst (LoadRange mem));
>>>
>>>   ins_cost(125); // XXX
>>>   format %{ "movl    $dst, $mem\t# range" %}
>>>   opcode(0x8B);
>>>   ins_encode(REX_reg_mem(dst, mem), OpcP, reg_mem(dst, mem));
>>>   ins_pipe(ialu_reg_mem);
>>> %}
>>>
>>> That's not a range check just yet; the real check, if any, should come
>>> after the null check, in the form of comparing
>>> something else with RSI. But you didn't show what's after the null
>>> check, how RSI is used, so it's hard to say what
>>> you're seeing in your example.
>>>
>>> As for the two test examples, could you paste the entire source code,
>>> with the PrintOptoAssembly output of method1() and
>>> method2() ? The first example looks weird, maybe it's a typo but you're
>>> using "j < cols" as the loop condition for the
>>> inner loop.
>>>
>>>
>>
>>> - Kris
>>>
>>> On Mon, Jan 16, 2012 at 1:59 AM, Manohar Jonnalagedda <
>>> manojo10386 at gmail.com <mailto:manojo10386 at gmail.com>> wrote:
>>>
>>>    Hello,
>>>
>>>    following this reference on Range Check Elimination done by the
>>> Hotspot compiler [1], I was keen in knowing how I
>>>    can detect whether range checks are taking place in loops by
>>> inspecting output using the PrintAssembly flag; with
>>>    the old PrintOptoAssembly flag, I have seen output such as the
>>> following, which I assume to be range checks :
>>>
>>>    B11: #  B73 B12 &lt;- B10  Freq: 1.21365
>>>    139     movq    RAX, [rsp + #24]  # spill
>>>    13e     movl    RSI, [RAX + #12 (8-bit)]  # range
>>>    141     NullCheck RAX
>>>
>>>    What is the equivalent with the new PrintAssembly flag (using hsdis)?
>>>
>>>    Moreover, as stated on the wiki page [1], loops are optimized if the
>>> stride is a compile-time constant. I performed
>>>    a few tests on a kmeans program, with 3 nested loops, having the
>>> following (high-level) structure:
>>>
>>>    ===
>>>    void method1(){
>>>    //loop 1
>>>    for(int i = 0; i< rows1; i++){
>>>      //...
>>>       for(int j = 0; j< rows2; j++){
>>>       //...
>>>        for(int k = 0; j < cols; k++){ array[j * cols + k] = //...}
>>>       }
>>>    }
>>>    }
>>>
>>>    void method2(){
>>>    //loop 2
>>>    for(int i =0; i < rows1; i++){
>>>       for(int j=0 ; i< rows2; j++){
>>>         for(int k=0 ; k< cols; k++){
>>>           array[i*cols+k] = //...
>>>         }
>>>       }
>>>    }
>>>    }
>>>
>>>    void main(){
>>>
>>>      do{
>>>        method1(); method2();
>>>      }while(!converged)
>>>
>>>    }
>>>    ====
>>>
>>>    In the first test, cols is an int whose value is determined at
>>> runtime (by reading a file), in the second test, it
>>>    is given as a compile-time constant(3). In the second test, there is
>>> a */significant*/ speed-up (around 40%).
>>>
>>>    However, when studying the diff of the output of PrintOptoAssembly
>>> for both method1 and method2, there is no
>>>    difference (apart from slight value changes in frequency). Would you
>>> have any hints as to where I could look for
>>>    differences?
>>>
>>>    Thanks a lot,
>>>    Manohar
>>>
>>>    [1]
>>> https://wikis.oracle.com/display/HotSpotInternals/RangeCheckElimination
>>>
>>>
>>>
>> As Kris pointed you need to fix your example:
>
>


-- 
> Vitaly
> 617-548-7007 (mobile)
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120119/860bfc12/attachment-0001.html 

From aph at redhat.com  Thu Jan 19 09:37:37 2012
From: aph at redhat.com (Andrew Haley)
Date: Thu, 19 Jan 2012 17:37:37 +0000
Subject: hsdis and caller options
Message-ID: <4F1854E1.7030008@redhat.com>

I'm trying to pass an argument to hsdis as a caller option.  It's
not being handled correctly, and I think this is the reason:

static void parse_caller_options(struct hsdis_app_data* app_data, const char* caller_options) {
  char* iop_base = app_data->insn_options;
  char* iop_limit = iop_base + sizeof(app_data->insn_options) - 1;
  char* iop = iop_base;
  const char* p;
  for (p = caller_options; p != NULL; ) {
    const char* q = strchr(p, ',');
    size_t plen = (q == NULL) ? strlen(p) : ((q++) - p);
    if (plen == 4 && strncmp(p, "help", plen) == 0) {
      print_help(app_data, NULL, NULL);
    } else if (plen >= 5 && strncmp(p, "mach=", 5) == 0) {
      char*  mach_option = app_data->mach_option;
      size_t mach_size   = sizeof(app_data->mach_option);
      mach_size -= 1;           /*leave room for the null*/
      if (plen > mach_size)  plen = mach_size;
      strncpy(mach_option, p, plen);
      mach_option[plen] = '\0';
    } else if (plen > 6 && strncmp(p, "hsdis-", 6)) {

Should this be
    ! strncmp(p, "hsdis-", 6)) {

Surely you only want this argument not to be passed if it begins
with "hsdis-" rather than if it does not begin with "hsdis-"?
I can't really tell because I don't know what is intended.

      // do not pass these to the next level
    } else {
      /* just copy it; {i386,sparc}-dis.c might like to see it  */
      if (iop > iop_base && iop < iop_limit)  (*iop++) = ',';
      if (iop + plen > iop_limit)
        plen = iop_limit - iop;
      strncpy(iop, p, plen);
      iop += plen;
    }
    p = q;
  }
}

Thanks,
Andrew.

From vladimir.kozlov at oracle.com  Thu Jan 19 10:07:12 2012
From: vladimir.kozlov at oracle.com (Vladimir Kozlov)
Date: Thu, 19 Jan 2012 10:07:12 -0800
Subject: Detecting range check elimination with PrintAssembly
In-Reply-To: <CABhSbLyzsAssScg+6oCDJPfzac5SdyoZMAMAxr6Vv-WjG8amqg@mail.gmail.com>
References: <CABhSbLyAwFLFwhmTzPE=0DiXK8e8kbmvr3tr-0AQ=KyFz1Mjsw@mail.gmail.com>	<CA+cQ+tTd_yhyu1=06JSgskU-kJ_qh=cXmnYdE6a7TgCR7s=uxg@mail.gmail.com>	<4F135374.8020004@oracle.com>	<CABhSbLy0gd3BFmBRX-aUgEt7h2HBHer_AQENMGWYTRN4p5-wEg@mail.gmail.com>	<4F149D88.2060305@oracle.com>
	<CABhSbLyzsAssScg+6oCDJPfzac5SdyoZMAMAxr6Vv-WjG8amqg@mail.gmail.com>
Message-ID: <4F185BD0.4040408@oracle.com>

Manohar Jonnalagedda wrote:
> Hi Vladimir, Vitaly,
> 
>     In such situations we usually use some visual tools to see
>     difference between log outputs. 
> 
> 
> Would you recommend any? I have had troubles using IdealGraphVisualizer: 
> either the overhead of using it makes the methods not get JIT compiled, 
> or, if I decrese the CompileThreshold, the VM crashes with an out of 
> memory error.

PrintOptoAssembly output is simple text. You can use any text compare tools 
available on net (google it). IdealGraphVisualizer is C2 tool to trace C2 
optimization, you don't need it to compare text outputs.

> 
>     At least you can use 'diff'. You may need to replace instructions
>     addresses in outputs (number at the beginning of lines) with the
>     same value to match. There are few tricks you may use to get similar
>     PrintOptoAssembly output. Use next flags to avoid mixing output from
>     program output and from 2 compiler threads (flags stop program until
>     a method is compiled and run only one compiler thread):
> 
>     -Xbatch -XX:CICompilerCount=1
> 
>     Also add -XX:+PrintCompilation -XX:+PrintInlining to see what method
>     is compiled and inlined. Note that you may see similar output for
>     individual methods but could be big difference in compiled caller
>     (computeAll()) method where 2 loop methods could be inlined. So you
>     need to compare all compiled methods.
> 
> 
> PrintInlining tells me that constructNearestClusterVector is inlined 
> into ComputeAll, and from what I understand, computeNewCentroids is not:
> 
> @ 109   Kmeans::computeNewCentroids (163 bytes)   already compiled into 
> a big method

Correct, it is big method and C2 decide to not inline it. You can change it by 
increasing InlineSmallCode value, for example -XX:InlineSmallCode=1500.

> 
> I have this message 3 times, do these belong to the Pre-Main-Post of the 
> loop in the computeAll method?
>  
> 
>     In general, to have constant as loop limit is always win because
>     some checks in generated code could be avoided and more
>     optimizations could be done for such loops. Use -XX:+TraceLoopOpts
>     to see what loop optimizations are done in both cases.
> 
> I am new to this flag: should I use it in conjunction with the 
> PrintOptoAssembly output for identifying loop numbers with the code they 
> represent? Would there be any other way to interpret these?

I suggested it to see what different loop optimizations are performed when a 
constant is used vs not. For example, if you get "MaxUnroll" instead of 
"PreMainPost" the generated code should be better.

Vladimir

> 
> 
> Thanks,
> Manohar
>  
> 
> 
>     For example, in your code you set 'x_col = 3', as result the next
>     loop in constructNearestClusterVector() will be fully unrolled when
>     this method is inlined into computeAll() and x_col is replaced with '3':
> 
>            for(k = 0; k < x_col; k++) {
>              double tmp = x[i*x_col + k] - mu[j* mu_col + k];
>              dist += tmp * tmp;
>            }
> 
>     Vladimir
> 
> 
>     On 1/16/12 1:39 AM, Manohar Jonnalagedda wrote:
> 
>         Hi Kris, Vladimir,
> 
>         thanks for both your responses.
> 
>            Second, your two test methods are different so you can't
>         directly compare them. method1() iterates over rows using
>            middle loop index 'j' and method2() uses external loop index
>         'i'. Unless they are typos again.
> 
> 
>         You are right, these are indeed typos. As Kris suggested, I have
>         the code printed here: http://pastebin.com/xRFD1Nt1.
>         The methods corresponding to method1, and method2 are
>         constructNearestClusterVector and computeNewCentroids. Their
>         PrintOptoAssembly outputs are respectively at
>         http://pastebin.com/1evN8b3K and http://pastebin.com/FxkVWTD5
> 
>         Also, it seems I have not explained myself correctly. I am not
>         trying to compare the performance of method1 with respect
>         to that of method2: method1 and method2 both run in the same
>         program. What I am trying to compare is their performance
>         in two cases:
>         - when cols is a compile-time constant (much faster)
>         - when cols is a value determined at run-time
> 
>            If you are using jdk7 there are few flags you can use to
>         print loop optimizations information. They need debug
>            version of VM but it is not problem for you, I think, since
>         you can use debug PrintOptoAssembly flag.
> 
>            -XX:+TraceLoopOpts prints all happened loop optimizations and
>         loop tree after each round of loop opts,
>            -XX:+TraceLoopPredicate prints RC information when it is
>         moved from a loop,
>            -XX:+TraceRangeLimitCheck prints additional information for
>         RC elimination optimization.
> 
> 
>         Thanks for these, I will have a look at what they output.
> 
>            Fourth, range check expression in your example is not what
>         you think. RC expression should be next:
>            (i*stride+offset) where 'i' is loop variable, 'stride' is
>         constant and 'offset' is loop invariant.
> 
>            In your example 'offset' is (j * cols) since it is loop
>         invariant, 'k' is loop variable and stride is '1' (one).
>            In both your methods RC will be moved out of inner loop so
>         the code for it will be the same. The only difference in
>            these methods will be where and how (j * cols) and (i * cols)
>         expressions are calculated.
> 
> 
>                I'd guess it's the difference in locality that made the
>         difference in performance in your two tests.
> 
>          Thanks for the explanation. I understand from the above that
>         the assembly output in both cases mentioned above may not
>         be different, because the expressions are similar. The
>         difference in runtime (due to cols being a compile-time constant)
>         will be visible elsewhere. Is that right? If so, where would I
>         be able to detect this?
> 
>         Cheers,
>         Manohar
> 
>                In your PrintOptoAssembly output snippet, the instruction
>         at 0x13e is a LoadRange, which loads the range from
>                the header
>                of an array:
> 
>                (from x86_64.ad <http://x86_64.ad> <http://x86_64.ad>
>         <http://x86_64.ad>)
> 
> 
>                // Load Range
>                instruct loadRange(rRegI dst, memory mem)
>                %{
>                   match(Set dst (LoadRange mem));
> 
>                   ins_cost(125); // XXX
>                   format %{ "movl    $dst, $mem\t# range" %}
>                   opcode(0x8B);
>                   ins_encode(REX_reg_mem(dst, mem), OpcP, reg_mem(dst,
>         mem));
>                   ins_pipe(ialu_reg_mem);
>                %}
> 
>                That's not a range check just yet; the real check, if
>         any, should come after the null check, in the form of
>                comparing
>                something else with RSI. But you didn't show what's after
>         the null check, how RSI is used, so it's hard to say what
>                you're seeing in your example.
> 
>                As for the two test examples, could you paste the entire
>         source code, with the PrintOptoAssembly output of
>                method1() and
>                method2() ? The first example looks weird, maybe it's a
>         typo but you're using "j < cols" as the loop condition
>                for the
>                inner loop.
> 
> 
>                - Kris
> 
>                On Mon, Jan 16, 2012 at 1:59 AM, Manohar Jonnalagedda
>         <manojo10386 at gmail.com <mailto:manojo10386 at gmail.com>
>         <mailto:manojo10386 at gmail.com <mailto:manojo10386 at gmail.com>>
>                <mailto:manojo10386 at gmail.com
>         <mailto:manojo10386 at gmail.com> <mailto:manojo10386 at gmail.com
>         <mailto:manojo10386 at gmail.com>>>> wrote:
> 
>                    Hello,
> 
>                    following this reference on Range Check Elimination
>         done by the Hotspot compiler [1], I was keen in knowing
>                how I
>                    can detect whether range checks are taking place in
>         loops by inspecting output using the PrintAssembly flag;
>                with
>                    the old PrintOptoAssembly flag, I have seen output
>         such as the following, which I assume to be range checks :
> 
>                    B11: #  B73 B12 &lt;- B10  Freq: 1.21365
>                    139     movq    RAX, [rsp + #24]  # spill
>                    13e     movl    RSI, [RAX + #12 (8-bit)]  # range
>                    141     NullCheck RAX
> 
>                    What is the equivalent with the new PrintAssembly
>         flag (using hsdis)?
> 
>                    Moreover, as stated on the wiki page [1], loops are
>         optimized if the stride is a compile-time constant. I
>                performed
>                    a few tests on a kmeans program, with 3 nested loops,
>         having the following (high-level) structure:
> 
>                    ===
>                    void method1(){
>                    //loop 1
>                    for(int i = 0; i< rows1; i++){
>                      //...
>                       for(int j = 0; j< rows2; j++){
>                       //...
>                        for(int k = 0; j < cols; k++){ array[j * cols +
>         k] = //...}
>                       }
>                    }
>                    }
> 
>                    void method2(){
>                    //loop 2
>                    for(int i =0; i < rows1; i++){
>                       for(int j=0 ; i< rows2; j++){
>                         for(int k=0 ; k< cols; k++){
>                           array[i*cols+k] = //...
>                         }
>                       }
>                    }
>                    }
> 
>                    void main(){
> 
>                      do{
>                        method1(); method2();
>                      }while(!converged)
> 
>                    }
>                    ====
> 
>                    In the first test, cols is an int whose value is
>         determined at runtime (by reading a file), in the second
>                test, it
>                    is given as a compile-time constant(3). In the second
>         test, there is a */significant*/ speed-up (around 40%).
> 
>                    However, when studying the diff of the output of
>         PrintOptoAssembly for both method1 and method2, there is no
>                    difference (apart from slight value changes in
>         frequency). Would you have any hints as to where I could look for
>                    differences?
> 
>                    Thanks a lot,
>                    Manohar
> 
>                    [1]
>         https://wikis.oracle.com/display/HotSpotInternals/RangeCheckElimination
> 
> 
> 
>         As Kris pointed you need to fix your example:
> 
> 
> 

From john.r.rose at oracle.com  Thu Jan 19 12:02:48 2012
From: john.r.rose at oracle.com (John Rose)
Date: Thu, 19 Jan 2012 12:02:48 -0800
Subject: hsdis and caller options
In-Reply-To: <4F1854E1.7030008@redhat.com>
References: <4F1854E1.7030008@redhat.com>
Message-ID: <E396B7AC-3E57-479E-8C3A-3494B215F581@oracle.com>

On Jan 19, 2012, at 9:37 AM, Andrew Haley wrote:

> I'm trying to pass an argument to hsdis as a caller option.  It's
> not being handled correctly, and I think this is the reason:
> 

...

> Should this be
>    ! strncmp(p, "hsdis-", 6)) {

You are correct; that's a bug.

-- John

From john.r.rose at oracle.com  Thu Jan 19 15:01:42 2012
From: john.r.rose at oracle.com (john.r.rose at oracle.com)
Date: Thu, 19 Jan 2012 23:01:42 +0000
Subject: hg: hsx/hotspot-main/jdk: 3 new changesets
Message-ID: <20120119230230.3C2D7470AE@hg.openjdk.java.net>

Changeset: db189e2f3cdb
Author:    jrose
Date:      2012-01-18 17:34 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/db189e2f3cdb

7117167: Misc warnings in java.lang.invoke and sun.invoke.*
Reviewed-by: smarks

! src/share/classes/java/lang/invoke/AdapterMethodHandle.java
! src/share/classes/java/lang/invoke/MemberName.java
! src/share/classes/java/lang/invoke/MethodHandleImpl.java
! src/share/classes/java/lang/invoke/MethodHandleProxies.java
! src/share/classes/java/lang/invoke/MethodHandles.java
! src/share/classes/sun/invoke/util/ValueConversions.java
! src/share/classes/sun/invoke/util/Wrapper.java
! test/java/lang/invoke/CallSiteTest.java
! test/java/lang/invoke/ClassValueTest.java
! test/java/lang/invoke/InvokeGenericTest.java
! test/java/lang/invoke/JavaDocExamplesTest.java
! test/java/lang/invoke/MethodHandlesTest.java
! test/java/lang/invoke/MethodTypeTest.java
! test/java/lang/invoke/PermuteArgsTest.java
! test/java/lang/invoke/RicochetTest.java
! test/java/lang/invoke/ThrowExceptionsTest.java
! test/sun/invoke/util/ValueConversionsTest.java

Changeset: 01014596ada1
Author:    jrose
Date:      2012-01-18 17:34 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/01014596ada1

7077803: java.lang.InternalError in java.lang.invoke.MethodHandleNatives.init
Summary: Use correct access token for unreflecting MHs where setAccessible(true)
Reviewed-by: never, twisti

! src/share/classes/java/lang/invoke/MethodHandles.java

Changeset: 92d2cba30f08
Author:    jrose
Date:      2012-01-18 17:34 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/92d2cba30f08

7030453: JSR 292 ClassValue.get method is too slow
Summary: Implement ClassValue cooperatively with Class like ThreadLocal with Thread.
Reviewed-by: twisti, mduigou

! src/share/classes/java/lang/Class.java
! src/share/classes/java/lang/ClassValue.java
! test/java/lang/invoke/ClassValueTest.java


From aph at redhat.com  Thu Jan 19 23:42:17 2012
From: aph at redhat.com (Andrew Haley)
Date: Fri, 20 Jan 2012 07:42:17 +0000
Subject: hsdis and caller options
In-Reply-To: <E396B7AC-3E57-479E-8C3A-3494B215F581@oracle.com>
References: <4F1854E1.7030008@redhat.com>
	<E396B7AC-3E57-479E-8C3A-3494B215F581@oracle.com>
Message-ID: <4F191AD9.5010701@redhat.com>

On 01/19/2012 08:02 PM, John Rose wrote:
> On Jan 19, 2012, at 9:37 AM, Andrew Haley wrote:
> 
>> > I'm trying to pass an argument to hsdis as a caller option.  It's
>> > not being handled correctly, and I think this is the reason:
>> > 
> ...
> 
>> > Should this be
>> >    ! strncmp(p, "hsdis-", 6)) {
> You are correct; that's a bug.

OK.  Should I create a bug report for this?

Andrew.


From john.r.rose at oracle.com  Fri Jan 20 00:30:43 2012
From: john.r.rose at oracle.com (John Rose)
Date: Fri, 20 Jan 2012 00:30:43 -0800
Subject: hsdis and caller options
In-Reply-To: <4F191AD9.5010701@redhat.com>
References: <4F1854E1.7030008@redhat.com>
	<E396B7AC-3E57-479E-8C3A-3494B215F581@oracle.com>
	<4F191AD9.5010701@redhat.com>
Message-ID: <1623C71E-7E7E-4F2F-98A1-2B50A456E465@oracle.com>

On Jan 19, 2012, at 11:42 PM, Andrew Haley wrote:

>> You are correct; that's a bug.
> 
> OK.  Should I create a bug report for this?

Yes, please.  -- John
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120120/3ccaea33/attachment.html 

From john.r.rose at oracle.com  Fri Jan 20 05:18:18 2012
From: john.r.rose at oracle.com (john.r.rose at oracle.com)
Date: Fri, 20 Jan 2012 13:18:18 +0000
Subject: hg: hsx/hotspot-main/hotspot: 5 new changesets
Message-ID: <20120120131836.26EBA470C2@hg.openjdk.java.net>

Changeset: eaa9557116a2
Author:    bdelsart
Date:      2012-01-18 16:18 +0100
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/eaa9557116a2

7120448: Fix FP values for compiled frames in frame::describe
Summary: fix for debug method frame::describe
Reviewed-by: never, kvn

! src/cpu/sparc/vm/frame_sparc.inline.hpp
! src/cpu/x86/vm/frame_x86.cpp
! src/cpu/x86/vm/frame_x86.hpp
! src/cpu/zero/vm/frame_zero.inline.hpp
! src/share/vm/runtime/frame.cpp
! src/share/vm/runtime/frame.hpp

Changeset: 15d394228cfa
Author:    jrose
Date:      2012-01-19 13:00 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/15d394228cfa

7111138: delete the obsolete flag -XX:+UseRicochetFrames
Reviewed-by: dholmes, bdelsart, kvn, twisti

! src/cpu/sparc/vm/methodHandles_sparc.cpp
! src/cpu/x86/vm/methodHandles_x86.cpp
! src/cpu/zero/vm/methodHandles_zero.hpp
! src/share/vm/prims/methodHandles.cpp
! src/share/vm/prims/methodHandles.hpp
! src/share/vm/runtime/globals.hpp
! src/share/vm/runtime/sharedRuntime.cpp

Changeset: 898522ae3c32
Author:    iveresov
Date:      2012-01-19 10:56 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/898522ae3c32

7131288: COMPILE SKIPPED: deopt handler overflow (retry at different tier)
Summary: Fix exception handler stub size, enable guarantees to check for the correct deopt and exception stub sizes in the future
Reviewed-by: kvn, never, twisti

! src/cpu/sparc/vm/c1_LIRAssembler_sparc.cpp
! src/cpu/sparc/vm/c1_LIRAssembler_sparc.hpp
! src/cpu/x86/vm/c1_LIRAssembler_x86.cpp

Changeset: 469e0a46f2fe
Author:    jrose
Date:      2012-01-19 17:20 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/469e0a46f2fe

Merge


Changeset: 50d9b7a0072c
Author:    jrose
Date:      2012-01-19 18:35 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/50d9b7a0072c

Merge



From staffan.larsen at oracle.com  Fri Jan 20 11:17:35 2012
From: staffan.larsen at oracle.com (Staffan Larsen)
Date: Fri, 20 Jan 2012 20:17:35 +0100
Subject: RFR (S): 7125793: MAC: test_gamma should always work
In-Reply-To: <4F0CA809.7020001@oracle.com>
References: <4EFECA5D.6010905@oracle.com> <4F034B51.3070609@oracle.com>
	<4F087516.40505@oracle.com> <4F0B7A34.5070406@oracle.com>
	<4F0CA809.7020001@oracle.com>
Message-ID: <F9C3D956-42B9-4B90-BFF5-2250FFA429F9@oracle.com>

Sorry if I am late to the game, but I was just wishing for this functionality - and there was the web rev! 

I've looked at this version of the patch: http://cr.openjdk.java.net/~jmelvin/7125793/webrev.03/

It looks good. Only two small comments:

java_md.c:GetJVMPath - I don't think the code inside "#ifndef GAMMA" needs to be changed. The code is always compiled with GAMMA defined. The "#ifndef GAMMA" is just there to facilitate comparison with the original code in the JDK.

os_bsd.cpp:2580 - (nit) incorrect indentation

I have also verified that the "hotspot" launcher works after this change.

/Staffan


On 10 jan 2012, at 22:05, James Melvin wrote:

> Hi Dan,
> 
> Final webrev to reflect your comments...
> 
> http://cr.openjdk.java.net/~jmelvin/7125793/webrev.02
> 
> Minor changes this round:
> 
>  make/bsd/makefiles/buildtree.make   # Fail gracefully on Apple BOOTDIR
>  make/bsd/makefiles/launcher.make    # Link with framework only on Mac
>  src/os/bsd/vm/os_bsd.cpp            # Just spelling fix
> 
> Lastly, I wanted to reply to John Coomes comments earlier about the
> test_gamma script simplification. Although I also value economy of
> expression, in this case I think the use of more advanced shell
> constructs increases the time for fresh eyes to decipher. Given
> performance and such is not an issue, I'd prefer to keep the simpler
> version I'm proposing with this change on Mac OS X, to make it easier on
> future maintenance. This should be a model for the other platforms when
> we reconcile. I've attached the before and after copies should there be
> further comments on the simplified short script.
> 
> Thanks,
> 
> Jim
> 
> 
> On 1/9/12 6:37 PM, Daniel D. Daugherty wrote:
>> On 1/7/12 9:38 AM, James Melvin wrote:
>>> WEBREV:
>>> http://cr.openjdk.java.net/~jmelvin/7125793/webrev.01
>> 
>> make/bsd/Makefile
>> No comments.
>> 
>> make/bsd/makefiles/buildtree.make
>> No comments.
>> 
>> make/bsd/makefiles/defs.make
>> Thanks for fixing this one for BSD platforms.
>> 
>> make/bsd/makefiles/launcher.make
>> line 60: typo: 'inadvertenly' -> 'inadvertently'
>> 
>> Sorry I missed this in my first review, but the addition
>> of '-framework CoreFoundation' to LFLAGS_LAUNCHER is
>> probably MacOS X specific. I think:
>> 
>> ifeq ($(OS_VENDOR), Darwin)
>> else
>> endif
>> 
>> will work in launcher.make also.
>> 
>> make/bsd/makefiles/vm.make
>> No comments.
>> 
>> src/os/bsd/vm/os_bsd.cpp
>> line 2544: typo: 'overriden' -> 'overridden'
>> line 2588: typo: 'overriden' -> 'overridden'
>> 
>> Looks like old code line 2576 depended on the 'hotspot'
>> symlink to refer to either 'client' or 'server' or whatever
>> JVM you wanted to run. I'm fairly certain that the 'hotspot'
>> symlink was retired; I'm just not sure when.
>> 
>> src/os/posix/launcher/java_md.c
>> No comments.
>> 
>> Dan
>> 
>> 
> <test_gamma.before><test_gamma.after>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120120/fe0a3f7e/attachment.html 

From john.coomes at oracle.com  Fri Jan 20 14:48:45 2012
From: john.coomes at oracle.com (john.coomes at oracle.com)
Date: Fri, 20 Jan 2012 22:48:45 +0000
Subject: hg: hsx/hotspot-main: 2 new changesets
Message-ID: <20120120224845.E7118470DC@hg.openjdk.java.net>

Changeset: 7ad075c80995
Author:    katleman
Date:      2012-01-13 10:05 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/rev/7ad075c80995

Added tag jdk8-b21 for changeset cc771d92284f

! .hgtags

Changeset: 60d6f64a86b1
Author:    katleman
Date:      2012-01-20 13:08 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/rev/60d6f64a86b1

Added tag jdk8-b22 for changeset 7ad075c80995

! .hgtags


From john.coomes at oracle.com  Fri Jan 20 14:48:51 2012
From: john.coomes at oracle.com (john.coomes at oracle.com)
Date: Fri, 20 Jan 2012 22:48:51 +0000
Subject: hg: hsx/hotspot-main/corba: 2 new changesets
Message-ID: <20120120224854.7633A470DD@hg.openjdk.java.net>

Changeset: a11d0062c445
Author:    katleman
Date:      2012-01-13 10:05 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/corba/rev/a11d0062c445

Added tag jdk8-b21 for changeset f157fc2a71a3

! .hgtags

Changeset: 5218eb256658
Author:    katleman
Date:      2012-01-20 13:08 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/corba/rev/5218eb256658

Added tag jdk8-b22 for changeset a11d0062c445

! .hgtags


From john.coomes at oracle.com  Fri Jan 20 14:49:00 2012
From: john.coomes at oracle.com (john.coomes at oracle.com)
Date: Fri, 20 Jan 2012 22:49:00 +0000
Subject: hg: hsx/hotspot-main/jaxp: 2 new changesets
Message-ID: <20120120224900.420F5470DE@hg.openjdk.java.net>

Changeset: cf9d6ec44f89
Author:    katleman
Date:      2012-01-13 10:05 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jaxp/rev/cf9d6ec44f89

Added tag jdk8-b21 for changeset d41eeadf5c13

! .hgtags

Changeset: 95102fd33418
Author:    katleman
Date:      2012-01-20 13:08 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jaxp/rev/95102fd33418

Added tag jdk8-b22 for changeset cf9d6ec44f89

! .hgtags


From john.coomes at oracle.com  Fri Jan 20 14:49:05 2012
From: john.coomes at oracle.com (john.coomes at oracle.com)
Date: Fri, 20 Jan 2012 22:49:05 +0000
Subject: hg: hsx/hotspot-main/jaxws: 4 new changesets
Message-ID: <20120120224905.BB09C470DF@hg.openjdk.java.net>

Changeset: e67d51254533
Author:    ohair
Date:      2012-01-09 09:22 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jaxws/rev/e67d51254533

7096063: /META-INF/mimetypes.default missing in jre\lib\resources.jar
Reviewed-by: dholmes

! build-defs.xml

Changeset: c266cab0e3ff
Author:    katleman
Date:      2012-01-11 16:12 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jaxws/rev/c266cab0e3ff

Merge


Changeset: 8d3df89b0f2d
Author:    katleman
Date:      2012-01-13 10:05 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jaxws/rev/8d3df89b0f2d

Added tag jdk8-b21 for changeset c266cab0e3ff

! .hgtags

Changeset: 25ce7a000487
Author:    katleman
Date:      2012-01-20 13:08 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jaxws/rev/25ce7a000487

Added tag jdk8-b22 for changeset 8d3df89b0f2d

! .hgtags


From john.coomes at oracle.com  Fri Jan 20 14:49:38 2012
From: john.coomes at oracle.com (john.coomes at oracle.com)
Date: Fri, 20 Jan 2012 22:49:38 +0000
Subject: hg: hsx/hotspot-main/jdk: 20 new changesets
Message-ID: <20120120225310.A80AB470E0@hg.openjdk.java.net>

Changeset: 1c4fffa22930
Author:    okutsu
Date:      2011-12-21 17:09 +0900
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/1c4fffa22930

7122054: (tz) Windows-only: tzmappings needs update for KB2633952
Reviewed-by: peytoia

! src/windows/lib/tzmappings

Changeset: b1814b3ea6d3
Author:    michaelm
Date:      2011-12-21 10:06 +0000
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/b1814b3ea6d3

7078386: NetworkInterface.getNetworkInterfaces() may return corrupted results on linux
Reviewed-by: michaelm, alanb, chegar
Contributed-by: brandon.passanisi at oracle.com

! src/solaris/native/java/net/NetworkInterface.c

Changeset: a9dfdc523c2c
Author:    valeriep
Date:      2011-12-21 14:08 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/a9dfdc523c2c

6839886: Array overrun in pkcs11
Summary: Fix the wrong value when dealing w/ month and day.
Reviewed-by: mullan

! src/share/native/sun/security/pkcs11/wrapper/p11_convert.c

Changeset: 11698dedbe79
Author:    weijun
Date:      2011-12-22 15:35 +0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/11698dedbe79

7122169: TcpTimeout fail for various reasons
Reviewed-by: alanb

! test/sun/security/krb5/auto/TcpTimeout.java

Changeset: 559e07ed1f56
Author:    alanb
Date:      2011-12-22 10:52 +0000
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/559e07ed1f56

7076310: (file) AclEntry.Builder setFlags throws IllegalArgumentException if set argument is empty
Reviewed-by: alanb
Contributed-by: stephen.flores at oracle.com

! src/share/classes/java/nio/file/attribute/AclEntry.java
+ test/java/nio/file/attribute/AclEntry/EmptySet.java

Changeset: 3c1ab134db71
Author:    dcubed
Date:      2011-12-22 18:35 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/3c1ab134db71

7121600: Instrumentation.redefineClasses() leaks class bytes
Summary: Call JNI ReleaseByteArrayElements() on memory returned by JNI GetByteArrayElements(). Also push test for 7122253.
Reviewed-by: acorn, poonam

! src/share/instrument/JPLISAgent.c
+ test/java/lang/instrument/BigClass.java
+ test/java/lang/instrument/MakeJAR4.sh
+ test/java/lang/instrument/RedefineBigClass.sh
+ test/java/lang/instrument/RedefineBigClassAgent.java
+ test/java/lang/instrument/RedefineBigClassApp.java
+ test/java/lang/instrument/RetransformBigClass.sh
+ test/java/lang/instrument/RetransformBigClassAgent.java
+ test/java/lang/instrument/RetransformBigClassApp.java

Changeset: 437255d15e76
Author:    lana
Date:      2011-12-28 10:51 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/437255d15e76

Merge

- src/share/classes/sun/awt/FocusingTextField.java
- src/share/classes/sun/awt/HorizBagLayout.java
- src/share/classes/sun/awt/OrientableFlowLayout.java
- src/share/classes/sun/awt/VariableGridLayout.java
- src/share/classes/sun/awt/VerticalBagLayout.java

Changeset: 3a7ea63302f8
Author:    smarks
Date:      2011-12-29 16:39 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/3a7ea63302f8

7122061: add -Xlint:all -Werror to warning-free build steps
Reviewed-by: chegar, alanb, dholmes, ohair

! make/com/sun/demo/jvmti/hprof/Makefile
! make/com/sun/java/browser/net/Makefile
! make/com/sun/tools/Makefile
! make/com/sun/tools/attach/Makefile
! make/com/sun/tracing/Makefile
! make/com/sun/tracing/dtrace/Makefile
! make/java/instrument/Makefile
! make/java/rmi/Makefile
! make/java/text/base/Makefile
! make/java/text/bidi/Makefile
! make/java/util/Makefile
! make/javax/accessibility/Makefile
! make/javax/others/Makefile
! make/javax/security/Makefile
! make/jpda/tty/Makefile
! make/sun/launcher/Makefile
! make/sun/serialver/Makefile
! make/sun/text/Makefile
! make/sun/util/Makefile

Changeset: 5aeefe0e5d8c
Author:    chegar
Date:      2012-01-01 09:24 +0000
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/5aeefe0e5d8c

7125055: ContentHandler.getContent API changed in error
Reviewed-by: alanb

! src/share/classes/java/net/ContentHandler.java
! src/share/classes/sun/net/www/content/image/gif.java
! src/share/classes/sun/net/www/content/image/jpeg.java
! src/share/classes/sun/net/www/content/image/png.java
! src/share/classes/sun/net/www/content/image/x_xbitmap.java
! src/share/classes/sun/net/www/content/image/x_xpixmap.java

Changeset: 8952a5f494f9
Author:    ksrini
Date:      2012-01-03 08:27 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/8952a5f494f9

7123582: (launcher) display the -version and -XshowSettings
Reviewed-by: alanb

! src/share/bin/java.c
! test/tools/launcher/Settings.java

Changeset: 5e34726cb4bb
Author:    ksrini
Date:      2012-01-03 08:33 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/5e34726cb4bb

7124443: (launcher) test DefaultsLocaleTest fails with Windows shells.
Reviewed-by: darcy

! test/tools/launcher/DefaultLocaleTest.java
- test/tools/launcher/DefaultLocaleTest.sh
+ test/tools/launcher/DefaultLocaleTestRun.java
! test/tools/launcher/TestHelper.java

Changeset: 0194fe5ca404
Author:    fparain
Date:      2012-01-04 03:49 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/0194fe5ca404

7104647: Adding a diagnostic command framework
Reviewed-by: mchung, dholmes

! make/common/Release.gmk
! make/java/management/mapfile-vers
! make/launchers/Makefile
! make/sun/tools/Makefile
+ src/linux/doc/man/jcmd.1
+ src/share/classes/com/sun/management/DiagnosticCommandArgumentInfo.java
+ src/share/classes/com/sun/management/DiagnosticCommandInfo.java
! src/share/classes/com/sun/management/HotSpotDiagnosticMXBean.java
! src/share/classes/sun/management/HotSpotDiagnostic.java
! src/share/classes/sun/tools/attach/HotSpotVirtualMachine.java
+ src/share/classes/sun/tools/jcmd/Arguments.java
+ src/share/classes/sun/tools/jcmd/JCmd.java
! src/share/javavm/export/jmm.h
! src/share/native/sun/management/HotSpotDiagnostic.c
+ src/solaris/doc/sun/man/man1/jcmd.1
+ test/com/sun/management/HotSpotDiagnosticMXBean/ExecuteDiagnosticCommand.java
+ test/com/sun/management/HotSpotDiagnosticMXBean/GetDiagnosticCommandInfo.java
+ test/com/sun/management/HotSpotDiagnosticMXBean/GetDiagnosticCommands.java
! test/sun/tools/common/CommonSetup.sh
+ test/sun/tools/jcmd/dcmd-script.txt
+ test/sun/tools/jcmd/help_help.out
+ test/sun/tools/jcmd/jcmd-Defaults.sh
+ test/sun/tools/jcmd/jcmd-f.sh
+ test/sun/tools/jcmd/jcmd-help-help.sh
+ test/sun/tools/jcmd/jcmd-help.sh
+ test/sun/tools/jcmd/jcmd-pid.sh
+ test/sun/tools/jcmd/jcmd_Output1.awk
+ test/sun/tools/jcmd/jcmd_pid_Output1.awk
+ test/sun/tools/jcmd/jcmd_pid_Output2.awk
+ test/sun/tools/jcmd/usage.out

Changeset: 38a318502e19
Author:    lana
Date:      2012-01-04 10:57 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/38a318502e19

Merge

! make/common/Release.gmk
- test/tools/launcher/DefaultLocaleTest.sh

Changeset: 93ab1df09d11
Author:    lana
Date:      2012-01-09 19:12 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/93ab1df09d11

Merge

- test/tools/launcher/DefaultLocaleTest.sh

Changeset: ddb97d4fa83d
Author:    ohair
Date:      2012-01-04 17:42 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/ddb97d4fa83d

7127104: Build issue with prtconf and zones, also using := to avoid extra execs
Reviewed-by: katleman

! make/common/shared/Platform.gmk

Changeset: 7c8c16f2c05e
Author:    ohair
Date:      2012-01-09 09:18 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/7c8c16f2c05e

7128320: Fix freetype sanity check to make it more generic
Reviewed-by: luchsh, katleman
Contributed-by: Jonathan Lu <luchsh at linux.vnet.ibm.com>

! make/common/Defs-linux.gmk
! make/common/Defs-solaris.gmk
! make/common/Defs-windows.gmk
! make/common/Demo.gmk
! make/tools/freetypecheck/Makefile

Changeset: 664fa4fb0ee4
Author:    katleman
Date:      2012-01-11 16:13 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/664fa4fb0ee4

Merge


Changeset: dda27c73d8db
Author:    katleman
Date:      2012-01-13 10:05 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/dda27c73d8db

Added tag jdk8-b21 for changeset 664fa4fb0ee4

! .hgtags

Changeset: 76bfd08d8cc5
Author:    katleman
Date:      2012-01-20 13:08 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/76bfd08d8cc5

Added tag jdk8-b22 for changeset dda27c73d8db

! .hgtags

Changeset: 81a2629aa2a2
Author:    amurillo
Date:      2012-01-20 14:31 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/81a2629aa2a2

Merge



From john.coomes at oracle.com  Fri Jan 20 14:54:17 2012
From: john.coomes at oracle.com (john.coomes at oracle.com)
Date: Fri, 20 Jan 2012 22:54:17 +0000
Subject: hg: hsx/hotspot-main/langtools: 8 new changesets
Message-ID: <20120120225436.11B65470E1@hg.openjdk.java.net>

Changeset: 116f68a5e677
Author:    jjg
Date:      2011-12-23 22:30 +0000
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/langtools/rev/116f68a5e677

7124605: typos in javac comments
Reviewed-by: ksrini

! test/tools/javac/generics/diamond/7046778/DiamondAndInnerClassTest.java
! test/tools/javac/generics/inference/7086601/T7086601b.java
! test/tools/javac/generics/rawOverride/7062745/GenericOverrideTest.java
! test/tools/javac/lambda/LambdaParserTest.java

Changeset: 67512b631961
Author:    lana
Date:      2011-12-28 10:52 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/langtools/rev/67512b631961

Merge


Changeset: 7a836147b266
Author:    jjg
Date:      2012-01-03 11:37 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/langtools/rev/7a836147b266

4881269: improve diagnostic for ill-formed tokens
Reviewed-by: mcimadamore

! src/share/classes/com/sun/tools/javac/parser/JavaTokenizer.java
! src/share/classes/com/sun/tools/javac/resources/compiler.properties
+ test/tools/javac/diags/examples/IllegalDot.java
+ test/tools/javac/parser/T4881269.java
+ test/tools/javac/parser/T4881269.out

Changeset: a07eef109532
Author:    jjh
Date:      2012-01-03 17:18 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/langtools/rev/a07eef109532

7046929: tools/javac/api/T6397104.java fails
Reviewed-by: jjg

! test/tools/javac/api/T6397104.java

Changeset: 4e8aa6eca726
Author:    lana
Date:      2012-01-04 10:58 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/langtools/rev/4e8aa6eca726

Merge


Changeset: bcb21abf1c41
Author:    lana
Date:      2012-01-09 19:13 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/langtools/rev/bcb21abf1c41

Merge


Changeset: 390a7828ae18
Author:    katleman
Date:      2012-01-13 10:05 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/langtools/rev/390a7828ae18

Added tag jdk8-b21 for changeset bcb21abf1c41

! .hgtags

Changeset: f6191bad139a
Author:    katleman
Date:      2012-01-20 13:08 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/langtools/rev/f6191bad139a

Added tag jdk8-b22 for changeset 390a7828ae18

! .hgtags


From karen.kinnear at oracle.com  Fri Jan 20 15:06:58 2012
From: karen.kinnear at oracle.com (Karen Kinnear)
Date: Fri, 20 Jan 2012 18:06:58 -0500
Subject: Code and Design Feedback request: 7114376: tune system dictionary size
Message-ID: <73E0E1AF-1806-439B-B826-AE4E59590BFD@oracle.com>


Please review initial proposal for: 7114376: tune system dictionary size

http://oklahoma.us.oracle.com/~kmkinnea/webrev/sdresize/webrev/

In addition to a code review, I would appreciate feedback on the tuning flags.

The underlying goal is to optimize performance for customers loading lots of classes
by dynamically resizing the system dictionary (loaded class cache) to reduce lookup times for "find" calls.
Today the system dictionary is implemented as a hash table of buckets and all loaded classes that hash
into the bucket are linked off of that bucket. 

average_depth = # loaded classes/number of buckets.
average_lookup_length = number of links walked/number of lookups

If you load 10,000 classes with the default 1009 buckets, you have an average_depth of 10.

Longer term we will want to enhance this to allow trade-offs between customers who want
to optimize for minimum size and those who want faster performance, so please keep that in
mind - if there is a way to specify that in the interface now, without implementing the shrinking 
heuristics yet, that would be appreciated.

Proposal:

1) Dynamic resizing
If the lookup path for classes is "too long", then the system dictionary is automatically resized.

2) Tunable heuristics
  1. starting size
      -XX:SystemDictionarySizeIndex (0-7)
      This sets the initial system dictionary size. The sizes you get are:
       1009 (default = current value), 2017, 4049, 50501, 10103, 20201, 40423, 99991

   2. "too long"
       The default value for too long was taken from the existing verification assertion which used to
       print a warning, i.e. if (average_lookup_length) > 2 * (average_depth).
       Actually, unless you have just unloaded > 20% of the current classes.

   3. frequency of checking
       This prototype checks for resizing when we check for class_unloading, which I believe is
       called during full GCs.
       I added the flag SystemDictionaryAverageDepth=# to allow running benchmarks to see
       if there is too much overhead if we check at every safepoint. I am gathering those results in parallel
       with this feedback request. My current goal is to remove this flag and check at every safepoint
       if this doesnt' cost too much.

   4. Total allowable sizes
      The max allowable size with the prototype is 99991. Do we need larger numbers today?
       If you load 1 million classes you would have an average_depth of ~10.

    5. I changed -XX:+PrintSystemDictionaryAtExit to a diagnostic flag so customers could find
        out the number of classes they are loading, and added additional information to that dump.

3) Tuning alternatives
  1. Starting Size
      * The algorithm for hashing optimizes even distribution for a table with a prime number of buckets.
      Would customers prefer to enter their own explicit table size, which may or may not be a prime number,
      rather than selecting a size index?

       * OR Would it be easier for backward compatibility as we allow both smaller and larger sizes if the customer
       were to specify:
       XS, S, M, L, XL for instance - and this would allow us to add XXS, XXL, etc. to the command-line later
       and to fit additional incremental sizes for dynamic tuning in between without changing the required command-line?
        - I prefer this approach, but would like feedback.

   2. Resize or Not? Or Allow/disallow Growing? Allow/disallow Shrinking? Optimize for small size? Optimize for speed?
       Do we need different and/or better controls? Suggestions please?
       For the optimize for speed/optimize for small size - I would suggest that there might be more general flags that
       specify those in the future, and that the system dictionary resizing take those more general flags into account, rather than
       having specific flags for optimizing the system dictionary size/look up speed. 

   3. SystemDictionaryAverageDepth=#
       Should I remove this - is this adding a level of unnecessary complexity?
       Or should I leave this and use it to have the dynamic resize use this to determine the new size
       rather than simply incrementing each time? My preference is to remove this at this time to simplify
       the implementation until we have more experience in the field.

thanks,
Karen

From john.coomes at oracle.com  Fri Jan 20 21:36:13 2012
From: john.coomes at oracle.com (john.coomes at oracle.com)
Date: Sat, 21 Jan 2012 05:36:13 +0000
Subject: hg: hsx/hsx23/hotspot: 15 new changesets
Message-ID: <20120121053646.79D0E470F5@hg.openjdk.java.net>

Changeset: 338d438ee229
Author:    katleman
Date:      2012-01-20 13:08 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/338d438ee229

Added tag jdk8-b22 for changeset 24727fb37561

! .hgtags

Changeset: 4e80db53c323
Author:    amurillo
Date:      2012-01-14 00:52 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/4e80db53c323

7129512: new hotspot build - hs23-b11
Reviewed-by: jcoomes

! make/hotspot_version

Changeset: 94ec88ca68e2
Author:    phh
Date:      2012-01-11 17:34 -0500
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/94ec88ca68e2

7115199: Add event tracing hooks and Java Flight Recorder infrastructure
Summary: Added a nop tracing infrastructure, JFR makefile changes and other infrastructure used only by JFR.
Reviewed-by: acorn, sspitsyn
Contributed-by: markus.gronlund at oracle.com

! make/Makefile
! make/bsd/makefiles/vm.make
! make/defs.make
! make/linux/makefiles/vm.make
! make/solaris/makefiles/vm.make
! make/windows/build.bat
! make/windows/create_obj_files.sh
! make/windows/makefiles/projectcreator.make
! make/windows/makefiles/vm.make
! src/share/vm/classfile/symbolTable.cpp
! src/share/vm/classfile/symbolTable.hpp
! src/share/vm/classfile/systemDictionary.cpp
! src/share/vm/oops/klass.cpp
! src/share/vm/oops/klass.hpp
! src/share/vm/oops/methodKlass.cpp
! src/share/vm/oops/methodOop.hpp
! src/share/vm/prims/jni.cpp
+ src/share/vm/prims/jniExport.hpp
! src/share/vm/runtime/java.cpp
! src/share/vm/runtime/mutexLocker.cpp
! src/share/vm/runtime/mutexLocker.hpp
! src/share/vm/runtime/os.cpp
! src/share/vm/runtime/thread.cpp
! src/share/vm/runtime/thread.hpp
! src/share/vm/runtime/vm_operations.hpp
+ src/share/vm/trace/traceEventTypes.hpp
+ src/share/vm/trace/traceMacros.hpp
+ src/share/vm/trace/tracing.hpp
! src/share/vm/utilities/globalDefinitions.hpp

Changeset: 4f3ce9284781
Author:    phh
Date:      2012-01-11 17:58 -0500
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/4f3ce9284781

Merge

! src/share/vm/oops/klass.cpp
! src/share/vm/oops/klass.hpp

Changeset: f1cd52d6ce02
Author:    kamg
Date:      2012-01-17 10:16 -0500
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/f1cd52d6ce02

Merge


Changeset: d7e3846464d0
Author:    zgu
Date:      2012-01-17 13:08 -0500
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/d7e3846464d0

7071311: Decoder enhancement
Summary: Made decoder thread-safe
Reviewed-by: coleenp, kamg

- src/os/bsd/vm/decoder_bsd.cpp
+ src/os/bsd/vm/decoder_machO.cpp
+ src/os/bsd/vm/decoder_machO.hpp
! src/os/linux/vm/decoder_linux.cpp
! src/os/linux/vm/os_linux.cpp
! src/os/solaris/vm/decoder_solaris.cpp
! src/os/solaris/vm/os_solaris.cpp
! src/os/windows/vm/decoder_windows.cpp
+ src/os/windows/vm/decoder_windows.hpp
! src/os/windows/vm/os_windows.cpp
! src/share/vm/utilities/decoder.cpp
! src/share/vm/utilities/decoder.hpp
+ src/share/vm/utilities/decoder_elf.cpp
+ src/share/vm/utilities/decoder_elf.hpp
! src/share/vm/utilities/elfFile.cpp
! src/share/vm/utilities/elfFile.hpp
! src/share/vm/utilities/elfStringTable.cpp
! src/share/vm/utilities/elfStringTable.hpp
! src/share/vm/utilities/elfSymbolTable.cpp
! src/share/vm/utilities/elfSymbolTable.hpp
! src/share/vm/utilities/vmError.cpp

Changeset: 6520f9861937
Author:    kamg
Date:      2012-01-17 21:25 -0500
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/6520f9861937

Merge


Changeset: db18ca98d237
Author:    zgu
Date:      2012-01-18 11:45 -0500
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/db18ca98d237

7131050: fix for "7071311 Decoder enhancement" does not build on MacOS X
Summary: Decoder API changes did not reflect in os_bsd
Reviewed-by: kamg, dcubed

! src/os/bsd/vm/os_bsd.cpp

Changeset: eaa9557116a2
Author:    bdelsart
Date:      2012-01-18 16:18 +0100
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/eaa9557116a2

7120448: Fix FP values for compiled frames in frame::describe
Summary: fix for debug method frame::describe
Reviewed-by: never, kvn

! src/cpu/sparc/vm/frame_sparc.inline.hpp
! src/cpu/x86/vm/frame_x86.cpp
! src/cpu/x86/vm/frame_x86.hpp
! src/cpu/zero/vm/frame_zero.inline.hpp
! src/share/vm/runtime/frame.cpp
! src/share/vm/runtime/frame.hpp

Changeset: 15d394228cfa
Author:    jrose
Date:      2012-01-19 13:00 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/15d394228cfa

7111138: delete the obsolete flag -XX:+UseRicochetFrames
Reviewed-by: dholmes, bdelsart, kvn, twisti

! src/cpu/sparc/vm/methodHandles_sparc.cpp
! src/cpu/x86/vm/methodHandles_x86.cpp
! src/cpu/zero/vm/methodHandles_zero.hpp
! src/share/vm/prims/methodHandles.cpp
! src/share/vm/prims/methodHandles.hpp
! src/share/vm/runtime/globals.hpp
! src/share/vm/runtime/sharedRuntime.cpp

Changeset: 898522ae3c32
Author:    iveresov
Date:      2012-01-19 10:56 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/898522ae3c32

7131288: COMPILE SKIPPED: deopt handler overflow (retry at different tier)
Summary: Fix exception handler stub size, enable guarantees to check for the correct deopt and exception stub sizes in the future
Reviewed-by: kvn, never, twisti

! src/cpu/sparc/vm/c1_LIRAssembler_sparc.cpp
! src/cpu/sparc/vm/c1_LIRAssembler_sparc.hpp
! src/cpu/x86/vm/c1_LIRAssembler_x86.cpp

Changeset: 469e0a46f2fe
Author:    jrose
Date:      2012-01-19 17:20 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/469e0a46f2fe

Merge


Changeset: 50d9b7a0072c
Author:    jrose
Date:      2012-01-19 18:35 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/50d9b7a0072c

Merge


Changeset: dcc292399a39
Author:    amurillo
Date:      2012-01-20 16:56 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/dcc292399a39

Merge

- src/os/bsd/vm/decoder_bsd.cpp

Changeset: e850d8e7ea54
Author:    amurillo
Date:      2012-01-20 16:56 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/e850d8e7ea54

Added tag hs23-b11 for changeset dcc292399a39

! .hgtags


From john.coomes at oracle.com  Sat Jan 21 02:26:07 2012
From: john.coomes at oracle.com (john.coomes at oracle.com)
Date: Sat, 21 Jan 2012 10:26:07 +0000
Subject: hg: hsx/hotspot-main/hotspot: 4 new changesets
Message-ID: <20120121102624.7EC6B470F9@hg.openjdk.java.net>

Changeset: 338d438ee229
Author:    katleman
Date:      2012-01-20 13:08 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/338d438ee229

Added tag jdk8-b22 for changeset 24727fb37561

! .hgtags

Changeset: dcc292399a39
Author:    amurillo
Date:      2012-01-20 16:56 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/dcc292399a39

Merge

- src/os/bsd/vm/decoder_bsd.cpp

Changeset: e850d8e7ea54
Author:    amurillo
Date:      2012-01-20 16:56 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/e850d8e7ea54

Added tag hs23-b11 for changeset dcc292399a39

! .hgtags

Changeset: 5f3fcd591768
Author:    amurillo
Date:      2012-01-20 17:07 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/5f3fcd591768

7131979: new hotspot build - hs23-b12
Reviewed-by: jcoomes

! make/hotspot_version


From james.melvin at oracle.com  Sat Jan 21 07:01:51 2012
From: james.melvin at oracle.com (James Melvin)
Date: Sat, 21 Jan 2012 10:01:51 -0500
Subject: RFR (S): 7125793: MAC: test_gamma should always work
In-Reply-To: <F9C3D956-42B9-4B90-BFF5-2250FFA429F9@oracle.com>
References: <4EFECA5D.6010905@oracle.com> <4F034B51.3070609@oracle.com>
	<4F087516.40505@oracle.com> <4F0B7A34.5070406@oracle.com>
	<4F0CA809.7020001@oracle.com>
	<F9C3D956-42B9-4B90-BFF5-2250FFA429F9@oracle.com>
Message-ID: <4F1AD35F.70005@oracle.com>

Staffan,

Thanks for the review.  Coming back to this work after a few other
initiatives.  Comments inline...

On 1/20/12 2:17 PM, Staffan Larsen wrote:
> Sorry if I am late to the game, but I was just wishing for this
> functionality - and there was the web rev!
>
> I've looked at this version of the patch:
> http://cr.openjdk.java.net/~jmelvin/7125793/webrev.03/
>
> It looks good. Only two small comments:
>
> java_md.c:GetJVMPath - I don't think the code inside "#ifndef GAMMA"
> needs to be changed. The code is always compiled with GAMMA defined. The
> "#ifndef GAMMA" is just there to facilitate comparison with the original
> code in the JDK.

Correct. The code in the jdk has also changed to something similar, but
with ifdefs instead of code logic. I'm not sure of the history behind
the gamma launcher and how much effort has been expended to keep the jdk
and hotspot launchers in sync in these sections. There are additional
changes proposed to the jdk launcher. They are in review at the moment.
I'd like to keep the current code in place and revise later to reflect
the jdk changes when the dust settles.

>
> os_bsd.cpp:2580 - (nit) incorrect indentation

Hmmm... This must be a webrev/browser thing. I'm looking in xemacs and
the webrev via Firefox on Mac and I don't see an indentation problem.
I'll send you the source file in a private email to inspect. Let me know
if you still see the problem.

>
> I have also verified that the "hotspot" launcher works after this change.

Great! Note, the gamma launcher expects an OpenJDK JAVA_HOME layout.
Therefore, it will not work with the Apple layout. This is detected in
the test_gamma script and a more appropriate error message is displayed.
Just change JAVA_HOME and you will see what I mean.

Again, thanks for the review!

Jim


>
> /Staffan
>
>
> On 10 jan 2012, at 22:05, James Melvin wrote:
>
>> Hi Dan,
>>
>> Final webrev to reflect your comments...
>>
>> http://cr.openjdk.java.net/~jmelvin/7125793/webrev.02
>>
>> Minor changes this round:
>>
>> make/bsd/makefiles/buildtree.make # Fail gracefully on Apple BOOTDIR
>> make/bsd/makefiles/launcher.make # Link with framework only on Mac
>> src/os/bsd/vm/os_bsd.cpp # Just spelling fix
>>
>> Lastly, I wanted to reply to John Coomes comments earlier about the
>> test_gamma script simplification. Although I also value economy of
>> expression, in this case I think the use of more advanced shell
>> constructs increases the time for fresh eyes to decipher. Given
>> performance and such is not an issue, I'd prefer to keep the simpler
>> version I'm proposing with this change on Mac OS X, to make it easier on
>> future maintenance. This should be a model for the other platforms when
>> we reconcile. I've attached the before and after copies should there be
>> further comments on the simplified short script.
>>
>> Thanks,
>>
>> Jim
>>
>>
>> On 1/9/12 6:37 PM, Daniel D. Daugherty wrote:
>>> On 1/7/12 9:38 AM, James Melvin wrote:
>>>> WEBREV:
>>>> http://cr.openjdk.java.net/~jmelvin/7125793/webrev.01
>>>
>>> make/bsd/Makefile
>>> No comments.
>>>
>>> make/bsd/makefiles/buildtree.make
>>> No comments.
>>>
>>> make/bsd/makefiles/defs.make
>>> Thanks for fixing this one for BSD platforms.
>>>
>>> make/bsd/makefiles/launcher.make
>>> line 60: typo: 'inadvertenly' -> 'inadvertently'
>>>
>>> Sorry I missed this in my first review, but the addition
>>> of '-framework CoreFoundation' to LFLAGS_LAUNCHER is
>>> probably MacOS X specific. I think:
>>>
>>> ifeq ($(OS_VENDOR), Darwin)
>>> else
>>> endif
>>>
>>> will work in launcher.make also.
>>>
>>> make/bsd/makefiles/vm.make
>>> No comments.
>>>
>>> src/os/bsd/vm/os_bsd.cpp
>>> line 2544: typo: 'overriden' -> 'overridden'
>>> line 2588: typo: 'overriden' -> 'overridden'
>>>
>>> Looks like old code line 2576 depended on the 'hotspot'
>>> symlink to refer to either 'client' or 'server' or whatever
>>> JVM you wanted to run. I'm fairly certain that the 'hotspot'
>>> symlink was retired; I'm just not sure when.
>>>
>>> src/os/posix/launcher/java_md.c
>>> No comments.
>>>
>>> Dan
>>>
>>>
>> <test_gamma.before><test_gamma.after>
>

From schlosna at gmail.com  Sat Jan 21 09:04:06 2012
From: schlosna at gmail.com (David Schlosnagle)
Date: Sat, 21 Jan 2012 12:04:06 -0500
Subject: Code and Design Feedback request: 7114376: tune system dictionary
	size
In-Reply-To: <73E0E1AF-1806-439B-B826-AE4E59590BFD@oracle.com>
References: <73E0E1AF-1806-439B-B826-AE4E59590BFD@oracle.com>
Message-ID: <CAGVnyzRHxg8SD5kx4duFCMuf7xzzaBGng_CBxC5n+DfmrzekZw@mail.gmail.com>

On Fri, Jan 20, 2012 at 6:06 PM, Karen Kinnear <karen.kinnear at oracle.com> wrote:
> Please review initial proposal for: 7114376: tune system dictionary size
>
> http://oklahoma.us.oracle.com/~kmkinnea/webrev/sdresize/webrev/

Karen,

Is there a URL where this webrev is accessible outside of Oracle?

Thanks,
Dave

From karen.kinnear at oracle.com  Sat Jan 21 11:04:06 2012
From: karen.kinnear at oracle.com (Karen Kinnear)
Date: Sat, 21 Jan 2012 14:04:06 -0500
Subject: Code and Design Feedback request: 7114376: tune system dictionary
	size
In-Reply-To: <CAGVnyzRHxg8SD5kx4duFCMuf7xzzaBGng_CBxC5n+DfmrzekZw@mail.gmail.com>
References: <73E0E1AF-1806-439B-B826-AE4E59590BFD@oracle.com>
	<CAGVnyzRHxg8SD5kx4duFCMuf7xzzaBGng_CBxC5n+DfmrzekZw@mail.gmail.com>
Message-ID: <F45DFA65-925A-446D-99A4-1F0D7AEF8496@oracle.com>

David,

Thank you for asking, my mistake. Here is an accessible webrev.

Webrev: http://cr.openjdk.java.net/~acorn/7114376/webrev/

An alternative possibility for supplying a single flag might be: -XX:LoadedClasses=#

This could be an anticipated count of expected total classes, which we could use
internally to set the initial SystemDictionary size. This might outlast any internal changes
in data structures.

Tests run:
vmsqe: nsk.quick.testlist: solaris-sparc, fastdbg - this includes sa testing
jck: vm,lang,api: solaris-sparc, fastdbg
JPRT - this includes CDS testing
performance runs in progress, including frequency of checking need to resize

thanks,
Karen

On Jan 21, 2012, at 12:04 PM, David Schlosnagle wrote:

> On Fri, Jan 20, 2012 at 6:06 PM, Karen Kinnear <karen.kinnear at oracle.com> wrote:
>> Please review initial proposal for: 7114376: tune system dictionary size
>> 
>> http://oklahoma.us.oracle.com/~kmkinnea/webrev/sdresize/webrev/
> 
> Karen,
> 
> Is there a URL where this webrev is accessible outside of Oracle?
> 
> Thanks,
> Dave


From staffan.larsen at oracle.com  Sun Jan 22 03:43:34 2012
From: staffan.larsen at oracle.com (Staffan Larsen)
Date: Sun, 22 Jan 2012 12:43:34 +0100
Subject: RFR (S): 7125793: MAC: test_gamma should always work
In-Reply-To: <4F1AD35F.70005@oracle.com>
References: <4EFECA5D.6010905@oracle.com> <4F034B51.3070609@oracle.com>
	<4F087516.40505@oracle.com> <4F0B7A34.5070406@oracle.com>
	<4F0CA809.7020001@oracle.com>
	<F9C3D956-42B9-4B90-BFF5-2250FFA429F9@oracle.com>
	<4F1AD35F.70005@oracle.com>
Message-ID: <3955B287-2568-4937-855B-72A6CD003E07@oracle.com>

Inline.
 
On 21 jan 2012, at 16:01, James Melvin wrote:
>> 
>> java_md.c:GetJVMPath - I don't think the code inside "#ifndef GAMMA"
>> needs to be changed. The code is always compiled with GAMMA defined. The
>> "#ifndef GAMMA" is just there to facilitate comparison with the original
>> code in the JDK.
> 
> Correct. The code in the jdk has also changed to something similar, but
> with ifdefs instead of code logic. I'm not sure of the history behind
> the gamma launcher and how much effort has been expended to keep the jdk
> and hotspot launchers in sync in these sections. There are additional
> changes proposed to the jdk launcher. They are in review at the moment.
> I'd like to keep the current code in place and revise later to reflect
> the jdk changes when the dust settles.
> 

I think the last update was about a year ago, when I rebased the launcher code in hotspot to 6u22 code. At that point, I believe the only changes to the code was inside "#ifdef GAMMA". 

>> 
>> os_bsd.cpp:2580 - (nit) incorrect indentation
> 
> Hmmm... This must be a webrev/browser thing. I'm looking in xemacs and
> the webrev via Firefox on Mac and I don't see an indentation problem.
> I'll send you the source file in a private email to inspect. Let me know
> if you still see the problem.

Could be a tabsize issue. I think the line was preceded by a tab instead of spaces.
 
Thanks,
/Staffan

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120122/3105ea71/attachment.html 

From james.melvin at oracle.com  Sun Jan 22 17:43:13 2012
From: james.melvin at oracle.com (James Melvin)
Date: Sun, 22 Jan 2012 20:43:13 -0500
Subject: RFR (S): 7125793: MAC: test_gamma should always work
In-Reply-To: <3955B287-2568-4937-855B-72A6CD003E07@oracle.com>
References: <4EFECA5D.6010905@oracle.com> <4F034B51.3070609@oracle.com>
	<4F087516.40505@oracle.com> <4F0B7A34.5070406@oracle.com>
	<4F0CA809.7020001@oracle.com>
	<F9C3D956-42B9-4B90-BFF5-2250FFA429F9@oracle.com>
	<4F1AD35F.70005@oracle.com>
	<3955B287-2568-4937-855B-72A6CD003E07@oracle.com>
Message-ID: <4F1CBB31.7040103@oracle.com>

Hi Staffan,

Comments inline...

On 1/22/12 6:43 AM, Staffan Larsen wrote:
> Inline.
>
> On 21 jan 2012, at 16:01, James Melvin wrote:
>>>
>>> java_md.c:GetJVMPath - I don't think the code inside "#ifndef GAMMA"
>>> needs to be changed. The code is always compiled with GAMMA defined. The
>>> "#ifndef GAMMA" is just there to facilitate comparison with the original
>>> code in the JDK.
>>
>> Correct. The code in the jdk has also changed to something similar, but
>> with ifdefs instead of code logic. I'm not sure of the history behind
>> the gamma launcher and how much effort has been expended to keep the jdk
>> and hotspot launchers in sync in these sections. There are additional
>> changes proposed to the jdk launcher. They are in review at the moment.
>> I'd like to keep the current code in place and revise later to reflect
>> the jdk changes when the dust settles.
>>
>
> I think the last update was about a year ago, when I rebased the
> launcher code in hotspot to 6u22 code. At that point, I believe the only
> changes to the code was inside "#ifdef GAMMA".

OK, I agree.  I've reverted the code in the #ifndef GAMMA section.  We
can rebase after the JDK launcher changes.  No problem.


>>> os_bsd.cpp:2580 - (nit) incorrect indentation
>>
>> Hmmm... This must be a webrev/browser thing. I'm looking in xemacs and
>> the webrev via Firefox on Mac and I don't see an indentation problem.
>> I'll send you the source file in a private email to inspect. Let me know
>> if you still see the problem.
>
> Could be a tabsize issue. I think the line was preceded by a tab instead
> of spaces.

Indeed, there was a tab at the beginning of the line.  Hg jcheck would
have discovered it after commit.  Thanks!

New webrev and JPRT available, including builds on Mac OS X...

WEBREV:  http://cr.openjdk.java.net/~jmelvin/7125793/webrev.04
JPRT: 
http://prt-web.us.oracle.com//archive/2012/01/2012-01-22-205611.jmelvin.7125793/

Thanks,

Jim

> Thanks,
> /Staffan
>

From david.holmes at oracle.com  Sun Jan 22 17:50:20 2012
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 23 Jan 2012 11:50:20 +1000
Subject: Code and Design Feedback request: 7114376: tune system dictionary
	size
In-Reply-To: <F45DFA65-925A-446D-99A4-1F0D7AEF8496@oracle.com>
References: <73E0E1AF-1806-439B-B826-AE4E59590BFD@oracle.com>
	<CAGVnyzRHxg8SD5kx4duFCMuf7xzzaBGng_CBxC5n+DfmrzekZw@mail.gmail.com>
	<F45DFA65-925A-446D-99A4-1F0D7AEF8496@oracle.com>
Message-ID: <4F1CBCDC.4060808@oracle.com>

Hi Karen,

On 22/01/2012 5:04 AM, Karen Kinnear wrote:
> Thank you for asking, my mistake. Here is an accessible webrev.
>
> Webrev: http://cr.openjdk.java.net/~acorn/7114376/webrev/

systemDictionary.cpp:

I notice that in systemDictionary the calculation of the index has been 
moved inside the locked regions. This impacted some of the SD method 
signatures, depending on whether they grabbed the lock internally, or 
whether the lock was first acquired externally. Seems to me we could 
avoid a lot of duplicate index calculating code if we went a step 
further and changed all the methods to only take the hash and do their 
own index calculation if needed - in particular if find_class did the 
calculation then I think most of the other index calculations would 
disappear.

1757 // Check if system dictionary should be resized to speed up find() 
calls
1758 // Default is to check on class unloading
1759 // SystemDictionarySizeIndex flag allows starting with a larger 
initial size
1760 // if SystemDictionaryAverageDepth is non-zero, also check on 
safepoint cleanup
1761 // and allow customers to select acceptable average depth to pass
1762 // to verify_lookup_length, otherwise use number_of_classes/table_size
1763 // verify_lookup_length compares actual average lookup depth: 
lookup_length/lookup_count
1764 // with theoretical or user-specified.
1765 // In future we may need to periodically calculate lookup_length to see
1766 // if we need to force a safepoint to resize system dictionary
1767 void SystemDictionary::should_resize_dictionary(int unloaded_count) {

I found the above comment block difficult to parse - perhaps some 
Capitals missing to indicate new points/sentences?

Also should this method assert that either the SD lock is held or else 
we're at a safepoint?

---

dictionary.hpp/cpp

+   // Change size of system dictionary
+   // Initially for growing when lookup time takes too long
+   Dictionary* dictionary_resize(int newsize);

Is "system dictionary" distinct from SystemDictionary?

I don't like having this as an instance method as it doesn't resize the 
current dictionary but returns a new-one, leaving the caller to free the 
original, yet has the side-effect of clearing all entries from the 
current one. Seems to me we should simply have the caller do:

Dictionary* newdict = new Dictionary(newsize);
olddict->copy_all_entries(newdict);
delete olddict;
olddict = newdict;

though perhaps copy_all_entries should really be move_all_entries?

  477 // This routine does not lock the system dictionary.
  ...
  487 // Actually, the locking here is a bit trickier:
  ...
  493 DictionaryEntry* Dictionary::get_entry(int index, unsigned int hash,
  494                                        Symbol* class_name,
  495                                        Handle class_loader) {
  496   oop loader = class_loader();
  497   _lookup_count++;
  498
  499   for (DictionaryEntry* entry = bucket(index);
  500                         entry != NULL;
  501                         entry = entry->next()) {
  502     if (entry->hash() == hash && entry->equals(class_name, loader)) {
  503       return entry;
  504     }
  505     _lookup_length++;

Your now non-debug counters seem to be being updated in a non-threadsafe 
manner - or are we assuming the lock has been taken in the caller?

> An alternative possibility for supplying a single flag might be: -XX:LoadedClasses=#
>
> This could be an anticipated count of expected total classes, which we could use
> internally to set the initial SystemDictionary size. This might outlast any internal changes
> in data structures.

I think this might be a simpler starting point for things. Dynamically 
resizing the SD is transiently a bad thing for everyone:
- pause while it happens
- excessive memory use while it happens

better in my opinion to try to get the initial number of buckets right 
and not dynamically resize.

Cheers,
David
-----

> Tests run:
> vmsqe: nsk.quick.testlist: solaris-sparc, fastdbg - this includes sa testing
> jck: vm,lang,api: solaris-sparc, fastdbg
> JPRT - this includes CDS testing
> performance runs in progress, including frequency of checking need to resize
>
> thanks,
> Karen
>
> On Jan 21, 2012, at 12:04 PM, David Schlosnagle wrote:
>
>> On Fri, Jan 20, 2012 at 6:06 PM, Karen Kinnear<karen.kinnear at oracle.com>  wrote:
>>> Please review initial proposal for: 7114376: tune system dictionary size
>>>
>>> http://oklahoma.us.oracle.com/~kmkinnea/webrev/sdresize/webrev/
>>
>> Karen,
>>
>> Is there a URL where this webrev is accessible outside of Oracle?
>>
>> Thanks,
>> Dave
>

From staffan.larsen at oracle.com  Sun Jan 22 23:04:09 2012
From: staffan.larsen at oracle.com (Staffan Larsen)
Date: Mon, 23 Jan 2012 08:04:09 +0100
Subject: RFR (S): 7125793: MAC: test_gamma should always work
In-Reply-To: <4F1CBB31.7040103@oracle.com>
References: <4EFECA5D.6010905@oracle.com> <4F034B51.3070609@oracle.com>
	<4F087516.40505@oracle.com> <4F0B7A34.5070406@oracle.com>
	<4F0CA809.7020001@oracle.com>
	<F9C3D956-42B9-4B90-BFF5-2250FFA429F9@oracle.com>
	<4F1AD35F.70005@oracle.com>
	<3955B287-2568-4937-855B-72A6CD003E07@oracle.com>
	<4F1CBB31.7040103@oracle.com>
Message-ID: <914DDA21-DD6A-416D-A6A0-6C0B13DA8E7B@oracle.com>

Looks great!

On 23 jan 2012, at 02:43, James Melvin wrote:

> Hi Staffan,
> 
> Comments inline...
> 
> On 1/22/12 6:43 AM, Staffan Larsen wrote:
>> Inline.
>> 
>> On 21 jan 2012, at 16:01, James Melvin wrote:
>>>> 
>>>> java_md.c:GetJVMPath - I don't think the code inside "#ifndef GAMMA"
>>>> needs to be changed. The code is always compiled with GAMMA defined. The
>>>> "#ifndef GAMMA" is just there to facilitate comparison with the original
>>>> code in the JDK.
>>> 
>>> Correct. The code in the jdk has also changed to something similar, but
>>> with ifdefs instead of code logic. I'm not sure of the history behind
>>> the gamma launcher and how much effort has been expended to keep the jdk
>>> and hotspot launchers in sync in these sections. There are additional
>>> changes proposed to the jdk launcher. They are in review at the moment.
>>> I'd like to keep the current code in place and revise later to reflect
>>> the jdk changes when the dust settles.
>>> 
>> 
>> I think the last update was about a year ago, when I rebased the
>> launcher code in hotspot to 6u22 code. At that point, I believe the only
>> changes to the code was inside "#ifdef GAMMA".
> 
> OK, I agree.  I've reverted the code in the #ifndef GAMMA section.  We
> can rebase after the JDK launcher changes.  No problem.
> 
> 
>>>> os_bsd.cpp:2580 - (nit) incorrect indentation
>>> 
>>> Hmmm... This must be a webrev/browser thing. I'm looking in xemacs and
>>> the webrev via Firefox on Mac and I don't see an indentation problem.
>>> I'll send you the source file in a private email to inspect. Let me know
>>> if you still see the problem.
>> 
>> Could be a tabsize issue. I think the line was preceded by a tab instead
>> of spaces.
> 
> Indeed, there was a tab at the beginning of the line.  Hg jcheck would
> have discovered it after commit.  Thanks!
> 
> New webrev and JPRT available, including builds on Mac OS X...
> 
> WEBREV:  http://cr.openjdk.java.net/~jmelvin/7125793/webrev.04
> JPRT: http://prt-web.us.oracle.com//archive/2012/01/2012-01-22-205611.jmelvin.7125793/
> 
> Thanks,
> 
> Jim
> 
>> Thanks,
>> /Staffan
>> 


From karen.kinnear at oracle.com  Mon Jan 23 06:47:22 2012
From: karen.kinnear at oracle.com (Karen Kinnear)
Date: Mon, 23 Jan 2012 09:47:22 -0500
Subject: Code and Design Feedback request: 7114376: tune system dictionary
	size
In-Reply-To: <4F1CBCDC.4060808@oracle.com>
References: <73E0E1AF-1806-439B-B826-AE4E59590BFD@oracle.com>
	<CAGVnyzRHxg8SD5kx4duFCMuf7xzzaBGng_CBxC5n+DfmrzekZw@mail.gmail.com>
	<F45DFA65-925A-446D-99A4-1F0D7AEF8496@oracle.com>
	<4F1CBCDC.4060808@oracle.com>
Message-ID: <E2FD2C70-3F3B-4EB1-84E6-CFA81F17AFA1@oracle.com>

David,

Thank you for the detailed feedback. More below.

On Jan 22, 2012, at 8:50 PM, David Holmes wrote:

> Hi Karen,
> 
> On 22/01/2012 5:04 AM, Karen Kinnear wrote:
>> Thank you for asking, my mistake. Here is an accessible webrev.
>> 
>> Webrev: http://cr.openjdk.java.net/~acorn/7114376/webrev/
> 
> systemDictionary.cpp:
> 
> I notice that in systemDictionary the calculation of the index has been moved inside the locked regions. This impacted some of the SD method signatures, depending on whether they grabbed the lock internally, or whether the lock was first acquired externally. Seems to me we could avoid a lot of duplicate index calculating code if we went a step further and changed all the methods to only take the hash and do their own index calculation if needed - in particular if find_class did the calculation then I think most of the other index calculations would disappear.

Great suggestion - I will do that and send for re-review.
> 
> 1757 // Check if system dictionary should be resized to speed up find() calls
> 1758 // Default is to check on class unloading
> 1759 // SystemDictionarySizeIndex flag allows starting with a larger initial size
> 1760 // if SystemDictionaryAverageDepth is non-zero, also check on safepoint cleanup
> 1761 // and allow customers to select acceptable average depth to pass
> 1762 // to verify_lookup_length, otherwise use number_of_classes/table_size
> 1763 // verify_lookup_length compares actual average lookup depth: lookup_length/lookup_count
> 1764 // with theoretical or user-specified.
> 1765 // In future we may need to periodically calculate lookup_length to see
> 1766 // if we need to force a safepoint to resize system dictionary
> 1767 void SystemDictionary::should_resize_dictionary(int unloaded_count) {
> 
> I found the above comment block difficult to parse - perhaps some Capitals missing to indicate new points/sentences?
I am going to simplify the flags to just -XX:LoadedClasses=# so this comment will be rewritten and
sent out for re-review.
> 
> Also should this method assert that either the SD lock is held or else we're at a safepoint?
Will add the assertion that we are at a safepoint.
> 
> ---
> 
> dictionary.hpp/cpp
> 
> +   // Change size of system dictionary
> +   // Initially for growing when lookup time takes too long
> +   Dictionary* dictionary_resize(int newsize);
> 
> Is "system dictionary" distinct from SystemDictionary?
> 
> I don't like having this as an instance method as it doesn't resize the current dictionary but returns a new-one, leaving the caller to free the original, yet has the side-effect of clearing all entries from the current one. Seems to me we should simply have the caller do:
> 
> Dictionary* newdict = new Dictionary(newsize);
> olddict->copy_all_entries(newdict);
> delete olddict;
> olddict = newdict;
> 
> though perhaps copy_all_entries should really be move_all_entries?
I will make that change. dictionary_resize used to do a lot more work, but the way it turned out
your suggestion is much better.
move_all_entries is a better name.
> 
> 477 // This routine does not lock the system dictionary.
> ...
> 487 // Actually, the locking here is a bit trickier:
> ...
> 493 DictionaryEntry* Dictionary::get_entry(int index, unsigned int hash,
> 494                                        Symbol* class_name,
> 495                                        Handle class_loader) {
> 496   oop loader = class_loader();
> 497   _lookup_count++;
> 498
> 499   for (DictionaryEntry* entry = bucket(index);
> 500                         entry != NULL;
> 501                         entry = entry->next()) {
> 502     if (entry->hash() == hash && entry->equals(class_name, loader)) {
> 503       return entry;
> 504     }
> 505     _lookup_length++;
> 
> Your now non-debug counters seem to be being updated in a non-threadsafe manner - or are we assuming the lock has been taken in the caller?
Good catch. This is called without a lock. 
So the question is - should I make this an atomic update, or since these are approximations used for heuristics
only, add a comment that we actually don't need them to be completely accurate? I can measure
cost of the atomic update.
> 
>> An alternative possibility for supplying a single flag might be: -XX:LoadedClasses=#
>> 
>> This could be an anticipated count of expected total classes, which we could use
>> internally to set the initial SystemDictionary size. This might outlast any internal changes
>> in data structures.
> 
> I think this might be a simpler starting point for things. Dynamically resizing the SD is transiently a bad thing for everyone:
> - pause while it happens
> - excessive memory use while it happens
> 
> better in my opinion to try to get the initial number of buckets right and not dynamically resize.
I believe we need both. It is simpler to just start with a better size - for now I will add the -XX:LoadedClasses=#
and use that as a starting point, and the only new flag, for customers that want to add a command-line argument.

Our goal is to have good performance without requiring custom tuning, so the dynamic resizing is valuable for
customers who do not want to figure out their usage and do not want to add a command-line argument.
I will simplify the ergonomic tuning to be based on the dynamically calculated LoadedClasses  as well, so
that if we do resize based on slower lookup times, that we resize to the current need rather than incrementally.

I expect the resizing to be beneficial both for customers with high numbers of loaded classes and I expect
the embedded customers to take advantage of resizing to potentially shrink the system dictionary. And I tried
to add the logic so that this could be reused for resizing other internal metadata structures.
> 
> Cheers,
> David
> -----

thank you for the very helpful feedback,
Karen

> 
>> Tests run:
>> vmsqe: nsk.quick.testlist: solaris-sparc, fastdbg - this includes sa testing
>> jck: vm,lang,api: solaris-sparc, fastdbg
>> JPRT - this includes CDS testing
>> performance runs in progress, including frequency of checking need to resize
>> 
>> thanks,
>> Karen
>> 
>> On Jan 21, 2012, at 12:04 PM, David Schlosnagle wrote:
>> 
>>> On Fri, Jan 20, 2012 at 6:06 PM, Karen Kinnear<karen.kinnear at oracle.com>  wrote:
>>>> Please review initial proposal for: 7114376: tune system dictionary size
>>>> 
>>>> http://oklahoma.us.oracle.com/~kmkinnea/webrev/sdresize/webrev/
>>> 
>>> Karen,
>>> 
>>> Is there a URL where this webrev is accessible outside of Oracle?
>>> 
>>> Thanks,
>>> Dave
>> 


From vitalyd at gmail.com  Mon Jan 23 07:26:17 2012
From: vitalyd at gmail.com (Vitaly Davidovich)
Date: Mon, 23 Jan 2012 10:26:17 -0500
Subject: Code and Design Feedback request: 7114376: tune system dictionary
	size
In-Reply-To: <E2FD2C70-3F3B-4EB1-84E6-CFA81F17AFA1@oracle.com>
References: <73E0E1AF-1806-439B-B826-AE4E59590BFD@oracle.com>
	<CAGVnyzRHxg8SD5kx4duFCMuf7xzzaBGng_CBxC5n+DfmrzekZw@mail.gmail.com>
	<F45DFA65-925A-446D-99A4-1F0D7AEF8496@oracle.com>
	<4F1CBCDC.4060808@oracle.com>
	<E2FD2C70-3F3B-4EB1-84E6-CFA81F17AFA1@oracle.com>
Message-ID: <CAHjP37H4HnvwE5WEvwtFUntcO2wOeuHh+NryhuCNqkYFbUR+Ow@mail.gmail.com>

Hi Karen,

Regarding the counters David mentioned, is there also any concern with them
overflowing, on a long-lived jvm process? Would the code deal with negative
values properly? Or do you not expect this to be a concern? The
lookup_count is incremented on each lookup so that looks a bit suspect to
my uninformed eyes.

As for atomic or not, I think the other issue here is if every bit of
performance is important, might it make sense to have per-cpu counters
(with sufficient cache line padding) and then aggregate them when needed?
This would reduce coherence traffic on these writes.

Vitaly

Sent from my phone
On Jan 23, 2012 9:48 AM, "Karen Kinnear" <karen.kinnear at oracle.com> wrote:

> David,
>
> Thank you for the detailed feedback. More below.
>
> On Jan 22, 2012, at 8:50 PM, David Holmes wrote:
>
> > Hi Karen,
> >
> > On 22/01/2012 5:04 AM, Karen Kinnear wrote:
> >> Thank you for asking, my mistake. Here is an accessible webrev.
> >>
> >> Webrev: http://cr.openjdk.java.net/~acorn/7114376/webrev/
> >
> > systemDictionary.cpp:
> >
> > I notice that in systemDictionary the calculation of the index has been
> moved inside the locked regions. This impacted some of the SD method
> signatures, depending on whether they grabbed the lock internally, or
> whether the lock was first acquired externally. Seems to me we could avoid
> a lot of duplicate index calculating code if we went a step further and
> changed all the methods to only take the hash and do their own index
> calculation if needed - in particular if find_class did the calculation
> then I think most of the other index calculations would disappear.
>
> Great suggestion - I will do that and send for re-review.
> >
> > 1757 // Check if system dictionary should be resized to speed up find()
> calls
> > 1758 // Default is to check on class unloading
> > 1759 // SystemDictionarySizeIndex flag allows starting with a larger
> initial size
> > 1760 // if SystemDictionaryAverageDepth is non-zero, also check on
> safepoint cleanup
> > 1761 // and allow customers to select acceptable average depth to pass
> > 1762 // to verify_lookup_length, otherwise use
> number_of_classes/table_size
> > 1763 // verify_lookup_length compares actual average lookup depth:
> lookup_length/lookup_count
> > 1764 // with theoretical or user-specified.
> > 1765 // In future we may need to periodically calculate lookup_length to
> see
> > 1766 // if we need to force a safepoint to resize system dictionary
> > 1767 void SystemDictionary::should_resize_dictionary(int unloaded_count)
> {
> >
> > I found the above comment block difficult to parse - perhaps some
> Capitals missing to indicate new points/sentences?
> I am going to simplify the flags to just -XX:LoadedClasses=# so this
> comment will be rewritten and
> sent out for re-review.
> >
> > Also should this method assert that either the SD lock is held or else
> we're at a safepoint?
> Will add the assertion that we are at a safepoint.
> >
> > ---
> >
> > dictionary.hpp/cpp
> >
> > +   // Change size of system dictionary
> > +   // Initially for growing when lookup time takes too long
> > +   Dictionary* dictionary_resize(int newsize);
> >
> > Is "system dictionary" distinct from SystemDictionary?
> >
> > I don't like having this as an instance method as it doesn't resize the
> current dictionary but returns a new-one, leaving the caller to free the
> original, yet has the side-effect of clearing all entries from the current
> one. Seems to me we should simply have the caller do:
> >
> > Dictionary* newdict = new Dictionary(newsize);
> > olddict->copy_all_entries(newdict);
> > delete olddict;
> > olddict = newdict;
> >
> > though perhaps copy_all_entries should really be move_all_entries?
> I will make that change. dictionary_resize used to do a lot more work, but
> the way it turned out
> your suggestion is much better.
> move_all_entries is a better name.
> >
> > 477 // This routine does not lock the system dictionary.
> > ...
> > 487 // Actually, the locking here is a bit trickier:
> > ...
> > 493 DictionaryEntry* Dictionary::get_entry(int index, unsigned int hash,
> > 494                                        Symbol* class_name,
> > 495                                        Handle class_loader) {
> > 496   oop loader = class_loader();
> > 497   _lookup_count++;
> > 498
> > 499   for (DictionaryEntry* entry = bucket(index);
> > 500                         entry != NULL;
> > 501                         entry = entry->next()) {
> > 502     if (entry->hash() == hash && entry->equals(class_name, loader)) {
> > 503       return entry;
> > 504     }
> > 505     _lookup_length++;
> >
> > Your now non-debug counters seem to be being updated in a non-threadsafe
> manner - or are we assuming the lock has been taken in the caller?
> Good catch. This is called without a lock.
> So the question is - should I make this an atomic update, or since these
> are approximations used for heuristics
> only, add a comment that we actually don't need them to be completely
> accurate? I can measure
> cost of the atomic update.
> >
> >> An alternative possibility for supplying a single flag might be:
> -XX:LoadedClasses=#
> >>
> >> This could be an anticipated count of expected total classes, which we
> could use
> >> internally to set the initial SystemDictionary size. This might outlast
> any internal changes
> >> in data structures.
> >
> > I think this might be a simpler starting point for things. Dynamically
> resizing the SD is transiently a bad thing for everyone:
> > - pause while it happens
> > - excessive memory use while it happens
> >
> > better in my opinion to try to get the initial number of buckets right
> and not dynamically resize.
> I believe we need both. It is simpler to just start with a better size -
> for now I will add the -XX:LoadedClasses=#
> and use that as a starting point, and the only new flag, for customers
> that want to add a command-line argument.
>
> Our goal is to have good performance without requiring custom tuning, so
> the dynamic resizing is valuable for
> customers who do not want to figure out their usage and do not want to add
> a command-line argument.
> I will simplify the ergonomic tuning to be based on the dynamically
> calculated LoadedClasses  as well, so
> that if we do resize based on slower lookup times, that we resize to the
> current need rather than incrementally.
>
> I expect the resizing to be beneficial both for customers with high
> numbers of loaded classes and I expect
> the embedded customers to take advantage of resizing to potentially shrink
> the system dictionary. And I tried
> to add the logic so that this could be reused for resizing other internal
> metadata structures.
> >
> > Cheers,
> > David
> > -----
>
> thank you for the very helpful feedback,
> Karen
>
> >
> >> Tests run:
> >> vmsqe: nsk.quick.testlist: solaris-sparc, fastdbg - this includes sa
> testing
> >> jck: vm,lang,api: solaris-sparc, fastdbg
> >> JPRT - this includes CDS testing
> >> performance runs in progress, including frequency of checking need to
> resize
> >>
> >> thanks,
> >> Karen
> >>
> >> On Jan 21, 2012, at 12:04 PM, David Schlosnagle wrote:
> >>
> >>> On Fri, Jan 20, 2012 at 6:06 PM, Karen Kinnear<
> karen.kinnear at oracle.com>  wrote:
> >>>> Please review initial proposal for: 7114376: tune system dictionary
> size
> >>>>
> >>>> http://oklahoma.us.oracle.com/~kmkinnea/webrev/sdresize/webrev/
> >>>
> >>> Karen,
> >>>
> >>> Is there a URL where this webrev is accessible outside of Oracle?
> >>>
> >>> Thanks,
> >>> Dave
> >>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120123/65a72b00/attachment-0001.html 

From paul.hohensee at oracle.com  Mon Jan 23 13:24:19 2012
From: paul.hohensee at oracle.com (Paul Hohensee)
Date: Mon, 23 Jan 2012 16:24:19 -0500
Subject: Code and Design Feedback request: 7114376: tune system dictionary
	size
In-Reply-To: <CAHjP37H4HnvwE5WEvwtFUntcO2wOeuHh+NryhuCNqkYFbUR+Ow@mail.gmail.com>
References: <73E0E1AF-1806-439B-B826-AE4E59590BFD@oracle.com>
	<CAGVnyzRHxg8SD5kx4duFCMuf7xzzaBGng_CBxC5n+DfmrzekZw@mail.gmail.com>
	<F45DFA65-925A-446D-99A4-1F0D7AEF8496@oracle.com>
	<4F1CBCDC.4060808@oracle.com>
	<E2FD2C70-3F3B-4EB1-84E6-CFA81F17AFA1@oracle.com>
	<CAHjP37H4HnvwE5WEvwtFUntcO2wOeuHh+NryhuCNqkYFbUR+Ow@mail.gmail.com>
Message-ID: <4F1DD003.1090408@oracle.com>

Per-software-thread counters would be easier to implement.  We don't
really know how many active hw threads we have, and the hw thread
count can change unpredictably or be unknowable (e.g., virtualized
environments).

Paul

On 1/23/12 10:26 AM, Vitaly Davidovich wrote:
>
> Hi Karen,
>
> Regarding the counters David mentioned, is there also any concern with 
> them overflowing, on a long-lived jvm process? Would the code deal 
> with negative values properly? Or do you not expect this to be a 
> concern? The lookup_count is incremented on each lookup so that looks 
> a bit suspect to my uninformed eyes.
>
> As for atomic or not, I think the other issue here is if every bit of 
> performance is important, might it make sense to have per-cpu counters 
> (with sufficient cache line padding) and then aggregate them when 
> needed? This would reduce coherence traffic on these writes.
>
> Vitaly
>
> Sent from my phone
>
> On Jan 23, 2012 9:48 AM, "Karen Kinnear" <karen.kinnear at oracle.com 
> <mailto:karen.kinnear at oracle.com>> wrote:
>
>     David,
>
>     Thank you for the detailed feedback. More below.
>
>     On Jan 22, 2012, at 8:50 PM, David Holmes wrote:
>
>     > Hi Karen,
>     >
>     > On 22/01/2012 5:04 AM, Karen Kinnear wrote:
>     >> Thank you for asking, my mistake. Here is an accessible webrev.
>     >>
>     >> Webrev: http://cr.openjdk.java.net/~acorn/7114376/webrev/
>     <http://cr.openjdk.java.net/%7Eacorn/7114376/webrev/>
>     >
>     > systemDictionary.cpp:
>     >
>     > I notice that in systemDictionary the calculation of the index
>     has been moved inside the locked regions. This impacted some of
>     the SD method signatures, depending on whether they grabbed the
>     lock internally, or whether the lock was first acquired
>     externally. Seems to me we could avoid a lot of duplicate index
>     calculating code if we went a step further and changed all the
>     methods to only take the hash and do their own index calculation
>     if needed - in particular if find_class did the calculation then I
>     think most of the other index calculations would disappear.
>
>     Great suggestion - I will do that and send for re-review.
>     >
>     > 1757 // Check if system dictionary should be resized to speed up
>     find() calls
>     > 1758 // Default is to check on class unloading
>     > 1759 // SystemDictionarySizeIndex flag allows starting with a
>     larger initial size
>     > 1760 // if SystemDictionaryAverageDepth is non-zero, also check
>     on safepoint cleanup
>     > 1761 // and allow customers to select acceptable average depth
>     to pass
>     > 1762 // to verify_lookup_length, otherwise use
>     number_of_classes/table_size
>     > 1763 // verify_lookup_length compares actual average lookup
>     depth: lookup_length/lookup_count
>     > 1764 // with theoretical or user-specified.
>     > 1765 // In future we may need to periodically calculate
>     lookup_length to see
>     > 1766 // if we need to force a safepoint to resize system dictionary
>     > 1767 void SystemDictionary::should_resize_dictionary(int
>     unloaded_count) {
>     >
>     > I found the above comment block difficult to parse - perhaps
>     some Capitals missing to indicate new points/sentences?
>     I am going to simplify the flags to just -XX:LoadedClasses=# so
>     this comment will be rewritten and
>     sent out for re-review.
>     >
>     > Also should this method assert that either the SD lock is held
>     or else we're at a safepoint?
>     Will add the assertion that we are at a safepoint.
>     >
>     > ---
>     >
>     > dictionary.hpp/cpp
>     >
>     > +   // Change size of system dictionary
>     > +   // Initially for growing when lookup time takes too long
>     > +   Dictionary* dictionary_resize(int newsize);
>     >
>     > Is "system dictionary" distinct from SystemDictionary?
>     >
>     > I don't like having this as an instance method as it doesn't
>     resize the current dictionary but returns a new-one, leaving the
>     caller to free the original, yet has the side-effect of clearing
>     all entries from the current one. Seems to me we should simply
>     have the caller do:
>     >
>     > Dictionary* newdict = new Dictionary(newsize);
>     > olddict->copy_all_entries(newdict);
>     > delete olddict;
>     > olddict = newdict;
>     >
>     > though perhaps copy_all_entries should really be move_all_entries?
>     I will make that change. dictionary_resize used to do a lot more
>     work, but the way it turned out
>     your suggestion is much better.
>     move_all_entries is a better name.
>     >
>     > 477 // This routine does not lock the system dictionary.
>     > ...
>     > 487 // Actually, the locking here is a bit trickier:
>     > ...
>     > 493 DictionaryEntry* Dictionary::get_entry(int index, unsigned
>     int hash,
>     > 494                                        Symbol* class_name,
>     > 495                                        Handle class_loader) {
>     > 496   oop loader = class_loader();
>     > 497   _lookup_count++;
>     > 498
>     > 499   for (DictionaryEntry* entry = bucket(index);
>     > 500                         entry != NULL;
>     > 501                         entry = entry->next()) {
>     > 502     if (entry->hash() == hash && entry->equals(class_name,
>     loader)) {
>     > 503       return entry;
>     > 504     }
>     > 505     _lookup_length++;
>     >
>     > Your now non-debug counters seem to be being updated in a
>     non-threadsafe manner - or are we assuming the lock has been taken
>     in the caller?
>     Good catch. This is called without a lock.
>     So the question is - should I make this an atomic update, or since
>     these are approximations used for heuristics
>     only, add a comment that we actually don't need them to be
>     completely accurate? I can measure
>     cost of the atomic update.
>     >
>     >> An alternative possibility for supplying a single flag might
>     be: -XX:LoadedClasses=#
>     >>
>     >> This could be an anticipated count of expected total classes,
>     which we could use
>     >> internally to set the initial SystemDictionary size. This might
>     outlast any internal changes
>     >> in data structures.
>     >
>     > I think this might be a simpler starting point for things.
>     Dynamically resizing the SD is transiently a bad thing for everyone:
>     > - pause while it happens
>     > - excessive memory use while it happens
>     >
>     > better in my opinion to try to get the initial number of buckets
>     right and not dynamically resize.
>     I believe we need both. It is simpler to just start with a better
>     size - for now I will add the -XX:LoadedClasses=#
>     and use that as a starting point, and the only new flag, for
>     customers that want to add a command-line argument.
>
>     Our goal is to have good performance without requiring custom
>     tuning, so the dynamic resizing is valuable for
>     customers who do not want to figure out their usage and do not
>     want to add a command-line argument.
>     I will simplify the ergonomic tuning to be based on the
>     dynamically calculated LoadedClasses  as well, so
>     that if we do resize based on slower lookup times, that we resize
>     to the current need rather than incrementally.
>
>     I expect the resizing to be beneficial both for customers with
>     high numbers of loaded classes and I expect
>     the embedded customers to take advantage of resizing to
>     potentially shrink the system dictionary. And I tried
>     to add the logic so that this could be reused for resizing other
>     internal metadata structures.
>     >
>     > Cheers,
>     > David
>     > -----
>
>     thank you for the very helpful feedback,
>     Karen
>
>     >
>     >> Tests run:
>     >> vmsqe: nsk.quick.testlist: solaris-sparc, fastdbg - this
>     includes sa testing
>     >> jck: vm,lang,api: solaris-sparc, fastdbg
>     >> JPRT - this includes CDS testing
>     >> performance runs in progress, including frequency of checking
>     need to resize
>     >>
>     >> thanks,
>     >> Karen
>     >>
>     >> On Jan 21, 2012, at 12:04 PM, David Schlosnagle wrote:
>     >>
>     >>> On Fri, Jan 20, 2012 at 6:06 PM, Karen
>     Kinnear<karen.kinnear at oracle.com
>     <mailto:karen.kinnear at oracle.com>>  wrote:
>     >>>> Please review initial proposal for: 7114376: tune system
>     dictionary size
>     >>>>
>     >>>>
>     http://oklahoma.us.oracle.com/~kmkinnea/webrev/sdresize/webrev/
>     <http://oklahoma.us.oracle.com/%7Ekmkinnea/webrev/sdresize/webrev/>
>     >>>
>     >>> Karen,
>     >>>
>     >>> Is there a URL where this webrev is accessible outside of Oracle?
>     >>>
>     >>> Thanks,
>     >>> Dave
>     >>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120123/2eb2c242/attachment.html 

From karen.kinnear at oracle.com  Mon Jan 23 13:55:45 2012
From: karen.kinnear at oracle.com (Karen Kinnear)
Date: Mon, 23 Jan 2012 16:55:45 -0500
Subject: Code and Design Feedback request: 7114376: tune system dictionary
	size
In-Reply-To: <4F1DD003.1090408@oracle.com>
References: <73E0E1AF-1806-439B-B826-AE4E59590BFD@oracle.com>
	<CAGVnyzRHxg8SD5kx4duFCMuf7xzzaBGng_CBxC5n+DfmrzekZw@mail.gmail.com>
	<F45DFA65-925A-446D-99A4-1F0D7AEF8496@oracle.com>
	<4F1CBCDC.4060808@oracle.com>
	<E2FD2C70-3F3B-4EB1-84E6-CFA81F17AFA1@oracle.com>
	<CAHjP37H4HnvwE5WEvwtFUntcO2wOeuHh+NryhuCNqkYFbUR+Ow@mail.gmail.com>
	<4F1DD003.1090408@oracle.com>
Message-ID: <382C3F6F-6BB7-41E8-B956-0689F9B9D4E7@oracle.com>

Vitaly,

Thank you for the input. Based on the feedback from you, David and John Pampuch, I have simplified the
implementation. It no longer relies on the lookup_count - which you are absolutely correct - would have a risk of overflowing.

As soon as this passes my internal testing I will send it out again. 

thank you so much,
Karen

On Jan 23, 2012, at 4:24 PM, Paul Hohensee wrote:

> Per-software-thread counters would be easier to implement.  We don't
> really know how many active hw threads we have, and the hw thread
> count can change unpredictably or be unknowable (e.g., virtualized
> environments).
> 
> Paul
> 
> On 1/23/12 10:26 AM, Vitaly Davidovich wrote:
>> 
>> Hi Karen,
>> 
>> Regarding the counters David mentioned, is there also any concern with them overflowing, on a long-lived jvm process? Would the code deal with negative values properly? Or do you not expect this to be a concern? The lookup_count is incremented on each lookup so that looks a bit suspect to my uninformed eyes.
>> 
>> As for atomic or not, I think the other issue here is if every bit of performance is important, might it make sense to have per-cpu counters (with sufficient cache line padding) and then aggregate them when needed? This would reduce coherence traffic on these writes.
>> 
>> Vitaly
>> 
>> Sent from my phone
>> 
>> On Jan 23, 2012 9:48 AM, "Karen Kinnear" <karen.kinnear at oracle.com> wrote:
>> David,
>> 
>> Thank you for the detailed feedback. More below.
>> 
>> On Jan 22, 2012, at 8:50 PM, David Holmes wrote:
>> 
>> > Hi Karen,
>> >
>> > On 22/01/2012 5:04 AM, Karen Kinnear wrote:
>> >> Thank you for asking, my mistake. Here is an accessible webrev.
>> >>
>> >> Webrev: http://cr.openjdk.java.net/~acorn/7114376/webrev/
>> >
>> > systemDictionary.cpp:
>> >
>> > I notice that in systemDictionary the calculation of the index has been moved inside the locked regions. This impacted some of the SD method signatures, depending on whether they grabbed the lock internally, or whether the lock was first acquired externally. Seems to me we could avoid a lot of duplicate index calculating code if we went a step further and changed all the methods to only take the hash and do their own index calculation if needed - in particular if find_class did the calculation then I think most of the other index calculations would disappear.
>> 
>> Great suggestion - I will do that and send for re-review.
>> >
>> > 1757 // Check if system dictionary should be resized to speed up find() calls
>> > 1758 // Default is to check on class unloading
>> > 1759 // SystemDictionarySizeIndex flag allows starting with a larger initial size
>> > 1760 // if SystemDictionaryAverageDepth is non-zero, also check on safepoint cleanup
>> > 1761 // and allow customers to select acceptable average depth to pass
>> > 1762 // to verify_lookup_length, otherwise use number_of_classes/table_size
>> > 1763 // verify_lookup_length compares actual average lookup depth: lookup_length/lookup_count
>> > 1764 // with theoretical or user-specified.
>> > 1765 // In future we may need to periodically calculate lookup_length to see
>> > 1766 // if we need to force a safepoint to resize system dictionary
>> > 1767 void SystemDictionary::should_resize_dictionary(int unloaded_count) {
>> >
>> > I found the above comment block difficult to parse - perhaps some Capitals missing to indicate new points/sentences?
>> I am going to simplify the flags to just -XX:LoadedClasses=# so this comment will be rewritten and
>> sent out for re-review.
>> >
>> > Also should this method assert that either the SD lock is held or else we're at a safepoint?
>> Will add the assertion that we are at a safepoint.
>> >
>> > ---
>> >
>> > dictionary.hpp/cpp
>> >
>> > +   // Change size of system dictionary
>> > +   // Initially for growing when lookup time takes too long
>> > +   Dictionary* dictionary_resize(int newsize);
>> >
>> > Is "system dictionary" distinct from SystemDictionary?
>> >
>> > I don't like having this as an instance method as it doesn't resize the current dictionary but returns a new-one, leaving the caller to free the original, yet has the side-effect of clearing all entries from the current one. Seems to me we should simply have the caller do:
>> >
>> > Dictionary* newdict = new Dictionary(newsize);
>> > olddict->copy_all_entries(newdict);
>> > delete olddict;
>> > olddict = newdict;
>> >
>> > though perhaps copy_all_entries should really be move_all_entries?
>> I will make that change. dictionary_resize used to do a lot more work, but the way it turned out
>> your suggestion is much better.
>> move_all_entries is a better name.
>> >
>> > 477 // This routine does not lock the system dictionary.
>> > ...
>> > 487 // Actually, the locking here is a bit trickier:
>> > ...
>> > 493 DictionaryEntry* Dictionary::get_entry(int index, unsigned int hash,
>> > 494                                        Symbol* class_name,
>> > 495                                        Handle class_loader) {
>> > 496   oop loader = class_loader();
>> > 497   _lookup_count++;
>> > 498
>> > 499   for (DictionaryEntry* entry = bucket(index);
>> > 500                         entry != NULL;
>> > 501                         entry = entry->next()) {
>> > 502     if (entry->hash() == hash && entry->equals(class_name, loader)) {
>> > 503       return entry;
>> > 504     }
>> > 505     _lookup_length++;
>> >
>> > Your now non-debug counters seem to be being updated in a non-threadsafe manner - or are we assuming the lock has been taken in the caller?
>> Good catch. This is called without a lock.
>> So the question is - should I make this an atomic update, or since these are approximations used for heuristics
>> only, add a comment that we actually don't need them to be completely accurate? I can measure
>> cost of the atomic update.
>> >
>> >> An alternative possibility for supplying a single flag might be: -XX:LoadedClasses=#
>> >>
>> >> This could be an anticipated count of expected total classes, which we could use
>> >> internally to set the initial SystemDictionary size. This might outlast any internal changes
>> >> in data structures.
>> >
>> > I think this might be a simpler starting point for things. Dynamically resizing the SD is transiently a bad thing for everyone:
>> > - pause while it happens
>> > - excessive memory use while it happens
>> >
>> > better in my opinion to try to get the initial number of buckets right and not dynamically resize.
>> I believe we need both. It is simpler to just start with a better size - for now I will add the -XX:LoadedClasses=#
>> and use that as a starting point, and the only new flag, for customers that want to add a command-line argument.
>> 
>> Our goal is to have good performance without requiring custom tuning, so the dynamic resizing is valuable for
>> customers who do not want to figure out their usage and do not want to add a command-line argument.
>> I will simplify the ergonomic tuning to be based on the dynamically calculated LoadedClasses  as well, so
>> that if we do resize based on slower lookup times, that we resize to the current need rather than incrementally.
>> 
>> I expect the resizing to be beneficial both for customers with high numbers of loaded classes and I expect
>> the embedded customers to take advantage of resizing to potentially shrink the system dictionary. And I tried
>> to add the logic so that this could be reused for resizing other internal metadata structures.
>> >
>> > Cheers,
>> > David
>> > -----
>> 
>> thank you for the very helpful feedback,
>> Karen
>> 
>> >
>> >> Tests run:
>> >> vmsqe: nsk.quick.testlist: solaris-sparc, fastdbg - this includes sa testing
>> >> jck: vm,lang,api: solaris-sparc, fastdbg
>> >> JPRT - this includes CDS testing
>> >> performance runs in progress, including frequency of checking need to resize
>> >>
>> >> thanks,
>> >> Karen
>> >>
>> >> On Jan 21, 2012, at 12:04 PM, David Schlosnagle wrote:
>> >>
>> >>> On Fri, Jan 20, 2012 at 6:06 PM, Karen Kinnear<karen.kinnear at oracle.com>  wrote:
>> >>>> Please review initial proposal for: 7114376: tune system dictionary size
>> >>>>
>> >>>> http://oklahoma.us.oracle.com/~kmkinnea/webrev/sdresize/webrev/
>> >>>
>> >>> Karen,
>> >>>
>> >>> Is there a URL where this webrev is accessible outside of Oracle?
>> >>>
>> >>> Thanks,
>> >>> Dave
>> >>
>> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120123/2ae38d69/attachment-0001.html 

From paul.hohensee at oracle.com  Mon Jan 23 14:05:33 2012
From: paul.hohensee at oracle.com (Paul Hohensee)
Date: Mon, 23 Jan 2012 17:05:33 -0500
Subject: Code and Design Feedback request: 7114376: tune system dictionary
	size
In-Reply-To: <4F1DD003.1090408@oracle.com>
References: <73E0E1AF-1806-439B-B826-AE4E59590BFD@oracle.com>
	<CAGVnyzRHxg8SD5kx4duFCMuf7xzzaBGng_CBxC5n+DfmrzekZw@mail.gmail.com>
	<F45DFA65-925A-446D-99A4-1F0D7AEF8496@oracle.com>
	<4F1CBCDC.4060808@oracle.com>
	<E2FD2C70-3F3B-4EB1-84E6-CFA81F17AFA1@oracle.com>
	<CAHjP37H4HnvwE5WEvwtFUntcO2wOeuHh+NryhuCNqkYFbUR+Ow@mail.gmail.com>
	<4F1DD003.1090408@oracle.com>
Message-ID: <4F1DD9AD.3020908@oracle.com>

Another approach you might consider is to move the resizing functionality
from the SystemDictionary into a generic ResizableHashtable, which
latter would be a subclass of BasicHashTable.  That would make it
a utility available to any part of Hotspot.

Paul

On 1/23/12 4:24 PM, Paul Hohensee wrote:
> Per-software-thread counters would be easier to implement.  We don't
> really know how many active hw threads we have, and the hw thread
> count can change unpredictably or be unknowable (e.g., virtualized
> environments).
>
> Paul
>
> On 1/23/12 10:26 AM, Vitaly Davidovich wrote:
>>
>> Hi Karen,
>>
>> Regarding the counters David mentioned, is there also any concern 
>> with them overflowing, on a long-lived jvm process? Would the code 
>> deal with negative values properly? Or do you not expect this to be a 
>> concern? The lookup_count is incremented on each lookup so that looks 
>> a bit suspect to my uninformed eyes.
>>
>> As for atomic or not, I think the other issue here is if every bit of 
>> performance is important, might it make sense to have per-cpu 
>> counters (with sufficient cache line padding) and then aggregate them 
>> when needed? This would reduce coherence traffic on these writes.
>>
>> Vitaly
>>
>> Sent from my phone
>>
>> On Jan 23, 2012 9:48 AM, "Karen Kinnear" <karen.kinnear at oracle.com 
>> <mailto:karen.kinnear at oracle.com>> wrote:
>>
>>     David,
>>
>>     Thank you for the detailed feedback. More below.
>>
>>     On Jan 22, 2012, at 8:50 PM, David Holmes wrote:
>>
>>     > Hi Karen,
>>     >
>>     > On 22/01/2012 5:04 AM, Karen Kinnear wrote:
>>     >> Thank you for asking, my mistake. Here is an accessible webrev.
>>     >>
>>     >> Webrev: http://cr.openjdk.java.net/~acorn/7114376/webrev/
>>     <http://cr.openjdk.java.net/%7Eacorn/7114376/webrev/>
>>     >
>>     > systemDictionary.cpp:
>>     >
>>     > I notice that in systemDictionary the calculation of the index
>>     has been moved inside the locked regions. This impacted some of
>>     the SD method signatures, depending on whether they grabbed the
>>     lock internally, or whether the lock was first acquired
>>     externally. Seems to me we could avoid a lot of duplicate index
>>     calculating code if we went a step further and changed all the
>>     methods to only take the hash and do their own index calculation
>>     if needed - in particular if find_class did the calculation then
>>     I think most of the other index calculations would disappear.
>>
>>     Great suggestion - I will do that and send for re-review.
>>     >
>>     > 1757 // Check if system dictionary should be resized to speed
>>     up find() calls
>>     > 1758 // Default is to check on class unloading
>>     > 1759 // SystemDictionarySizeIndex flag allows starting with a
>>     larger initial size
>>     > 1760 // if SystemDictionaryAverageDepth is non-zero, also check
>>     on safepoint cleanup
>>     > 1761 // and allow customers to select acceptable average depth
>>     to pass
>>     > 1762 // to verify_lookup_length, otherwise use
>>     number_of_classes/table_size
>>     > 1763 // verify_lookup_length compares actual average lookup
>>     depth: lookup_length/lookup_count
>>     > 1764 // with theoretical or user-specified.
>>     > 1765 // In future we may need to periodically calculate
>>     lookup_length to see
>>     > 1766 // if we need to force a safepoint to resize system dictionary
>>     > 1767 void SystemDictionary::should_resize_dictionary(int
>>     unloaded_count) {
>>     >
>>     > I found the above comment block difficult to parse - perhaps
>>     some Capitals missing to indicate new points/sentences?
>>     I am going to simplify the flags to just -XX:LoadedClasses=# so
>>     this comment will be rewritten and
>>     sent out for re-review.
>>     >
>>     > Also should this method assert that either the SD lock is held
>>     or else we're at a safepoint?
>>     Will add the assertion that we are at a safepoint.
>>     >
>>     > ---
>>     >
>>     > dictionary.hpp/cpp
>>     >
>>     > +   // Change size of system dictionary
>>     > +   // Initially for growing when lookup time takes too long
>>     > +   Dictionary* dictionary_resize(int newsize);
>>     >
>>     > Is "system dictionary" distinct from SystemDictionary?
>>     >
>>     > I don't like having this as an instance method as it doesn't
>>     resize the current dictionary but returns a new-one, leaving the
>>     caller to free the original, yet has the side-effect of clearing
>>     all entries from the current one. Seems to me we should simply
>>     have the caller do:
>>     >
>>     > Dictionary* newdict = new Dictionary(newsize);
>>     > olddict->copy_all_entries(newdict);
>>     > delete olddict;
>>     > olddict = newdict;
>>     >
>>     > though perhaps copy_all_entries should really be move_all_entries?
>>     I will make that change. dictionary_resize used to do a lot more
>>     work, but the way it turned out
>>     your suggestion is much better.
>>     move_all_entries is a better name.
>>     >
>>     > 477 // This routine does not lock the system dictionary.
>>     > ...
>>     > 487 // Actually, the locking here is a bit trickier:
>>     > ...
>>     > 493 DictionaryEntry* Dictionary::get_entry(int index, unsigned
>>     int hash,
>>     > 494                                        Symbol* class_name,
>>     > 495                                        Handle class_loader) {
>>     > 496   oop loader = class_loader();
>>     > 497   _lookup_count++;
>>     > 498
>>     > 499   for (DictionaryEntry* entry = bucket(index);
>>     > 500                         entry != NULL;
>>     > 501                         entry = entry->next()) {
>>     > 502     if (entry->hash() == hash && entry->equals(class_name,
>>     loader)) {
>>     > 503       return entry;
>>     > 504     }
>>     > 505     _lookup_length++;
>>     >
>>     > Your now non-debug counters seem to be being updated in a
>>     non-threadsafe manner - or are we assuming the lock has been
>>     taken in the caller?
>>     Good catch. This is called without a lock.
>>     So the question is - should I make this an atomic update, or
>>     since these are approximations used for heuristics
>>     only, add a comment that we actually don't need them to be
>>     completely accurate? I can measure
>>     cost of the atomic update.
>>     >
>>     >> An alternative possibility for supplying a single flag might
>>     be: -XX:LoadedClasses=#
>>     >>
>>     >> This could be an anticipated count of expected total classes,
>>     which we could use
>>     >> internally to set the initial SystemDictionary size. This
>>     might outlast any internal changes
>>     >> in data structures.
>>     >
>>     > I think this might be a simpler starting point for things.
>>     Dynamically resizing the SD is transiently a bad thing for everyone:
>>     > - pause while it happens
>>     > - excessive memory use while it happens
>>     >
>>     > better in my opinion to try to get the initial number of
>>     buckets right and not dynamically resize.
>>     I believe we need both. It is simpler to just start with a better
>>     size - for now I will add the -XX:LoadedClasses=#
>>     and use that as a starting point, and the only new flag, for
>>     customers that want to add a command-line argument.
>>
>>     Our goal is to have good performance without requiring custom
>>     tuning, so the dynamic resizing is valuable for
>>     customers who do not want to figure out their usage and do not
>>     want to add a command-line argument.
>>     I will simplify the ergonomic tuning to be based on the
>>     dynamically calculated LoadedClasses  as well, so
>>     that if we do resize based on slower lookup times, that we resize
>>     to the current need rather than incrementally.
>>
>>     I expect the resizing to be beneficial both for customers with
>>     high numbers of loaded classes and I expect
>>     the embedded customers to take advantage of resizing to
>>     potentially shrink the system dictionary. And I tried
>>     to add the logic so that this could be reused for resizing other
>>     internal metadata structures.
>>     >
>>     > Cheers,
>>     > David
>>     > -----
>>
>>     thank you for the very helpful feedback,
>>     Karen
>>
>>     >
>>     >> Tests run:
>>     >> vmsqe: nsk.quick.testlist: solaris-sparc, fastdbg - this
>>     includes sa testing
>>     >> jck: vm,lang,api: solaris-sparc, fastdbg
>>     >> JPRT - this includes CDS testing
>>     >> performance runs in progress, including frequency of checking
>>     need to resize
>>     >>
>>     >> thanks,
>>     >> Karen
>>     >>
>>     >> On Jan 21, 2012, at 12:04 PM, David Schlosnagle wrote:
>>     >>
>>     >>> On Fri, Jan 20, 2012 at 6:06 PM, Karen
>>     Kinnear<karen.kinnear at oracle.com
>>     <mailto:karen.kinnear at oracle.com>>  wrote:
>>     >>>> Please review initial proposal for: 7114376: tune system
>>     dictionary size
>>     >>>>
>>     >>>>
>>     http://oklahoma.us.oracle.com/~kmkinnea/webrev/sdresize/webrev/
>>     <http://oklahoma.us.oracle.com/%7Ekmkinnea/webrev/sdresize/webrev/>
>>     >>>
>>     >>> Karen,
>>     >>>
>>     >>> Is there a URL where this webrev is accessible outside of Oracle?
>>     >>>
>>     >>> Thanks,
>>     >>> Dave
>>     >>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120123/4db27703/attachment.html 

From Karen.Kinnear at oracle.com  Mon Jan 23 14:16:25 2012
From: Karen.Kinnear at oracle.com (Karen Kinnear)
Date: Mon, 23 Jan 2012 17:16:25 -0500
Subject: Code and Design Feedback request: 7114376: tune system dictionary
	size
In-Reply-To: <4F1DD9AD.3020908@oracle.com>
References: <73E0E1AF-1806-439B-B826-AE4E59590BFD@oracle.com>
	<CAGVnyzRHxg8SD5kx4duFCMuf7xzzaBGng_CBxC5n+DfmrzekZw@mail.gmail.com>
	<F45DFA65-925A-446D-99A4-1F0D7AEF8496@oracle.com>
	<4F1CBCDC.4060808@oracle.com>
	<E2FD2C70-3F3B-4EB1-84E6-CFA81F17AFA1@oracle.com>
	<CAHjP37H4HnvwE5WEvwtFUntcO2wOeuHh+NryhuCNqkYFbUR+Ow@mail.gmail.com>
	<4F1DD003.1090408@oracle.com> <4F1DD9AD.3020908@oracle.com>
Message-ID: <02E6E0EF-45B3-4677-99E2-E0D88D692741@oracle.com>

Paul,

The common logic for resizing a hashtable is not the per-table calculation of when to do
the resizing, but the work of moving the entries. That was explicitly put in the 
BasicHashtable class to make it usable by other hash tables in the future. A ResizeableHashtable
may make sense in the future - today the subtleties of handling locks are dealt with
on a per-table basis.

thanks,
Karen

On Jan 23, 2012, at 5:05 PM, Paul Hohensee wrote:

> Another approach you might consider is to move the resizing functionality
> from the SystemDictionary into a generic ResizableHashtable, which
> latter would be a subclass of BasicHashTable.  That would make it
> a utility available to any part of Hotspot.
> 
> Paul
> 
> On 1/23/12 4:24 PM, Paul Hohensee wrote:
>> 
>> Per-software-thread counters would be easier to implement.  We don't
>> really know how many active hw threads we have, and the hw thread
>> count can change unpredictably or be unknowable (e.g., virtualized
>> environments).
>> 
>> Paul
>> 
>> On 1/23/12 10:26 AM, Vitaly Davidovich wrote:
>>> 
>>> Hi Karen,
>>> 
>>> Regarding the counters David mentioned, is there also any concern with them overflowing, on a long-lived jvm process? Would the code deal with negative values properly? Or do you not expect this to be a concern? The lookup_count is incremented on each lookup so that looks a bit suspect to my uninformed eyes.
>>> 
>>> As for atomic or not, I think the other issue here is if every bit of performance is important, might it make sense to have per-cpu counters (with sufficient cache line padding) and then aggregate them when needed? This would reduce coherence traffic on these writes.
>>> 
>>> Vitaly
>>> 
>>> Sent from my phone
>>> 
>>> On Jan 23, 2012 9:48 AM, "Karen Kinnear" <karen.kinnear at oracle.com> wrote:
>>> David,
>>> 
>>> Thank you for the detailed feedback. More below.
>>> 
>>> On Jan 22, 2012, at 8:50 PM, David Holmes wrote:
>>> 
>>> > Hi Karen,
>>> >
>>> > On 22/01/2012 5:04 AM, Karen Kinnear wrote:
>>> >> Thank you for asking, my mistake. Here is an accessible webrev.
>>> >>
>>> >> Webrev: http://cr.openjdk.java.net/~acorn/7114376/webrev/
>>> >
>>> > systemDictionary.cpp:
>>> >
>>> > I notice that in systemDictionary the calculation of the index has been moved inside the locked regions. This impacted some of the SD method signatures, depending on whether they grabbed the lock internally, or whether the lock was first acquired externally. Seems to me we could avoid a lot of duplicate index calculating code if we went a step further and changed all the methods to only take the hash and do their own index calculation if needed - in particular if find_class did the calculation then I think most of the other index calculations would disappear.
>>> 
>>> Great suggestion - I will do that and send for re-review.
>>> >
>>> > 1757 // Check if system dictionary should be resized to speed up find() calls
>>> > 1758 // Default is to check on class unloading
>>> > 1759 // SystemDictionarySizeIndex flag allows starting with a larger initial size
>>> > 1760 // if SystemDictionaryAverageDepth is non-zero, also check on safepoint cleanup
>>> > 1761 // and allow customers to select acceptable average depth to pass
>>> > 1762 // to verify_lookup_length, otherwise use number_of_classes/table_size
>>> > 1763 // verify_lookup_length compares actual average lookup depth: lookup_length/lookup_count
>>> > 1764 // with theoretical or user-specified.
>>> > 1765 // In future we may need to periodically calculate lookup_length to see
>>> > 1766 // if we need to force a safepoint to resize system dictionary
>>> > 1767 void SystemDictionary::should_resize_dictionary(int unloaded_count) {
>>> >
>>> > I found the above comment block difficult to parse - perhaps some Capitals missing to indicate new points/sentences?
>>> I am going to simplify the flags to just -XX:LoadedClasses=# so this comment will be rewritten and
>>> sent out for re-review.
>>> >
>>> > Also should this method assert that either the SD lock is held or else we're at a safepoint?
>>> Will add the assertion that we are at a safepoint.
>>> >
>>> > ---
>>> >
>>> > dictionary.hpp/cpp
>>> >
>>> > +   // Change size of system dictionary
>>> > +   // Initially for growing when lookup time takes too long
>>> > +   Dictionary* dictionary_resize(int newsize);
>>> >
>>> > Is "system dictionary" distinct from SystemDictionary?
>>> >
>>> > I don't like having this as an instance method as it doesn't resize the current dictionary but returns a new-one, leaving the caller to free the original, yet has the side-effect of clearing all entries from the current one. Seems to me we should simply have the caller do:
>>> >
>>> > Dictionary* newdict = new Dictionary(newsize);
>>> > olddict->copy_all_entries(newdict);
>>> > delete olddict;
>>> > olddict = newdict;
>>> >
>>> > though perhaps copy_all_entries should really be move_all_entries?
>>> I will make that change. dictionary_resize used to do a lot more work, but the way it turned out
>>> your suggestion is much better.
>>> move_all_entries is a better name.
>>> >
>>> > 477 // This routine does not lock the system dictionary.
>>> > ...
>>> > 487 // Actually, the locking here is a bit trickier:
>>> > ...
>>> > 493 DictionaryEntry* Dictionary::get_entry(int index, unsigned int hash,
>>> > 494                                        Symbol* class_name,
>>> > 495                                        Handle class_loader) {
>>> > 496   oop loader = class_loader();
>>> > 497   _lookup_count++;
>>> > 498
>>> > 499   for (DictionaryEntry* entry = bucket(index);
>>> > 500                         entry != NULL;
>>> > 501                         entry = entry->next()) {
>>> > 502     if (entry->hash() == hash && entry->equals(class_name, loader)) {
>>> > 503       return entry;
>>> > 504     }
>>> > 505     _lookup_length++;
>>> >
>>> > Your now non-debug counters seem to be being updated in a non-threadsafe manner - or are we assuming the lock has been taken in the caller?
>>> Good catch. This is called without a lock.
>>> So the question is - should I make this an atomic update, or since these are approximations used for heuristics
>>> only, add a comment that we actually don't need them to be completely accurate? I can measure
>>> cost of the atomic update.
>>> >
>>> >> An alternative possibility for supplying a single flag might be: -XX:LoadedClasses=#
>>> >>
>>> >> This could be an anticipated count of expected total classes, which we could use
>>> >> internally to set the initial SystemDictionary size. This might outlast any internal changes
>>> >> in data structures.
>>> >
>>> > I think this might be a simpler starting point for things. Dynamically resizing the SD is transiently a bad thing for everyone:
>>> > - pause while it happens
>>> > - excessive memory use while it happens
>>> >
>>> > better in my opinion to try to get the initial number of buckets right and not dynamically resize.
>>> I believe we need both. It is simpler to just start with a better size - for now I will add the -XX:LoadedClasses=#
>>> and use that as a starting point, and the only new flag, for customers that want to add a command-line argument.
>>> 
>>> Our goal is to have good performance without requiring custom tuning, so the dynamic resizing is valuable for
>>> customers who do not want to figure out their usage and do not want to add a command-line argument.
>>> I will simplify the ergonomic tuning to be based on the dynamically calculated LoadedClasses  as well, so
>>> that if we do resize based on slower lookup times, that we resize to the current need rather than incrementally.
>>> 
>>> I expect the resizing to be beneficial both for customers with high numbers of loaded classes and I expect
>>> the embedded customers to take advantage of resizing to potentially shrink the system dictionary. And I tried
>>> to add the logic so that this could be reused for resizing other internal metadata structures.
>>> >
>>> > Cheers,
>>> > David
>>> > -----
>>> 
>>> thank you for the very helpful feedback,
>>> Karen
>>> 
>>> >
>>> >> Tests run:
>>> >> vmsqe: nsk.quick.testlist: solaris-sparc, fastdbg - this includes sa testing
>>> >> jck: vm,lang,api: solaris-sparc, fastdbg
>>> >> JPRT - this includes CDS testing
>>> >> performance runs in progress, including frequency of checking need to resize
>>> >>
>>> >> thanks,
>>> >> Karen
>>> >>
>>> >> On Jan 21, 2012, at 12:04 PM, David Schlosnagle wrote:
>>> >>
>>> >>> On Fri, Jan 20, 2012 at 6:06 PM, Karen Kinnear<karen.kinnear at oracle.com>  wrote:
>>> >>>> Please review initial proposal for: 7114376: tune system dictionary size
>>> >>>>
>>> >>>> http://oklahoma.us.oracle.com/~kmkinnea/webrev/sdresize/webrev/
>>> >>>
>>> >>> Karen,
>>> >>>
>>> >>> Is there a URL where this webrev is accessible outside of Oracle?
>>> >>>
>>> >>> Thanks,
>>> >>> Dave
>>> >>
>>> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120123/ea07b3f8/attachment-0001.html 

From paul.hohensee at oracle.com  Mon Jan 23 14:18:01 2012
From: paul.hohensee at oracle.com (Paul Hohensee)
Date: Mon, 23 Jan 2012 17:18:01 -0500
Subject: Code and Design Feedback request: 7114376: tune system dictionary
	size
In-Reply-To: <02E6E0EF-45B3-4677-99E2-E0D88D692741@oracle.com>
References: <73E0E1AF-1806-439B-B826-AE4E59590BFD@oracle.com>
	<CAGVnyzRHxg8SD5kx4duFCMuf7xzzaBGng_CBxC5n+DfmrzekZw@mail.gmail.com>
	<F45DFA65-925A-446D-99A4-1F0D7AEF8496@oracle.com>
	<4F1CBCDC.4060808@oracle.com>
	<E2FD2C70-3F3B-4EB1-84E6-CFA81F17AFA1@oracle.com>
	<CAHjP37H4HnvwE5WEvwtFUntcO2wOeuHh+NryhuCNqkYFbUR+Ow@mail.gmail.com>
	<4F1DD003.1090408@oracle.com> <4F1DD9AD.3020908@oracle.com>
	<02E6E0EF-45B3-4677-99E2-E0D88D692741@oracle.com>
Message-ID: <4F1DDC99.2060609@oracle.com>

Ok.  You could have a ConcurrentResizeableHashTable for that.  Might 
file an RFE.

Paul

On 1/23/12 5:16 PM, Karen Kinnear wrote:
> Paul,
>
> The common logic for resizing a hashtable is not the per-table 
> calculation of when to do
> the resizing, but the work of moving the entries. That was explicitly 
> put in the
> BasicHashtable class to make it usable by other hash tables in the 
> future. A ResizeableHashtable
> may make sense in the future - today the subtleties of handling locks 
> are dealt with
> on a per-table basis.
>
> thanks,
> Karen
>
> On Jan 23, 2012, at 5:05 PM, Paul Hohensee wrote:
>
>> Another approach you might consider is to move the resizing functionality
>> from the SystemDictionary into a generic ResizableHashtable, which
>> latter would be a subclass of BasicHashTable.  That would make it
>> a utility available to any part of Hotspot.
>>
>> Paul
>>
>> On 1/23/12 4:24 PM, Paul Hohensee wrote:
>>> Per-software-thread counters would be easier to implement.  We don't
>>> really know how many active hw threads we have, and the hw thread
>>> count can change unpredictably or be unknowable (e.g., virtualized
>>> environments).
>>>
>>> Paul
>>>
>>> On 1/23/12 10:26 AM, Vitaly Davidovich wrote:
>>>>
>>>> Hi Karen,
>>>>
>>>> Regarding the counters David mentioned, is there also any concern 
>>>> with them overflowing, on a long-lived jvm process? Would the code 
>>>> deal with negative values properly? Or do you not expect this to be 
>>>> a concern? The lookup_count is incremented on each lookup so that 
>>>> looks a bit suspect to my uninformed eyes.
>>>>
>>>> As for atomic or not, I think the other issue here is if every bit 
>>>> of performance is important, might it make sense to have per-cpu 
>>>> counters (with sufficient cache line padding) and then aggregate 
>>>> them when needed? This would reduce coherence traffic on these writes.
>>>>
>>>> Vitaly
>>>>
>>>> Sent from my phone
>>>>
>>>> On Jan 23, 2012 9:48 AM, "Karen Kinnear" <karen.kinnear at oracle.com 
>>>> <mailto:karen.kinnear at oracle.com>> wrote:
>>>>
>>>>     David,
>>>>
>>>>     Thank you for the detailed feedback. More below.
>>>>
>>>>     On Jan 22, 2012, at 8:50 PM, David Holmes wrote:
>>>>
>>>>     > Hi Karen,
>>>>     >
>>>>     > On 22/01/2012 5:04 AM, Karen Kinnear wrote:
>>>>     >> Thank you for asking, my mistake. Here is an accessible webrev.
>>>>     >>
>>>>     >> Webrev: http://cr.openjdk.java.net/~acorn/7114376/webrev/
>>>>     <http://cr.openjdk.java.net/%7Eacorn/7114376/webrev/>
>>>>     >
>>>>     > systemDictionary.cpp:
>>>>     >
>>>>     > I notice that in systemDictionary the calculation of the
>>>>     index has been moved inside the locked regions. This impacted
>>>>     some of the SD method signatures, depending on whether they
>>>>     grabbed the lock internally, or whether the lock was first
>>>>     acquired externally. Seems to me we could avoid a lot of
>>>>     duplicate index calculating code if we went a step further and
>>>>     changed all the methods to only take the hash and do their own
>>>>     index calculation if needed - in particular if find_class did
>>>>     the calculation then I think most of the other index
>>>>     calculations would disappear.
>>>>
>>>>     Great suggestion - I will do that and send for re-review.
>>>>     >
>>>>     > 1757 // Check if system dictionary should be resized to speed
>>>>     up find() calls
>>>>     > 1758 // Default is to check on class unloading
>>>>     > 1759 // SystemDictionarySizeIndex flag allows starting with a
>>>>     larger initial size
>>>>     > 1760 // if SystemDictionaryAverageDepth is non-zero, also
>>>>     check on safepoint cleanup
>>>>     > 1761 // and allow customers to select acceptable average
>>>>     depth to pass
>>>>     > 1762 // to verify_lookup_length, otherwise use
>>>>     number_of_classes/table_size
>>>>     > 1763 // verify_lookup_length compares actual average lookup
>>>>     depth: lookup_length/lookup_count
>>>>     > 1764 // with theoretical or user-specified.
>>>>     > 1765 // In future we may need to periodically calculate
>>>>     lookup_length to see
>>>>     > 1766 // if we need to force a safepoint to resize system
>>>>     dictionary
>>>>     > 1767 void SystemDictionary::should_resize_dictionary(int
>>>>     unloaded_count) {
>>>>     >
>>>>     > I found the above comment block difficult to parse - perhaps
>>>>     some Capitals missing to indicate new points/sentences?
>>>>     I am going to simplify the flags to just -XX:LoadedClasses=# so
>>>>     this comment will be rewritten and
>>>>     sent out for re-review.
>>>>     >
>>>>     > Also should this method assert that either the SD lock is
>>>>     held or else we're at a safepoint?
>>>>     Will add the assertion that we are at a safepoint.
>>>>     >
>>>>     > ---
>>>>     >
>>>>     > dictionary.hpp/cpp
>>>>     >
>>>>     > +   // Change size of system dictionary
>>>>     > +   // Initially for growing when lookup time takes too long
>>>>     > +   Dictionary* dictionary_resize(int newsize);
>>>>     >
>>>>     > Is "system dictionary" distinct from SystemDictionary?
>>>>     >
>>>>     > I don't like having this as an instance method as it doesn't
>>>>     resize the current dictionary but returns a new-one, leaving
>>>>     the caller to free the original, yet has the side-effect of
>>>>     clearing all entries from the current one. Seems to me we
>>>>     should simply have the caller do:
>>>>     >
>>>>     > Dictionary* newdict = new Dictionary(newsize);
>>>>     > olddict->copy_all_entries(newdict);
>>>>     > delete olddict;
>>>>     > olddict = newdict;
>>>>     >
>>>>     > though perhaps copy_all_entries should really be
>>>>     move_all_entries?
>>>>     I will make that change. dictionary_resize used to do a lot
>>>>     more work, but the way it turned out
>>>>     your suggestion is much better.
>>>>     move_all_entries is a better name.
>>>>     >
>>>>     > 477 // This routine does not lock the system dictionary.
>>>>     > ...
>>>>     > 487 // Actually, the locking here is a bit trickier:
>>>>     > ...
>>>>     > 493 DictionaryEntry* Dictionary::get_entry(int index,
>>>>     unsigned int hash,
>>>>     > 494                                        Symbol* class_name,
>>>>     > 495                                        Handle class_loader) {
>>>>     > 496   oop loader = class_loader();
>>>>     > 497   _lookup_count++;
>>>>     > 498
>>>>     > 499   for (DictionaryEntry* entry = bucket(index);
>>>>     > 500                         entry != NULL;
>>>>     > 501                         entry = entry->next()) {
>>>>     > 502     if (entry->hash() == hash &&
>>>>     entry->equals(class_name, loader)) {
>>>>     > 503       return entry;
>>>>     > 504     }
>>>>     > 505     _lookup_length++;
>>>>     >
>>>>     > Your now non-debug counters seem to be being updated in a
>>>>     non-threadsafe manner - or are we assuming the lock has been
>>>>     taken in the caller?
>>>>     Good catch. This is called without a lock.
>>>>     So the question is - should I make this an atomic update, or
>>>>     since these are approximations used for heuristics
>>>>     only, add a comment that we actually don't need them to be
>>>>     completely accurate? I can measure
>>>>     cost of the atomic update.
>>>>     >
>>>>     >> An alternative possibility for supplying a single flag might
>>>>     be: -XX:LoadedClasses=#
>>>>     >>
>>>>     >> This could be an anticipated count of expected total
>>>>     classes, which we could use
>>>>     >> internally to set the initial SystemDictionary size. This
>>>>     might outlast any internal changes
>>>>     >> in data structures.
>>>>     >
>>>>     > I think this might be a simpler starting point for things.
>>>>     Dynamically resizing the SD is transiently a bad thing for
>>>>     everyone:
>>>>     > - pause while it happens
>>>>     > - excessive memory use while it happens
>>>>     >
>>>>     > better in my opinion to try to get the initial number of
>>>>     buckets right and not dynamically resize.
>>>>     I believe we need both. It is simpler to just start with a
>>>>     better size - for now I will add the -XX:LoadedClasses=#
>>>>     and use that as a starting point, and the only new flag, for
>>>>     customers that want to add a command-line argument.
>>>>
>>>>     Our goal is to have good performance without requiring custom
>>>>     tuning, so the dynamic resizing is valuable for
>>>>     customers who do not want to figure out their usage and do not
>>>>     want to add a command-line argument.
>>>>     I will simplify the ergonomic tuning to be based on the
>>>>     dynamically calculated LoadedClasses  as well, so
>>>>     that if we do resize based on slower lookup times, that we
>>>>     resize to the current need rather than incrementally.
>>>>
>>>>     I expect the resizing to be beneficial both for customers with
>>>>     high numbers of loaded classes and I expect
>>>>     the embedded customers to take advantage of resizing to
>>>>     potentially shrink the system dictionary. And I tried
>>>>     to add the logic so that this could be reused for resizing
>>>>     other internal metadata structures.
>>>>     >
>>>>     > Cheers,
>>>>     > David
>>>>     > -----
>>>>
>>>>     thank you for the very helpful feedback,
>>>>     Karen
>>>>
>>>>     >
>>>>     >> Tests run:
>>>>     >> vmsqe: nsk.quick.testlist: solaris-sparc, fastdbg - this
>>>>     includes sa testing
>>>>     >> jck: vm,lang,api: solaris-sparc, fastdbg
>>>>     >> JPRT - this includes CDS testing
>>>>     >> performance runs in progress, including frequency of
>>>>     checking need to resize
>>>>     >>
>>>>     >> thanks,
>>>>     >> Karen
>>>>     >>
>>>>     >> On Jan 21, 2012, at 12:04 PM, David Schlosnagle wrote:
>>>>     >>
>>>>     >>> On Fri, Jan 20, 2012 at 6:06 PM, Karen
>>>>     Kinnear<karen.kinnear at oracle.com
>>>>     <mailto:karen.kinnear at oracle.com>>  wrote:
>>>>     >>>> Please review initial proposal for: 7114376: tune system
>>>>     dictionary size
>>>>     >>>>
>>>>     >>>>
>>>>     http://oklahoma.us.oracle.com/~kmkinnea/webrev/sdresize/webrev/
>>>>     <http://oklahoma.us.oracle.com/%7Ekmkinnea/webrev/sdresize/webrev/>
>>>>     >>>
>>>>     >>> Karen,
>>>>     >>>
>>>>     >>> Is there a URL where this webrev is accessible outside of
>>>>     Oracle?
>>>>     >>>
>>>>     >>> Thanks,
>>>>     >>> Dave
>>>>     >>
>>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120123/cad55970/attachment-0001.html 

From dean.long at oracle.com  Mon Jan 23 15:22:49 2012
From: dean.long at oracle.com (Dean Long)
Date: Mon, 23 Jan 2012 15:22:49 -0800
Subject: Code and Design Feedback request: 7114376: tune system dictionary
	size
In-Reply-To: <F45DFA65-925A-446D-99A4-1F0D7AEF8496@oracle.com>
References: <73E0E1AF-1806-439B-B826-AE4E59590BFD@oracle.com>
	<CAGVnyzRHxg8SD5kx4duFCMuf7xzzaBGng_CBxC5n+DfmrzekZw@mail.gmail.com>
	<F45DFA65-925A-446D-99A4-1F0D7AEF8496@oracle.com>
Message-ID: <4F1DEBC9.9030306@oracle.com>

It is probably out of the scope of this CR, but is it possible that the 
internal data structure might one day change from monolithic hash table 
to something more hierarchical?  For example having a separate data 
structure per package or per classloader.  I was reading online about 
something called a HAT-trie that deals with shared prefixes and was 
designed with cache locality in mind.

dl

On 1/21/2012 11:04 AM, Karen Kinnear wrote:
> David,
>
> Thank you for asking, my mistake. Here is an accessible webrev.
>
> Webrev: http://cr.openjdk.java.net/~acorn/7114376/webrev/
>
> An alternative possibility for supplying a single flag might be: -XX:LoadedClasses=#
>
> This could be an anticipated count of expected total classes, which we could use
> internally to set the initial SystemDictionary size. This might outlast any internal changes
> in data structures.
>
> Tests run:
> vmsqe: nsk.quick.testlist: solaris-sparc, fastdbg - this includes sa testing
> jck: vm,lang,api: solaris-sparc, fastdbg
> JPRT - this includes CDS testing
> performance runs in progress, including frequency of checking need to resize
>
> thanks,
> Karen
>
> On Jan 21, 2012, at 12:04 PM, David Schlosnagle wrote:
>
>> On Fri, Jan 20, 2012 at 6:06 PM, Karen Kinnear<karen.kinnear at oracle.com>  wrote:
>>> Please review initial proposal for: 7114376: tune system dictionary size
>>>
>>> http://oklahoma.us.oracle.com/~kmkinnea/webrev/sdresize/webrev/
>> Karen,
>>
>> Is there a URL where this webrev is accessible outside of Oracle?
>>
>> Thanks,
>> Dave

From karen.kinnear at oracle.com  Tue Jan 24 18:42:34 2012
From: karen.kinnear at oracle.com (Karen Kinnear)
Date: Tue, 24 Jan 2012 21:42:34 -0500
Subject: Code and Design Feedback request: 7114376: tune system dictionary
	size
In-Reply-To: <4F1DEBC9.9030306@oracle.com>
References: <73E0E1AF-1806-439B-B826-AE4E59590BFD@oracle.com>
	<CAGVnyzRHxg8SD5kx4duFCMuf7xzzaBGng_CBxC5n+DfmrzekZw@mail.gmail.com>
	<F45DFA65-925A-446D-99A4-1F0D7AEF8496@oracle.com>
	<4F1DEBC9.9030306@oracle.com>
Message-ID: <61714338-2FBC-4B5A-9A22-6A3C0306DA56@oracle.com>

Dean,

That is actually a really good idea. True, it is out of scope for the current release timeframe, but once we have the permgen removal
changes in, we will want to move more metadata to per classloader data structures that will more quickly free up when we unload classes/classloader.

I filed RFE to capture this idea - 7133093 and put my name on it because it sounds quite promising :-)

thanks!
Karen

On Jan 23, 2012, at 6:22 PM, Dean Long wrote:

> It is probably out of the scope of this CR, but is it possible that the internal data structure might one day change from monolithic hash table to something more hierarchical?  For example having a separate data structure per package or per classloader.  I was reading online about something called a HAT-trie that deals with shared prefixes and was designed with cache locality in mind.
> 
> dl
> 
> On 1/21/2012 11:04 AM, Karen Kinnear wrote:
>> David,
>> 
>> Thank you for asking, my mistake. Here is an accessible webrev.
>> 
>> Webrev: http://cr.openjdk.java.net/~acorn/7114376/webrev/
>> 
>> An alternative possibility for supplying a single flag might be: -XX:LoadedClasses=#
>> 
>> This could be an anticipated count of expected total classes, which we could use
>> internally to set the initial SystemDictionary size. This might outlast any internal changes
>> in data structures.
>> 
>> Tests run:
>> vmsqe: nsk.quick.testlist: solaris-sparc, fastdbg - this includes sa testing
>> jck: vm,lang,api: solaris-sparc, fastdbg
>> JPRT - this includes CDS testing
>> performance runs in progress, including frequency of checking need to resize
>> 
>> thanks,
>> Karen
>> 
>> On Jan 21, 2012, at 12:04 PM, David Schlosnagle wrote:
>> 
>>> On Fri, Jan 20, 2012 at 6:06 PM, Karen Kinnear<karen.kinnear at oracle.com>  wrote:
>>>> Please review initial proposal for: 7114376: tune system dictionary size
>>>> 
>>>> http://oklahoma.us.oracle.com/~kmkinnea/webrev/sdresize/webrev/
>>> Karen,
>>> 
>>> Is there a URL where this webrev is accessible outside of Oracle?
>>> 
>>> Thanks,
>>> Dave


From dean.long at oracle.com  Tue Jan 24 20:32:51 2012
From: dean.long at oracle.com (Dean Long)
Date: Tue, 24 Jan 2012 20:32:51 -0800
Subject: Code and Design Feedback request: 7114376: tune system dictionary
	size
In-Reply-To: <61714338-2FBC-4B5A-9A22-6A3C0306DA56@oracle.com>
References: <73E0E1AF-1806-439B-B826-AE4E59590BFD@oracle.com>
	<CAGVnyzRHxg8SD5kx4duFCMuf7xzzaBGng_CBxC5n+DfmrzekZw@mail.gmail.com>
	<F45DFA65-925A-446D-99A4-1F0D7AEF8496@oracle.com>
	<4F1DEBC9.9030306@oracle.com>
	<61714338-2FBC-4B5A-9A22-6A3C0306DA56@oracle.com>
Message-ID: <4F1F85F3.8000707@oracle.com>

OK :-)

dl

On 1/24/2012 6:42 PM, Karen Kinnear wrote:
> Dean,
>
> That is actually a really good idea. True, it is out of scope for the current release timeframe, but once we have the permgen removal
> changes in, we will want to move more metadata to per classloader data structures that will more quickly free up when we unload classes/classloader.
>
> I filed RFE to capture this idea - 7133093 and put my name on it because it sounds quite promising :-)
>
> thanks!
> Karen
>
> On Jan 23, 2012, at 6:22 PM, Dean Long wrote:
>
>> It is probably out of the scope of this CR, but is it possible that the internal data structure might one day change from monolithic hash table to something more hierarchical?  For example having a separate data structure per package or per classloader.  I was reading online about something called a HAT-trie that deals with shared prefixes and was designed with cache locality in mind.
>>
>> dl
>>
>> On 1/21/2012 11:04 AM, Karen Kinnear wrote:
>>> David,
>>>
>>> Thank you for asking, my mistake. Here is an accessible webrev.
>>>
>>> Webrev: http://cr.openjdk.java.net/~acorn/7114376/webrev/
>>>
>>> An alternative possibility for supplying a single flag might be: -XX:LoadedClasses=#
>>>
>>> This could be an anticipated count of expected total classes, which we could use
>>> internally to set the initial SystemDictionary size. This might outlast any internal changes
>>> in data structures.
>>>
>>> Tests run:
>>> vmsqe: nsk.quick.testlist: solaris-sparc, fastdbg - this includes sa testing
>>> jck: vm,lang,api: solaris-sparc, fastdbg
>>> JPRT - this includes CDS testing
>>> performance runs in progress, including frequency of checking need to resize
>>>
>>> thanks,
>>> Karen
>>>
>>> On Jan 21, 2012, at 12:04 PM, David Schlosnagle wrote:
>>>
>>>> On Fri, Jan 20, 2012 at 6:06 PM, Karen Kinnear<karen.kinnear at oracle.com>   wrote:
>>>>> Please review initial proposal for: 7114376: tune system dictionary size
>>>>>
>>>>> http://oklahoma.us.oracle.com/~kmkinnea/webrev/sdresize/webrev/
>>>> Karen,
>>>>
>>>> Is there a URL where this webrev is accessible outside of Oracle?
>>>>
>>>> Thanks,
>>>> Dave

From james.melvin at oracle.com  Wed Jan 25 06:22:35 2012
From: james.melvin at oracle.com (James Melvin)
Date: Wed, 25 Jan 2012 09:22:35 -0500
Subject: NOTICE: JPRT now requires HotSpot build/test on Mac OS X
Message-ID: <4F20102B.1060403@oracle.com>

Hi,

Just a quick note that the following change has been pushed to
hotspot-rt and will soon percolate up to hotspot-main...

7126732 MAC: Require Mac OS X builds/tests for JPRT integrate jobs for 
HotSpot

This effectively adds Mac OS X to the default set of platforms upon
which to build HotSpot. All 'integrate' jobs must now pass builds and
tests on Mac OS X as a prerequisite to integrating any changeset(s). As
with other platforms, this will keep the Mac OS X port fresh and at Beta
quality or better at all times.

Lastly, this change will not affect job times. In fact, the mac-minis
we've deployed as JPRT clients are quite zippy and have demonstrated
some of the fastest builds of HotSpot in the farm.

Let me know if there are any immediate issues with this change.

Thanks,

Jim

From karen.kinnear at oracle.com  Wed Jan 25 07:55:06 2012
From: karen.kinnear at oracle.com (Karen Kinnear)
Date: Wed, 25 Jan 2012 10:55:06 -0500
Subject: Code and Design Feedback request: 7114376: tune system dictionary
	size
In-Reply-To: <4F1CBCDC.4060808@oracle.com>
References: <73E0E1AF-1806-439B-B826-AE4E59590BFD@oracle.com>
	<CAGVnyzRHxg8SD5kx4duFCMuf7xzzaBGng_CBxC5n+DfmrzekZw@mail.gmail.com>
	<F45DFA65-925A-446D-99A4-1F0D7AEF8496@oracle.com>
	<4F1CBCDC.4060808@oracle.com>
Message-ID: <400030A9-3FC7-4318-85F9-3097DC5638C3@oracle.com>

Folks,

Based on your helpful feedback, and help with benchmarking, I have redone the flags and resizing heuristics.
Webrev updated at:
    http://cr.openjdk.java.net/~acorn/7114376.01/webrev/

New flag: -XX:PredictedLoadedClassCount=#

I've left just the one flag - users can start with their actual loaded class count, but they can set that
number to whatever gives them the best performance. 

Testing:

Passed nsks (which included SA), jcks, jprt (which included CDS).
No performance difference for specjbb2005 on linux-amd64. 
volano2 appears to run 13% faster on linux-amd64 with dynamic resizing.
Large application runs in progress to determine initial tuning for default depth, so that may change.

thanks,
Karen

p.s. I filed 7133093 for suggestions on future improvements - thank you for those

On Jan 22, 2012, at 8:50 PM, David Holmes wrote:

> Hi Karen,
> 
> On 22/01/2012 5:04 AM, Karen Kinnear wrote:
>> Thank you for asking, my mistake. Here is an accessible webrev.
>> 
>> Webrev: http://cr.openjdk.java.net/~acorn/7114376/webrev/
> 
> systemDictionary.cpp:
> 
> I notice that in systemDictionary the calculation of the index has been moved inside the locked regions. This impacted some of the SD method signatures, depending on whether they grabbed the lock internally, or whether the lock was first acquired externally. Seems to me we could avoid a lot of duplicate index calculating code if we went a step further and changed all the methods to only take the hash and do their own index calculation if needed - in particular if find_class did the calculation then I think most of the other index calculations would disappear.
> 
> 1757 // Check if system dictionary should be resized to speed up find() calls
> 1758 // Default is to check on class unloading
> 1759 // SystemDictionarySizeIndex flag allows starting with a larger initial size
> 1760 // if SystemDictionaryAverageDepth is non-zero, also check on safepoint cleanup
> 1761 // and allow customers to select acceptable average depth to pass
> 1762 // to verify_lookup_length, otherwise use number_of_classes/table_size
> 1763 // verify_lookup_length compares actual average lookup depth: lookup_length/lookup_count
> 1764 // with theoretical or user-specified.
> 1765 // In future we may need to periodically calculate lookup_length to see
> 1766 // if we need to force a safepoint to resize system dictionary
> 1767 void SystemDictionary::should_resize_dictionary(int unloaded_count) {
> 
> I found the above comment block difficult to parse - perhaps some Capitals missing to indicate new points/sentences?
> 
> Also should this method assert that either the SD lock is held or else we're at a safepoint?
> 
> ---
> 
> dictionary.hpp/cpp
> 
> +   // Change size of system dictionary
> +   // Initially for growing when lookup time takes too long
> +   Dictionary* dictionary_resize(int newsize);
> 
> Is "system dictionary" distinct from SystemDictionary?
> 
> I don't like having this as an instance method as it doesn't resize the current dictionary but returns a new-one, leaving the caller to free the original, yet has the side-effect of clearing all entries from the current one. Seems to me we should simply have the caller do:
> 
> Dictionary* newdict = new Dictionary(newsize);
> olddict->copy_all_entries(newdict);
> delete olddict;
> olddict = newdict;
> 
> though perhaps copy_all_entries should really be move_all_entries?
> 
> 477 // This routine does not lock the system dictionary.
> ...
> 487 // Actually, the locking here is a bit trickier:
> ...
> 493 DictionaryEntry* Dictionary::get_entry(int index, unsigned int hash,
> 494                                        Symbol* class_name,
> 495                                        Handle class_loader) {
> 496   oop loader = class_loader();
> 497   _lookup_count++;
> 498
> 499   for (DictionaryEntry* entry = bucket(index);
> 500                         entry != NULL;
> 501                         entry = entry->next()) {
> 502     if (entry->hash() == hash && entry->equals(class_name, loader)) {
> 503       return entry;
> 504     }
> 505     _lookup_length++;
> 
> Your now non-debug counters seem to be being updated in a non-threadsafe manner - or are we assuming the lock has been taken in the caller?
> 
>> An alternative possibility for supplying a single flag might be: -XX:LoadedClasses=#
>> 
>> This could be an anticipated count of expected total classes, which we could use
>> internally to set the initial SystemDictionary size. This might outlast any internal changes
>> in data structures.
> 
> I think this might be a simpler starting point for things. Dynamically resizing the SD is transiently a bad thing for everyone:
> - pause while it happens
> - excessive memory use while it happens
> 
> better in my opinion to try to get the initial number of buckets right and not dynamically resize.
> 
> Cheers,
> David
> -----
> 
>> Tests run:
>> vmsqe: nsk.quick.testlist: solaris-sparc, fastdbg - this includes sa testing
>> jck: vm,lang,api: solaris-sparc, fastdbg
>> JPRT - this includes CDS testing
>> performance runs in progress, including frequency of checking need to resize
>> 
>> thanks,
>> Karen
>> 
>> On Jan 21, 2012, at 12:04 PM, David Schlosnagle wrote:
>> 
>>> On Fri, Jan 20, 2012 at 6:06 PM, Karen Kinnear<karen.kinnear at oracle.com>  wrote:
>>>> Please review initial proposal for: 7114376: tune system dictionary size
>>>> 
>>>> http://oklahoma.us.oracle.com/~kmkinnea/webrev/sdresize/webrev/
>>> 
>>> Karen,
>>> 
>>> Is there a URL where this webrev is accessible outside of Oracle?
>>> 
>>> Thanks,
>>> Dave
>> 


From tom.rodriguez at oracle.com  Wed Jan 25 10:40:46 2012
From: tom.rodriguez at oracle.com (Tom Rodriguez)
Date: Wed, 25 Jan 2012 10:40:46 -0800
Subject: cr.openjdk.java.net
Message-ID: <42E64EB1-462E-4102-BA7D-AE192ADECCDC@oracle.com>

Is anyone else having trouble uploading webrevs?  I tried uploading a webrev and got this:

rsync: mkstemp "/oj/home/never/7129164/src/share/vm/runtime/.safepoint.hpp.sdiff.html.hCaO1Z" failed: No space left on device (28)
rsync: mkstemp "/oj/home/never/7129164/src/share/vm/runtime/.safepoint.hpp.udiff.html.iCaO1Z" failed: No space left on device (28)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1039) [sender=3.0.6]

so I thought I'd try to delete some webrevs, rm'ing individual files doesn't work and neither does renaming directories to .trash.

sftp> rm test.c.html
Removing /oj/home/never/7089790_bsd_only/agent/src/os/bsd/test.c.html
Couldn't delete file: Failure

sftp> rename wmt .trash/wmt
Couldn't rename file "/oj/home/never/wmt" to "/oj/home/never/.trash/wmt": Failure

and the error messages are useless.

tom

From tony.printezis at oracle.com  Wed Jan 25 10:52:16 2012
From: tony.printezis at oracle.com (Tony Printezis)
Date: Wed, 25 Jan 2012 13:52:16 -0500
Subject: cr.openjdk.java.net
In-Reply-To: <42E64EB1-462E-4102-BA7D-AE192ADECCDC@oracle.com>
References: <42E64EB1-462E-4102-BA7D-AE192ADECCDC@oracle.com>
Message-ID: <4F204F60.6060009@oracle.com>

Tom,

I hit the same issue too.

Tony

On 1/25/2012 1:40 PM, Tom Rodriguez wrote:
> Is anyone else having trouble uploading webrevs?  I tried uploading a webrev and got this:
>
> rsync: mkstemp "/oj/home/never/7129164/src/share/vm/runtime/.safepoint.hpp.sdiff.html.hCaO1Z" failed: No space left on device (28)
> rsync: mkstemp "/oj/home/never/7129164/src/share/vm/runtime/.safepoint.hpp.udiff.html.iCaO1Z" failed: No space left on device (28)
> rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1039) [sender=3.0.6]
>
> so I thought I'd try to delete some webrevs, rm'ing individual files doesn't work and neither does renaming directories to .trash.
>
> sftp>  rm test.c.html
> Removing /oj/home/never/7089790_bsd_only/agent/src/os/bsd/test.c.html
> Couldn't delete file: Failure
>
> sftp>  rename wmt .trash/wmt
> Couldn't rename file "/oj/home/never/wmt" to "/oj/home/never/.trash/wmt": Failure
>
> and the error messages are useless.
>
> tom

From jiangli.zhou at oracle.com  Wed Jan 25 10:54:45 2012
From: jiangli.zhou at oracle.com (Jiangli Zhou)
Date: Wed, 25 Jan 2012 10:54:45 -0800
Subject: cr.openjdk.java.net
In-Reply-To: <42E64EB1-462E-4102-BA7D-AE192ADECCDC@oracle.com>
References: <42E64EB1-462E-4102-BA7D-AE192ADECCDC@oracle.com>
Message-ID: <4F204FF5.3030206@oracle.com>

Hi Tom,

I just ran into the same failure when trying to upload a webrev. 
Deleting a existing file or directory doesn't work either. Not sure who 
should be contacted for the problem.

sftp> rm 7117052 .trash/7117052
Removing /oj/home/jiangli/7117052
Couldn't delete file: Permission denied

Thanks,
Jiangli

On 01/25/2012 10:40 AM, Tom Rodriguez wrote:
> Is anyone else having trouble uploading webrevs?  I tried uploading a webrev and got this:
>
> rsync: mkstemp "/oj/home/never/7129164/src/share/vm/runtime/.safepoint.hpp.sdiff.html.hCaO1Z" failed: No space left on device (28)
> rsync: mkstemp "/oj/home/never/7129164/src/share/vm/runtime/.safepoint.hpp.udiff.html.iCaO1Z" failed: No space left on device (28)
> rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1039) [sender=3.0.6]
>
> so I thought I'd try to delete some webrevs, rm'ing individual files doesn't work and neither does renaming directories to .trash.
>
> sftp>  rm test.c.html
> Removing /oj/home/never/7089790_bsd_only/agent/src/os/bsd/test.c.html
> Couldn't delete file: Failure
>
> sftp>  rename wmt .trash/wmt
> Couldn't rename file "/oj/home/never/wmt" to "/oj/home/never/.trash/wmt": Failure
>
> and the error messages are useless.
>
> tom


From mark.reinhold at oracle.com  Wed Jan 25 10:56:13 2012
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Wed, 25 Jan 2012 10:56:13 -0800
Subject: cr.openjdk.java.net
In-Reply-To: tony.printezis@oracle.com; Wed, 25 Jan 2012 13:52:16 EST;
	<4F204F60.6060009@oracle.com>
Message-ID: <20120125185613.EFAE4D18@eggemoggin.niobe.net>

Like the message said, no space left on device.

Clearing old stuff out now ... try again in a few minutes.

- Mark

From paul.hohensee at oracle.com  Wed Jan 25 11:04:59 2012
From: paul.hohensee at oracle.com (Paul Hohensee)
Date: Wed, 25 Jan 2012 14:04:59 -0500
Subject: cr.openjdk.java.net
In-Reply-To: <42E64EB1-462E-4102-BA7D-AE192ADECCDC@oracle.com>
References: <42E64EB1-462E-4102-BA7D-AE192ADECCDC@oracle.com>
Message-ID: <4F20525B.9090706@oracle.com>

I just successfully renamed a bunch of old webrevs to .trash.

Paul

On 1/25/12 1:40 PM, Tom Rodriguez wrote:
> Is anyone else having trouble uploading webrevs?  I tried uploading a webrev and got this:
>
> rsync: mkstemp "/oj/home/never/7129164/src/share/vm/runtime/.safepoint.hpp.sdiff.html.hCaO1Z" failed: No space left on device (28)
> rsync: mkstemp "/oj/home/never/7129164/src/share/vm/runtime/.safepoint.hpp.udiff.html.iCaO1Z" failed: No space left on device (28)
> rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1039) [sender=3.0.6]
>
> so I thought I'd try to delete some webrevs, rm'ing individual files doesn't work and neither does renaming directories to .trash.
>
> sftp>  rm test.c.html
> Removing /oj/home/never/7089790_bsd_only/agent/src/os/bsd/test.c.html
> Couldn't delete file: Failure
>
> sftp>  rename wmt .trash/wmt
> Couldn't rename file "/oj/home/never/wmt" to "/oj/home/never/.trash/wmt": Failure
>
> and the error messages are useless.
>
> tom

From tom.rodriguez at oracle.com  Wed Jan 25 11:15:11 2012
From: tom.rodriguez at oracle.com (Tom Rodriguez)
Date: Wed, 25 Jan 2012 11:15:11 -0800
Subject: cr.openjdk.java.net
In-Reply-To: <20120125185613.EFAE4D18@eggemoggin.niobe.net>
References: <20120125185613.EFAE4D18@eggemoggin.niobe.net>
Message-ID: <BBAD0062-63B2-4BC9-BF3B-9AF039649161@oracle.com>

Thanks!  Is the lack of space why rename and rm don't work for me or is that unrelated?

tom

On Jan 25, 2012, at 10:56 AM, mark.reinhold at oracle.com wrote:

> Like the message said, no space left on device.
> 
> Clearing old stuff out now ... try again in a few minutes.
> 
> - Mark


From mark.reinhold at oracle.com  Wed Jan 25 11:20:45 2012
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Wed, 25 Jan 2012 11:20:45 -0800
Subject: cr.openjdk.java.net
In-Reply-To: tom.rodriguez@oracle.com; Wed, 25 Jan 2012 11:15:11 PST;
	<BBAD0062-63B2-4BC9-BF3B-9AF039649161@oracle.com>
Message-ID: <20120125192045.3B667D18@eggemoggin.niobe.net>

2012/1/25 11:15 -0800, tom.rodriguez at oracle.com:
> Thanks!  Is the lack of space why rename and rm don't work for me or is that unrelated?

Same problem.

- Mark

From daniel.daugherty at oracle.com  Wed Jan 25 16:58:55 2012
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Wed, 25 Jan 2012 17:58:55 -0700
Subject: Code and Design Feedback request: 7114376: tune system dictionary
	size
In-Reply-To: <400030A9-3FC7-4318-85F9-3097DC5638C3@oracle.com>
References: <73E0E1AF-1806-439B-B826-AE4E59590BFD@oracle.com>	<CAGVnyzRHxg8SD5kx4duFCMuf7xzzaBGng_CBxC5n+DfmrzekZw@mail.gmail.com>	<F45DFA65-925A-446D-99A4-1F0D7AEF8496@oracle.com>	<4F1CBCDC.4060808@oracle.com>
	<400030A9-3FC7-4318-85F9-3097DC5638C3@oracle.com>
Message-ID: <4F20A54F.8050604@oracle.com>

On 1/25/12 8:55 AM, Karen Kinnear wrote:
> Folks,
>
> Based on your helpful feedback, and help with benchmarking, I have redone the flags and resizing heuristics.
> Webrev updated at:
>      http://cr.openjdk.java.net/~acorn/7114376.01/webrev/

- feel free to ignore the nits
- thanks for updating the copyright lines

agent/src/share/classes/sun/jvm/hotspot/memory/LoaderConstraintTable.java
     No comments.

agent/src/share/classes/sun/jvm/hotspot/memory/SystemDictionary.java
     Should there be support for fetching and storing the new
     _sdgeneration and _primelist fields here?

src/share/vm/classfile/dictionary.hpp
     line 108: my brain is wondering why rename find() to find_unlocked()

src/share/vm/classfile/dictionary.cpp
     Now I see why find() was renamed to find_unlocked().

     nit: line 439 - extra blank before ','

     line 443 - do you want to assert() that the SD lock is not held?

     lines 448-451:  if find_unlocked() is called with search criteria
         that doesn't match anything and the SystemDictionary is resized,
         won't we get stuck in this loop since we never refetch an
         updated sdgeneration value?

     lines 444-451:
         I know that HotSpot style does not like do ... while loops, but
         this one seems to be begging for one. Please consider:

  444 DictionaryEntry* entry = NULL;
  445   do {
  446     int sdgeneration = SystemDictionary::generation();
  447     int index = hash_to_index(hash);
  448     entry = get_entry(index, hash, name, loader);
  449     // if NULL was returned and generation changed, then a resize
  450     // happened so we need to try again
  451   } while (entry == NULL && SystemDictionary::generation() > 
sdgeneration);

     nit: line 453 - extra blank after '='

     lines 468-474:
         I'm finding the new comment difficult to understand. Please 
consider:

     //
     // With resizable dictionary support, the locking requirements on
     // callers to this routine are a bit trickier:
     // - uses of this routine on shared_system_dictionary and placeholders
     //   do not need a lock (these are not resizable)
     // - uses of this routine on resizable dictionaries have to have a lock
     //   unless find_unlocked() is used to call this routine
     // - the find_unlocked() routine handles the case where an entry could
     //   not be found because the dictionary was resized and does a retry
     //   automatically

     nit line 642: you have spaces around one '=' in the output but not the
         other. The original code didn't have spaces around '='.

src/share/vm/classfile/systemDictionary.hpp
nit: 282 - why add a blank line?

     line 548 uses "SystemDictionary"; lines 553 and 591 use
         "system dictionary"; is there some semantic difference?

     line 555: does _sdgeneration have to be an "int" or is it bounded
         by PRIMEARRAYSIZE?

     line 592: if _sdgeneration type changes, then return type should
         change also.

     lines 668, 671: instead of "unsigned int", should the hash parameter
         be a new typedef'ed value to make future experiments easier?
         However, changing that would require updates in many places...

src/share/vm/classfile/systemDictionary.cpp
     line 67: if _sdgeneration type in .hpp file changes, then this def
         also has to change.

     nit: line 68: indent off by 1

nit: 86 - why add a blank line?

     nit: deleted blank line on 863 - why?

     lines 1772-1773: comment is not indented, but should be

     line 1780: return seems redundant here

     line 1908: indent of '= 0' is off by 1 space

src/share/vm/runtime/globals.hpp
     No comments.

src/share/vm/runtime/vmStructs.cpp
     No comments.

src/share/vm/utilities/hashtable.hpp
     nit: 150     if (NULL != _buckets) {
         This: 'if (_buckets != NULL)' seem more traditional.
         Any particular reason for your order?

     nit: 184 - why add a blank line?

src/share/vm/utilities/hashtable.inline.hpp
     nit: deleted blank lines on 67 and 92 - why?

     No other comments.

src/share/vm/utilities/hashtable.cpp
     No comments.


From john.pampuch at oracle.com  Thu Jan 26 12:26:43 2012
From: john.pampuch at oracle.com (John Pampuch)
Date: Thu, 26 Jan 2012 12:26:43 -0800
Subject: Code and Design Feedback request: 7114376: tune system dictionary
	size
In-Reply-To: <400030A9-3FC7-4318-85F9-3097DC5638C3@oracle.com>
References: <73E0E1AF-1806-439B-B826-AE4E59590BFD@oracle.com>	<CAGVnyzRHxg8SD5kx4duFCMuf7xzzaBGng_CBxC5n+DfmrzekZw@mail.gmail.com>	<F45DFA65-925A-446D-99A4-1F0D7AEF8496@oracle.com>	<4F1CBCDC.4060808@oracle.com>
	<400030A9-3FC7-4318-85F9-3097DC5638C3@oracle.com>
Message-ID: <4F21B703.3040202@oracle.com>

An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120126/fcbebd0d/attachment.html 

From karen.kinnear at oracle.com  Thu Jan 26 14:13:02 2012
From: karen.kinnear at oracle.com (Karen Kinnear)
Date: Thu, 26 Jan 2012 17:13:02 -0500
Subject: Code and Design Feedback request: 7114376: tune system dictionary
	size
In-Reply-To: <4F21B703.3040202@oracle.com>
References: <73E0E1AF-1806-439B-B826-AE4E59590BFD@oracle.com>	<CAGVnyzRHxg8SD5kx4duFCMuf7xzzaBGng_CBxC5n+DfmrzekZw@mail.gmail.com>	<F45DFA65-925A-446D-99A4-1F0D7AEF8496@oracle.com>	<4F1CBCDC.4060808@oracle.com>
	<400030A9-3FC7-4318-85F9-3097DC5638C3@oracle.com>
	<4F21B703.3040202@oracle.com>
Message-ID: <A60D8143-87B5-4811-A486-11DB9AE3492A@oracle.com>

John,

Good questions.

In order to make 7u4, I am going to back off the dynamic resizing. We will revisit that after we get the
permgen removal logic into jdk8.

For now, I am going to back off to just allowing setting the command-line value so we can tune
the fixed size of the system dictionary.
On Jan 26, 2012, at 3:26 PM, John Pampuch wrote:

> Karen-
> 
> I would have guessed that this change would have more affect on startup, and less on throughput on a benchmark like Volano.  Does this improvement make sense to you?  (In other words, what am I missing?)
I would not expect this to make much difference in startup - the issue arises when you load lots and lots of classes and do
lots and lots of Class.forName() or other lookups of loaded classes. At startup time, there are usually not that many classes loaded -
the overhead comes from looking up the classes when there have been hash conflicts.

So, yes, it is throughput that would benefit. That said, I would need to do more detailed analysis of the volano results before
I believed them.
> 
> Also, I'd be curious to know how much of an effect occurs by setting the parameter vs. leaving the default and letting the resizing work?  I'll talk to Jerry and see if we can find a little time to investigate. 
That would be good to get on the calendar for the next round.
> 
> Last thing, (and not urgent!): do we attempt to estimate the table size ergonomically, maybe based on things like core counts, memory available, -Xmx values, etc.? 
We estimate the table size based on the number of loaded classes at this point. There are lots of additional factors we
could consider in future, when we have time to do much more significant benchmarking and tuning. 
> 
> Thanks,
> 
> -John
thanks,
Karen
> 
> On 1/25/12 7:55 AM, Karen Kinnear wrote:
>> 
>> Folks,
>> 
>> Based on your helpful feedback, and help with benchmarking, I have redone the flags and resizing heuristics.
>> Webrev updated at:
>>     http://cr.openjdk.java.net/~acorn/7114376.01/webrev/
>> 
>> New flag: -XX:PredictedLoadedClassCount=#
>> 
>> I've left just the one flag - users can start with their actual loaded class count, but they can set that
>> number to whatever gives them the best performance. 
>> 
>> Testing:
>> 
>> Passed nsks (which included SA), jcks, jprt (which included CDS).
>> No performance difference for specjbb2005 on linux-amd64. 
>> volano2 appears to run 13% faster on linux-amd64 with dynamic resizing.
>> Large application runs in progress to determine initial tuning for default depth, so that may change.
>> 
>> thanks,
>> Karen
>> 
>> p.s. I filed 7133093 for suggestions on future improvements - thank you for those
>> 
>> On Jan 22, 2012, at 8:50 PM, David Holmes wrote:
>> 
>>> Hi Karen,
>>> 
>>> On 22/01/2012 5:04 AM, Karen Kinnear wrote:
>>>> Thank you for asking, my mistake. Here is an accessible webrev.
>>>> 
>>>> Webrev: http://cr.openjdk.java.net/~acorn/7114376/webrev/
>>> systemDictionary.cpp:
>>> 
>>> I notice that in systemDictionary the calculation of the index has been moved inside the locked regions. This impacted some of the SD method signatures, depending on whether they grabbed the lock internally, or whether the lock was first acquired externally. Seems to me we could avoid a lot of duplicate index calculating code if we went a step further and changed all the methods to only take the hash and do their own index calculation if needed - in particular if find_class did the calculation then I think most of the other index calculations would disappear.
>>> 
>>> 1757 // Check if system dictionary should be resized to speed up find() calls
>>> 1758 // Default is to check on class unloading
>>> 1759 // SystemDictionarySizeIndex flag allows starting with a larger initial size
>>> 1760 // if SystemDictionaryAverageDepth is non-zero, also check on safepoint cleanup
>>> 1761 // and allow customers to select acceptable average depth to pass
>>> 1762 // to verify_lookup_length, otherwise use number_of_classes/table_size
>>> 1763 // verify_lookup_length compares actual average lookup depth: lookup_length/lookup_count
>>> 1764 // with theoretical or user-specified.
>>> 1765 // In future we may need to periodically calculate lookup_length to see
>>> 1766 // if we need to force a safepoint to resize system dictionary
>>> 1767 void SystemDictionary::should_resize_dictionary(int unloaded_count) {
>>> 
>>> I found the above comment block difficult to parse - perhaps some Capitals missing to indicate new points/sentences?
>>> 
>>> Also should this method assert that either the SD lock is held or else we're at a safepoint?
>>> 
>>> ---
>>> 
>>> dictionary.hpp/cpp
>>> 
>>> +   // Change size of system dictionary
>>> +   // Initially for growing when lookup time takes too long
>>> +   Dictionary* dictionary_resize(int newsize);
>>> 
>>> Is "system dictionary" distinct from SystemDictionary?
>>> 
>>> I don't like having this as an instance method as it doesn't resize the current dictionary but returns a new-one, leaving the caller to free the original, yet has the side-effect of clearing all entries from the current one. Seems to me we should simply have the caller do:
>>> 
>>> Dictionary* newdict = new Dictionary(newsize);
>>> olddict->copy_all_entries(newdict);
>>> delete olddict;
>>> olddict = newdict;
>>> 
>>> though perhaps copy_all_entries should really be move_all_entries?
>>> 
>>> 477 // This routine does not lock the system dictionary.
>>> ...
>>> 487 // Actually, the locking here is a bit trickier:
>>> ...
>>> 493 DictionaryEntry* Dictionary::get_entry(int index, unsigned int hash,
>>> 494                                        Symbol* class_name,
>>> 495                                        Handle class_loader) {
>>> 496   oop loader = class_loader();
>>> 497   _lookup_count++;
>>> 498
>>> 499   for (DictionaryEntry* entry = bucket(index);
>>> 500                         entry != NULL;
>>> 501                         entry = entry->next()) {
>>> 502     if (entry->hash() == hash && entry->equals(class_name, loader)) {
>>> 503       return entry;
>>> 504     }
>>> 505     _lookup_length++;
>>> 
>>> Your now non-debug counters seem to be being updated in a non-threadsafe manner - or are we assuming the lock has been taken in the caller?
>>> 
>>>> An alternative possibility for supplying a single flag might be: -XX:LoadedClasses=#
>>>> 
>>>> This could be an anticipated count of expected total classes, which we could use
>>>> internally to set the initial SystemDictionary size. This might outlast any internal changes
>>>> in data structures.
>>> I think this might be a simpler starting point for things. Dynamically resizing the SD is transiently a bad thing for everyone:
>>> - pause while it happens
>>> - excessive memory use while it happens
>>> 
>>> better in my opinion to try to get the initial number of buckets right and not dynamically resize.
>>> 
>>> Cheers,
>>> David
>>> -----
>>> 
>>>> Tests run:
>>>> vmsqe: nsk.quick.testlist: solaris-sparc, fastdbg - this includes sa testing
>>>> jck: vm,lang,api: solaris-sparc, fastdbg
>>>> JPRT - this includes CDS testing
>>>> performance runs in progress, including frequency of checking need to resize
>>>> 
>>>> thanks,
>>>> Karen
>>>> 
>>>> On Jan 21, 2012, at 12:04 PM, David Schlosnagle wrote:
>>>> 
>>>>> On Fri, Jan 20, 2012 at 6:06 PM, Karen Kinnear<karen.kinnear at oracle.com>  wrote:
>>>>>> Please review initial proposal for: 7114376: tune system dictionary size
>>>>>> 
>>>>>> http://oklahoma.us.oracle.com/~kmkinnea/webrev/sdresize/webrev/
>>>>> Karen,
>>>>> 
>>>>> Is there a URL where this webrev is accessible outside of Oracle?
>>>>> 
>>>>> Thanks,
>>>>> Dave

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120126/b4192155/attachment-0001.html 

From john.coomes at oracle.com  Thu Jan 26 20:32:03 2012
From: john.coomes at oracle.com (john.coomes at oracle.com)
Date: Fri, 27 Jan 2012 04:32:03 +0000
Subject: hg: hsx/hotspot-main: Added tag jdk8-b23 for changeset 60d6f64a86b1
Message-ID: <20120127043203.75DC8471FF@hg.openjdk.java.net>

Changeset: 1a5f1d6b98d6
Author:    katleman
Date:      2012-01-26 18:23 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/rev/1a5f1d6b98d6

Added tag jdk8-b23 for changeset 60d6f64a86b1

! .hgtags


From john.coomes at oracle.com  Thu Jan 26 20:32:09 2012
From: john.coomes at oracle.com (john.coomes at oracle.com)
Date: Fri, 27 Jan 2012 04:32:09 +0000
Subject: hg: hsx/hotspot-main/corba: Added tag jdk8-b23 for changeset
	5218eb256658
Message-ID: <20120127043212.2FCDB47200@hg.openjdk.java.net>

Changeset: b98f0e6dddf9
Author:    katleman
Date:      2012-01-26 18:23 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/corba/rev/b98f0e6dddf9

Added tag jdk8-b23 for changeset 5218eb256658

! .hgtags


From john.coomes at oracle.com  Thu Jan 26 20:32:18 2012
From: john.coomes at oracle.com (john.coomes at oracle.com)
Date: Fri, 27 Jan 2012 04:32:18 +0000
Subject: hg: hsx/hotspot-main/jaxp: Added tag jdk8-b23 for changeset
	95102fd33418
Message-ID: <20120127043219.1ADA947201@hg.openjdk.java.net>

Changeset: 7836655e2495
Author:    katleman
Date:      2012-01-26 18:23 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jaxp/rev/7836655e2495

Added tag jdk8-b23 for changeset 95102fd33418

! .hgtags


From john.coomes at oracle.com  Thu Jan 26 20:32:27 2012
From: john.coomes at oracle.com (john.coomes at oracle.com)
Date: Fri, 27 Jan 2012 04:32:27 +0000
Subject: hg: hsx/hotspot-main/jaxws: Added tag jdk8-b23 for changeset
	25ce7a000487
Message-ID: <20120127043228.00A8B47202@hg.openjdk.java.net>

Changeset: e0d90803439b
Author:    katleman
Date:      2012-01-26 18:23 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jaxws/rev/e0d90803439b

Added tag jdk8-b23 for changeset 25ce7a000487

! .hgtags


From john.coomes at oracle.com  Thu Jan 26 20:33:35 2012
From: john.coomes at oracle.com (john.coomes at oracle.com)
Date: Fri, 27 Jan 2012 04:33:35 +0000
Subject: hg: hsx/hotspot-main/jdk: 43 new changesets
Message-ID: <20120127044106.C253447208@hg.openjdk.java.net>

Changeset: 44bd765c22f4
Author:    prr
Date:      2012-01-13 13:11 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/44bd765c22f4

7127827: JRE8: javaws fails to launch on oracle linux due to XRender
Reviewed-by: bae, jgodinez

! src/solaris/classes/sun/java2d/xr/XRCompositeManager.java

Changeset: b566004bcb1a
Author:    dbuck
Date:      2012-01-16 11:52 +0400
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/b566004bcb1a

7083621: Add fontconfig file for OEL6 and rename RH/O EL 5 file so that it is picked up for all 5.x updates
Reviewed-by: bae, prr

! make/sun/awt/Makefile

Changeset: 397667460892
Author:    lana
Date:      2012-01-18 11:27 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/397667460892

Merge

- test/tools/launcher/DefaultLocaleTest.sh

Changeset: e0f94b9c53a8
Author:    alexsch
Date:      2012-01-10 15:46 +0400
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/e0f94b9c53a8

7110815: closed/javax/swing/JSplitPane/4885629/bug4885629.java unstable on MacOS
Reviewed-by: kizune

+ test/javax/swing/JSplitPane/4885629/bug4885629.java

Changeset: 79d14e328670
Author:    alexsch
Date:      2012-01-10 17:11 +0400
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/79d14e328670

6505523: NullPointerException in BasicTreeUI when a node is removed by expansion listener
Reviewed-by: rupashka

! src/share/classes/javax/swing/plaf/basic/BasicTreeUI.java
+ test/javax/swing/JTree/6505523/bug6505523.java

Changeset: ce32a4e1be1d
Author:    alexsch
Date:      2012-01-13 12:39 +0400
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/ce32a4e1be1d

7121765: closed/javax/swing/JTextArea/4697612/bug4697612.java fails on MacOS on Aqua L&F
Reviewed-by: rupashka

+ test/javax/swing/JTextArea/4697612/bug4697612.java
+ test/javax/swing/JTextArea/4697612/bug4697612.txt

Changeset: 59b8875949e1
Author:    malenkov
Date:      2012-01-16 18:28 +0400
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/59b8875949e1

7122740: PropertyDescriptor Performance Slow
Reviewed-by: rupashka

! src/share/classes/com/sun/beans/TypeResolver.java

Changeset: 3e9d35e6ee4f
Author:    denis
Date:      2012-01-17 19:09 +0400
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/3e9d35e6ee4f

7110590: DnDMerlinQLTestsuite_DnDJTextArea test fails with an java.awt.dnd.InvalidDnDOperationException
Reviewed-by: art

! src/share/classes/java/awt/AWTKeyStroke.java

Changeset: 89bc9d08fe82
Author:    anthony
Date:      2012-01-18 19:09 +0400
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/89bc9d08fe82

7130662: GTK file dialog crashes with a NPE
Summary: Guard adding a back slash to the directory name with an if (!= null) check
Reviewed-by: anthony, art
Contributed-by: Matt <melkor at orangepalantir.org>

! src/solaris/classes/sun/awt/X11/GtkFileDialogPeer.java

Changeset: fe1278123fbb
Author:    lana
Date:      2012-01-18 11:41 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/fe1278123fbb

Merge

- test/tools/launcher/DefaultLocaleTest.sh

Changeset: 4d8b49a45cff
Author:    lana
Date:      2012-01-18 20:23 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/4d8b49a45cff

Merge


Changeset: 400cc379adb5
Author:    alanb
Date:      2012-01-06 15:00 +0000
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/400cc379adb5

7127235: (fs) NPE in Files.walkFileTree if cached attributes are GC'ed
Reviewed-by: forax, chegar

! src/share/classes/java/nio/file/FileTreeWalker.java

Changeset: cdc128128044
Author:    valeriep
Date:      2012-01-05 18:18 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/cdc128128044

6414899: P11Digest  should support cloning
Summary: Enhanced the PKCS11 Digest implementation to support cloning
Reviewed-by: vinnie

! make/sun/security/pkcs11/mapfile-vers
! src/share/classes/sun/security/pkcs11/P11Digest.java
! src/share/classes/sun/security/pkcs11/wrapper/PKCS11.java
! src/share/lib/security/sunpkcs11-solaris.cfg
! src/share/native/sun/security/pkcs11/wrapper/pkcs11wrapper.h
+ test/sun/security/pkcs11/MessageDigest/TestCloning.java

Changeset: e6ef778c1df4
Author:    valeriep
Date:      2012-01-06 11:02 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/e6ef778c1df4

Merge


Changeset: 6720ae7b1448
Author:    valeriep
Date:      2012-01-06 16:06 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/6720ae7b1448

7033170: Cipher.getMaxAllowedKeyLength(String) throws NoSuchAlgorithmException
Summary: Changed to always use full transformation as provider properties.
Reviewed-by: mullan

! src/share/classes/sun/security/pkcs11/SunPKCS11.java
! test/javax/crypto/Cipher/GetMaxAllowed.java

Changeset: 2050ff9dfc92
Author:    darcy
Date:      2012-01-06 18:47 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/2050ff9dfc92

7123649: Remove public modifier from Math.powerOfTwoF.
Reviewed-by: smarks, alanb

! src/share/classes/java/lang/Math.java

Changeset: 74c92c3e66ad
Author:    gadams
Date:      2012-01-09 19:33 +0000
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/74c92c3e66ad

7030573: test/java/io/FileInputStream/LargeFileAvailable.java fails when there is insufficient disk space
Reviewed-by: alanb

! test/java/io/FileInputStream/LargeFileAvailable.java

Changeset: 858038d89fd5
Author:    darcy
Date:      2012-01-09 15:54 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/858038d89fd5

7128441: StrictMath performance improvement note shared with Math
Reviewed-by: darcy
Contributed-by: Martin Desruisseaux <martin.desruisseaux at geomatys.fr>

! src/share/classes/java/lang/Math.java
! src/share/classes/java/lang/StrictMath.java

Changeset: dd69d3695cee
Author:    darcy
Date:      2012-01-09 20:14 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/dd69d3695cee

7128512: Javadoc typo in java.lang.invoke.MethodHandle
Reviewed-by: mduigou

! src/share/classes/java/lang/invoke/MethodHandle.java

Changeset: d72de8b3fe36
Author:    chegar
Date:      2012-01-10 10:57 +0000
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/d72de8b3fe36

7123415: Some cases of network interface indexes being read incorrectly
Reviewed-by: chegar
Contributed-by: brandon.passanisi at oracle.com

! src/solaris/native/java/net/net_util_md.c

Changeset: bba276a6aa0d
Author:    chegar
Date:      2012-01-10 12:48 +0000
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/bba276a6aa0d

7128584: Typo in sun.misc.VM's private directMemory field comment
Reviewed-by: forax, chegar
Contributed-by: Krystal Mok <sajia at taobao.com>

! src/share/classes/sun/misc/VM.java

Changeset: 49e64a8fc18f
Author:    darcy
Date:      2012-01-10 17:12 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/49e64a8fc18f

7112008: Javadoc for j.l.Object.finalize() vs JLS 12.6 Finalization of Class Instances
Reviewed-by: mduigou

! src/share/classes/java/lang/Object.java

Changeset: 62dbcbe4c446
Author:    darcy
Date:      2012-01-10 17:46 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/62dbcbe4c446

7128931: Bad HTML escaping in java.lang.Throwable javadoc
Reviewed-by: mduigou

! src/share/classes/java/lang/Throwable.java

Changeset: 31a1fc60a895
Author:    chegar
Date:      2012-01-11 10:52 +0000
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/31a1fc60a895

7128648: HttpURLConnection.getHeaderFields should return an unmodifiable Map
Reviewed-by: michaelm

! src/share/classes/sun/net/www/protocol/http/HttpURLConnection.java
+ test/java/net/HttpURLConnection/UnmodifiableMaps.java

Changeset: 82144054d2d8
Author:    alanb
Date:      2012-01-11 13:07 +0000
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/82144054d2d8

7068856: (fs) Typo in Files.isSameFile() javadoc
7099208: (fs) Files.newBufferedReader has typo in javadoc
Reviewed-by: forax

! src/share/classes/java/nio/file/Files.java
! src/share/classes/java/nio/file/Path.java

Changeset: 96fe796fd242
Author:    ksrini
Date:      2012-01-11 08:14 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/96fe796fd242

7125442: jar application located in two bytes character named folder cannot be run with JRE 7 u1/u2
Reviewed-by: sherman, mchung, darcy

! src/share/bin/java.c
+ test/tools/launcher/I18NJarTest.java
! test/tools/launcher/TestHelper.java

Changeset: 11e52d5ba64e
Author:    xuelei
Date:      2012-01-12 03:39 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/11e52d5ba64e

7106773: 512 bits RSA key cannot work with SHA384 and SHA512
Reviewed-by: weijun

! src/share/classes/sun/security/pkcs11/P11Cipher.java
! src/share/classes/sun/security/pkcs11/P11Key.java
! src/share/classes/sun/security/pkcs11/P11RSACipher.java
! src/share/classes/sun/security/pkcs11/P11Signature.java
! src/share/classes/sun/security/ssl/ClientHandshaker.java
! src/share/classes/sun/security/ssl/ServerHandshaker.java
! src/share/classes/sun/security/ssl/SignatureAndHashAlgorithm.java
! src/share/classes/sun/security/util/DisabledAlgorithmConstraints.java
+ src/share/classes/sun/security/util/KeyLength.java
+ src/share/classes/sun/security/util/Length.java
! src/windows/classes/sun/security/mscapi/Key.java
! src/windows/classes/sun/security/mscapi/RSACipher.java
! src/windows/classes/sun/security/mscapi/RSASignature.java
+ test/sun/security/mscapi/ShortRSAKey1024.sh
+ test/sun/security/mscapi/ShortRSAKey512.sh
+ test/sun/security/mscapi/ShortRSAKey768.sh
+ test/sun/security/mscapi/ShortRSAKeyWithinTLS.java
! test/sun/security/pkcs11/KeyStore/ClientAuth.java
! test/sun/security/pkcs11/KeyStore/ClientAuth.sh
! test/sun/security/ssl/javax/net/ssl/SSLContextVersion.java
+ test/sun/security/ssl/javax/net/ssl/TLSv12/ShortRSAKey512.java

Changeset: 38bf1e9b6979
Author:    weijun
Date:      2012-01-13 09:50 +0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/38bf1e9b6979

7090565: Move test/closed/javax/security/auth/x500/X500Principal/Parse.java to open tests
Reviewed-by: mullan

+ test/javax/security/auth/x500/X500Principal/NameFormat.java

Changeset: ef3b6736c074
Author:    valeriep
Date:      2012-01-12 16:04 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/ef3b6736c074

7088989: Improve the performance for T4 by utilizing the newly provided crypto APIs
Summary: Added the OracleUcrypto provider for utilizing the Solaris ucrypto API.
Reviewed-by: weijun

! make/com/oracle/Makefile
+ make/com/oracle/net/Makefile
+ make/com/oracle/nio/Makefile
+ make/com/oracle/security/ucrypto/FILES_c.gmk
+ make/com/oracle/security/ucrypto/Makefile
+ make/com/oracle/security/ucrypto/mapfile-vers
+ make/com/oracle/util/Makefile
! src/share/lib/security/java.security-solaris
! test/Makefile
+ test/com/oracle/security/ucrypto/TestAES.java
+ test/com/oracle/security/ucrypto/TestDigest.java
+ test/com/oracle/security/ucrypto/TestRSA.java
+ test/com/oracle/security/ucrypto/UcryptoTest.java
! test/java/security/Provider/DefaultPKCS11.java

Changeset: a7ad2fcd7291
Author:    valeriep
Date:      2012-01-12 18:49 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/a7ad2fcd7291

Merge


Changeset: 7e593aa6ad41
Author:    littlee
Date:      2012-01-13 13:20 +0000
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/7e593aa6ad41

7129029: (fs) Unix file system provider should be buildable on platforms that don't support O_NOFOLLOW
Reviewed-by: alanb

! src/solaris/classes/sun/nio/fs/UnixChannelFactory.java
! src/solaris/classes/sun/nio/fs/UnixFileSystemProvider.java
! src/solaris/classes/sun/nio/fs/UnixNativeDispatcher.java
! src/solaris/classes/sun/nio/fs/UnixPath.java
! src/solaris/native/sun/nio/fs/genUnixConstants.c

Changeset: e8e08d46cc37
Author:    weijun
Date:      2012-01-16 10:10 +0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/e8e08d46cc37

7118809: rcache deadlock
Reviewed-by: valeriep

! src/share/classes/sun/security/krb5/internal/rcache/CacheTable.java
! src/share/classes/sun/security/krb5/internal/rcache/ReplayCache.java
! test/sun/security/krb5/auto/Context.java
+ test/sun/security/krb5/auto/ReplayCache.java

Changeset: d1b0bda3a3c7
Author:    alanb
Date:      2012-01-16 16:30 +0000
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/d1b0bda3a3c7

7130398: ProblemList.txt updates (1/2012)
Reviewed-by: chegar

! test/ProblemList.txt

Changeset: e8a143213c65
Author:    chegar
Date:      2012-01-16 18:05 +0000
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/e8a143213c65

7129083: CookieManager does not store cookies if url is read before setting cookie manager
Reviewed-by: michaelm

! src/share/classes/sun/net/www/http/HttpClient.java
! src/share/classes/sun/net/www/protocol/http/HttpURLConnection.java
! src/share/classes/sun/net/www/protocol/https/HttpsClient.java
+ test/sun/net/www/http/HttpClient/CookieHttpClientTest.java
+ test/sun/security/ssl/sun/net/www/protocol/https/HttpsURLConnection/CookieHttpsClientTest.java

Changeset: 40d699d7f6a1
Author:    chegar
Date:      2012-01-17 14:10 +0000
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/40d699d7f6a1

6671616: TEST_BUG: java/io/File/BlockIsDirectory.java fails when /dev/dsk empty (sol)
Reviewed-by: alanb

! test/ProblemList.txt
- test/java/io/File/BlockIsDirectory.java

Changeset: 2f096eb72520
Author:    mchung
Date:      2012-01-17 15:55 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/2f096eb72520

7117570: Warnings in sun.mangement.* and its subpackages
Reviewed-by: mchung, dsamersoff
Contributed-by: kurchi.subhra.hazra at oracle.com

! src/share/classes/sun/management/Agent.java
! src/share/classes/sun/management/ConnectorAddressLink.java
! src/share/classes/sun/management/Flag.java
! src/share/classes/sun/management/GarbageCollectionNotifInfoCompositeData.java
! src/share/classes/sun/management/GarbageCollectorImpl.java
! src/share/classes/sun/management/GcInfoBuilder.java
! src/share/classes/sun/management/GcInfoCompositeData.java
! src/share/classes/sun/management/HotSpotDiagnostic.java
! src/share/classes/sun/management/HotspotCompilation.java
! src/share/classes/sun/management/HotspotThread.java
! src/share/classes/sun/management/LazyCompositeData.java
! src/share/classes/sun/management/ManagementFactoryHelper.java
! src/share/classes/sun/management/MappedMXBeanType.java
! src/share/classes/sun/management/MonitorInfoCompositeData.java
! src/share/classes/sun/management/NotificationEmitterSupport.java
! src/share/classes/sun/management/RuntimeImpl.java
! src/share/classes/sun/management/ThreadInfoCompositeData.java
! src/share/classes/sun/management/counter/perf/PerfInstrumentation.java
! src/share/classes/sun/management/jmxremote/ConnectorBootstrap.java
! src/share/classes/sun/management/snmp/AdaptorBootstrap.java
! src/share/classes/sun/management/snmp/jvminstr/JVM_MANAGEMENT_MIB_IMPL.java
! src/share/classes/sun/management/snmp/jvminstr/JvmMemGCTableMetaImpl.java
! src/share/classes/sun/management/snmp/jvminstr/JvmMemManagerTableMetaImpl.java
! src/share/classes/sun/management/snmp/jvminstr/JvmMemMgrPoolRelTableMetaImpl.java
! src/share/classes/sun/management/snmp/jvminstr/JvmMemPoolTableMetaImpl.java
! src/share/classes/sun/management/snmp/jvminstr/JvmMemoryImpl.java
! src/share/classes/sun/management/snmp/jvminstr/JvmMemoryMetaImpl.java
! src/share/classes/sun/management/snmp/jvminstr/JvmOSImpl.java
! src/share/classes/sun/management/snmp/jvminstr/JvmRTBootClassPathEntryImpl.java
! src/share/classes/sun/management/snmp/jvminstr/JvmRTBootClassPathTableMetaImpl.java
! src/share/classes/sun/management/snmp/jvminstr/JvmRTClassPathEntryImpl.java
! src/share/classes/sun/management/snmp/jvminstr/JvmRTClassPathTableMetaImpl.java
! src/share/classes/sun/management/snmp/jvminstr/JvmRTInputArgsEntryImpl.java
! src/share/classes/sun/management/snmp/jvminstr/JvmRTInputArgsTableMetaImpl.java
! src/share/classes/sun/management/snmp/jvminstr/JvmRTLibraryPathEntryImpl.java
! src/share/classes/sun/management/snmp/jvminstr/JvmRTLibraryPathTableMetaImpl.java
! src/share/classes/sun/management/snmp/jvminstr/JvmRuntimeMetaImpl.java
! src/share/classes/sun/management/snmp/jvminstr/JvmThreadInstanceEntryImpl.java
! src/share/classes/sun/management/snmp/jvminstr/JvmThreadInstanceTableMetaImpl.java
! src/share/classes/sun/management/snmp/jvminstr/JvmThreadingMetaImpl.java
! src/share/classes/sun/management/snmp/jvmmib/EnumJvmClassesVerboseLevel.java
! src/share/classes/sun/management/snmp/jvmmib/EnumJvmJITCompilerTimeMonitoring.java
! src/share/classes/sun/management/snmp/jvmmib/EnumJvmMemManagerState.java
! src/share/classes/sun/management/snmp/jvmmib/EnumJvmMemPoolCollectThreshdSupport.java
! src/share/classes/sun/management/snmp/jvmmib/EnumJvmMemPoolState.java
! src/share/classes/sun/management/snmp/jvmmib/EnumJvmMemPoolThreshdSupport.java
! src/share/classes/sun/management/snmp/jvmmib/EnumJvmMemPoolType.java
! src/share/classes/sun/management/snmp/jvmmib/EnumJvmMemoryGCCall.java
! src/share/classes/sun/management/snmp/jvmmib/EnumJvmMemoryGCVerboseLevel.java
! src/share/classes/sun/management/snmp/jvmmib/EnumJvmRTBootClassPathSupport.java
! src/share/classes/sun/management/snmp/jvmmib/EnumJvmThreadContentionMonitoring.java
! src/share/classes/sun/management/snmp/jvmmib/EnumJvmThreadCpuTimeMonitoring.java
! src/share/classes/sun/management/snmp/jvmmib/JVM_MANAGEMENT_MIB.java
! src/share/classes/sun/management/snmp/jvmmib/JVM_MANAGEMENT_MIBOidTable.java
! src/share/classes/sun/management/snmp/jvmmib/JvmClassLoadingMeta.java
! src/share/classes/sun/management/snmp/jvmmib/JvmCompilationMeta.java
! src/share/classes/sun/management/snmp/jvmmib/JvmMemGCEntryMeta.java
! src/share/classes/sun/management/snmp/jvmmib/JvmMemGCTableMeta.java
! src/share/classes/sun/management/snmp/jvmmib/JvmMemManagerEntryMeta.java
! src/share/classes/sun/management/snmp/jvmmib/JvmMemManagerTableMeta.java
! src/share/classes/sun/management/snmp/jvmmib/JvmMemMgrPoolRelEntryMeta.java
! src/share/classes/sun/management/snmp/jvmmib/JvmMemMgrPoolRelTableMeta.java
! src/share/classes/sun/management/snmp/jvmmib/JvmMemPoolEntryMeta.java
! src/share/classes/sun/management/snmp/jvmmib/JvmMemPoolTableMeta.java
! src/share/classes/sun/management/snmp/jvmmib/JvmOSMeta.java
! src/share/classes/sun/management/snmp/jvmmib/JvmRTBootClassPathEntryMeta.java
! src/share/classes/sun/management/snmp/jvmmib/JvmRTBootClassPathTableMeta.java
! src/share/classes/sun/management/snmp/jvmmib/JvmRTClassPathEntryMeta.java
! src/share/classes/sun/management/snmp/jvmmib/JvmRTClassPathTableMeta.java
! src/share/classes/sun/management/snmp/jvmmib/JvmRTInputArgsEntryMeta.java
! src/share/classes/sun/management/snmp/jvmmib/JvmRTInputArgsTableMeta.java
! src/share/classes/sun/management/snmp/jvmmib/JvmRTLibraryPathEntryMeta.java
! src/share/classes/sun/management/snmp/jvmmib/JvmRTLibraryPathTableMeta.java
! src/share/classes/sun/management/snmp/jvmmib/JvmRuntimeMeta.java
! src/share/classes/sun/management/snmp/jvmmib/JvmThreadInstanceEntryMeta.java
! src/share/classes/sun/management/snmp/jvmmib/JvmThreadInstanceTableMeta.java
! src/share/classes/sun/management/snmp/jvmmib/JvmThreadingMeta.java
! src/share/classes/sun/management/snmp/util/MibLogger.java
! src/share/classes/sun/management/snmp/util/SnmpListTableCache.java
! src/share/classes/sun/management/snmp/util/SnmpNamedListTableCache.java
! src/share/classes/sun/management/snmp/util/SnmpTableCache.java

Changeset: b14e13237498
Author:    lana
Date:      2012-01-18 11:00 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/b14e13237498

Merge


Changeset: e6614f361127
Author:    lana
Date:      2012-01-18 20:24 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/e6614f361127

Merge

- test/java/io/File/BlockIsDirectory.java

Changeset: 227fcf5d0bec
Author:    lana
Date:      2012-01-24 13:43 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/227fcf5d0bec

Merge

- test/java/io/File/BlockIsDirectory.java

Changeset: 954a1c535730
Author:    amurillo
Date:      2012-01-25 12:36 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/954a1c535730

Merge

- test/java/io/File/BlockIsDirectory.java

Changeset: d3b334e376d3
Author:    mr
Date:      2012-01-23 12:39 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/d3b334e376d3

7110396: Sound code fails to build with gcc 4.6 on multiarch Linux systems
Reviewed-by: ohair

! make/javax/sound/jsoundalsa/Makefile

Changeset: 54202e0148ec
Author:    katleman
Date:      2012-01-25 13:54 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/54202e0148ec

Merge


Changeset: 34029a0c69bb
Author:    katleman
Date:      2012-01-26 18:23 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/jdk/rev/34029a0c69bb

Added tag jdk8-b23 for changeset 54202e0148ec

! .hgtags


From john.r.rose at oracle.com  Thu Jan 26 23:36:23 2012
From: john.r.rose at oracle.com (john.r.rose at oracle.com)
Date: Fri, 27 Jan 2012 07:36:23 +0000
Subject: hg: hsx/hotspot-main/hotspot: 12 new changesets
Message-ID: <20120127073649.3BA1A4721D@hg.openjdk.java.net>

Changeset: 53a127075045
Author:    kvn
Date:      2012-01-20 09:43 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/53a127075045

7131302: connode.cpp:205 Error: ShouldNotReachHere()
Summary: Add Value() methods to short and byte Load nodes to truncate constants which does not fit.
Reviewed-by: jrose

! src/share/vm/opto/memnode.cpp
! src/share/vm/opto/memnode.hpp

Changeset: 9164b8236699
Author:    iveresov
Date:      2012-01-20 15:02 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/9164b8236699

7131028: Switch statement takes wrong path
Summary: Pass correct type to branch in LIRGenerator::do_SwitchRanges()
Reviewed-by: kvn, never

! src/share/vm/c1/c1_LIR.hpp
! src/share/vm/c1/c1_LIRGenerator.cpp

Changeset: a81f60ddab06
Author:    never
Date:      2012-01-22 14:03 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/a81f60ddab06

7130676: Tiered: assert(bci == 0 || 0<= bci && bci<code_size()) during stack trace construction
Reviewed-by: kvn, iveresov

! src/share/vm/classfile/javaClasses.cpp

Changeset: 82e5a84b7436
Author:    bdelsart
Date:      2012-01-24 15:41 +0100
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/82e5a84b7436

7120450: complete information dumped by frame_describe
Summary: improvements of frame_describe
Reviewed-by: never, twisti

! src/cpu/sparc/vm/frame_sparc.cpp
! src/cpu/sparc/vm/methodHandles_sparc.cpp
! src/cpu/sparc/vm/methodHandles_sparc.hpp
! src/cpu/x86/vm/frame_x86.cpp
! src/cpu/x86/vm/methodHandles_x86.cpp
! src/cpu/x86/vm/methodHandles_x86.hpp
! src/share/vm/runtime/frame.cpp

Changeset: dddf0be88eb1
Author:    iveresov
Date:      2012-01-24 17:00 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/dddf0be88eb1

7132945: Tiered: adjust OSR threshold of level 3
Summary: Change the value of Tier3BackEdgeThreshold from 7000 to 60000
Reviewed-by: kvn, never

! src/share/vm/runtime/globals.hpp

Changeset: cf407b7d3d78
Author:    roland
Date:      2012-01-25 09:31 +0100
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/cf407b7d3d78

7116050: C2/ARM: memory stomping error with DivideMcTests
Summary: Block::schedule_local() may write beyond end of ready_cnt array
Reviewed-by: never, kvn

! src/share/vm/opto/block.hpp
! src/share/vm/opto/gcm.cpp
! src/share/vm/opto/lcm.cpp

Changeset: 52474ec73861
Author:    kvn
Date:      2012-01-24 17:04 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/52474ec73861

7132936: guarantee(t != NULL) failed: must be con
Summary: Use !higher_equal() check to skip top and values which fit.
Reviewed-by: never

! src/share/vm/opto/memnode.cpp

Changeset: 94f0ce74d48e
Author:    kvn
Date:      2012-01-25 08:10 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/94f0ce74d48e

Merge


Changeset: 9a28ddfc1f4a
Author:    roland
Date:      2012-01-26 09:38 +0100
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/9a28ddfc1f4a

7123910: Some CTW tests crash VM: is_loaded() && that->is_loaded()
Summary: handle not loaded array klass in Parse::do_checkcast().
Reviewed-by: kvn, never

! src/share/vm/opto/parseHelper.cpp

Changeset: 5dbed2f542ff
Author:    bdelsart
Date:      2012-01-26 16:49 +0100
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/5dbed2f542ff

7120468: SPARC/x86: use frame::describe to enhance trace_method_handle
Summary: improvements of TraceMethodHandles for JSR292
Reviewed-by: never, twisti

! src/cpu/sparc/vm/frame_sparc.cpp
! src/cpu/sparc/vm/methodHandles_sparc.cpp
! src/cpu/sparc/vm/methodHandles_sparc.hpp
! src/cpu/x86/vm/frame_x86.cpp
! src/cpu/x86/vm/methodHandles_x86.cpp
! src/cpu/x86/vm/methodHandles_x86.hpp
! src/cpu/zero/vm/frame_zero.cpp
! src/share/vm/runtime/frame.cpp
! src/share/vm/runtime/frame.hpp

Changeset: 20334ed5ed3c
Author:    iveresov
Date:      2012-01-26 12:15 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/20334ed5ed3c

7131259: compile_method and CompilationPolicy::event shouldn't be declared TRAPS
Summary: Make sure that CompilationPolicy::event() doesn't throw exceptions
Reviewed-by: kvn, never

! src/share/vm/c1/c1_Runtime1.cpp
! src/share/vm/compiler/compileBroker.cpp
! src/share/vm/compiler/compileBroker.hpp
! src/share/vm/interpreter/interpreterRuntime.cpp
! src/share/vm/runtime/advancedThresholdPolicy.cpp
! src/share/vm/runtime/advancedThresholdPolicy.hpp
! src/share/vm/runtime/compilationPolicy.cpp
! src/share/vm/runtime/compilationPolicy.hpp
! src/share/vm/runtime/simpleThresholdPolicy.cpp
! src/share/vm/runtime/simpleThresholdPolicy.hpp
! src/share/vm/utilities/exceptions.hpp

Changeset: 072384a61312
Author:    jrose
Date:      2012-01-26 19:39 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/072384a61312

Merge



From david.holmes at oracle.com  Thu Jan 26 23:45:08 2012
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 27 Jan 2012 17:45:08 +1000
Subject: Code and Design Feedback request: 7114376: tune system dictionary
	size
In-Reply-To: <4F20A54F.8050604@oracle.com>
References: <73E0E1AF-1806-439B-B826-AE4E59590BFD@oracle.com>	<CAGVnyzRHxg8SD5kx4duFCMuf7xzzaBGng_CBxC5n+DfmrzekZw@mail.gmail.com>	<F45DFA65-925A-446D-99A4-1F0D7AEF8496@oracle.com>	<4F1CBCDC.4060808@oracle.com>
	<400030A9-3FC7-4318-85F9-3097DC5638C3@oracle.com>
	<4F20A54F.8050604@oracle.com>
Message-ID: <4F225604.5080004@oracle.com>

Hi Karen,

Just a couple of comments building on Dan's comments.

On 26/01/2012 10:58 AM, Daniel D. Daugherty wrote:
> On 1/25/12 8:55 AM, Karen Kinnear wrote:
>> Folks,
>>
>> Based on your helpful feedback, and help with benchmarking, I have
>> redone the flags and resizing heuristics.
>> Webrev updated at:
>> http://cr.openjdk.java.net/~acorn/7114376.01/webrev/
> src/share/vm/classfile/dictionary.cpp
>
> lines 448-451: if find_unlocked() is called with search criteria
> that doesn't match anything and the SystemDictionary is resized,
> won't we get stuck in this loop since we never refetch an
> updated sdgeneration value?

I agree with Dan, in principle we'll get stuck. But I'm a little puzzled 
as to how, given:

  444   int sdgeneration = SystemDictionary::generation();
  445   int index = hash_to_index(hash);
  446   DictionaryEntry* entry = get_entry(index, hash, name, loader);
  447
  448   while (entry == NULL && SystemDictionary::generation() > 
sdgeneration) {

we can have reached a safepoint between reading the current generation 
at 444 and re-checking it at 448? Is this just being conservative incase 
there is something in hash_to_index or get_entry that might do a 
safepoint check? I presume/assume that get_entry can not be executing 
concurrently with a resize operation.

> lines 444-451:
> I know that HotSpot style does not like do ... while loops, but
> this one seems to be begging for one. Please consider:
>
> 444 DictionaryEntry* entry = NULL;
> 445 do {
> 446   int sdgeneration = SystemDictionary::generation();
> 447   int index = hash_to_index(hash);
> 448   entry = get_entry(index, hash, name, loader);
> 449   // if NULL was returned and generation changed, then a resize
> 450   // happened so we need to try again
> 451 } while (entry == NULL && SystemDictionary::generation() >
> sdgeneration);

+1

>
> nit: line 453 - extra blank after '='
>
> lines 468-474:
> I'm finding the new comment difficult to understand. Please consider:
>
> //
> // With resizable dictionary support, the locking requirements on
> // callers to this routine are a bit trickier:
> // - uses of this routine on shared_system_dictionary and placeholders
> // do not need a lock (these are not resizable)
> // - uses of this routine on resizable dictionaries have to have a lock
> // unless find_unlocked() is used to call this routine
> // - the find_unlocked() routine handles the case where an entry could
> // not be found because the dictionary was resized and does a retry
> // automatically

+1

Cheers,
David

From bengt.rutisson at oracle.com  Fri Jan 27 04:36:17 2012
From: bengt.rutisson at oracle.com (bengt.rutisson at oracle.com)
Date: Fri, 27 Jan 2012 12:36:17 +0000
Subject: hg: hsx/hotspot-main/hotspot: 14 new changesets
Message-ID: <20120127123650.5AE5F47226@hg.openjdk.java.net>

Changeset: 2e966d967c5c
Author:    johnc
Date:      2012-01-13 13:27 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/2e966d967c5c

7121547: G1: High number mispredicted branches while iterating over the marking bitmap
Summary: There is a high number of mispredicted branches associated with calling BitMap::iteratate() from within CMBitMapRO::iterate(). Implement a version of CMBitMapRO::iterate() directly using inline-able routines.
Reviewed-by: tonyp, iveresov

! src/share/vm/gc_implementation/g1/concurrentMark.cpp
! src/share/vm/gc_implementation/g1/concurrentMark.hpp
! src/share/vm/gc_implementation/g1/concurrentMark.inline.hpp
! src/share/vm/utilities/bitMap.inline.hpp

Changeset: 851b58c26def
Author:    brutisso
Date:      2012-01-16 11:21 +0100
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/851b58c26def

7130334: G1: Change comments and error messages that refer to CMS in g1/concurrentMark.cpp/hpp
Summary: Removed references to CMS in the concurrentMark.cpp/hpp files.
Reviewed-by: tonyp, jmasa, johnc

! src/share/vm/gc_implementation/g1/concurrentMark.cpp
! src/share/vm/gc_implementation/g1/concurrentMark.hpp

Changeset: 9509c20bba28
Author:    brutisso
Date:      2012-01-16 22:10 +0100
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/9509c20bba28

6976060: G1: humongous object allocations should initiate marking cycles when necessary
Reviewed-by: tonyp, johnc

! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp
! src/share/vm/gc_implementation/g1/g1CollectedHeap.hpp
! src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp
! src/share/vm/gc_implementation/g1/g1CollectorPolicy.hpp
! src/share/vm/gc_implementation/g1/vm_operations_g1.cpp
! src/share/vm/gc_interface/gcCause.cpp
! src/share/vm/gc_interface/gcCause.hpp

Changeset: 0b3d1ec6eaee
Author:    tonyp
Date:      2012-01-18 10:30 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/0b3d1ec6eaee

7097586: G1: improve the per-space output when using jmap -heap
Summary: Extend the jmap -heap output for G1 to include some more G1-specific information.
Reviewed-by: brutisso, johnc, poonam

! agent/src/share/classes/sun/jvm/hotspot/gc_implementation/g1/G1CollectedHeap.java
! agent/src/share/classes/sun/jvm/hotspot/gc_implementation/g1/G1MonitoringSupport.java
+ agent/src/share/classes/sun/jvm/hotspot/gc_implementation/g1/HeapRegionSetBase.java
! agent/src/share/classes/sun/jvm/hotspot/tools/HeapSummary.java
! src/share/vm/gc_implementation/g1/concurrentMark.cpp
! src/share/vm/gc_implementation/g1/heapRegionSet.hpp
! src/share/vm/gc_implementation/g1/vmStructs_g1.hpp

Changeset: 7ca7be5a6a0b
Author:    johnc
Date:      2012-01-17 10:21 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/7ca7be5a6a0b

7129271: G1: Interference from multiple threads in PrintGC/PrintGCDetails output
Summary: During an initial mark pause, signal the Concurrent Mark thread after the pause output from PrintGC/PrintGCDetails is complete.
Reviewed-by: tonyp, brutisso

! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp

Changeset: a8a126788ea0
Author:    tonyp
Date:      2012-01-19 09:13 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/a8a126788ea0

7078465: G1: Don't use the undefined value (-1) for the G1 old memory pool max size
Reviewed-by: johnc, brutisso

! src/share/vm/gc_implementation/g1/g1MonitoringSupport.hpp
! src/share/vm/services/g1MemoryPool.hpp

Changeset: 57025542827f
Author:    brutisso
Date:      2012-01-20 18:01 +0100
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/57025542827f

7131791: G1: Asserts in nightly testing due to 6976060
Summary: Create a handle and fake an object to make sure that we don't loose the memory we just allocated
Reviewed-by: tonyp, stefank

! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp

Changeset: 6a78aa6ac1ff
Author:    brutisso
Date:      2012-01-23 20:36 +0100
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/6a78aa6ac1ff

7132311: G1: assert((s == klass->oop_size(this)) || (Universe::heap()->is_gc_active() && ((is_typeArray()...
Summary: Move the check for when to call collect() to before we do a humongous object allocation
Reviewed-by: stefank, tonyp

! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp
! src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp
! src/share/vm/gc_implementation/g1/g1CollectorPolicy.hpp

Changeset: 877914d90c57
Author:    tonyp
Date:      2012-01-24 17:08 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/877914d90c57

7132398: G1: java.lang.IllegalArgumentException: Invalid threshold: 9223372036854775807 > max (1073741824)
Summary: Was not passing the right old pool max to the memory pool constructor in the fix for 7078465.
Reviewed-by: brutisso, johnc

! src/share/vm/services/g1MemoryPool.cpp

Changeset: d30fa85f9994
Author:    johnc
Date:      2012-01-12 00:06 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/d30fa85f9994

6484965: G1: piggy-back liveness accounting phase on marking
Summary: Remove the separate counting phase of concurrent marking by tracking the amount of marked bytes and the cards spanned by marked objects in marking task/worker thread local data structures, which are updated as individual objects are marked.
Reviewed-by: brutisso, tonyp

! src/share/vm/gc_implementation/g1/concurrentMark.cpp
! src/share/vm/gc_implementation/g1/concurrentMark.hpp
! src/share/vm/gc_implementation/g1/concurrentMark.inline.hpp
! src/share/vm/gc_implementation/g1/concurrentMarkThread.cpp
! src/share/vm/gc_implementation/g1/concurrentMarkThread.hpp
! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp
! src/share/vm/gc_implementation/g1/g1CollectedHeap.hpp
! src/share/vm/gc_implementation/g1/g1EvacFailure.hpp
! src/share/vm/gc_implementation/g1/g1OopClosures.hpp
! src/share/vm/gc_implementation/g1/heapRegion.hpp

Changeset: eff609af17d7
Author:    tonyp
Date:      2012-01-25 12:58 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/eff609af17d7

7127706: G1: re-enable survivors during the initial-mark pause
Summary: Re-enable survivors during the initial-mark pause. Afterwards, the concurrent marking threads have to scan them and mark everything reachable from them. The next GC will have to wait for the survivors to be scanned.
Reviewed-by: brutisso, johnc

! src/share/vm/gc_implementation/g1/concurrentMark.cpp
! src/share/vm/gc_implementation/g1/concurrentMark.hpp
! src/share/vm/gc_implementation/g1/concurrentMark.inline.hpp
! src/share/vm/gc_implementation/g1/concurrentMarkThread.cpp
! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp
! src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp
! src/share/vm/gc_implementation/g1/g1CollectorPolicy.hpp
! src/share/vm/gc_implementation/g1/g1EvacFailure.hpp
! src/share/vm/gc_implementation/g1/g1OopClosures.hpp
! src/share/vm/gc_implementation/g1/g1OopClosures.inline.hpp
! src/share/vm/gc_implementation/g1/heapRegion.inline.hpp
! src/share/vm/runtime/mutexLocker.cpp
! src/share/vm/runtime/mutexLocker.hpp

Changeset: a5244e07b761
Author:    jcoomes
Date:      2012-01-25 21:14 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/a5244e07b761

7112413: JVM Crash, possibly GC-related
Summary: disable UseAdaptiveSizePolicy with the CMS and ParNew
Reviewed-by: johnc, brutisso

! src/share/vm/runtime/arguments.cpp

Changeset: b4ebad3520bb
Author:    johnc
Date:      2012-01-26 14:14 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/b4ebad3520bb

7133038: G1: Some small profile based optimizations
Summary: Some minor profile based optimizations. Reduce the number of branches and branch mispredicts by removing some virtual calls, through closure specalization, and refactoring some conditional statements.
Reviewed-by: brutisso, tonyp

! src/share/vm/gc_implementation/g1/g1OopClosures.hpp
! src/share/vm/gc_implementation/g1/g1OopClosures.inline.hpp
! src/share/vm/gc_implementation/g1/g1RemSet.cpp
! src/share/vm/gc_implementation/g1/g1RemSet.hpp
! src/share/vm/gc_implementation/g1/g1RemSet.inline.hpp
! src/share/vm/gc_implementation/g1/g1_specialized_oop_closures.hpp
! src/share/vm/gc_implementation/g1/heapRegion.cpp

Changeset: 0a10d80352d5
Author:    brutisso
Date:      2012-01-27 09:04 +0100
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/0a10d80352d5

Merge

- src/os/bsd/vm/decoder_bsd.cpp
! src/share/vm/runtime/arguments.cpp
! src/share/vm/runtime/mutexLocker.cpp
! src/share/vm/runtime/mutexLocker.hpp


From keith.mcguigan at oracle.com  Fri Jan 27 11:17:27 2012
From: keith.mcguigan at oracle.com (keith.mcguigan at oracle.com)
Date: Fri, 27 Jan 2012 19:17:27 +0000
Subject: hg: hsx/hotspot-main/hotspot: 15 new changesets
Message-ID: <20120127191759.B11E647236@hg.openjdk.java.net>

Changeset: af739d5ab23c
Author:    bpittore
Date:      2012-01-21 23:02 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/af739d5ab23c

6972759: Step over not working after thrown exception and Pop
Summary: reset jvmtithreadstate exception state after frame pop and forceearlyreturn processed
Reviewed-by: minqi, dholmes, dlong
Contributed-by: bill.pittore at oracle.com

! src/share/vm/prims/jvmtiThreadState.cpp
! src/share/vm/prims/jvmtiThreadState.hpp

Changeset: 583b428aa858
Author:    coleenp
Date:      2012-01-23 17:45 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/583b428aa858

Merge

- src/os/bsd/vm/decoder_bsd.cpp

Changeset: d6660fedbab5
Author:    phh
Date:      2012-01-24 14:07 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/d6660fedbab5

7126732: MAC: Require Mac OS X builds/tests for JPRT integrate jobs for HotSpot
Summary: Modify jprt.properties to run OSX builds and tests.
Reviewed-by: dcubed, kamg, ohair, dholmes
Contributed-by: james.melvin at oracle.com

! make/jprt.properties

Changeset: bf864f701a4a
Author:    dsamersoff
Date:      2012-01-25 02:29 +0400
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/bf864f701a4a

7066129: GarbageCollectorMXBean#getLastGcInfo leaks native memory
Summary: Make GCStatInfo a resource object
Reviewed-by: phh, coleenp

! src/share/vm/services/gcNotifier.cpp
! src/share/vm/services/management.cpp
! src/share/vm/services/memoryManager.cpp
! src/share/vm/services/memoryManager.hpp

Changeset: df88f58f3b61
Author:    dsamersoff
Date:      2012-01-24 20:15 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/df88f58f3b61

Merge


Changeset: e8a4934564b2
Author:    phh
Date:      2012-01-24 19:33 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/e8a4934564b2

7125793: MAC: test_gamma should always work
Summary: Fix gamma launcher on Mac OS X and reconcile test_gamma script on Unix platforms
Reviewed-by: dcubed, ohair, jcoomes, dholmes, ksrini
Contributed-by: james.melvin at oracle.com

! make/bsd/Makefile
! make/bsd/makefiles/buildtree.make
! make/bsd/makefiles/defs.make
! make/bsd/makefiles/launcher.make
! make/bsd/makefiles/vm.make
! make/linux/makefiles/buildtree.make
! make/solaris/makefiles/buildtree.make
! src/os/bsd/vm/os_bsd.cpp
! src/os/posix/launcher/java_md.c

Changeset: 78dadb7b16ab
Author:    phh
Date:      2012-01-25 01:16 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/78dadb7b16ab

Merge


Changeset: d708a8cdd022
Author:    kamg
Date:      2012-01-25 10:08 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/d708a8cdd022

Merge


Changeset: 520830f632e7
Author:    fparain
Date:      2012-01-25 10:32 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/520830f632e7

7131346: Parsing of boolean arguments to diagnostic commands is broken
Reviewed-by: dholmes, dcubed

! src/share/vm/services/diagnosticArgument.cpp
! src/share/vm/utilities/globalDefinitions_visCPP.hpp

Changeset: 24ec1a6d6ef3
Author:    fparain
Date:      2012-01-25 16:33 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/24ec1a6d6ef3

Merge


Changeset: a42c07c38c47
Author:    dsamersoff
Date:      2012-01-25 21:10 +0400
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/a42c07c38c47

7132515: Add dcmd to manage UnlockingCommercialFeature flag
Summary: Added dcmd to unlock or check status of UnlockingCommercialFeature flag
Reviewed-by: fparain, rottenha

! src/share/vm/services/diagnosticCommand.cpp
! src/share/vm/services/diagnosticCommand.hpp
+ src/share/vm/services/diagnosticCommand_ext.hpp
! src/share/vm/services/diagnosticFramework.hpp
! src/share/vm/services/management.cpp

Changeset: 6d00795f99a1
Author:    dsamersoff
Date:      2012-01-25 15:03 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/6d00795f99a1

Merge


Changeset: 6db63e782d3d
Author:    dsamersoff
Date:      2012-01-25 18:58 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/6db63e782d3d

Merge


Changeset: de268c8a8075
Author:    phh
Date:      2012-01-26 20:06 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/de268c8a8075

7082553: Interpret Thread.setPriority(Thread.MAX_PRIORITY) to mean FX60 on Solaris 10 and 11
Summary: Add CriticalPriority == MaxPriority+1 and enable scheduling class as well as thread priority to change on Solaris.
Reviewed-by: dholmes, dcubed

! src/os/bsd/vm/os_bsd.cpp
! src/os/linux/vm/os_linux.cpp
! src/os/solaris/vm/osThread_solaris.hpp
! src/os/solaris/vm/os_solaris.cpp
! src/os/windows/vm/os_windows.cpp
! src/share/vm/compiler/compileBroker.cpp
! src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepThread.cpp
! src/share/vm/runtime/globals.hpp
! src/share/vm/runtime/os.hpp

Changeset: bf5da1648543
Author:    kamg
Date:      2012-01-27 10:42 -0500
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/bf5da1648543

Merge

! src/share/vm/compiler/compileBroker.cpp
! src/share/vm/runtime/globals.hpp


From vladimir.kozlov at oracle.com  Fri Jan 27 11:25:22 2012
From: vladimir.kozlov at oracle.com (Vladimir Kozlov)
Date: Fri, 27 Jan 2012 11:25:22 -0800
Subject: review for 7129164: JNI Get/ReleasePrimitiveArrayCritical doesn't
	scale
In-Reply-To: <4DDD98EA-07C9-44D9-AF7E-C938E086C6FC@oracle.com>
References: <4DDD98EA-07C9-44D9-AF7E-C938E086C6FC@oracle.com>
Message-ID: <4F22FA22.3000106@oracle.com>

I change alias to hotspot-dev since it affects everyone.

In GC_locker::check_active_before_gc() move wait_begin initialization under 
Print* check since it used only under such checks. Could method 
check_active_before_gc() be called  from different threads concurrently? Does it 
need lock? Note that other methods have lock.

Thanks,
Vladimir

Tom Rodriguez wrote:
> http://cr.openjdk.java.net/~never/7129164
> 200 lines changed: 126 ins; 33 del; 41 mod; 3958 unchg
> 
> 7129164: JNI Get/ReleasePrimitiveArrayCritical doesn't scale
> Summary:
> Reviewed-by:
> 
> The machinery for GC_locker which supports GetPrimitiveArrayCritical
> maintains a count of the number of threads that currently have raw
> pointers exposed.  This is used to allow existing critical sections to
> drain before creating new ones so that a GC can occur.  Currently
> these counts are updated using atomic all the time, even if a GC isn't
> being requested.  This creates scalability problem when a lot of
> threads are hammering atomic operations on the jni_lock_count.  The
> count only needs to be valid when checking if a critical section is
> currently active and when draining the existing sections.  The fix is
> to make the count be computed as part of the safepointing machinery
> and to only decrement the counters when _needs_gc is true.  In debug
> mode the old count is maintained and validated against the lazily
> computed count.
> 
> On a microbenchmark that stresses GetPrimitiveArrayCritical with many
> threads and relatively short critical section it can more than double
> the throughput.  This also slightly speeds up the normal
> GetPrimtiveArrayCritical calls.  Mostly it's not easily measurable
> with larger scale programs.
> 
> Tested with microbenchmark that stresses GetPrimtiveArrayCritical and
> the crypto benchmarks of specjvm2008 on x86 and sparc.  I also ran the
> java.io regression tests from the JDK.
> 

From tom.rodriguez at oracle.com  Fri Jan 27 11:53:17 2012
From: tom.rodriguez at oracle.com (Tom Rodriguez)
Date: Fri, 27 Jan 2012 11:53:17 -0800
Subject: review for 7129164: JNI Get/ReleasePrimitiveArrayCritical doesn't
	scale
In-Reply-To: <4F22FA22.3000106@oracle.com>
References: <4DDD98EA-07C9-44D9-AF7E-C938E086C6FC@oracle.com>
	<4F22FA22.3000106@oracle.com>
Message-ID: <77A3F194-FB36-446A-8691-FB7C4F45C021@oracle.com>


On Jan 27, 2012, at 11:25 AM, Vladimir Kozlov wrote:

> I change alias to hotspot-dev since it affects everyone.

Thanks.  I meant to send it to hotspot-dev but my auto pilot had other ideas.

> 
> In GC_locker::check_active_before_gc() move wait_begin initialization under Print* check since it used only under such checks.

Done.

> Could method check_active_before_gc() be called  from different threads concurrently? Does it need lock? Note that other methods have lock.

check_active_before_gc is only called at a safepoint:

  assert(SafepointSynchronize::is_at_safepoint(), "only read at safepoint");

And it's normally only called at the beginning of the collection in the VMThread.  The existing one had no locking and I don't think this changes anything in regards to the safety of it.

tom

> 
> Thanks,
> Vladimir
> 
> Tom Rodriguez wrote:
>> http://cr.openjdk.java.net/~never/7129164
>> 200 lines changed: 126 ins; 33 del; 41 mod; 3958 unchg
>> 7129164: JNI Get/ReleasePrimitiveArrayCritical doesn't scale
>> Summary:
>> Reviewed-by:
>> The machinery for GC_locker which supports GetPrimitiveArrayCritical
>> maintains a count of the number of threads that currently have raw
>> pointers exposed.  This is used to allow existing critical sections to
>> drain before creating new ones so that a GC can occur.  Currently
>> these counts are updated using atomic all the time, even if a GC isn't
>> being requested.  This creates scalability problem when a lot of
>> threads are hammering atomic operations on the jni_lock_count.  The
>> count only needs to be valid when checking if a critical section is
>> currently active and when draining the existing sections.  The fix is
>> to make the count be computed as part of the safepointing machinery
>> and to only decrement the counters when _needs_gc is true.  In debug
>> mode the old count is maintained and validated against the lazily
>> computed count.
>> On a microbenchmark that stresses GetPrimitiveArrayCritical with many
>> threads and relatively short critical section it can more than double
>> the throughput.  This also slightly speeds up the normal
>> GetPrimtiveArrayCritical calls.  Mostly it's not easily measurable
>> with larger scale programs.
>> Tested with microbenchmark that stresses GetPrimtiveArrayCritical and
>> the crypto benchmarks of specjvm2008 on x86 and sparc.  I also ran the
>> java.io regression tests from the JDK.


From tom.rodriguez at oracle.com  Fri Jan 27 11:56:24 2012
From: tom.rodriguez at oracle.com (Tom Rodriguez)
Date: Fri, 27 Jan 2012 11:56:24 -0800
Subject: review for 7129164: JNI Get/ReleasePrimitiveArrayCritical doesn't
	scale
In-Reply-To: <59B07013FD3B4667AD4C062A3DEAF324@oracle.com>
References: <4DDD98EA-07C9-44D9-AF7E-C938E086C6FC@oracle.com>
	<59B07013FD3B4667AD4C062A3DEAF324@oracle.com>
Message-ID: <00343F27-B713-42EA-BB18-B1807110E08F@oracle.com>


On Jan 27, 2012, at 11:49 AM, Igor Veresov wrote:

> gcLocker.cpp:  
> 
> 70 wait_begin = os::javaTimeMillis(); 
> 71 if (PrintJNIGCStalls && PrintGCDetails) {
> 
> 
> 
> I think the call to javaTimeMillis() can be moved inside the scope of the if since wait_begin is used only for printing. Also, may be wait_begin could be a member of GCLocker ?

Sure.  Just trying to avoid recompiles during development.

+  static jlong         _wait_begin;      // Timestamp for the setting of _needs_gc.                                                                    
+                                         // Used only by printing code.                                                                                

> 
> Otherwise looks good!

Thanks.

tom

> 
> igor
> 
> 
> On Friday, January 27, 2012 at 10:30 AM, Tom Rodriguez wrote:
> 
>> http://cr.openjdk.java.net/~never/7129164
>> 200 lines changed: 126 ins; 33 del; 41 mod; 3958 unchg
>> 
>> 7129164: JNI Get/ReleasePrimitiveArrayCritical doesn't scale
>> Summary:
>> Reviewed-by:
>> 
>> The machinery for GC_locker which supports GetPrimitiveArrayCritical
>> maintains a count of the number of threads that currently have raw
>> pointers exposed. This is used to allow existing critical sections to
>> drain before creating new ones so that a GC can occur. Currently
>> these counts are updated using atomic all the time, even if a GC isn't
>> being requested. This creates scalability problem when a lot of
>> threads are hammering atomic operations on the jni_lock_count. The
>> count only needs to be valid when checking if a critical section is
>> currently active and when draining the existing sections. The fix is
>> to make the count be computed as part of the safepointing machinery
>> and to only decrement the counters when _needs_gc is true. In debug
>> mode the old count is maintained and validated against the lazily
>> computed count.
>> 
>> On a microbenchmark that stresses GetPrimitiveArrayCritical with many
>> threads and relatively short critical section it can more than double
>> the throughput. This also slightly speeds up the normal
>> GetPrimtiveArrayCritical calls. Mostly it's not easily measurable
>> with larger scale programs.
>> 
>> Tested with microbenchmark that stresses GetPrimtiveArrayCritical and
>> the crypto benchmarks of specjvm2008 on x86 and sparc. I also ran the
>> java.io (http://java.io) regression tests from the JDK.
> 
> 
> 


From john.coomes at oracle.com  Fri Jan 27 12:23:05 2012
From: john.coomes at oracle.com (john.coomes at oracle.com)
Date: Fri, 27 Jan 2012 20:23:05 +0000
Subject: hg: hsx/hotspot-main/langtools: 8 new changesets
Message-ID: <20120127202325.8D05847237@hg.openjdk.java.net>

Changeset: 70d92518063e
Author:    mcimadamore
Date:      2012-01-11 18:23 +0000
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/langtools/rev/70d92518063e

7126754: Generics compilation failure casting List<? extends Set...> to List<Set...>
Summary: Problems with Types.rewriteQuantifiers not preserving variance
Reviewed-by: jjg

! src/share/classes/com/sun/tools/javac/code/Types.java
+ test/tools/javac/cast/7126754/T7126754.java
+ test/tools/javac/cast/7126754/T7126754.out

Changeset: 133744729455
Author:    mcimadamore
Date:      2012-01-12 15:28 +0000
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/langtools/rev/133744729455

7123100: javac fails with java.lang.StackOverflowError
Summary: Inference of under-constrained type-variables creates erroneous recursive wildcard types
Reviewed-by: jjg

! src/share/classes/com/sun/tools/javac/comp/Infer.java
+ test/tools/javac/cast/7123100/T7123100a.java
+ test/tools/javac/cast/7123100/T7123100a.out
+ test/tools/javac/cast/7123100/T7123100b.java
+ test/tools/javac/cast/7123100/T7123100b.out
+ test/tools/javac/cast/7123100/T7123100c.java
+ test/tools/javac/cast/7123100/T7123100c.out
+ test/tools/javac/cast/7123100/T7123100d.java
+ test/tools/javac/cast/7123100/T7123100d.out

Changeset: 1e2f4f4fb9f7
Author:    jjh
Date:      2012-01-17 17:14 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/langtools/rev/1e2f4f4fb9f7

7127924: langtools regression tests sometimes fail en-masse on windows
Reviewed-by: jjg

! test/tools/javac/diags/CheckExamples.java
! test/tools/javac/diags/MessageInfo.java
! test/tools/javac/diags/RunExamples.java

Changeset: f00afa80f1f0
Author:    lana
Date:      2012-01-18 11:00 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/langtools/rev/f00afa80f1f0

Merge


Changeset: cf2496340fef
Author:    darcy
Date:      2012-01-18 16:43 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/langtools/rev/cf2496340fef

7130768: Clarify behavior of Element.getEnclosingElements in subtypes
Reviewed-by: mcimadamore, jjg

! src/share/classes/javax/lang/model/element/Element.java
! src/share/classes/javax/lang/model/element/PackageElement.java
! src/share/classes/javax/lang/model/element/TypeElement.java

Changeset: 99261fc7d95d
Author:    jjh
Date:      2012-01-18 18:26 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/langtools/rev/99261fc7d95d

7131308: Three regression tests fail due to bad fix for 7127924
Reviewed-by: jjg

! test/tools/javac/diags/CheckExamples.java
! test/tools/javac/diags/MessageInfo.java
! test/tools/javac/diags/RunExamples.java

Changeset: 601ffcc6551d
Author:    lana
Date:      2012-01-24 13:44 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/langtools/rev/601ffcc6551d

Merge


Changeset: 6c9d21ca92c4
Author:    katleman
Date:      2012-01-26 18:23 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/langtools/rev/6c9d21ca92c4

Added tag jdk8-b23 for changeset 601ffcc6551d

! .hgtags


From daniel.daugherty at oracle.com  Fri Jan 27 12:41:44 2012
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Fri, 27 Jan 2012 13:41:44 -0700
Subject: NOTICE: JPRT now requires HotSpot build/test on Mac OS X
In-Reply-To: <4F20102B.1060403@oracle.com>
References: <4F20102B.1060403@oracle.com>
Message-ID: <4F230C08.9050702@oracle.com>

The following changeset is now in Main_Baseline:

Changeset: d6660fedbab5
Author:    phh
Date:      2012-01-24 14:07 -0500
URL:       
http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/d6660fedbab5

7126732: MAC: Require Mac OS X builds/tests for JPRT integrate jobs for 
HotSpot
Summary: Modify jprt.properties to run OSX builds and tests.
Reviewed-by: dcubed, kamg, ohair, dholmes
Contributed-by: james.melvin at oracle.com


This means that JPRT jobs destined for Main_Baseline now have
to build (and test) on MacOS X...

MacOS X is one step closer to being a fully supported HSX platform.

Dan


On 1/25/12 7:22 AM, James Melvin wrote:
> Hi,
>
> Just a quick note that the following change has been pushed to
> hotspot-rt and will soon percolate up to hotspot-main...
>
> 7126732 MAC: Require Mac OS X builds/tests for JPRT integrate jobs for 
> HotSpot
>
> This effectively adds Mac OS X to the default set of platforms upon
> which to build HotSpot. All 'integrate' jobs must now pass builds and
> tests on Mac OS X as a prerequisite to integrating any changeset(s). As
> with other platforms, this will keep the Mac OS X port fresh and at Beta
> quality or better at all times.
>
> Lastly, this change will not affect job times. In fact, the mac-minis
> we've deployed as JPRT clients are quite zippy and have demonstrated
> some of the fastest builds of HotSpot in the farm.
>
> Let me know if there are any immediate issues with this change.
>
> Thanks,
>
> Jim
>

From dbhole at redhat.com  Fri Jan 27 13:45:42 2012
From: dbhole at redhat.com (Deepak Bhole)
Date: Fri, 27 Jan 2012 16:45:42 -0500
Subject: Patch to fix build breakage with GCC 4.7
Message-ID: <20120127214542.GQ3418@redhat.com>

Hi,

HotSpot currently fails to build with GCC 4.7 due to 2 points where it
is using boolean rather than NULL. GCC 4.6 treated it as a warning --
4.7 treats it as an error.

Attached patch fixes the issue and makes it build again.

Please feel free to commit if OK, or let me know if it needs re-work.

Cheers,
Deepak
-------------- next part --------------
diff -up openjdk/hotspot/src/share/vm/oops/constantPoolOop.cpp.sav openjdk/hotspot/src/share/vm/oops/constantPoolOop.cpp
--- openjdk/hotspot/src/share/vm/oops/constantPoolOop.cpp.sav	2012-01-27 13:55:15.486395713 -0500
+++ openjdk/hotspot/src/share/vm/oops/constantPoolOop.cpp	2012-01-27 13:55:32.531114310 -0500
@@ -269,7 +269,7 @@ klassOop constantPoolOopDesc::klass_ref_
 methodOop constantPoolOopDesc::method_at_if_loaded(constantPoolHandle cpool,
                                                    int which, Bytecodes::Code invoke_code) {
   assert(!constantPoolCacheOopDesc::is_secondary_index(which), "no indy instruction here");
-  if (cpool->cache() == NULL)  return false;  // nothing to load yet
+  if (cpool->cache() == NULL)  return NULL;  // nothing to load yet
   int cache_index = which - CPCACHE_INDEX_TAG;
   if (!(cache_index >= 0 && cache_index < cpool->cache()->length())) {
     if (PrintMiscellaneous && (Verbose||WizardMode)) {
diff -up openjdk/hotspot/src/share/vm/opto/loopnode.cpp.sav openjdk/hotspot/src/share/vm/opto/loopnode.cpp
--- openjdk/hotspot/src/share/vm/opto/loopnode.cpp.sav	2012-01-27 13:55:43.251937313 -0500
+++ openjdk/hotspot/src/share/vm/opto/loopnode.cpp	2012-01-27 13:55:53.546767356 -0500
@@ -893,7 +893,7 @@ Node *LoopLimitNode::Identity( PhaseTran
 Node* CountedLoopNode::match_incr_with_optional_truncation(
                       Node* expr, Node** trunc1, Node** trunc2, const TypeInt** trunc_type) {
   // Quick cutouts:
-  if (expr == NULL || expr->req() != 3)  return false;
+  if (expr == NULL || expr->req() != 3)  return NULL;
 
   Node *t1 = NULL;
   Node *t2 = NULL;

From dbhole at redhat.com  Fri Jan 27 14:07:48 2012
From: dbhole at redhat.com (Deepak Bhole)
Date: Fri, 27 Jan 2012 17:07:48 -0500
Subject: Patch to fix build breakage with GCC 4.7
In-Reply-To: <20120127214542.GQ3418@redhat.com>
References: <20120127214542.GQ3418@redhat.com>
Message-ID: <20120127220748.GA12417@redhat.com>

* Deepak Bhole <dbhole at redhat.com> [2012-01-27 16:48]:
> Hi,
> 
> HotSpot currently fails to build with GCC 4.7 due to 2 points where it
> is using boolean rather than NULL. GCC 4.6 treated it as a warning --
> 4.7 treats it as an error.
> 
> Attached patch fixes the issue and makes it build again.
> 

Sorry, old habit of posting just the patch.. webrev is here for those
who prefer it:
http://cr.openjdk.java.net/~dbhole/GCC-4.7-HS23.00/

Cheers,
Deepak


From karen.kinnear at oracle.com  Fri Jan 27 15:13:04 2012
From: karen.kinnear at oracle.com (Karen Kinnear)
Date: Fri, 27 Jan 2012 18:13:04 -0500
Subject: Code and Design Feedback request: 7114376: tune system dictionary
	size
In-Reply-To: <4F20A54F.8050604@oracle.com>
References: <73E0E1AF-1806-439B-B826-AE4E59590BFD@oracle.com>	<CAGVnyzRHxg8SD5kx4duFCMuf7xzzaBGng_CBxC5n+DfmrzekZw@mail.gmail.com>	<F45DFA65-925A-446D-99A4-1F0D7AEF8496@oracle.com>	<4F1CBCDC.4060808@oracle.com>
	<400030A9-3FC7-4318-85F9-3097DC5638C3@oracle.com>
	<4F20A54F.8050604@oracle.com>
Message-ID: <C56B8E6F-383E-4ADB-873E-1660411D850D@oracle.com>

Dan & David - 

Thank you both for the detailed reviews.

I will make the changes suggested - but I will put the code changes on hold. I think
the best approach right now is to just add the command-line flag and then after
we get the permanent generation changes, rethink the loaded class cache storage
options - and do a whole lot more measurements than I am able to do in this timeframe -
even thanks to the help I have had.

Details below,
thanks,
Karen

On Jan 25, 2012, at 7:58 PM, Daniel D. Daugherty wrote:

> On 1/25/12 8:55 AM, Karen Kinnear wrote:
>> Folks,
>> 
>> Based on your helpful feedback, and help with benchmarking, I have redone the flags and resizing heuristics.
>> Webrev updated at:
>>     http://cr.openjdk.java.net/~acorn/7114376.01/webrev/
> 
> - feel free to ignore the nits
> - thanks for updating the copyright lines
Teachable :-)
> 
> agent/src/share/classes/sun/jvm/hotspot/memory/LoaderConstraintTable.java
>    No comments.
> 
> agent/src/share/classes/sun/jvm/hotspot/memory/SystemDictionary.java
>    Should there be support for fetching and storing the new
>    _sdgeneration and _primelist fields here?
good point. I will save the comment for when we revisit this.
> 
> src/share/vm/classfile/dictionary.hpp
>    line 108: my brain is wondering why rename find() to find_unlocked()
> 
> src/share/vm/classfile/dictionary.cpp
>    Now I see why find() was renamed to find_unlocked().
:-)
> 
>    nit: line 439 - extra blank before ','
fixed.
> 
>    line 443 - do you want to assert() that the SD lock is not held?
> 
>    lines 448-451:  if find_unlocked() is called with search criteria
>        that doesn't match anything and the SystemDictionary is resized,
>        won't we get stuck in this loop since we never refetch an
>        updated sdgeneration value?
Thank you - I will update the sdgeneration each time through. And
yes, David, this is paranoia in case of a safepoint or a change in when
we do the resizing in future.
> 
>    lines 444-451:
>        I know that HotSpot style does not like do ... while loops, but
>        this one seems to be begging for one. Please consider:
> 
> 444 DictionaryEntry* entry = NULL;
> 445   do {
> 446     int sdgeneration = SystemDictionary::generation();
> 447     int index = hash_to_index(hash);
> 448     entry = get_entry(index, hash, name, loader);
> 449     // if NULL was returned and generation changed, then a resize
> 450     // happened so we need to try again
> 451   } while (entry == NULL && SystemDictionary::generation() > sdgeneration);
Much better - thanks!
> 
>    nit: line 453 - extra blank after '='
fixed.
> 
>    lines 468-474:
>        I'm finding the new comment difficult to understand. Please consider:
> 
>    //
>    // With resizable dictionary support, the locking requirements on
>    // callers to this routine are a bit trickier:
>    // - uses of this routine on shared_system_dictionary and placeholders
>    //   do not need a lock (these are not resizable)
>    // - uses of this routine on resizable dictionaries have to have a lock
>    //   unless find_unlocked() is used to call this routine
// They also need to calculate the index while holding the lock, since
// the index might change if the system dictionary is resized.
>    // - the find_unlocked() routine handles the case where an entry could
>    //   not be found because the dictionary was resized and does a retry
>    //   automatically
Updated - and thanks.
> 
>    nit line 642: you have spaces around one '=' in the output but not the
>        other. The original code didn't have spaces around '='.
removed spaces.
> 
> src/share/vm/classfile/systemDictionary.hpp
> nit: 282 - why add a blank line?
fixed
> 
>    line 548 uses "SystemDictionary"; lines 553 and 591 use
>        "system dictionary"; is there some semantic difference?
made all lower case
> 
>    line 555: does _sdgeneration have to be an "int" or is it bounded
>        by PRIMEARRAYSIZE?
I'll leave this for now and save the comment for when we revisit this.
> 
>    line 592: if _sdgeneration type changes, then return type should
>        change also.
> 
>    lines 668, 671: instead of "unsigned int", should the hash parameter
>        be a new typedef'ed value to make future experiments easier?
>        However, changing that would require updates in many places...
Not touching hash parameter. We want to leave that as unsigned int.
> 
> src/share/vm/classfile/systemDictionary.cpp
>    line 67: if _sdgeneration type in .hpp file changes, then this def
>   also has to change.
True. Also true for all values initialized here.
> 
>    nit: line 68: indent off by 1
fixed
> 
> nit: 86 - why add a blank line?
fixed.
> 
>    nit: deleted blank line on 863 - why?
fixed.
> 
>    lines 1772-1773: comment is not indented, but should be
fixed.
> 
>    line 1780: return seems redundant here
Thanks - it used to return a value.
> 
>    line 1908: indent of '= 0' is off by 1 space
fixed.
> 
> src/share/vm/runtime/globals.hpp
>    No comments.
> 
> src/share/vm/runtime/vmStructs.cpp
>    No comments.
> 
> src/share/vm/utilities/hashtable.hpp
>    nit: 150     if (NULL != _buckets) {
>        This: 'if (_buckets != NULL)' seem more traditional.
>        Any particular reason for your order?
I put it to the more traditional. I tend to like to do my NULL comparisons backwards,
particularly for the == case for the times I accidentally type =. But no big deal.
> 
>    nit: 184 - why add a blank line?
Fixed. code moving/removing.
> 
> src/share/vm/utilities/hashtable.inline.hpp
>    nit: deleted blank lines on 67 and 92 - why?
Changes that didn't make the latest version. Undone.
> 
>    No other comments.
> 
> src/share/vm/utilities/hashtable.cpp
>    No comments.
> 


From john.coomes at oracle.com  Fri Jan 27 17:51:50 2012
From: john.coomes at oracle.com (john.coomes at oracle.com)
Date: Sat, 28 Jan 2012 01:51:50 +0000
Subject: hg: hsx/hsx23/hotspot: 45 new changesets
Message-ID: <20120128015327.81C8547242@hg.openjdk.java.net>

Changeset: 6edfe6e42a68
Author:    katleman
Date:      2012-01-26 18:23 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/6edfe6e42a68

Added tag jdk8-b23 for changeset e850d8e7ea54

! .hgtags

Changeset: 5f3fcd591768
Author:    amurillo
Date:      2012-01-20 17:07 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/5f3fcd591768

7131979: new hotspot build - hs23-b12
Reviewed-by: jcoomes

! make/hotspot_version

Changeset: 53a127075045
Author:    kvn
Date:      2012-01-20 09:43 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/53a127075045

7131302: connode.cpp:205 Error: ShouldNotReachHere()
Summary: Add Value() methods to short and byte Load nodes to truncate constants which does not fit.
Reviewed-by: jrose

! src/share/vm/opto/memnode.cpp
! src/share/vm/opto/memnode.hpp

Changeset: 9164b8236699
Author:    iveresov
Date:      2012-01-20 15:02 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/9164b8236699

7131028: Switch statement takes wrong path
Summary: Pass correct type to branch in LIRGenerator::do_SwitchRanges()
Reviewed-by: kvn, never

! src/share/vm/c1/c1_LIR.hpp
! src/share/vm/c1/c1_LIRGenerator.cpp

Changeset: a81f60ddab06
Author:    never
Date:      2012-01-22 14:03 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/a81f60ddab06

7130676: Tiered: assert(bci == 0 || 0<= bci && bci<code_size()) during stack trace construction
Reviewed-by: kvn, iveresov

! src/share/vm/classfile/javaClasses.cpp

Changeset: 82e5a84b7436
Author:    bdelsart
Date:      2012-01-24 15:41 +0100
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/82e5a84b7436

7120450: complete information dumped by frame_describe
Summary: improvements of frame_describe
Reviewed-by: never, twisti

! src/cpu/sparc/vm/frame_sparc.cpp
! src/cpu/sparc/vm/methodHandles_sparc.cpp
! src/cpu/sparc/vm/methodHandles_sparc.hpp
! src/cpu/x86/vm/frame_x86.cpp
! src/cpu/x86/vm/methodHandles_x86.cpp
! src/cpu/x86/vm/methodHandles_x86.hpp
! src/share/vm/runtime/frame.cpp

Changeset: dddf0be88eb1
Author:    iveresov
Date:      2012-01-24 17:00 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/dddf0be88eb1

7132945: Tiered: adjust OSR threshold of level 3
Summary: Change the value of Tier3BackEdgeThreshold from 7000 to 60000
Reviewed-by: kvn, never

! src/share/vm/runtime/globals.hpp

Changeset: cf407b7d3d78
Author:    roland
Date:      2012-01-25 09:31 +0100
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/cf407b7d3d78

7116050: C2/ARM: memory stomping error with DivideMcTests
Summary: Block::schedule_local() may write beyond end of ready_cnt array
Reviewed-by: never, kvn

! src/share/vm/opto/block.hpp
! src/share/vm/opto/gcm.cpp
! src/share/vm/opto/lcm.cpp

Changeset: 52474ec73861
Author:    kvn
Date:      2012-01-24 17:04 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/52474ec73861

7132936: guarantee(t != NULL) failed: must be con
Summary: Use !higher_equal() check to skip top and values which fit.
Reviewed-by: never

! src/share/vm/opto/memnode.cpp

Changeset: 94f0ce74d48e
Author:    kvn
Date:      2012-01-25 08:10 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/94f0ce74d48e

Merge


Changeset: 9a28ddfc1f4a
Author:    roland
Date:      2012-01-26 09:38 +0100
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/9a28ddfc1f4a

7123910: Some CTW tests crash VM: is_loaded() && that->is_loaded()
Summary: handle not loaded array klass in Parse::do_checkcast().
Reviewed-by: kvn, never

! src/share/vm/opto/parseHelper.cpp

Changeset: 5dbed2f542ff
Author:    bdelsart
Date:      2012-01-26 16:49 +0100
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/5dbed2f542ff

7120468: SPARC/x86: use frame::describe to enhance trace_method_handle
Summary: improvements of TraceMethodHandles for JSR292
Reviewed-by: never, twisti

! src/cpu/sparc/vm/frame_sparc.cpp
! src/cpu/sparc/vm/methodHandles_sparc.cpp
! src/cpu/sparc/vm/methodHandles_sparc.hpp
! src/cpu/x86/vm/frame_x86.cpp
! src/cpu/x86/vm/methodHandles_x86.cpp
! src/cpu/x86/vm/methodHandles_x86.hpp
! src/cpu/zero/vm/frame_zero.cpp
! src/share/vm/runtime/frame.cpp
! src/share/vm/runtime/frame.hpp

Changeset: 20334ed5ed3c
Author:    iveresov
Date:      2012-01-26 12:15 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/20334ed5ed3c

7131259: compile_method and CompilationPolicy::event shouldn't be declared TRAPS
Summary: Make sure that CompilationPolicy::event() doesn't throw exceptions
Reviewed-by: kvn, never

! src/share/vm/c1/c1_Runtime1.cpp
! src/share/vm/compiler/compileBroker.cpp
! src/share/vm/compiler/compileBroker.hpp
! src/share/vm/interpreter/interpreterRuntime.cpp
! src/share/vm/runtime/advancedThresholdPolicy.cpp
! src/share/vm/runtime/advancedThresholdPolicy.hpp
! src/share/vm/runtime/compilationPolicy.cpp
! src/share/vm/runtime/compilationPolicy.hpp
! src/share/vm/runtime/simpleThresholdPolicy.cpp
! src/share/vm/runtime/simpleThresholdPolicy.hpp
! src/share/vm/utilities/exceptions.hpp

Changeset: 072384a61312
Author:    jrose
Date:      2012-01-26 19:39 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/072384a61312

Merge


Changeset: 2e966d967c5c
Author:    johnc
Date:      2012-01-13 13:27 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/2e966d967c5c

7121547: G1: High number mispredicted branches while iterating over the marking bitmap
Summary: There is a high number of mispredicted branches associated with calling BitMap::iteratate() from within CMBitMapRO::iterate(). Implement a version of CMBitMapRO::iterate() directly using inline-able routines.
Reviewed-by: tonyp, iveresov

! src/share/vm/gc_implementation/g1/concurrentMark.cpp
! src/share/vm/gc_implementation/g1/concurrentMark.hpp
! src/share/vm/gc_implementation/g1/concurrentMark.inline.hpp
! src/share/vm/utilities/bitMap.inline.hpp

Changeset: 851b58c26def
Author:    brutisso
Date:      2012-01-16 11:21 +0100
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/851b58c26def

7130334: G1: Change comments and error messages that refer to CMS in g1/concurrentMark.cpp/hpp
Summary: Removed references to CMS in the concurrentMark.cpp/hpp files.
Reviewed-by: tonyp, jmasa, johnc

! src/share/vm/gc_implementation/g1/concurrentMark.cpp
! src/share/vm/gc_implementation/g1/concurrentMark.hpp

Changeset: 9509c20bba28
Author:    brutisso
Date:      2012-01-16 22:10 +0100
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/9509c20bba28

6976060: G1: humongous object allocations should initiate marking cycles when necessary
Reviewed-by: tonyp, johnc

! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp
! src/share/vm/gc_implementation/g1/g1CollectedHeap.hpp
! src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp
! src/share/vm/gc_implementation/g1/g1CollectorPolicy.hpp
! src/share/vm/gc_implementation/g1/vm_operations_g1.cpp
! src/share/vm/gc_interface/gcCause.cpp
! src/share/vm/gc_interface/gcCause.hpp

Changeset: 0b3d1ec6eaee
Author:    tonyp
Date:      2012-01-18 10:30 -0500
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/0b3d1ec6eaee

7097586: G1: improve the per-space output when using jmap -heap
Summary: Extend the jmap -heap output for G1 to include some more G1-specific information.
Reviewed-by: brutisso, johnc, poonam

! agent/src/share/classes/sun/jvm/hotspot/gc_implementation/g1/G1CollectedHeap.java
! agent/src/share/classes/sun/jvm/hotspot/gc_implementation/g1/G1MonitoringSupport.java
+ agent/src/share/classes/sun/jvm/hotspot/gc_implementation/g1/HeapRegionSetBase.java
! agent/src/share/classes/sun/jvm/hotspot/tools/HeapSummary.java
! src/share/vm/gc_implementation/g1/concurrentMark.cpp
! src/share/vm/gc_implementation/g1/heapRegionSet.hpp
! src/share/vm/gc_implementation/g1/vmStructs_g1.hpp

Changeset: 7ca7be5a6a0b
Author:    johnc
Date:      2012-01-17 10:21 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/7ca7be5a6a0b

7129271: G1: Interference from multiple threads in PrintGC/PrintGCDetails output
Summary: During an initial mark pause, signal the Concurrent Mark thread after the pause output from PrintGC/PrintGCDetails is complete.
Reviewed-by: tonyp, brutisso

! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp

Changeset: a8a126788ea0
Author:    tonyp
Date:      2012-01-19 09:13 -0500
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/a8a126788ea0

7078465: G1: Don't use the undefined value (-1) for the G1 old memory pool max size
Reviewed-by: johnc, brutisso

! src/share/vm/gc_implementation/g1/g1MonitoringSupport.hpp
! src/share/vm/services/g1MemoryPool.hpp

Changeset: 57025542827f
Author:    brutisso
Date:      2012-01-20 18:01 +0100
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/57025542827f

7131791: G1: Asserts in nightly testing due to 6976060
Summary: Create a handle and fake an object to make sure that we don't loose the memory we just allocated
Reviewed-by: tonyp, stefank

! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp

Changeset: 6a78aa6ac1ff
Author:    brutisso
Date:      2012-01-23 20:36 +0100
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/6a78aa6ac1ff

7132311: G1: assert((s == klass->oop_size(this)) || (Universe::heap()->is_gc_active() && ((is_typeArray()...
Summary: Move the check for when to call collect() to before we do a humongous object allocation
Reviewed-by: stefank, tonyp

! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp
! src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp
! src/share/vm/gc_implementation/g1/g1CollectorPolicy.hpp

Changeset: 877914d90c57
Author:    tonyp
Date:      2012-01-24 17:08 -0500
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/877914d90c57

7132398: G1: java.lang.IllegalArgumentException: Invalid threshold: 9223372036854775807 > max (1073741824)
Summary: Was not passing the right old pool max to the memory pool constructor in the fix for 7078465.
Reviewed-by: brutisso, johnc

! src/share/vm/services/g1MemoryPool.cpp

Changeset: d30fa85f9994
Author:    johnc
Date:      2012-01-12 00:06 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/d30fa85f9994

6484965: G1: piggy-back liveness accounting phase on marking
Summary: Remove the separate counting phase of concurrent marking by tracking the amount of marked bytes and the cards spanned by marked objects in marking task/worker thread local data structures, which are updated as individual objects are marked.
Reviewed-by: brutisso, tonyp

! src/share/vm/gc_implementation/g1/concurrentMark.cpp
! src/share/vm/gc_implementation/g1/concurrentMark.hpp
! src/share/vm/gc_implementation/g1/concurrentMark.inline.hpp
! src/share/vm/gc_implementation/g1/concurrentMarkThread.cpp
! src/share/vm/gc_implementation/g1/concurrentMarkThread.hpp
! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp
! src/share/vm/gc_implementation/g1/g1CollectedHeap.hpp
! src/share/vm/gc_implementation/g1/g1EvacFailure.hpp
! src/share/vm/gc_implementation/g1/g1OopClosures.hpp
! src/share/vm/gc_implementation/g1/heapRegion.hpp

Changeset: eff609af17d7
Author:    tonyp
Date:      2012-01-25 12:58 -0500
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/eff609af17d7

7127706: G1: re-enable survivors during the initial-mark pause
Summary: Re-enable survivors during the initial-mark pause. Afterwards, the concurrent marking threads have to scan them and mark everything reachable from them. The next GC will have to wait for the survivors to be scanned.
Reviewed-by: brutisso, johnc

! src/share/vm/gc_implementation/g1/concurrentMark.cpp
! src/share/vm/gc_implementation/g1/concurrentMark.hpp
! src/share/vm/gc_implementation/g1/concurrentMark.inline.hpp
! src/share/vm/gc_implementation/g1/concurrentMarkThread.cpp
! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp
! src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp
! src/share/vm/gc_implementation/g1/g1CollectorPolicy.hpp
! src/share/vm/gc_implementation/g1/g1EvacFailure.hpp
! src/share/vm/gc_implementation/g1/g1OopClosures.hpp
! src/share/vm/gc_implementation/g1/g1OopClosures.inline.hpp
! src/share/vm/gc_implementation/g1/heapRegion.inline.hpp
! src/share/vm/runtime/mutexLocker.cpp
! src/share/vm/runtime/mutexLocker.hpp

Changeset: a5244e07b761
Author:    jcoomes
Date:      2012-01-25 21:14 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/a5244e07b761

7112413: JVM Crash, possibly GC-related
Summary: disable UseAdaptiveSizePolicy with the CMS and ParNew
Reviewed-by: johnc, brutisso

! src/share/vm/runtime/arguments.cpp

Changeset: b4ebad3520bb
Author:    johnc
Date:      2012-01-26 14:14 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/b4ebad3520bb

7133038: G1: Some small profile based optimizations
Summary: Some minor profile based optimizations. Reduce the number of branches and branch mispredicts by removing some virtual calls, through closure specalization, and refactoring some conditional statements.
Reviewed-by: brutisso, tonyp

! src/share/vm/gc_implementation/g1/g1OopClosures.hpp
! src/share/vm/gc_implementation/g1/g1OopClosures.inline.hpp
! src/share/vm/gc_implementation/g1/g1RemSet.cpp
! src/share/vm/gc_implementation/g1/g1RemSet.hpp
! src/share/vm/gc_implementation/g1/g1RemSet.inline.hpp
! src/share/vm/gc_implementation/g1/g1_specialized_oop_closures.hpp
! src/share/vm/gc_implementation/g1/heapRegion.cpp

Changeset: 0a10d80352d5
Author:    brutisso
Date:      2012-01-27 09:04 +0100
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/0a10d80352d5

Merge

- src/os/bsd/vm/decoder_bsd.cpp
! src/share/vm/runtime/arguments.cpp
! src/share/vm/runtime/mutexLocker.cpp
! src/share/vm/runtime/mutexLocker.hpp

Changeset: af739d5ab23c
Author:    bpittore
Date:      2012-01-21 23:02 -0500
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/af739d5ab23c

6972759: Step over not working after thrown exception and Pop
Summary: reset jvmtithreadstate exception state after frame pop and forceearlyreturn processed
Reviewed-by: minqi, dholmes, dlong
Contributed-by: bill.pittore at oracle.com

! src/share/vm/prims/jvmtiThreadState.cpp
! src/share/vm/prims/jvmtiThreadState.hpp

Changeset: 583b428aa858
Author:    coleenp
Date:      2012-01-23 17:45 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/583b428aa858

Merge

- src/os/bsd/vm/decoder_bsd.cpp

Changeset: d6660fedbab5
Author:    phh
Date:      2012-01-24 14:07 -0500
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/d6660fedbab5

7126732: MAC: Require Mac OS X builds/tests for JPRT integrate jobs for HotSpot
Summary: Modify jprt.properties to run OSX builds and tests.
Reviewed-by: dcubed, kamg, ohair, dholmes
Contributed-by: james.melvin at oracle.com

! make/jprt.properties

Changeset: bf864f701a4a
Author:    dsamersoff
Date:      2012-01-25 02:29 +0400
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/bf864f701a4a

7066129: GarbageCollectorMXBean#getLastGcInfo leaks native memory
Summary: Make GCStatInfo a resource object
Reviewed-by: phh, coleenp

! src/share/vm/services/gcNotifier.cpp
! src/share/vm/services/management.cpp
! src/share/vm/services/memoryManager.cpp
! src/share/vm/services/memoryManager.hpp

Changeset: df88f58f3b61
Author:    dsamersoff
Date:      2012-01-24 20:15 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/df88f58f3b61

Merge


Changeset: e8a4934564b2
Author:    phh
Date:      2012-01-24 19:33 -0500
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/e8a4934564b2

7125793: MAC: test_gamma should always work
Summary: Fix gamma launcher on Mac OS X and reconcile test_gamma script on Unix platforms
Reviewed-by: dcubed, ohair, jcoomes, dholmes, ksrini
Contributed-by: james.melvin at oracle.com

! make/bsd/Makefile
! make/bsd/makefiles/buildtree.make
! make/bsd/makefiles/defs.make
! make/bsd/makefiles/launcher.make
! make/bsd/makefiles/vm.make
! make/linux/makefiles/buildtree.make
! make/solaris/makefiles/buildtree.make
! src/os/bsd/vm/os_bsd.cpp
! src/os/posix/launcher/java_md.c

Changeset: 78dadb7b16ab
Author:    phh
Date:      2012-01-25 01:16 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/78dadb7b16ab

Merge


Changeset: d708a8cdd022
Author:    kamg
Date:      2012-01-25 10:08 -0500
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/d708a8cdd022

Merge


Changeset: 520830f632e7
Author:    fparain
Date:      2012-01-25 10:32 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/520830f632e7

7131346: Parsing of boolean arguments to diagnostic commands is broken
Reviewed-by: dholmes, dcubed

! src/share/vm/services/diagnosticArgument.cpp
! src/share/vm/utilities/globalDefinitions_visCPP.hpp

Changeset: 24ec1a6d6ef3
Author:    fparain
Date:      2012-01-25 16:33 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/24ec1a6d6ef3

Merge


Changeset: a42c07c38c47
Author:    dsamersoff
Date:      2012-01-25 21:10 +0400
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/a42c07c38c47

7132515: Add dcmd to manage UnlockingCommercialFeature flag
Summary: Added dcmd to unlock or check status of UnlockingCommercialFeature flag
Reviewed-by: fparain, rottenha

! src/share/vm/services/diagnosticCommand.cpp
! src/share/vm/services/diagnosticCommand.hpp
+ src/share/vm/services/diagnosticCommand_ext.hpp
! src/share/vm/services/diagnosticFramework.hpp
! src/share/vm/services/management.cpp

Changeset: 6d00795f99a1
Author:    dsamersoff
Date:      2012-01-25 15:03 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/6d00795f99a1

Merge


Changeset: 6db63e782d3d
Author:    dsamersoff
Date:      2012-01-25 18:58 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/6db63e782d3d

Merge


Changeset: de268c8a8075
Author:    phh
Date:      2012-01-26 20:06 -0500
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/de268c8a8075

7082553: Interpret Thread.setPriority(Thread.MAX_PRIORITY) to mean FX60 on Solaris 10 and 11
Summary: Add CriticalPriority == MaxPriority+1 and enable scheduling class as well as thread priority to change on Solaris.
Reviewed-by: dholmes, dcubed

! src/os/bsd/vm/os_bsd.cpp
! src/os/linux/vm/os_linux.cpp
! src/os/solaris/vm/osThread_solaris.hpp
! src/os/solaris/vm/os_solaris.cpp
! src/os/windows/vm/os_windows.cpp
! src/share/vm/compiler/compileBroker.cpp
! src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepThread.cpp
! src/share/vm/runtime/globals.hpp
! src/share/vm/runtime/os.hpp

Changeset: bf5da1648543
Author:    kamg
Date:      2012-01-27 10:42 -0500
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/bf5da1648543

Merge

! src/share/vm/compiler/compileBroker.cpp
! src/share/vm/runtime/globals.hpp

Changeset: 9e177d44b10f
Author:    amurillo
Date:      2012-01-27 14:44 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/9e177d44b10f

Merge


Changeset: a80fd4f45d7a
Author:    amurillo
Date:      2012-01-27 14:44 -0800
URL:       http://hg.openjdk.java.net/hsx/hsx23/hotspot/rev/a80fd4f45d7a

Added tag hs23-b12 for changeset 9e177d44b10f

! .hgtags


From john.coomes at oracle.com  Fri Jan 27 22:34:13 2012
From: john.coomes at oracle.com (john.coomes at oracle.com)
Date: Sat, 28 Jan 2012 06:34:13 +0000
Subject: hg: hsx/hotspot-main/hotspot: 4 new changesets
Message-ID: <20120128063425.B07EE47247@hg.openjdk.java.net>

Changeset: 6edfe6e42a68
Author:    katleman
Date:      2012-01-26 18:23 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/6edfe6e42a68

Added tag jdk8-b23 for changeset e850d8e7ea54

! .hgtags

Changeset: 9e177d44b10f
Author:    amurillo
Date:      2012-01-27 14:44 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/9e177d44b10f

Merge


Changeset: a80fd4f45d7a
Author:    amurillo
Date:      2012-01-27 14:44 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/a80fd4f45d7a

Added tag hs23-b12 for changeset 9e177d44b10f

! .hgtags

Changeset: 9f1c2b7cdfb6
Author:    amurillo
Date:      2012-01-27 14:49 -0800
URL:       http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/9f1c2b7cdfb6

7135385: new hotspot build - hs23-b13
Reviewed-by: jcoomes

! make/hotspot_version


From david.holmes at oracle.com  Sun Jan 29 15:53:59 2012
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 30 Jan 2012 09:53:59 +1000
Subject: review for 7129164: JNI Get/ReleasePrimitiveArrayCritical doesn't
	scale
In-Reply-To: <4F22FA22.3000106@oracle.com>
References: <4DDD98EA-07C9-44D9-AF7E-C938E086C6FC@oracle.com>
	<4F22FA22.3000106@oracle.com>
Message-ID: <4F25DC17.60309@oracle.com>

On 28/01/2012 5:25 AM, Vladimir Kozlov wrote:
> I change alias to hotspot-dev since it affects everyone.

Thanks Vladimir!

Overall I find it difficult to follow the protocol that applies here - 
in particular how safepoints can and can not appear between different 
methods and the way in which needs_gc() might change value between the 
enter and exit of a critical region. Without that understanding it is 
hard to evaluate the correctness of the new approach.

---

All: copyright dates need updating.

---

src/share/vm/runtime/safepoint.hpp

Can we add an assert that the SafepointLock is held in 
increment_jni_active_count() please.

----

src/share/vm/memory/gcLocker.hpp

54   // The _jni_lock_count is keeps

Typo: "is keeps"

---

src/share/vm/memory/gcLocker.cpp

  72       _wait_begin = os::javaTimeMillis();
  135      os::javaTimeMillis() - _wait_begin,

Are you sure you want to use the non-monotonic javaTimeMillis here? Use 
of this was recently changed elsewhere in the GC code.


Cheers,
David
-----



> In GC_locker::check_active_before_gc() move wait_begin initialization
> under Print* check since it used only under such checks. Could method
> check_active_before_gc() be called from different threads concurrently?
> Does it need lock? Note that other methods have lock.
>
> Thanks,
> Vladimir
>
> Tom Rodriguez wrote:
>> http://cr.openjdk.java.net/~never/7129164
>> 200 lines changed: 126 ins; 33 del; 41 mod; 3958 unchg
>>
>> 7129164: JNI Get/ReleasePrimitiveArrayCritical doesn't scale
>> Summary:
>> Reviewed-by:
>>
>> The machinery for GC_locker which supports GetPrimitiveArrayCritical
>> maintains a count of the number of threads that currently have raw
>> pointers exposed. This is used to allow existing critical sections to
>> drain before creating new ones so that a GC can occur. Currently
>> these counts are updated using atomic all the time, even if a GC isn't
>> being requested. This creates scalability problem when a lot of
>> threads are hammering atomic operations on the jni_lock_count. The
>> count only needs to be valid when checking if a critical section is
>> currently active and when draining the existing sections. The fix is
>> to make the count be computed as part of the safepointing machinery
>> and to only decrement the counters when _needs_gc is true. In debug
>> mode the old count is maintained and validated against the lazily
>> computed count.
>>
>> On a microbenchmark that stresses GetPrimitiveArrayCritical with many
>> threads and relatively short critical section it can more than double
>> the throughput. This also slightly speeds up the normal
>> GetPrimtiveArrayCritical calls. Mostly it's not easily measurable
>> with larger scale programs.
>>
>> Tested with microbenchmark that stresses GetPrimtiveArrayCritical and
>> the crypto benchmarks of specjvm2008 on x86 and sparc. I also ran the
>> java.io regression tests from the JDK.
>>

From tom.rodriguez at oracle.com  Sun Jan 29 16:30:00 2012
From: tom.rodriguez at oracle.com (Tom Rodriguez)
Date: Sun, 29 Jan 2012 16:30:00 -0800
Subject: review for 7129164: JNI Get/ReleasePrimitiveArrayCritical doesn't
	scale
In-Reply-To: <4F25DC17.60309@oracle.com>
References: <4DDD98EA-07C9-44D9-AF7E-C938E086C6FC@oracle.com>
	<4F22FA22.3000106@oracle.com> <4F25DC17.60309@oracle.com>
Message-ID: <C592823E-DB9A-47B1-8E4E-26E11AAA36E4@oracle.com>


On Jan 29, 2012, at 3:53 PM, David Holmes wrote:

> On 28/01/2012 5:25 AM, Vladimir Kozlov wrote:
>> I change alias to hotspot-dev since it affects everyone.
> 
> Thanks Vladimir!
> 
> Overall I find it difficult to follow the protocol that applies here - in particular how safepoints can and can not appear between different methods and the way in which needs_gc() might change value between the enter and exit of a critical region. Without that understanding it is hard to evaluate the correctness of the new approach.

I tried to put in enough asserts to ensure that it was clear what invariants are in force when updating the state.  Safepoints can appear wherever they normally can appear, while locking and waiting on mutexes.  _needs_gc can only change from false to true during a safepoint.  For the most part everything operates exactly as it used to once _needs_gc is set and the only trick is computing the correct value of _jni_active_count when setting _needs_gc to be true.  The debug code that still performs the atomics attempts to verify that the computed value and actual value are in sync.  Is there any more that I can explain about how it operates?

> 
> ---
> 
> All: copyright dates need updating.

Fixed.

> 
> ---
> 
> src/share/vm/runtime/safepoint.hpp
> 
> Can we add an assert that the SafepointLock is held in increment_jni_active_count() please.

Fixed.

> 
> ----
> 
> src/share/vm/memory/gcLocker.hpp
> 
> 54   // The _jni_lock_count is keeps
> 
> Typo: "is keeps"

Fixed.

> 
> ---
> 
> src/share/vm/memory/gcLocker.cpp
> 
> 72       _wait_begin = os::javaTimeMillis();
> 135      os::javaTimeMillis() - _wait_begin,
> 
> Are you sure you want to use the non-monotonic javaTimeMillis here? Use of this was recently changed elsewhere in the GC code.

It's only printing code, but I'll switch it to use tty->timestamp().milliseconds() which will make it match our internal time stamps, which seems like a useful property.  Is that ok?

I've updated the webrev.

tom

> 
> 
> Cheers,
> David
> -----
> 
> 
> 
>> In GC_locker::check_active_before_gc() move wait_begin initialization
>> under Print* check since it used only under such checks. Could method
>> check_active_before_gc() be called from different threads concurrently?
>> Does it need lock? Note that other methods have lock.
>> 
>> Thanks,
>> Vladimir
>> 
>> Tom Rodriguez wrote:
>>> http://cr.openjdk.java.net/~never/7129164
>>> 200 lines changed: 126 ins; 33 del; 41 mod; 3958 unchg
>>> 
>>> 7129164: JNI Get/ReleasePrimitiveArrayCritical doesn't scale
>>> Summary:
>>> Reviewed-by:
>>> 
>>> The machinery for GC_locker which supports GetPrimitiveArrayCritical
>>> maintains a count of the number of threads that currently have raw
>>> pointers exposed. This is used to allow existing critical sections to
>>> drain before creating new ones so that a GC can occur. Currently
>>> these counts are updated using atomic all the time, even if a GC isn't
>>> being requested. This creates scalability problem when a lot of
>>> threads are hammering atomic operations on the jni_lock_count. The
>>> count only needs to be valid when checking if a critical section is
>>> currently active and when draining the existing sections. The fix is
>>> to make the count be computed as part of the safepointing machinery
>>> and to only decrement the counters when _needs_gc is true. In debug
>>> mode the old count is maintained and validated against the lazily
>>> computed count.
>>> 
>>> On a microbenchmark that stresses GetPrimitiveArrayCritical with many
>>> threads and relatively short critical section it can more than double
>>> the throughput. This also slightly speeds up the normal
>>> GetPrimtiveArrayCritical calls. Mostly it's not easily measurable
>>> with larger scale programs.
>>> 
>>> Tested with microbenchmark that stresses GetPrimtiveArrayCritical and
>>> the crypto benchmarks of specjvm2008 on x86 and sparc. I also ran the
>>> java.io regression tests from the JDK.
>>> 


From david.holmes at oracle.com  Sun Jan 29 18:26:09 2012
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 30 Jan 2012 12:26:09 +1000
Subject: review for 7129164: JNI Get/ReleasePrimitiveArrayCritical doesn't
	scale
In-Reply-To: <C592823E-DB9A-47B1-8E4E-26E11AAA36E4@oracle.com>
References: <4DDD98EA-07C9-44D9-AF7E-C938E086C6FC@oracle.com>
	<4F22FA22.3000106@oracle.com> <4F25DC17.60309@oracle.com>
	<C592823E-DB9A-47B1-8E4E-26E11AAA36E4@oracle.com>
Message-ID: <4F25FFC1.5020904@oracle.com>

Hi Tom,

On 30/01/2012 10:30 AM, Tom Rodriguez wrote:
> On Jan 29, 2012, at 3:53 PM, David Holmes wrote:
>
>> On 28/01/2012 5:25 AM, Vladimir Kozlov wrote:
>>> I change alias to hotspot-dev since it affects everyone.
>>
>> Thanks Vladimir!
>>
>> Overall I find it difficult to follow the protocol that applies here - in particular how safepoints can and can not appear between different methods and the way in which needs_gc() might change value between the enter and exit of a critical region. Without that understanding it is hard to evaluate the correctness of the new approach.
>
> I tried to put in enough asserts to ensure that it was clear what invariants are in force when updating the state.  Safepoints can appear wherever they normally can appear, while locking and waiting on mutexes.  _needs_gc can only change from false to true during a safepoint.  For the most part everything operates exactly as it used to once _needs_gc is set and the only trick is computing the correct value of _jni_active_count when setting _needs_gc to be true.  The debug code that still performs the atomics attempts to verify that the computed value and actual value are in sync.  Is there any more that I can explain about how it operates?

It's my ignorance of the overall operation that makes evaluating this 
change difficult for me. But thanks to your side-band emails I now have 
a much clear understanding of things - thanks. And the protocol seems safe.

>> src/share/vm/memory/gcLocker.cpp
>>
>> 72       _wait_begin = os::javaTimeMillis();
>> 135      os::javaTimeMillis() - _wait_begin,
>>
>> Are you sure you want to use the non-monotonic javaTimeMillis here? Use of this was recently changed elsewhere in the GC code.
>
> It's only printing code, but I'll switch it to use tty->timestamp().milliseconds() which will make it match our internal time stamps, which seems like a useful property.  Is that ok?

The recent GC changes switched to using javaTimeNanos() converted to 
millis - see 7117303 and 7129514. The TimeStamp is based on 
elapsedCounter which is still a time-of-day source on some platforms and 
so not monotonic.

Thanks,
David

> I've updated the webrev.
>
> tom
>
>>
>>
>> Cheers,
>> David
>> -----
>>
>>
>>
>>> In GC_locker::check_active_before_gc() move wait_begin initialization
>>> under Print* check since it used only under such checks. Could method
>>> check_active_before_gc() be called from different threads concurrently?
>>> Does it need lock? Note that other methods have lock.
>>>
>>> Thanks,
>>> Vladimir
>>>
>>> Tom Rodriguez wrote:
>>>> http://cr.openjdk.java.net/~never/7129164
>>>> 200 lines changed: 126 ins; 33 del; 41 mod; 3958 unchg
>>>>
>>>> 7129164: JNI Get/ReleasePrimitiveArrayCritical doesn't scale
>>>> Summary:
>>>> Reviewed-by:
>>>>
>>>> The machinery for GC_locker which supports GetPrimitiveArrayCritical
>>>> maintains a count of the number of threads that currently have raw
>>>> pointers exposed. This is used to allow existing critical sections to
>>>> drain before creating new ones so that a GC can occur. Currently
>>>> these counts are updated using atomic all the time, even if a GC isn't
>>>> being requested. This creates scalability problem when a lot of
>>>> threads are hammering atomic operations on the jni_lock_count. The
>>>> count only needs to be valid when checking if a critical section is
>>>> currently active and when draining the existing sections. The fix is
>>>> to make the count be computed as part of the safepointing machinery
>>>> and to only decrement the counters when _needs_gc is true. In debug
>>>> mode the old count is maintained and validated against the lazily
>>>> computed count.
>>>>
>>>> On a microbenchmark that stresses GetPrimitiveArrayCritical with many
>>>> threads and relatively short critical section it can more than double
>>>> the throughput. This also slightly speeds up the normal
>>>> GetPrimtiveArrayCritical calls. Mostly it's not easily measurable
>>>> with larger scale programs.
>>>>
>>>> Tested with microbenchmark that stresses GetPrimtiveArrayCritical and
>>>> the crypto benchmarks of specjvm2008 on x86 and sparc. I also ran the
>>>> java.io regression tests from the JDK.
>>>>
>

From karen.kinnear at oracle.com  Sun Jan 29 19:12:50 2012
From: karen.kinnear at oracle.com (Karen Kinnear)
Date: Sun, 29 Jan 2012 22:12:50 -0500
Subject: Code and Design Feedback request: 7114376: tune system dictionary
	size
In-Reply-To: <C56B8E6F-383E-4ADB-873E-1660411D850D@oracle.com>
References: <73E0E1AF-1806-439B-B826-AE4E59590BFD@oracle.com>	<CAGVnyzRHxg8SD5kx4duFCMuf7xzzaBGng_CBxC5n+DfmrzekZw@mail.gmail.com>	<F45DFA65-925A-446D-99A4-1F0D7AEF8496@oracle.com>	<4F1CBCDC.4060808@oracle.com>
	<400030A9-3FC7-4318-85F9-3097DC5638C3@oracle.com>
	<4F20A54F.8050604@oracle.com>
	<C56B8E6F-383E-4ADB-873E-1660411D850D@oracle.com>
Message-ID: <93AA3E6D-352F-4D95-A932-70DB5CA9FE32@oracle.com>

Final review please -

I reduced this to just add the command-line flag -XX:PredictedLoadedClassCount=#
and left the dynamic resizing for later.

webrev: http://cr.openjdk.java.net/~acorn/7114376.02/webrev/

Testing:
Manual testing of command-line options, CDS, scimark.
In parallel: JPRT, nsk, benchmarks

thanks,
Karen

p.s. the following files did not change between webrev.01 and webrev.02
agent/...
globals.hpp
vmStructs.cpp

On Jan 27, 2012, at 6:13 PM, Karen Kinnear wrote:

> Dan & David - 
> 
> Thank you both for the detailed reviews.
> 
> I will make the changes suggested - but I will put the code changes on hold. I think
> the best approach right now is to just add the command-line flag and then after
> we get the permanent generation changes, rethink the loaded class cache storage
> options - and do a whole lot more measurements than I am able to do in this timeframe -
> even thanks to the help I have had.
> 
> Details below,
> thanks,
> Karen


From david.holmes at oracle.com  Sun Jan 29 20:55:33 2012
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 30 Jan 2012 14:55:33 +1000
Subject: Patch to fix build breakage with GCC 4.7
In-Reply-To: <20120127214542.GQ3418@redhat.com>
References: <20120127214542.GQ3418@redhat.com>
Message-ID: <4F2622C5.3060804@oracle.com>

Hi Deepak,

Fix looks good to me. So you have my "approval" as a Reviewer but will 
need someone else to commit it for you - sorry.

David

On 28/01/2012 7:45 AM, Deepak Bhole wrote:
> Hi,
>
> HotSpot currently fails to build with GCC 4.7 due to 2 points where it
> is using boolean rather than NULL. GCC 4.6 treated it as a warning --
> 4.7 treats it as an error.
>
> Attached patch fixes the issue and makes it build again.
>
> Please feel free to commit if OK, or let me know if it needs re-work.
>
> Cheers,
> Deepak

From david.holmes at oracle.com  Sun Jan 29 21:04:33 2012
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 30 Jan 2012 15:04:33 +1000
Subject: Code and Design Feedback request: 7114376: tune system dictionary
	size
In-Reply-To: <93AA3E6D-352F-4D95-A932-70DB5CA9FE32@oracle.com>
References: <73E0E1AF-1806-439B-B826-AE4E59590BFD@oracle.com>	<CAGVnyzRHxg8SD5kx4duFCMuf7xzzaBGng_CBxC5n+DfmrzekZw@mail.gmail.com>	<F45DFA65-925A-446D-99A4-1F0D7AEF8496@oracle.com>	<4F1CBCDC.4060808@oracle.com>
	<400030A9-3FC7-4318-85F9-3097DC5638C3@oracle.com>
	<4F20A54F.8050604@oracle.com>
	<C56B8E6F-383E-4ADB-873E-1660411D850D@oracle.com>
	<93AA3E6D-352F-4D95-A932-70DB5CA9FE32@oracle.com>
Message-ID: <4F2624E1.1020001@oracle.com>

Hi Karen,

On 30/01/2012 1:12 PM, Karen Kinnear wrote:
> Final review please -
>
> I reduced this to just add the command-line flag -XX:PredictedLoadedClassCount=#
> and left the dynamic resizing for later.

Ok.

systemDictionary.cpp:

1171 void SystemDictionary::set_shared_dictionary(HashtableBucket* t, 
int length,
1172                                              int number_of_entries) {
1173   int newsize = length / sizeof(HashtableBucket);
1174   _shared_dictionary = new Dictionary(_nof_buckets, t, 
number_of_entries);
1175 }

newSize is calculated but not used.

David
-----


> webrev: http://cr.openjdk.java.net/~acorn/7114376.02/webrev/
>
> Testing:
> Manual testing of command-line options, CDS, scimark.
> In parallel: JPRT, nsk, benchmarks
>
> thanks,
> Karen
>
> p.s. the following files did not change between webrev.01 and webrev.02
> agent/...
> globals.hpp
> vmStructs.cpp
>
> On Jan 27, 2012, at 6:13 PM, Karen Kinnear wrote:
>
>> Dan&  David -
>>
>> Thank you both for the detailed reviews.
>>
>> I will make the changes suggested - but I will put the code changes on hold. I think
>> the best approach right now is to just add the command-line flag and then after
>> we get the permanent generation changes, rethink the loaded class cache storage
>> options - and do a whole lot more measurements than I am able to do in this timeframe -
>> even thanks to the help I have had.
>>
>> Details below,
>> thanks,
>> Karen
>

From bengt.rutisson at oracle.com  Mon Jan 30 00:01:49 2012
From: bengt.rutisson at oracle.com (Bengt Rutisson)
Date: Mon, 30 Jan 2012 09:01:49 +0100
Subject: Patch to fix build breakage with GCC 4.7
In-Reply-To: <4F2622C5.3060804@oracle.com>
References: <20120127214542.GQ3418@redhat.com> <4F2622C5.3060804@oracle.com>
Message-ID: <4F264E6D.3040903@oracle.com>


Hi Deepak and David,

Looks good to me too. I can commit the fix, but since the changes are to 
runtime code it might be best to push it through the runtime repository, 
right?

I am not sure about the state of hotspot-rt at the moment. Is it open 
for hs23 bug fixes? Can I go ahead an push there? I can also push it 
through the hotspot-gc repository if that is ok.

Bengt




On 2012-01-30 05:55, David Holmes wrote:
> Hi Deepak,
>
> Fix looks good to me. So you have my "approval" as a Reviewer but will 
> need someone else to commit it for you - sorry.
>
> David
>
> On 28/01/2012 7:45 AM, Deepak Bhole wrote:
>> Hi,
>>
>> HotSpot currently fails to build with GCC 4.7 due to 2 points where it
>> is using boolean rather than NULL. GCC 4.6 treated it as a warning --
>> 4.7 treats it as an error.
>>
>> Attached patch fixes the issue and makes it build again.
>>
>> Please feel free to commit if OK, or let me know if it needs re-work.
>>
>> Cheers,
>> Deepak


From robert.ottenhag at oracle.com  Mon Jan 30 02:11:13 2012
From: robert.ottenhag at oracle.com (Robert Ottenhag)
Date: Mon, 30 Jan 2012 11:11:13 +0100
Subject: Patch to fix build breakage with GCC 4.7
In-Reply-To: <4F264E6D.3040903@oracle.com>
References: <20120127214542.GQ3418@redhat.com> <4F2622C5.3060804@oracle.com>
	<4F264E6D.3040903@oracle.com>
Message-ID: <4F266CC1.1010407@oracle.com>

Yes, I also agree on the fix (that s/false/NULL/ follows the rest of 
each function's fail control logic)

Bengt, please create a bug and do a quick follow up post for proper bug 
review. There should be no problem pushing it through hotspot-rt at this 
point.

/Robert

On 01/30/2012 09:01 AM, Bengt Rutisson wrote:
>
> Hi Deepak and David,
>
> Looks good to me too. I can commit the fix, but since the changes are 
> to runtime code it might be best to push it through the runtime 
> repository, right?
>
> I am not sure about the state of hotspot-rt at the moment. Is it open 
> for hs23 bug fixes? Can I go ahead an push there? I can also push it 
> through the hotspot-gc repository if that is ok.
>
> Bengt
>
>
>
>
> On 2012-01-30 05:55, David Holmes wrote:
>> Hi Deepak,
>>
>> Fix looks good to me. So you have my "approval" as a Reviewer but 
>> will need someone else to commit it for you - sorry.
>>
>> David
>>
>> On 28/01/2012 7:45 AM, Deepak Bhole wrote:
>>> Hi,
>>>
>>> HotSpot currently fails to build with GCC 4.7 due to 2 points where it
>>> is using boolean rather than NULL. GCC 4.6 treated it as a warning --
>>> 4.7 treats it as an error.
>>>
>>> Attached patch fixes the issue and makes it build again.
>>>
>>> Please feel free to commit if OK, or let me know if it needs re-work.
>>>
>>> Cheers,
>>> Deepak
>


-- 
Oracle
Robert Ottenhag | Senior Member of Technical Staff
Phone: +46850630961 | Fax: +46850630911 | Mobile: +46707106161
Oracle Java HotSpot Virtual Machine
ORACLE Sweden | Folkungagatan 122 | SE-116 30 Stockholm

Oracle Svenska AB, Kronborgsgr?nd 17, S-164 28 KISTA, reg.no. 556254-6746

Green Oracle

Oracle is committed to developing practices and products that help protect the environment
--


From bengt.rutisson at oracle.com  Mon Jan 30 02:14:43 2012
From: bengt.rutisson at oracle.com (Bengt Rutisson)
Date: Mon, 30 Jan 2012 11:14:43 +0100
Subject: review for 7129164: JNI Get/ReleasePrimitiveArrayCritical doesn't
	scale
In-Reply-To: <4F25FFC1.5020904@oracle.com>
References: <4DDD98EA-07C9-44D9-AF7E-C938E086C6FC@oracle.com>	<4F22FA22.3000106@oracle.com>
	<4F25DC17.60309@oracle.com>	<C592823E-DB9A-47B1-8E4E-26E11AAA36E4@oracle.com>
	<4F25FFC1.5020904@oracle.com>
Message-ID: <4F266D93.9080308@oracle.com>


Hi Tom,

Looked at your latest webrev. Looks good to me.

Some minor comments:


gcLocker.cpp

You still use tty->timestamp().milliseconds() instead of 
javaTimeNanos(). Not sure whether you just didn't get to fixing it yet 
or if you decided against it.



gcLocker.hpp

   84   static void verify_critical_count() NOT_DEBUG_RETURN;

Not sure what the correct thing is here, but the GC code seems to use 
PRODUCT_RETURN more than NOT_DEBUG_RETURN. I guess the only difference 
is for optimized builds (and of course using #ifndef PRODUCT instead of 
#ifdef ASSERT).



safepoint.cpp

Not strictly related to your change, but the 
SafepointSynchronize::begin() method is missing one indentation level 
due to the the scoping that starts on line 139:

  139   {
  140   MutexLocker mu(Safepoint_lock);
  141
  142   // Reset the count of active JNI critical threads
  143   _current_jni_active_count = 0;

  This scope ends at the end of the method:

  398   if (PrintSafepointStatistics) {
  399     // Record how much time spend on the above cleanup tasks
  400     update_statistics_on_cleanup_end(os::javaTimeNanos());
  401   }
  402   }
  403 }

It seems to me that the scope is not really necessary. In that case, can 
we remove it? It would be nice to have the method correctly indented. If 
we decide to keep the scope I'd prefer a comment on line 402, to 
indicate that it is the end of the MutexLocker scope.

Bengt

On 2012-01-30 03:26, David Holmes wrote:
> Hi Tom,
>
> On 30/01/2012 10:30 AM, Tom Rodriguez wrote:
>> On Jan 29, 2012, at 3:53 PM, David Holmes wrote:
>>
>>> On 28/01/2012 5:25 AM, Vladimir Kozlov wrote:
>>>> I change alias to hotspot-dev since it affects everyone.
>>>
>>> Thanks Vladimir!
>>>
>>> Overall I find it difficult to follow the protocol that applies here 
>>> - in particular how safepoints can and can not appear between 
>>> different methods and the way in which needs_gc() might change value 
>>> between the enter and exit of a critical region. Without that 
>>> understanding it is hard to evaluate the correctness of the new 
>>> approach.
>>
>> I tried to put in enough asserts to ensure that it was clear what 
>> invariants are in force when updating the state.  Safepoints can 
>> appear wherever they normally can appear, while locking and waiting 
>> on mutexes.  _needs_gc can only change from false to true during a 
>> safepoint.  For the most part everything operates exactly as it used 
>> to once _needs_gc is set and the only trick is computing the correct 
>> value of _jni_active_count when setting _needs_gc to be true.  The 
>> debug code that still performs the atomics attempts to verify that 
>> the computed value and actual value are in sync.  Is there any more 
>> that I can explain about how it operates?
>
> It's my ignorance of the overall operation that makes evaluating this 
> change difficult for me. But thanks to your side-band emails I now 
> have a much clear understanding of things - thanks. And the protocol 
> seems safe.
>
>>> src/share/vm/memory/gcLocker.cpp
>>>
>>> 72       _wait_begin = os::javaTimeMillis();
>>> 135      os::javaTimeMillis() - _wait_begin,
>>>
>>> Are you sure you want to use the non-monotonic javaTimeMillis here? 
>>> Use of this was recently changed elsewhere in the GC code.
>>
>> It's only printing code, but I'll switch it to use 
>> tty->timestamp().milliseconds() which will make it match our internal 
>> time stamps, which seems like a useful property.  Is that ok?
>
> The recent GC changes switched to using javaTimeNanos() converted to 
> millis - see 7117303 and 7129514. The TimeStamp is based on 
> elapsedCounter which is still a time-of-day source on some platforms 
> and so not monotonic.
>
> Thanks,
> David
>
>> I've updated the webrev.
>>
>> tom
>>
>>>
>>>
>>> Cheers,
>>> David
>>> -----
>>>
>>>
>>>
>>>> In GC_locker::check_active_before_gc() move wait_begin initialization
>>>> under Print* check since it used only under such checks. Could method
>>>> check_active_before_gc() be called from different threads 
>>>> concurrently?
>>>> Does it need lock? Note that other methods have lock.
>>>>
>>>> Thanks,
>>>> Vladimir
>>>>
>>>> Tom Rodriguez wrote:
>>>>> http://cr.openjdk.java.net/~never/7129164
>>>>> 200 lines changed: 126 ins; 33 del; 41 mod; 3958 unchg
>>>>>
>>>>> 7129164: JNI Get/ReleasePrimitiveArrayCritical doesn't scale
>>>>> Summary:
>>>>> Reviewed-by:
>>>>>
>>>>> The machinery for GC_locker which supports GetPrimitiveArrayCritical
>>>>> maintains a count of the number of threads that currently have raw
>>>>> pointers exposed. This is used to allow existing critical sections to
>>>>> drain before creating new ones so that a GC can occur. Currently
>>>>> these counts are updated using atomic all the time, even if a GC 
>>>>> isn't
>>>>> being requested. This creates scalability problem when a lot of
>>>>> threads are hammering atomic operations on the jni_lock_count. The
>>>>> count only needs to be valid when checking if a critical section is
>>>>> currently active and when draining the existing sections. The fix is
>>>>> to make the count be computed as part of the safepointing machinery
>>>>> and to only decrement the counters when _needs_gc is true. In debug
>>>>> mode the old count is maintained and validated against the lazily
>>>>> computed count.
>>>>>
>>>>> On a microbenchmark that stresses GetPrimitiveArrayCritical with many
>>>>> threads and relatively short critical section it can more than double
>>>>> the throughput. This also slightly speeds up the normal
>>>>> GetPrimtiveArrayCritical calls. Mostly it's not easily measurable
>>>>> with larger scale programs.
>>>>>
>>>>> Tested with microbenchmark that stresses GetPrimtiveArrayCritical and
>>>>> the crypto benchmarks of specjvm2008 on x86 and sparc. I also ran the
>>>>> java.io regression tests from the JDK.
>>>>>
>>


From bengt.rutisson at oracle.com  Mon Jan 30 04:17:30 2012
From: bengt.rutisson at oracle.com (Bengt Rutisson)
Date: Mon, 30 Jan 2012 13:17:30 +0100
Subject: Review request (S): 7140882 (Was:  Re: Patch to fix build breakage
	with GCC 4.7)
In-Reply-To: <4F266CC1.1010407@oracle.com>
References: <20120127214542.GQ3418@redhat.com> <4F2622C5.3060804@oracle.com>
	<4F264E6D.3040903@oracle.com> <4F266CC1.1010407@oracle.com>
Message-ID: <4F268A5A.4070806@oracle.com>


Robert,

On 2012-01-30 11:11, Robert Ottenhag wrote:
> Yes, I also agree on the fix (that s/false/NULL/ follows the rest of 
> each function's fail control logic)
>
> Bengt, please create a bug and do a quick follow up post for proper 
> bug review. There should be no problem pushing it through hotspot-rt 
> at this point.

I filed:

7140882 Don't return booleans from methods returning pointers
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7140882

The push job is now in the queue.

Deepak, I saw that you are an "author" for the JDK projects, but not for 
the HSX project. Thus, I can't create the changeset as your user. 
Instead I list you as "Contributed-by". That field only supports email 
addresses. Here is what the changeset will look like:

changeset:   3021:f457154eee8b
tag:         tip
user:        brutisso
date:        Mon Jan 30 12:36:49 2012 +0100
files:       src/share/vm/oops/constantPoolOop.cpp 
src/share/vm/opto/loopnode.cpp
description:
7140882: Don't return booleans from methods returning pointers
Summary: Changed "return false" to "return NULL"
Reviewed-by: dholmes, rottenha
Contributed-by: dbhole at redhat.com

Bengt

>
> /Robert
>
> On 01/30/2012 09:01 AM, Bengt Rutisson wrote:
>>
>> Hi Deepak and David,
>>
>> Looks good to me too. I can commit the fix, but since the changes are 
>> to runtime code it might be best to push it through the runtime 
>> repository, right?
>>
>> I am not sure about the state of hotspot-rt at the moment. Is it open 
>> for hs23 bug fixes? Can I go ahead an push there? I can also push it 
>> through the hotspot-gc repository if that is ok.
>>
>> Bengt
>>
>>
>>
>>
>> On 2012-01-30 05:55, David Holmes wrote:
>>> Hi Deepak,
>>>
>>> Fix looks good to me. So you have my "approval" as a Reviewer but 
>>> will need someone else to commit it for you - sorry.
>>>
>>> David
>>>
>>> On 28/01/2012 7:45 AM, Deepak Bhole wrote:
>>>> Hi,
>>>>
>>>> HotSpot currently fails to build with GCC 4.7 due to 2 points where it
>>>> is using boolean rather than NULL. GCC 4.6 treated it as a warning --
>>>> 4.7 treats it as an error.
>>>>
>>>> Attached patch fixes the issue and makes it build again.
>>>>
>>>> Please feel free to commit if OK, or let me know if it needs re-work.
>>>>
>>>> Cheers,
>>>> Deepak
>>
>
>


From paul.hohensee at oracle.com  Mon Jan 30 05:57:16 2012
From: paul.hohensee at oracle.com (Paul Hohensee)
Date: Mon, 30 Jan 2012 08:57:16 -0500
Subject: Review request (S): 7140882 (Was: Re: Patch to fix build breakage
	with GCC 4.7)
In-Reply-To: <4F268A5A.4070806@oracle.com>
References: <20120127214542.GQ3418@redhat.com> <4F2622C5.3060804@oracle.com>
	<4F264E6D.3040903@oracle.com> <4F266CC1.1010407@oracle.com>
	<4F268A5A.4070806@oracle.com>
Message-ID: <4F26A1BC.8050708@oracle.com>

Deepak, let me know if you'd like Author status on the HSX project.

Paul

On 1/30/12 7:17 AM, Bengt Rutisson wrote:
>
> Robert,
>
> On 2012-01-30 11:11, Robert Ottenhag wrote:
>> Yes, I also agree on the fix (that s/false/NULL/ follows the rest of 
>> each function's fail control logic)
>>
>> Bengt, please create a bug and do a quick follow up post for proper 
>> bug review. There should be no problem pushing it through hotspot-rt 
>> at this point.
>
> I filed:
>
> 7140882 Don't return booleans from methods returning pointers
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7140882
>
> The push job is now in the queue.
>
> Deepak, I saw that you are an "author" for the JDK projects, but not 
> for the HSX project. Thus, I can't create the changeset as your user. 
> Instead I list you as "Contributed-by". That field only supports email 
> addresses. Here is what the changeset will look like:
>
> changeset:   3021:f457154eee8b
> tag:         tip
> user:        brutisso
> date:        Mon Jan 30 12:36:49 2012 +0100
> files:       src/share/vm/oops/constantPoolOop.cpp 
> src/share/vm/opto/loopnode.cpp
> description:
> 7140882: Don't return booleans from methods returning pointers
> Summary: Changed "return false" to "return NULL"
> Reviewed-by: dholmes, rottenha
> Contributed-by: dbhole at redhat.com
>
> Bengt
>
>>
>> /Robert
>>
>> On 01/30/2012 09:01 AM, Bengt Rutisson wrote:
>>>
>>> Hi Deepak and David,
>>>
>>> Looks good to me too. I can commit the fix, but since the changes 
>>> are to runtime code it might be best to push it through the runtime 
>>> repository, right?
>>>
>>> I am not sure about the state of hotspot-rt at the moment. Is it 
>>> open for hs23 bug fixes? Can I go ahead an push there? I can also 
>>> push it through the hotspot-gc repository if that is ok.
>>>
>>> Bengt
>>>
>>>
>>>
>>>
>>> On 2012-01-30 05:55, David Holmes wrote:
>>>> Hi Deepak,
>>>>
>>>> Fix looks good to me. So you have my "approval" as a Reviewer but 
>>>> will need someone else to commit it for you - sorry.
>>>>
>>>> David
>>>>
>>>> On 28/01/2012 7:45 AM, Deepak Bhole wrote:
>>>>> Hi,
>>>>>
>>>>> HotSpot currently fails to build with GCC 4.7 due to 2 points 
>>>>> where it
>>>>> is using boolean rather than NULL. GCC 4.6 treated it as a warning --
>>>>> 4.7 treats it as an error.
>>>>>
>>>>> Attached patch fixes the issue and makes it build again.
>>>>>
>>>>> Please feel free to commit if OK, or let me know if it needs re-work.
>>>>>
>>>>> Cheers,
>>>>> Deepak
>>>
>>
>>
>

From paul.hohensee at oracle.com  Mon Jan 30 06:10:37 2012
From: paul.hohensee at oracle.com (Paul Hohensee)
Date: Mon, 30 Jan 2012 09:10:37 -0500
Subject: Code and Design Feedback request: 7114376: tune system dictionary
	size
In-Reply-To: <93AA3E6D-352F-4D95-A932-70DB5CA9FE32@oracle.com>
References: <73E0E1AF-1806-439B-B826-AE4E59590BFD@oracle.com>	<CAGVnyzRHxg8SD5kx4duFCMuf7xzzaBGng_CBxC5n+DfmrzekZw@mail.gmail.com>	<F45DFA65-925A-446D-99A4-1F0D7AEF8496@oracle.com>	<4F1CBCDC.4060808@oracle.com>
	<400030A9-3FC7-4318-85F9-3097DC5638C3@oracle.com>
	<4F20A54F.8050604@oracle.com>
	<C56B8E6F-383E-4ADB-873E-1660411D850D@oracle.com>
	<93AA3E6D-352F-4D95-A932-70DB5CA9FE32@oracle.com>
Message-ID: <4F26A4DD.7040703@oracle.com>

Some minor things.

I'd define PRIMEARRAYSIZE, OLDDEFAULTSDSIZE and AVERAGEDEPTHGOAL
as const static int rather than macros.  Though I'm not sure anymore whether
the latter can be used as an array size.  If not, then I'd define them 
as enums
after _nof_buckets.

Rather than use OLDDEFAULTSDSIZE, I'd use _nof_buckets directly, or set
OLDDEFAULTSDSIZE equal to _nof_buckets.

I'd replace the name "calculate_systemdictionary_size" with 
"calculate_size",
since we know the object is a SystemDictionary.

There's extra blanks in a few places in the definition of
calculcate_systemdictionary_size.

Otherwise looks great.

Paul

On 1/29/12 10:12 PM, Karen Kinnear wrote:
> Final review please -
>
> I reduced this to just add the command-line flag -XX:PredictedLoadedClassCount=#
> and left the dynamic resizing for later.
>
> webrev: http://cr.openjdk.java.net/~acorn/7114376.02/webrev/
>
> Testing:
> Manual testing of command-line options, CDS, scimark.
> In parallel: JPRT, nsk, benchmarks
>
> thanks,
> Karen
>
> p.s. the following files did not change between webrev.01 and webrev.02
> agent/...
> globals.hpp
> vmStructs.cpp
>
> On Jan 27, 2012, at 6:13 PM, Karen Kinnear wrote:
>
>> Dan&  David -
>>
>> Thank you both for the detailed reviews.
>>
>> I will make the changes suggested - but I will put the code changes on hold. I think
>> the best approach right now is to just add the command-line flag and then after
>> we get the permanent generation changes, rethink the loaded class cache storage
>> options - and do a whole lot more measurements than I am able to do in this timeframe -
>> even thanks to the help I have had.
>>
>> Details below,
>> thanks,
>> Karen

From vitalyd at gmail.com  Mon Jan 30 07:28:38 2012
From: vitalyd at gmail.com (Vitaly Davidovich)
Date: Mon, 30 Jan 2012 10:28:38 -0500
Subject: Code and Design Feedback request: 7114376: tune system dictionary
	size
In-Reply-To: <93AA3E6D-352F-4D95-A932-70DB5CA9FE32@oracle.com>
References: <73E0E1AF-1806-439B-B826-AE4E59590BFD@oracle.com>
	<CAGVnyzRHxg8SD5kx4duFCMuf7xzzaBGng_CBxC5n+DfmrzekZw@mail.gmail.com>
	<F45DFA65-925A-446D-99A4-1F0D7AEF8496@oracle.com>
	<4F1CBCDC.4060808@oracle.com>
	<400030A9-3FC7-4318-85F9-3097DC5638C3@oracle.com>
	<4F20A54F.8050604@oracle.com>
	<C56B8E6F-383E-4ADB-873E-1660411D850D@oracle.com>
	<93AA3E6D-352F-4D95-A932-70DB5CA9FE32@oracle.com>
Message-ID: <CAHjP37FbDnkkJUzVnDWT_uUHN+ku4CFHsfj+o0EGivKMFoYx=A@mail.gmail.com>

Hi Karen,

Do you think it's worthwhile to assert that _sdgeneration is within array
bounds inside calculate_system dictionary_size? Precaution against a bad
change later on that would do the initial read outside the bounds.

Also, it seems that you can declare the prime sizes array as a static const
array inside the same calculate size function since it's only used there as
far as I can see.  You can then initialize it there without specifying size
and can compute its length via sizeof(array)/sizeof(array[0]) and don't
need the size DEFINE.  Not sure if that's against hotspot guidelines though.

Vitaly

Sent from my phone
On Jan 29, 2012 10:14 PM, "Karen Kinnear" <karen.kinnear at oracle.com> wrote:

> Final review please -
>
> I reduced this to just add the command-line flag
> -XX:PredictedLoadedClassCount=#
> and left the dynamic resizing for later.
>
> webrev: http://cr.openjdk.java.net/~acorn/7114376.02/webrev/
>
> Testing:
> Manual testing of command-line options, CDS, scimark.
> In parallel: JPRT, nsk, benchmarks
>
> thanks,
> Karen
>
> p.s. the following files did not change between webrev.01 and webrev.02
> agent/...
> globals.hpp
> vmStructs.cpp
>
> On Jan 27, 2012, at 6:13 PM, Karen Kinnear wrote:
>
> > Dan & David -
> >
> > Thank you both for the detailed reviews.
> >
> > I will make the changes suggested - but I will put the code changes on
> hold. I think
> > the best approach right now is to just add the command-line flag and
> then after
> > we get the permanent generation changes, rethink the loaded class cache
> storage
> > options - and do a whole lot more measurements than I am able to do in
> this timeframe -
> > even thanks to the help I have had.
> >
> > Details below,
> > thanks,
> > Karen
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120130/a3bcc2ec/attachment.html 

From ahughes at redhat.com  Mon Jan 30 07:33:06 2012
From: ahughes at redhat.com (Andrew Hughes)
Date: Mon, 30 Jan 2012 10:33:06 -0500 (EST)
Subject: Patch to fix build breakage with GCC 4.7
In-Reply-To: <20120127220748.GA12417@redhat.com>
Message-ID: <cb43049c-50f8-4261-b6f0-55675d2c9696@zmail16.collab.prod.int.phx2.redhat.com>

----- Original Message -----
> * Deepak Bhole <dbhole at redhat.com> [2012-01-27 16:48]:
> > Hi,
> > 
> > HotSpot currently fails to build with GCC 4.7 due to 2 points where
> > it
> > is using boolean rather than NULL. GCC 4.6 treated it as a warning
> > --
> > 4.7 treats it as an error.
> > 
> > Attached patch fixes the issue and makes it build again.
> > 
> 
> Sorry, old habit of posting just the patch.. webrev is here for those
> who prefer it:
> http://cr.openjdk.java.net/~dbhole/GCC-4.7-HS23.00/
> 
> Cheers,
> Deepak
> 
> 

The right habit, IMHO ;-)
-- 
Andrew :)

Free Java Software Engineer
Red Hat, Inc. (http://www.redhat.com)

PGP Key: 248BDC07 (https://keys.indymedia.org/)
Fingerprint = EC5A 1F5E C0AD 1D15 8F1F  8F91 3B96 A578 248B DC07


From ahughes at redhat.com  Mon Jan 30 07:34:20 2012
From: ahughes at redhat.com (Andrew Hughes)
Date: Mon, 30 Jan 2012 10:34:20 -0500 (EST)
Subject: Patch to fix build breakage with GCC 4.7
In-Reply-To: <4F2622C5.3060804@oracle.com>
Message-ID: <f7f579de-804b-4eae-b34b-6188adb43fdd@zmail16.collab.prod.int.phx2.redhat.com>

----- Original Message -----
> Hi Deepak,
> 
> Fix looks good to me. So you have my "approval" as a Reviewer but
> will
> need someone else to commit it for you - sorry.
> 

Is there any news on when external developers will finally be able to commit
to HotSpot?  This situation is ridiculous.

> David
> 
> 

-- 
Andrew :)

Free Java Software Engineer
Red Hat, Inc. (http://www.redhat.com)

PGP Key: 248BDC07 (https://keys.indymedia.org/)
Fingerprint = EC5A 1F5E C0AD 1D15 8F1F  8F91 3B96 A578 248B DC07


From dbhole at redhat.com  Mon Jan 30 07:36:03 2012
From: dbhole at redhat.com (Deepak Bhole)
Date: Mon, 30 Jan 2012 10:36:03 -0500
Subject: Review request (S): 7140882 (Was: Re: Patch to fix build
	breakage with GCC 4.7)
In-Reply-To: <4F26A1BC.8050708@oracle.com>
References: <20120127214542.GQ3418@redhat.com> <4F2622C5.3060804@oracle.com>
	<4F264E6D.3040903@oracle.com> <4F266CC1.1010407@oracle.com>
	<4F268A5A.4070806@oracle.com> <4F26A1BC.8050708@oracle.com>
Message-ID: <20120130153601.GH3179@redhat.com>

* Paul Hohensee <paul.hohensee at oracle.com> [2012-01-30 09:07]:
> Deepak, let me know if you'd like Author status on the HSX project.
> 

Hi Paul,

I have only posted a handful of patches to HS (probably 2-3). I usually
post for JDK. But if few is enough for HS author status.. sure! No
worries if not. I am happy as long as the fix goes in :)

Cheers,
Deepak

> Paul
> 
> On 1/30/12 7:17 AM, Bengt Rutisson wrote:
> >
> >Robert,
> >
> >On 2012-01-30 11:11, Robert Ottenhag wrote:
> >>Yes, I also agree on the fix (that s/false/NULL/ follows the
> >>rest of each function's fail control logic)
> >>
> >>Bengt, please create a bug and do a quick follow up post for
> >>proper bug review. There should be no problem pushing it through
> >>hotspot-rt at this point.
> >
> >I filed:
> >
> >7140882 Don't return booleans from methods returning pointers
> >http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7140882
> >
> >The push job is now in the queue.
> >
> >Deepak, I saw that you are an "author" for the JDK projects, but
> >not for the HSX project. Thus, I can't create the changeset as
> >your user. Instead I list you as "Contributed-by". That field only
> >supports email addresses. Here is what the changeset will look
> >like:
> >
> >changeset:   3021:f457154eee8b
> >tag:         tip
> >user:        brutisso
> >date:        Mon Jan 30 12:36:49 2012 +0100
> >files:       src/share/vm/oops/constantPoolOop.cpp
> >src/share/vm/opto/loopnode.cpp
> >description:
> >7140882: Don't return booleans from methods returning pointers
> >Summary: Changed "return false" to "return NULL"
> >Reviewed-by: dholmes, rottenha
> >Contributed-by: dbhole at redhat.com
> >
> >Bengt
> >
> >>
> >>/Robert
> >>
> >>On 01/30/2012 09:01 AM, Bengt Rutisson wrote:
> >>>
> >>>Hi Deepak and David,
> >>>
> >>>Looks good to me too. I can commit the fix, but since the
> >>>changes are to runtime code it might be best to push it
> >>>through the runtime repository, right?
> >>>
> >>>I am not sure about the state of hotspot-rt at the moment. Is
> >>>it open for hs23 bug fixes? Can I go ahead an push there? I
> >>>can also push it through the hotspot-gc repository if that is
> >>>ok.
> >>>
> >>>Bengt
> >>>
> >>>
> >>>
> >>>
> >>>On 2012-01-30 05:55, David Holmes wrote:
> >>>>Hi Deepak,
> >>>>
> >>>>Fix looks good to me. So you have my "approval" as a
> >>>>Reviewer but will need someone else to commit it for you -
> >>>>sorry.
> >>>>
> >>>>David
> >>>>
> >>>>On 28/01/2012 7:45 AM, Deepak Bhole wrote:
> >>>>>Hi,
> >>>>>
> >>>>>HotSpot currently fails to build with GCC 4.7 due to 2
> >>>>>points where it
> >>>>>is using boolean rather than NULL. GCC 4.6 treated it as a warning --
> >>>>>4.7 treats it as an error.
> >>>>>
> >>>>>Attached patch fixes the issue and makes it build again.
> >>>>>
> >>>>>Please feel free to commit if OK, or let me know if it needs re-work.
> >>>>>
> >>>>>Cheers,
> >>>>>Deepak
> >>>
> >>
> >>
> >

From ahughes at redhat.com  Mon Jan 30 07:38:51 2012
From: ahughes at redhat.com (Andrew Hughes)
Date: Mon, 30 Jan 2012 10:38:51 -0500 (EST)
Subject: Review request (S): 7140882 (Was: Re: Patch to fix build
	breakage	with GCC 4.7)
In-Reply-To: <4F268A5A.4070806@oracle.com>
Message-ID: <b9ed18b8-07f4-43b4-ab02-76ba0a6946fc@zmail16.collab.prod.int.phx2.redhat.com>

----- Original Message -----
> 

> Deepak, I saw that you are an "author" for the JDK projects, but not
> for
> the HSX project. Thus, I can't create the changeset as your user.
> Instead I list you as "Contributed-by". That field only supports
> email
> addresses. Here is what the changeset will look like:
> 
> changeset:   3021:f457154eee8b
> tag:         tip
> user:        brutisso
> date:        Mon Jan 30 12:36:49 2012 +0100
> files:       src/share/vm/oops/constantPoolOop.cpp
> src/share/vm/opto/loopnode.cpp
> description:
> 7140882: Don't return booleans from methods returning pointers
> Summary: Changed "return false" to "return NULL"
> Reviewed-by: dholmes, rottenha
> Contributed-by: dbhole at redhat.com
> 
> Bengt
> 
> >

(ccing discuss)

That seems wrong.  No-one commits to the jdk projects directly, so
what's the point in being a committer/reviewer there?

Both Deepak and I have pushed patches to various repositories before.
Why are our rights being taken away like this?
-- 
Andrew :)

Free Java Software Engineer
Red Hat, Inc. (http://www.redhat.com)

PGP Key: 248BDC07 (https://keys.indymedia.org/)
Fingerprint = EC5A 1F5E C0AD 1D15 8F1F  8F91 3B96 A578 248B DC07


From tom.rodriguez at oracle.com  Mon Jan 30 08:14:00 2012
From: tom.rodriguez at oracle.com (Tom Rodriguez)
Date: Mon, 30 Jan 2012 08:14:00 -0800
Subject: review for 7129164: JNI Get/ReleasePrimitiveArrayCritical doesn't
	scale
In-Reply-To: <4F266D93.9080308@oracle.com>
References: <4DDD98EA-07C9-44D9-AF7E-C938E086C6FC@oracle.com>	<4F22FA22.3000106@oracle.com>
	<4F25DC17.60309@oracle.com>	<C592823E-DB9A-47B1-8E4E-26E11AAA36E4@oracle.com>
	<4F25FFC1.5020904@oracle.com> <4F266D93.9080308@oracle.com>
Message-ID: <76326728-2D3E-4690-8F32-F6C79189B6CC@oracle.com>


On Jan 30, 2012, at 2:14 AM, Bengt Rutisson wrote:

> 
> Hi Tom,
> 
> Looked at your latest webrev. Looks good to me.
> 
> Some minor comments:
> 
> 
> gcLocker.cpp
> 
> You still use tty->timestamp().milliseconds() instead of javaTimeNanos(). Not sure whether you just didn't get to fixing it yet or if you decided against it.

I decided against it.  It's a printing timestamp so matching the time stamp in tty seemed to make more sense to me.

> 
> 
> 
> gcLocker.hpp
> 
>  84   static void verify_critical_count() NOT_DEBUG_RETURN;
> 
> Not sure what the correct thing is here, but the GC code seems to use PRODUCT_RETURN more than NOT_DEBUG_RETURN. I guess the only difference is for optimized builds (and of course using #ifndef PRODUCT instead of #ifdef ASSERT).

I didn't want it doing any work in optimized builds so that's why it's NOT_DEBUG_RETURN.

> 
> 
> 
> safepoint.cpp
> 
> Not strictly related to your change, but the SafepointSynchronize::begin() method is missing one indentation level due to the the scoping that starts on line 139:

Yes, I noticed that too.  The whole thing could be factored better too.  I already pushed so I can't fix it.  I've got another change coming so maybe I'll fix it in there.  Thanks.

tom

> 
> 139   {
> 140   MutexLocker mu(Safepoint_lock);
> 141
> 142   // Reset the count of active JNI critical threads
> 143   _current_jni_active_count = 0;
> 
> This scope ends at the end of the method:
> 
> 398   if (PrintSafepointStatistics) {
> 399     // Record how much time spend on the above cleanup tasks
> 400     update_statistics_on_cleanup_end(os::javaTimeNanos());
> 401   }
> 402   }
> 403 }
> 
> It seems to me that the scope is not really necessary. In that case, can we remove it? It would be nice to have the method correctly indented. If we decide to keep the scope I'd prefer a comment on line 402, to indicate that it is the end of the MutexLocker scope.
> 
> Bengt
> 
> On 2012-01-30 03:26, David Holmes wrote:
>> Hi Tom,
>> 
>> On 30/01/2012 10:30 AM, Tom Rodriguez wrote:
>>> On Jan 29, 2012, at 3:53 PM, David Holmes wrote:
>>> 
>>>> On 28/01/2012 5:25 AM, Vladimir Kozlov wrote:
>>>>> I change alias to hotspot-dev since it affects everyone.
>>>> 
>>>> Thanks Vladimir!
>>>> 
>>>> Overall I find it difficult to follow the protocol that applies here - in particular how safepoints can and can not appear between different methods and the way in which needs_gc() might change value between the enter and exit of a critical region. Without that understanding it is hard to evaluate the correctness of the new approach.
>>> 
>>> I tried to put in enough asserts to ensure that it was clear what invariants are in force when updating the state.  Safepoints can appear wherever they normally can appear, while locking and waiting on mutexes.  _needs_gc can only change from false to true during a safepoint.  For the most part everything operates exactly as it used to once _needs_gc is set and the only trick is computing the correct value of _jni_active_count when setting _needs_gc to be true.  The debug code that still performs the atomics attempts to verify that the computed value and actual value are in sync.  Is there any more that I can explain about how it operates?
>> 
>> It's my ignorance of the overall operation that makes evaluating this change difficult for me. But thanks to your side-band emails I now have a much clear understanding of things - thanks. And the protocol seems safe.
>> 
>>>> src/share/vm/memory/gcLocker.cpp
>>>> 
>>>> 72       _wait_begin = os::javaTimeMillis();
>>>> 135      os::javaTimeMillis() - _wait_begin,
>>>> 
>>>> Are you sure you want to use the non-monotonic javaTimeMillis here? Use of this was recently changed elsewhere in the GC code.
>>> 
>>> It's only printing code, but I'll switch it to use tty->timestamp().milliseconds() which will make it match our internal time stamps, which seems like a useful property.  Is that ok?
>> 
>> The recent GC changes switched to using javaTimeNanos() converted to millis - see 7117303 and 7129514. The TimeStamp is based on elapsedCounter which is still a time-of-day source on some platforms and so not monotonic.
>> 
>> Thanks,
>> David
>> 
>>> I've updated the webrev.
>>> 
>>> tom
>>> 
>>>> 
>>>> 
>>>> Cheers,
>>>> David
>>>> -----
>>>> 
>>>> 
>>>> 
>>>>> In GC_locker::check_active_before_gc() move wait_begin initialization
>>>>> under Print* check since it used only under such checks. Could method
>>>>> check_active_before_gc() be called from different threads concurrently?
>>>>> Does it need lock? Note that other methods have lock.
>>>>> 
>>>>> Thanks,
>>>>> Vladimir
>>>>> 
>>>>> Tom Rodriguez wrote:
>>>>>> http://cr.openjdk.java.net/~never/7129164
>>>>>> 200 lines changed: 126 ins; 33 del; 41 mod; 3958 unchg
>>>>>> 
>>>>>> 7129164: JNI Get/ReleasePrimitiveArrayCritical doesn't scale
>>>>>> Summary:
>>>>>> Reviewed-by:
>>>>>> 
>>>>>> The machinery for GC_locker which supports GetPrimitiveArrayCritical
>>>>>> maintains a count of the number of threads that currently have raw
>>>>>> pointers exposed. This is used to allow existing critical sections to
>>>>>> drain before creating new ones so that a GC can occur. Currently
>>>>>> these counts are updated using atomic all the time, even if a GC isn't
>>>>>> being requested. This creates scalability problem when a lot of
>>>>>> threads are hammering atomic operations on the jni_lock_count. The
>>>>>> count only needs to be valid when checking if a critical section is
>>>>>> currently active and when draining the existing sections. The fix is
>>>>>> to make the count be computed as part of the safepointing machinery
>>>>>> and to only decrement the counters when _needs_gc is true. In debug
>>>>>> mode the old count is maintained and validated against the lazily
>>>>>> computed count.
>>>>>> 
>>>>>> On a microbenchmark that stresses GetPrimitiveArrayCritical with many
>>>>>> threads and relatively short critical section it can more than double
>>>>>> the throughput. This also slightly speeds up the normal
>>>>>> GetPrimtiveArrayCritical calls. Mostly it's not easily measurable
>>>>>> with larger scale programs.
>>>>>> 
>>>>>> Tested with microbenchmark that stresses GetPrimtiveArrayCritical and
>>>>>> the crypto benchmarks of specjvm2008 on x86 and sparc. I also ran the
>>>>>> java.io regression tests from the JDK.
>>>>>> 
>>> 
> 


From aph at redhat.com  Mon Jan 30 09:22:23 2012
From: aph at redhat.com (Andrew Haley)
Date: Mon, 30 Jan 2012 17:22:23 +0000
Subject: hsdis and caller options
In-Reply-To: <1623C71E-7E7E-4F2F-98A1-2B50A456E465@oracle.com>
References: <4F1854E1.7030008@redhat.com>
	<E396B7AC-3E57-479E-8C3A-3494B215F581@oracle.com>
	<4F191AD9.5010701@redhat.com>
	<1623C71E-7E7E-4F2F-98A1-2B50A456E465@oracle.com>
Message-ID: <4F26D1CF.3010805@redhat.com>

On 01/20/2012 08:30 AM, John Rose wrote:
>> OK.  Should I create a bug report for this?
> Yes, please.  -- John

OK, done.  Title is "HSDIS does not handle caller options correctly"

I have no ID for that bug.

Andrew.


From aph at redhat.com  Mon Jan 30 09:23:16 2012
From: aph at redhat.com (Andrew Haley)
Date: Mon, 30 Jan 2012 17:23:16 +0000
Subject: hsdis and caller options
In-Reply-To: <4F26D1CF.3010805@redhat.com>
References: <4F1854E1.7030008@redhat.com>
	<E396B7AC-3E57-479E-8C3A-3494B215F581@oracle.com>
	<4F191AD9.5010701@redhat.com>
	<1623C71E-7E7E-4F2F-98A1-2B50A456E465@oracle.com>
	<4F26D1CF.3010805@redhat.com>
Message-ID: <4F26D204.5070300@redhat.com>

On 01/30/2012 05:22 PM, Andrew Haley wrote:
> On 01/20/2012 08:30 AM, John Rose wrote:
>>> OK.  Should I create a bug report for this?
>> Yes, please.  -- John
> 
> OK, done.  Title is "HSDIS does not handle caller options correctly"
> 
> I have no ID for that bug.

I do now:

http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7140985

Andrew.

From karen.kinnear at oracle.com  Mon Jan 30 10:35:14 2012
From: karen.kinnear at oracle.com (Karen Kinnear)
Date: Mon, 30 Jan 2012 13:35:14 -0500
Subject: Code and Design Feedback request: 7114376: tune system dictionary
	size
In-Reply-To: <4F2624E1.1020001@oracle.com>
References: <73E0E1AF-1806-439B-B826-AE4E59590BFD@oracle.com>	<CAGVnyzRHxg8SD5kx4duFCMuf7xzzaBGng_CBxC5n+DfmrzekZw@mail.gmail.com>	<F45DFA65-925A-446D-99A4-1F0D7AEF8496@oracle.com>	<4F1CBCDC.4060808@oracle.com>
	<400030A9-3FC7-4318-85F9-3097DC5638C3@oracle.com>
	<4F20A54F.8050604@oracle.com>
	<C56B8E6F-383E-4ADB-873E-1660411D850D@oracle.com>
	<93AA3E6D-352F-4D95-A932-70DB5CA9FE32@oracle.com>
	<4F2624E1.1020001@oracle.com>
Message-ID: <76B15119-559B-4174-9F8E-3720CC069B18@oracle.com>

David et al,

Latest webrev:
http://cr.openjdk.java.net/~acorn/7114376.03/webrev/

Good catch. I put this back - the shared archive system dictionary size doesn't change.

Paul also asked for #defines to be moved into the Constants enum, so I did that as well.
And I made this experimental, so it requires the -XX:+UnlockExperimentalVMOptions.

thanks,
Karen

p.s. again: manually tested. JPRT in progress, nsk's resubmitted.

On Jan 30, 2012, at 12:04 AM, David Holmes wrote:

> Hi Karen,
> 
> On 30/01/2012 1:12 PM, Karen Kinnear wrote:
>> Final review please -
>> 
>> I reduced this to just add the command-line flag -XX:PredictedLoadedClassCount=#
>> and left the dynamic resizing for later.
> 
> Ok.
> 
> systemDictionary.cpp:
> 
> 1171 void SystemDictionary::set_shared_dictionary(HashtableBucket* t, int length,
> 1172                                              int number_of_entries) {
> 1173   int newsize = length / sizeof(HashtableBucket);
> 1174   _shared_dictionary = new Dictionary(_nof_buckets, t, number_of_entries);
> 1175 }
> 
> newSize is calculated but not used.
> 
> David
> -----
> 
> 
>> webrev: http://cr.openjdk.java.net/~acorn/7114376.02/webrev/
>> 
>> Testing:
>> Manual testing of command-line options, CDS, scimark.
>> In parallel: JPRT, nsk, benchmarks
>> 
>> thanks,
>> Karen
>> 
>> p.s. the following files did not change between webrev.01 and webrev.02
>> agent/...
>> globals.hpp
>> vmStructs.cpp
>> 
>> On Jan 27, 2012, at 6:13 PM, Karen Kinnear wrote:
>> 
>>> Dan&  David -
>>> 
>>> Thank you both for the detailed reviews.
>>> 
>>> I will make the changes suggested - but I will put the code changes on hold. I think
>>> the best approach right now is to just add the command-line flag and then after
>>> we get the permanent generation changes, rethink the loaded class cache storage
>>> options - and do a whole lot more measurements than I am able to do in this timeframe -
>>> even thanks to the help I have had.
>>> 
>>> Details below,
>>> thanks,
>>> Karen
>> 


From paul.hohensee at oracle.com  Mon Jan 30 11:02:54 2012
From: paul.hohensee at oracle.com (Paul Hohensee)
Date: Mon, 30 Jan 2012 14:02:54 -0500
Subject: Code and Design Feedback request: 7114376: tune system dictionary
	size
In-Reply-To: <76B15119-559B-4174-9F8E-3720CC069B18@oracle.com>
References: <73E0E1AF-1806-439B-B826-AE4E59590BFD@oracle.com>	<CAGVnyzRHxg8SD5kx4duFCMuf7xzzaBGng_CBxC5n+DfmrzekZw@mail.gmail.com>	<F45DFA65-925A-446D-99A4-1F0D7AEF8496@oracle.com>	<4F1CBCDC.4060808@oracle.com>
	<400030A9-3FC7-4318-85F9-3097DC5638C3@oracle.com>
	<4F20A54F.8050604@oracle.com>
	<C56B8E6F-383E-4ADB-873E-1660411D850D@oracle.com>
	<93AA3E6D-352F-4D95-A932-70DB5CA9FE32@oracle.com>
	<4F2624E1.1020001@oracle.com>
	<76B15119-559B-4174-9F8E-3720CC069B18@oracle.com>
Message-ID: <4F26E95E.2010808@oracle.com>

On 1/30/12 1:35 PM, Karen Kinnear wrote:
> David et al,
>
> Latest webrev:
> http://cr.openjdk.java.net/~acorn/7114376.03/webrev/
>
> Good catch. I put this back - the shared archive system dictionary size doesn't change.
>
> Paul also asked for #defines to be moved into the Constants enum, so I did that as well.
> And I made this experimental, so it requires the -XX:+UnlockExperimentalVMOptions.

Thanks for making those changes.  Looks good to go.

Paul

>
> thanks,
> Karen
>
> p.s. again: manually tested. JPRT in progress, nsk's resubmitted.
>
> On Jan 30, 2012, at 12:04 AM, David Holmes wrote:
>
>> Hi Karen,
>>
>> On 30/01/2012 1:12 PM, Karen Kinnear wrote:
>>> Final review please -
>>>
>>> I reduced this to just add the command-line flag -XX:PredictedLoadedClassCount=#
>>> and left the dynamic resizing for later.
>> Ok.
>>
>> systemDictionary.cpp:
>>
>> 1171 void SystemDictionary::set_shared_dictionary(HashtableBucket* t, int length,
>> 1172                                              int number_of_entries) {
>> 1173   int newsize = length / sizeof(HashtableBucket);
>> 1174   _shared_dictionary = new Dictionary(_nof_buckets, t, number_of_entries);
>> 1175 }
>>
>> newSize is calculated but not used.
>>
>> David
>> -----
>>
>>
>>> webrev: http://cr.openjdk.java.net/~acorn/7114376.02/webrev/
>>>
>>> Testing:
>>> Manual testing of command-line options, CDS, scimark.
>>> In parallel: JPRT, nsk, benchmarks
>>>
>>> thanks,
>>> Karen
>>>
>>> p.s. the following files did not change between webrev.01 and webrev.02
>>> agent/...
>>> globals.hpp
>>> vmStructs.cpp
>>>
>>> On Jan 27, 2012, at 6:13 PM, Karen Kinnear wrote:
>>>
>>>> Dan&   David -
>>>>
>>>> Thank you both for the detailed reviews.
>>>>
>>>> I will make the changes suggested - but I will put the code changes on hold. I think
>>>> the best approach right now is to just add the command-line flag and then after
>>>> we get the permanent generation changes, rethink the loaded class cache storage
>>>> options - and do a whole lot more measurements than I am able to do in this timeframe -
>>>> even thanks to the help I have had.
>>>>
>>>> Details below,
>>>> thanks,
>>>> Karen

From daniel.daugherty at oracle.com  Mon Jan 30 14:02:56 2012
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Mon, 30 Jan 2012 15:02:56 -0700
Subject: Code and Design Feedback request: 7114376: tune system dictionary
	size
In-Reply-To: <76B15119-559B-4174-9F8E-3720CC069B18@oracle.com>
References: <73E0E1AF-1806-439B-B826-AE4E59590BFD@oracle.com>	<CAGVnyzRHxg8SD5kx4duFCMuf7xzzaBGng_CBxC5n+DfmrzekZw@mail.gmail.com>	<F45DFA65-925A-446D-99A4-1F0D7AEF8496@oracle.com>	<4F1CBCDC.4060808@oracle.com>	<400030A9-3FC7-4318-85F9-3097DC5638C3@oracle.com>	<4F20A54F.8050604@oracle.com>	<C56B8E6F-383E-4ADB-873E-1660411D850D@oracle.com>	<93AA3E6D-352F-4D95-A932-70DB5CA9FE32@oracle.com>	<4F2624E1.1020001@oracle.com>
	<76B15119-559B-4174-9F8E-3720CC069B18@oracle.com>
Message-ID: <4F271390.9060203@oracle.com>

On 1/30/12 11:35 AM, Karen Kinnear wrote:
> David et al,
>
> Latest webrev:
> http://cr.openjdk.java.net/~acorn/7114376.03/webrev/

Thumbs up!
Only naming suggestions and nits. Feel free to ignore.

agent/src/share/classes/sun/jvm/hotspot/memory/LoaderConstraintTable.java
     No comments.

agent/src/share/classes/sun/jvm/hotspot/memory/SystemDictionary.java
     No comments.

src/share/vm/classfile/dictionary.cpp
     No comments.

src/share/vm/classfile/systemDictionary.hpp
     Now that these constants are all in the enum, it is more clear
     that your naming style is different than the existing stuff:

  539   enum Constants {
  540     _loader_constraint_size = 107,                     // number 
of entries in constraint table
  541     _resolution_error_size  = 107,                     // number 
of entries in resolution error table
  542     _invoke_method_size     = 139,                     // number 
of entries in invoke method table
  543     _nof_buckets            = 1009,                    // number 
of buckets in hash table for placeholders
  544     _olddefaultsdsize       = 1009,                    // backward 
compat for system dictionary size
  545     _primearraysize         = 8,                       // array of 
primes for system dictionary size
  546     _averagedepthgoal       = 3                        // goal for 
lookup length
  547   };

     _olddefaultsdsize => _old_default_sdsize
     _primearraysize   => _prime_array_size
     _averagedepthgoal => _average_depth_goal

     Of course, feel free to say "no more renaming!".

src/share/vm/classfile/systemDictionary.cpp
     nit line 68: one space of on '=' alignment
     nit line 69: can you indent those numbers? Inside the "System..." by
         two blanks would be great.

src/share/vm/runtime/globals.hpp
     No comments.

src/share/vm/runtime/vmStructs.cpp
     No comments.

src/share/vm/utilities/hashtable.hpp
     No comments.


From david.holmes at oracle.com  Mon Jan 30 16:33:10 2012
From: david.holmes at oracle.com (David Holmes)
Date: Tue, 31 Jan 2012 10:33:10 +1000
Subject: Review request (S): 7140882 (Was: Re: Patch to fix build breakage
	with GCC 4.7)
In-Reply-To: <b9ed18b8-07f4-43b4-ab02-76ba0a6946fc@zmail16.collab.prod.int.phx2.redhat.com>
References: <b9ed18b8-07f4-43b4-ab02-76ba0a6946fc@zmail16.collab.prod.int.phx2.redhat.com>
Message-ID: <4F2736C6.70805@oracle.com>

Hi Andrew,

On 31/01/2012 1:38 AM, Andrew Hughes wrote:
> ----- Original Message -----
>
>> Deepak, I saw that you are an "author" for the JDK projects, but not
>> for
>> the HSX project. Thus, I can't create the changeset as your user.
>> Instead I list you as "Contributed-by". That field only supports
>> email
>> addresses. Here is what the changeset will look like:
<snip>>
>>>
>
> (ccing discuss)
>
> That seems wrong.  No-one commits to the jdk projects directly, so
> what's the point in being a committer/reviewer there?
>
> Both Deepak and I have pushed patches to various repositories before.
> Why are our rights being taken away like this?

I think you have this backwards. No one externally can commit to hotspot 
(HSX) directly due to the need to go through JPRT. But commits to JDK 
repos (jdk8/*, jdk7/*) can be done directly (the jdkN/hotspot repos only 
get populated by getting changes from the hsx/hotspot repo).

To submit hotspot patches you need the relevant role in the HSX project. 
But until external pushes can be done the whole Contributor vs Author vs 
Committer issue is somewhat moot.

David Holmes

From david.holmes at oracle.com  Mon Jan 30 16:39:34 2012
From: david.holmes at oracle.com (David Holmes)
Date: Tue, 31 Jan 2012 10:39:34 +1000
Subject: Code and Design Feedback request: 7114376: tune system dictionary
	size
In-Reply-To: <76B15119-559B-4174-9F8E-3720CC069B18@oracle.com>
References: <73E0E1AF-1806-439B-B826-AE4E59590BFD@oracle.com>	<CAGVnyzRHxg8SD5kx4duFCMuf7xzzaBGng_CBxC5n+DfmrzekZw@mail.gmail.com>	<F45DFA65-925A-446D-99A4-1F0D7AEF8496@oracle.com>	<4F1CBCDC.4060808@oracle.com>
	<400030A9-3FC7-4318-85F9-3097DC5638C3@oracle.com>
	<4F20A54F.8050604@oracle.com>
	<C56B8E6F-383E-4ADB-873E-1660411D850D@oracle.com>
	<93AA3E6D-352F-4D95-A932-70DB5CA9FE32@oracle.com>
	<4F2624E1.1020001@oracle.com>
	<76B15119-559B-4174-9F8E-3720CC069B18@oracle.com>
Message-ID: <4F273846.1070905@oracle.com>

Hi Karen,

Other than that I don't see why this needs to be experimental I don't 
have any further comments.

Thanks,
David

On 31/01/2012 4:35 AM, Karen Kinnear wrote:
> David et al,
>
> Latest webrev:
> http://cr.openjdk.java.net/~acorn/7114376.03/webrev/
>
> Good catch. I put this back - the shared archive system dictionary size doesn't change.
>
> Paul also asked for #defines to be moved into the Constants enum, so I did that as well.
> And I made this experimental, so it requires the -XX:+UnlockExperimentalVMOptions.
>
> thanks,
> Karen
>
> p.s. again: manually tested. JPRT in progress, nsk's resubmitted.
>
> On Jan 30, 2012, at 12:04 AM, David Holmes wrote:
>
>> Hi Karen,
>>
>> On 30/01/2012 1:12 PM, Karen Kinnear wrote:
>>> Final review please -
>>>
>>> I reduced this to just add the command-line flag -XX:PredictedLoadedClassCount=#
>>> and left the dynamic resizing for later.
>>
>> Ok.
>>
>> systemDictionary.cpp:
>>
>> 1171 void SystemDictionary::set_shared_dictionary(HashtableBucket* t, int length,
>> 1172                                              int number_of_entries) {
>> 1173   int newsize = length / sizeof(HashtableBucket);
>> 1174   _shared_dictionary = new Dictionary(_nof_buckets, t, number_of_entries);
>> 1175 }
>>
>> newSize is calculated but not used.
>>
>> David
>> -----
>>
>>
>>> webrev: http://cr.openjdk.java.net/~acorn/7114376.02/webrev/
>>>
>>> Testing:
>>> Manual testing of command-line options, CDS, scimark.
>>> In parallel: JPRT, nsk, benchmarks
>>>
>>> thanks,
>>> Karen
>>>
>>> p.s. the following files did not change between webrev.01 and webrev.02
>>> agent/...
>>> globals.hpp
>>> vmStructs.cpp
>>>
>>> On Jan 27, 2012, at 6:13 PM, Karen Kinnear wrote:
>>>
>>>> Dan&   David -
>>>>
>>>> Thank you both for the detailed reviews.
>>>>
>>>> I will make the changes suggested - but I will put the code changes on hold. I think
>>>> the best approach right now is to just add the command-line flag and then after
>>>> we get the permanent generation changes, rethink the loaded class cache storage
>>>> options - and do a whole lot more measurements than I am able to do in this timeframe -
>>>> even thanks to the help I have had.
>>>>
>>>> Details below,
>>>> thanks,
>>>> Karen
>>>
>

From karen.kinnear at oracle.com  Mon Jan 30 17:07:07 2012
From: karen.kinnear at oracle.com (Karen Kinnear)
Date: Mon, 30 Jan 2012 20:07:07 -0500
Subject: Code and Design Feedback request: 7114376: tune system dictionary
	size
In-Reply-To: <4F273846.1070905@oracle.com>
References: <73E0E1AF-1806-439B-B826-AE4E59590BFD@oracle.com>	<CAGVnyzRHxg8SD5kx4duFCMuf7xzzaBGng_CBxC5n+DfmrzekZw@mail.gmail.com>	<F45DFA65-925A-446D-99A4-1F0D7AEF8496@oracle.com>	<4F1CBCDC.4060808@oracle.com>
	<400030A9-3FC7-4318-85F9-3097DC5638C3@oracle.com>
	<4F20A54F.8050604@oracle.com>
	<C56B8E6F-383E-4ADB-873E-1660411D850D@oracle.com>
	<93AA3E6D-352F-4D95-A932-70DB5CA9FE32@oracle.com>
	<4F2624E1.1020001@oracle.com>
	<76B15119-559B-4174-9F8E-3720CC069B18@oracle.com>
	<4F273846.1070905@oracle.com>
Message-ID: <BD9B3051-56C8-42FE-98DC-798B19ECAC72@oracle.com>

Thanks David. The amount of benchmarking for using this new flag has been more limited than
I would like for a fully supported flag. And given that we are going to be revising the underlying
data structure later, it may be more appropriate to offer a different flag, so this sets different
expectations on whether this flag will still be supported.

thanks for all the reviews,
Karen

On Jan 30, 2012, at 7:39 PM, David Holmes wrote:

> Hi Karen,
> 
> Other than that I don't see why this needs to be experimental I don't have any further comments.
> 
> Thanks,
> David
> 
> On 31/01/2012 4:35 AM, Karen Kinnear wrote:
>> David et al,
>> 
>> Latest webrev:
>> http://cr.openjdk.java.net/~acorn/7114376.03/webrev/
>> 
>> Good catch. I put this back - the shared archive system dictionary size doesn't change.
>> 
>> Paul also asked for #defines to be moved into the Constants enum, so I did that as well.
>> And I made this experimental, so it requires the -XX:+UnlockExperimentalVMOptions.
>> 
>> thanks,
>> Karen
>> 
>> p.s. again: manually tested. JPRT in progress, nsk's resubmitted.
>> 
>> On Jan 30, 2012, at 12:04 AM, David Holmes wrote:
>> 
>>> Hi Karen,
>>> 
>>> On 30/01/2012 1:12 PM, Karen Kinnear wrote:
>>>> Final review please -
>>>> 
>>>> I reduced this to just add the command-line flag -XX:PredictedLoadedClassCount=#
>>>> and left the dynamic resizing for later.
>>> 
>>> Ok.
>>> 
>>> systemDictionary.cpp:
>>> 
>>> 1171 void SystemDictionary::set_shared_dictionary(HashtableBucket* t, int length,
>>> 1172                                              int number_of_entries) {
>>> 1173   int newsize = length / sizeof(HashtableBucket);
>>> 1174   _shared_dictionary = new Dictionary(_nof_buckets, t, number_of_entries);
>>> 1175 }
>>> 
>>> newSize is calculated but not used.
>>> 
>>> David
>>> -----
>>> 
>>> 
>>>> webrev: http://cr.openjdk.java.net/~acorn/7114376.02/webrev/
>>>> 
>>>> Testing:
>>>> Manual testing of command-line options, CDS, scimark.
>>>> In parallel: JPRT, nsk, benchmarks
>>>> 
>>>> thanks,
>>>> Karen
>>>> 
>>>> p.s. the following files did not change between webrev.01 and webrev.02
>>>> agent/...
>>>> globals.hpp
>>>> vmStructs.cpp
>>>> 
>>>> On Jan 27, 2012, at 6:13 PM, Karen Kinnear wrote:
>>>> 
>>>>> Dan&   David -
>>>>> 
>>>>> Thank you both for the detailed reviews.
>>>>> 
>>>>> I will make the changes suggested - but I will put the code changes on hold. I think
>>>>> the best approach right now is to just add the command-line flag and then after
>>>>> we get the permanent generation changes, rethink the loaded class cache storage
>>>>> options - and do a whole lot more measurements than I am able to do in this timeframe -
>>>>> even thanks to the help I have had.
>>>>> 
>>>>> Details below,
>>>>> thanks,
>>>>> Karen
>>>> 
>> 


From tom.rodriguez at oracle.com  Mon Jan 30 17:42:09 2012
From: tom.rodriguez at oracle.com (Tom Rodriguez)
Date: Mon, 30 Jan 2012 17:42:09 -0800
Subject: review for 7129164: JNI Get/ReleasePrimitiveArrayCritical doesn't
	scale
In-Reply-To: <76326728-2D3E-4690-8F32-F6C79189B6CC@oracle.com>
References: <4DDD98EA-07C9-44D9-AF7E-C938E086C6FC@oracle.com>	<4F22FA22.3000106@oracle.com>
	<4F25DC17.60309@oracle.com>	<C592823E-DB9A-47B1-8E4E-26E11AAA36E4@oracle.com>
	<4F25FFC1.5020904@oracle.com> <4F266D93.9080308@oracle.com>
	<76326728-2D3E-4690-8F32-F6C79189B6CC@oracle.com>
Message-ID: <359A5416-4618-44C9-8588-1C2BEC55A6CC@oracle.com>


On Jan 30, 2012, at 8:14 AM, Tom Rodriguez wrote:

> 
> On Jan 30, 2012, at 2:14 AM, Bengt Rutisson wrote:
> 
>> 
>> Hi Tom,
>> 
>> Looked at your latest webrev. Looks good to me.
>> 
>> Some minor comments:
>> 
>> 
>> gcLocker.cpp
>> 
>> You still use tty->timestamp().milliseconds() instead of javaTimeNanos(). Not sure whether you just didn't get to fixing it yet or if you decided against it.
> 
> I decided against it.  It's a printing timestamp so matching the time stamp in tty seemed to make more sense to me.

I'm about to send out a webrev for some other changes and I'm going to correct this to use javaTimeNanos to be consistent with the other GC timestamps.  I'm also going to remove the extra braces in safepoint.cpp.

tom

> 
>> 
>> 
>> 
>> gcLocker.hpp
>> 
>> 84   static void verify_critical_count() NOT_DEBUG_RETURN;
>> 
>> Not sure what the correct thing is here, but the GC code seems to use PRODUCT_RETURN more than NOT_DEBUG_RETURN. I guess the only difference is for optimized builds (and of course using #ifndef PRODUCT instead of #ifdef ASSERT).
> 
> I didn't want it doing any work in optimized builds so that's why it's NOT_DEBUG_RETURN.
> 
>> 
>> 
>> 
>> safepoint.cpp
>> 
>> Not strictly related to your change, but the SafepointSynchronize::begin() method is missing one indentation level due to the the scoping that starts on line 139:
> 
> Yes, I noticed that too.  The whole thing could be factored better too.  I already pushed so I can't fix it.  I've got another change coming so maybe I'll fix it in there.  Thanks.
> 
> tom
> 
>> 
>> 139   {
>> 140   MutexLocker mu(Safepoint_lock);
>> 141
>> 142   // Reset the count of active JNI critical threads
>> 143   _current_jni_active_count = 0;
>> 
>> This scope ends at the end of the method:
>> 
>> 398   if (PrintSafepointStatistics) {
>> 399     // Record how much time spend on the above cleanup tasks
>> 400     update_statistics_on_cleanup_end(os::javaTimeNanos());
>> 401   }
>> 402   }
>> 403 }
>> 
>> It seems to me that the scope is not really necessary. In that case, can we remove it? It would be nice to have the method correctly indented. If we decide to keep the scope I'd prefer a comment on line 402, to indicate that it is the end of the MutexLocker scope.
>> 
>> Bengt
>> 
>> On 2012-01-30 03:26, David Holmes wrote:
>>> Hi Tom,
>>> 
>>> On 30/01/2012 10:30 AM, Tom Rodriguez wrote:
>>>> On Jan 29, 2012, at 3:53 PM, David Holmes wrote:
>>>> 
>>>>> On 28/01/2012 5:25 AM, Vladimir Kozlov wrote:
>>>>>> I change alias to hotspot-dev since it affects everyone.
>>>>> 
>>>>> Thanks Vladimir!
>>>>> 
>>>>> Overall I find it difficult to follow the protocol that applies here - in particular how safepoints can and can not appear between different methods and the way in which needs_gc() might change value between the enter and exit of a critical region. Without that understanding it is hard to evaluate the correctness of the new approach.
>>>> 
>>>> I tried to put in enough asserts to ensure that it was clear what invariants are in force when updating the state.  Safepoints can appear wherever they normally can appear, while locking and waiting on mutexes.  _needs_gc can only change from false to true during a safepoint.  For the most part everything operates exactly as it used to once _needs_gc is set and the only trick is computing the correct value of _jni_active_count when setting _needs_gc to be true.  The debug code that still performs the atomics attempts to verify that the computed value and actual value are in sync.  Is there any more that I can explain about how it operates?
>>> 
>>> It's my ignorance of the overall operation that makes evaluating this change difficult for me. But thanks to your side-band emails I now have a much clear understanding of things - thanks. And the protocol seems safe.
>>> 
>>>>> src/share/vm/memory/gcLocker.cpp
>>>>> 
>>>>> 72       _wait_begin = os::javaTimeMillis();
>>>>> 135      os::javaTimeMillis() - _wait_begin,
>>>>> 
>>>>> Are you sure you want to use the non-monotonic javaTimeMillis here? Use of this was recently changed elsewhere in the GC code.
>>>> 
>>>> It's only printing code, but I'll switch it to use tty->timestamp().milliseconds() which will make it match our internal time stamps, which seems like a useful property.  Is that ok?
>>> 
>>> The recent GC changes switched to using javaTimeNanos() converted to millis - see 7117303 and 7129514. The TimeStamp is based on elapsedCounter which is still a time-of-day source on some platforms and so not monotonic.
>>> 
>>> Thanks,
>>> David
>>> 
>>>> I've updated the webrev.
>>>> 
>>>> tom
>>>> 
>>>>> 
>>>>> 
>>>>> Cheers,
>>>>> David
>>>>> -----
>>>>> 
>>>>> 
>>>>> 
>>>>>> In GC_locker::check_active_before_gc() move wait_begin initialization
>>>>>> under Print* check since it used only under such checks. Could method
>>>>>> check_active_before_gc() be called from different threads concurrently?
>>>>>> Does it need lock? Note that other methods have lock.
>>>>>> 
>>>>>> Thanks,
>>>>>> Vladimir
>>>>>> 
>>>>>> Tom Rodriguez wrote:
>>>>>>> http://cr.openjdk.java.net/~never/7129164
>>>>>>> 200 lines changed: 126 ins; 33 del; 41 mod; 3958 unchg
>>>>>>> 
>>>>>>> 7129164: JNI Get/ReleasePrimitiveArrayCritical doesn't scale
>>>>>>> Summary:
>>>>>>> Reviewed-by:
>>>>>>> 
>>>>>>> The machinery for GC_locker which supports GetPrimitiveArrayCritical
>>>>>>> maintains a count of the number of threads that currently have raw
>>>>>>> pointers exposed. This is used to allow existing critical sections to
>>>>>>> drain before creating new ones so that a GC can occur. Currently
>>>>>>> these counts are updated using atomic all the time, even if a GC isn't
>>>>>>> being requested. This creates scalability problem when a lot of
>>>>>>> threads are hammering atomic operations on the jni_lock_count. The
>>>>>>> count only needs to be valid when checking if a critical section is
>>>>>>> currently active and when draining the existing sections. The fix is
>>>>>>> to make the count be computed as part of the safepointing machinery
>>>>>>> and to only decrement the counters when _needs_gc is true. In debug
>>>>>>> mode the old count is maintained and validated against the lazily
>>>>>>> computed count.
>>>>>>> 
>>>>>>> On a microbenchmark that stresses GetPrimitiveArrayCritical with many
>>>>>>> threads and relatively short critical section it can more than double
>>>>>>> the throughput. This also slightly speeds up the normal
>>>>>>> GetPrimtiveArrayCritical calls. Mostly it's not easily measurable
>>>>>>> with larger scale programs.
>>>>>>> 
>>>>>>> Tested with microbenchmark that stresses GetPrimtiveArrayCritical and
>>>>>>> the crypto benchmarks of specjvm2008 on x86 and sparc. I also ran the
>>>>>>> java.io regression tests from the JDK.
>>>>>>> 
>>>> 
>> 
> 


From karen.kinnear at oracle.com  Mon Jan 30 17:47:49 2012
From: karen.kinnear at oracle.com (Karen Kinnear)
Date: Mon, 30 Jan 2012 20:47:49 -0500
Subject: Code and Design Feedback request: 7114376: tune system dictionary
	size
In-Reply-To: <4F271390.9060203@oracle.com>
References: <73E0E1AF-1806-439B-B826-AE4E59590BFD@oracle.com>	<CAGVnyzRHxg8SD5kx4duFCMuf7xzzaBGng_CBxC5n+DfmrzekZw@mail.gmail.com>	<F45DFA65-925A-446D-99A4-1F0D7AEF8496@oracle.com>	<4F1CBCDC.4060808@oracle.com>	<400030A9-3FC7-4318-85F9-3097DC5638C3@oracle.com>	<4F20A54F.8050604@oracle.com>	<C56B8E6F-383E-4ADB-873E-1660411D850D@oracle.com>	<93AA3E6D-352F-4D95-A932-70DB5CA9FE32@oracle.com>	<4F2624E1.1020001@oracle.com>
	<76B15119-559B-4174-9F8E-3720CC069B18@oracle.com>
	<4F271390.9060203@oracle.com>
Message-ID: <29051759-8407-4E67-854E-06C2C8D5CDB2@oracle.com>

Dan,

Took all of your suggestions but didn't make a new webrev for this.

thanks for all the reviews,
Karen

On Jan 30, 2012, at 5:02 PM, Daniel D. Daugherty wrote:

> On 1/30/12 11:35 AM, Karen Kinnear wrote:
>> David et al,
>> 
>> Latest webrev:
>> http://cr.openjdk.java.net/~acorn/7114376.03/webrev/
> 
> Thumbs up!
> Only naming suggestions and nits. Feel free to ignore.
> 
> agent/src/share/classes/sun/jvm/hotspot/memory/LoaderConstraintTable.java
>    No comments.
> 
> agent/src/share/classes/sun/jvm/hotspot/memory/SystemDictionary.java
>    No comments.
> 
> src/share/vm/classfile/dictionary.cpp
>    No comments.
> 
> src/share/vm/classfile/systemDictionary.hpp
>    Now that these constants are all in the enum, it is more clear
>    that your naming style is different than the existing stuff:
> 
> 539   enum Constants {
> 540     _loader_constraint_size = 107,                     // number of entries in constraint table
> 541     _resolution_error_size  = 107,                     // number of entries in resolution error table
> 542     _invoke_method_size     = 139,                     // number of entries in invoke method table
> 543     _nof_buckets            = 1009,                    // number of buckets in hash table for placeholders
> 544     _olddefaultsdsize       = 1009,                    // backward compat for system dictionary size
> 545     _primearraysize         = 8,                       // array of primes for system dictionary size
> 546     _averagedepthgoal       = 3                        // goal for lookup length
> 547   };
> 
>    _olddefaultsdsize => _old_default_sdsize
>    _primearraysize   => _prime_array_size
>    _averagedepthgoal => _average_depth_goal
> 
>    Of course, feel free to say "no more renaming!".
> 
> src/share/vm/classfile/systemDictionary.cpp
>    nit line 68: one space of on '=' alignment
>    nit line 69: can you indent those numbers? Inside the "System..." by
>        two blanks would be great.
> 
> src/share/vm/runtime/globals.hpp
>    No comments.
> 
> src/share/vm/runtime/vmStructs.cpp
>    No comments.
> 
> src/share/vm/utilities/hashtable.hpp
>    No comments.
> 


From tom.rodriguez at oracle.com  Mon Jan 30 17:49:38 2012
From: tom.rodriguez at oracle.com (Tom Rodriguez)
Date: Mon, 30 Jan 2012 17:49:38 -0800
Subject: 7013347: allow crypto functions to be called inline to enhance
	performance 
Message-ID: <F890E827-AD93-4683-88D5-B7ADDBF545F1@oracle.com>

http://cr.openjdk.java.net/~never/7013347
1133 lines changed: 979 ins; 56 del; 98 mod; 35796 unchg

7013347: allow crypto functions to be called inline to enhance performance
Reviewed-by:

This is a long one.

The synopsis of this is slightly misleading.  This doens't allow
direct calls to native routines from Java but it does attempt to
reduce the overhead of using JNI for specific use cases while still
maintaining the safety invariants that JNI provdies.  For native code
that runs in a bounded time JNI provides a function called
GetPrimtiveArrayCritical which may provide direct access to the body
of Java arrays of primitive.  In Hotspot this is accomplished by
suppressing garbage collection while these pointers are exposed to
native code.  This is accomplished with the GC_locker class which is
basically a readers/writers lock.  Note that the GC_locker doesn't
suppress safepointing, just garbage collections.  There are many
operations which require a safepoint to make forward progress, so
suppressing them indefinitely isn't acceptable.

This RFE provides is a shorthand for the use of
GetPrimtiveArrayCritical by defining an alternate native calling
convention that only allows the use of primitive or arrays of
primtive.  The native method must also be static since non-static
methods are passed the receiver as an argument and Java objects aren't
allowed.  Synchronization and exceptions aren't allowed either.  The
Java code calling these natives is fee to use all of those features so
it's not that onerous of a restriction.

The benefits of this approach are that JVM can more quickly do the
work inline that would normally be done by the
GetPrimtiveArrayCritical/ReleasePrimtiveArrayCritical function calls.
Calling back into the JVM through JNI requires synchronization with
the JVM and each upcall adds a minimum overhead to the native routine.
This helps to reduce the overhead to a more fixed cost per call.  It
also simplifies the work that the caller must do since synchronization
and exceptions aren't allowed.  For now this work is being done in the
existing native wrapper generation but with some more simplification
this could be more easily inlined directly into the caller.

The signature of the native routine follows the same name mangling as
normal JNI methods but they start with JavaCritical_ instead of Java_.
Any array arguments are unpacked into a pair of arguments, the length
followed by a pointer to the body of the array.  If the incoming array
is NULL then the body pointer is NULL and the length is 0.

Currently this is a JDK private interface while we gain some
experience with it but it will likely become a more standard
extension.  It's also an optional extension so a native library is
required to provide the normal point in addition to the alternate
entry point.

The changes consist of three parts.  The first is the lookup logic
that finds the alternate native entry point.  JNI critical natives
currently can only be found through dynamic lookup.  JNI
RegisterNatives doesn't know about these functions so there's no way
to provide the alternate entry point.

The second part is the lazy critical entry logic.  The fix for 7129164
introduced code that computed the JNI active count during
safepointing.  Now as part of that computation, if a thread is seen to
be in thread_in_native state and the nmethod on the top of stack is a
critical native wrapper, then the critical count for that thread is
incremented and the suspend flags are set so that when the nmethod
returns the native code it will call back into the runtime and do the
unlock of the critical native.

The last part are the native wrappers themselves.  When compiling a
critical native wrapper, they emit a new check of GC_locker::_needs_gc
and they call into the runtime if it's true.  This keeps them from
starting new JNI critical sections if a GC has been requested.  The
arguments are unpacked following the alternate calling convention and
the method is called as it normally would be.

On return the wrapper checks the suspend flags as it normally would
and calls back into the runtime where is might have to block and force
a GC if it's the last thread exiting the GC_locker.  This required
some slightly different handling of the final transition back to
thread_in_Java since we have to allow blocking.

The wrappers are only generated differently if they are compiling a
critical native so it shouldn't have much effect on normal execution.
The only library currently taking advantage of this is the new ucrypto
provider on Solaris.  For some crypto operations it improves
throughput by 20% or more because the crypto routines are fast enough
that the JNi overhead is significant.  It's expected that other parts
of the JDK will take advantage of it going forward and hopefully it
can be tightened up further.

Tested with new crypto provider and microbenchmark test case.  Also
ran runthese.



From vladimir.kozlov at oracle.com  Mon Jan 30 20:10:39 2012
From: vladimir.kozlov at oracle.com (Vladimir Kozlov)
Date: Mon, 30 Jan 2012 20:10:39 -0800
Subject: 7013347: allow crypto functions to be called inline to enhance
	performance
In-Reply-To: <F890E827-AD93-4683-88D5-B7ADDBF545F1@oracle.com>
References: <F890E827-AD93-4683-88D5-B7ADDBF545F1@oracle.com>
Message-ID: <4F2769BF.9020109@oracle.com>

In NativeLookup::lookup_critical_style() jni_name id only used when (dll != 
NULL). Should the name construction be moved inside this check?

The same for critical_name in lookup_critical_entry().

In check_for_lazy_critical_native() should you check that stub_cb != NULL?
Also this code has typo (ifdef?):
+ #ifndef ASSERT


Why you don't use new branch instructions in your sparc assembler code?
Next code loads one byte but you tests all bits:

+   __ load_bool_contents(sync_state, G3_scratch);
+   __ cmp(G3_scratch, 0);
+   __ brx(Assembler::equal, false, Assembler::pt, cont);
+   __ delayed()->nop();

you can replace it with:

+   __ load_bool_contents(sync_state, G3_scratch);
+   __ cmp_zero_and_br(Assembler::equal, G3_scratch, cont);

----
+   __ ba(done);
+   __ delayed()->nop();

with:

+   __ ba_short(done);


Don't use next short forward branch in both x86_32 and x86_64 files since the 
code could be large depending on number of arguments:

+   __ cmp8(ExternalAddress((address)GC_locker::needs_gc_address()), false);
+   __ jccb(Assembler::equal, cont);


Thanks,
Vladimir

Tom Rodriguez wrote:
> http://cr.openjdk.java.net/~never/7013347
> 1133 lines changed: 979 ins; 56 del; 98 mod; 35796 unchg
> 
> 7013347: allow crypto functions to be called inline to enhance performance
> Reviewed-by:
> 
> This is a long one.
> 
> The synopsis of this is slightly misleading.  This doens't allow
> direct calls to native routines from Java but it does attempt to
> reduce the overhead of using JNI for specific use cases while still
> maintaining the safety invariants that JNI provdies.  For native code
> that runs in a bounded time JNI provides a function called
> GetPrimtiveArrayCritical which may provide direct access to the body
> of Java arrays of primitive.  In Hotspot this is accomplished by
> suppressing garbage collection while these pointers are exposed to
> native code.  This is accomplished with the GC_locker class which is
> basically a readers/writers lock.  Note that the GC_locker doesn't
> suppress safepointing, just garbage collections.  There are many
> operations which require a safepoint to make forward progress, so
> suppressing them indefinitely isn't acceptable.
> 
> This RFE provides is a shorthand for the use of
> GetPrimtiveArrayCritical by defining an alternate native calling
> convention that only allows the use of primitive or arrays of
> primtive.  The native method must also be static since non-static
> methods are passed the receiver as an argument and Java objects aren't
> allowed.  Synchronization and exceptions aren't allowed either.  The
> Java code calling these natives is fee to use all of those features so
> it's not that onerous of a restriction.
> 
> The benefits of this approach are that JVM can more quickly do the
> work inline that would normally be done by the
> GetPrimtiveArrayCritical/ReleasePrimtiveArrayCritical function calls.
> Calling back into the JVM through JNI requires synchronization with
> the JVM and each upcall adds a minimum overhead to the native routine.
> This helps to reduce the overhead to a more fixed cost per call.  It
> also simplifies the work that the caller must do since synchronization
> and exceptions aren't allowed.  For now this work is being done in the
> existing native wrapper generation but with some more simplification
> this could be more easily inlined directly into the caller.
> 
> The signature of the native routine follows the same name mangling as
> normal JNI methods but they start with JavaCritical_ instead of Java_.
> Any array arguments are unpacked into a pair of arguments, the length
> followed by a pointer to the body of the array.  If the incoming array
> is NULL then the body pointer is NULL and the length is 0.
> 
> Currently this is a JDK private interface while we gain some
> experience with it but it will likely become a more standard
> extension.  It's also an optional extension so a native library is
> required to provide the normal point in addition to the alternate
> entry point.
> 
> The changes consist of three parts.  The first is the lookup logic
> that finds the alternate native entry point.  JNI critical natives
> currently can only be found through dynamic lookup.  JNI
> RegisterNatives doesn't know about these functions so there's no way
> to provide the alternate entry point.
> 
> The second part is the lazy critical entry logic.  The fix for 7129164
> introduced code that computed the JNI active count during
> safepointing.  Now as part of that computation, if a thread is seen to
> be in thread_in_native state and the nmethod on the top of stack is a
> critical native wrapper, then the critical count for that thread is
> incremented and the suspend flags are set so that when the nmethod
> returns the native code it will call back into the runtime and do the
> unlock of the critical native.
> 
> The last part are the native wrappers themselves.  When compiling a
> critical native wrapper, they emit a new check of GC_locker::_needs_gc
> and they call into the runtime if it's true.  This keeps them from
> starting new JNI critical sections if a GC has been requested.  The
> arguments are unpacked following the alternate calling convention and
> the method is called as it normally would be.
> 
> On return the wrapper checks the suspend flags as it normally would
> and calls back into the runtime where is might have to block and force
> a GC if it's the last thread exiting the GC_locker.  This required
> some slightly different handling of the final transition back to
> thread_in_Java since we have to allow blocking.
> 
> The wrappers are only generated differently if they are compiling a
> critical native so it shouldn't have much effect on normal execution.
> The only library currently taking advantage of this is the new ucrypto
> provider on Solaris.  For some crypto operations it improves
> throughput by 20% or more because the crypto routines are fast enough
> that the JNi overhead is significant.  It's expected that other parts
> of the JDK will take advantage of it going forward and hopefully it
> can be tightened up further.
> 
> Tested with new crypto provider and microbenchmark test case.  Also
> ran runthese.
> 
> 

From tom.rodriguez at oracle.com  Mon Jan 30 22:25:11 2012
From: tom.rodriguez at oracle.com (Tom Rodriguez)
Date: Mon, 30 Jan 2012 22:25:11 -0800
Subject: 7013347: allow crypto functions to be called inline to enhance
	performance
In-Reply-To: <4F2769BF.9020109@oracle.com>
References: <F890E827-AD93-4683-88D5-B7ADDBF545F1@oracle.com>
	<4F2769BF.9020109@oracle.com>
Message-ID: <5FA37A5B-C60F-456D-BCC1-44864E9C6B56@oracle.com>


On Jan 30, 2012, at 8:10 PM, Vladimir Kozlov wrote:

> In NativeLookup::lookup_critical_style() jni_name id only used when (dll != NULL). Should the name construction be moved inside this check?

I rearranged it.

address NativeLookup::lookup_critical_style(methodHandle method, char* pure_name, const char* long_name, int args_size, bool os_style) {
  if (!method->has_native_function()) {
    return NULL;
  }

  address current_entry = method->native_function();

  char dll_name[MAXPATHLEN];
  int offset;
  if (os::dll_address_to_library_name(current_entry, dll_name, sizeof(dll_name), &offset)) {
    char ebuf[32];
    void* dll = os::dll_load(dll_name, ebuf, sizeof(ebuf));
    if (dll != NULL) {
      // Compute complete JNI name for style                                                                                                            
      stringStream st;
      if (os_style) os::print_jni_name_prefix_on(&st, args_size);
      st.print_raw(pure_name);
      st.print_raw(long_name);
      if (os_style) os::print_jni_name_suffix_on(&st, args_size);
      char* jni_name = st.as_string();
      return (address)os::dll_lookup(dll, jni_name);
    }
  }

  return NULL;
}


> 
> The same for critical_name in lookup_critical_entry().

I moved it down below the test for object.

> 
> In check_for_lazy_critical_native() should you check that stub_cb != NULL?

I can.

> Also this code has typo (ifdef?):
> + #ifndef ASSERT

Yes.  Thanks.

> 
> 
> Why you don't use new branch instructions in your sparc assembler code?

I forgot it existed.

> Next code loads one byte but you tests all bits:
> 
> +   __ load_bool_contents(sync_state, G3_scratch);
> +   __ cmp(G3_scratch, 0);
> +   __ brx(Assembler::equal, false, Assembler::pt, cont);
> +   __ delayed()->nop();
> 
> you can replace it with:
> 
> +   __ load_bool_contents(sync_state, G3_scratch);
> +   __ cmp_zero_and_br(Assembler::equal, G3_scratch, cont);
> 
> ----
> +   __ ba(done);
> +   __ delayed()->nop();
> 
> with:
> 
> +   __ ba_short(done);

Thanks.

> 
> 
> Don't use next short forward branch in both x86_32 and x86_64 files since the code could be large depending on number of arguments:
> 
> +   __ cmp8(ExternalAddress((address)GC_locker::needs_gc_address()), false);
> +   __ jccb(Assembler::equal, cont);

Code is only emitted for arguments that are in registers so it should be relatively small but maybe it's not small enough.

Thanks.

> 
> 
> Thanks,
> Vladimir
> 
> Tom Rodriguez wrote:
>> http://cr.openjdk.java.net/~never/7013347
>> 1133 lines changed: 979 ins; 56 del; 98 mod; 35796 unchg
>> 7013347: allow crypto functions to be called inline to enhance performance
>> Reviewed-by:
>> This is a long one.
>> The synopsis of this is slightly misleading.  This doens't allow
>> direct calls to native routines from Java but it does attempt to
>> reduce the overhead of using JNI for specific use cases while still
>> maintaining the safety invariants that JNI provdies.  For native code
>> that runs in a bounded time JNI provides a function called
>> GetPrimtiveArrayCritical which may provide direct access to the body
>> of Java arrays of primitive.  In Hotspot this is accomplished by
>> suppressing garbage collection while these pointers are exposed to
>> native code.  This is accomplished with the GC_locker class which is
>> basically a readers/writers lock.  Note that the GC_locker doesn't
>> suppress safepointing, just garbage collections.  There are many
>> operations which require a safepoint to make forward progress, so
>> suppressing them indefinitely isn't acceptable.
>> This RFE provides is a shorthand for the use of
>> GetPrimtiveArrayCritical by defining an alternate native calling
>> convention that only allows the use of primitive or arrays of
>> primtive.  The native method must also be static since non-static
>> methods are passed the receiver as an argument and Java objects aren't
>> allowed.  Synchronization and exceptions aren't allowed either.  The
>> Java code calling these natives is fee to use all of those features so
>> it's not that onerous of a restriction.
>> The benefits of this approach are that JVM can more quickly do the
>> work inline that would normally be done by the
>> GetPrimtiveArrayCritical/ReleasePrimtiveArrayCritical function calls.
>> Calling back into the JVM through JNI requires synchronization with
>> the JVM and each upcall adds a minimum overhead to the native routine.
>> This helps to reduce the overhead to a more fixed cost per call.  It
>> also simplifies the work that the caller must do since synchronization
>> and exceptions aren't allowed.  For now this work is being done in the
>> existing native wrapper generation but with some more simplification
>> this could be more easily inlined directly into the caller.
>> The signature of the native routine follows the same name mangling as
>> normal JNI methods but they start with JavaCritical_ instead of Java_.
>> Any array arguments are unpacked into a pair of arguments, the length
>> followed by a pointer to the body of the array.  If the incoming array
>> is NULL then the body pointer is NULL and the length is 0.
>> Currently this is a JDK private interface while we gain some
>> experience with it but it will likely become a more standard
>> extension.  It's also an optional extension so a native library is
>> required to provide the normal point in addition to the alternate
>> entry point.
>> The changes consist of three parts.  The first is the lookup logic
>> that finds the alternate native entry point.  JNI critical natives
>> currently can only be found through dynamic lookup.  JNI
>> RegisterNatives doesn't know about these functions so there's no way
>> to provide the alternate entry point.
>> The second part is the lazy critical entry logic.  The fix for 7129164
>> introduced code that computed the JNI active count during
>> safepointing.  Now as part of that computation, if a thread is seen to
>> be in thread_in_native state and the nmethod on the top of stack is a
>> critical native wrapper, then the critical count for that thread is
>> incremented and the suspend flags are set so that when the nmethod
>> returns the native code it will call back into the runtime and do the
>> unlock of the critical native.
>> The last part are the native wrappers themselves.  When compiling a
>> critical native wrapper, they emit a new check of GC_locker::_needs_gc
>> and they call into the runtime if it's true.  This keeps them from
>> starting new JNI critical sections if a GC has been requested.  The
>> arguments are unpacked following the alternate calling convention and
>> the method is called as it normally would be.
>> On return the wrapper checks the suspend flags as it normally would
>> and calls back into the runtime where is might have to block and force
>> a GC if it's the last thread exiting the GC_locker.  This required
>> some slightly different handling of the final transition back to
>> thread_in_Java since we have to allow blocking.
>> The wrappers are only generated differently if they are compiling a
>> critical native so it shouldn't have much effect on normal execution.
>> The only library currently taking advantage of this is the new ucrypto
>> provider on Solaris.  For some crypto operations it improves
>> throughput by 20% or more because the crypto routines are fast enough
>> that the JNi overhead is significant.  It's expected that other parts
>> of the JDK will take advantage of it going forward and hopefully it
>> can be tightened up further.
>> Tested with new crypto provider and microbenchmark test case.  Also
>> ran runthese.


From tom.rodriguez at oracle.com  Mon Jan 30 23:50:23 2012
From: tom.rodriguez at oracle.com (Tom Rodriguez)
Date: Mon, 30 Jan 2012 23:50:23 -0800
Subject: Request for review 7141200: log some interesting information in ring
	buffers for crashes 
Message-ID: <190D13C4-541E-470A-901F-EACB0D6A148C@oracle.com>

http://cr.openjdk.java.net/~never/7141200
664 lines changed: 389 ins; 185 del; 90 mod; 37188 unchg

7141200: log some interesting information in ring buffers for crashes
Reviewed-by:

As part of convergence with JRockit it was requested that Hotspot
capture some internal events in ring buffer so that they could be
dumped into the error log to provide more context at crash time.  I
chose to make these logs be mostly text based for simplicity of
implementation.  Eventually more of the flight recorder infrastructure
will be in place and the recording of this kind of data should be more
integrated the infrastructure for that.  So my current implementation
is somewhat ad hoc since I expect it will be revisited.

This also doesn't include one of the more important features, dumping
crashing context when scanning the heap.  Capturing that information
without impacting GC performance is difficult and would require broad
changes in the heap iteration code.  That can possibly be address later.

I've chosen to tear out the existing Events class which was a debug
only feature of limited use and extend that into a collection of
product ring buffers.  They are registered during JVM startup and
dumped into the hs_err log when the JVM crashes.  There are a couple
different categories maintained: compilation, deopt, internal
exceptions, other uncategorized messages and GC events.  Each ring
buffer has it's own mutex to reduce contention across different kinds
of events and generally logging is performed for relatively infrequent
or expensive events to minimize any performance penalty.  I also chose
to maintain separate logs for the categories so that higher frequency
events one kind don't hide other kinds.

I eliminated some of the less useful Events::log calls as well and
move some others around.

The GC logging keeps track of GC history using GCStatInfos.  This has
the benefit that it's GC independent and provides before and after
information.  It could also or instead capture Universe::print_on
though that's pretty verbose output.  The GC section is fairly long
and I'm not sure what the best output would be.  Opinions would be
appreciated.

There are some EventMarks in GC code that I'm tempted to delete.  Any
objections?

I put a couple example logs at:

http://cr.openjdk.java.net/~never/7141200/example1.log
http://cr.openjdk.java.net/~never/7141200/example2.log

I can produce others if needed.

Performance seems to be largely unchanged according to refworkload but
I'm still running some benchmarks.  Tested with JCK vm and lang, and
the nsk tests.



From bengt.rutisson at oracle.com  Tue Jan 31 00:12:30 2012
From: bengt.rutisson at oracle.com (Bengt Rutisson)
Date: Tue, 31 Jan 2012 09:12:30 +0100
Subject: review for 7129164: JNI Get/ReleasePrimitiveArrayCritical doesn't
	scale
In-Reply-To: <359A5416-4618-44C9-8588-1C2BEC55A6CC@oracle.com>
References: <4DDD98EA-07C9-44D9-AF7E-C938E086C6FC@oracle.com>	<4F22FA22.3000106@oracle.com>
	<4F25DC17.60309@oracle.com>	<C592823E-DB9A-47B1-8E4E-26E11AAA36E4@oracle.com>
	<4F25FFC1.5020904@oracle.com> <4F266D93.9080308@oracle.com>
	<76326728-2D3E-4690-8F32-F6C79189B6CC@oracle.com>
	<359A5416-4618-44C9-8588-1C2BEC55A6CC@oracle.com>
Message-ID: <4F27A26E.5020803@oracle.com>


Tom,

On 2012-01-31 02:42, Tom Rodriguez wrote:
> On Jan 30, 2012, at 8:14 AM, Tom Rodriguez wrote:
>
>> On Jan 30, 2012, at 2:14 AM, Bengt Rutisson wrote:
>>
>>> Hi Tom,
>>>
>>> Looked at your latest webrev. Looks good to me.
>>>
>>> Some minor comments:
>>>
>>>
>>> gcLocker.cpp
>>>
>>> You still use tty->timestamp().milliseconds() instead of javaTimeNanos(). Not sure whether you just didn't get to fixing it yet or if you decided against it.
>> I decided against it.  It's a printing timestamp so matching the time stamp in tty seemed to make more sense to me.
> I'm about to send out a webrev for some other changes and I'm going to correct this to use javaTimeNanos to be consistent with the other GC timestamps.  I'm also going to remove the extra braces in safepoint.cpp.

Had a quick look at the changes you sent out for 7013347. I see that you 
have included these fixes there. Thanks!

Bengt

> tom
>
>>>
>>>
>>> gcLocker.hpp
>>>
>>> 84   static void verify_critical_count() NOT_DEBUG_RETURN;
>>>
>>> Not sure what the correct thing is here, but the GC code seems to use PRODUCT_RETURN more than NOT_DEBUG_RETURN. I guess the only difference is for optimized builds (and of course using #ifndef PRODUCT instead of #ifdef ASSERT).
>> I didn't want it doing any work in optimized builds so that's why it's NOT_DEBUG_RETURN.
>>
>>>
>>>
>>> safepoint.cpp
>>>
>>> Not strictly related to your change, but the SafepointSynchronize::begin() method is missing one indentation level due to the the scoping that starts on line 139:
>> Yes, I noticed that too.  The whole thing could be factored better too.  I already pushed so I can't fix it.  I've got another change coming so maybe I'll fix it in there.  Thanks.
>>
>> tom
>>
>>> 139   {
>>> 140   MutexLocker mu(Safepoint_lock);
>>> 141
>>> 142   // Reset the count of active JNI critical threads
>>> 143   _current_jni_active_count = 0;
>>>
>>> This scope ends at the end of the method:
>>>
>>> 398   if (PrintSafepointStatistics) {
>>> 399     // Record how much time spend on the above cleanup tasks
>>> 400     update_statistics_on_cleanup_end(os::javaTimeNanos());
>>> 401   }
>>> 402   }
>>> 403 }
>>>
>>> It seems to me that the scope is not really necessary. In that case, can we remove it? It would be nice to have the method correctly indented. If we decide to keep the scope I'd prefer a comment on line 402, to indicate that it is the end of the MutexLocker scope.
>>>
>>> Bengt
>>>
>>> On 2012-01-30 03:26, David Holmes wrote:
>>>> Hi Tom,
>>>>
>>>> On 30/01/2012 10:30 AM, Tom Rodriguez wrote:
>>>>> On Jan 29, 2012, at 3:53 PM, David Holmes wrote:
>>>>>
>>>>>> On 28/01/2012 5:25 AM, Vladimir Kozlov wrote:
>>>>>>> I change alias to hotspot-dev since it affects everyone.
>>>>>> Thanks Vladimir!
>>>>>>
>>>>>> Overall I find it difficult to follow the protocol that applies here - in particular how safepoints can and can not appear between different methods and the way in which needs_gc() might change value between the enter and exit of a critical region. Without that understanding it is hard to evaluate the correctness of the new approach.
>>>>> I tried to put in enough asserts to ensure that it was clear what invariants are in force when updating the state.  Safepoints can appear wherever they normally can appear, while locking and waiting on mutexes.  _needs_gc can only change from false to true during a safepoint.  For the most part everything operates exactly as it used to once _needs_gc is set and the only trick is computing the correct value of _jni_active_count when setting _needs_gc to be true.  The debug code that still performs the atomics attempts to verify that the computed value and actual value are in sync.  Is there any more that I can explain about how it operates?
>>>> It's my ignorance of the overall operation that makes evaluating this change difficult for me. But thanks to your side-band emails I now have a much clear understanding of things - thanks. And the protocol seems safe.
>>>>
>>>>>> src/share/vm/memory/gcLocker.cpp
>>>>>>
>>>>>> 72       _wait_begin = os::javaTimeMillis();
>>>>>> 135      os::javaTimeMillis() - _wait_begin,
>>>>>>
>>>>>> Are you sure you want to use the non-monotonic javaTimeMillis here? Use of this was recently changed elsewhere in the GC code.
>>>>> It's only printing code, but I'll switch it to use tty->timestamp().milliseconds() which will make it match our internal time stamps, which seems like a useful property.  Is that ok?
>>>> The recent GC changes switched to using javaTimeNanos() converted to millis - see 7117303 and 7129514. The TimeStamp is based on elapsedCounter which is still a time-of-day source on some platforms and so not monotonic.
>>>>
>>>> Thanks,
>>>> David
>>>>
>>>>> I've updated the webrev.
>>>>>
>>>>> tom
>>>>>
>>>>>>
>>>>>> Cheers,
>>>>>> David
>>>>>> -----
>>>>>>
>>>>>>
>>>>>>
>>>>>>> In GC_locker::check_active_before_gc() move wait_begin initialization
>>>>>>> under Print* check since it used only under such checks. Could method
>>>>>>> check_active_before_gc() be called from different threads concurrently?
>>>>>>> Does it need lock? Note that other methods have lock.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Vladimir
>>>>>>>
>>>>>>> Tom Rodriguez wrote:
>>>>>>>> http://cr.openjdk.java.net/~never/7129164
>>>>>>>> 200 lines changed: 126 ins; 33 del; 41 mod; 3958 unchg
>>>>>>>>
>>>>>>>> 7129164: JNI Get/ReleasePrimitiveArrayCritical doesn't scale
>>>>>>>> Summary:
>>>>>>>> Reviewed-by:
>>>>>>>>
>>>>>>>> The machinery for GC_locker which supports GetPrimitiveArrayCritical
>>>>>>>> maintains a count of the number of threads that currently have raw
>>>>>>>> pointers exposed. This is used to allow existing critical sections to
>>>>>>>> drain before creating new ones so that a GC can occur. Currently
>>>>>>>> these counts are updated using atomic all the time, even if a GC isn't
>>>>>>>> being requested. This creates scalability problem when a lot of
>>>>>>>> threads are hammering atomic operations on the jni_lock_count. The
>>>>>>>> count only needs to be valid when checking if a critical section is
>>>>>>>> currently active and when draining the existing sections. The fix is
>>>>>>>> to make the count be computed as part of the safepointing machinery
>>>>>>>> and to only decrement the counters when _needs_gc is true. In debug
>>>>>>>> mode the old count is maintained and validated against the lazily
>>>>>>>> computed count.
>>>>>>>>
>>>>>>>> On a microbenchmark that stresses GetPrimitiveArrayCritical with many
>>>>>>>> threads and relatively short critical section it can more than double
>>>>>>>> the throughput. This also slightly speeds up the normal
>>>>>>>> GetPrimtiveArrayCritical calls. Mostly it's not easily measurable
>>>>>>>> with larger scale programs.
>>>>>>>>
>>>>>>>> Tested with microbenchmark that stresses GetPrimtiveArrayCritical and
>>>>>>>> the crypto benchmarks of specjvm2008 on x86 and sparc. I also ran the
>>>>>>>> java.io regression tests from the JDK.
>>>>>>>>


From christian.thalinger at oracle.com  Tue Jan 31 03:01:44 2012
From: christian.thalinger at oracle.com (Christian Thalinger)
Date: Tue, 31 Jan 2012 12:01:44 +0100
Subject: Request for review 7141200: log some interesting information in
	ring buffers for crashes
In-Reply-To: <190D13C4-541E-470A-901F-EACB0D6A148C@oracle.com>
References: <190D13C4-541E-470A-901F-EACB0D6A148C@oracle.com>
Message-ID: <89EE5D00-CC05-4093-B149-A5173C281BDB@oracle.com>


On Jan 31, 2012, at 8:50 AM, Tom Rodriguez wrote:

> http://cr.openjdk.java.net/~never/7141200
> 664 lines changed: 389 ins; 185 del; 90 mod; 37188 unchg
> 
> 7141200: log some interesting information in ring buffers for crashes
> Reviewed-by:

This looks good.  Since you removed the debug methods to print the events I guess now the way to print them is to call Events::print_all.  Could we have a default outputStream for easier calling in the debugger:

+   static void print_all(outputStream* out = tty);

(Although in my experience this doesn't work very good in GDB.)

> 
> As part of convergence with JRockit it was requested that Hotspot
> capture some internal events in ring buffer so that they could be
> dumped into the error log to provide more context at crash time.  I
> chose to make these logs be mostly text based for simplicity of
> implementation.  Eventually more of the flight recorder infrastructure
> will be in place and the recording of this kind of data should be more
> integrated the infrastructure for that.  So my current implementation
> is somewhat ad hoc since I expect it will be revisited.
> 
> This also doesn't include one of the more important features, dumping
> crashing context when scanning the heap.  Capturing that information
> without impacting GC performance is difficult and would require broad
> changes in the heap iteration code.  That can possibly be address later.
> 
> I've chosen to tear out the existing Events class which was a debug
> only feature of limited use and extend that into a collection of
> product ring buffers.  They are registered during JVM startup and
> dumped into the hs_err log when the JVM crashes.  There are a couple
> different categories maintained: compilation, deopt, internal
> exceptions, other uncategorized messages and GC events.  Each ring
> buffer has it's own mutex to reduce contention across different kinds
> of events and generally logging is performed for relatively infrequent
> or expensive events to minimize any performance penalty.  I also chose
> to maintain separate logs for the categories so that higher frequency
> events one kind don't hide other kinds.
> 
> I eliminated some of the less useful Events::log calls as well and
> move some others around.
> 
> The GC logging keeps track of GC history using GCStatInfos.  This has
> the benefit that it's GC independent and provides before and after
> information.  It could also or instead capture Universe::print_on
> though that's pretty verbose output.  The GC section is fairly long
> and I'm not sure what the best output would be.  Opinions would be
> appreciated.
> 
> There are some EventMarks in GC code that I'm tempted to delete.  Any
> objections?
> 
> I put a couple example logs at:
> 
> http://cr.openjdk.java.net/~never/7141200/example1.log
> http://cr.openjdk.java.net/~never/7141200/example2.log

Unfortunately these are not there.

-- Chris

> 
> I can produce others if needed.
> 
> Performance seems to be largely unchanged according to refworkload but
> I'm still running some benchmarks.  Tested with JCK vm and lang, and
> the nsk tests.
> 
> 


From christian.thalinger at oracle.com  Tue Jan 31 03:31:07 2012
From: christian.thalinger at oracle.com (Christian Thalinger)
Date: Tue, 31 Jan 2012 12:31:07 +0100
Subject: Request for review 7141200: log some interesting information in
	ring buffers for crashes
In-Reply-To: <89EE5D00-CC05-4093-B149-A5173C281BDB@oracle.com>
References: <190D13C4-541E-470A-901F-EACB0D6A148C@oracle.com>
	<89EE5D00-CC05-4093-B149-A5173C281BDB@oracle.com>
Message-ID: <7CA349E0-312D-4C15-8516-C29FD2DDF5CB@oracle.com>


On Jan 31, 2012, at 12:01 PM, Christian Thalinger wrote:

> 
> On Jan 31, 2012, at 8:50 AM, Tom Rodriguez wrote:
> 
>> http://cr.openjdk.java.net/~never/7141200
>> 664 lines changed: 389 ins; 185 del; 90 mod; 37188 unchg
>> 
>> 7141200: log some interesting information in ring buffers for crashes
>> Reviewed-by:
> 
> This looks good.  Since you removed the debug methods to print the events I guess now the way to print them is to call Events::print_all.  Could we have a default outputStream for easier calling in the debugger:
> 
> +   static void print_all(outputStream* out = tty);

Or maybe even just name it Events::print.

-- Chris

> 
> (Although in my experience this doesn't work very good in GDB.)
> 
>> 
>> As part of convergence with JRockit it was requested that Hotspot
>> capture some internal events in ring buffer so that they could be
>> dumped into the error log to provide more context at crash time.  I
>> chose to make these logs be mostly text based for simplicity of
>> implementation.  Eventually more of the flight recorder infrastructure
>> will be in place and the recording of this kind of data should be more
>> integrated the infrastructure for that.  So my current implementation
>> is somewhat ad hoc since I expect it will be revisited.
>> 
>> This also doesn't include one of the more important features, dumping
>> crashing context when scanning the heap.  Capturing that information
>> without impacting GC performance is difficult and would require broad
>> changes in the heap iteration code.  That can possibly be address later.
>> 
>> I've chosen to tear out the existing Events class which was a debug
>> only feature of limited use and extend that into a collection of
>> product ring buffers.  They are registered during JVM startup and
>> dumped into the hs_err log when the JVM crashes.  There are a couple
>> different categories maintained: compilation, deopt, internal
>> exceptions, other uncategorized messages and GC events.  Each ring
>> buffer has it's own mutex to reduce contention across different kinds
>> of events and generally logging is performed for relatively infrequent
>> or expensive events to minimize any performance penalty.  I also chose
>> to maintain separate logs for the categories so that higher frequency
>> events one kind don't hide other kinds.
>> 
>> I eliminated some of the less useful Events::log calls as well and
>> move some others around.
>> 
>> The GC logging keeps track of GC history using GCStatInfos.  This has
>> the benefit that it's GC independent and provides before and after
>> information.  It could also or instead capture Universe::print_on
>> though that's pretty verbose output.  The GC section is fairly long
>> and I'm not sure what the best output would be.  Opinions would be
>> appreciated.
>> 
>> There are some EventMarks in GC code that I'm tempted to delete.  Any
>> objections?
>> 
>> I put a couple example logs at:
>> 
>> http://cr.openjdk.java.net/~never/7141200/example1.log
>> http://cr.openjdk.java.net/~never/7141200/example2.log
> 
> Unfortunately these are not there.
> 
> -- Chris
> 
>> 
>> I can produce others if needed.
>> 
>> Performance seems to be largely unchanged according to refworkload but
>> I'm still running some benchmarks.  Tested with JCK vm and lang, and
>> the nsk tests.
>> 
>> 
> 


From paul.hohensee at oracle.com  Tue Jan 31 07:27:01 2012
From: paul.hohensee at oracle.com (Paul Hohensee)
Date: Tue, 31 Jan 2012 10:27:01 -0500
Subject: CFV: New HSX Committer
Message-ID: <4F280845.2030400@oracle.com>

I'd like to nominate Staffan Larsen to be an OpenJDK HSX project 
Committer [0].

Staffan is a Committer on the JDK 6, 7, 7u and 8 projects, has pushed seven
changes to the Hotspot forest and is the Oracle Serviceability group 
lead (the
Serviceability group's responsibility includes functionality in both the JDK
libraries and Hotspot).  Staffan is thus well-qualified to be a Committer on
the HSX project.

Votes are due by 12:01AM U.S.A. Eastern time, February 8th, 2012.

Only current HSX Committers [1] are eligible to vote on this nomination.

For Lazy Consensus voting instructions, see [2].

Thanks,

Paul

[0] http://openjdk.java.net/bylaws#committer
[1] http://openjdk.java.net/census/#hsx
[2] http://openjdk.java.net/projects/#committer-vote

From daniel.daugherty at oracle.com  Tue Jan 31 07:33:46 2012
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Tue, 31 Jan 2012 08:33:46 -0700
Subject: CFV: New HSX Committer
In-Reply-To: <4F280845.2030400@oracle.com>
References: <4F280845.2030400@oracle.com>
Message-ID: <4F2809DA.1090500@oracle.com>

Vote: yes

Dan


On 1/31/12 8:27 AM, Paul Hohensee wrote:
> I'd like to nominate Staffan Larsen to be an OpenJDK HSX project 
> Committer [0].
>
> Staffan is a Committer on the JDK 6, 7, 7u and 8 projects, has pushed 
> seven
> changes to the Hotspot forest and is the Oracle Serviceability group 
> lead (the
> Serviceability group's responsibility includes functionality in both 
> the JDK
> libraries and Hotspot).  Staffan is thus well-qualified to be a 
> Committer on
> the HSX project.
>
> Votes are due by 12:01AM U.S.A. Eastern time, February 8th, 2012.
>
> Only current HSX Committers [1] are eligible to vote on this nomination.
>
> For Lazy Consensus voting instructions, see [2].
>
> Thanks,
>
> Paul
>
> [0] http://openjdk.java.net/bylaws#committer
> [1] http://openjdk.java.net/census/#hsx
> [2] http://openjdk.java.net/projects/#committer-vote

From vladimir.kozlov at oracle.com  Tue Jan 31 07:38:02 2012
From: vladimir.kozlov at oracle.com (Vladimir Kozlov)
Date: Tue, 31 Jan 2012 07:38:02 -0800
Subject: CFV: New HSX Committer
In-Reply-To: <4F280845.2030400@oracle.com>
References: <4F280845.2030400@oracle.com>
Message-ID: <4F280ADA.8000001@oracle.com>

Vote: yes

Paul Hohensee wrote:
> I'd like to nominate Staffan Larsen to be an OpenJDK HSX project 
> Committer [0].
> 
> Staffan is a Committer on the JDK 6, 7, 7u and 8 projects, has pushed seven
> changes to the Hotspot forest and is the Oracle Serviceability group 
> lead (the
> Serviceability group's responsibility includes functionality in both the 
> JDK
> libraries and Hotspot).  Staffan is thus well-qualified to be a 
> Committer on
> the HSX project.
> 
> Votes are due by 12:01AM U.S.A. Eastern time, February 8th, 2012.
> 
> Only current HSX Committers [1] are eligible to vote on this nomination.
> 
> For Lazy Consensus voting instructions, see [2].
> 
> Thanks,
> 
> Paul
> 
> [0] http://openjdk.java.net/bylaws#committer
> [1] http://openjdk.java.net/census/#hsx
> [2] http://openjdk.java.net/projects/#committer-vote

From jesper.wilhelmsson at oracle.com  Tue Jan 31 07:39:17 2012
From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson)
Date: Tue, 31 Jan 2012 16:39:17 +0100
Subject: CFV: New HSX Committer
In-Reply-To: <4F280845.2030400@oracle.com>
References: <4F280845.2030400@oracle.com>
Message-ID: <4F280B25.9000508@oracle.com>

vote: yes

/Jesper

On 01/31/2012 04:27 PM, Paul Hohensee wrote:
> I'd like to nominate Staffan Larsen to be an OpenJDK HSX project Committer [0].
>
> Staffan is a Committer on the JDK 6, 7, 7u and 8 projects, has pushed seven
> changes to the Hotspot forest and is the Oracle Serviceability group lead (the
> Serviceability group's responsibility includes functionality in both the JDK
> libraries and Hotspot). Staffan is thus well-qualified to be a Committer on
> the HSX project.
>
> Votes are due by 12:01AM U.S.A. Eastern time, February 8th, 2012.
>
> Only current HSX Committers [1] are eligible to vote on this nomination.
>
> For Lazy Consensus voting instructions, see [2].
>
> Thanks,
>
> Paul
>
> [0] http://openjdk.java.net/bylaws#committer
> [1] http://openjdk.java.net/census/#hsx
> [2] http://openjdk.java.net/projects/#committer-vote

From Dmitry.Samersoff at oracle.com  Tue Jan 31 07:41:52 2012
From: Dmitry.Samersoff at oracle.com (Dmitry Samersoff)
Date: Tue, 31 Jan 2012 19:41:52 +0400
Subject: CFV: New HSX Committer
In-Reply-To: <4F280845.2030400@oracle.com>
References: <4F280845.2030400@oracle.com>
Message-ID: <4F280BC0.7050901@oracle.com>

Vote: yes


On 2012-01-31 19:27, Paul Hohensee wrote:
> I'd like to nominate Staffan Larsen to be an OpenJDK HSX project
> Committer [0].
> 
> Staffan is a Committer on the JDK 6, 7, 7u and 8 projects, has pushed seven
> changes to the Hotspot forest and is the Oracle Serviceability group
> lead (the
> Serviceability group's responsibility includes functionality in both the
> JDK
> libraries and Hotspot).  Staffan is thus well-qualified to be a
> Committer on
> the HSX project.
> 
> Votes are due by 12:01AM U.S.A. Eastern time, February 8th, 2012.
> 
> Only current HSX Committers [1] are eligible to vote on this nomination.
> 
> For Lazy Consensus voting instructions, see [2].
> 
> Thanks,
> 
> Paul
> 
> [0] http://openjdk.java.net/bylaws#committer
> [1] http://openjdk.java.net/census/#hsx
> [2] http://openjdk.java.net/projects/#committer-vote


-- 
Dmitry Samersoff
Java Hotspot development team, SPB04
* There will come soft rains ...

From vladimir.kozlov at oracle.com  Tue Jan 31 07:46:22 2012
From: vladimir.kozlov at oracle.com (Vladimir Kozlov)
Date: Tue, 31 Jan 2012 07:46:22 -0800
Subject: Request for review 7141200: log some interesting information
	in ring buffers for crashes
In-Reply-To: <190D13C4-541E-470A-901F-EACB0D6A148C@oracle.com>
References: <190D13C4-541E-470A-901F-EACB0D6A148C@oracle.com>
Message-ID: <4F280CCE.90009@oracle.com>

In next class definition "T data;" should be "X data;". Right?

+   template <class X> class EventRecord {
+    public:
+     jlong   timestamp;
+     Thread* thread;
+     T       data;
+   };

In general changes look good.

Thanks,
Vladimir

Tom Rodriguez wrote:
> http://cr.openjdk.java.net/~never/7141200
> 664 lines changed: 389 ins; 185 del; 90 mod; 37188 unchg
> 
> 7141200: log some interesting information in ring buffers for crashes
> Reviewed-by:
> 
> As part of convergence with JRockit it was requested that Hotspot
> capture some internal events in ring buffer so that they could be
> dumped into the error log to provide more context at crash time.  I
> chose to make these logs be mostly text based for simplicity of
> implementation.  Eventually more of the flight recorder infrastructure
> will be in place and the recording of this kind of data should be more
> integrated the infrastructure for that.  So my current implementation
> is somewhat ad hoc since I expect it will be revisited.
> 
> This also doesn't include one of the more important features, dumping
> crashing context when scanning the heap.  Capturing that information
> without impacting GC performance is difficult and would require broad
> changes in the heap iteration code.  That can possibly be address later.
> 
> I've chosen to tear out the existing Events class which was a debug
> only feature of limited use and extend that into a collection of
> product ring buffers.  They are registered during JVM startup and
> dumped into the hs_err log when the JVM crashes.  There are a couple
> different categories maintained: compilation, deopt, internal
> exceptions, other uncategorized messages and GC events.  Each ring
> buffer has it's own mutex to reduce contention across different kinds
> of events and generally logging is performed for relatively infrequent
> or expensive events to minimize any performance penalty.  I also chose
> to maintain separate logs for the categories so that higher frequency
> events one kind don't hide other kinds.
> 
> I eliminated some of the less useful Events::log calls as well and
> move some others around.
> 
> The GC logging keeps track of GC history using GCStatInfos.  This has
> the benefit that it's GC independent and provides before and after
> information.  It could also or instead capture Universe::print_on
> though that's pretty verbose output.  The GC section is fairly long
> and I'm not sure what the best output would be.  Opinions would be
> appreciated.
> 
> There are some EventMarks in GC code that I'm tempted to delete.  Any
> objections?
> 
> I put a couple example logs at:
> 
> http://cr.openjdk.java.net/~never/7141200/example1.log
> http://cr.openjdk.java.net/~never/7141200/example2.log
> 
> I can produce others if needed.
> 
> Performance seems to be largely unchanged according to refworkload but
> I'm still running some benchmarks.  Tested with JCK vm and lang, and
> the nsk tests.
> 
> 

From tony.printezis at oracle.com  Tue Jan 31 07:52:46 2012
From: tony.printezis at oracle.com (Tony Printezis)
Date: Tue, 31 Jan 2012 10:52:46 -0500
Subject: CFV: New HSX Committer
In-Reply-To: <4F280845.2030400@oracle.com>
References: <4F280845.2030400@oracle.com>
Message-ID: <4F280E4E.8060401@oracle.com>

Vote: yes

On 1/31/2012 10:27 AM, Paul Hohensee wrote:
> I'd like to nominate Staffan Larsen to be an OpenJDK HSX project 
> Committer [0].
>
> Staffan is a Committer on the JDK 6, 7, 7u and 8 projects, has pushed 
> seven
> changes to the Hotspot forest and is the Oracle Serviceability group 
> lead (the
> Serviceability group's responsibility includes functionality in both 
> the JDK
> libraries and Hotspot).  Staffan is thus well-qualified to be a 
> Committer on
> the HSX project.
>
> Votes are due by 12:01AM U.S.A. Eastern time, February 8th, 2012.
>
> Only current HSX Committers [1] are eligible to vote on this nomination.
>
> For Lazy Consensus voting instructions, see [2].
>
> Thanks,
>
> Paul
>
> [0] http://openjdk.java.net/bylaws#committer
> [1] http://openjdk.java.net/census/#hsx
> [2] http://openjdk.java.net/projects/#committer-vote

From christian.tornqvist at oracle.com  Tue Jan 31 07:59:14 2012
From: christian.tornqvist at oracle.com (=?iso-8859-1?B?Q2hyaXN0aWFuIFT2cm5xdmlzdA==?=)
Date: Tue, 31 Jan 2012 07:59:14 -0800 (PST)
Subject: CFV: New HSX Committer
In-Reply-To: <4F280845.2030400@oracle.com>
References: <4F280845.2030400@oracle.com>
Message-ID: <9d9fc038-096c-4afc-82d4-9add0345e11f@default>

Vote: yes

-----Original Message-----
From: Paul Hohensee 
Sent: den 31 januari 2012 16:27
To: HotSpot Open Source Developers
Subject: CFV: New HSX Committer

I'd like to nominate Staffan Larsen to be an OpenJDK HSX project 
Committer [0].

Staffan is a Committer on the JDK 6, 7, 7u and 8 projects, has pushed seven
changes to the Hotspot forest and is the Oracle Serviceability group 
lead (the
Serviceability group's responsibility includes functionality in both the JDK
libraries and Hotspot).  Staffan is thus well-qualified to be a Committer on
the HSX project.

Votes are due by 12:01AM U.S.A. Eastern time, February 8th, 2012.

Only current HSX Committers [1] are eligible to vote on this nomination.

For Lazy Consensus voting instructions, see [2].

Thanks,

Paul

[0] http://openjdk.java.net/bylaws#committer
[1] http://openjdk.java.net/census/#hsx
[2] http://openjdk.java.net/projects/#committer-vote

From coleen.phillimore at oracle.com  Tue Jan 31 08:46:32 2012
From: coleen.phillimore at oracle.com (Coleen Phillimore)
Date: Tue, 31 Jan 2012 11:46:32 -0500
Subject: CFV: New HSX Committer
In-Reply-To: <4F280845.2030400@oracle.com>
References: <4F280845.2030400@oracle.com>
Message-ID: <4F281AE8.3090101@oracle.com>

Vote: yes

On 1/31/2012 10:27 AM, Paul Hohensee wrote:
> I'd like to nominate Staffan Larsen to be an OpenJDK HSX project 
> Committer [0].
>
> Staffan is a Committer on the JDK 6, 7, 7u and 8 projects, has pushed 
> seven
> changes to the Hotspot forest and is the Oracle Serviceability group 
> lead (the
> Serviceability group's responsibility includes functionality in both 
> the JDK
> libraries and Hotspot).  Staffan is thus well-qualified to be a 
> Committer on
> the HSX project.
>
> Votes are due by 12:01AM U.S.A. Eastern time, February 8th, 2012.
>
> Only current HSX Committers [1] are eligible to vote on this nomination.
>
> For Lazy Consensus voting instructions, see [2].
>
> Thanks,
>
> Paul
>
> [0] http://openjdk.java.net/bylaws#committer
> [1] http://openjdk.java.net/census/#hsx
> [2] http://openjdk.java.net/projects/#committer-vote

From tom.rodriguez at oracle.com  Tue Jan 31 09:00:23 2012
From: tom.rodriguez at oracle.com (Tom Rodriguez)
Date: Tue, 31 Jan 2012 09:00:23 -0800
Subject: CFV: New HSX Committer
In-Reply-To: <4F280845.2030400@oracle.com>
References: <4F280845.2030400@oracle.com>
Message-ID: <3EB10967-0002-4192-B4DC-4129D827A2B3@oracle.com>

Vote: yes

On Jan 31, 2012, at 7:27 AM, Paul Hohensee wrote:

> I'd like to nominate Staffan Larsen to be an OpenJDK HSX project Committer [0].
> 
> Staffan is a Committer on the JDK 6, 7, 7u and 8 projects, has pushed seven
> changes to the Hotspot forest and is the Oracle Serviceability group lead (the
> Serviceability group's responsibility includes functionality in both the JDK
> libraries and Hotspot).  Staffan is thus well-qualified to be a Committer on
> the HSX project.
> 
> Votes are due by 12:01AM U.S.A. Eastern time, February 8th, 2012.
> 
> Only current HSX Committers [1] are eligible to vote on this nomination.
> 
> For Lazy Consensus voting instructions, see [2].
> 
> Thanks,
> 
> Paul
> 
> [0] http://openjdk.java.net/bylaws#committer
> [1] http://openjdk.java.net/census/#hsx
> [2] http://openjdk.java.net/projects/#committer-vote


From keith.mcguigan at oracle.com  Tue Jan 31 09:05:36 2012
From: keith.mcguigan at oracle.com (keith mcguigan)
Date: Tue, 31 Jan 2012 12:05:36 -0500
Subject: CFV: New HSX Committer
In-Reply-To: <4F280845.2030400@oracle.com>
References: <4F280845.2030400@oracle.com>
Message-ID: <4F281F60.9020707@oracle.com>

Vote: yes

Paul Hohensee wrote:
> I'd like to nominate Staffan Larsen to be an OpenJDK HSX project 
> Committer [0].
> 
> Staffan is a Committer on the JDK 6, 7, 7u and 8 projects, has pushed seven
> changes to the Hotspot forest and is the Oracle Serviceability group 
> lead (the
> Serviceability group's responsibility includes functionality in both the 
> JDK
> libraries and Hotspot).  Staffan is thus well-qualified to be a 
> Committer on
> the HSX project.
> 
> Votes are due by 12:01AM U.S.A. Eastern time, February 8th, 2012.
> 
> Only current HSX Committers [1] are eligible to vote on this nomination.
> 
> For Lazy Consensus voting instructions, see [2].
> 
> Thanks,
> 
> Paul
> 
> [0] http://openjdk.java.net/bylaws#committer
> [1] http://openjdk.java.net/census/#hsx
> [2] http://openjdk.java.net/projects/#committer-vote

From tom.rodriguez at oracle.com  Tue Jan 31 09:05:00 2012
From: tom.rodriguez at oracle.com (Tom Rodriguez)
Date: Tue, 31 Jan 2012 09:05:00 -0800
Subject: Request for review 7141200: log some interesting information in
	ring buffers for crashes
In-Reply-To: <89EE5D00-CC05-4093-B149-A5173C281BDB@oracle.com>
References: <190D13C4-541E-470A-901F-EACB0D6A148C@oracle.com>
	<89EE5D00-CC05-4093-B149-A5173C281BDB@oracle.com>
Message-ID: <3554A634-9664-4DFA-BBC4-537008DC316B@oracle.com>


On Jan 31, 2012, at 3:01 AM, Christian Thalinger wrote:

> 
> On Jan 31, 2012, at 8:50 AM, Tom Rodriguez wrote:
> 
>> http://cr.openjdk.java.net/~never/7141200
>> 664 lines changed: 389 ins; 185 del; 90 mod; 37188 unchg
>> 
>> 7141200: log some interesting information in ring buffers for crashes
>> Reviewed-by:
> 
> This looks good.  Since you removed the debug methods to print the events I guess now the way to print them is to call Events::print_all.  Could we have a default outputStream for easier calling in the debugger:
> 
> +   static void print_all(outputStream* out = tty);
> 
> (Although in my experience this doesn't work very good in GDB.)

I'll add something like that, probably Events::print.

>> 
>> There are some EventMarks in GC code that I'm tempted to delete.  Any
>> objections?
>> 
>> I put a couple example logs at:
>> 
>> http://cr.openjdk.java.net/~never/7141200/example1.log
>> http://cr.openjdk.java.net/~never/7141200/example2.log
> 
> Unfortunately these are not there.

I think they got deleted by rsync.  They are there now.

tom

> 
> -- Chris
> 
>> 
>> I can produce others if needed.
>> 
>> Performance seems to be largely unchanged according to refworkload but
>> I'm still running some benchmarks.  Tested with JCK vm and lang, and
>> the nsk tests.
>> 
>> 
> 


From tom.rodriguez at oracle.com  Tue Jan 31 09:06:34 2012
From: tom.rodriguez at oracle.com (Tom Rodriguez)
Date: Tue, 31 Jan 2012 09:06:34 -0800
Subject: Request for review 7141200: log some interesting information in
	ring buffers for crashes
In-Reply-To: <4F280CCE.90009@oracle.com>
References: <190D13C4-541E-470A-901F-EACB0D6A148C@oracle.com>
	<4F280CCE.90009@oracle.com>
Message-ID: <26C6884F-7099-4BCC-9444-0D7FDA30A8B5@oracle.com>


On Jan 31, 2012, at 7:46 AM, Vladimir Kozlov wrote:

> In next class definition "T data;" should be "X data;". Right?
> 
> +   template <class X> class EventRecord {
> +    public:
> +     jlong   timestamp;
> +     Thread* thread;
> +     T       data;
> +   };

Yes. I originally wanted to reuse T for EventRecord but gcc didn't like that.  I've fixed that.

> 
> In general changes look good.

Thanks.

tom

> 
> Thanks,
> Vladimir
> 
> Tom Rodriguez wrote:
>> http://cr.openjdk.java.net/~never/7141200
>> 664 lines changed: 389 ins; 185 del; 90 mod; 37188 unchg
>> 7141200: log some interesting information in ring buffers for crashes
>> Reviewed-by:
>> As part of convergence with JRockit it was requested that Hotspot
>> capture some internal events in ring buffer so that they could be
>> dumped into the error log to provide more context at crash time.  I
>> chose to make these logs be mostly text based for simplicity of
>> implementation.  Eventually more of the flight recorder infrastructure
>> will be in place and the recording of this kind of data should be more
>> integrated the infrastructure for that.  So my current implementation
>> is somewhat ad hoc since I expect it will be revisited.
>> This also doesn't include one of the more important features, dumping
>> crashing context when scanning the heap.  Capturing that information
>> without impacting GC performance is difficult and would require broad
>> changes in the heap iteration code.  That can possibly be address later.
>> I've chosen to tear out the existing Events class which was a debug
>> only feature of limited use and extend that into a collection of
>> product ring buffers.  They are registered during JVM startup and
>> dumped into the hs_err log when the JVM crashes.  There are a couple
>> different categories maintained: compilation, deopt, internal
>> exceptions, other uncategorized messages and GC events.  Each ring
>> buffer has it's own mutex to reduce contention across different kinds
>> of events and generally logging is performed for relatively infrequent
>> or expensive events to minimize any performance penalty.  I also chose
>> to maintain separate logs for the categories so that higher frequency
>> events one kind don't hide other kinds.
>> I eliminated some of the less useful Events::log calls as well and
>> move some others around.
>> The GC logging keeps track of GC history using GCStatInfos.  This has
>> the benefit that it's GC independent and provides before and after
>> information.  It could also or instead capture Universe::print_on
>> though that's pretty verbose output.  The GC section is fairly long
>> and I'm not sure what the best output would be.  Opinions would be
>> appreciated.
>> There are some EventMarks in GC code that I'm tempted to delete.  Any
>> objections?
>> I put a couple example logs at:
>> http://cr.openjdk.java.net/~never/7141200/example1.log
>> http://cr.openjdk.java.net/~never/7141200/example2.log
>> I can produce others if needed.
>> Performance seems to be largely unchanged according to refworkload but
>> I'm still running some benchmarks.  Tested with JCK vm and lang, and
>> the nsk tests.


From jon.masamitsu at oracle.com  Tue Jan 31 09:36:06 2012
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Tue, 31 Jan 2012 09:36:06 -0800
Subject: Request for review 7141200: log some interesting information
	in ring buffers for crashes
In-Reply-To: <190D13C4-541E-470A-901F-EACB0D6A148C@oracle.com>
References: <190D13C4-541E-470A-901F-EACB0D6A148C@oracle.com>
Message-ID: <4F282686.7010908@oracle.com>

Tom,

I only looked at the files from

src/share/vm/services/memoryManager.*
src/share/vm/utilities/debug.*
|| src/share/vm/utilities/events.*
src/share/vm/utilities/exceptions.cpp
src/share/vm/utilities/vmError.cpp*

*In events.hpp if the variable _filled maybe would be
better named _entries to indicate the number of entries
written.  I initially thought with _filled you were trying
to record the number of times the buffer wrapped.

Copyrights need updating.

These are supposed to be start and end times?

| Time: 0.000000.3 - 0.000000.3


Jon


On 01/30/12 23:50, Tom Rodriguez wrote:
> http://cr.openjdk.java.net/~never/7141200
> 664 lines changed: 389 ins; 185 del; 90 mod; 37188 unchg
>
> 7141200: log some interesting information in ring buffers for crashes
> Reviewed-by:
>
> As part of convergence with JRockit it was requested that Hotspot
> capture some internal events in ring buffer so that they could be
> dumped into the error log to provide more context at crash time.  I
> chose to make these logs be mostly text based for simplicity of
> implementation.  Eventually more of the flight recorder infrastructure
> will be in place and the recording of this kind of data should be more
> integrated the infrastructure for that.  So my current implementation
> is somewhat ad hoc since I expect it will be revisited.
>
> This also doesn't include one of the more important features, dumping
> crashing context when scanning the heap.  Capturing that information
> without impacting GC performance is difficult and would require broad
> changes in the heap iteration code.  That can possibly be address later.
>
> I've chosen to tear out the existing Events class which was a debug
> only feature of limited use and extend that into a collection of
> product ring buffers.  They are registered during JVM startup and
> dumped into the hs_err log when the JVM crashes.  There are a couple
> different categories maintained: compilation, deopt, internal
> exceptions, other uncategorized messages and GC events.  Each ring
> buffer has it's own mutex to reduce contention across different kinds
> of events and generally logging is performed for relatively infrequent
> or expensive events to minimize any performance penalty.  I also chose
> to maintain separate logs for the categories so that higher frequency
> events one kind don't hide other kinds.
>
> I eliminated some of the less useful Events::log calls as well and
> move some others around.
>
> The GC logging keeps track of GC history using GCStatInfos.  This has
> the benefit that it's GC independent and provides before and after
> information.  It could also or instead capture Universe::print_on
> though that's pretty verbose output.  The GC section is fairly long
> and I'm not sure what the best output would be.  Opinions would be
> appreciated.
>
> There are some EventMarks in GC code that I'm tempted to delete.  Any
> objections?
>
> I put a couple example logs at:
>
> http://cr.openjdk.java.net/~never/7141200/example1.log
> http://cr.openjdk.java.net/~never/7141200/example2.log
>
> I can produce others if needed.
>
> Performance seems to be largely unchanged according to refworkload but
> I'm still running some benchmarks.  Tested with JCK vm and lang, and
> the nsk tests.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120131/819b1c6e/attachment.html 

From christian.thalinger at oracle.com  Tue Jan 31 09:37:36 2012
From: christian.thalinger at oracle.com (Christian Thalinger)
Date: Tue, 31 Jan 2012 18:37:36 +0100
Subject: CFV: New HSX Committer
In-Reply-To: <4F280845.2030400@oracle.com>
References: <4F280845.2030400@oracle.com>
Message-ID: <2F597852-CBFF-40AA-9D3A-51D88E153025@oracle.com>

Vote: yes

On Jan 31, 2012, at 4:27 PM, Paul Hohensee wrote:

> I'd like to nominate Staffan Larsen to be an OpenJDK HSX project Committer [0].
> 
> Staffan is a Committer on the JDK 6, 7, 7u and 8 projects, has pushed seven
> changes to the Hotspot forest and is the Oracle Serviceability group lead (the
> Serviceability group's responsibility includes functionality in both the JDK
> libraries and Hotspot).  Staffan is thus well-qualified to be a Committer on
> the HSX project.
> 
> Votes are due by 12:01AM U.S.A. Eastern time, February 8th, 2012.
> 
> Only current HSX Committers [1] are eligible to vote on this nomination.
> 
> For Lazy Consensus voting instructions, see [2].
> 
> Thanks,
> 
> Paul
> 
> [0] http://openjdk.java.net/bylaws#committer
> [1] http://openjdk.java.net/census/#hsx
> [2] http://openjdk.java.net/projects/#committer-vote


From karen.kinnear at oracle.com  Tue Jan 31 09:44:37 2012
From: karen.kinnear at oracle.com (Karen Kinnear)
Date: Tue, 31 Jan 2012 12:44:37 -0500
Subject: CFV: New HSX Committer
In-Reply-To: <4F280845.2030400@oracle.com>
References: <4F280845.2030400@oracle.com>
Message-ID: <E5D6E233-594A-45FD-A3DD-DD8704FFABCF@oracle.com>

Vote: yes

On Jan 31, 2012, at 10:27 AM, Paul Hohensee wrote:

> I'd like to nominate Staffan Larsen to be an OpenJDK HSX project Committer [0].
> 
> Staffan is a Committer on the JDK 6, 7, 7u and 8 projects, has pushed seven
> changes to the Hotspot forest and is the Oracle Serviceability group lead (the
> Serviceability group's responsibility includes functionality in both the JDK
> libraries and Hotspot).  Staffan is thus well-qualified to be a Committer on
> the HSX project.
> 
> Votes are due by 12:01AM U.S.A. Eastern time, February 8th, 2012.
> 
> Only current HSX Committers [1] are eligible to vote on this nomination.
> 
> For Lazy Consensus voting instructions, see [2].
> 
> Thanks,
> 
> Paul
> 
> [0] http://openjdk.java.net/bylaws#committer
> [1] http://openjdk.java.net/census/#hsx
> [2] http://openjdk.java.net/projects/#committer-vote


From tom.rodriguez at oracle.com  Tue Jan 31 09:51:23 2012
From: tom.rodriguez at oracle.com (Tom Rodriguez)
Date: Tue, 31 Jan 2012 09:51:23 -0800
Subject: Request for review 7141200: log some interesting information in
	ring buffers for crashes
In-Reply-To: <4F282686.7010908@oracle.com>
References: <190D13C4-541E-470A-901F-EACB0D6A148C@oracle.com>
	<4F282686.7010908@oracle.com>
Message-ID: <4AC50618-1521-42C1-A428-63E3E7E46515@oracle.com>


On Jan 31, 2012, at 9:36 AM, Jon Masamitsu wrote:

> Tom,
> 
> I only looked at the files from 
> 
> src/share/vm/services/memoryManager.*
> src/share/vm/utilities/debug.*
> src/share/vm/utilities/events.*
> src/share/vm/utilities/exceptions.cpp
> src/share/vm/utilities/vmError.cpp
> 
> In events.hpp if the variable _filled maybe would be
> better named _entries to indicate the number of entries
> written.  I initially thought with _filled you were trying
> to record the number of times the buffer wrapped.

Yes it's not a good name.  _entries is better.

> 
> Copyrights need updating.

Yes.

> 
> These are supposed to be start and end times?
> | Time: 0.000000.3 - 0.000000.3

Yes.  They come from the Management API which appears to use TimeStamp which bottoms out in os::elapsed_counter.  There are converted so seconds in the printing.  Maybe printing the long would be better?

tom

> 
> Jon
> 
> 
> On 01/30/12 23:50, Tom Rodriguez wrote:
>> http://cr.openjdk.java.net/~never/7141200
>> 
>> 664 lines changed: 389 ins; 185 del; 90 mod; 37188 unchg
>> 
>> 7141200: log some interesting information in ring buffers for crashes
>> Reviewed-by:
>> 
>> As part of convergence with JRockit it was requested that Hotspot
>> capture some internal events in ring buffer so that they could be
>> dumped into the error log to provide more context at crash time.  I
>> chose to make these logs be mostly text based for simplicity of
>> implementation.  Eventually more of the flight recorder infrastructure
>> will be in place and the recording of this kind of data should be more
>> integrated the infrastructure for that.  So my current implementation
>> is somewhat ad hoc since I expect it will be revisited.
>> 
>> This also doesn't include one of the more important features, dumping
>> crashing context when scanning the heap.  Capturing that information
>> without impacting GC performance is difficult and would require broad
>> changes in the heap iteration code.  That can possibly be address later.
>> 
>> I've chosen to tear out the existing Events class which was a debug
>> only feature of limited use and extend that into a collection of
>> product ring buffers.  They are registered during JVM startup and
>> dumped into the hs_err log when the JVM crashes.  There are a couple
>> different categories maintained: compilation, deopt, internal
>> exceptions, other uncategorized messages and GC events.  Each ring
>> buffer has it's own mutex to reduce contention across different kinds
>> of events and generally logging is performed for relatively infrequent
>> or expensive events to minimize any performance penalty.  I also chose
>> to maintain separate logs for the categories so that higher frequency
>> events one kind don't hide other kinds.
>> 
>> I eliminated some of the less useful Events::log calls as well and
>> move some others around.
>> 
>> The GC logging keeps track of GC history using GCStatInfos.  This has
>> the benefit that it's GC independent and provides before and after
>> information.  It could also or instead capture Universe::print_on
>> though that's pretty verbose output.  The GC section is fairly long
>> and I'm not sure what the best output would be.  Opinions would be
>> appreciated.
>> 
>> There are some EventMarks in GC code that I'm tempted to delete.  Any
>> objections?
>> 
>> I put a couple example logs at:
>> 
>> 
>> http://cr.openjdk.java.net/~never/7141200/example1.log
>> http://cr.openjdk.java.net/~never/7141200/example2.log
>> 
>> 
>> I can produce others if needed.
>> 
>> Performance seems to be largely unchanged according to refworkload but
>> I'm still running some benchmarks.  Tested with JCK vm and lang, and
>> the nsk tests.
>> 
>> 
>> 


From kevin.walls at oracle.com  Tue Jan 31 10:08:24 2012
From: kevin.walls at oracle.com (Kevin Walls)
Date: Tue, 31 Jan 2012 18:08:24 +0000
Subject: Request for review 7141200: log some interesting information
	in ring buffers for crashes
In-Reply-To: <190D13C4-541E-470A-901F-EACB0D6A148C@oracle.com>
References: <190D13C4-541E-470A-901F-EACB0D6A148C@oracle.com>
Message-ID: <4F282E18.2060904@oracle.com>


Hi Tom,

Thanks for the examples.

Really like that we have a timestamp so we can see the ordering of 
separate types
of events.  But we will get questions on how to convert a big number like
9617182121 into a time, and how it relates to times we see in other log 
files:
can we convert it, e.g the same format as the GC logs?

(worst case, I was thinking we log the value of first_hrtime so we have 
some baseline,
but I'm sure we can make it prettier than that 8-) )


Why are the GC events all happening in the time period: Time: 0.000000.3 
- 0.000000.3

That must be the format string %f.3 in memoryManager.cpp.  Again maybe we
can make this more readable, this figure will no doubt be used to search 
GC logs.


Trivial typos:
events.hpp line 108, maybe "Should only be called while mutex is held."
events.hpp line 58 subclassed rather than subclasses.


Thanks!
Kevin


On 31/01/12 07:50, Tom Rodriguez wrote:
> http://cr.openjdk.java.net/~never/7141200
> 664 lines changed: 389 ins; 185 del; 90 mod; 37188 unchg
>
> 7141200: log some interesting information in ring buffers for crashes
> Reviewed-by:
>
> As part of convergence with JRockit it was requested that Hotspot
> capture some internal events in ring buffer so that they could be
> dumped into the error log to provide more context at crash time.  I
> chose to make these logs be mostly text based for simplicity of
> implementation.  Eventually more of the flight recorder infrastructure
> will be in place and the recording of this kind of data should be more
> integrated the infrastructure for that.  So my current implementation
> is somewhat ad hoc since I expect it will be revisited.
>
> This also doesn't include one of the more important features, dumping
> crashing context when scanning the heap.  Capturing that information
> without impacting GC performance is difficult and would require broad
> changes in the heap iteration code.  That can possibly be address later.
>
> I've chosen to tear out the existing Events class which was a debug
> only feature of limited use and extend that into a collection of
> product ring buffers.  They are registered during JVM startup and
> dumped into the hs_err log when the JVM crashes.  There are a couple
> different categories maintained: compilation, deopt, internal
> exceptions, other uncategorized messages and GC events.  Each ring
> buffer has it's own mutex to reduce contention across different kinds
> of events and generally logging is performed for relatively infrequent
> or expensive events to minimize any performance penalty.  I also chose
> to maintain separate logs for the categories so that higher frequency
> events one kind don't hide other kinds.
>
> I eliminated some of the less useful Events::log calls as well and
> move some others around.
>
> The GC logging keeps track of GC history using GCStatInfos.  This has
> the benefit that it's GC independent and provides before and after
> information.  It could also or instead capture Universe::print_on
> though that's pretty verbose output.  The GC section is fairly long
> and I'm not sure what the best output would be.  Opinions would be
> appreciated.
>
> There are some EventMarks in GC code that I'm tempted to delete.  Any
> objections?
>
> I put a couple example logs at:
>
> http://cr.openjdk.java.net/~never/7141200/example1.log
> http://cr.openjdk.java.net/~never/7141200/example2.log
>
> I can produce others if needed.
>
> Performance seems to be largely unchanged according to refworkload but
> I'm still running some benchmarks.  Tested with JCK vm and lang, and
> the nsk tests.
>
>


From tom.rodriguez at oracle.com  Tue Jan 31 10:15:05 2012
From: tom.rodriguez at oracle.com (Tom Rodriguez)
Date: Tue, 31 Jan 2012 10:15:05 -0800
Subject: Request for review 7141200: log some interesting information in
	ring buffers for crashes
In-Reply-To: <4F282E18.2060904@oracle.com>
References: <190D13C4-541E-470A-901F-EACB0D6A148C@oracle.com>
	<4F282E18.2060904@oracle.com>
Message-ID: <0E709DDE-60A7-4A01-BE2B-C258EA10C814@oracle.com>


On Jan 31, 2012, at 10:08 AM, Kevin Walls wrote:

> 
> Hi Tom,
> 
> Thanks for the examples.
> 
> Really like that we have a timestamp so we can see the ordering of separate types
> of events.  But we will get questions on how to convert a big number like
> 9617182121 into a time, and how it relates to times we see in other log files:
> can we convert it, e.g the same format as the GC logs?

I'm using javaTimeNanos() / 1000 which I believe should match the time stamps in GC logs, though it won't necessarily match the timestamps generated by TimeStamp, which is used by tty and LogCompilation.  The Management API use TimeStamp as well.  We have too many different sources, plus relative to JVM begin vs. absolute.  I'm honestly unsure which is truly best here.

> 
> (worst case, I was thinking we log the value of first_hrtime so we have some baseline,
> but I'm sure we can make it prettier than that 8-) )
> 
> 
> Why are the GC events all happening in the time period: Time: 0.000000.3 - 0.000000.3

I think it's a printing bug.  I'm looking into it.

> 
> That must be the format string %f.3 in memoryManager.cpp.  Again maybe we
> can make this more readable, this figure will no doubt be used to search GC logs.
> 
> 
> Trivial typos:
> events.hpp line 108, maybe "Should only be called while mutex is held."
> events.hpp line 58 subclassed rather than subclasses.

Thanks.

tom

> 
> 
> Thanks!
> Kevin
> 
> 
> On 31/01/12 07:50, Tom Rodriguez wrote:
>> http://cr.openjdk.java.net/~never/7141200
>> 664 lines changed: 389 ins; 185 del; 90 mod; 37188 unchg
>> 
>> 7141200: log some interesting information in ring buffers for crashes
>> Reviewed-by:
>> 
>> As part of convergence with JRockit it was requested that Hotspot
>> capture some internal events in ring buffer so that they could be
>> dumped into the error log to provide more context at crash time.  I
>> chose to make these logs be mostly text based for simplicity of
>> implementation.  Eventually more of the flight recorder infrastructure
>> will be in place and the recording of this kind of data should be more
>> integrated the infrastructure for that.  So my current implementation
>> is somewhat ad hoc since I expect it will be revisited.
>> 
>> This also doesn't include one of the more important features, dumping
>> crashing context when scanning the heap.  Capturing that information
>> without impacting GC performance is difficult and would require broad
>> changes in the heap iteration code.  That can possibly be address later.
>> 
>> I've chosen to tear out the existing Events class which was a debug
>> only feature of limited use and extend that into a collection of
>> product ring buffers.  They are registered during JVM startup and
>> dumped into the hs_err log when the JVM crashes.  There are a couple
>> different categories maintained: compilation, deopt, internal
>> exceptions, other uncategorized messages and GC events.  Each ring
>> buffer has it's own mutex to reduce contention across different kinds
>> of events and generally logging is performed for relatively infrequent
>> or expensive events to minimize any performance penalty.  I also chose
>> to maintain separate logs for the categories so that higher frequency
>> events one kind don't hide other kinds.
>> 
>> I eliminated some of the less useful Events::log calls as well and
>> move some others around.
>> 
>> The GC logging keeps track of GC history using GCStatInfos.  This has
>> the benefit that it's GC independent and provides before and after
>> information.  It could also or instead capture Universe::print_on
>> though that's pretty verbose output.  The GC section is fairly long
>> and I'm not sure what the best output would be.  Opinions would be
>> appreciated.
>> 
>> There are some EventMarks in GC code that I'm tempted to delete.  Any
>> objections?
>> 
>> I put a couple example logs at:
>> 
>> http://cr.openjdk.java.net/~never/7141200/example1.log
>> http://cr.openjdk.java.net/~never/7141200/example2.log
>> 
>> I can produce others if needed.
>> 
>> Performance seems to be largely unchanged according to refworkload but
>> I'm still running some benchmarks.  Tested with JCK vm and lang, and
>> the nsk tests.
>> 
>> 
> 


From tom.rodriguez at oracle.com  Tue Jan 31 10:17:09 2012
From: tom.rodriguez at oracle.com (Tom Rodriguez)
Date: Tue, 31 Jan 2012 10:17:09 -0800
Subject: Request for review 7141200: log some interesting information in
	ring buffers for crashes
In-Reply-To: <4F282686.7010908@oracle.com>
References: <190D13C4-541E-470A-901F-EACB0D6A148C@oracle.com>
	<4F282686.7010908@oracle.com>
Message-ID: <EABF73D0-5802-48E6-905A-356655D0DA65@oracle.com>


On Jan 31, 2012, at 9:36 AM, Jon Masamitsu wrote:

> Tom,
> 
> I only looked at the files from 
> 
> src/share/vm/services/memoryManager.*
> src/share/vm/utilities/debug.*
> src/share/vm/utilities/events.*
> src/share/vm/utilities/exceptions.cpp
> src/share/vm/utilities/vmError.cpp
> 
> In events.hpp if the variable _filled maybe would be
> better named _entries to indicate the number of entries
> written.  I initially thought with _filled you were trying
> to record the number of times the buffer wrapped.
> 
> Copyrights need updating.
> 
> These are supposed to be start and end times?
> | Time: 0.000000.3 - 0.000000.3

Also what's the opinion on using GCStatInfo instead of something like Universe::print_on.  Jesper had given me code which did both, but I left it out because the two combined seemed like too much output.  Maybe I'm over thinking it though.

tom

> 
> Jon
> 
> 
> On 01/30/12 23:50, Tom Rodriguez wrote:
>> http://cr.openjdk.java.net/~never/7141200
>> 
>> 664 lines changed: 389 ins; 185 del; 90 mod; 37188 unchg
>> 
>> 7141200: log some interesting information in ring buffers for crashes
>> Reviewed-by:
>> 
>> As part of convergence with JRockit it was requested that Hotspot
>> capture some internal events in ring buffer so that they could be
>> dumped into the error log to provide more context at crash time.  I
>> chose to make these logs be mostly text based for simplicity of
>> implementation.  Eventually more of the flight recorder infrastructure
>> will be in place and the recording of this kind of data should be more
>> integrated the infrastructure for that.  So my current implementation
>> is somewhat ad hoc since I expect it will be revisited.
>> 
>> This also doesn't include one of the more important features, dumping
>> crashing context when scanning the heap.  Capturing that information
>> without impacting GC performance is difficult and would require broad
>> changes in the heap iteration code.  That can possibly be address later.
>> 
>> I've chosen to tear out the existing Events class which was a debug
>> only feature of limited use and extend that into a collection of
>> product ring buffers.  They are registered during JVM startup and
>> dumped into the hs_err log when the JVM crashes.  There are a couple
>> different categories maintained: compilation, deopt, internal
>> exceptions, other uncategorized messages and GC events.  Each ring
>> buffer has it's own mutex to reduce contention across different kinds
>> of events and generally logging is performed for relatively infrequent
>> or expensive events to minimize any performance penalty.  I also chose
>> to maintain separate logs for the categories so that higher frequency
>> events one kind don't hide other kinds.
>> 
>> I eliminated some of the less useful Events::log calls as well and
>> move some others around.
>> 
>> The GC logging keeps track of GC history using GCStatInfos.  This has
>> the benefit that it's GC independent and provides before and after
>> information.  It could also or instead capture Universe::print_on
>> though that's pretty verbose output.  The GC section is fairly long
>> and I'm not sure what the best output would be.  Opinions would be
>> appreciated.
>> 
>> There are some EventMarks in GC code that I'm tempted to delete.  Any
>> objections?
>> 
>> I put a couple example logs at:
>> 
>> 
>> http://cr.openjdk.java.net/~never/7141200/example1.log
>> http://cr.openjdk.java.net/~never/7141200/example2.log
>> 
>> 
>> I can produce others if needed.
>> 
>> Performance seems to be largely unchanged according to refworkload but
>> I'm still running some benchmarks.  Tested with JCK vm and lang, and
>> the nsk tests.
>> 
>> 
>> 


From tom.rodriguez at oracle.com  Tue Jan 31 10:34:41 2012
From: tom.rodriguez at oracle.com (Tom Rodriguez)
Date: Tue, 31 Jan 2012 10:34:41 -0800
Subject: Request for review 7141200: log some interesting information in
	ring buffers for crashes
In-Reply-To: <4F282E18.2060904@oracle.com>
References: <190D13C4-541E-470A-901F-EACB0D6A148C@oracle.com>
	<4F282E18.2060904@oracle.com>
Message-ID: <9E57BE2B-EAC9-4F5D-8FBC-21A27E322E0F@oracle.com>


On Jan 31, 2012, at 10:08 AM, Kevin Walls wrote:

> 
> Hi Tom,
> 
> Thanks for the examples.
> 
> Really like that we have a timestamp so we can see the ordering of separate types
> of events.  But we will get questions on how to convert a big number like
> 9617182121 into a time, and how it relates to times we see in other log files:
> can we convert it, e.g the same format as the GC logs?
> 
> (worst case, I was thinking we log the value of first_hrtime so we have some baseline,
> but I'm sure we can make it prettier than that 8-) )
> 
> 
> Why are the GC events all happening in the time period: Time: 0.000000.3 - 0.000000.3
> 
> That must be the format string %f.3 in memoryManager.cpp.  Again maybe we
> can make this more readable, this figure will no doubt be used to search GC logs.

It was that plus a missing conversion to double.  It looks right now.

| GC Index: 1  Type: Minor
| Time: 5.693 - 5.707
| Usage:
|    Pool #0: 1137600 -> 1137600
|    Pool #1: 1040288 -> 0
|    Pool #2: 0 -> 327744
|    Pool #3: 0 -> 0
|    Pool #4: 2292048 -> 2292048


> 
> 
> Trivial typos:
> events.hpp line 108, maybe "Should only be called while mutex is held."
> events.hpp line 58 subclassed rather than subclasses.

Fixed.  Thanks.

tom

> 
> 
> Thanks!
> Kevin
> 
> 
> On 31/01/12 07:50, Tom Rodriguez wrote:
>> http://cr.openjdk.java.net/~never/7141200
>> 664 lines changed: 389 ins; 185 del; 90 mod; 37188 unchg
>> 
>> 7141200: log some interesting information in ring buffers for crashes
>> Reviewed-by:
>> 
>> As part of convergence with JRockit it was requested that Hotspot
>> capture some internal events in ring buffer so that they could be
>> dumped into the error log to provide more context at crash time.  I
>> chose to make these logs be mostly text based for simplicity of
>> implementation.  Eventually more of the flight recorder infrastructure
>> will be in place and the recording of this kind of data should be more
>> integrated the infrastructure for that.  So my current implementation
>> is somewhat ad hoc since I expect it will be revisited.
>> 
>> This also doesn't include one of the more important features, dumping
>> crashing context when scanning the heap.  Capturing that information
>> without impacting GC performance is difficult and would require broad
>> changes in the heap iteration code.  That can possibly be address later.
>> 
>> I've chosen to tear out the existing Events class which was a debug
>> only feature of limited use and extend that into a collection of
>> product ring buffers.  They are registered during JVM startup and
>> dumped into the hs_err log when the JVM crashes.  There are a couple
>> different categories maintained: compilation, deopt, internal
>> exceptions, other uncategorized messages and GC events.  Each ring
>> buffer has it's own mutex to reduce contention across different kinds
>> of events and generally logging is performed for relatively infrequent
>> or expensive events to minimize any performance penalty.  I also chose
>> to maintain separate logs for the categories so that higher frequency
>> events one kind don't hide other kinds.
>> 
>> I eliminated some of the less useful Events::log calls as well and
>> move some others around.
>> 
>> The GC logging keeps track of GC history using GCStatInfos.  This has
>> the benefit that it's GC independent and provides before and after
>> information.  It could also or instead capture Universe::print_on
>> though that's pretty verbose output.  The GC section is fairly long
>> and I'm not sure what the best output would be.  Opinions would be
>> appreciated.
>> 
>> There are some EventMarks in GC code that I'm tempted to delete.  Any
>> objections?
>> 
>> I put a couple example logs at:
>> 
>> http://cr.openjdk.java.net/~never/7141200/example1.log
>> http://cr.openjdk.java.net/~never/7141200/example2.log
>> 
>> I can produce others if needed.
>> 
>> Performance seems to be largely unchanged according to refworkload but
>> I'm still running some benchmarks.  Tested with JCK vm and lang, and
>> the nsk tests.
>> 
>> 
> 


From volker.simonis at gmail.com  Tue Jan 31 11:07:10 2012
From: volker.simonis at gmail.com (Volker Simonis)
Date: Tue, 31 Jan 2012 20:07:10 +0100
Subject: Request for review (S): MH spread invoker crashes with NULL argument
	on x86_32
Message-ID: <CA+3eh13z7yZ6TyEhCzn8H7typvq1+TzMcbY6EsNUBrUty240Jw@mail.gmail.com>

Hi,

as the test in the webrev demonstrates, the MH spread invoker
will crashes the VM if invoked with a NULL argument on x86_32
platforms.

This is because nn 32-bit Intel platforms, the spread invoker
uses the 'rsi' register as temporary. But because 'rsi' also
contains the 'saved_last_sp' on x86_32 platforms, 'rsi' has to be
temporary saved on the stack.  If this saving is done before a
failing 'NULL' check in the spread adapter, the following
excpetion handling routine will be confused by the extra value on
the stack, because it expects to find the return address here.

To fix this problem, I've exchanged the usage of the temporary
registers 'rsi' and 'rdi' in the spread adapter such that we only
have to save the value of 'rsi' to the stack after the NULL check
was done:

http://cr.openjdk.java.net/~simonis/SpreadNullArg/

Regards,
Volker

PS: please don't forget to update the test directory name after a
bug id was assigned to this issue.

From james.melvin at oracle.com  Tue Jan 31 12:28:03 2012
From: james.melvin at oracle.com (James Melvin)
Date: Tue, 31 Jan 2012 15:28:03 -0500
Subject: RFR(S): 7123386: RFE: Preserve universal builds of HotSpot on Mac OS X
Message-ID: <4F284ED3.3070303@oracle.com>

Hi,

Please review the change to HotSpot to package JVM libraries in a
universal binary on Mac OS X. The current plan of record is to only
support a 64-bit JVM on Mac OS X. This solution preserves the option to
additionally include other architectures in the universal binary
in the future. It also further encapsulates platform specific makefile
changes for Mac OS X builds.

Feedback welcome...

WEBREV:
   http://cr.openjdk.java.net/~jmelvin/7123386/webrev.00

TESTS RUN:
   JPRT  2012-01-31-152453.jmelvin.7123386
   local Mac OS X builds/tests

Thanks,

Jim

From john.r.rose at oracle.com  Tue Jan 31 12:30:02 2012
From: john.r.rose at oracle.com (John Rose)
Date: Tue, 31 Jan 2012 12:30:02 -0800
Subject: CFV: New HSX Committer
In-Reply-To: <4F280845.2030400@oracle.com>
References: <4F280845.2030400@oracle.com>
Message-ID: <D5570512-93EC-4236-9738-D29AB83B7EDB@oracle.com>

Vote: yes

On Jan 31, 2012, at 7:27 AM, Paul Hohensee wrote:

> I'd like to nominate Staffan Larsen to be an OpenJDK HSX project Committer [0].


From bengt.rutisson at oracle.com  Tue Jan 31 12:50:06 2012
From: bengt.rutisson at oracle.com (Bengt Rutisson)
Date: Tue, 31 Jan 2012 21:50:06 +0100
Subject: CFV: New HSX Committer
In-Reply-To: <4F280845.2030400@oracle.com>
References: <4F280845.2030400@oracle.com>
Message-ID: <4F2853FE.1080204@oracle.com>


Vote: yes

Bengt

On 2012-01-31 16:27, Paul Hohensee wrote:
> I'd like to nominate Staffan Larsen to be an OpenJDK HSX project 
> Committer [0].
>
> Staffan is a Committer on the JDK 6, 7, 7u and 8 projects, has pushed 
> seven
> changes to the Hotspot forest and is the Oracle Serviceability group 
> lead (the
> Serviceability group's responsibility includes functionality in both 
> the JDK
> libraries and Hotspot).  Staffan is thus well-qualified to be a 
> Committer on
> the HSX project.
>
> Votes are due by 12:01AM U.S.A. Eastern time, February 8th, 2012.
>
> Only current HSX Committers [1] are eligible to vote on this nomination.
>
> For Lazy Consensus voting instructions, see [2].
>
> Thanks,
>
> Paul
>
> [0] http://openjdk.java.net/bylaws#committer
> [1] http://openjdk.java.net/census/#hsx
> [2] http://openjdk.java.net/projects/#committer-vote


From daniel.daugherty at oracle.com  Tue Jan 31 12:50:59 2012
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Tue, 31 Jan 2012 13:50:59 -0700
Subject: RFR(S): 7123386: RFE: Preserve universal builds of HotSpot on
	Mac OS X
In-Reply-To: <4F284ED3.3070303@oracle.com>
References: <4F284ED3.3070303@oracle.com>
Message-ID: <4F285433.4030106@oracle.com>

On 1/31/12 1:28 PM, James Melvin wrote:
> Hi,
>
> Please review the change to HotSpot to package JVM libraries in a
> universal binary on Mac OS X. The current plan of record is to only
> support a 64-bit JVM on Mac OS X. This solution preserves the option to
> additionally include other architectures in the universal binary
> in the future. It also further encapsulates platform specific makefile
> changes for Mac OS X builds.
>
> Feedback welcome...
>
> WEBREV:
>   http://cr.openjdk.java.net/~jmelvin/7123386/webrev.00

Thumbs up!

Please update copyright years to 2012.

make/Makefile
     No comments.

make/bsd/makefiles/defs.make
     nit: line 174: 'settins -> 'settings'

     line 192, 196, 197 - those '.txt' files aren't binaries... :-)

     line 198: Is there a reason not to have the Client VM on the
         "LIPO" list? We may not support a 64-bit Client VM, but
         that doesn't mean someone in the OpenJDK world wouldn't
         want/like/support one...

make/defs.make
     No comments.

make/bsd/makefiles/universal.gmk
     line 79: '-xfp' instead of '-xf'? Preserves timestamps...

From david.holmes at oracle.com  Tue Jan 31 13:21:58 2012
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 01 Feb 2012 07:21:58 +1000
Subject: CFV: New HSX Committer
In-Reply-To: <4F280845.2030400@oracle.com>
References: <4F280845.2030400@oracle.com>
Message-ID: <4F285B76.1070009@oracle.com>

Vote: yes

David

On 1/02/2012 1:27 AM, Paul Hohensee wrote:
> I'd like to nominate Staffan Larsen to be an OpenJDK HSX project
> Committer [0].
>
> Staffan is a Committer on the JDK 6, 7, 7u and 8 projects, has pushed seven
> changes to the Hotspot forest and is the Oracle Serviceability group
> lead (the
> Serviceability group's responsibility includes functionality in both the
> JDK
> libraries and Hotspot). Staffan is thus well-qualified to be a Committer on
> the HSX project.
>
> Votes are due by 12:01AM U.S.A. Eastern time, February 8th, 2012.
>
> Only current HSX Committers [1] are eligible to vote on this nomination.
>
> For Lazy Consensus voting instructions, see [2].
>
> Thanks,
>
> Paul
>
> [0] http://openjdk.java.net/bylaws#committer
> [1] http://openjdk.java.net/census/#hsx
> [2] http://openjdk.java.net/projects/#committer-vote

From zhengyu.gu at oracle.com  Tue Jan 31 14:01:38 2012
From: zhengyu.gu at oracle.com (Zhengyu Gu)
Date: Tue, 31 Jan 2012 17:01:38 -0500
Subject: CFV: New HSX Committer
In-Reply-To: <4F280845.2030400@oracle.com>
References: <4F280845.2030400@oracle.com>
Message-ID: <4F2864C2.3060701@oracle.com>

Vote: yes

On 1/31/2012 10:27 AM, Paul Hohensee wrote:
> I'd like to nominate Staffan Larsen to be an OpenJDK HSX project 
> Committer [0].
>
> Staffan is a Committer on the JDK 6, 7, 7u and 8 projects, has pushed 
> seven
> changes to the Hotspot forest and is the Oracle Serviceability group 
> lead (the
> Serviceability group's responsibility includes functionality in both 
> the JDK
> libraries and Hotspot).  Staffan is thus well-qualified to be a 
> Committer on
> the HSX project.
>
> Votes are due by 12:01AM U.S.A. Eastern time, February 8th, 2012.
>
> Only current HSX Committers [1] are eligible to vote on this nomination.
>
> For Lazy Consensus voting instructions, see [2].
>
> Thanks,
>
> Paul
>
> [0] http://openjdk.java.net/bylaws#committer
> [1] http://openjdk.java.net/census/#hsx
> [2] http://openjdk.java.net/projects/#committer-vote

From tom.rodriguez at oracle.com  Tue Jan 31 14:27:34 2012
From: tom.rodriguez at oracle.com (Tom Rodriguez)
Date: Tue, 31 Jan 2012 14:27:34 -0800
Subject: Request for review 7141200: log some interesting information in
	ring buffers for crashes
In-Reply-To: <4F282E18.2060904@oracle.com>
References: <190D13C4-541E-470A-901F-EACB0D6A148C@oracle.com>
	<4F282E18.2060904@oracle.com>
Message-ID: <532C3F2D-5D52-4337-9100-7177662007F7@oracle.com>

I've got a new version of the GC logging code.  It's now using PrintHeapAtGC routines which will give more useful and better formatted output.  I fixed all the review comments, deleted the EventMarks in GC code since no one objected, renamed _count to _length and _filled to _count in EventLogBase.  Each output line is preceded by Event: to help with greppability.  I've updated the webrev and the examples.

Thanks for all the good feedback.

Here's a sample of the new GC output:

GC Heap History (10):
Event: 9672681252 GC heap before
{Heap before GC invocations=25 (full 12):
 PSYoungGen      total 19712K, used 338K [0xe1e00000, 0xe3400000, 0xf7400000)
  eden space 16896K, 2% used [0xe1e00000,0xe1e54870,0xe2e80000)
  from space 2816K, 0% used [0xe3140000,0xe3140000,0xe3400000)
  to   space 2816K, 0% used [0xe2e80000,0xe2e80000,0xe3140000)
 PSOldGen        total 43008K, used 366K [0xb7400000, 0xb9e00000, 0xe1e00000)
  object space 43008K, 0% used [0xb7400000,0xb745bbf8,0xb9e00000)
 PSPermGen       total 16384K, used 2398K [0xb3400000, 0xb4400000, 0xb7400000)
  object space 16384K, 14% used [0xb3400000,0xb36578e0,0xb4400000)
Event: 9672681255 GC heap after
Heap after GC invocations=25 (full 12):
 PSYoungGen      total 19712K, used 48K [0xe1e00000, 0xe3400000, 0xf7400000)
  eden space 16896K, 0% used [0xe1e00000,0xe1e00000,0xe2e80000)
  from space 2816K, 1% used [0xe2e80000,0xe2e8c000,0xe3140000)
  to   space 2816K, 0% used [0xe3140000,0xe3140000,0xe3400000)
 PSOldGen        total 43008K, used 366K [0xb7400000, 0xb9e00000, 0xe1e00000)
  object space 43008K, 0% used [0xb7400000,0xb745bbf8,0xb9e00000)
 PSPermGen       total 16384K, used 2398K [0xb3400000, 0xb4400000, 0xb7400000)
  object space 16384K, 14% used [0xb3400000,0xb36578e0,0xb4400000)
}

A full example is at http://cr.openjdk.java.net/~never/7141200/example1.log.

tom

On Jan 31, 2012, at 10:08 AM, Kevin Walls wrote:

> 
> Hi Tom,
> 
> Thanks for the examples.
> 
> Really like that we have a timestamp so we can see the ordering of separate types
> of events.  But we will get questions on how to convert a big number like
> 9617182121 into a time, and how it relates to times we see in other log files:
> can we convert it, e.g the same format as the GC logs?
> 
> (worst case, I was thinking we log the value of first_hrtime so we have some baseline,
> but I'm sure we can make it prettier than that 8-) )
> 
> 
> Why are the GC events all happening in the time period: Time: 0.000000.3 - 0.000000.3
> 
> That must be the format string %f.3 in memoryManager.cpp.  Again maybe we
> can make this more readable, this figure will no doubt be used to search GC logs.
> 
> 
> Trivial typos:
> events.hpp line 108, maybe "Should only be called while mutex is held."
> events.hpp line 58 subclassed rather than subclasses.
> 
> 
> Thanks!
> Kevin
> 
> 
> On 31/01/12 07:50, Tom Rodriguez wrote:
>> http://cr.openjdk.java.net/~never/7141200
>> 664 lines changed: 389 ins; 185 del; 90 mod; 37188 unchg
>> 
>> 7141200: log some interesting information in ring buffers for crashes
>> Reviewed-by:
>> 
>> As part of convergence with JRockit it was requested that Hotspot
>> capture some internal events in ring buffer so that they could be
>> dumped into the error log to provide more context at crash time.  I
>> chose to make these logs be mostly text based for simplicity of
>> implementation.  Eventually more of the flight recorder infrastructure
>> will be in place and the recording of this kind of data should be more
>> integrated the infrastructure for that.  So my current implementation
>> is somewhat ad hoc since I expect it will be revisited.
>> 
>> This also doesn't include one of the more important features, dumping
>> crashing context when scanning the heap.  Capturing that information
>> without impacting GC performance is difficult and would require broad
>> changes in the heap iteration code.  That can possibly be address later.
>> 
>> I've chosen to tear out the existing Events class which was a debug
>> only feature of limited use and extend that into a collection of
>> product ring buffers.  They are registered during JVM startup and
>> dumped into the hs_err log when the JVM crashes.  There are a couple
>> different categories maintained: compilation, deopt, internal
>> exceptions, other uncategorized messages and GC events.  Each ring
>> buffer has it's own mutex to reduce contention across different kinds
>> of events and generally logging is performed for relatively infrequent
>> or expensive events to minimize any performance penalty.  I also chose
>> to maintain separate logs for the categories so that higher frequency
>> events one kind don't hide other kinds.
>> 
>> I eliminated some of the less useful Events::log calls as well and
>> move some others around.
>> 
>> The GC logging keeps track of GC history using GCStatInfos.  This has
>> the benefit that it's GC independent and provides before and after
>> information.  It could also or instead capture Universe::print_on
>> though that's pretty verbose output.  The GC section is fairly long
>> and I'm not sure what the best output would be.  Opinions would be
>> appreciated.
>> 
>> There are some EventMarks in GC code that I'm tempted to delete.  Any
>> objections?
>> 
>> I put a couple example logs at:
>> 
>> http://cr.openjdk.java.net/~never/7141200/example1.log
>> http://cr.openjdk.java.net/~never/7141200/example2.log
>> 
>> I can produce others if needed.
>> 
>> Performance seems to be largely unchanged according to refworkload but
>> I'm still running some benchmarks.  Tested with JCK vm and lang, and
>> the nsk tests.
>> 
>> 
> 


From paul.hohensee at oracle.com  Tue Jan 31 15:35:27 2012
From: paul.hohensee at oracle.com (Paul Hohensee)
Date: Tue, 31 Jan 2012 18:35:27 -0500
Subject: RFR(S): 7123386: RFE: Preserve universal builds of HotSpot on
	Mac OS X
In-Reply-To: <4F284ED3.3070303@oracle.com>
References: <4F284ED3.3070303@oracle.com>
Message-ID: <4F287ABF.8040300@oracle.com>

Looks good.

Paul

On 1/31/12 3:28 PM, James Melvin wrote:
> Hi,
>
> Please review the change to HotSpot to package JVM libraries in a
> universal binary on Mac OS X. The current plan of record is to only
> support a 64-bit JVM on Mac OS X. This solution preserves the option to
> additionally include other architectures in the universal binary
> in the future. It also further encapsulates platform specific makefile
> changes for Mac OS X builds.
>
> Feedback welcome...
>
> WEBREV:
>   http://cr.openjdk.java.net/~jmelvin/7123386/webrev.00
>
> TESTS RUN:
>   JPRT  2012-01-31-152453.jmelvin.7123386
>   local Mac OS X builds/tests
>
> Thanks,
>
> Jim