August 2013 - hotspot-gc-dev

RFR (S): 8023021: Unnecessary clearing of the card table introduced by the fix for JDK-8023013
by Bengt Rutisson 15 Aug '13

15 Aug '13

Hi all, Erik discovered a bug with the change for JDK-8023013 that I just pushed. With my change we will do some unnecessary clearing of the card table. Here's a webrev to fix that: http://cr.openjdk.java.net/~brutisso/8023021/webrev.00/ Thanks Erik for finding this! From the bug report: The fix for JDK-8023013 "Use specific generations rather than generation iteration" introduced unnecessary clearing of the card table. This loop (which existed both in CardTableRS::clear_into_younger() and CardTableRS::invalidate_or_clear()): Generation* g = gen; for(Generation* prev_gen = gch->prev_gen(g); prev_gen != NULL; g = prev_gen, prev_gen = gch->prev_gen(g)) { ... } was translated in to one call for the old generation and one for the young generation. But this is wrong. The loop is actually not a loop when we only have two generations. These methods are only called with gen being the old generation, so the translation should be to only do one call to the old generation. No work should be done for the young generation. Thanks, Bengt

2 2

Request for review (m) - 8022725:Refactor code in concurrentMarkSweepGeneration.cpp
by Jon Masamitsu 15 Aug '13

15 Aug '13

8022725:Refactor code in concurrentMarkSweepGeneration.cpp concurrentMarkSweepGeneration.cpp is a rather large file and should be refactored (split into multiple files) for easier maintenance. There are no intended semantic changes. Only reorganization and moving of code. I've been investigating some issues with compiler optimizations on this file and the refactoring will make that doable. http://cr.openjdk.java.net/~jmasa/8022725/webrev.00/

3 4

Deferral justification: JDK-8022892
by Tao Mao 15 Aug '13

15 Aug '13

Hi GC team, The CR is here https://jbs.oracle.com/bugs/browse/JDK-8022892. First of all, Jon pointed out it's not a regression in hs24 since it reported against hs23. (Thank you, Jon) I think this is right. But i'm not sure, only with that, whether I can get through the deferral process. So, let me put down what I have found so far. 1. In the CR, Exception Code:c0000005 is seen and it means EXCEPTION_ACCESS_VIOLATION. This type of error is reported many times in different places inside and outside VM. Also, in the CR, it shows " Fault Module Name:msvcr100.dll", which indicates the top of stack frame is C native code/3rd party library from Microsoft. Thus, it may be not a VM problem at all, in this case. 2. Let's look at the ILW priority mapping. The impact is high since it's crash. But there's ambiguity about its likelihood and workaround. I would think the likelihood is low according to the definition (http://wiki.se.oracle.com/display/JPGRM/ILW+and+priority+mapping+for+bugs#I…) " The defect is encountered in an uncommon (or unsupported) use case or is intermittent in nature with a low frequency of occurrence" Starting with initial heap size 140GB seems to be an uncommon use case (to me). Workaround: use other collector as the report stated or lower heap size. Low. ILW = HLL = P4. 3. In fact, it's hard to find a windows machine configured with 140GB memory. Probably there's no such machine if you refer to RAM/CPU limits of Windows machine stated in Microsoft website (http://support.microsoft.com/kb/2860880) The RAM limit for Windows HPC Server 2008 R2 is 128GB. That's the reason why I think the likelihood is low. What I'm trying to do here is to use the Linux machine sthdev05.se.oracle.com listed here (http://wiki.se.oracle.com/display/JRPG/Development+Servers) This machine have about 256GB memory. But the VM cannot allocate(commit) enough memory if I set -Xms to be 35g or higher. In this attempt, we are not yet close to reproducing. Ok, that's all I've found. Among all these, I'd like to add the 128GB RAM limit of this type of Windows server as the second justification. Could you read the email and look through the bug report to help obtain at least one more evidence for the deferral justification if possible? Thanks. Tao

1 1

Re: JIT stops compiling after a while (java 7)
by Vladimir Kozlov 14 Aug '13

14 Aug '13

Hi, Martin Thank you for very nice test! If it was not 7u40 I would say -XX:+UseCodeCacheFlushing will help. But it is on by default in jdk8 and in 7u40. I don't understand why it is not triggered. And you need debug VM to run with -XX:+PrintMethodFlushing. VM stops compiling when it CodeCache is full, it is normal (no space left for compiled code). PermGen clean up means some classes were unloaded. During class unloading its methods are removed from CodeCache leaving some space for new compilations. jdk8 unload classes more aggressively because in metaspace classes allocated per classloader. And when classloader is gone, all its classes and their methods are gone - no need to wait for full GC. I CCing to GC group to get confirmation/explanation. That may explain difference between 7u40 and 8. Drop in Invocations/s is due to running instance.run() in Interpreter when VM stops compiling because new 'instance' uses new classes and previous compilations does not help. I think it is not dropping totally down (as in second phase) because the method Util.pow10() is compiled separately and used by interpreted method instance.run(). During second phase Util.pow10() could be also removed from codecaceh (UseCodeCacheFlushing can do that) and everything run in Interpreter. All above is speculation based on your data and test. You can run -XX:CICompilerCount=1 -XX:+PrintCompilation to see compilation events and when methods removed from codecache (become 'not entrant' and then 'zombie'). We also did several fixes in CodeCache code in jdk8 which were not backported to 7u40. They may also have some effect. But I see big correlation between metaspace and codecache sizes in jdk8 so I think it is the main reason for difference (different classes unloading behavior). Regards, Vladimir On 8/13/13 9:47 PM, Martin Traverso wrote: > We're seeing some strange behavior with the JIT compiler in jdk7. As far as we can tell, at some point in the lifetime > of the process the vm decides to stop generating native code. > > I wrote a simple program that reproduces the behavior. It generates classes dynamically, with a method that loops and > performs some computation. It does this over and over and it calls the method each time to make the vm optimize and > generate native code. Separately, it gathers and prints the following stats: cumulative compilation time, code cache > size, perm gen size and invocations per second. > > You can take a look at the code here: http://github.com/martint/jittest. Here's a chart that shows how those stats > behave over time: https://raw.github.com/martint/jittest/master/stats-7u40.png. > > As you can see in the chart, the code cache grows steadily in the beginning. So does the cumulative compilation time. At > some point, the cache fills up and the vm stops compiling (my interpretation) and invocation throughput drops > significantly. Nothing much happens for a while after that, until the perm gen fills up and goes through a GC cycle. > Some time after (not sure if related to the perm gen GC), the code cache flushes and the vm starts compiling again. > Eventually, the code cache fills up again, but at a lower level that the first time. The vm stops compiling once again > and performance drops to an even lower level that before. I haven't seen it recover from this condition, which is > consistent with what we see in our production system. > > I've been able to reproduce it in both 7u10 and the 7u40 early access that's available in the openjdk site. > Interestingly, java 8 does not exhibit this issue. > > Is this a known problem? Is there any way around it (other than restarting the vm) or things to try out while we wait > for java 8 to come out? > > Thanks! > > Martin >

2 1

Deterministic memory manager of constant complexity
by Phil Bouchard 14 Aug '13

14 Aug '13

Hello, I am a software engineer who wrote the following: http://finitetheory.com/author.html I would like to propose Java a new deterministic memory manager of constant complexity that will make applications run smoothly and never to lag again. Its documentation is available at: https://svn.boost.org/svn/boost/sandbox/block_ptr/libs/smart_ptr/doc/index.… As a practical example, the applications on the Android will run much better. Sincerely yours, Phil Bouchard

1 0

hg: hsx/hotspot-gc/hotspot: 8022899: SunStudio compiler can not handle EXCEPTION_MARK and inlining
by erik.helin＠oracle.com 14 Aug '13

14 Aug '13

Changeset: 5d9995d16b26 Author: ehelin Date: 2013-08-14 13:49 +0200 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/5d9995d16b26 8022899: SunStudio compiler can not handle EXCEPTION_MARK and inlining Reviewed-by: coleenp, mgerdin ! src/share/vm/utilities/exceptions.hpp

1 0

RFR (L): 7145569 G1: optimize nmethods scanning
by Thomas Schatzl 14 Aug '13

14 Aug '13

Hi all, can I have some reviews for the latest version for the 7145569 patch? If possible from one person from the compiler team too. This RFR is a continuation of a thread from June [1], where John Cuthbertson proposed this change for the first time. We performed many stability and performance tests with it in the meantime, and found a bug - hence the re-review request. Summarizing the original issue: We found cases where G1 code cache scan to be the dominator of pause time. An additional problem has been that entire code cache scan has been performed by a single thread, other threads waiting for it. Since G1 can only determine that an nmethod does not need to to be scanned any more during full gc, this also resulted in linearly increasing gc times until the next full gc. The original change attempts to fix this by keeping a per-region list of nmethods (compiled code) that have references into that particular regions. The lists are updated when the state of the compiled code changes. During pause time, at normal gc, instead of scanning the entire code cache, the gc only scans the ones of regions that are affected by the evacuation. During initial mark we still scan all regions, but on a per-region basis, and during full gc the per-region lists are simply rebuilt. This change can be found at [2]. The problem with the original patch has been in removing nmethods from the regions while code cache memory is reclaimed: this update has not been synchronized with the appropriate lock (the CodeCache_lock), so it happened that these lists got corrupted. The fix for this can be found at [3]: it adds this synchronization. This fix is somewhat complicated by the fact that while holding the Patching_lock (where the original call to the deregistration was located) we cannot grab the CodeCache_lock, as this would potentially cause a deadlock. So the change remembers that we need to unregister the nmethod later, and does it later. Other changes are mostly concerned with adding or fixing asserts that fail now because we try to iterate over the oops of the nmethod later than expected. There is a webrev containing the complete patch at [4]. Testing result summary: - no performance changes - negligible overhead of the additional per region nmethod lists ("remembered sets") - very small code cache scanning times during regular gc - much more balanced code scan times during initial mark pause. There is still imbalance due to sometimes heavily biased distribution of nmethods across regions. This will probably need to be fixed in a follow-up CR. Bugs.sun.com link: http://bugs.sun.com/view_bug.do?bug_id=7145569 Testing (in addition to the one performed using the original change): Specjbb2013, JPRT, Weblogic, nashorn javascript benchmark suite with reduced code cache size, internal test suites Thanks, Thomas [1] http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2013-June/007591.html [2] http://cr.openjdk.java.net/~tschatzl/7145569/webrev.johnc/ [3] http://cr.openjdk.java.net/~tschatzl/7145569/webrev.1.updates/ [4] http://cr.openjdk.java.net/~tschatzl/7145569/webrev.1.complete/

1 0

hg: hsx/hotspot-gc/hotspot: 2 new changesets
by bengt.rutisson＠oracle.com 14 Aug '13

14 Aug '13

Changeset: 330dfb0476f4 Author: brutisso Date: 2013-08-14 09:02 +0200 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/330dfb0476f4 8022800: Use specific generations rather than generation iteration Reviewed-by: jmasa, ehelin ! src/share/vm/gc_implementation/parNew/parNewGeneration.cpp ! src/share/vm/memory/cardTableRS.cpp ! src/share/vm/memory/cardTableRS.hpp ! src/share/vm/memory/defNewGeneration.cpp ! src/share/vm/memory/genCollectedHeap.cpp ! src/share/vm/memory/genCollectedHeap.hpp ! src/share/vm/memory/genMarkSweep.cpp ! src/share/vm/memory/genRemSet.hpp Changeset: 3f22cbf5275d Author: brutisso Date: 2013-08-14 10:55 +0200 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/3f22cbf5275d Merge

1 0

hg: hsx/hotspot-gc/hotspot: 8022880: False sharing between PSPromotionManager instances
by stefan.karlsson＠oracle.com 14 Aug '13

14 Aug '13

Changeset: 9766f73e770d Author: stefank Date: 2013-05-31 14:32 +0200 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/9766f73e770d 8022880: False sharing between PSPromotionManager instances Summary: Pad the PSPromotionManager instances in the manager array. Reviewed-by: brutisso, jmasa ! src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.cpp ! src/share/vm/gc_implementation/parNew/parNewGeneration.hpp ! src/share/vm/gc_implementation/parNew/parOopClosures.hpp ! src/share/vm/gc_implementation/parallelScavenge/psPromotionManager.cpp ! src/share/vm/gc_implementation/parallelScavenge/psPromotionManager.hpp ! src/share/vm/gc_implementation/parallelScavenge/psPromotionManager.inline.hpp + src/share/vm/memory/padded.hpp + src/share/vm/memory/padded.inline.hpp ! src/share/vm/utilities/debug.hpp ! src/share/vm/utilities/globalDefinitions.hpp

1 0

RFR (M): 8022800: Use specific generations rather than generation iteration
by Bengt Rutisson 14 Aug '13

14 Aug '13

Hi everyone, Could I have a couple of reviews of this change? http://cr.openjdk.java.net/~brutisso/8022800/webrev.00/ The interesting change is in genCollectedHeap.hpp, where I change the methods prev_gen(), next_gen() and get_gen() to never return NULL. The rest of the changes are ripple effects of this change. From the CR: The generations abstraction in the GC code was intended to be very general. In practice we now always have exactly two generations - the young and the old. This means that the GC code can be simplified quite a bit. A first step is to change from iteration over generations to explicitly specify which generation is needed. This makes the code easier to follow and it also helps tools like Parfait to understand the code better. This bug targets some simple clean-ups. A natural follow up of this is probably to change the name of GenCollectedHeap to TwoGenerationCollectedHeap (similar to the existing TwoGenerationCollectorPolicy). But that will be handled separately. Thanks, Bengt

1 1