From John.Coomes at oracle.com Thu May 1 01:11:03 2014 From: John.Coomes at oracle.com (John Coomes) Date: Wed, 30 Apr 2014 18:11:03 -0700 Subject: review request (S): 8042255 make gc src file exclusion more automatic Message-ID: <21345.40743.531779.25170@mykonos.us.oracle.com> Please review this small makefile change to reduce the need to update the list of files in make/excludeSrc.make. http://cr.openjdk.java.net/~jcoomes/8042255-gc-exclude/ -John From jesper.wilhelmsson at oracle.com Fri May 2 00:26:02 2014 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Fri, 02 May 2014 02:26:02 +0200 Subject: RFR: 8027643 - Merge GenCollectorPolicy and TwoGenerationCollectorPolicy In-Reply-To: <5360DEA8.1070608@oracle.com> References: <53602775.4090400@oracle.com> <5360DEA8.1070608@oracle.com> Message-ID: <5362E61A.7010803@oracle.com> Hi Stefan, Thanks for the review! Yes, the _generations array is going away. /Jesper Stefan Johansson skrev 30/4/14 13:29: > Hi Jesper, > > The change looks good. > > Do the further cleanups you mention include getting rid of the _generations > array in GenCollectorPolicy and instead having a GenerationSpec for young and > one for old? If not, I think that is something that should be added in upcoming > one, since the policy no longer supports anything but 2 generations. > > Cheers, > Stefan > > On 2014-04-30 00:28, Jesper Wilhelmsson wrote: >> Hi, >> >> This is another step in cleaning up the collector policy code. It is the first >> of a few related patches. This change merges the GenCollectorPolicy class with >> TwoGenerationalCollectorPolicy. >> >> I have deliberately refrained from doing any other cleanups that is made >> possible by this change. That will come in later changes. That makes this >> change really easy to review :-) >> >> Webrev: http://cr.openjdk.java.net/~jwilhelm/8027643/webrev/ >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8027643 >> >> Thanks! >> /Jesper > From jesper.wilhelmsson at oracle.com Fri May 2 01:17:30 2014 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Fri, 02 May 2014 03:17:30 +0200 Subject: RFR(S): 8026396 - Remove information duplication in the collector policy Message-ID: <5362F22A.1030505@oracle.com> Hi, Another step towards cleaner collector policy code. This cleanup removes the need to keep the generation sizing flags in sync with the collector policy version of the same variables during setup. The collector policy variables are initialized in the start and then used throughout the setup code. In the end we write the values back to the flags if needed. This change builds upon the merged collector policy (8027643) currently in review. Webrev: http://cr.openjdk.java.net/~jwilhelm/8026396/webrev/ Bug: https://bugs.openjdk.java.net/browse/JDK-8026396 Thanks, /Jesper From jesper.wilhelmsson at oracle.com Fri May 2 01:39:39 2014 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Fri, 02 May 2014 03:39:39 +0200 Subject: RFR: 8042298 - Remove the names gen0 and gen1 from the GC code Message-ID: <5362F75B.9020201@oracle.com> Hi, This is a step towards removing the _generations array. The names gen0 and gen1 are replaced with young and old to use a more standardized vocabulary. Webrev: http://cr.openjdk.java.net/~jwilhelm/8042298/webrev/ Bug: https://bugs.openjdk.java.net/browse/JDK-8042298 Thanks, /Jesper From bengt.rutisson at oracle.com Fri May 2 08:33:54 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Fri, 02 May 2014 10:33:54 +0200 Subject: RFR(S): 8040803: G1: Concurrent mark hangs when mark stack overflows In-Reply-To: <5360F4CB.9040106@oracle.com> References: <5360F4CB.9040106@oracle.com> Message-ID: <53635872.4030107@oracle.com> Hi Per, On 2014-04-30 15:04, Per Liden wrote: > Hi, > > Could I please have a couple of reviews in this bug fix: > > Summary: G1's concurrent marking can potentially hang forever if the > global mark stack overflows and immediately after that a Full GC > happens, which tries to abort the marking. The reason is that there's > a race between detecting the overflow situation and detecting the > abort signal. Threads detecting the overflow situation first will go > into the overflow protocol and wait on a barrier for all threads to > reach this state. However, threads detecting the abort signal first > will terminate and never participate in the barrier. > > This patch introduces an abort state and function on the > WorkGangBarrierSync class, to unblock any threads waiting for the > barrier to complete when the concurrent mark is aborted. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8040803 > Webrev: http://cr.openjdk.java.net/~pliden/8040803/webrev.0/ The change looks good. One minor improvement would be to try to use an early exit in ConcurrentMark::enter_first_sync_barrier() when the barrier fails. Thanks, Bengt > > /Per From stefan.johansson at oracle.com Fri May 2 12:36:31 2014 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Fri, 02 May 2014 14:36:31 +0200 Subject: RFR(S): 8026396 - Remove information duplication in the collector policy In-Reply-To: <5362F22A.1030505@oracle.com> References: <5362F22A.1030505@oracle.com> Message-ID: <5363914F.9040202@oracle.com> Looks good, Stefan On 2014-05-02 03:17, Jesper Wilhelmsson wrote: > Hi, > > Another step towards cleaner collector policy code. > > This cleanup removes the need to keep the generation sizing flags in > sync with the collector policy version of the same variables during > setup. The collector policy variables are initialized in the start and > then used throughout the setup code. In the end we write the values > back to the flags if needed. > > This change builds upon the merged collector policy (8027643) > currently in review. > > Webrev: http://cr.openjdk.java.net/~jwilhelm/8026396/webrev/ > > Bug: https://bugs.openjdk.java.net/browse/JDK-8026396 > > Thanks, > /Jesper From jesper.wilhelmsson at oracle.com Fri May 2 12:52:43 2014 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Fri, 02 May 2014 14:52:43 +0200 Subject: RFR(S): 8026396 - Remove information duplication in the collector policy In-Reply-To: <5363914F.9040202@oracle.com> References: <5362F22A.1030505@oracle.com> <5363914F.9040202@oracle.com> Message-ID: <5363951B.4090003@oracle.com> Thanks! /Jesper Stefan Johansson skrev 2/5/14 14:36: > Looks good, > > Stefan > > On 2014-05-02 03:17, Jesper Wilhelmsson wrote: >> Hi, >> >> Another step towards cleaner collector policy code. >> >> This cleanup removes the need to keep the generation sizing flags in sync with >> the collector policy version of the same variables during setup. The collector >> policy variables are initialized in the start and then used throughout the >> setup code. In the end we write the values back to the flags if needed. >> >> This change builds upon the merged collector policy (8027643) currently in >> review. >> >> Webrev: http://cr.openjdk.java.net/~jwilhelm/8026396/webrev/ >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8026396 >> >> Thanks, >> /Jesper > From John.Coomes at oracle.com Fri May 2 16:05:37 2014 From: John.Coomes at oracle.com (John Coomes) Date: Fri, 2 May 2014 09:05:37 -0700 Subject: RFR: 8027643 - Merge GenCollectorPolicy and TwoGenerationCollectorPolicy In-Reply-To: <53602775.4090400@oracle.com> References: <53602775.4090400@oracle.com> Message-ID: <21347.49745.501768.48858@mykonos.us.oracle.com> Jesper Wilhelmsson (jesper.wilhelmsson at oracle.com) wrote: > Hi, > > This is another step in cleaning up the collector policy code. It is the first > of a few related patches. This change merges the GenCollectorPolicy class with > TwoGenerationalCollectorPolicy. > > I have deliberately refrained from doing any other cleanups that is made > possible by this change. That will come in later changes. That makes this change > really easy to review :-) I, for one, heartily appreciate it :-). > Webrev: http://cr.openjdk.java.net/~jwilhelm/8027643/webrev/ > > Bug: https://bugs.openjdk.java.net/browse/JDK-8027643 Looks good to me. -John From jon.masamitsu at oracle.com Fri May 2 17:25:43 2014 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Fri, 02 May 2014 10:25:43 -0700 Subject: Request for review (s) - 8031323: Optionally align objects copied to survivor spaces In-Reply-To: <53284A7E.9060106@oracle.com> References: <53267CBB.9020804@oracle.com> <53284A7E.9060106@oracle.com> Message-ID: <5363D517.80200@oracle.com> New webrev with problems addressed except as noted below. http://cr.openjdk.java.net/~jmasa/8031323/webrev.01/ Thanks. On 03/18/2014 06:30 AM, Bengt Rutisson wrote: > > Hi Jon, > > On 2014-03-17 05:40, Jon Masamitsu wrote: >> 8031323: Optionally align objects copied to survivor spaces >> >> Add the option to specify an alignment for allocations into the >> survivor spaces. Survivor space allocations will overflow if the >> survivor spaces have not been increased. The expected use >> is to align survivor space allocations to cache line sizes. >> >> http://cr.openjdk.java.net/~jmasa/8031323/webrev.00/ > > For G1 it looks to me like we are aligning objects in the old regions > as well. I thought the intent was to only align in the survivor > regions. Am I missing something? Yes, that was an error. Fixed in the new webrev. > > Note that it is not enough to check the alloc purpose for G1 since > G1CollectedHeap::par_allocate_during_gc() only takes the purpose as a > hint and may still return buffers from other regions than expected for > a purpose. But I guess that in this case it is not too important to > get it exactly right, so it may be enough to check the alloc purpose > value. I did use the purpose and there are cases where alignments in the survivor spaces can be missed. I think the number missed is a very small percentage of the total so I'm going to let those slip by. I thought that preferable to the more extensive coding changes it would take to catch them. > > > The code in PSYoungPromotionLAB::allcoate(), > ParGCAllocBuffer::allocate_aligned() and > ContiguousSpace::allocate_aligned() is very similar. Is there a way to > consolidate these methods? > I looked at this but did not come up with a suitable way to fix it. I've opened CR to track the issue. https://bugs.openjdk.java.net/browse/JDK-8042321 > > The code in CollectedHeap::align_allocation_or_fail() looks quite > different from other places in the code base where we do padding. What > do you think about doing something like this instead? > > inline HeapWord* CollectedHeap::align_allocation_or_fail(HeapWord* > addr, HeapWord* end, unsigned short alignment_in_bytes) { > if (alignment_in_bytes <= ObjectAlignmentInBytes) { > return addr; > } > > assert(is_ptr_aligned(addr, HeapWordSize), ""); > assert(is_size_aligned(alignment_in_bytes, HeapWordSize), ""); > > HeapWord* new_addr = (HeapWord*) align_pointer_up(addr, > alignment_in_bytes); > size_t padding = pointer_delta(new_addr, addr); > https://bugs.openjdk.java.net/browse/JDK-8042321 > if (padding == 0) { > return addr; > } > > if (padding < CollectedHeap::min_fill_size()) { > padding += alignment_in_bytes / HeapWordSize; > assert(padding >= CollectedHeap::min_fill_size(), ""); > new_addr = addr + padding; > } > > if(new_addr > addr && new_addr < end) { > CollectedHeap::fill_with_object(addr, padding); > return new_addr; > } else { > return NULL; > } > } Fixed that as suggested. Jon > > Thanks, > Bengt > >> >> Jon > From jesper.wilhelmsson at oracle.com Fri May 2 19:38:40 2014 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Fri, 02 May 2014 21:38:40 +0200 Subject: RFR: 8027643 - Merge GenCollectorPolicy and TwoGenerationCollectorPolicy In-Reply-To: <21347.49745.501768.48858@mykonos.us.oracle.com> References: <53602775.4090400@oracle.com> <21347.49745.501768.48858@mykonos.us.oracle.com> Message-ID: <5363F440.8090002@oracle.com> Thanks John! /Jesper John Coomes skrev 2/5/14 18:05: > Jesper Wilhelmsson (jesper.wilhelmsson at oracle.com) wrote: >> Hi, >> >> This is another step in cleaning up the collector policy code. It is the first >> of a few related patches. This change merges the GenCollectorPolicy class with >> TwoGenerationalCollectorPolicy. >> >> I have deliberately refrained from doing any other cleanups that is made >> possible by this change. That will come in later changes. That makes this change >> really easy to review :-) > > I, for one, heartily appreciate it :-). > >> Webrev: http://cr.openjdk.java.net/~jwilhelm/8027643/webrev/ >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8027643 > > Looks good to me. > > -John > From per.liden at oracle.com Mon May 5 08:24:38 2014 From: per.liden at oracle.com (Per Liden) Date: Mon, 05 May 2014 10:24:38 +0200 Subject: RFR(XS): G1: Concurrent mark stuck in loop calling os::elapsedVTime() In-Reply-To: <5361874E.1080504@oracle.com> References: <5360F6E4.6040801@oracle.com> <5361874E.1080504@oracle.com> Message-ID: <53674AC6.5060400@oracle.com> Hi Jon, On 05/01/2014 01:29 AM, Jon Masamitsu wrote: > Per, > > Are you saying that when the heap expands, "_heap_end" might not be > updated but > "_finger" will be updated for an allocation in the newly expanded part > of the heap? Yes, that's correct. The exact scenario when this happens tends to be this: 1) Heap has N regions, where N is below max heap size so there's still room for expansion. 2) An humongous allocation of two regions is done. 3) In this particular case, to satisfy the allocation the heap need to be expanded by one region, and the humongous allocation ends up at [N-1, N+1). 4) At this point _heap_end still points to N, but the heap size is actually N+1. 5) When CM claims the humongous region starting at N-1 the _finger will be bumped to N+1, and _finger is now past _heap_end. /Per > > Jon > > On 4/30/2014 6:13 AM, Per Liden wrote: >> Hi, >> >> Here's another G1 concurrent mark fix I'd like to have reviewed. >> >> Summary: If the heap is expended when a concurrent mark is in progress >> there's a window where the global marking finger (_cm->_finger) points >> past the heap end (_cm->_heap_end). When this happens the marking >> threads will get stuck in a loop spinning, because >> _cm->out_of_regions() will still return false. out_of_regions() is >> currently implemented as "return _finger == _heap_end;". >> >> This patch fixes this problem by adjusting out_of_region() to instead >> be "return _finger >= _heap_end;". This is safe because objects in >> those new regions will be considered live anyway, because their TAMS >> will be equal to bottom. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8040804 >> Webrev: http://cr.openjdk.java.net/~pliden/8040804/webrev.0/ >> >> /Per > From per.liden at oracle.com Mon May 5 08:25:25 2014 From: per.liden at oracle.com (Per Liden) Date: Mon, 05 May 2014 10:25:25 +0200 Subject: RFR(XS): G1: Concurrent mark stuck in loop calling os::elapsedVTime() In-Reply-To: <1398874191.2766.0.camel@cirrus> References: <5360F6E4.6040801@oracle.com> <1398874191.2766.0.camel@cirrus> Message-ID: <53674AF5.7080303@oracle.com> Thanks Bengt and Thomas! /Per On 04/30/2014 06:09 PM, Thomas Schatzl wrote: > Hi, > > On Wed, 2014-04-30 at 15:13 +0200, Per Liden wrote: >> Hi, >> >> Here's another G1 concurrent mark fix I'd like to have reviewed. >> >> Summary: If the heap is expended when a concurrent mark is in progress >> there's a window where the global marking finger (_cm->_finger) points >> past the heap end (_cm->_heap_end). When this happens the marking >> threads will get stuck in a loop spinning, because _cm->out_of_regions() >> will still return false. out_of_regions() is currently implemented as >> "return _finger == _heap_end;". >> >> This patch fixes this problem by adjusting out_of_region() to instead be >> "return _finger >= _heap_end;". This is safe because objects in those >> new regions will be considered live anyway, because their TAMS will be >> equal to bottom. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8040804 >> Webrev: http://cr.openjdk.java.net/~pliden/8040804/webrev.0/ >> > > Looks good. > > Thomas > > From per.liden at oracle.com Mon May 5 11:57:52 2014 From: per.liden at oracle.com (Per Liden) Date: Mon, 05 May 2014 13:57:52 +0200 Subject: RFR(S): 8040803: G1: Concurrent mark hangs when mark stack overflows In-Reply-To: <53613848.2000409@oracle.com> References: <5360F4CB.9040106@oracle.com> <53613848.2000409@oracle.com> Message-ID: <53677CC0.6000300@oracle.com> Hi Jon, On 04/30/2014 07:52 PM, Jon Masamitsu wrote: > Per, > > Adding a new flag sometimes is like adding a new degree > of freedom and sometimes can make a complicated situation > more complicated. > > Before I review this can you help me understand the > problem. Is the window for the race condition this > code in do_marking_step()? > > 4108 if (_cm->has_overflown()) { > 4109 // This can happen if the mark stack overflows during a GC > pause > 4110 // and this task, after a yield point, restarts. We have to > abort > 4111 // as we need to get into the overflow protocol which happens > 4112 // right at the end of this task. > 4113 set_has_aborted(); > 4114 } > > The window being between the time _has_overflown is set and when > _has_aborted is set? The race is between checking _cm->has_overflown() and checking _cm->has_aborted(). Both of these are checked in a few places during marking (typically in regular_clock_call() and some other place). Since this code is executed by several threads in parallel, without synchronization, different threads can see one or the other state first depending on where a particular thread happens to be executing when the abort and overflow happens. Note that the set_has_aborted() in the code above sets the CMTask local abort state, which is not part of the race here. _cm->has_aborted() is the global abort state, which is set when a Full GC happens. /Per > > Jon > > On 4/30/2014 6:04 AM, Per Liden wrote: >> Hi, >> >> Could I please have a couple of reviews in this bug fix: >> >> Summary: G1's concurrent marking can potentially hang forever if the >> global mark stack overflows and immediately after that a Full GC >> happens, which tries to abort the marking. The reason is that there's >> a race between detecting the overflow situation and detecting the >> abort signal. Threads detecting the overflow situation first will go >> into the overflow protocol and wait on a barrier for all threads to >> reach this state. However, threads detecting the abort signal first >> will terminate and never participate in the barrier. >> >> This patch introduces an abort state and function on the >> WorkGangBarrierSync class, to unblock any threads waiting for the >> barrier to complete when the concurrent mark is aborted. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8040803 >> Webrev: http://cr.openjdk.java.net/~pliden/8040803/webrev.0/ >> >> /Per > From bengt.rutisson at oracle.com Mon May 5 11:52:00 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Mon, 05 May 2014 13:52:00 +0200 Subject: Request for review (s) - 8031323: Optionally align objects copied to survivor spaces In-Reply-To: <5363D517.80200@oracle.com> References: <53267CBB.9020804@oracle.com> <53284A7E.9060106@oracle.com> <5363D517.80200@oracle.com> Message-ID: <53677B60.6030709@oracle.com> Hi Jon, On 2014-05-02 19:25, Jon Masamitsu wrote: > New webrev with problems addressed except as noted below. > > http://cr.openjdk.java.net/~jmasa/8031323/webrev.01/ To make it compile I had to remove the inline keyword from the declaration of CollectedHeap::align_allocation_or_fail(): diff --git a/src/share/vm/gc_interface/collectedHeap.hpp b/src/share/vm/gc_interface/collectedHeap.hpp --- a/src/share/vm/gc_interface/collectedHeap.hpp +++ b/src/share/vm/gc_interface/collectedHeap.hpp @@ -353,7 +353,7 @@ // Return the address "addr" aligned by "alignment_in_bytes" if such // an address is below "end". Return NULL otherwise. - inline static HeapWord* align_allocation_or_fail(HeapWord* addr, + static HeapWord* align_allocation_or_fail(HeapWord* addr, HeapWord* end, unsigned short alignment_in_bytes); The includes for the usages of CollectedHeap::align_allocation_or_fail() need to be updated. space.cpp should include collectedHeap.inline.hpp. psPromotionLAB.hpp is using CollectedHeap::align_allocation_or_fail() and thus would need the include too, but we should not include inline files from hpp files, so you need to move PSYoungPromotionLAB::allocate() into psPromotionLAB.inline.hpp and have that include collectedHeap.inline.hpp. Similarly parGCAllocBuffer.hpp would need the include too since ParGCAllocBuffer::allocate_aligned() uses CollectedHeap::align_allocation_or_fail(). But again we should not include inline files in hpp files so ParGCAllocBuffer::allocate_aligned() should probably be moved in to the cpp file. Or, if we are worried about performance regressions we need to introduce a parGCAllocBuffer.inline.hpp file and move it there. In CollectedHeap::align_allocation_or_fail(), I think we can replace this: if(new_addr > addr && new_addr < end) { with: assert(new_addr > addr, "some good err_msg"); if(new_addr < end) { In ContiguousSpace::allocate_aligned() I think we can make the assert for the alignment of obj a bit stronger. Now we do: assert(is_aligned(obj) && is_aligned(new_top), "checking alignment"); But we have just gone through some trouble to make sure that obj is aligned to SurvivorAlignmentInBytes, so I think we should assert that rather than just the 8 byte alignment that Space::is_aligned() gives us. PSYoungPromotionLAB::allocate() and ContiguousSpace::allocate_aligned() is part of the code duplication that you plan on fixing with JDK-8042321. I'm fine with fixing that in a separate change, but it would be nice if we can make these methods look more similar. Right now they use different return patterns. PSYoungPromotionLAB::allocate() uses: if (allocation worked) { return obj; } ... return NULL; But ContiguousSpace::allocate_aligned() uses an else statement: if (allocation worked) { return obj; } else { ... return NULL; } Similarly to the comment about the assert in ContiguousSpace::allocate_aligned() I think the assert in PSYoungPromotionLAB::allocate() can be strengthened to assert that obj is SurvivorAlignmentInBytes aligned. A minor thing about oop.pcgc.inline.hpp -// Used by parallel old GC. +// forward_to_atomic() is used by parallel old and G1 GCs. You updated this comment to include G1, but the method is actually also used by ParNew (see ParNewGeneration::copy_to_survivor_space_avoiding_promotion_undo()) and I don't find any usages in ParallelOld. I would suggest to just get rid of the comment. It is not important to have a comment saying who uses it. Another very minor thing. The extra line feed added around line 1200 in parNewGeneration.cpp does not seem related to this change: http://cr.openjdk.java.net/~jmasa/8031323/webrev.01/src/share/vm/gc_implementation/parNew/parNewGeneration.cpp.cdiff.html Testing. There are tests for this feature coming, right? What kind of testing have you done so far on the current patch? Thanks, Bengt > > Thanks. > > > On 03/18/2014 06:30 AM, Bengt Rutisson wrote: >> >> Hi Jon, >> >> On 2014-03-17 05:40, Jon Masamitsu wrote: >>> 8031323: Optionally align objects copied to survivor spaces >>> >>> Add the option to specify an alignment for allocations into the >>> survivor spaces. Survivor space allocations will overflow if the >>> survivor spaces have not been increased. The expected use >>> is to align survivor space allocations to cache line sizes. >>> >>> http://cr.openjdk.java.net/~jmasa/8031323/webrev.00/ >> >> For G1 it looks to me like we are aligning objects in the old regions >> as well. I thought the intent was to only align in the survivor >> regions. Am I missing something? > > Yes, that was an error. Fixed in the new webrev. > >> >> Note that it is not enough to check the alloc purpose for G1 since >> G1CollectedHeap::par_allocate_during_gc() only takes the purpose as a >> hint and may still return buffers from other regions than expected >> for a purpose. But I guess that in this case it is not too important >> to get it exactly right, so it may be enough to check the alloc >> purpose value. > > I did use the purpose and there are cases where alignments in the > survivor spaces > can be missed. I think the number missed is a very small percentage > of the total > so I'm going to let those slip by. I thought that preferable to the > more extensive > coding changes it would take to catch them. >> >> >> The code in PSYoungPromotionLAB::allcoate(), >> ParGCAllocBuffer::allocate_aligned() and >> ContiguousSpace::allocate_aligned() is very similar. Is there a way >> to consolidate these methods? >> > I looked at this but did not come up with a suitable way to fix it. > I've opened > CR to track the issue. > > https://bugs.openjdk.java.net/browse/JDK-8042321 > > >> >> The code in CollectedHeap::align_allocation_or_fail() looks quite >> different from other places in the code base where we do padding. >> What do you think about doing something like this instead? >> >> inline HeapWord* CollectedHeap::align_allocation_or_fail(HeapWord* >> addr, HeapWord* end, unsigned short alignment_in_bytes) { >> if (alignment_in_bytes <= ObjectAlignmentInBytes) { >> return addr; >> } >> >> assert(is_ptr_aligned(addr, HeapWordSize), ""); >> assert(is_size_aligned(alignment_in_bytes, HeapWordSize), ""); >> >> HeapWord* new_addr = (HeapWord*) align_pointer_up(addr, >> alignment_in_bytes); >> size_t padding = pointer_delta(new_addr, addr); >> https://bugs.openjdk.java.net/browse/JDK-8042321 >> if (padding == 0) { >> return addr; >> } >> >> if (padding < CollectedHeap::min_fill_size()) { >> padding += alignment_in_bytes / HeapWordSize; >> assert(padding >= CollectedHeap::min_fill_size(), ""); >> new_addr = addr + padding; >> } >> >> if(new_addr > addr && new_addr < end) { >> CollectedHeap::fill_with_object(addr, padding); >> return new_addr; >> } else { >> return NULL; >> } >> } > > Fixed that as suggested. > > Jon >> >> Thanks, >> Bengt >> >>> >>> Jon >> > From per.liden at oracle.com Mon May 5 13:27:27 2014 From: per.liden at oracle.com (Per Liden) Date: Mon, 05 May 2014 15:27:27 +0200 Subject: RFR(S): 8040803: G1: Concurrent mark hangs when mark stack overflows In-Reply-To: <53635872.4030107@oracle.com> References: <5360F4CB.9040106@oracle.com> <53635872.4030107@oracle.com> Message-ID: <536791BF.4050408@oracle.com> Hi Bengt, On 05/02/2014 10:33 AM, Bengt Rutisson wrote: > > Hi Per, > > On 2014-04-30 15:04, Per Liden wrote: >> Hi, >> >> Could I please have a couple of reviews in this bug fix: >> >> Summary: G1's concurrent marking can potentially hang forever if the >> global mark stack overflows and immediately after that a Full GC >> happens, which tries to abort the marking. The reason is that there's >> a race between detecting the overflow situation and detecting the >> abort signal. Threads detecting the overflow situation first will go >> into the overflow protocol and wait on a barrier for all threads to >> reach this state. However, threads detecting the abort signal first >> will terminate and never participate in the barrier. >> >> This patch introduces an abort state and function on the >> WorkGangBarrierSync class, to unblock any threads waiting for the >> barrier to complete when the concurrent mark is aborted. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8040803 >> Webrev: http://cr.openjdk.java.net/~pliden/8040803/webrev.0/ > > The change looks good. Thanks for looking at the patch Bengt. > > One minor improvement would be to try to use an early exit in > ConcurrentMark::enter_first_sync_barrier() when the barrier fails. Here's an updated webrev with early exit: http://cr.openjdk.java.net/~pliden/8040803/webrev.1/ /Per > > Thanks, > Bengt > >> >> /Per > From andreas.sjoberg at oracle.com Mon May 5 13:33:39 2014 From: andreas.sjoberg at oracle.com (=?ISO-8859-1?Q?Andreas_Sj=F6berg?=) Date: Mon, 05 May 2014 15:33:39 +0200 Subject: RFR (XXS): JDK-8026784 Error message in AdaptiveFreeList::verify_stats is wrong Message-ID: <53679333.9090003@oracle.com> Hi all, I'd like to get a review for this tiny fix of an error message in the CMS adaptive free list verification code. Bug: https://bugs.openjdk.java.net/browse/JDK-8026784 Webrev: http://cr.openjdk.java.net/~ehelin/andreas/8026784/webrev.00/ Testing: jprt[1] Thanks. / Andreas [1] http://sthjprt.se.oracle.com/archives/2014/05/2014-05-05-122946.ansjob.hotspot//JobStatus.txt From stefan.karlsson at oracle.com Mon May 5 13:35:40 2014 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 05 May 2014 15:35:40 +0200 Subject: RFR (XXS): JDK-8026784 Error message in AdaptiveFreeList::verify_stats is wrong In-Reply-To: <53679333.9090003@oracle.com> References: <53679333.9090003@oracle.com> Message-ID: <536793AC.2040606@oracle.com> On 2014-05-05 15:33, Andreas Sj?berg wrote: > Hi all, > > I'd like to get a review for this tiny fix of an error message in the > CMS adaptive free list verification code. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8026784 > > Webrev: http://cr.openjdk.java.net/~ehelin/andreas/8026784/webrev.00/ > Looks good. StefanK > > Testing: jprt[1] > > Thanks. > > / Andreas > > [1] > http://sthjprt.se.oracle.com/archives/2014/05/2014-05-05-122946.ansjob.hotspot//JobStatus.txt From thomas.schatzl at oracle.com Mon May 5 13:45:45 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 05 May 2014 15:45:45 +0200 Subject: RFR (XXS): JDK-8026784 Error message in AdaptiveFreeList::verify_stats is wrong In-Reply-To: <53679333.9090003@oracle.com> References: <53679333.9090003@oracle.com> Message-ID: <1399297545.2567.18.camel@cirrus> Hi, On Mon, 2014-05-05 at 15:33 +0200, Andreas Sj?berg wrote: > Hi all, > > I'd like to get a review for this tiny fix of an error message in the > CMS adaptive free list verification code. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8026784 > > Webrev: http://cr.openjdk.java.net/~ehelin/andreas/8026784/webrev.00/ > > > Testing: jprt[1] Looks good. Thomas From erik.helin at oracle.com Mon May 5 14:11:34 2014 From: erik.helin at oracle.com (Erik Helin) Date: Mon, 05 May 2014 16:11:34 +0200 Subject: RFR (XXS): JDK-8026784 Error message in AdaptiveFreeList::verify_stats is wrong In-Reply-To: <53679333.9090003@oracle.com> References: <53679333.9090003@oracle.com> Message-ID: <53679C16.1030609@oracle.com> Looks good. Thanks, Erik On 2014-05-05 15:33, Andreas Sj?berg wrote: > Hi all, > > I'd like to get a review for this tiny fix of an error message in the > CMS adaptive free list verification code. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8026784 > > Webrev: http://cr.openjdk.java.net/~ehelin/andreas/8026784/webrev.00/ > > > Testing: jprt[1] > > Thanks. > > / Andreas > > [1] > http://sthjprt.se.oracle.com/archives/2014/05/2014-05-05-122946.ansjob.hotspot//JobStatus.txt > From bengt.rutisson at oracle.com Mon May 5 16:11:29 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Mon, 5 May 2014 18:11:29 +0200 Subject: RFR(S): 8040803: G1: Concurrent mark hangs when mark stack overflows In-Reply-To: <536791BF.4050408@oracle.com> References: <5360F4CB.9040106@oracle.com> <53635872.4030107@oracle.com> <536791BF.4050408@oracle.com> Message-ID: <046ED8B9-DA6D-4014-987F-584B849935D3@oracle.com> Hi Per, Looks good. Bengt > 5 maj 2014 kl. 15:27 skrev Per Liden : > > Hi Bengt, > >> On 05/02/2014 10:33 AM, Bengt Rutisson wrote: >> >> Hi Per, >> >>> On 2014-04-30 15:04, Per Liden wrote: >>> Hi, >>> >>> Could I please have a couple of reviews in this bug fix: >>> >>> Summary: G1's concurrent marking can potentially hang forever if the >>> global mark stack overflows and immediately after that a Full GC >>> happens, which tries to abort the marking. The reason is that there's >>> a race between detecting the overflow situation and detecting the >>> abort signal. Threads detecting the overflow situation first will go >>> into the overflow protocol and wait on a barrier for all threads to >>> reach this state. However, threads detecting the abort signal first >>> will terminate and never participate in the barrier. >>> >>> This patch introduces an abort state and function on the >>> WorkGangBarrierSync class, to unblock any threads waiting for the >>> barrier to complete when the concurrent mark is aborted. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8040803 >>> Webrev: http://cr.openjdk.java.net/~pliden/8040803/webrev.0/ >> >> The change looks good. > > Thanks for looking at the patch Bengt. > >> >> One minor improvement would be to try to use an early exit in >> ConcurrentMark::enter_first_sync_barrier() when the barrier fails. > > Here's an updated webrev with early exit: > > http://cr.openjdk.java.net/~pliden/8040803/webrev.1/ > > /Per > >> >> Thanks, >> Bengt >> >>> >>> /Per > From John.Coomes at oracle.com Mon May 5 16:14:40 2014 From: John.Coomes at oracle.com (John Coomes) Date: Mon, 5 May 2014 09:14:40 -0700 Subject: RFR (L) 8034246: remove CMS and ParNew adaptive size policy code Message-ID: <21351.47344.517204.861951@mykonos.us.oracle.com> I'd appreciate reviews of this change to delete unused adaptive size policy code for cms and parnew. It's big in terms of line count, but most comes from the deletion of 6 files. http://cr.openjdk.java.net/~jcoomes/8034246-rm-cms-asp/ -John From jon.masamitsu at oracle.com Mon May 5 17:29:08 2014 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Mon, 05 May 2014 10:29:08 -0700 Subject: RFR(XS): G1: Concurrent mark stuck in loop calling os::elapsedVTime() In-Reply-To: <53674AC6.5060400@oracle.com> References: <5360F6E4.6040801@oracle.com> <5361874E.1080504@oracle.com> <53674AC6.5060400@oracle.com> Message-ID: <5367CA64.8060000@oracle.com> On 05/05/2014 01:24 AM, Per Liden wrote: > Hi Jon, > > On 05/01/2014 01:29 AM, Jon Masamitsu wrote: >> Per, >> >> Are you saying that when the heap expands, "_heap_end" might not be >> updated but >> "_finger" will be updated for an allocation in the newly expanded part >> of the heap? > > Yes, that's correct. The exact scenario when this happens tends to be > this: > > 1) Heap has N regions, where N is below max heap size so there's still > room for expansion. > 2) An humongous allocation of two regions is done. > 3) In this particular case, to satisfy the allocation the heap need to > be expanded by one region, and the humongous allocation ends up at > [N-1, N+1). > 4) At this point _heap_end still points to N, but the heap size is > actually N+1. Can that be fixed? Jon > 5) When CM claims the humongous region starting at N-1 the _finger > will be bumped to N+1, and _finger is now past _heap_end. > > /Per > >> >> Jon >> >> On 4/30/2014 6:13 AM, Per Liden wrote: >>> Hi, >>> >>> Here's another G1 concurrent mark fix I'd like to have reviewed. >>> >>> Summary: If the heap is expended when a concurrent mark is in progress >>> there's a window where the global marking finger (_cm->_finger) points >>> past the heap end (_cm->_heap_end). When this happens the marking >>> threads will get stuck in a loop spinning, because >>> _cm->out_of_regions() will still return false. out_of_regions() is >>> currently implemented as "return _finger == _heap_end;". >>> >>> This patch fixes this problem by adjusting out_of_region() to instead >>> be "return _finger >= _heap_end;". This is safe because objects in >>> those new regions will be considered live anyway, because their TAMS >>> will be equal to bottom. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8040804 >>> Webrev: http://cr.openjdk.java.net/~pliden/8040804/webrev.0/ >>> >>> /Per >> > From jesper.wilhelmsson at oracle.com Tue May 6 00:24:32 2014 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Tue, 6 May 2014 02:24:32 +0200 Subject: RFR (L) 8034246: remove CMS and ParNew adaptive size policy code In-Reply-To: <21351.47344.517204.861951@mykonos.us.oracle.com> References: <21351.47344.517204.861951@mykonos.us.oracle.com> Message-ID: <9F088E4C-99E2-4556-8E18-A8B3FC4A3CE2@oracle.com> Looks good. Thanks for cleaning this up! /Jesper > 5 maj 2014 kl. 18:14 skrev John Coomes : > > I'd appreciate reviews of this change to delete unused adaptive size > policy code for cms and parnew. It's big in terms of line count, but > most comes from the deletion of 6 files. > > http://cr.openjdk.java.net/~jcoomes/8034246-rm-cms-asp/ > > -John From John.Coomes at oracle.com Tue May 6 01:35:04 2014 From: John.Coomes at oracle.com (John Coomes) Date: Mon, 5 May 2014 18:35:04 -0700 Subject: RFR (L) 8034246: remove CMS and ParNew adaptive size policy code In-Reply-To: <9F088E4C-99E2-4556-8E18-A8B3FC4A3CE2@oracle.com> References: <21351.47344.517204.861951@mykonos.us.oracle.com> <9F088E4C-99E2-4556-8E18-A8B3FC4A3CE2@oracle.com> Message-ID: <21352.15432.623240.370984@mykonos.us.oracle.com> Jesper Wilhelmsson (jesper.wilhelmsson at oracle.com) wrote: > Looks good. > Thanks for cleaning this up! Many thanks for taking a look. -John > > 5 maj 2014 kl. 18:14 skrev John Coomes : > > > > I'd appreciate reviews of this change to delete unused adaptive size > > policy code for cms and parnew. It's big in terms of line count, but > > most comes from the deletion of 6 files. > > > > http://cr.openjdk.java.net/~jcoomes/8034246-rm-cms-asp/ > > > > -John From thomas.schatzl at oracle.com Tue May 6 08:45:12 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 06 May 2014 10:45:12 +0200 Subject: RFR (L) 8034246: remove CMS and ParNew adaptive size policy code In-Reply-To: <21351.47344.517204.861951@mykonos.us.oracle.com> References: <21351.47344.517204.861951@mykonos.us.oracle.com> Message-ID: <1399365912.2595.12.camel@cirrus> Hi John, On Mon, 2014-05-05 at 09:14 -0700, John Coomes wrote: > I'd appreciate reviews of this change to delete unused adaptive size > policy code for cms and parnew. It's big in terms of line count, but > most comes from the deletion of 6 files. > > http://cr.openjdk.java.net/~jcoomes/8034246-rm-cms-asp/ > > -John some notes: - the GCPauseTimer class can be removed now too I think. This change removes its last uses. - in CMSCollector::reset(), I am not sure why the AdaptiveSizePolicyOutput use has been removed, and the change in AdaptiveSizePolicyOutput::print_test(). Same in GenCollectedHeap::do_collection(). If that is intendeded, can we remove the local "sp" variable in CMSCollector::reset(), i.e. inline it? It's only used once quite far below. That's just a suggestion, I am fine with either way. Otherwise looks good. Thanks, Thomas From mikael.gerdin at oracle.com Tue May 6 08:52:08 2014 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Tue, 06 May 2014 10:52:08 +0200 Subject: RFR (L) 8034246: remove CMS and ParNew adaptive size policy code In-Reply-To: <21351.47344.517204.861951@mykonos.us.oracle.com> References: <21351.47344.517204.861951@mykonos.us.oracle.com> Message-ID: <1598836.bStRTXLZya@mgerdin03> John, On Monday 05 May 2014 09.14.40 John Coomes wrote: > I'd appreciate reviews of this change to delete unused adaptive size > policy code for cms and parnew. It's big in terms of line count, but > most comes from the deletion of 6 files. > > http://cr.openjdk.java.net/~jcoomes/8034246-rm-cms-asp/ Good to get rid of code that hasn't been working for a while. In src/share/vm/memory/collectorPolicy.cpp the include statement: #include "gc_implementation/shared/adaptiveSizePolicy.hpp" is already present a few lines up. CMSCollector::size_policy and ConcurrentMarkSweepGeneration::size_policy are basically identical now, do you want to fix this now or leave this as just a code removal change? (personally, I'm fine either way) Consider the change reviewed nevertheless. /Mikael > > -John From per.liden at oracle.com Tue May 6 08:57:19 2014 From: per.liden at oracle.com (Per Liden) Date: Tue, 06 May 2014 10:57:19 +0200 Subject: RFR(XS): G1: Concurrent mark stuck in loop calling os::elapsedVTime() In-Reply-To: <5367CA64.8060000@oracle.com> References: <5360F6E4.6040801@oracle.com> <5361874E.1080504@oracle.com> <53674AC6.5060400@oracle.com> <5367CA64.8060000@oracle.com> Message-ID: <5368A3EF.2050702@oracle.com> Jon, On 05/05/2014 07:29 PM, Jon Masamitsu wrote: > > On 05/05/2014 01:24 AM, Per Liden wrote: >> Hi Jon, >> >> On 05/01/2014 01:29 AM, Jon Masamitsu wrote: >>> Per, >>> >>> Are you saying that when the heap expands, "_heap_end" might not be >>> updated but >>> "_finger" will be updated for an allocation in the newly expanded part >>> of the heap? >> >> Yes, that's correct. The exact scenario when this happens tends to be >> this: >> >> 1) Heap has N regions, where N is below max heap size so there's still >> room for expansion. >> 2) An humongous allocation of two regions is done. >> 3) In this particular case, to satisfy the allocation the heap need to >> be expanded by one region, and the humongous allocation ends up at >> [N-1, N+1). >> 4) At this point _heap_end still points to N, but the heap size is >> actually N+1. > Can that be fixed? In theory that could be done, but we would have to introduce locking or other means of synchronization between the mark threads and the mutator expanding the heap. But more importantly, when marking starts we want a snapshot of the heap and we shouldn't have to adjust CM's view of the heap size after that. There's currently a function to update the heap size called update_g1_committed(). This is called when the heap expands during a GC safepoint. However, as Bengt pointed out, that function is superfluous and should be removed (he created an RFE to do that, JDK-8042097). /Per > > Jon > >> 5) When CM claims the humongous region starting at N-1 the _finger >> will be bumped to N+1, and _finger is now past _heap_end. >> >> /Per >> >>> >>> Jon >>> >>> On 4/30/2014 6:13 AM, Per Liden wrote: >>>> Hi, >>>> >>>> Here's another G1 concurrent mark fix I'd like to have reviewed. >>>> >>>> Summary: If the heap is expended when a concurrent mark is in progress >>>> there's a window where the global marking finger (_cm->_finger) points >>>> past the heap end (_cm->_heap_end). When this happens the marking >>>> threads will get stuck in a loop spinning, because >>>> _cm->out_of_regions() will still return false. out_of_regions() is >>>> currently implemented as "return _finger == _heap_end;". >>>> >>>> This patch fixes this problem by adjusting out_of_region() to instead >>>> be "return _finger >= _heap_end;". This is safe because objects in >>>> those new regions will be considered live anyway, because their TAMS >>>> will be equal to bottom. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8040804 >>>> Webrev: http://cr.openjdk.java.net/~pliden/8040804/webrev.0/ >>>> >>>> /Per >>> >> > From dmitry.fazunenko at oracle.com Tue May 6 10:58:48 2014 From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko) Date: Tue, 06 May 2014 14:58:48 +0400 Subject: test/.../testlibrary/DynamicVMOption.java - common lib for dynamic VM options Message-ID: <5368C068.8000400@oracle.com> Hi, Would please look at the new version of a common library for testing dynamic VM options: http://cr.openjdk.java.net/~iignatyev/dfazunenko/8039489/webrev.00/ Updated tests using this library will look like: http://cr.openjdk.java.net/~iignatyev/dfazunenko/8041358/webrev.00/ Any your comments will be much appreciated. Thanks, Dima From jon.masamitsu at oracle.com Wed May 7 03:01:58 2014 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Tue, 06 May 2014 20:01:58 -0700 Subject: Request for Review (xs) : 8038928 - gc/g1/TestGCLogMessages.java fail with "[Evacuation Failure' found" In-Reply-To: <535EC568.7060207@oracle.com> References: <535EC568.7060207@oracle.com> Message-ID: <5369A226.1020506@oracle.com> I now do not think there is a good enough way to save the part of test that tries to verify that no unexpected evacuation failure has occurred. Making it meaningful and reliable eludes me so I'm simply deleting that part of the test. http://cr.openjdk.java.net/~jmasa/8038928/webrev.01/ Thank you to those that helped me get to this point and thank you to those that put up with my first attempt. Jon On 4/28/2014 2:17 PM, Jon Masamitsu wrote: > The requirement that an evacuation failure not happen during this > test is based on the expected behavior of the GC and is not a > required behavior. In some instance the evacuation failure will > happen, but it is a not a GC failure if it does and is only an > unexpected path being followed. > > The test is not reliable but before removing it, I've made > some changes to try and save it. I've modified the > test to slow down the allocations and changed the allocation to > allocate smaller objects (which also has a side effect of slowing > allocations). The goal is to detect gross breakages of > evacuation failure while risking only very, very rare spurious > failures. > > I had reproduced the failure with the unmodified test and it > would fail within 30 minutes. With the modifications, I haven't > seen the failure in a day of testing. > > If the modifications don't work, I'll remove the test. > > http://cr.openjdk.java.net/~jmasa/8038928/webrev.00/ > > https://bugs.openjdk.java.net/browse/JDK-8038928 > > Thanks. > > Jon From John.Coomes at oracle.com Wed May 7 04:36:04 2014 From: John.Coomes at oracle.com (John Coomes) Date: Tue, 6 May 2014 21:36:04 -0700 Subject: RFR (L) 8034246: remove CMS and ParNew adaptive size policy code In-Reply-To: <1399365912.2595.12.camel@cirrus> References: <21351.47344.517204.861951@mykonos.us.oracle.com> <1399365912.2595.12.camel@cirrus> Message-ID: <21353.47156.497944.698473@mykonos.us.oracle.com> Thomas Schatzl (thomas.schatzl at oracle.com) wrote: > Hi John, > > On Mon, 2014-05-05 at 09:14 -0700, John Coomes wrote: > > I'd appreciate reviews of this change to delete unused adaptive size > > policy code for cms and parnew. It's big in terms of line count, but > > most comes from the deletion of 6 files. > > > > http://cr.openjdk.java.net/~jcoomes/8034246-rm-cms-asp/ > > > > -John > > some notes: > > - the GCPauseTimer class can be removed now too I think. This change > removes its last uses. Thanks for spotting that; I removed it. > - in CMSCollector::reset(), I am not sure why the > AdaptiveSizePolicyOutput use has been removed, and the change in > AdaptiveSizePolicyOutput::print_test(). Same in > GenCollectedHeap::do_collection(). The ASPOutput class was only active when AdaptiveSizePolicy and either ParallelGC or CMS was enabled. CMS forces AdaptiveSizePolicy off, so it is effectively ParallelGC only. > If that is intendeded, can we remove > the local "sp" variable in CMSCollector::reset(), i.e. inline it? It's > only used once quite far below. > That's just a suggestion, I am fine with either way. Sure, I removed sp. Updated webrev at the same location. > Otherwise looks good. Thanks! -John From John.Coomes at oracle.com Wed May 7 04:38:11 2014 From: John.Coomes at oracle.com (John Coomes) Date: Tue, 6 May 2014 21:38:11 -0700 Subject: RFR (L) 8034246: remove CMS and ParNew adaptive size policy code In-Reply-To: <1598836.bStRTXLZya@mgerdin03> References: <21351.47344.517204.861951@mykonos.us.oracle.com> <1598836.bStRTXLZya@mgerdin03> Message-ID: <21353.47283.610771.899532@mykonos.us.oracle.com> Mikael Gerdin (mikael.gerdin at oracle.com) wrote: > John, > > On Monday 05 May 2014 09.14.40 John Coomes wrote: > > I'd appreciate reviews of this change to delete unused adaptive size > > policy code for cms and parnew. It's big in terms of line count, but > > most comes from the deletion of 6 files. > > > > http://cr.openjdk.java.net/~jcoomes/8034246-rm-cms-asp/ > > Good to get rid of code that hasn't been working for a while. > > In src/share/vm/memory/collectorPolicy.cpp the include statement: > #include "gc_implementation/shared/adaptiveSizePolicy.hpp" > is already present a few lines up. Yes; fixed. > CMSCollector::size_policy and ConcurrentMarkSweepGeneration::size_policy are > basically identical now, do you want to fix this now or leave this as just a > code removal change? (personally, I'm fine either way) The ConcurrentMarkSweepGeneration one is no longer used, so I removed it. Updated webrev is at the same location. > Consider the change reviewed nevertheless. Thanks! -John From ysr1729 at gmail.com Wed May 7 06:37:53 2014 From: ysr1729 at gmail.com (Srinivas Ramakrishna) Date: Tue, 6 May 2014 23:37:53 -0700 Subject: RFR (L) 8034246: remove CMS and ParNew adaptive size policy code In-Reply-To: <21353.47283.610771.899532@mykonos.us.oracle.com> References: <21351.47344.517204.861951@mykonos.us.oracle.com> <1598836.bStRTXLZya@mgerdin03> <21353.47283.610771.899532@mykonos.us.oracle.com> Message-ID: Looks good to me too, and good that we are able to remove code that hasn't been used and seems unlikely now, with G1 available, to have the resources to make it work well. Please make sure that this has also been approved by Jon. Reviewed. -- Ramki ysr1729 > On May 6, 2014, at 21:38, John Coomes wrote: > > Mikael Gerdin (mikael.gerdin at oracle.com) wrote: >> John, >> >>> On Monday 05 May 2014 09.14.40 John Coomes wrote: >>> I'd appreciate reviews of this change to delete unused adaptive size >>> policy code for cms and parnew. It's big in terms of line count, but >>> most comes from the deletion of 6 files. >>> >>> http://cr.openjdk.java.net/~jcoomes/8034246-rm-cms-asp/ >> >> Good to get rid of code that hasn't been working for a while. >> >> In src/share/vm/memory/collectorPolicy.cpp the include statement: >> #include "gc_implementation/shared/adaptiveSizePolicy.hpp" >> is already present a few lines up. > > Yes; fixed. > >> CMSCollector::size_policy and ConcurrentMarkSweepGeneration::size_policy are >> basically identical now, do you want to fix this now or leave this as just a >> code removal change? (personally, I'm fine either way) > > The ConcurrentMarkSweepGeneration one is no longer used, so I removed > it. Updated webrev is at the same location. > >> Consider the change reviewed nevertheless. > > Thanks! > > -John From ysr1729 at gmail.com Wed May 7 06:39:26 2014 From: ysr1729 at gmail.com (Srinivas Ramakrishna) Date: Tue, 6 May 2014 23:39:26 -0700 Subject: Request for Review (xs) : 8038928 - gc/g1/TestGCLogMessages.java fail with "[Evacuation Failure' found" In-Reply-To: <5369A226.1020506@oracle.com> References: <535EC568.7060207@oracle.com> <5369A226.1020506@oracle.com> Message-ID: <271FD693-EB11-42FD-870A-04E33F34EE50@gmail.com> Looks good to me. ysr1729 > On May 6, 2014, at 20:01, Jon Masamitsu wrote: > > I now do not think there is a good enough way to save the > part of test that tries to verify that no unexpected evacuation > failure has occurred. Making it meaningful and reliable > eludes me so I'm simply deleting that part of the test. > > http://cr.openjdk.java.net/~jmasa/8038928/webrev.01/ > > Thank you to those that helped me get to this point and > thank you to those that put up with my first attempt. > > Jon > >> On 4/28/2014 2:17 PM, Jon Masamitsu wrote: >> The requirement that an evacuation failure not happen during this >> test is based on the expected behavior of the GC and is not a >> required behavior. In some instance the evacuation failure will >> happen, but it is a not a GC failure if it does and is only an >> unexpected path being followed. >> >> The test is not reliable but before removing it, I've made >> some changes to try and save it. I've modified the >> test to slow down the allocations and changed the allocation to >> allocate smaller objects (which also has a side effect of slowing >> allocations). The goal is to detect gross breakages of >> evacuation failure while risking only very, very rare spurious >> failures. >> >> I had reproduced the failure with the unmodified test and it >> would fail within 30 minutes. With the modifications, I haven't >> seen the failure in a day of testing. >> >> If the modifications don't work, I'll remove the test. >> >> http://cr.openjdk.java.net/~jmasa/8038928/webrev.00/ >> >> https://bugs.openjdk.java.net/browse/JDK-8038928 >> >> Thanks. >> >> Jon > From bengt.rutisson at oracle.com Wed May 7 08:34:57 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 07 May 2014 10:34:57 +0200 Subject: Request for Review (xs) : 8038928 - gc/g1/TestGCLogMessages.java fail with "[Evacuation Failure' found" In-Reply-To: <5369A226.1020506@oracle.com> References: <535EC568.7060207@oracle.com> <5369A226.1020506@oracle.com> Message-ID: <5369F031.3040903@oracle.com> Hi Jon, On 2014-05-07 05:01, Jon Masamitsu wrote: > I now do not think there is a good enough way to save the > part of test that tries to verify that no unexpected evacuation > failure has occurred. Making it meaningful and reliable > eludes me so I'm simply deleting that part of the test. > > http://cr.openjdk.java.net/~jmasa/8038928/webrev.01/ > > Thank you to those that helped me get to this point and > thank you to those that put up with my first attempt. I'm not sure I understand why it is not possible to make the test stable by just setting the young gen size and heap size appropriately. But I don't have strong opinions about the test. Maybe just removing the check is the simplest solution. It was Thomas who added the check for evacuation failure as part of this change: http://hg.openjdk.java.net/jdk9/hs-gc/hotspot/rev/70a6a3c4cc3e If Thomas is fine with just removing this check, I'm fine with it too. Thomas? Thanks, Bengt > > Jon > > On 4/28/2014 2:17 PM, Jon Masamitsu wrote: >> The requirement that an evacuation failure not happen during this >> test is based on the expected behavior of the GC and is not a >> required behavior. In some instance the evacuation failure will >> happen, but it is a not a GC failure if it does and is only an >> unexpected path being followed. >> >> The test is not reliable but before removing it, I've made >> some changes to try and save it. I've modified the >> test to slow down the allocations and changed the allocation to >> allocate smaller objects (which also has a side effect of slowing >> allocations). The goal is to detect gross breakages of >> evacuation failure while risking only very, very rare spurious >> failures. >> >> I had reproduced the failure with the unmodified test and it >> would fail within 30 minutes. With the modifications, I haven't >> seen the failure in a day of testing. >> >> If the modifications don't work, I'll remove the test. >> >> http://cr.openjdk.java.net/~jmasa/8038928/webrev.00/ >> >> https://bugs.openjdk.java.net/browse/JDK-8038928 >> >> Thanks. >> >> Jon > From kirk at kodewerk.com Wed May 7 11:03:56 2014 From: kirk at kodewerk.com (Kirk Pepperdine) Date: Wed, 7 May 2014 13:03:56 +0200 Subject: RFR (L) 8034246: remove CMS and ParNew adaptive size policy code In-Reply-To: References: <21351.47344.517204.861951@mykonos.us.oracle.com> <1598836.bStRTXLZya@mgerdin03> <21353.47283.610771.899532@mykonos.us.oracle.com> Message-ID: <876494FC-D3B6-43BF-B0D2-830B509646F1@kodewerk.com> Hi, So my understanding correct (without digging into the code) is that CMS no longer uses any adaptive sizing? In the past I?ve provided charts that demonstrated adaptive sizing shrank memory in response to lower object creation rates. All well and good but in these cases the lower object creation rates were a result of increased frequency and duration of collections which should have resulted in a larger heap! There are/were a number of other degenerative cases where top of heap aggressively tracked top of live set causing all kinds of grief so backing off of this implementation seems reasonable, especially if there is little chance of it being fixed. Regards, Kirk On May 7, 2014, at 8:37 AM, Srinivas Ramakrishna wrote: > > Looks good to me too, and good that we are able to remove code that hasn't been used and seems unlikely now, with G1 available, to have the resources to make it work well. Please make sure that this has also been approved by Jon. > > Reviewed. > -- Ramki > > ysr1729 > >> On May 6, 2014, at 21:38, John Coomes wrote: >> >> Mikael Gerdin (mikael.gerdin at oracle.com) wrote: >>> John, >>> >>>> On Monday 05 May 2014 09.14.40 John Coomes wrote: >>>> I'd appreciate reviews of this change to delete unused adaptive size >>>> policy code for cms and parnew. It's big in terms of line count, but >>>> most comes from the deletion of 6 files. >>>> >>>> http://cr.openjdk.java.net/~jcoomes/8034246-rm-cms-asp/ >>> >>> Good to get rid of code that hasn't been working for a while. >>> >>> In src/share/vm/memory/collectorPolicy.cpp the include statement: >>> #include "gc_implementation/shared/adaptiveSizePolicy.hpp" >>> is already present a few lines up. >> >> Yes; fixed. >> >>> CMSCollector::size_policy and ConcurrentMarkSweepGeneration::size_policy are >>> basically identical now, do you want to fix this now or leave this as just a >>> code removal change? (personally, I'm fine either way) >> >> The ConcurrentMarkSweepGeneration one is no longer used, so I removed >> it. Updated webrev is at the same location. >> >>> Consider the change reviewed nevertheless. >> >> Thanks! >> >> -John From bengt.rutisson at oracle.com Wed May 7 11:18:29 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 07 May 2014 13:18:29 +0200 Subject: RFR (M/L): JDK-8035400: Move G1ParScanThreadState into its own files In-Reply-To: <1397829156.2717.24.camel@cirrus> References: <1397829156.2717.24.camel@cirrus> Message-ID: <536A1685.7080704@oracle.com> Hi Thomas, On 2014-04-18 15:52, Thomas Schatzl wrote: > Hi all, > > can I have reviews for the above change? It moves G1ParScanThreadState > into G1ParScanThreadState*pp files. > > The only changes are limited to: > - adding a "#pragma warning( disable:4355 ) // 'this' : used in base > member initializer list" to shut visual C up about the problem (which > should be cleaned up at some point - I found an issue that slipped > through because of that, JDK-8040977) As I commented in the review of JDK-8040977 I would prefer to make the change to not pass this as a parameter to the constructor. That would also remove the need for disabling the warning. Maybe in that case base this review on top of the fix for JDK-8040977 rather than the other way around? > - added necessary include file references; I hope the AIX guys can > compile that change to avoid troubles. It compiles fine with all Oracle > supported archs. You also moved the definition of the destructor of G1ParScanThreadState from the hpp file to the cpp file. Makes sense, but was not strictly needed for this change, right? > > There will be another CR for fixing up visibility and cleaning up stuff > a little. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8035400 > > Webrev: > http://cr.openjdk.java.net/~tschatzl/8035400/webrev/ It is a bit hard to review moved code. But except for the comment regarding JDK-8040977 above I think it looks good. I think you can clean up the includes a bit more if you have time. Seems like these includes in g1CollectedHeap.cpp are for example not needed anymore: #include "oops/oop.inline.hpp" #include "oops/oop.pcgc.inline.hpp" Thanks, Bengt > > Testing: > perf testing indicated no changes, jprt > > Thanks, > Thomas > > > From John.Coomes at oracle.com Wed May 7 23:29:12 2014 From: John.Coomes at oracle.com (John Coomes) Date: Wed, 7 May 2014 16:29:12 -0700 Subject: RFR (L) 8034246: remove CMS and ParNew adaptive size policy code In-Reply-To: References: <21351.47344.517204.861951@mykonos.us.oracle.com> <1598836.bStRTXLZya@mgerdin03> <21353.47283.610771.899532@mykonos.us.oracle.com> Message-ID: <21354.49608.277624.35777@mykonos.us.oracle.com> Srinivas Ramakrishna (ysr1729 at gmail.com) wrote: > > Looks good to me too, and good that we are able to remove code that hasn't been used and seems unlikely now, with G1 available, to have the resources to make it work well. Please make sure that this has also been approved by Jon. Yes, Jon & I discussed this just the other day :-). > Reviewed. Thanks! -John > > On May 6, 2014, at 21:38, John Coomes wrote: > > > > Mikael Gerdin (mikael.gerdin at oracle.com) wrote: > >> John, > >> > >>> On Monday 05 May 2014 09.14.40 John Coomes wrote: > >>> I'd appreciate reviews of this change to delete unused adaptive size > >>> policy code for cms and parnew. It's big in terms of line count, but > >>> most comes from the deletion of 6 files. > >>> > >>> http://cr.openjdk.java.net/~jcoomes/8034246-rm-cms-asp/ > >> > >> Good to get rid of code that hasn't been working for a while. > >> > >> In src/share/vm/memory/collectorPolicy.cpp the include statement: > >> #include "gc_implementation/shared/adaptiveSizePolicy.hpp" > >> is already present a few lines up. > > > > Yes; fixed. > > > >> CMSCollector::size_policy and ConcurrentMarkSweepGeneration::size_policy are > >> basically identical now, do you want to fix this now or leave this as just a > >> code removal change? (personally, I'm fine either way) > > > > The ConcurrentMarkSweepGeneration one is no longer used, so I removed > > it. Updated webrev is at the same location. > > > >> Consider the change reviewed nevertheless. > > > > Thanks! > > > > -John -- John Coomes Oracle, MS USCA22-3?? john.coomes at oracle.com 4220 Network Circle 408-276-7048 Santa Clara, CA 95054-1778 *** Support GreenPeace and we'll all breathe easier. *** From John.Coomes at oracle.com Wed May 7 23:44:04 2014 From: John.Coomes at oracle.com (John Coomes) Date: Wed, 7 May 2014 16:44:04 -0700 Subject: RFR (L) 8034246: remove CMS and ParNew adaptive size policy code In-Reply-To: <876494FC-D3B6-43BF-B0D2-830B509646F1@kodewerk.com> References: <21351.47344.517204.861951@mykonos.us.oracle.com> <1598836.bStRTXLZya@mgerdin03> <21353.47283.610771.899532@mykonos.us.oracle.com> <876494FC-D3B6-43BF-B0D2-830B509646F1@kodewerk.com> Message-ID: <21354.50500.838480.593745@mykonos.us.oracle.com> Kirk Pepperdine (kirk at kodewerk.com) wrote: > Hi, > > So my understanding correct (without digging into the code) is that CMS no longer uses any adaptive sizing? > > In the past I?ve provided charts that demonstrated adaptive sizing shrank memory in response to lower object creation rates. All well and good but in these cases the lower object creation rates were a result of increased frequency and duration of collections which should have resulted in a larger heap! There are/were a number of other degenerative cases where top of heap aggressively tracked top of live set causing all kinds of grief so backing off of this implementation seems reasonable, especially if there is little chance of it being fixed. Hi Kirk, Actually, CMS will still use the same sizing code that it has for many years. The option -XX:+UseAdaptiveSizePolicy enabled code that was intended to do more "ergonomic" sizing of the survivor spaces, generations and tenuring threshold than is currently done, to reduce the need to tune those on the command line. However, the CMS implementation was incomplete and it was never enabled by default for CMS (it is enabled by default for ParallelGC, where it works well). Some time ago [1] UseAdaptiveSizePolicy had to be completely disabled for CMS (i.e., it is disabled even if you add -XX:+UseAdaptiveSizePolicy on the command line) because it would cause the VM to crash. So this change just removes code that cannot be executed. -John [1] https://bugs.openjdk.java.net/browse/JDK-7112413 > On May 7, 2014, at 8:37 AM, Srinivas Ramakrishna wrote: > > > > > Looks good to me too, and good that we are able to remove code that hasn't been used and seems unlikely now, with G1 available, to have the resources to make it work well. Please make sure that this has also been approved by Jon. > > > > Reviewed. > > -- Ramki > > > > ysr1729 > > > >> On May 6, 2014, at 21:38, John Coomes wrote: > >> > >> Mikael Gerdin (mikael.gerdin at oracle.com) wrote: > >>> John, > >>> > >>>> On Monday 05 May 2014 09.14.40 John Coomes wrote: > >>>> I'd appreciate reviews of this change to delete unused adaptive size > >>>> policy code for cms and parnew. It's big in terms of line count, but > >>>> most comes from the deletion of 6 files. > >>>> > >>>> http://cr.openjdk.java.net/~jcoomes/8034246-rm-cms-asp/ > >>> > >>> Good to get rid of code that hasn't been working for a while. > >>> > >>> In src/share/vm/memory/collectorPolicy.cpp the include statement: > >>> #include "gc_implementation/shared/adaptiveSizePolicy.hpp" > >>> is already present a few lines up. > >> > >> Yes; fixed. > >> > >>> CMSCollector::size_policy and ConcurrentMarkSweepGeneration::size_policy are > >>> basically identical now, do you want to fix this now or leave this as just a > >>> code removal change? (personally, I'm fine either way) > >> > >> The ConcurrentMarkSweepGeneration one is no longer used, so I removed > >> it. Updated webrev is at the same location. > >> > >>> Consider the change reviewed nevertheless. > >> > >> Thanks! > >> > >> -John > -- John Coomes Oracle, MS USCA22-3?? john.coomes at oracle.com 4220 Network Circle 408-276-7048 Santa Clara, CA 95054-1778 *** Support GreenPeace and we'll all breathe easier. *** From John.Coomes at oracle.com Wed May 7 23:51:43 2014 From: John.Coomes at oracle.com (John Coomes) Date: Wed, 7 May 2014 16:51:43 -0700 Subject: review request (S): 8042255 make gc src file exclusion more automatic In-Reply-To: <21345.40743.531779.25170@mykonos.us.oracle.com> References: <21345.40743.531779.25170@mykonos.us.oracle.com> Message-ID: <21354.50959.243762.185432@mykonos.us.oracle.com> John Coomes (John.Coomes at oracle.com) wrote: > Please review this small makefile change to reduce the need to update > the list of files in make/excludeSrc.make. > > http://cr.openjdk.java.net/~jcoomes/8042255-gc-exclude/ Anyone? This was motivated the removal of the CMS AdaptiveSizePolicy code [1]. If I can push this first, then I don't have to worry about updating excludeSrc.make. And neither will you, when you add or remove cms, g1 or parnew files in the future :-). -John [1] http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2014-May/010036.html From kirk at kodewerk.com Thu May 8 04:11:37 2014 From: kirk at kodewerk.com (Kirk Pepperdine) Date: Thu, 8 May 2014 06:11:37 +0200 Subject: RFR (L) 8034246: remove CMS and ParNew adaptive size policy code In-Reply-To: <21354.50500.838480.593745@mykonos.us.oracle.com> References: <21351.47344.517204.861951@mykonos.us.oracle.com> <1598836.bStRTXLZya@mgerdin03> <21353.47283.610771.899532@mykonos.us.oracle.com> <876494FC-D3B6-43BF-B0D2-830B509646F1@kodewerk.com> <21354.50500.838480.593745@mykonos.us.oracle.com> Message-ID: <0FE17FCE-4E0C-4986-9FB7-93344006FB3F@kodewerk.com> Thanks John. On May 8, 2014, at 1:44 AM, John Coomes wrote: > Kirk Pepperdine (kirk at kodewerk.com) wrote: >> Hi, >> >> So my understanding correct (without digging into the code) is that CMS no longer uses any adaptive sizing? >> >> In the past I?ve provided charts that demonstrated adaptive sizing shrank memory in response to lower object creation rates. All well and good but in these cases the lower object creation rates were a result of increased frequency and duration of collections which should have resulted in a larger heap! There are/were a number of other degenerative cases where top of heap aggressively tracked top of live set causing all kinds of grief so backing off of this implementation seems reasonable, especially if there is little chance of it being fixed. > > Hi Kirk, > > Actually, CMS will still use the same sizing code that it has for many > years. > > The option -XX:+UseAdaptiveSizePolicy enabled code that was intended > to do more "ergonomic" sizing of the survivor spaces, generations and > tenuring threshold than is currently done, to reduce the need to tune > those on the command line. However, the CMS implementation was > incomplete and it was never enabled by default for CMS (it is enabled > by default for ParallelGC, where it works well). > > Some time ago [1] UseAdaptiveSizePolicy had to be completely disabled > for CMS (i.e., it is disabled even if you add > -XX:+UseAdaptiveSizePolicy on the command line) because it would cause > the VM to crash. So this change just removes code that cannot be > executed. > > -John > > [1] https://bugs.openjdk.java.net/browse/JDK-7112413 > >> On May 7, 2014, at 8:37 AM, Srinivas Ramakrishna wrote: >> >>> >>> Looks good to me too, and good that we are able to remove code that hasn't been used and seems unlikely now, with G1 available, to have the resources to make it work well. Please make sure that this has also been approved by Jon. >>> >>> Reviewed. >>> -- Ramki >>> >>> ysr1729 >>> >>>> On May 6, 2014, at 21:38, John Coomes wrote: >>>> >>>> Mikael Gerdin (mikael.gerdin at oracle.com) wrote: >>>>> John, >>>>> >>>>>> On Monday 05 May 2014 09.14.40 John Coomes wrote: >>>>>> I'd appreciate reviews of this change to delete unused adaptive size >>>>>> policy code for cms and parnew. It's big in terms of line count, but >>>>>> most comes from the deletion of 6 files. >>>>>> >>>>>> http://cr.openjdk.java.net/~jcoomes/8034246-rm-cms-asp/ >>>>> >>>>> Good to get rid of code that hasn't been working for a while. >>>>> >>>>> In src/share/vm/memory/collectorPolicy.cpp the include statement: >>>>> #include "gc_implementation/shared/adaptiveSizePolicy.hpp" >>>>> is already present a few lines up. >>>> >>>> Yes; fixed. >>>> >>>>> CMSCollector::size_policy and ConcurrentMarkSweepGeneration::size_policy are >>>>> basically identical now, do you want to fix this now or leave this as just a >>>>> code removal change? (personally, I'm fine either way) >>>> >>>> The ConcurrentMarkSweepGeneration one is no longer used, so I removed >>>> it. Updated webrev is at the same location. >>>> >>>>> Consider the change reviewed nevertheless. >>>> >>>> Thanks! >>>> >>>> -John >> > > -- > John Coomes Oracle, MS USCA22-3?? > john.coomes at oracle.com 4220 Network Circle > 408-276-7048 Santa Clara, CA 95054-1778 > *** Support GreenPeace and we'll all breathe easier. *** From andreas.sjoberg at oracle.com Thu May 8 08:01:20 2014 From: andreas.sjoberg at oracle.com (=?ISO-8859-1?Q?Andreas_Sj=F6berg?=) Date: Thu, 08 May 2014 10:01:20 +0200 Subject: RFR(S): JDK-8042474: Clean up duplicated code in RSHashTable Message-ID: <536B39D0.1070801@oracle.com> Hi all, could I please have reviews for this small cleanup? For RSHashTable, it - removes entry_for_region_ind(RegionIdx_t) - changes get_entry to be public and declared const - changes the sites that called entry_for_region to instead use get_entry, since they have the same implementation. - changes get_cards to make use of get_entry instead of very similar code contained inline. webrev: http://cr.openjdk.java.net/~brutisso/JDK-8042474/01/webrev/ It seems get_cards is not used. Maybe it should be removed as a separate CR? testing: jprt. Best regards, Andreas -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.rutisson at oracle.com Thu May 8 08:20:41 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Thu, 08 May 2014 10:20:41 +0200 Subject: RFR(S): JDK-8042474: Clean up duplicated code in RSHashTable In-Reply-To: <536B39D0.1070801@oracle.com> References: <536B39D0.1070801@oracle.com> Message-ID: <536B3E59.6050203@oracle.com> Hi Andreas, On 2014-05-08 10:01, Andreas Sj?berg wrote: > Hi all, > > could I please have reviews for this small cleanup? > For RSHashTable, it > - removes entry_for_region_ind(RegionIdx_t) > - changes get_entry to be public and declared const > - changes the sites that called entry_for_region to instead use > get_entry, since they have the same implementation. > - changes get_cards to make use of get_entry instead of very similar > code contained inline. > > webrev: http://cr.openjdk.java.net/~brutisso/JDK-8042474/01/webrev/ Looks good. Nice cleanup! One minor thing: These asserts in RSHashTable::get_cards(): 204 assert(entry->r_ind() == region_ind, "Postcondition of loop + test above."); 205 assert(entry->num_valid_cards() > 0, "Inv"); are already done in RSHashTable::get_entry(). I think you can remove them too. > > It seems get_cards is not used. Maybe it should be removed as a > separate CR? Yes it is unused, so it would be good to remove it. I'm fine with doing that as a separate CR. Maybe in that case also go through all methods in sparsePRT.hpp and make sure that they are used. Thanks, Bengt > > testing: jprt. > > Best regards, > Andreas -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schatzl at oracle.com Thu May 8 08:49:44 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 08 May 2014 10:49:44 +0200 Subject: RFR(S): JDK-8042474: Clean up duplicated code in RSHashTable In-Reply-To: <536B39D0.1070801@oracle.com> References: <536B39D0.1070801@oracle.com> Message-ID: <1399538984.2600.25.camel@cirrus> Hi, On Thu, 2014-05-08 at 10:01 +0200, Andreas Sj?berg wrote: > Hi all, > > could I please have reviews for this small cleanup? > For RSHashTable, it > - removes entry_for_region_ind(RegionIdx_t) > - changes get_entry to be public and declared const Get_entry() has already been public. That does not seem to be required though, it seems to be only used within the SparsePRT class. > - changes the sites that called entry_for_region to instead use > get_entry, since they have the same implementation. > - changes get_cards to make use of get_entry instead of very similar > code contained inline. > > webrev: http://cr.openjdk.java.net/~brutisso/JDK-8042474/01/webrev/ Looks good except for the duplicate asserts Bengt mentioned :) > > It seems get_cards is not used. Maybe it should be removed as a > separate CR? Like Bengt I do not mind either way - please also fix up visibility of the methods :) Thanks, Thomas From andreas.sjoberg at oracle.com Thu May 8 11:33:22 2014 From: andreas.sjoberg at oracle.com (=?ISO-8859-1?Q?Andreas_Sj=F6berg?=) Date: Thu, 08 May 2014 13:33:22 +0200 Subject: RFR(S): JDK-8042474: Clean up duplicated code in RSHashTable In-Reply-To: <536B3E59.6050203@oracle.com> References: <536B39D0.1070801@oracle.com> <536B3E59.6050203@oracle.com> Message-ID: <536B6B82.3050502@oracle.com> Hi, I removed the duplicate asserts. Here's a new webrev: http://cr.openjdk.java.net/~brutisso/JDK-8042474/02/webrev/ As for the general cleanup of unused methods, I will do that as a separate CR. Thanks for looking at this, Andreas On 05/08/2014 10:20 AM, Bengt Rutisson wrote: > > Hi Andreas, > > On 2014-05-08 10:01, Andreas Sj?berg wrote: >> Hi all, >> >> could I please have reviews for this small cleanup? >> For RSHashTable, it >> - removes entry_for_region_ind(RegionIdx_t) >> - changes get_entry to be public and declared const >> - changes the sites that called entry_for_region to instead use >> get_entry, since they have the same implementation. >> - changes get_cards to make use of get_entry instead of very similar >> code contained inline. >> >> webrev: http://cr.openjdk.java.net/~brutisso/JDK-8042474/01/webrev/ > > Looks good. Nice cleanup! > > One minor thing: > > These asserts in RSHashTable::get_cards(): > > 204 assert(entry->r_ind() == region_ind, "Postcondition of loop + > test above."); > 205 assert(entry->num_valid_cards() > 0, "Inv"); > > are already done in RSHashTable::get_entry(). I think you can remove > them too. > > >> >> It seems get_cards is not used. Maybe it should be removed as a >> separate CR? > > Yes it is unused, so it would be good to remove it. I'm fine with > doing that as a separate CR. Maybe in that case also go through all > methods in sparsePRT.hpp and make sure that they are used. > > Thanks, > Bengt > >> >> testing: jprt. >> >> Best regards, >> Andreas > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.rutisson at oracle.com Thu May 8 11:50:00 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Thu, 08 May 2014 13:50:00 +0200 Subject: RFR(S): JDK-8042474: Clean up duplicated code in RSHashTable In-Reply-To: <536B6B82.3050502@oracle.com> References: <536B39D0.1070801@oracle.com> <536B3E59.6050203@oracle.com> <536B6B82.3050502@oracle.com> Message-ID: <536B6F68.7060805@oracle.com> On 2014-05-08 13:33, Andreas Sj?berg wrote: > Hi, > > I removed the duplicate asserts. > Here's a new webrev: > http://cr.openjdk.java.net/~brutisso/JDK-8042474/02/webrev/ Looks good. > > As for the general cleanup of unused methods, I will do that as a > separate CR. Sounds good to me. Bengt > > Thanks for looking at this, > Andreas > On 05/08/2014 10:20 AM, Bengt Rutisson wrote: >> >> Hi Andreas, >> >> On 2014-05-08 10:01, Andreas Sj?berg wrote: >>> Hi all, >>> >>> could I please have reviews for this small cleanup? >>> For RSHashTable, it >>> - removes entry_for_region_ind(RegionIdx_t) >>> - changes get_entry to be public and declared const >>> - changes the sites that called entry_for_region to instead use >>> get_entry, since they have the same implementation. >>> - changes get_cards to make use of get_entry instead of very >>> similar code contained inline. >>> >>> webrev: http://cr.openjdk.java.net/~brutisso/JDK-8042474/01/webrev/ >> >> Looks good. Nice cleanup! >> >> One minor thing: >> >> These asserts in RSHashTable::get_cards(): >> >> 204 assert(entry->r_ind() == region_ind, "Postcondition of loop + >> test above."); >> 205 assert(entry->num_valid_cards() > 0, "Inv"); >> >> are already done in RSHashTable::get_entry(). I think you can remove >> them too. >> >> >>> >>> It seems get_cards is not used. Maybe it should be removed as a >>> separate CR? >> >> Yes it is unused, so it would be good to remove it. I'm fine with >> doing that as a separate CR. Maybe in that case also go through all >> methods in sparsePRT.hpp and make sure that they are used. >> >> Thanks, >> Bengt >> >>> >>> testing: jprt. >>> >>> Best regards, >>> Andreas >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrey.x.zakharov at oracle.com Thu May 8 12:47:44 2014 From: andrey.x.zakharov at oracle.com (Andrey Zakharov) Date: Thu, 08 May 2014 16:47:44 +0400 Subject: RFR: 8011397: JTREG needs to copy additional WhiteBox class file to JTwork/scratch/sun/hotspot Message-ID: <536B7CF0.6010508@oracle.com> Hi! Suggesting patch with fixes for https://bugs.openjdk.java.net/browse/JDK-8011397 webrev: https://bugs.openjdk.java.net/secure/attachment/20275/8011397.tgz patch: https://bugs.openjdk.java.net/secure/attachment/20274/8011397.WhiteBoxPermission Thanks. From kirk at kodewerk.com Thu May 8 04:11:37 2014 From: kirk at kodewerk.com (Kirk Pepperdine) Date: Thu, 8 May 2014 06:11:37 +0200 Subject: RFR (L) 8034246: remove CMS and ParNew adaptive size policy code In-Reply-To: <21354.50500.838480.593745@mykonos.us.oracle.com> References: <21351.47344.517204.861951@mykonos.us.oracle.com> <1598836.bStRTXLZya@mgerdin03> <21353.47283.610771.899532@mykonos.us.oracle.com> <876494FC-D3B6-43BF-B0D2-830B509646F1@kodewerk.com> <21354.50500.838480.593745@mykonos.us.oracle.com> Message-ID: <0FE17FCE-4E0C-4986-9FB7-93344006FB3F@kodewerk.com> Thanks John. On May 8, 2014, at 1:44 AM, John Coomes wrote: > Kirk Pepperdine (kirk at kodewerk.com) wrote: >> Hi, >> >> So my understanding correct (without digging into the code) is that CMS no longer uses any adaptive sizing? >> >> In the past I?ve provided charts that demonstrated adaptive sizing shrank memory in response to lower object creation rates. All well and good but in these cases the lower object creation rates were a result of increased frequency and duration of collections which should have resulted in a larger heap! There are/were a number of other degenerative cases where top of heap aggressively tracked top of live set causing all kinds of grief so backing off of this implementation seems reasonable, especially if there is little chance of it being fixed. > > Hi Kirk, > > Actually, CMS will still use the same sizing code that it has for many > years. > > The option -XX:+UseAdaptiveSizePolicy enabled code that was intended > to do more "ergonomic" sizing of the survivor spaces, generations and > tenuring threshold than is currently done, to reduce the need to tune > those on the command line. However, the CMS implementation was > incomplete and it was never enabled by default for CMS (it is enabled > by default for ParallelGC, where it works well). > > Some time ago [1] UseAdaptiveSizePolicy had to be completely disabled > for CMS (i.e., it is disabled even if you add > -XX:+UseAdaptiveSizePolicy on the command line) because it would cause > the VM to crash. So this change just removes code that cannot be > executed. > > -John > > [1] https://bugs.openjdk.java.net/browse/JDK-7112413 > >> On May 7, 2014, at 8:37 AM, Srinivas Ramakrishna wrote: >> >>> >>> Looks good to me too, and good that we are able to remove code that hasn't been used and seems unlikely now, with G1 available, to have the resources to make it work well. Please make sure that this has also been approved by Jon. >>> >>> Reviewed. >>> -- Ramki >>> >>> ysr1729 >>> >>>> On May 6, 2014, at 21:38, John Coomes wrote: >>>> >>>> Mikael Gerdin (mikael.gerdin at oracle.com) wrote: >>>>> John, >>>>> >>>>>> On Monday 05 May 2014 09.14.40 John Coomes wrote: >>>>>> I'd appreciate reviews of this change to delete unused adaptive size >>>>>> policy code for cms and parnew. It's big in terms of line count, but >>>>>> most comes from the deletion of 6 files. >>>>>> >>>>>> http://cr.openjdk.java.net/~jcoomes/8034246-rm-cms-asp/ >>>>> >>>>> Good to get rid of code that hasn't been working for a while. >>>>> >>>>> In src/share/vm/memory/collectorPolicy.cpp the include statement: >>>>> #include "gc_implementation/shared/adaptiveSizePolicy.hpp" >>>>> is already present a few lines up. >>>> >>>> Yes; fixed. >>>> >>>>> CMSCollector::size_policy and ConcurrentMarkSweepGeneration::size_policy are >>>>> basically identical now, do you want to fix this now or leave this as just a >>>>> code removal change? (personally, I'm fine either way) >>>> >>>> The ConcurrentMarkSweepGeneration one is no longer used, so I removed >>>> it. Updated webrev is at the same location. >>>> >>>>> Consider the change reviewed nevertheless. >>>> >>>> Thanks! >>>> >>>> -John >> > > -- > John Coomes Oracle, MS USCA22-3?? > john.coomes at oracle.com 4220 Network Circle > 408-276-7048 Santa Clara, CA 95054-1778 > *** Support GreenPeace and we'll all breathe easier. *** From filipp.zhinkin at oracle.com Thu May 8 15:09:42 2014 From: filipp.zhinkin at oracle.com (Filipp Zhinkin) Date: Thu, 08 May 2014 19:09:42 +0400 Subject: RFR: 8011397: JTREG needs to copy additional WhiteBox class file to JTwork/scratch/sun/hotspot In-Reply-To: <536B7CF0.6010508@oracle.com> References: <536B7CF0.6010508@oracle.com> Message-ID: <536B9E36.6090802@oracle.com> Andrey, I've CC'ed compiler and runtime mailing list, because you're changes affect test for other components as too. I don't like your solution (but I'm not a reviewer, so treat my words just as suggestion), because we'll have to write more meta information for each test and it is very easy to forget to install WhiteBoxPermission if you don't test your test with some security manager. From my point of view, it will be better to extend ClassFileInstaller so it will copy not only a class whose name was passed as an arguments, but also all inner classes of that class. And if someone want copy only specified class without inner classes, then some option could be added to ClassFileInstaller to force such behaviour. Thanks, Filipp. On 05/08/2014 04:47 PM, Andrey Zakharov wrote: > Hi! > Suggesting patch with fixes for > https://bugs.openjdk.java.net/browse/JDK-8011397 > > webrev: https://bugs.openjdk.java.net/secure/attachment/20275/8011397.tgz > patch: > https://bugs.openjdk.java.net/secure/attachment/20274/8011397.WhiteBoxPermission > > > Thanks. From per.liden at oracle.com Fri May 9 07:39:36 2014 From: per.liden at oracle.com (Per Liden) Date: Fri, 09 May 2014 09:39:36 +0200 Subject: RFR(S): 8040803: G1: Concurrent mark hangs when mark stack overflows In-Reply-To: <046ED8B9-DA6D-4014-987F-584B849935D3@oracle.com> References: <5360F4CB.9040106@oracle.com> <53635872.4030107@oracle.com> <536791BF.4050408@oracle.com> <046ED8B9-DA6D-4014-987F-584B849935D3@oracle.com> Message-ID: <536C8638.7030408@oracle.com> Thanks Bengt, Still need a second reviewer on this. /Per On 05/05/2014 06:11 PM, Bengt Rutisson wrote: > > Hi Per, > > Looks good. > > Bengt > >> 5 maj 2014 kl. 15:27 skrev Per Liden : >> >> Hi Bengt, >> >>> On 05/02/2014 10:33 AM, Bengt Rutisson wrote: >>> >>> Hi Per, >>> >>>> On 2014-04-30 15:04, Per Liden wrote: >>>> Hi, >>>> >>>> Could I please have a couple of reviews in this bug fix: >>>> >>>> Summary: G1's concurrent marking can potentially hang forever if the >>>> global mark stack overflows and immediately after that a Full GC >>>> happens, which tries to abort the marking. The reason is that there's >>>> a race between detecting the overflow situation and detecting the >>>> abort signal. Threads detecting the overflow situation first will go >>>> into the overflow protocol and wait on a barrier for all threads to >>>> reach this state. However, threads detecting the abort signal first >>>> will terminate and never participate in the barrier. >>>> >>>> This patch introduces an abort state and function on the >>>> WorkGangBarrierSync class, to unblock any threads waiting for the >>>> barrier to complete when the concurrent mark is aborted. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8040803 >>>> Webrev: http://cr.openjdk.java.net/~pliden/8040803/webrev.0/ >>> >>> The change looks good. >> >> Thanks for looking at the patch Bengt. >> >>> >>> One minor improvement would be to try to use an early exit in >>> ConcurrentMark::enter_first_sync_barrier() when the barrier fails. >> >> Here's an updated webrev with early exit: >> >> http://cr.openjdk.java.net/~pliden/8040803/webrev.1/ >> >> /Per >> >>> >>> Thanks, >>> Bengt >>> >>>> >>>> /Per >> From kirk at kodewerk.com Fri May 9 09:07:31 2014 From: kirk at kodewerk.com (Kirk Pepperdine) Date: Fri, 9 May 2014 11:07:31 +0200 Subject: GC log file corruption Message-ID: Hi, I found this in 1.7.0_51 (OSX) 12.590: [GC concurrent-root-region-scan-start] 12.590: [Full GC (System.gc())12.591: [GC concurrent-root-region-scan-end, 0.0012580 secs] 12.591: [GC concurrent-mark-start] 5221K->4963K(16M), 0.0284460 secs] [Eden: 1024.0K(5120.0K)->0.0B(6144.0K) Survivors: 1024.0K->0.0B Heap: 5221.0K(16.0M)->4963.9K(16.0M)] [Times: user=0.04 sys=0.00, real=0.03 secs] 12.619: [GC concurrent-mark-abort] The Full GC has been corrupted by the concurrent-root-region-scan-end. Is it worthwhile filing a bug on this or is this type of record corruption considered to be acceptable? Kind regards, Kirk From kirk at kodewerk.com Fri May 9 09:15:42 2014 From: kirk at kodewerk.com (Kirk Pepperdine) Date: Fri, 9 May 2014 11:15:42 +0200 Subject: one more question Message-ID: <3C7B6CF7-2F55-4379-9B4C-D0F5EC7ADF04@kodewerk.com> Hi again, With regards to the last log file I forgot to ask the obvious question, 12.590: [GC concurrent-root-region-scan-start] 12.590: [Full GC (System.gc())12.591: [GC concurrent-root-region-scan-end, 0.0012580 secs] 12.591: [GC concurrent-mark-start] 5221K->4963K(16M), 0.0284460 secs] [Eden: 1024.0K(5120.0K)->0.0B(6144.0K) Survivors: 1024.0K->0.0B Heap: 5221.0K(16.0M)->4963.9K(16.0M)] [Times: user=0.04 sys=0.00, real=0.03 secs] 12.619: [GC concurrent-mark-abort] The concurrent root region scan started @ 12.590 and I imagine the call to System.gc() came in shortly after that. So closing down the root region scan is expected. However, if these time stamps are to be believed (and I?m guessing there are some funky timing issues in here), the concurrent mark start is unexpected. Shouldn?t the Full GC cycle have prevented the concurrent mark from starting? Or does it need the abort to cleanup or reset for the next concurrent cycle? Regards, Kirk From erik.helin at oracle.com Mon May 12 14:46:34 2014 From: erik.helin at oracle.com (Erik Helin) Date: Mon, 12 May 2014 16:46:34 +0200 Subject: RFR(S): 8040803: G1: Concurrent mark hangs when mark stack overflows In-Reply-To: <536C8638.7030408@oracle.com> References: <5360F4CB.9040106@oracle.com> <53635872.4030107@oracle.com> <536791BF.4050408@oracle.com> <046ED8B9-DA6D-4014-987F-584B849935D3@oracle.com> <536C8638.7030408@oracle.com> Message-ID: <5370DECA.8080101@oracle.com> Looks good. Thanks, Erik On 2014-05-09 09:39, Per Liden wrote: > Thanks Bengt, > > Still need a second reviewer on this. > > /Per > > On 05/05/2014 06:11 PM, Bengt Rutisson wrote: >> >> Hi Per, >> >> Looks good. >> >> Bengt >> >>> 5 maj 2014 kl. 15:27 skrev Per Liden : >>> >>> Hi Bengt, >>> >>>> On 05/02/2014 10:33 AM, Bengt Rutisson wrote: >>>> >>>> Hi Per, >>>> >>>>> On 2014-04-30 15:04, Per Liden wrote: >>>>> Hi, >>>>> >>>>> Could I please have a couple of reviews in this bug fix: >>>>> >>>>> Summary: G1's concurrent marking can potentially hang forever if the >>>>> global mark stack overflows and immediately after that a Full GC >>>>> happens, which tries to abort the marking. The reason is that there's >>>>> a race between detecting the overflow situation and detecting the >>>>> abort signal. Threads detecting the overflow situation first will go >>>>> into the overflow protocol and wait on a barrier for all threads to >>>>> reach this state. However, threads detecting the abort signal first >>>>> will terminate and never participate in the barrier. >>>>> >>>>> This patch introduces an abort state and function on the >>>>> WorkGangBarrierSync class, to unblock any threads waiting for the >>>>> barrier to complete when the concurrent mark is aborted. >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8040803 >>>>> Webrev: http://cr.openjdk.java.net/~pliden/8040803/webrev.0/ >>>> >>>> The change looks good. >>> >>> Thanks for looking at the patch Bengt. >>> >>>> >>>> One minor improvement would be to try to use an early exit in >>>> ConcurrentMark::enter_first_sync_barrier() when the barrier fails. >>> >>> Here's an updated webrev with early exit: >>> >>> http://cr.openjdk.java.net/~pliden/8040803/webrev.1/ >>> >>> /Per >>> >>>> >>>> Thanks, >>>> Bengt >>>> >>>>> >>>>> /Per >>> > From per.liden at oracle.com Mon May 12 15:18:41 2014 From: per.liden at oracle.com (Per Liden) Date: Mon, 12 May 2014 17:18:41 +0200 Subject: RFR(S): 8040803: G1: Concurrent mark hangs when mark stack overflows In-Reply-To: <5370DECA.8080101@oracle.com> References: <5360F4CB.9040106@oracle.com> <53635872.4030107@oracle.com> <536791BF.4050408@oracle.com> <046ED8B9-DA6D-4014-987F-584B849935D3@oracle.com> <536C8638.7030408@oracle.com> <5370DECA.8080101@oracle.com> Message-ID: <5370E651.4030309@oracle.com> Thanks Erik! /Per On 05/12/2014 04:46 PM, Erik Helin wrote: > Looks good. > > Thanks, > Erik > > On 2014-05-09 09:39, Per Liden wrote: >> Thanks Bengt, >> >> Still need a second reviewer on this. >> >> /Per >> >> On 05/05/2014 06:11 PM, Bengt Rutisson wrote: >>> >>> Hi Per, >>> >>> Looks good. >>> >>> Bengt >>> >>>> 5 maj 2014 kl. 15:27 skrev Per Liden : >>>> >>>> Hi Bengt, >>>> >>>>> On 05/02/2014 10:33 AM, Bengt Rutisson wrote: >>>>> >>>>> Hi Per, >>>>> >>>>>> On 2014-04-30 15:04, Per Liden wrote: >>>>>> Hi, >>>>>> >>>>>> Could I please have a couple of reviews in this bug fix: >>>>>> >>>>>> Summary: G1's concurrent marking can potentially hang forever if the >>>>>> global mark stack overflows and immediately after that a Full GC >>>>>> happens, which tries to abort the marking. The reason is that there's >>>>>> a race between detecting the overflow situation and detecting the >>>>>> abort signal. Threads detecting the overflow situation first will go >>>>>> into the overflow protocol and wait on a barrier for all threads to >>>>>> reach this state. However, threads detecting the abort signal first >>>>>> will terminate and never participate in the barrier. >>>>>> >>>>>> This patch introduces an abort state and function on the >>>>>> WorkGangBarrierSync class, to unblock any threads waiting for the >>>>>> barrier to complete when the concurrent mark is aborted. >>>>>> >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8040803 >>>>>> Webrev: http://cr.openjdk.java.net/~pliden/8040803/webrev.0/ >>>>> >>>>> The change looks good. >>>> >>>> Thanks for looking at the patch Bengt. >>>> >>>>> >>>>> One minor improvement would be to try to use an early exit in >>>>> ConcurrentMark::enter_first_sync_barrier() when the barrier fails. >>>> >>>> Here's an updated webrev with early exit: >>>> >>>> http://cr.openjdk.java.net/~pliden/8040803/webrev.1/ >>>> >>>> /Per >>>> >>>>> >>>>> Thanks, >>>>> Bengt >>>>> >>>>>> >>>>>> /Per >>>> >> From per.liden at oracle.com Tue May 13 10:34:25 2014 From: per.liden at oracle.com (Per Liden) Date: Tue, 13 May 2014 12:34:25 +0200 Subject: RFR(S): 8039042: G1: Phantom zeros in cardtable Message-ID: <5371F531.7040008@oracle.com> Summary: We use memset to initialize the cardtable. On Sparc T4 (and later) memset uses some special instructions with the side-effect that the memory can temporarily be filled with zeros before the actual value is set. These phantom zeros can be observed by concurrent readers of this memory, which can trick the G1 post-barrier to incorrectly dirty a card in a young region. Please see the bug for a more details. A similar memset-problem has been observed and fixed before (JDK-6948537) when using memset on the BlockOffsetTable. Similar to the BlockOffsetTable, I'm using the UseMemSetInBOT flag as an indicator of whether it's safe to use memset or not. UseMemSetInBOT is not a perfect name for this flag as code other than the BOT might want to use it. It's also questionable of this should be a command-line flag at all. I've filed a separate RFE, JDK-8042930, to do a bit of clean up related to this flag. Bug: https://bugs.openjdk.java.net/browse/JDK-8039042 Webrev: http://cr.openjdk.java.net/~pliden/8039042/webrev.0/ Thanks! /Per From mikael.gerdin at oracle.com Tue May 13 12:41:32 2014 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Tue, 13 May 2014 14:41:32 +0200 Subject: RFR(S): 8039042: G1: Phantom zeros in cardtable In-Reply-To: <5371F531.7040008@oracle.com> References: <5371F531.7040008@oracle.com> Message-ID: <1829588.qp7YZWs7PG@mgerdin03> Per, On Tuesday 13 May 2014 12.34.25 Per Liden wrote: > Summary: We use memset to initialize the cardtable. On Sparc T4 (and > later) memset uses some special instructions with the side-effect that > the memory can temporarily be filled with zeros before the actual value > is set. These phantom zeros can be observed by concurrent readers of > this memory, which can trick the G1 post-barrier to incorrectly dirty a > card in a young region. Please see the bug for a more details. > > A similar memset-problem has been observed and fixed before > (JDK-6948537) when using memset on the BlockOffsetTable. Similar to the > BlockOffsetTable, I'm using the UseMemSetInBOT flag as an indicator of > whether it's safe to use memset or not. UseMemSetInBOT is not a perfect > name for this flag as code other than the BOT might want to use it. It's > also questionable of this should be a command-line flag at all. I've > filed a separate RFE, JDK-8042930, to do a bit of clean up related to > this flag. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8039042 > Webrev: http://cr.openjdk.java.net/~pliden/8039042/webrev.0/ The change looks good. A fun bug to investigate indeed :) I agree that setting of UseMemSetInBOT should be retired, we don't need a -XX: +RandomlyCrashOnSolarisSparc flag. /Mikael > > Thanks! > /Per From stefan.johansson at oracle.com Wed May 14 13:11:19 2014 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Wed, 14 May 2014 15:11:19 +0200 Subject: RFR: 8042298 - Remove the names gen0 and gen1 from the GC code In-Reply-To: <5362F75B.9020201@oracle.com> References: <5362F75B.9020201@oracle.com> Message-ID: <53736B77.9030206@oracle.com> Hi Jesper, Thanks for doing this work. Here are some comments: src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp: * TraceGen0Time and TraceGen1Time should be changed to TraceYoungTime and TraceOldTime for consistency, and I guess that will need a CCC request. src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.hpp * CollectGen0First -> CollectYoungFirst, same as above. Also used in tenuredGeneration.hpp. src/share/vm/gc_implementation/g1/g1CollectorPolicy.hpp * Comment starting on line 41 mentions TraceGen0Time and gen1. As an additional comment, I think that GenCollectedHeap::get_gen(int) should be broken up into two functions get_young() and get_old(). I get that this was left to a later patch to avoid having code/logic changes in this patch but I think the change is so little that it makes sense to do it in this patch. I would like to avoid having code like, even if it is for a brief period of time: Generation* young = gch->get_gen(0); Thanks, Stefan On 2014-05-02 03:39, Jesper Wilhelmsson wrote: > Hi, > > This is a step towards removing the _generations array. The names gen0 > and gen1 are replaced with young and old to use a more standardized > vocabulary. > > Webrev: http://cr.openjdk.java.net/~jwilhelm/8042298/webrev/ > > Bug: https://bugs.openjdk.java.net/browse/JDK-8042298 > > Thanks, > /Jesper From jon.masamitsu at oracle.com Wed May 14 14:15:28 2014 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Wed, 14 May 2014 07:15:28 -0700 Subject: Request for review (s) - 8031323: Optionally align objects copied to survivor spaces In-Reply-To: <53677B60.6030709@oracle.com> References: <53267CBB.9020804@oracle.com> <53284A7E.9060106@oracle.com> <5363D517.80200@oracle.com> <53677B60.6030709@oracle.com> Message-ID: <53737A80.8070704@oracle.com> Bengt, Thanks for looking at this. Responses in-line. On 5/5/2014 4:52 AM, Bengt Rutisson wrote: > > Hi Jon, > > On 2014-05-02 19:25, Jon Masamitsu wrote: >> New webrev with problems addressed except as noted below. >> >> http://cr.openjdk.java.net/~jmasa/8031323/webrev.01/ > > > To make it compile I had to remove the inline keyword from the > declaration of CollectedHeap::align_allocation_or_fail(): > > diff --git a/src/share/vm/gc_interface/collectedHeap.hpp > b/src/share/vm/gc_interface/collectedHeap.hpp > --- a/src/share/vm/gc_interface/collectedHeap.hpp > +++ b/src/share/vm/gc_interface/collectedHeap.hpp > @@ -353,7 +353,7 @@ > > // Return the address "addr" aligned by "alignment_in_bytes" if such > // an address is below "end". Return NULL otherwise. > - inline static HeapWord* align_allocation_or_fail(HeapWord* addr, > + static HeapWord* align_allocation_or_fail(HeapWord* addr, > HeapWord* end, > unsigned short > alignment_in_bytes); > > > The includes for the usages of > CollectedHeap::align_allocation_or_fail() need to be updated. > > space.cpp should include collectedHeap.inline.hpp. > > psPromotionLAB.hpp is using CollectedHeap::align_allocation_or_fail() > and thus would need the include too, but we should not include inline > files from hpp files, so you need to move > PSYoungPromotionLAB::allocate() into psPromotionLAB.inline.hpp and > have that include collectedHeap.inline.hpp. > > Similarly parGCAllocBuffer.hpp would need the include too since > ParGCAllocBuffer::allocate_aligned() uses > CollectedHeap::align_allocation_or_fail(). But again we should not > include inline files in hpp files so > ParGCAllocBuffer::allocate_aligned() should probably be moved in to > the cpp file. Or, if we are worried about performance regressions we > need to introduce a parGCAllocBuffer.inline.hpp file and move it there. > Now compiles with and without USE_PRECOMPILED_HEADER=0. I left the inline methods as inline. > > > In CollectedHeap::align_allocation_or_fail(), I think we can replace > this: > > if(new_addr > addr && new_addr < end) { > > with: > > assert(new_addr > addr, "some good err_msg"); > if(new_addr < end) { > > Fixed. > > In ContiguousSpace::allocate_aligned() I think we can make the assert > for the alignment of obj a bit stronger. Now we do: > > assert(is_aligned(obj) && is_aligned(new_top), "checking alignment"); > > But we have just gone through some trouble to make sure that obj is > aligned to SurvivorAlignmentInBytes, so I think we should assert that > rather than just the 8 byte alignment that Space::is_aligned() gives us. > Fixed. > > PSYoungPromotionLAB::allocate() and > ContiguousSpace::allocate_aligned() is part of the code duplication > that you plan on fixing with JDK-8042321. I'm fine with fixing that in > a separate change, but it would be nice if we can make these methods > look more similar. Right now they use different return patterns. > > PSYoungPromotionLAB::allocate() uses: > > if (allocation worked) { > return obj; > } > ... > return NULL; > > But ContiguousSpace::allocate_aligned() uses an else statement: > > if (allocation worked) { > return obj; > } else { > ... > return NULL; > } > Fixed. Both have an else clause. > Similarly to the comment about the assert in > ContiguousSpace::allocate_aligned() I think the assert in > PSYoungPromotionLAB::allocate() can be strengthened to assert that obj > is SurvivorAlignmentInBytes aligned. > Fixed. > > A minor thing about oop.pcgc.inline.hpp > > -// Used by parallel old GC. > +// forward_to_atomic() is used by parallel old and G1 GCs. > > You updated this comment to include G1, but the method is actually > also used by ParNew (see > ParNewGeneration::copy_to_survivor_space_avoiding_promotion_undo()) > and I don't find any usages in ParallelOld. I would suggest to just > get rid of the comment. It is not important to have a comment saying > who uses it. > > Deleted comment. > > Another very minor thing. The extra line feed added around line 1200 > in parNewGeneration.cpp does not seem related to this change: I think the extra blank line was in the original. I added it back. > > http://cr.openjdk.java.net/~jmasa/8031323/webrev.01/src/share/vm/gc_implementation/parNew/parNewGeneration.cpp.cdiff.html > > > > Testing. There are tests for this feature coming, right? What kind of > testing have you done so far on the current patch? I added diagnostics to count the number of allocations that were aligned to 64 bytes. Then ran with different values of SurvivorAlignmentInBytes. In particular 8 bytes and 64 bytes. I printed the fraction of survivor allocations and promoted allocations that were aligned and not aligned after each collection. I looked at the output such as (for 8byte alignment) Promoted addresses aligned 462 missed 3178 p_fraction 0.126923 Survivor addresses aligned 933 missed 6050 ss_fraction 0.133610 and (for 64 byte alignment) Promoted addresses aligned 471 missed 3168 p_fraction 0.129431 Survivor addresses aligned 7107 missed 0 ss_fraction 1.000000 to see if it was as I expected it to be. The webrev with the changes are at http://cr.openjdk.java.net/~jmasa/8031323/webrev_delta.02/ The webrev shows the changes but also contains (if you look at other than just the changes), the diagnostic printing code I described above. You can ignore that since it is not in the complete webrev. The complete webrev is at http://cr.openjdk.java.net/~jmasa/8031323/webrev.02/ Thanks again. Jon > > Thanks, > Bengt > > >> >> Thanks. >> >> >> On 03/18/2014 06:30 AM, Bengt Rutisson wrote: >>> >>> Hi Jon, >>> >>> On 2014-03-17 05:40, Jon Masamitsu wrote: >>>> 8031323: Optionally align objects copied to survivor spaces >>>> >>>> Add the option to specify an alignment for allocations into the >>>> survivor spaces. Survivor space allocations will overflow if the >>>> survivor spaces have not been increased. The expected use >>>> is to align survivor space allocations to cache line sizes. >>>> >>>> http://cr.openjdk.java.net/~jmasa/8031323/webrev.00/ >>> >>> For G1 it looks to me like we are aligning objects in the old >>> regions as well. I thought the intent was to only align in the >>> survivor regions. Am I missing something? >> >> Yes, that was an error. Fixed in the new webrev. >> >>> >>> Note that it is not enough to check the alloc purpose for G1 since >>> G1CollectedHeap::par_allocate_during_gc() only takes the purpose as >>> a hint and may still return buffers from other regions than expected >>> for a purpose. But I guess that in this case it is not too important >>> to get it exactly right, so it may be enough to check the alloc >>> purpose value. >> >> I did use the purpose and there are cases where alignments in the >> survivor spaces >> can be missed. I think the number missed is a very small percentage >> of the total >> so I'm going to let those slip by. I thought that preferable to the >> more extensive >> coding changes it would take to catch them. >>> >>> >>> The code in PSYoungPromotionLAB::allcoate(), >>> ParGCAllocBuffer::allocate_aligned() and >>> ContiguousSpace::allocate_aligned() is very similar. Is there a way >>> to consolidate these methods? >>> >> I looked at this but did not come up with a suitable way to fix it. >> I've opened >> CR to track the issue. >> >> https://bugs.openjdk.java.net/browse/JDK-8042321 >> >> >>> >>> The code in CollectedHeap::align_allocation_or_fail() looks quite >>> different from other places in the code base where we do padding. >>> What do you think about doing something like this instead? >>> >>> inline HeapWord* CollectedHeap::align_allocation_or_fail(HeapWord* >>> addr, HeapWord* end, unsigned short alignment_in_bytes) { >>> if (alignment_in_bytes <= ObjectAlignmentInBytes) { >>> return addr; >>> } >>> >>> assert(is_ptr_aligned(addr, HeapWordSize), ""); >>> assert(is_size_aligned(alignment_in_bytes, HeapWordSize), ""); >>> >>> HeapWord* new_addr = (HeapWord*) align_pointer_up(addr, >>> alignment_in_bytes); >>> size_t padding = pointer_delta(new_addr, addr); >>> https://bugs.openjdk.java.net/browse/JDK-8042321 >>> if (padding == 0) { >>> return addr; >>> } >>> >>> if (padding < CollectedHeap::min_fill_size()) { >>> padding += alignment_in_bytes / HeapWordSize; >>> assert(padding >= CollectedHeap::min_fill_size(), ""); >>> new_addr = addr + padding; >>> } >>> >>> if(new_addr > addr && new_addr < end) { >>> CollectedHeap::fill_with_object(addr, padding); >>> return new_addr; >>> } else { >>> return NULL; >>> } >>> } >> >> Fixed that as suggested. >> >> Jon >>> >>> Thanks, >>> Bengt >>> >>>> >>>> Jon >>> >> > From dmitry.fazunenko at oracle.com Wed May 14 14:28:19 2014 From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko) Date: Wed, 14 May 2014 18:28:19 +0400 Subject: RFR: 8039489: Refactor test framework for dynamic VM options In-Reply-To: <535A524D.1000701@oracle.com> References: <535A524D.1000701@oracle.com> Message-ID: <53737D83.2050205@oracle.com> Hi, Would you please look at the new version of a common library for testing dynamic VM options. Description: Some time ago a library for testing VM flags which could be changed dynamically was integrated into the hs repository: test/testlibrary/com/oracle/java/testlibrary/DynamicVMOptionChecker.java test/testlibrary/com/oracle/java/testlibrary/TestDynamicVMOption.java The library was designed in not the most optimal way. This change is an attempt to redesign it: test/testlibrary/com/oracle/java/testlibrary/DynamicVMOption.java is a new version which provides a convenient access to the HS API on changing VM flags. The tests based on the library are also refactored to meet new API. You comments will be much appreciated. http://cr.openjdk.java.net/~ehelin/dmitry/8039489/webrev.00/ https://bugs.openjdk.java.net/browse/JDK-8039489 I want to thank Erik Helin for his incredible assistance with working on this change. Thanks, Dima From jesper.wilhelmsson at oracle.com Wed May 14 14:50:21 2014 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Wed, 14 May 2014 16:50:21 +0200 Subject: RFR: 8039489: Refactor test framework for dynamic VM options In-Reply-To: <53737D83.2050205@oracle.com> References: <535A524D.1000701@oracle.com> <53737D83.2050205@oracle.com> Message-ID: <537382AD.6090107@oracle.com> Looks good! /Jesper Dmitry Fazunenko skrev 14/5/14 16:28: > Hi, > > Would you please look at the new version of a common library for testing dynamic > VM options. > > Description: > Some time ago a library for testing VM flags which could be changed dynamically > was integrated into the hs repository: > test/testlibrary/com/oracle/java/testlibrary/DynamicVMOptionChecker.java > test/testlibrary/com/oracle/java/testlibrary/TestDynamicVMOption.java > > The library was designed in not the most optimal way. > This change is an attempt to redesign it: > > test/testlibrary/com/oracle/java/testlibrary/DynamicVMOption.java > > is a new version which provides a convenient access to the HS API on changing VM > flags. > The tests based on the library are also refactored to meet new API. > > You comments will be much appreciated. > > http://cr.openjdk.java.net/~ehelin/dmitry/8039489/webrev.00/ > https://bugs.openjdk.java.net/browse/JDK-8039489 > > I want to thank Erik Helin for his incredible assistance with working on this > change. > > Thanks, > Dima From bengt.rutisson at oracle.com Wed May 14 15:38:34 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 14 May 2014 17:38:34 +0200 Subject: Request for review (s) - 8031323: Optionally align objects copied to survivor spaces In-Reply-To: <53737A80.8070704@oracle.com> References: <53267CBB.9020804@oracle.com> <53284A7E.9060106@oracle.com> <5363D517.80200@oracle.com> <53677B60.6030709@oracle.com> <53737A80.8070704@oracle.com> Message-ID: <53738DFA.6090603@oracle.com> Hi Jon, Thanks for fixing all of this! Looks good. Reviewed. Bengt On 5/14/14 4:15 PM, Jon Masamitsu wrote: > Bengt, > > Thanks for looking at this. Responses in-line. > > On 5/5/2014 4:52 AM, Bengt Rutisson wrote: >> >> Hi Jon, >> >> On 2014-05-02 19:25, Jon Masamitsu wrote: >>> New webrev with problems addressed except as noted below. >>> >>> http://cr.openjdk.java.net/~jmasa/8031323/webrev.01/ >> >> >> To make it compile I had to remove the inline keyword from the >> declaration of CollectedHeap::align_allocation_or_fail(): >> >> diff --git a/src/share/vm/gc_interface/collectedHeap.hpp >> b/src/share/vm/gc_interface/collectedHeap.hpp >> --- a/src/share/vm/gc_interface/collectedHeap.hpp >> +++ b/src/share/vm/gc_interface/collectedHeap.hpp >> @@ -353,7 +353,7 @@ >> >> // Return the address "addr" aligned by "alignment_in_bytes" if such >> // an address is below "end". Return NULL otherwise. >> - inline static HeapWord* align_allocation_or_fail(HeapWord* addr, >> + static HeapWord* align_allocation_or_fail(HeapWord* addr, >> HeapWord* end, >> unsigned short >> alignment_in_bytes); >> >> >> The includes for the usages of >> CollectedHeap::align_allocation_or_fail() need to be updated. >> >> space.cpp should include collectedHeap.inline.hpp. >> >> psPromotionLAB.hpp is using CollectedHeap::align_allocation_or_fail() >> and thus would need the include too, but we should not include inline >> files from hpp files, so you need to move >> PSYoungPromotionLAB::allocate() into psPromotionLAB.inline.hpp and >> have that include collectedHeap.inline.hpp. >> >> Similarly parGCAllocBuffer.hpp would need the include too since >> ParGCAllocBuffer::allocate_aligned() uses >> CollectedHeap::align_allocation_or_fail(). But again we should not >> include inline files in hpp files so >> ParGCAllocBuffer::allocate_aligned() should probably be moved in to >> the cpp file. Or, if we are worried about performance regressions we >> need to introduce a parGCAllocBuffer.inline.hpp file and move it there. >> > Now compiles with and without USE_PRECOMPILED_HEADER=0. I left the > inline methods as inline. > >> >> >> In CollectedHeap::align_allocation_or_fail(), I think we can replace >> this: >> >> if(new_addr > addr && new_addr < end) { >> >> with: >> >> assert(new_addr > addr, "some good err_msg"); >> if(new_addr < end) { >> >> > > Fixed. > >> >> In ContiguousSpace::allocate_aligned() I think we can make the assert >> for the alignment of obj a bit stronger. Now we do: >> >> assert(is_aligned(obj) && is_aligned(new_top), "checking alignment"); >> >> But we have just gone through some trouble to make sure that obj is >> aligned to SurvivorAlignmentInBytes, so I think we should assert that >> rather than just the 8 byte alignment that Space::is_aligned() gives us. >> > > Fixed. > >> >> PSYoungPromotionLAB::allocate() and >> ContiguousSpace::allocate_aligned() is part of the code duplication >> that you plan on fixing with JDK-8042321. I'm fine with fixing that >> in a separate change, but it would be nice if we can make these >> methods look more similar. Right now they use different return patterns. >> >> PSYoungPromotionLAB::allocate() uses: >> >> if (allocation worked) { >> return obj; >> } >> ... >> return NULL; >> >> But ContiguousSpace::allocate_aligned() uses an else statement: >> >> if (allocation worked) { >> return obj; >> } else { >> ... >> return NULL; >> } >> > Fixed. Both have an else clause. > >> Similarly to the comment about the assert in >> ContiguousSpace::allocate_aligned() I think the assert in >> PSYoungPromotionLAB::allocate() can be strengthened to assert that >> obj is SurvivorAlignmentInBytes aligned. >> > > Fixed. > >> >> A minor thing about oop.pcgc.inline.hpp >> >> -// Used by parallel old GC. >> +// forward_to_atomic() is used by parallel old and G1 GCs. >> >> You updated this comment to include G1, but the method is actually >> also used by ParNew (see >> ParNewGeneration::copy_to_survivor_space_avoiding_promotion_undo()) >> and I don't find any usages in ParallelOld. I would suggest to just >> get rid of the comment. It is not important to have a comment saying >> who uses it. >> >> > Deleted comment. > >> >> Another very minor thing. The extra line feed added around line 1200 >> in parNewGeneration.cpp does not seem related to this change: > > I think the extra blank line was in the original. I added it back. > >> >> http://cr.openjdk.java.net/~jmasa/8031323/webrev.01/src/share/vm/gc_implementation/parNew/parNewGeneration.cpp.cdiff.html >> >> >> >> Testing. There are tests for this feature coming, right? What kind of >> testing have you done so far on the current patch? > > I added diagnostics to count the number of allocations that were aligned > to 64 bytes. Then ran with different values of SurvivorAlignmentInBytes. > In particular 8 bytes and 64 bytes. I printed the fraction of > survivor allocations > and promoted allocations that were aligned and not aligned after each > collection. > I looked at the output such as (for 8byte alignment) > > Promoted addresses aligned 462 missed 3178 p_fraction 0.126923 > Survivor addresses aligned 933 missed 6050 ss_fraction 0.133610 > > and (for 64 byte alignment) > > Promoted addresses aligned 471 missed 3168 p_fraction 0.129431 > Survivor addresses aligned 7107 missed 0 ss_fraction 1.000000 > > to see if it was as I expected it to be. > > The webrev with the changes are at > > http://cr.openjdk.java.net/~jmasa/8031323/webrev_delta.02/ > > The webrev shows the changes but also contains (if you look at other > than just the changes), the diagnostic printing code I described above. > You can ignore that since it is not in the complete webrev. > > The complete webrev is at > > http://cr.openjdk.java.net/~jmasa/8031323/webrev.02/ > > Thanks again. > > Jon > > > >> >> Thanks, >> Bengt >> >> >>> >>> Thanks. >>> >>> >>> On 03/18/2014 06:30 AM, Bengt Rutisson wrote: >>>> >>>> Hi Jon, >>>> >>>> On 2014-03-17 05:40, Jon Masamitsu wrote: >>>>> 8031323: Optionally align objects copied to survivor spaces >>>>> >>>>> Add the option to specify an alignment for allocations into the >>>>> survivor spaces. Survivor space allocations will overflow if the >>>>> survivor spaces have not been increased. The expected use >>>>> is to align survivor space allocations to cache line sizes. >>>>> >>>>> http://cr.openjdk.java.net/~jmasa/8031323/webrev.00/ >>>> >>>> For G1 it looks to me like we are aligning objects in the old >>>> regions as well. I thought the intent was to only align in the >>>> survivor regions. Am I missing something? >>> >>> Yes, that was an error. Fixed in the new webrev. >>> >>>> >>>> Note that it is not enough to check the alloc purpose for G1 since >>>> G1CollectedHeap::par_allocate_during_gc() only takes the purpose as >>>> a hint and may still return buffers from other regions than >>>> expected for a purpose. But I guess that in this case it is not too >>>> important to get it exactly right, so it may be enough to check the >>>> alloc purpose value. >>> >>> I did use the purpose and there are cases where alignments in the >>> survivor spaces >>> can be missed. I think the number missed is a very small >>> percentage of the total >>> so I'm going to let those slip by. I thought that preferable to the >>> more extensive >>> coding changes it would take to catch them. >>>> >>>> >>>> The code in PSYoungPromotionLAB::allcoate(), >>>> ParGCAllocBuffer::allocate_aligned() and >>>> ContiguousSpace::allocate_aligned() is very similar. Is there a way >>>> to consolidate these methods? >>>> >>> I looked at this but did not come up with a suitable way to fix it. >>> I've opened >>> CR to track the issue. >>> >>> https://bugs.openjdk.java.net/browse/JDK-8042321 >>> >>> >>>> >>>> The code in CollectedHeap::align_allocation_or_fail() looks quite >>>> different from other places in the code base where we do padding. >>>> What do you think about doing something like this instead? >>>> >>>> inline HeapWord* CollectedHeap::align_allocation_or_fail(HeapWord* >>>> addr, HeapWord* end, unsigned short alignment_in_bytes) { >>>> if (alignment_in_bytes <= ObjectAlignmentInBytes) { >>>> return addr; >>>> } >>>> >>>> assert(is_ptr_aligned(addr, HeapWordSize), ""); >>>> assert(is_size_aligned(alignment_in_bytes, HeapWordSize), ""); >>>> >>>> HeapWord* new_addr = (HeapWord*) align_pointer_up(addr, >>>> alignment_in_bytes); >>>> size_t padding = pointer_delta(new_addr, addr); >>>> https://bugs.openjdk.java.net/browse/JDK-8042321 >>>> if (padding == 0) { >>>> return addr; >>>> } >>>> >>>> if (padding < CollectedHeap::min_fill_size()) { >>>> padding += alignment_in_bytes / HeapWordSize; >>>> assert(padding >= CollectedHeap::min_fill_size(), ""); >>>> new_addr = addr + padding; >>>> } >>>> >>>> if(new_addr > addr && new_addr < end) { >>>> CollectedHeap::fill_with_object(addr, padding); >>>> return new_addr; >>>> } else { >>>> return NULL; >>>> } >>>> } >>> >>> Fixed that as suggested. >>> >>> Jon >>>> >>>> Thanks, >>>> Bengt >>>> >>>>> >>>>> Jon >>>> >>> >> > From jon.masamitsu at oracle.com Wed May 14 20:34:14 2014 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Wed, 14 May 2014 13:34:14 -0700 Subject: RFR(S): 8039042: G1: Phantom zeros in cardtable In-Reply-To: <5371F531.7040008@oracle.com> References: <5371F531.7040008@oracle.com> Message-ID: <5373D346.4010805@oracle.com> Per, Could you use a check on the sparc version to decide on when to use memset. VM_Version::is_T4() in cpu/sparc/vm/vm_version_sparc.hpp Jon On 5/13/2014 3:34 AM, Per Liden wrote: > Summary: We use memset to initialize the cardtable. On Sparc T4 (and > later) memset uses some special instructions with the side-effect that > the memory can temporarily be filled with zeros before the actual > value is set. These phantom zeros can be observed by concurrent > readers of this memory, which can trick the G1 post-barrier to > incorrectly dirty a card in a young region. Please see the bug for a > more details. > > A similar memset-problem has been observed and fixed before > (JDK-6948537) when using memset on the BlockOffsetTable. Similar to > the BlockOffsetTable, I'm using the UseMemSetInBOT flag as an > indicator of whether it's safe to use memset or not. UseMemSetInBOT is > not a perfect name for this flag as code other than the BOT might want > to use it. It's also questionable of this should be a command-line > flag at all. I've filed a separate RFE, JDK-8042930, to do a bit of > clean up related to this flag. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8039042 > Webrev: http://cr.openjdk.java.net/~pliden/8039042/webrev.0/ > > Thanks! > /Per From jesper.wilhelmsson at oracle.com Wed May 14 21:21:02 2014 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Wed, 14 May 2014 23:21:02 +0200 Subject: RFR: 8042298 - Remove the names gen0 and gen1 from the GC code In-Reply-To: <53736B77.9030206@oracle.com> References: <5362F75B.9020201@oracle.com> <53736B77.9030206@oracle.com> Message-ID: <5373DE3E.9080803@oracle.com> Hi Stefan, Thanks for looking at this! See comments inline. Stefan Johansson skrev 14/5/14 15:11: > Hi Jesper, > > Thanks for doing this work. Here are some comments: > > src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp: > * TraceGen0Time and TraceGen1Time should be changed to TraceYoungTime and > TraceOldTime for consistency, and I guess that will need a CCC request. > > src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.hpp > > * CollectGen0First -> CollectYoungFirst, same as above. Also used in > tenuredGeneration.hpp. > > src/share/vm/gc_implementation/g1/g1CollectorPolicy.hpp > * Comment starting on line 41 mentions TraceGen0Time and gen1. I agree that the names gen0 and gen1 should be completely removed, which includes changing these flags too. Initially I wanted this change to be a simple name change so that it would be easy to review it. Changing the flags is borderline to be more than a name change, but I'm fine with including that change. I'll file a CCC to change the flags. > As an additional comment, I think that GenCollectedHeap::get_gen(int) should be > broken up into two functions get_young() and get_old(). I get that this was left > to a later patch to avoid having code/logic changes in this patch but I think > the change is so little that it makes sense to do it in this patch. I would like > to avoid having code like, even if it is for a brief period of time: > Generation* young = gch->get_gen(0); This change is part of the larger removal of the generation array which is the last part of this series of collector policy cleanups. Initially I was OK with moving get_young() and get_old() to this change even though that would mean that the change would grow considerably in size and wouldn't be a simple name change anymore. After all the only reason I wanted to keep it small was to make it easier for the reviewers :-) However, when I started to look at it in more detail I realized that removing get_gen(int) requires non-trivial changes in the serviceability agent which is the only reason why the last change in this series isn't done yet. It would be possible to introduce get_young() and get_old() and replace all usages of get_gen(int) within the VM, but leave get_gen(int) for the SA to use. I don't really like this approach, but if you prefer it over just changing the names it's a possibility. Thanks, /Jesper > > Thanks, > Stefan > > On 2014-05-02 03:39, Jesper Wilhelmsson wrote: >> Hi, >> >> This is a step towards removing the _generations array. The names gen0 and >> gen1 are replaced with young and old to use a more standardized vocabulary. >> >> Webrev: http://cr.openjdk.java.net/~jwilhelm/8042298/webrev/ >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8042298 >> >> Thanks, >> /Jesper > From vladimir.kozlov at oracle.com Wed May 14 22:01:10 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 14 May 2014 15:01:10 -0700 Subject: RFR(S): 8039042: G1: Phantom zeros in cardtable In-Reply-To: <5373D346.4010805@oracle.com> References: <5371F531.7040008@oracle.com> <5373D346.4010805@oracle.com> Message-ID: <5373E7A6.6090500@oracle.com> Jon, We already do that for UseMemSetInBOT flag in vm_version_sparc.cpp. And it is correctly use has_blk_init() check because is_T4() is not reliable. Vladimir On 5/14/14 1:34 PM, Jon Masamitsu wrote: > Per, > > Could you use a check on the sparc version to decide > on when to use memset. > > VM_Version::is_T4() > > in cpu/sparc/vm/vm_version_sparc.hpp > > Jon > > On 5/13/2014 3:34 AM, Per Liden wrote: >> Summary: We use memset to initialize the cardtable. On Sparc T4 (and >> later) memset uses some special instructions with the side-effect that >> the memory can temporarily be filled with zeros before the actual >> value is set. These phantom zeros can be observed by concurrent >> readers of this memory, which can trick the G1 post-barrier to >> incorrectly dirty a card in a young region. Please see the bug for a >> more details. >> >> A similar memset-problem has been observed and fixed before >> (JDK-6948537) when using memset on the BlockOffsetTable. Similar to >> the BlockOffsetTable, I'm using the UseMemSetInBOT flag as an >> indicator of whether it's safe to use memset or not. UseMemSetInBOT is >> not a perfect name for this flag as code other than the BOT might want >> to use it. It's also questionable of this should be a command-line >> flag at all. I've filed a separate RFE, JDK-8042930, to do a bit of >> clean up related to this flag. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8039042 >> Webrev: http://cr.openjdk.java.net/~pliden/8039042/webrev.0/ >> >> Thanks! >> /Per > From erik.helin at oracle.com Thu May 15 07:09:15 2014 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 15 May 2014 09:09:15 +0200 Subject: RFR: 8039489: Refactor test framework for dynamic VM options In-Reply-To: <53737D83.2050205@oracle.com> References: <535A524D.1000701@oracle.com> <53737D83.2050205@oracle.com> Message-ID: <2853847.YKHQLIsHKd@ehelin-laptop> Hi Dmitry, the patch looks good to me, thanks for taking care of these tests and also updating the testlibrary! Thanks, Erik On Wednesday 14 May 2014 18:28:19 PM Dmitry Fazunenko wrote: > Hi, > > Would you please look at the new version of a common library for testing > dynamic VM options. > > Description: > Some time ago a library for testing VM flags which could be changed > dynamically was integrated into the hs repository: > test/testlibrary/com/oracle/java/testlibrary/DynamicVMOptionChecker.java > test/testlibrary/com/oracle/java/testlibrary/TestDynamicVMOption.java > > The library was designed in not the most optimal way. > This change is an attempt to redesign it: > > test/testlibrary/com/oracle/java/testlibrary/DynamicVMOption.java > > is a new version which provides a convenient access to the HS API on > changing VM flags. > The tests based on the library are also refactored to meet new API. > > You comments will be much appreciated. > > http://cr.openjdk.java.net/~ehelin/dmitry/8039489/webrev.00/ > https://bugs.openjdk.java.net/browse/JDK-8039489 > > I want to thank Erik Helin for his incredible assistance with working on > this change. > > Thanks, > Dima From stefan.johansson at oracle.com Thu May 15 10:09:10 2014 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Thu, 15 May 2014 12:09:10 +0200 Subject: RFR: 8042298 - Remove the names gen0 and gen1 from the GC code In-Reply-To: <5373DE3E.9080803@oracle.com> References: <5362F75B.9020201@oracle.com> <53736B77.9030206@oracle.com> <5373DE3E.9080803@oracle.com> Message-ID: <53749246.4070201@oracle.com> On 2014-05-14 23:21, Jesper Wilhelmsson wrote: > Hi Stefan, > > Thanks for looking at this! See comments inline. > > Stefan Johansson skrev 14/5/14 15:11: >> Hi Jesper, >> >> Thanks for doing this work. Here are some comments: >> >> src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp: >> * TraceGen0Time and TraceGen1Time should be changed to TraceYoungTime >> and >> TraceOldTime for consistency, and I guess that will need a CCC request. >> >> src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.hpp >> >> >> * CollectGen0First -> CollectYoungFirst, same as above. Also used in >> tenuredGeneration.hpp. >> >> src/share/vm/gc_implementation/g1/g1CollectorPolicy.hpp >> * Comment starting on line 41 mentions TraceGen0Time and gen1. > > I agree that the names gen0 and gen1 should be completely removed, > which includes changing these flags too. Initially I wanted this > change to be a simple name change so that it would be easy to review > it. Changing the flags is borderline to be more than a name change, > but I'm fine with including that change. > > I'll file a CCC to change the flags. > Great, thanks. >> As an additional comment, I think that GenCollectedHeap::get_gen(int) >> should be >> broken up into two functions get_young() and get_old(). I get that >> this was left >> to a later patch to avoid having code/logic changes in this patch but >> I think >> the change is so little that it makes sense to do it in this patch. I >> would like >> to avoid having code like, even if it is for a brief period of time: >> Generation* young = gch->get_gen(0); > > This change is part of the larger removal of the generation array > which is the last part of this series of collector policy cleanups. > Initially I was OK with moving get_young() and get_old() to this > change even though that would mean that the change would grow > considerably in size and wouldn't be a simple name change anymore. > After all the only reason I wanted to keep it small was to make it > easier for the reviewers :-) > That's good, thanks =) > However, when I started to look at it in more detail I realized that > removing get_gen(int) requires non-trivial changes in the > serviceability agent which is the only reason why the last change in > this series isn't done yet. > I understand that you might want to do all the changes in the SA at once, and since this patch doesn't require you to change the SA I'm fine with leaving it as is. > It would be possible to introduce get_young() and get_old() and > replace all usages of get_gen(int) within the VM, but leave > get_gen(int) for the SA to use. I don't really like this approach, but > if you prefer it over just changing the names it's a possibility. > I'm fine with any of the approaches, none of them are perfect but both are good so I'll let you decide how you want to handle it. Thanks, Stefan > Thanks, > /Jesper > >> >> Thanks, >> Stefan >> >> On 2014-05-02 03:39, Jesper Wilhelmsson wrote: >>> Hi, >>> >>> This is a step towards removing the _generations array. The names >>> gen0 and >>> gen1 are replaced with young and old to use a more standardized >>> vocabulary. >>> >>> Webrev: http://cr.openjdk.java.net/~jwilhelm/8042298/webrev/ >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8042298 >>> >>> Thanks, >>> /Jesper >> From andrey.x.zakharov at oracle.com Thu May 15 11:29:06 2014 From: andrey.x.zakharov at oracle.com (Andrey Zakharov) Date: Thu, 15 May 2014 15:29:06 +0400 Subject: RFR: 8041506 - The test gc/g1/TestHumongousShrinkHeap.java reports that memory is not de-committed Message-ID: <5374A502.5030409@oracle.com> Hi. To proper testing of free list sorting we need to defragment memory with small young and humongous objects This is test scenario: Make enough space for new objects to prevent it going old. - allocate bunch of small objects, and a bit of humongous several times. Free almost all of allocated stuff. Check that heap shrinks after GC. webrev: http://cr.openjdk.java.net/~jwilhelm/8041506/webrev.02/ bug: https://bugs.openjdk.java.net/browse/JDK-8041506 Thanks. From jesper.wilhelmsson at oracle.com Thu May 15 11:36:01 2014 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Thu, 15 May 2014 13:36:01 +0200 Subject: RFR: 8041506 - The test gc/g1/TestHumongousShrinkHeap.java reports that memory is not de-committed In-Reply-To: <5374A502.5030409@oracle.com> References: <5374A502.5030409@oracle.com> Message-ID: <5374A6A1.6010404@oracle.com> Looks good! /Jesper Andrey Zakharov skrev 15/5/14 13:29: > Hi. > To proper testing of free list sorting we need to defragment memory with small > young and humongous objects > This is test scenario: > Make enough space for new objects to prevent it going old. > - allocate bunch of small objects, and a bit of humongous > several times. > > Free almost all of allocated stuff. Check that heap shrinks after GC. > > webrev: http://cr.openjdk.java.net/~jwilhelm/8041506/webrev.02/ > bug: https://bugs.openjdk.java.net/browse/JDK-8041506 > > Thanks. > From jesper.wilhelmsson at oracle.com Fri May 16 00:10:01 2014 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Fri, 16 May 2014 02:10:01 +0200 Subject: RFR: 8042298 - Remove the names gen0 and gen1 from the GC code In-Reply-To: <53749246.4070201@oracle.com> References: <5362F75B.9020201@oracle.com> <53736B77.9030206@oracle.com> <5373DE3E.9080803@oracle.com> <53749246.4070201@oracle.com> Message-ID: <53755759.7040506@oracle.com> Hi, A new webrev is available here: http://cr.openjdk.java.net/~jwilhelm/8042298/webrev.2/ In addition to the last webrev the flags TraceGen0Time, TraceGen1Time and CollectGen0First have been renamed to get rid of the Gen0/Gen1. Also, since CollectGen0First and ScavengeBeforeFullGC was doing the same thing for different collectors, ScavengeBeforeFullGC has been merged with CollectGen0First. A CCC request has been filed for the flag changes. Thanks, /Jesper Stefan Johansson skrev 15/5/14 12:09: > On 2014-05-14 23:21, Jesper Wilhelmsson wrote: >> Hi Stefan, >> >> Thanks for looking at this! See comments inline. >> >> Stefan Johansson skrev 14/5/14 15:11: >>> Hi Jesper, >>> >>> Thanks for doing this work. Here are some comments: >>> >>> src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp: >>> * TraceGen0Time and TraceGen1Time should be changed to TraceYoungTime and >>> TraceOldTime for consistency, and I guess that will need a CCC request. >>> >>> src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.hpp >>> >>> >>> * CollectGen0First -> CollectYoungFirst, same as above. Also used in >>> tenuredGeneration.hpp. >>> >>> src/share/vm/gc_implementation/g1/g1CollectorPolicy.hpp >>> * Comment starting on line 41 mentions TraceGen0Time and gen1. >> >> I agree that the names gen0 and gen1 should be completely removed, which >> includes changing these flags too. Initially I wanted this change to be a >> simple name change so that it would be easy to review it. Changing the flags >> is borderline to be more than a name change, but I'm fine with including that >> change. >> >> I'll file a CCC to change the flags. >> > Great, thanks. >>> As an additional comment, I think that GenCollectedHeap::get_gen(int) should be >>> broken up into two functions get_young() and get_old(). I get that this was left >>> to a later patch to avoid having code/logic changes in this patch but I think >>> the change is so little that it makes sense to do it in this patch. I would like >>> to avoid having code like, even if it is for a brief period of time: >>> Generation* young = gch->get_gen(0); >> >> This change is part of the larger removal of the generation array which is the >> last part of this series of collector policy cleanups. Initially I was OK with >> moving get_young() and get_old() to this change even though that would mean >> that the change would grow considerably in size and wouldn't be a simple name >> change anymore. After all the only reason I wanted to keep it small was to >> make it easier for the reviewers :-) >> > That's good, thanks =) >> However, when I started to look at it in more detail I realized that removing >> get_gen(int) requires non-trivial changes in the serviceability agent which is >> the only reason why the last change in this series isn't done yet. >> > I understand that you might want to do all the changes in the SA at once, and > since this patch doesn't require you to change the SA I'm fine with leaving it > as is. >> It would be possible to introduce get_young() and get_old() and replace all >> usages of get_gen(int) within the VM, but leave get_gen(int) for the SA to >> use. I don't really like this approach, but if you prefer it over just >> changing the names it's a possibility. >> > I'm fine with any of the approaches, none of them are perfect but both are good > so I'll let you decide how you want to handle it. > > Thanks, > Stefan >> Thanks, >> /Jesper >> >>> >>> Thanks, >>> Stefan >>> >>> On 2014-05-02 03:39, Jesper Wilhelmsson wrote: >>>> Hi, >>>> >>>> This is a step towards removing the _generations array. The names gen0 and >>>> gen1 are replaced with young and old to use a more standardized vocabulary. >>>> >>>> Webrev: http://cr.openjdk.java.net/~jwilhelm/8042298/webrev/ >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8042298 >>>> >>>> Thanks, >>>> /Jesper >>> > From volker.simonis at gmail.com Fri May 16 08:12:46 2014 From: volker.simonis at gmail.com (Volker Simonis) Date: Fri, 16 May 2014 10:12:46 +0200 Subject: RFR(S): 8039042: G1: Phantom zeros in cardtable In-Reply-To: <5373E7A6.6090500@oracle.com> References: <5371F531.7040008@oracle.com> <5373D346.4010805@oracle.com> <5373E7A6.6090500@oracle.com> Message-ID: Hi, I've updated "8042930: Consider removing UseMemSetInBOT" (https://bugs.openjdk.java.net/browse/JDK-8042930) with the below comments to document the problem we had with SPARC CPU detection for FUJITSU Sparc64-X CPUs. Regards, Volker When we really remove UseMemSetInBOT we should pay close attention to also consider other SPARC CPUs like for example Fujitsu Sparc64-X The following mail threads and bugs discussed the problems of checking the relevant CPU-features on Fujitsu CPUs "CPU-feature detection is broken for new Fujitsu Sparc64-X CPUs" http://mail.openjdk.java.net/pipermail/hotspot-dev/2013-November/thread.html#11834 "RFR(S): 8029190 : VM_Version::determine_features() asserts on Fujitsu Sparc64 CPUs" http://mail.openjdk.java.net/pipermail/hotspot-dev/2013-December/thread.html#11892 "VM_Version::determine_features() asserts on Fujitsu Sparc64 CPUs" https://bugs.openjdk.java.net/browse/JDK-8029190 I think we still couldn't find out until now how memset() is implemented on Solaris/Fujitsu Sparc-X and/or how BIS is working on Fujitsu Sparc-X. On Thu, May 15, 2014 at 12:01 AM, Vladimir Kozlov wrote: > Jon, > > We already do that for UseMemSetInBOT flag in vm_version_sparc.cpp. And it > is correctly use has_blk_init() check because is_T4() is not reliable. > > Vladimir > > > On 5/14/14 1:34 PM, Jon Masamitsu wrote: >> >> Per, >> >> Could you use a check on the sparc version to decide >> on when to use memset. >> >> VM_Version::is_T4() >> >> in cpu/sparc/vm/vm_version_sparc.hpp >> >> Jon >> >> On 5/13/2014 3:34 AM, Per Liden wrote: >>> >>> Summary: We use memset to initialize the cardtable. On Sparc T4 (and >>> later) memset uses some special instructions with the side-effect that >>> the memory can temporarily be filled with zeros before the actual >>> value is set. These phantom zeros can be observed by concurrent >>> readers of this memory, which can trick the G1 post-barrier to >>> incorrectly dirty a card in a young region. Please see the bug for a >>> more details. >>> >>> A similar memset-problem has been observed and fixed before >>> (JDK-6948537) when using memset on the BlockOffsetTable. Similar to >>> the BlockOffsetTable, I'm using the UseMemSetInBOT flag as an >>> indicator of whether it's safe to use memset or not. UseMemSetInBOT is >>> not a perfect name for this flag as code other than the BOT might want >>> to use it. It's also questionable of this should be a command-line >>> flag at all. I've filed a separate RFE, JDK-8042930, to do a bit of >>> clean up related to this flag. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8039042 >>> Webrev: http://cr.openjdk.java.net/~pliden/8039042/webrev.0/ >>> >>> Thanks! >>> /Per >> >> > From thomas.viessmann at oracle.com Fri May 16 08:26:36 2014 From: thomas.viessmann at oracle.com (Thomas Viessmann) Date: Fri, 16 May 2014 10:26:36 +0200 Subject: G1:long remark pauses Message-ID: <5375CBBC.2030904@oracle.com> Hi, I've been tuning a Java 7u51, Solaris 10, T4 system with 24G heap. My customer is not very happy with the remark pauses of up to 2 seconds. -XX:ParallelGCThreads=48 turned out to be the optimum. Here is the log file which contains the java args at the top: http://aubing.de.oracle.com/gclog/gc_log_03052014.log Any idea to drive the remark stop times further down? Thanks and Regards Thomas From bengt.rutisson at oracle.com Fri May 16 10:58:34 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Fri, 16 May 2014 12:58:34 +0200 Subject: G1:long remark pauses In-Reply-To: <5375CBBC.2030904@oracle.com> References: <5375CBBC.2030904@oracle.com> Message-ID: <5375EF5A.3050707@oracle.com> Hi Thomas, It looks like the application is using a lot of Reference objects. The time spent in remark is dominated by reference processing. See the attached graph generated from the log file you sent. You can try to see if adding -XX:+ParallelRefProcEnabled improves the situation. If the customer is interested in updating their application they might want to see if they can reduce the number of java.lang.ref.Reference objects they use. Hths, Bengt On 2014-05-16 10:26, Thomas Viessmann wrote: > Hi, > > > I've been tuning a Java 7u51, Solaris 10, T4 system with 24G heap. > My customer is not very happy with the remark pauses of up to 2 seconds. > -XX:ParallelGCThreads=48 turned out to be the optimum. Here is the > log file > which contains the java args at the top: > > http://aubing.de.oracle.com/gclog/gc_log_03052014.log > > Any idea to drive the remark stop times further down? > > > Thanks and Regards > > Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: remarks.png Type: image/png Size: 130904 bytes Desc: not available URL: From thomas.viessmann at oracle.com Fri May 16 11:10:38 2014 From: thomas.viessmann at oracle.com (Thomas Viessmann) Date: Fri, 16 May 2014 13:10:38 +0200 Subject: G1:long remark pauses In-Reply-To: <5375EF5A.3050707@oracle.com> References: <5375CBBC.2030904@oracle.com> <5375EF5A.3050707@oracle.com> Message-ID: <5375F22E.90003@oracle.com> Hi Bengt, Well, that's already done and it did improve things. argv[21]: -XX:+ParallelRefProcEnabled argv[22]: -XX:ParallelGCThreads=48 before -XX:+ParallelRefProcEnabled was set the stop times were in the range of 20 to 60 seconds. The application is a Cacao by Oracle. So they cannot change it. Thanks and Regards Thomas On 05/16/14 12:58, Bengt Rutisson wrote: > > Hi Thomas, > > It looks like the application is using a lot of Reference objects. The > time spent in remark is dominated by reference processing. See the > attached graph generated from the log file you sent. > > You can try to see if adding -XX:+ParallelRefProcEnabled improves the > situation. > > If the customer is interested in updating their application they might > want to see if they can reduce the number of java.lang.ref.Reference > objects they use. > > Hths, > Bengt > > > On 2014-05-16 10:26, Thomas Viessmann wrote: >> Hi, >> >> >> I've been tuning a Java 7u51, Solaris 10, T4 system with 24G heap. >> My customer is not very happy with the remark pauses of up to 2 >> seconds. >> -XX:ParallelGCThreads=48 turned out to be the optimum. Here is the >> log file >> which contains the java args at the top: >> >> http://aubing.de.oracle.com/gclog/gc_log_03052014.log >> >> Any idea to drive the remark stop times further down? >> >> >> Thanks and Regards >> >> Thomas > -- Oracle THOMAS VIESSMANN | Senior Principal Technical Support Engineer - Java Phone: +498914302496 | Mobile: +491743005467 Oracle Customer Technical Support - Java ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen ORACLE Deutschland B.V. & Co. KG Hauptverwaltung: Riesstr. 25, D-80992 Muenchen Registergericht: Amtsgericht Muenchen, HRA 95603 Gesch?ftsf?hrere: Juergen Kunz Komplement?rin: ORACLE Deutschland Verwaltung B.V. Hertogswetering 163/167, 3543 AS Utrecht, Niederlande Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher ------------------------------------------------------------------------ ------------------------------------------------------------------------ Green Oracle Oracle is committed to developing practices and products that help protect the environment -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: green-for-email-sig_0.gif Type: image/gif Size: 356 bytes Desc: not available URL: From bengt.rutisson at oracle.com Fri May 16 11:18:13 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Fri, 16 May 2014 13:18:13 +0200 Subject: G1:long remark pauses In-Reply-To: <5375F22E.90003@oracle.com> References: <5375CBBC.2030904@oracle.com> <5375EF5A.3050707@oracle.com> <5375F22E.90003@oracle.com> Message-ID: <5375F3F5.7030808@oracle.com> Hi Thomas, On 2014-05-16 13:10, Thomas Viessmann wrote: > Hi Bengt, > > Well, that's already done and it did improve things. > argv[21]: -XX:+ParallelRefProcEnabled > argv[22]: -XX:ParallelGCThreads=48 Sorry, I missed that. > > before -XX:+ParallelRefProcEnabled was set the stop times were in the range of 20 to 60 seconds. OK. Glad it helped some. :) > The application is a Cacao by Oracle. So they cannot change it. Is there some way of reducing the amount of reference objects that Cacao uses? Does it have cache sizes or similar that can be tuned. With a JFR recording we might be able to figure out where the reference objects come from. Thanks, Bengt > > Thanks and Regards > > Thomas > > > > > > > On 05/16/14 12:58, Bengt Rutisson wrote: >> >> Hi Thomas, >> >> It looks like the application is using a lot of Reference objects. >> The time spent in remark is dominated by reference processing. See >> the attached graph generated from the log file you sent. >> >> You can try to see if adding -XX:+ParallelRefProcEnabled improves the >> situation. >> >> If the customer is interested in updating their application they >> might want to see if they can reduce the number of >> java.lang.ref.Reference objects they use. >> >> Hths, >> Bengt >> >> >> On 2014-05-16 10:26, Thomas Viessmann wrote: >>> Hi, >>> >>> >>> I've been tuning a Java 7u51, Solaris 10, T4 system with 24G heap. >>> My customer is not very happy with the remark pauses of up to 2 >>> seconds. >>> -XX:ParallelGCThreads=48 turned out to be the optimum. Here is the >>> log file >>> which contains the java args at the top: >>> >>> http://aubing.de.oracle.com/gclog/gc_log_03052014.log >>> >>> Any idea to drive the remark stop times further down? >>> >>> >>> Thanks and Regards >>> >>> Thomas >> > > -- > Oracle > THOMAS VIESSMANN | Senior Principal Technical Support Engineer - Java > Phone: +498914302496 | Mobile: +491743005467 > > Oracle Customer Technical Support - Java > > ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen > > ORACLE Deutschland B.V. & Co. KG > Hauptverwaltung: Riesstr. 25, D-80992 Muenchen > Registergericht: Amtsgericht Muenchen, HRA 95603 > Gesch?ftsf?hrere: Juergen Kunz > > Komplement?rin: ORACLE Deutschland Verwaltung B.V. > Hertogswetering 163/167, 3543 AS Utrecht, Niederlande > Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 > Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher > > ------------------------------------------------------------------------ > ------------------------------------------------------------------------ > Green Oracle Oracle is committed to > developing practices and products that help protect the environment -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/gif Size: 658 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/gif Size: 356 bytes Desc: not available URL: From thomas.viessmann at oracle.com Fri May 16 11:34:03 2014 From: thomas.viessmann at oracle.com (Thomas Viessmann) Date: Fri, 16 May 2014 13:34:03 +0200 Subject: G1:long remark pauses In-Reply-To: <5375F3F5.7030808@oracle.com> References: <5375CBBC.2030904@oracle.com> <5375EF5A.3050707@oracle.com> <5375F22E.90003@oracle.com> <5375F3F5.7030808@oracle.com> Message-ID: <5375F7AB.1070908@oracle.com> Hi Bengt, Sure, the application has lots of objects and references. Downsizing the application has been tried The heap size of 24 g is already at minimum. A smaller heap gave OutOfmemory really quick. My question was more whether the remark phases could be optimized further. I assume this is not the case and we have reached the limitations of G1, right? Thanks and Regards Thomas On 05/16/14 13:18, Bengt Rutisson wrote: > > Hi Thomas, > > On 2014-05-16 13:10, Thomas Viessmann wrote: >> Hi Bengt, >> >> Well, that's already done and it did improve things. >> argv[21]: -XX:+ParallelRefProcEnabled >> argv[22]: -XX:ParallelGCThreads=48 > > Sorry, I missed that. > >> before -XX:+ParallelRefProcEnabled was set the stop times were in the range of 20 to 60 seconds. > > OK. Glad it helped some. :) > >> The application is a Cacao by Oracle. So they cannot change it. > > Is there some way of reducing the amount of reference objects that > Cacao uses? Does it have cache sizes or similar that can be tuned. > With a JFR recording we might be able to figure out where the > reference objects come from. > > Thanks, > Bengt > >> Thanks and Regards >> >> Thomas >> >> >> >> >> >> >> On 05/16/14 12:58, Bengt Rutisson wrote: >>> >>> Hi Thomas, >>> >>> It looks like the application is using a lot of Reference objects. >>> The time spent in remark is dominated by reference processing. See >>> the attached graph generated from the log file you sent. >>> >>> You can try to see if adding -XX:+ParallelRefProcEnabled improves >>> the situation. >>> >>> If the customer is interested in updating their application they >>> might want to see if they can reduce the number of >>> java.lang.ref.Reference objects they use. >>> >>> Hths, >>> Bengt >>> >>> >>> On 2014-05-16 10:26, Thomas Viessmann wrote: >>>> Hi, >>>> >>>> >>>> I've been tuning a Java 7u51, Solaris 10, T4 system with 24G heap. >>>> My customer is not very happy with the remark pauses of up to 2 >>>> seconds. >>>> -XX:ParallelGCThreads=48 turned out to be the optimum. Here is the >>>> log file >>>> which contains the java args at the top: >>>> >>>> http://aubing.de.oracle.com/gclog/gc_log_03052014.log >>>> >>>> Any idea to drive the remark stop times further down? >>>> >>>> >>>> Thanks and Regards >>>> >>>> Thomas >>> >> >> -- >> Oracle >> THOMAS VIESSMANN | Senior Principal Technical Support Engineer - Java >> Phone: +498914302496 | Mobile: +491743005467 >> >> Oracle Customer Technical Support - Java >> >> ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen >> >> ORACLE Deutschland B.V. & Co. KG >> Hauptverwaltung: Riesstr. 25, D-80992 Muenchen >> Registergericht: Amtsgericht Muenchen, HRA 95603 >> Gesch?ftsf?hrere: Juergen Kunz >> >> Komplement?rin: ORACLE Deutschland Verwaltung B.V. >> Hertogswetering 163/167, 3543 AS Utrecht, Niederlande >> Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 >> Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher >> >> ------------------------------------------------------------------------ >> ------------------------------------------------------------------------ >> Green Oracle Oracle is committed >> to developing practices and products that help protect the environment > -- Oracle THOMAS VIESSMANN | Senior Principal Technical Support Engineer - Java Phone: +498914302496 | Mobile: +491743005467 Oracle Customer Technical Support - Java ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen ORACLE Deutschland B.V. & Co. KG Hauptverwaltung: Riesstr. 25, D-80992 Muenchen Registergericht: Amtsgericht Muenchen, HRA 95603 Gesch?ftsf?hrere: Juergen Kunz Komplement?rin: ORACLE Deutschland Verwaltung B.V. Hertogswetering 163/167, 3543 AS Utrecht, Niederlande Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher ------------------------------------------------------------------------ ------------------------------------------------------------------------ Green Oracle Oracle is committed to developing practices and products that help protect the environment -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/gif Size: 658 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/gif Size: 356 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: green-for-email-sig_0.gif Type: image/gif Size: 356 bytes Desc: not available URL: From stefan.johansson at oracle.com Fri May 16 11:45:56 2014 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Fri, 16 May 2014 13:45:56 +0200 Subject: RFR: 8042298 - Remove the names gen0 and gen1 from the GC code In-Reply-To: <53755759.7040506@oracle.com> References: <5362F75B.9020201@oracle.com> <53736B77.9030206@oracle.com> <5373DE3E.9080803@oracle.com> <53749246.4070201@oracle.com> <53755759.7040506@oracle.com> Message-ID: <5375FA74.4000701@oracle.com> Thanks Jesper, On 2014-05-16 02:10, Jesper Wilhelmsson wrote: > Hi, > > A new webrev is available here: > http://cr.openjdk.java.net/~jwilhelm/8042298/webrev.2/ > > In addition to the last webrev the flags TraceGen0Time, TraceGen1Time > and CollectGen0First have been renamed to get rid of the Gen0/Gen1. > > Also, since CollectGen0First and ScavengeBeforeFullGC was doing the > same thing for different collectors, ScavengeBeforeFullGC has been > merged with CollectGen0First. > Looks good! Stefan > A CCC request has been filed for the flag changes. > > Thanks, > /Jesper > > > Stefan Johansson skrev 15/5/14 12:09: >> On 2014-05-14 23:21, Jesper Wilhelmsson wrote: >>> Hi Stefan, >>> >>> Thanks for looking at this! See comments inline. >>> >>> Stefan Johansson skrev 14/5/14 15:11: >>>> Hi Jesper, >>>> >>>> Thanks for doing this work. Here are some comments: >>>> >>>> src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp: >>>> * TraceGen0Time and TraceGen1Time should be changed to >>>> TraceYoungTime and >>>> TraceOldTime for consistency, and I guess that will need a CCC >>>> request. >>>> >>>> src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.hpp >>>> >>>> >>>> >>>> * CollectGen0First -> CollectYoungFirst, same as above. Also used in >>>> tenuredGeneration.hpp. >>>> >>>> src/share/vm/gc_implementation/g1/g1CollectorPolicy.hpp >>>> * Comment starting on line 41 mentions TraceGen0Time and gen1. >>> >>> I agree that the names gen0 and gen1 should be completely removed, >>> which >>> includes changing these flags too. Initially I wanted this change to >>> be a >>> simple name change so that it would be easy to review it. Changing >>> the flags >>> is borderline to be more than a name change, but I'm fine with >>> including that >>> change. >>> >>> I'll file a CCC to change the flags. >>> >> Great, thanks. >>>> As an additional comment, I think that >>>> GenCollectedHeap::get_gen(int) should be >>>> broken up into two functions get_young() and get_old(). I get that >>>> this was left >>>> to a later patch to avoid having code/logic changes in this patch >>>> but I think >>>> the change is so little that it makes sense to do it in this patch. >>>> I would like >>>> to avoid having code like, even if it is for a brief period of time: >>>> Generation* young = gch->get_gen(0); >>> >>> This change is part of the larger removal of the generation array >>> which is the >>> last part of this series of collector policy cleanups. Initially I >>> was OK with >>> moving get_young() and get_old() to this change even though that >>> would mean >>> that the change would grow considerably in size and wouldn't be a >>> simple name >>> change anymore. After all the only reason I wanted to keep it small >>> was to >>> make it easier for the reviewers :-) >>> >> That's good, thanks =) >>> However, when I started to look at it in more detail I realized that >>> removing >>> get_gen(int) requires non-trivial changes in the serviceability >>> agent which is >>> the only reason why the last change in this series isn't done yet. >>> >> I understand that you might want to do all the changes in the SA at >> once, and >> since this patch doesn't require you to change the SA I'm fine with >> leaving it >> as is. >>> It would be possible to introduce get_young() and get_old() and >>> replace all >>> usages of get_gen(int) within the VM, but leave get_gen(int) for the >>> SA to >>> use. I don't really like this approach, but if you prefer it over just >>> changing the names it's a possibility. >>> >> I'm fine with any of the approaches, none of them are perfect but >> both are good >> so I'll let you decide how you want to handle it. >> >> Thanks, >> Stefan >>> Thanks, >>> /Jesper >>> >>>> >>>> Thanks, >>>> Stefan >>>> >>>> On 2014-05-02 03:39, Jesper Wilhelmsson wrote: >>>>> Hi, >>>>> >>>>> This is a step towards removing the _generations array. The names >>>>> gen0 and >>>>> gen1 are replaced with young and old to use a more standardized >>>>> vocabulary. >>>>> >>>>> Webrev: http://cr.openjdk.java.net/~jwilhelm/8042298/webrev/ >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8042298 >>>>> >>>>> Thanks, >>>>> /Jesper >>>> >> From R.E.Jones at kent.ac.uk Fri May 16 11:46:47 2014 From: R.E.Jones at kent.ac.uk (Richard Jones) Date: Fri, 16 May 2014 12:46:47 +0100 Subject: G1:long remark pauses In-Reply-To: <5375EF5A.3050707@oracle.com> References: <5375CBBC.2030904@oracle.com> <5375EF5A.3050707@oracle.com> Message-ID: <69B8E9A3-3E9B-47BD-9804-45D98EFA6304@kent.ac.uk> Thomas et al Interesting. We took a look at reference processing in the context f building a fully on-the-fly copying collector for Jikes RVM. Our bottom line was that we found that the time to process references, in either stop-the-world fashion or by blocking any mutator that calls Reference.get(), was significant for the benchmarks we used (DaCapo) though not nearly as long as this. However, in our fully on-the-fly collector, although the length of the reference processing phase was longer, mutators ran concurrently so we eliminated pauses. It was also important to use deletion (snapshot at the beginning) rather than insertion (incremental update) barriers. While I realise this is probably of no immediate help to you :-), details will be published at ISMM 2014. You can find our paper at http://www.cs.kent.ac.uk/projects/gc/mirrorgc/. Regards Richard Richard Jones | Professor of Computer Systems | University of Kent School of Computing, University of Kent, Canterbury, Kent CT2 7NF, UK T +44 1227 827943 | F +44 1227 762811 | W http://www.cs.kent.ac.uk/~rej/ The Garbage Collection Handbook: The Art of Automatic Memory Management On 16 May 2014, at 11:58, Bengt Rutisson wrote: > > Hi Thomas, > > It looks like the application is using a lot of Reference objects. The time spent in remark is dominated by reference processing. See the attached graph generated from the log file you sent. > > You can try to see if adding -XX:+ParallelRefProcEnabled improves the situation. > > If the customer is interested in updating their application they might want to see if they can reduce the number of java.lang.ref.Reference objects they use. > > Hths, > Bengt > > > On 2014-05-16 10:26, Thomas Viessmann wrote: >> Hi, >> >> >> I've been tuning a Java 7u51, Solaris 10, T4 system with 24G heap. >> My customer is not very happy with the remark pauses of up to 2 seconds. >> -XX:ParallelGCThreads=48 turned out to be the optimum. Here is the log file >> which contains the java args at the top: >> >> http://aubing.de.oracle.com/gclog/gc_log_03052014.log >> >> Any idea to drive the remark stop times further down? >> >> >> Thanks and Regards >> >> Thomas > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.rutisson at oracle.com Fri May 16 11:53:07 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Fri, 16 May 2014 13:53:07 +0200 Subject: G1:long remark pauses In-Reply-To: <5375F7AB.1070908@oracle.com> References: <5375CBBC.2030904@oracle.com> <5375EF5A.3050707@oracle.com> <5375F22E.90003@oracle.com> <5375F3F5.7030808@oracle.com> <5375F7AB.1070908@oracle.com> Message-ID: <5375FC23.5040004@oracle.com> Hi again Thomas, On 2014-05-16 13:34, Thomas Viessmann wrote: > Hi Bengt, > > Sure, the application has lots of objects and references. > Downsizing the application has been tried The heap size of 24 g is > already at minimum. A smaller heap gave OutOfmemory really quick. > My question was more whether the remark phases could be optimized > further. I assume this is not the case and we have reached the > limitations > of G1, right? How many reference objects does the application use? Can you run it with -XX:+PrintReferenceGC to see how many there are? If there are a lot of them I don't think there is much more that can be done for the remark phase. But if there are not that many I guess it means that the remark phase is inefficient. Have you tried any of the other GCs? How do they behave with this application? Thanks, Bengt > > Thanks and Regards > > Thomas > > > > On 05/16/14 13:18, Bengt Rutisson wrote: >> >> Hi Thomas, >> >> On 2014-05-16 13:10, Thomas Viessmann wrote: >>> Hi Bengt, >>> >>> Well, that's already done and it did improve things. >>> argv[21]: -XX:+ParallelRefProcEnabled >>> argv[22]: -XX:ParallelGCThreads=48 >> >> Sorry, I missed that. >> >>> before -XX:+ParallelRefProcEnabled was set the stop times were in the range of 20 to 60 seconds. >> >> OK. Glad it helped some. :) >> >>> The application is a Cacao by Oracle. So they cannot change it. >> >> Is there some way of reducing the amount of reference objects that >> Cacao uses? Does it have cache sizes or similar that can be tuned. >> With a JFR recording we might be able to figure out where the >> reference objects come from. >> >> Thanks, >> Bengt >> >>> Thanks and Regards >>> >>> Thomas >>> >>> >>> >>> >>> >>> >>> On 05/16/14 12:58, Bengt Rutisson wrote: >>>> >>>> Hi Thomas, >>>> >>>> It looks like the application is using a lot of Reference objects. >>>> The time spent in remark is dominated by reference processing. See >>>> the attached graph generated from the log file you sent. >>>> >>>> You can try to see if adding -XX:+ParallelRefProcEnabled improves >>>> the situation. >>>> >>>> If the customer is interested in updating their application they >>>> might want to see if they can reduce the number of >>>> java.lang.ref.Reference objects they use. >>>> >>>> Hths, >>>> Bengt >>>> >>>> >>>> On 2014-05-16 10:26, Thomas Viessmann wrote: >>>>> Hi, >>>>> >>>>> >>>>> I've been tuning a Java 7u51, Solaris 10, T4 system with 24G heap. >>>>> My customer is not very happy with the remark pauses of up to 2 >>>>> seconds. >>>>> -XX:ParallelGCThreads=48 turned out to be the optimum. Here is >>>>> the log file >>>>> which contains the java args at the top: >>>>> >>>>> http://aubing.de.oracle.com/gclog/gc_log_03052014.log >>>>> >>>>> Any idea to drive the remark stop times further down? >>>>> >>>>> >>>>> Thanks and Regards >>>>> >>>>> Thomas >>>> >>> >>> -- >>> Oracle >>> THOMAS VIESSMANN | Senior Principal Technical Support Engineer - Java >>> Phone: +498914302496 | Mobile: +491743005467 >>> >>> Oracle Customer Technical Support - Java >>> >>> ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen >>> >>> ORACLE Deutschland B.V. & Co. KG >>> Hauptverwaltung: Riesstr. 25, D-80992 Muenchen >>> Registergericht: Amtsgericht Muenchen, HRA 95603 >>> Gesch?ftsf?hrere: Juergen Kunz >>> >>> Komplement?rin: ORACLE Deutschland Verwaltung B.V. >>> Hertogswetering 163/167, 3543 AS Utrecht, Niederlande >>> Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 >>> Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher >>> >>> ------------------------------------------------------------------------ >>> ------------------------------------------------------------------------ >>> Green Oracle Oracle is committed >>> to developing practices and products that help protect the environment >> > > -- > Oracle > THOMAS VIESSMANN | Senior Principal Technical Support Engineer - Java > Phone: +498914302496 | Mobile: +491743005467 > > Oracle Customer Technical Support - Java > > ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen > > ORACLE Deutschland B.V. & Co. KG > Hauptverwaltung: Riesstr. 25, D-80992 Muenchen > Registergericht: Amtsgericht Muenchen, HRA 95603 > Gesch?ftsf?hrere: Juergen Kunz > > Komplement?rin: ORACLE Deutschland Verwaltung B.V. > Hertogswetering 163/167, 3543 AS Utrecht, Niederlande > Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 > Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher > > ------------------------------------------------------------------------ > ------------------------------------------------------------------------ > Green Oracle Oracle is committed to > developing practices and products that help protect the environment -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/gif Size: 658 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/gif Size: 356 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/gif Size: 658 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/gif Size: 356 bytes Desc: not available URL: From thomas.viessmann at oracle.com Fri May 16 12:01:42 2014 From: thomas.viessmann at oracle.com (Thomas Viessmann) Date: Fri, 16 May 2014 14:01:42 +0200 Subject: G1:long remark pauses In-Reply-To: <5375FC23.5040004@oracle.com> References: <5375CBBC.2030904@oracle.com> <5375EF5A.3050707@oracle.com> <5375F22E.90003@oracle.com> <5375F3F5.7030808@oracle.com> <5375F7AB.1070908@oracle.com> <5375FC23.5040004@oracle.com> Message-ID: <5375FE26.4080009@oracle.com> Hi Bengt, Thanks for confirming. ParallelOld had stop pauses in the range of 20 to 30 seconds. CMS was a disaster due to extreme fragmentation and high promotion rate even with huge eden and survivors. There are definitely lots of references. I can find out details. Thanks and Regards Thomas On 05/16/14 13:53, Bengt Rutisson wrote: > > Hi again Thomas, > > > On 2014-05-16 13:34, Thomas Viessmann wrote: >> Hi Bengt, >> >> Sure, the application has lots of objects and references. >> Downsizing the application has been tried The heap size of 24 g is >> already at minimum. A smaller heap gave OutOfmemory really quick. >> My question was more whether the remark phases could be optimized >> further. I assume this is not the case and we have reached the >> limitations >> of G1, right? > > > How many reference objects does the application use? Can you run it > with -XX:+PrintReferenceGC to see how many there are? > > If there are a lot of them I don't think there is much more that can > be done for the remark phase. But if there are not that many I guess > it means that the remark phase is inefficient. > > Have you tried any of the other GCs? How do they behave with this > application? > > Thanks, > Bengt > > >> >> Thanks and Regards >> >> Thomas >> >> >> >> On 05/16/14 13:18, Bengt Rutisson wrote: >>> >>> Hi Thomas, >>> >>> On 2014-05-16 13:10, Thomas Viessmann wrote: >>>> Hi Bengt, >>>> >>>> Well, that's already done and it did improve things. >>>> argv[21]: -XX:+ParallelRefProcEnabled >>>> argv[22]: -XX:ParallelGCThreads=48 >>> >>> Sorry, I missed that. >>> >>>> before -XX:+ParallelRefProcEnabled was set the stop times were in the range of 20 to 60 seconds. >>> >>> OK. Glad it helped some. :) >>> >>>> The application is a Cacao by Oracle. So they cannot change it. >>> >>> Is there some way of reducing the amount of reference objects that >>> Cacao uses? Does it have cache sizes or similar that can be tuned. >>> With a JFR recording we might be able to figure out where the >>> reference objects come from. >>> >>> Thanks, >>> Bengt >>> >>>> Thanks and Regards >>>> >>>> Thomas >>>> >>>> >>>> >>>> >>>> >>>> >>>> On 05/16/14 12:58, Bengt Rutisson wrote: >>>>> >>>>> Hi Thomas, >>>>> >>>>> It looks like the application is using a lot of Reference objects. >>>>> The time spent in remark is dominated by reference processing. See >>>>> the attached graph generated from the log file you sent. >>>>> >>>>> You can try to see if adding -XX:+ParallelRefProcEnabled improves >>>>> the situation. >>>>> >>>>> If the customer is interested in updating their application they >>>>> might want to see if they can reduce the number of >>>>> java.lang.ref.Reference objects they use. >>>>> >>>>> Hths, >>>>> Bengt >>>>> >>>>> >>>>> On 2014-05-16 10:26, Thomas Viessmann wrote: >>>>>> Hi, >>>>>> >>>>>> >>>>>> I've been tuning a Java 7u51, Solaris 10, T4 system with 24G heap. >>>>>> My customer is not very happy with the remark pauses of up to 2 >>>>>> seconds. >>>>>> -XX:ParallelGCThreads=48 turned out to be the optimum. Here is >>>>>> the log file >>>>>> which contains the java args at the top: >>>>>> >>>>>> http://aubing.de.oracle.com/gclog/gc_log_03052014.log >>>>>> >>>>>> Any idea to drive the remark stop times further down? >>>>>> >>>>>> >>>>>> Thanks and Regards >>>>>> >>>>>> Thomas >>>>> >>>> >>>> -- >>>> Oracle >>>> THOMAS VIESSMANN | Senior Principal Technical Support Engineer - Java >>>> Phone: +498914302496 | Mobile: +491743005467 >>>> >>>> Oracle Customer Technical Support - Java >>>> >>>> ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen >>>> >>>> ORACLE Deutschland B.V. & Co. KG >>>> Hauptverwaltung: Riesstr. 25, D-80992 Muenchen >>>> Registergericht: Amtsgericht Muenchen, HRA 95603 >>>> Gesch?ftsf?hrere: Juergen Kunz >>>> >>>> Komplement?rin: ORACLE Deutschland Verwaltung B.V. >>>> Hertogswetering 163/167, 3543 AS Utrecht, Niederlande >>>> Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 >>>> Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher >>>> >>>> ------------------------------------------------------------------------ >>>> ------------------------------------------------------------------------ >>>> Green Oracle Oracle is committed >>>> to developing practices and products that help protect the environment >>> >> >> -- >> Oracle >> THOMAS VIESSMANN | Senior Principal Technical Support Engineer - Java >> Phone: +498914302496 | Mobile: +491743005467 >> >> Oracle Customer Technical Support - Java >> >> ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen >> >> ORACLE Deutschland B.V. & Co. KG >> Hauptverwaltung: Riesstr. 25, D-80992 Muenchen >> Registergericht: Amtsgericht Muenchen, HRA 95603 >> Gesch?ftsf?hrere: Juergen Kunz >> >> Komplement?rin: ORACLE Deutschland Verwaltung B.V. >> Hertogswetering 163/167, 3543 AS Utrecht, Niederlande >> Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 >> Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher >> >> ------------------------------------------------------------------------ >> ------------------------------------------------------------------------ >> Green Oracle Oracle is committed >> to developing practices and products that help protect the environment > -- Oracle THOMAS VIESSMANN | Senior Principal Technical Support Engineer - Java Phone: +498914302496 | Mobile: +491743005467 Oracle Customer Technical Support - Java ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen ORACLE Deutschland B.V. & Co. KG Hauptverwaltung: Riesstr. 25, D-80992 Muenchen Registergericht: Amtsgericht Muenchen, HRA 95603 Gesch?ftsf?hrere: Juergen Kunz Komplement?rin: ORACLE Deutschland Verwaltung B.V. Hertogswetering 163/167, 3543 AS Utrecht, Niederlande Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher ------------------------------------------------------------------------ ------------------------------------------------------------------------ Green Oracle Oracle is committed to developing practices and products that help protect the environment -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/gif Size: 658 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/gif Size: 356 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/gif Size: 658 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/gif Size: 356 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: green-for-email-sig_0.gif Type: image/gif Size: 356 bytes Desc: not available URL: From bengt.rutisson at oracle.com Fri May 16 12:32:05 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Fri, 16 May 2014 14:32:05 +0200 Subject: G1:long remark pauses In-Reply-To: <5375FE26.4080009@oracle.com> References: <5375CBBC.2030904@oracle.com> <5375EF5A.3050707@oracle.com> <5375F22E.90003@oracle.com> <5375F3F5.7030808@oracle.com> <5375F7AB.1070908@oracle.com> <5375FC23.5040004@oracle.com> <5375FE26.4080009@oracle.com> Message-ID: Hi Thomas, > 16 maj 2014 kl. 14:01 skrev Thomas Viessmann : > > Hi Bengt, > > > Thanks for confirming. ParallelOld had stop pauses in the range of 20 to 30 seconds. > CMS was a disaster due to extreme fragmentation and high promotion rate even with > huge eden and survivors. Ok, so even with the long remark pauses G1 is performing better than the other GCs? > There are definitely lots of references. I can find out > details. Thanks, it would be interesting to get this data. Thanks, Bengt > > Thanks and Regards > > Thomas > > > >> On 05/16/14 13:53, Bengt Rutisson wrote: >> >> Hi again Thomas, >> >> >>> On 2014-05-16 13:34, Thomas Viessmann wrote: >>> Hi Bengt, >>> >>> Sure, the application has lots of objects and references. >>> Downsizing the application has been tried The heap size of 24 g is >>> already at minimum. A smaller heap gave OutOfmemory really quick. >>> My question was more whether the remark phases could be optimized >>> further. I assume this is not the case and we have reached the limitations >>> of G1, right? >> >> >> How many reference objects does the application use? Can you run it with -XX:+PrintReferenceGC to see how many there are? >> >> If there are a lot of them I don't think there is much more that can be done for the remark phase. But if there are not that many I guess it means that the remark phase is inefficient. >> >> Have you tried any of the other GCs? How do they behave with this application? >> >> Thanks, >> Bengt >> >> >>> >>> Thanks and Regards >>> >>> Thomas >>> >>> >>> >>>> On 05/16/14 13:18, Bengt Rutisson wrote: >>>> >>>> Hi Thomas, >>>> >>>>> On 2014-05-16 13:10, Thomas Viessmann wrote: >>>>> Hi Bengt, >>>>> >>>>> Well, that's already done and it did improve things. >>>>> argv[21]: -XX:+ParallelRefProcEnabled >>>>> argv[22]: -XX:ParallelGCThreads=48 >>>> >>>> Sorry, I missed that. >>>> >>>>> before -XX:+ParallelRefProcEnabled was set the stop times were in the range of 20 to 60 seconds. >>>> >>>> OK. Glad it helped some. :) >>>> >>>>> The application is a Cacao by Oracle. So they cannot change it. >>>> >>>> Is there some way of reducing the amount of reference objects that Cacao uses? Does it have cache sizes or similar that can be tuned. With a JFR recording we might be able to figure out where the reference objects come from. >>>> >>>> Thanks, >>>> Bengt >>>> >>>>> Thanks and Regards >>>>> >>>>> Thomas >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>>> On 05/16/14 12:58, Bengt Rutisson wrote: >>>>>> >>>>>> Hi Thomas, >>>>>> >>>>>> It looks like the application is using a lot of Reference objects. The time spent in remark is dominated by reference processing. See the attached graph generated from the log file you sent. >>>>>> >>>>>> You can try to see if adding -XX:+ParallelRefProcEnabled improves the situation. >>>>>> >>>>>> If the customer is interested in updating their application they might want to see if they can reduce the number of java.lang.ref.Reference objects they use. >>>>>> >>>>>> Hths, >>>>>> Bengt >>>>>> >>>>>> >>>>>>> On 2014-05-16 10:26, Thomas Viessmann wrote: >>>>>>> Hi, >>>>>>> >>>>>>> >>>>>>> I've been tuning a Java 7u51, Solaris 10, T4 system with 24G heap. >>>>>>> My customer is not very happy with the remark pauses of up to 2 seconds. >>>>>>> -XX:ParallelGCThreads=48 turned out to be the optimum. Here is the log file >>>>>>> which contains the java args at the top: >>>>>>> >>>>>>> http://aubing.de.oracle.com/gclog/gc_log_03052014.log >>>>>>> >>>>>>> Any idea to drive the remark stop times further down? >>>>>>> >>>>>>> >>>>>>> Thanks and Regards >>>>>>> >>>>>>> Thomas >>>>> >>>>> -- >>>>> >>>>> THOMAS VIESSMANN | Senior Principal Technical Support Engineer - Java >>>>> Phone: +498914302496 | Mobile: +491743005467 >>>>> Oracle Customer Technical Support - Java >>>>> >>>>> ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen >>>>> >>>>> ORACLE Deutschland B.V. & Co. KG >>>>> Hauptverwaltung: Riesstr. 25, D-80992 Muenchen >>>>> Registergericht: Amtsgericht Muenchen, HRA 95603 >>>>> Gesch?ftsf?hrere: Juergen Kunz >>>>> >>>>> Komplement?rin: ORACLE Deutschland Verwaltung B.V. >>>>> Hertogswetering 163/167, 3543 AS Utrecht, Niederlande >>>>> Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 >>>>> Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher >>>>> >>>>> Oracle is committed to developing practices and products that help protect the environment >>> >>> -- >>> >>> THOMAS VIESSMANN | Senior Principal Technical Support Engineer - Java >>> Phone: +498914302496 | Mobile: +491743005467 >>> Oracle Customer Technical Support - Java >>> >>> ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen >>> >>> ORACLE Deutschland B.V. & Co. KG >>> Hauptverwaltung: Riesstr. 25, D-80992 Muenchen >>> Registergericht: Amtsgericht Muenchen, HRA 95603 >>> Gesch?ftsf?hrere: Juergen Kunz >>> >>> Komplement?rin: ORACLE Deutschland Verwaltung B.V. >>> Hertogswetering 163/167, 3543 AS Utrecht, Niederlande >>> Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 >>> Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher >>> >>> Oracle is committed to developing practices and products that help protect the environment > > -- > > THOMAS VIESSMANN | Senior Principal Technical Support Engineer - Java > Phone: +498914302496 | Mobile: +491743005467 > Oracle Customer Technical Support - Java > > ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen > > ORACLE Deutschland B.V. & Co. KG > Hauptverwaltung: Riesstr. 25, D-80992 Muenchen > Registergericht: Amtsgericht Muenchen, HRA 95603 > Gesch?ftsf?hrere: Juergen Kunz > > Komplement?rin: ORACLE Deutschland Verwaltung B.V. > Hertogswetering 163/167, 3543 AS Utrecht, Niederlande > Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 > Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher > > Oracle is committed to developing practices and products that help protect the environment -------------- next part -------------- An HTML attachment was scrubbed... URL: From charlie.hunt at oracle.com Fri May 16 12:40:03 2014 From: charlie.hunt at oracle.com (charlie hunt) Date: Fri, 16 May 2014 07:40:03 -0500 Subject: G1:long remark pauses In-Reply-To: References: <5375CBBC.2030904@oracle.com> <5375EF5A.3050707@oracle.com> <5375F22E.90003@oracle.com> <5375F3F5.7030808@oracle.com> <5375F7AB.1070908@oracle.com> <5375FC23.5040004@oracle.com> <5375FE26.4080009@oracle.com> Message-ID: <53760723.3070002@oracle.com> From the sound of what Thomas is describing, this might be one of those apps that's making heavy use of SoftReferences. Output from -XX:+PrintReferenceGC as Bengt suggested will show if that's the case. If we see a large number of SoftReferences being processed per GC, we may get further help with tuning the SoftReference reclamation policy, (-XX:SoftRefLRUPolicyMSPerMB). charlie On 05/16/2014 07:32 AM, Bengt Rutisson wrote: > > Hi Thomas, > > > 16 maj 2014 kl. 14:01 skrev Thomas Viessmann > >: > >> Hi Bengt, >> >> >> Thanks for confirming. ParallelOld had stop pauses in the range of 20 >> to 30 seconds. >> CMS was a disaster due to extreme fragmentation and high promotion >> rate even with >> huge eden and survivors. > > Ok, so even with the long remark pauses G1 is performing better than > the other GCs? > > >> There are definitely lots of references. I can find out >> details. > > Thanks, it would be interesting to get this data. > > Thanks, > Bengt > >> >> Thanks and Regards >> >> Thomas >> >> >> >> On 05/16/14 13:53, Bengt Rutisson wrote: >>> >>> Hi again Thomas, >>> >>> >>> On 2014-05-16 13:34, Thomas Viessmann wrote: >>>> Hi Bengt, >>>> >>>> Sure, the application has lots of objects and references. >>>> Downsizing the application has been tried The heap size of 24 g is >>>> already at minimum. A smaller heap gave OutOfmemory really quick. >>>> My question was more whether the remark phases could be optimized >>>> further. I assume this is not the case and we have reached the >>>> limitations >>>> of G1, right? >>> >>> >>> How many reference objects does the application use? Can you run it >>> with -XX:+PrintReferenceGC to see how many there are? >>> >>> If there are a lot of them I don't think there is much more that can >>> be done for the remark phase. But if there are not that many I guess >>> it means that the remark phase is inefficient. >>> >>> Have you tried any of the other GCs? How do they behave with this >>> application? >>> >>> Thanks, >>> Bengt >>> >>> >>>> >>>> Thanks and Regards >>>> >>>> Thomas >>>> >>>> >>>> >>>> On 05/16/14 13:18, Bengt Rutisson wrote: >>>>> >>>>> Hi Thomas, >>>>> >>>>> On 2014-05-16 13:10, Thomas Viessmann wrote: >>>>>> Hi Bengt, >>>>>> >>>>>> Well, that's already done and it did improve things. >>>>>> argv[21]: -XX:+ParallelRefProcEnabled >>>>>> argv[22]: -XX:ParallelGCThreads=48 >>>>> >>>>> Sorry, I missed that. >>>>> >>>>>> before -XX:+ParallelRefProcEnabled was set the stop times were in the range of 20 to 60 seconds. >>>>> >>>>> OK. Glad it helped some. :) >>>>> >>>>>> The application is a Cacao by Oracle. So they cannot change it. >>>>> >>>>> Is there some way of reducing the amount of reference objects that >>>>> Cacao uses? Does it have cache sizes or similar that can be tuned. >>>>> With a JFR recording we might be able to figure out where the >>>>> reference objects come from. >>>>> >>>>> Thanks, >>>>> Bengt >>>>> >>>>>> Thanks and Regards >>>>>> >>>>>> Thomas >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On 05/16/14 12:58, Bengt Rutisson wrote: >>>>>>> >>>>>>> Hi Thomas, >>>>>>> >>>>>>> It looks like the application is using a lot of Reference >>>>>>> objects. The time spent in remark is dominated by reference >>>>>>> processing. See the attached graph generated from the log file >>>>>>> you sent. >>>>>>> >>>>>>> You can try to see if adding -XX:+ParallelRefProcEnabled >>>>>>> improves the situation. >>>>>>> >>>>>>> If the customer is interested in updating their application they >>>>>>> might want to see if they can reduce the number of >>>>>>> java.lang.ref.Reference objects they use. >>>>>>> >>>>>>> Hths, >>>>>>> Bengt >>>>>>> >>>>>>> >>>>>>> On 2014-05-16 10:26, Thomas Viessmann wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> >>>>>>>> I've been tuning a Java 7u51, Solaris 10, T4 system with 24G heap. >>>>>>>> My customer is not very happy with the remark pauses of up to >>>>>>>> 2 seconds. >>>>>>>> -XX:ParallelGCThreads=48 turned out to be the optimum. Here is >>>>>>>> the log file >>>>>>>> which contains the java args at the top: >>>>>>>> >>>>>>>> http://aubing.de.oracle.com/gclog/gc_log_03052014.log >>>>>>>> >>>>>>>> Any idea to drive the remark stop times further down? >>>>>>>> >>>>>>>> >>>>>>>> Thanks and Regards >>>>>>>> >>>>>>>> Thomas >>>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> THOMAS VIESSMANN | Senior Principal Technical Support Engineer - Java >>>>>> Phone: +498914302496 | Mobile: +491743005467 >>>>>> >>>>>> Oracle Customer Technical Support - Java >>>>>> >>>>>> ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen >>>>>> >>>>>> ORACLE Deutschland B.V. & Co. KG >>>>>> Hauptverwaltung: Riesstr. 25, D-80992 Muenchen >>>>>> Registergericht: Amtsgericht Muenchen, HRA 95603 >>>>>> Gesch?ftsf?hrere: Juergen Kunz >>>>>> >>>>>> Komplement?rin: ORACLE Deutschland Verwaltung B.V. >>>>>> Hertogswetering 163/167, 3543 AS Utrecht, Niederlande >>>>>> Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 >>>>>> Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher >>>>>> >>>>>> ------------------------------------------------------------------------ >>>>>> ------------------------------------------------------------------------ >>>>>> Oracle >>>>>> is committed to developing practices and products that help >>>>>> protect the environment >>>>> >>>> >>>> -- >>>> >>>> THOMAS VIESSMANN | Senior Principal Technical Support Engineer - Java >>>> Phone: +498914302496 | Mobile: +491743005467 >>>> >>>> Oracle Customer Technical Support - Java >>>> >>>> ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen >>>> >>>> ORACLE Deutschland B.V. & Co. KG >>>> Hauptverwaltung: Riesstr. 25, D-80992 Muenchen >>>> Registergericht: Amtsgericht Muenchen, HRA 95603 >>>> Gesch?ftsf?hrere: Juergen Kunz >>>> >>>> Komplement?rin: ORACLE Deutschland Verwaltung B.V. >>>> Hertogswetering 163/167, 3543 AS Utrecht, Niederlande >>>> Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 >>>> Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher >>>> >>>> ------------------------------------------------------------------------ >>>> ------------------------------------------------------------------------ >>>> Oracle is >>>> committed to developing practices and products that help protect >>>> the environment >>> >> >> -- >> >> THOMAS VIESSMANN | Senior Principal Technical Support Engineer - Java >> Phone: +498914302496 | Mobile: +491743005467 >> >> Oracle Customer Technical Support - Java >> >> ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen >> >> ORACLE Deutschland B.V. & Co. KG >> Hauptverwaltung: Riesstr. 25, D-80992 Muenchen >> Registergericht: Amtsgericht Muenchen, HRA 95603 >> Gesch?ftsf?hrere: Juergen Kunz >> >> Komplement?rin: ORACLE Deutschland Verwaltung B.V. >> Hertogswetering 163/167, 3543 AS Utrecht, Niederlande >> Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 >> Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher >> >> ------------------------------------------------------------------------ >> ------------------------------------------------------------------------ >> Oracle >> is committed to developing practices and products that help protect >> the environment -------------- next part -------------- An HTML attachment was scrubbed... URL: From per.liden at oracle.com Fri May 16 14:06:45 2014 From: per.liden at oracle.com (Per Liden) Date: Fri, 16 May 2014 16:06:45 +0200 Subject: RFR(S): 8039042: G1: Phantom zeros in cardtable In-Reply-To: References: <5371F531.7040008@oracle.com> <5373D346.4010805@oracle.com> <5373E7A6.6090500@oracle.com> Message-ID: <53761B75.2060102@oracle.com> Thanks, good information there. Just to avoid any potential misunderstands around the RFE I filled. I'm basically just saying we should consider not exposing this as a command like flag, but we obviously need to keep it (with a better name) as an internal flag somewhere. cheers, /Per On 2014-05-16 10:12, Volker Simonis wrote: > Hi, > > I've updated "8042930: Consider removing UseMemSetInBOT" > (https://bugs.openjdk.java.net/browse/JDK-8042930) with the below > comments to document the problem we had with SPARC CPU detection for > FUJITSU Sparc64-X CPUs. > > Regards, > Volker > > > When we really remove UseMemSetInBOT we should pay close attention to > also consider other SPARC CPUs like for example Fujitsu Sparc64-X > > The following mail threads and bugs discussed the problems of checking > the relevant CPU-features on Fujitsu CPUs > > "CPU-feature detection is broken for new Fujitsu Sparc64-X CPUs" > http://mail.openjdk.java.net/pipermail/hotspot-dev/2013-November/thread.html#11834 > > "RFR(S): 8029190 : VM_Version::determine_features() asserts on Fujitsu > Sparc64 CPUs" > http://mail.openjdk.java.net/pipermail/hotspot-dev/2013-December/thread.html#11892 > > "VM_Version::determine_features() asserts on Fujitsu Sparc64 CPUs" > https://bugs.openjdk.java.net/browse/JDK-8029190 > > I think we still couldn't find out until now how memset() is > implemented on Solaris/Fujitsu Sparc-X and/or how BIS is working on > Fujitsu Sparc-X. > > > On Thu, May 15, 2014 at 12:01 AM, Vladimir Kozlov > wrote: >> Jon, >> >> We already do that for UseMemSetInBOT flag in vm_version_sparc.cpp. And it >> is correctly use has_blk_init() check because is_T4() is not reliable. >> >> Vladimir >> >> >> On 5/14/14 1:34 PM, Jon Masamitsu wrote: >>> Per, >>> >>> Could you use a check on the sparc version to decide >>> on when to use memset. >>> >>> VM_Version::is_T4() >>> >>> in cpu/sparc/vm/vm_version_sparc.hpp >>> >>> Jon >>> >>> On 5/13/2014 3:34 AM, Per Liden wrote: >>>> Summary: We use memset to initialize the cardtable. On Sparc T4 (and >>>> later) memset uses some special instructions with the side-effect that >>>> the memory can temporarily be filled with zeros before the actual >>>> value is set. These phantom zeros can be observed by concurrent >>>> readers of this memory, which can trick the G1 post-barrier to >>>> incorrectly dirty a card in a young region. Please see the bug for a >>>> more details. >>>> >>>> A similar memset-problem has been observed and fixed before >>>> (JDK-6948537) when using memset on the BlockOffsetTable. Similar to >>>> the BlockOffsetTable, I'm using the UseMemSetInBOT flag as an >>>> indicator of whether it's safe to use memset or not. UseMemSetInBOT is >>>> not a perfect name for this flag as code other than the BOT might want >>>> to use it. It's also questionable of this should be a command-line >>>> flag at all. I've filed a separate RFE, JDK-8042930, to do a bit of >>>> clean up related to this flag. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8039042 >>>> Webrev: http://cr.openjdk.java.net/~pliden/8039042/webrev.0/ >>>> >>>> Thanks! >>>> /Per >>> From jesper.wilhelmsson at oracle.com Mon May 19 21:17:35 2014 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Mon, 19 May 2014 23:17:35 +0200 Subject: RFR: 8042298 - Remove the names gen0 and gen1 from the GC code In-Reply-To: <5375FA74.4000701@oracle.com> References: <5362F75B.9020201@oracle.com> <53736B77.9030206@oracle.com> <5373DE3E.9080803@oracle.com> <53749246.4070201@oracle.com> <53755759.7040506@oracle.com> <5375FA74.4000701@oracle.com> Message-ID: <537A74EF.9040101@oracle.com> Hi, After some discussions we have concluded that the merged flag used to trigger young GCs before a full GC should keep the name ScavengeBeforeFullGC. There were a couple of arguments for this, the most important from my perspective that ScavengeBeforeFullGC is a publicly documented flag. So changes since last webrev is that all former usages of CollectGen0First is now using ScavengeBeforeFullGC, and the removal of ScavengeBeforeFullGC is reverted. There is also a check in arguments.cpp to set the correct default value if not running ParallelGC. CollectGen0First and ScavengeBeforeFullGC had different default values. An updated webrev is available here: http://cr.openjdk.java.net/~jwilhelm/8042298/webrev.3/ Thanks, /Jesper Stefan Johansson skrev 16/5/14 13:45: > Thanks Jesper, > > On 2014-05-16 02:10, Jesper Wilhelmsson wrote: >> Hi, >> >> A new webrev is available here: >> http://cr.openjdk.java.net/~jwilhelm/8042298/webrev.2/ >> >> In addition to the last webrev the flags TraceGen0Time, TraceGen1Time and >> CollectGen0First have been renamed to get rid of the Gen0/Gen1. >> >> Also, since CollectGen0First and ScavengeBeforeFullGC was doing the same thing >> for different collectors, ScavengeBeforeFullGC has been merged with >> CollectGen0First. >> > Looks good! > > Stefan >> A CCC request has been filed for the flag changes. >> >> Thanks, >> /Jesper >> >> >> Stefan Johansson skrev 15/5/14 12:09: >>> On 2014-05-14 23:21, Jesper Wilhelmsson wrote: >>>> Hi Stefan, >>>> >>>> Thanks for looking at this! See comments inline. >>>> >>>> Stefan Johansson skrev 14/5/14 15:11: >>>>> Hi Jesper, >>>>> >>>>> Thanks for doing this work. Here are some comments: >>>>> >>>>> src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp: >>>>> * TraceGen0Time and TraceGen1Time should be changed to TraceYoungTime and >>>>> TraceOldTime for consistency, and I guess that will need a CCC request. >>>>> >>>>> src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.hpp >>>>> >>>>> >>>>> >>>>> * CollectGen0First -> CollectYoungFirst, same as above. Also used in >>>>> tenuredGeneration.hpp. >>>>> >>>>> src/share/vm/gc_implementation/g1/g1CollectorPolicy.hpp >>>>> * Comment starting on line 41 mentions TraceGen0Time and gen1. >>>> >>>> I agree that the names gen0 and gen1 should be completely removed, which >>>> includes changing these flags too. Initially I wanted this change to be a >>>> simple name change so that it would be easy to review it. Changing the flags >>>> is borderline to be more than a name change, but I'm fine with including that >>>> change. >>>> >>>> I'll file a CCC to change the flags. >>>> >>> Great, thanks. >>>>> As an additional comment, I think that GenCollectedHeap::get_gen(int) >>>>> should be >>>>> broken up into two functions get_young() and get_old(). I get that this was >>>>> left >>>>> to a later patch to avoid having code/logic changes in this patch but I think >>>>> the change is so little that it makes sense to do it in this patch. I would >>>>> like >>>>> to avoid having code like, even if it is for a brief period of time: >>>>> Generation* young = gch->get_gen(0); >>>> >>>> This change is part of the larger removal of the generation array which is the >>>> last part of this series of collector policy cleanups. Initially I was OK with >>>> moving get_young() and get_old() to this change even though that would mean >>>> that the change would grow considerably in size and wouldn't be a simple name >>>> change anymore. After all the only reason I wanted to keep it small was to >>>> make it easier for the reviewers :-) >>>> >>> That's good, thanks =) >>>> However, when I started to look at it in more detail I realized that removing >>>> get_gen(int) requires non-trivial changes in the serviceability agent which is >>>> the only reason why the last change in this series isn't done yet. >>>> >>> I understand that you might want to do all the changes in the SA at once, and >>> since this patch doesn't require you to change the SA I'm fine with leaving it >>> as is. >>>> It would be possible to introduce get_young() and get_old() and replace all >>>> usages of get_gen(int) within the VM, but leave get_gen(int) for the SA to >>>> use. I don't really like this approach, but if you prefer it over just >>>> changing the names it's a possibility. >>>> >>> I'm fine with any of the approaches, none of them are perfect but both are good >>> so I'll let you decide how you want to handle it. >>> >>> Thanks, >>> Stefan >>>> Thanks, >>>> /Jesper >>>> >>>>> >>>>> Thanks, >>>>> Stefan >>>>> >>>>> On 2014-05-02 03:39, Jesper Wilhelmsson wrote: >>>>>> Hi, >>>>>> >>>>>> This is a step towards removing the _generations array. The names gen0 and >>>>>> gen1 are replaced with young and old to use a more standardized vocabulary. >>>>>> >>>>>> Webrev: http://cr.openjdk.java.net/~jwilhelm/8042298/webrev/ >>>>>> >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8042298 >>>>>> >>>>>> Thanks, >>>>>> /Jesper >>>>> >>> > From per.liden at oracle.com Tue May 20 06:19:37 2014 From: per.liden at oracle.com (Per Liden) Date: Tue, 20 May 2014 08:19:37 +0200 Subject: RFR(S): 8039042: G1: Phantom zeros in cardtable In-Reply-To: <1829588.qp7YZWs7PG@mgerdin03> References: <5371F531.7040008@oracle.com> <1829588.qp7YZWs7PG@mgerdin03> Message-ID: <537AF3F9.5070505@oracle.com> Thanks for reviewing Mikael, and thanks for the co-debugging this. Indeed an interesting bug :) Still looking for a capital R review on this patch. Thanks! /Per On 05/13/2014 02:41 PM, Mikael Gerdin wrote: > Per, > > On Tuesday 13 May 2014 12.34.25 Per Liden wrote: >> Summary: We use memset to initialize the cardtable. On Sparc T4 (and >> later) memset uses some special instructions with the side-effect that >> the memory can temporarily be filled with zeros before the actual value >> is set. These phantom zeros can be observed by concurrent readers of >> this memory, which can trick the G1 post-barrier to incorrectly dirty a >> card in a young region. Please see the bug for a more details. >> >> A similar memset-problem has been observed and fixed before >> (JDK-6948537) when using memset on the BlockOffsetTable. Similar to the >> BlockOffsetTable, I'm using the UseMemSetInBOT flag as an indicator of >> whether it's safe to use memset or not. UseMemSetInBOT is not a perfect >> name for this flag as code other than the BOT might want to use it. It's >> also questionable of this should be a command-line flag at all. I've >> filed a separate RFE, JDK-8042930, to do a bit of clean up related to >> this flag. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8039042 >> Webrev: http://cr.openjdk.java.net/~pliden/8039042/webrev.0/ > > The change looks good. > A fun bug to investigate indeed :) > > I agree that setting of UseMemSetInBOT should be retired, we don't need a -XX: > +RandomlyCrashOnSolarisSparc flag. > > /Mikael > >> >> Thanks! >> /Per > From thomas.schatzl at oracle.com Tue May 20 07:06:29 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 20 May 2014 09:06:29 +0200 Subject: RFR(S): 8039042: G1: Phantom zeros in cardtable In-Reply-To: <537AF3F9.5070505@oracle.com> References: <5371F531.7040008@oracle.com> <1829588.qp7YZWs7PG@mgerdin03> <537AF3F9.5070505@oracle.com> Message-ID: <1400569589.2672.1.camel@cirrus> Hi, On Tue, 2014-05-20 at 08:19 +0200, Per Liden wrote: > Thanks for reviewing Mikael, and thanks for the co-debugging this. > Indeed an interesting bug :) > > Still looking for a capital R review on this patch. > > Thanks! > /Per > > On 05/13/2014 02:41 PM, Mikael Gerdin wrote: > > Per, > > > > On Tuesday 13 May 2014 12.34.25 Per Liden wrote: > >> Summary: We use memset to initialize the cardtable. On Sparc T4 (and > >> later) memset uses some special instructions with the side-effect that > >> the memory can temporarily be filled with zeros before the actual value > >> is set. These phantom zeros can be observed by concurrent readers of > >> this memory, which can trick the G1 post-barrier to incorrectly dirty a > >> card in a young region. Please see the bug for a more details. > >> > >> A similar memset-problem has been observed and fixed before > >> (JDK-6948537) when using memset on the BlockOffsetTable. Similar to the > >> BlockOffsetTable, I'm using the UseMemSetInBOT flag as an indicator of > >> whether it's safe to use memset or not. UseMemSetInBOT is not a perfect > >> name for this flag as code other than the BOT might want to use it. It's > >> also questionable of this should be a command-line flag at all. I've > >> filed a separate RFE, JDK-8042930, to do a bit of clean up related to > >> this flag. > >> > >> Bug: https://bugs.openjdk.java.net/browse/JDK-8039042 > >> Webrev: http://cr.openjdk.java.net/~pliden/8039042/webrev.0/ looks good. Thomas From per.liden at oracle.com Tue May 20 08:14:15 2014 From: per.liden at oracle.com (Per Liden) Date: Tue, 20 May 2014 10:14:15 +0200 Subject: RFR(S): 8039042: G1: Phantom zeros in cardtable In-Reply-To: <1400569589.2672.1.camel@cirrus> References: <5371F531.7040008@oracle.com> <1829588.qp7YZWs7PG@mgerdin03> <537AF3F9.5070505@oracle.com> <1400569589.2672.1.camel@cirrus> Message-ID: <537B0ED7.7030202@oracle.com> Thanks Thomas! /Per On 05/20/2014 09:06 AM, Thomas Schatzl wrote: > Hi, > > On Tue, 2014-05-20 at 08:19 +0200, Per Liden wrote: >> Thanks for reviewing Mikael, and thanks for the co-debugging this. >> Indeed an interesting bug :) >> >> Still looking for a capital R review on this patch. >> >> Thanks! >> /Per >> >> On 05/13/2014 02:41 PM, Mikael Gerdin wrote: >>> Per, >>> >>> On Tuesday 13 May 2014 12.34.25 Per Liden wrote: >>>> Summary: We use memset to initialize the cardtable. On Sparc T4 (and >>>> later) memset uses some special instructions with the side-effect that >>>> the memory can temporarily be filled with zeros before the actual value >>>> is set. These phantom zeros can be observed by concurrent readers of >>>> this memory, which can trick the G1 post-barrier to incorrectly dirty a >>>> card in a young region. Please see the bug for a more details. >>>> >>>> A similar memset-problem has been observed and fixed before >>>> (JDK-6948537) when using memset on the BlockOffsetTable. Similar to the >>>> BlockOffsetTable, I'm using the UseMemSetInBOT flag as an indicator of >>>> whether it's safe to use memset or not. UseMemSetInBOT is not a perfect >>>> name for this flag as code other than the BOT might want to use it. It's >>>> also questionable of this should be a command-line flag at all. I've >>>> filed a separate RFE, JDK-8042930, to do a bit of clean up related to >>>> this flag. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8039042 >>>> Webrev: http://cr.openjdk.java.net/~pliden/8039042/webrev.0/ > > looks good. > > Thomas > > From stefan.johansson at oracle.com Tue May 20 08:42:55 2014 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Tue, 20 May 2014 10:42:55 +0200 Subject: RFR: 8042298 - Remove the names gen0 and gen1 from the GC code In-Reply-To: <537A74EF.9040101@oracle.com> References: <5362F75B.9020201@oracle.com> <53736B77.9030206@oracle.com> <5373DE3E.9080803@oracle.com> <53749246.4070201@oracle.com> <53755759.7040506@oracle.com> <5375FA74.4000701@oracle.com> <537A74EF.9040101@oracle.com> Message-ID: <537B158F.7040409@oracle.com> I followed the discussion and agree. Change looks good, Stefan On 2014-05-19 23:17, Jesper Wilhelmsson wrote: > Hi, > > After some discussions we have concluded that the merged flag used to > trigger young GCs before a full GC should keep the name > ScavengeBeforeFullGC. There were a couple of arguments for this, the > most important from my perspective that ScavengeBeforeFullGC is a > publicly documented flag. > > So changes since last webrev is that all former usages of > CollectGen0First is now using ScavengeBeforeFullGC, and the removal of > ScavengeBeforeFullGC is reverted. There is also a check in > arguments.cpp to set the correct default value if not running > ParallelGC. CollectGen0First and ScavengeBeforeFullGC had different > default values. > > An updated webrev is available here: > http://cr.openjdk.java.net/~jwilhelm/8042298/webrev.3/ > > Thanks, > /Jesper > > > Stefan Johansson skrev 16/5/14 13:45: >> Thanks Jesper, >> >> On 2014-05-16 02:10, Jesper Wilhelmsson wrote: >>> Hi, >>> >>> A new webrev is available here: >>> http://cr.openjdk.java.net/~jwilhelm/8042298/webrev.2/ >>> >>> In addition to the last webrev the flags TraceGen0Time, >>> TraceGen1Time and >>> CollectGen0First have been renamed to get rid of the Gen0/Gen1. >>> >>> Also, since CollectGen0First and ScavengeBeforeFullGC was doing the >>> same thing >>> for different collectors, ScavengeBeforeFullGC has been merged with >>> CollectGen0First. >>> >> Looks good! >> >> Stefan >>> A CCC request has been filed for the flag changes. >>> >>> Thanks, >>> /Jesper >>> >>> >>> Stefan Johansson skrev 15/5/14 12:09: >>>> On 2014-05-14 23:21, Jesper Wilhelmsson wrote: >>>>> Hi Stefan, >>>>> >>>>> Thanks for looking at this! See comments inline. >>>>> >>>>> Stefan Johansson skrev 14/5/14 15:11: >>>>>> Hi Jesper, >>>>>> >>>>>> Thanks for doing this work. Here are some comments: >>>>>> >>>>>> src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp: >>>>>> * TraceGen0Time and TraceGen1Time should be changed to >>>>>> TraceYoungTime and >>>>>> TraceOldTime for consistency, and I guess that will need a CCC >>>>>> request. >>>>>> >>>>>> src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.hpp >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> * CollectGen0First -> CollectYoungFirst, same as above. Also used in >>>>>> tenuredGeneration.hpp. >>>>>> >>>>>> src/share/vm/gc_implementation/g1/g1CollectorPolicy.hpp >>>>>> * Comment starting on line 41 mentions TraceGen0Time and gen1. >>>>> >>>>> I agree that the names gen0 and gen1 should be completely removed, >>>>> which >>>>> includes changing these flags too. Initially I wanted this change >>>>> to be a >>>>> simple name change so that it would be easy to review it. Changing >>>>> the flags >>>>> is borderline to be more than a name change, but I'm fine with >>>>> including that >>>>> change. >>>>> >>>>> I'll file a CCC to change the flags. >>>>> >>>> Great, thanks. >>>>>> As an additional comment, I think that >>>>>> GenCollectedHeap::get_gen(int) >>>>>> should be >>>>>> broken up into two functions get_young() and get_old(). I get >>>>>> that this was >>>>>> left >>>>>> to a later patch to avoid having code/logic changes in this patch >>>>>> but I think >>>>>> the change is so little that it makes sense to do it in this >>>>>> patch. I would >>>>>> like >>>>>> to avoid having code like, even if it is for a brief period of time: >>>>>> Generation* young = gch->get_gen(0); >>>>> >>>>> This change is part of the larger removal of the generation array >>>>> which is the >>>>> last part of this series of collector policy cleanups. Initially I >>>>> was OK with >>>>> moving get_young() and get_old() to this change even though that >>>>> would mean >>>>> that the change would grow considerably in size and wouldn't be a >>>>> simple name >>>>> change anymore. After all the only reason I wanted to keep it >>>>> small was to >>>>> make it easier for the reviewers :-) >>>>> >>>> That's good, thanks =) >>>>> However, when I started to look at it in more detail I realized >>>>> that removing >>>>> get_gen(int) requires non-trivial changes in the serviceability >>>>> agent which is >>>>> the only reason why the last change in this series isn't done yet. >>>>> >>>> I understand that you might want to do all the changes in the SA at >>>> once, and >>>> since this patch doesn't require you to change the SA I'm fine with >>>> leaving it >>>> as is. >>>>> It would be possible to introduce get_young() and get_old() and >>>>> replace all >>>>> usages of get_gen(int) within the VM, but leave get_gen(int) for >>>>> the SA to >>>>> use. I don't really like this approach, but if you prefer it over >>>>> just >>>>> changing the names it's a possibility. >>>>> >>>> I'm fine with any of the approaches, none of them are perfect but >>>> both are good >>>> so I'll let you decide how you want to handle it. >>>> >>>> Thanks, >>>> Stefan >>>>> Thanks, >>>>> /Jesper >>>>> >>>>>> >>>>>> Thanks, >>>>>> Stefan >>>>>> >>>>>> On 2014-05-02 03:39, Jesper Wilhelmsson wrote: >>>>>>> Hi, >>>>>>> >>>>>>> This is a step towards removing the _generations array. The >>>>>>> names gen0 and >>>>>>> gen1 are replaced with young and old to use a more standardized >>>>>>> vocabulary. >>>>>>> >>>>>>> Webrev: http://cr.openjdk.java.net/~jwilhelm/8042298/webrev/ >>>>>>> >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8042298 >>>>>>> >>>>>>> Thanks, >>>>>>> /Jesper >>>>>> >>>> >> From per.liden at oracle.com Tue May 20 11:59:56 2014 From: per.liden at oracle.com (Per Liden) Date: Tue, 20 May 2014 13:59:56 +0200 Subject: RFR(S): 8040807: G1: Enable G1CollectedHeap::stop() Message-ID: <537B43BC.2090004@oracle.com> Looking for a couple of reviews in this patch. Summary: This patch re-enables the controlled stopping of G1's concurrent threads at VM shutdown. This could potentially cause hangs during VM shutdown because the G1 marking threads could get stuck in various places and fail to terminate. JDK-8040803 and JDK-8040804 fixed these issues, so this is the final step to re-enable the actual stopping of those threads. This patch also moves the call to CollectedHeap::stop() a few lines down to group the GC related stuff together. It also adjusts/removes some comments that are no longer correct. Bug: https://bugs.openjdk.java.net/browse/JDK-8040807 Webrev: http://cr.openjdk.java.net/~pliden/8040807/webrev.0/ Testing: - GC nightlies. 5 tests in this suite used to timeout because of the issue with hanging threads. They now pass. - JPRT Thanks! /Per From bengt.rutisson at oracle.com Tue May 20 12:20:04 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Tue, 20 May 2014 14:20:04 +0200 Subject: RFR(S): 8040807: G1: Enable G1CollectedHeap::stop() In-Reply-To: <537B43BC.2090004@oracle.com> References: <537B43BC.2090004@oracle.com> Message-ID: <537B4874.6070407@oracle.com> Looks good, Per! Thanks, Bengt On 2014-05-20 13:59, Per Liden wrote: > Looking for a couple of reviews in this patch. > > Summary: This patch re-enables the controlled stopping of G1's > concurrent threads at VM shutdown. This could potentially cause hangs > during VM shutdown because the G1 marking threads could get stuck in > various places and fail to terminate. JDK-8040803 and JDK-8040804 > fixed these issues, so this is the final step to re-enable the actual > stopping of those threads. This patch also moves the call to > CollectedHeap::stop() a few lines down to group the GC related stuff > together. It also adjusts/removes some comments that are no longer > correct. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8040807 > Webrev: http://cr.openjdk.java.net/~pliden/8040807/webrev.0/ > > Testing: > - GC nightlies. 5 tests in this suite used to timeout because of the > issue with hanging threads. They now pass. > - JPRT > > Thanks! > /Per From per.liden at oracle.com Tue May 20 12:22:54 2014 From: per.liden at oracle.com (Per Liden) Date: Tue, 20 May 2014 14:22:54 +0200 Subject: RFR(S): 8040807: G1: Enable G1CollectedHeap::stop() In-Reply-To: <537B4874.6070407@oracle.com> References: <537B43BC.2090004@oracle.com> <537B4874.6070407@oracle.com> Message-ID: <537B491E.4030503@oracle.com> Thanks Bengt! /Per On 05/20/2014 02:20 PM, Bengt Rutisson wrote: > > Looks good, Per! > > Thanks, > Bengt > > On 2014-05-20 13:59, Per Liden wrote: >> Looking for a couple of reviews in this patch. >> >> Summary: This patch re-enables the controlled stopping of G1's >> concurrent threads at VM shutdown. This could potentially cause hangs >> during VM shutdown because the G1 marking threads could get stuck in >> various places and fail to terminate. JDK-8040803 and JDK-8040804 >> fixed these issues, so this is the final step to re-enable the actual >> stopping of those threads. This patch also moves the call to >> CollectedHeap::stop() a few lines down to group the GC related stuff >> together. It also adjusts/removes some comments that are no longer >> correct. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8040807 >> Webrev: http://cr.openjdk.java.net/~pliden/8040807/webrev.0/ >> >> Testing: >> - GC nightlies. 5 tests in this suite used to timeout because of the >> issue with hanging threads. They now pass. >> - JPRT >> >> Thanks! >> /Per > From andreas.eriksson at oracle.com Tue May 20 14:26:40 2014 From: andreas.eriksson at oracle.com (Andreas Eriksson) Date: Tue, 20 May 2014 16:26:40 +0200 Subject: RFR: 8038925: Java with G1 crashes in dump_instance_fields using jmap or jcmd without fullgc Message-ID: <537B6620.6020600@oracle.com> Hi, Could I have a review for this G1 jdk7 only fix please? (There is a related problem in jdk8 and jdk9, where an assert can fail because of this problem. I have logged another bug for this.) Description: Due to the marking cleanup reclaiming empty regions, and having stale references a crash can occur when doing a heap dump. The code tries to do an is_klass check on the object, which crashes the VM. The fix is to add an is_perm check before doing the check, since is_perm will do a bounds check on the oop and if it's in the perm gen we know it's safe to look at it since G1 only ever does full compactions of the perm gen. For more information, and a more in-depth analysis, please see the jira bug. http://cr.openjdk.java.net/~aeriksso/8038925/webrev.01/ https://bugs.openjdk.java.net/browse/JDK-8038925 Regards, Andreas From jon.masamitsu at oracle.com Tue May 20 14:45:30 2014 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Tue, 20 May 2014 07:45:30 -0700 Subject: RFR(S): 8040807: G1: Enable G1CollectedHeap::stop() In-Reply-To: <537B43BC.2090004@oracle.com> References: <537B43BC.2090004@oracle.com> Message-ID: <537B6A8A.5030607@oracle.com> Looks good. Reviewed. Jon On 05/20/2014 04:59 AM, Per Liden wrote: > Looking for a couple of reviews in this patch. > > Summary: This patch re-enables the controlled stopping of G1's > concurrent threads at VM shutdown. This could potentially cause hangs > during VM shutdown because the G1 marking threads could get stuck in > various places and fail to terminate. JDK-8040803 and JDK-8040804 > fixed these issues, so this is the final step to re-enable the actual > stopping of those threads. This patch also moves the call to > CollectedHeap::stop() a few lines down to group the GC related stuff > together. It also adjusts/removes some comments that are no longer > correct. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8040807 > Webrev: http://cr.openjdk.java.net/~pliden/8040807/webrev.0/ > > Testing: > - GC nightlies. 5 tests in this suite used to timeout because of the > issue with hanging threads. They now pass. > - JPRT > > Thanks! > /Per From andrey.x.zakharov at oracle.com Tue May 20 15:13:07 2014 From: andrey.x.zakharov at oracle.com (Andrey Zakharov) Date: Tue, 20 May 2014 19:13:07 +0400 Subject: RFR: 8041506 - The test gc/g1/TestHumongousShrinkHeap.java reports that memory is not de-committed In-Reply-To: <5374A6A1.6010404@oracle.com> References: <5374A502.5030409@oracle.com> <5374A6A1.6010404@oracle.com> Message-ID: <537B7103.8030701@oracle.com> Hi Any comments are welcome Thanks. On 15.05.2014 15:36, Jesper Wilhelmsson wrote: > Looks good! > /Jesper > > Andrey Zakharov skrev 15/5/14 13:29: >> Hi. >> To proper testing of free list sorting we need to defragment memory >> with small >> young and humongous objects >> This is test scenario: >> Make enough space for new objects to prevent it going old. >> - allocate bunch of small objects, and a bit of humongous >> several times. >> >> Free almost all of allocated stuff. Check that heap shrinks after GC. >> >> webrev: http://cr.openjdk.java.net/~jwilhelm/8041506/webrev.02/ >> bug: https://bugs.openjdk.java.net/browse/JDK-8041506 >> >> Thanks. >> From mikael.gerdin at oracle.com Wed May 21 08:46:28 2014 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Wed, 21 May 2014 10:46:28 +0200 Subject: RFR: 8038925: Java with G1 crashes in dump_instance_fields using jmap or jcmd without fullgc In-Reply-To: <537B6620.6020600@oracle.com> References: <537B6620.6020600@oracle.com> Message-ID: <13880974.pHCQdHd51K@mgerdin03> Hi Andreas, On Tuesday 20 May 2014 16.26.40 Andreas Eriksson wrote: > Hi, > > Could I have a review for this G1 jdk7 only fix please? > (There is a related problem in jdk8 and jdk9, where an assert can fail > because of this problem. I have logged another bug for this.) > > Description: > Due to the marking cleanup reclaiming empty regions, and having stale > references a crash can occur when doing a heap dump. > The code tries to do an is_klass check on the object, which crashes the VM. > The fix is to add an is_perm check before doing the check, since is_perm > will do a bounds check on the oop and if it's in the perm gen we know > it's safe to look at it since G1 only ever does full compactions of the > perm gen. > > For more information, and a more in-depth analysis, please see the jira bug. > > http://cr.openjdk.java.net/~aeriksso/8038925/webrev.01/ The change looks good to me. /Mikael > https://bugs.openjdk.java.net/browse/JDK-8038925 > > Regards, > Andreas From andreas.eriksson at oracle.com Wed May 21 09:36:47 2014 From: andreas.eriksson at oracle.com (Andreas Eriksson) Date: Wed, 21 May 2014 11:36:47 +0200 Subject: RFR: 8038925: Java with G1 crashes in dump_instance_fields using jmap or jcmd without fullgc In-Reply-To: <13880974.pHCQdHd51K@mgerdin03> References: <537B6620.6020600@oracle.com> <13880974.pHCQdHd51K@mgerdin03> Message-ID: <537C73AF.8040001@oracle.com> Thanks Mikael. Could I have a review from a jdk7u Reviewer as well please? /Andreas On 2014-05-21 10:46, Mikael Gerdin wrote: > Hi Andreas, > > On Tuesday 20 May 2014 16.26.40 Andreas Eriksson wrote: >> Hi, >> >> Could I have a review for this G1 jdk7 only fix please? >> (There is a related problem in jdk8 and jdk9, where an assert can fail >> because of this problem. I have logged another bug for this.) >> >> Description: >> Due to the marking cleanup reclaiming empty regions, and having stale >> references a crash can occur when doing a heap dump. >> The code tries to do an is_klass check on the object, which crashes the VM. >> The fix is to add an is_perm check before doing the check, since is_perm >> will do a bounds check on the oop and if it's in the perm gen we know >> it's safe to look at it since G1 only ever does full compactions of the >> perm gen. >> >> For more information, and a more in-depth analysis, please see the jira bug. >> >> http://cr.openjdk.java.net/~aeriksso/8038925/webrev.01/ > The change looks good to me. > > /Mikael > >> https://bugs.openjdk.java.net/browse/JDK-8038925 >> >> Regards, >> Andreas From bengt.rutisson at oracle.com Wed May 21 09:55:32 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 21 May 2014 11:55:32 +0200 Subject: RFR: 8038925: Java with G1 crashes in dump_instance_fields using jmap or jcmd without fullgc In-Reply-To: <537B6620.6020600@oracle.com> References: <537B6620.6020600@oracle.com> Message-ID: <537C7814.8020806@oracle.com> Hi Andreas, The fix looks good. One comment about the test. It does not verify that no full GC happens. The way the test is set up I guess that should not happen and I am not sure it is worth the effort to add a check for it. Just wanted to mention it if you want to make test more resilient to future changes in the JVM that for some reason can trigger a full GC for this test. I'm fine with leaving the test as it is. Thanks, Bengt On 5/20/14 4:26 PM, Andreas Eriksson wrote: > Hi, > > Could I have a review for this G1 jdk7 only fix please? > (There is a related problem in jdk8 and jdk9, where an assert can fail > because of this problem. I have logged another bug for this.) > > Description: > Due to the marking cleanup reclaiming empty regions, and having stale > references a crash can occur when doing a heap dump. > The code tries to do an is_klass check on the object, which crashes > the VM. > The fix is to add an is_perm check before doing the check, since > is_perm will do a bounds check on the oop and if it's in the perm gen > we know it's safe to look at it since G1 only ever does full > compactions of the perm gen. > > For more information, and a more in-depth analysis, please see the > jira bug. > > http://cr.openjdk.java.net/~aeriksso/8038925/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8038925 > > Regards, > Andreas From andreas.eriksson at oracle.com Wed May 21 10:12:59 2014 From: andreas.eriksson at oracle.com (Andreas Eriksson) Date: Wed, 21 May 2014 12:12:59 +0200 Subject: RFR: 8038925: Java with G1 crashes in dump_instance_fields using jmap or jcmd without fullgc In-Reply-To: <537C7814.8020806@oracle.com> References: <537B6620.6020600@oracle.com> <537C7814.8020806@oracle.com> Message-ID: <537C7C2B.1080105@oracle.com> Hi Bengt, thanks for looking at this. I agree that verifying that no full GC has happened would be good. One thing I see as problematic though, is what to do if a full GC has happened. Should I make the test fail? Or is there some way to signal that the test was inconclusive / couldn't finish? Regards, Andreas On 2014-05-21 11:55, Bengt Rutisson wrote: > > Hi Andreas, > > The fix looks good. > > One comment about the test. It does not verify that no full GC > happens. The way the test is set up I guess that should not happen and > I am not sure it is worth the effort to add a check for it. Just > wanted to mention it if you want to make test more resilient to future > changes in the JVM that for some reason can trigger a full GC for this > test. > > I'm fine with leaving the test as it is. > > Thanks, > Bengt > > > On 5/20/14 4:26 PM, Andreas Eriksson wrote: >> Hi, >> >> Could I have a review for this G1 jdk7 only fix please? >> (There is a related problem in jdk8 and jdk9, where an assert can >> fail because of this problem. I have logged another bug for this.) >> >> Description: >> Due to the marking cleanup reclaiming empty regions, and having stale >> references a crash can occur when doing a heap dump. >> The code tries to do an is_klass check on the object, which crashes >> the VM. >> The fix is to add an is_perm check before doing the check, since >> is_perm will do a bounds check on the oop and if it's in the perm gen >> we know it's safe to look at it since G1 only ever does full >> compactions of the perm gen. >> >> For more information, and a more in-depth analysis, please see the >> jira bug. >> >> http://cr.openjdk.java.net/~aeriksso/8038925/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8038925 >> >> Regards, >> Andreas > From bengt.rutisson at oracle.com Wed May 21 10:31:19 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 21 May 2014 12:31:19 +0200 Subject: RFR: 8038925: Java with G1 crashes in dump_instance_fields using jmap or jcmd without fullgc In-Reply-To: <537C7C2B.1080105@oracle.com> References: <537B6620.6020600@oracle.com> <537C7814.8020806@oracle.com> <537C7C2B.1080105@oracle.com> Message-ID: <537C8077.8030400@oracle.com> On 5/21/14 12:12 PM, Andreas Eriksson wrote: > Hi Bengt, thanks for looking at this. > > I agree that verifying that no full GC has happened would be good. > One thing I see as problematic though, is what to do if a full GC has > happened. > Should I make the test fail? Or is there some way to signal that the > test was inconclusive / couldn't finish? Personally I would prefer to make the test fail. JTreg only has two states, PASS or FAIL, and I think that if we allow it to pass we might not notice if there is some change that makes the test always get a full GC and then never actually testing anything. I don't think it will fail intermittently by getting full GCs. I think the test is pretty stable. But I think it would be good to have a way of noticing if it stops testing what it is supposed to test. (By the way, I would really like a "SKIPPED" state in JTreg but I haven't had any luck pushing for that. I think it could be useful for other things as well. There is for example no good reason for your test to be run 4 times each night with the exact same binary. But that is what happens since we can't say that we should skip this test if we run with any other GC than G1.) Thanks, Bengt > > Regards, > Andreas > > On 2014-05-21 11:55, Bengt Rutisson wrote: >> >> Hi Andreas, >> >> The fix looks good. >> >> One comment about the test. It does not verify that no full GC >> happens. The way the test is set up I guess that should not happen >> and I am not sure it is worth the effort to add a check for it. Just >> wanted to mention it if you want to make test more resilient to >> future changes in the JVM that for some reason can trigger a full GC >> for this test. >> >> I'm fine with leaving the test as it is. >> >> Thanks, >> Bengt >> >> >> On 5/20/14 4:26 PM, Andreas Eriksson wrote: >>> Hi, >>> >>> Could I have a review for this G1 jdk7 only fix please? >>> (There is a related problem in jdk8 and jdk9, where an assert can >>> fail because of this problem. I have logged another bug for this.) >>> >>> Description: >>> Due to the marking cleanup reclaiming empty regions, and having >>> stale references a crash can occur when doing a heap dump. >>> The code tries to do an is_klass check on the object, which crashes >>> the VM. >>> The fix is to add an is_perm check before doing the check, since >>> is_perm will do a bounds check on the oop and if it's in the perm >>> gen we know it's safe to look at it since G1 only ever does full >>> compactions of the perm gen. >>> >>> For more information, and a more in-depth analysis, please see the >>> jira bug. >>> >>> http://cr.openjdk.java.net/~aeriksso/8038925/webrev.01/ >>> https://bugs.openjdk.java.net/browse/JDK-8038925 >>> >>> Regards, >>> Andreas >> > From andreas.eriksson at oracle.com Wed May 21 12:05:41 2014 From: andreas.eriksson at oracle.com (Andreas Eriksson) Date: Wed, 21 May 2014 14:05:41 +0200 Subject: RFR: 8038925: Java with G1 crashes in dump_instance_fields using jmap or jcmd without fullgc In-Reply-To: <537C8077.8030400@oracle.com> References: <537B6620.6020600@oracle.com> <537C7814.8020806@oracle.com> <537C7C2B.1080105@oracle.com> <537C8077.8030400@oracle.com> Message-ID: <537C9695.5010902@oracle.com> A new webrev is up: http://cr.openjdk.java.net/~aeriksso/8038925/webrev.02/ The test now verifies that no full GC has been done after doing the heap dump. I also modified the test to not run if it for some reason is not using G1. Regards, Andreas On 2014-05-21 12:31, Bengt Rutisson wrote: > On 5/21/14 12:12 PM, Andreas Eriksson wrote: >> Hi Bengt, thanks for looking at this. >> >> I agree that verifying that no full GC has happened would be good. >> One thing I see as problematic though, is what to do if a full GC has >> happened. >> Should I make the test fail? Or is there some way to signal that the >> test was inconclusive / couldn't finish? > > Personally I would prefer to make the test fail. JTreg only has two > states, PASS or FAIL, and I think that if we allow it to pass we might > not notice if there is some change that makes the test always get a > full GC and then never actually testing anything. > > I don't think it will fail intermittently by getting full GCs. I think > the test is pretty stable. But I think it would be good to have a way > of noticing if it stops testing what it is supposed to test. > > (By the way, I would really like a "SKIPPED" state in JTreg but I > haven't had any luck pushing for that. I think it could be useful for > other things as well. There is for example no good reason for your > test to be run 4 times each night with the exact same binary. But that > is what happens since we can't say that we should skip this test if we > run with any other GC than G1.) > > Thanks, > Bengt > >> >> Regards, >> Andreas >> >> On 2014-05-21 11:55, Bengt Rutisson wrote: >>> >>> Hi Andreas, >>> >>> The fix looks good. >>> >>> One comment about the test. It does not verify that no full GC >>> happens. The way the test is set up I guess that should not happen >>> and I am not sure it is worth the effort to add a check for it. Just >>> wanted to mention it if you want to make test more resilient to >>> future changes in the JVM that for some reason can trigger a full GC >>> for this test. >>> >>> I'm fine with leaving the test as it is. >>> >>> Thanks, >>> Bengt >>> >>> >>> On 5/20/14 4:26 PM, Andreas Eriksson wrote: >>>> Hi, >>>> >>>> Could I have a review for this G1 jdk7 only fix please? >>>> (There is a related problem in jdk8 and jdk9, where an assert can >>>> fail because of this problem. I have logged another bug for this.) >>>> >>>> Description: >>>> Due to the marking cleanup reclaiming empty regions, and having >>>> stale references a crash can occur when doing a heap dump. >>>> The code tries to do an is_klass check on the object, which crashes >>>> the VM. >>>> The fix is to add an is_perm check before doing the check, since >>>> is_perm will do a bounds check on the oop and if it's in the perm >>>> gen we know it's safe to look at it since G1 only ever does full >>>> compactions of the perm gen. >>>> >>>> For more information, and a more in-depth analysis, please see the >>>> jira bug. >>>> >>>> http://cr.openjdk.java.net/~aeriksso/8038925/webrev.01/ >>>> https://bugs.openjdk.java.net/browse/JDK-8038925 >>>> >>>> Regards, >>>> Andreas >>> >> > From erik.helin at oracle.com Wed May 21 14:20:13 2014 From: erik.helin at oracle.com (Erik Helin) Date: Wed, 21 May 2014 16:20:13 +0200 Subject: RFR: 8043639: Backout JDK-8034852: Shrinking of Metaspace high-water-mark causes incorrect OutOfMemoryErrors or back-to-back GCs Message-ID: <1679331.2y5zeXlrKh@ehelin-desktop> Hi all, this is an "anti-patch" that backs out: https://bugs.openjdk.java.net/browse/JDK-8034852 (commit dd531b743c04 in the jdk9/hs-gc repo, this commit has *not* been pushed upstream to jdk9/hs). The patch was created by `hg rollback dd531b743c04` and it applied cleanly. Webrev: http://cr.openjdk.java.net/~ehelin/8043639/webrev.00/ I'm currently working on fixing this, but the fix was slightly trickier than expected, so I will need some more time. Thanks, Erik From stefan.karlsson at oracle.com Wed May 21 16:30:17 2014 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 21 May 2014 18:30:17 +0200 Subject: RFR: 8043639: Backout JDK-8034852: Shrinking of Metaspace high-water-mark causes incorrect OutOfMemoryErrors or back-to-back GCs In-Reply-To: <1679331.2y5zeXlrKh@ehelin-desktop> References: <1679331.2y5zeXlrKh@ehelin-desktop> Message-ID: <537CD499.8050405@oracle.com> On 2014-05-21 16:20, Erik Helin wrote: > Hi all, > > this is an "anti-patch" that backs out: > https://bugs.openjdk.java.net/browse/JDK-8034852 > > (commit dd531b743c04 in the jdk9/hs-gc repo, this commit has *not* > been pushed upstream to jdk9/hs). > > The patch was created by `hg rollback dd531b743c04` and it applied > cleanly. > > Webrev: > http://cr.openjdk.java.net/~ehelin/8043639/webrev.00/ Looks OK. StefanK > > I'm currently working on fixing this, but the fix was slightly > trickier than expected, so I will need some more time. > > Thanks, > Erik > From ilya.dergalin at oracle.com Wed May 21 14:26:36 2014 From: ilya.dergalin at oracle.com (Ilya Dergalin) Date: Wed, 21 May 2014 18:26:36 +0400 Subject: RFR: 8041506 - The test gc/g1/TestHumongousShrinkHeap.java reports that memory is not de-committed In-Reply-To: <537B7103.8030701@oracle.com> References: <5374A502.5030409@oracle.com> <5374A6A1.6010404@oracle.com> <537B7103.8030701@oracle.com> Message-ID: <537CB79C.8080003@oracle.com> Colleagues, Andrey has 1 review from Jesper for this request. He needs two to push the test. Please review. On 20.05.2014 19:13, Andrey Zakharov wrote: > Hi > Any comments are welcome > > Thanks. > > On 15.05.2014 15:36, Jesper Wilhelmsson wrote: >> Looks good! >> /Jesper >> >> Andrey Zakharov skrev 15/5/14 13:29: >>> Hi. >>> To proper testing of free list sorting we need to defragment memory >>> with small >>> young and humongous objects >>> This is test scenario: >>> Make enough space for new objects to prevent it going old. >>> - allocate bunch of small objects, and a bit of humongous >>> several times. >>> >>> Free almost all of allocated stuff. Check that heap shrinks after GC. >>> >>> webrev: http://cr.openjdk.java.net/~jwilhelm/8041506/webrev.02/ >>> bug: https://bugs.openjdk.java.net/browse/JDK-8041506 >>> >>> Thanks. >>> > -- Best Regards, Ilya From thomas.schatzl at oracle.com Wed May 21 17:35:53 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 21 May 2014 19:35:53 +0200 Subject: RFR: 8041506 - The test gc/g1/TestHumongousShrinkHeap.java reports that memory is not de-committed In-Reply-To: <537CB79C.8080003@oracle.com> References: <5374A502.5030409@oracle.com> <5374A6A1.6010404@oracle.com> <537B7103.8030701@oracle.com> <537CB79C.8080003@oracle.com> Message-ID: <1400693753.2632.0.camel@cirrus> Hi, On Wed, 2014-05-21 at 18:26 +0400, Ilya Dergalin wrote: > Colleagues, > > Andrey has 1 review from Jesper for this request. He needs two to push > the test. Please review. looks okay. Thanks, Thomas From bengt.rutisson at oracle.com Thu May 22 08:02:04 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Thu, 22 May 2014 10:02:04 +0200 Subject: RFR: 8038925: Java with G1 crashes in dump_instance_fields using jmap or jcmd without fullgc In-Reply-To: <537C9695.5010902@oracle.com> References: <537B6620.6020600@oracle.com> <537C7814.8020806@oracle.com> <537C7C2B.1080105@oracle.com> <537C8077.8030400@oracle.com> <537C9695.5010902@oracle.com> Message-ID: <537DAEFC.2000508@oracle.com> Hi Andreas, On 5/21/14 2:05 PM, Andreas Eriksson wrote: > A new webrev is up: > http://cr.openjdk.java.net/~aeriksso/8038925/webrev.02/ > > The test now verifies that no full GC has been done after doing the > heap dump. > I also modified the test to not run if it for some reason is not using > G1. Thanks for the update, Andreas. The test looks good except for the @run tag. @run main/othervm -Xms512m -Xmx1024m -XX:+UseG1GC -XX:+ExplicitGCInvokesConcurrent TestG1ConcurrentGCHeapDump The problem is that more GC flags will be added when the test is run in nightly testing. Some GC flags will conflict with UseG1GC. On the other hand at some point UseG1GC will be one of the flags that is added. So, I think what you need to do is just remove "-XX:+UseG1GC" from the @run tag. Then your test will report log "skipped" when it is run in the nightly testing except for the nightly testing done with G1 where it will actually do something. Bengt > > Regards, > Andreas > > On 2014-05-21 12:31, Bengt Rutisson wrote: >> On 5/21/14 12:12 PM, Andreas Eriksson wrote: >>> Hi Bengt, thanks for looking at this. >>> >>> I agree that verifying that no full GC has happened would be good. >>> One thing I see as problematic though, is what to do if a full GC >>> has happened. >>> Should I make the test fail? Or is there some way to signal that the >>> test was inconclusive / couldn't finish? >> >> Personally I would prefer to make the test fail. JTreg only has two >> states, PASS or FAIL, and I think that if we allow it to pass we >> might not notice if there is some change that makes the test always >> get a full GC and then never actually testing anything. >> >> I don't think it will fail intermittently by getting full GCs. I >> think the test is pretty stable. But I think it would be good to have >> a way of noticing if it stops testing what it is supposed to test. >> >> (By the way, I would really like a "SKIPPED" state in JTreg but I >> haven't had any luck pushing for that. I think it could be useful for >> other things as well. There is for example no good reason for your >> test to be run 4 times each night with the exact same binary. But >> that is what happens since we can't say that we should skip this test >> if we run with any other GC than G1.) >> >> Thanks, >> Bengt >> >>> >>> Regards, >>> Andreas >>> >>> On 2014-05-21 11:55, Bengt Rutisson wrote: >>>> >>>> Hi Andreas, >>>> >>>> The fix looks good. >>>> >>>> One comment about the test. It does not verify that no full GC >>>> happens. The way the test is set up I guess that should not happen >>>> and I am not sure it is worth the effort to add a check for it. >>>> Just wanted to mention it if you want to make test more resilient >>>> to future changes in the JVM that for some reason can trigger a >>>> full GC for this test. >>>> >>>> I'm fine with leaving the test as it is. >>>> >>>> Thanks, >>>> Bengt >>>> >>>> >>>> On 5/20/14 4:26 PM, Andreas Eriksson wrote: >>>>> Hi, >>>>> >>>>> Could I have a review for this G1 jdk7 only fix please? >>>>> (There is a related problem in jdk8 and jdk9, where an assert can >>>>> fail because of this problem. I have logged another bug for this.) >>>>> >>>>> Description: >>>>> Due to the marking cleanup reclaiming empty regions, and having >>>>> stale references a crash can occur when doing a heap dump. >>>>> The code tries to do an is_klass check on the object, which >>>>> crashes the VM. >>>>> The fix is to add an is_perm check before doing the check, since >>>>> is_perm will do a bounds check on the oop and if it's in the perm >>>>> gen we know it's safe to look at it since G1 only ever does full >>>>> compactions of the perm gen. >>>>> >>>>> For more information, and a more in-depth analysis, please see the >>>>> jira bug. >>>>> >>>>> http://cr.openjdk.java.net/~aeriksso/8038925/webrev.01/ >>>>> https://bugs.openjdk.java.net/browse/JDK-8038925 >>>>> >>>>> Regards, >>>>> Andreas >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From andreas.eriksson at oracle.com Thu May 22 08:14:42 2014 From: andreas.eriksson at oracle.com (Andreas Eriksson) Date: Thu, 22 May 2014 10:14:42 +0200 Subject: RFR: 8038925: Java with G1 crashes in dump_instance_fields using jmap or jcmd without fullgc In-Reply-To: <537DAEFC.2000508@oracle.com> References: <537B6620.6020600@oracle.com> <537C7814.8020806@oracle.com> <537C7C2B.1080105@oracle.com> <537C8077.8030400@oracle.com> <537C9695.5010902@oracle.com> <537DAEFC.2000508@oracle.com> Message-ID: <537DB1F2.6020406@oracle.com> OK, I'll remove it. Thanks, Andreas On 2014-05-22 10:02, Bengt Rutisson wrote: > > > Hi Andreas, > > On 5/21/14 2:05 PM, Andreas Eriksson wrote: >> A new webrev is up: >> http://cr.openjdk.java.net/~aeriksso/8038925/webrev.02/ >> >> The test now verifies that no full GC has been done after doing the >> heap dump. >> I also modified the test to not run if it for some reason is not >> using G1. > > Thanks for the update, Andreas. > > The test looks good except for the @run tag. > > @run main/othervm -Xms512m -Xmx1024m -XX:+UseG1GC > -XX:+ExplicitGCInvokesConcurrent TestG1ConcurrentGCHeapDump > > The problem is that more GC flags will be added when the test is run > in nightly testing. Some GC flags will conflict with UseG1GC. On the > other hand at some point UseG1GC will be one of the flags that is added. > > So, I think what you need to do is just remove "-XX:+UseG1GC" from the > @run tag. Then your test will report log "skipped" when it is run in > the nightly testing except for the nightly testing done with G1 where > it will actually do something. > > Bengt > >> >> Regards, >> Andreas >> >> On 2014-05-21 12:31, Bengt Rutisson wrote: >>> On 5/21/14 12:12 PM, Andreas Eriksson wrote: >>>> Hi Bengt, thanks for looking at this. >>>> >>>> I agree that verifying that no full GC has happened would be good. >>>> One thing I see as problematic though, is what to do if a full GC >>>> has happened. >>>> Should I make the test fail? Or is there some way to signal that >>>> the test was inconclusive / couldn't finish? >>> >>> Personally I would prefer to make the test fail. JTreg only has two >>> states, PASS or FAIL, and I think that if we allow it to pass we >>> might not notice if there is some change that makes the test always >>> get a full GC and then never actually testing anything. >>> >>> I don't think it will fail intermittently by getting full GCs. I >>> think the test is pretty stable. But I think it would be good to >>> have a way of noticing if it stops testing what it is supposed to test. >>> >>> (By the way, I would really like a "SKIPPED" state in JTreg but I >>> haven't had any luck pushing for that. I think it could be useful >>> for other things as well. There is for example no good reason for >>> your test to be run 4 times each night with the exact same binary. >>> But that is what happens since we can't say that we should skip this >>> test if we run with any other GC than G1.) >>> >>> Thanks, >>> Bengt >>> >>>> >>>> Regards, >>>> Andreas >>>> >>>> On 2014-05-21 11:55, Bengt Rutisson wrote: >>>>> >>>>> Hi Andreas, >>>>> >>>>> The fix looks good. >>>>> >>>>> One comment about the test. It does not verify that no full GC >>>>> happens. The way the test is set up I guess that should not happen >>>>> and I am not sure it is worth the effort to add a check for it. >>>>> Just wanted to mention it if you want to make test more resilient >>>>> to future changes in the JVM that for some reason can trigger a >>>>> full GC for this test. >>>>> >>>>> I'm fine with leaving the test as it is. >>>>> >>>>> Thanks, >>>>> Bengt >>>>> >>>>> >>>>> On 5/20/14 4:26 PM, Andreas Eriksson wrote: >>>>>> Hi, >>>>>> >>>>>> Could I have a review for this G1 jdk7 only fix please? >>>>>> (There is a related problem in jdk8 and jdk9, where an assert can >>>>>> fail because of this problem. I have logged another bug for this.) >>>>>> >>>>>> Description: >>>>>> Due to the marking cleanup reclaiming empty regions, and having >>>>>> stale references a crash can occur when doing a heap dump. >>>>>> The code tries to do an is_klass check on the object, which >>>>>> crashes the VM. >>>>>> The fix is to add an is_perm check before doing the check, since >>>>>> is_perm will do a bounds check on the oop and if it's in the perm >>>>>> gen we know it's safe to look at it since G1 only ever does full >>>>>> compactions of the perm gen. >>>>>> >>>>>> For more information, and a more in-depth analysis, please see >>>>>> the jira bug. >>>>>> >>>>>> http://cr.openjdk.java.net/~aeriksso/8038925/webrev.01/ >>>>>> https://bugs.openjdk.java.net/browse/JDK-8038925 >>>>>> >>>>>> Regards, >>>>>> Andreas >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From andreas.eriksson at oracle.com Thu May 22 08:43:29 2014 From: andreas.eriksson at oracle.com (Andreas Eriksson) Date: Thu, 22 May 2014 10:43:29 +0200 Subject: RFR: 8038925: Java with G1 crashes in dump_instance_fields using jmap or jcmd without fullgc In-Reply-To: <537DB1F2.6020406@oracle.com> References: <537B6620.6020600@oracle.com> <537C7814.8020806@oracle.com> <537C7C2B.1080105@oracle.com> <537C8077.8030400@oracle.com> <537C9695.5010902@oracle.com> <537DAEFC.2000508@oracle.com> <537DB1F2.6020406@oracle.com> Message-ID: <537DB8B1.5080905@oracle.com> Hi, Adding jdk7u-dev. Could I have a jdk7u Reviewer look at this as well please? This is a jdk7 only fix. (There is a related problem in jdk8 and jdk9, where an assert can fail because of this problem. I have logged another bug for this.) Description: Due to the marking cleanup reclaiming empty regions, and having stale references a crash can occur when doing a heap dump. The code tries to do an is_klass check on the object, which crashes the VM. The fix is to add an is_perm check before doing the check, since is_perm will do a bounds check on the oop and if it's in the perm gen we know it's safe to look at it since G1 only ever does full compactions of the perm gen. For more information, and a more in-depth analysis, please see the jira bug. http://cr.openjdk.java.net/~aeriksso/8038925/webrev.03/ https://bugs.openjdk.java.net/browse/JDK-8038925 Regards, Andreas On 2014-05-22 10:14, Andreas Eriksson wrote: > OK, I'll remove it. > > Thanks, > Andreas > > On 2014-05-22 10:02, Bengt Rutisson wrote: >> >> >> Hi Andreas, >> >> On 5/21/14 2:05 PM, Andreas Eriksson wrote: >>> A new webrev is up: >>> http://cr.openjdk.java.net/~aeriksso/8038925/webrev.02/ >>> >>> The test now verifies that no full GC has been done after doing the >>> heap dump. >>> I also modified the test to not run if it for some reason is not >>> using G1. >> >> Thanks for the update, Andreas. >> >> The test looks good except for the @run tag. >> >> @run main/othervm -Xms512m -Xmx1024m -XX:+UseG1GC >> -XX:+ExplicitGCInvokesConcurrent TestG1ConcurrentGCHeapDump >> >> The problem is that more GC flags will be added when the test is run >> in nightly testing. Some GC flags will conflict with UseG1GC. On the >> other hand at some point UseG1GC will be one of the flags that is added. >> >> So, I think what you need to do is just remove "-XX:+UseG1GC" from >> the @run tag. Then your test will report log "skipped" when it is run >> in the nightly testing except for the nightly testing done with G1 >> where it will actually do something. >> >> Bengt >> >>> >>> Regards, >>> Andreas >>> >>> On 2014-05-21 12:31, Bengt Rutisson wrote: >>>> On 5/21/14 12:12 PM, Andreas Eriksson wrote: >>>>> Hi Bengt, thanks for looking at this. >>>>> >>>>> I agree that verifying that no full GC has happened would be good. >>>>> One thing I see as problematic though, is what to do if a full GC >>>>> has happened. >>>>> Should I make the test fail? Or is there some way to signal that >>>>> the test was inconclusive / couldn't finish? >>>> >>>> Personally I would prefer to make the test fail. JTreg only has two >>>> states, PASS or FAIL, and I think that if we allow it to pass we >>>> might not notice if there is some change that makes the test always >>>> get a full GC and then never actually testing anything. >>>> >>>> I don't think it will fail intermittently by getting full GCs. I >>>> think the test is pretty stable. But I think it would be good to >>>> have a way of noticing if it stops testing what it is supposed to >>>> test. >>>> >>>> (By the way, I would really like a "SKIPPED" state in JTreg but I >>>> haven't had any luck pushing for that. I think it could be useful >>>> for other things as well. There is for example no good reason for >>>> your test to be run 4 times each night with the exact same binary. >>>> But that is what happens since we can't say that we should skip >>>> this test if we run with any other GC than G1.) >>>> >>>> Thanks, >>>> Bengt >>>> >>>>> >>>>> Regards, >>>>> Andreas >>>>> >>>>> On 2014-05-21 11:55, Bengt Rutisson wrote: >>>>>> >>>>>> Hi Andreas, >>>>>> >>>>>> The fix looks good. >>>>>> >>>>>> One comment about the test. It does not verify that no full GC >>>>>> happens. The way the test is set up I guess that should not >>>>>> happen and I am not sure it is worth the effort to add a check >>>>>> for it. Just wanted to mention it if you want to make test more >>>>>> resilient to future changes in the JVM that for some reason can >>>>>> trigger a full GC for this test. >>>>>> >>>>>> I'm fine with leaving the test as it is. >>>>>> >>>>>> Thanks, >>>>>> Bengt >>>>>> >>>>>> >>>>>> On 5/20/14 4:26 PM, Andreas Eriksson wrote: >>>>>>> Hi, >>>>>>> >>>>>>> Could I have a review for this G1 jdk7 only fix please? >>>>>>> (There is a related problem in jdk8 and jdk9, where an assert >>>>>>> can fail because of this problem. I have logged another bug for >>>>>>> this.) >>>>>>> >>>>>>> Description: >>>>>>> Due to the marking cleanup reclaiming empty regions, and having >>>>>>> stale references a crash can occur when doing a heap dump. >>>>>>> The code tries to do an is_klass check on the object, which >>>>>>> crashes the VM. >>>>>>> The fix is to add an is_perm check before doing the check, since >>>>>>> is_perm will do a bounds check on the oop and if it's in the >>>>>>> perm gen we know it's safe to look at it since G1 only ever does >>>>>>> full compactions of the perm gen. >>>>>>> >>>>>>> For more information, and a more in-depth analysis, please see >>>>>>> the jira bug. >>>>>>> >>>>>>> http://cr.openjdk.java.net/~aeriksso/8038925/webrev.01/ >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8038925 >>>>>>> >>>>>>> Regards, >>>>>>> Andreas >>>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.rutisson at oracle.com Thu May 22 11:02:22 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Thu, 22 May 2014 13:02:22 +0200 Subject: RFR: 8038925: Java with G1 crashes in dump_instance_fields using jmap or jcmd without fullgc In-Reply-To: <537DB8B1.5080905@oracle.com> References: <537B6620.6020600@oracle.com> <537C7814.8020806@oracle.com> <537C7C2B.1080105@oracle.com> <537C8077.8030400@oracle.com> <537C9695.5010902@oracle.com> <537DAEFC.2000508@oracle.com> <537DB1F2.6020406@oracle.com> <537DB8B1.5080905@oracle.com> Message-ID: <537DD93E.1010303@oracle.com> Hi Andreas, This is a HotSpot change and for JDK7 HotSpot was developed in the hsx project. I am a Reviewer in the hsx project (http://openjdk.java.net/census#brutisso) isn't that enough to review this change? Anyway, last webrev looks good. Thanks for fixing the test! Bengt On 5/22/14 10:43 AM, Andreas Eriksson wrote: > Hi, > > Adding jdk7u-dev. > > Could I have a jdk7u Reviewer look at this as well please? This is a > jdk7 only fix. > (There is a related problem in jdk8 and jdk9, where an assert can fail > because of this problem. I have logged another bug for this.) > > Description: > Due to the marking cleanup reclaiming empty regions, and having stale > references a crash can occur when doing a heap dump. > The code tries to do an is_klass check on the object, which crashes > the VM. > The fix is to add an is_perm check before doing the check, since > is_perm will do a bounds check on the oop and if it's in the perm gen > we know it's safe to look at it since G1 only ever does full > compactions of the perm gen. > > For more information, and a more in-depth analysis, please see the > jira bug. > > http://cr.openjdk.java.net/~aeriksso/8038925/webrev.03/ > https://bugs.openjdk.java.net/browse/JDK-8038925 > > Regards, > Andreas > > On 2014-05-22 10:14, Andreas Eriksson wrote: >> OK, I'll remove it. >> >> Thanks, >> Andreas >> >> On 2014-05-22 10:02, Bengt Rutisson wrote: >>> >>> >>> Hi Andreas, >>> >>> On 5/21/14 2:05 PM, Andreas Eriksson wrote: >>>> A new webrev is up: >>>> http://cr.openjdk.java.net/~aeriksso/8038925/webrev.02/ >>>> >>>> The test now verifies that no full GC has been done after doing the >>>> heap dump. >>>> I also modified the test to not run if it for some reason is not >>>> using G1. >>> >>> Thanks for the update, Andreas. >>> >>> The test looks good except for the @run tag. >>> >>> @run main/othervm -Xms512m -Xmx1024m -XX:+UseG1GC >>> -XX:+ExplicitGCInvokesConcurrent TestG1ConcurrentGCHeapDump >>> >>> The problem is that more GC flags will be added when the test is run >>> in nightly testing. Some GC flags will conflict with UseG1GC. On the >>> other hand at some point UseG1GC will be one of the flags that is added. >>> >>> So, I think what you need to do is just remove "-XX:+UseG1GC" from >>> the @run tag. Then your test will report log "skipped" when it is >>> run in the nightly testing except for the nightly testing done with >>> G1 where it will actually do something. >>> >>> Bengt >>> >>>> >>>> Regards, >>>> Andreas >>>> >>>> On 2014-05-21 12:31, Bengt Rutisson wrote: >>>>> On 5/21/14 12:12 PM, Andreas Eriksson wrote: >>>>>> Hi Bengt, thanks for looking at this. >>>>>> >>>>>> I agree that verifying that no full GC has happened would be good. >>>>>> One thing I see as problematic though, is what to do if a full GC >>>>>> has happened. >>>>>> Should I make the test fail? Or is there some way to signal that >>>>>> the test was inconclusive / couldn't finish? >>>>> >>>>> Personally I would prefer to make the test fail. JTreg only has >>>>> two states, PASS or FAIL, and I think that if we allow it to pass >>>>> we might not notice if there is some change that makes the test >>>>> always get a full GC and then never actually testing anything. >>>>> >>>>> I don't think it will fail intermittently by getting full GCs. I >>>>> think the test is pretty stable. But I think it would be good to >>>>> have a way of noticing if it stops testing what it is supposed to >>>>> test. >>>>> >>>>> (By the way, I would really like a "SKIPPED" state in JTreg but I >>>>> haven't had any luck pushing for that. I think it could be useful >>>>> for other things as well. There is for example no good reason for >>>>> your test to be run 4 times each night with the exact same binary. >>>>> But that is what happens since we can't say that we should skip >>>>> this test if we run with any other GC than G1.) >>>>> >>>>> Thanks, >>>>> Bengt >>>>> >>>>>> >>>>>> Regards, >>>>>> Andreas >>>>>> >>>>>> On 2014-05-21 11:55, Bengt Rutisson wrote: >>>>>>> >>>>>>> Hi Andreas, >>>>>>> >>>>>>> The fix looks good. >>>>>>> >>>>>>> One comment about the test. It does not verify that no full GC >>>>>>> happens. The way the test is set up I guess that should not >>>>>>> happen and I am not sure it is worth the effort to add a check >>>>>>> for it. Just wanted to mention it if you want to make test more >>>>>>> resilient to future changes in the JVM that for some reason can >>>>>>> trigger a full GC for this test. >>>>>>> >>>>>>> I'm fine with leaving the test as it is. >>>>>>> >>>>>>> Thanks, >>>>>>> Bengt >>>>>>> >>>>>>> >>>>>>> On 5/20/14 4:26 PM, Andreas Eriksson wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> Could I have a review for this G1 jdk7 only fix please? >>>>>>>> (There is a related problem in jdk8 and jdk9, where an assert >>>>>>>> can fail because of this problem. I have logged another bug for >>>>>>>> this.) >>>>>>>> >>>>>>>> Description: >>>>>>>> Due to the marking cleanup reclaiming empty regions, and having >>>>>>>> stale references a crash can occur when doing a heap dump. >>>>>>>> The code tries to do an is_klass check on the object, which >>>>>>>> crashes the VM. >>>>>>>> The fix is to add an is_perm check before doing the check, >>>>>>>> since is_perm will do a bounds check on the oop and if it's in >>>>>>>> the perm gen we know it's safe to look at it since G1 only ever >>>>>>>> does full compactions of the perm gen. >>>>>>>> >>>>>>>> For more information, and a more in-depth analysis, please see >>>>>>>> the jira bug. >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~aeriksso/8038925/webrev.01/ >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8038925 >>>>>>>> >>>>>>>> Regards, >>>>>>>> Andreas >>>>>>> >>>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From andreas.eriksson at oracle.com Thu May 22 11:39:13 2014 From: andreas.eriksson at oracle.com (Andreas Eriksson) Date: Thu, 22 May 2014 13:39:13 +0200 Subject: RFR: 8038925: Java with G1 crashes in dump_instance_fields using jmap or jcmd without fullgc In-Reply-To: <537DD93E.1010303@oracle.com> References: <537B6620.6020600@oracle.com> <537C7814.8020806@oracle.com> <537C7C2B.1080105@oracle.com> <537C8077.8030400@oracle.com> <537C9695.5010902@oracle.com> <537DAEFC.2000508@oracle.com> <537DB1F2.6020406@oracle.com> <537DB8B1.5080905@oracle.com> <537DD93E.1010303@oracle.com> Message-ID: <537DE1E1.60803@oracle.com> Hi, Bengt: Right, that should be enough, thanks. Mikael: Can I use you as a reviewer for this latest version as well? Regards, Andreas On 2014-05-22 13:02, Bengt Rutisson wrote: > > Hi Andreas, > > This is a HotSpot change and for JDK7 HotSpot was developed in the hsx > project. I am a Reviewer in the hsx project > (http://openjdk.java.net/census#brutisso) isn't that enough to review > this change? > > Anyway, last webrev looks good. Thanks for fixing the test! > > Bengt > > On 5/22/14 10:43 AM, Andreas Eriksson wrote: >> Hi, >> >> Adding jdk7u-dev. >> >> Could I have a jdk7u Reviewer look at this as well please? This is a >> jdk7 only fix. >> (There is a related problem in jdk8 and jdk9, where an assert can >> fail because of this problem. I have logged another bug for this.) >> >> Description: >> Due to the marking cleanup reclaiming empty regions, and having stale >> references a crash can occur when doing a heap dump. >> The code tries to do an is_klass check on the object, which crashes >> the VM. >> The fix is to add an is_perm check before doing the check, since >> is_perm will do a bounds check on the oop and if it's in the perm gen >> we know it's safe to look at it since G1 only ever does full >> compactions of the perm gen. >> >> For more information, and a more in-depth analysis, please see the >> jira bug. >> >> http://cr.openjdk.java.net/~aeriksso/8038925/webrev.03/ >> https://bugs.openjdk.java.net/browse/JDK-8038925 >> >> Regards, >> Andreas >> >> On 2014-05-22 10:14, Andreas Eriksson wrote: >>> OK, I'll remove it. >>> >>> Thanks, >>> Andreas >>> >>> On 2014-05-22 10:02, Bengt Rutisson wrote: >>>> >>>> >>>> Hi Andreas, >>>> >>>> On 5/21/14 2:05 PM, Andreas Eriksson wrote: >>>>> A new webrev is up: >>>>> http://cr.openjdk.java.net/~aeriksso/8038925/webrev.02/ >>>>> >>>>> The test now verifies that no full GC has been done after doing >>>>> the heap dump. >>>>> I also modified the test to not run if it for some reason is not >>>>> using G1. >>>> >>>> Thanks for the update, Andreas. >>>> >>>> The test looks good except for the @run tag. >>>> >>>> @run main/othervm -Xms512m -Xmx1024m -XX:+UseG1GC >>>> -XX:+ExplicitGCInvokesConcurrent TestG1ConcurrentGCHeapDump >>>> >>>> The problem is that more GC flags will be added when the test is >>>> run in nightly testing. Some GC flags will conflict with UseG1GC. >>>> On the other hand at some point UseG1GC will be one of the flags >>>> that is added. >>>> >>>> So, I think what you need to do is just remove "-XX:+UseG1GC" from >>>> the @run tag. Then your test will report log "skipped" when it is >>>> run in the nightly testing except for the nightly testing done with >>>> G1 where it will actually do something. >>>> >>>> Bengt >>>> >>>>> >>>>> Regards, >>>>> Andreas >>>>> >>>>> On 2014-05-21 12:31, Bengt Rutisson wrote: >>>>>> On 5/21/14 12:12 PM, Andreas Eriksson wrote: >>>>>>> Hi Bengt, thanks for looking at this. >>>>>>> >>>>>>> I agree that verifying that no full GC has happened would be good. >>>>>>> One thing I see as problematic though, is what to do if a full >>>>>>> GC has happened. >>>>>>> Should I make the test fail? Or is there some way to signal that >>>>>>> the test was inconclusive / couldn't finish? >>>>>> >>>>>> Personally I would prefer to make the test fail. JTreg only has >>>>>> two states, PASS or FAIL, and I think that if we allow it to pass >>>>>> we might not notice if there is some change that makes the test >>>>>> always get a full GC and then never actually testing anything. >>>>>> >>>>>> I don't think it will fail intermittently by getting full GCs. I >>>>>> think the test is pretty stable. But I think it would be good to >>>>>> have a way of noticing if it stops testing what it is supposed to >>>>>> test. >>>>>> >>>>>> (By the way, I would really like a "SKIPPED" state in JTreg but I >>>>>> haven't had any luck pushing for that. I think it could be useful >>>>>> for other things as well. There is for example no good reason for >>>>>> your test to be run 4 times each night with the exact same >>>>>> binary. But that is what happens since we can't say that we >>>>>> should skip this test if we run with any other GC than G1.) >>>>>> >>>>>> Thanks, >>>>>> Bengt >>>>>> >>>>>>> >>>>>>> Regards, >>>>>>> Andreas >>>>>>> >>>>>>> On 2014-05-21 11:55, Bengt Rutisson wrote: >>>>>>>> >>>>>>>> Hi Andreas, >>>>>>>> >>>>>>>> The fix looks good. >>>>>>>> >>>>>>>> One comment about the test. It does not verify that no full GC >>>>>>>> happens. The way the test is set up I guess that should not >>>>>>>> happen and I am not sure it is worth the effort to add a check >>>>>>>> for it. Just wanted to mention it if you want to make test more >>>>>>>> resilient to future changes in the JVM that for some reason can >>>>>>>> trigger a full GC for this test. >>>>>>>> >>>>>>>> I'm fine with leaving the test as it is. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Bengt >>>>>>>> >>>>>>>> >>>>>>>> On 5/20/14 4:26 PM, Andreas Eriksson wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> Could I have a review for this G1 jdk7 only fix please? >>>>>>>>> (There is a related problem in jdk8 and jdk9, where an assert >>>>>>>>> can fail because of this problem. I have logged another bug >>>>>>>>> for this.) >>>>>>>>> >>>>>>>>> Description: >>>>>>>>> Due to the marking cleanup reclaiming empty regions, and >>>>>>>>> having stale references a crash can occur when doing a heap dump. >>>>>>>>> The code tries to do an is_klass check on the object, which >>>>>>>>> crashes the VM. >>>>>>>>> The fix is to add an is_perm check before doing the check, >>>>>>>>> since is_perm will do a bounds check on the oop and if it's in >>>>>>>>> the perm gen we know it's safe to look at it since G1 only >>>>>>>>> ever does full compactions of the perm gen. >>>>>>>>> >>>>>>>>> For more information, and a more in-depth analysis, please see >>>>>>>>> the jira bug. >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~aeriksso/8038925/webrev.01/ >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8038925 >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> Andreas >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From per.liden at oracle.com Thu May 22 13:10:21 2014 From: per.liden at oracle.com (Per Liden) Date: Thu, 22 May 2014 15:10:21 +0200 Subject: RFR(S): 8040807: G1: Enable G1CollectedHeap::stop() In-Reply-To: <537B6A8A.5030607@oracle.com> References: <537B43BC.2090004@oracle.com> <537B6A8A.5030607@oracle.com> Message-ID: <537DF73D.6080204@oracle.com> Thanks Jon! /Per On 2014-05-20 16:45, Jon Masamitsu wrote: > Looks good. > > Reviewed. > > Jon > > On 05/20/2014 04:59 AM, Per Liden wrote: >> Looking for a couple of reviews in this patch. >> >> Summary: This patch re-enables the controlled stopping of G1's >> concurrent threads at VM shutdown. This could potentially cause hangs >> during VM shutdown because the G1 marking threads could get stuck in >> various places and fail to terminate. JDK-8040803 and JDK-8040804 >> fixed these issues, so this is the final step to re-enable the actual >> stopping of those threads. This patch also moves the call to >> CollectedHeap::stop() a few lines down to group the GC related stuff >> together. It also adjusts/removes some comments that are no longer >> correct. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8040807 >> Webrev: http://cr.openjdk.java.net/~pliden/8040807/webrev.0/ >> >> Testing: >> - GC nightlies. 5 tests in this suite used to timeout because of the >> issue with hanging threads. They now pass. >> - JPRT >> >> Thanks! >> /Per > From dmitry.fazunenko at oracle.com Thu May 22 15:58:59 2014 From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko) Date: Thu, 22 May 2014 19:58:59 +0400 Subject: RFR: 8040250: The test test/gc/parallelScavenge/TestDynShrinkHeap.java fails with OOME In-Reply-To: <53737D83.2050205@oracle.com> References: <53737D83.2050205@oracle.com> Message-ID: <537E1EC3.9080607@oracle.com> Hello, Would you review a very simple test fix, please. Description: Test allocated ~1G memory and failed with OOM before any checks. The fixed version specifies -Xmx1G and allocates 0.5G. The test checks that memory could be decommited, so for the test it's doesn't meter how much memory to allocate initially. Bug: https://bugs.openjdk.java.net/browse/JDK-8040250 Webrev: http://cr.openjdk.java.net/~iignatyev/dfazunenko/8040250/webrev.00/ Testing: This is the very simple fix. I tested it locally running jtreg. The test failed before fix and passes after. Any your comments are very welcome. Thanks, Dima From jesper.wilhelmsson at oracle.com Thu May 22 16:46:00 2014 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Thu, 22 May 2014 18:46:00 +0200 Subject: RFR: 8040250: The test test/gc/parallelScavenge/TestDynShrinkHeap.java fails with OOME In-Reply-To: <537E1EC3.9080607@oracle.com> References: <53737D83.2050205@oracle.com> <537E1EC3.9080607@oracle.com> Message-ID: <537E29C8.8080906@oracle.com> Looks good! /Jesper Dmitry Fazunenko skrev 22/5/14 17:58: > Hello, > > Would you review a very simple test fix, please. > > Description: > Test allocated ~1G memory and failed with OOM before any checks. > The fixed version specifies -Xmx1G and allocates 0.5G. > The test checks that memory could be decommited, so for the test it's doesn't > meter how much memory to allocate initially. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8040250 > Webrev: http://cr.openjdk.java.net/~iignatyev/dfazunenko/8040250/webrev.00/ > > > Testing: > This is the very simple fix. I tested it locally running jtreg. The test failed > before fix and passes after. > > Any your comments are very welcome. > > Thanks, > Dima From jon.masamitsu at oracle.com Thu May 22 21:31:35 2014 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Thu, 22 May 2014 14:31:35 -0700 Subject: RFR: 8040250: The test test/gc/parallelScavenge/TestDynShrinkHeap.java fails with OOME In-Reply-To: <537E1EC3.9080607@oracle.com> References: <53737D83.2050205@oracle.com> <537E1EC3.9080607@oracle.com> Message-ID: <537E6CB7.6040305@oracle.com> Dima, Change looks good. I noticed that the @run has UseParallelGC on it. @run main/othervm -XX:+UseAdaptiveSizePolicyWithSystemGC -XX:+UseParallelGC -XX:MinHeapFreeRatio=0 -XX:MaxHeapFreeRatio=100 -Xmx1g -verbose:gc TestDynShrinkHeap Why doesn't this test run into the problem of multiple GC's specified on the command line when run with other GC's? For example, I would expect this test to be running with all the GC's when nightly testing is done. I'm asking because that seems to be the problem with 8042051 - gc/g1/TestHumongousShrinkHeap.java: Conflicting collector combinations in option list ? Thanks. Jon On 05/22/2014 08:58 AM, Dmitry Fazunenko wrote: > Hello, > > Would you review a very simple test fix, please. > > Description: > Test allocated ~1G memory and failed with OOM before any checks. > The fixed version specifies -Xmx1G and allocates 0.5G. > The test checks that memory could be decommited, so for the test it's > doesn't meter how much memory to allocate initially. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8040250 > Webrev: > http://cr.openjdk.java.net/~iignatyev/dfazunenko/8040250/webrev.00/ > > > Testing: > This is the very simple fix. I tested it locally running jtreg. The > test failed before fix and passes after. > > Any your comments are very welcome. > > Thanks, > Dima From mikael.gerdin at oracle.com Fri May 23 10:37:25 2014 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Fri, 23 May 2014 12:37:25 +0200 Subject: RFR: 8038925: Java with G1 crashes in dump_instance_fields using jmap or jcmd without fullgc In-Reply-To: <537DE1E1.60803@oracle.com> References: <537B6620.6020600@oracle.com> <537DD93E.1010303@oracle.com> <537DE1E1.60803@oracle.com> Message-ID: <26952292.MpAIKFi7Cd@mgerdin03> On Thursday 22 May 2014 13.39.13 Andreas Eriksson wrote: > Hi, > > Bengt: > > Right, that should be enough, thanks. > > Mikael: > > Can I use you as a reviewer for this latest version as well? Yes, this looks fine. /Mikael > > Regards, > Andreas > > On 2014-05-22 13:02, Bengt Rutisson wrote: > > Hi Andreas, > > > > This is a HotSpot change and for JDK7 HotSpot was developed in the hsx > > project. I am a Reviewer in the hsx project > > (http://openjdk.java.net/census#brutisso) isn't that enough to review > > this change? > > > > Anyway, last webrev looks good. Thanks for fixing the test! > > > > Bengt > > > > On 5/22/14 10:43 AM, Andreas Eriksson wrote: > >> Hi, > >> > >> Adding jdk7u-dev. > >> > >> Could I have a jdk7u Reviewer look at this as well please? This is a > >> jdk7 only fix. > >> (There is a related problem in jdk8 and jdk9, where an assert can > >> fail because of this problem. I have logged another bug for this.) > >> > >> Description: > >> Due to the marking cleanup reclaiming empty regions, and having stale > >> references a crash can occur when doing a heap dump. > >> The code tries to do an is_klass check on the object, which crashes > >> the VM. > >> The fix is to add an is_perm check before doing the check, since > >> is_perm will do a bounds check on the oop and if it's in the perm gen > >> we know it's safe to look at it since G1 only ever does full > >> compactions of the perm gen. > >> > >> For more information, and a more in-depth analysis, please see the > >> jira bug. > >> > >> http://cr.openjdk.java.net/~aeriksso/8038925/webrev.03/ > >> https://bugs.openjdk.java.net/browse/JDK-8038925 > >> > >> Regards, > >> Andreas > >> > >> On 2014-05-22 10:14, Andreas Eriksson wrote: > >>> OK, I'll remove it. > >>> > >>> Thanks, > >>> Andreas > >>> > >>> On 2014-05-22 10:02, Bengt Rutisson wrote: > >>>> Hi Andreas, > >>>> > >>>> On 5/21/14 2:05 PM, Andreas Eriksson wrote: > >>>>> A new webrev is up: > >>>>> http://cr.openjdk.java.net/~aeriksso/8038925/webrev.02/ > >>>>> > >>>>> The test now verifies that no full GC has been done after doing > >>>>> the heap dump. > >>>>> I also modified the test to not run if it for some reason is not > >>>>> using G1. > >>>> > >>>> Thanks for the update, Andreas. > >>>> > >>>> The test looks good except for the @run tag. > >>>> > >>>> @run main/othervm -Xms512m -Xmx1024m -XX:+UseG1GC > >>>> -XX:+ExplicitGCInvokesConcurrent TestG1ConcurrentGCHeapDump > >>>> > >>>> The problem is that more GC flags will be added when the test is > >>>> run in nightly testing. Some GC flags will conflict with UseG1GC. > >>>> On the other hand at some point UseG1GC will be one of the flags > >>>> that is added. > >>>> > >>>> So, I think what you need to do is just remove "-XX:+UseG1GC" from > >>>> the @run tag. Then your test will report log "skipped" when it is > >>>> run in the nightly testing except for the nightly testing done with > >>>> G1 where it will actually do something. > >>>> > >>>> Bengt > >>>> > >>>>> Regards, > >>>>> Andreas > >>>>> > >>>>> On 2014-05-21 12:31, Bengt Rutisson wrote: > >>>>>> On 5/21/14 12:12 PM, Andreas Eriksson wrote: > >>>>>>> Hi Bengt, thanks for looking at this. > >>>>>>> > >>>>>>> I agree that verifying that no full GC has happened would be good. > >>>>>>> One thing I see as problematic though, is what to do if a full > >>>>>>> GC has happened. > >>>>>>> Should I make the test fail? Or is there some way to signal that > >>>>>>> the test was inconclusive / couldn't finish? > >>>>>> > >>>>>> Personally I would prefer to make the test fail. JTreg only has > >>>>>> two states, PASS or FAIL, and I think that if we allow it to pass > >>>>>> we might not notice if there is some change that makes the test > >>>>>> always get a full GC and then never actually testing anything. > >>>>>> > >>>>>> I don't think it will fail intermittently by getting full GCs. I > >>>>>> think the test is pretty stable. But I think it would be good to > >>>>>> have a way of noticing if it stops testing what it is supposed to > >>>>>> test. > >>>>>> > >>>>>> (By the way, I would really like a "SKIPPED" state in JTreg but I > >>>>>> haven't had any luck pushing for that. I think it could be useful > >>>>>> for other things as well. There is for example no good reason for > >>>>>> your test to be run 4 times each night with the exact same > >>>>>> binary. But that is what happens since we can't say that we > >>>>>> should skip this test if we run with any other GC than G1.) > >>>>>> > >>>>>> Thanks, > >>>>>> Bengt > >>>>>> > >>>>>>> Regards, > >>>>>>> Andreas > >>>>>>> > >>>>>>> On 2014-05-21 11:55, Bengt Rutisson wrote: > >>>>>>>> Hi Andreas, > >>>>>>>> > >>>>>>>> The fix looks good. > >>>>>>>> > >>>>>>>> One comment about the test. It does not verify that no full GC > >>>>>>>> happens. The way the test is set up I guess that should not > >>>>>>>> happen and I am not sure it is worth the effort to add a check > >>>>>>>> for it. Just wanted to mention it if you want to make test more > >>>>>>>> resilient to future changes in the JVM that for some reason can > >>>>>>>> trigger a full GC for this test. > >>>>>>>> > >>>>>>>> I'm fine with leaving the test as it is. > >>>>>>>> > >>>>>>>> Thanks, > >>>>>>>> Bengt > >>>>>>>> > >>>>>>>> On 5/20/14 4:26 PM, Andreas Eriksson wrote: > >>>>>>>>> Hi, > >>>>>>>>> > >>>>>>>>> Could I have a review for this G1 jdk7 only fix please? > >>>>>>>>> (There is a related problem in jdk8 and jdk9, where an assert > >>>>>>>>> can fail because of this problem. I have logged another bug > >>>>>>>>> for this.) > >>>>>>>>> > >>>>>>>>> Description: > >>>>>>>>> Due to the marking cleanup reclaiming empty regions, and > >>>>>>>>> having stale references a crash can occur when doing a heap dump. > >>>>>>>>> The code tries to do an is_klass check on the object, which > >>>>>>>>> crashes the VM. > >>>>>>>>> The fix is to add an is_perm check before doing the check, > >>>>>>>>> since is_perm will do a bounds check on the oop and if it's in > >>>>>>>>> the perm gen we know it's safe to look at it since G1 only > >>>>>>>>> ever does full compactions of the perm gen. > >>>>>>>>> > >>>>>>>>> For more information, and a more in-depth analysis, please see > >>>>>>>>> the jira bug. > >>>>>>>>> > >>>>>>>>> http://cr.openjdk.java.net/~aeriksso/8038925/webrev.01/ > >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8038925 > >>>>>>>>> > >>>>>>>>> Regards, > >>>>>>>>> Andreas From andreas.eriksson at oracle.com Fri May 23 10:38:19 2014 From: andreas.eriksson at oracle.com (Andreas Eriksson) Date: Fri, 23 May 2014 12:38:19 +0200 Subject: RFR: 8038925: Java with G1 crashes in dump_instance_fields using jmap or jcmd without fullgc In-Reply-To: <26952292.MpAIKFi7Cd@mgerdin03> References: <537B6620.6020600@oracle.com> <537DD93E.1010303@oracle.com> <537DE1E1.60803@oracle.com> <26952292.MpAIKFi7Cd@mgerdin03> Message-ID: <537F251B.1020200@oracle.com> Great, thanks. /Andreas On 2014-05-23 12:37, Mikael Gerdin wrote: > On Thursday 22 May 2014 13.39.13 Andreas Eriksson wrote: >> Hi, >> >> Bengt: >> >> Right, that should be enough, thanks. >> >> Mikael: >> >> Can I use you as a reviewer for this latest version as well? > Yes, this looks fine. > > /Mikael > >> Regards, >> Andreas >> >> On 2014-05-22 13:02, Bengt Rutisson wrote: >>> Hi Andreas, >>> >>> This is a HotSpot change and for JDK7 HotSpot was developed in the hsx >>> project. I am a Reviewer in the hsx project >>> (http://openjdk.java.net/census#brutisso) isn't that enough to review >>> this change? >>> >>> Anyway, last webrev looks good. Thanks for fixing the test! >>> >>> Bengt >>> >>> On 5/22/14 10:43 AM, Andreas Eriksson wrote: >>>> Hi, >>>> >>>> Adding jdk7u-dev. >>>> >>>> Could I have a jdk7u Reviewer look at this as well please? This is a >>>> jdk7 only fix. >>>> (There is a related problem in jdk8 and jdk9, where an assert can >>>> fail because of this problem. I have logged another bug for this.) >>>> >>>> Description: >>>> Due to the marking cleanup reclaiming empty regions, and having stale >>>> references a crash can occur when doing a heap dump. >>>> The code tries to do an is_klass check on the object, which crashes >>>> the VM. >>>> The fix is to add an is_perm check before doing the check, since >>>> is_perm will do a bounds check on the oop and if it's in the perm gen >>>> we know it's safe to look at it since G1 only ever does full >>>> compactions of the perm gen. >>>> >>>> For more information, and a more in-depth analysis, please see the >>>> jira bug. >>>> >>>> http://cr.openjdk.java.net/~aeriksso/8038925/webrev.03/ >>>> https://bugs.openjdk.java.net/browse/JDK-8038925 >>>> >>>> Regards, >>>> Andreas >>>> >>>> On 2014-05-22 10:14, Andreas Eriksson wrote: >>>>> OK, I'll remove it. >>>>> >>>>> Thanks, >>>>> Andreas >>>>> >>>>> On 2014-05-22 10:02, Bengt Rutisson wrote: >>>>>> Hi Andreas, >>>>>> >>>>>> On 5/21/14 2:05 PM, Andreas Eriksson wrote: >>>>>>> A new webrev is up: >>>>>>> http://cr.openjdk.java.net/~aeriksso/8038925/webrev.02/ >>>>>>> >>>>>>> The test now verifies that no full GC has been done after doing >>>>>>> the heap dump. >>>>>>> I also modified the test to not run if it for some reason is not >>>>>>> using G1. >>>>>> Thanks for the update, Andreas. >>>>>> >>>>>> The test looks good except for the @run tag. >>>>>> >>>>>> @run main/othervm -Xms512m -Xmx1024m -XX:+UseG1GC >>>>>> -XX:+ExplicitGCInvokesConcurrent TestG1ConcurrentGCHeapDump >>>>>> >>>>>> The problem is that more GC flags will be added when the test is >>>>>> run in nightly testing. Some GC flags will conflict with UseG1GC. >>>>>> On the other hand at some point UseG1GC will be one of the flags >>>>>> that is added. >>>>>> >>>>>> So, I think what you need to do is just remove "-XX:+UseG1GC" from >>>>>> the @run tag. Then your test will report log "skipped" when it is >>>>>> run in the nightly testing except for the nightly testing done with >>>>>> G1 where it will actually do something. >>>>>> >>>>>> Bengt >>>>>> >>>>>>> Regards, >>>>>>> Andreas >>>>>>> >>>>>>> On 2014-05-21 12:31, Bengt Rutisson wrote: >>>>>>>> On 5/21/14 12:12 PM, Andreas Eriksson wrote: >>>>>>>>> Hi Bengt, thanks for looking at this. >>>>>>>>> >>>>>>>>> I agree that verifying that no full GC has happened would be good. >>>>>>>>> One thing I see as problematic though, is what to do if a full >>>>>>>>> GC has happened. >>>>>>>>> Should I make the test fail? Or is there some way to signal that >>>>>>>>> the test was inconclusive / couldn't finish? >>>>>>>> Personally I would prefer to make the test fail. JTreg only has >>>>>>>> two states, PASS or FAIL, and I think that if we allow it to pass >>>>>>>> we might not notice if there is some change that makes the test >>>>>>>> always get a full GC and then never actually testing anything. >>>>>>>> >>>>>>>> I don't think it will fail intermittently by getting full GCs. I >>>>>>>> think the test is pretty stable. But I think it would be good to >>>>>>>> have a way of noticing if it stops testing what it is supposed to >>>>>>>> test. >>>>>>>> >>>>>>>> (By the way, I would really like a "SKIPPED" state in JTreg but I >>>>>>>> haven't had any luck pushing for that. I think it could be useful >>>>>>>> for other things as well. There is for example no good reason for >>>>>>>> your test to be run 4 times each night with the exact same >>>>>>>> binary. But that is what happens since we can't say that we >>>>>>>> should skip this test if we run with any other GC than G1.) >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Bengt >>>>>>>> >>>>>>>>> Regards, >>>>>>>>> Andreas >>>>>>>>> >>>>>>>>> On 2014-05-21 11:55, Bengt Rutisson wrote: >>>>>>>>>> Hi Andreas, >>>>>>>>>> >>>>>>>>>> The fix looks good. >>>>>>>>>> >>>>>>>>>> One comment about the test. It does not verify that no full GC >>>>>>>>>> happens. The way the test is set up I guess that should not >>>>>>>>>> happen and I am not sure it is worth the effort to add a check >>>>>>>>>> for it. Just wanted to mention it if you want to make test more >>>>>>>>>> resilient to future changes in the JVM that for some reason can >>>>>>>>>> trigger a full GC for this test. >>>>>>>>>> >>>>>>>>>> I'm fine with leaving the test as it is. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Bengt >>>>>>>>>> >>>>>>>>>> On 5/20/14 4:26 PM, Andreas Eriksson wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> Could I have a review for this G1 jdk7 only fix please? >>>>>>>>>>> (There is a related problem in jdk8 and jdk9, where an assert >>>>>>>>>>> can fail because of this problem. I have logged another bug >>>>>>>>>>> for this.) >>>>>>>>>>> >>>>>>>>>>> Description: >>>>>>>>>>> Due to the marking cleanup reclaiming empty regions, and >>>>>>>>>>> having stale references a crash can occur when doing a heap dump. >>>>>>>>>>> The code tries to do an is_klass check on the object, which >>>>>>>>>>> crashes the VM. >>>>>>>>>>> The fix is to add an is_perm check before doing the check, >>>>>>>>>>> since is_perm will do a bounds check on the oop and if it's in >>>>>>>>>>> the perm gen we know it's safe to look at it since G1 only >>>>>>>>>>> ever does full compactions of the perm gen. >>>>>>>>>>> >>>>>>>>>>> For more information, and a more in-depth analysis, please see >>>>>>>>>>> the jira bug. >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~aeriksso/8038925/webrev.01/ >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8038925 >>>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> Andreas From thomas.viessmann at oracle.com Mon May 26 08:03:56 2014 From: thomas.viessmann at oracle.com (Thomas Viessmann) Date: Mon, 26 May 2014 10:03:56 +0200 Subject: any plans to lock heap memory to RAM? Message-ID: <5382F56C.9060004@oracle.com> Hi, are there any plans to(optionally) lock memory to RAM e.g. via memctl() in order to avoid paging during GC runs? IMO this becomes more of an issue (and not less) the bigger applications become and the bigger heaps get sized. With a small heap of 1 or 2 G paging was rarely an issue. With a heap of 10 or 20 G paging is more often observed than one might think. Thanks and Regards Thomas -- Oracle THOMAS VIESSMANN | Senior Principal Technical Support Engineer - Java Phone: +498914302496 | Mobile: +491743005467 Oracle Customer Technical Support - Java ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen ORACLE Deutschland B.V. & Co. KG Hauptverwaltung: Riesstr. 25, D-80992 Muenchen Registergericht: Amtsgericht Muenchen, HRA 95603 Gesch?ftsf?hrere: Juergen Kunz Komplement?rin: ORACLE Deutschland Verwaltung B.V. Hertogswetering 163/167, 3543 AS Utrecht, Niederlande Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher ------------------------------------------------------------------------ ------------------------------------------------------------------------ Green Oracle Oracle is committed to developing practices and products that help protect the environment -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: green-for-email-sig_0.gif Type: image/gif Size: 356 bytes Desc: not available URL: From dmitry.fazunenko at oracle.com Mon May 26 13:17:51 2014 From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko) Date: Mon, 26 May 2014 17:17:51 +0400 Subject: RFR: 8040250: The test test/gc/parallelScavenge/TestDynShrinkHeap.java fails with OOME In-Reply-To: <537E6CB7.6040305@oracle.com> References: <53737D83.2050205@oracle.com> <537E1EC3.9080607@oracle.com> <537E6CB7.6040305@oracle.com> Message-ID: <53833EFF.7080500@oracle.com> Hi Jon, Thanks for the review! Actually, gc/parallelScavenge/TestDynShrinkHeap.java experiences the same issue as gc/g1/TestHumongousShrinkHeap.java does, if run with different GC. 8042051 matches both failures... The problem exists only in Main-baseline, when all HS tests are executed with multiple GCs. In GC nighlties the HS tests are executed without extra flags. I hope that adding support of @requires will allow to solve the issue. I also submitted INTJDK-7611364 to fix this problem locally for our nightly testing. Thanks, Dima On 23.05.2014 1:31, Jon Masamitsu wrote: > Dima, > > Change looks good. > > I noticed that the @run has UseParallelGC on it. > > @run main/othervm -XX:+UseAdaptiveSizePolicyWithSystemGC > -XX:+UseParallelGC -XX:MinHeapFreeRatio=0 -XX:MaxHeapFreeRatio=100 > -Xmx1g -verbose:gc TestDynShrinkHeap > > > Why doesn't this test run into the problem of multiple GC's specified > on the command line when run with other GC's? For example, > I would expect this test to be running with all the GC's when > nightly testing is done. > > I'm asking because that seems to be the problem with > > 8042051 - gc/g1/TestHumongousShrinkHeap.java: Conflicting collector > combinations in option list ? > > Thanks. > > Jon > > > On 05/22/2014 08:58 AM, Dmitry Fazunenko wrote: >> Hello, >> >> Would you review a very simple test fix, please. >> >> Description: >> Test allocated ~1G memory and failed with OOM before any checks. >> The fixed version specifies -Xmx1G and allocates 0.5G. >> The test checks that memory could be decommited, so for the test it's >> doesn't meter how much memory to allocate initially. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8040250 >> Webrev: >> http://cr.openjdk.java.net/~iignatyev/dfazunenko/8040250/webrev.00/ >> >> >> Testing: >> This is the very simple fix. I tested it locally running jtreg. The >> test failed before fix and passes after. >> >> Any your comments are very welcome. >> >> Thanks, >> Dima > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmitry.fazunenko at oracle.com Mon May 26 15:32:43 2014 From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko) Date: Mon, 26 May 2014 19:32:43 +0400 Subject: RFR: 8041506 - The test gc/g1/TestHumongousShrinkHeap.java reports that memory is not de-committed In-Reply-To: <5374A502.5030409@oracle.com> References: <5374A502.5030409@oracle.com> Message-ID: <53835E9B.7080102@oracle.com> Hi Andrey, Sorry, it took too long from me to review you change. I have several comments: - overall fix looks good - I think you need to change the subject: you fix 8041946, not 8041506 - Replace 'TestHumongousShrinkHeap' with 'TestShrinkDefragmentedHeap' - Make MemoryUsagePrinter as static inner class of test (to avoid possible conflicts with other tests) - It would be good if you provide more text description to the test, like * allocate small objects mixed with humongous ones "ssssHssssHssssHssssHssssH" * release all allocated object except the last humongous one ".............................................H" * invoke gc and check that memory returned to the system (amount of committed memory got down) - I'm not sure that you can predict the expected amount of committed memory at the end... I wouldn't use the expectedCommitted in the test (there are many memory consumers, not only your test, so the final committed should be either less or greater than expectedCommitted ) - I think you don't need to touch 'test/TEST.groups'. There is :needs_g1gc tests group (hs/test/closed/TEST.group) which lists all g1 specific tests. - Please provide information on how you tested your change. Thanks, Dima On 15.05.2014 15:29, Andrey Zakharov wrote: > Hi. > To proper testing of free list sorting we need to defragment memory > with small young and humongous objects > This is test scenario: > Make enough space for new objects to prevent it going old. > - allocate bunch of small objects, and a bit of humongous > several times. > > Free almost all of allocated stuff. Check that heap shrinks after GC. > > webrev: http://cr.openjdk.java.net/~jwilhelm/8041506/webrev.02/ > bug: https://bugs.openjdk.java.net/browse/JDK-8041506 > > Thanks. > From thomas.schatzl at oracle.com Tue May 27 10:39:16 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 27 May 2014 12:39:16 +0200 Subject: RFR (XS): JDK-8040977: G1 crashes when run with -XX:-G1DeferredRSUpdate In-Reply-To: <5360D563.1070303@oracle.com> References: <1398171391.3002.24.camel@cirrus> <5360D563.1070303@oracle.com> Message-ID: <1401187156.2682.12.camel@cirrus> Hi Bengt, thanks for the review. On Wed, 2014-04-30 at 12:50 +0200, Bengt Rutisson wrote: > Hi Thomas, > > On 2014-04-22 14:56, Thomas Schatzl wrote: > > Hi all, > > > > can I have reviews for this change? It fixes wrong order of > > declaration of members of G1ParScanThreadState that causes crashes when > > G1DeferredRSUpdate is disabled. > > > > The change is based on the changes for 8035400 and8035401 posted recently. > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8040977 > > > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8040977/webrev/ > > I realize that this fixes the code but I would really appreciate a more > stable way of handling the dependencies. > > As it it now we end up calling methods on a G1ParScanThreadState > instance while we are setting it up. This seems broken to me and will > probably lead to similar initialization order issues again. Best would > be to not pass "this" to the constructor of G1ParScanClosure and instead > manage the circular dependency between G1ParScanClosure and > G1ParScanThreadState more explicitly after they have both been properly > set up. > > Second best would be to at least pass the worker id/queue num as a > separate parameter to avoid having to call methods on an uninitialized > object. I fixed this implementing the former idea. Also added some New webrev at http://cr.openjdk.java.net/~tschatzl/8040977/webrev.1/ (Sorry, I already had merged the changes before making a diff webrev - however, most changes in the VM code have been redone anyway. The test case stayed the same). Thanks, Thomas From thomas.schatzl at oracle.com Tue May 27 10:39:21 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 27 May 2014 12:39:21 +0200 Subject: RFR (M/L): JDK-8035400: Move G1ParScanThreadState into its own files In-Reply-To: <536A1685.7080704@oracle.com> References: <1397829156.2717.24.camel@cirrus> <536A1685.7080704@oracle.com> Message-ID: <1401187161.2682.13.camel@cirrus> Hi Bengt, thanks for the review and sorry for the long delay... On Wed, 2014-05-07 at 13:18 +0200, Bengt Rutisson wrote: > Hi Thomas, > > On 2014-04-18 15:52, Thomas Schatzl wrote: > > Hi all, > > > > can I have reviews for the above change? It moves G1ParScanThreadState > > into G1ParScanThreadState*pp files. > > > > The only changes are limited to: > > - adding a "#pragma warning( disable:4355 ) // 'this' : used in base > > member initializer list" to shut visual C up about the problem (which > > should be cleaned up at some point - I found an issue that slipped > > through because of that, JDK-8040977) > > As I commented in the review of JDK-8040977 I would prefer to make the > change to not pass this as a parameter to the constructor. That would > also remove the need for disabling the warning. Maybe in that case base > this review on top of the fix for JDK-8040977 rather than the other way > around? I do not see an advantage either way. Since this would require me make significant changes to all patches, I would prefer keeping the order this way if you do not mind. In the latest JDK-8040977 I removed the need for the pragma as requested. > > - added necessary include file references; I hope the AIX guys can > > compile that change to avoid troubles. It compiles fine with all Oracle > > supported archs. > > You also moved the definition of the destructor of G1ParScanThreadState > from the hpp file to the cpp file. Makes sense, but was not strictly > needed for this change, right? Fixed that. This has been an oversight when separating out the changes. > > There will be another CR for fixing up visibility and cleaning up stuff > > a little. > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8035400 > > > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8035400/webrev/ > > It is a bit hard to review moved code. But except for the comment > regarding JDK-8040977 above I think it looks good. > > I think you can clean up the includes a bit more if you have time. Seems > like these includes in g1CollectedHeap.cpp are for example not needed > anymore: > > #include "oops/oop.inline.hpp" > #include "oops/oop.pcgc.inline.hpp" I tried to clean up the includes a little more. However you cannot move these particular includes because they are still needed for evacuation failure handling. I also rebased the change on the current hotspot jdk9 gc repo. Diff webrev at http://cr.openjdk.java.net/~tschatzl/8035400/webrev.1_to_2/ Complete webrev at http://cr.openjdk.java.net/~tschatzl/8035400/webrev.2/ Thanks, Thomas From thomas.schatzl at oracle.com Tue May 27 10:39:25 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 27 May 2014 12:39:25 +0200 Subject: RFR (M): JDK-8035401: Fix visibility of G1ParScanThreadState members In-Reply-To: <5360E01E.7020906@oracle.com> References: <1397827791.2717.23.camel@cirrus> <5356CC15.7070507@oracle.com> <1398198974.2532.32.camel@cirrus> <5356DFB3.7050400@oracle.com> <5360E01E.7020906@oracle.com> Message-ID: <1401187165.2682.14.camel@cirrus> Hi Bengt, thanks for the review. On Wed, 2014-04-30 at 13:35 +0200, Bengt Rutisson wrote: > Hi Thomas, > > Over all this looks good to me too. > > One question for g1ParScanThreadState.cpp. You have marked the > deal_with_refrence() methods as "inline" even though they are in the > same cpp file. Does that have any effect? I moved them to the inline files. > > 394 > 395 template inline void > G1ParScanThreadState::deal_with_reference(T* ref_to_scan) { > 396 if (!has_partial_array_mask(ref_to_scan)) { > 397 // Note: we can use "raw" versions of "region_containing" because > 398 // "obj_to_scan" is definitely in the heap, and is not in a > 399 // humongous region. > 400 HeapRegion* r = _g1h->heap_region_containing_raw(ref_to_scan); > 401 do_oop_evac(ref_to_scan, r); > 402 } else { > 403 do_oop_partial_array((oop*)ref_to_scan); > 404 } > 405 } > 406 > 407 inline void G1ParScanThreadState::deal_with_reference(StarTask ref) { > 408 assert(verify_task(ref), "sanity"); > 409 if (ref.is_narrow()) { > 410 deal_with_reference((narrowOop*)ref); > 411 } else { > 412 deal_with_reference((oop*)ref); > 413 } > 414 } > > Also, I think that you have to declare methods that should be inlined > before the place where they are being used on some platforms (Solaris). > In this case I think it means that they should be declared before > steal_and_trim_queue(). Moved them to the inline file. > > Personally I also find the new deal_with_reference(StarTask ref) a > little confusing. With that method and the two methods generated by > deal_with_reference(T* ref_to_scan) I get kind of unsure which method > that will be executed by a call like: > > 156 StarTask stolen_task; > 157 while (task_queues->steal(queue_num(), hash_seed(), stolen_task)) { > 158 assert(verify_task(stolen_task), "sanity"); > 159 deal_with_reference(stolen_task); > > All three deal_with_reference() methods are potential matches. I assume > the compiler prefers the deal_with_reference(StarTask ref) but it makes > me unsure when I read the code. Changed to dispatch_reference(). > > One minor nit: > > g1ParScanThreadState.hpp > You have changed the indentation of private/protected/public keywords to > have one space indentation. That's fine as I think that is the standard, > but since the whole file used no space indentation I would also have > been fine with leaving that. However now the last "public" keyword is > still having no space before it. Can you indent that too? > > 218 public: Fixed. Also removed superfluous newlines at the end of files. Also re-checked again for performance regressions, none found. Diff to last revision http://cr.openjdk.java.net/~tschatzl/8035401/webrev.1_to_2/ Full diff: http://cr.openjdk.java.net/~tschatzl/8035401/webrev.2/ (based on 8035400) Thanks, Thomas > > Thanks, > Bengt > > > On 2014-04-22 23:31, Jon Masamitsu wrote: > > > > On 4/22/14 1:36 PM, Thomas Schatzl wrote: > >> Hi Jon, > >> > >> On Tue, 2014-04-22 at 13:07 -0700, Jon Masamitsu wrote: > >>> Thomas, > >>> > >>> What I see in these changes are > >>> > >>> 1) no semantic changes > >> No. > >> > >>> 2) some methods in .hpp files moved to .cpp files > >> Yes, because they were only referenced by the cpp file, so I thought it > >> would be good to move them there. They will be inlined as needed anyway > >> (and I think for some of them they were never inlined due to their > >> size). > >> > >> I will do some more runs with the inline's added again. > >> > >>> 3) creation of steal_and_trim_queue() with definition in > >>> a .cpp file (I may have missed additional such new > >>> methods) > >> There are none except queue_is_empty(), see below. > >> > >>> 4) change in visibility as the CR says > >> That's the main change. > >> > >>> 5) no performance regressions as stated in your RFR > >> No. Checked the results for the usual benchmarks (specjvm2008, > >> specjbb05/2013) again right now, and there are no significant > >> differences in the scores (on x64 and sparc), and for specjbb05/2013 the > >> average gc pause time, and the object copy time (assuming that this is > >> the part that will be affected most) stay the same as in the baseline. > >> > >>> If that's what constitutes the change, looks good. > >> Thanks. > >> > >>> Reviewed. > >>> > >>> If there is something more significant that I have > >>> overlooked, please point me at it and I'll look again. > >> There is not. Sorry, I should have pointed out the changes in more > >> detail instead of you making guesses. > >> > >> Additional minor changes: > >> > >> - G1ParScanThreadState accesses members directly instead of using > >> getters (e.g. _refs instead of refs()). > >> > >> - fixed some newlines in method declarations, removing newlines > >> > >> - removed refs() to avoid direct access from outside, and adding a new > >> method queue_is_empty() (only used in asserts as refs()->is_empty(), and > >> I did not want to expose refs() just for the asserts). > > > > All looks good. > > > > Reviewed. > > > > Jon > > > >> > >> Thanks, > >> Thomas > >> > >>> On 4/18/14 6:29 AM, Thomas Schatzl wrote: > >>>> Hi all, > >>>> > >>>> can I have reviews for this change? After moving > >>>> G1ParScanThreadState, > >>>> this change cleans up visibility, making a whole lot of stuff private. > >>>> > >>>> CR: > >>>> https://bugs.openjdk.java.net/browse/JDK-8035401 > >>>> > >>>> Webrev: > >>>> http://cr.openjdk.java.net/~tschatzl/8035401/webrev/ > >>>> > >>>> Testing: > >>>> perf testing indicated no changes, jprt > >>>> > >>>> Thanks, > >>>> Thomas > >>>> > >>>> > >> > > > From per.liden at oracle.com Tue May 27 11:15:07 2014 From: per.liden at oracle.com (Per Liden) Date: Tue, 27 May 2014 13:15:07 +0200 Subject: RFR(S): 8040807: G1: Enable G1CollectedHeap::stop() In-Reply-To: <537DF73D.6080204@oracle.com> References: <537B43BC.2090004@oracle.com> <537B6A8A.5030607@oracle.com> <537DF73D.6080204@oracle.com> Message-ID: <538473BB.9070300@oracle.com> Hi, I did some additional testing and eyeballing of this fix and noticed that it would be a good idea to also tell concurrent mark to abort, otherwise we will always wait until concurrent mark has finished, which is unnecessary (and could potentially take some time if the live set is large). So, I added a call to _cm->set_has_aborted() to abort any ongoing concurrent mark. Updated webrev: http://cr.openjdk.java.net/~pliden/8040807/webrev.1/ Diff against previous webrev: http://cr.openjdk.java.net/~pliden/8040807/webrev.diff_0vs1/ Testing: Wrote a simple test to provoke an concurrent mark followed by an immediate exit. With the first version of the patch, we would always wait until concurrent mark completes. Now it will instead show an concurrent-mark-abort, which happens much earlier. /Per On 05/22/2014 03:10 PM, Per Liden wrote: > Thanks Jon! > > /Per > > On 2014-05-20 16:45, Jon Masamitsu wrote: >> Looks good. >> >> Reviewed. >> >> Jon >> >> On 05/20/2014 04:59 AM, Per Liden wrote: >>> Looking for a couple of reviews in this patch. >>> >>> Summary: This patch re-enables the controlled stopping of G1's >>> concurrent threads at VM shutdown. This could potentially cause hangs >>> during VM shutdown because the G1 marking threads could get stuck in >>> various places and fail to terminate. JDK-8040803 and JDK-8040804 >>> fixed these issues, so this is the final step to re-enable the actual >>> stopping of those threads. This patch also moves the call to >>> CollectedHeap::stop() a few lines down to group the GC related stuff >>> together. It also adjusts/removes some comments that are no longer >>> correct. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8040807 >>> Webrev: http://cr.openjdk.java.net/~pliden/8040807/webrev.0/ >>> >>> Testing: >>> - GC nightlies. 5 tests in this suite used to timeout because of the >>> issue with hanging threads. They now pass. >>> - JPRT >>> >>> Thanks! >>> /Per >> > From John.Coomes at oracle.com Tue May 27 14:26:17 2014 From: John.Coomes at oracle.com (John Coomes) Date: Tue, 27 May 2014 07:26:17 -0700 Subject: RFR: 8042298 - Remove the names gen0 and gen1 from the GC code In-Reply-To: <537A74EF.9040101@oracle.com> References: <5362F75B.9020201@oracle.com> <53736B77.9030206@oracle.com> <5373DE3E.9080803@oracle.com> <53749246.4070201@oracle.com> <53755759.7040506@oracle.com> <5375FA74.4000701@oracle.com> <537A74EF.9040101@oracle.com> Message-ID: <21380.41097.250774.837985@mykonos.us.oracle.com> Jesper Wilhelmsson (jesper.wilhelmsson at oracle.com) wrote: > ... > So changes since last webrev is that all former usages of CollectGen0First is > now using ScavengeBeforeFullGC, and the removal of ScavengeBeforeFullGC is > reverted. There is also a check in arguments.cpp to set the correct default > value if not running ParallelGC. CollectGen0First and ScavengeBeforeFullGC had > different default values. > > An updated webrev is available here: > http://cr.openjdk.java.net/~jwilhelm/8042298/webrev.3/ This looks good to me. -John > Stefan Johansson skrev 16/5/14 13:45: > > Thanks Jesper, > > > > On 2014-05-16 02:10, Jesper Wilhelmsson wrote: > >> Hi, > >> > >> A new webrev is available here: > >> http://cr.openjdk.java.net/~jwilhelm/8042298/webrev.2/ > >> > >> In addition to the last webrev the flags TraceGen0Time, TraceGen1Time and > >> CollectGen0First have been renamed to get rid of the Gen0/Gen1. > >> > >> Also, since CollectGen0First and ScavengeBeforeFullGC was doing the same thing > >> for different collectors, ScavengeBeforeFullGC has been merged with > >> CollectGen0First. > >> > > Looks good! > > > > Stefan > >> A CCC request has been filed for the flag changes. > >> > >> Thanks, > >> /Jesper > >> > >> > >> Stefan Johansson skrev 15/5/14 12:09: > >>> On 2014-05-14 23:21, Jesper Wilhelmsson wrote: > >>>> Hi Stefan, > >>>> > >>>> Thanks for looking at this! See comments inline. > >>>> > >>>> Stefan Johansson skrev 14/5/14 15:11: > >>>>> Hi Jesper, > >>>>> > >>>>> Thanks for doing this work. Here are some comments: > >>>>> > >>>>> src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp: > >>>>> * TraceGen0Time and TraceGen1Time should be changed to TraceYoungTime and > >>>>> TraceOldTime for consistency, and I guess that will need a CCC request. > >>>>> > >>>>> src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.hpp > >>>>> > >>>>> > >>>>> > >>>>> * CollectGen0First -> CollectYoungFirst, same as above. Also used in > >>>>> tenuredGeneration.hpp. > >>>>> > >>>>> src/share/vm/gc_implementation/g1/g1CollectorPolicy.hpp > >>>>> * Comment starting on line 41 mentions TraceGen0Time and gen1. > >>>> > >>>> I agree that the names gen0 and gen1 should be completely removed, which > >>>> includes changing these flags too. Initially I wanted this change to be a > >>>> simple name change so that it would be easy to review it. Changing the flags > >>>> is borderline to be more than a name change, but I'm fine with including that > >>>> change. > >>>> > >>>> I'll file a CCC to change the flags. > >>>> > >>> Great, thanks. > >>>>> As an additional comment, I think that GenCollectedHeap::get_gen(int) > >>>>> should be > >>>>> broken up into two functions get_young() and get_old(). I get that this was > >>>>> left > >>>>> to a later patch to avoid having code/logic changes in this patch but I think > >>>>> the change is so little that it makes sense to do it in this patch. I would > >>>>> like > >>>>> to avoid having code like, even if it is for a brief period of time: > >>>>> Generation* young = gch->get_gen(0); > >>>> > >>>> This change is part of the larger removal of the generation array which is the > >>>> last part of this series of collector policy cleanups. Initially I was OK with > >>>> moving get_young() and get_old() to this change even though that would mean > >>>> that the change would grow considerably in size and wouldn't be a simple name > >>>> change anymore. After all the only reason I wanted to keep it small was to > >>>> make it easier for the reviewers :-) > >>>> > >>> That's good, thanks =) > >>>> However, when I started to look at it in more detail I realized that removing > >>>> get_gen(int) requires non-trivial changes in the serviceability agent which is > >>>> the only reason why the last change in this series isn't done yet. > >>>> > >>> I understand that you might want to do all the changes in the SA at once, and > >>> since this patch doesn't require you to change the SA I'm fine with leaving it > >>> as is. > >>>> It would be possible to introduce get_young() and get_old() and replace all > >>>> usages of get_gen(int) within the VM, but leave get_gen(int) for the SA to > >>>> use. I don't really like this approach, but if you prefer it over just > >>>> changing the names it's a possibility. > >>>> > >>> I'm fine with any of the approaches, none of them are perfect but both are good > >>> so I'll let you decide how you want to handle it. > >>> > >>> Thanks, > >>> Stefan > >>>> Thanks, > >>>> /Jesper > >>>> > >>>>> > >>>>> Thanks, > >>>>> Stefan > >>>>> > >>>>> On 2014-05-02 03:39, Jesper Wilhelmsson wrote: > >>>>>> Hi, > >>>>>> > >>>>>> This is a step towards removing the _generations array. The names gen0 and > >>>>>> gen1 are replaced with young and old to use a more standardized vocabulary. > >>>>>> > >>>>>> Webrev: http://cr.openjdk.java.net/~jwilhelm/8042298/webrev/ > >>>>>> > >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8042298 > >>>>>> > >>>>>> Thanks, > >>>>>> /Jesper > >>>>> > >>> > > From jesper.wilhelmsson at oracle.com Tue May 27 17:05:10 2014 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Tue, 27 May 2014 19:05:10 +0200 Subject: RFR: 8042298 - Remove the names gen0 and gen1 from the GC code In-Reply-To: <21380.41097.250774.837985@mykonos.us.oracle.com> References: <5362F75B.9020201@oracle.com> <53736B77.9030206@oracle.com> <5373DE3E.9080803@oracle.com> <53749246.4070201@oracle.com> <53755759.7040506@oracle.com> <5375FA74.4000701@oracle.com> <537A74EF.9040101@oracle.com> <21380.41097.250774.837985@mykonos.us.oracle.com> Message-ID: <5384C5C6.80302@oracle.com> Thanks John! /Jesper John Coomes skrev 27/5/14 16:26: > Jesper Wilhelmsson (jesper.wilhelmsson at oracle.com) wrote: >> ... >> So changes since last webrev is that all former usages of CollectGen0First is >> now using ScavengeBeforeFullGC, and the removal of ScavengeBeforeFullGC is >> reverted. There is also a check in arguments.cpp to set the correct default >> value if not running ParallelGC. CollectGen0First and ScavengeBeforeFullGC had >> different default values. >> >> An updated webrev is available here: >> http://cr.openjdk.java.net/~jwilhelm/8042298/webrev.3/ > > This looks good to me. > > -John > >> Stefan Johansson skrev 16/5/14 13:45: >>> Thanks Jesper, >>> >>> On 2014-05-16 02:10, Jesper Wilhelmsson wrote: >>>> Hi, >>>> >>>> A new webrev is available here: >>>> http://cr.openjdk.java.net/~jwilhelm/8042298/webrev.2/ >>>> >>>> In addition to the last webrev the flags TraceGen0Time, TraceGen1Time and >>>> CollectGen0First have been renamed to get rid of the Gen0/Gen1. >>>> >>>> Also, since CollectGen0First and ScavengeBeforeFullGC was doing the same thing >>>> for different collectors, ScavengeBeforeFullGC has been merged with >>>> CollectGen0First. >>>> >>> Looks good! >>> >>> Stefan >>>> A CCC request has been filed for the flag changes. >>>> >>>> Thanks, >>>> /Jesper >>>> >>>> >>>> Stefan Johansson skrev 15/5/14 12:09: >>>>> On 2014-05-14 23:21, Jesper Wilhelmsson wrote: >>>>>> Hi Stefan, >>>>>> >>>>>> Thanks for looking at this! See comments inline. >>>>>> >>>>>> Stefan Johansson skrev 14/5/14 15:11: >>>>>>> Hi Jesper, >>>>>>> >>>>>>> Thanks for doing this work. Here are some comments: >>>>>>> >>>>>>> src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp: >>>>>>> * TraceGen0Time and TraceGen1Time should be changed to TraceYoungTime and >>>>>>> TraceOldTime for consistency, and I guess that will need a CCC request. >>>>>>> >>>>>>> src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.hpp >>>>>>> >>>>>>> >>>>>>> >>>>>>> * CollectGen0First -> CollectYoungFirst, same as above. Also used in >>>>>>> tenuredGeneration.hpp. >>>>>>> >>>>>>> src/share/vm/gc_implementation/g1/g1CollectorPolicy.hpp >>>>>>> * Comment starting on line 41 mentions TraceGen0Time and gen1. >>>>>> >>>>>> I agree that the names gen0 and gen1 should be completely removed, which >>>>>> includes changing these flags too. Initially I wanted this change to be a >>>>>> simple name change so that it would be easy to review it. Changing the flags >>>>>> is borderline to be more than a name change, but I'm fine with including that >>>>>> change. >>>>>> >>>>>> I'll file a CCC to change the flags. >>>>>> >>>>> Great, thanks. >>>>>>> As an additional comment, I think that GenCollectedHeap::get_gen(int) >>>>>>> should be >>>>>>> broken up into two functions get_young() and get_old(). I get that this was >>>>>>> left >>>>>>> to a later patch to avoid having code/logic changes in this patch but I think >>>>>>> the change is so little that it makes sense to do it in this patch. I would >>>>>>> like >>>>>>> to avoid having code like, even if it is for a brief period of time: >>>>>>> Generation* young = gch->get_gen(0); >>>>>> >>>>>> This change is part of the larger removal of the generation array which is the >>>>>> last part of this series of collector policy cleanups. Initially I was OK with >>>>>> moving get_young() and get_old() to this change even though that would mean >>>>>> that the change would grow considerably in size and wouldn't be a simple name >>>>>> change anymore. After all the only reason I wanted to keep it small was to >>>>>> make it easier for the reviewers :-) >>>>>> >>>>> That's good, thanks =) >>>>>> However, when I started to look at it in more detail I realized that removing >>>>>> get_gen(int) requires non-trivial changes in the serviceability agent which is >>>>>> the only reason why the last change in this series isn't done yet. >>>>>> >>>>> I understand that you might want to do all the changes in the SA at once, and >>>>> since this patch doesn't require you to change the SA I'm fine with leaving it >>>>> as is. >>>>>> It would be possible to introduce get_young() and get_old() and replace all >>>>>> usages of get_gen(int) within the VM, but leave get_gen(int) for the SA to >>>>>> use. I don't really like this approach, but if you prefer it over just >>>>>> changing the names it's a possibility. >>>>>> >>>>> I'm fine with any of the approaches, none of them are perfect but both are good >>>>> so I'll let you decide how you want to handle it. >>>>> >>>>> Thanks, >>>>> Stefan >>>>>> Thanks, >>>>>> /Jesper >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Stefan >>>>>>> >>>>>>> On 2014-05-02 03:39, Jesper Wilhelmsson wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> This is a step towards removing the _generations array. The names gen0 and >>>>>>>> gen1 are replaced with young and old to use a more standardized vocabulary. >>>>>>>> >>>>>>>> Webrev: http://cr.openjdk.java.net/~jwilhelm/8042298/webrev/ >>>>>>>> >>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8042298 >>>>>>>> >>>>>>>> Thanks, >>>>>>>> /Jesper >>>>>>> >>>>> >>> From thomas.viessmann at oracle.com Wed May 28 07:29:33 2014 From: thomas.viessmann at oracle.com (Thomas Viessmann) Date: Wed, 28 May 2014 09:29:33 +0200 Subject: G1:long remark pauses In-Reply-To: <53760723.3070002@oracle.com> References: <5375CBBC.2030904@oracle.com> <5375EF5A.3050707@oracle.com> <5375F22E.90003@oracle.com> <5375F3F5.7030808@oracle.com> <5375F7AB.1070908@oracle.com> <5375FC23.5040004@oracle.com> <5375FE26.4080009@oracle.com> <53760723.3070002@oracle.com> Message-ID: <5385905D.6000704@oracle.com> Hi Charlie and all, you were pretty close. It's the weak references which stops the world, not the SoftRefs: 248766.241: [GC remark 248766.246: [GC ref-proc248766.246: [SoftReference, 184918 refs, 0.1139337 secs]248766.360: *[WeakReference, 2104678 refs, 1.9516481 secs]248768.312:* [FinalReference, 45 refs, 0.0033450 secs]248768.315: [PhantomReference, 106 refs, 0.0029866 secs]248768.318: [JNI Weak Reference, 0.0008403 secs], 2.0846210 secs], 2.3878287 secs] [Times: user=25.82 sys=0.87, real=2.39 secs] 248768.630: Total time for which application threads were stopped: 2.4019674 seconds There are also a lot of SoftRefs around but those are not the culprit. Varying -XX:SoftRefLRUPolicyMSPerMB between 100 and 10000 did not change anything. Any idea how this could be addressed? Thanks and Regards Thomas On 05/16/14 14:40, charlie hunt wrote: > From the sound of what Thomas is describing, this might be one of > those apps that's making heavy use of SoftReferences. Output from > -XX:+PrintReferenceGC as Bengt suggested will show if that's the case. > > If we see a large number of SoftReferences being processed per GC, we > may get further help with tuning the SoftReference reclamation policy, > (-XX:SoftRefLRUPolicyMSPerMB). > > charlie > > On 05/16/2014 07:32 AM, Bengt Rutisson wrote: >> >> Hi Thomas, >> >> >> 16 maj 2014 kl. 14:01 skrev Thomas Viessmann >> >: >> >>> Hi Bengt, >>> >>> >>> Thanks for confirming. ParallelOld had stop pauses in the range of >>> 20 to 30 seconds. >>> CMS was a disaster due to extreme fragmentation and high promotion >>> rate even with >>> huge eden and survivors. >> >> Ok, so even with the long remark pauses G1 is performing better than >> the other GCs? >> >> >>> There are definitely lots of references. I can find out >>> details. >> >> Thanks, it would be interesting to get this data. >> >> Thanks, >> Bengt >> >>> >>> Thanks and Regards >>> >>> Thomas >>> >>> >>> >>> On 05/16/14 13:53, Bengt Rutisson wrote: >>>> >>>> Hi again Thomas, >>>> >>>> >>>> On 2014-05-16 13:34, Thomas Viessmann wrote: >>>>> Hi Bengt, >>>>> >>>>> Sure, the application has lots of objects and references. >>>>> Downsizing the application has been tried The heap size of 24 g is >>>>> already at minimum. A smaller heap gave OutOfmemory really quick. >>>>> My question was more whether the remark phases could be optimized >>>>> further. I assume this is not the case and we have reached the >>>>> limitations >>>>> of G1, right? >>>> >>>> >>>> How many reference objects does the application use? Can you run it >>>> with -XX:+PrintReferenceGC to see how many there are? >>>> >>>> If there are a lot of them I don't think there is much more that >>>> can be done for the remark phase. But if there are not that many I >>>> guess it means that the remark phase is inefficient. >>>> >>>> Have you tried any of the other GCs? How do they behave with this >>>> application? >>>> >>>> Thanks, >>>> Bengt >>>> >>>> >>>>> >>>>> Thanks and Regards >>>>> >>>>> Thomas >>>>> >>>>> >>>>> >>>>> On 05/16/14 13:18, Bengt Rutisson wrote: >>>>>> >>>>>> Hi Thomas, >>>>>> >>>>>> On 2014-05-16 13:10, Thomas Viessmann wrote: >>>>>>> Hi Bengt, >>>>>>> >>>>>>> Well, that's already done and it did improve things. >>>>>>> argv[21]: -XX:+ParallelRefProcEnabled >>>>>>> argv[22]: -XX:ParallelGCThreads=48 >>>>>> >>>>>> Sorry, I missed that. >>>>>> >>>>>>> before -XX:+ParallelRefProcEnabled was set the stop times were in the range of 20 to 60 seconds. >>>>>> >>>>>> OK. Glad it helped some. :) >>>>>> >>>>>>> The application is a Cacao by Oracle. So they cannot change it. >>>>>> >>>>>> Is there some way of reducing the amount of reference objects >>>>>> that Cacao uses? Does it have cache sizes or similar that can be >>>>>> tuned. With a JFR recording we might be able to figure out where >>>>>> the reference objects come from. >>>>>> >>>>>> Thanks, >>>>>> Bengt >>>>>> >>>>>>> Thanks and Regards >>>>>>> >>>>>>> Thomas >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 05/16/14 12:58, Bengt Rutisson wrote: >>>>>>>> >>>>>>>> Hi Thomas, >>>>>>>> >>>>>>>> It looks like the application is using a lot of Reference >>>>>>>> objects. The time spent in remark is dominated by reference >>>>>>>> processing. See the attached graph generated from the log file >>>>>>>> you sent. >>>>>>>> >>>>>>>> You can try to see if adding -XX:+ParallelRefProcEnabled >>>>>>>> improves the situation. >>>>>>>> >>>>>>>> If the customer is interested in updating their application >>>>>>>> they might want to see if they can reduce the number of >>>>>>>> java.lang.ref.Reference objects they use. >>>>>>>> >>>>>>>> Hths, >>>>>>>> Bengt >>>>>>>> >>>>>>>> >>>>>>>> On 2014-05-16 10:26, Thomas Viessmann wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> >>>>>>>>> I've been tuning a Java 7u51, Solaris 10, T4 system with 24G >>>>>>>>> heap. >>>>>>>>> My customer is not very happy with the remark pauses of up to >>>>>>>>> 2 seconds. >>>>>>>>> -XX:ParallelGCThreads=48 turned out to be the optimum. Here >>>>>>>>> is the log file >>>>>>>>> which contains the java args at the top: >>>>>>>>> >>>>>>>>> http://aubing.de.oracle.com/gclog/gc_log_03052014.log >>>>>>>>> >>>>>>>>> Any idea to drive the remark stop times further down? >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks and Regards >>>>>>>>> >>>>>>>>> Thomas >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> THOMAS VIESSMANN | Senior Principal Technical Support Engineer - >>>>>>> Java >>>>>>> Phone: +498914302496 | Mobile: +491743005467 >>>>>>> >>>>>>> Oracle Customer Technical Support - Java >>>>>>> >>>>>>> ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen >>>>>>> >>>>>>> ORACLE Deutschland B.V. & Co. KG >>>>>>> Hauptverwaltung: Riesstr. 25, D-80992 Muenchen >>>>>>> Registergericht: Amtsgericht Muenchen, HRA 95603 >>>>>>> Gesch?ftsf?hrere: Juergen Kunz >>>>>>> >>>>>>> Komplement?rin: ORACLE Deutschland Verwaltung B.V. >>>>>>> Hertogswetering 163/167, 3543 AS Utrecht, Niederlande >>>>>>> Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 >>>>>>> Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher >>>>>>> >>>>>>> ------------------------------------------------------------------------ >>>>>>> ------------------------------------------------------------------------ >>>>>>> Oracle >>>>>>> is committed to developing practices and products that help >>>>>>> protect the environment >>>>>> >>>>> >>>>> -- >>>>> >>>>> THOMAS VIESSMANN | Senior Principal Technical Support Engineer - Java >>>>> Phone: +498914302496 | Mobile: +491743005467 >>>>> >>>>> Oracle Customer Technical Support - Java >>>>> >>>>> ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen >>>>> >>>>> ORACLE Deutschland B.V. & Co. KG >>>>> Hauptverwaltung: Riesstr. 25, D-80992 Muenchen >>>>> Registergericht: Amtsgericht Muenchen, HRA 95603 >>>>> Gesch?ftsf?hrere: Juergen Kunz >>>>> >>>>> Komplement?rin: ORACLE Deutschland Verwaltung B.V. >>>>> Hertogswetering 163/167, 3543 AS Utrecht, Niederlande >>>>> Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 >>>>> Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher >>>>> >>>>> ------------------------------------------------------------------------ >>>>> ------------------------------------------------------------------------ >>>>> Oracle is >>>>> committed to developing practices and products that help protect >>>>> the environment >>>> >>> >>> -- >>> >>> THOMAS VIESSMANN | Senior Principal Technical Support Engineer - Java >>> Phone: +498914302496 | Mobile: +491743005467 >>> >>> Oracle Customer Technical Support - Java >>> >>> ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen >>> >>> ORACLE Deutschland B.V. & Co. KG >>> Hauptverwaltung: Riesstr. 25, D-80992 Muenchen >>> Registergericht: Amtsgericht Muenchen, HRA 95603 >>> Gesch?ftsf?hrere: Juergen Kunz >>> >>> Komplement?rin: ORACLE Deutschland Verwaltung B.V. >>> Hertogswetering 163/167, 3543 AS Utrecht, Niederlande >>> Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 >>> Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher >>> >>> ------------------------------------------------------------------------ >>> ------------------------------------------------------------------------ >>> >>> Oracle is committed to developing practices and products that help >>> protect the environment > -- Oracle THOMAS VIESSMANN | Senior Principal Technical Support Engineer - Java Phone: +498914302496 | Mobile: +491743005467 Oracle Customer Technical Support - Java ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen ORACLE Deutschland B.V. & Co. KG Hauptverwaltung: Riesstr. 25, D-80992 Muenchen Registergericht: Amtsgericht Muenchen, HRA 95603 Gesch?ftsf?hrere: Juergen Kunz Komplement?rin: ORACLE Deutschland Verwaltung B.V. Hertogswetering 163/167, 3543 AS Utrecht, Niederlande Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher ------------------------------------------------------------------------ ------------------------------------------------------------------------ Green Oracle Oracle is committed to developing practices and products that help protect the environment -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: green-for-email-sig_0.gif Type: image/gif Size: 356 bytes Desc: not available URL: From charlie.hunt at oracle.com Wed May 28 12:20:59 2014 From: charlie.hunt at oracle.com (charlie hunt) Date: Wed, 28 May 2014 07:20:59 -0500 Subject: G1:long remark pauses In-Reply-To: <5385905D.6000704@oracle.com> References: <5375CBBC.2030904@oracle.com> <5375EF5A.3050707@oracle.com> <5375F22E.90003@oracle.com> <5375F3F5.7030808@oracle.com> <5375F7AB.1070908@oracle.com> <5375FC23.5040004@oracle.com> <5375FE26.4080009@oracle.com> <53760723.3070002@oracle.com> <5385905D.6000704@oracle.com> Message-ID: <5385D4AB.1040907@oracle.com> Hi Thomas, About the only thing I can suggest is running the concurrent cycle more frequently so as to run the remark more frequently. Perhaps setting InitiatingHeapOccupancyPercent low enough that the concurrent cycle runs all the time. The remark is showing good parallelism since User=25.82 and real=2.39. I suppose a refactoring of the application to reduce usage of WeakReferences is another alternative. Btw, does ParallelOld GC or CMS GC show a similar behavior with a high number of WeakReferences as a GC issue? charlie On 05/28/2014 02:29 AM, Thomas Viessmann wrote: > Hi Charlie and all, > > you were pretty close. It's the weak references which stops the world, > not the SoftRefs: > > 248766.241: [GC remark 248766.246: [GC ref-proc248766.246: > [SoftReference, 184918 refs, 0.1139337 secs]248766.360: > *[WeakReference, 2104678 refs, 1.9516481 secs]248768.312:* > [FinalReference, 45 refs, 0.0033450 secs]248768.315: > [PhantomReference, 106 refs, 0.0029866 secs]248768.318: [JNI Weak > Reference, 0.0008403 secs], 2.0846210 secs], 2.3878287 secs] > [Times: user=25.82 sys=0.87, real=2.39 secs] > 248768.630: Total time for which application threads were stopped: > 2.4019674 seconds > > There are also a lot of SoftRefs around but those are not the culprit. > Varying -XX:SoftRefLRUPolicyMSPerMB between 100 and 10000 did not > change anything. > > Any idea how this could be addressed? > > Thanks and Regards > > Thomas > > > > > On 05/16/14 14:40, charlie hunt wrote: >> From the sound of what Thomas is describing, this might be one of >> those apps that's making heavy use of SoftReferences. Output from >> -XX:+PrintReferenceGC as Bengt suggested will show if that's the case. >> >> If we see a large number of SoftReferences being processed per GC, we >> may get further help with tuning the SoftReference reclamation >> policy, (-XX:SoftRefLRUPolicyMSPerMB). >> >> charlie >> >> On 05/16/2014 07:32 AM, Bengt Rutisson wrote: >>> >>> Hi Thomas, >>> >>> >>> 16 maj 2014 kl. 14:01 skrev Thomas Viessmann >>> >: >>> >>>> Hi Bengt, >>>> >>>> >>>> Thanks for confirming. ParallelOld had stop pauses in the range of >>>> 20 to 30 seconds. >>>> CMS was a disaster due to extreme fragmentation and high promotion >>>> rate even with >>>> huge eden and survivors. >>> >>> Ok, so even with the long remark pauses G1 is performing better than >>> the other GCs? >>> >>> >>>> There are definitely lots of references. I can find out >>>> details. >>> >>> Thanks, it would be interesting to get this data. >>> >>> Thanks, >>> Bengt >>> >>>> >>>> Thanks and Regards >>>> >>>> Thomas >>>> >>>> >>>> >>>> On 05/16/14 13:53, Bengt Rutisson wrote: >>>>> >>>>> Hi again Thomas, >>>>> >>>>> >>>>> On 2014-05-16 13:34, Thomas Viessmann wrote: >>>>>> Hi Bengt, >>>>>> >>>>>> Sure, the application has lots of objects and references. >>>>>> Downsizing the application has been tried The heap size of 24 g is >>>>>> already at minimum. A smaller heap gave OutOfmemory really quick. >>>>>> My question was more whether the remark phases could be optimized >>>>>> further. I assume this is not the case and we have reached the >>>>>> limitations >>>>>> of G1, right? >>>>> >>>>> >>>>> How many reference objects does the application use? Can you run >>>>> it with -XX:+PrintReferenceGC to see how many there are? >>>>> >>>>> If there are a lot of them I don't think there is much more that >>>>> can be done for the remark phase. But if there are not that many I >>>>> guess it means that the remark phase is inefficient. >>>>> >>>>> Have you tried any of the other GCs? How do they behave with this >>>>> application? >>>>> >>>>> Thanks, >>>>> Bengt >>>>> >>>>> >>>>>> >>>>>> Thanks and Regards >>>>>> >>>>>> Thomas >>>>>> >>>>>> >>>>>> >>>>>> On 05/16/14 13:18, Bengt Rutisson wrote: >>>>>>> >>>>>>> Hi Thomas, >>>>>>> >>>>>>> On 2014-05-16 13:10, Thomas Viessmann wrote: >>>>>>>> Hi Bengt, >>>>>>>> >>>>>>>> Well, that's already done and it did improve things. >>>>>>>> argv[21]: -XX:+ParallelRefProcEnabled >>>>>>>> argv[22]: -XX:ParallelGCThreads=48 >>>>>>> >>>>>>> Sorry, I missed that. >>>>>>> >>>>>>>> before -XX:+ParallelRefProcEnabled was set the stop times were in the range of 20 to 60 seconds. >>>>>>> >>>>>>> OK. Glad it helped some. :) >>>>>>> >>>>>>>> The application is a Cacao by Oracle. So they cannot change it. >>>>>>> >>>>>>> Is there some way of reducing the amount of reference objects >>>>>>> that Cacao uses? Does it have cache sizes or similar that can be >>>>>>> tuned. With a JFR recording we might be able to figure out where >>>>>>> the reference objects come from. >>>>>>> >>>>>>> Thanks, >>>>>>> Bengt >>>>>>> >>>>>>>> Thanks and Regards >>>>>>>> >>>>>>>> Thomas >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 05/16/14 12:58, Bengt Rutisson wrote: >>>>>>>>> >>>>>>>>> Hi Thomas, >>>>>>>>> >>>>>>>>> It looks like the application is using a lot of Reference >>>>>>>>> objects. The time spent in remark is dominated by reference >>>>>>>>> processing. See the attached graph generated from the log file >>>>>>>>> you sent. >>>>>>>>> >>>>>>>>> You can try to see if adding -XX:+ParallelRefProcEnabled >>>>>>>>> improves the situation. >>>>>>>>> >>>>>>>>> If the customer is interested in updating their application >>>>>>>>> they might want to see if they can reduce the number of >>>>>>>>> java.lang.ref.Reference objects they use. >>>>>>>>> >>>>>>>>> Hths, >>>>>>>>> Bengt >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2014-05-16 10:26, Thomas Viessmann wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> I've been tuning a Java 7u51, Solaris 10, T4 system with 24G >>>>>>>>>> heap. >>>>>>>>>> My customer is not very happy with the remark pauses of up >>>>>>>>>> to 2 seconds. >>>>>>>>>> -XX:ParallelGCThreads=48 turned out to be the optimum. Here >>>>>>>>>> is the log file >>>>>>>>>> which contains the java args at the top: >>>>>>>>>> >>>>>>>>>> http://aubing.de.oracle.com/gclog/gc_log_03052014.log >>>>>>>>>> >>>>>>>>>> Any idea to drive the remark stop times further down? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks and Regards >>>>>>>>>> >>>>>>>>>> Thomas >>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> THOMAS VIESSMANN | Senior Principal Technical Support Engineer >>>>>>>> - Java >>>>>>>> Phone: +498914302496 | Mobile: +491743005467 >>>>>>>> >>>>>>>> Oracle Customer Technical Support - Java >>>>>>>> >>>>>>>> ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen >>>>>>>> >>>>>>>> ORACLE Deutschland B.V. & Co. KG >>>>>>>> Hauptverwaltung: Riesstr. 25, D-80992 Muenchen >>>>>>>> Registergericht: Amtsgericht Muenchen, HRA 95603 >>>>>>>> Gesch?ftsf?hrere: Juergen Kunz >>>>>>>> >>>>>>>> Komplement?rin: ORACLE Deutschland Verwaltung B.V. >>>>>>>> Hertogswetering 163/167, 3543 AS Utrecht, Niederlande >>>>>>>> Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 >>>>>>>> Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher >>>>>>>> >>>>>>>> ------------------------------------------------------------------------ >>>>>>>> ------------------------------------------------------------------------ >>>>>>>> Oracle >>>>>>>> is committed to developing practices and products that help >>>>>>>> protect the environment >>>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> THOMAS VIESSMANN | Senior Principal Technical Support Engineer - Java >>>>>> Phone: +498914302496 | Mobile: +491743005467 >>>>>> >>>>>> Oracle Customer Technical Support - Java >>>>>> >>>>>> ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen >>>>>> >>>>>> ORACLE Deutschland B.V. & Co. KG >>>>>> Hauptverwaltung: Riesstr. 25, D-80992 Muenchen >>>>>> Registergericht: Amtsgericht Muenchen, HRA 95603 >>>>>> Gesch?ftsf?hrere: Juergen Kunz >>>>>> >>>>>> Komplement?rin: ORACLE Deutschland Verwaltung B.V. >>>>>> Hertogswetering 163/167, 3543 AS Utrecht, Niederlande >>>>>> Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 >>>>>> Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher >>>>>> >>>>>> ------------------------------------------------------------------------ >>>>>> ------------------------------------------------------------------------ >>>>>> Oracle >>>>>> is committed to developing practices and products that help >>>>>> protect the environment >>>>> >>>> >>>> -- >>>> >>>> THOMAS VIESSMANN | Senior Principal Technical Support Engineer - Java >>>> Phone: +498914302496 | Mobile: +491743005467 >>>> >>>> Oracle Customer Technical Support - Java >>>> >>>> ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen >>>> >>>> ORACLE Deutschland B.V. & Co. KG >>>> Hauptverwaltung: Riesstr. 25, D-80992 Muenchen >>>> Registergericht: Amtsgericht Muenchen, HRA 95603 >>>> Gesch?ftsf?hrere: Juergen Kunz >>>> >>>> Komplement?rin: ORACLE Deutschland Verwaltung B.V. >>>> Hertogswetering 163/167, 3543 AS Utrecht, Niederlande >>>> Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 >>>> Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher >>>> >>>> ------------------------------------------------------------------------ >>>> ------------------------------------------------------------------------ >>>> >>>> Oracle is committed to developing practices and products that help >>>> protect the environment >> > > -- > Oracle > THOMAS VIESSMANN | Senior Principal Technical Support Engineer - Java > Phone: +498914302496 | Mobile: +491743005467 > > Oracle Customer Technical Support - Java > > ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen > > ORACLE Deutschland B.V. & Co. KG > Hauptverwaltung: Riesstr. 25, D-80992 Muenchen > Registergericht: Amtsgericht Muenchen, HRA 95603 > Gesch?ftsf?hrere: Juergen Kunz > > Komplement?rin: ORACLE Deutschland Verwaltung B.V. > Hertogswetering 163/167, 3543 AS Utrecht, Niederlande > Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 > Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher > > ------------------------------------------------------------------------ > ------------------------------------------------------------------------ > Green Oracle Oracle is committed to > developing practices and products that help protect the environment -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/gif Size: 658 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/gif Size: 356 bytes Desc: not available URL: From mikael.gerdin at oracle.com Wed May 28 12:43:12 2014 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Wed, 28 May 2014 14:43:12 +0200 Subject: G1:long remark pauses In-Reply-To: <5385D4AB.1040907@oracle.com> References: <5375CBBC.2030904@oracle.com> <5385905D.6000704@oracle.com> <5385D4AB.1040907@oracle.com> Message-ID: <4173678.fBVK9KDdpU@mgerdin03> On Wednesday 28 May 2014 07.20.59 charlie hunt wrote: > Hi Thomas, > > About the only thing I can suggest is running the concurrent cycle more > frequently so as to run the remark more frequently. Perhaps setting > InitiatingHeapOccupancyPercent low enough that the concurrent cycle runs > all the time. If the weakly reachable objects are reasonably short-lived an increased survivor size and tenuring threshold could allow more weak references to be dealt with by incremental young collections instead. /Mikael > > The remark is showing good parallelism since User=25.82 and real=2.39. > > I suppose a refactoring of the application to reduce usage of > WeakReferences is another alternative. > > Btw, does ParallelOld GC or CMS GC show a similar behavior with a high > number of WeakReferences as a GC issue? > > charlie > > On 05/28/2014 02:29 AM, Thomas Viessmann wrote: > > Hi Charlie and all, > > > > you were pretty close. It's the weak references which stops the world, > > not the SoftRefs: > > > > 248766.241: [GC remark 248766.246: [GC ref-proc248766.246: > > [SoftReference, 184918 refs, 0.1139337 secs]248766.360: > > *[WeakReference, 2104678 refs, 1.9516481 secs]248768.312:* > > [FinalReference, 45 refs, 0.0033450 secs]248768.315: > > [PhantomReference, 106 refs, 0.0029866 secs]248768.318: [JNI Weak > > Reference, 0.0008403 secs], 2.0846210 secs], 2.3878287 secs] > > > > [Times: user=25.82 sys=0.87, real=2.39 secs] > > > > 248768.630: Total time for which application threads were stopped: > > 2.4019674 seconds > > > > There are also a lot of SoftRefs around but those are not the culprit. > > Varying -XX:SoftRefLRUPolicyMSPerMB between 100 and 10000 did not > > change anything. > > > > Any idea how this could be addressed? > > > > Thanks and Regards > > > > Thomas > > > > On 05/16/14 14:40, charlie hunt wrote: > >> From the sound of what Thomas is describing, this might be one of > >> those apps that's making heavy use of SoftReferences. Output from > >> -XX:+PrintReferenceGC as Bengt suggested will show if that's the case. > >> > >> If we see a large number of SoftReferences being processed per GC, we > >> may get further help with tuning the SoftReference reclamation > >> policy, (-XX:SoftRefLRUPolicyMSPerMB). > >> > >> charlie > >> > >> On 05/16/2014 07:32 AM, Bengt Rutisson wrote: > >>> Hi Thomas, > >>> > >>> > >>> 16 maj 2014 kl. 14:01 skrev Thomas Viessmann > >>> > >>> >: > >>>> Hi Bengt, > >>>> > >>>> > >>>> Thanks for confirming. ParallelOld had stop pauses in the range of > >>>> 20 to 30 seconds. > >>>> CMS was a disaster due to extreme fragmentation and high promotion > >>>> rate even with > >>>> huge eden and survivors. > >>> > >>> Ok, so even with the long remark pauses G1 is performing better than > >>> the other GCs? > >>> > >>>> There are definitely lots of references. I can find out > >>>> details. > >>> > >>> Thanks, it would be interesting to get this data. > >>> > >>> Thanks, > >>> Bengt > >>> > >>>> Thanks and Regards > >>>> > >>>> Thomas > >>>> > >>>> On 05/16/14 13:53, Bengt Rutisson wrote: > >>>>> Hi again Thomas, > >>>>> > >>>>> On 2014-05-16 13:34, Thomas Viessmann wrote: > >>>>>> Hi Bengt, > >>>>>> > >>>>>> Sure, the application has lots of objects and references. > >>>>>> Downsizing the application has been tried The heap size of 24 g is > >>>>>> already at minimum. A smaller heap gave OutOfmemory really quick. > >>>>>> My question was more whether the remark phases could be optimized > >>>>>> further. I assume this is not the case and we have reached the > >>>>>> limitations > >>>>>> of G1, right? > >>>>> > >>>>> How many reference objects does the application use? Can you run > >>>>> it with -XX:+PrintReferenceGC to see how many there are? > >>>>> > >>>>> If there are a lot of them I don't think there is much more that > >>>>> can be done for the remark phase. But if there are not that many I > >>>>> guess it means that the remark phase is inefficient. > >>>>> > >>>>> Have you tried any of the other GCs? How do they behave with this > >>>>> application? > >>>>> > >>>>> Thanks, > >>>>> Bengt > >>>>> > >>>>>> Thanks and Regards > >>>>>> > >>>>>> Thomas > >>>>>> > >>>>>> On 05/16/14 13:18, Bengt Rutisson wrote: > >>>>>>> Hi Thomas, > >>>>>>> > >>>>>>> On 2014-05-16 13:10, Thomas Viessmann wrote: > >>>>>>>> Hi Bengt, > >>>>>>>> > >>>>>>>> Well, that's already done and it did improve things. > >>>>>>>> argv[21]: -XX:+ParallelRefProcEnabled > >>>>>>>> argv[22]: -XX:ParallelGCThreads=48 > >>>>>>> > >>>>>>> Sorry, I missed that. > >>>>>>> > >>>>>>>> before -XX:+ParallelRefProcEnabled was set the stop times were in > >>>>>>>> the range of 20 to 60 seconds.>>>>>>> > >>>>>>> OK. Glad it helped some. :) > >>>>>>> > >>>>>>>> The application is a Cacao by Oracle. So they cannot change it. > >>>>>>> > >>>>>>> Is there some way of reducing the amount of reference objects > >>>>>>> that Cacao uses? Does it have cache sizes or similar that can be > >>>>>>> tuned. With a JFR recording we might be able to figure out where > >>>>>>> the reference objects come from. > >>>>>>> > >>>>>>> Thanks, > >>>>>>> Bengt > >>>>>>> > >>>>>>>> Thanks and Regards > >>>>>>>> > >>>>>>>> Thomas > >>>>>>>> > >>>>>>>> On 05/16/14 12:58, Bengt Rutisson wrote: > >>>>>>>>> Hi Thomas, > >>>>>>>>> > >>>>>>>>> It looks like the application is using a lot of Reference > >>>>>>>>> objects. The time spent in remark is dominated by reference > >>>>>>>>> processing. See the attached graph generated from the log file > >>>>>>>>> you sent. > >>>>>>>>> > >>>>>>>>> You can try to see if adding -XX:+ParallelRefProcEnabled > >>>>>>>>> improves the situation. > >>>>>>>>> > >>>>>>>>> If the customer is interested in updating their application > >>>>>>>>> they might want to see if they can reduce the number of > >>>>>>>>> java.lang.ref.Reference objects they use. > >>>>>>>>> > >>>>>>>>> Hths, > >>>>>>>>> Bengt > >>>>>>>>> > >>>>>>>>> On 2014-05-16 10:26, Thomas Viessmann wrote: > >>>>>>>>>> Hi, > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> I've been tuning a Java 7u51, Solaris 10, T4 system with 24G > >>>>>>>>>> heap. > >>>>>>>>>> My customer is not very happy with the remark pauses of up > >>>>>>>>>> to 2 seconds. > >>>>>>>>>> > >>>>>>>>>> -XX:ParallelGCThreads=48 turned out to be the optimum. Here > >>>>>>>>>> > >>>>>>>>>> is the log file > >>>>>>>>>> which contains the java args at the top: > >>>>>>>>>> > >>>>>>>>>> http://aubing.de.oracle.com/gclog/gc_log_03052014.log > >>>>>>>>>> > >>>>>>>>>> Any idea to drive the remark stop times further down? > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> Thanks and Regards > >>>>>>>>>> > >>>>>>>>>> Thomas > >>>>>>>> > >>>>>>>> ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen > >>>>>>>> > >>>>>>>> ORACLE Deutschland B.V. & Co. KG > >>>>>>>> Hauptverwaltung: Riesstr. 25, D-80992 Muenchen > >>>>>>>> Registergericht: Amtsgericht Muenchen, HRA 95603 > >>>>>>>> Gesch?ftsf?hrere: Juergen Kunz > >>>>>>>> > >>>>>>>> Komplement?rin: ORACLE Deutschland Verwaltung B.V. > >>>>>>>> Hertogswetering 163/167, 3543 AS Utrecht, Niederlande > >>>>>>>> Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 > >>>>>>>> Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher > >>>>>>>> > >>>>>>>> ------------------------------------------------------------------- > >>>>>>>> ----- > >>>>>>>> ------------------------------------------------------------------- > >>>>>>>> ----- > >>>>>>>> Oracle > >>>>>>>> is committed to developing practices and products that help > >>>>>>>> protect the environment > >>>>>> > >>>>>> ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen > >>>>>> > >>>>>> ORACLE Deutschland B.V. & Co. KG > >>>>>> Hauptverwaltung: Riesstr. 25, D-80992 Muenchen > >>>>>> Registergericht: Amtsgericht Muenchen, HRA 95603 > >>>>>> Gesch?ftsf?hrere: Juergen Kunz > >>>>>> > >>>>>> Komplement?rin: ORACLE Deutschland Verwaltung B.V. > >>>>>> Hertogswetering 163/167, 3543 AS Utrecht, Niederlande > >>>>>> Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 > >>>>>> Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher > >>>>>> > >>>>>> --------------------------------------------------------------------- > >>>>>> --- > >>>>>> --------------------------------------------------------------------- > >>>>>> --- > >>>>>> Oracle > >>>>>> is committed to developing practices and products that help > >>>>>> protect the environment > >>>> > >>>> ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen > >>>> > >>>> ORACLE Deutschland B.V. & Co. KG > >>>> Hauptverwaltung: Riesstr. 25, D-80992 Muenchen > >>>> Registergericht: Amtsgericht Muenchen, HRA 95603 > >>>> Gesch?ftsf?hrere: Juergen Kunz > >>>> > >>>> Komplement?rin: ORACLE Deutschland Verwaltung B.V. > >>>> Hertogswetering 163/167, 3543 AS Utrecht, Niederlande > >>>> Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 > >>>> Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher > >>>> > >>>> ----------------------------------------------------------------------- > >>>> - > >>>> ----------------------------------------------------------------------- > >>>> - > >>>> > >>>> Oracle is committed to developing practices and products that help > >>>> protect the environment > > > > ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen > > > > ORACLE Deutschland B.V. & Co. KG > > Hauptverwaltung: Riesstr. 25, D-80992 Muenchen > > Registergericht: Amtsgericht Muenchen, HRA 95603 > > Gesch?ftsf?hrere: Juergen Kunz > > > > Komplement?rin: ORACLE Deutschland Verwaltung B.V. > > Hertogswetering 163/167, 3543 AS Utrecht, Niederlande > > Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 > > Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher > > > > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------ > > Green Oracle Oracle is committed to > > developing practices and products that help protect the environment From per.liden at oracle.com Wed May 28 13:30:34 2014 From: per.liden at oracle.com (Per Liden) Date: Wed, 28 May 2014 15:30:34 +0200 Subject: RFR(s): 8042310: TestStringDeduplicationMemoryUsage test failing Message-ID: <5385E4FA.7080304@oracle.com> Could I please have this patch reviewed. Summary: This test is unstable and is failing intermittently in nightly testing. The test relies on getting the heap usage via an MXBean, but since the test doesn't have complete control over what ends up on the heap the heap usage is sometimes slightly higher than the test expects. There are other (stable) tests which already verifies than string deduplication actually happens when it's supposed to, so this test doesn't really add much value. This patch removes the unstable test completely. Bug: https://bugs.openjdk.java.net/browse/JDK-8042310 Webrev: http://cr.openjdk.java.net/~pliden/8042310/webrev.0/ Thanks! /Per From per.liden at oracle.com Wed May 28 14:11:19 2014 From: per.liden at oracle.com (Per Liden) Date: Wed, 28 May 2014 16:11:19 +0200 Subject: RFR(s): 8044132: Quarantine unstable/broken GC tests Message-ID: <5385EE87.2030404@oracle.com> Summary: This patch adds the @ignore tag on four GC tests. These tests are unstable or broken. Until these tests have been fixed we quarantine them to avoid noise in the test results. Bug: https://bugs.openjdk.java.net/browse/JDK-8044132 Webrev: http://cr.openjdk.java.net/~pliden/8044132/webrev.0/ Thanks! /Per From charlie.hunt at oracle.com Wed May 28 14:13:49 2014 From: charlie.hunt at oracle.com (charlie hunt) Date: Wed, 28 May 2014 09:13:49 -0500 Subject: G1:long remark pauses In-Reply-To: <4173678.fBVK9KDdpU@mgerdin03> References: <5375CBBC.2030904@oracle.com> <5385905D.6000704@oracle.com> <5385D4AB.1040907@oracle.com> <4173678.fBVK9KDdpU@mgerdin03> Message-ID: <5385EF1D.4030506@oracle.com> On 05/28/2014 07:43 AM, Mikael Gerdin wrote: > On Wednesday 28 May 2014 07.20.59 charlie hunt wrote: >> Hi Thomas, >> >> About the only thing I can suggest is running the concurrent cycle more >> frequently so as to run the remark more frequently. Perhaps setting >> InitiatingHeapOccupancyPercent low enough that the concurrent cycle runs >> all the time. > If the weakly reachable objects are reasonably short-lived an increased > survivor size and tenuring threshold could allow more weak references to be > dealt with by incremental young collections instead. Mikael, Doesn't the suggestion of tuning survivor size go against the general advice of tuning G1, (and more generally tuning young generation sizing with G1), since you're essentially overriding the G1 the young GC pause time heuristics? Fwiw, max tenuring threshold defaults to 15 with G1. Perhaps an alternative to explicit sizing of survivor spaces (or young gen) would be increasing MaxPauseTimeMillis which should generally increase the size of eden and survivor. charlie > > /Mikael > >> The remark is showing good parallelism since User=25.82 and real=2.39. >> >> I suppose a refactoring of the application to reduce usage of >> WeakReferences is another alternative. >> >> Btw, does ParallelOld GC or CMS GC show a similar behavior with a high >> number of WeakReferences as a GC issue? >> >> charlie >> >> On 05/28/2014 02:29 AM, Thomas Viessmann wrote: >>> Hi Charlie and all, >>> >>> you were pretty close. It's the weak references which stops the world, >>> not the SoftRefs: >>> >>> 248766.241: [GC remark 248766.246: [GC ref-proc248766.246: >>> [SoftReference, 184918 refs, 0.1139337 secs]248766.360: >>> *[WeakReference, 2104678 refs, 1.9516481 secs]248768.312:* >>> [FinalReference, 45 refs, 0.0033450 secs]248768.315: >>> [PhantomReference, 106 refs, 0.0029866 secs]248768.318: [JNI Weak >>> Reference, 0.0008403 secs], 2.0846210 secs], 2.3878287 secs] >>> >>> [Times: user=25.82 sys=0.87, real=2.39 secs] >>> >>> 248768.630: Total time for which application threads were stopped: >>> 2.4019674 seconds >>> >>> There are also a lot of SoftRefs around but those are not the culprit. >>> Varying -XX:SoftRefLRUPolicyMSPerMB between 100 and 10000 did not >>> change anything. >>> >>> Any idea how this could be addressed? >>> >>> Thanks and Regards >>> >>> Thomas >>> >>> On 05/16/14 14:40, charlie hunt wrote: >>>> From the sound of what Thomas is describing, this might be one of >>>> those apps that's making heavy use of SoftReferences. Output from >>>> -XX:+PrintReferenceGC as Bengt suggested will show if that's the case. >>>> >>>> If we see a large number of SoftReferences being processed per GC, we >>>> may get further help with tuning the SoftReference reclamation >>>> policy, (-XX:SoftRefLRUPolicyMSPerMB). >>>> >>>> charlie >>>> >>>> On 05/16/2014 07:32 AM, Bengt Rutisson wrote: >>>>> Hi Thomas, >>>>> >>>>> >>>>> 16 maj 2014 kl. 14:01 skrev Thomas Viessmann >>>>> >>>>> >: >>>>>> Hi Bengt, >>>>>> >>>>>> >>>>>> Thanks for confirming. ParallelOld had stop pauses in the range of >>>>>> 20 to 30 seconds. >>>>>> CMS was a disaster due to extreme fragmentation and high promotion >>>>>> rate even with >>>>>> huge eden and survivors. >>>>> Ok, so even with the long remark pauses G1 is performing better than >>>>> the other GCs? >>>>> >>>>>> There are definitely lots of references. I can find out >>>>>> details. >>>>> Thanks, it would be interesting to get this data. >>>>> >>>>> Thanks, >>>>> Bengt >>>>> >>>>>> Thanks and Regards >>>>>> >>>>>> Thomas >>>>>> >>>>>> On 05/16/14 13:53, Bengt Rutisson wrote: >>>>>>> Hi again Thomas, >>>>>>> >>>>>>> On 2014-05-16 13:34, Thomas Viessmann wrote: >>>>>>>> Hi Bengt, >>>>>>>> >>>>>>>> Sure, the application has lots of objects and references. >>>>>>>> Downsizing the application has been tried The heap size of 24 g is >>>>>>>> already at minimum. A smaller heap gave OutOfmemory really quick. >>>>>>>> My question was more whether the remark phases could be optimized >>>>>>>> further. I assume this is not the case and we have reached the >>>>>>>> limitations >>>>>>>> of G1, right? >>>>>>> How many reference objects does the application use? Can you run >>>>>>> it with -XX:+PrintReferenceGC to see how many there are? >>>>>>> >>>>>>> If there are a lot of them I don't think there is much more that >>>>>>> can be done for the remark phase. But if there are not that many I >>>>>>> guess it means that the remark phase is inefficient. >>>>>>> >>>>>>> Have you tried any of the other GCs? How do they behave with this >>>>>>> application? >>>>>>> >>>>>>> Thanks, >>>>>>> Bengt >>>>>>> >>>>>>>> Thanks and Regards >>>>>>>> >>>>>>>> Thomas >>>>>>>> >>>>>>>> On 05/16/14 13:18, Bengt Rutisson wrote: >>>>>>>>> Hi Thomas, >>>>>>>>> >>>>>>>>> On 2014-05-16 13:10, Thomas Viessmann wrote: >>>>>>>>>> Hi Bengt, >>>>>>>>>> >>>>>>>>>> Well, that's already done and it did improve things. >>>>>>>>>> argv[21]: -XX:+ParallelRefProcEnabled >>>>>>>>>> argv[22]: -XX:ParallelGCThreads=48 >>>>>>>>> Sorry, I missed that. >>>>>>>>> >>>>>>>>>> before -XX:+ParallelRefProcEnabled was set the stop times were in >>>>>>>>>> the range of 20 to 60 seconds.>>>>>>> >>>>>>>>> OK. Glad it helped some. :) >>>>>>>>> >>>>>>>>>> The application is a Cacao by Oracle. So they cannot change it. >>>>>>>>> Is there some way of reducing the amount of reference objects >>>>>>>>> that Cacao uses? Does it have cache sizes or similar that can be >>>>>>>>> tuned. With a JFR recording we might be able to figure out where >>>>>>>>> the reference objects come from. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Bengt >>>>>>>>> >>>>>>>>>> Thanks and Regards >>>>>>>>>> >>>>>>>>>> Thomas >>>>>>>>>> >>>>>>>>>> On 05/16/14 12:58, Bengt Rutisson wrote: >>>>>>>>>>> Hi Thomas, >>>>>>>>>>> >>>>>>>>>>> It looks like the application is using a lot of Reference >>>>>>>>>>> objects. The time spent in remark is dominated by reference >>>>>>>>>>> processing. See the attached graph generated from the log file >>>>>>>>>>> you sent. >>>>>>>>>>> >>>>>>>>>>> You can try to see if adding -XX:+ParallelRefProcEnabled >>>>>>>>>>> improves the situation. >>>>>>>>>>> >>>>>>>>>>> If the customer is interested in updating their application >>>>>>>>>>> they might want to see if they can reduce the number of >>>>>>>>>>> java.lang.ref.Reference objects they use. >>>>>>>>>>> >>>>>>>>>>> Hths, >>>>>>>>>>> Bengt >>>>>>>>>>> >>>>>>>>>>> On 2014-05-16 10:26, Thomas Viessmann wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I've been tuning a Java 7u51, Solaris 10, T4 system with 24G >>>>>>>>>>>> heap. >>>>>>>>>>>> My customer is not very happy with the remark pauses of up >>>>>>>>>>>> to 2 seconds. >>>>>>>>>>>> >>>>>>>>>>>> -XX:ParallelGCThreads=48 turned out to be the optimum. Here >>>>>>>>>>>> >>>>>>>>>>>> is the log file >>>>>>>>>>>> which contains the java args at the top: >>>>>>>>>>>> >>>>>>>>>>>> http://aubing.de.oracle.com/gclog/gc_log_03052014.log >>>>>>>>>>>> >>>>>>>>>>>> Any idea to drive the remark stop times further down? >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks and Regards >>>>>>>>>>>> >>>>>>>>>>>> Thomas >>>>>>>>>> ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen >>>>>>>>>> >>>>>>>>>> ORACLE Deutschland B.V. & Co. KG >>>>>>>>>> Hauptverwaltung: Riesstr. 25, D-80992 Muenchen >>>>>>>>>> Registergericht: Amtsgericht Muenchen, HRA 95603 >>>>>>>>>> Gesch?ftsf?hrere: Juergen Kunz >>>>>>>>>> >>>>>>>>>> Komplement?rin: ORACLE Deutschland Verwaltung B.V. >>>>>>>>>> Hertogswetering 163/167, 3543 AS Utrecht, Niederlande >>>>>>>>>> Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 >>>>>>>>>> Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher >>>>>>>>>> >>>>>>>>>> ------------------------------------------------------------------- >>>>>>>>>> ----- >>>>>>>>>> ------------------------------------------------------------------- >>>>>>>>>> ----- >>>>>>>>>> Oracle >>>>>>>>>> is committed to developing practices and products that help >>>>>>>>>> protect the environment >>>>>>>> ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen >>>>>>>> >>>>>>>> ORACLE Deutschland B.V. & Co. KG >>>>>>>> Hauptverwaltung: Riesstr. 25, D-80992 Muenchen >>>>>>>> Registergericht: Amtsgericht Muenchen, HRA 95603 >>>>>>>> Gesch?ftsf?hrere: Juergen Kunz >>>>>>>> >>>>>>>> Komplement?rin: ORACLE Deutschland Verwaltung B.V. >>>>>>>> Hertogswetering 163/167, 3543 AS Utrecht, Niederlande >>>>>>>> Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 >>>>>>>> Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher >>>>>>>> >>>>>>>> --------------------------------------------------------------------- >>>>>>>> --- >>>>>>>> --------------------------------------------------------------------- >>>>>>>> --- >>>>>>>> Oracle >>>>>>>> is committed to developing practices and products that help >>>>>>>> protect the environment >>>>>> ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen >>>>>> >>>>>> ORACLE Deutschland B.V. & Co. KG >>>>>> Hauptverwaltung: Riesstr. 25, D-80992 Muenchen >>>>>> Registergericht: Amtsgericht Muenchen, HRA 95603 >>>>>> Gesch?ftsf?hrere: Juergen Kunz >>>>>> >>>>>> Komplement?rin: ORACLE Deutschland Verwaltung B.V. >>>>>> Hertogswetering 163/167, 3543 AS Utrecht, Niederlande >>>>>> Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 >>>>>> Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher >>>>>> >>>>>> ----------------------------------------------------------------------- >>>>>> - >>>>>> ----------------------------------------------------------------------- >>>>>> - >>>>>> >>>>>> Oracle is committed to developing practices and products that help >>>>>> protect the environment >>> ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen >>> >>> ORACLE Deutschland B.V. & Co. KG >>> Hauptverwaltung: Riesstr. 25, D-80992 Muenchen >>> Registergericht: Amtsgericht Muenchen, HRA 95603 >>> Gesch?ftsf?hrere: Juergen Kunz >>> >>> Komplement?rin: ORACLE Deutschland Verwaltung B.V. >>> Hertogswetering 163/167, 3543 AS Utrecht, Niederlande >>> Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 >>> Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher >>> >>> ------------------------------------------------------------------------ >>> ------------------------------------------------------------------------ >>> Green Oracle Oracle is committed to >>> developing practices and products that help protect the environment From jesper.wilhelmsson at oracle.com Wed May 28 14:19:09 2014 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Wed, 28 May 2014 16:19:09 +0200 Subject: RFR(s): 8042310: TestStringDeduplicationMemoryUsage test failing In-Reply-To: <5385E4FA.7080304@oracle.com> References: <5385E4FA.7080304@oracle.com> Message-ID: <5385F05D.8060802@oracle.com> Looks good. /Jesper Per Liden skrev 28/5/14 15:30: > Could I please have this patch reviewed. > > Summary: This test is unstable and is failing intermittently in nightly testing. > The test relies on getting the heap usage via an MXBean, but since the test > doesn't have complete control over what ends up on the heap the heap usage is > sometimes slightly higher than the test expects. There are other (stable) tests > which already verifies than string deduplication actually happens when it's > supposed to, so this test doesn't really add much value. This patch removes the > unstable test completely. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8042310 > Webrev: http://cr.openjdk.java.net/~pliden/8042310/webrev.0/ > > Thanks! > /Per From jesper.wilhelmsson at oracle.com Wed May 28 14:22:50 2014 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Wed, 28 May 2014 16:22:50 +0200 Subject: RFR(s): 8044132: Quarantine unstable/broken GC tests In-Reply-To: <5385EE87.2030404@oracle.com> References: <5385EE87.2030404@oracle.com> Message-ID: <5385F13A.3030803@oracle.com> Looks good. /Jesper Per Liden skrev 28/5/14 16:11: > Summary: This patch adds the @ignore tag on four GC tests. These tests are > unstable or broken. Until these tests have been fixed we quarantine them to > avoid noise in the test results. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8044132 > Webrev: http://cr.openjdk.java.net/~pliden/8044132/webrev.0/ > > Thanks! > /Per From mikael.gerdin at oracle.com Wed May 28 14:35:57 2014 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Wed, 28 May 2014 16:35:57 +0200 Subject: G1:long remark pauses In-Reply-To: <5385EF1D.4030506@oracle.com> References: <5375CBBC.2030904@oracle.com> <4173678.fBVK9KDdpU@mgerdin03> <5385EF1D.4030506@oracle.com> Message-ID: <5434546.VSIa7BJNvm@mgerdin03> On Wednesday 28 May 2014 09.13.49 charlie hunt wrote: > On 05/28/2014 07:43 AM, Mikael Gerdin wrote: > > On Wednesday 28 May 2014 07.20.59 charlie hunt wrote: > >> Hi Thomas, > >> > >> About the only thing I can suggest is running the concurrent cycle more > >> frequently so as to run the remark more frequently. Perhaps setting > >> InitiatingHeapOccupancyPercent low enough that the concurrent cycle runs > >> all the time. > > > > If the weakly reachable objects are reasonably short-lived an increased > > survivor size and tenuring threshold could allow more weak references to > > be > > dealt with by incremental young collections instead. > > Mikael, > > Doesn't the suggestion of tuning survivor size go against the general > advice of tuning G1, (and more generally tuning young generation sizing > with G1), since you're essentially overriding the G1 the young GC pause > time heuristics? It does, but if the reference processing during remark is a large problem then there are only two possible approaches for reducing the time: 1) Allocate less WeakReference objects. 2) Allow WeakReference objects to die before ConcurrentMark has to deal with them. > > Fwiw, max tenuring threshold defaults to 15 with G1. > > Perhaps an alternative to explicit sizing of survivor spaces (or young > gen) would be increasing MaxPauseTimeMillis which should generally > increase the size of eden and survivor. That is probably a reasonable approach as well. /Mikael > > charlie > > > /Mikael > > > >> The remark is showing good parallelism since User=25.82 and real=2.39. > >> > >> I suppose a refactoring of the application to reduce usage of > >> WeakReferences is another alternative. > >> > >> Btw, does ParallelOld GC or CMS GC show a similar behavior with a high > >> number of WeakReferences as a GC issue? > >> > >> charlie > >> > >> On 05/28/2014 02:29 AM, Thomas Viessmann wrote: > >>> Hi Charlie and all, > >>> > >>> you were pretty close. It's the weak references which stops the world, > >>> not the SoftRefs: > >>> > >>> 248766.241: [GC remark 248766.246: [GC ref-proc248766.246: > >>> [SoftReference, 184918 refs, 0.1139337 secs]248766.360: > >>> *[WeakReference, 2104678 refs, 1.9516481 secs]248768.312:* > >>> [FinalReference, 45 refs, 0.0033450 secs]248768.315: > >>> [PhantomReference, 106 refs, 0.0029866 secs]248768.318: [JNI Weak > >>> Reference, 0.0008403 secs], 2.0846210 secs], 2.3878287 secs] > >>> > >>> [Times: user=25.82 sys=0.87, real=2.39 secs] > >>> > >>> 248768.630: Total time for which application threads were stopped: > >>> 2.4019674 seconds > >>> > >>> There are also a lot of SoftRefs around but those are not the culprit. > >>> Varying -XX:SoftRefLRUPolicyMSPerMB between 100 and 10000 did not > >>> change anything. > >>> > >>> Any idea how this could be addressed? > >>> > >>> Thanks and Regards > >>> > >>> Thomas > >>> > >>> On 05/16/14 14:40, charlie hunt wrote: > >>>> From the sound of what Thomas is describing, this might be one of > >>>> > >>>> those apps that's making heavy use of SoftReferences. Output from > >>>> -XX:+PrintReferenceGC as Bengt suggested will show if that's the case. > >>>> > >>>> If we see a large number of SoftReferences being processed per GC, we > >>>> may get further help with tuning the SoftReference reclamation > >>>> policy, (-XX:SoftRefLRUPolicyMSPerMB). > >>>> > >>>> charlie > >>>> > >>>> On 05/16/2014 07:32 AM, Bengt Rutisson wrote: > >>>>> Hi Thomas, > >>>>> > >>>>> > >>>>> 16 maj 2014 kl. 14:01 skrev Thomas Viessmann > >>>>> > >>>>> >: > >>>>>> Hi Bengt, > >>>>>> > >>>>>> > >>>>>> Thanks for confirming. ParallelOld had stop pauses in the range of > >>>>>> 20 to 30 seconds. > >>>>>> CMS was a disaster due to extreme fragmentation and high promotion > >>>>>> rate even with > >>>>>> huge eden and survivors. > >>>>> > >>>>> Ok, so even with the long remark pauses G1 is performing better than > >>>>> the other GCs? > >>>>> > >>>>>> There are definitely lots of references. I can find out > >>>>>> details. > >>>>> > >>>>> Thanks, it would be interesting to get this data. > >>>>> > >>>>> Thanks, > >>>>> Bengt > >>>>> > >>>>>> Thanks and Regards > >>>>>> > >>>>>> Thomas > >>>>>> > >>>>>> On 05/16/14 13:53, Bengt Rutisson wrote: > >>>>>>> Hi again Thomas, > >>>>>>> > >>>>>>> On 2014-05-16 13:34, Thomas Viessmann wrote: > >>>>>>>> Hi Bengt, > >>>>>>>> > >>>>>>>> Sure, the application has lots of objects and references. > >>>>>>>> Downsizing the application has been tried The heap size of 24 g is > >>>>>>>> already at minimum. A smaller heap gave OutOfmemory really quick. > >>>>>>>> My question was more whether the remark phases could be optimized > >>>>>>>> further. I assume this is not the case and we have reached the > >>>>>>>> limitations > >>>>>>>> of G1, right? > >>>>>>> > >>>>>>> How many reference objects does the application use? Can you run > >>>>>>> it with -XX:+PrintReferenceGC to see how many there are? > >>>>>>> > >>>>>>> If there are a lot of them I don't think there is much more that > >>>>>>> can be done for the remark phase. But if there are not that many I > >>>>>>> guess it means that the remark phase is inefficient. > >>>>>>> > >>>>>>> Have you tried any of the other GCs? How do they behave with this > >>>>>>> application? > >>>>>>> > >>>>>>> Thanks, > >>>>>>> Bengt > >>>>>>> > >>>>>>>> Thanks and Regards > >>>>>>>> > >>>>>>>> Thomas > >>>>>>>> > >>>>>>>> On 05/16/14 13:18, Bengt Rutisson wrote: > >>>>>>>>> Hi Thomas, > >>>>>>>>> > >>>>>>>>> On 2014-05-16 13:10, Thomas Viessmann wrote: > >>>>>>>>>> Hi Bengt, > >>>>>>>>>> > >>>>>>>>>> Well, that's already done and it did improve things. > >>>>>>>>>> argv[21]: -XX:+ParallelRefProcEnabled > >>>>>>>>>> argv[22]: -XX:ParallelGCThreads=48 > >>>>>>>>> > >>>>>>>>> Sorry, I missed that. > >>>>>>>>> > >>>>>>>>>> before -XX:+ParallelRefProcEnabled was set the stop times were in > >>>>>>>>>> the range of 20 to 60 seconds.>>>>>>> > >>>>>>>>> > >>>>>>>>> OK. Glad it helped some. :) > >>>>>>>>> > >>>>>>>>>> The application is a Cacao by Oracle. So they cannot change it. > >>>>>>>>> > >>>>>>>>> Is there some way of reducing the amount of reference objects > >>>>>>>>> that Cacao uses? Does it have cache sizes or similar that can be > >>>>>>>>> tuned. With a JFR recording we might be able to figure out where > >>>>>>>>> the reference objects come from. > >>>>>>>>> > >>>>>>>>> Thanks, > >>>>>>>>> Bengt > >>>>>>>>> > >>>>>>>>>> Thanks and Regards > >>>>>>>>>> > >>>>>>>>>> Thomas > >>>>>>>>>> > >>>>>>>>>> On 05/16/14 12:58, Bengt Rutisson wrote: > >>>>>>>>>>> Hi Thomas, > >>>>>>>>>>> > >>>>>>>>>>> It looks like the application is using a lot of Reference > >>>>>>>>>>> objects. The time spent in remark is dominated by reference > >>>>>>>>>>> processing. See the attached graph generated from the log file > >>>>>>>>>>> you sent. > >>>>>>>>>>> > >>>>>>>>>>> You can try to see if adding -XX:+ParallelRefProcEnabled > >>>>>>>>>>> improves the situation. > >>>>>>>>>>> > >>>>>>>>>>> If the customer is interested in updating their application > >>>>>>>>>>> they might want to see if they can reduce the number of > >>>>>>>>>>> java.lang.ref.Reference objects they use. > >>>>>>>>>>> > >>>>>>>>>>> Hths, > >>>>>>>>>>> Bengt > >>>>>>>>>>> > >>>>>>>>>>> On 2014-05-16 10:26, Thomas Viessmann wrote: > >>>>>>>>>>>> Hi, > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> I've been tuning a Java 7u51, Solaris 10, T4 system with 24G > >>>>>>>>>>>> heap. > >>>>>>>>>>>> My customer is not very happy with the remark pauses of up > >>>>>>>>>>>> to 2 seconds. > >>>>>>>>>>>> > >>>>>>>>>>>> -XX:ParallelGCThreads=48 turned out to be the optimum. Here > >>>>>>>>>>>> > >>>>>>>>>>>> is the log file > >>>>>>>>>>>> which contains the java args at the top: > >>>>>>>>>>>> > >>>>>>>>>>>> http://aubing.de.oracle.com/gclog/gc_log_03052014.log > >>>>>>>>>>>> > >>>>>>>>>>>> Any idea to drive the remark stop times further down? > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> Thanks and Regards > >>>>>>>>>>>> > >>>>>>>>>>>> Thomas > >>>>>>>>>> > >>>>>>>>>> ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen > >>>>>>>>>> > >>>>>>>>>> ORACLE Deutschland B.V. & Co. KG > >>>>>>>>>> Hauptverwaltung: Riesstr. 25, D-80992 Muenchen > >>>>>>>>>> Registergericht: Amtsgericht Muenchen, HRA 95603 > >>>>>>>>>> Gesch?ftsf?hrere: Juergen Kunz > >>>>>>>>>> > >>>>>>>>>> Komplement?rin: ORACLE Deutschland Verwaltung B.V. > >>>>>>>>>> Hertogswetering 163/167, 3543 AS Utrecht, Niederlande > >>>>>>>>>> Handelsregister der Handelskammer Midden-Niederlande, Nr. > >>>>>>>>>> 30143697 > >>>>>>>>>> Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher > >>>>>>>>>> > >>>>>>>>>> ----------------------------------------------------------------- > >>>>>>>>>> -- > >>>>>>>>>> ----- > >>>>>>>>>> ----------------------------------------------------------------- > >>>>>>>>>> -- > >>>>>>>>>> ----- > >>>>>>>>>> Oracle > >>>>>>>>>> is committed to developing practices and products that help > >>>>>>>>>> protect the environment > >>>>>>>> > >>>>>>>> ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen > >>>>>>>> > >>>>>>>> ORACLE Deutschland B.V. & Co. KG > >>>>>>>> Hauptverwaltung: Riesstr. 25, D-80992 Muenchen > >>>>>>>> Registergericht: Amtsgericht Muenchen, HRA 95603 > >>>>>>>> Gesch?ftsf?hrere: Juergen Kunz > >>>>>>>> > >>>>>>>> Komplement?rin: ORACLE Deutschland Verwaltung B.V. > >>>>>>>> Hertogswetering 163/167, 3543 AS Utrecht, Niederlande > >>>>>>>> Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 > >>>>>>>> Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher > >>>>>>>> > >>>>>>>> ------------------------------------------------------------------- > >>>>>>>> -- > >>>>>>>> --- > >>>>>>>> ------------------------------------------------------------------- > >>>>>>>> -- > >>>>>>>> --- > >>>>>>>> Oracle > >>>>>>>> is committed to developing practices and products that help > >>>>>>>> protect the environment > >>>>>> > >>>>>> ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen > >>>>>> > >>>>>> ORACLE Deutschland B.V. & Co. KG > >>>>>> Hauptverwaltung: Riesstr. 25, D-80992 Muenchen > >>>>>> Registergericht: Amtsgericht Muenchen, HRA 95603 > >>>>>> Gesch?ftsf?hrere: Juergen Kunz > >>>>>> > >>>>>> Komplement?rin: ORACLE Deutschland Verwaltung B.V. > >>>>>> Hertogswetering 163/167, 3543 AS Utrecht, Niederlande > >>>>>> Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 > >>>>>> Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher > >>>>>> > >>>>>> --------------------------------------------------------------------- > >>>>>> -- > >>>>>> - > >>>>>> --------------------------------------------------------------------- > >>>>>> -- > >>>>>> - > >>>>>> > >>>>>> Oracle is committed to developing practices and products that help > >>>>>> protect the environment > >>> > >>> ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen > >>> > >>> ORACLE Deutschland B.V. & Co. KG > >>> Hauptverwaltung: Riesstr. 25, D-80992 Muenchen > >>> Registergericht: Amtsgericht Muenchen, HRA 95603 > >>> Gesch?ftsf?hrere: Juergen Kunz > >>> > >>> Komplement?rin: ORACLE Deutschland Verwaltung B.V. > >>> Hertogswetering 163/167, 3543 AS Utrecht, Niederlande > >>> Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 > >>> Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher > >>> > >>> ------------------------------------------------------------------------ > >>> ------------------------------------------------------------------------ > >>> Green Oracle Oracle is committed to > >>> developing practices and products that help protect the environment From thomas.viessmann at oracle.com Wed May 28 14:36:42 2014 From: thomas.viessmann at oracle.com (Thomas Viessmann) Date: Wed, 28 May 2014 16:36:42 +0200 Subject: G1:long remark pauses In-Reply-To: <5385EF1D.4030506@oracle.com> References: <5375CBBC.2030904@oracle.com> <5385905D.6000704@oracle.com> <5385D4AB.1040907@oracle.com> <4173678.fBVK9KDdpU@mgerdin03> <5385EF1D.4030506@oracle.com> Message-ID: <5385F47A.9000200@oracle.com> Hi all, FYI: this application has lots of long living objects. I quickly gave up CMS due to high promotion rate and extreme fragmentation. Even with huge eden and survivors configured. I have already set InitiatingHeapOccupancyPercent=45 while heap is around 60% full on average. Well, redesigning an application with fewer weak refs is not really a short term option after all ;-) Thanks and Regards Thomas On 05/28/14 16:13, charlie hunt wrote: > On 05/28/2014 07:43 AM, Mikael Gerdin wrote: >> On Wednesday 28 May 2014 07.20.59 charlie hunt wrote: >>> Hi Thomas, >>> >>> About the only thing I can suggest is running the concurrent cycle more >>> frequently so as to run the remark more frequently. Perhaps setting >>> InitiatingHeapOccupancyPercent low enough that the concurrent cycle >>> runs >>> all the time. >> If the weakly reachable objects are reasonably short-lived an increased >> survivor size and tenuring threshold could allow more weak references >> to be >> dealt with by incremental young collections instead. > Mikael, > > Doesn't the suggestion of tuning survivor size go against the general > advice of tuning G1, (and more generally tuning young generation > sizing with G1), since you're essentially overriding the G1 the young > GC pause time heuristics? > > Fwiw, max tenuring threshold defaults to 15 with G1. > > Perhaps an alternative to explicit sizing of survivor spaces (or young > gen) would be increasing MaxPauseTimeMillis which should generally > increase the size of eden and survivor. > > charlie > >> >> /Mikael >> >>> The remark is showing good parallelism since User=25.82 and real=2.39. >>> >>> I suppose a refactoring of the application to reduce usage of >>> WeakReferences is another alternative. >>> >>> Btw, does ParallelOld GC or CMS GC show a similar behavior with a high >>> number of WeakReferences as a GC issue? >>> >>> charlie >>> >>> On 05/28/2014 02:29 AM, Thomas Viessmann wrote: >>>> Hi Charlie and all, >>>> >>>> you were pretty close. It's the weak references which stops the world, >>>> not the SoftRefs: >>>> >>>> 248766.241: [GC remark 248766.246: [GC ref-proc248766.246: >>>> [SoftReference, 184918 refs, 0.1139337 secs]248766.360: >>>> *[WeakReference, 2104678 refs, 1.9516481 secs]248768.312:* >>>> [FinalReference, 45 refs, 0.0033450 secs]248768.315: >>>> [PhantomReference, 106 refs, 0.0029866 secs]248768.318: [JNI Weak >>>> Reference, 0.0008403 secs], 2.0846210 secs], 2.3878287 secs] >>>> >>>> [Times: user=25.82 sys=0.87, real=2.39 secs] >>>> >>>> 248768.630: Total time for which application threads were stopped: >>>> 2.4019674 seconds >>>> >>>> There are also a lot of SoftRefs around but those are not the culprit. >>>> Varying -XX:SoftRefLRUPolicyMSPerMB between 100 and 10000 did not >>>> change anything. >>>> >>>> Any idea how this could be addressed? >>>> >>>> Thanks and Regards >>>> >>>> Thomas >>>> >>>> On 05/16/14 14:40, charlie hunt wrote: >>>>> From the sound of what Thomas is describing, this might be one of >>>>> those apps that's making heavy use of SoftReferences. Output from >>>>> -XX:+PrintReferenceGC as Bengt suggested will show if that's the >>>>> case. >>>>> >>>>> If we see a large number of SoftReferences being processed per GC, we >>>>> may get further help with tuning the SoftReference reclamation >>>>> policy, (-XX:SoftRefLRUPolicyMSPerMB). >>>>> >>>>> charlie >>>>> >>>>> On 05/16/2014 07:32 AM, Bengt Rutisson wrote: >>>>>> Hi Thomas, >>>>>> >>>>>> >>>>>> 16 maj 2014 kl. 14:01 skrev Thomas Viessmann >>>>>> >>>>>> >: >>>>>>> Hi Bengt, >>>>>>> >>>>>>> >>>>>>> Thanks for confirming. ParallelOld had stop pauses in the range of >>>>>>> 20 to 30 seconds. >>>>>>> CMS was a disaster due to extreme fragmentation and high promotion >>>>>>> rate even with >>>>>>> huge eden and survivors. >>>>>> Ok, so even with the long remark pauses G1 is performing better than >>>>>> the other GCs? >>>>>> >>>>>>> There are definitely lots of references. I can find out >>>>>>> details. >>>>>> Thanks, it would be interesting to get this data. >>>>>> >>>>>> Thanks, >>>>>> Bengt >>>>>> >>>>>>> Thanks and Regards >>>>>>> >>>>>>> Thomas >>>>>>> >>>>>>> On 05/16/14 13:53, Bengt Rutisson wrote: >>>>>>>> Hi again Thomas, >>>>>>>> >>>>>>>> On 2014-05-16 13:34, Thomas Viessmann wrote: >>>>>>>>> Hi Bengt, >>>>>>>>> >>>>>>>>> Sure, the application has lots of objects and references. >>>>>>>>> Downsizing the application has been tried The heap size of 24 >>>>>>>>> g is >>>>>>>>> already at minimum. A smaller heap gave OutOfmemory really >>>>>>>>> quick. >>>>>>>>> My question was more whether the remark phases could be optimized >>>>>>>>> further. I assume this is not the case and we have reached the >>>>>>>>> limitations >>>>>>>>> of G1, right? >>>>>>>> How many reference objects does the application use? Can you run >>>>>>>> it with -XX:+PrintReferenceGC to see how many there are? >>>>>>>> >>>>>>>> If there are a lot of them I don't think there is much more that >>>>>>>> can be done for the remark phase. But if there are not that many I >>>>>>>> guess it means that the remark phase is inefficient. >>>>>>>> >>>>>>>> Have you tried any of the other GCs? How do they behave with this >>>>>>>> application? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Bengt >>>>>>>> >>>>>>>>> Thanks and Regards >>>>>>>>> >>>>>>>>> Thomas >>>>>>>>> >>>>>>>>> On 05/16/14 13:18, Bengt Rutisson wrote: >>>>>>>>>> Hi Thomas, >>>>>>>>>> >>>>>>>>>> On 2014-05-16 13:10, Thomas Viessmann wrote: >>>>>>>>>>> Hi Bengt, >>>>>>>>>>> >>>>>>>>>>> Well, that's already done and it did improve things. >>>>>>>>>>> argv[21]: -XX:+ParallelRefProcEnabled >>>>>>>>>>> argv[22]: -XX:ParallelGCThreads=48 >>>>>>>>>> Sorry, I missed that. >>>>>>>>>> >>>>>>>>>>> before -XX:+ParallelRefProcEnabled was set the stop times >>>>>>>>>>> were in >>>>>>>>>>> the range of 20 to 60 seconds.>>>>>>> >>>>>>>>>> OK. Glad it helped some. :) >>>>>>>>>> >>>>>>>>>>> The application is a Cacao by Oracle. So they cannot change it. >>>>>>>>>> Is there some way of reducing the amount of reference objects >>>>>>>>>> that Cacao uses? Does it have cache sizes or similar that can be >>>>>>>>>> tuned. With a JFR recording we might be able to figure out where >>>>>>>>>> the reference objects come from. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Bengt >>>>>>>>>> >>>>>>>>>>> Thanks and Regards >>>>>>>>>>> >>>>>>>>>>> Thomas >>>>>>>>>>> >>>>>>>>>>> On 05/16/14 12:58, Bengt Rutisson wrote: >>>>>>>>>>>> Hi Thomas, >>>>>>>>>>>> >>>>>>>>>>>> It looks like the application is using a lot of Reference >>>>>>>>>>>> objects. The time spent in remark is dominated by reference >>>>>>>>>>>> processing. See the attached graph generated from the log file >>>>>>>>>>>> you sent. >>>>>>>>>>>> >>>>>>>>>>>> You can try to see if adding -XX:+ParallelRefProcEnabled >>>>>>>>>>>> improves the situation. >>>>>>>>>>>> >>>>>>>>>>>> If the customer is interested in updating their application >>>>>>>>>>>> they might want to see if they can reduce the number of >>>>>>>>>>>> java.lang.ref.Reference objects they use. >>>>>>>>>>>> >>>>>>>>>>>> Hths, >>>>>>>>>>>> Bengt >>>>>>>>>>>> >>>>>>>>>>>> On 2014-05-16 10:26, Thomas Viessmann wrote: >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> I've been tuning a Java 7u51, Solaris 10, T4 system with 24G >>>>>>>>>>>>> heap. >>>>>>>>>>>>> My customer is not very happy with the remark pauses of up >>>>>>>>>>>>> to 2 seconds. >>>>>>>>>>>>> >>>>>>>>>>>>> -XX:ParallelGCThreads=48 turned out to be the optimum. Here >>>>>>>>>>>>> >>>>>>>>>>>>> is the log file >>>>>>>>>>>>> which contains the java args at the top: >>>>>>>>>>>>> >>>>>>>>>>>>> http://aubing.de.oracle.com/gclog/gc_log_03052014.log >>>>>>>>>>>>> >>>>>>>>>>>>> Any idea to drive the remark stop times further down? >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks and Regards >>>>>>>>>>>>> >>>>>>>>>>>>> Thomas >>>>>>>>>>> ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 >>>>>>>>>>> Muenchen >>>>>>>>>>> >>>>>>>>>>> ORACLE Deutschland B.V. & Co. KG >>>>>>>>>>> Hauptverwaltung: Riesstr. 25, D-80992 Muenchen >>>>>>>>>>> Registergericht: Amtsgericht Muenchen, HRA 95603 >>>>>>>>>>> Gesch?ftsf?hrere: Juergen Kunz >>>>>>>>>>> >>>>>>>>>>> Komplement?rin: ORACLE Deutschland Verwaltung B.V. >>>>>>>>>>> Hertogswetering 163/167, 3543 AS Utrecht, Niederlande >>>>>>>>>>> Handelsregister der Handelskammer Midden-Niederlande, Nr. >>>>>>>>>>> 30143697 >>>>>>>>>>> Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val >>>>>>>>>>> Maher >>>>>>>>>>> >>>>>>>>>>> ------------------------------------------------------------------- >>>>>>>>>>> >>>>>>>>>>> ----- >>>>>>>>>>> ------------------------------------------------------------------- >>>>>>>>>>> >>>>>>>>>>> ----- >>>>>>>>>>> Oracle >>>>>>>>>>> is committed to developing practices and products that help >>>>>>>>>>> protect the environment >>>>>>>>> ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen >>>>>>>>> >>>>>>>>> ORACLE Deutschland B.V. & Co. KG >>>>>>>>> Hauptverwaltung: Riesstr. 25, D-80992 Muenchen >>>>>>>>> Registergericht: Amtsgericht Muenchen, HRA 95603 >>>>>>>>> Gesch?ftsf?hrere: Juergen Kunz >>>>>>>>> >>>>>>>>> Komplement?rin: ORACLE Deutschland Verwaltung B.V. >>>>>>>>> Hertogswetering 163/167, 3543 AS Utrecht, Niederlande >>>>>>>>> Handelsregister der Handelskammer Midden-Niederlande, Nr. >>>>>>>>> 30143697 >>>>>>>>> Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher >>>>>>>>> >>>>>>>>> --------------------------------------------------------------------- >>>>>>>>> >>>>>>>>> --- >>>>>>>>> --------------------------------------------------------------------- >>>>>>>>> >>>>>>>>> --- >>>>>>>>> Oracle >>>>>>>>> is committed to developing practices and products that help >>>>>>>>> protect the environment >>>>>>> ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen >>>>>>> >>>>>>> ORACLE Deutschland B.V. & Co. KG >>>>>>> Hauptverwaltung: Riesstr. 25, D-80992 Muenchen >>>>>>> Registergericht: Amtsgericht Muenchen, HRA 95603 >>>>>>> Gesch?ftsf?hrere: Juergen Kunz >>>>>>> >>>>>>> Komplement?rin: ORACLE Deutschland Verwaltung B.V. >>>>>>> Hertogswetering 163/167, 3543 AS Utrecht, Niederlande >>>>>>> Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 >>>>>>> Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher >>>>>>> >>>>>>> ----------------------------------------------------------------------- >>>>>>> >>>>>>> - >>>>>>> ----------------------------------------------------------------------- >>>>>>> >>>>>>> - >>>>>>> >>>>>>> Oracle is committed to developing practices and products that help >>>>>>> protect the environment >>>> ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen >>>> >>>> ORACLE Deutschland B.V. & Co. KG >>>> Hauptverwaltung: Riesstr. 25, D-80992 Muenchen >>>> Registergericht: Amtsgericht Muenchen, HRA 95603 >>>> Gesch?ftsf?hrere: Juergen Kunz >>>> >>>> Komplement?rin: ORACLE Deutschland Verwaltung B.V. >>>> Hertogswetering 163/167, 3543 AS Utrecht, Niederlande >>>> Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 >>>> Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher >>>> >>>> ------------------------------------------------------------------------ >>>> >>>> ------------------------------------------------------------------------ >>>> >>>> Green Oracle Oracle is committed to >>>> developing practices and products that help protect the environment > -- Oracle THOMAS VIESSMANN | Senior Principal Technical Support Engineer - Java Phone: +498914302496 | Mobile: +491743005467 Oracle Customer Technical Support - Java ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen ORACLE Deutschland B.V. & Co. KG Hauptverwaltung: Riesstr. 25, D-80992 Muenchen Registergericht: Amtsgericht Muenchen, HRA 95603 Gesch?ftsf?hrere: Juergen Kunz Komplement?rin: ORACLE Deutschland Verwaltung B.V. Hertogswetering 163/167, 3543 AS Utrecht, Niederlande Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher ------------------------------------------------------------------------ ------------------------------------------------------------------------ Green Oracle Oracle is committed to developing practices and products that help protect the environment -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: green-for-email-sig_0.gif Type: image/gif Size: 356 bytes Desc: not available URL: From per.liden at oracle.com Wed May 28 14:38:38 2014 From: per.liden at oracle.com (Per Liden) Date: Wed, 28 May 2014 16:38:38 +0200 Subject: RFR(s): 8044132: Quarantine unstable/broken GC tests In-Reply-To: <5385F13A.3030803@oracle.com> References: <5385EE87.2030404@oracle.com> <5385F13A.3030803@oracle.com> Message-ID: <5385F4EE.5010804@oracle.com> Thanks Jesper! /Per On 05/28/2014 04:22 PM, Jesper Wilhelmsson wrote: > Looks good. > /Jesper > > > Per Liden skrev 28/5/14 16:11: >> Summary: This patch adds the @ignore tag on four GC tests. These tests >> are >> unstable or broken. Until these tests have been fixed we quarantine >> them to >> avoid noise in the test results. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8044132 >> Webrev: http://cr.openjdk.java.net/~pliden/8044132/webrev.0/ >> >> Thanks! >> /Per From per.liden at oracle.com Wed May 28 14:38:52 2014 From: per.liden at oracle.com (Per Liden) Date: Wed, 28 May 2014 16:38:52 +0200 Subject: RFR(s): 8042310: TestStringDeduplicationMemoryUsage test failing In-Reply-To: <5385F05D.8060802@oracle.com> References: <5385E4FA.7080304@oracle.com> <5385F05D.8060802@oracle.com> Message-ID: <5385F4FC.9090108@oracle.com> Thanks Jesper! /Per On 05/28/2014 04:19 PM, Jesper Wilhelmsson wrote: > Looks good. > /Jesper > > Per Liden skrev 28/5/14 15:30: >> Could I please have this patch reviewed. >> >> Summary: This test is unstable and is failing intermittently in >> nightly testing. >> The test relies on getting the heap usage via an MXBean, but since the >> test >> doesn't have complete control over what ends up on the heap the heap >> usage is >> sometimes slightly higher than the test expects. There are other >> (stable) tests >> which already verifies than string deduplication actually happens when >> it's >> supposed to, so this test doesn't really add much value. This patch >> removes the >> unstable test completely. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8042310 >> Webrev: http://cr.openjdk.java.net/~pliden/8042310/webrev.0/ >> >> Thanks! >> /Per From erik.helin at oracle.com Thu May 29 12:26:02 2014 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 29 May 2014 14:26:02 +0200 Subject: RFR(s): 8044132: Quarantine unstable/broken GC tests In-Reply-To: <5385EE87.2030404@oracle.com> References: <5385EE87.2030404@oracle.com> Message-ID: <2101294.GZaqlr0Zao@ehelin-laptop> Hi Per, looks good, reviewed. Thanks, Erik On Wednesday 28 May 2014 16:11:19 PM Per Liden wrote: > Summary: This patch adds the @ignore tag on four GC tests. These tests > are unstable or broken. Until these tests have been fixed we quarantine > them to avoid noise in the test results. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8044132 > Webrev: http://cr.openjdk.java.net/~pliden/8044132/webrev.0/ > > Thanks! > /Per From erik.helin at oracle.com Thu May 29 12:30:52 2014 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 29 May 2014 14:30:52 +0200 Subject: RFR(s): 8042310: TestStringDeduplicationMemoryUsage test failing In-Reply-To: <5385E4FA.7080304@oracle.com> References: <5385E4FA.7080304@oracle.com> Message-ID: <1581556.2UetVeFjdY@ehelin-laptop> Hi Per, looks good, reviewed. Thanks, Erik On Wednesday 28 May 2014 15:30:34 PM Per Liden wrote: > Could I please have this patch reviewed. > > Summary: This test is unstable and is failing intermittently in nightly > testing. The test relies on getting the heap usage via an MXBean, but > since the test doesn't have complete control over what ends up on the > heap the heap usage is sometimes slightly higher than the test expects. > There are other (stable) tests which already verifies than string > deduplication actually happens when it's supposed to, so this test > doesn't really add much value. This patch removes the unstable test > completely. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8042310 > Webrev: http://cr.openjdk.java.net/~pliden/8042310/webrev.0/ > > Thanks! > /Per From per.liden at oracle.com Fri May 30 08:30:43 2014 From: per.liden at oracle.com (Per Liden) Date: Fri, 30 May 2014 10:30:43 +0200 Subject: RFR(s): 8042310: TestStringDeduplicationMemoryUsage test failing In-Reply-To: <1581556.2UetVeFjdY@ehelin-laptop> References: <5385E4FA.7080304@oracle.com> <1581556.2UetVeFjdY@ehelin-laptop> Message-ID: <538841B3.4080409@oracle.com> Thanks Erik! /Per On 05/29/2014 02:30 PM, Erik Helin wrote: > Hi Per, > > looks good, reviewed. > > Thanks, > Erik > > On Wednesday 28 May 2014 15:30:34 PM Per Liden wrote: >> Could I please have this patch reviewed. >> >> Summary: This test is unstable and is failing intermittently in nightly >> testing. The test relies on getting the heap usage via an MXBean, but >> since the test doesn't have complete control over what ends up on the >> heap the heap usage is sometimes slightly higher than the test expects. >> There are other (stable) tests which already verifies than string >> deduplication actually happens when it's supposed to, so this test >> doesn't really add much value. This patch removes the unstable test >> completely. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8042310 >> Webrev: http://cr.openjdk.java.net/~pliden/8042310/webrev.0/ >> >> Thanks! >> /Per > From per.liden at oracle.com Fri May 30 08:30:58 2014 From: per.liden at oracle.com (Per Liden) Date: Fri, 30 May 2014 10:30:58 +0200 Subject: RFR(s): 8044132: Quarantine unstable/broken GC tests In-Reply-To: <2101294.GZaqlr0Zao@ehelin-laptop> References: <5385EE87.2030404@oracle.com> <2101294.GZaqlr0Zao@ehelin-laptop> Message-ID: <538841C2.8040603@oracle.com> Thanks Erik! /Per On 05/29/2014 02:26 PM, Erik Helin wrote: > Hi Per, > > looks good, reviewed. > > Thanks, > Erik > > On Wednesday 28 May 2014 16:11:19 PM Per Liden wrote: >> Summary: This patch adds the @ignore tag on four GC tests. These tests >> are unstable or broken. Until these tests have been fixed we quarantine >> them to avoid noise in the test results. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8044132 >> Webrev: http://cr.openjdk.java.net/~pliden/8044132/webrev.0/ >> >> Thanks! >> /Per >