From jwha at google.com  Wed Jul  1 01:25:05 2015
From: jwha at google.com (Jungwoo Ha)
Date: Tue, 30 Jun 2015 18:25:05 -0700
Subject: RFR (XL): JDK-8086706 Parallel Full GC for CMS
In-Reply-To: <CA+n_jhjRd3_rm9v7n5qMGWptDqPcQ-ax4w4eHT91R=r1Tyj6mA@mail.gmail.com>
References: <CA+n_jhjw=T2O3pquZU++N5miGdXxwh=Kp_TBDTmk2ud79SJCgg@mail.gmail.com>
	<5579B1DB.9090804@oracle.com>
	<CA+3eh10-MyRSJ8gcievY_=hTXUjshGnUfwmZFGMsd9ZMD-xBuw@mail.gmail.com>
	<C9B58FD5-C1E4-4143-A85B-4AB322246822@oracle.com>
	<CA+n_jhjRd3_rm9v7n5qMGWptDqPcQ-ax4w4eHT91R=r1Tyj6mA@mail.gmail.com>
Message-ID: <CA+n_jhiX+wMmNo7-7j5Qr+_ZR+gMRaEpw+P9g34RgtWHN5s7qw@mail.gmail.com>

JEP: https://bugs.openjdk.java.net/browse/JDK-8130200

The bug id is different. Should we continue on this thread or start a new
one?
- Jungwoo
On Jun 27, 2015 5:48 PM, "Jungwoo Ha" <jwha at google.com> wrote:

> Sorry for the long wait. So many things have cleaned up on JDK9.
> Here is the webrev against jdk9/hs-rt.
> http://cr.openjdk.java.net/~jwha/8086706/webrev.jdk9.00
>
> I am preparing for the JEP doc.
>
>
> On Thu, Jun 11, 2015 at 12:31 PM, Kim Barrett <kim.barrett at oracle.com>
> wrote:
>
>> On Jun 11, 2015, at 12:33 PM, Volker Simonis <volker.simonis at gmail.com>
>> wrote:
>> >
>> > Hi,
>> >
>> > if nobody has already started doing it, I'll try to forward port the
>> > patch to jdk9/hs-gc/hotspot.
>>
>> You?ll want to use jdk9/hs-rt rather than jdk9/hs-gc.
>>
>>
>>
>
>
> --
> Jungwoo Ha | Java Platform Team | jwha at google.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150630/f6db8619/attachment.htm>

From thomas.schatzl at oracle.com  Wed Jul  1 12:18:45 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 01 Jul 2015 14:18:45 +0200
Subject: RFR (XS): 8129977: TestSummarizeRSetStats.java fails: Incorrect
	amount of per-period RSet summaries at the end
Message-ID: <1435753125.2641.19.camel@oracle.com>

Hi all,

  can I have reviews for the following tiny change that fixes the test
TestSummarizeRSetStats on machines with a "small" amount of available
physical memory?

The problem is that when run on machines with little memory, it issues
two young-only garbage collections instead of one.

The problem is that the test does not fix the initial heap size, which
is calculated according to available memory. This causes an additional
young GC after the first one induced by the test, messing up the
expected amount of log messages.

The reported behavior can be reproduced locally by either manually
setting -Xms to a significantly lower value than -Xmx (like 12m) or
setting MaxRAM to the same size as the machine's total physical memory
(512m - which results in a too small initial heap size of 10M, causing
this exact error).

After forcing -Xms to -Xmx in the test, it is good again regardless of
other options.

Thanks to StefanJ for observing the issue with automatic heap sizing on
the same machine for JDK-8129590 and getting me on the right track.

CR:
https://bugs.openjdk.java.net/browse/JDK-8129977

Webrev:
http://cr.openjdk.java.net/~tschatzl/8129977/webrev/

Testing:
jprt

Thanks,
  Thomas




From tom.benson at oracle.com  Wed Jul  1 12:32:48 2015
From: tom.benson at oracle.com (Tom Benson)
Date: Wed, 01 Jul 2015 08:32:48 -0400
Subject: RFR (XS): 8129977: TestSummarizeRSetStats.java fails: Incorrect
	amount of per-period RSet summaries at the end
In-Reply-To: <1435753125.2641.19.camel@oracle.com>
References: <1435753125.2641.19.camel@oracle.com>
Message-ID: <5593DDF0.5010307@oracle.com>

Looks good to me.
Tom

On 7/1/2015 8:18 AM, Thomas Schatzl wrote:
> Hi all,
>
>    can I have reviews for the following tiny change that fixes the test
> TestSummarizeRSetStats on machines with a "small" amount of available
> physical memory?
>
> The problem is that when run on machines with little memory, it issues
> two young-only garbage collections instead of one.
>
> The problem is that the test does not fix the initial heap size, which
> is calculated according to available memory. This causes an additional
> young GC after the first one induced by the test, messing up the
> expected amount of log messages.
>
> The reported behavior can be reproduced locally by either manually
> setting -Xms to a significantly lower value than -Xmx (like 12m) or
> setting MaxRAM to the same size as the machine's total physical memory
> (512m - which results in a too small initial heap size of 10M, causing
> this exact error).
>
> After forcing -Xms to -Xmx in the test, it is good again regardless of
> other options.
>
> Thanks to StefanJ for observing the issue with automatic heap sizing on
> the same machine for JDK-8129590 and getting me on the right track.
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8129977
>
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8129977/webrev/
>
> Testing:
> jprt
>
> Thanks,
>    Thomas
>
>



From dmitry.fazunenko at oracle.com  Wed Jul  1 13:39:06 2015
From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko)
Date: Wed, 01 Jul 2015 16:39:06 +0300
Subject: RFR (XS): 8129977: TestSummarizeRSetStats.java fails: Incorrect
	amount of per-period RSet summaries at the end
In-Reply-To: <1435753125.2641.19.camel@oracle.com>
References: <1435753125.2641.19.camel@oracle.com>
Message-ID: <5593ED7A.5070104@oracle.com>

The fix looks good to me as well.
Thanks for detailed description of your analysis.

-- Dima

On 01.07.2015 15:18, Thomas Schatzl wrote:
> Hi all,
>
>    can I have reviews for the following tiny change that fixes the test
> TestSummarizeRSetStats on machines with a "small" amount of available
> physical memory?
>
> The problem is that when run on machines with little memory, it issues
> two young-only garbage collections instead of one.
>
> The problem is that the test does not fix the initial heap size, which
> is calculated according to available memory. This causes an additional
> young GC after the first one induced by the test, messing up the
> expected amount of log messages.
>
> The reported behavior can be reproduced locally by either manually
> setting -Xms to a significantly lower value than -Xmx (like 12m) or
> setting MaxRAM to the same size as the machine's total physical memory
> (512m - which results in a too small initial heap size of 10M, causing
> this exact error).
>
> After forcing -Xms to -Xmx in the test, it is good again regardless of
> other options.
>
> Thanks to StefanJ for observing the issue with automatic heap sizing on
> the same machine for JDK-8129590 and getting me on the right track.
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8129977
>
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8129977/webrev/
>
> Testing:
> jprt
>
> Thanks,
>    Thomas
>
>



From stefan.johansson at oracle.com  Wed Jul  1 13:51:25 2015
From: stefan.johansson at oracle.com (Stefan Johansson)
Date: Wed, 01 Jul 2015 15:51:25 +0200
Subject: RFR: 8129590: TestShrinkDefragmentedHeap.java runs out of memory
Message-ID: <5593F05D.9030300@oracle.com>

Hi,

Please review this test-fix to avoid:
https://bugs.openjdk.java.net/browse/JDK-8129590

Webrev:
http://cr.openjdk.java.net/~sjohanss/8129590/hotspot.00/

Summary:
The test specified the initial heap size and the young generation size 
but not the maximum heap size. It needs more memory than the initial 
heap size and in most cases this is not a problem, because the heap can 
be expanded. But on hosts with very little memory it becomes a problem, 
because the maximum heap size gets limited to the same size as the 
initial heap size.

A simple fix for the issue is to set the the maximum heap size to 
something a little larger than the initial heap size to make sure the 
expansion is possible. This might come with other problems if the host 
can't support the needed heap size, but in such cases I think we should 
avoid running this test.

Testing:
* Verified fix on failing host.
* Reproduced failure locally by setting small Xmx and verified that 
making it larger fixes the issue.

Thanks,
Stefan


From jesper.wilhelmsson at oracle.com  Wed Jul  1 14:10:03 2015
From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson)
Date: Wed, 01 Jul 2015 16:10:03 +0200
Subject: RFR: 8129590: TestShrinkDefragmentedHeap.java runs out of memory
In-Reply-To: <5593F05D.9030300@oracle.com>
References: <5593F05D.9030300@oracle.com>
Message-ID: <5593F4BB.3010402@oracle.com>

Looks good!
/Jesper

Stefan Johansson skrev den 1/7/15 15:51:
> Hi,
>
> Please review this test-fix to avoid:
> https://bugs.openjdk.java.net/browse/JDK-8129590
>
> Webrev:
> http://cr.openjdk.java.net/~sjohanss/8129590/hotspot.00/
>
> Summary:
> The test specified the initial heap size and the young generation size but not
> the maximum heap size. It needs more memory than the initial heap size and in
> most cases this is not a problem, because the heap can be expanded. But on hosts
> with very little memory it becomes a problem, because the maximum heap size gets
> limited to the same size as the initial heap size.
>
> A simple fix for the issue is to set the the maximum heap size to something a
> little larger than the initial heap size to make sure the expansion is possible.
> This might come with other problems if the host can't support the needed heap
> size, but in such cases I think we should avoid running this test.
>
> Testing:
> * Verified fix on failing host.
> * Reproduced failure locally by setting small Xmx and verified that making it
> larger fixes the issue.
>
> Thanks,
> Stefan


From thomas.schatzl at oracle.com  Wed Jul  1 14:34:18 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 01 Jul 2015 16:34:18 +0200
Subject: RFR: (XS) 8129430 - tests that requrie G1 should be excluded
	from execution on embedded platfomrs where g1 is not supported
In-Reply-To: <5592D8F2.50403@oracle.com>
References: <5592D8F2.50403@oracle.com>
Message-ID: <1435761258.2641.20.camel@oracle.com>

Hi Dmitrij,

On Tue, 2015-06-30 at 20:59 +0300, Dmitrij Pochepko wrote:
> Hi,
> 
> Please take a look at small fix into TEST.groups
> 
> Description:
> a "needs_g1gc" group description has a bug - missing "\" symbol at the 
> end of string, which is not last.
> This error leads to wrong groups definitions. So, a simple fix is to add 
> this symbol.
> 
> A bug: https://bugs.openjdk.java.net/browse/JDK-8129430
> A webrev: http://cr.openjdk.java.net/~dpochepk/8129430/webrev.01/
> 

  looks good.

Thomas



From thomas.schatzl at oracle.com  Wed Jul  1 14:37:02 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 01 Jul 2015 16:37:02 +0200
Subject: RFR: 8129590: TestShrinkDefragmentedHeap.java runs out of memory
In-Reply-To: <5593F05D.9030300@oracle.com>
References: <5593F05D.9030300@oracle.com>
Message-ID: <1435761422.2641.22.camel@oracle.com>

Hi Stefan,

On Wed, 2015-07-01 at 15:51 +0200, Stefan Johansson wrote:
> Hi,
> 
> Please review this test-fix to avoid:
> https://bugs.openjdk.java.net/browse/JDK-8129590
> 
> Webrev:
> http://cr.openjdk.java.net/~sjohanss/8129590/hotspot.00/
> 
> Summary:
> The test specified the initial heap size and the young generation size 
> but not the maximum heap size. It needs more memory than the initial 
> heap size and in most cases this is not a problem, because the heap can 
> be expanded. But on hosts with very little memory it becomes a problem, 
> because the maximum heap size gets limited to the same size as the 
> initial heap size.
> 
> A simple fix for the issue is to set the the maximum heap size to 
> something a little larger than the initial heap size to make sure the 
> expansion is possible. This might come with other problems if the host 
> can't support the needed heap size, but in such cases I think we should 
> avoid running this test.
> 
> Testing:
> * Verified fix on failing host.
> * Reproduced failure locally by setting small Xmx and verified that 
> making it larger fixes the issue.

  looks good. Could you add this bug id to the @bug line?

Thomas




From derek.white at oracle.com  Wed Jul  1 14:40:37 2015
From: derek.white at oracle.com (Derek White)
Date: Wed, 1 Jul 2015 10:40:37 -0400
Subject: RFR: (XS) 8129430 - tests that requrie G1 should be excluded from
	execution on embedded platfomrs where g1 is not supported
In-Reply-To: <1435761258.2641.20.camel@oracle.com>
References: <5592D8F2.50403@oracle.com> <1435761258.2641.20.camel@oracle.com>
Message-ID: <5593FBE5.5040801@oracle.com>

On 7/1/15 10:34 AM, Thomas Schatzl wrote:
> Hi Dmitrij,
>
> On Tue, 2015-06-30 at 20:59 +0300, Dmitrij Pochepko wrote:
>> Hi,
>>
>> Please take a look at small fix into TEST.groups
>>
>> Description:
>> a "needs_g1gc" group description has a bug - missing "\" symbol at the
>> end of string, which is not last.
>> This error leads to wrong groups definitions. So, a simple fix is to add
>> this symbol.
>>
>> A bug: https://bugs.openjdk.java.net/browse/JDK-8129430
>> A webrev: http://cr.openjdk.java.net/~dpochepk/8129430/webrev.01/
>>
>    looks good.
>
> Thomas
>
Yikes, that was me. Fix looks good!

  - Derek


From stefan.johansson at oracle.com  Wed Jul  1 14:36:16 2015
From: stefan.johansson at oracle.com (Stefan Johansson)
Date: Wed, 01 Jul 2015 16:36:16 +0200
Subject: RFR: 8129590: TestShrinkDefragmentedHeap.java runs out of memory
In-Reply-To: <1435761422.2641.22.camel@oracle.com>
References: <5593F05D.9030300@oracle.com> <1435761422.2641.22.camel@oracle.com>
Message-ID: <5593FAE0.50608@oracle.com>

Thanks for the reviews Jesper and Thomas,

I will add the bug id and push right away.

Stefan

On 2015-07-01 16:37, Thomas Schatzl wrote:
> Hi Stefan,
>
> On Wed, 2015-07-01 at 15:51 +0200, Stefan Johansson wrote:
>> Hi,
>>
>> Please review this test-fix to avoid:
>> https://bugs.openjdk.java.net/browse/JDK-8129590
>>
>> Webrev:
>> http://cr.openjdk.java.net/~sjohanss/8129590/hotspot.00/
>>
>> Summary:
>> The test specified the initial heap size and the young generation size
>> but not the maximum heap size. It needs more memory than the initial
>> heap size and in most cases this is not a problem, because the heap can
>> be expanded. But on hosts with very little memory it becomes a problem,
>> because the maximum heap size gets limited to the same size as the
>> initial heap size.
>>
>> A simple fix for the issue is to set the the maximum heap size to
>> something a little larger than the initial heap size to make sure the
>> expansion is possible. This might come with other problems if the host
>> can't support the needed heap size, but in such cases I think we should
>> avoid running this test.
>>
>> Testing:
>> * Verified fix on failing host.
>> * Reproduced failure locally by setting small Xmx and verified that
>> making it larger fixes the issue.
>    looks good. Could you add this bug id to the @bug line?
>
> Thomas
>
>



From jon.masamitsu at oracle.com  Wed Jul  1 14:48:50 2015
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Wed, 01 Jul 2015 07:48:50 -0700
Subject: RFR: 8129590: TestShrinkDefragmentedHeap.java runs out of memory
In-Reply-To: <5593F05D.9030300@oracle.com>
References: <5593F05D.9030300@oracle.com>
Message-ID: <5593FDD2.5000902@oracle.com>

Stefan,

You raised the question about possible problems when the
maximum heap size is too large for the platform.   Can the number
and size of the small objects be reduced to the minimum that
still makes the test worthwhile and then reduce the maximum
and initial heap sizes accordingly to reduce the chance of
running on a machine with not enough memory?

Jon

On 7/1/2015 6:51 AM, Stefan Johansson wrote:
> Hi,
>
> Please review this test-fix to avoid:
> https://bugs.openjdk.java.net/browse/JDK-8129590
>
> Webrev:
> http://cr.openjdk.java.net/~sjohanss/8129590/hotspot.00/
>
> Summary:
> The test specified the initial heap size and the young generation size 
> but not the maximum heap size. It needs more memory than the initial 
> heap size and in most cases this is not a problem, because the heap 
> can be expanded. But on hosts with very little memory it becomes a 
> problem, because the maximum heap size gets limited to the same size 
> as the initial heap size.
>
> A simple fix for the issue is to set the the maximum heap size to 
> something a little larger than the initial heap size to make sure the 
> expansion is possible. This might come with other problems if the host 
> can't support the needed heap size, but in such cases I think we 
> should avoid running this test.
>
> Testing:
> * Verified fix on failing host.
> * Reproduced failure locally by setting small Xmx and verified that 
> making it larger fixes the issue.
>
> Thanks,
> Stefan



From jon.masamitsu at oracle.com  Wed Jul  1 16:31:25 2015
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Wed, 01 Jul 2015 09:31:25 -0700
Subject: RFR: 8087324: Use semaphores when starting and stopping GC task
	threads
In-Reply-To: <557AF21B.2090102@oracle.com>
References: <557AF21B.2090102@oracle.com>
Message-ID: <559415DD.7030604@oracle.com>



On 6/12/2015 7:52 AM, Stefan Karlsson wrote:
> Hi all,
>
> The current implementation to distribute tasks to GC worker threads 
> often cause long latencies (multiple milliseconds) when the threads 
> are started and stopped.
>
> The main reason is that the worker threads have to fight over the 
> Monitor lock when they are woken up from the call to Monitor::wait. 
> Another reason is that all worker threads call notify_all when they 
> finish a task and there wakes all all sleeping worker threads, which 
> will yet again force the worker threads to fight over the lock.
>
> I propose that we use semaphores instead, so that the worker threads 
> don't have to fight over a lock when they are woken up.
>
>
> The patches build upon the following patch which introduces a 
> Semaphore utility class. This patch will sent out for review on the 
> hotspot-dev, since it affects non-GC parts of the code:
> http://cr.openjdk.java.net/~stefank/8087322/webrev.00/
> https://bugs.openjdk.java.net/browse/JDK-8087322
>
>
> The first patch that I would like to get reviewed is:
> http://cr.openjdk.java.net/~stefank/8087323/webrev.00/
> https://bugs.openjdk.java.net/browse/JDK-8087323 - Unify and split the 
> work gang classes
>
> It prepares for JDK-8087324, by separating the generic WorkGang 
> implementation from the more elaborate YieldingFlexibleWorkGang (CMS) 
> implementation. By having this part as a separate patch, I hope it 
> will be easier to review JDK-8087324. The patch changes the work gang 
> inheritance from:
>
> AbstractWorkGang
>  WorkGang
>   FlexibleWorkGang
>    YieldingFlexibleWorkGang
>
> to:
>
> AbstractWorkGang
>  WorkGang
>  YieldingFlexibleWorkGang
>
> Parts of the FlexibleWorkGang and WorkGang code that is going to be 
> used by both concrete work gang classes, has been moved into 
> AbstractWorkGang. I've duplicated some code in WorkGang and 
> YieldingFlexibleWorkGang, but that code will be removed from WorkGang 
> in the following patch.

http://cr.openjdk.java.net/~stefank/8087323/webrev.00/src/share/vm/gc/cms/yieldingWorkgroup.hpp.frames.html

There seems to be only one definition of is_YieldingFlexibleGang_task() 
now.  Is that right?  Is that useful?

  131   NOT_PRODUCT(virtual bool is_YieldingFlexibleGang_task() const {
  132     return true;
  133   })


Not a change  in your patch but

   86   AbstractWorkGang(const char* name, uint workers, bool are_GC_task_threads, bool are_ConcurrentGC_threads) :
   87       _name(name),
   88       _total_workers(workers),
   89       _active_workers(UseDynamicNumberOfGCThreads ? 1U : workers),
   90       _are_GC_task_threads(are_GC_task_threads),
   91       _are_Concurren


_active_workers is always calculated as >= 2 unless _total_workers is 
only 1.
So line 89 should be

_active_workers(UseDynamicNumberOfGCThreads ? MIN2(2, workers) : workers)

Should I file a CR for that?  Or do you want to include it.

Have you considered (maybe for a later patch) changing 
YieldingFlexibleWorkGang to
simply YieldingWorkGang?  The "Flexible" attribute of 
YieldingFlexibleWorkGang having
been moved into AbstractWorkGang.

http://cr.openjdk.java.net/~stefank/8087323/webrev.00/src/share/vm/gc/cms/yieldingWorkgroup.cpp.frames.html

Is the cast at 53 necessary? I see it in the original code too.

   50 AbstractGangWorker* YieldingFlexibleWorkGang::allocate_worker(uint which) {
   51   YieldingFlexibleGangWorker* new_member =
   52       new YieldingFlexibleGangWorker(this, which);
   53   return (YieldingFlexibleGangWorker*) new_member;
   54 }


The rest looks good.

I'll do the second patch next.

Jon

>
>
> The second patch I'd like to get reviewed is:
> http://cr.openjdk.java.net/~stefank/8087324/webrev.00/
> https://bugs.openjdk.java.net/browse/JDK-8087324 - Use semaphores when 
> starting and stopping GC task threads
>
> It first simplifies the way we distribute the tasks to the GC worker 
> threads. For example, the coordinator thread dispatches a task to a 
> specific number of workers, and then waits for all work to be 
> completed. There's no risk that multiple tasks will be scheduled 
> simultaneously, so there's no need for the sequences number that is 
> used in the current implementation.
>
> The patch contains two task dispatch / thread synchronization 
> implementations:
>
> The first implementation uses Monitors, similar to what we did before 
> the patch, but with a slightly lower overhead since the code calls 
> notify_all less often. It still suffers from the "thundering heard" 
> problem. When the coordinator thread signals that the worker threads 
> should start, they all wake up from Monitor::wait and they all try to 
> lock the Monitor.
>
> The second, and the more interesting, implementation uses semaphores. 
> When the worker threads wake up from the semaphore wait, they don't 
> have to serialize the execution by taking a lock. This greatly 
> decreases the time it takes to start and stop the worker threads.
>
> The semaphore implementation is used on all platforms where the 
> Semaphore class has been implemented in JDK-8087322. So, on some OS:es 
> the code will revert to the Monitor-based solution until a Semaphore 
> class has been implemented for that OS. So, porters might want to 
> consider implementing the Sempahore class.
>
> There's also a diagnostic vm option 
> (-XX:+/-UseSemaphoreGCThreadsSynchronization) to turn off the 
> Semaphore-based implementation, which can be used to debug this new 
> code. It's mainly targeted towards support and sustaining engineering.
>
>
> The patches have been performance tested on Linux, Solaris, OSX, and 
> Windows.
>
> The effects of the patch can be seen by running benchmarks with small 
> young gen sizes, which triggers frequent and short GCs.
>
> For example, here are runs from the SPECjvm2008 xml.transform 
> benchmark with:
> -Xmx1g -Xms1g -Xmn64m -XX:+PrintGC -XX:+UseG1GC -jar SPECjvm2008.jar 
> -ikv xml.transform -it 30 -wt 30
>
> I got the following GC times:
>
>             Average    Median    99.9 percentile   Max
> Baseline: 8.76ms    8.44 ms   25.9 ms 34.7 ms
> Monitor:   6.17 ms 5.88 ms   26.0 ms 49.1 ms
> Semaphore: 3.43 ms 3.26 ms   13.4 ms           33.4 ms
>
> If I run an empty GC task 10 times per GC, by running the following code:
> http://cr.openjdk.java.net/~stefank/8087324/timedTask/
>
> I get the following numbers to complete the empty GC tasks:
>
>             Average    Median    99.9 percentile   Max
> Baseline: 1.43 ms    0.92 ms   3.43 ms           9.30ms
> Monitor:    0.75ms 0.72 ms   1.74 ms           2.78ms
> Semaphore: 0.07 ms 0.07 ms   0.17 ms           0.26 ms
>
>
>
> The code has been tested with JPRT and our nightly testing suites.
>
> I've created a unit test to run a small test with both the semaphore 
> implementation and the monitor implementation:
> http://cr.openjdk.java.net/~stefank/8087324/workgangTest/
>
> But since we currently don't have code to shutdown worker threads 
> after they have been started, I don't want to push this test (or clean 
> it up) until we have that in place. I created this bug for that:
> https://bugs.openjdk.java.net/browse/JDK-8087340
>
> Thanks,
> StefanK
>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150701/11f7b7ce/attachment.htm>

From tprintezis at twitter.com  Wed Jul  1 22:02:00 2015
From: tprintezis at twitter.com (Tony Printezis)
Date: Wed, 1 Jul 2015 18:02:00 -0400
Subject: RFR: JDK-8086056: ParNew: auto-tune
 ParGCCardsPerStrideChunk
In-Reply-To: <558D6128.6020501@oracle.com>
References: <etPan.5581f51b.699a1ea0.1215a@tw-mbp-tprintezis>
	<55826D13.1030200@oracle.com>
	<etPan.5582cc8d.1bcb7375.1215a@tw-mbp-tprintezis>
	<5587B9F6.3010306@oracle.com>
	<etPan.558986ff.16ab4e1c.1215a@tw-mbp-tprintezis>
	<etPan.558af9e9.4affa36.1215a@tw-mbp-tprintezis>
	<558BB5A8.3060002@oracle.com>
	<etPan.558c022e.23da99d5.1215a@tw-mbp-tprintezis>
	<558CF399.1090705@oracle.com>
	<etPan.558d5405.68777fa0.1215a@tw-mbp-tprintezis>
	<558D6128.6020501@oracle.com>
Message-ID: <etPan.55946358.47295612.289@tw-mbp-tprintezis>

Gerard,

Meant to say thanks for the explanation. The range and constraint features were easy to use.

Tony

On June 26, 2015 at 10:26:51 AM, Gerard Ziemski (gerard.ziemski at oracle.com) wrote:

hi guys, 

On 6/26/2015 8:30 AM, Tony Printezis wrote: 
> 
> The JEP says: 
> 
> "The range and constraints checks are done every time a flag changes, 
> as well as late in the JVM initialization routine (i.e., in 
> init_globals() after stubRoutines_init2()) at the time when all flags 
> have their final values set. We will continue to check the manageable 
> flags as long as the JVM runs." 
> 
> I haven't actively been using this myself, but I assume that the 
> constraints will be checked every time a managaeble flag is changed 
> and that the one trying to change it will get an error reported back 
> if they try to change it in an invalid way. 

The JEP required that in the case of manageable flags, when an attempt 
is made to change value of a flag from outside the Java process, that an 
error message be printed in the outside process and the value be left 
unchanaged. 

If the Java process itself tries to make such change, however, then the 
error message is printed in the process itself and the VM exits. 

There is a window of time, however, during the initialization process, 
when flags are being still set, so during that time the range/constraint 
check is disabled, but changes after Arguments::apply_ergo() are fatal 
to the VM if check fails. 


cheers 
-----

Tony Printezis | JVM/GC Engineer / VM Team | Twitter

@TonyPrintezis
tprintezis at twitter.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150701/7c7ad4de/attachment.htm>

From tprintezis at twitter.com  Wed Jul  1 22:05:05 2015
From: tprintezis at twitter.com (Tony Printezis)
Date: Wed, 1 Jul 2015 18:05:05 -0400
Subject: RFR: JDK-8086056: ParNew: auto-tune
 ParGCCardsPerStrideChunk
In-Reply-To: <1435325859.2599.11.camel@oracle.com>
References: <etPan.5581f51b.699a1ea0.1215a@tw-mbp-tprintezis>
	<55826D13.1030200@oracle.com>
	<etPan.5582cc8d.1bcb7375.1215a@tw-mbp-tprintezis>
	<5587B9F6.3010306@oracle.com>
	<etPan.558986ff.16ab4e1c.1215a@tw-mbp-tprintezis>
	<etPan.558af9e9.4affa36.1215a@tw-mbp-tprintezis>
	<558BB5A8.3060002@oracle.com>
	<etPan.558c022e.23da99d5.1215a@tw-mbp-tprintezis>
	<558CF399.1090705@oracle.com>
	<etPan.558d5405.68777fa0.1215a@tw-mbp-tprintezis>
	<1435325859.2599.11.camel@oracle.com>
Message-ID: <etPan.55946411.211dea1b.289@tw-mbp-tprintezis>

Latest changes, as discussed with Bengt and Thomas,

http://cr.openjdk.java.net/~tonyp/8086056/webrev.2/

Tony

On June 26, 2015 at 9:37:45 AM, Thomas Schatzl (thomas.schatzl at oracle.com) wrote:

Hi Tony,  

On Fri, 2015-06-26 at 09:30 -0400, Tony Printezis wrote:  

[...]  
> >  
> >  
> > The JEP says:  
> >  
> > "The range and constraints checks are done every time a flag  
> > changes, as well as late in the JVM initialization routine (i.e., in  
> > init_globals() after stubRoutines_init2()) at the time when all  
> > flags have their final values set. We will continue to check the  
> > manageable flags as long as the JVM runs."  
> >  
> > I haven't actively been using this myself, but I assume that the  
> > constraints will be checked every time a managaeble flag is changed  
> > and that the one trying to change it will get an error reported back  
> > if they try to change it in an invalid way.  
> >  
> > I'm copying Gerard on this email. He has built this support and can  
> > surely answer the details.  
>  
> Hi, thanks. I?ll definitely look at it and change the code  
> appropriately. BTW, has the change been integrated into the hs gc repo  
> yet? (I don?t see it so I assume it hasn't.)  

You overlooked that a week or two ago we started to try out merging the  
hs-rt and hs-gc repos to test whether doing so is beneficial in terms of  
maintenance etc.  

So the hs-gc repo is currently discontinued. Please rebase your changes  
to hs-rt. :)  

[...]  
> >  
> > Finally, just a heads up. I will be on vacation for the coming three  
> > weeks. I doubt that I will be able to review any new version during  
> > that time. If you get reviews from others I'm perfectly fine with  
> > this being pushed. Depending on how different the changes that you  
> > push are compared to the versions I've reviewed I'll leave it up to  
> > you to decide whether you want to list me as a reviewer or not. I'm  
> > fine either way. It's not a problem to leave me out of the reviewer  
> > list if you are unsure.  
>  
> OK, thanks for letting me know! And enjoy your time off!!!!  

I will take over :)  

Thanks,  
Thomas  



-----

Tony Printezis | JVM/GC Engineer / VM Team | Twitter

@TonyPrintezis
tprintezis at twitter.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150701/3715a091/attachment.htm>

From sangheon.kim at oracle.com  Thu Jul  2 00:03:28 2015
From: sangheon.kim at oracle.com (sangheon.kim)
Date: Wed, 01 Jul 2015 17:03:28 -0700
Subject: RFR: JDK-8086056: ParNew: auto-tune ParGCCardsPerStrideChunk
In-Reply-To: <etPan.55946411.211dea1b.289@tw-mbp-tprintezis>
References: <etPan.5581f51b.699a1ea0.1215a@tw-mbp-tprintezis>	<55826D13.1030200@oracle.com>	<etPan.5582cc8d.1bcb7375.1215a@tw-mbp-tprintezis>	<5587B9F6.3010306@oracle.com>	<etPan.558986ff.16ab4e1c.1215a@tw-mbp-tprintezis>	<etPan.558af9e9.4affa36.1215a@tw-mbp-tprintezis>	<558BB5A8.3060002@oracle.com>	<etPan.558c022e.23da99d5.1215a@tw-mbp-tprintezis>	<558CF399.1090705@oracle.com>	<etPan.558d5405.68777fa0.1215a@tw-mbp-tprintezis>	<1435325859.2599.11.camel@oracle.com>
	<etPan.55946411.211dea1b.289@tw-mbp-tprintezis>
Message-ID: <55947FD0.7070208@oracle.com>

Hi Tony,

I just looked the calculation routine and if we set 
'DynamicParGCStridesMinOldGenCapacity = 
DynamicParGCStridesMaxOldGenCapacity', division by zero would happen.
So if I understand correctly, the constraint of 'max' should be always 
greater than 'min', isn't it?

http://cr.openjdk.java.net/~tonyp/8086056/webrev.2/src/share/vm/gc/cms/parNewGeneration.cpp.frames.html

  885   const size_t min_old_gen_capacity = 
DynamicParGCStridesMinOldGenCapacity;
  886   const size_t max_old_gen_capacity = 
DynamicParGCStridesMaxOldGenCapacity;

  902     const size_t capacity_diff = max_old_gen_capacity - 
min_old_gen_capacity;
  903     const size_t capacity_offset = capacity - min_old_gen_capacity;
  904     const double r = (double) capacity_offset / (double) 
*capacity_diff**;*

Thanks,
Sangheon


On 07/01/2015 03:05 PM, Tony Printezis wrote:
> Latest changes, as discussed with Bengt and Thomas,
>
> http://cr.openjdk.java.net/~tonyp/8086056/webrev.2/ 
> <http://cr.openjdk.java.net/%7Etonyp/8086056/webrev.2/>
>
> Tony
>
> On June 26, 2015 at 9:37:45 AM, Thomas Schatzl 
> (thomas.schatzl at oracle.com <mailto:thomas.schatzl at oracle.com>) wrote:
>
>> Hi Tony,
>>
>> On Fri, 2015-06-26 at 09:30 -0400, Tony Printezis wrote:
>>
>> [...]
>> > >
>> > >
>> > > The JEP says:
>> > >
>> > > "The range and constraints checks are done every time a flag
>> > > changes, as well as late in the JVM initialization routine (i.e., in
>> > > init_globals() after stubRoutines_init2()) at the time when all
>> > > flags have their final values set. We will continue to check the
>> > > manageable flags as long as the JVM runs."
>> > >
>> > > I haven't actively been using this myself, but I assume that the
>> > > constraints will be checked every time a managaeble flag is changed
>> > > and that the one trying to change it will get an error reported back
>> > > if they try to change it in an invalid way.
>> > >
>> > > I'm copying Gerard on this email. He has built this support and can
>> > > surely answer the details.
>> >
>> > Hi, thanks. I?ll definitely look at it and change the code
>> > appropriately. BTW, has the change been integrated into the hs gc repo
>> > yet? (I don?t see it so I assume it hasn't.)
>>
>> You overlooked that a week or two ago we started to try out merging the
>> hs-rt and hs-gc repos to test whether doing so is beneficial in terms of
>> maintenance etc.
>>
>> So the hs-gc repo is currently discontinued. Please rebase your changes
>> to hs-rt. :)
>>
>> [...]
>> > >
>> > > Finally, just a heads up. I will be on vacation for the coming three
>> > > weeks. I doubt that I will be able to review any new version during
>> > > that time. If you get reviews from others I'm perfectly fine with
>> > > this being pushed. Depending on how different the changes that you
>> > > push are compared to the versions I've reviewed I'll leave it up to
>> > > you to decide whether you want to list me as a reviewer or not. I'm
>> > > fine either way. It's not a problem to leave me out of the reviewer
>> > > list if you are unsure.
>> >
>> > OK, thanks for letting me know! And enjoy your time off!!!!
>>
>> I will take over :)
>>
>> Thanks,
>> Thomas
>>
>>
>>
> -----
>
> Tony Printezis | JVM/GC Engineer / VM Team | Twitter
>
> @TonyPrintezis
> tprintezis at twitter.com <mailto:tprintezis at twitter.com>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150701/0230aac3/attachment.htm>

From thomas.schatzl at oracle.com  Thu Jul  2 08:43:36 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 02 Jul 2015 10:43:36 +0200
Subject: RFR: JDK-8086056: ParNew: auto-tune ParGCCardsPerStrideChunk
In-Reply-To: <etPan.55946411.211dea1b.289@tw-mbp-tprintezis>
References: <etPan.5581f51b.699a1ea0.1215a@tw-mbp-tprintezis>
	<55826D13.1030200@oracle.com>
	<etPan.5582cc8d.1bcb7375.1215a@tw-mbp-tprintezis>
	<5587B9F6.3010306@oracle.com>
	<etPan.558986ff.16ab4e1c.1215a@tw-mbp-tprintezis>
	<etPan.558af9e9.4affa36.1215a@tw-mbp-tprintezis>
	<558BB5A8.3060002@oracle.com>
	<etPan.558c022e.23da99d5.1215a@tw-mbp-tprintezis>
	<558CF399.1090705@oracle.com>
	<etPan.558d5405.68777fa0.1215a@tw-mbp-tprintezis>
	<1435325859.2599.11.camel@oracle.com>
	<etPan.55946411.211dea1b.289@tw-mbp-tprintezis>
Message-ID: <1435826616.2615.18.camel@oracle.com>

Hi Tony,

On Wed, 2015-07-01 at 18:05 -0400, Tony Printezis wrote:
> Latest changes, as discussed with Bengt and Thomas,
>
> http://cr.openjdk.java.net/~tonyp/8086056/webrev.2/

  a few suggestions:

- please use braces even for simple statements in if-clauses, e.g. 

parNewGeneration.cpp:

 883   if (!UseDynamicParGCStrides) return;

- Comments should start with upper case:

parNewGeneration.hpp:

 354   // automatically calculate ParGCCardsPerStrideChunk based on the
old

- parameter calculation in ParNewGeneration::adjust_cards_per_stride:
  - DynamicParGCStridesMinSize should be > 0 (only if !
UseDynamicParGCStrides is enabled, not sure if that can be encoded
within the framework)
  - the issues with the Dynamic*Capacity variables Sangheon mentioned,
although I think it is fine that they are equal, just that "r" needs to
be set correctly.

Just wanting to check back if it was intended that the code always
rounds down, achieving the maximum stride only at the end of the
interval. E.g.
 
 912     const int stride_size_log = log2_long((jlong) stride_size);
 913     res = (size_t) 1 << stride_size_log;

It's fine with me, just asking.

Thanks,
  Thomas





From stefan.karlsson at oracle.com  Thu Jul  2 11:48:48 2015
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Thu, 02 Jul 2015 13:48:48 +0200
Subject: RFR: 8087324: Use semaphores when starting and stopping GC task
	threads
In-Reply-To: <559415DD.7030604@oracle.com>
References: <557AF21B.2090102@oracle.com> <559415DD.7030604@oracle.com>
Message-ID: <55952520.6030609@oracle.com>



On 2015-07-01 18:31, Jon Masamitsu wrote:
>
>
> On 6/12/2015 7:52 AM, Stefan Karlsson wrote:
>> Hi all,
>>
>> The current implementation to distribute tasks to GC worker threads 
>> often cause long latencies (multiple milliseconds) when the threads 
>> are started and stopped.
>>
>> The main reason is that the worker threads have to fight over the 
>> Monitor lock when they are woken up from the call to Monitor::wait. 
>> Another reason is that all worker threads call notify_all when they 
>> finish a task and there wakes all all sleeping worker threads, which 
>> will yet again force the worker threads to fight over the lock.
>>
>> I propose that we use semaphores instead, so that the worker threads 
>> don't have to fight over a lock when they are woken up.
>>
>>
>> The patches build upon the following patch which introduces a 
>> Semaphore utility class. This patch will sent out for review on the 
>> hotspot-dev, since it affects non-GC parts of the code:
>> http://cr.openjdk.java.net/~stefank/8087322/webrev.00/
>> https://bugs.openjdk.java.net/browse/JDK-8087322
>>
>>
>> The first patch that I would like to get reviewed is:
>> http://cr.openjdk.java.net/~stefank/8087323/webrev.00/
>> https://bugs.openjdk.java.net/browse/JDK-8087323 - Unify and split 
>> the work gang classes
>>
>> It prepares for JDK-8087324, by separating the generic WorkGang 
>> implementation from the more elaborate YieldingFlexibleWorkGang (CMS) 
>> implementation. By having this part as a separate patch, I hope it 
>> will be easier to review JDK-8087324. The patch changes the work gang 
>> inheritance from:
>>
>> AbstractWorkGang
>>  WorkGang
>>   FlexibleWorkGang
>>    YieldingFlexibleWorkGang
>>
>> to:
>>
>> AbstractWorkGang
>>  WorkGang
>>  YieldingFlexibleWorkGang
>>
>> Parts of the FlexibleWorkGang and WorkGang code that is going to be 
>> used by both concrete work gang classes, has been moved into 
>> AbstractWorkGang. I've duplicated some code in WorkGang and 
>> YieldingFlexibleWorkGang, but that code will be removed from WorkGang 
>> in the following patch.
>
> http://cr.openjdk.java.net/~stefank/8087323/webrev.00/src/share/vm/gc/cms/yieldingWorkgroup.hpp.frames.html
>
> There seems to be only one definition of 
> is_YieldingFlexibleGang_task() now.  Is that right?  Is that useful?
>
>   131   NOT_PRODUCT(virtual bool is_YieldingFlexibleGang_task() const {
>   132     return true;
>   133   })

I agree. I don't think we need it anymore.


>
> Not a change  in your patch but
>
>    86   AbstractWorkGang(const char* name, uint workers, bool are_GC_task_threads, bool are_ConcurrentGC_threads) :
>    87       _name(name),
>    88       _total_workers(workers),
>    89       _active_workers(UseDynamicNumberOfGCThreads ? 1U : workers),
>    90       _are_GC_task_threads(are_GC_task_threads),
>    91       _are_Concurren
>
> _active_workers is always calculated as >= 2 unless _total_workers is 
> only 1.
> So line 89 should be
>
> _active_workers(UseDynamicNumberOfGCThreads ? MIN2(2, workers) : workers)
>
> Should I file a CR for that?  Or do you want to include it.

I'm not sure that what is proposed above is correct. I see that 
AdaptiveSizePolicy::calc_active_workers returns 2 as a minimum, but both 
ConcurrentMark::calc_parallel_marking_threads and 
AdaptiveSizePolicy::calc_active_conc_workers can return 1.

I also don't think it should be AbstractWorkGang's responsibility to 
have the knowledge about the minimum number of worker threads that are 
used when UseDynamicNumberOfGCThreads are turned on. Maybe we should set 
it to 0, and let the calc_*_active_workers setup the default value.

I would prefer to handle any changes, to this part of the code, as 
separate RFEs.

> Have you considered (maybe for a later patch) changing 
> YieldingFlexibleWorkGang to
> simply YieldingWorkGang?  The "Flexible" attribute of 
> YieldingFlexibleWorkGang having
> been moved into AbstractWorkGang.

I thought about it, but didn't think it was important enough to warrant 
that change in this patch. I wouldn't mind if a RFE was created to 
change the name.

>
> http://cr.openjdk.java.net/~stefank/8087323/webrev.00/src/share/vm/gc/cms/yieldingWorkgroup.cpp.frames.html
>
> Is the cast at 53 necessary? I see it in the original code too.
>
>    50 AbstractGangWorker* YieldingFlexibleWorkGang::allocate_worker(uint which) {
>    51   YieldingFlexibleGangWorker* new_member =
>    52       new YieldingFlexibleGangWorker(this, which);
>    53   return (YieldingFlexibleGangWorker*) new_member;
>    54 }

Yes, this is unnecessary.

>
> The rest looks good.

Thanks.

>
> I'll do the second patch next.

Great.

StefanK

>
> Jon
>
>>
>>
>> The second patch I'd like to get reviewed is:
>> http://cr.openjdk.java.net/~stefank/8087324/webrev.00/
>> https://bugs.openjdk.java.net/browse/JDK-8087324 - Use semaphores 
>> when starting and stopping GC task threads
>>
>> It first simplifies the way we distribute the tasks to the GC worker 
>> threads. For example, the coordinator thread dispatches a task to a 
>> specific number of workers, and then waits for all work to be 
>> completed. There's no risk that multiple tasks will be scheduled 
>> simultaneously, so there's no need for the sequences number that is 
>> used in the current implementation.
>>
>> The patch contains two task dispatch / thread synchronization 
>> implementations:
>>
>> The first implementation uses Monitors, similar to what we did before 
>> the patch, but with a slightly lower overhead since the code calls 
>> notify_all less often. It still suffers from the "thundering heard" 
>> problem. When the coordinator thread signals that the worker threads 
>> should start, they all wake up from Monitor::wait and they all try to 
>> lock the Monitor.
>>
>> The second, and the more interesting, implementation uses semaphores. 
>> When the worker threads wake up from the semaphore wait, they don't 
>> have to serialize the execution by taking a lock. This greatly 
>> decreases the time it takes to start and stop the worker threads.
>>
>> The semaphore implementation is used on all platforms where the 
>> Semaphore class has been implemented in JDK-8087322. So, on some 
>> OS:es the code will revert to the Monitor-based solution until a 
>> Semaphore class has been implemented for that OS. So, porters might 
>> want to consider implementing the Sempahore class.
>>
>> There's also a diagnostic vm option 
>> (-XX:+/-UseSemaphoreGCThreadsSynchronization) to turn off the 
>> Semaphore-based implementation, which can be used to debug this new 
>> code. It's mainly targeted towards support and sustaining engineering.
>>
>>
>> The patches have been performance tested on Linux, Solaris, OSX, and 
>> Windows.
>>
>> The effects of the patch can be seen by running benchmarks with small 
>> young gen sizes, which triggers frequent and short GCs.
>>
>> For example, here are runs from the SPECjvm2008 xml.transform 
>> benchmark with:
>> -Xmx1g -Xms1g -Xmn64m -XX:+PrintGC -XX:+UseG1GC -jar SPECjvm2008.jar 
>> -ikv xml.transform -it 30 -wt 30
>>
>> I got the following GC times:
>>
>>             Average    Median    99.9 percentile   Max
>> Baseline: 8.76ms    8.44 ms   25.9 ms 34.7 ms
>> Monitor:   6.17 ms 5.88 ms   26.0 ms 49.1 ms
>> Semaphore: 3.43 ms 3.26 ms   13.4 ms           33.4 ms
>>
>> If I run an empty GC task 10 times per GC, by running the following code:
>> http://cr.openjdk.java.net/~stefank/8087324/timedTask/
>>
>> I get the following numbers to complete the empty GC tasks:
>>
>>             Average    Median    99.9 percentile   Max
>> Baseline: 1.43 ms    0.92 ms   3.43 ms           9.30ms
>> Monitor:    0.75ms 0.72 ms   1.74 ms           2.78ms
>> Semaphore: 0.07 ms 0.07 ms   0.17 ms           0.26 ms
>>
>>
>>
>> The code has been tested with JPRT and our nightly testing suites.
>>
>> I've created a unit test to run a small test with both the semaphore 
>> implementation and the monitor implementation:
>> http://cr.openjdk.java.net/~stefank/8087324/workgangTest/
>>
>> But since we currently don't have code to shutdown worker threads 
>> after they have been started, I don't want to push this test (or 
>> clean it up) until we have that in place. I created this bug for that:
>> https://bugs.openjdk.java.net/browse/JDK-8087340
>>
>> Thanks,
>> StefanK
>>
>>
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150702/7d186501/attachment.htm>

From stefan.johansson at oracle.com  Thu Jul  2 14:52:16 2015
From: stefan.johansson at oracle.com (Stefan Johansson)
Date: Thu, 02 Jul 2015 16:52:16 +0200
Subject: RFR: 8129417: Oop iteration clean-up to remove
	oop_ms_follow_contents
In-Reply-To: <5592B8DC.4000903@oracle.com>
References: <5588217F.9090201@oracle.com> <5592B8DC.4000903@oracle.com>
Message-ID: <55955020.1000504@oracle.com>

Thansk for reviewing this Per,

New full webrev:
http://cr.openjdk.java.net/~sjohanss/8129417/hotspot.01

Incremental webrevs:
http://cr.openjdk.java.net/~sjohanss/8129417/1-remove-oop_ms_follow_contents/hotspot.00-01/
http://cr.openjdk.java.net/~sjohanss/8129417/2-add-oop_iterate_size/hotspot.00-01/
http://cr.openjdk.java.net/~sjohanss/8129417/3-re-structure-mark-sweep/hotspot.00-01/
http://cr.openjdk.java.net/~sjohanss/8129417/4-compiler-hints-for-inlining/hotspot.00-01/

See comments inline.

On 2015-06-30 17:42, Per Liden wrote:
> Hi Stefan,
>
> I think this looks good in general, some comments below. Question: any 
> plans to address the remaining oopDesc::ms_* and oopDesc::pc_* parts, 
> or do you see anything that would stop us from converting those too in 
> the future?
I think this is cleanup that we also want to do, especially the ms_*. 
Haven't looked as much at the pc_* parts but I guess they cloud be 
handled in a similar way.

>
> On 2015-06-22 16:53, Stefan Johansson wrote:
>> Hi,
>>
>> Please review these changes for RFE:
>> https://bugs.openjdk.java.net/browse/JDK-8129417
>>
>> Webrev for the full change:
>> http://cr.openjdk.java.net/~sjohanss/8129417/hotspot.00/
>>
>> Summary:
>> To allow further cleanups and later remove G1 specific code from
>> mark-sweep, we want to remove the mark-sweep specific visitor
>> oop_ms_follow_contents. Doing this cleanup has proven to be a little
>> more complicated than I first anticipated and to make the change easier
>> to review I've split it into four different parts.
>>
>> Part 1 - removing oop_ms_follow_contents:
>> http://cr.openjdk.java.net/~sjohanss/8129417/1-remove-oop_ms_follow_contents/hotspot.00/ 
>>
>
> iterator.hpp:
> -------------
> - I can't see any reason for MetadataAwareOopClosure::do_klass() to be 
> in iterator.inline.hpp, so I'd suggest we move it to iterator.hpp.
>
Moved and also did some re-arranging in iterator.hpp.

>
> instanceMirrorKlass.inline.hpp
> ------------------------------
> - A little typo in comment on line 64, "sowhen ..."
>
Fixed.

> - Looks like we lost this assert when code was moved from markSweep.cpp:
>
>     // If klass is NULL then this a mirror for a primitive type.
>     // We don't have to follow them, since they are handled as strong
>     // roots in Universe::oops_do.
>     assert(java_lang_Class::is_primitive(obj), "Sanity check");
>
Added back.

>
> specialized_oop_closures.hpp
> ----------------------------
> - I'd prefer if the forwarding declaration comes after the DefNew part 
> rather then in the middle of the ParNew/CMS parts.
>
Fixed.

>
>>
>>
>> This is more or less what the RFE is all about, the other changes are
>> needed to avoid introducing a regression. To be able to remove the
>> mark-sweep specific visitor oop_ms_follow_contents the generic visitor
>> oop_oop_iterate for all *Klass classes needs to be able to handle the
>> mark-sweep case correctly. This is done by adding more functionality to
>> the Devirtualizer and make use of it in the generic visitors.
>>
>> The MarkAndPushClosure is also added to the set of closures that are
>> specialized to avoid using virtual calls during the iteration.
>> ---
>>
>> Part 2 - introducing oop_iterate_size:
>> http://cr.openjdk.java.net/~sjohanss/8129417/2-add-oop_iterate_size/hotspot.00/ 
>>
>
> space.cpp
> ---------
> - In ContiguousSpaceDCTOC::walk_mem_region_with_cl() we should be 
> using oop_iterate_size() in the loop as well and get rid of the 
> oop->size() call.
>
Fixed.

>
>>
>>
>> As stated above the cleanup to remove oop_ms_follow_content give a
>> slight regression. One reason for this regression was the fact the
>> oop_ms_follow_content didn't care about calculating the size for the
>> object it iterated, but the generic visitors do. When looking through
>> the code base it is clear that only a subset of the users of
>> oop_iterate() really cares about the size and to solve this regression
>> and make it possible to iterate without calculating the size I've added
>> oop_iterate_size(). It is implemented using oop_iterate() (now void) but
>> also calculates the size.
>> ---
>>
>> Part 3 - restructure mark-sweep:
>> http://cr.openjdk.java.net/~sjohanss/8129417/3-re-structure-mark-sweep/hotspot.00/ 
>>
>
> markSweep.hpp
> -------------
> - It looks like MarkSweep has a lot of functions that should be 
> private now. I didn't check them all, but here's some that I think 
> could be made private.
>
>   static void mark_object(oop obj);
>   template <class T> static inline void follow_root(T* p);
>   static inline void push_objarray(oop obj, size_t index);
>   static void follow_stack();
>   static void follow_object(oop obj);
>   static void follow_array(objArrayOop array, int index);
>
Made all but follow_stack private (it's used by PSMarkSweep).

>
> - In MarkSweep::follow_stack/follow_root, could we move the 
> is_objArray() check into follow_object() instead and void the 
> duplication?
>
Did this, but it gives a small regression on linux_x64. Not entirely 
sure what to do with this, will do yet another re-run with all changes.

>>
>>
>> Both a cleanup and a way to avoid part of the regression on some
>> platforms. We want to avoid having too much code in the inline.hpp
>> files, but we also want all compilers to be able to inline call in a
>> good manor. This part of the change moves a lot of code from
>> markSweep.inline.hpp to markSweep.cpp and also try to structure it to
>> allow good inlining.
>> ---
>>
>> Part 4 - compiler hints for inlining:
>> http://cr.openjdk.java.net/~sjohanss/8129417/4-compiler-hints-for-inlining/hotspot.00/ 
>>
>
> stack.inline.hpp
> ----------------
> - Please #undef NOINLINE somewhere in the end of that file.
>
Fixed.

Thanks,
Stefan

> cheers,
> /Per
>
>>
>>
>> To avoid having regressions it seems to be very important that certain
>> code-paths are correctly inline. To achieve this we need to hint the
>> compiler in some parts of the code.
>>
>> The change does two things:
>> * Avoid inlining the slow-path for Stack::push() when using gcc
>> * Add always_inline for Solaris in instanceKlass.inline.hpp (same as
>> what is already present for Windows compiler)
>> ---
>>
>> Testing:
>> * Adhoc RBT run for functionality - no new failures.
>> * Adhoc aurora.se run for performance - no obvious regression.
>> * Many local runs tuned to do Full GCs a lot to verify that there is no
>> obvious regression.
>>
>> Thanks,
>> Stefan



From jesper.wilhelmsson at oracle.com  Thu Jul  2 15:07:36 2015
From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson)
Date: Thu, 02 Jul 2015 17:07:36 +0200
Subject: RFR(XXS): JDK-8130330 - Quarantine
	gc/survivorAlignment/TestPromotionFromSurvivorToTenuredAfterMinorGC.java
Message-ID: <559553B8.9080502@oracle.com>

Hi,

Please review this tiny change to quarantine 
TestPromotionFromSurvivorToTenuredAfterMinorGC.java

Due to JDK-8130308 - "Too low memory usage in 
TestPromotionFromSurvivorToTenuredAfterMinorGC.java" we have not been able to 
integrate main into hs-rt. I have not been able to reproduce the failure 
locally, but it fails every time in jprt.

Quarantining the test to be able to integrate changes in main to hs-rt. This may 
allow us to find other scenarios where the test fails that may be easier to 
debug than in jprt.

Bug: https://bugs.openjdk.java.net/browse/JDK-8130330
Webrev: http://cr.openjdk.java.net/~jwilhelm/8130330/

Thanks,
/Jesper


From jon.masamitsu at oracle.com  Thu Jul  2 15:03:05 2015
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Thu, 02 Jul 2015 08:03:05 -0700
Subject: RFR: 8087324: Use semaphores when starting and stopping GC task
	threads
In-Reply-To: <55952520.6030609@oracle.com>
References: <557AF21B.2090102@oracle.com> <559415DD.7030604@oracle.com>
	<55952520.6030609@oracle.com>
Message-ID: <559552A9.8030008@oracle.com>



On 07/02/2015 04:48 AM, Stefan Karlsson wrote:
>
>
> On 2015-07-01 18:31, Jon Masamitsu wrote:
>>
>>
>> On 6/12/2015 7:52 AM, Stefan Karlsson wrote:
>>> Hi all,
>>>
>>> The current implementation to distribute tasks to GC worker threads 
>>> often cause long latencies (multiple milliseconds) when the threads 
>>> are started and stopped.
>>>
>>> The main reason is that the worker threads have to fight over the 
>>> Monitor lock when they are woken up from the call to Monitor::wait. 
>>> Another reason is that all worker threads call notify_all when they 
>>> finish a task and there wakes all all sleeping worker threads, which 
>>> will yet again force the worker threads to fight over the lock.
>>>
>>> I propose that we use semaphores instead, so that the worker threads 
>>> don't have to fight over a lock when they are woken up.
>>>
>>>
>>> The patches build upon the following patch which introduces a 
>>> Semaphore utility class. This patch will sent out for review on the 
>>> hotspot-dev, since it affects non-GC parts of the code:
>>> http://cr.openjdk.java.net/~stefank/8087322/webrev.00/
>>> https://bugs.openjdk.java.net/browse/JDK-8087322
>>>
>>>
>>> The first patch that I would like to get reviewed is:
>>> http://cr.openjdk.java.net/~stefank/8087323/webrev.00/
>>> https://bugs.openjdk.java.net/browse/JDK-8087323 - Unify and split 
>>> the work gang classes
>>>
>>> It prepares for JDK-8087324, by separating the generic WorkGang 
>>> implementation from the more elaborate YieldingFlexibleWorkGang 
>>> (CMS) implementation. By having this part as a separate patch, I 
>>> hope it will be easier to review JDK-8087324. The patch changes the 
>>> work gang inheritance from:
>>>
>>> AbstractWorkGang
>>>  WorkGang
>>>   FlexibleWorkGang
>>>    YieldingFlexibleWorkGang
>>>
>>> to:
>>>
>>> AbstractWorkGang
>>>  WorkGang
>>>  YieldingFlexibleWorkGang
>>>
>>> Parts of the FlexibleWorkGang and WorkGang code that is going to be 
>>> used by both concrete work gang classes, has been moved into 
>>> AbstractWorkGang. I've duplicated some code in WorkGang and 
>>> YieldingFlexibleWorkGang, but that code will be removed from 
>>> WorkGang in the following patch.
>>
>> http://cr.openjdk.java.net/~stefank/8087323/webrev.00/src/share/vm/gc/cms/yieldingWorkgroup.hpp.frames.html
>>
>> There seems to be only one definition of 
>> is_YieldingFlexibleGang_task() now.  Is that right?  Is that useful?
>>
>>   131   NOT_PRODUCT(virtual bool is_YieldingFlexibleGang_task() const {
>>   132     return true;
>>   133   })
>
> I agree. I don't think we need it anymore.
>
Thanks.

>
>>
>> Not a change  in your patch but
>>
>>    86   AbstractWorkGang(const char* name, uint workers, bool are_GC_task_threads, bool are_ConcurrentGC_threads) :
>>    87       _name(name),
>>    88       _total_workers(workers),
>>    89       _active_workers(UseDynamicNumberOfGCThreads ? 1U : workers),
>>    90       _are_GC_task_threads(are_GC_task_threads),
>>    91       _are_Concurren
>>
>> _active_workers is always calculated as >= 2 unless _total_workers is 
>> only 1.
>> So line 89 should be
>>
>> _active_workers(UseDynamicNumberOfGCThreads ? MIN2(2, workers) : workers)
>>
>> Should I file a CR for that?  Or do you want to include it.
>
> I'm not sure that what is proposed above is correct. I see that 
> AdaptiveSizePolicy::calc_active_workers returns 2 as a minimum, but 
> both ConcurrentMark::calc_parallel_marking_threads and 
> AdaptiveSizePolicy::calc_active_conc_workers can return 1.
>
> I also don't think it should be AbstractWorkGang's responsibility to 
> have the knowledge about the minimum number of worker threads that are 
> used when UseDynamicNumberOfGCThreads are turned on. Maybe we should 
> set it to 0, and let the calc_*_active_workers setup the default value.

I think that at one time I had tried to set the default to 0 and 
something failed.  I can see the point though.

>
> I would prefer to handle any changes, to this part of the code, as 
> separate RFEs.

Fair enough.  If I think it's worth doing, I'll file and RFE. Probably 
something more than just setting
it to 2 (maybe picking a default value with the help of AdaptiveSizePolicy).


>
>> Have you considered (maybe for a later patch) changing 
>> YieldingFlexibleWorkGang to
>> simply YieldingWorkGang?  The "Flexible" attribute of 
>> YieldingFlexibleWorkGang having
>> been moved into AbstractWorkGang.
>
> I thought about it, but didn't think it was important enough to 
> warrant that change in this patch. I wouldn't mind if a RFE was 
> created to change the name.

I'll file the RFE  if you agree with the point that "Flexible" describes 
what AbstractWorkGang does now.

>
>>
>> http://cr.openjdk.java.net/~stefank/8087323/webrev.00/src/share/vm/gc/cms/yieldingWorkgroup.cpp.frames.html
>>
>> Is the cast at 53 necessary? I see it in the original code too.
>>
>>    50 AbstractGangWorker* YieldingFlexibleWorkGang::allocate_worker(uint which) {
>>    51   YieldingFlexibleGangWorker* new_member =
>>    52       new YieldingFlexibleGangWorker(this, which);
>>    53   return (YieldingFlexibleGangWorker*) new_member;
>>    54 }
>
> Yes, this is unnecessary.

Thanks.

Jon

>
>>
>> The rest looks good.
>
> Thanks.
>
>>
>> I'll do the second patch next.
>
> Great.
>
> StefanK
>
>>
>> Jon
>>
>>>
>>>
>>> The second patch I'd like to get reviewed is:
>>> http://cr.openjdk.java.net/~stefank/8087324/webrev.00/
>>> https://bugs.openjdk.java.net/browse/JDK-8087324 - Use semaphores 
>>> when starting and stopping GC task threads
>>>
>>> It first simplifies the way we distribute the tasks to the GC worker 
>>> threads. For example, the coordinator thread dispatches a task to a 
>>> specific number of workers, and then waits for all work to be 
>>> completed. There's no risk that multiple tasks will be scheduled 
>>> simultaneously, so there's no need for the sequences number that is 
>>> used in the current implementation.
>>>
>>> The patch contains two task dispatch / thread synchronization 
>>> implementations:
>>>
>>> The first implementation uses Monitors, similar to what we did 
>>> before the patch, but with a slightly lower overhead since the code 
>>> calls notify_all less often. It still suffers from the "thundering 
>>> heard" problem. When the coordinator thread signals that the worker 
>>> threads should start, they all wake up from Monitor::wait and they 
>>> all try to lock the Monitor.
>>>
>>> The second, and the more interesting, implementation uses 
>>> semaphores. When the worker threads wake up from the semaphore wait, 
>>> they don't have to serialize the execution by taking a lock. This 
>>> greatly decreases the time it takes to start and stop the worker 
>>> threads.
>>>
>>> The semaphore implementation is used on all platforms where the 
>>> Semaphore class has been implemented in JDK-8087322. So, on some 
>>> OS:es the code will revert to the Monitor-based solution until a 
>>> Semaphore class has been implemented for that OS. So, porters might 
>>> want to consider implementing the Sempahore class.
>>>
>>> There's also a diagnostic vm option 
>>> (-XX:+/-UseSemaphoreGCThreadsSynchronization) to turn off the 
>>> Semaphore-based implementation, which can be used to debug this new 
>>> code. It's mainly targeted towards support and sustaining engineering.
>>>
>>>
>>> The patches have been performance tested on Linux, Solaris, OSX, and 
>>> Windows.
>>>
>>> The effects of the patch can be seen by running benchmarks with 
>>> small young gen sizes, which triggers frequent and short GCs.
>>>
>>> For example, here are runs from the SPECjvm2008 xml.transform 
>>> benchmark with:
>>> -Xmx1g -Xms1g -Xmn64m -XX:+PrintGC -XX:+UseG1GC -jar SPECjvm2008.jar 
>>> -ikv xml.transform -it 30 -wt 30
>>>
>>> I got the following GC times:
>>>
>>>             Average    Median    99.9 percentile   Max
>>> Baseline: 8.76ms    8.44 ms   25.9 ms 34.7 ms
>>> Monitor:   6.17 ms 5.88 ms   26.0 ms 49.1 ms
>>> Semaphore: 3.43 ms 3.26 ms   13.4 ms           33.4 ms
>>>
>>> If I run an empty GC task 10 times per GC, by running the following 
>>> code:
>>> http://cr.openjdk.java.net/~stefank/8087324/timedTask/
>>>
>>> I get the following numbers to complete the empty GC tasks:
>>>
>>>             Average    Median    99.9 percentile   Max
>>> Baseline: 1.43 ms    0.92 ms   3.43 ms           9.30ms
>>> Monitor:    0.75ms 0.72 ms   1.74 ms           2.78ms
>>> Semaphore: 0.07 ms 0.07 ms   0.17 ms           0.26 ms
>>>
>>>
>>>
>>> The code has been tested with JPRT and our nightly testing suites.
>>>
>>> I've created a unit test to run a small test with both the semaphore 
>>> implementation and the monitor implementation:
>>> http://cr.openjdk.java.net/~stefank/8087324/workgangTest/
>>>
>>> But since we currently don't have code to shutdown worker threads 
>>> after they have been started, I don't want to push this test (or 
>>> clean it up) until we have that in place. I created this bug for that:
>>> https://bugs.openjdk.java.net/browse/JDK-8087340
>>>
>>> Thanks,
>>> StefanK
>>>
>>>
>>>
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150702/ea42ab57/attachment.htm>

From dmitry.fazunenko at oracle.com  Thu Jul  2 15:14:48 2015
From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko)
Date: Thu, 02 Jul 2015 18:14:48 +0300
Subject: RFR(XXS): JDK-8130330 - Quarantine
	gc/survivorAlignment/TestPromotionFromSurvivorToTenuredAfterMinorGC.java
In-Reply-To: <559553B8.9080502@oracle.com>
References: <559553B8.9080502@oracle.com>
Message-ID: <55955568.7010306@oracle.com>

Hi Jesper,

One comment: @ignore should go after @library

Thanks,
Dima

On 02.07.2015 18:07, Jesper Wilhelmsson wrote:
> Hi,
>
> Please review this tiny change to quarantine 
> TestPromotionFromSurvivorToTenuredAfterMinorGC.java
>
> Due to JDK-8130308 - "Too low memory usage in 
> TestPromotionFromSurvivorToTenuredAfterMinorGC.java" we have not been 
> able to integrate main into hs-rt. I have not been able to reproduce 
> the failure locally, but it fails every time in jprt.
>
> Quarantining the test to be able to integrate changes in main to 
> hs-rt. This may allow us to find other scenarios where the test fails 
> that may be easier to debug than in jprt.
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8130330
> Webrev: http://cr.openjdk.java.net/~jwilhelm/8130330/
>
> Thanks,
> /Jesper



From jesper.wilhelmsson at oracle.com  Thu Jul  2 15:34:42 2015
From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson)
Date: Thu, 02 Jul 2015 17:34:42 +0200
Subject: RFR(XXS): JDK-8130330 - Quarantine
	gc/survivorAlignment/TestPromotionFromSurvivorToTenuredAfterMinorGC.java
In-Reply-To: <55955568.7010306@oracle.com>
References: <559553B8.9080502@oracle.com> <55955568.7010306@oracle.com>
Message-ID: <55955A12.9030404@oracle.com>

Fixed:

http://cr.openjdk.java.net/~jwilhelm/8130330/webrev.01/

Thanks Dima!
/Jesper


Dmitry Fazunenko skrev den 2/7/15 17:14:
> Hi Jesper,
>
> One comment: @ignore should go after @library
>
> Thanks,
> Dima
>
> On 02.07.2015 18:07, Jesper Wilhelmsson wrote:
>> Hi,
>>
>> Please review this tiny change to quarantine
>> TestPromotionFromSurvivorToTenuredAfterMinorGC.java
>>
>> Due to JDK-8130308 - "Too low memory usage in
>> TestPromotionFromSurvivorToTenuredAfterMinorGC.java" we have not been able to
>> integrate main into hs-rt. I have not been able to reproduce the failure
>> locally, but it fails every time in jprt.
>>
>> Quarantining the test to be able to integrate changes in main to hs-rt. This
>> may allow us to find other scenarios where the test fails that may be easier
>> to debug than in jprt.
>>
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8130330
>> Webrev: http://cr.openjdk.java.net/~jwilhelm/8130330/
>>
>> Thanks,
>> /Jesper
>


From dmitry.fazunenko at oracle.com  Thu Jul  2 15:39:45 2015
From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko)
Date: Thu, 02 Jul 2015 18:39:45 +0300
Subject: RFR(XXS): JDK-8130330 - Quarantine
	gc/survivorAlignment/TestPromotionFromSurvivorToTenuredAfterMinorGC.java
In-Reply-To: <55955A12.9030404@oracle.com>
References: <559553B8.9080502@oracle.com> <55955568.7010306@oracle.com>
	<55955A12.9030404@oracle.com>
Message-ID: <55955B41.303@oracle.com>

looks good!

Thanks
Dima

On 02.07.2015 18:34, Jesper Wilhelmsson wrote:
> Fixed:
>
> http://cr.openjdk.java.net/~jwilhelm/8130330/webrev.01/
>
> Thanks Dima!
> /Jesper
>
>
> Dmitry Fazunenko skrev den 2/7/15 17:14:
>> Hi Jesper,
>>
>> One comment: @ignore should go after @library
>>
>> Thanks,
>> Dima
>>
>> On 02.07.2015 18:07, Jesper Wilhelmsson wrote:
>>> Hi,
>>>
>>> Please review this tiny change to quarantine
>>> TestPromotionFromSurvivorToTenuredAfterMinorGC.java
>>>
>>> Due to JDK-8130308 - "Too low memory usage in
>>> TestPromotionFromSurvivorToTenuredAfterMinorGC.java" we have not 
>>> been able to
>>> integrate main into hs-rt. I have not been able to reproduce the 
>>> failure
>>> locally, but it fails every time in jprt.
>>>
>>> Quarantining the test to be able to integrate changes in main to 
>>> hs-rt. This
>>> may allow us to find other scenarios where the test fails that may 
>>> be easier
>>> to debug than in jprt.
>>>
>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8130330
>>> Webrev: http://cr.openjdk.java.net/~jwilhelm/8130330/
>>>
>>> Thanks,
>>> /Jesper
>>



From thomas.schatzl at oracle.com  Thu Jul  2 15:49:14 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 02 Jul 2015 17:49:14 +0200
Subject: RFR(XXS): JDK-8130330 - Quarantine
	gc/survivorAlignment/TestPromotionFromSurvivorToTenuredAfterMinorGC.java
In-Reply-To: <55955B41.303@oracle.com>
References: <559553B8.9080502@oracle.com> <55955568.7010306@oracle.com>
	<55955A12.9030404@oracle.com> <55955B41.303@oracle.com>
Message-ID: <1435852154.2596.8.camel@oracle.com>

Hi,

On Thu, 2015-07-02 at 18:39 +0300, Dmitry Fazunenko wrote:
> looks good!
> 
> Thanks
> Dima
> 
> On 02.07.2015 18:34, Jesper Wilhelmsson wrote:
> > Fixed:
> >
> > http://cr.openjdk.java.net/~jwilhelm/8130330/webrev.01/

  looks good.

Thomas




From jesper.wilhelmsson at oracle.com  Thu Jul  2 15:50:47 2015
From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson)
Date: Thu, 02 Jul 2015 17:50:47 +0200
Subject: RFR(XXS): JDK-8130330 - Quarantine
	gc/survivorAlignment/TestPromotionFromSurvivorToTenuredAfterMinorGC.java
In-Reply-To: <1435852154.2596.8.camel@oracle.com>
References: <559553B8.9080502@oracle.com> <55955568.7010306@oracle.com>	
	<55955A12.9030404@oracle.com> <55955B41.303@oracle.com>
	<1435852154.2596.8.camel@oracle.com>
Message-ID: <55955DD7.20107@oracle.com>

Thanks Thomas and Dima!
I'm pushing this change now and will start a new sync main->hs-rt directly 
thereafter.
/Jesper

Thomas Schatzl skrev den 2/7/15 17:49:
> Hi,
>
> On Thu, 2015-07-02 at 18:39 +0300, Dmitry Fazunenko wrote:
>> looks good!
>>
>> Thanks
>> Dima
>>
>> On 02.07.2015 18:34, Jesper Wilhelmsson wrote:
>>> Fixed:
>>>
>>> http://cr.openjdk.java.net/~jwilhelm/8130330/webrev.01/
>
>    looks good.
>
> Thomas
>
>


From daniel.daugherty at oracle.com  Thu Jul  2 17:43:05 2015
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Thu, 02 Jul 2015 11:43:05 -0600
Subject: RFR (XS): 8129977: TestSummarizeRSetStats.java fails: Incorrect
	amount of per-period RSet summaries at the end
Message-ID: <55957829.3010706@oracle.com>

 > http://cr.openjdk.java.net/~tschatzl/8129977/webrev/

test/gc/g1/TestSummarizeRSetStats.java
test/gc/g1/TestSummarizeRSetStatsPerRegion.java
     No comments on the @bug additions.

test/gc/g1/TestSummarizeRSetStatsTools.java
     '-Xms' addition looks fine.

Thumbs up!

Dan


On 7/1/2015 8:18 AM, Thomas Schatzl wrote:
>/  Hi all,
/>/
/>/     can I have reviews for the following tiny change that fixes the test
/>/  TestSummarizeRSetStats on machines with a "small" amount of available
/>/  physical memory?
/>/
/>/  The problem is that when run on machines with little memory, it issues
/>/  two young-only garbage collections instead of one.
/>/
/>/  The problem is that the test does not fix the initial heap size, which
/>/  is calculated according to available memory. This causes an additional
/>/  young GC after the first one induced by the test, messing up the
/>/  expected amount of log messages.
/>/
/>/  The reported behavior can be reproduced locally by either manually
/>/  setting -Xms to a significantly lower value than -Xmx (like 12m) or
/>/  setting MaxRAM to the same size as the machine's total physical memory
/>/  (512m - which results in a too small initial heap size of 10M, causing
/>/  this exact error).
/>/
/>/  After forcing -Xms to -Xmx in the test, it is good again regardless of
/>/  other options.
/>/
/>/  Thanks to StefanJ for observing the issue with automatic heap sizing on
/>/  the same machine for JDK-8129590 and getting me on the right track.
/>/
/>/  CR:
/>/  https://bugs.openjdk.java.net/browse/JDK-8129977
/>/
/>/  Webrev:
/>/  http://cr.openjdk.java.net/~tschatzl/8129977/webrev/  <http://cr.openjdk.java.net/%7Etschatzl/8129977/webrev/>
/>/
/>/  Testing:
/>/  jprt
/>/
/>/  Thanks,
/>/     Thomas
/>/
/>


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150702/a1ebf132/attachment.htm>

From jon.masamitsu at oracle.com  Thu Jul  2 17:43:38 2015
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Thu, 02 Jul 2015 10:43:38 -0700
Subject: RFR: 8087324: Use semaphores when starting and stopping GC task
	threads
In-Reply-To: <5591122F.2000704@oracle.com>
References: <557AF21B.2090102@oracle.com> <5591122F.2000704@oracle.com>
Message-ID: <5595784A.9050903@oracle.com>



On 6/29/2015 2:38 AM, Stefan Karlsson wrote:
> Hi all,
>
> "8087322: Implement a Semaphore utility class" has now been pushed, so 
> I've updated the patch to reflect the changes.
>
> http://cr.openjdk.java.net/~stefank/8087324/webrev.01.delta
> http://cr.openjdk.java.net/~stefank/8087324/webrev.01
http://cr.openjdk.java.net/~stefank/8087324/webrev.01/src/share/vm/gc/shared/workgroup.hpp.frames.html

Are these used?

  194   void print_worker_started_task(AbstractGangTask* task, uint worker_id);
  195   void print_worker_finished_task(AbstractGangTask* task, uint worker_id);


Rest looks good.  Just one question (just for my education).
  

http://cr.openjdk.java.net/~stefank/8087324/webrev.01/src/share/vm/gc/shared/workgroup.cpp.frames.html

  336 void GangWorker::loop() {
  337   while (true) {
  338     WorkData data = wait_for_task();
  339
  340     run_task(data);
  341
  342     signal_task_done();
  343   }
  344 }

Does this allow the same thread to execute more than 1
task ("data" here)?  Meaning if 2 threads are  requested but
1 thread is not scheduled to a cpu, will the other thread
do both chunks of work?

Jon


>
> - The IMPLEMENTS_SEMAPHORE_CLASS define was removed, since all 
> platforms need to provide a Semaphore implementation.
>
> - Removed the need to pass down "max number of workers" to the 
> Semaphore constructor.
>
> - Updated semaphore.hpp include path
>
> Thanks,
> StefanK
>
>
> On 2015-06-12 16:52, Stefan Karlsson wrote:
>> Hi all,
>>
>> The current implementation to distribute tasks to GC worker threads 
>> often cause long latencies (multiple milliseconds) when the threads 
>> are started and stopped.
>>
>> The main reason is that the worker threads have to fight over the 
>> Monitor lock when they are woken up from the call to Monitor::wait. 
>> Another reason is that all worker threads call notify_all when they 
>> finish a task and there wakes all all sleeping worker threads, which 
>> will yet again force the worker threads to fight over the lock.
>>
>> I propose that we use semaphores instead, so that the worker threads 
>> don't have to fight over a lock when they are woken up.
>>
>>
>> The patches build upon the following patch which introduces a 
>> Semaphore utility class. This patch will sent out for review on the 
>> hotspot-dev, since it affects non-GC parts of the code:
>> http://cr.openjdk.java.net/~stefank/8087322/webrev.00/
>> https://bugs.openjdk.java.net/browse/JDK-8087322
>>
>>
>> The first patch that I would like to get reviewed is:
>> http://cr.openjdk.java.net/~stefank/8087323/webrev.00/
>> https://bugs.openjdk.java.net/browse/JDK-8087323 - Unify and split 
>> the work gang classes
>>
>> It prepares for JDK-8087324, by separating the generic WorkGang 
>> implementation from the more elaborate YieldingFlexibleWorkGang (CMS) 
>> implementation. By having this part as a separate patch, I hope it 
>> will be easier to review JDK-8087324. The patch changes the work gang 
>> inheritance from:
>>
>> AbstractWorkGang
>>  WorkGang
>>   FlexibleWorkGang
>>    YieldingFlexibleWorkGang
>>
>> to:
>>
>> AbstractWorkGang
>>  WorkGang
>>  YieldingFlexibleWorkGang
>>
>> Parts of the FlexibleWorkGang and WorkGang code that is going to be 
>> used by both concrete work gang classes, has been moved into 
>> AbstractWorkGang. I've duplicated some code in WorkGang and 
>> YieldingFlexibleWorkGang, but that code will be removed from WorkGang 
>> in the following patch.
>>
>>
>> The second patch I'd like to get reviewed is:
>> http://cr.openjdk.java.net/~stefank/8087324/webrev.00/
>> https://bugs.openjdk.java.net/browse/JDK-8087324 - Use semaphores 
>> when starting and stopping GC task threads
>>
>> It first simplifies the way we distribute the tasks to the GC worker 
>> threads. For example, the coordinator thread dispatches a task to a 
>> specific number of workers, and then waits for all work to be 
>> completed. There's no risk that multiple tasks will be scheduled 
>> simultaneously, so there's no need for the sequences number that is 
>> used in the current implementation.
>>
>> The patch contains two task dispatch / thread synchronization 
>> implementations:
>>
>> The first implementation uses Monitors, similar to what we did before 
>> the patch, but with a slightly lower overhead since the code calls 
>> notify_all less often. It still suffers from the "thundering heard" 
>> problem. When the coordinator thread signals that the worker threads 
>> should start, they all wake up from Monitor::wait and they all try to 
>> lock the Monitor.
>>
>> The second, and the more interesting, implementation uses semaphores. 
>> When the worker threads wake up from the semaphore wait, they don't 
>> have to serialize the execution by taking a lock. This greatly 
>> decreases the time it takes to start and stop the worker threads.
>>
>> The semaphore implementation is used on all platforms where the 
>> Semaphore class has been implemented in JDK-8087322. So, on some 
>> OS:es the code will revert to the Monitor-based solution until a 
>> Semaphore class has been implemented for that OS. So, porters might 
>> want to consider implementing the Sempahore class.
>>
>> There's also a diagnostic vm option 
>> (-XX:+/-UseSemaphoreGCThreadsSynchronization) to turn off the 
>> Semaphore-based implementation, which can be used to debug this new 
>> code. It's mainly targeted towards support and sustaining engineering.
>>
>>
>> The patches have been performance tested on Linux, Solaris, OSX, and 
>> Windows.
>>
>> The effects of the patch can be seen by running benchmarks with small 
>> young gen sizes, which triggers frequent and short GCs.
>>
>> For example, here are runs from the SPECjvm2008 xml.transform 
>> benchmark with:
>> -Xmx1g -Xms1g -Xmn64m -XX:+PrintGC -XX:+UseG1GC -jar SPECjvm2008.jar 
>> -ikv xml.transform -it 30 -wt 30
>>
>> I got the following GC times:
>>
>>             Average    Median    99.9 percentile   Max
>> Baseline: 8.76ms    8.44 ms   25.9 ms 34.7 ms
>> Monitor:   6.17 ms 5.88 ms   26.0 ms 49.1 ms
>> Semaphore: 3.43 ms 3.26 ms   13.4 ms           33.4 ms
>>
>> If I run an empty GC task 10 times per GC, by running the following code:
>> http://cr.openjdk.java.net/~stefank/8087324/timedTask/
>>
>> I get the following numbers to complete the empty GC tasks:
>>
>>             Average    Median    99.9 percentile   Max
>> Baseline: 1.43 ms    0.92 ms   3.43 ms           9.30ms
>> Monitor:    0.75ms 0.72 ms   1.74 ms           2.78ms
>> Semaphore: 0.07 ms 0.07 ms   0.17 ms           0.26 ms
>>
>>
>>
>> The code has been tested with JPRT and our nightly testing suites.
>>
>> I've created a unit test to run a small test with both the semaphore 
>> implementation and the monitor implementation:
>> http://cr.openjdk.java.net/~stefank/8087324/workgangTest/
>>
>> But since we currently don't have code to shutdown worker threads 
>> after they have been started, I don't want to push this test (or 
>> clean it up) until we have that in place. I created this bug for that:
>> https://bugs.openjdk.java.net/browse/JDK-8087340
>>
>> Thanks,
>> StefanK
>>
>>
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150702/870f4d63/attachment.htm>

From stefan.karlsson at oracle.com  Thu Jul  2 17:58:37 2015
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Thu, 02 Jul 2015 19:58:37 +0200
Subject: RFR: 8087324: Use semaphores when starting and stopping GC task
	threads
In-Reply-To: <5595784A.9050903@oracle.com>
References: <557AF21B.2090102@oracle.com> <5591122F.2000704@oracle.com>
	<5595784A.9050903@oracle.com>
Message-ID: <55957BCD.9020604@oracle.com>

On 2015-07-02 19:43, Jon Masamitsu wrote:
>
>
> On 6/29/2015 2:38 AM, Stefan Karlsson wrote:
>> Hi all,
>>
>> "8087322: Implement a Semaphore utility class" has now been pushed, 
>> so I've updated the patch to reflect the changes.
>>
>> http://cr.openjdk.java.net/~stefank/8087324/webrev.01.delta
>> http://cr.openjdk.java.net/~stefank/8087324/webrev.01
> http://cr.openjdk.java.net/~stefank/8087324/webrev.01/src/share/vm/gc/shared/workgroup.hpp.frames.html
>
> Are these used?
>
>   194   void print_worker_started_task(AbstractGangTask* task, uint worker_id);
>   195   void print_worker_finished_task(AbstractGangTask* task, uint worker_id);

No. I'll remove them.

>
>
> Rest looks good.  Just one question (just for my education).
>   
> http://cr.openjdk.java.net/~stefank/8087324/webrev.01/src/share/vm/gc/shared/workgroup.cpp.frames.html
>
>   336 void GangWorker::loop() {
>   337   while (true) {
>   338     WorkData data = wait_for_task();
>   339
>   340     run_task(data);
>   341
>   342     signal_task_done();
>   343   }
>   344 }
>
> Does this allow the same thread to execute more than 1
> task ("data" here)?  Meaning if 2 threads are  requested but
> 1 thread is not scheduled to a cpu, will the other thread
> do both chunks of work?

Yes, that's correct.

Thanks for reviewing!
StefanK

>
> Jon
>
>
>>
>> - The IMPLEMENTS_SEMAPHORE_CLASS define was removed, since all 
>> platforms need to provide a Semaphore implementation.
>>
>> - Removed the need to pass down "max number of workers" to the 
>> Semaphore constructor.
>>
>> - Updated semaphore.hpp include path
>>
>> Thanks,
>> StefanK
>>
>>
>> On 2015-06-12 16:52, Stefan Karlsson wrote:
>>> Hi all,
>>>
>>> The current implementation to distribute tasks to GC worker threads 
>>> often cause long latencies (multiple milliseconds) when the threads 
>>> are started and stopped.
>>>
>>> The main reason is that the worker threads have to fight over the 
>>> Monitor lock when they are woken up from the call to Monitor::wait. 
>>> Another reason is that all worker threads call notify_all when they 
>>> finish a task and there wakes all all sleeping worker threads, which 
>>> will yet again force the worker threads to fight over the lock.
>>>
>>> I propose that we use semaphores instead, so that the worker threads 
>>> don't have to fight over a lock when they are woken up.
>>>
>>>
>>> The patches build upon the following patch which introduces a 
>>> Semaphore utility class. This patch will sent out for review on the 
>>> hotspot-dev, since it affects non-GC parts of the code:
>>> http://cr.openjdk.java.net/~stefank/8087322/webrev.00/
>>> https://bugs.openjdk.java.net/browse/JDK-8087322
>>>
>>>
>>> The first patch that I would like to get reviewed is:
>>> http://cr.openjdk.java.net/~stefank/8087323/webrev.00/
>>> https://bugs.openjdk.java.net/browse/JDK-8087323 - Unify and split 
>>> the work gang classes
>>>
>>> It prepares for JDK-8087324, by separating the generic WorkGang 
>>> implementation from the more elaborate YieldingFlexibleWorkGang 
>>> (CMS) implementation. By having this part as a separate patch, I 
>>> hope it will be easier to review JDK-8087324. The patch changes the 
>>> work gang inheritance from:
>>>
>>> AbstractWorkGang
>>>  WorkGang
>>>   FlexibleWorkGang
>>>    YieldingFlexibleWorkGang
>>>
>>> to:
>>>
>>> AbstractWorkGang
>>>  WorkGang
>>>  YieldingFlexibleWorkGang
>>>
>>> Parts of the FlexibleWorkGang and WorkGang code that is going to be 
>>> used by both concrete work gang classes, has been moved into 
>>> AbstractWorkGang. I've duplicated some code in WorkGang and 
>>> YieldingFlexibleWorkGang, but that code will be removed from 
>>> WorkGang in the following patch.
>>>
>>>
>>> The second patch I'd like to get reviewed is:
>>> http://cr.openjdk.java.net/~stefank/8087324/webrev.00/
>>> https://bugs.openjdk.java.net/browse/JDK-8087324 - Use semaphores 
>>> when starting and stopping GC task threads
>>>
>>> It first simplifies the way we distribute the tasks to the GC worker 
>>> threads. For example, the coordinator thread dispatches a task to a 
>>> specific number of workers, and then waits for all work to be 
>>> completed. There's no risk that multiple tasks will be scheduled 
>>> simultaneously, so there's no need for the sequences number that is 
>>> used in the current implementation.
>>>
>>> The patch contains two task dispatch / thread synchronization 
>>> implementations:
>>>
>>> The first implementation uses Monitors, similar to what we did 
>>> before the patch, but with a slightly lower overhead since the code 
>>> calls notify_all less often. It still suffers from the "thundering 
>>> heard" problem. When the coordinator thread signals that the worker 
>>> threads should start, they all wake up from Monitor::wait and they 
>>> all try to lock the Monitor.
>>>
>>> The second, and the more interesting, implementation uses 
>>> semaphores. When the worker threads wake up from the semaphore wait, 
>>> they don't have to serialize the execution by taking a lock. This 
>>> greatly decreases the time it takes to start and stop the worker 
>>> threads.
>>>
>>> The semaphore implementation is used on all platforms where the 
>>> Semaphore class has been implemented in JDK-8087322. So, on some 
>>> OS:es the code will revert to the Monitor-based solution until a 
>>> Semaphore class has been implemented for that OS. So, porters might 
>>> want to consider implementing the Sempahore class.
>>>
>>> There's also a diagnostic vm option 
>>> (-XX:+/-UseSemaphoreGCThreadsSynchronization) to turn off the 
>>> Semaphore-based implementation, which can be used to debug this new 
>>> code. It's mainly targeted towards support and sustaining engineering.
>>>
>>>
>>> The patches have been performance tested on Linux, Solaris, OSX, and 
>>> Windows.
>>>
>>> The effects of the patch can be seen by running benchmarks with 
>>> small young gen sizes, which triggers frequent and short GCs.
>>>
>>> For example, here are runs from the SPECjvm2008 xml.transform 
>>> benchmark with:
>>> -Xmx1g -Xms1g -Xmn64m -XX:+PrintGC -XX:+UseG1GC -jar SPECjvm2008.jar 
>>> -ikv xml.transform -it 30 -wt 30
>>>
>>> I got the following GC times:
>>>
>>>             Average    Median    99.9 percentile   Max
>>> Baseline: 8.76ms    8.44 ms   25.9 ms 34.7 ms
>>> Monitor:   6.17 ms 5.88 ms   26.0 ms 49.1 ms
>>> Semaphore: 3.43 ms 3.26 ms   13.4 ms           33.4 ms
>>>
>>> If I run an empty GC task 10 times per GC, by running the following 
>>> code:
>>> http://cr.openjdk.java.net/~stefank/8087324/timedTask/
>>>
>>> I get the following numbers to complete the empty GC tasks:
>>>
>>>             Average    Median    99.9 percentile   Max
>>> Baseline: 1.43 ms    0.92 ms   3.43 ms           9.30ms
>>> Monitor:    0.75ms 0.72 ms   1.74 ms           2.78ms
>>> Semaphore: 0.07 ms 0.07 ms   0.17 ms           0.26 ms
>>>
>>>
>>>
>>> The code has been tested with JPRT and our nightly testing suites.
>>>
>>> I've created a unit test to run a small test with both the semaphore 
>>> implementation and the monitor implementation:
>>> http://cr.openjdk.java.net/~stefank/8087324/workgangTest/
>>>
>>> But since we currently don't have code to shutdown worker threads 
>>> after they have been started, I don't want to push this test (or 
>>> clean it up) until we have that in place. I created this bug for that:
>>> https://bugs.openjdk.java.net/browse/JDK-8087340
>>>
>>> Thanks,
>>> StefanK
>>>
>>>
>>>
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150702/347d7625/attachment.htm>

From jon.masamitsu at oracle.com  Thu Jul  2 17:50:52 2015
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Thu, 02 Jul 2015 10:50:52 -0700
Subject: RFR: 8087324: Use semaphores when starting and stopping GC task
	threads
In-Reply-To: <55957BCD.9020604@oracle.com>
References: <557AF21B.2090102@oracle.com> <5591122F.2000704@oracle.com>
	<5595784A.9050903@oracle.com> <55957BCD.9020604@oracle.com>
Message-ID: <559579FC.1000303@oracle.com>



On 07/02/2015 10:58 AM, Stefan Karlsson wrote:
> On 2015-07-02 19:43, Jon Masamitsu wrote:
>>
>>
>> On 6/29/2015 2:38 AM, Stefan Karlsson wrote:
>>> Hi all,
>>>
>>> "8087322: Implement a Semaphore utility class" has now been pushed, 
>>> so I've updated the patch to reflect the changes.
>>>
>>> http://cr.openjdk.java.net/~stefank/8087324/webrev.01.delta
>>> http://cr.openjdk.java.net/~stefank/8087324/webrev.01
>> http://cr.openjdk.java.net/~stefank/8087324/webrev.01/src/share/vm/gc/shared/workgroup.hpp.frames.html
>>
>> Are these used?
>>
>>   194   void print_worker_started_task(AbstractGangTask* task, uint worker_id);
>>   195   void print_worker_finished_task(AbstractGangTask* task, uint worker_id);
>
> No. I'll remove them.
>
>>
>>
>> Rest looks good.  Just one question (just for my education).
>>   
>> http://cr.openjdk.java.net/~stefank/8087324/webrev.01/src/share/vm/gc/shared/workgroup.cpp.frames.html
>>
>>   336 void GangWorker::loop() {
>>   337   while (true) {
>>   338     WorkData data = wait_for_task();
>>   339
>>   340     run_task(data);
>>   341
>>   342     signal_task_done();
>>   343   }
>>   344 }
>>
>> Does this allow the same thread to execute more than 1
>> task ("data" here)?  Meaning if 2 threads are  requested but
>> 1 thread is not scheduled to a cpu, will the other thread
>> do both chunks of work?
>
> Yes, that's correct.

That is really cool.  Thanks.

Jon
>
> Thanks for reviewing!
> StefanK
>
>> Jon
>>
>>
>>>
>>> - The IMPLEMENTS_SEMAPHORE_CLASS define was removed, since all 
>>> platforms need to provide a Semaphore implementation.
>>>
>>> - Removed the need to pass down "max number of workers" to the 
>>> Semaphore constructor.
>>>
>>> - Updated semaphore.hpp include path
>>>
>>> Thanks,
>>> StefanK
>>>
>>>
>>> On 2015-06-12 16:52, Stefan Karlsson wrote:
>>>> Hi all,
>>>>
>>>> The current implementation to distribute tasks to GC worker threads 
>>>> often cause long latencies (multiple milliseconds) when the threads 
>>>> are started and stopped.
>>>>
>>>> The main reason is that the worker threads have to fight over the 
>>>> Monitor lock when they are woken up from the call to Monitor::wait. 
>>>> Another reason is that all worker threads call notify_all when they 
>>>> finish a task and there wakes all all sleeping worker threads, 
>>>> which will yet again force the worker threads to fight over the lock.
>>>>
>>>> I propose that we use semaphores instead, so that the worker 
>>>> threads don't have to fight over a lock when they are woken up.
>>>>
>>>>
>>>> The patches build upon the following patch which introduces a 
>>>> Semaphore utility class. This patch will sent out for review on the 
>>>> hotspot-dev, since it affects non-GC parts of the code:
>>>> http://cr.openjdk.java.net/~stefank/8087322/webrev.00/
>>>> https://bugs.openjdk.java.net/browse/JDK-8087322
>>>>
>>>>
>>>> The first patch that I would like to get reviewed is:
>>>> http://cr.openjdk.java.net/~stefank/8087323/webrev.00/
>>>> https://bugs.openjdk.java.net/browse/JDK-8087323 - Unify and split 
>>>> the work gang classes
>>>>
>>>> It prepares for JDK-8087324, by separating the generic WorkGang 
>>>> implementation from the more elaborate YieldingFlexibleWorkGang 
>>>> (CMS) implementation. By having this part as a separate patch, I 
>>>> hope it will be easier to review JDK-8087324. The patch changes the 
>>>> work gang inheritance from:
>>>>
>>>> AbstractWorkGang
>>>>  WorkGang
>>>>   FlexibleWorkGang
>>>>    YieldingFlexibleWorkGang
>>>>
>>>> to:
>>>>
>>>> AbstractWorkGang
>>>>  WorkGang
>>>>  YieldingFlexibleWorkGang
>>>>
>>>> Parts of the FlexibleWorkGang and WorkGang code that is going to be 
>>>> used by both concrete work gang classes, has been moved into 
>>>> AbstractWorkGang. I've duplicated some code in WorkGang and 
>>>> YieldingFlexibleWorkGang, but that code will be removed from 
>>>> WorkGang in the following patch.
>>>>
>>>>
>>>> The second patch I'd like to get reviewed is:
>>>> http://cr.openjdk.java.net/~stefank/8087324/webrev.00/
>>>> https://bugs.openjdk.java.net/browse/JDK-8087324 - Use semaphores 
>>>> when starting and stopping GC task threads
>>>>
>>>> It first simplifies the way we distribute the tasks to the GC 
>>>> worker threads. For example, the coordinator thread dispatches a 
>>>> task to a specific number of workers, and then waits for all work 
>>>> to be completed. There's no risk that multiple tasks will be 
>>>> scheduled simultaneously, so there's no need for the sequences 
>>>> number that is used in the current implementation.
>>>>
>>>> The patch contains two task dispatch / thread synchronization 
>>>> implementations:
>>>>
>>>> The first implementation uses Monitors, similar to what we did 
>>>> before the patch, but with a slightly lower overhead since the code 
>>>> calls notify_all less often. It still suffers from the "thundering 
>>>> heard" problem. When the coordinator thread signals that the worker 
>>>> threads should start, they all wake up from Monitor::wait and they 
>>>> all try to lock the Monitor.
>>>>
>>>> The second, and the more interesting, implementation uses 
>>>> semaphores. When the worker threads wake up from the semaphore 
>>>> wait, they don't have to serialize the execution by taking a lock. 
>>>> This greatly decreases the time it takes to start and stop the 
>>>> worker threads.
>>>>
>>>> The semaphore implementation is used on all platforms where the 
>>>> Semaphore class has been implemented in JDK-8087322. So, on some 
>>>> OS:es the code will revert to the Monitor-based solution until a 
>>>> Semaphore class has been implemented for that OS. So, porters might 
>>>> want to consider implementing the Sempahore class.
>>>>
>>>> There's also a diagnostic vm option 
>>>> (-XX:+/-UseSemaphoreGCThreadsSynchronization) to turn off the 
>>>> Semaphore-based implementation, which can be used to debug this new 
>>>> code. It's mainly targeted towards support and sustaining engineering.
>>>>
>>>>
>>>> The patches have been performance tested on Linux, Solaris, OSX, 
>>>> and Windows.
>>>>
>>>> The effects of the patch can be seen by running benchmarks with 
>>>> small young gen sizes, which triggers frequent and short GCs.
>>>>
>>>> For example, here are runs from the SPECjvm2008 xml.transform 
>>>> benchmark with:
>>>> -Xmx1g -Xms1g -Xmn64m -XX:+PrintGC -XX:+UseG1GC -jar 
>>>> SPECjvm2008.jar -ikv xml.transform -it 30 -wt 30
>>>>
>>>> I got the following GC times:
>>>>
>>>>             Average    Median    99.9 percentile   Max
>>>> Baseline: 8.76ms    8.44 ms   25.9 ms 34.7 ms
>>>> Monitor:   6.17 ms 5.88 ms   26.0 ms 49.1 ms
>>>> Semaphore: 3.43 ms 3.26 ms   13.4 ms           33.4 ms
>>>>
>>>> If I run an empty GC task 10 times per GC, by running the following 
>>>> code:
>>>> http://cr.openjdk.java.net/~stefank/8087324/timedTask/
>>>>
>>>> I get the following numbers to complete the empty GC tasks:
>>>>
>>>>             Average    Median    99.9 percentile   Max
>>>> Baseline: 1.43 ms    0.92 ms   3.43 ms           9.30ms
>>>> Monitor:    0.75ms 0.72 ms   1.74 ms           2.78ms
>>>> Semaphore: 0.07 ms 0.07 ms   0.17 ms           0.26 ms
>>>>
>>>>
>>>>
>>>> The code has been tested with JPRT and our nightly testing suites.
>>>>
>>>> I've created a unit test to run a small test with both the 
>>>> semaphore implementation and the monitor implementation:
>>>> http://cr.openjdk.java.net/~stefank/8087324/workgangTest/
>>>>
>>>> But since we currently don't have code to shutdown worker threads 
>>>> after they have been started, I don't want to push this test (or 
>>>> clean it up) until we have that in place. I created this bug for that:
>>>> https://bugs.openjdk.java.net/browse/JDK-8087340
>>>>
>>>> Thanks,
>>>> StefanK
>>>>
>>>>
>>>>
>>>
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150702/aae0e868/attachment.htm>

From tprintezis at twitter.com  Thu Jul  2 19:27:17 2015
From: tprintezis at twitter.com (Tony Printezis)
Date: Thu, 2 Jul 2015 15:27:17 -0400
Subject: RFR: JDK-8086056: ParNew: auto-tune
 ParGCCardsPerStrideChunk
In-Reply-To: <55947FD0.7070208@oracle.com>
References: <etPan.5581f51b.699a1ea0.1215a@tw-mbp-tprintezis>
	<55826D13.1030200@oracle.com>
	<etPan.5582cc8d.1bcb7375.1215a@tw-mbp-tprintezis>
	<5587B9F6.3010306@oracle.com>
	<etPan.558986ff.16ab4e1c.1215a@tw-mbp-tprintezis>
	<etPan.558af9e9.4affa36.1215a@tw-mbp-tprintezis>
	<558BB5A8.3060002@oracle.com>
	<etPan.558c022e.23da99d5.1215a@tw-mbp-tprintezis>
	<558CF399.1090705@oracle.com>
	<etPan.558d5405.68777fa0.1215a@tw-mbp-tprintezis>
	<1435325859.2599.11.camel@oracle.com>
	<etPan.55946411.211dea1b.289@tw-mbp-tprintezis>
	<55947FD0.7070208@oracle.com>
Message-ID: <etPan.55959095.76507531.386@tw-mbp-tprintezis>

Sangheon,

Thanks, well spotted. I did what Thomas suggested: allow the diff to be 0 and make sure r is 0.0 when that happens.

FYA: I tried to cause the issue by setting min_capacity == max_capacity and I couldn?t. Note the start of that method: the calculation only takes place when min_capacity <= capacity <= max_capacity. So, if min_capacity == max_capacity, both have to be == capacity for the calculation to happen which will cause the div-by-0. :-)

Anyway, thanks again for bringing this up,

Tony

On July 1, 2015 at 8:03:32 PM, sangheon.kim (sangheon.kim at oracle.com) wrote:

Hi Tony,

I just looked the calculation routine and if we set 'DynamicParGCStridesMinOldGenCapacity = DynamicParGCStridesMaxOldGenCapacity', division by zero would happen.
So if I understand correctly, the constraint of 'max' should be always greater than 'min', isn't it?

http://cr.openjdk.java.net/~tonyp/8086056/webrev.2/src/share/vm/gc/cms/parNewGeneration.cpp.frames.html

?885?? const size_t min_old_gen_capacity = DynamicParGCStridesMinOldGenCapacity;
?886?? const size_t max_old_gen_capacity = DynamicParGCStridesMaxOldGenCapacity;

?902???? const size_t capacity_diff = max_old_gen_capacity - min_old_gen_capacity;
?903???? const size_t capacity_offset = capacity - min_old_gen_capacity;
?904???? const double r = (double) capacity_offset / (double) capacity_diff;

Thanks,
Sangheon


On 07/01/2015 03:05 PM, Tony Printezis wrote:
Latest changes, as discussed with Bengt and Thomas,

http://cr.openjdk.java.net/~tonyp/8086056/webrev.2/

Tony

On June 26, 2015 at 9:37:45 AM, Thomas Schatzl (thomas.schatzl at oracle.com) wrote:

Hi Tony,

On Fri, 2015-06-26 at 09:30 -0400, Tony Printezis wrote:

[...]
> >
> >
> > The JEP says:
> >
> > "The range and constraints checks are done every time a flag
> > changes, as well as late in the JVM initialization routine (i.e., in
> > init_globals() after stubRoutines_init2()) at the time when all
> > flags have their final values set. We will continue to check the
> > manageable flags as long as the JVM runs."
> >
> > I haven't actively been using this myself, but I assume that the
> > constraints will be checked every time a managaeble flag is changed
> > and that the one trying to change it will get an error reported back
> > if they try to change it in an invalid way.
> >
> > I'm copying Gerard on this email. He has built this support and can
> > surely answer the details.
>
> Hi, thanks. I?ll definitely look at it and change the code
> appropriately. BTW, has the change been integrated into the hs gc repo
> yet? (I don?t see it so I assume it hasn't.)

You overlooked that a week or two ago we started to try out merging the
hs-rt and hs-gc repos to test whether doing so is beneficial in terms of
maintenance etc.

So the hs-gc repo is currently discontinued. Please rebase your changes
to hs-rt. :)

[...]
> >
> > Finally, just a heads up. I will be on vacation for the coming three
> > weeks. I doubt that I will be able to review any new version during
> > that time. If you get reviews from others I'm perfectly fine with
> > this being pushed. Depending on how different the changes that you
> > push are compared to the versions I've reviewed I'll leave it up to
> > you to decide whether you want to list me as a reviewer or not. I'm
> > fine either way. It's not a problem to leave me out of the reviewer
> > list if you are unsure.
>
> OK, thanks for letting me know! And enjoy your time off!!!!

I will take over :)

Thanks,
Thomas



-----

Tony Printezis | JVM/GC Engineer / VM Team | Twitter

@TonyPrintezis
tprintezis at twitter.com


-----

Tony Printezis | JVM/GC Engineer / VM Team | Twitter

@TonyPrintezis
tprintezis at twitter.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150702/0288a2d7/attachment.htm>

From tprintezis at twitter.com  Thu Jul  2 19:39:38 2015
From: tprintezis at twitter.com (Tony Printezis)
Date: Thu, 2 Jul 2015 15:39:38 -0400
Subject: RFR: JDK-8086056: ParNew: auto-tune
 ParGCCardsPerStrideChunk
In-Reply-To: <1435826616.2615.18.camel@oracle.com>
References: <etPan.5581f51b.699a1ea0.1215a@tw-mbp-tprintezis>
	<55826D13.1030200@oracle.com>
	<etPan.5582cc8d.1bcb7375.1215a@tw-mbp-tprintezis>
	<5587B9F6.3010306@oracle.com>
	<etPan.558986ff.16ab4e1c.1215a@tw-mbp-tprintezis>
	<etPan.558af9e9.4affa36.1215a@tw-mbp-tprintezis>
	<558BB5A8.3060002@oracle.com>
	<etPan.558c022e.23da99d5.1215a@tw-mbp-tprintezis>
	<558CF399.1090705@oracle.com>
	<etPan.558d5405.68777fa0.1215a@tw-mbp-tprintezis>
	<1435325859.2599.11.camel@oracle.com>
	<etPan.55946411.211dea1b.289@tw-mbp-tprintezis>
	<1435826616.2615.18.camel@oracle.com>
Message-ID: <etPan.5595937a.283b46fd.386@tw-mbp-tprintezis>

Thomas,

Latest version:

http://cr.openjdk.java.net/~tonyp/8086056/webrev.3/

See below for comments.

On July 2, 2015 at 4:43:42 AM, Thomas Schatzl (thomas.schatzl at oracle.com) wrote:

Hi Tony,?

On Wed, 2015-07-01 at 18:05 -0400, Tony Printezis wrote:?
> Latest changes, as discussed with Bengt and Thomas,?
>?
> http://cr.openjdk.java.net/~tonyp/8086056/webrev.2/?

a few suggestions:?

- please use braces even for simple statements in if-clauses, e.g.?

parNewGeneration.cpp:?

883 if (!UseDynamicParGCStrides) return;?


I did that. But I actually added an extra verbose statement in the body of the if-statement (I thought it?d be helpful to also print the value of ParGCCardsPerStrideChunk when the dynamic calculation is off). So, I don?t have a simple return any more, but I?ll keep that in mind in the future (was it anywhere else? I didn?t immediately spot another occurrence of that).




- Comments should start with upper case:?

parNewGeneration.hpp:?

354 // automatically calculate ParGCCardsPerStrideChunk based on the?
old?


Done.




- parameter calculation in ParNewGeneration::adjust_cards_per_stride:?
- DynamicParGCStridesMinSize should be > 0 (only if !?
UseDynamicParGCStrides is enabled, not sure if that can be encoded?
within the framework)?


If !UseDynamicParGCStrides, then both sets of min / max parameters are not used. I think it?d make the constraint checking a pain having to sanity-check parameters whether a flag is on or off (unless they mean different things in either case, of course).

I added a range check to the min / max stride size args, added a lower bound of 1, and decided to add an artificial upper bound of 32K (I have tested with up to 8K). Is that reasonable? We can revisit that if we need to increse it. Note that you can still set ParGCCardsPerStrideChunk to something higher if required.




- the issues with the Dynamic*Capacity variables Sangheon mentioned,?
although I think it is fine that they are equal, just that "r" needs to?
be set correctly.?


Yeah, fixed. (See previous e-mail.)




Just wanting to check back if it was intended that the code always?
rounds down, achieving the maximum stride only at the end of the?
interval. E.g.?

912 const int stride_size_log = log2_long((jlong) stride_size);?
913 res = (size_t) 1 << stride_size_log;?

It's fine with me, just asking.?


I did the simplest calculation. I can revise it if you want. Also note that if capacity > max_capacity, then stride size will be set to the max stride size.

Tony




Thanks,?
Thomas?









-----

Tony Printezis | JVM/GC Engineer / VM Team | Twitter

@TonyPrintezis
tprintezis at twitter.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150702/ee5d7e60/attachment.htm>

From mandy.chung at oracle.com  Fri Jul  3 07:08:02 2015
From: mandy.chung at oracle.com (Mandy Chung)
Date: Fri, 3 Jul 2015 00:08:02 -0700
Subject: JEP 132: More-prompt finalization
In-Reply-To: <558E6D72.1050505@gmail.com>
References: <55674C6E.8050509@gmail.com> <5567CC5E.7000102@oracle.com>
	<8DBC7345-09D8-458F-9017-298D1E620F86@gmail.com>
	<6882C9A35DFB9B4FA2779F7BF5B9757D20761169A2@GSCMAMP06EX.firmwide.corp.gs.com>
	<556B61B0.5030708@gmail.com>
	<A71299B6-5F49-458D-8A12-ABCAADC37F5B@oracle.com>
	<55767D37.4080100@gmail.com> <5587BA4E.3050103@gmail.com>
	<73CD8857-E400-4F30-9651-39ECBE19CC51@oracle.com>
	<558E6D72.1050505@gmail.com>
Message-ID: <3E5CCF95-845B-49B8-BAA8-9949C527312C@oracle.com>


> On Jun 27, 2015, at 2:31 AM, Peter Levart <peter.levart at gmail.com> wrote:
> 
>>> Are any of the features of this prototype interesting for you and worth pursuing further?
>> This mostly appears to be a question for core-libs; that's where the
>> proposed changes are located.  GC is involved mostly to ensure the
>> internal APIs between the collector and reference processing are
>> maintained.
>> 
>> My personal opinion is that we should be working toward a day when we
>> can remove the finalize function, rather than trying to make the
>> infrastructure around it faster.  But clearly that day won't be soon,
>> if ever.
>> 


I agree.  We should put efforts in removing our JDK use of finalizer, as many as we can, and that will help tune the design of any new Cleaner-like API.   

> 
> The infrastructure is shared for all Reference kinds. Improving it impacts all reference processing, not only finalization. If it makes finalization faster too, it can be viewed as just a coincidence ;-)
> 

Improving the reference processing is definitely attractive work.   This work is not a small task and also due to its complexity, we will have to look at this closely after several months once we complete some feature works targeted for JDK 9.  We probably should write a JEP for this when that time comes.  


> But I agree. We should move internal code away from finalization and also give users viable alternatives that would make finalize() method obsolete one day.



+1. 

Mandy

From thomas.schatzl at oracle.com  Fri Jul  3 09:38:18 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Fri, 03 Jul 2015 11:38:18 +0200
Subject: RFR: JDK-8086056: ParNew: auto-tune ParGCCardsPerStrideChunk
In-Reply-To: <etPan.5595937a.283b46fd.386@tw-mbp-tprintezis>
References: <etPan.5581f51b.699a1ea0.1215a@tw-mbp-tprintezis>
	<55826D13.1030200@oracle.com>
	<etPan.5582cc8d.1bcb7375.1215a@tw-mbp-tprintezis>
	<5587B9F6.3010306@oracle.com>
	<etPan.558986ff.16ab4e1c.1215a@tw-mbp-tprintezis>
	<etPan.558af9e9.4affa36.1215a@tw-mbp-tprintezis>
	<558BB5A8.3060002@oracle.com>
	<etPan.558c022e.23da99d5.1215a@tw-mbp-tprintezis>
	<558CF399.1090705@oracle.com>
	<etPan.558d5405.68777fa0.1215a@tw-mbp-tprintezis>
	<1435325859.2599.11.camel@oracle.com>
	<etPan.55946411.211dea1b.289@tw-mbp-tprintezis>
	<1435826616.2615.18.camel@oracle.com>
	<etPan.5595937a.283b46fd.386@tw-mbp-tprintezis>
Message-ID: <1435916298.2565.5.camel@oracle.com>

Hi Tony,

On Thu, 2015-07-02 at 15:39 -0400, Tony Printezis wrote:
> Thomas,
> 
> 
> Latest version:
> 
> 
> http://cr.openjdk.java.net/~tonyp/8086056/webrev.3/

  looks good, thanks for your changes. I will push on Monday if there
are no further objections.
> 
> See below for comments.
> 
> On July 2, 2015 at 4:43:42 AM, Thomas Schatzl
> (thomas.schatzl at oracle.com) wrote:
> 
> > 
> > Hi Tony, 
> > 
> > On Wed, 2015-07-01 at 18:05 -0400, Tony Printezis wrote: 
> > > Latest changes, as discussed with Bengt and Thomas, 
> > > 
> > > http://cr.openjdk.java.net/~tonyp/8086056/webrev.2/ 
> > 
> > a few suggestions: 
> > 
> > - please use braces even for simple statements in if-clauses, e.g. 
> > 
> > parNewGeneration.cpp: 
> > 
> > 883 if (!UseDynamicParGCStrides) return; 
> 
> I did that. But I actually added an extra verbose statement in the
> body of the if-statement (I thought it?d be helpful to also print the
> value of ParGCCardsPerStrideChunk when the dynamic calculation is
> off). So, I don?t have a simple return any more, but I?ll keep that in
> mind in the future (was it anywhere else? I didn?t immediately spot
> another occurrence of that).

Okay :)
> > 
> > - parameter calculation in
> > ParNewGeneration::adjust_cards_per_stride: 
> > - DynamicParGCStridesMinSize should be > 0 (only if ! 
> > UseDynamicParGCStrides is enabled, not sure if that can be encoded 
> > within the framework) 
> 
> If !UseDynamicParGCStrides, then both sets of min / max parameters are
> not used. I think it?d make the constraint checking a pain having to
> sanity-check parameters whether a flag is on or off (unless they mean
> different things in either case, of course).
> 
> I added a range check to the min / max stride size args, added a lower
> bound of 1, and decided to add an artificial upper bound of 32K (I
> have tested with up to 8K). Is that reasonable? We can revisit that if
> we need to increse it. Note that you can still set
> ParGCCardsPerStrideChunk to something higher if required.

I agree.
> > 
> > - the issues with the Dynamic*Capacity variables Sangheon
> > mentioned, 
> > although I think it is fine that they are equal, just that "r" needs
> > to be set correctly. 
> 
> Yeah, fixed. (See previous e-mail.)
> > 
> > Just wanting to check back if it was intended that the code always 
> > rounds down, achieving the maximum stride only at the end of the 
> > interval. E.g. 
> > 
> > 912 const int stride_size_log = log2_long((jlong) stride_size); 
> > 913 res = (size_t) 1 << stride_size_log; 
> > 
> > It's fine with me, just asking. 
> 
> I did the simplest calculation. I can revise it if you want. Also note
> that if capacity > max_capacity, then stride size will be set to the
> max stride size.

No, it's fine, just asking.

Thanks,
  Thomas





From stefan.johansson at oracle.com  Fri Jul  3 12:48:07 2015
From: stefan.johansson at oracle.com (Stefan Johansson)
Date: Fri, 03 Jul 2015 14:48:07 +0200
Subject: RFR: 8129417: Oop iteration clean-up to remove
	oop_ms_follow_contents
In-Reply-To: <55955020.1000504@oracle.com>
References: <5588217F.9090201@oracle.com> <5592B8DC.4000903@oracle.com>
	<55955020.1000504@oracle.com>
Message-ID: <55968487.20606@oracle.com>

Hi all,

Please disregard:
http://cr.openjdk.java.net/~sjohanss/8129417/2-add-oop_iterate_size/hotspot.00-01/ 


This fix was incorrect and will not be part of the change.

Stefan

On 2015-07-02 16:52, Stefan Johansson wrote:
> Thansk for reviewing this Per,
>
> New full webrev:
> http://cr.openjdk.java.net/~sjohanss/8129417/hotspot.01
>
> Incremental webrevs:
> http://cr.openjdk.java.net/~sjohanss/8129417/1-remove-oop_ms_follow_contents/hotspot.00-01/ 
>
> http://cr.openjdk.java.net/~sjohanss/8129417/2-add-oop_iterate_size/hotspot.00-01/ 
>
> http://cr.openjdk.java.net/~sjohanss/8129417/3-re-structure-mark-sweep/hotspot.00-01/ 
>
> http://cr.openjdk.java.net/~sjohanss/8129417/4-compiler-hints-for-inlining/hotspot.00-01/ 
>
>
> See comments inline.
>
> On 2015-06-30 17:42, Per Liden wrote:
>> Hi Stefan,
>>
>> I think this looks good in general, some comments below. Question: 
>> any plans to address the remaining oopDesc::ms_* and oopDesc::pc_* 
>> parts, or do you see anything that would stop us from converting 
>> those too in the future?
> I think this is cleanup that we also want to do, especially the ms_*. 
> Haven't looked as much at the pc_* parts but I guess they cloud be 
> handled in a similar way.
>
>>
>> On 2015-06-22 16:53, Stefan Johansson wrote:
>>> Hi,
>>>
>>> Please review these changes for RFE:
>>> https://bugs.openjdk.java.net/browse/JDK-8129417
>>>
>>> Webrev for the full change:
>>> http://cr.openjdk.java.net/~sjohanss/8129417/hotspot.00/
>>>
>>> Summary:
>>> To allow further cleanups and later remove G1 specific code from
>>> mark-sweep, we want to remove the mark-sweep specific visitor
>>> oop_ms_follow_contents. Doing this cleanup has proven to be a little
>>> more complicated than I first anticipated and to make the change easier
>>> to review I've split it into four different parts.
>>>
>>> Part 1 - removing oop_ms_follow_contents:
>>> http://cr.openjdk.java.net/~sjohanss/8129417/1-remove-oop_ms_follow_contents/hotspot.00/ 
>>>
>>
>> iterator.hpp:
>> -------------
>> - I can't see any reason for MetadataAwareOopClosure::do_klass() to 
>> be in iterator.inline.hpp, so I'd suggest we move it to iterator.hpp.
>>
> Moved and also did some re-arranging in iterator.hpp.
>
>>
>> instanceMirrorKlass.inline.hpp
>> ------------------------------
>> - A little typo in comment on line 64, "sowhen ..."
>>
> Fixed.
>
>> - Looks like we lost this assert when code was moved from markSweep.cpp:
>>
>>     // If klass is NULL then this a mirror for a primitive type.
>>     // We don't have to follow them, since they are handled as strong
>>     // roots in Universe::oops_do.
>>     assert(java_lang_Class::is_primitive(obj), "Sanity check");
>>
> Added back.
>
>>
>> specialized_oop_closures.hpp
>> ----------------------------
>> - I'd prefer if the forwarding declaration comes after the DefNew 
>> part rather then in the middle of the ParNew/CMS parts.
>>
> Fixed.
>
>>
>>>
>>>
>>> This is more or less what the RFE is all about, the other changes are
>>> needed to avoid introducing a regression. To be able to remove the
>>> mark-sweep specific visitor oop_ms_follow_contents the generic visitor
>>> oop_oop_iterate for all *Klass classes needs to be able to handle the
>>> mark-sweep case correctly. This is done by adding more functionality to
>>> the Devirtualizer and make use of it in the generic visitors.
>>>
>>> The MarkAndPushClosure is also added to the set of closures that are
>>> specialized to avoid using virtual calls during the iteration.
>>> ---
>>>
>>> Part 2 - introducing oop_iterate_size:
>>> http://cr.openjdk.java.net/~sjohanss/8129417/2-add-oop_iterate_size/hotspot.00/ 
>>>
>>
>> space.cpp
>> ---------
>> - In ContiguousSpaceDCTOC::walk_mem_region_with_cl() we should be 
>> using oop_iterate_size() in the loop as well and get rid of the 
>> oop->size() call.
>>
> Fixed.
>
>>
>>>
>>>
>>> As stated above the cleanup to remove oop_ms_follow_content give a
>>> slight regression. One reason for this regression was the fact the
>>> oop_ms_follow_content didn't care about calculating the size for the
>>> object it iterated, but the generic visitors do. When looking through
>>> the code base it is clear that only a subset of the users of
>>> oop_iterate() really cares about the size and to solve this regression
>>> and make it possible to iterate without calculating the size I've added
>>> oop_iterate_size(). It is implemented using oop_iterate() (now void) 
>>> but
>>> also calculates the size.
>>> ---
>>>
>>> Part 3 - restructure mark-sweep:
>>> http://cr.openjdk.java.net/~sjohanss/8129417/3-re-structure-mark-sweep/hotspot.00/ 
>>>
>>
>> markSweep.hpp
>> -------------
>> - It looks like MarkSweep has a lot of functions that should be 
>> private now. I didn't check them all, but here's some that I think 
>> could be made private.
>>
>>   static void mark_object(oop obj);
>>   template <class T> static inline void follow_root(T* p);
>>   static inline void push_objarray(oop obj, size_t index);
>>   static void follow_stack();
>>   static void follow_object(oop obj);
>>   static void follow_array(objArrayOop array, int index);
>>
> Made all but follow_stack private (it's used by PSMarkSweep).
>
>>
>> - In MarkSweep::follow_stack/follow_root, could we move the 
>> is_objArray() check into follow_object() instead and void the 
>> duplication?
>>
> Did this, but it gives a small regression on linux_x64. Not entirely 
> sure what to do with this, will do yet another re-run with all changes.
>
>>>
>>>
>>> Both a cleanup and a way to avoid part of the regression on some
>>> platforms. We want to avoid having too much code in the inline.hpp
>>> files, but we also want all compilers to be able to inline call in a
>>> good manor. This part of the change moves a lot of code from
>>> markSweep.inline.hpp to markSweep.cpp and also try to structure it to
>>> allow good inlining.
>>> ---
>>>
>>> Part 4 - compiler hints for inlining:
>>> http://cr.openjdk.java.net/~sjohanss/8129417/4-compiler-hints-for-inlining/hotspot.00/ 
>>>
>>
>> stack.inline.hpp
>> ----------------
>> - Please #undef NOINLINE somewhere in the end of that file.
>>
> Fixed.
>
> Thanks,
> Stefan
>
>> cheers,
>> /Per
>>
>>>
>>>
>>> To avoid having regressions it seems to be very important that certain
>>> code-paths are correctly inline. To achieve this we need to hint the
>>> compiler in some parts of the code.
>>>
>>> The change does two things:
>>> * Avoid inlining the slow-path for Stack::push() when using gcc
>>> * Add always_inline for Solaris in instanceKlass.inline.hpp (same as
>>> what is already present for Windows compiler)
>>> ---
>>>
>>> Testing:
>>> * Adhoc RBT run for functionality - no new failures.
>>> * Adhoc aurora.se run for performance - no obvious regression.
>>> * Many local runs tuned to do Full GCs a lot to verify that there is no
>>> obvious regression.
>>>
>>> Thanks,
>>> Stefan
>



From ionutb83 at yahoo.com  Sat Jul  4 10:51:51 2015
From: ionutb83 at yahoo.com (Ionut)
Date: Sat, 4 Jul 2015 10:51:51 +0000 (UTC)
Subject: Dirty Cards Write Barrier
Message-ID: <542048556.2359614.1436007111298.JavaMail.yahoo@mail.yahoo.com>

Hello Everybody,
? I have just joined this group and since last year I started to study HotSpot more in detail. ?I have a question for you, so your input will be greatly appreciated.
? Based on my understanding Dirty Cards Write Barrier is a way of keeping track of references from Old to Young Generation. ?This technique is used during minor GC to search for objects referred from Old generation.?? Is this technique used by all minor GC: Serial, Parallel, Parallel Old, CMS, G1? Are there any other Write Barrier techniques?
Regards,Ionut Balosin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150704/40a39979/attachment.htm>

From sangheon.kim at oracle.com  Mon Jul  6 08:19:14 2015
From: sangheon.kim at oracle.com (sangheon.kim)
Date: Mon, 06 Jul 2015 01:19:14 -0700
Subject: RFR: JDK-8086056: ParNew: auto-tune ParGCCardsPerStrideChunk
In-Reply-To: <etPan.55959095.76507531.386@tw-mbp-tprintezis>
References: <etPan.5581f51b.699a1ea0.1215a@tw-mbp-tprintezis>
	<55826D13.1030200@oracle.com>
	<etPan.5582cc8d.1bcb7375.1215a@tw-mbp-tprintezis>
	<5587B9F6.3010306@oracle.com>
	<etPan.558986ff.16ab4e1c.1215a@tw-mbp-tprintezis>
	<etPan.558af9e9.4affa36.1215a@tw-mbp-tprintezis>
	<558BB5A8.3060002@oracle.com>
	<etPan.558c022e.23da99d5.1215a@tw-mbp-tprintezis>
	<558CF399.1090705@oracle.com>
	<etPan.558d5405.68777fa0.1215a@tw-mbp-tprintezis>
	<1435325859.2599.11.camel@oracle.com>
	<etPan.55946411.211dea1b.289@tw-mbp-tprintezis>
	<55947FD0.7070208@oracle.com>
	<etPan.55959095.76507531.386@tw-mbp-tprintezis>
Message-ID: <559A3A02.3070301@oracle.com>

Hi Tony,

On 07/02/2015 12:27 PM, Tony Printezis wrote:
> Sangheon,
>
> Thanks, well spotted. I did what Thomas suggested: allow the diff to 
> be 0 and make sure r is 0.0 when that happens.
Okay.

>
> FYA: I tried to cause the issue by setting min_capacity == 
> max_capacity and I couldn?t. Note the start of that method: the 
> calculation only takes place when min_capacity <= capacity <= 
> max_capacity. So, if min_capacity == max_capacity, both have to be == 
> capacity for the calculation to happen which will cause the div-by-0. :-)
Oh, I see.
Thanks for the result!

Sangheon


>
> Anyway, thanks again for bringing this up,
>
> Tony
>
> On July 1, 2015 at 8:03:32 PM, sangheon.kim (sangheon.kim at oracle.com 
> <mailto:sangheon.kim at oracle.com>) wrote:
>
>> Hi Tony,
>>
>> I just looked the calculation routine and if we set 
>> 'DynamicParGCStridesMinOldGenCapacity = 
>> DynamicParGCStridesMaxOldGenCapacity', division by zero would happen.
>> So if I understand correctly, the constraint of 'max' should be 
>> always greater than 'min', isn't it?
>>
>> http://cr.openjdk.java.net/~tonyp/8086056/webrev.2/src/share/vm/gc/cms/parNewGeneration.cpp.frames.html
>>
>>  885   const size_t min_old_gen_capacity = 
>> DynamicParGCStridesMinOldGenCapacity;
>>  886   const size_t max_old_gen_capacity = 
>> DynamicParGCStridesMaxOldGenCapacity;
>>
>>  902     const size_t capacity_diff = max_old_gen_capacity - 
>> min_old_gen_capacity;
>>  903     const size_t capacity_offset = capacity - min_old_gen_capacity;
>>  904     const double r = (double) capacity_offset / (double) 
>> *capacity_diff**;*
>>
>> Thanks,
>> Sangheon
>>
>>
>> On 07/01/2015 03:05 PM, Tony Printezis wrote:
>>> Latest changes, as discussed with Bengt and Thomas,
>>>
>>> http://cr.openjdk.java.net/~tonyp/8086056/webrev.2/ 
>>> <http://cr.openjdk.java.net/%7Etonyp/8086056/webrev.2/>
>>>
>>> Tony
>>>
>>> On June 26, 2015 at 9:37:45 AM, Thomas Schatzl 
>>> (thomas.schatzl at oracle.com <mailto:thomas.schatzl at oracle.com>) wrote:
>>>
>>>> Hi Tony,
>>>>
>>>> On Fri, 2015-06-26 at 09:30 -0400, Tony Printezis wrote:
>>>>
>>>> [...]
>>>> > >
>>>> > >
>>>> > > The JEP says:
>>>> > >
>>>> > > "The range and constraints checks are done every time a flag
>>>> > > changes, as well as late in the JVM initialization routine 
>>>> (i.e., in
>>>> > > init_globals() after stubRoutines_init2()) at the time when all
>>>> > > flags have their final values set. We will continue to check the
>>>> > > manageable flags as long as the JVM runs."
>>>> > >
>>>> > > I haven't actively been using this myself, but I assume that the
>>>> > > constraints will be checked every time a managaeble flag is changed
>>>> > > and that the one trying to change it will get an error reported 
>>>> back
>>>> > > if they try to change it in an invalid way.
>>>> > >
>>>> > > I'm copying Gerard on this email. He has built this support and can
>>>> > > surely answer the details.
>>>> >
>>>> > Hi, thanks. I?ll definitely look at it and change the code
>>>> > appropriately. BTW, has the change been integrated into the hs gc 
>>>> repo
>>>> > yet? (I don?t see it so I assume it hasn't.)
>>>>
>>>> You overlooked that a week or two ago we started to try out merging the
>>>> hs-rt and hs-gc repos to test whether doing so is beneficial in 
>>>> terms of
>>>> maintenance etc.
>>>>
>>>> So the hs-gc repo is currently discontinued. Please rebase your changes
>>>> to hs-rt. :)
>>>>
>>>> [...]
>>>> > >
>>>> > > Finally, just a heads up. I will be on vacation for the coming 
>>>> three
>>>> > > weeks. I doubt that I will be able to review any new version during
>>>> > > that time. If you get reviews from others I'm perfectly fine with
>>>> > > this being pushed. Depending on how different the changes that you
>>>> > > push are compared to the versions I've reviewed I'll leave it up to
>>>> > > you to decide whether you want to list me as a reviewer or not. I'm
>>>> > > fine either way. It's not a problem to leave me out of the reviewer
>>>> > > list if you are unsure.
>>>> >
>>>> > OK, thanks for letting me know! And enjoy your time off!!!!
>>>>
>>>> I will take over :)
>>>>
>>>> Thanks,
>>>> Thomas
>>>>
>>>>
>>>>
>>> -----
>>>
>>> Tony Printezis | JVM/GC Engineer / VM Team | Twitter
>>>
>>> @TonyPrintezis
>>> tprintezis at twitter.com <mailto:tprintezis at twitter.com>
>>>
>>
> -----
>
> Tony Printezis | JVM/GC Engineer / VM Team | Twitter
>
> @TonyPrintezis
> tprintezis at twitter.com <mailto:tprintezis at twitter.com>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150706/a274905b/attachment.htm>

From thomas.schatzl at oracle.com  Mon Jul  6 10:41:11 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 06 Jul 2015 12:41:11 +0200
Subject: RFR (XS): 8129977: TestSummarizeRSetStats.java fails: Incorrect
	amount of per-period RSet summaries at the end
In-Reply-To: <55957829.3010706@oracle.com>
References: <55957829.3010706@oracle.com>
Message-ID: <1436179271.2568.6.camel@oracle.com>

Hi Daniel,

On Thu, 2015-07-02 at 11:43 -0600, Daniel D. Daugherty wrote:
> > http://cr.openjdk.java.net/~tschatzl/8129977/webrev/
> 
> test/gc/g1/TestSummarizeRSetStats.java
> test/gc/g1/TestSummarizeRSetStatsPerRegion.java
>     No comments on the @bug additions.
> 
> test/gc/g1/TestSummarizeRSetStatsTools.java
>     '-Xms' addition looks fine.
> 
> Thumbs up!

  thanks a lot. (A bit late here with my response, I pushed last week
already).

Thanks,
  Thomas




From thomas.schatzl at oracle.com  Mon Jul  6 10:46:53 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 06 Jul 2015 12:46:53 +0200
Subject: RFR (M): 8129558: Coalesce dead objects during removal of
	self-forwarded pointers
In-Reply-To: <1435589501.2700.27.camel@oracle.com>
References: <1435219625.2431.22.camel@oracle.com>
	<558C0006.7020408@oracle.com> <1435246215.2431.69.camel@oracle.com>
	<etPan.558c2db9.578e5c3b.1215a@tw-mbp-tprintezis>
	<1435256305.2201.3.camel@oracle.com>
	<etPan.558c5f3c.6f01ff09.1215a@tw-mbp-tprintezis>
	<1435264558.5300.9.camel@oracle.com>
	<etPan.558c75ad.3a5de73e.1215a@tw-mbp-tprintezis>
	<1435352543.4283.53.camel@oracle.com> <558DC6CB.8010406@oracle.com>
	<1435589501.2700.27.camel@oracle.com>
Message-ID: <1436179613.2568.7.camel@oracle.com>

Hi all,

On Mon, 2015-06-29 at 16:51 +0200, Thomas Schatzl wrote:
> Hi,
> 
> On Fri, 2015-06-26 at 17:40 -0400, Tom Benson wrote:
> > Hi Thomas,
> > Looks good to me.  Just an optional nit:  New comment says "eventually 
> > also update...".  I wondered if you meant "conditionally...", otherwise 
> > I'm not sure why "eventually."
> 
>   I will remove that word. :)

Done here:

http://cr.openjdk.java.net/~tschatzl/8129558/webrev.3 (full)
http://cr.openjdk.java.net/~tschatzl/8129558/webrev.2_to_3 (diff)

if nobody objects I would like to push this change then asap.

Thanks,
  Thomas




From thomas.schatzl at oracle.com  Mon Jul  6 11:51:05 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 06 Jul 2015 13:51:05 +0200
Subject: RFR: JDK-8086056: ParNew: auto-tune ParGCCardsPerStrideChunk
In-Reply-To: <558D6128.6020501@oracle.com>
References: <etPan.5581f51b.699a1ea0.1215a@tw-mbp-tprintezis>
	<55826D13.1030200@oracle.com>
	<etPan.5582cc8d.1bcb7375.1215a@tw-mbp-tprintezis>
	<5587B9F6.3010306@oracle.com>
	<etPan.558986ff.16ab4e1c.1215a@tw-mbp-tprintezis>
	<etPan.558af9e9.4affa36.1215a@tw-mbp-tprintezis>
	<558BB5A8.3060002@oracle.com>
	<etPan.558c022e.23da99d5.1215a@tw-mbp-tprintezis>
	<558CF399.1090705@oracle.com>
	<etPan.558d5405.68777fa0.1215a@tw-mbp-tprintezis>
	<558D6128.6020501@oracle.com>
Message-ID: <1436183465.2568.30.camel@oracle.com>

Hi Gerard,

On Fri, 2015-06-26 at 09:26 -0500, Gerard Ziemski wrote:
> hi guys,
> 
> On 6/26/2015 8:30 AM, Tony Printezis wrote:
> >
> > The JEP says:
> >
> > "The range and constraints checks are done every time a flag changes, 
> > as well as late in the JVM initialization routine (i.e., in 
> > init_globals() after stubRoutines_init2()) at the time when all flags 
> > have their final values set. We will continue to check the manageable 
> > flags as long as the JVM runs."
> >
> > I haven't actively been using this myself, but I assume that the 
> > constraints will be checked every time a managaeble flag is changed 
> > and that the one trying to change it will get an error reported back 
> > if they try to change it in an invalid way.
> 
> The JEP required that in the case of manageable flags, when an attempt 
> is made to change value of a flag from outside the Java process, that an 
> error message be printed in the outside process and the value be left 
> unchanaged.
> 
> If the Java process itself tries to make such change, however, then the 
> error message is printed in the process itself and the VM exits.
> 
> There is a window of time, however, during the initialization process, 
> when flags are being still set, so during that time the range/constraint 
> check is disabled, but changes after Arguments::apply_ergo() are fatal 
> to the VM if check fails.

  currently looking at some failures in the TestOptionsWithRangesDynamic
jtreg test. The test seems to fail because it tries to set the various
options independently of the options it depends on (in the constraint
functions).

E.g. sets DynamicParGCStridesMaxSize to 1, which then fails ("Can not
change option to avlid value "1" via jcmd") because of "value violates
its flags constraints" which include DynamicParGCStridesMinSize, which
default value is 256, making the max size < min size.

What is the recommened way to fix this problem?

Thanks,
  Thomas




From volker.simonis at gmail.com  Mon Jul  6 13:03:05 2015
From: volker.simonis at gmail.com (Volker Simonis)
Date: Mon, 6 Jul 2015 15:03:05 +0200
Subject: RFR(XS): JDK-8129440 G1 crash during concurrent root region scan.
In-Reply-To: <02D5D45C1F8DB848A7AE20E80EE61A5C39945B54@DEWDFEMB20C.global.corp.sap>
References: <02D5D45C1F8DB848A7AE20E80EE61A5C39945B54@DEWDFEMB20C.global.corp.sap>
Message-ID: <CA+3eh12JaZRDsvmbtssi7wqAsbR7kO3F2_j6vjCWqbu0LBJwdA@mail.gmail.com>

On Tue, Jun 23, 2015 at 8:45 AM, Siebenborn, Axel
<axel.siebenborn at sap.com> wrote:
>
>
> Hi Volker,
>
> thanks for your comment.Indeed, this was my first idea for a fix. However,
> there are more than 50 callers of  load_heap_oop, plus some generated by
> preprocessor defines like this one:
>
>
>
> #define DO_OOP_WORK_IMPL(closureName) \
>
>   template <class T> inline void closureName##Closure::do_oop_work(T* p) {
> \
>
>     T heap_oop = oopDesc::load_heap_oop(p);               \
>
>     if (!oopDesc::is_null(heap_oop)) {                    \
>
>       oop obj = oopDesc::decode_heap_oop_not_null(heap_oop);       \
>
>       do_oop(obj);                                        \
>
>     }                                                     \
>
>   }
>
>
>
> We would have to decide for each of the callers, if there might be a
> concurrency problem or not.
>
>
>
> Do you think, it is worth the effort and the risk of missing one?
>

OK, you're probably right.

If you haven?t seen any performance regressions caused by this fix I'm
fine with your solution.

Unfortunately I can not push this shared-code change so can we please
get a sponsor from within Oracle for this change?

Thanks,
Volker

>
> Regards,
>
> Axel
>
>
>
>
>
>
>
>
>
> On 22.06.2015 at 18:17 Volker Simonis wrote:
>
>> Hi Axel,
>
>>
>
>> the change looks good, but shouldn't we better provide two versions of
>
>> load_heap_oop()
>
>>
>
>> inline oop       oopDesc::load_heap_oop(oop* p)
>
>> inline oop       oopDesc::load_heap_oop(volatile oop* p)
>
>>
>
>> such that the callers can decide on their own which versions they require?
>
>>
>
>> Or do the callers of load_heap_oop() not always know if there exist
>
>> concurrent mutator threads?
>
>>
>
>> Regards,
>
>> Volker
>
>>
>
>>
>
>> On Mon, Jun 22, 2015 at 5:33 PM, Siebenborn, Axel
>
>> <axel.siebenborn at sap.com> wrote:
>
>>> Hi,
>
>>> we have seen crashes with G1 during concurrent root region scan.
>
>>> It turned out, that the reason for the crashes was reloading an oop after
>>> a null check.
>
>>> The compiler may legally do this, as the pointer is not declared as
>>> volatile.
>
>>> The code runs concurrently to mutator threads.
>
>>>
>
>>> template <class T>
>
>>> inline void G1RootRegionScanClosure::do_oop_nv(T* p) {
>
>>>   T heap_oop = oopDesc::load_heap_oop(p);
>>> // 1. Load oop
>
>>>   if (!oopDesc::is_null(heap_oop)) {
>
>>>     oop obj = oopDesc::decode_heap_oop_not_null(heap_oop);    // 2.
>>> Compiler decides to re-load the oop
>
>>>     HeapRegion* hr = _g1h->heap_region_containing((HeapWord*) obj);
>
>>>     _cm->grayRoot(obj, obj->size(), _worker_id, hr);
>
>>>   }
>
>>> }
>
>>>
>
>>>
>
>>> We have seen the problem on AIX with the xlC compiler. However, we have
>>> seen similar optimizations on other platforms and other platforms.
>
>>>
>
>>> As this code pattern is used quite often, I would suggest a global fix in
>>> oopDesc::load_heap_oop(p).
>
>>>
>
>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8129440
>
>>> Webrev: http://cr.openjdk.java.net/~asiebenborn/8129440/webrev.00/
>
>>>
>
>>> Thanks,
>
>>> Axel
>
>>>
>
>>>
>
>>>


From erik.helin at oracle.com  Mon Jul  6 15:15:37 2015
From: erik.helin at oracle.com (Erik Helin)
Date: Mon, 6 Jul 2015 17:15:37 +0200
Subject: RFR: 8129417: Oop iteration clean-up to remove
	oop_ms_follow_contents
In-Reply-To: <5588217F.9090201@oracle.com>
References: <5588217F.9090201@oracle.com>
Message-ID: <20150706151537.GP3266@ehelin.jrpg.bea.com>

Hi Stefan,

thanks a lot for taking on this work. Please see comments inline.

On 2015-06-22, Stefan Johansson wrote:
> Hi,
> 
> Please review these changes for RFE:
> https://bugs.openjdk.java.net/browse/JDK-8129417
> 
> Webrev for the full change:
> http://cr.openjdk.java.net/~sjohanss/8129417/hotspot.00/
> 
> Summary:
> To allow further cleanups and later remove G1 specific code from mark-sweep,
> we want to remove the mark-sweep specific visitor oop_ms_follow_contents.
> Doing this cleanup has proven to be a little more complicated than I first
> anticipated and to make the change easier to review I've split it into four
> different parts.
> 
> Part 1 - removing oop_ms_follow_contents:
> http://cr.openjdk.java.net/~sjohanss/8129417/1-remove-oop_ms_follow_contents/hotspot.00/

- Looking in specialized_oop_closures.hpp, the comment
   71 // This is split into several because of a Visual C++ 6.0 compiler bug
   72 // where very long macros cause the compiler to crash
  seems a bit dated, given that Visual C++ 6.0 was released in 1998. I
  think we should try to merge ALL_OOP_OOP_ITERATE_CLOSURES_1 and 2 (and
  their corresponding macros). Do you think like doing that in this
  patch or do you want to file a follow-up bug?

- The part of the patch that changes instanceMirrorKlass.inline.hpp
  might impact more GCs than just any of the marksweep ones. Have you
  seen any performance improvements/regressions with G1 or CMS?

> Part 2 - introducing oop_iterate_size:
> http://cr.openjdk.java.net/~sjohanss/8129417/2-add-oop_iterate_size/hotspot.00/

- In arrayKlass.hpp, why add a space to:
  +#define OOP_OOP_ITERATE_DECL_NO_BACKWARDS(OopClosureType, nv_suffix)         \

- In arrayKlass.hpp, the following lines seems aligned strangely:
  +void KlassType::oop_oop_iterate_range##nv_suffix(oop obj, OopClosureType* closure, int start, int end) {  \
  +  oop_oop_iterate_range<nvs_to_bool(nv_suffix)>(obj, closure, start, end);                               \

- In oop.inline.hpp, why must the size be computed before applying the
  closure? In the world of perm gen, this might have been important in
  order to not follow stale klass pointers, but with Metaspace the Klass*
  won't move.

- Can we remove the method MutableSpace::oop_iterate?

> Part 3 - restructure mark-sweep:
> http://cr.openjdk.java.net/~sjohanss/8129417/3-re-structure-mark-sweep/hotspot.00/

- Can you please remove the commented assert in mark_and_push:
   +//  assert(Universe::heap()->is_in_reserved(p), "should be in object space");
  or should it be enabled again?

- I agree with Per about moving the check for object arrays into
  follow_object.

> Part 4 - compiler hints for inlining:
> http://cr.openjdk.java.net/~sjohanss/8129417/4-compiler-hints-for-inlining/hotspot.00/

- I would like to see the definiton of FORCE_INLINE in the compiler
  specific globalDefinitions file. Then the code in
  instanceKlass.inline.hpp can look like:
   #if defined(TARGET_COMPILER_visCPP) || defined(TARGET_COMPILER_sparcWorks)
     #define INLINE FORCE_INLINE
   #else
     #define INLINE inline
   #endif

- The same comment for stack.inline.hpp, but with NO_INLINE:
  #if defined(TARGET_COMPILER_sparcWorks)
    #define STACK_NO_INLINE NO_INLINE
  #else
    #define STACK_NO_INLINE
  #endif

- I agree with Per about adding an #undef in stack.inline.hpp

Thanks!
Erik

> Testing:
> * Adhoc RBT run for functionality - no new failures.
> * Adhoc aurora.se run for performance - no obvious regression.
> * Many local runs tuned to do Full GCs a lot to verify that there is no
> obvious regression.
> 
> Thanks,
> Stefan


From dmitry.dmitriev at oracle.com  Mon Jul  6 16:18:19 2015
From: dmitry.dmitriev at oracle.com (Dmitry Dmitriev)
Date: Mon, 6 Jul 2015 19:18:19 +0300
Subject: RFR: JDK-8086056: ParNew: auto-tune ParGCCardsPerStrideChunk
In-Reply-To: <559A9BE2.9030807@oracle.com>
References: <etPan.5581f51b.699a1ea0.1215a@tw-mbp-tprintezis>
	<55826D13.1030200@oracle.com>
	<etPan.5582cc8d.1bcb7375.1215a@tw-mbp-tprintezis>
	<5587B9F6.3010306@oracle.com>
	<etPan.558986ff.16ab4e1c.1215a@tw-mbp-tprintezis>
	<etPan.558af9e9.4affa36.1215a@tw-mbp-tprintezis>
	<558BB5A8.3060002@oracle.com>
	<etPan.558c022e.23da99d5.1215a@tw-mbp-tprintezis>
	<558CF399.1090705@oracle.com>
	<etPan.558d5405.68777fa0.1215a@tw-mbp-tprintezis>
	<558D6128.6020501@oracle.com> <1436183465.2568.30.camel@oracle.com>
	<559A9BE2.9030807@oracle.com>
Message-ID: <559AAA4B.4060209@oracle.com>

Hi Gerard and Thomas,

Yes, we need special case in this case. In case of dynamic options the 
simplest way is just add this depenency to the test command line. Test 
located here:
test/runtime/CommandLine/OptionsValidation/TestOptionsWithRangesDynamic.java

If you open it, you can found following line:

* @run main/othervm -XX:MinHeapFreeRatio=0 -XX:MaxHeapFreeRatio=100 
TestOptionsWithRangesDynamic

So, this line already contains special case for 
MinHeapFreeRatio/MaxHeapFreeRatio. So, you can add 
DynamicParGCStridesMaxSize and DynamicParGCStridesMinSize to this 
command line - add DynamicParGCStridesMinSize=<min_value> and 
DynamicParGCStridesMaxSize=<max_value>. As I understand the valid range 
for these flags is range(1, 32*K). Thus, you can chage it to the following:

* @run main/othervm -XX:MinHeapFreeRatio=0 -XX:MaxHeapFreeRatio=100 
-XX:DynamicParGCStridesMinSize=1 -XX:DynamicParGCStridesMaxSize=32768 
TestOptionsWithRangesDynamic

Also, I found your JPRT job and I will try to test this approach. And I 
will think how test for manageable options with dependecies can be 
improved.

Thank you,
Dmitry


On 06.07.2015 18:16, Gerard Ziemski wrote:
> hiThomas,
>
> On 7/6/2015 6:51 AM, Thomas Schatzl wrote:
>> Hi Gerard,
>>
>> On Fri, 2015-06-26 at 09:26 -0500, Gerard Ziemski wrote:
>>> hi guys,
>>>
>>> On 6/26/2015 8:30 AM, Tony Printezis wrote:
>>>> The JEP says:
>>>>
>>>> "The range and constraints checks are done every time a flag changes,
>>>> as well as late in the JVM initialization routine (i.e., in
>>>> init_globals() after stubRoutines_init2()) at the time when all flags
>>>> have their final values set. We will continue to check the manageable
>>>> flags as long as the JVM runs."
>>>>
>>>> I haven't actively been using this myself, but I assume that the
>>>> constraints will be checked every time a managaeble flag is changed
>>>> and that the one trying to change it will get an error reported back
>>>> if they try to change it in an invalid way.
>>> The JEP required that in the case of manageable flags, when an attempt
>>> is made to change value of a flag from outside the Java process, 
>>> that an
>>> error message be printed in the outside process and the value be left
>>> unchanaged.
>>>
>>> If the Java process itself tries to make such change, however, then the
>>> error message is printed in the process itself and the VM exits.
>>>
>>> There is a window of time, however, during the initialization process,
>>> when flags are being still set, so during that time the 
>>> range/constraint
>>> check is disabled, but changes after Arguments::apply_ergo() are fatal
>>> to the VM if check fails.
>>    currently looking at some failures in the 
>> TestOptionsWithRangesDynamic
>> jtreg test. The test seems to fail because it tries to set the various
>> options independently of the options it depends on (in the constraint
>> functions).
>>
>> E.g. sets DynamicParGCStridesMaxSize to 1, which then fails ("Can not
>> change option to avlid value "1" via jcmd") because of "value violates
>> its flags constraints" which include DynamicParGCStridesMinSize, which
>> default value is 256, making the max size < min size.
>>
>> What is the recommened way to fix this problem?
>
> Good question.
>
> I'm including Dmitry, who wrote the testing framework.
>
> Dmitry: do we need special cases in the JTREG test for those flags 
> that have constraints that further limit the available range beyond 
> that of its defined range?Should Thomas jump into the JTREG test code 
> and do this himself? Can you give him some pointers on how and where 
> to insert such additional logic?
>
> The JEP has specifically said that we were not allowed to 
> automatically change a flag's value that violated the range or 
> constraint, even though for ranges it would have been trivial to do 
> so, and even for constraints quite doable. If we were allowed to do 
> that, then this issue wouldn't exist.
>
>
> cheers
>
>



From gerard.ziemski at oracle.com  Mon Jul  6 15:16:50 2015
From: gerard.ziemski at oracle.com (Gerard Ziemski)
Date: Mon, 06 Jul 2015 10:16:50 -0500
Subject: RFR: JDK-8086056: ParNew: auto-tune ParGCCardsPerStrideChunk
In-Reply-To: <1436183465.2568.30.camel@oracle.com>
References: <etPan.5581f51b.699a1ea0.1215a@tw-mbp-tprintezis>	
	<55826D13.1030200@oracle.com>	
	<etPan.5582cc8d.1bcb7375.1215a@tw-mbp-tprintezis>	
	<5587B9F6.3010306@oracle.com>	
	<etPan.558986ff.16ab4e1c.1215a@tw-mbp-tprintezis>	
	<etPan.558af9e9.4affa36.1215a@tw-mbp-tprintezis>	
	<558BB5A8.3060002@oracle.com>	
	<etPan.558c022e.23da99d5.1215a@tw-mbp-tprintezis>	
	<558CF399.1090705@oracle.com>	
	<etPan.558d5405.68777fa0.1215a@tw-mbp-tprintezis>	
	<558D6128.6020501@oracle.com> <1436183465.2568.30.camel@oracle.com>
Message-ID: <559A9BE2.9030807@oracle.com>

hiThomas,

On 7/6/2015 6:51 AM, Thomas Schatzl wrote:
> Hi Gerard,
>
> On Fri, 2015-06-26 at 09:26 -0500, Gerard Ziemski wrote:
>> hi guys,
>>
>> On 6/26/2015 8:30 AM, Tony Printezis wrote:
>>> The JEP says:
>>>
>>> "The range and constraints checks are done every time a flag changes,
>>> as well as late in the JVM initialization routine (i.e., in
>>> init_globals() after stubRoutines_init2()) at the time when all flags
>>> have their final values set. We will continue to check the manageable
>>> flags as long as the JVM runs."
>>>
>>> I haven't actively been using this myself, but I assume that the
>>> constraints will be checked every time a managaeble flag is changed
>>> and that the one trying to change it will get an error reported back
>>> if they try to change it in an invalid way.
>> The JEP required that in the case of manageable flags, when an attempt
>> is made to change value of a flag from outside the Java process, that an
>> error message be printed in the outside process and the value be left
>> unchanaged.
>>
>> If the Java process itself tries to make such change, however, then the
>> error message is printed in the process itself and the VM exits.
>>
>> There is a window of time, however, during the initialization process,
>> when flags are being still set, so during that time the range/constraint
>> check is disabled, but changes after Arguments::apply_ergo() are fatal
>> to the VM if check fails.
>    currently looking at some failures in the TestOptionsWithRangesDynamic
> jtreg test. The test seems to fail because it tries to set the various
> options independently of the options it depends on (in the constraint
> functions).
>
> E.g. sets DynamicParGCStridesMaxSize to 1, which then fails ("Can not
> change option to avlid value "1" via jcmd") because of "value violates
> its flags constraints" which include DynamicParGCStridesMinSize, which
> default value is 256, making the max size < min size.
>
> What is the recommened way to fix this problem?

Good question.

I'm including Dmitry, who wrote the testing framework.

Dmitry: do we need special cases in the JTREG test for those flags that 
have constraints that further limit the available range beyond that of 
its defined range?Should Thomas jump into the JTREG test code and do 
this himself? Can you give him some pointers on how and where to insert 
such additional logic?

The JEP has specifically said that we were not allowed to automatically 
change a flag's value that violated the range or constraint, even though 
for ranges it would have been trivial to do so, and even for constraints 
quite doable. If we were allowed to do that, then this issue wouldn't exist.


cheers




From dmitry.dmitriev at oracle.com  Mon Jul  6 16:42:40 2015
From: dmitry.dmitriev at oracle.com (Dmitry Dmitriev)
Date: Mon, 6 Jul 2015 19:42:40 +0300
Subject: RFR: JDK-8086056: ParNew: auto-tune ParGCCardsPerStrideChunk
In-Reply-To: <559AAA4B.4060209@oracle.com>
References: <etPan.5581f51b.699a1ea0.1215a@tw-mbp-tprintezis>
	<55826D13.1030200@oracle.com>
	<etPan.5582cc8d.1bcb7375.1215a@tw-mbp-tprintezis>
	<5587B9F6.3010306@oracle.com>
	<etPan.558986ff.16ab4e1c.1215a@tw-mbp-tprintezis>
	<etPan.558af9e9.4affa36.1215a@tw-mbp-tprintezis>
	<558BB5A8.3060002@oracle.com>
	<etPan.558c022e.23da99d5.1215a@tw-mbp-tprintezis>
	<558CF399.1090705@oracle.com>
	<etPan.558d5405.68777fa0.1215a@tw-mbp-tprintezis>
	<558D6128.6020501@oracle.com> <1436183465.2568.30.camel@oracle.com>
	<559A9BE2.9030807@oracle.com> <559AAA4B.4060209@oracle.com>
Message-ID: <559AB000.9060101@oracle.com>

I check and test will pass with following line:
* @run main/othervm -XX:MinHeapFreeRatio=0 -XX:MaxHeapFreeRatio=100 
-XX:DynamicParGCStridesMinSize=1 -XX:DynamicParGCStridesMaxSize=32768 
TestOptionsWithRangesDynamic

Also, Tony, I have few comments about patch:

In src/share/vm/runtime/commandLineFlagConstraintsGC.cpp module:
1) All new 4 constraint functions not add new line when print about error.
2) DynamicParGCStridesMinSizeConstraintFunc print value of 
DynamicParGCStridesMaxOldGenCapacity instead of 
DynamicParGCStridesMaxSize(line 81).
3) DynamicParGCStridesMaxSizeConstraintFunc print value of 
DynamicParGCStridesMinOldGenCapacity instead of 
DynamicParGCStridesMinSize(line 97).

Thanks,
Dmitry


On 06.07.2015 19:18, Dmitry Dmitriev wrote:
> Hi Gerard and Thomas,
>
> Yes, we need special case in this case. In case of dynamic options the 
> simplest way is just add this depenency to the test command line. Test 
> located here:
> test/runtime/CommandLine/OptionsValidation/TestOptionsWithRangesDynamic.java 
>
>
> If you open it, you can found following line:
>
> * @run main/othervm -XX:MinHeapFreeRatio=0 -XX:MaxHeapFreeRatio=100 
> TestOptionsWithRangesDynamic
>
> So, this line already contains special case for 
> MinHeapFreeRatio/MaxHeapFreeRatio. So, you can add 
> DynamicParGCStridesMaxSize and DynamicParGCStridesMinSize to this 
> command line - add DynamicParGCStridesMinSize=<min_value> and 
> DynamicParGCStridesMaxSize=<max_value>. As I understand the valid 
> range for these flags is range(1, 32*K). Thus, you can chage it to the 
> following:
>
> * @run main/othervm -XX:MinHeapFreeRatio=0 -XX:MaxHeapFreeRatio=100 
> -XX:DynamicParGCStridesMinSize=1 -XX:DynamicParGCStridesMaxSize=32768 
> TestOptionsWithRangesDynamic
>
> Also, I found your JPRT job and I will try to test this approach. And 
> I will think how test for manageable options with dependecies can be 
> improved.
>
> Thank you,
> Dmitry
>
>
> On 06.07.2015 18:16, Gerard Ziemski wrote:
>> hiThomas,
>>
>> On 7/6/2015 6:51 AM, Thomas Schatzl wrote:
>>> Hi Gerard,
>>>
>>> On Fri, 2015-06-26 at 09:26 -0500, Gerard Ziemski wrote:
>>>> hi guys,
>>>>
>>>> On 6/26/2015 8:30 AM, Tony Printezis wrote:
>>>>> The JEP says:
>>>>>
>>>>> "The range and constraints checks are done every time a flag changes,
>>>>> as well as late in the JVM initialization routine (i.e., in
>>>>> init_globals() after stubRoutines_init2()) at the time when all flags
>>>>> have their final values set. We will continue to check the manageable
>>>>> flags as long as the JVM runs."
>>>>>
>>>>> I haven't actively been using this myself, but I assume that the
>>>>> constraints will be checked every time a managaeble flag is changed
>>>>> and that the one trying to change it will get an error reported back
>>>>> if they try to change it in an invalid way.
>>>> The JEP required that in the case of manageable flags, when an attempt
>>>> is made to change value of a flag from outside the Java process, 
>>>> that an
>>>> error message be printed in the outside process and the value be left
>>>> unchanaged.
>>>>
>>>> If the Java process itself tries to make such change, however, then 
>>>> the
>>>> error message is printed in the process itself and the VM exits.
>>>>
>>>> There is a window of time, however, during the initialization process,
>>>> when flags are being still set, so during that time the 
>>>> range/constraint
>>>> check is disabled, but changes after Arguments::apply_ergo() are fatal
>>>> to the VM if check fails.
>>>    currently looking at some failures in the 
>>> TestOptionsWithRangesDynamic
>>> jtreg test. The test seems to fail because it tries to set the various
>>> options independently of the options it depends on (in the constraint
>>> functions).
>>>
>>> E.g. sets DynamicParGCStridesMaxSize to 1, which then fails ("Can not
>>> change option to avlid value "1" via jcmd") because of "value violates
>>> its flags constraints" which include DynamicParGCStridesMinSize, which
>>> default value is 256, making the max size < min size.
>>>
>>> What is the recommened way to fix this problem?
>>
>> Good question.
>>
>> I'm including Dmitry, who wrote the testing framework.
>>
>> Dmitry: do we need special cases in the JTREG test for those flags 
>> that have constraints that further limit the available range beyond 
>> that of its defined range?Should Thomas jump into the JTREG test code 
>> and do this himself? Can you give him some pointers on how and where 
>> to insert such additional logic?
>>
>> The JEP has specifically said that we were not allowed to 
>> automatically change a flag's value that violated the range or 
>> constraint, even though for ranges it would have been trivial to do 
>> so, and even for constraints quite doable. If we were allowed to do 
>> that, then this issue wouldn't exist.
>>
>>
>> cheers
>>
>>
>



From coleen.phillimore at oracle.com  Mon Jul  6 21:00:48 2015
From: coleen.phillimore at oracle.com (Coleen Phillimore)
Date: Mon, 06 Jul 2015 17:00:48 -0400
Subject: Runtime command-line deprecation (Was Re: RFR: 8066821(S) Enhance
	command line processing to manage deprecating and obsoleting -XX
	command line arguments
In-Reply-To: <87B07B03-3423-4738-9617-0C7B72A74A6E@oracle.com>
References: <54B44661.8070007@oracle.com>
	<54B84E91.1050708@oracle.com>	<F68EBF6B-38A5-4461-9ADD-41BB198FF943@oracle.com>	<54D15A0B.6030501@oracle.com>
	<87B07B03-3423-4738-9617-0C7B72A74A6E@oracle.com>
Message-ID: <559AEC80.3010907@oracle.com>


Hi,

I'm happy for more clarity in the process of removing command line 
arguments.  I have some questions and comments below.

On 6/26/15 4:09 PM, Karen Kinnear wrote:
> Derek,
>
> I am really glad you are looking into this. I expanded this to the runtime folks in case many of them have not yet seen this.
> Mary just forwarded this and I believe you haven't checked it in yet, so perhaps still time to discuss and make sure
> we all are together on the deprecation policy.
>
> The runtime team was discussing at staff this week how we handle deprecating jvm command-line options as
> we are looking to do more clean up in the future.
>
> So our internal change control process classifies our exported interfaces into three categories:
> External: command-line flags e.g. -verbose:gc, -Xmx, ...
>      This includes any commercial flags
> Private: -XX flags documented (e.g. performance tuning or troubleshooting)
>      I would assume this would include -XX product, experimental flags and manageable flags
> Internal: undocumented -XX flags
>      I would assume this would include -XX develop and diagnostic flags
>
> (please correct me if my assumptions are wrong folks)

This is a good categorization.  Although I think there's some grey area 
between External and Private, where some XX options are so commonly used 
they should be considered External.   Some of the GC options may fall 
into this category like UseParNewGC.

>
> The way I understand that we handle private -XX options today is a 2-step removal: (obsolete_jvm_flags - where the
> release number is in a table and could be undefined)
>
> Using your helpful taxonomy from https://bugs.openjdk.java.net/browse/JDK-806682:
>
> Today: private -XX options use 2-step removal (obsolete_jvm_flags)
>      release 1: Deprecate & Obsolete - warn about the option but do nothing with it (we can remove all code that supports it)
>      release 2: Dead - unrecognized
>    - the point of the 2-step is to give customers time to modify any scripts they use
>
> I believe we have very rarely removed External flags - since customers, licensees, etc. may expect them.
>
> Mini-proposal:
> 1) I would recommend that we add a future set of changes to add consistent handling for the External flags -
> so that they would follow a three-step removal:
>      release 1: Deprecate & Handle - warn and keep supporting
>      release 2: Deprecate & Obsolete - warn and do nothing
>      release 3: Dead - unrecognized
>
> 2) For the Internal flags - I think it would be kindest to customers and not a huge amount of additional work if
> we were to follow the Private model of using a 2 step.
>
> 3) leave the Private flags with the current 2-step removal

Yes, this reflects our current model.
>
> 4) add support for aliasing - for any of them
>      So that if you are doing aliasing, you would follow the model you are adding
>       release 1: Deprecated & Handled - i.e. warn and still support (and set the information for the new alias)
>       release 2: Dead - unrecognized
>
> 5) Could we possibly take the two flags that followed a different model already, i.e. moving to
> Deprecated & Handled and handle those as mistakes rather than part of a new general model?

So this is my question which is around the code review in question.   
Note, Derek, can you resend the RFR to hotspot-dev at openjdk.java.net so 
it gets to all of us (even the compiler team may want to share in the 
mechanism).

Why are UseParNewGC and MaxGCMinorPauseMillis deprecated and handled and 
not deprecated and obsolete?  I don't actually see why have the 
distinction here?

Did these flags follow the deprecation procedure below?  Why not just 
make them obsolete, why have this other mechanism?  I may be asking the 
same question as Karen.

I think the aliased flags could have a deprecate and handle model (where 
handle == alias) in which case you only need one table for the aliased 
flags, and you need to keep them in globals.hpp so they're parsed properly.
>
> Or do you think we will see more cases other than aliasing in which we would want to
>      release 1: Deprecate & Handle and then release 2: Dead
>      rather than release 1: Deprecate & Obsolete and then 2: Dead
>      or rather than a 3 step like the External option proposal above?

I think for the aliased flags you want the first option here 
(deprecate&handle), right?  But that's only because you need to keep the 
option in globals.hpp so it parses correctly, otherwise the second 
option (deprecate&obsolete) would be the preference?

The options in the GC that are being deprecated and handled have had 
warnings about them for a while, so making them obsolete doesn't feel 
too soon.

Also, I agree with Kim's comment below that your comment lines are too 
long.  My fonts are too big to have windows this wide.

thanks,
Coleen
>
> thanks,
> Karen
>
> p.s. Note that all of the deprecation needs to
> 1) work with licensee engineering to ensure we give licensee's a head's up and get feedback
> 2) file a change control request
>
> - we try to do these both as bulk requests to reduce the processing overhead.
>
> p.p.s. Details
>
> 1.  Do the warnings print today or are they silent? Just want to make sure we are conscious of how
> those are handled if any of this moves to the new unified logging mechanism for which the new default
> for "warning" level is to print.
>
> 2. "will likely be removed in a future release"? If we have already set the release it will be removed - is this a bit
> vague or did I not read closely enough?
>
>
>
> On Feb 3, 2015, at 6:30 PM, Derek White wrote:
>
>> Request for review (again):
>>
>> - Updated with Kim's suggestion.
>> - Stopped double-printing warnings in some cases.
>> - Initial caps on warning() messages.
>>
>> Webrev:  http://cr.openjdk.java.net/~drwhite/8066821/webrev.04/
>> CR:         https://bugs.openjdk.java.net/browse/JDK-8066821
>> Tests:     jtreg, jprt
>>
>> Thanks for looking!
>>
>> - Derek
>>
>> On 1/28/15 3:47 PM, Kim Barrett wrote:
>>> On Jan 15, 2015, at 6:34 PM, Derek White <derek.white at oracle.com> wrote:
>>>> Hi All,
>>>>
>>>> I need another review, especially someone from runtime. Coleen, do you want crack at it or can you forward as needed?
>>>>
>>>> Another version for review:
>>>> http://cr.openjdk.java.net/~drwhite/8066821/webrev.01
>>> I noticed a bunch of formatting changes that were in the previous
>>> webrev have been eliminated in the .01 webrev.  Since I was going to
>>> comment or complain about at least some of them, I have no objection
>>> to having them backed out.
>>>
>>> This is not a complete review; I've only gotten part way through the
>>> first file!  Unfortunately, I'm swamped with other things right now
>>> and don't seem to be making progress toward finishing.  So here's what
>>> I've got so far.
>>>
>>> ------------------------------------------------------------------------------
>>> src/share/vm/runtime/arguments.cpp
>>>   132     int len = (int)strlen(name);
>>>
>>> Pre-existing issue.
>>>
>>> Why is this using int and casting to int, rather than just using
>>> size_t?  The return type for strlen is size_t, as is the argument for
>>> strncmp where len is used, and the other use of len also can/should be
>>> size_t.
>>>
>>> [I only noticed this because an earlier webrev tweaked the formatting
>>> of this line.]
>>>
>>> ------------------------------------------------------------------------------
>>> src/share/vm/runtime/arguments.cpp
>>>   235     const char* spec_vendor = "Sun Microsystems Inc.";
>>>   236     uint32_t spec_version = 0;
>>>
>>> Dead initializers; the variables are immediately reassigned new
>>> values.
>>>
>>> This is a pre-existing defect that I wouldn't have even noticed had
>>> the enum for bufsz not been reformatted in a previous webrev.  (How's
>>> that for a lesson in being lazy. :-)
>>>
>>> ------------------------------------------------------------------------------
>>> src/share/vm/runtime/arguments.cpp
>>>   253  *  -XX arguments are usually defined in globals.hpp, globals_<cpu>.hpp, globals_<os>.hpp, <compiler>_globals.hpp, or <gc>_globals.hpp.
>>>
>>> Rather than "are usually defined in" I think better would be something
>>> like "are defined several places, such as".
>>>
>>> ------------------------------------------------------------------------------
>>> src/share/vm/runtime/arguments.cpp
>>>   251  *  -XX argument processing:
>>>
>>> While the Hotspot style guidelines explicitly don't specify a line
>>> length limit, I find lines that are long enough that they don't fit in
>>> one side of a side-by-side webrev on a reasonable-sized landscape
>>> monitor with a font size I can read without having to scroll the frame
>>> back and forth to be pretty annoying.
>>>
>>> ------------------------------------------------------------------------------
>>> src/share/vm/runtime/arguments.cpp
>>>   419     if (((strncmp(flag_status.name, s, len) == 0) &&
>>>   420           (strlen(s) == len)) ||
>>>   421          ((s[0] == '+' || s[0] == '-') &&
>>>   422           (strncmp(flag_status.name, &s[1], len) == 0) &&
>>>   423           (strlen(&s[1]) == len))) {
>>>
>>> Pre-existing issue.
>>>
>>> The calls to strlen and comparisons to len on lines 420 and 423 could
>>> be replaced with
>>>
>>> 420:  s[len] == '\0'
>>>
>>> 423:  s[len+1] == '\0'
>>>
>>> ------------------------------------------------------------------------------
>>> src/share/vm/runtime/arguments.cpp
>>>   449     size_t len = strlen(flag_status.alias_name);
>>>   450     if (((strncmp(flag_status.alias_name, flag_name, len) == 0) &&
>>>   451          (strlen(flag_name) == len))) {
>>>
>>> There is no point to using strlen() and strncmp() here.  Just use
>>> strcmp(), e.g.
>>>
>>>    if (strcmp(flag_status.alias_name, flag_name) == 0) {
>>>
>>> http://stackoverflow.com/questions/14885000/does-strlen-in-a-strncmp-expression-defeat-the-purpose-of-using-strncmp-ov
>>>
>>> ------------------------------------------------------------------------------
>>> src/share/vm/runtime/arguments.cpp
>>>   412   int i = 0;
>>>   413   assert(version != NULL, "Must provide a version buffer");
>>>   414   while (special_table[i].name != NULL) {
>>> ...
>>>   432     i++;
>>>
>>> Pre-existing issue.
>>>
>>> More readable would be to use a for-loop, e.g.
>>>
>>>    for (size_t i = 0; special_table[i].name != NULL; ++i) {
>>>      ...
>>>    }
>>>
>>> size_t is a better type for i too.
>>>
>>> ------------------------------------------------------------------------------
>>> src/share/vm/runtime/arguments.cpp
>>>   446   int i = 0;
>>>   447   while (aliased_jvm_flags[i].alias_name != NULL) {
>>> ...
>>>   454     i++;
>>>
>>> As above, a for-loop would be more readable.
>>>
>>> ------------------------------------------------------------------------------
>>>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150706/46d8bb87/attachment.htm>

From jon.masamitsu at oracle.com  Mon Jul  6 21:15:40 2015
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Mon, 06 Jul 2015 14:15:40 -0700
Subject: Runtime command-line deprecation (Was Re: RFR: 8066821(S) Enhance
	command line processing to manage deprecating and obsoleting -XX
	command line arguments
In-Reply-To: <559AEC80.3010907@oracle.com>
References: <54B44661.8070007@oracle.com>	<54B84E91.1050708@oracle.com>	<F68EBF6B-38A5-4461-9ADD-41BB198FF943@oracle.com>	<54D15A0B.6030501@oracle.com>	<87B07B03-3423-4738-9617-0C7B72A74A6E@oracle.com>
	<559AEC80.3010907@oracle.com>
Message-ID: <559AEFFC.8050304@oracle.com>



On 07/06/2015 02:00 PM, Coleen Phillimore wrote:
>
> Hi,
>
> I'm happy for more clarity in the process of removing command line 
> arguments.  I have some questions and comments below.
>
> On 6/26/15 4:09 PM, Karen Kinnear wrote:
>> Derek,
>>
>> I am really glad you are looking into this. I expanded this to the runtime folks in case many of them have not yet seen this.
>> Mary just forwarded this and I believe you haven't checked it in yet, so perhaps still time to discuss and make sure
>> we all are together on the deprecation policy.
>>
>> The runtime team was discussing at staff this week how we handle deprecating jvm command-line options as
>> we are looking to do more clean up in the future.
>>
>> So our internal change control process classifies our exported interfaces into three categories:
>> External: command-line flags e.g. -verbose:gc, -Xmx, ...
>>      This includes any commercial flags
>> Private: -XX flags documented (e.g. performance tuning or troubleshooting)
>>      I would assume this would include -XX product, experimental flags and manageable flags
>> Internal: undocumented -XX flags
>>      I would assume this would include -XX develop and diagnostic flags
>>
>> (please correct me if my assumptions are wrong folks)
>
> This is a good categorization.  Although I think there's some grey 
> area between External and Private, where some XX options are so 
> commonly used they should be considered External.   Some of the GC 
> options may fall into this category like UseParNewGC.
>
>> The way I understand that we handle private -XX options today is a 2-step removal: (obsolete_jvm_flags - where the
>> release number is in a table and could be undefined)
>>
>> Using your helpful taxonomy fromhttps://bugs.openjdk.java.net/browse/JDK-806682:
>>
>> Today: private -XX options use 2-step removal (obsolete_jvm_flags)
>>      release 1: Deprecate & Obsolete - warn about the option but do nothing with it (we can remove all code that supports it)
>>      release 2: Dead - unrecognized
>>    - the point of the 2-step is to give customers time to modify any scripts they use
>>
>> I believe we have very rarely removed External flags - since customers, licensees, etc. may expect them.
>>
>> Mini-proposal:
>> 1) I would recommend that we add a future set of changes to add consistent handling for the External flags -
>> so that they would follow a three-step removal:
>>      release 1: Deprecate & Handle - warn and keep supporting
>>      release 2: Deprecate & Obsolete - warn and do nothing
>>      release 3: Dead - unrecognized
>>
>> 2) For the Internal flags - I think it would be kindest to customers and not a huge amount of additional work if
>> we were to follow the Private model of using a 2 step.
>>
>> 3) leave the Private flags with the current 2-step removal
>
> Yes, this reflects our current model.
>> 4) add support for aliasing - for any of them
>>      So that if you are doing aliasing, you would follow the model you are adding
>>       release 1: Deprecated & Handled - i.e. warn and still support (and set the information for the new alias)
>>       release 2: Dead - unrecognized
>>
>> 5) Could we possibly take the two flags that followed a different model already, i.e. moving to
>> Deprecated & Handled and handle those as mistakes rather than part of a new general model?
>
> So this is my question which is around the code review in question.   
> Note, Derek, can you resend the RFR to hotspot-dev at openjdk.java.net so 
> it gets to all of us (even the compiler team may want to share in the 
> mechanism).
>
> Why are UseParNewGC and MaxGCMinorPauseMillis deprecated and handled 
> and not deprecated and obsolete?  I don't actually see why have the 
> distinction here?

If we "deprecate and obsolete" what GC do we run?  Last month we would
have run UseParallelGC which the user may or may not have liked. Today
we would run UseG1GC.  Probably a little more surprising.  I think the 
selection
of the GC is critical enough such that we don't run if we don't handle 
it.  Of
course, I'm a GC guy and maybe the rest of the  world doesn't think like 
that.

Jon

>
> Did these flags follow the deprecation procedure below?  Why not just 
> make them obsolete, why have this other mechanism?  I may be asking 
> the same question as Karen.
>
> I think the aliased flags could have a deprecate and handle model 
> (where handle == alias) in which case you only need one table for the 
> aliased flags, and you need to keep them in globals.hpp so they're 
> parsed properly.
>> Or do you think we will see more cases other than aliasing in which we would want to
>>      release 1: Deprecate & Handle and then release 2: Dead
>>      rather than release 1: Deprecate & Obsolete and then 2: Dead
>>      or rather than a 3 step like the External option proposal above?
>
> I think for the aliased flags you want the first option here 
> (deprecate&handle), right?  But that's only because you need to keep 
> the option in globals.hpp so it parses correctly, otherwise the second 
> option (deprecate&obsolete) would be the preference?
>
> The options in the GC that are being deprecated and handled have had 
> warnings about them for a while, so making them obsolete doesn't feel 
> too soon.
>
> Also, I agree with Kim's comment below that your comment lines are too 
> long.  My fonts are too big to have windows this wide.
>
> thanks,
> Coleen
>> thanks,
>> Karen
>>
>> p.s. Note that all of the deprecation needs to
>> 1) work with licensee engineering to ensure we give licensee's a head's up and get feedback
>> 2) file a change control request
>>
>> - we try to do these both as bulk requests to reduce the processing overhead.
>>
>> p.p.s. Details
>>
>> 1.  Do the warnings print today or are they silent? Just want to make sure we are conscious of how
>> those are handled if any of this moves to the new unified logging mechanism for which the new default
>> for "warning" level is to print.
>>
>> 2. "will likely be removed in a future release"? If we have already set the release it will be removed - is this a bit
>> vague or did I not read closely enough?
>>
>>
>>
>> On Feb 3, 2015, at 6:30 PM, Derek White wrote:
>>
>>> Request for review (again):
>>>
>>> - Updated with Kim's suggestion.
>>> - Stopped double-printing warnings in some cases.
>>> - Initial caps on warning() messages.
>>>
>>> Webrev:http://cr.openjdk.java.net/~drwhite/8066821/webrev.04/
>>> CR:https://bugs.openjdk.java.net/browse/JDK-8066821
>>> Tests:     jtreg, jprt
>>>
>>> Thanks for looking!
>>>
>>> - Derek
>>>
>>> On 1/28/15 3:47 PM, Kim Barrett wrote:
>>>> On Jan 15, 2015, at 6:34 PM, Derek White<derek.white at oracle.com>  wrote:
>>>>> Hi All,
>>>>>
>>>>> I need another review, especially someone from runtime. Coleen, do you want crack at it or can you forward as needed?
>>>>>
>>>>> Another version for review:
>>>>> http://cr.openjdk.java.net/~drwhite/8066821/webrev.01
>>>> I noticed a bunch of formatting changes that were in the previous
>>>> webrev have been eliminated in the .01 webrev.  Since I was going to
>>>> comment or complain about at least some of them, I have no objection
>>>> to having them backed out.
>>>>
>>>> This is not a complete review; I've only gotten part way through the
>>>> first file!  Unfortunately, I'm swamped with other things right now
>>>> and don't seem to be making progress toward finishing.  So here's what
>>>> I've got so far.
>>>>
>>>> ------------------------------------------------------------------------------
>>>> src/share/vm/runtime/arguments.cpp
>>>>   132     int len = (int)strlen(name);
>>>>
>>>> Pre-existing issue.
>>>>
>>>> Why is this using int and casting to int, rather than just using
>>>> size_t?  The return type for strlen is size_t, as is the argument for
>>>> strncmp where len is used, and the other use of len also can/should be
>>>> size_t.
>>>>
>>>> [I only noticed this because an earlier webrev tweaked the formatting
>>>> of this line.]
>>>>
>>>> ------------------------------------------------------------------------------
>>>> src/share/vm/runtime/arguments.cpp
>>>>   235     const char* spec_vendor = "Sun Microsystems Inc.";
>>>>   236     uint32_t spec_version = 0;
>>>>
>>>> Dead initializers; the variables are immediately reassigned new
>>>> values.
>>>>
>>>> This is a pre-existing defect that I wouldn't have even noticed had
>>>> the enum for bufsz not been reformatted in a previous webrev.  (How's
>>>> that for a lesson in being lazy. :-)
>>>>
>>>> ------------------------------------------------------------------------------
>>>> src/share/vm/runtime/arguments.cpp
>>>>   253  *  -XX arguments are usually defined in globals.hpp, globals_<cpu>.hpp, globals_<os>.hpp, <compiler>_globals.hpp, or <gc>_globals.hpp.
>>>>
>>>> Rather than "are usually defined in" I think better would be something
>>>> like "are defined several places, such as".
>>>>
>>>> ------------------------------------------------------------------------------
>>>> src/share/vm/runtime/arguments.cpp
>>>>   251  *  -XX argument processing:
>>>>
>>>> While the Hotspot style guidelines explicitly don't specify a line
>>>> length limit, I find lines that are long enough that they don't fit in
>>>> one side of a side-by-side webrev on a reasonable-sized landscape
>>>> monitor with a font size I can read without having to scroll the frame
>>>> back and forth to be pretty annoying.
>>>>
>>>> ------------------------------------------------------------------------------
>>>> src/share/vm/runtime/arguments.cpp
>>>>   419     if (((strncmp(flag_status.name, s, len) == 0) &&
>>>>   420           (strlen(s) == len)) ||
>>>>   421          ((s[0] == '+' || s[0] == '-') &&
>>>>   422           (strncmp(flag_status.name, &s[1], len) == 0) &&
>>>>   423           (strlen(&s[1]) == len))) {
>>>>
>>>> Pre-existing issue.
>>>>
>>>> The calls to strlen and comparisons to len on lines 420 and 423 could
>>>> be replaced with
>>>>
>>>> 420:  s[len] == '\0'
>>>>
>>>> 423:  s[len+1] == '\0'
>>>>
>>>> ------------------------------------------------------------------------------
>>>> src/share/vm/runtime/arguments.cpp
>>>>   449     size_t len = strlen(flag_status.alias_name);
>>>>   450     if (((strncmp(flag_status.alias_name, flag_name, len) == 0) &&
>>>>   451          (strlen(flag_name) == len))) {
>>>>
>>>> There is no point to using strlen() and strncmp() here.  Just use
>>>> strcmp(), e.g.
>>>>
>>>>    if (strcmp(flag_status.alias_name, flag_name) == 0) {
>>>>
>>>> http://stackoverflow.com/questions/14885000/does-strlen-in-a-strncmp-expression-defeat-the-purpose-of-using-strncmp-ov
>>>>
>>>> ------------------------------------------------------------------------------
>>>> src/share/vm/runtime/arguments.cpp
>>>>   412   int i = 0;
>>>>   413   assert(version != NULL, "Must provide a version buffer");
>>>>   414   while (special_table[i].name != NULL) {
>>>> ...
>>>>   432     i++;
>>>>
>>>> Pre-existing issue.
>>>>
>>>> More readable would be to use a for-loop, e.g.
>>>>
>>>>    for (size_t i = 0; special_table[i].name != NULL; ++i) {
>>>>      ...
>>>>    }
>>>>
>>>> size_t is a better type for i too.
>>>>
>>>> ------------------------------------------------------------------------------
>>>> src/share/vm/runtime/arguments.cpp
>>>>   446   int i = 0;
>>>>   447   while (aliased_jvm_flags[i].alias_name != NULL) {
>>>> ...
>>>>   454     i++;
>>>>
>>>> As above, a for-loop would be more readable.
>>>>
>>>> ------------------------------------------------------------------------------
>>>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150706/e8e9ee42/attachment.htm>

From jon.masamitsu at oracle.com  Mon Jul  6 21:18:03 2015
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Mon, 06 Jul 2015 14:18:03 -0700
Subject: RFR (S): 8079555: REDO - Determining the desired PLAB size adjusts
	to the the number of threads at the wrong place
In-Reply-To: <55722403.6010208@oracle.com>
References: <55722403.6010208@oracle.com>
Message-ID: <559AF08B.3010904@oracle.com>

Sangheon,

Changes look good.

Jon


On 06/05/2015 03:34 PM, sangheon.kim wrote:
> Hi all,
>
> Please review this REDO change to determine the desired PLAB size for 
> current gc worker threads.
> The reason for back out of previous 
> change(https://bugs.openjdk.java.net/browse/JDK-8073204) was it 
> wouldn't guarantee to be '>= PLAB minimum size'. So added 
> MAX2(min_size(),) to guarantee that.
>
> Currently we calculate an optimal PLAB size with current number of gc 
> workers.
> When the number of workers changes dynamically 
> (-XX:+UseDynamicNumberOfGCThreads), the desired PLAB size returned(by 
> desired_plab_sz()) is still tuned to the number of gc workers that has 
> been used previously.
>
> This change is, firstly calculate the desired PLAB value for a single 
> gc worker and then return desired PLAB size which is bigger(or equal) 
> than PLAB min size according to the current number of threads.
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8079555
>
> Webrev:
> http://cr.openjdk.java.net/~sangheki/8079555/webrev.00/
>
> Test:
> JPRT
>
> Thanks,
> Sangheon



From sangheon.kim at oracle.com  Mon Jul  6 21:44:05 2015
From: sangheon.kim at oracle.com (sangheon.kim)
Date: Mon, 06 Jul 2015 14:44:05 -0700
Subject: RFR (S): 8079555: REDO - Determining the desired PLAB size adjusts
	to the the number of threads at the wrong place
In-Reply-To: <559AF08B.3010904@oracle.com>
References: <55722403.6010208@oracle.com> <559AF08B.3010904@oracle.com>
Message-ID: <559AF6A5.40401@oracle.com>

Hi Jon,

Thanks for reviewing this.
And I need a sponsor for this change.
Would you sponsor this?

Thanks,
Sangheon


On 07/06/2015 02:18 PM, Jon Masamitsu wrote:
> Sangheon,
>
> Changes look good.
>
> Jon
>
>
> On 06/05/2015 03:34 PM, sangheon.kim wrote:
>> Hi all,
>>
>> Please review this REDO change to determine the desired PLAB size for 
>> current gc worker threads.
>> The reason for back out of previous 
>> change(https://bugs.openjdk.java.net/browse/JDK-8073204) was it 
>> wouldn't guarantee to be '>= PLAB minimum size'. So added 
>> MAX2(min_size(),) to guarantee that.
>>
>> Currently we calculate an optimal PLAB size with current number of gc 
>> workers.
>> When the number of workers changes dynamically 
>> (-XX:+UseDynamicNumberOfGCThreads), the desired PLAB size returned(by 
>> desired_plab_sz()) is still tuned to the number of gc workers that 
>> has been used previously.
>>
>> This change is, firstly calculate the desired PLAB value for a single 
>> gc worker and then return desired PLAB size which is bigger(or equal) 
>> than PLAB min size according to the current number of threads.
>>
>> CR:
>> https://bugs.openjdk.java.net/browse/JDK-8079555
>>
>> Webrev:
>> http://cr.openjdk.java.net/~sangheki/8079555/webrev.00/
>>
>> Test:
>> JPRT
>>
>> Thanks,
>> Sangheon
>



From jon.masamitsu at oracle.com  Tue Jul  7 04:05:51 2015
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Mon, 06 Jul 2015 21:05:51 -0700
Subject: RFR (S): 8079555: REDO - Determining the desired PLAB size adjusts
	to the the number of threads at the wrong place
In-Reply-To: <559AF6A5.40401@oracle.com>
References: <55722403.6010208@oracle.com> <559AF08B.3010904@oracle.com>
	<559AF6A5.40401@oracle.com>
Message-ID: <559B501F.40204@oracle.com>

Sangheon,

Send me the updated patch and I'll integrate it.

Jon

On 7/6/2015 2:44 PM, sangheon.kim wrote:
> Hi Jon,
>
> Thanks for reviewing this.
> And I need a sponsor for this change.
> Would you sponsor this?
>
> Thanks,
> Sangheon
>
>
> On 07/06/2015 02:18 PM, Jon Masamitsu wrote:
>> Sangheon,
>>
>> Changes look good.
>>
>> Jon
>>
>>
>> On 06/05/2015 03:34 PM, sangheon.kim wrote:
>>> Hi all,
>>>
>>> Please review this REDO change to determine the desired PLAB size 
>>> for current gc worker threads.
>>> The reason for back out of previous 
>>> change(https://bugs.openjdk.java.net/browse/JDK-8073204) was it 
>>> wouldn't guarantee to be '>= PLAB minimum size'. So added 
>>> MAX2(min_size(),) to guarantee that.
>>>
>>> Currently we calculate an optimal PLAB size with current number of 
>>> gc workers.
>>> When the number of workers changes dynamically 
>>> (-XX:+UseDynamicNumberOfGCThreads), the desired PLAB size 
>>> returned(by desired_plab_sz()) is still tuned to the number of gc 
>>> workers that has been used previously.
>>>
>>> This change is, firstly calculate the desired PLAB value for a 
>>> single gc worker and then return desired PLAB size which is 
>>> bigger(or equal) than PLAB min size according to the current number 
>>> of threads.
>>>
>>> CR:
>>> https://bugs.openjdk.java.net/browse/JDK-8079555
>>>
>>> Webrev:
>>> http://cr.openjdk.java.net/~sangheki/8079555/webrev.00/
>>>
>>> Test:
>>> JPRT
>>>
>>> Thanks,
>>> Sangheon
>>
>



From stefan.johansson at oracle.com  Tue Jul  7 12:58:57 2015
From: stefan.johansson at oracle.com (Stefan Johansson)
Date: Tue, 07 Jul 2015 14:58:57 +0200
Subject: RFR: 8129417: Oop iteration clean-up to remove
	oop_ms_follow_contents
In-Reply-To: <20150706151537.GP3266@ehelin.jrpg.bea.com>
References: <5588217F.9090201@oracle.com>
	<20150706151537.GP3266@ehelin.jrpg.bea.com>
Message-ID: <559BCD11.3010801@oracle.com>

Thanks Erik for reviewing,

New webrevs:
Part 2 - full: 
http://cr.openjdk.java.net/~sjohanss/8129417/2-add-oop_iterate_size/hotspot.01/
Part 2 - inc: 
http://cr.openjdk.java.net/~sjohanss/8129417/2-add-oop_iterate_size/hotspot.00-01/

Part 3 - full: 
http://cr.openjdk.java.net/~sjohanss/8129417/3-re-structure-mark-sweep/hotspot.02/
Part 3 - inc: 
http://cr.openjdk.java.net/~sjohanss/8129417/3-re-structure-mark-sweep/hotspot.00-02/
Also includes an updated assert, which was missed in previous webrev.

Comments inline.

On 2015-07-06 17:15, Erik Helin wrote:
> Hi Stefan,
>
> thanks a lot for taking on this work. Please see comments inline.
>
> On 2015-06-22, Stefan Johansson wrote:
>> Hi,
>>
>> Please review these changes for RFE:
>> https://bugs.openjdk.java.net/browse/JDK-8129417
>>
>> Webrev for the full change:
>> http://cr.openjdk.java.net/~sjohanss/8129417/hotspot.00/
>>
>> Summary:
>> To allow further cleanups and later remove G1 specific code from mark-sweep,
>> we want to remove the mark-sweep specific visitor oop_ms_follow_contents.
>> Doing this cleanup has proven to be a little more complicated than I first
>> anticipated and to make the change easier to review I've split it into four
>> different parts.
>>
>> Part 1 - removing oop_ms_follow_contents:
>> http://cr.openjdk.java.net/~sjohanss/8129417/1-remove-oop_ms_follow_contents/hotspot.00/
> - Looking in specialized_oop_closures.hpp, the comment
>     71 // This is split into several because of a Visual C++ 6.0 compiler bug
>     72 // where very long macros cause the compiler to crash
>    seems a bit dated, given that Visual C++ 6.0 was released in 1998. I
>    think we should try to merge ALL_OOP_OOP_ITERATE_CLOSURES_1 and 2 (and
>    their corresponding macros). Do you think like doing that in this
>    patch or do you want to file a follow-up bug?
I would like to do/investigate this as a separate RFE, I have a list of 
things that should follow this clean up and I plan to file those before 
pushing this change.

>
> - The part of the patch that changes instanceMirrorKlass.inline.hpp
>    might impact more GCs than just any of the marksweep ones. Have you
>    seen any performance improvements/regressions with G1 or CMS?
No, but I have not done as many runs and done as much analysis of this 
as I've done on the serial full gc. I've not seen any regression in 
aurora.se performance runs.
>> Part 2 - introducing oop_iterate_size:
>> http://cr.openjdk.java.net/~sjohanss/8129417/2-add-oop_iterate_size/hotspot.00/
> - In arrayKlass.hpp, why add a space to:
>    +#define OOP_OOP_ITERATE_DECL_NO_BACKWARDS(OopClosureType, nv_suffix)         \
I've followed '\' alignment "rule" for these macros, which seems to be 
two spaces after the longest line. I just realized that I should also 
update OOP_OOP_ITERATE_DECL_RANGE with one extra space to be consistent.

> - In arrayKlass.hpp, the following lines seems aligned strangely:
>    +void KlassType::oop_oop_iterate_range##nv_suffix(oop obj, OopClosureType* closure, int start, int end) {  \
>    +  oop_oop_iterate_range<nvs_to_bool(nv_suffix)>(obj, closure, start, end);                               \
Nice catch, missed fixing the alignment when making it void.

> - In oop.inline.hpp, why must the size be computed before applying the
>    closure? In the world of perm gen, this might have been important in
>    order to not follow stale klass pointers, but with Metaspace the Klass*
>    won't move.
As we've discussed offline, this is most likely safe. I've done an 
ad-hoc RBT run with some random testing when I assert on the size being 
the same both before and after the call to oop_oop_iterate.

I would like to do this change as a separate change as well, so it's 
been added to my list of RFEs to file.

>
> - Can we remove the method MutableSpace::oop_iterate?
Looks like it, no one seems to be using it. I'll remove it instead of 
updating it to use oop_iterate_size().

>
>> Part 3 - restructure mark-sweep:
>> http://cr.openjdk.java.net/~sjohanss/8129417/3-re-structure-mark-sweep/hotspot.00/
> - Can you please remove the commented assert in mark_and_push:
>     +//  assert(Universe::heap()->is_in_reserved(p), "should be in object space");
>    or should it be enabled again?
Removed.

> - I agree with Per about moving the check for object arrays into
>    follow_object.
Already done, but gives a slight regression. I still think we can live 
with this small regression for now, to avoid people doing wrong and call 
follow_object on objArrays.

>> Part 4 - compiler hints for inlining:
>> http://cr.openjdk.java.net/~sjohanss/8129417/4-compiler-hints-for-inlining/hotspot.00/
> - I would like to see the definiton of FORCE_INLINE in the compiler
>    specific globalDefinitions file. Then the code in
>    instanceKlass.inline.hpp can look like:
>     #if defined(TARGET_COMPILER_visCPP) || defined(TARGET_COMPILER_sparcWorks)
>       #define INLINE FORCE_INLINE
>     #else
>       #define INLINE inline
>     #endif
>
> - The same comment for stack.inline.hpp, but with NO_INLINE:
>    #if defined(TARGET_COMPILER_sparcWorks)
>      #define STACK_NO_INLINE NO_INLINE
>    #else
>      #define STACK_NO_INLINE
>    #endif
I agree, we should come up with a better and cleaner way to do this. I 
will file a follow up RFE.

> - I agree with Per about adding an #undef in stack.inline.hpp
Already fixed.

Thanks again Erik,
Stefan

> Thanks!
> Erik
>
>> Testing:
>> * Adhoc RBT run for functionality - no new failures.
>> * Adhoc aurora.se run for performance - no obvious regression.
>> * Many local runs tuned to do Full GCs a lot to verify that there is no
>> obvious regression.
>>
>> Thanks,
>> Stefan



From sangheon.kim at oracle.com  Tue Jul  7 13:40:17 2015
From: sangheon.kim at oracle.com (sangheon.kim)
Date: Tue, 07 Jul 2015 06:40:17 -0700
Subject: RFR (S): 8079555: REDO - Determining the desired PLAB size adjusts
	to the the number of threads at the wrong place
In-Reply-To: <559B501F.40204@oracle.com>
References: <55722403.6010208@oracle.com>
	<559AF08B.3010904@oracle.com>	<559AF6A5.40401@oracle.com>
	<559B501F.40204@oracle.com>
Message-ID: <559BD6C1.9030408@oracle.com>

Thank you Jon.
I'll send it at separate email.

Sangheon


On 07/06/2015 09:05 PM, Jon Masamitsu wrote:
> Sangheon,
>
> Send me the updated patch and I'll integrate it.
>
> Jon
>
> On 7/6/2015 2:44 PM, sangheon.kim wrote:
>> Hi Jon,
>>
>> Thanks for reviewing this.
>> And I need a sponsor for this change.
>> Would you sponsor this?
>>
>> Thanks,
>> Sangheon
>>
>>
>> On 07/06/2015 02:18 PM, Jon Masamitsu wrote:
>>> Sangheon,
>>>
>>> Changes look good.
>>>
>>> Jon
>>>
>>>
>>> On 06/05/2015 03:34 PM, sangheon.kim wrote:
>>>> Hi all,
>>>>
>>>> Please review this REDO change to determine the desired PLAB size 
>>>> for current gc worker threads.
>>>> The reason for back out of previous 
>>>> change(https://bugs.openjdk.java.net/browse/JDK-8073204) was it 
>>>> wouldn't guarantee to be '>= PLAB minimum size'. So added 
>>>> MAX2(min_size(),) to guarantee that.
>>>>
>>>> Currently we calculate an optimal PLAB size with current number of 
>>>> gc workers.
>>>> When the number of workers changes dynamically 
>>>> (-XX:+UseDynamicNumberOfGCThreads), the desired PLAB size 
>>>> returned(by desired_plab_sz()) is still tuned to the number of gc 
>>>> workers that has been used previously.
>>>>
>>>> This change is, firstly calculate the desired PLAB value for a 
>>>> single gc worker and then return desired PLAB size which is 
>>>> bigger(or equal) than PLAB min size according to the current number 
>>>> of threads.
>>>>
>>>> CR:
>>>> https://bugs.openjdk.java.net/browse/JDK-8079555
>>>>
>>>> Webrev:
>>>> http://cr.openjdk.java.net/~sangheki/8079555/webrev.00/
>>>>
>>>> Test:
>>>> JPRT
>>>>
>>>> Thanks,
>>>> Sangheon
>>>
>>
>



From jon.masamitsu at oracle.com  Tue Jul  7 18:49:24 2015
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Tue, 07 Jul 2015 11:49:24 -0700
Subject: RFR: JDK-8086056: ParNew: auto-tune ParGCCardsPerStrideChunk
In-Reply-To: <etPan.5595937a.283b46fd.386@tw-mbp-tprintezis>
References: <etPan.5581f51b.699a1ea0.1215a@tw-mbp-tprintezis>	<55826D13.1030200@oracle.com>	<etPan.5582cc8d.1bcb7375.1215a@tw-mbp-tprintezis>	<5587B9F6.3010306@oracle.com>	<etPan.558986ff.16ab4e1c.1215a@tw-mbp-tprintezis>	<etPan.558af9e9.4affa36.1215a@tw-mbp-tprintezis>	<558BB5A8.3060002@oracle.com>	<etPan.558c022e.23da99d5.1215a@tw-mbp-tprintezis>	<558CF399.1090705@oracle.com>	<etPan.558d5405.68777fa0.1215a@tw-mbp-tprintezis>	<1435325859.2599.11.camel@oracle.com>	<etPan.55946411.211dea1b.289@tw-mbp-tprintezis>	<1435826616.2615.18.camel@oracle.com>
	<etPan.5595937a.283b46fd.386@tw-mbp-tprintezis>
Message-ID: <559C1F34.5040108@oracle.com>

Tony,

Are you setup to run specjbb2013?  I did a run with

the PerStride changes (original patch) vs a baseline and
saw an increase in ParNew times.  I attached a plot of the
pause times.  I used

-server -Xmx30g -Xms30g -Xmn20g -XX:+UseConcMarkSweepGC -XX:+UseLargePages

This was run on a linux system with 8 cores.

 From a run of specjbb2005 on the same system there is an improvement
in the ParNew times.

I'll repeat the experiment with the latest patch.   If you can try to 
repeat the experiment,
that would give me some confidence that it's not just me.

Thanks.

Jon


On 07/02/2015 12:39 PM, Tony Printezis wrote:
> Thomas,
>
> Latest version:
>
> http://cr.openjdk.java.net/~tonyp/8086056/webrev.3/ 
> <http://cr.openjdk.java.net/%7Etonyp/8086056/webrev.3/>
>
> See below for comments.
>
> On July 2, 2015 at 4:43:42 AM, Thomas Schatzl 
> (thomas.schatzl at oracle.com <mailto:thomas.schatzl at oracle.com>) wrote:
>
>> Hi Tony,
>>
>> On Wed, 2015-07-01 at 18:05 -0400, Tony Printezis wrote:
>> > Latest changes, as discussed with Bengt and Thomas,
>> >
>> > http://cr.openjdk.java.net/~tonyp/8086056/webrev.2/
>>
>> a few suggestions:
>>
>> - please use braces even for simple statements in if-clauses, e.g.
>>
>> parNewGeneration.cpp:
>>
>> 883 if (!UseDynamicParGCStrides) return;
>
>
> I did that. But I actually added an extra verbose statement in the 
> body of the if-statement (I thought it?d be helpful to also print the 
> value of ParGCCardsPerStrideChunk when the dynamic calculation is 
> off). So, I don?t have a simple return any more, but I?ll keep that in 
> mind in the future (was it anywhere else? I didn?t immediately spot 
> another occurrence of that).
>
>
>>
>> - Comments should start with upper case:
>>
>> parNewGeneration.hpp:
>>
>> 354 // automatically calculate ParGCCardsPerStrideChunk based on the
>> old
>
>
> Done.
>
>
>>
>> - parameter calculation in ParNewGeneration::adjust_cards_per_stride:
>> - DynamicParGCStridesMinSize should be > 0 (only if !
>> UseDynamicParGCStrides is enabled, not sure if that can be encoded
>> within the framework)
>
>
> If !UseDynamicParGCStrides, then both sets of min / max parameters are 
> not used. I think it?d make the constraint checking a pain having to 
> sanity-check parameters whether a flag is on or off (unless they mean 
> different things in either case, of course).
>
> I added a range check to the min / max stride size args, added a lower 
> bound of 1, and decided to add an artificial upper bound of 32K (I 
> have tested with up to 8K). Is that reasonable? We can revisit that if 
> we need to increse it. Note that you can still set 
> ParGCCardsPerStrideChunk to something higher if required.
>
>
>>
>> - the issues with the Dynamic*Capacity variables Sangheon mentioned,
>> although I think it is fine that they are equal, just that "r" needs to
>> be set correctly.
>
>
> Yeah, fixed. (See previous e-mail.)
>
>
>>
>> Just wanting to check back if it was intended that the code always
>> rounds down, achieving the maximum stride only at the end of the
>> interval. E.g.
>>
>> 912 const int stride_size_log = log2_long((jlong) stride_size);
>> 913 res = (size_t) 1 << stride_size_log;
>>
>> It's fine with me, just asking.
>
>
> I did the simplest calculation. I can revise it if you want. Also note 
> that if capacity > max_capacity, then stride size will be set to the 
> max stride size.
>
> Tony
>
>
>>
>> Thanks,
>> Thomas
>>
>>
>>
>
>
>
>
>
>
> -----
>
> Tony Printezis | JVM/GC Engineer / VM Team | Twitter
>
> @TonyPrintezis
> tprintezis at twitter.com <mailto:tprintezis at twitter.com>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150707/e432d208/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ParNew.png
Type: image/png
Size: 46637 bytes
Desc: not available
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150707/e432d208/ParNew.png>

From yiyeguhu at gmail.com  Tue Jul  7 21:14:53 2015
From: yiyeguhu at gmail.com (Tao Mao)
Date: Tue, 7 Jul 2015 14:14:53 -0700
Subject: Region-level compaction and related considerations for G1 to
	better facilitate H-objects
In-Reply-To: <1435580949.2700.19.camel@oracle.com>
References: <CANrGW1xWLa5BoiZn4eSD1n3tJ2ZKDb0jRw+pnEptoXkrDxMZsA@mail.gmail.com>
	<1435580949.2700.19.camel@oracle.com>
Message-ID: <CANrGW1xG57ddWYG3NCu_BGyP6T-b-aM+G78+T8r5Lad2CSpuGA@mail.gmail.com>

Thank you for reply, Thomas! Please see inline. It looks there are at least
two ideas that can be good suggestions here.

On Mon, Jun 29, 2015 at 5:29 AM, Thomas Schatzl <thomas.schatzl at oracle.com>
wrote:

> Hi Tao,
>
> On Fri, 2015-06-26 at 17:46 -0700, Tao Mao wrote:
> > Hi there,
> >
> > Recently in using G1 on our realtime-focused production servers, I
> > noticed some full GCs were triggered by humongous allocation failure
> > due to fragmentation of free regions while it's still having enough
> > space in number.
> >
> > I'm not suggesting that we move H-objects around. Just wonder if we
> > have the following or similar in place or plan to have. e.g.,
>
> In such cases, moving around (small) humongous objects might actually be
> a good idea (compared to a full gc).
> >
> > (1) For evacuation, we grab free regions closer to the start of heap
> > and/or try to grab free regions with fewer neighboring free regions.
>
> 1a) has been implemented in some 8u release, either u20 or u40. If I
> remember correctly, new old gen regions (that includes humongous) are
> allocated from the end of the heap downwards, new young regions from the
> start of the heap.
>

I saw that piece of code. Nice to have.


>
> The second option has not been explored (try to grab free regions
> according to other policies), but seems straightforward (but some work)
> to do. Check out G1CollectedHeap::new_region().
>

Might not be that significant if the above is there. The above mechanism
realizes that in the end, IMO.


>
> Controlling the destination regions only solves half of the problem
> though, but a useful mechanism also requires control of the source
> regions: in a given evacuation there are often a lot of almost equally
> expensive regions to collect, so any change according to e.g. "try to
> create as much as possible contiguous free areas" might help a lot
> without a lot of performance impact.
>

Good idea! You may want to record this idea somewhere.


>
> > (2) Humongous object allocation tries to find the most suitable
> > location, i.e., least waste of its neighboring free regions after
> > H-allocation.
>
> Check out G1CollectedHeap::humongous_obj_allocate(). The entry point
> that might be most interesting would be to change
> HeapRegionManager::find_contiguous_empty_or_available().
>

I checked the latest code. It looks like the second point (finding the most
suitable contiguous regions for humongous allocation) would be
straightforward to implement in HeapRegionManager::find_contiguous().


>
> > Is there something along this line to fight region-level
> > fragmentation, or related fixed bugs? If so, any pointers?
>
> Above mentioned is what I know in that area.
>
> Thanks,
>   Thomas
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150707/bd6fc010/attachment.htm>

From jwha at google.com  Tue Jul  7 21:59:09 2015
From: jwha at google.com (Jungwoo Ha)
Date: Wed, 8 Jul 2015 06:59:09 +0900
Subject: RFR: JEP: Parallelize the Full GC Phase in CMS
Message-ID: <CA+n_jhj=3oodvDHDhWC7x5EhHPj71GTttFcF5WW16OxvGik-LA@mail.gmail.com>

JEP: https://bugs.openjdk.java.net/browse/JDK-8130200
Webrev: http://cr.openjdk.java.net/~jwha/8086706/webrev.jdk9.00

This JEP is parallelizing the full GC phase of CMS.
Please review this proposal and let me know.

-- 
Jungwoo Ha
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150708/66e7f1b1/attachment.htm>

From jwha at google.com  Tue Jul  7 21:59:52 2015
From: jwha at google.com (Jungwoo Ha)
Date: Wed, 8 Jul 2015 06:59:52 +0900
Subject: RFR (XL): JDK-8086706 Parallel Full GC for CMS
In-Reply-To: <CA+n_jhiX+wMmNo7-7j5Qr+_ZR+gMRaEpw+P9g34RgtWHN5s7qw@mail.gmail.com>
References: <CA+n_jhjw=T2O3pquZU++N5miGdXxwh=Kp_TBDTmk2ud79SJCgg@mail.gmail.com>
	<5579B1DB.9090804@oracle.com>
	<CA+3eh10-MyRSJ8gcievY_=hTXUjshGnUfwmZFGMsd9ZMD-xBuw@mail.gmail.com>
	<C9B58FD5-C1E4-4143-A85B-4AB322246822@oracle.com>
	<CA+n_jhjRd3_rm9v7n5qMGWptDqPcQ-ax4w4eHT91R=r1Tyj6mA@mail.gmail.com>
	<CA+n_jhiX+wMmNo7-7j5Qr+_ZR+gMRaEpw+P9g34RgtWHN5s7qw@mail.gmail.com>
Message-ID: <CA+n_jhidEOH1dh4YrR0OQxtXUmU1Mh6-zvJXheHdj+M2zYNAAw@mail.gmail.com>

I started a new thread. Let's close this one.

On Wed, Jul 1, 2015 at 10:25 AM, Jungwoo Ha <jwha at google.com> wrote:

> JEP: https://bugs.openjdk.java.net/browse/JDK-8130200
>
> The bug id is different. Should we continue on this thread or start a new
> one?
> - Jungwoo
> On Jun 27, 2015 5:48 PM, "Jungwoo Ha" <jwha at google.com> wrote:
>
>> Sorry for the long wait. So many things have cleaned up on JDK9.
>> Here is the webrev against jdk9/hs-rt.
>> http://cr.openjdk.java.net/~jwha/8086706/webrev.jdk9.00
>>
>> I am preparing for the JEP doc.
>>
>>
>> On Thu, Jun 11, 2015 at 12:31 PM, Kim Barrett <kim.barrett at oracle.com>
>> wrote:
>>
>>> On Jun 11, 2015, at 12:33 PM, Volker Simonis <volker.simonis at gmail.com>
>>> wrote:
>>> >
>>> > Hi,
>>> >
>>> > if nobody has already started doing it, I'll try to forward port the
>>> > patch to jdk9/hs-gc/hotspot.
>>>
>>> You?ll want to use jdk9/hs-rt rather than jdk9/hs-gc.
>>>
>>>
>>>
>>
>>
>> --
>> Jungwoo Ha | Java Platform Team | jwha at google.com
>>
>>


-- 
Jungwoo Ha | Java Platform Team | jwha at google.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150708/b741dd93/attachment.htm>

From tprintezis at twitter.com  Wed Jul  8 07:39:16 2015
From: tprintezis at twitter.com (Tony Printezis)
Date: Wed, 8 Jul 2015 09:39:16 +0200
Subject: RFR: JDK-8086056: ParNew: auto-tune
 ParGCCardsPerStrideChunk
In-Reply-To: <559C1F34.5040108@oracle.com>
References: <etPan.5581f51b.699a1ea0.1215a@tw-mbp-tprintezis>
	<55826D13.1030200@oracle.com>
	<etPan.5582cc8d.1bcb7375.1215a@tw-mbp-tprintezis>
	<5587B9F6.3010306@oracle.com>
	<etPan.558986ff.16ab4e1c.1215a@tw-mbp-tprintezis>
	<etPan.558af9e9.4affa36.1215a@tw-mbp-tprintezis>
	<558BB5A8.3060002@oracle.com>
	<etPan.558c022e.23da99d5.1215a@tw-mbp-tprintezis>
	<558CF399.1090705@oracle.com>
	<etPan.558d5405.68777fa0.1215a@tw-mbp-tprintezis>
	<1435325859.2599.11.camel@oracle.com>
	<etPan.55946411.211dea1b.289@tw-mbp-tprintezis>
	<1435826616.2615.18.camel@oracle.com>
	<etPan.5595937a.283b46fd.386@tw-mbp-tprintezis>
	<559C1F34.5040108@oracle.com>
Message-ID: <etPan.559cd3a5.1c9689f6.23ec@tw-mbp-tprintezis>

Hi Jon,

Thanks for giving it a spin. Yes, we do have specjbb2013. I?m at ECOOP / CurryOn this week. I?ll try this on Monday after I get back home. Thanks again,

Tony

On July 7, 2015 at 9:10:12 PM, Jon Masamitsu (jon.masamitsu at oracle.com) wrote:

Tony,

Are you setup to run specjbb2013?? I did a run with

the PerStride changes (original patch) vs a baseline and
saw an increase in ParNew times.? I attached a plot of the
pause times.? I used

-server -Xmx30g -Xms30g -Xmn20g -XX:+UseConcMarkSweepGC -XX:+UseLargePages

This was run on a linux system with 8 cores.

From a run of specjbb2005 on the same system there is an improvement
in the ParNew times.

I'll repeat the experiment with the latest patch.?? If you can try to repeat the experiment,
that would give me some confidence that it's not just me.

Thanks.

Jon


On 07/02/2015 12:39 PM, Tony Printezis wrote:
Thomas,

Latest version:

http://cr.openjdk.java.net/~tonyp/8086056/webrev.3/

See below for comments.

On July 2, 2015 at 4:43:42 AM, Thomas Schatzl (thomas.schatzl at oracle.com) wrote:

Hi Tony,?

On Wed, 2015-07-01 at 18:05 -0400, Tony Printezis wrote:?
> Latest changes, as discussed with Bengt and Thomas,?
>?
> http://cr.openjdk.java.net/~tonyp/8086056/webrev.2/?

a few suggestions:?

- please use braces even for simple statements in if-clauses, e.g.?

parNewGeneration.cpp:?

883 if (!UseDynamicParGCStrides) return;?


I did that. But I actually added an extra verbose statement in the body of the if-statement (I thought it?d be helpful to also print the value of ParGCCardsPerStrideChunk when the dynamic calculation is off). So, I don?t have a simple return any more, but I?ll keep that in mind in the future (was it anywhere else? I didn?t immediately spot another occurrence of that).




- Comments should start with upper case:?

parNewGeneration.hpp:?

354 // automatically calculate ParGCCardsPerStrideChunk based on the?
old?


Done.




- parameter calculation in ParNewGeneration::adjust_cards_per_stride:?
- DynamicParGCStridesMinSize should be > 0 (only if !?
UseDynamicParGCStrides is enabled, not sure if that can be encoded?
within the framework)?


If !UseDynamicParGCStrides, then both sets of min / max parameters are not used. I think it?d make the constraint checking a pain having to sanity-check parameters whether a flag is on or off (unless they mean different things in either case, of course).

I added a range check to the min / max stride size args, added a lower bound of 1, and decided to add an artificial upper bound of 32K (I have tested with up to 8K). Is that reasonable? We can revisit that if we need to increse it. Note that you can still set ParGCCardsPerStrideChunk to something higher if required.




- the issues with the Dynamic*Capacity variables Sangheon mentioned,?
although I think it is fine that they are equal, just that "r" needs to?
be set correctly.?


Yeah, fixed. (See previous e-mail.)




Just wanting to check back if it was intended that the code always?
rounds down, achieving the maximum stride only at the end of the?
interval. E.g.?

912 const int stride_size_log = log2_long((jlong) stride_size);?
913 res = (size_t) 1 << stride_size_log;?

It's fine with me, just asking.?


I did the simplest calculation. I can revise it if you want. Also note that if capacity > max_capacity, then stride size will be set to the max stride size.

Tony




Thanks,?
Thomas?









-----

Tony Printezis | JVM/GC Engineer / VM Team | Twitter

@TonyPrintezis
tprintezis at twitter.com


-----

Tony Printezis | JVM/GC Engineer / VM Team | Twitter

@TonyPrintezis
tprintezis at twitter.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150708/4e994844/attachment.htm>

From thomas.schatzl at oracle.com  Wed Jul  8 10:34:54 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 08 Jul 2015 12:34:54 +0200
Subject: Request for code review  (M) -  8081629: CMS split_block() does
	not correctly fix up block-offset-table for large blocks
In-Reply-To: <558868A3.2070604@oracle.com>
References: <558868A3.2070604@oracle.com>
Message-ID: <1436351694.2275.5.camel@oracle.com>

Hi Jon,

  sorry for the late review...

On Mon, 2015-06-22 at 12:57 -0700, Jon Masamitsu wrote:
> This bug is causing large young GC times for ParNew when
> there are large objects in the heap and the logarithmic strides
> are being used (badly).
> 
> https://bugs.openjdk.java.net/browse/JDK-8081629
> 
> The change in webrev_min corrects the problem
> with the minimum code change (least risk).
> 
> http://cr.openjdk.java.net/~jmasa/8081629/webrev_min.02/
> 
> A rewriting of split_block() in included in a second webrev
> 
> http://cr.openjdk.java.net/~jmasa/8081629/webrev.02/
> 
> I've willing to go with the minimum change but could also
> be encouraged to push the rewrite of split_block().
> 
> Vote for the one you like with your code review.

Voting for webrev.02 as it seems easier to understand to me.

I would suggest to fix one comment in both version that read 

         // Fill in the remainder of this "power block", if it
         // is non-null.

replacing non-null by "empty". A "null" range seems confusing to me in
this context.

Both seem okay to me (with some extra spaces in line 469 that should be
fixed before pushing in the "min" fix).

Thanks,
  Thomas




From thomas.schatzl at oracle.com  Wed Jul  8 10:59:24 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 08 Jul 2015 12:59:24 +0200
Subject: Request for Review (s) - 7012980: PSOldGen is increased if
	there is no space in Metaspace
In-Reply-To: <5582EBA4.2060504@oracle.com>
References: <554CDD1C.9070200@oracle.com> <55511CF2.60103@oracle.com>
	<555CC33E.2040106@oracle.com> <5582EBA4.2060504@oracle.com>
Message-ID: <1436353164.2275.8.camel@oracle.com>

Hi Jon,

On Thu, 2015-06-18 at 09:02 -0700, Jon Masamitsu wrote:
> A recent integration collided with my patch so I had to merge
> and redo the webrev.  Previously in addition to the fix there was
> a clean up that introduce a method is_system_gc().  The need
> for is_system_gc() was overridden by the already added
> is_user_requested_gc().
> 
> New webrev
> 
> http://cr.openjdk.java.net/~jmasa/7012980/webrev.02/
> 
> Thanks.

  looks good.

One minor nit I can see is the naming of "should_update_promo_stats" - I
am not sure why it is has "promo" in the name.

The other, with the "eden" in the name can be immediately associated
with young gen, the other not so much imo.

Maybe there is a better name of it, but maybe it fits because other
variables related to allocation in the old/tenured gen also have "promo"
in their name. So it is probably best to keep it for uniformity.

Thanks,
  Thomas




From tom.benson at oracle.com  Wed Jul  8 14:17:10 2015
From: tom.benson at oracle.com (Tom Benson)
Date: Wed, 08 Jul 2015 10:17:10 -0400
Subject: Request for Review (s) - 7012980: PSOldGen is increased if there
	is no space in Metaspace
In-Reply-To: <1436353164.2275.8.camel@oracle.com>
References: <554CDD1C.9070200@oracle.com>
	<55511CF2.60103@oracle.com>	<555CC33E.2040106@oracle.com>
	<5582EBA4.2060504@oracle.com> <1436353164.2275.8.camel@oracle.com>
Message-ID: <559D30E6.8070400@oracle.com>

Hi Jon,
Looks good to me as well.  One thing I noticed is that this will also 
prevent growth when GC cause is _gc_locker, which I think would have 
happened before.   I think that's OK, and possibly even good, since the 
real reason for the GC is lost, but just noting that it is a difference.
Tom

On 7/8/2015 6:59 AM, Thomas Schatzl wrote:
> Hi Jon,
>
> On Thu, 2015-06-18 at 09:02 -0700, Jon Masamitsu wrote:
>> A recent integration collided with my patch so I had to merge
>> and redo the webrev.  Previously in addition to the fix there was
>> a clean up that introduce a method is_system_gc().  The need
>> for is_system_gc() was overridden by the already added
>> is_user_requested_gc().
>>
>> New webrev
>>
>> http://cr.openjdk.java.net/~jmasa/7012980/webrev.02/
>>
>> Thanks.
>    looks good.
>
> One minor nit I can see is the naming of "should_update_promo_stats" - I
> am not sure why it is has "promo" in the name.
>
> The other, with the "eden" in the name can be immediately associated
> with young gen, the other not so much imo.
>
> Maybe there is a better name of it, but maybe it fits because other
> variables related to allocation in the old/tenured gen also have "promo"
> in their name. So it is probably best to keep it for uniformity.
>
> Thanks,
>    Thomas
>
>



From jon.masamitsu at oracle.com  Wed Jul  8 15:50:23 2015
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Wed, 08 Jul 2015 08:50:23 -0700
Subject: Request for Review (s) - 7012980: PSOldGen is increased if there
	is no space in Metaspace
In-Reply-To: <1436353164.2275.8.camel@oracle.com>
References: <554CDD1C.9070200@oracle.com> <55511CF2.60103@oracle.com>	
	<555CC33E.2040106@oracle.com> <5582EBA4.2060504@oracle.com>
	<1436353164.2275.8.camel@oracle.com>
Message-ID: <559D46BF.30805@oracle.com>

Thomas,

Thanks for the review.

On 07/08/2015 03:59 AM, Thomas Schatzl wrote:
> Hi Jon,
>
> On Thu, 2015-06-18 at 09:02 -0700, Jon Masamitsu wrote:
>> A recent integration collided with my patch so I had to merge
>> and redo the webrev.  Previously in addition to the fix there was
>> a clean up that introduce a method is_system_gc().  The need
>> for is_system_gc() was overridden by the already added
>> is_user_requested_gc().
>>
>> New webrev
>>
>> http://cr.openjdk.java.net/~jmasa/7012980/webrev.02/
>>
>> Thanks.
>    looks good.
>
> One minor nit I can see is the naming of "should_update_promo_stats" - I
> am not sure why it is has "promo" in the name.
>
> The other, with the "eden" in the name can be immediately associated
> with young gen, the other not so much imo.
>
> Maybe there is a better name of it, but maybe it fits because other
> variables related to allocation in the old/tenured gen also have "promo"
> in their name. So it is probably best to keep it for uniformity.

Yes, the "promo" name is historic.   The
ParallelGC ergonomics tries to manage the amount of space in
the old gen available for promotions (i.e., the free space in the
old gen).  With eden always being empty at the end of a young
GC, the size of eden relates to how much time until the next
young GC.  Similarly "promo" relates to how long until the
next full GC.

Jon
>
> Thanks,
>    Thomas
>
>



From jon.masamitsu at oracle.com  Wed Jul  8 16:05:20 2015
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Wed, 08 Jul 2015 09:05:20 -0700
Subject: Request for Review (s) - 7012980: PSOldGen is increased if there
	is no space in Metaspace
In-Reply-To: <559D30E6.8070400@oracle.com>
References: <554CDD1C.9070200@oracle.com>
	<55511CF2.60103@oracle.com>	<555CC33E.2040106@oracle.com>
	<5582EBA4.2060504@oracle.com> <1436353164.2275.8.camel@oracle.com>
	<559D30E6.8070400@oracle.com>
Message-ID: <559D4A40.2080003@oracle.com>

Tom,

Thanks for the review.

On 07/08/2015 07:17 AM, Tom Benson wrote:
> Hi Jon,
> Looks good to me as well.  One thing I noticed is that this will also 
> prevent growth when GC cause is _gc_locker, which I think would have 
> happened before.   I think that's OK, and possibly even good, since 
> the real reason for the GC is lost, but just noting that it is a 
> difference.

Yes, switching from a negative test to a positive test does
change the result.  I think it is OK because the intervals between
GC's due to _gc_locker is not directly related to the amount
of free space in the young gen.  At least not related in a simple
way and trying to include those intervals would not be consistent
with the way the statistics are used.

Jon

> Tom
>
> On 7/8/2015 6:59 AM, Thomas Schatzl wrote:
>> Hi Jon,
>>
>> On Thu, 2015-06-18 at 09:02 -0700, Jon Masamitsu wrote:
>>> A recent integration collided with my patch so I had to merge
>>> and redo the webrev.  Previously in addition to the fix there was
>>> a clean up that introduce a method is_system_gc().  The need
>>> for is_system_gc() was overridden by the already added
>>> is_user_requested_gc().
>>>
>>> New webrev
>>>
>>> http://cr.openjdk.java.net/~jmasa/7012980/webrev.02/
>>>
>>> Thanks.
>>    looks good.
>>
>> One minor nit I can see is the naming of "should_update_promo_stats" - I
>> am not sure why it is has "promo" in the name.
>>
>> The other, with the "eden" in the name can be immediately associated
>> with young gen, the other not so much imo.
>>
>> Maybe there is a better name of it, but maybe it fits because other
>> variables related to allocation in the old/tenured gen also have "promo"
>> in their name. So it is probably best to keep it for uniformity.
>>
>> Thanks,
>>    Thomas
>>
>>
>



From axel.siebenborn at sap.com  Thu Jul  9 08:29:51 2015
From: axel.siebenborn at sap.com (Siebenborn, Axel)
Date: Thu, 9 Jul 2015 08:29:51 +0000
Subject: RFR(XS): JDK-8129440 G1 crash during concurrent root region scan.
In-Reply-To: <CA+3eh12JaZRDsvmbtssi7wqAsbR7kO3F2_j6vjCWqbu0LBJwdA@mail.gmail.com>
References: <02D5D45C1F8DB848A7AE20E80EE61A5C39945B54@DEWDFEMB20C.global.corp.sap>
	<CA+3eh12JaZRDsvmbtssi7wqAsbR7kO3F2_j6vjCWqbu0LBJwdA@mail.gmail.com>
Message-ID: <02D5D45C1F8DB848A7AE20E80EE61A5C3994BB9A@DEWDFEMB20C.global.corp.sap>


On 06.07.2015 at 15:03 Volker Simonis wrote:
> On Tue, Jun 23, 2015 at 8:45 AM, Siebenborn, Axel
> <axel.siebenborn at sap.com> wrote:
>>
>>
>> Hi Volker,
>>
>> thanks for your comment.Indeed, this was my first idea for a fix. However,
>> there are more than 50 callers of  load_heap_oop, plus some generated by
>> preprocessor defines like this one:
>>
>> #define DO_OOP_WORK_IMPL(closureName) \
>>   template <class T> inline void closureName##Closure::do_oop_work(T* p) {
>> \
>>     T heap_oop = oopDesc::load_heap_oop(p);               \
>>     if (!oopDesc::is_null(heap_oop)) {                    \
>>       oop obj = oopDesc::decode_heap_oop_not_null(heap_oop);       \
>>       do_oop(obj);                                        \
>>     }                                                     \
>>   }
>>
>> We would have to decide for each of the callers, if there might be a
>> concurrency problem or not.
>>
>> Do you think, it is worth the effort and the risk of missing one?
>>
>
> OK, you're probably right.
>
> If you haven?t seen any performance regressions caused by this fix I'm
> fine with your solution.
>
> Unfortunately I can not push this shared-code change so can we please
> get a sponsor from within Oracle for this change?
>
> Thanks,
> Volker

We did not see any performance regressions.
On Itanium compilers add release and acquire semantics to stores and loads of volatile variables. The additional memory barriers really could have a performance impact. But even on Itanium we don?t see any performance regression in our benchmarks.
On other platforms, optimizations prevented by the volatile qualifier are optimizations that we really don't want.    

Thanks,
Axel

>>
>> Regards,
>>
>> Axel
>>
>> On 22.06.2015 at 18:17 Volker Simonis wrote:
>>> Hi Axel,
>>>
>>> the change looks good, but shouldn't we better provide two versions of
>>> load_heap_oop()
>>>
>>> inline oop       oopDesc::load_heap_oop(oop* p)
>>> inline oop       oopDesc::load_heap_oop(volatile oop* p)
>>>
>>> such that the callers can decide on their own which versions they require?
>>>
>>> Or do the callers of load_heap_oop() not always know if there exist
>>> concurrent mutator threads?
>>>
>>> Regards,
>>> Volker
>>>
>>>
>>> On Mon, Jun 22, 2015 at 5:33 PM, Siebenborn, Axel
>>> <axel.siebenborn at sap.com> wrote:
>>>> Hi,
>>>> we have seen crashes with G1 during concurrent root region scan.
>>>> It turned out, that the reason for the crashes was reloading an oop after
>>>> a null check.
>>>> The compiler may legally do this, as the pointer is not declared as
>>>> volatile.
>>>> The code runs concurrently to mutator threads.
>>>>
>>>> template <class T>
>>>> inline void G1RootRegionScanClosure::do_oop_nv(T* p) {
>>>>   T heap_oop = oopDesc::load_heap_oop(p);
>>>> // 1. Load oop
>>>>   if (!oopDesc::is_null(heap_oop)) {
>>>>     oop obj = oopDesc::decode_heap_oop_not_null(heap_oop);    // 2.
>>>> Compiler decides to re-load the oop
>>>>     HeapRegion* hr = _g1h->heap_region_containing((HeapWord*) obj);
>>>>     _cm->grayRoot(obj, obj->size(), _worker_id, hr);
>>>>   }
>>>> }
>>>>
>>>>
>>>> We have seen the problem on AIX with the xlC compiler. However, we have
>>>> seen similar optimizations on other platforms and other platforms.
>>>>
>>>> As this code pattern is used quite often, I would suggest a global fix in
>>>> oopDesc::load_heap_oop(p).
>>>>
>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8129440
>>>> Webrev: http://cr.openjdk.java.net/~asiebenborn/8129440/webrev.00/
>>>>
>>>> Thanks,
>>>> Axel
>>>>

From jon.masamitsu at oracle.com  Thu Jul  9 16:50:28 2015
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Thu, 09 Jul 2015 09:50:28 -0700
Subject: RFR(XS): JDK-8129440 G1 crash during concurrent root region scan.
In-Reply-To: <02D5D45C1F8DB848A7AE20E80EE61A5C3994BB9A@DEWDFEMB20C.global.corp.sap>
References: <02D5D45C1F8DB848A7AE20E80EE61A5C39945B54@DEWDFEMB20C.global.corp.sap>	<CA+3eh12JaZRDsvmbtssi7wqAsbR7kO3F2_j6vjCWqbu0LBJwdA@mail.gmail.com>
	<02D5D45C1F8DB848A7AE20E80EE61A5C3994BB9A@DEWDFEMB20C.global.corp.sap>
Message-ID: <559EA654.40009@oracle.com>



On 7/9/2015 1:29 AM, Siebenborn, Axel wrote:
> On 06.07.2015 at 15:03 Volker Simonis wrote:
>> On Tue, Jun 23, 2015 at 8:45 AM, Siebenborn, Axel
>> <axel.siebenborn at sap.com> wrote:
>>>
>>> Hi Volker,
>>>
>>> thanks for your comment.Indeed, this was my first idea for a fix. However,
>>> there are more than 50 callers of  load_heap_oop, plus some generated by
>>> preprocessor defines like this one:
>>>
>>> #define DO_OOP_WORK_IMPL(closureName) \
>>>    template <class T> inline void closureName##Closure::do_oop_work(T* p) {
>>> \
>>>      T heap_oop = oopDesc::load_heap_oop(p);               \
>>>      if (!oopDesc::is_null(heap_oop)) {                    \
>>>        oop obj = oopDesc::decode_heap_oop_not_null(heap_oop);       \
>>>        do_oop(obj);                                        \
>>>      }                                                     \
>>>    }
>>>
>>> We would have to decide for each of the callers, if there might be a
>>> concurrency problem or not.
>>>
>>> Do you think, it is worth the effort and the risk of missing one?
>>>
>> OK, you're probably right.
>>
>> If you haven?t seen any performance regressions caused by this fix I'm
>> fine with your solution.
>>
>> Unfortunately I can not push this shared-code change so can we please
>> get a sponsor from within Oracle for this change?
>>
>> Thanks,
>> Volker
> We did not see any performance regressions.
> On Itanium compilers add release and acquire semantics to stores and loads of volatile variables. The additional memory barriers really could have a performance impact. But even on Itanium we don?t see any performance regression in our benchmarks.
> On other platforms, optimizations prevented by the volatile qualifier are optimizations that we really don't want.

Did you do performance measurements on GC pause times?

Jon

>   
>
> Thanks,
> Axel
>
>>> Regards,
>>>
>>> Axel
>>>
>>> On 22.06.2015 at 18:17 Volker Simonis wrote:
>>>> Hi Axel,
>>>>
>>>> the change looks good, but shouldn't we better provide two versions of
>>>> load_heap_oop()
>>>>
>>>> inline oop       oopDesc::load_heap_oop(oop* p)
>>>> inline oop       oopDesc::load_heap_oop(volatile oop* p)
>>>>
>>>> such that the callers can decide on their own which versions they require?
>>>>
>>>> Or do the callers of load_heap_oop() not always know if there exist
>>>> concurrent mutator threads?
>>>>
>>>> Regards,
>>>> Volker
>>>>
>>>>
>>>> On Mon, Jun 22, 2015 at 5:33 PM, Siebenborn, Axel
>>>> <axel.siebenborn at sap.com> wrote:
>>>>> Hi,
>>>>> we have seen crashes with G1 during concurrent root region scan.
>>>>> It turned out, that the reason for the crashes was reloading an oop after
>>>>> a null check.
>>>>> The compiler may legally do this, as the pointer is not declared as
>>>>> volatile.
>>>>> The code runs concurrently to mutator threads.
>>>>>
>>>>> template <class T>
>>>>> inline void G1RootRegionScanClosure::do_oop_nv(T* p) {
>>>>>    T heap_oop = oopDesc::load_heap_oop(p);
>>>>> // 1. Load oop
>>>>>    if (!oopDesc::is_null(heap_oop)) {
>>>>>      oop obj = oopDesc::decode_heap_oop_not_null(heap_oop);    // 2.
>>>>> Compiler decides to re-load the oop
>>>>>      HeapRegion* hr = _g1h->heap_region_containing((HeapWord*) obj);
>>>>>      _cm->grayRoot(obj, obj->size(), _worker_id, hr);
>>>>>    }
>>>>> }
>>>>>
>>>>>
>>>>> We have seen the problem on AIX with the xlC compiler. However, we have
>>>>> seen similar optimizations on other platforms and other platforms.
>>>>>
>>>>> As this code pattern is used quite often, I would suggest a global fix in
>>>>> oopDesc::load_heap_oop(p).
>>>>>
>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8129440
>>>>> Webrev: http://cr.openjdk.java.net/~asiebenborn/8129440/webrev.00/
>>>>>
>>>>> Thanks,
>>>>> Axel
>>>>>



From axel.siebenborn at sap.com  Fri Jul 10 07:38:57 2015
From: axel.siebenborn at sap.com (Siebenborn, Axel)
Date: Fri, 10 Jul 2015 07:38:57 +0000
Subject: RFR(XS): JDK-8129440 G1 crash during concurrent root region scan.
In-Reply-To: <559EA654.40009@oracle.com>
References: <02D5D45C1F8DB848A7AE20E80EE61A5C39945B54@DEWDFEMB20C.global.corp.sap>
	<CA+3eh12JaZRDsvmbtssi7wqAsbR7kO3F2_j6vjCWqbu0LBJwdA@mail.gmail.com>
	<02D5D45C1F8DB848A7AE20E80EE61A5C3994BB9A@DEWDFEMB20C.global.corp.sap>
	<559EA654.40009@oracle.com>
Message-ID: <02D5D45C1F8DB848A7AE20E80EE61A5C3994C017@DEWDFEMB20C.global.corp.sap>



On 09.07.2015 at 18:50 Jon Masamitsu wrote:
>
> On 7/9/2015 1:29 AM, Siebenborn, Axel wrote:
>> On 06.07.2015 at 15:03 Volker Simonis wrote:
>>> On Tue, Jun 23, 2015 at 8:45 AM, Siebenborn, Axel
>>> <axel.siebenborn at sap.com> wrote:
>>>>
>>>> Hi Volker,
>>>>
>>>> thanks for your comment.Indeed, this was my first idea for a fix. However,
>>>> there are more than 50 callers of  load_heap_oop, plus some generated by
>>>> preprocessor defines like this one:
>>>>
>>>> #define DO_OOP_WORK_IMPL(closureName) \
>>>>    template <class T> inline void closureName##Closure::do_oop_work(T* p) {
>>>> \
>>>>      T heap_oop = oopDesc::load_heap_oop(p);               \
>>>>      if (!oopDesc::is_null(heap_oop)) {                    \
>>>>        oop obj = oopDesc::decode_heap_oop_not_null(heap_oop);       \
>>>>        do_oop(obj);                                        \
>>>>      }                                                     \
>>>>    }
>>>>
>>>> We would have to decide for each of the callers, if there might be a
>>>> concurrency problem or not.
>>>>
>>>> Do you think, it is worth the effort and the risk of missing one?
>>> OK, you're probably right.
>>>
>>> If you haven?t seen any performance regressions caused by this fix I'm
>>> fine with your solution.
>>>
>>> Unfortunately I can not push this shared-code change so can we please
>>> get a sponsor from within Oracle for this change?
>>>
>>> Thanks,
>>> Volker 
>> We did not see any performance regressions.
>> On Itanium compilers add release and acquire semantics to stores and loads of volatile variables. The additional memory barriers really could have a performance impact. But even on Itanium we don?t see any performance regression in our benchmarks.
>> On other platforms, optimizations prevented by the volatile qualifier are optimizations that we really don't want. 
>
> Did you do performance measurements on GC pause times?
>
> Jon
I checked the GC pause times in standard benchmarks for ParrallelGC, CMS and G1. I don't see regressions here.

Axel

>>  
>> Thanks,
>> Axel
>>>> Regards,
>>>>
>>>> Axel
>>>>
>>>> On 22.06.2015 at 18:17 Volker Simonis wrote:
>>>>> Hi Axel,
>>>>>
>>>>> the change looks good, but shouldn't we better provide two versions of
>>>>> load_heap_oop()
>>>>>
>>>>> inline oop       oopDesc::load_heap_oop(oop* p)
>>>>> inline oop       oopDesc::load_heap_oop(volatile oop* p)
>>>>>
>>>>> such that the callers can decide on their own which versions they require?
>>>>>
>>>>> Or do the callers of load_heap_oop() not always know if there exist
>>>>> concurrent mutator threads?
>>>>>
>>>>> Regards,
>>>>> Volker
>>>>>
>>>>>
>>>>> On Mon, Jun 22, 2015 at 5:33 PM, Siebenborn, Axel
>>>>> <axel.siebenborn at sap.com> wrote:
>>>>>> Hi,
>>>>>> we have seen crashes with G1 during concurrent root region scan.
>>>>>> It turned out, that the reason for the crashes was reloading an oop after
>>>>>> a null check.
>>>>>> The compiler may legally do this, as the pointer is not declared as
>>>>>> volatile.
>>>>>> The code runs concurrently to mutator threads.
>>>>>>
>>>>>> template <class T>
>>>>>> inline void G1RootRegionScanClosure::do_oop_nv(T* p) {
>>>>>>    T heap_oop = oopDesc::load_heap_oop(p);
>>>>>> // 1. Load oop
>>>>>>    if (!oopDesc::is_null(heap_oop)) {
>>>>>>      oop obj = oopDesc::decode_heap_oop_not_null(heap_oop);    // 2.
>>>>>> Compiler decides to re-load the oop
>>>>>>      HeapRegion* hr = _g1h->heap_region_containing((HeapWord*) obj);
>>>>>>      _cm->grayRoot(obj, obj->size(), _worker_id, hr);
>>>>>>    }
>>>>>> }
>>>>>>
>>>>>>
>>>>>> We have seen the problem on AIX with the xlC compiler. However, we have
>>>>>> seen similar optimizations on other platforms and other platforms.
>>>>>>
>>>>>> As this code pattern is used quite often, I would suggest a global fix in
>>>>>> oopDesc::load_heap_oop(p).
>>>>>>
>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8129440
>>>>>> Webrev: http://cr.openjdk.java.net/~asiebenborn/8129440/webrev.00/
>>>>>>
>>>>>> Thanks,
>>>>>> Axel

From derek.white at oracle.com  Fri Jul 10 15:28:11 2015
From: derek.white at oracle.com (Derek White)
Date: Fri, 10 Jul 2015 11:28:11 -0400
Subject: RFR (XS): 8078673: Update TEST.groups for recent GC tests
In-Reply-To: <47A58D96-A29E-44B4-84EE-F8FBA162B4C4@oracle.com>
References: <553FFC31.8030101@oracle.com> <5540EE43.6060101@oracle.com>
	<5540F209.9040700@oracle.com> <554105DE.70601@oracle.com>
	<55410DB4.8030007@oracle.com> <55411182.70805@oracle.com>
	<55411D9B.6090308@oracle.com>
	<42D796DF-4043-459E-999B-E1AAA365827A@oracle.com>
	<55414139.3010201@oracle.com>
	<47A58D96-A29E-44B4-84EE-F8FBA162B4C4@oracle.com>
Message-ID: <559FE48B.1030003@oracle.com>

Please review this partial fix for GC tests that require certain 
collectors (e.g. shouldn't run in embedded).

This is an updated webrev to account for Leonid's fix for "8079134: 
[TESTBUG] Remove applicable_*gc and needs_*gc groups from TEST.groups", 
which removed a pile of TEST.groups lists including needs_gc, 
needs_serialgc, needs_parallelgc, and needs_cmsgc.

Now the fix simply updates the needs_g1gc list in TEST.groups and adds 
appropriate "@requires vm.gc" annotations to a few GC tests.

Note that the "@requires vm.gc" changes are /almost/ purely documentary. 
These are ProcessBuilder-based tests, so any "-XX:+UsexxxGC" flags 
passed in by jtreg are ignored. It's very confusing, as well as 
unnecessary, for a jtreg run specifying -XX:+UseG1GC to be running a 
test that then replaces the flag with "-XX:+UseParallelGC" (etc).

*CR:*
https://bugs.openjdk.java.net/browse/JDK-8078673

*Webrev:*
http://cr.openjdk.java.net/~drwhite/8078673/webrev.03/

*Testing:*
  JPRT

*Open review comments:*
David H: Your last comments on this subject requested changes to the 
"needs_gc" list, which has been removed by 8079134.

Kim and Jesper: I agree with your comments about wanting some better for 
both testing multiple GCs and dealing with SE Embedded properly in the 
testing infrastructure. This webrev is simply a good fix within the 
existing infrastructure.

Thanks,
  - Derek

On 4/29/15 5:03 PM, Kim Barrett wrote:
> On Apr 29, 2015, at 4:38 PM, Jesper Wilhelmsson <jesper.wilhelmsson at oracle.com> wrote:
>> Hi,
>>
>> There are tests like hotspot/test/gc/g1/TestShrinkAuxiliaryData**.java where there is a base class that provides the test and a bunch of test classes that only starts the base test with different arguments. This case would be similar but slightly more ugly since the actual code would be the same and trigger the same base test, but with different @requires in the comment above the test.
>>
>> I'm not sure it would help though. What we really would need here is a @requires that could check the host name or the hardware platform or OS.
> @requires has os.{name, family, arch, simpleArch} properties that can be tested.  But I?m not sure any of those are really right for testing for ?embedded system? (whatever that actually means).
>
>> Kim Barrett skrev den 29/4/15 20:35:
>>> On Apr 29, 2015, at 2:06 PM, Derek White <derek.white at oracle.com> wrote:
>>>> So most of these tests use ProcessBuilder to specify a command line, and explicitly mention a GC to use. A single test might actually run each collector (gc/logging/TestGCId, for example). Does @requires matter in this case?
>>>>
>>>> Oh, maybe they should all have @requires vm.gc=="null", becuase the actual test is ignoring GC passed in by the test harness GC and running as a separate process anyway. It's misleading if the harness said UseG1GC and the test was actually running UseParallelGC, for example.
>>> That?s one solution.  A different solution would be to clone into multiple tests, one for each relevant collector, where the vm.gc can be ?null? or the corresponding collector.  That cloning is kind of ugly though; it?s too bad there can only be one @requires constraint per test, rather than per @run line or the like.  But we didn?t get any traction with such a suggestion: https://bugs.openjdk.java.net/browse/CODETOOLS-7901090.
>>>
>>>> FYI, it sounds like my original problem does require the exclude lists to keep the embedded JVM from running GC tests that it shouldn't.
>>> I?m not sure how to address this problem.  For example, we don?t want to exclude TestSmallHeap.java on embedded JVMs, we just want to exclude its sub-cases that would attempt to use a gc that isn?t supported.
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150710/842a8bfd/attachment.htm>

From derek.white at oracle.com  Fri Jul 10 15:40:30 2015
From: derek.white at oracle.com (Derek White)
Date: Fri, 10 Jul 2015 11:40:30 -0400
Subject: Runtime command-line deprecation (Was Re: RFR: 8066821(S) Enhance
	command line processing to manage deprecating and obsoleting -XX
	command line arguments
In-Reply-To: <559AEC80.3010907@oracle.com>
References: <54B44661.8070007@oracle.com> <54B84E91.1050708@oracle.com>
	<F68EBF6B-38A5-4461-9ADD-41BB198FF943@oracle.com>
	<54D15A0B.6030501@oracle.com>
	<87B07B03-3423-4738-9617-0C7B72A74A6E@oracle.com>
	<559AEC80.3010907@oracle.com>
Message-ID: <559FE76E.2060009@oracle.com>

Hi Karen, Coleen,

FYI - I'm getting back to this (was waiting for Gerard's big change). 
Once I have this merged and tested I'll send it out for review sometime 
next week. To the right alias this time :-)

  - Derek

On 7/6/15 5:00 PM, Coleen Phillimore wrote:
>
> Hi,
>
> I'm happy for more clarity in the process of removing command line 
> arguments.  I have some questions and comments below.
>
> On 6/26/15 4:09 PM, Karen Kinnear wrote:
>> Derek,
>>
>> ...
> So this is my question which is around the code review in question.   
> Note, Derek, can you resend the RFR to hotspot-dev at openjdk.java.net so 
> it gets to all of us (even the compiler team may want to share in the 
> mechanism).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150710/27bfff76/attachment.htm>

From jon.masamitsu at oracle.com  Fri Jul 10 15:32:54 2015
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Fri, 10 Jul 2015 08:32:54 -0700
Subject: RFR(XS): JDK-8129440 G1 crash during concurrent root region scan.
In-Reply-To: <02D5D45C1F8DB848A7AE20E80EE61A5C3994C017@DEWDFEMB20C.global.corp.sap>
References: <02D5D45C1F8DB848A7AE20E80EE61A5C39945B54@DEWDFEMB20C.global.corp.sap>	<CA+3eh12JaZRDsvmbtssi7wqAsbR7kO3F2_j6vjCWqbu0LBJwdA@mail.gmail.com>	<02D5D45C1F8DB848A7AE20E80EE61A5C3994BB9A@DEWDFEMB20C.global.corp.sap>
	<559EA654.40009@oracle.com>
	<02D5D45C1F8DB848A7AE20E80EE61A5C3994C017@DEWDFEMB20C.global.corp.sap>
Message-ID: <559FE5A6.4030204@oracle.com>



On 07/10/2015 12:38 AM, Siebenborn, Axel wrote:
>
> On 09.07.2015 at 18:50 Jon Masamitsu wrote:
>> On 7/9/2015 1:29 AM, Siebenborn, Axel wrote:
>>> On 06.07.2015 at 15:03 Volker Simonis wrote:
>>>> On Tue, Jun 23, 2015 at 8:45 AM, Siebenborn, Axel
>>>> <axel.siebenborn at sap.com> wrote:
>>>>> Hi Volker,
>>>>>
>>>>> thanks for your comment.Indeed, this was my first idea for a fix. However,
>>>>> there are more than 50 callers of  load_heap_oop, plus some generated by
>>>>> preprocessor defines like this one:
>>>>>
>>>>> #define DO_OOP_WORK_IMPL(closureName) \
>>>>>     template <class T> inline void closureName##Closure::do_oop_work(T* p) {
>>>>> \
>>>>>       T heap_oop = oopDesc::load_heap_oop(p);               \
>>>>>       if (!oopDesc::is_null(heap_oop)) {                    \
>>>>>         oop obj = oopDesc::decode_heap_oop_not_null(heap_oop);       \
>>>>>         do_oop(obj);                                        \
>>>>>       }                                                     \
>>>>>     }
>>>>>
>>>>> We would have to decide for each of the callers, if there might be a
>>>>> concurrency problem or not.
>>>>>
>>>>> Do you think, it is worth the effort and the risk of missing one?
>>>> OK, you're probably right.
>>>>
>>>> If you haven?t seen any performance regressions caused by this fix I'm
>>>> fine with your solution.
>>>>
>>>> Unfortunately I can not push this shared-code change so can we please
>>>> get a sponsor from within Oracle for this change?
>>>>
>>>> Thanks,
>>>> Volker
>>> We did not see any performance regressions.
>>> On Itanium compilers add release and acquire semantics to stores and loads of volatile variables. The additional memory barriers really could have a performance impact. But even on Itanium we don?t see any performance regression in our benchmarks.
>>> On other platforms, optimizations prevented by the volatile qualifier are optimizations that we really don't want.
>> Did you do performance measurements on GC pause times?
>>
>> Jon
> I checked the GC pause times in standard benchmarks for ParrallelGC, CMS and G1. I don't see regressions here.

Great.  Can you send me the GC log files.  Probably just to me is OK 
unless someone
else want to look at them.

Jon

>
> Axel
>
>>>   
>>> Thanks,
>>> Axel
>>>>> Regards,
>>>>>
>>>>> Axel
>>>>>
>>>>> On 22.06.2015 at 18:17 Volker Simonis wrote:
>>>>>> Hi Axel,
>>>>>>
>>>>>> the change looks good, but shouldn't we better provide two versions of
>>>>>> load_heap_oop()
>>>>>>
>>>>>> inline oop       oopDesc::load_heap_oop(oop* p)
>>>>>> inline oop       oopDesc::load_heap_oop(volatile oop* p)
>>>>>>
>>>>>> such that the callers can decide on their own which versions they require?
>>>>>>
>>>>>> Or do the callers of load_heap_oop() not always know if there exist
>>>>>> concurrent mutator threads?
>>>>>>
>>>>>> Regards,
>>>>>> Volker
>>>>>>
>>>>>>
>>>>>> On Mon, Jun 22, 2015 at 5:33 PM, Siebenborn, Axel
>>>>>> <axel.siebenborn at sap.com> wrote:
>>>>>>> Hi,
>>>>>>> we have seen crashes with G1 during concurrent root region scan.
>>>>>>> It turned out, that the reason for the crashes was reloading an oop after
>>>>>>> a null check.
>>>>>>> The compiler may legally do this, as the pointer is not declared as
>>>>>>> volatile.
>>>>>>> The code runs concurrently to mutator threads.
>>>>>>>
>>>>>>> template <class T>
>>>>>>> inline void G1RootRegionScanClosure::do_oop_nv(T* p) {
>>>>>>>     T heap_oop = oopDesc::load_heap_oop(p);
>>>>>>> // 1. Load oop
>>>>>>>     if (!oopDesc::is_null(heap_oop)) {
>>>>>>>       oop obj = oopDesc::decode_heap_oop_not_null(heap_oop);    // 2.
>>>>>>> Compiler decides to re-load the oop
>>>>>>>       HeapRegion* hr = _g1h->heap_region_containing((HeapWord*) obj);
>>>>>>>       _cm->grayRoot(obj, obj->size(), _worker_id, hr);
>>>>>>>     }
>>>>>>> }
>>>>>>>
>>>>>>>
>>>>>>> We have seen the problem on AIX with the xlC compiler. However, we have
>>>>>>> seen similar optimizations on other platforms and other platforms.
>>>>>>>
>>>>>>> As this code pattern is used quite often, I would suggest a global fix in
>>>>>>> oopDesc::load_heap_oop(p).
>>>>>>>
>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8129440
>>>>>>> Webrev: http://cr.openjdk.java.net/~asiebenborn/8129440/webrev.00/
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Axel



From eric.caspole at oracle.com  Fri Jul 10 19:52:39 2015
From: eric.caspole at oracle.com (Eric Caspole)
Date: Fri, 10 Jul 2015 15:52:39 -0400
Subject: RFR: JDK-8129961 : SIGSEGV when copying to survivor space
Message-ID: <55A02287.20004@oracle.com>

Hi everybody,
Please review this fix for JDK-8129961

  https://bugs.openjdk.java.net/browse/JDK-8129961

  http://cr.openjdk.java.net/~ecaspole/JDK-8129961/webrev/

The problem was that the enum GenCollectedHeap::YoungGen == 0  and it 
was interpreted as "false" in no_allocs_since_save_marks() after a 
recent refactoring.

Since there is no place in the code either serial gc or parNew where it 
ever passes "false" to avoid rescanning young gens, I want to remove the 
parameter so it will always scan young and old gens. That is simpler to 
read and removes this type of problem.
Passed JPRT.
Let me know what you think.
Thanks,
Eric



From kim.barrett at oracle.com  Fri Jul 10 20:10:45 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Fri, 10 Jul 2015 16:10:45 -0400
Subject: RFR: JDK-8129961 : SIGSEGV when copying to survivor space
In-Reply-To: <55A02287.20004@oracle.com>
References: <55A02287.20004@oracle.com>
Message-ID: <99D3340E-0365-4D0F-BB2C-7EBBF5608AF1@oracle.com>

On Jul 10, 2015, at 3:52 PM, Eric Caspole <eric.caspole at oracle.com> wrote:
> 
> Hi everybody,
> Please review this fix for JDK-8129961
> 
> https://bugs.openjdk.java.net/browse/JDK-8129961
> 
> http://cr.openjdk.java.net/~ecaspole/JDK-8129961/webrev/
> 
> The problem was that the enum GenCollectedHeap::YoungGen == 0  and it was interpreted as "false" in no_allocs_since_save_marks() after a recent refactoring.
> 
> Since there is no place in the code either serial gc or parNew where it ever passes "false" to avoid rescanning young gens, I want to remove the parameter so it will always scan young and old gens. That is simpler to read and removes this type of problem.
> Passed JPRT.
> Let me know what you think.

------------------------------------------------------------------------------
src/share/vm/gc/shared/genCollectedHeap.cpp
 744 bool GenCollectedHeap::no_allocs_since_save_marks() {
 745   if (!_young_gen->no_allocs_since_save_marks()) {
 746     return false;
 747   }
 748   return _old_gen->no_allocs_since_save_marks();
 749 }

This looks rather contorted now.  Why not just

  return
    _young_gen->no_allocs_since_save_marks() &&
    _old_gen->no_allocs_since_save_marks();

------------------------------------------------------------------------------

Otherwise, looks good.



From eric.caspole at oracle.com  Fri Jul 10 21:36:30 2015
From: eric.caspole at oracle.com (Eric Caspole)
Date: Fri, 10 Jul 2015 17:36:30 -0400
Subject: RFR: JDK-8129961 : SIGSEGV when copying to survivor space
In-Reply-To: <99D3340E-0365-4D0F-BB2C-7EBBF5608AF1@oracle.com>
References: <55A02287.20004@oracle.com>
	<99D3340E-0365-4D0F-BB2C-7EBBF5608AF1@oracle.com>
Message-ID: <55A03ADE.7030306@oracle.com>

Good idea, I'll do that.
Eric

On 7/10/2015 4:10 PM, Kim Barrett wrote:
> On Jul 10, 2015, at 3:52 PM, Eric Caspole <eric.caspole at oracle.com> wrote:
>> Hi everybody,
>> Please review this fix for JDK-8129961
>>
>> https://bugs.openjdk.java.net/browse/JDK-8129961
>>
>> http://cr.openjdk.java.net/~ecaspole/JDK-8129961/webrev/
>>
>> The problem was that the enum GenCollectedHeap::YoungGen == 0  and it was interpreted as "false" in no_allocs_since_save_marks() after a recent refactoring.
>>
>> Since there is no place in the code either serial gc or parNew where it ever passes "false" to avoid rescanning young gens, I want to remove the parameter so it will always scan young and old gens. That is simpler to read and removes this type of problem.
>> Passed JPRT.
>> Let me know what you think.
> ------------------------------------------------------------------------------
> src/share/vm/gc/shared/genCollectedHeap.cpp
>   744 bool GenCollectedHeap::no_allocs_since_save_marks() {
>   745   if (!_young_gen->no_allocs_since_save_marks()) {
>   746     return false;
>   747   }
>   748   return _old_gen->no_allocs_since_save_marks();
>   749 }
>
> This looks rather contorted now.  Why not just
>
>    return
>      _young_gen->no_allocs_since_save_marks() &&
>      _old_gen->no_allocs_since_save_marks();
>
> ------------------------------------------------------------------------------
>
> Otherwise, looks good.
>



From kim.barrett at oracle.com  Sat Jul 11 00:23:48 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Fri, 10 Jul 2015 20:23:48 -0400
Subject: RFR: 8130931: refactor CardTableModRefBS[ForCTRS]
Message-ID: <BF2A976B-4C2D-4AC9-9627-181D1B866735@oracle.com>

Please review this refactoring of CardTableModRefBS and
CardTableModRefBSForCTRS.  The changes are in two stages, with
separate webrevs to make it easier to review.

The first stage eliminates some friend declarations from
CardTableModRefBS.

- BytecodeInterpreter doesn't need to be a friend.
- CheckForUnmarkedOops changed to use public byte_for_const().
- SharkBuilder changed to use public dirty_card_val().
- GuaranteeNotModClosure does not exist.
- CardTableRS changed to be a friend of CardTableModRefBSForCTRS. 
  Changed some references to static members of CardTableModRefBS
  to work around bugs in clang and Visual Studio [1].

Only the VMStructs friend declaration is retained.

The second stage moves some functionality that is specific to or only
used with the ForCTRS class from the base class to the ForCTRS
subclass. It also moves the ForCTRS class into its own files, separate
from the base class.

We could further split below CardTableModRefBSForCTRS, with support
for parallel card scanning (used only by CMS) being in another class.
That split already partially exists in the source code structure, with
some relevant functions being defined in CMS-specific
parCardTableModRefBS.cpp. That restructuring is being left for later.

CR:
https://bugs.openjdk.java.net/browse/JDK-8130931

Full webrev:
http://cr.openjdk.java.net/~kbarrett/8130931/webrev.00/

De-friending webrev:
http://cr.openjdk.java.net/~kbarrett/8130931/remove-friends.00/

Refactoring webrev:
http://cr.openjdk.java.net/~kbarrett/8130931/refactor.00/

Testing:
JPRT
local specjbb2000 & specjbb2005 with G1, CMS, and Parallel GCs
Aurora Runtime+GC Nightly & VM quicklist, using CMS (in progress) 
Aurora Runtime+GC Nightly & VM quicklist, using G1 (in progress) 

[1] Both clang and Visual Studio are more restrictive than the
standard specifies for friend access to static members of base classes
of the class granting friendship. The relevant standardese is C++03
11.5 [class.protected] (C++11 11.4). At least in the case of clang,
this non-conformance is intentional. For gory details, see:

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=52136
https://llvm.org/bugs/show_bug.cgi?id=6840
http://lists.cs.uiuc.edu/pipermail/cfe-dev/2010-June/thread.html#9301

Having read all that, I'm inclined to agree with the clang folks - the
special dispensation for static members is very strange. But so far
the clang folks seem to be just ignoring the problem, and don't seem
to have opened a DR for it.

I don't have any similar background information for Visual Studio's
non-conformance.



From jon.masamitsu at oracle.com  Mon Jul 13 01:57:39 2015
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Sun, 12 Jul 2015 18:57:39 -0700
Subject: RFR: JDK-8129961 : SIGSEGV when copying to survivor space
In-Reply-To: <55A02287.20004@oracle.com>
References: <55A02287.20004@oracle.com>
Message-ID: <55A31B13.5080001@oracle.com>

Eric,

With Kim's suggestion it looks good.

Reviewed.

Jon

On 7/10/2015 12:52 PM, Eric Caspole wrote:
> Hi everybody,
> Please review this fix for JDK-8129961
>
>  https://bugs.openjdk.java.net/browse/JDK-8129961
>
>  http://cr.openjdk.java.net/~ecaspole/JDK-8129961/webrev/
>
> The problem was that the enum GenCollectedHeap::YoungGen == 0  and it 
> was interpreted as "false" in no_allocs_since_save_marks() after a 
> recent refactoring.
>
> Since there is no place in the code either serial gc or parNew where 
> it ever passes "false" to avoid rescanning young gens, I want to 
> remove the parameter so it will always scan young and old gens. That 
> is simpler to read and removes this type of problem.
> Passed JPRT.
> Let me know what you think.
> Thanks,
> Eric
>



From david.holmes at oracle.com  Mon Jul 13 02:38:45 2015
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 13 Jul 2015 12:38:45 +1000
Subject: RFR (XS): 8078673: Update TEST.groups for recent GC tests
In-Reply-To: <559FE48B.1030003@oracle.com>
References: <553FFC31.8030101@oracle.com> <5540EE43.6060101@oracle.com>
	<5540F209.9040700@oracle.com> <554105DE.70601@oracle.com>
	<55410DB4.8030007@oracle.com> <55411182.70805@oracle.com>
	<55411D9B.6090308@oracle.com>
	<42D796DF-4043-459E-999B-E1AAA365827A@oracle.com>
	<55414139.3010201@oracle.com>
	<47A58D96-A29E-44B4-84EE-F8FBA162B4C4@oracle.com>
	<559FE48B.1030003@oracle.com>
Message-ID: <55A324B5.106@oracle.com>

Hi Derek,

On 11/07/2015 1:28 AM, Derek White wrote:
> Please review this partial fix for GC tests that require certain
> collectors (e.g. shouldn't run in embedded).
>
> This is an updated webrev to account for Leonid's fix for "8079134:
> [TESTBUG] Remove applicable_*gc and needs_*gc groups from TEST.groups",
> which removed a pile of TEST.groups lists including needs_gc,
> needs_serialgc, needs_parallelgc, and needs_cmsgc.
>
> Now the fix simply updates the needs_g1gc list in TEST.groups and adds
> appropriate "@requires vm.gc" annotations to a few GC tests.
>
> Note that the "@requires vm.gc" changes are /almost/ purely documentary.
> These are ProcessBuilder-based tests, so any "-XX:+UsexxxGC" flags
> passed in by jtreg are ignored. It's very confusing, as well as
> unnecessary, for a jtreg run specifying -XX:+UseG1GC to be running a
> test that then replaces the flag with "-XX:+UseParallelGC" (etc).

So even though we would never pass through the jtreg specific GC option 
we will skip these tests if that option doesn't match with the GC's the 
test will be testing. As long as that doesn't lead to these tests being 
untested that seems okay.

But really this highlights a basic problem we have with our approach to 
testing with different VM options. Unless the GC option is the only 
option that changes across the runs you would want the other options to 
be passed through to the actual tests in many cases. :(

Cheers,
David


> *CR:*
> https://bugs.openjdk.java.net/browse/JDK-8078673
>
> *Webrev:*
> http://cr.openjdk.java.net/~drwhite/8078673/webrev.03/
>
> *Testing:*
>   JPRT
>
> *Open review comments:*
> David H: Your last comments on this subject requested changes to the
> "needs_gc" list, which has been removed by 8079134.
>
> Kim and Jesper: I agree with your comments about wanting some better for
> both testing multiple GCs and dealing with SE Embedded properly in the
> testing infrastructure. This webrev is simply a good fix within the
> existing infrastructure.
>
> Thanks,
>   - Derek
>
> On 4/29/15 5:03 PM, Kim Barrett wrote:
>> On Apr 29, 2015, at 4:38 PM, Jesper Wilhelmsson<jesper.wilhelmsson at oracle.com>  wrote:
>>> Hi,
>>>
>>> There are tests like hotspot/test/gc/g1/TestShrinkAuxiliaryData**.java where there is a base class that provides the test and a bunch of test classes that only starts the base test with different arguments. This case would be similar but slightly more ugly since the actual code would be the same and trigger the same base test, but with different @requires in the comment above the test.
>>>
>>> I'm not sure it would help though. What we really would need here is a @requires that could check the host name or the hardware platform or OS.
>> @requires has os.{name, family, arch, simpleArch} properties that can be tested.  But I?m not sure any of those are really right for testing for ?embedded system? (whatever that actually means).
>>
>>> Kim Barrett skrev den 29/4/15 20:35:
>>>> On Apr 29, 2015, at 2:06 PM, Derek White<derek.white at oracle.com>  wrote:
>>>>> So most of these tests use ProcessBuilder to specify a command line, and explicitly mention a GC to use. A single test might actually run each collector (gc/logging/TestGCId, for example). Does @requires matter in this case?
>>>>>
>>>>> Oh, maybe they should all have @requires vm.gc=="null", becuase the actual test is ignoring GC passed in by the test harness GC and running as a separate process anyway. It's misleading if the harness said UseG1GC and the test was actually running UseParallelGC, for example.
>>>> That?s one solution.  A different solution would be to clone into multiple tests, one for each relevant collector, where the vm.gc can be ?null? or the corresponding collector.  That cloning is kind of ugly though; it?s too bad there can only be one @requires constraint per test, rather than per @run line or the like.  But we didn?t get any traction with such a suggestion:https://bugs.openjdk.java.net/browse/CODETOOLS-7901090.
>>>>
>>>>> FYI, it sounds like my original problem does require the exclude lists to keep the embedded JVM from running GC tests that it shouldn't.
>>>> I?m not sure how to address this problem.  For example, we don?t want to exclude TestSmallHeap.java on embedded JVMs, we just want to exclude its sub-cases that would attempt to use a gc that isn?t supported.
>>
>


From thomas.schatzl at oracle.com  Mon Jul 13 08:47:04 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 13 Jul 2015 10:47:04 +0200
Subject: RFR: 8130931: refactor CardTableModRefBS[ForCTRS]
In-Reply-To: <BF2A976B-4C2D-4AC9-9627-181D1B866735@oracle.com>
References: <BF2A976B-4C2D-4AC9-9627-181D1B866735@oracle.com>
Message-ID: <1436777224.2275.18.camel@oracle.com>

Hi,

On Fri, 2015-07-10 at 20:23 -0400, Kim Barrett wrote:
> Please review this refactoring of CardTableModRefBS and
> CardTableModRefBSForCTRS.  The changes are in two stages, with
> separate webrevs to make it easier to review.
> 
> The first stage eliminates some friend declarations from
> CardTableModRefBS.
> 
> - BytecodeInterpreter doesn't need to be a friend.
> - CheckForUnmarkedOops changed to use public byte_for_const().

Actually the whole member _unmarked_card can be removed.

> - SharkBuilder changed to use public dirty_card_val().
> - GuaranteeNotModClosure does not exist.
> - CardTableRS changed to be a friend of CardTableModRefBSForCTRS. 
>   Changed some references to static members of CardTableModRefBS
>   to work around bugs in clang and Visual Studio [1].
> 
> Only the VMStructs friend declaration is retained.
> 
> The second stage moves some functionality that is specific to or only
> used with the ForCTRS class from the base class to the ForCTRS
> subclass. It also moves the ForCTRS class into its own files, separate
> from the base class.

I would prefer if the parCardTableModRefBS.cpp file would be removed,
its contents moved to the new CardTableModRefBSForCTRS.cpp as it only
contains CardTableModRefBSForCTRS methods (and both seem only used for
CMS).
Iow, is there some reason to keep this file, or is this part of the
"left for later" task?

> We could further split below CardTableModRefBSForCTRS, with support
> for parallel card scanning (used only by CMS) being in another class.
> That split already partially exists in the source code structure, with
> some relevant functions being defined in CMS-specific
> parCardTableModRefBS.cpp. That restructuring is being left for later.

Looks good otherwise. Thanks for the cleanup.

Thanks,
  Thomas




From derek.white at oracle.com  Mon Jul 13 21:37:52 2015
From: derek.white at oracle.com (Derek White)
Date: Mon, 13 Jul 2015 17:37:52 -0400
Subject: RFR (XS): 8078673: Update TEST.groups for recent GC tests
In-Reply-To: <55A324B5.106@oracle.com>
References: <553FFC31.8030101@oracle.com> <5540EE43.6060101@oracle.com>
	<5540F209.9040700@oracle.com> <554105DE.70601@oracle.com>
	<55410DB4.8030007@oracle.com> <55411182.70805@oracle.com>
	<55411D9B.6090308@oracle.com>
	<42D796DF-4043-459E-999B-E1AAA365827A@oracle.com>
	<55414139.3010201@oracle.com>
	<47A58D96-A29E-44B4-84EE-F8FBA162B4C4@oracle.com>
	<559FE48B.1030003@oracle.com> <55A324B5.106@oracle.com>
Message-ID: <55A42FB0.5050909@oracle.com>

On 7/12/15 10:38 PM, David Holmes wrote:
> Hi Derek,
>
> On 11/07/2015 1:28 AM, Derek White wrote:
>> Please review this partial fix for GC tests that require certain
>> collectors (e.g. shouldn't run in embedded).
>>
>> This is an updated webrev to account for Leonid's fix for "8079134:
>> [TESTBUG] Remove applicable_*gc and needs_*gc groups from TEST.groups",
>> which removed a pile of TEST.groups lists including needs_gc,
>> needs_serialgc, needs_parallelgc, and needs_cmsgc.
>>
>> Now the fix simply updates the needs_g1gc list in TEST.groups and adds
>> appropriate "@requires vm.gc" annotations to a few GC tests.
>>
>> Note that the "@requires vm.gc" changes are /almost/ purely documentary.
>> These are ProcessBuilder-based tests, so any "-XX:+UsexxxGC" flags
>> passed in by jtreg are ignored. It's very confusing, as well as
>> unnecessary, for a jtreg run specifying -XX:+UseG1GC to be running a
>> test that then replaces the flag with "-XX:+UseParallelGC" (etc).
>
> So even though we would never pass through the jtreg specific GC 
> option we will skip these tests if that option doesn't match with the 
> GC's the test will be testing. As long as that doesn't lead to these 
> tests being untested that seems okay.
In all of the tests I modified, it always runs the test if jtreg did not 
specify any GC. Most of the tests will also run if jtreg specifies the 
same GC as the test. So we should have test coverage.
>
> But really this highlights a basic problem we have with our approach 
> to testing with different VM options. Unless the GC option is the only 
> option that changes across the runs you would want the other options 
> to be passed through to the actual tests in many cases. :(

Some test do pass along the outer (jtreg) arguments, and some don't. So 
the fix with "@requires vm.gc" removes the possibility of conflicting 
GC's being specified in that case. There could still be conflicts 
between other arguments specified by jtreg vs. the ProcessBuilder. If 
the ProcessBuilder arguments were constant, they could also be specified 
by |"@requires vm.opt." annotations, but this leaves out many cases.

I'm also not satisfied by the current state of affairs, but I think this 
fix gets us slightly closer to what we want.

  - Derek

|
>
>> *CR:*
>> https://bugs.openjdk.java.net/browse/JDK-8078673
>>
>> *Webrev:*
>> http://cr.openjdk.java.net/~drwhite/8078673/webrev.03/
>>
>> *Testing:*
>>   JPRT
>>
>> *Open review comments:*
>> David H: Your last comments on this subject requested changes to the
>> "needs_gc" list, which has been removed by 8079134.
>>
>> Kim and Jesper: I agree with your comments about wanting some better for
>> both testing multiple GCs and dealing with SE Embedded properly in the
>> testing infrastructure. This webrev is simply a good fix within the
>> existing infrastructure.
>>
>> Thanks,
>>   - Derek
>>
>> On 4/29/15 5:03 PM, Kim Barrett wrote:
>>> On Apr 29, 2015, at 4:38 PM, Jesper 
>>> Wilhelmsson<jesper.wilhelmsson at oracle.com>  wrote:
>>>> Hi,
>>>>
>>>> There are tests like 
>>>> hotspot/test/gc/g1/TestShrinkAuxiliaryData**.java where there is a 
>>>> base class that provides the test and a bunch of test classes that 
>>>> only starts the base test with different arguments. This case would 
>>>> be similar but slightly more ugly since the actual code would be 
>>>> the same and trigger the same base test, but with different 
>>>> @requires in the comment above the test.
>>>>
>>>> I'm not sure it would help though. What we really would need here 
>>>> is a @requires that could check the host name or the hardware 
>>>> platform or OS.
>>> @requires has os.{name, family, arch, simpleArch} properties that 
>>> can be tested.  But I?m not sure any of those are really right for 
>>> testing for ?embedded system? (whatever that actually means).
>>>
>>>> Kim Barrett skrev den 29/4/15 20:35:
>>>>> On Apr 29, 2015, at 2:06 PM, Derek White<derek.white at oracle.com>  
>>>>> wrote:
>>>>>> So most of these tests use ProcessBuilder to specify a command 
>>>>>> line, and explicitly mention a GC to use. A single test might 
>>>>>> actually run each collector (gc/logging/TestGCId, for example). 
>>>>>> Does @requires matter in this case?
>>>>>>
>>>>>> Oh, maybe they should all have @requires vm.gc=="null", becuase 
>>>>>> the actual test is ignoring GC passed in by the test harness GC 
>>>>>> and running as a separate process anyway. It's misleading if the 
>>>>>> harness said UseG1GC and the test was actually running 
>>>>>> UseParallelGC, for example.
>>>>> That?s one solution.  A different solution would be to clone into 
>>>>> multiple tests, one for each relevant collector, where the vm.gc 
>>>>> can be ?null? or the corresponding collector.  That cloning is 
>>>>> kind of ugly though; it?s too bad there can only be one @requires 
>>>>> constraint per test, rather than per @run line or the like.  But 
>>>>> we didn?t get any traction with such a 
>>>>> suggestion:https://bugs.openjdk.java.net/browse/CODETOOLS-7901090.
>>>>>
>>>>>> FYI, it sounds like my original problem does require the exclude 
>>>>>> lists to keep the embedded JVM from running GC tests that it 
>>>>>> shouldn't.
>>>>> I?m not sure how to address this problem.  For example, we don?t 
>>>>> want to exclude TestSmallHeap.java on embedded JVMs, we just want 
>>>>> to exclude its sub-cases that would attempt to use a gc that isn?t 
>>>>> supported.
>>>
>>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150713/17bbdaf7/attachment.htm>

From thomas.schatzl at oracle.com  Tue Jul 14 15:23:47 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Tue, 14 Jul 2015 17:23:47 +0200
Subject: RFR (XXS): 8131166: Remove additional whitespace in G1Allocator
Message-ID: <1436887427.2286.18.camel@oracle.com>

Hi all,

  can I have reviews for the following trivial change that removes a
single whitespace in front of every line from the G1Allocator class body
to make them more conforming to other code?

CR:
https://bugs.openjdk.java.net/browse/JDK-8131166

Webrev:
http://cr.openjdk.java.net/~tschatzl/8131166/webrev/
(look at diff, whitespace only changes)

Testing:
jprt

Thanks,
  Thomas




From tom.benson at oracle.com  Tue Jul 14 15:36:52 2015
From: tom.benson at oracle.com (Tom Benson)
Date: Tue, 14 Jul 2015 11:36:52 -0400
Subject: RFR (XXS): 8131166: Remove additional whitespace in G1Allocator
In-Reply-To: <1436887427.2286.18.camel@oracle.com>
References: <1436887427.2286.18.camel@oracle.com>
Message-ID: <55A52C94.9060809@oracle.com>

Looks good to me.
Tom

On 7/14/2015 11:23 AM, Thomas Schatzl wrote:
> Hi all,
>
>    can I have reviews for the following trivial change that removes a
> single whitespace in front of every line from the G1Allocator class body
> to make them more conforming to other code?
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8131166
>
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8131166/webrev/
> (look at diff, whitespace only changes)
>
> Testing:
> jprt
>
> Thanks,
>    Thomas
>
>



From ionutb83 at yahoo.com  Tue Jul 14 17:12:46 2015
From: ionutb83 at yahoo.com (ionutb83)
Date: Tue, 14 Jul 2015 20:12:46 +0300
Subject: Dirty Cards Write Barrier
Message-ID: <ygtwf8rbbnwbgg136asyyimi.1436893966880@email.android.com>

Hello All,

? ?Could somebody please answer my question below?

Best Regards
Ionut

<div>-------- Original message --------</div><div>From: Ionut <ionutb83 at yahoo.com> </div><div>Date:04/07/2015  13:51  (GMT+02:00) </div><div>To: hotspot-gc-dev at openjdk.java.net </div><div>Subject: Dirty Cards Write Barrier </div><div>
</div>Hello Everybody,

  I have just joined this group and since last year I started to study HotSpot more in detail.  I have a question for you, so your input will be greatly appreciated.

  Based on my understanding Dirty Cards Write Barrier is a way of keeping track of references from Old to Young Generation.  This technique is used during minor GC to search for objects referred from Old generation. 
  Is this technique used by all minor GC: Serial, Parallel, Parallel Old, CMS, G1? Are there any other Write Barrier techniques?

Regards,
Ionut Balosin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150714/b79798a8/attachment.htm>

From jon.masamitsu at oracle.com  Tue Jul 14 17:27:26 2015
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Tue, 14 Jul 2015 10:27:26 -0700
Subject: RFR (XXS): 8131166: Remove additional whitespace in G1Allocator
In-Reply-To: <1436887427.2286.18.camel@oracle.com>
References: <1436887427.2286.18.camel@oracle.com>
Message-ID: <55A5467E.20708@oracle.com>

Changes look good.

Reviewed.

Jon

On 07/14/2015 08:23 AM, Thomas Schatzl wrote:
> Hi all,
>
>    can I have reviews for the following trivial change that removes a
> single whitespace in front of every line from the G1Allocator class body
> to make them more conforming to other code?
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8131166
>
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8131166/webrev/
> (look at diff, whitespace only changes)
>
> Testing:
> jprt
>
> Thanks,
>    Thomas
>
>



From thomas.schatzl at oracle.com  Wed Jul 15 07:37:01 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 15 Jul 2015 09:37:01 +0200
Subject: Dirty Cards Write Barrier
In-Reply-To: <ygtwf8rbbnwbgg136asyyimi.1436893966880@email.android.com>
References: <ygtwf8rbbnwbgg136asyyimi.1436893966880@email.android.com>
Message-ID: <1436945821.2282.11.camel@oracle.com>

Hi Ionut,

On Tue, 2015-07-14 at 20:12 +0300, ionutb83 wrote:
> Hello All,
> 
> 
>    Could somebody please answer my question below?
> 
> 
> Best Regards
> Ionut
> 
> 
> -------- Original message --------
> From: Ionut 
> Date:04/07/2015 13:51 (GMT+02:00) 
> To: hotspot-gc-dev at openjdk.java.net 
> Subject: Dirty Cards Write Barrier 
> 
> 
> Hello Everybody,
>
>   I have just joined this group and since last year I started to study
> HotSpot more in detail.  I have a question for you, so your input will
> be greatly appreciated.
>
>   Based on my understanding Dirty Cards Write Barrier is a way of
> keeping track of references from Old to Young Generation.  This
> technique is used during minor GC to search for objects referred from
> Old generation. 
>   Is this technique used by all minor GC: Serial, Parallel, Parallel
> Old, CMS, G1?

All but G1 use this technique to track references between generations.
G1 uses card marking to track references between regions, not
generations. Also the time and place of how these references are
converted into remembered sets (and the use of the extra data structure
to store remembered sets itself) is quite different.

G1 uses a technique roughly based on what

David Detlefs, Ross Knippel, William D. Clinger, and Matthias Jacob.
2002. Concurrent Remembered Set Refinement in Generational Garbage
Collection. In Proceedings of the 2nd Java? Virtual Machine Research and
Technology Symposium, Samuel P. Midkiff (Ed.). USENIX Association,
Berkeley, CA, USA, 13-26. 

describe.

Note that both G1 and CMS use write barriers for concurrent marking too
(and actually G1 also has a special case where it uses a read barrier).

>  Are there any other Write Barrier techniques?

Not sure what the question is: certainly there are other uses of write
barriers than card marking (see above), and different techniques to get
the same effect without using card marks (ie. maintain remembered sets).

E.g. Xi Yang, Stephen M. Blackburn, Daniel Frampton, and Antony L.
Hosking. 2012. Barriers reconsidered, friendlier still!. SIGPLAN Not.
47, 11 (June 2012), 37-48. DOI=10.1145/2426642.2259004
http://doi.acm.org/10.1145/2426642.2259004 gives an overview of some
kinds of software write barriers, but it maybe also contains a few
useful references for further research.

Thanks,
  Thomas





From dawid.weiss at gmail.com  Wed Jul 15 08:12:58 2015
From: dawid.weiss at gmail.com (Dawid Weiss)
Date: Wed, 15 Jul 2015 10:12:58 +0200
Subject: [JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.9.0-ea-b72) - Build
	# 13461 - Failure!
In-Reply-To: <378730283.143.1436907784184.JavaMail.jenkins@serv1>
References: <378730283.143.1436907784184.JavaMail.jenkins@serv1>
Message-ID: <CAM21Rt_DPDwAFPKGocW3S839tG2H_WkMEf1iUPVuKUw+1biaDg@mail.gmail.com>

Hi Rory, gc-compiler folks,

We've hit odd JVM/gc failures after upgrading to jdk1.9.0-ea-b72,
check out the issue and hs_err files here:

https://issues.apache.org/jira/browse/LUCENE-6678

Don't know if it's a known issue, just collected logs for now.

Dawid


On Tue, Jul 14, 2015 at 11:02 PM, Policeman Jenkins Server
<jenkins at thetaphi.de> wrote:
> Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/13461/
> Java: 64bit/jdk1.9.0-ea-b72 -XX:-UseCompressedOops -XX:+UseSerialGC -Djava.locale.providers=JRE,SPI
>
> All tests passed
>
> Build Log:
> [...truncated 9822 lines...]
>    [junit4] JVM J1: stdout was not empty, see: /home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/temp/junit4-J1-20150714_205152_761.sysout
>    [junit4] >>> JVM J1: stdout (verbatim) ----
>    [junit4] #
>    [junit4] # A fatal error has been detected by the Java Runtime Environment:
>    [junit4] #
>    [junit4] #  SIGSEGV (0xb) at pc=0x00007f827f856da5, pid=7407, tid=0x00007f827d311700
>    [junit4] #
>    [junit4] # JRE version: Java(TM) SE Runtime Environment (9.0-b72) (build 1.9.0-ea-b72)
>    [junit4] # Java VM: Java HotSpot(TM) 64-Bit Server VM (1.9.0-ea-b72 mixed mode linux-amd64 )
>    [junit4] # Problematic frame:
>    [junit4] # V  [libjvm.so+0x537da5]  DefNewGeneration::copy_to_survivor_space(oopDesc*)+0x25
>    [junit4] #
>    [junit4] # No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
>    [junit4] #
>    [junit4] # An error report file with more information is saved as:
>    [junit4] # /home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/hs_err_pid7407.log
>    [junit4] #
>    [junit4] # If you would like to submit a bug report, please visit:
>    [junit4] #   http://bugreport.java.com/bugreport/crash.jsp
>    [junit4] #
>    [junit4] <<< JVM J1: EOF ----
>
>    [junit4] JVM J2: stdout was not empty, see: /home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/temp/junit4-J2-20150714_205152_761.sysout
>    [junit4] >>> JVM J2: stdout (verbatim) ----
>    [junit4] #
>    [junit4] # A fatal error has been detected by the Java Runtime Environment:
>    [junit4] #
>    [junit4] #  SIGSEGV (0xb) at pc=0x00007fb44c05dda5, pid=7402, tid=0x00007fb449b18700
>    [junit4] #
>    [junit4] # JRE version: Java(TM) SE Runtime Environment (9.0-b72) (build 1.9.0-ea-b72)
>    [junit4] # Java VM: Java HotSpot(TM) 64-Bit Server VM (1.9.0-ea-b72 mixed mode linux-amd64 )
>    [junit4] # Problematic frame:
>    [junit4] # V  [libjvm.so+0x537da5]  DefNewGeneration::copy_to_survivor_space(oopDesc*)+0x25
>    [junit4] #
>    [junit4] # No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
>    [junit4] #
>    [junit4] # An error report file with more information is saved as:
>    [junit4] # /home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J2/hs_err_pid7402.log
>    [junit4] #
>    [junit4] # If you would like to submit a bug report, please visit:
>    [junit4] #   http://bugreport.java.com/bugreport/crash.jsp
>    [junit4] #
>    [junit4] <<< JVM J2: EOF ----
>
> [...truncated 207 lines...]
>    [junit4] JVM J0: stdout was not empty, see: /home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/temp/junit4-J0-20150714_205152_761.sysout
>    [junit4] >>> JVM J0: stdout (verbatim) ----
>    [junit4] #
>    [junit4] # A fatal error has been detected by the Java Runtime Environment:
>    [junit4] #
>    [junit4] #  SIGSEGV (0xb) at pc=0x00007fecf93696ef, pid=7408, tid=0x00007fecc3d3a700
>    [junit4] #
>    [junit4] # JRE version: Java(TM) SE Runtime Environment (9.0-b72) (build 1.9.0-ea-b72)
>    [junit4] # Java VM: Java HotSpot(TM) 64-Bit Server VM (1.9.0-ea-b72 mixed mode linux-amd64 )
>    [junit4] # Problematic frame:
>    [junit4] # V  [libjvm.so+0x5376ef]  DefNewGeneration::drain_promo_failure_scan_stack()+0x3f
>    [junit4] #
>    [junit4] # No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
>    [junit4] #
>    [junit4] # An error report file with more information is saved as:
>    [junit4] # /home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/hs_err_pid7408.log
>    [junit4] #
>    [junit4] # If you would like to submit a bug report, please visit:
>    [junit4] #   http://bugreport.java.com/bugreport/crash.jsp
>    [junit4] #
>    [junit4] <<< JVM J0: EOF ----
>
> [...truncated 1 lines...]
>    [junit4] ERROR: JVM J0 ended with an exception, command line: /home/jenkins/tools/java/64bit/jdk1.9.0-ea-b72/bin/java -XX:-UseCompressedOops -XX:+UseSerialGC -Djava.locale.providers=JRE,SPI -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/heapdumps -ea -esa -Dtests.prefix=tests -Dtests.seed=90C16F4AC280153C -Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random -Dtests.postingsformat=random -Dtests.docvaluesformat=random -Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random -Dtests.linedocsfile=europarl.lines.txt.gz -Dtests.luceneMatchVersion=6.0.0 -Dtests.cleanthreads=perClass -Djava.util.logging.config.file=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/tools/junit4/logging.properties -Dtests.nightly=false -Dtests.weekly=false -Dtests.monster=false -Dtests.slow=true -Dtests.asserts=true -Dtests.multiplier=3 -DtempDir=./temp -Djava.io.tmpdir=./temp -Djunit4.tempDir=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/temp -Dcommon.dir=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene -Dclover.db.dir=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/clover/db -Djava.security.policy=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/tools/junit4/solr-tests.policy -Dtests.LUCENE_VERSION=6.0.0 -Djetty.testMode=1 -Djetty.insecurerandom=1 -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -Djava.awt.headless=true -Djdk.map.althashing.threshold=0 -Dtests.leaveTemporary=false -Dtests.filterstacks=true -Dtests.disableHdfs=true -Djava.security.manager=org.apache.lucene.util.TestSecurityManager -classpath /home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/classes/test:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-test-framework/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/test-framework/lib/junit4-ant-2.1.13.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/test-framework/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/codecs/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/analysis/common/lucene-analyzers-common-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/codecs/lucene-codecs-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/backward-codecs/lucene-backward-codecs-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/highlighter/lucene-highlighter-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/memory/lucene-memory-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/misc/lucene-misc-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/spatial/lucene-spatial-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/expressions/lucene-expressions-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/suggest/lucene-suggest-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/grouping/lucene-grouping-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/queries/lucene-queries-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/queryparser/lucene-queryparser-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/join/lucene-join-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/sandbox/lucene-sandbox-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/antlr-runtime-3.5.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/antlr4-runtime-4.5.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/asm-4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/asm-commons-4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/caffeine-1.0.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/commons-cli-1.2.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/commons-codec-1.10.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/commons-collections-3.2.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/commons-configuration-1.6.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/commons-fileupload-1.2.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/commons-lang-2.6.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/dom4j-1.6.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/fastutil-6.5.11.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/guava-14.0.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/hadoop-annotations-2.6.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/hadoop-auth-2.6.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/hadoop-common-2.6.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/hadoop-hdfs-2.6.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/hll-1.6.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/hppc-0.5.2.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/htrace-core-3.0.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/jackson-core-2.5.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/jackson-dataformat-smile-2.5.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/joda-time-2.2.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/log4j-1.2.17.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/org.restlet-2.3.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/org.restlet.ext.servlet-2.3.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/presto-parser-0.108.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/protobuf-java-2.5.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/slf4j-log4j12-1.7.7.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/slice-0.10.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/spatial4j-0.4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/t-digest-3.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/commons-io-2.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/httpclient-4.4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/httpcore-4.4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/httpmime-4.4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/noggit-0.6.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/slf4j-api-1.7.7.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/stax2-api-3.1.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/woodstox-core-asl-4.4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/zookeeper-3.4.6.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/javax.servlet-api-3.1.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-continuation-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-deploy-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-http-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-io-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-jmx-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-rewrite-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-security-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-server-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-servlet-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-servlets-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-util-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-webapp-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-xml-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/example-DIH/solr/db/lib/derby-10.9.1.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/core/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/test-framework/lib/junit-4.10.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/test-framework/lib/randomizedtesting-runner-2.1.13.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/antlr-2.7.7.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/antlr4-runtime-4.5.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-core-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-core-api-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-core-avl-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-core-shared-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-i18n-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptor-kerberos-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-admin-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/api-all-1.0.0-M20.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/bcprov-jdk15-1.45.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/cglib-nodep-2.2.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/commons-collections-3.2.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/commons-math3-3.4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/easymock-3.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/ehcache-core-2.4.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jackson-annotations-2.5.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jackson-databind-2.5.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jcl-over-slf4j-1.7.7.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jersey-core-1.9.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jersey-server-1.9.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jetty-6.1.26.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jetty-util-6.1.26.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/mina-core-2.0.0-M5.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/objenesis-1.2.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/presto-parser-0.108.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/slice-0.10.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/analysis/icu/lucene-analyzers-icu-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/contrib/solr-analysis-extras/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-launcher.jar:/var/lib/jenkins/.ant/lib/ivy-2.3.0.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jdepend.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bcel.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jmf.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit4.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-xalan2.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-javamail.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jai.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bsf.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-logging.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-net.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-resolver.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-log4j.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-oro.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-antlr.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jsch.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-regexp.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-swing.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-testutil.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-netrexx.jar:/var/lib/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.1.13.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/temp/junit4-J0-20150714_205152_761.events @/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/temp/junit4-J0-20150714_205152_761.suites
>    [junit4] ERROR: JVM J0 ended with an exception: Forked process returned with error code: 134. Very likely a JVM crash.  Process output piped in logs above.
>    [junit4]     at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1484)
>    [junit4]     at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:133)
>    [junit4]     at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:964)
>    [junit4]     at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:961)
>    [junit4]     at java.util.concurrent.FutureTask.run(FutureTask.java:265)
>    [junit4]     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>    [junit4]     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>    [junit4]     at java.lang.Thread.run(Thread.java:745)
>    [junit4] ERROR: JVM J1 ended with an exception, command line: /home/jenkins/tools/java/64bit/jdk1.9.0-ea-b72/bin/java -XX:-UseCompressedOops -XX:+UseSerialGC -Djava.locale.providers=JRE,SPI -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/heapdumps -ea -esa -Dtests.prefix=tests -Dtests.seed=90C16F4AC280153C -Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random -Dtests.postingsformat=random -Dtests.docvaluesformat=random -Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random -Dtests.linedocsfile=europarl.lines.txt.gz -Dtests.luceneMatchVersion=6.0.0 -Dtests.cleanthreads=perClass -Djava.util.logging.config.file=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/tools/junit4/logging.properties -Dtests.nightly=false -Dtests.weekly=false -Dtests.monster=false -Dtests.slow=true -Dtests.asserts=true -Dtests.multiplier=3 -DtempDir=./temp -Djava.io.tmpdir=./temp -Djunit4.tempDir=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/temp -Dcommon.dir=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene -Dclover.db.dir=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/clover/db -Djava.security.policy=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/tools/junit4/solr-tests.policy -Dtests.LUCENE_VERSION=6.0.0 -Djetty.testMode=1 -Djetty.insecurerandom=1 -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -Djava.awt.headless=true -Djdk.map.althashing.threshold=0 -Dtests.leaveTemporary=false -Dtests.filterstacks=true -Dtests.disableHdfs=true -Djava.security.manager=org.apache.lucene.util.TestSecurityManager -classpath /home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/classes/test:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-test-framework/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/test-framework/lib/junit4-ant-2.1.13.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/test-framework/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/codecs/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/analysis/common/lucene-analyzers-common-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/codecs/lucene-codecs-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/backward-codecs/lucene-backward-codecs-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/highlighter/lucene-highlighter-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/memory/lucene-memory-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/misc/lucene-misc-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/spatial/lucene-spatial-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/expressions/lucene-expressions-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/suggest/lucene-suggest-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/grouping/lucene-grouping-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/queries/lucene-queries-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/queryparser/lucene-queryparser-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/join/lucene-join-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/sandbox/lucene-sandbox-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/antlr-runtime-3.5.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/antlr4-runtime-4.5.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/asm-4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/asm-commons-4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/caffeine-1.0.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/commons-cli-1.2.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/commons-codec-1.10.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/commons-collections-3.2.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/commons-configuration-1.6.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/commons-fileupload-1.2.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/commons-lang-2.6.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/dom4j-1.6.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/fastutil-6.5.11.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/guava-14.0.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/hadoop-annotations-2.6.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/hadoop-auth-2.6.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/hadoop-common-2.6.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/hadoop-hdfs-2.6.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/hll-1.6.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/hppc-0.5.2.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/htrace-core-3.0.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/jackson-core-2.5.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/jackson-dataformat-smile-2.5.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/joda-time-2.2.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/log4j-1.2.17.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/org.restlet-2.3.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/org.restlet.ext.servlet-2.3.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/presto-parser-0.108.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/protobuf-java-2.5.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/slf4j-log4j12-1.7.7.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/slice-0.10.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/spatial4j-0.4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/t-digest-3.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/commons-io-2.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/httpclient-4.4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/httpcore-4.4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/httpmime-4.4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/noggit-0.6.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/slf4j-api-1.7.7.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/stax2-api-3.1.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/woodstox-core-asl-4.4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/zookeeper-3.4.6.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/javax.servlet-api-3.1.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-continuation-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-deploy-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-http-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-io-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-jmx-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-rewrite-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-security-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-server-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-servlet-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-servlets-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-util-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-webapp-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-xml-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/example-DIH/solr/db/lib/derby-10.9.1.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/core/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/test-framework/lib/junit-4.10.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/test-framework/lib/randomizedtesting-runner-2.1.13.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/antlr-2.7.7.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/antlr4-runtime-4.5.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-core-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-core-api-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-core-avl-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-core-shared-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-i18n-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptor-kerberos-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-admin-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/api-all-1.0.0-M20.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/bcprov-jdk15-1.45.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/cglib-nodep-2.2.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/commons-collections-3.2.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/commons-math3-3.4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/easymock-3.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/ehcache-core-2.4.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jackson-annotations-2.5.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jackson-databind-2.5.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jcl-over-slf4j-1.7.7.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jersey-core-1.9.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jersey-server-1.9.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jetty-6.1.26.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jetty-util-6.1.26.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/mina-core-2.0.0-M5.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/objenesis-1.2.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/presto-parser-0.108.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/slice-0.10.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/analysis/icu/lucene-analyzers-icu-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/contrib/solr-analysis-extras/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-launcher.jar:/var/lib/jenkins/.ant/lib/ivy-2.3.0.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jdepend.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bcel.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jmf.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit4.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-xalan2.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-javamail.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jai.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bsf.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-logging.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-net.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-resolver.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-log4j.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-oro.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-antlr.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jsch.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-regexp.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-swing.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-testutil.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-netrexx.jar:/var/lib/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.1.13.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/temp/junit4-J1-20150714_205152_761.events @/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/temp/junit4-J1-20150714_205152_761.suites
>    [junit4] ERROR: JVM J1 ended with an exception: Forked process returned with error code: 134. Very likely a JVM crash.  Process output piped in logs above.
>    [junit4]     at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1484)
>    [junit4]     at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:133)
>    [junit4]     at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:964)
>    [junit4]     at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:961)
>    [junit4]     at java.util.concurrent.FutureTask.run(FutureTask.java:265)
>    [junit4]     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>    [junit4]     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>    [junit4]     at java.lang.Thread.run(Thread.java:745)
>    [junit4] ERROR: JVM J2 ended with an exception, command line: /home/jenkins/tools/java/64bit/jdk1.9.0-ea-b72/bin/java -XX:-UseCompressedOops -XX:+UseSerialGC -Djava.locale.providers=JRE,SPI -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/heapdumps -ea -esa -Dtests.prefix=tests -Dtests.seed=90C16F4AC280153C -Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random -Dtests.postingsformat=random -Dtests.docvaluesformat=random -Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random -Dtests.linedocsfile=europarl.lines.txt.gz -Dtests.luceneMatchVersion=6.0.0 -Dtests.cleanthreads=perClass -Djava.util.logging.config.file=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/tools/junit4/logging.properties -Dtests.nightly=false -Dtests.weekly=false -Dtests.monster=false -Dtests.slow=true -Dtests.asserts=true -Dtests.multiplier=3 -DtempDir=./temp -Djava.io.tmpdir=./temp -Djunit4.tempDir=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/temp -Dcommon.dir=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene -Dclover.db.dir=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/clover/db -Djava.security.policy=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/tools/junit4/solr-tests.policy -Dtests.LUCENE_VERSION=6.0.0 -Djetty.testMode=1 -Djetty.insecurerandom=1 -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -Djava.awt.headless=true -Djdk.map.althashing.threshold=0 -Dtests.leaveTemporary=false -Dtests.filterstacks=true -Dtests.disableHdfs=true -Djava.security.manager=org.apache.lucene.util.TestSecurityManager -classpath /home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/classes/test:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-test-framework/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/test-framework/lib/junit4-ant-2.1.13.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/test-framework/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/codecs/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/analysis/common/lucene-analyzers-common-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/codecs/lucene-codecs-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/backward-codecs/lucene-backward-codecs-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/highlighter/lucene-highlighter-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/memory/lucene-memory-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/misc/lucene-misc-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/spatial/lucene-spatial-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/expressions/lucene-expressions-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/suggest/lucene-suggest-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/grouping/lucene-grouping-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/queries/lucene-queries-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/queryparser/lucene-queryparser-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/join/lucene-join-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/sandbox/lucene-sandbox-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/antlr-runtime-3.5.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/antlr4-runtime-4.5.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/asm-4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/asm-commons-4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/caffeine-1.0.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/commons-cli-1.2.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/commons-codec-1.10.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/commons-collections-3.2.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/commons-configuration-1.6.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/commons-fileupload-1.2.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/commons-lang-2.6.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/dom4j-1.6.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/fastutil-6.5.11.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/guava-14.0.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/hadoop-annotations-2.6.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/hadoop-auth-2.6.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/hadoop-common-2.6.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/hadoop-hdfs-2.6.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/hll-1.6.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/hppc-0.5.2.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/htrace-core-3.0.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/jackson-core-2.5.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/jackson-dataformat-smile-2.5.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/joda-time-2.2.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/log4j-1.2.17.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/org.restlet-2.3.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/org.restlet.ext.servlet-2.3.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/presto-parser-0.108.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/protobuf-java-2.5.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/slf4j-log4j12-1.7.7.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/slice-0.10.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/spatial4j-0.4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/t-digest-3.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/commons-io-2.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/httpclient-4.4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/httpcore-4.4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/httpmime-4.4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/noggit-0.6.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/slf4j-api-1.7.7.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/stax2-api-3.1.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/woodstox-core-asl-4.4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/zookeeper-3.4.6.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/javax.servlet-api-3.1.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-continuation-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-deploy-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-http-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-io-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-jmx-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-rewrite-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-security-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-server-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-servlet-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-servlets-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-util-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-webapp-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-xml-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/example-DIH/solr/db/lib/derby-10.9.1.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/core/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/test-framework/lib/junit-4.10.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/test-framework/lib/randomizedtesting-runner-2.1.13.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/antlr-2.7.7.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/antlr4-runtime-4.5.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-core-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-core-api-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-core-avl-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-core-shared-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-i18n-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptor-kerberos-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-admin-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/api-all-1.0.0-M20.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/bcprov-jdk15-1.45.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/cglib-nodep-2.2.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/commons-collections-3.2.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/commons-math3-3.4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/easymock-3.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/ehcache-core-2.4.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jackson-annotations-2.5.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jackson-databind-2.5.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jcl-over-slf4j-1.7.7.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jersey-core-1.9.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jersey-server-1.9.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jetty-6.1.26.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jetty-util-6.1.26.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/mina-core-2.0.0-M5.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/objenesis-1.2.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/presto-parser-0.108.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/slice-0.10.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/analysis/icu/lucene-analyzers-icu-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/contrib/solr-analysis-extras/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-launcher.jar:/var/lib/jenkins/.ant/lib/ivy-2.3.0.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jdepend.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bcel.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jmf.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit4.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-xalan2.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-javamail.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jai.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bsf.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-logging.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-net.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-resolver.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-log4j.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-oro.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-antlr.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jsch.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-regexp.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-swing.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-testutil.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-netrexx.jar:/var/lib/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.1.13.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/temp/junit4-J2-20150714_205152_761.events @/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/temp/junit4-J2-20150714_205152_761.suites
>    [junit4] ERROR: JVM J2 ended with an exception: Forked process returned with error code: 134. Very likely a JVM crash.  Process output piped in logs above.
>    [junit4]     at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1484)
>    [junit4]     at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:133)
>    [junit4]     at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:964)
>    [junit4]     at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:961)
>    [junit4]     at java.util.concurrent.FutureTask.run(FutureTask.java:265)
>    [junit4]     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>    [junit4]     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>    [junit4]     at java.lang.Thread.run(Thread.java:745)
>
> BUILD FAILED
> /home/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:526: The following error occurred while executing this line:
> /home/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:474: The following error occurred while executing this line:
> /home/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:61: The following error occurred while executing this line:
> /home/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:39: The following error occurred while executing this line:
> /home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:230: The following error occurred while executing this line:
> /home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:513: The following error occurred while executing this line:
> /home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1426: The following error occurred while executing this line:
> /home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:981: At least one slave process threw an exception, first: Forked process returned with error code: 134. Very likely a JVM crash.  Process output piped in logs above.
>
> Total time: 35 minutes 44 seconds
> Build step 'Invoke Ant' marked build as failure
> Archiving artifacts
> Recording test results
> Email was triggered for: Failure - Any
> Sending email for trigger: Failure - Any
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe at lucene.apache.org
> For additional commands, e-mail: dev-help at lucene.apache.org


From thomas.schatzl at oracle.com  Wed Jul 15 08:57:57 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 15 Jul 2015 10:57:57 +0200
Subject: [JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.9.0-ea-b72) -
	Build # 13461 - Failure!
In-Reply-To: <CAM21Rt_DPDwAFPKGocW3S839tG2H_WkMEf1iUPVuKUw+1biaDg@mail.gmail.com>
References: <378730283.143.1436907784184.JavaMail.jenkins@serv1>
	<CAM21Rt_DPDwAFPKGocW3S839tG2H_WkMEf1iUPVuKUw+1biaDg@mail.gmail.com>
Message-ID: <1436950677.2282.18.camel@oracle.com>

Hi,

On Wed, 2015-07-15 at 10:12 +0200, Dawid Weiss wrote:
> Hi Rory, gc-compiler folks,
> 
> We've hit odd JVM/gc failures after upgrading to jdk1.9.0-ea-b72,
> check out the issue and hs_err files here:
> 
> https://issues.apache.org/jira/browse/LUCENE-6678
> 
> Don't know if it's a known issue, just collected logs for now.

  the crash looks like "8129961 : SIGSEGV when copying to survivor
space" that has slipped through. The fix has been pushed to the group
repos this Monday, but will take a while to propagate.

Others may have more information on when exactly it will hit an EA
build.

You can use b71 which does not have the change that caused this
particular bug in the meantime.

Thanks,
  Thomas




From dawid.weiss at gmail.com  Wed Jul 15 09:05:11 2015
From: dawid.weiss at gmail.com (Dawid Weiss)
Date: Wed, 15 Jul 2015 11:05:11 +0200
Subject: [JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.9.0-ea-b72) - Build
	# 13461 - Failure!
In-Reply-To: <1436950677.2282.18.camel@oracle.com>
References: <378730283.143.1436907784184.JavaMail.jenkins@serv1>
	<CAM21Rt_DPDwAFPKGocW3S839tG2H_WkMEf1iUPVuKUw+1biaDg@mail.gmail.com>
	<1436950677.2282.18.camel@oracle.com>
Message-ID: <CAM21Rt8JJ5h6DqMF61RrjO9HLoyHWU4CctjRZtcfn5fAZV4xdQ@mail.gmail.com>

Thanks Thomas, I've added a note about it to our issue.
.
Dawid

On Wed, Jul 15, 2015 at 10:57 AM, Thomas Schatzl
<thomas.schatzl at oracle.com> wrote:
> Hi,
>
> On Wed, 2015-07-15 at 10:12 +0200, Dawid Weiss wrote:
>> Hi Rory, gc-compiler folks,
>>
>> We've hit odd JVM/gc failures after upgrading to jdk1.9.0-ea-b72,
>> check out the issue and hs_err files here:
>>
>> https://issues.apache.org/jira/browse/LUCENE-6678
>>
>> Don't know if it's a known issue, just collected logs for now.
>
>   the crash looks like "8129961 : SIGSEGV when copying to survivor
> space" that has slipped through. The fix has been pushed to the group
> repos this Monday, but will take a while to propagate.
>
> Others may have more information on when exactly it will hit an EA
> build.
>
> You can use b71 which does not have the change that caused this
> particular bug in the meantime.
>
> Thanks,
>   Thomas
>
>


From ionutb83 at yahoo.com  Wed Jul 15 09:24:42 2015
From: ionutb83 at yahoo.com (Ionut)
Date: Wed, 15 Jul 2015 09:24:42 +0000 (UTC)
Subject: Dirty Cards Write Barrier
In-Reply-To: <1436945821.2282.11.camel@oracle.com>
References: <1436945821.2282.11.camel@oracle.com>
Message-ID: <1622410903.1046361.1436952282556.JavaMail.yahoo@mail.yahoo.com>

Hi Thomas,
? ? Thank a lot for your input! I will go over the articles you mentioned, it seems to be a bit more clear now.
ThanksIonut Balosin 


     On Wednesday, July 15, 2015 10:37 AM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
   

 Hi Ionut,

On Tue, 2015-07-14 at 20:12 +0300, ionutb83 wrote:
> Hello All,
> 
> 
>? ? Could somebody please answer my question below?
> 
> 
> Best Regards
> Ionut
> 
> 
> -------- Original message --------
> From: Ionut 
> Date:04/07/2015 13:51 (GMT+02:00) 
> To: hotspot-gc-dev at openjdk.java.net 
> Subject: Dirty Cards Write Barrier 
> 
> 
> Hello Everybody,
>
>? I have just joined this group and since last year I started to study
> HotSpot more in detail.? I have a question for you, so your input will
> be greatly appreciated.
>
>? Based on my understanding Dirty Cards Write Barrier is a way of
> keeping track of references from Old to Young Generation.? This
> technique is used during minor GC to search for objects referred from
> Old generation. 
>? Is this technique used by all minor GC: Serial, Parallel, Parallel
> Old, CMS, G1?

All but G1 use this technique to track references between generations.
G1 uses card marking to track references between regions, not
generations. Also the time and place of how these references are
converted into remembered sets (and the use of the extra data structure
to store remembered sets itself) is quite different.

G1 uses a technique roughly based on what

David Detlefs, Ross Knippel, William D. Clinger, and Matthias Jacob.
2002. Concurrent Remembered Set Refinement in Generational Garbage
Collection. In Proceedings of the 2nd Java? Virtual Machine Research and
Technology Symposium, Samuel P. Midkiff (Ed.). USENIX Association,
Berkeley, CA, USA, 13-26. 

describe.

Note that both G1 and CMS use write barriers for concurrent marking too
(and actually G1 also has a special case where it uses a read barrier).

>? Are there any other Write Barrier techniques?

Not sure what the question is: certainly there are other uses of write
barriers than card marking (see above), and different techniques to get
the same effect without using card marks (ie. maintain remembered sets).

E.g. Xi Yang, Stephen M. Blackburn, Daniel Frampton, and Antony L.
Hosking. 2012. Barriers reconsidered, friendlier still!. SIGPLAN Not.
47, 11 (June 2012), 37-48. DOI=10.1145/2426642.2259004
http://doi.acm.org/10.1145/2426642.2259004 gives an overview of some
kinds of software write barriers, but it maybe also contains a few
useful references for further research.

Thanks,
? Thomas




  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150715/48986f88/attachment.htm>

From thomas.schatzl at oracle.com  Wed Jul 15 09:43:51 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 15 Jul 2015 11:43:51 +0200
Subject: RFR (XXS): 8131166: Remove additional whitespace in G1Allocator
In-Reply-To: <55A5467E.20708@oracle.com>
References: <1436887427.2286.18.camel@oracle.com> <55A5467E.20708@oracle.com>
Message-ID: <1436953431.2282.19.camel@oracle.com>

Hi Jon, Tom,

On Tue, 2015-07-14 at 10:27 -0700, Jon Masamitsu wrote:
> Changes look good.
> 
> Reviewed.
> 
> Jon
> 

  thanks for the reviews.

Thomas



From thomas.schatzl at oracle.com  Wed Jul 15 10:47:46 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 15 Jul 2015 12:47:46 +0200
Subject: RFR (S): 8131319: Move G1Allocator::_summary_bytes_used back to
	G1CollectedHeap
Message-ID: <1436957266.2282.35.camel@oracle.com>

Hi all,

  can I have comments and reviews for the following change to move
G1Allocator::_summary_bytes_used to a more fitting place?

The problem from my POV is, that the member
G1Allocator::_summary_bytes_used, that contains the number of bytes held
by the heap outside of the allocators is located within G1Allocator.

That sounds odd, particularly because G1Allocator actually never uses
it, it's managed completely by G1CollectedHeap.

Before the addition of the G1ArchiveAllocator one could have made the
argument that G1Allocator holds all memory contained in the heap, but
this is not true any more: the memory occupied by G1ArchiveAllocator
needs to be added to the total sum of all managed space anyway, so
moving _summary_bytes_used out of G1Allocator seems sensible (to me).

I talked to StefanJ about this change who originally moved
_summary_used_bytes into G1Allocator, and we could not find an argument
for keeping _summary_used_bytes in G1Allocator.

This allows us to also refine the responsibilities of G1Allocator a
little more, to be the owner of the current allocation areas (regions)
only.

CR:
https://bugs.openjdk.java.net/browse/JDK-8131319

Webrev:
http://cr.openjdk.java.net/~tschatzl/8131319/webrev/

Testing:
jprt

Thanks,
  Thomas




From rory.odonnell at oracle.com  Wed Jul 15 08:17:01 2015
From: rory.odonnell at oracle.com (Rory O'Donnell)
Date: Wed, 15 Jul 2015 09:17:01 +0100
Subject: [JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.9.0-ea-b72) - Build
	# 13461 - Failure!
In-Reply-To: <CAM21Rt_DPDwAFPKGocW3S839tG2H_WkMEf1iUPVuKUw+1biaDg@mail.gmail.com>
References: <378730283.143.1436907784184.JavaMail.jenkins@serv1>
	<CAM21Rt_DPDwAFPKGocW3S839tG2H_WkMEf1iUPVuKUw+1biaDg@mail.gmail.com>
Message-ID: <55A616FD.90403@oracle.com>

Hi Dawid,

Could you log an incident at bugs.java.com and let us know the incident id.

Thanks, Rory

On 15/07/2015 09:12, Dawid Weiss wrote:
> Hi Rory, gc-compiler folks,
>
> We've hit odd JVM/gc failures after upgrading to jdk1.9.0-ea-b72,
> check out the issue and hs_err files here:
>
> https://issues.apache.org/jira/browse/LUCENE-6678
>
> Don't know if it's a known issue, just collected logs for now.
>
> Dawid
>
>
> On Tue, Jul 14, 2015 at 11:02 PM, Policeman Jenkins Server
> <jenkins at thetaphi.de> wrote:
>> Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/13461/
>> Java: 64bit/jdk1.9.0-ea-b72 -XX:-UseCompressedOops -XX:+UseSerialGC -Djava.locale.providers=JRE,SPI
>>
>> All tests passed
>>
>> Build Log:
>> [...truncated 9822 lines...]
>>     [junit4] JVM J1: stdout was not empty, see: /home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/temp/junit4-J1-20150714_205152_761.sysout
>>     [junit4] >>> JVM J1: stdout (verbatim) ----
>>     [junit4] #
>>     [junit4] # A fatal error has been detected by the Java Runtime Environment:
>>     [junit4] #
>>     [junit4] #  SIGSEGV (0xb) at pc=0x00007f827f856da5, pid=7407, tid=0x00007f827d311700
>>     [junit4] #
>>     [junit4] # JRE version: Java(TM) SE Runtime Environment (9.0-b72) (build 1.9.0-ea-b72)
>>     [junit4] # Java VM: Java HotSpot(TM) 64-Bit Server VM (1.9.0-ea-b72 mixed mode linux-amd64 )
>>     [junit4] # Problematic frame:
>>     [junit4] # V  [libjvm.so+0x537da5]  DefNewGeneration::copy_to_survivor_space(oopDesc*)+0x25
>>     [junit4] #
>>     [junit4] # No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
>>     [junit4] #
>>     [junit4] # An error report file with more information is saved as:
>>     [junit4] # /home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/hs_err_pid7407.log
>>     [junit4] #
>>     [junit4] # If you would like to submit a bug report, please visit:
>>     [junit4] #   http://bugreport.java.com/bugreport/crash.jsp
>>     [junit4] #
>>     [junit4] <<< JVM J1: EOF ----
>>
>>     [junit4] JVM J2: stdout was not empty, see: /home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/temp/junit4-J2-20150714_205152_761.sysout
>>     [junit4] >>> JVM J2: stdout (verbatim) ----
>>     [junit4] #
>>     [junit4] # A fatal error has been detected by the Java Runtime Environment:
>>     [junit4] #
>>     [junit4] #  SIGSEGV (0xb) at pc=0x00007fb44c05dda5, pid=7402, tid=0x00007fb449b18700
>>     [junit4] #
>>     [junit4] # JRE version: Java(TM) SE Runtime Environment (9.0-b72) (build 1.9.0-ea-b72)
>>     [junit4] # Java VM: Java HotSpot(TM) 64-Bit Server VM (1.9.0-ea-b72 mixed mode linux-amd64 )
>>     [junit4] # Problematic frame:
>>     [junit4] # V  [libjvm.so+0x537da5]  DefNewGeneration::copy_to_survivor_space(oopDesc*)+0x25
>>     [junit4] #
>>     [junit4] # No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
>>     [junit4] #
>>     [junit4] # An error report file with more information is saved as:
>>     [junit4] # /home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J2/hs_err_pid7402.log
>>     [junit4] #
>>     [junit4] # If you would like to submit a bug report, please visit:
>>     [junit4] #   http://bugreport.java.com/bugreport/crash.jsp
>>     [junit4] #
>>     [junit4] <<< JVM J2: EOF ----
>>
>> [...truncated 207 lines...]
>>     [junit4] JVM J0: stdout was not empty, see: /home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/temp/junit4-J0-20150714_205152_761.sysout
>>     [junit4] >>> JVM J0: stdout (verbatim) ----
>>     [junit4] #
>>     [junit4] # A fatal error has been detected by the Java Runtime Environment:
>>     [junit4] #
>>     [junit4] #  SIGSEGV (0xb) at pc=0x00007fecf93696ef, pid=7408, tid=0x00007fecc3d3a700
>>     [junit4] #
>>     [junit4] # JRE version: Java(TM) SE Runtime Environment (9.0-b72) (build 1.9.0-ea-b72)
>>     [junit4] # Java VM: Java HotSpot(TM) 64-Bit Server VM (1.9.0-ea-b72 mixed mode linux-amd64 )
>>     [junit4] # Problematic frame:
>>     [junit4] # V  [libjvm.so+0x5376ef]  DefNewGeneration::drain_promo_failure_scan_stack()+0x3f
>>     [junit4] #
>>     [junit4] # No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
>>     [junit4] #
>>     [junit4] # An error report file with more information is saved as:
>>     [junit4] # /home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/hs_err_pid7408.log
>>     [junit4] #
>>     [junit4] # If you would like to submit a bug report, please visit:
>>     [junit4] #   http://bugreport.java.com/bugreport/crash.jsp
>>     [junit4] #
>>     [junit4] <<< JVM J0: EOF ----
>>
>> [...truncated 1 lines...]
>>     [junit4] ERROR: JVM J0 ended with an exception, command line: /home/jenkins/tools/java/64bit/jdk1.9.0-ea-b72/bin/java -XX:-UseCompressedOops -XX:+UseSerialGC -Djava.locale.providers=JRE,SPI -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/heapdumps -ea -esa -Dtests.prefix=tests -Dtests.seed=90C16F4AC280153C -Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random -Dtests.postingsformat=random -Dtests.docvaluesformat=random -Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random -Dtests.linedocsfile=europarl.lines.txt.gz -Dtests.luceneMatchVersion=6.0.0 -Dtests.cleanthreads=perClass -Djava.util.logging.config.file=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/tools/junit4/logging.properties -Dtests.nightly=false -Dtests.weekly=false -Dtests.monster=false -Dtests.slow=true -Dtests.asserts=true -Dtests.multiplier=3 -DtempDir=./temp -Djava.io.tmpdir=./temp -Djunit4.tempDir=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/temp -Dcommon.dir=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene -Dclover.db.dir=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/clover/db -Djava.security.policy=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/tools/junit4/solr-tests.policy -Dtests.LUCENE_VERSION=6.0.0 -Djetty.testMode=1 -Djetty.insecurerandom=1 -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -Djava.awt.headless=true -Djdk.map.althashing.threshold=0 -Dtests.leaveTemporary=false -Dtests.filterstacks=true -Dtests.disableHdfs=true -Djava.security.manager=org.apache.lucene.util.TestSecurityManager -classpath /home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/classes/test:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-test-framework/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/test-framework/lib/junit4-ant-2.1.13.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/test-framework/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/codecs/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/analysis/common/lucene-analyzers-common-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/codecs/lucene-codecs-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/backward-codecs/lucene-backward-codecs-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/highlighter/lucene-highlighter-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/memory/lucene-memory-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/misc/lucene-misc-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/spatial/lucene-spatial-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/expressions/lucene-expressions-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/suggest/lucene-suggest-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/grouping/lucene-grouping-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/queries/lucene-queries-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/queryparser/lucene-queryparser-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/join/lucene-join-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/sandbox/lucene-sandbox-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/antlr-runtime-3.5.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/antlr4-runtime-4.5.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/asm-4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/asm-commons-4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/caffeine-1.0.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/commons-cli-1.2.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/commons-codec-1.10.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/commons-collections-3.2.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/commons-configuration-1.6.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/commons-fileupload-1.2.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/commons-lang-2.6.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/dom4j-1.6.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/fastutil-6.5.11.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/guava-14.0.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/hadoop-annotations-2.6.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/hadoop-auth-2.6.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/hadoop-common-2.6.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/hadoop-hdfs-2.6.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/hll-1.6.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/hppc-0.5.2.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/htrace-core-3.0.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/jackson-core-2.5.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/jackson-dataformat-smile-2.5.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/joda-time-2.2.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/log4j-1.2.17.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/org.restlet-2.3.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/org.restlet.ext.servlet-2.3.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/presto-parser-0.108.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/protobuf-java-2.5.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/slf4j-log4j12-1.7.7.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/slice-0.10.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/spatial4j-0.4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/t-digest-3.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/commons-io-2.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/httpclient-4.4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/httpcore-4.4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/httpmime-4.4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/noggit-0.6.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/slf4j-api-1.7.7.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/stax2-api-3.1.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/woodstox-core-asl-4.4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/zookeeper-3.4.6.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/javax.servlet-api-3.1.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-continuation-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-deploy-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-http-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-io-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-jmx-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-rewrite-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-security-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-server-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-servlet-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-servlets-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-util-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-webapp-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-xml-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/example-DIH/solr/db/lib/derby-10.9.1.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/core/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/test-framework/lib/junit-4.10.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/test-framework/lib/randomizedtesting-runner-2.1.13.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/antlr-2.7.7.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/antlr4-runtime-4.5.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-core-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-core-api-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-core-avl-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-core-shared-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-i18n-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptor-kerberos-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-admin-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/api-all-1.0.0-M20.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/bcprov-jdk15-1.45.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/cglib-nodep-2.2.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/commons-collections-3.2.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/commons-math3-3.4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/easymock-3.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/ehcache-core-2.4.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jackson-annotations-2.5.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jackson-databind-2.5.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jcl-over-slf4j-1.7.7.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jersey-core-1.9.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jersey-server-1.9.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jetty-6.1.26.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jetty-util-6.1.26.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/mina-core-2.0.0-M5.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/objenesis-1.2.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/presto-parser-0.108.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/slice-0.10.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/analysis/icu/lucene-analyzers-icu-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/contrib/solr-analysis-extras/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-launcher.jar:/var/lib/jenkins/.ant/lib/ivy-2.3.0.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jdepend.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bcel.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jmf.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit4.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-xalan2.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-javamail.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jai.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bsf.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-logging.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-net.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-resolver.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-log4j.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-oro.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-antlr.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jsch.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-regexp.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-swing.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-testutil.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-netrexx.jar:/var/lib/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.1.13.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/temp/junit4-J0-20150714_205152_761.events @/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/temp/junit4-J0-20150714_205152_761.suites
>>     [junit4] ERROR: JVM J0 ended with an exception: Forked process returned with error code: 134. Very likely a JVM crash.  Process output piped in logs above.
>>     [junit4]     at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1484)
>>     [junit4]     at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:133)
>>     [junit4]     at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:964)
>>     [junit4]     at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:961)
>>     [junit4]     at java.util.concurrent.FutureTask.run(FutureTask.java:265)
>>     [junit4]     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>     [junit4]     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>     [junit4]     at java.lang.Thread.run(Thread.java:745)
>>     [junit4] ERROR: JVM J1 ended with an exception, command line: /home/jenkins/tools/java/64bit/jdk1.9.0-ea-b72/bin/java -XX:-UseCompressedOops -XX:+UseSerialGC -Djava.locale.providers=JRE,SPI -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/heapdumps -ea -esa -Dtests.prefix=tests -Dtests.seed=90C16F4AC280153C -Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random -Dtests.postingsformat=random -Dtests.docvaluesformat=random -Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random -Dtests.linedocsfile=europarl.lines.txt.gz -Dtests.luceneMatchVersion=6.0.0 -Dtests.cleanthreads=perClass -Djava.util.logging.config.file=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/tools/junit4/logging.properties -Dtests.nightly=false -Dtests.weekly=false -Dtests.monster=false -Dtests.slow=true -Dtests.asserts=true -Dtests.multiplier=3 -DtempDir=./temp -Djava.io.tmpdir=./temp -Djunit4.tempDir=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/temp -Dcommon.dir=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene -Dclover.db.dir=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/clover/db -Djava.security.policy=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/tools/junit4/solr-tests.policy -Dtests.LUCENE_VERSION=6.0.0 -Djetty.testMode=1 -Djetty.insecurerandom=1 -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -Djava.awt.headless=true -Djdk.map.althashing.threshold=0 -Dtests.leaveTemporary=false -Dtests.filterstacks=true -Dtests.disableHdfs=true -Djava.security.manager=org.apache.lucene.util.TestSecurityManager -classpath /home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/classes/test:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-test-framework/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/test-framework/lib/junit4-ant-2.1.13.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/test-framework/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/codecs/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/analysis/common/lucene-analyzers-common-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/codecs/lucene-codecs-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/backward-codecs/lucene-backward-codecs-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/highlighter/lucene-highlighter-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/memory/lucene-memory-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/misc/lucene-misc-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/spatial/lucene-spatial-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/expressions/lucene-expressions-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/suggest/lucene-suggest-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/grouping/lucene-grouping-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/queries/lucene-queries-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/queryparser/lucene-queryparser-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/join/lucene-join-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/sandbox/lucene-sandbox-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/antlr-runtime-3.5.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/antlr4-runtime-4.5.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/asm-4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/asm-commons-4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/caffeine-1.0.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/commons-cli-1.2.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/commons-codec-1.10.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/commons-collections-3.2.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/commons-configuration-1.6.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/commons-fileupload-1.2.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/commons-lang-2.6.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/dom4j-1.6.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/fastutil-6.5.11.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/guava-14.0.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/hadoop-annotations-2.6.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/hadoop-auth-2.6.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/hadoop-common-2.6.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/hadoop-hdfs-2.6.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/hll-1.6.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/hppc-0.5.2.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/htrace-core-3.0.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/jackson-core-2.5.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/jackson-dataformat-smile-2.5.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/joda-time-2.2.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/log4j-1.2.17.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/org.restlet-2.3.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/org.restlet.ext.servlet-2.3.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/presto-parser-0.108.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/protobuf-java-2.5.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/slf4j-log4j12-1.7.7.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/slice-0.10.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/spatial4j-0.4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/t-digest-3.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/commons-io-2.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/httpclient-4.4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/httpcore-4.4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/httpmime-4.4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/noggit-0.6.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/slf4j-api-1.7.7.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/stax2-api-3.1.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/woodstox-core-asl-4.4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/zookeeper-3.4.6.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/javax.servlet-api-3.1.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-continuation-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-deploy-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-http-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-io-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-jmx-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-rewrite-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-security-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-server-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-servlet-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-servlets-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-util-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-webapp-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-xml-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/example-DIH/solr/db/lib/derby-10.9.1.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/core/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/test-framework/lib/junit-4.10.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/test-framework/lib/randomizedtesting-runner-2.1.13.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/antlr-2.7.7.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/antlr4-runtime-4.5.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-core-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-core-api-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-core-avl-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-core-shared-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-i18n-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptor-kerberos-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-admin-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/api-all-1.0.0-M20.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/bcprov-jdk15-1.45.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/cglib-nodep-2.2.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/commons-collections-3.2.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/commons-math3-3.4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/easymock-3.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/ehcache-core-2.4.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jackson-annotations-2.5.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jackson-databind-2.5.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jcl-over-slf4j-1.7.7.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jersey-core-1.9.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jersey-server-1.9.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jetty-6.1.26.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jetty-util-6.1.26.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/mina-core-2.0.0-M5.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/objenesis-1.2.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/presto-parser-0.108.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/slice-0.10.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/analysis/icu/lucene-analyzers-icu-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/contrib/solr-analysis-extras/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-launcher.jar:/var/lib/jenkins/.ant/lib/ivy-2.3.0.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jdepend.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bcel.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jmf.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit4.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-xalan2.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-javamail.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jai.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bsf.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-logging.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-net.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-resolver.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-log4j.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-oro.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-antlr.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jsch.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-regexp.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-swing.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-testutil.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-netrexx.jar:/var/lib/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.1.13.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/temp/junit4-J1-20150714_205152_761.events @/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/temp/junit4-J1-20150714_205152_761.suites
>>     [junit4] ERROR: JVM J1 ended with an exception: Forked process returned with error code: 134. Very likely a JVM crash.  Process output piped in logs above.
>>     [junit4]     at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1484)
>>     [junit4]     at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:133)
>>     [junit4]     at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:964)
>>     [junit4]     at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:961)
>>     [junit4]     at java.util.concurrent.FutureTask.run(FutureTask.java:265)
>>     [junit4]     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>     [junit4]     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>     [junit4]     at java.lang.Thread.run(Thread.java:745)
>>     [junit4] ERROR: JVM J2 ended with an exception, command line: /home/jenkins/tools/java/64bit/jdk1.9.0-ea-b72/bin/java -XX:-UseCompressedOops -XX:+UseSerialGC -Djava.locale.providers=JRE,SPI -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/heapdumps -ea -esa -Dtests.prefix=tests -Dtests.seed=90C16F4AC280153C -Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random -Dtests.postingsformat=random -Dtests.docvaluesformat=random -Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random -Dtests.linedocsfile=europarl.lines.txt.gz -Dtests.luceneMatchVersion=6.0.0 -Dtests.cleanthreads=perClass -Djava.util.logging.config.file=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/tools/junit4/logging.properties -Dtests.nightly=false -Dtests.weekly=false -Dtests.monster=false -Dtests.slow=true -Dtests.asserts=true -Dtests.multiplier=3 -DtempDir=./temp -Djava.io.tmpdir=./temp -Djunit4.tempDir=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/temp -Dcommon.dir=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene -Dclover.db.dir=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/clover/db -Djava.security.policy=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/tools/junit4/solr-tests.policy -Dtests.LUCENE_VERSION=6.0.0 -Djetty.testMode=1 -Djetty.insecurerandom=1 -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -Djava.awt.headless=true -Djdk.map.althashing.threshold=0 -Dtests.leaveTemporary=false -Dtests.filterstacks=true -Dtests.disableHdfs=true -Djava.security.manager=org.apache.lucene.util.TestSecurityManager -classpath /home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/classes/test:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-test-framework/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/test-framework/lib/junit4-ant-2.1.13.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/test-framework/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/codecs/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/analysis/common/lucene-analyzers-common-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/codecs/lucene-codecs-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/backward-codecs/lucene-backward-codecs-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/highlighter/lucene-highlighter-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/memory/lucene-memory-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/misc/lucene-misc-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/spatial/lucene-spatial-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/expressions/lucene-expressions-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/suggest/lucene-suggest-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/grouping/lucene-grouping-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/queries/lucene-queries-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/queryparser/lucene-queryparser-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/join/lucene-join-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/sandbox/lucene-sandbox-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/antlr-runtime-3.5.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/antlr4-runtime-4.5.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/asm-4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/asm-commons-4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/caffeine-1.0.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/commons-cli-1.2.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/commons-codec-1.10.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/commons-collections-3.2.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/commons-configuration-1.6.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/commons-fileupload-1.2.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/commons-lang-2.6.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/dom4j-1.6.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/fastutil-6.5.11.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/guava-14.0.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/hadoop-annotations-2.6.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/hadoop-auth-2.6.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/hadoop-common-2.6.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/hadoop-hdfs-2.6.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/hll-1.6.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/hppc-0.5.2.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/htrace-core-3.0.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/jackson-core-2.5.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/jackson-dataformat-smile-2.5.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/joda-time-2.2.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/log4j-1.2.17.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/org.restlet-2.3.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/org.restlet.ext.servlet-2.3.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/presto-parser-0.108.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/protobuf-java-2.5.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/slf4j-log4j12-1.7.7.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/slice-0.10.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/spatial4j-0.4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/lib/t-digest-3.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/commons-io-2.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/httpclient-4.4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/httpcore-4.4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/httpmime-4.4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/noggit-0.6.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/slf4j-api-1.7.7.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/stax2-api-3.1.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/woodstox-core-asl-4.4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/lib/zookeeper-3.4.6.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/javax.servlet-api-3.1.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-continuation-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-deploy-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-http-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-io-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-jmx-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-rewrite-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-security-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-server-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-servlet-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-servlets-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-util-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-webapp-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/server/lib/jetty-xml-9.2.11.v20150529.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/example-DIH/solr/db/lib/derby-10.9.1.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/core/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/test-framework/lib/junit-4.10.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/test-framework/lib/randomizedtesting-runner-2.1.13.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/antlr-2.7.7.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/antlr4-runtime-4.5.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-core-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-core-api-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-core-avl-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-core-shared-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-i18n-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptor-kerberos-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-admin-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/api-all-1.0.0-M20.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/bcprov-jdk15-1.45.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/cglib-nodep-2.2.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/commons-collections-3.2.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/commons-math3-3.4.1.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/easymock-3.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/ehcache-core-2.4.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jackson-annotations-2.5.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jackson-databind-2.5.4.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jcl-over-slf4j-1.7.7.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jersey-core-1.9.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jersey-server-1.9.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jetty-6.1.26.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/jetty-util-6.1.26.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/mina-core-2.0.0-M5.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/objenesis-1.2.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/presto-parser-0.108.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/test-lib/slice-0.10.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/analysis/icu/lucene-analyzers-icu-6.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/contrib/solr-analysis-extras/classes/java:/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-launcher.jar:/var/lib/jenkins/.ant/lib/ivy-2.3.0.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jdepend.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bcel.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jmf.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit4.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-xalan2.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-javamail.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jai.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bsf.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-logging.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-net.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-resolver.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-log4j.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-oro.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-antlr.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jsch.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-regexp.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-swing.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-testutil.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-netrexx.jar:/var/lib/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.1.13.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/temp/junit4-J2-20150714_205152_761.events @/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/temp/junit4-J2-20150714_205152_761.suites
>>     [junit4] ERROR: JVM J2 ended with an exception: Forked process returned with error code: 134. Very likely a JVM crash.  Process output piped in logs above.
>>     [junit4]     at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1484)
>>     [junit4]     at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:133)
>>     [junit4]     at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:964)
>>     [junit4]     at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:961)
>>     [junit4]     at java.util.concurrent.FutureTask.run(FutureTask.java:265)
>>     [junit4]     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>     [junit4]     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>     [junit4]     at java.lang.Thread.run(Thread.java:745)
>>
>> BUILD FAILED
>> /home/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:526: The following error occurred while executing this line:
>> /home/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:474: The following error occurred while executing this line:
>> /home/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:61: The following error occurred while executing this line:
>> /home/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:39: The following error occurred while executing this line:
>> /home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:230: The following error occurred while executing this line:
>> /home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:513: The following error occurred while executing this line:
>> /home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1426: The following error occurred while executing this line:
>> /home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:981: At least one slave process threw an exception, first: Forked process returned with error code: 134. Very likely a JVM crash.  Process output piped in logs above.
>>
>> Total time: 35 minutes 44 seconds
>> Build step 'Invoke Ant' marked build as failure
>> Archiving artifacts
>> Recording test results
>> Email was triggered for: Failure - Any
>> Sending email for trigger: Failure - Any
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe at lucene.apache.org
>> For additional commands, e-mail: dev-help at lucene.apache.org
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe at lucene.apache.org
> For additional commands, e-mail: dev-help at lucene.apache.org
>

-- 
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA , Dublin, Ireland




From michail.chernov at oracle.com  Wed Jul 15 15:10:31 2015
From: michail.chernov at oracle.com (Michail Chernov)
Date: Wed, 15 Jul 2015 18:10:31 +0300
Subject: RFR: 8131343: Remove unused imports from
	hotspot/test/testlibrary/jdk/test/lib/*.java
Message-ID: <55A677E7.3090405@oracle.com>

Hi,

Could you please review this small patch to 
hotspot/test/testlibrary/jdk/test/lib/*.java which removes unused import 
and fixes spelling.

http://cr.openjdk.java.net/~mchernov/8131343/webrev.00/
https://bugs.openjdk.java.net/browse/JDK-8131343

Thanks,
Michail





From tom.benson at oracle.com  Wed Jul 15 15:42:09 2015
From: tom.benson at oracle.com (Tom Benson)
Date: Wed, 15 Jul 2015 11:42:09 -0400
Subject: RFR (S): 8131319: Move G1Allocator::_summary_bytes_used back
	to G1CollectedHeap
In-Reply-To: <1436957266.2282.35.camel@oracle.com>
References: <1436957266.2282.35.camel@oracle.com>
Message-ID: <55A67F51.9010504@oracle.com>

Hi Thomas,
Makes sense and looks good to me, with one trivial comment. 
G1Allocator::used() was renamed to ::used_in_alloc_regions(), presumably 
to indicate it only returns the amount used in the current active 
allocation region(s).   But at present, at least, it only returns the 
size for the current mutator region.  Is the plural name looking forward 
to an expected future change?

Also an observation..  G1ArchiveAllocator still has its own 
_summary_bytes_used.  This has to be cleared when recalculate_used is 
used to reset the heap's _summary_bytes_used, in a couple of places, EG:

   4106           set_used(recalculate_used());
   4107           if (_archive_allocator != NULL) {
   4108             _archive_allocator->clear_used();
   4109           }

That's because recalculate used walks all the G1 regions so its total 
includes the space allocated by an ArchiveAllocator.   Now that 
_summary_bytes_used is in _g1h rather than the _allocator, we could 
consider having the ArchiveAllocator update that total, rather than its 
local one.  That should allow the somewhat odd clear_used() calls to go 
away, but we would then have ArchiveAllocator calling 
_g1h->increase_used, which also doesn't seem ideal.   What do you think?

Tom

On 7/15/2015 6:47 AM, Thomas Schatzl wrote:
> Hi all,
>
>    can I have comments and reviews for the following change to move
> G1Allocator::_summary_bytes_used to a more fitting place?
>
> The problem from my POV is, that the member
> G1Allocator::_summary_bytes_used, that contains the number of bytes held
> by the heap outside of the allocators is located within G1Allocator.
>
> That sounds odd, particularly because G1Allocator actually never uses
> it, it's managed completely by G1CollectedHeap.
>
> Before the addition of the G1ArchiveAllocator one could have made the
> argument that G1Allocator holds all memory contained in the heap, but
> this is not true any more: the memory occupied by G1ArchiveAllocator
> needs to be added to the total sum of all managed space anyway, so
> moving _summary_bytes_used out of G1Allocator seems sensible (to me).
>
> I talked to StefanJ about this change who originally moved
> _summary_used_bytes into G1Allocator, and we could not find an argument
> for keeping _summary_used_bytes in G1Allocator.
>
> This allows us to also refine the responsibilities of G1Allocator a
> little more, to be the owner of the current allocation areas (regions)
> only.
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8131319
>
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8131319/webrev/
>
> Testing:
> jprt
>
> Thanks,
>    Thomas
>
>



From dmitry.fazunenko at oracle.com  Wed Jul 15 17:03:46 2015
From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko)
Date: Wed, 15 Jul 2015 20:03:46 +0300
Subject: RFR: 8131343: Remove unused imports from
	hotspot/test/testlibrary/jdk/test/lib/*.java
In-Reply-To: <55A677E7.3090405@oracle.com>
References: <55A677E7.3090405@oracle.com>
Message-ID: <55A69272.3020702@oracle.com>

Michail,

the fix looks good to me.

Thanks,
Dima

On 15.07.2015 18:10, Michail Chernov wrote:
> Hi,
>
> Could you please review this small patch to 
> hotspot/test/testlibrary/jdk/test/lib/*.java which removes unused 
> import and fixes spelling.
>
> http://cr.openjdk.java.net/~mchernov/8131343/webrev.00/
> https://bugs.openjdk.java.net/browse/JDK-8131343
>
> Thanks,
> Michail
>
>
>



From thomas.schatzl at oracle.com  Thu Jul 16 08:40:56 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 16 Jul 2015 10:40:56 +0200
Subject: RFR (M): G1: Parallelize object self-forwarding and scanning during
	an evacuation failure
Message-ID: <1437036056.2361.8.camel@oracle.com>

Hi all,

  can I have reviews for the following change from a student (Walter
Gugenberger; he has signed the OCA) who worked on this issue last
semester?

The change parallelizes object self-forwarding and scanning during an
evacuation failure by:
- reuse the entire existing evacuation copy closure and work queue
mechanism in case of evacuation failure to store the work items.
This allows evacuation failure to automatically benefit from work
stealing etc.
- remove the dedicate evacuation failure handling closure because it is
not necessary any more.
  - there is one subtle change in behavior: the old evacuation failure
always had a NULL ReferenceProcessor, while now the standard reference
processor for references is applied. Since the NULL ReferenceProcessor
kept all references alive, now potentially less references will be kept
alive after an evacuation failure. I do not see a problem here.
- implementing per-thread preserved mark buffers

As for actual performance improvements, there is none: the main problem
is that in case of evacuation failure the code will simply serialize on
the FreeList_lock when trying to get a new allocation region instead of
the EvacFailureStack_lock.

Evacuation failure and the early-exit for evac failure as suggested in
the CR will only be really effective when the FreeList_lock can be
avoided. This will be added in a follow-up change.

CR:
https://bugs.openjdk.java.net/browse/JDK-8004687

Webrev:
http://cr.openjdk.java.net/~tschatzl/8004687/webrev/

Testing:
jprt, lots of internal evacuation failure testing, aurora runs with vm.quick/vm.gc nightly tests etc.

Thanks,
  THomas





From thomas.schatzl at oracle.com  Thu Jul 16 09:16:09 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 16 Jul 2015 11:16:09 +0200
Subject: RFR (S): 8131319: Move G1Allocator::_summary_bytes_used back to
	G1CollectedHeap
In-Reply-To: <55A67F51.9010504@oracle.com>
References: <1436957266.2282.35.camel@oracle.com> <55A67F51.9010504@oracle.com>
Message-ID: <1437038169.2361.26.camel@oracle.com>

Hi,

On Wed, 2015-07-15 at 11:42 -0400, Tom Benson wrote:
> Hi Thomas,
> Makes sense and looks good to me, with one trivial comment. 
> G1Allocator::used() was renamed to ::used_in_alloc_regions(), presumably 
> to indicate it only returns the amount used in the current active 
> allocation region(s).   But at present, at least, it only returns the 
> size for the current mutator region.  Is the plural name looking forward 
> to an expected future change?

Yes and no.

One further change (actually the next in line) is going to tighten the
G1Allocator interface so that it would be more easily possible to have
multiple mutator allocation regions at the same time by just hiding all
the details about how mutator allocation regions are managed from
G1CollectedHeap. We know of some applications where the contention on
the single mutator allocation region seems to be a problem (I created
JDK-8131668 for that because I could not find a completely suitable
existing one). So yes, there may be multiple allocation regions in the
future or some other mechanism to avoid that.

Such a change is not planned as part for the investigation about PLAB
fragmentation/waste in G1 (JDK-8030849) this change is about.

Also, there are not multiple regions to consider at the moment, because
the allocation regions for PLABs are actually temporary, at the end of
GC they are flushed to be regular regions, only remembering old gen
regions that may be reused in the next gc. I.e. at the time the
information is queried, these region allocators are always empty.

So, no in some aspects. :)

I have no particular opinion about the name, but G1Allocator at the
moment manages multiple allocation regions already, so it seems more
fitting. Because otherwise someone might ask why not be even more
specific and explicitly mention that at the moment there is exactly one
(mutator) alloc region we consider.

I was thinking of splitting the PLAB and TLAB allocator into separate
instances, but did not think it through. Maybe something that should be
considered in more detail in a separate CR.

> Also an observation..  G1ArchiveAllocator still has its own 
> _summary_bytes_used.  This has to be cleared when recalculate_used is 
> used to reset the heap's _summary_bytes_used, in a couple of places, EG:
> 
>    4106           set_used(recalculate_used());
>    4107           if (_archive_allocator != NULL) {
>    4108             _archive_allocator->clear_used();
>    4109           }

Yes, I saw that. I had thought about changing this, but then refrained
from it to make the change more straightforward.

> That's because recalculate used walks all the G1 regions so its total 
> includes the space allocated by an ArchiveAllocator.   Now that 
> _summary_bytes_used is in _g1h rather than the _allocator, we could 
> consider having the ArchiveAllocator update that total, rather than its 
> local one.  That should allow the somewhat odd clear_used() calls to go 
> away, but we would then have ArchiveAllocator calling 
> _g1h->increase_used, which also doesn't seem ideal.   What do you think?

Actually, the PLAB allocators do exactly that every time a region is
retired via updating the actual bytes copied in the collector policy
which is then later added to the total.

I will look into this since it seems it is not only me not liking the
"if (archive_allocator != NULL) { ... }" copy&paste code here as it
might be easy to forget.

Thanks,
  Thomas




From thomas.schatzl at oracle.com  Thu Jul 16 09:40:12 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 16 Jul 2015 11:40:12 +0200
Subject: RFR (M): 8004687: G1: Parallelize object self-forwarding and
	scanning during an evacuation failure
Message-ID: <1437039612.2361.27.camel@oracle.com>

(resend with CR number in the subject)
Hi all,

  can I have reviews for the following change from a student (Walter
Gugenberger; he has signed the OCA) who worked on this issue last
semester?

The change parallelizes object self-forwarding and scanning during an
evacuation failure by:
- reuse the entire existing evacuation copy closure and work queue
mechanism in case of evacuation failure to store the work items.
This allows evacuation failure to automatically benefit from work
stealing etc.
- remove the dedicate evacuation failure handling closure because it is
not necessary any more.
  - there is one subtle change in behavior: the old evacuation failure
always had a NULL ReferenceProcessor, while now the standard reference
processor for references is applied. Since the NULL ReferenceProcessor
kept all references alive, now potentially less references will be kept
alive after an evacuation failure. I do not see a problem here.
- implementing per-thread preserved mark buffers

As for actual performance improvements, there is none: the main problem
is that in case of evacuation failure the code will simply serialize on
the FreeList_lock when trying to get a new allocation region instead of
the EvacFailureStack_lock.

Evacuation failure and the early-exit for evac failure as suggested in
the CR will only be really effective when the FreeList_lock can be
avoided. This will be added in a follow-up change.

CR:
https://bugs.openjdk.java.net/browse/JDK-8004687

Webrev:
http://cr.openjdk.java.net/~tschatzl/8004687/webrev/

Testing:
jprt, lots of internal evacuation failure testing, aurora runs with vm.quick/vm.gc nightly tests etc.

Thanks,
  THomas







From thomas.schatzl at oracle.com  Thu Jul 16 10:23:56 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 16 Jul 2015 12:23:56 +0200
Subject: RFR (S): 8131319: Move G1Allocator::_summary_bytes_used back to
	G1CollectedHeap
In-Reply-To: <1437038169.2361.26.camel@oracle.com>
References: <1436957266.2282.35.camel@oracle.com>
	<55A67F51.9010504@oracle.com> <1437038169.2361.26.camel@oracle.com>
Message-ID: <1437042236.2361.36.camel@oracle.com>

Hi,

On Thu, 2015-07-16 at 11:16 +0200, Thomas Schatzl wrote:
> Hi,
> 
> On Wed, 2015-07-15 at 11:42 -0400, Tom Benson wrote:
> > Hi Thomas,
> [..]

> > Also an observation..  G1ArchiveAllocator still has its own 
> > _summary_bytes_used.  This has to be cleared when recalculate_used is 
> > used to reset the heap's _summary_bytes_used, in a couple of places, EG:
> > 
> >    4106           set_used(recalculate_used());
> >    4107           if (_archive_allocator != NULL) {
> >    4108             _archive_allocator->clear_used();
> >    4109           }
> 
> Yes, I saw that. I had thought about changing this, but then refrained
> from it to make the change more straightforward.
> 
> > That's because recalculate used walks all the G1 regions so its total 
> > includes the space allocated by an ArchiveAllocator.   Now that 
> > _summary_bytes_used is in _g1h rather than the _allocator, we could 
> > consider having the ArchiveAllocator update that total, rather than its 
> > local one.  That should allow the somewhat odd clear_used() calls to go 
> > away, but we would then have ArchiveAllocator calling 
> > _g1h->increase_used, which also doesn't seem ideal.   What do you think?
> 
> Actually, the PLAB allocators do exactly that every time a region is
> retired via updating the actual bytes copied in the collector policy
> which is then later added to the total.
> 
> I will look into this since it seems it is not only me not liking the
> "if (archive_allocator != NULL) { ... }" copy&paste code here as it
> might be easy to forget.

  so I looked through the code again, and one way of passing on the
amount of memory actually used would be to return that value in
G1Allocator::end_archive_alloc_range(), to be added by the caller to the
_summary_bytes_used.

What is problematic, and what I did not know was that
StringTable::copy_shared_string() may not actually call
G1Allocator::end_archive_alloc_range() in case some allocation was not
successful.
I would actually think that this is some conceptual flaw in the use of
the archive allocator, i.e. allow a begin() call without a mandatory
end(), but that is up to you to decide.
That could be fixed though. What do you think?

At the moment the VM will just exit, but it seems unsafe to me to not
terminate the scope in all cases.

Thanks,
  Thomas




From derek.white at oracle.com  Thu Jul 16 23:07:24 2015
From: derek.white at oracle.com (Derek White)
Date: Thu, 16 Jul 2015 19:07:24 -0400
Subject: Runtime command-line deprecation (Was Re: RFR: 8066821(S) Enhance
	command line processing to manage deprecating and obsoleting -XX
	command line arguments
In-Reply-To: <559AEC80.3010907@oracle.com>
References: <54B44661.8070007@oracle.com> <54B84E91.1050708@oracle.com>
	<F68EBF6B-38A5-4461-9ADD-41BB198FF943@oracle.com>
	<54D15A0B.6030501@oracle.com>
	<87B07B03-3423-4738-9617-0C7B72A74A6E@oracle.com>
	<559AEC80.3010907@oracle.com>
Message-ID: <55A8392C.3020404@oracle.com>

Hi Karen, Coleen,

I'll respond to one issue which you both brought up here, and respond to 
the rest inline.

I think you both were skeptical of the need to "deprecate & handle" an 
argument (where the argument isn't an alias for another argument), 
although Karen suggested that might be good for external option removal, 
which has a really long lead time.

 From the options I've looked at, the decision to ignore vs. handle a 
deprecated option depends on what the option does and what users expect. 
For simple tuning or diagnostic options, there is little harm if we 
don't handle "-XX:+ UseOldInlining". The user would be hard-pressed to 
tell the difference. But if we wanted to deprecate "-XX:+UseG1GC" (for 
an unlikely example), this would make a big difference if we simply 
ignored the option. In this case we should still handle the option after 
emitting a deprecation warning (for at least one release).

Should we ever go from "deprecate & handle" in one release to full 
removal in the next? Or should we always have a release that "deprecates 
& ignores" an option before removing it? Ignoring an old option is often 
a convenience for customers, but on the other hand it's kind of lying to 
the customers. Maybe they're asking for "XX:-MSG" and we say "sure boss" 
and pour on the monosodium glutamate :-)

On the other hand, really handling a deprecated option has a cost too, 
so code complexity (and reliability), or time and space overheads might 
call for simply ignoring a deprecated option instead of always handling 
it in one release and ignoring it in the next.

Coming up with some guidelines for this is a good idea, but I don't 
think hard rules will work.

Specific responses below.

FYI  - New webrev is going through last round of merge testing.

  - Derek

On 7/6/15 5:00 PM, Coleen Phillimore wrote:
>
> Hi,
>
> I'm happy for more clarity in the process of removing command line 
> arguments.  I have some questions and comments below.
>
> On 6/26/15 4:09 PM, Karen Kinnear wrote:
>> Derek,
>>
>> I am really glad you are looking into this. I expanded this to the runtime folks in case many of them have not yet seen this.
>> Mary just forwarded this and I believe you haven't checked it in yet, so perhaps still time to discuss and make sure
>> we all are together on the deprecation policy.
>>
>> The runtime team was discussing at staff this week how we handle deprecating jvm command-line options as
>> we are looking to do more clean up in the future.
>>
>> So our internal change control process classifies our exported interfaces into three categories:
>> External: command-line flags e.g. -verbose:gc, -Xmx, ...
>>      This includes any commercial flags
>> Private: -XX flags documented (e.g. performance tuning or troubleshooting)
>>      I would assume this would include -XX product, experimental flags and manageable flags
>> Internal: undocumented -XX flags
>>      I would assume this would include -XX develop and diagnostic flags
>>
>> (please correct me if my assumptions are wrong folks)
>
> This is a good categorization.  Although I think there's some grey 
> area between External and Private, where some XX options are so 
> commonly used they should be considered External.   Some of the GC 
> options may fall into this category like UseParNewGC.
Yes, I agree that there is unexpected flag promotion. It would be great 
to get a common understanding of where the lines are between External, 
Private, and Internal. The flag types like commercial, product, 
experimental, develop, diagnostic are important, although I'd argue that 
experimental flags are more likely "internal" not private. I think we 
also need a liberal definition of "Documented" to include not just 
official documentation but blog postings and even serious mentions in 
places like StackOverflow. If the only reference that Google can find to 
an -XX flag is in the openJDK source code, then chances are it is 
"Internal".
>> The way I understand that we handle private -XX options today is a 2-step removal: (obsolete_jvm_flags - where the
>> release number is in a table and could be undefined)
>>
>> Using your helpful taxonomy fromhttps://bugs.openjdk.java.net/browse/JDK-806682:
>>
>> Today: private -XX options use 2-step removal (obsolete_jvm_flags)
>>      release 1: Deprecate & Obsolete - warn about the option but do nothing with it (we can remove all code that supports it)
>>      release 2: Dead - unrecognized
>>    - the point of the 2-step is to give customers time to modify any scripts they use
>>
>> I believe we have very rarely removed External flags - since customers, licensees, etc. may expect them.
>>
>> Mini-proposal:
>> 1) I would recommend that we add a future set of changes to add consistent handling for the External flags -
>> so that they would follow a three-step removal:
>>      release 1: Deprecate & Handle - warn and keep supporting
>>      release 2: Deprecate & Obsolete - warn and do nothing
>>      release 3: Dead - unrecognized
>>
>> 2) For the Internal flags - I think it would be kindest to customers and not a huge amount of additional work if
>> we were to follow the Private model of using a 2 step.
>>
>> 3) leave the Private flags with the current 2-step removal
>
> Yes, this reflects our current model.
>> 4) add support for aliasing - for any of them
>>      So that if you are doing aliasing, you would follow the model you are adding
>>       release 1: Deprecated & Handled - i.e. warn and still support (and set the information for the new alias)
>>       release 2: Dead - unrecognized
>>
>> 5) Could we possibly take the two flags that followed a different model already, i.e. moving to
>> Deprecated & Handled and handle those as mistakes rather than part of a new general model?
>
> So this is my question which is around the code review in question.   
> Note, Derek, can you resend the RFR to hotspot-dev at openjdk.java.net so 
> it gets to all of us (even the compiler team may want to share in the 
> mechanism).
>
> Why are UseParNewGC and MaxGCMinorPauseMillis deprecated and handled 
> and not deprecated and obsolete?  I don't actually see why have the 
> distinction here?
>
> Did these flags follow the deprecation procedure below?  Why not just 
> make them obsolete, why have this other mechanism?  I may be asking 
> the same question as Karen.
>
> I think the aliased flags could have a deprecate and handle model 
> (where handle == alias) in which case you only need one table for the 
> aliased flags, and you need to keep them in globals.hpp so they're 
> parsed properly.

The real reason I wrote the code that way is because I was supporting 
the current policy (classic "mechanism not policy" checkin). I didn't 
(and don't) have the history on how we got there. But after thinking 
about it (see top post), I think there will be some cases where we'll 
have handle options instead of ignoring them and vice verse. And the 
final decision may come down to engineering, or customer, or partner 
concerns, or management whimsy, or whatever the current fad is. We're 
talking about code changes over multiple years here :-)
>> Or do you think we will see more cases other than aliasing in which we would want to
>>      release 1: Deprecate & Handle and then release 2: Dead
>>      rather than release 1: Deprecate & Obsolete and then 2: Dead
>>      or rather than a 3 step like the External option proposal above?
>
> I think for the aliased flags you want the first option here 
> (deprecate&handle), right?  But that's only because you need to keep 
> the option in globals.hpp so it parses correctly, otherwise the second 
> option (deprecate&obsolete) would be the preference?
>
> The options in the GC that are being deprecated and handled have had 
> warnings about them for a while, so making them obsolete doesn't feel 
> too soon.

Yes, but just went through the approval process.
>
> Also, I agree with Kim's comment below that your comment lines are too 
> long.  My fonts are too big to have windows this wide.
>
> thanks,
> Coleen
>> thanks,
>> Karen
>>
>> p.s. Note that all of the deprecation needs to
>> 1) work with licensee engineering to ensure we give licensee's a head's up and get feedback
>> 2) file a change control request
>>
>> - we try to do these both as bulk requests to reduce the processing overhead.
>>
>> p.p.s. Details
>>
>> 1.  Do the warnings print today or are they silent? Just want to make sure we are conscious of how
>> those are handled if any of this moves to the new unified logging mechanism for which the new default
>> for "warning" level is to print.
The deprecation messages go through warning(), which is controlled by 
PrintWarnings (defaults to true). This is how the obsolete flag warnings 
are printed. The new deprecation mechanism replaces a mishmash of calls 
to jio_fprintf() and warning(), with a variety of different ways of 
saying the same thing.
>> 2. "will likely be removed in a future release"? If we have already set the release it will be removed - is this a bit
>> vague or did I not read closely enough?
That text came from some of the deprecated GC options. If removal has 
been scheduled, we could say something more definite, or even the exact 
release. We don't print the exact "death" release for obsolete options 
currently though.
>>
>> On Feb 3, 2015, at 6:30 PM, Derek White wrote:
>>
>>> Request for review (again):
>>>
>>> - Updated with Kim's suggestion.
>>> - Stopped double-printing warnings in some cases.
>>> - Initial caps on warning() messages.
>>>
>>> Webrev:http://cr.openjdk.java.net/~drwhite/8066821/webrev.04/
>>> CR:https://bugs.openjdk.java.net/browse/JDK-8066821
>>> Tests:     jtreg, jprt
>>>
>>> Thanks for looking!
>>>
>>> - Derek
>>>
>>> On 1/28/15 3:47 PM, Kim Barrett wrote:
>>>
...
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150716/817d2a78/attachment.htm>

From david.holmes at oracle.com  Thu Jul 16 23:52:05 2015
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 17 Jul 2015 09:52:05 +1000
Subject: Runtime command-line deprecation (Was Re: RFR: 8066821(S) Enhance
	command line processing to manage deprecating and obsoleting -XX
	command line arguments
In-Reply-To: <55A8392C.3020404@oracle.com>
References: <54B44661.8070007@oracle.com> <54B84E91.1050708@oracle.com>
	<F68EBF6B-38A5-4461-9ADD-41BB198FF943@oracle.com>
	<54D15A0B.6030501@oracle.com>
	<87B07B03-3423-4738-9617-0C7B72A74A6E@oracle.com>
	<559AEC80.3010907@oracle.com> <55A8392C.3020404@oracle.com>
Message-ID: <55A843A5.1080108@oracle.com>

Hi Derek,

One general comment inline below that I meant to make to Karen's email 
earlier ...

On 17/07/2015 9:07 AM, Derek White wrote:
> Hi Karen, Coleen,
>
> I'll respond to one issue which you both brought up here, and respond to
> the rest inline.
>
> I think you both were skeptical of the need to "deprecate & handle" an
> argument (where the argument isn't an alias for another argument),
> although Karen suggested that might be good for external option removal,
> which has a really long lead time.
>
>  From the options I've looked at, the decision to ignore vs. handle a
> deprecated option depends on what the option does and what users expect.
> For simple tuning or diagnostic options, there is little harm if we
> don't handle "-XX:+ UseOldInlining". The user would be hard-pressed to
> tell the difference. But if we wanted to deprecate "-XX:+UseG1GC" (for
> an unlikely example), this would make a big difference if we simply
> ignored the option. In this case we should still handle the option after
> emitting a deprecation warning (for at least one release).
>
> Should we ever go from "deprecate & handle" in one release to full
> removal in the next? Or should we always have a release that "deprecates
> & ignores" an option before removing it? Ignoring an old option is often
> a convenience for customers, but on the other hand it's kind of lying to
> the customers. Maybe they're asking for "XX:-MSG" and we say "sure boss"
> and pour on the monosodium glutamate :-)
>
> On the other hand, really handling a deprecated option has a cost too,
> so code complexity (and reliability), or time and space overheads might
> call for simply ignoring a deprecated option instead of always handling
> it in one release and ignoring it in the next.
>
> Coming up with some guidelines for this is a good idea, but I don't
> think hard rules will work.
>
> Specific responses below.
>
> FYI  - New webrev is going through last round of merge testing.
>
>   - Derek
>
> On 7/6/15 5:00 PM, Coleen Phillimore wrote:
>>
>> Hi,
>>
>> I'm happy for more clarity in the process of removing command line
>> arguments.  I have some questions and comments below.
>>
>> On 6/26/15 4:09 PM, Karen Kinnear wrote:
>>> Derek,
>>>
>>> I am really glad you are looking into this. I expanded this to the
>>> runtime folks in case many of them have not yet seen this.
>>> Mary just forwarded this and I believe you haven't checked it in yet,
>>> so perhaps still time to discuss and make sure
>>> we all are together on the deprecation policy.
>>>
>>> The runtime team was discussing at staff this week how we handle
>>> deprecating jvm command-line options as
>>> we are looking to do more clean up in the future.
>>>
>>> So our internal change control process classifies our exported
>>> interfaces into three categories:
>>> External: command-line flags e.g. -verbose:gc, -Xmx, ...
>>>      This includes any commercial flags
>>> Private: -XX flags documented (e.g. performance tuning or
>>> troubleshooting)
>>>      I would assume this would include -XX product, experimental
>>> flags and manageable flags
>>> Internal: undocumented -XX flags
>>>      I would assume this would include -XX develop and diagnostic flags
>>>
>>> (please correct me if my assumptions are wrong folks)
>>
>> This is a good categorization.  Although I think there's some grey
>> area between External and Private, where some XX options are so
>> commonly used they should be considered External.   Some of the GC
>> options may fall into this category like UseParNewGC.
> Yes, I agree that there is unexpected flag promotion. It would be great
> to get a common understanding of where the lines are between External,
> Private, and Internal. The flag types like commercial, product,
> experimental, develop, diagnostic are important, although I'd argue that
> experimental flags are more likely "internal" not private. I think we
> also need a liberal definition of "Documented" to include not just
> official documentation but blog postings and even serious mentions in
> places like StackOverflow. If the only reference that Google can find to
> an -XX flag is in the openJDK source code, then chances are it is
> "Internal".

The classifications that Karen listed come from a document that 
pre-dates open-sourcing of Java by many years. In an OpenSource world 
all flags are "documented" - and there are a few webpages that serve as 
tables of VM options for everyone to see. Not to mention all the 
publicly visible bug reports. So a new definition of "documented" is 
certainly needed.

I also think we need definitions based more on the type of flag: 
product, develop, diagnostic, experimental, commercial - as a first 
level classification. With the concerns of external/internal/private 
applying as a secondary classification when needed. Obviously not all 
combinations make sense.

I agree that in the worst-case there could well be a three phase process:
  - deprecate and handle
  - deprecate and ignore
  - remove (give error on use)

but that should only be needed for external, product flags - those with 
the broadest use-base and exposure. Most will need the second two stages 
only. And for some we should be able to just rip them out whenever eg 
experimental-internal/private.

Aside: in the JDK deprecation has mostly been phase one only, 
unfortunately, with a handful of phase two. Alas no stage three.

Cheers,
David
------

>>> The way I understand that we handle private -XX options today is a
>>> 2-step removal: (obsolete_jvm_flags - where the
>>> release number is in a table and could be undefined)
>>>
>>> Using your helpful taxonomy
>>> fromhttps://bugs.openjdk.java.net/browse/JDK-806682:
>>>
>>> Today: private -XX options use 2-step removal (obsolete_jvm_flags)
>>>      release 1: Deprecate & Obsolete - warn about the option but do
>>> nothing with it (we can remove all code that supports it)
>>>      release 2: Dead - unrecognized
>>>    - the point of the 2-step is to give customers time to modify any
>>> scripts they use
>>>
>>> I believe we have very rarely removed External flags - since
>>> customers, licensees, etc. may expect them.
>>>
>>> Mini-proposal:
>>> 1) I would recommend that we add a future set of changes to add
>>> consistent handling for the External flags -
>>> so that they would follow a three-step removal:
>>>      release 1: Deprecate & Handle - warn and keep supporting
>>>      release 2: Deprecate & Obsolete - warn and do nothing
>>>      release 3: Dead - unrecognized
>>>
>>> 2) For the Internal flags - I think it would be kindest to customers
>>> and not a huge amount of additional work if
>>> we were to follow the Private model of using a 2 step.
>>>
>>> 3) leave the Private flags with the current 2-step removal
>>
>> Yes, this reflects our current model.
>>> 4) add support for aliasing - for any of them
>>>      So that if you are doing aliasing, you would follow the model
>>> you are adding
>>>       release 1: Deprecated & Handled - i.e. warn and still support
>>> (and set the information for the new alias)
>>>       release 2: Dead - unrecognized
>>>
>>> 5) Could we possibly take the two flags that followed a different
>>> model already, i.e. moving to
>>> Deprecated & Handled and handle those as mistakes rather than part of
>>> a new general model?
>>
>> So this is my question which is around the code review in question.
>> Note, Derek, can you resend the RFR to hotspot-dev at openjdk.java.net so
>> it gets to all of us (even the compiler team may want to share in the
>> mechanism).
>>
>> Why are UseParNewGC and MaxGCMinorPauseMillis deprecated and handled
>> and not deprecated and obsolete?  I don't actually see why have the
>> distinction here?
>>
>> Did these flags follow the deprecation procedure below?  Why not just
>> make them obsolete, why have this other mechanism?  I may be asking
>> the same question as Karen.
>>
>> I think the aliased flags could have a deprecate and handle model
>> (where handle == alias) in which case you only need one table for the
>> aliased flags, and you need to keep them in globals.hpp so they're
>> parsed properly.
>
> The real reason I wrote the code that way is because I was supporting
> the current policy (classic "mechanism not policy" checkin). I didn't
> (and don't) have the history on how we got there. But after thinking
> about it (see top post), I think there will be some cases where we'll
> have handle options instead of ignoring them and vice verse. And the
> final decision may come down to engineering, or customer, or partner
> concerns, or management whimsy, or whatever the current fad is. We're
> talking about code changes over multiple years here :-)
>>> Or do you think we will see more cases other than aliasing in which
>>> we would want to
>>>      release 1: Deprecate & Handle and then release 2: Dead
>>>      rather than release 1: Deprecate & Obsolete and then 2: Dead
>>>      or rather than a 3 step like the External option proposal above?
>>
>> I think for the aliased flags you want the first option here
>> (deprecate&handle), right?  But that's only because you need to keep
>> the option in globals.hpp so it parses correctly, otherwise the second
>> option (deprecate&obsolete) would be the preference?
>>
>> The options in the GC that are being deprecated and handled have had
>> warnings about them for a while, so making them obsolete doesn't feel
>> too soon.
>
> Yes, but just went through the approval process.
>>
>> Also, I agree with Kim's comment below that your comment lines are too
>> long.  My fonts are too big to have windows this wide.
>>
>> thanks,
>> Coleen
>>> thanks,
>>> Karen
>>>
>>> p.s. Note that all of the deprecation needs to
>>> 1) work with licensee engineering to ensure we give licensee's a
>>> head's up and get feedback
>>> 2) file a change control request
>>>
>>> - we try to do these both as bulk requests to reduce the processing
>>> overhead.
>>>
>>> p.p.s. Details
>>>
>>> 1.  Do the warnings print today or are they silent? Just want to make
>>> sure we are conscious of how
>>> those are handled if any of this moves to the new unified logging
>>> mechanism for which the new default
>>> for "warning" level is to print.
> The deprecation messages go through warning(), which is controlled by
> PrintWarnings (defaults to true). This is how the obsolete flag warnings
> are printed. The new deprecation mechanism replaces a mishmash of calls
> to jio_fprintf() and warning(), with a variety of different ways of
> saying the same thing.
>>> 2. "will likely be removed in a future release"? If we have already
>>> set the release it will be removed - is this a bit
>>> vague or did I not read closely enough?
> That text came from some of the deprecated GC options. If removal has
> been scheduled, we could say something more definite, or even the exact
> release. We don't print the exact "death" release for obsolete options
> currently though.
>>>
>>> On Feb 3, 2015, at 6:30 PM, Derek White wrote:
>>>
>>>> Request for review (again):
>>>>
>>>> - Updated with Kim's suggestion.
>>>> - Stopped double-printing warnings in some cases.
>>>> - Initial caps on warning() messages.
>>>>
>>>> Webrev:http://cr.openjdk.java.net/~drwhite/8066821/webrev.04/
>>>> CR:https://bugs.openjdk.java.net/browse/JDK-8066821
>>>> Tests:     jtreg, jprt
>>>>
>>>> Thanks for looking!
>>>>
>>>> - Derek
>>>>
>>>> On 1/28/15 3:47 PM, Kim Barrett wrote:
>>>>
> ...


From thomas.schatzl at oracle.com  Fri Jul 17 08:25:44 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Fri, 17 Jul 2015 10:25:44 +0200
Subject: RFR (M): 8004687: G1: Parallelize object self-forwarding and
	scanning during an evacuation failure
In-Reply-To: <1437039612.2361.27.camel@oracle.com>
References: <1437039612.2361.27.camel@oracle.com>
Message-ID: <1437121544.2363.12.camel@oracle.com>

Hi all,

  some additional comments for better understanding:

On Thu, 2015-07-16 at 11:40 +0200, Thomas Schatzl wrote:
> (resend with CR number in the subject)
> Hi all,
> 
>   can I have reviews for the following change from a student (Walter
> Gugenberger; he has signed the OCA) who worked on this issue last
> semester?
> 
> The change parallelizes object self-forwarding and scanning during an
> evacuation failure by:
> - reuse the entire existing evacuation copy closure and work queue
> mechanism in case of evacuation failure to store the work items.
> This allows evacuation failure to automatically benefit from work
> stealing etc.
> - remove the dedicate evacuation failure handling closure because it is
> not necessary any more.
>   - there is one subtle change in behavior: the old evacuation failure
> always had a NULL ReferenceProcessor, while now the standard reference
> processor for references is applied. Since the NULL ReferenceProcessor
> kept all references alive, now potentially less references will be kept
> alive after an evacuation failure. I do not see a problem here.
> - implementing per-thread preserved mark buffers
> 
> As for actual performance improvements, there is none: the main problem
> is that in case of evacuation failure the code will simply serialize on
> the FreeList_lock when trying to get a new allocation region instead of
> the EvacFailureStack_lock.

The reason why we serialize on the FreeList_lock is that the current
allocation path always falls through to in the end try to allocate a new
allocation region (because during evacuation failure there is no more
space left). This requires obtaining that FreeList_lock.

In situations where not many objects fail like if we only slightly
underestimate the amount of space needed for the collection, which is a
fairly common case (unless you force the problem due to misconfiguration
like I did for some of my measurements :) ), the FreeList_lock may not
be contended as much and so there are already significant visible
performance improvements in some cases.

> Evacuation failure and the early-exit for evac failure as suggested in
> the CR will only be really effective when the FreeList_lock can be
> avoided. This will be added in a follow-up change.

The main reason for this (arbitrary) split is that I wanted to make the
full change more easily reviewable.

> CR:
> https://bugs.openjdk.java.net/browse/JDK-8004687
> 
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8004687/webrev/
> 
> Testing:
> jprt, lots of internal evacuation failure testing, aurora runs with vm.quick/vm.gc nightly tests etc.
> 

Thanks,
  Thomas





From goetz.lindenmaier at sap.com  Fri Jul 17 09:52:14 2015
From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz)
Date: Fri, 17 Jul 2015 09:52:14 +0000
Subject: FW: RFR(S): 8130434: [TESTBUG] Harden TestLargePageUseForAuxMemory
	for more page size combinations.
Message-ID: <4295855A5C1DE049A61835A1887419CC2D006C48@DEWDFEMB12A.global.corp.sap>

Hi,

could I please get a review for this change?  I also please need a sponsor.
I posted to hotspot-dev before, gc-dev is probably better.
This change is crafted on top of "8079208:  gc/g1/TestLargePageUseForAuxMemory.java fails due to not considering page allocation granularity for setup".

Best regards,
  Goetz.

From: Lindenmaier, Goetz
Sent: Freitag, 10. Juli 2015 09:03
To: hotspot-dev at openjdk.java.net
Subject: RFR(S): 8130434: [TESTBUG] Harden TestLargePageUseForAuxMemory for more page size combinations.

Hi,

this test wants to enforce large page usage for the card table etc.
It fails for certain page sizes we see on ppc, linux and aix.

On some, large page size == page size holds and this test fails:
"To test we would require to use an invalid heap size (assert failed: 33554432 > 33554432)"
Fix: Just skip the test in this case as it's pointless.

We have a machine with page size = 64K and large page size 4M.
In this case heapSizeDiffForBitmap was too small to reduce the heap size.
Due to the big value for the normal page size heap alignment is > largePageSize
and the heap size was aligned up to the original value.
Error: "'G1 'Prev Bitmap': pg_sz=65536' missing from stdout/stderr"
Fix: consider heap alignement for heapSizeDiffForBitmap.

Please review this change.  I please need a sponsor.
http://cr.openjdk.java.net/~goetz/webrevs/8130434-lrgPg/webrev.01/

Besides ppc, I tested this change on  linuxx86_64, sun_64 and darwin_x86_64.

Best regards,
  Goetz
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150717/1d82be35/attachment.htm>

From sanhong.lsh at alibaba-inc.com  Fri Jul 17 12:12:30 2015
From: sanhong.lsh at alibaba-inc.com (=?GBK?B?wO7I/brsKMj9uuwp?=)
Date: Fri, 17 Jul 2015 20:12:30 +0800
Subject: Regarding to the fix for JDK-8048556
Message-ID: <007201d0c089$db008000$91018000$@alibaba-inc.com>

Hey folks,

This is San Hong, from Alipay.

We have a fix patch for Bug JDK-8048556  (we signed OCA with Oracle as
Alibaba group) and want to contribute it to OpenJDK.   I would ask here if
there is any interest and anyone can sponsor for this contribution?  Many
thanks in advance.

(I am cc??ing Ben, because  I just mentioned our work when Ben visited China
in May, also welcome your inputs on this)

 

More background information: 

we develop and deploy AlipayJDK which is  based on OpenJDK at Alipay.  We
run the stress performance test for our online systems and found that this
bug really put the performance impact on the rt(response time) of our
system, that??s why we have had to fix it in our JDK7.    

 

The basic idea of fixing is similar with the comments did by Tony in [1],
which needs the right gc counts, but we implemented it in a different way,
as we found it will require a lot of changes if we update the collect()
method on CollectedHeap to accept GC counts.   Instead, the pattern we used
is as follows: 

 

-        Remember the VM operation before initiating the gc??the example
code we implemented in VM_GenCollectFullConcurrent::doit()  in
vmCMSOperations.cpp:

 

  if (_gc_count_before == gch->total_collections()) {

    // The "full" of do_full_collection call below "forces"

    // a collection; the second arg, 0, below ensures that

    // only the young gen is collected. XXX In the future,

    // we'll probably need to have something in this interface

    // to say do this only if we are sure we will not bail

    // out to a full collection in this attempt, but that's

    // for the future.

    assert(SafepointSynchronize::is_at_safepoint(),

      "We can only be executing this arm of if at a safepoint");

GCCauseSetter gccs(gch, _gc_cause);

 

#ifdef ALIPAY_JDK

    VMOperationSetter vmop(gch, this);   // VMOperationSetter did the
similar thing with GCCauseSetter, which is used to record which vm operation
is used for current gc.

#endif

 

    gch->do_full_collection(gch->must_clear_all_soft_refs(),

                            0 /* collect only youngest gen */);

  } 

 

-        If the vm gc operation gets interrupted by GCLocker, we will save
it in GC_locker::check_active_before_gc() in gcLocker.cpp

 

#ifdef ALIPAY_JDK

bool GC_locker::check_active_before_gc(VM_GC_Operation * op) {

#else

bool GC_locker::check_active_before_gc() {

#endif

  assert(SafepointSynchronize::is_at_safepoint(), "only read at safepoint");

  if (is_active() && !_needs_gc) {

    verify_critical_count();

    

#ifdef ALIPAY_JDK

    assert(op != NULL, "VM_GC_Operation::doit must invoke
VMOperationSetter!");

    _interrupted_gc_op = op->clone_for_gclocker();

#endif

    

-        Finally, we invoke the saved vm gc operation in
GC_locker::jni_unlock(JavaThread* thread)  instead of calling to
Universe::heap()->collect(GCCause::_gc_locker);

 

#ifdef ALIPAY_JDK

        assert(_interrupted_gc_op != NULL, "VM_GC_Operation must implement
the clone_for_gclocker!");

        VM_GC_Operation* op =
const_cast<VM_GC_Operation*>(_interrupted_gc_op);

        VMThread::execute(op);

        delete _interrupted_gc_op;

        _interrupted_gc_op = NULL;

#else

        Universe::heap()->collect(GCCause::_gc_locker);

#endif            

 

The rationale here is:  the GCLocker is always trying to compensate the gc
operation which has been interrupted while it is active.

(the original implementation is doing the compensation blindly by calling
Universe::heap()->collect for all cases)

 

Your thoughts?   Looking forward to your replay and suggestion!

BTW ?C this patch passed our local FVT, SVT test suits on linux-xa64
platform, our team can port it to jdk9 as patch for review and also follow
up the tests for other platforms if necessary:)

 

[1]
https://bugs.openjdk.java.net/browse/JDK-8048556?page=com.atlassian.jira.plu
gin.system.issuetabpanels:comment-tabpanel

 

Thanks!

San Hong

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150717/f6566baa/attachment.htm>

From tom.benson at oracle.com  Fri Jul 17 13:06:59 2015
From: tom.benson at oracle.com (Tom Benson)
Date: Fri, 17 Jul 2015 09:06:59 -0400
Subject: RFR (S): 8131319: Move G1Allocator::_summary_bytes_used back
	to G1CollectedHeap
In-Reply-To: <1437038169.2361.26.camel@oracle.com>
References: <1436957266.2282.35.camel@oracle.com>	
	<55A67F51.9010504@oracle.com> <1437038169.2361.26.camel@oracle.com>
Message-ID: <55A8FDF3.4090006@oracle.com>

Hi Thomas,

On 7/16/2015 5:16 AM, Thomas Schatzl wrote:
> Hi,
>
> On Wed, 2015-07-15 at 11:42 -0400, Tom Benson wrote:
>> Hi Thomas,
>> Makes sense and looks good to me, with one trivial comment.
>> G1Allocator::used() was renamed to ::used_in_alloc_regions(), presumably
>> to indicate it only returns the amount used in the current active
>> allocation region(s).   But at present, at least, it only returns the
>> size for the current mutator region.  Is the plural name looking forward
>> to an expected future change?
> Yes and no.
>
> One further change (actually the next in line) is going to tighten the
> G1Allocator interface so that it would be more easily possible to have
> multiple mutator allocation regions at the same time by just hiding all
> the details about how mutator allocation regions are managed from
> G1CollectedHeap. We know of some applications where the contention on
> the single mutator allocation region seems to be a problem (I created
> JDK-8131668 for that because I could not find a completely suitable
> existing one). So yes, there may be multiple allocation regions in the
> future or some other mechanism to avoid that.
>
> Such a change is not planned as part for the investigation about PLAB
> fragmentation/waste in G1 (JDK-8030849) this change is about.
>
> Also, there are not multiple regions to consider at the moment, because
> the allocation regions for PLABs are actually temporary, at the end of
> GC they are flushed to be regular regions, only remembering old gen
> regions that may be reused in the next gc. I.e. at the time the
> information is queried, these region allocators are always empty.
>
> So, no in some aspects. :)
>
> I have no particular opinion about the name, but G1Allocator at the
> moment manages multiple allocation regions already, so it seems more
> fitting. Because otherwise someone might ask why not be even more
> specific and explicitly mention that at the moment there is exactly one
> (mutator) alloc region we consider.
>
> I was thinking of splitting the PLAB and TLAB allocator into separate
> instances, but did not think it through. Maybe something that should be
> considered in more detail in a separate CR.

OK.  Sounds like mostly yes, it's intentionally named that way for 
possible futures.

>
>> Also an observation..  G1ArchiveAllocator still has its own
>> _summary_bytes_used.  This has to be cleared when recalculate_used is
>> used to reset the heap's _summary_bytes_used, in a couple of places, EG:
>>
>>     4106           set_used(recalculate_used());
>>     4107           if (_archive_allocator != NULL) {
>>     4108             _archive_allocator->clear_used();
>>     4109           }
> Yes, I saw that. I had thought about changing this, but then refrained
> from it to make the change more straightforward.
>
>> That's because recalculate used walks all the G1 regions so its total
>> includes the space allocated by an ArchiveAllocator.   Now that
>> _summary_bytes_used is in _g1h rather than the _allocator, we could
>> consider having the ArchiveAllocator update that total, rather than its
>> local one.  That should allow the somewhat odd clear_used() calls to go
>> away, but we would then have ArchiveAllocator calling
>> _g1h->increase_used, which also doesn't seem ideal.   What do you think?
> Actually, the PLAB allocators do exactly that every time a region is
> retired via updating the actual bytes copied in the collector policy
> which is then later added to the total.
>
> I will look into this since it seems it is not only me not liking the
> "if (archive_allocator != NULL) { ... }" copy&paste code here as it
> might be easy to forget.

I agree it's worth a look.  Tnx,
Tom

> Thanks,
>    Thomas
>
>



From goetz.lindenmaier at sap.com  Fri Jul 17 13:24:54 2015
From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz)
Date: Fri, 17 Jul 2015 13:24:54 +0000
Subject: RFR(XS): 8131761: Fix merge error adding code that was removed in
	8077936.
Message-ID: <4295855A5C1DE049A61835A1887419CC2D006D99@DEWDFEMB12A.global.corp.sap>

Hi,

8077936 http://hg.openjdk.java.net/jdk9/dev/hotspot/rev/e57fce7b6ad2
removed the function obj_field_volatile(). This was added again
in this merge change: http://hg.openjdk.java.net/jdk9/dev/hotspot/rev/5f2ef612ed74.

This change removes it once more.

Please review this change. I please need a sponsor.
http://cr.openjdk.java.net/~goetz/webrevs/8131761-fixMerge/webrev.01/

Best regards,
  Goetz.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150717/782e6a7a/attachment.htm>

From daniel.daugherty at oracle.com  Fri Jul 17 13:29:27 2015
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Fri, 17 Jul 2015 07:29:27 -0600
Subject: Runtime command-line deprecation (Was Re: RFR: 8066821(S) Enhance
	command line processing to manage deprecating and obsoleting -XX
	command line arguments
In-Reply-To: <55A843A5.1080108@oracle.com>
References: <54B44661.8070007@oracle.com> <54B84E91.1050708@oracle.com>
	<F68EBF6B-38A5-4461-9ADD-41BB198FF943@oracle.com>
	<54D15A0B.6030501@oracle.com>
	<87B07B03-3423-4738-9617-0C7B72A74A6E@oracle.com>
	<559AEC80.3010907@oracle.com> <55A8392C.3020404@oracle.com>
	<55A843A5.1080108@oracle.com>
Message-ID: <55A90337.7030909@oracle.com>

One tiny correction below...

On 7/16/15 5:52 PM, David Holmes wrote:
> Hi Derek,
>
> One general comment inline below that I meant to make to Karen's email 
> earlier ...
>
> On 17/07/2015 9:07 AM, Derek White wrote:
>> Hi Karen, Coleen,
>>
>> I'll respond to one issue which you both brought up here, and respond to
>> the rest inline.
>>
>> I think you both were skeptical of the need to "deprecate & handle" an
>> argument (where the argument isn't an alias for another argument),
>> although Karen suggested that might be good for external option removal,
>> which has a really long lead time.
>>
>>  From the options I've looked at, the decision to ignore vs. handle a
>> deprecated option depends on what the option does and what users expect.
>> For simple tuning or diagnostic options, there is little harm if we
>> don't handle "-XX:+ UseOldInlining". The user would be hard-pressed to
>> tell the difference. But if we wanted to deprecate "-XX:+UseG1GC" (for
>> an unlikely example), this would make a big difference if we simply
>> ignored the option. In this case we should still handle the option after
>> emitting a deprecation warning (for at least one release).
>>
>> Should we ever go from "deprecate & handle" in one release to full
>> removal in the next? Or should we always have a release that "deprecates
>> & ignores" an option before removing it? Ignoring an old option is often
>> a convenience for customers, but on the other hand it's kind of lying to
>> the customers. Maybe they're asking for "XX:-MSG" and we say "sure boss"
>> and pour on the monosodium glutamate :-)
>>
>> On the other hand, really handling a deprecated option has a cost too,
>> so code complexity (and reliability), or time and space overheads might
>> call for simply ignoring a deprecated option instead of always handling
>> it in one release and ignoring it in the next.
>>
>> Coming up with some guidelines for this is a good idea, but I don't
>> think hard rules will work.
>>
>> Specific responses below.
>>
>> FYI  - New webrev is going through last round of merge testing.
>>
>>   - Derek
>>
>> On 7/6/15 5:00 PM, Coleen Phillimore wrote:
>>>
>>> Hi,
>>>
>>> I'm happy for more clarity in the process of removing command line
>>> arguments.  I have some questions and comments below.
>>>
>>> On 6/26/15 4:09 PM, Karen Kinnear wrote:
>>>> Derek,
>>>>
>>>> I am really glad you are looking into this. I expanded this to the
>>>> runtime folks in case many of them have not yet seen this.
>>>> Mary just forwarded this and I believe you haven't checked it in yet,
>>>> so perhaps still time to discuss and make sure
>>>> we all are together on the deprecation policy.
>>>>
>>>> The runtime team was discussing at staff this week how we handle
>>>> deprecating jvm command-line options as
>>>> we are looking to do more clean up in the future.
>>>>
>>>> So our internal change control process classifies our exported
>>>> interfaces into three categories:
>>>> External: command-line flags e.g. -verbose:gc, -Xmx, ...
>>>>      This includes any commercial flags
>>>> Private: -XX flags documented (e.g. performance tuning or
>>>> troubleshooting)
>>>>      I would assume this would include -XX product, experimental
>>>> flags and manageable flags
>>>> Internal: undocumented -XX flags
>>>>      I would assume this would include -XX develop and diagnostic 
>>>> flags
>>>>
>>>> (please correct me if my assumptions are wrong folks)
>>>
>>> This is a good categorization.  Although I think there's some grey
>>> area between External and Private, where some XX options are so
>>> commonly used they should be considered External.   Some of the GC
>>> options may fall into this category like UseParNewGC.
>> Yes, I agree that there is unexpected flag promotion. It would be great
>> to get a common understanding of where the lines are between External,
>> Private, and Internal. The flag types like commercial, product,
>> experimental, develop, diagnostic are important, although I'd argue that
>> experimental flags are more likely "internal" not private. I think we
>> also need a liberal definition of "Documented" to include not just
>> official documentation but blog postings and even serious mentions in
>> places like StackOverflow. If the only reference that Google can find to
>> an -XX flag is in the openJDK source code, then chances are it is
>> "Internal".
>
> The classifications that Karen listed come from a document that 
> pre-dates open-sourcing of Java by many years. In an OpenSource world 
> all flags are "documented" - and there are a few webpages that serve 
> as tables of VM options for everyone to see. Not to mention all the 
> publicly visible bug reports. So a new definition of "documented" is 
> certainly needed.
>
> I also think we need definitions based more on the type of flag: 
> product, develop, diagnostic, experimental, commercial - as a first 
> level classification. With the concerns of external/internal/private 
> applying as a secondary classification when needed. Obviously not all 
> combinations make sense.
>
> I agree that in the worst-case there could well be a three phase process:
>  - deprecate and handle
>  - deprecate and ignore
>  - remove (give error on use)
>
> but that should only be needed for external, product flags - those 
> with the broadest use-base and exposure. Most will need the second two 
> stages only. And for some we should be able to just rip them out 
> whenever eg experimental-internal/private.
>
> Aside: in the JDK deprecation has mostly been phase one only, 
> unfortunately, with a handful of phase two. Alas no stage three.

There's at least one stage 3 in the JDK:

java.lang.Thread.suspend() throws UnsupportedOperationException
since JDK8-B96.

https://bugs.openjdk.java.net/browse/JDK-7059085

http://hg.openjdk.java.net/jdk9/dev/jdk/annotate/669be1677ab7/src/share/classes/java/lang/Thread.java#l848

Dan


>
> Cheers,
> David
> ------
>
>>>> The way I understand that we handle private -XX options today is a
>>>> 2-step removal: (obsolete_jvm_flags - where the
>>>> release number is in a table and could be undefined)
>>>>
>>>> Using your helpful taxonomy
>>>> fromhttps://bugs.openjdk.java.net/browse/JDK-806682:
>>>>
>>>> Today: private -XX options use 2-step removal (obsolete_jvm_flags)
>>>>      release 1: Deprecate & Obsolete - warn about the option but do
>>>> nothing with it (we can remove all code that supports it)
>>>>      release 2: Dead - unrecognized
>>>>    - the point of the 2-step is to give customers time to modify any
>>>> scripts they use
>>>>
>>>> I believe we have very rarely removed External flags - since
>>>> customers, licensees, etc. may expect them.
>>>>
>>>> Mini-proposal:
>>>> 1) I would recommend that we add a future set of changes to add
>>>> consistent handling for the External flags -
>>>> so that they would follow a three-step removal:
>>>>      release 1: Deprecate & Handle - warn and keep supporting
>>>>      release 2: Deprecate & Obsolete - warn and do nothing
>>>>      release 3: Dead - unrecognized
>>>>
>>>> 2) For the Internal flags - I think it would be kindest to customers
>>>> and not a huge amount of additional work if
>>>> we were to follow the Private model of using a 2 step.
>>>>
>>>> 3) leave the Private flags with the current 2-step removal
>>>
>>> Yes, this reflects our current model.
>>>> 4) add support for aliasing - for any of them
>>>>      So that if you are doing aliasing, you would follow the model
>>>> you are adding
>>>>       release 1: Deprecated & Handled - i.e. warn and still support
>>>> (and set the information for the new alias)
>>>>       release 2: Dead - unrecognized
>>>>
>>>> 5) Could we possibly take the two flags that followed a different
>>>> model already, i.e. moving to
>>>> Deprecated & Handled and handle those as mistakes rather than part of
>>>> a new general model?
>>>
>>> So this is my question which is around the code review in question.
>>> Note, Derek, can you resend the RFR to hotspot-dev at openjdk.java.net so
>>> it gets to all of us (even the compiler team may want to share in the
>>> mechanism).
>>>
>>> Why are UseParNewGC and MaxGCMinorPauseMillis deprecated and handled
>>> and not deprecated and obsolete?  I don't actually see why have the
>>> distinction here?
>>>
>>> Did these flags follow the deprecation procedure below?  Why not just
>>> make them obsolete, why have this other mechanism?  I may be asking
>>> the same question as Karen.
>>>
>>> I think the aliased flags could have a deprecate and handle model
>>> (where handle == alias) in which case you only need one table for the
>>> aliased flags, and you need to keep them in globals.hpp so they're
>>> parsed properly.
>>
>> The real reason I wrote the code that way is because I was supporting
>> the current policy (classic "mechanism not policy" checkin). I didn't
>> (and don't) have the history on how we got there. But after thinking
>> about it (see top post), I think there will be some cases where we'll
>> have handle options instead of ignoring them and vice verse. And the
>> final decision may come down to engineering, or customer, or partner
>> concerns, or management whimsy, or whatever the current fad is. We're
>> talking about code changes over multiple years here :-)
>>>> Or do you think we will see more cases other than aliasing in which
>>>> we would want to
>>>>      release 1: Deprecate & Handle and then release 2: Dead
>>>>      rather than release 1: Deprecate & Obsolete and then 2: Dead
>>>>      or rather than a 3 step like the External option proposal above?
>>>
>>> I think for the aliased flags you want the first option here
>>> (deprecate&handle), right?  But that's only because you need to keep
>>> the option in globals.hpp so it parses correctly, otherwise the second
>>> option (deprecate&obsolete) would be the preference?
>>>
>>> The options in the GC that are being deprecated and handled have had
>>> warnings about them for a while, so making them obsolete doesn't feel
>>> too soon.
>>
>> Yes, but just went through the approval process.
>>>
>>> Also, I agree with Kim's comment below that your comment lines are too
>>> long.  My fonts are too big to have windows this wide.
>>>
>>> thanks,
>>> Coleen
>>>> thanks,
>>>> Karen
>>>>
>>>> p.s. Note that all of the deprecation needs to
>>>> 1) work with licensee engineering to ensure we give licensee's a
>>>> head's up and get feedback
>>>> 2) file a change control request
>>>>
>>>> - we try to do these both as bulk requests to reduce the processing
>>>> overhead.
>>>>
>>>> p.p.s. Details
>>>>
>>>> 1.  Do the warnings print today or are they silent? Just want to make
>>>> sure we are conscious of how
>>>> those are handled if any of this moves to the new unified logging
>>>> mechanism for which the new default
>>>> for "warning" level is to print.
>> The deprecation messages go through warning(), which is controlled by
>> PrintWarnings (defaults to true). This is how the obsolete flag warnings
>> are printed. The new deprecation mechanism replaces a mishmash of calls
>> to jio_fprintf() and warning(), with a variety of different ways of
>> saying the same thing.
>>>> 2. "will likely be removed in a future release"? If we have already
>>>> set the release it will be removed - is this a bit
>>>> vague or did I not read closely enough?
>> That text came from some of the deprecated GC options. If removal has
>> been scheduled, we could say something more definite, or even the exact
>> release. We don't print the exact "death" release for obsolete options
>> currently though.
>>>>
>>>> On Feb 3, 2015, at 6:30 PM, Derek White wrote:
>>>>
>>>>> Request for review (again):
>>>>>
>>>>> - Updated with Kim's suggestion.
>>>>> - Stopped double-printing warnings in some cases.
>>>>> - Initial caps on warning() messages.
>>>>>
>>>>> Webrev:http://cr.openjdk.java.net/~drwhite/8066821/webrev.04/
>>>>> CR:https://bugs.openjdk.java.net/browse/JDK-8066821
>>>>> Tests:     jtreg, jprt
>>>>>
>>>>> Thanks for looking!
>>>>>
>>>>> - Derek
>>>>>
>>>>> On 1/28/15 3:47 PM, Kim Barrett wrote:
>>>>>
>> ...



From tom.benson at oracle.com  Fri Jul 17 13:34:59 2015
From: tom.benson at oracle.com (Tom Benson)
Date: Fri, 17 Jul 2015 09:34:59 -0400
Subject: RFR (S): 8131319: Move G1Allocator::_summary_bytes_used back
	to G1CollectedHeap
In-Reply-To: <1437042236.2361.36.camel@oracle.com>
References: <1436957266.2282.35.camel@oracle.com>	
	<55A67F51.9010504@oracle.com> <1437038169.2361.26.camel@oracle.com>
	<1437042236.2361.36.camel@oracle.com>
Message-ID: <55A90483.9030308@oracle.com>

Hi,

On 7/16/2015 6:23 AM, Thomas Schatzl wrote:
> Hi,
>
> On Thu, 2015-07-16 at 11:16 +0200, Thomas Schatzl wrote:
>> Hi,
>>
>> On Wed, 2015-07-15 at 11:42 -0400, Tom Benson wrote:
>>> Hi Thomas,
>> [..]
>>> Also an observation..  G1ArchiveAllocator still has its own
>>> _summary_bytes_used.  This has to be cleared when recalculate_used is
>>> used to reset the heap's _summary_bytes_used, in a couple of places, EG:
>>>
>>>     4106           set_used(recalculate_used());
>>>     4107           if (_archive_allocator != NULL) {
>>>     4108             _archive_allocator->clear_used();
>>>     4109           }
>> Yes, I saw that. I had thought about changing this, but then refrained
>> from it to make the change more straightforward.
>>
>>> That's because recalculate used walks all the G1 regions so its total
>>> includes the space allocated by an ArchiveAllocator.   Now that
>>> _summary_bytes_used is in _g1h rather than the _allocator, we could
>>> consider having the ArchiveAllocator update that total, rather than its
>>> local one.  That should allow the somewhat odd clear_used() calls to go
>>> away, but we would then have ArchiveAllocator calling
>>> _g1h->increase_used, which also doesn't seem ideal.   What do you think?
>> Actually, the PLAB allocators do exactly that every time a region is
>> retired via updating the actual bytes copied in the collector policy
>> which is then later added to the total.
>>
>> I will look into this since it seems it is not only me not liking the
>> "if (archive_allocator != NULL) { ... }" copy&paste code here as it
>> might be easy to forget.
>    so I looked through the code again, and one way of passing on the
> amount of memory actually used would be to return that value in
> G1Allocator::end_archive_alloc_range(), to be added by the caller to the
> _summary_bytes_used.
>
> What is problematic, and what I did not know was that
> StringTable::copy_shared_string() may not actually call
> G1Allocator::end_archive_alloc_range() in case some allocation was not
> successful.
> I would actually think that this is some conceptual flaw in the use of
> the archive allocator, i.e. allow a begin() call without a mandatory
> end(), but that is up to you to decide.
> That could be fixed though. What do you think?
>
> At the moment the VM will just exit, but it seems unsafe to me to not
> terminate the scope in all cases.

I did test that scenario (where the archive is left 'un-finalized') 
somewhat, while testing with hardcoded calls to the archive region code, 
so I believe it would work.  Collections still happened without failure 
- the active archive region is parseable, and is already marked 
'archive.'  But it's true that the shared string support will just exit 
the VM if an allocation fails, so it's certainly not a problem that 
'end' is not called today.

I doesn't strike me as a 'conceptual flaw.'  You can keep allocating in 
the current archive region until you call 'end'. If you don't call 
'end,' you can keep doing so until the JVM exits.  That is, if you never 
need the summary of region addresses that end_ provides. I can't think 
of a use for that off the top of my head, but I guess it could happen. 8^)
Tom

>
> Thanks,
>    Thomas
>
>



From tom.benson at oracle.com  Fri Jul 17 14:02:19 2015
From: tom.benson at oracle.com (Tom Benson)
Date: Fri, 17 Jul 2015 10:02:19 -0400
Subject: RFR (S): 8131319: Move G1Allocator::_summary_bytes_used back
	to G1CollectedHeap
In-Reply-To: <1437042236.2361.36.camel@oracle.com>
References: <1436957266.2282.35.camel@oracle.com>	
	<55A67F51.9010504@oracle.com> <1437038169.2361.26.camel@oracle.com>
	<1437042236.2361.36.camel@oracle.com>
Message-ID: <55A90AEB.7090006@oracle.com>

Hi again,

Forgot to reply to this part:
>> Actually, the PLAB allocators do exactly that every time a region is
>> retired via updating the actual bytes copied in the collector policy
>> which is then later added to the total.
>>
>> I will look into this since it seems it is not only me not liking the
>> "if (archive_allocator != NULL) { ... }" copy&paste code here as it
>> might be easy to forget.
>    so I looked through the code again, and one way of passing on the
> amount of memory actually used would be to return that value in
> G1Allocator::end_archive_alloc_range(), to be added by the caller to the
> _summary_bytes_used.

That would work with archive regions used as they are today, where there 
will not be a GC while one is 'active.'   But if we want to continue to 
allow that possibility, doing it there won't eliminate the 'clear_use' 
calls.   I think calling g1h->increase_used would eliminate both the 
'clear_use' calls and the check to see if _archive_allocator exists when 
computing the used size,  while still allowing GCs while an archive 
region was active.
Really, I guess this could be the subject of a separate change later.
Tom


From thomas.schatzl at oracle.com  Fri Jul 17 15:55:43 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Fri, 17 Jul 2015 17:55:43 +0200
Subject: RFR (S): 8131319: Move G1Allocator::_summary_bytes_used back to
	G1CollectedHeap
In-Reply-To: <55A90AEB.7090006@oracle.com>
References: <1436957266.2282.35.camel@oracle.com>
	<55A67F51.9010504@oracle.com> <1437038169.2361.26.camel@oracle.com>
	<1437042236.2361.36.camel@oracle.com> <55A90AEB.7090006@oracle.com>
Message-ID: <1437148543.9068.25.camel@oracle.com>

Hi,

On Fri, 2015-07-17 at 10:02 -0400, Tom Benson wrote:
> Hi again,
> 
> Forgot to reply to this part:
> >> Actually, the PLAB allocators do exactly that every time a region is
> >> retired via updating the actual bytes copied in the collector policy
> >> which is then later added to the total.
> >>
> >> I will look into this since it seems it is not only me not liking the
> >> "if (archive_allocator != NULL) { ... }" copy&paste code here as it
> >> might be easy to forget.
> >    so I looked through the code again, and one way of passing on the
> > amount of memory actually used would be to return that value in
> > G1Allocator::end_archive_alloc_range(), to be added by the caller to the
> > _summary_bytes_used.
> 
> That would work with archive regions used as they are today, where there 
> will not be a GC while one is 'active.'   But if we want to continue to 
> allow that possibility, doing it there won't eliminate the 'clear_use' 
> calls.   I think calling g1h->increase_used would eliminate both the 
> 'clear_use' calls and the check to see if _archive_allocator exists when 
> computing the used size,  while still allowing GCs while an archive 
> region was active.

One option could be that the archive allocator got notified of GC and
did whatever it needs to to update global data like the G1Allocator
(with init/release_*_regions()).

The problem with immediate updates would obviously be concurrency, e.g.
if multiple threads were allocating into an archive region(s). Or
another allocator doing its updates directly like for pinned regions.

Not sure about how MT safe the current archive allocator is actually.

> Really, I guess this could be the subject of a separate change later.

 Okay.

Thanks,
  Thomas





From tom.benson at oracle.com  Fri Jul 17 17:20:11 2015
From: tom.benson at oracle.com (Tom Benson)
Date: Fri, 17 Jul 2015 13:20:11 -0400
Subject: RFR (S): 8131319: Move G1Allocator::_summary_bytes_used back
	to G1CollectedHeap
In-Reply-To: <1437148543.9068.25.camel@oracle.com>
References: <1436957266.2282.35.camel@oracle.com>		
	<55A67F51.9010504@oracle.com>
	<1437038169.2361.26.camel@oracle.com>	
	<1437042236.2361.36.camel@oracle.com> <55A90AEB.7090006@oracle.com>
	<1437148543.9068.25.camel@oracle.com>
Message-ID: <55A9394B.4040703@oracle.com>

Hi Thomas,

On 7/17/2015 11:55 AM, Thomas Schatzl wrote:
> Hi,
>
> On Fri, 2015-07-17 at 10:02 -0400, Tom Benson wrote:
>> Hi again,
>>
>> Forgot to reply to this part:
>>>> Actually, the PLAB allocators do exactly that every time a region is
>>>> retired via updating the actual bytes copied in the collector policy
>>>> which is then later added to the total.
>>>>
>>>> I will look into this since it seems it is not only me not liking the
>>>> "if (archive_allocator != NULL) { ... }" copy&paste code here as it
>>>> might be easy to forget.
>>>     so I looked through the code again, and one way of passing on the
>>> amount of memory actually used would be to return that value in
>>> G1Allocator::end_archive_alloc_range(), to be added by the caller to the
>>> _summary_bytes_used.
>> That would work with archive regions used as they are today, where there
>> will not be a GC while one is 'active.'   But if we want to continue to
>> allow that possibility, doing it there won't eliminate the 'clear_use'
>> calls.   I think calling g1h->increase_used would eliminate both the
>> 'clear_use' calls and the check to see if _archive_allocator exists when
>> computing the used size,  while still allowing GCs while an archive
>> region was active.
> One option could be that the archive allocator got notified of GC and
> did whatever it needs to to update global data like the G1Allocator
> (with init/release_*_regions()).
>
> The problem with immediate updates would obviously be concurrency, e.g.
> if multiple threads were allocating into an archive region(s). Or
> another allocator doing its updates directly like for pinned regions.
>
> Not sure about how MT safe the current archive allocator is actually.
It is designed for single threaded use from the VM thread, not general 
allocation.

Anyway,  your change for 8131319 still looks good to me.  Tnx,
Tom

>
>> Really, I guess this could be the subject of a separate change later.
>   Okay.
>
> Thanks,
>    Thomas
>
>
>



From derek.white at oracle.com  Fri Jul 17 17:30:27 2015
From: derek.white at oracle.com (Derek White)
Date: Fri, 17 Jul 2015 13:30:27 -0400
Subject: RFR: 8066821(S) Enhance command line processing to manage
	deprecating and obsoleting -XX command line arguments
In-Reply-To: <54D15A0B.6030501@oracle.com>
References: <54B44661.8070007@oracle.com> <54B84E91.1050708@oracle.com>
	<F68EBF6B-38A5-4461-9ADD-41BB198FF943@oracle.com>
	<54D15A0B.6030501@oracle.com>
Message-ID: <55A93BB3.4040402@oracle.com>

Request for review:

[This updated webrev is being sent to wider audience, and has been 
merged with Gerard's option constraints check-in. Also factored out 
-XX:+AggressiveHeap processing into it's own chapter. I mean function :-)]

http://cr.openjdk.java.net/~drwhite/8066821/webrev.06/
https://bugs.openjdk.java.net/browse/JDK-8066821

This webrev adds support for handling "/deprecated/" -XX options 
(options that still *do* something but are planned for removal) and 
"/alias/" options (alternate names for other -XX options) by simply 
adding entries to the deprecated_jvm_flags and/or aliased_jvm_flags 
tables. This follows the example of the existing obsolete_jvm_flags table.

This replaces a lot of ad-hoc and occasionally wrong code in 
arguments.cpp as well as supporting automatically disabling options 
after a certain version.

Tests:
Deprecated and alias options can be tested by adding entries to tables 
in new tests:
   VMAliasOptions.java
   VMDeprecatedOptions.java

The new tests subsume these existing tests:
   test/gc/startup_warnings/TestDefaultMaxRAMFraction.java
   test/gc/startup_warnings/TestNoParNew.java

Tests run:
     jprt
     jtreg (investigating errors in resource mgmt tests running on SE 
embedded that seems unrelated to these changes.)

Thanks,

  - Derek

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150717/f46ea2e7/attachment.htm>

From jon.masamitsu at oracle.com  Fri Jul 17 19:57:33 2015
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Fri, 17 Jul 2015 12:57:33 -0700
Subject: RFR (M): 8004687: G1: Parallelize object self-forwarding and
	scanning during an evacuation failure
In-Reply-To: <1437039612.2361.27.camel@oracle.com>
References: <1437039612.2361.27.camel@oracle.com>
Message-ID: <55A95E2D.1070500@oracle.com>

Thomas,

Changes look good.

Would it work to change

OopAndMarkOop
to
OopAndOldMarkOop

and change the constructor to

866     OopAndMarkOop(oop obj) : _o(obj), _m(markOop(obj))

so that OopAndOldMarkOop specifically saves the oop and it's
markword rather than just an oop and (not necessarily related)
markword?

An enhancement to be considered now or later. You have
OopAndMarkOop now.  Could you pass an OopAndMarkOop
around instead of  an oop and a markword pair.  For
example, starting with

201 oop copy_to_survivor_space(InCSetState const state, oop const obj, 
markOop const old_mark);

substitute passing an OopAndMarkOop in place of the obj and old_mark.

Reviewed.

Jon

On 7/16/2015 2:40 AM, Thomas Schatzl wrote:
> (resend with CR number in the subject)
> Hi all,
>
>    can I have reviews for the following change from a student (Walter
> Gugenberger; he has signed the OCA) who worked on this issue last
> semester?
>
> The change parallelizes object self-forwarding and scanning during an
> evacuation failure by:
> - reuse the entire existing evacuation copy closure and work queue
> mechanism in case of evacuation failure to store the work items.
> This allows evacuation failure to automatically benefit from work
> stealing etc.
> - remove the dedicate evacuation failure handling closure because it is
> not necessary any more.
>    - there is one subtle change in behavior: the old evacuation failure
> always had a NULL ReferenceProcessor, while now the standard reference
> processor for references is applied. Since the NULL ReferenceProcessor
> kept all references alive, now potentially less references will be kept
> alive after an evacuation failure. I do not see a problem here.
> - implementing per-thread preserved mark buffers
>
> As for actual performance improvements, there is none: the main problem
> is that in case of evacuation failure the code will simply serialize on
> the FreeList_lock when trying to get a new allocation region instead of
> the EvacFailureStack_lock.
>
> Evacuation failure and the early-exit for evac failure as suggested in
> the CR will only be really effective when the FreeList_lock can be
> avoided. This will be added in a follow-up change.
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8004687
>
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8004687/webrev/
>
> Testing:
> jprt, lots of internal evacuation failure testing, aurora runs with vm.quick/vm.gc nightly tests etc.
>
> Thanks,
>    THomas
>
>
>
>
>



From david.holmes at oracle.com  Mon Jul 20 02:09:21 2015
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 20 Jul 2015 12:09:21 +1000
Subject: Runtime command-line deprecation (Was Re: RFR: 8066821(S) Enhance
	command line processing to manage deprecating and obsoleting -XX
	command line arguments
In-Reply-To: <55A90337.7030909@oracle.com>
References: <54B44661.8070007@oracle.com> <54B84E91.1050708@oracle.com>
	<F68EBF6B-38A5-4461-9ADD-41BB198FF943@oracle.com>
	<54D15A0B.6030501@oracle.com>
	<87B07B03-3423-4738-9617-0C7B72A74A6E@oracle.com>
	<559AEC80.3010907@oracle.com> <55A8392C.3020404@oracle.com>
	<55A843A5.1080108@oracle.com> <55A90337.7030909@oracle.com>
Message-ID: <55AC5851.1040001@oracle.com>

On 17/07/2015 11:29 PM, Daniel D. Daugherty wrote:
> One tiny correction below...

A correction to your correction ...

> On 7/16/15 5:52 PM, David Holmes wrote:
>> Aside: in the JDK deprecation has mostly been phase one only,
>> unfortunately, with a handful of phase two. Alas no stage three.
>
> There's at least one stage 3 in the JDK:
>
> java.lang.Thread.suspend() throws UnsupportedOperationException
> since JDK8-B96.

I don't consider that a stage 3 - stage 3 would mean Thread.suspend 
doesn't even exist in the API.

Cheers,
David

> https://bugs.openjdk.java.net/browse/JDK-7059085
>
> http://hg.openjdk.java.net/jdk9/dev/jdk/annotate/669be1677ab7/src/share/classes/java/lang/Thread.java#l848
>
>
> Dan
>
>
>>
>> Cheers,
>> David
>> ------
>>
>>>>> The way I understand that we handle private -XX options today is a
>>>>> 2-step removal: (obsolete_jvm_flags - where the
>>>>> release number is in a table and could be undefined)
>>>>>
>>>>> Using your helpful taxonomy
>>>>> fromhttps://bugs.openjdk.java.net/browse/JDK-806682:
>>>>>
>>>>> Today: private -XX options use 2-step removal (obsolete_jvm_flags)
>>>>>      release 1: Deprecate & Obsolete - warn about the option but do
>>>>> nothing with it (we can remove all code that supports it)
>>>>>      release 2: Dead - unrecognized
>>>>>    - the point of the 2-step is to give customers time to modify any
>>>>> scripts they use
>>>>>
>>>>> I believe we have very rarely removed External flags - since
>>>>> customers, licensees, etc. may expect them.
>>>>>
>>>>> Mini-proposal:
>>>>> 1) I would recommend that we add a future set of changes to add
>>>>> consistent handling for the External flags -
>>>>> so that they would follow a three-step removal:
>>>>>      release 1: Deprecate & Handle - warn and keep supporting
>>>>>      release 2: Deprecate & Obsolete - warn and do nothing
>>>>>      release 3: Dead - unrecognized
>>>>>
>>>>> 2) For the Internal flags - I think it would be kindest to customers
>>>>> and not a huge amount of additional work if
>>>>> we were to follow the Private model of using a 2 step.
>>>>>
>>>>> 3) leave the Private flags with the current 2-step removal
>>>>
>>>> Yes, this reflects our current model.
>>>>> 4) add support for aliasing - for any of them
>>>>>      So that if you are doing aliasing, you would follow the model
>>>>> you are adding
>>>>>       release 1: Deprecated & Handled - i.e. warn and still support
>>>>> (and set the information for the new alias)
>>>>>       release 2: Dead - unrecognized
>>>>>
>>>>> 5) Could we possibly take the two flags that followed a different
>>>>> model already, i.e. moving to
>>>>> Deprecated & Handled and handle those as mistakes rather than part of
>>>>> a new general model?
>>>>
>>>> So this is my question which is around the code review in question.
>>>> Note, Derek, can you resend the RFR to hotspot-dev at openjdk.java.net so
>>>> it gets to all of us (even the compiler team may want to share in the
>>>> mechanism).
>>>>
>>>> Why are UseParNewGC and MaxGCMinorPauseMillis deprecated and handled
>>>> and not deprecated and obsolete?  I don't actually see why have the
>>>> distinction here?
>>>>
>>>> Did these flags follow the deprecation procedure below?  Why not just
>>>> make them obsolete, why have this other mechanism?  I may be asking
>>>> the same question as Karen.
>>>>
>>>> I think the aliased flags could have a deprecate and handle model
>>>> (where handle == alias) in which case you only need one table for the
>>>> aliased flags, and you need to keep them in globals.hpp so they're
>>>> parsed properly.
>>>
>>> The real reason I wrote the code that way is because I was supporting
>>> the current policy (classic "mechanism not policy" checkin). I didn't
>>> (and don't) have the history on how we got there. But after thinking
>>> about it (see top post), I think there will be some cases where we'll
>>> have handle options instead of ignoring them and vice verse. And the
>>> final decision may come down to engineering, or customer, or partner
>>> concerns, or management whimsy, or whatever the current fad is. We're
>>> talking about code changes over multiple years here :-)
>>>>> Or do you think we will see more cases other than aliasing in which
>>>>> we would want to
>>>>>      release 1: Deprecate & Handle and then release 2: Dead
>>>>>      rather than release 1: Deprecate & Obsolete and then 2: Dead
>>>>>      or rather than a 3 step like the External option proposal above?
>>>>
>>>> I think for the aliased flags you want the first option here
>>>> (deprecate&handle), right?  But that's only because you need to keep
>>>> the option in globals.hpp so it parses correctly, otherwise the second
>>>> option (deprecate&obsolete) would be the preference?
>>>>
>>>> The options in the GC that are being deprecated and handled have had
>>>> warnings about them for a while, so making them obsolete doesn't feel
>>>> too soon.
>>>
>>> Yes, but just went through the approval process.
>>>>
>>>> Also, I agree with Kim's comment below that your comment lines are too
>>>> long.  My fonts are too big to have windows this wide.
>>>>
>>>> thanks,
>>>> Coleen
>>>>> thanks,
>>>>> Karen
>>>>>
>>>>> p.s. Note that all of the deprecation needs to
>>>>> 1) work with licensee engineering to ensure we give licensee's a
>>>>> head's up and get feedback
>>>>> 2) file a change control request
>>>>>
>>>>> - we try to do these both as bulk requests to reduce the processing
>>>>> overhead.
>>>>>
>>>>> p.p.s. Details
>>>>>
>>>>> 1.  Do the warnings print today or are they silent? Just want to make
>>>>> sure we are conscious of how
>>>>> those are handled if any of this moves to the new unified logging
>>>>> mechanism for which the new default
>>>>> for "warning" level is to print.
>>> The deprecation messages go through warning(), which is controlled by
>>> PrintWarnings (defaults to true). This is how the obsolete flag warnings
>>> are printed. The new deprecation mechanism replaces a mishmash of calls
>>> to jio_fprintf() and warning(), with a variety of different ways of
>>> saying the same thing.
>>>>> 2. "will likely be removed in a future release"? If we have already
>>>>> set the release it will be removed - is this a bit
>>>>> vague or did I not read closely enough?
>>> That text came from some of the deprecated GC options. If removal has
>>> been scheduled, we could say something more definite, or even the exact
>>> release. We don't print the exact "death" release for obsolete options
>>> currently though.
>>>>>
>>>>> On Feb 3, 2015, at 6:30 PM, Derek White wrote:
>>>>>
>>>>>> Request for review (again):
>>>>>>
>>>>>> - Updated with Kim's suggestion.
>>>>>> - Stopped double-printing warnings in some cases.
>>>>>> - Initial caps on warning() messages.
>>>>>>
>>>>>> Webrev:http://cr.openjdk.java.net/~drwhite/8066821/webrev.04/
>>>>>> CR:https://bugs.openjdk.java.net/browse/JDK-8066821
>>>>>> Tests:     jtreg, jprt
>>>>>>
>>>>>> Thanks for looking!
>>>>>>
>>>>>> - Derek
>>>>>>
>>>>>> On 1/28/15 3:47 PM, Kim Barrett wrote:
>>>>>>
>>> ...
>


From kim.barrett at oracle.com  Mon Jul 20 06:27:19 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Mon, 20 Jul 2015 02:27:19 -0400
Subject: RFR: 8130931: refactor CardTableModRefBS[ForCTRS]
In-Reply-To: <1436777224.2275.18.camel@oracle.com>
References: <BF2A976B-4C2D-4AC9-9627-181D1B866735@oracle.com>
	<1436777224.2275.18.camel@oracle.com>
Message-ID: <94C2AED9-51B2-4452-B38E-C778C93D0FB6@oracle.com>

On Jul 13, 2015, at 4:47 AM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
> 
> On Fri, 2015-07-10 at 20:23 -0400, Kim Barrett wrote:
>> Please review this refactoring of CardTableModRefBS and
>> CardTableModRefBSForCTRS.  The changes are in two stages, with
>> separate webrevs to make it easier to review.
>> 
>> The first stage eliminates some friend declarations from
>> CardTableModRefBS.
>> 
>> - BytecodeInterpreter doesn't need to be a friend.
>> - CheckForUnmarkedOops changed to use public byte_for_const().
> 
> Actually the whole member _unmarked_card can be removed.

Good point. Here?s the diff for that change.  I can produce a new webrev if needed.

diff -r 53eb5948d161 src/share/vm/gc/parallel/cardTableExtension.cpp
--- a/src/share/vm/gc/parallel/cardTableExtension.cpp	Mon Jul 20 02:03:42 2015 -0400
+++ b/src/share/vm/gc/parallel/cardTableExtension.cpp	Mon Jul 20 02:10:34 2015 -0400
@@ -40,7 +40,6 @@
   PSYoungGen*         _young_gen;
   CardTableExtension* _card_table;
   HeapWord*           _unmarked_addr;
-  const jbyte*        _unmarked_card;
 
  protected:
   template <class T> void do_oop_work(T* p) {
@@ -50,7 +49,6 @@
       // Don't overwrite the first missing card mark
       if (_unmarked_addr == NULL) {
         _unmarked_addr = (HeapWord*)p;
-        _unmarked_card = _card_table->byte_for_const(p);
       }
     }
   }

> 
>> - SharkBuilder changed to use public dirty_card_val().
>> - GuaranteeNotModClosure does not exist.
>> - CardTableRS changed to be a friend of CardTableModRefBSForCTRS. 
>>  Changed some references to static members of CardTableModRefBS
>>  to work around bugs in clang and Visual Studio [1].
>> 
>> Only the VMStructs friend declaration is retained.
>> 
>> The second stage moves some functionality that is specific to or only
>> used with the ForCTRS class from the base class to the ForCTRS
>> subclass. It also moves the ForCTRS class into its own files, separate
>> from the base class.
> 
> I would prefer if the parCardTableModRefBS.cpp file would be removed,
> its contents moved to the new CardTableModRefBSForCTRS.cpp as it only
> contains CardTableModRefBSForCTRS methods (and both seem only used for
> CMS).
> Iow, is there some reason to keep this file, or is this part of the
> "left for later" task?

Yes, this is part of the ?left for later? task.  I don?t want to move those functions
from one file to another, only to later have them moved (more or less) back.

> 
>> We could further split below CardTableModRefBSForCTRS, with support
>> for parallel card scanning (used only by CMS) being in another class.
>> That split already partially exists in the source code structure, with
>> some relevant functions being defined in CMS-specific
>> parCardTableModRefBS.cpp. That restructuring is being left for later.
> 
> Looks good otherwise. Thanks for the cleanup.

Thanks for looking at it.



From thomas.schatzl at oracle.com  Mon Jul 20 08:32:51 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 20 Jul 2015 10:32:51 +0200
Subject: RFR: 8130931: refactor CardTableModRefBS[ForCTRS]
In-Reply-To: <94C2AED9-51B2-4452-B38E-C778C93D0FB6@oracle.com>
References: <BF2A976B-4C2D-4AC9-9627-181D1B866735@oracle.com>
	<1436777224.2275.18.camel@oracle.com>
	<94C2AED9-51B2-4452-B38E-C778C93D0FB6@oracle.com>
Message-ID: <1437381171.2272.10.camel@oracle.com>

Hi,

On Mon, 2015-07-20 at 02:27 -0400, Kim Barrett wrote:
> On Jul 13, 2015, at 4:47 AM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
> > 
> > On Fri, 2015-07-10 at 20:23 -0400, Kim Barrett wrote:
> >> Please review this refactoring of CardTableModRefBS and
> >> CardTableModRefBSForCTRS.  The changes are in two stages, with
> >> separate webrevs to make it easier to review.
> >> 
> >> The first stage eliminates some friend declarations from
> >> CardTableModRefBS.
> >> 
> >> - BytecodeInterpreter doesn't need to be a friend.
> >> - CheckForUnmarkedOops changed to use public byte_for_const().
> > 
> > Actually the whole member _unmarked_card can be removed.
> 
> Good point. Here?s the diff for that change.  I can produce a new webrev if needed.

 Please do. You do need a second reviewer, not sure if he will want to
dig through all the thread emails to get every bit of the change...

[...]

> > I would prefer if the parCardTableModRefBS.cpp file would be removed,
> > its contents moved to the new CardTableModRefBSForCTRS.cpp as it only
> > contains CardTableModRefBSForCTRS methods (and both seem only used for
> > CMS).
> > Iow, is there some reason to keep this file, or is this part of the
> > "left for later" task?
> 
> Yes, this is part of the ?left for later? task.  I don?t want to move those functions
> from one file to another, only to later have them moved (more or less) back.

Okay.

Thanks,
  Thomas




From thomas.schatzl at oracle.com  Mon Jul 20 09:04:08 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 20 Jul 2015 11:04:08 +0200
Subject: FW: RFR(S): 8130434: [TESTBUG] Harden
	TestLargePageUseForAuxMemory for more page size combinations.
In-Reply-To: <4295855A5C1DE049A61835A1887419CC2D006C48@DEWDFEMB12A.global.corp.sap>
References: <4295855A5C1DE049A61835A1887419CC2D006C48@DEWDFEMB12A.global.corp.sap>
Message-ID: <1437383048.2272.17.camel@oracle.com>

Hi Goetz,

  sorry for taking a while to pick this up...

On Fri, 2015-07-17 at 09:52 +0000, Lindenmaier, Goetz wrote:
> Hi,
>  
> could I please get a review for this change?  I also please need a
> sponsor.

I can sponsor the change.

> I posted to hotspot-dev before, gc-dev is probably better.
>
> This change is crafted on top of ?8079208:
> gc/g1/TestLargePageUseForAuxMemory.java fails due to not considering
> page allocation granularity for setup?.
>  
> From: Lindenmaier, Goetz 
> Sent: Freitag, 10. Juli 2015 09:03
> To: hotspot-dev at openjdk.java.net
> Subject: RFR(S): 8130434: [TESTBUG] Harden
> TestLargePageUseForAuxMemory for more page size combinations.
>  
> Hi,
>  
> this test wants to enforce large page usage for the card table etc.
> It fails for certain page sizes we see on ppc, linux and aix.
>  
> On some, large page size == page size holds and this test fails: 
> "To test we would require to use an invalid heap size (assert failed:
> 33554432 > 33554432)" 
> Fix: Just skip the test in this case as it's pointless. 

Okay, I can understand that change.

> We have a machine with page size = 64K and large page size 4M. 
> In this case heapSizeDiffForBitmap was too small to reduce the heap
> size. 
> Due to the big value for the normal page size heap alignment is >
> largePageSize 
> and the heap size was aligned up to the original value. 
> Error: "'G1 'Prev Bitmap': pg_sz=65536' missing from stdout/stderr" 
> Fix: consider heap alignement for heapSizeDiffForBitmap.

I am possibly missing something here:

So page size is 64k, large page size 4M.

In this case, heapSizeForBitmapUsingLargePages = 4M * 64 = 256M, and
heapSizeDiffForBitmap is (assuming allocGranularity = page size = 64k), 

  max(4M (=64k * 64), 1M, 4M)

= 4M, which does not seem to be an issue?

Further, the use of lcm indicates that the actual reason is that in some
of your setups, small-page-size * card size > largePageSize? It is
possible, so this is just another try to make the test more robust?

The change seems good to me.

Thanks,
  Thomas




From goetz.lindenmaier at sap.com  Mon Jul 20 09:27:18 2015
From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz)
Date: Mon, 20 Jul 2015 09:27:18 +0000
Subject: FW: RFR(S): 8130434: [TESTBUG] Harden
	TestLargePageUseForAuxMemory for more page size combinations.
In-Reply-To: <1437383048.2272.17.camel@oracle.com>
References: <4295855A5C1DE049A61835A1887419CC2D006C48@DEWDFEMB12A.global.corp.sap>
	<1437383048.2272.17.camel@oracle.com>
Message-ID: <4295855A5C1DE049A61835A1887419CC2D00824B@DEWDFEMB12A.global.corp.sap>

Hi Thomas,

Thanks for picking this up!
> I can sponsor the change.
That's great, thank you.

> > We have a machine with page size = 64K and large page size 4M. 
> > In this case heapSizeDiffForBitmap was too small to reduce the heap
> > size. 
> > Due to the big value for the normal page size heap alignment is >
> > largePageSize 
> > and the heap size was aligned up to the original value. 
> > Error: "'G1 'Prev Bitmap': pg_sz=65536' missing from stdout/stderr" 
> > Fix: consider heap alignement for heapSizeDiffForBitmap.
>
> I am possibly missing something here:
>
> So page size is 64k, large page size 4M.
>
> In this case, heapSizeForBitmapUsingLargePages = 4M * 64 = 256M, and
> heapSizeDiffForBitmap is (assuming allocGranularity = page size = 64k), 
>
>   max(4M (=64k * 64), 1M, 4M)
>
> = 4M, which does not seem to be an issue?
>
> Further, the use of lcm indicates that the actual reason is that in some
> of your setups, small-page-size * card size > largePageSize? It is
> possible, so this is just another try to make the test more robust?

To find out the heap alignment, I must do the same as is done in the VM.
The lcm stems from collectorPolicy.cpp:181, CollectorPolicy::compute_heap_alignment()
The VM runs with heap alignment 512*64K = 32M.

You are correct, the old test used heapSizeDiffForBitmap = 4M.
So test case6 does: heapSizeForBitmapUsingLargePages - heapSizeDiffForBitmap is 252M.  Because of 
the large heap alignment, this was rounded up by the VM to 256 again, failing the test.  Actually, 
it falls back to test case4.

After my fix, 
heapSizeDiffForBitmap = 32M.
heapSizeForBitmapUsingLargePages - heapSizeDiffForBitmap is 224M
and the VM comes up with that heap size, passing the test.

Thanks and best regards,
  Goetz

From david.holmes at oracle.com  Mon Jul 20 09:29:41 2015
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 20 Jul 2015 19:29:41 +1000
Subject: RFR: 8066821(S) Enhance command line processing to manage
	deprecating and obsoleting -XX command line arguments
In-Reply-To: <55A93BB3.4040402@oracle.com>
References: <54B44661.8070007@oracle.com> <54B84E91.1050708@oracle.com>
	<F68EBF6B-38A5-4461-9ADD-41BB198FF943@oracle.com>
	<54D15A0B.6030501@oracle.com> <55A93BB3.4040402@oracle.com>
Message-ID: <55ACBF85.9080404@oracle.com>

Hi Derek,

Sorry but I'm finding this a bit confused and inconsistent. Comments below.

On 18/07/2015 3:30 AM, Derek White wrote:
> Request for review:
>
> [This updated webrev is being sent to wider audience, and has been
> merged with Gerard's option constraints check-in. Also factored out
> -XX:+AggressiveHeap processing into it's own chapter. I mean function :-)]
>
> http://cr.openjdk.java.net/~drwhite/8066821/webrev.06/
> https://bugs.openjdk.java.net/browse/JDK-8066821

argument.cpp:

  258  *               been not scheduled).

-> not been scheduled

  260  *     OBSOLETE: An option may be removed (and deleted from 
globals.hpp), but still be accepted on the command
  261  *               line. A warning is printed to let the user know 
that support may be removed in the future.


Isn't this the stronger case that support has already been removed (the 
option does nothing) and it will be removed at the next major release. 
At the start of a major release we should be able to delete all entries 
from the obsolete table - else it wasn't obsolete but deprecated.

Not sure "obsolete" is the right term here - obsolete things still 
function. For us an obsolete option does nothing (it exists only in the 
obsolete table).

  276  * Tests:  Aliases are tested in VMAliasOptions.java.
  277  *         Deprecated options are tested in VMDeprecatedOptions.java.
  278  *         Obsolete options are tested in various files.

We don't normally document this kind of thing in the source.

  281 // Obsolete or deprecated -XX flag.

I can tell this is going to get confusing already.

  284   JDK_Version obsoleted_in; // When the warning started (obsolete 
or deprecated).

But there is a difference: deprecated == warn but still process; 
obsolete == warn and ignore.

  288 // When a flag is eliminated, it can be added to this list in order to
  289 // continue accepting this flag on the command-line, while issuing 
a warning
  290 // and ignoring the value.  Once the JDK version reaches the 
'accept_until'
  291 // limit, we flatly refuse to admit the existence of the flag.
  292 static SpecialFlag const obsolete_jvm_flags[] = {

When a flag is eliminated it is gone from this table. As soon as the 
accept_until version is the current version we wipe the table of all 
such entries. This should be one of the first things done in a new release.

320 // When a flag is deprecated, it can be added to this list in order 
to issuing a warning when the flag is used.
  321 // Once the JDK version reaches the 'accept_until' limit, we 
flatly refuse to admit the existence of the flag.
  322 static SpecialFlag const deprecated_jvm_flags[] = {

A deprecated flag should be obsoleted before it reaches the accept_until 
limit. Which suggests that when we start a new version we wipe the 
obsoleted table and move the deprecated options into it.


776     case 1: {
  777       if (warn) {
  778         char version[256];
  779         since.to_string(version, sizeof(version));
  780         if (real_name != arg) {
  781           warning("Option %s was deprecated in version %s and will 
likely be removed in a future release. Use option %s instead.",
  782                   arg, version, real_name);
  783         } else {
  784           warning("Option %s was deprecated in version %s and will 
likely be removed in a future release.",
  785                   arg, version);
  786         }

This isn't distinguishing between deprecated and obsoleted ???

997       warning("Ignoring option %s; support was removed in %s", 
stripped_argname, version);

You have changed a handful of warnings so they start with a capital 
letter, and have changed the associated tests. But this seems a 
pointless "convention" because we have dozens of warnings that don't 
start with a capital.

Thanks,
David
-----

> This webrev adds support for handling "/deprecated/" -XX options
> (options that still *do* something but are planned for removal) and
> "/alias/" options (alternate names for other -XX options) by simply
> adding entries to the deprecated_jvm_flags and/or aliased_jvm_flags
> tables. This follows the example of the existing obsolete_jvm_flags table.
>
> This replaces a lot of ad-hoc and occasionally wrong code in
> arguments.cpp as well as supporting automatically disabling options
> after a certain version.
>
> Tests:
> Deprecated and alias options can be tested by adding entries to tables
> in new tests:
>    VMAliasOptions.java
>    VMDeprecatedOptions.java
>
> The new tests subsume these existing tests:
>    test/gc/startup_warnings/TestDefaultMaxRAMFraction.java
>    test/gc/startup_warnings/TestNoParNew.java
>
> Tests run:
>      jprt
>      jtreg (investigating errors in resource mgmt tests running on SE
> embedded that seems unrelated to these changes.)
>
> Thanks,
>
>   - Derek
>


From thomas.schatzl at oracle.com  Mon Jul 20 10:10:31 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 20 Jul 2015 12:10:31 +0200
Subject: FW: RFR(S): 8130434: [TESTBUG] Harden
	TestLargePageUseForAuxMemory for more page size combinations.
In-Reply-To: <4295855A5C1DE049A61835A1887419CC2D00824B@DEWDFEMB12A.global.corp.sap>
References: <4295855A5C1DE049A61835A1887419CC2D006C48@DEWDFEMB12A.global.corp.sap>
	<1437383048.2272.17.camel@oracle.com>
	<4295855A5C1DE049A61835A1887419CC2D00824B@DEWDFEMB12A.global.corp.sap>
Message-ID: <1437387031.2272.20.camel@oracle.com>

Hi Goetz,

On Mon, 2015-07-20 at 09:26 +0000, Lindenmaier, Goetz wrote:
Hi Thomas,
> 
> Thanks for picking this up!
> > I can sponsor the change.
> That's great, thank you.
> 
> > > We have a machine with page size = 64K and large page size 4M. 
> > > In this case heapSizeDiffForBitmap was too small to reduce the
> > > heap
size. 
> > > Due to the big value for the normal page size heap alignment is >
> > > largePageSize 
> > > and the heap size was aligned up to the original value. 
> > > Error: "'G1 'Prev Bitmap': pg_sz=65536' missing from
> > > stdout/stderr" 
> > > Fix: consider heap alignement for heapSizeDiffForBitmap.
> >
> > I am possibly missing something here:
> >
> > So page size is 64k, large page size 4M.
> >
> > In this case, heapSizeForBitmapUsingLargePages = 4M * 64 = 256M,
> > and
> > heapSizeDiffForBitmap is (assuming allocGranularity = page
> > size = 64k), 
> >
> >   max(4M (=64k * 64), 1M, 4M)
> >
> > = 4M, which does not seem to be an issue?
> >
> > Further, the use of lcm indicates that the actual reason is that in
> > some
of your setups, small-page-size * card size > largePageSize? It
> > is
possible, so this is just another try to make the test more
> > robust?
> 
> To find out the heap alignment, I must do the same as is done in the
VM.
> The lcm stems from collectorPolicy.cpp:181,
CollectorPolicy::compute_heap_alignment()
> The VM runs with heap alignment 512*64K = 32M.
> 
> You are correct, the old test used heapSizeDiffForBitmap = 4M.
> So test case6 does: heapSizeForBitmapUsingLargePages -
heapSizeDiffForBitmap is 252M.  Because of 
> the large heap alignment, this was rounded up by the VM to 256 again,
failing the test.  Actually, 
> it falls back to test case4.
> 
> After my fix, 
> heapSizeDiffForBitmap = 32M.
> heapSizeForBitmapUsingLargePages - heapSizeDiffForBitmap is 224M
> and the VM comes up with that heap size, passing the test.
> 
  I overlooked the card table alignment requirement. With 64k small page
size this is 32M...

Looks good.

I will wait a bit until pushing, maybe someone else has some time
looking at this.

Thanks,
  Thomas




From goetz.lindenmaier at sap.com  Mon Jul 20 10:15:50 2015
From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz)
Date: Mon, 20 Jul 2015 10:15:50 +0000
Subject: FW: RFR(S): 8130434: [TESTBUG] Harden
	TestLargePageUseForAuxMemory for more page size combinations.
In-Reply-To: <1437387031.2272.20.camel@oracle.com>
References: <4295855A5C1DE049A61835A1887419CC2D006C48@DEWDFEMB12A.global.corp.sap>
	<1437383048.2272.17.camel@oracle.com>
	<4295855A5C1DE049A61835A1887419CC2D00824B@DEWDFEMB12A.global.corp.sap>
	<1437387031.2272.20.camel@oracle.com>
Message-ID: <4295855A5C1DE049A61835A1887419CC2D0082AE@DEWDFEMB12A.global.corp.sap>

Hi Thomas,

thanks a lot! 
> I will wait a bit until pushing, maybe someone else has some time
> looking at this.
That's ok.

Best regards,
  Goetz.


-----Original Message-----
From: Thomas Schatzl [mailto:thomas.schatzl at oracle.com] 
Sent: Montag, 20. Juli 2015 12:11
To: Lindenmaier, Goetz
Cc: hotspot-gc-dev at openjdk.java.net
Subject: Re: FW: RFR(S): 8130434: [TESTBUG] Harden TestLargePageUseForAuxMemory for more page size combinations.

Hi Goetz,

On Mon, 2015-07-20 at 09:26 +0000, Lindenmaier, Goetz wrote:
Hi Thomas,
> 
> Thanks for picking this up!
> > I can sponsor the change.
> That's great, thank you.
> 
> > > We have a machine with page size = 64K and large page size 4M. 
> > > In this case heapSizeDiffForBitmap was too small to reduce the
> > > heap
size. 
> > > Due to the big value for the normal page size heap alignment is >
> > > largePageSize 
> > > and the heap size was aligned up to the original value. 
> > > Error: "'G1 'Prev Bitmap': pg_sz=65536' missing from
> > > stdout/stderr" 
> > > Fix: consider heap alignement for heapSizeDiffForBitmap.
> >
> > I am possibly missing something here:
> >
> > So page size is 64k, large page size 4M.
> >
> > In this case, heapSizeForBitmapUsingLargePages = 4M * 64 = 256M,
> > and
> > heapSizeDiffForBitmap is (assuming allocGranularity = page
> > size = 64k), 
> >
> >   max(4M (=64k * 64), 1M, 4M)
> >
> > = 4M, which does not seem to be an issue?
> >
> > Further, the use of lcm indicates that the actual reason is that in
> > some
of your setups, small-page-size * card size > largePageSize? It
> > is
possible, so this is just another try to make the test more
> > robust?
> 
> To find out the heap alignment, I must do the same as is done in the
VM.
> The lcm stems from collectorPolicy.cpp:181,
CollectorPolicy::compute_heap_alignment()
> The VM runs with heap alignment 512*64K = 32M.
> 
> You are correct, the old test used heapSizeDiffForBitmap = 4M.
> So test case6 does: heapSizeForBitmapUsingLargePages -
heapSizeDiffForBitmap is 252M.  Because of 
> the large heap alignment, this was rounded up by the VM to 256 again,
failing the test.  Actually, 
> it falls back to test case4.
> 
> After my fix, 
> heapSizeDiffForBitmap = 32M.
> heapSizeForBitmapUsingLargePages - heapSizeDiffForBitmap is 224M
> and the VM comes up with that heap size, passing the test.
> 
  I overlooked the card table alignment requirement. With 64k small page
size this is 32M...

Looks good.

I will wait a bit until pushing, maybe someone else has some time
looking at this.

Thanks,
  Thomas



From adinn at redhat.com  Mon Jul 20 13:22:28 2015
From: adinn at redhat.com (Andrew Dinn)
Date: Mon, 20 Jul 2015 14:22:28 +0100
Subject: Query regarding ordering in G1 post-write barrier
Message-ID: <55ACF614.5020103@redhat.com>

$SUBJECT is in relation to the following code in method
GraphKit::g1_mark_card()

  . . .
  __ storeCM(__ ctrl(), card_adr, zero, oop_store, oop_alias_idx,
card_bt, Compile::AliasIdxRaw);

  //  Now do the queue work
  __ if_then(index, BoolTest::ne, zeroX); {

    Node* next_index = _gvn.transform(new SubXNode(index, __
ConX(sizeof(intptr_t))));
    Node* log_addr = __ AddP(no_base, buffer, next_index);

    // Order, see storeCM.
    __ store(__ ctrl(), log_addr, card_adr, T_ADDRESS,
Compile::AliasIdxRaw, MemNode::unordered);
    __ store(__ ctrl(), index_adr, next_index, TypeX_X->basic_type(),
Compile::AliasIdxRaw, MemNode::unordered);

  } __ else_(); {
    __ make_leaf_call(tf, CAST_FROM_FN_PTR(address,
SharedRuntime::g1_wb_post), "g1_wb_post", card_adr, __ thread());
  } __ end_if();
  . . .

The 3 stores StoreCM -> StoreP -> StoreX which mark the card and then
push the card address into the dirty queue appear to end up being
emitted in that order -- I assume by virtue of the memory links between
them (output of StoreCM is the mem input of StoreP, output of StoreP is
the mem input of StoreX). That order makes sense if the queue were to be
observed concurrently i.e. mark card to ensure write is flagged before
it is performed, write value, then decrement index to make value write
visible. So, that's ok on x86 where TCO means this is also the order of
visibility.

The StoreCM is 'ordered' i.e. it is flagged with mem ordering type =
mo_release. However, the latter pair of instructions are 'unordered'. I
looked at the G1 code which processes dirty queues and could not make
head nor tail (oops, apologies for any undercurrent of a pun) of when it
gets run.

So, the question is this: does dirty queue processing only happen in GC
threads when mutators cannot be writing them? or is there a need on
non-TCO architectures to maintain some sort of consistency in the queue
update via by serializing these writes?

Hmm, ok that seems to be two questions. Let's make that a starter for 10
and a bonus for whoever buzzes fastest.

regards,


Andrew Dinn
-----------
Senior Principal Software Engineer
Red Hat UK Ltd
Registered in UK and Wales under Company Registration No. 3798903
Directors: Michael Cunningham (USA), Matt Parson (USA), Charlie Peters
(USA), Michael O'Neill (Ireland)


From thomas.schatzl at oracle.com  Mon Jul 20 13:44:47 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 20 Jul 2015 15:44:47 +0200
Subject: Query regarding ordering in G1 post-write barrier
In-Reply-To: <55ACF614.5020103@redhat.com>
References: <55ACF614.5020103@redhat.com>
Message-ID: <1437399887.2272.76.camel@oracle.com>

Hi,

On Mon, 2015-07-20 at 14:22 +0100, Andrew Dinn wrote:
> $SUBJECT is in relation to the following code in method
> GraphKit::g1_mark_card()
[...]
> The StoreCM is 'ordered' i.e. it is flagged with mem ordering type =
> mo_release. However, the latter pair of instructions are 'unordered'. I
> looked at the G1 code which processes dirty queues and could not make
> head nor tail (oops, apologies for any undercurrent of a pun) of when it
> gets run.
>
> So, the question is this: does dirty queue processing only happen in GC
> threads when mutators cannot be writing them? or is there a need on
> non-TCO architectures to maintain some sort of consistency in the queue
> update via by serializing these writes?

Mutator threads and refinement threads are running concurrently. While
GC threads are working on the queues, mutators never run.

While refinement threads may set and clear the mark on the card table
concurrently because they were processing that card concurrently, there
is afaik sufficient synchronization between the card mark and the
reference write on that in the code (in
HeapRegion::oops_on_card_seq_iterate_careful()). The refinement threads
do not modify or assume any value of the buffer's members, so there does
not seem to be a need in the write barrier for further synchronization.

Refinement threads and mutator threads take a lock to push and pop a new
buffer (in SharedRuntime::g1_wb_post) to and from the buffer queue,
which means that there is at least one cmpxchg/synchronization between
the thread pushing the buffer and the refinement thread popping it.

Which should be sufficient that the buffer's members are visible to all
participants of this exchange.

Note that GC threads do access the buffers, but there is a safepointing
operation between the accesses.

That should cover the eventualities. You can now tell us where I am
wrong :)

> Hmm, ok that seems to be two questions. Let's make that a starter for 10
> and a bonus for whoever buzzes fastest.

Not sure if that answers your question. So what did I win? :-)

Thanks,
  Thomas




From daniel.daugherty at oracle.com  Mon Jul 20 13:54:20 2015
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Mon, 20 Jul 2015 07:54:20 -0600
Subject: Runtime command-line deprecation (Was Re: RFR: 8066821(S) Enhance
	command line processing to manage deprecating and obsoleting -XX
	command line arguments
In-Reply-To: <55AC5851.1040001@oracle.com>
References: <54B44661.8070007@oracle.com> <54B84E91.1050708@oracle.com>
	<F68EBF6B-38A5-4461-9ADD-41BB198FF943@oracle.com>
	<54D15A0B.6030501@oracle.com>
	<87B07B03-3423-4738-9617-0C7B72A74A6E@oracle.com>
	<559AEC80.3010907@oracle.com> <55A8392C.3020404@oracle.com>
	<55A843A5.1080108@oracle.com> <55A90337.7030909@oracle.com>
	<55AC5851.1040001@oracle.com>
Message-ID: <55ACFD8C.6060907@oracle.com>

On 7/19/15 8:09 PM, David Holmes wrote:
> On 17/07/2015 11:29 PM, Daniel D. Daugherty wrote:
>> One tiny correction below...
>
> A correction to your correction ...
>
>> On 7/16/15 5:52 PM, David Holmes wrote:
>>> Aside: in the JDK deprecation has mostly been phase one only,
>>> unfortunately, with a handful of phase two. Alas no stage three.
>>
>> There's at least one stage 3 in the JDK:
>>
>> java.lang.Thread.suspend() throws UnsupportedOperationException
>> since JDK8-B96.
>
> I don't consider that a stage 3 - stage 3 would mean Thread.suspend 
> doesn't even exist in the API.

Your three stages from your original reply which you trimmed
from this reply:

 > I agree that in the worst-case there could well be a three phase process:
 >  - deprecate and handle
 >  - deprecate and ignore
 >  - remove (give error on use)

I was responding to the "give error on use" part and not so much
on the "remove" part.

Dan


>
> Cheers,
> David
>
>> https://bugs.openjdk.java.net/browse/JDK-7059085
>>
>> http://hg.openjdk.java.net/jdk9/dev/jdk/annotate/669be1677ab7/src/share/classes/java/lang/Thread.java#l848 
>>
>>
>>
>> Dan
>>
>>
>>>
>>> Cheers,
>>> David
>>> ------
>>>
>>>>>> The way I understand that we handle private -XX options today is a
>>>>>> 2-step removal: (obsolete_jvm_flags - where the
>>>>>> release number is in a table and could be undefined)
>>>>>>
>>>>>> Using your helpful taxonomy
>>>>>> fromhttps://bugs.openjdk.java.net/browse/JDK-806682:
>>>>>>
>>>>>> Today: private -XX options use 2-step removal (obsolete_jvm_flags)
>>>>>>      release 1: Deprecate & Obsolete - warn about the option but do
>>>>>> nothing with it (we can remove all code that supports it)
>>>>>>      release 2: Dead - unrecognized
>>>>>>    - the point of the 2-step is to give customers time to modify any
>>>>>> scripts they use
>>>>>>
>>>>>> I believe we have very rarely removed External flags - since
>>>>>> customers, licensees, etc. may expect them.
>>>>>>
>>>>>> Mini-proposal:
>>>>>> 1) I would recommend that we add a future set of changes to add
>>>>>> consistent handling for the External flags -
>>>>>> so that they would follow a three-step removal:
>>>>>>      release 1: Deprecate & Handle - warn and keep supporting
>>>>>>      release 2: Deprecate & Obsolete - warn and do nothing
>>>>>>      release 3: Dead - unrecognized
>>>>>>
>>>>>> 2) For the Internal flags - I think it would be kindest to customers
>>>>>> and not a huge amount of additional work if
>>>>>> we were to follow the Private model of using a 2 step.
>>>>>>
>>>>>> 3) leave the Private flags with the current 2-step removal
>>>>>
>>>>> Yes, this reflects our current model.
>>>>>> 4) add support for aliasing - for any of them
>>>>>>      So that if you are doing aliasing, you would follow the model
>>>>>> you are adding
>>>>>>       release 1: Deprecated & Handled - i.e. warn and still support
>>>>>> (and set the information for the new alias)
>>>>>>       release 2: Dead - unrecognized
>>>>>>
>>>>>> 5) Could we possibly take the two flags that followed a different
>>>>>> model already, i.e. moving to
>>>>>> Deprecated & Handled and handle those as mistakes rather than 
>>>>>> part of
>>>>>> a new general model?
>>>>>
>>>>> So this is my question which is around the code review in question.
>>>>> Note, Derek, can you resend the RFR to 
>>>>> hotspot-dev at openjdk.java.net so
>>>>> it gets to all of us (even the compiler team may want to share in the
>>>>> mechanism).
>>>>>
>>>>> Why are UseParNewGC and MaxGCMinorPauseMillis deprecated and handled
>>>>> and not deprecated and obsolete?  I don't actually see why have the
>>>>> distinction here?
>>>>>
>>>>> Did these flags follow the deprecation procedure below? Why not just
>>>>> make them obsolete, why have this other mechanism?  I may be asking
>>>>> the same question as Karen.
>>>>>
>>>>> I think the aliased flags could have a deprecate and handle model
>>>>> (where handle == alias) in which case you only need one table for the
>>>>> aliased flags, and you need to keep them in globals.hpp so they're
>>>>> parsed properly.
>>>>
>>>> The real reason I wrote the code that way is because I was supporting
>>>> the current policy (classic "mechanism not policy" checkin). I didn't
>>>> (and don't) have the history on how we got there. But after thinking
>>>> about it (see top post), I think there will be some cases where we'll
>>>> have handle options instead of ignoring them and vice verse. And the
>>>> final decision may come down to engineering, or customer, or partner
>>>> concerns, or management whimsy, or whatever the current fad is. We're
>>>> talking about code changes over multiple years here :-)
>>>>>> Or do you think we will see more cases other than aliasing in which
>>>>>> we would want to
>>>>>>      release 1: Deprecate & Handle and then release 2: Dead
>>>>>>      rather than release 1: Deprecate & Obsolete and then 2: Dead
>>>>>>      or rather than a 3 step like the External option proposal 
>>>>>> above?
>>>>>
>>>>> I think for the aliased flags you want the first option here
>>>>> (deprecate&handle), right?  But that's only because you need to keep
>>>>> the option in globals.hpp so it parses correctly, otherwise the 
>>>>> second
>>>>> option (deprecate&obsolete) would be the preference?
>>>>>
>>>>> The options in the GC that are being deprecated and handled have had
>>>>> warnings about them for a while, so making them obsolete doesn't feel
>>>>> too soon.
>>>>
>>>> Yes, but just went through the approval process.
>>>>>
>>>>> Also, I agree with Kim's comment below that your comment lines are 
>>>>> too
>>>>> long.  My fonts are too big to have windows this wide.
>>>>>
>>>>> thanks,
>>>>> Coleen
>>>>>> thanks,
>>>>>> Karen
>>>>>>
>>>>>> p.s. Note that all of the deprecation needs to
>>>>>> 1) work with licensee engineering to ensure we give licensee's a
>>>>>> head's up and get feedback
>>>>>> 2) file a change control request
>>>>>>
>>>>>> - we try to do these both as bulk requests to reduce the processing
>>>>>> overhead.
>>>>>>
>>>>>> p.p.s. Details
>>>>>>
>>>>>> 1.  Do the warnings print today or are they silent? Just want to 
>>>>>> make
>>>>>> sure we are conscious of how
>>>>>> those are handled if any of this moves to the new unified logging
>>>>>> mechanism for which the new default
>>>>>> for "warning" level is to print.
>>>> The deprecation messages go through warning(), which is controlled by
>>>> PrintWarnings (defaults to true). This is how the obsolete flag 
>>>> warnings
>>>> are printed. The new deprecation mechanism replaces a mishmash of 
>>>> calls
>>>> to jio_fprintf() and warning(), with a variety of different ways of
>>>> saying the same thing.
>>>>>> 2. "will likely be removed in a future release"? If we have already
>>>>>> set the release it will be removed - is this a bit
>>>>>> vague or did I not read closely enough?
>>>> That text came from some of the deprecated GC options. If removal has
>>>> been scheduled, we could say something more definite, or even the 
>>>> exact
>>>> release. We don't print the exact "death" release for obsolete options
>>>> currently though.
>>>>>>
>>>>>> On Feb 3, 2015, at 6:30 PM, Derek White wrote:
>>>>>>
>>>>>>> Request for review (again):
>>>>>>>
>>>>>>> - Updated with Kim's suggestion.
>>>>>>> - Stopped double-printing warnings in some cases.
>>>>>>> - Initial caps on warning() messages.
>>>>>>>
>>>>>>> Webrev:http://cr.openjdk.java.net/~drwhite/8066821/webrev.04/
>>>>>>> CR:https://bugs.openjdk.java.net/browse/JDK-8066821
>>>>>>> Tests:     jtreg, jprt
>>>>>>>
>>>>>>> Thanks for looking!
>>>>>>>
>>>>>>> - Derek
>>>>>>>
>>>>>>> On 1/28/15 3:47 PM, Kim Barrett wrote:
>>>>>>>
>>>> ...
>>
>
>



From adinn at redhat.com  Mon Jul 20 14:14:42 2015
From: adinn at redhat.com (Andrew Dinn)
Date: Mon, 20 Jul 2015 15:14:42 +0100
Subject: Query regarding ordering in G1 post-write barrier
In-Reply-To: <1437399887.2272.76.camel@oracle.com>
References: <55ACF614.5020103@redhat.com> <1437399887.2272.76.camel@oracle.com>
Message-ID: <55AD0252.3040007@redhat.com>

On 20/07/15 14:44, Thomas Schatzl wrote:
> 
> . . .
> 
> Refinement threads and mutator threads take a lock to push and pop a new
> buffer (in SharedRuntime::g1_wb_post) to and from the buffer queue,
> which means that there is at least one cmpxchg/synchronization between
> the thread pushing the buffer and the refinement thread popping it.
> 
> . . .
> 
> Note that GC threads do access the buffers, but there is a safepointing
> operation between the accesses.
> 
> That should cover the eventualities. You can now tell us where I am
> wrong :)

Thanks for the explanation. I don't doubt that you (and the code) are
right -- but I wanted to understand why.

>> Hmm, ok that seems to be two questions. Let's make that a starter for 10
>> and a bonus for whoever buzzes fastest.
> 
> Not sure if that answers your question. So what did I win? :-)

Why glory***, naturally.

regards,


Andrew Dinn
-----------
*** n.b. to find out what 'glory' means see Through The Looking Glass


From thomas.schatzl at oracle.com  Mon Jul 20 14:16:11 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 20 Jul 2015 16:16:11 +0200
Subject: RFR (M): 8004687: G1: Parallelize object self-forwarding and
	scanning during an evacuation failure
In-Reply-To: <55A95E2D.1070500@oracle.com>
References: <1437039612.2361.27.camel@oracle.com> <55A95E2D.1070500@oracle.com>
Message-ID: <1437401771.2272.90.camel@oracle.com>

Hi Jon,


On Fri, 2015-07-17 at 12:57 -0700, Jon Masamitsu wrote:
> Thomas,
> 
> Changes look good.

  thanks for the review.

> 
> Would it work to change
> 
> OopAndMarkOop
> to
> OopAndOldMarkOop
> 
> and change the constructor to
> 
> 866     OopAndMarkOop(oop obj) : _o(obj), _m(markOop(obj))
> 
> so that OopAndOldMarkOop specifically saves the oop and it's
> markword rather than just an oop and (not necessarily related)
> markword?

  the only problem I see is that we earlier (8006025) changed the code
to explicitly not re-read the markoop from the obj multiple times. The
markoop is volatile, so it prevents some optimizations which we want
here.

It may not be a huge problem or one at all in this particular case,
depending on where the OopAndOldMarkOop is instantiated, but requires
checking.

I put a webrev with pure renaming of the struct at
http://cr.openjdk.java.net/~tschatzl/8004687/webrev.1/ (diff webrev:
http://cr.openjdk.java.net/~tschatzl/8004687/webrev.0_to_1/).

I do not mind either version.

> An enhancement to be considered now or later. You have
> OopAndMarkOop now.  Could you pass an OopAndMarkOop
> around instead of  an oop and a markword pair.  For
> example, starting with
> 
> 201 oop copy_to_survivor_space(InCSetState const state, oop const obj, 
> markOop const old_mark);
> 
> substitute passing an OopAndMarkOop in place of the obj and old_mark.

Prototyping this it does not seem to be huge gain. The use and passing
of a struct is only really advantageous in a few places (in
copy_to_survivor_spaces()). In many places the existing code manipulates
the old object and old mark quite a bit, adding a "obj_and_mark" (if we
call the parameter obj_and_mark) prefix in quite a few places only.

I put that change at
http://cr.openjdk.java.net/~tschatzl/8004687/webrev.refactor/.
Particularly lines 250-265 in g1ParScanThreadState.cpp seem to be more
obfuscated than before. There is opportunity to move the entire mark
update method into OopAndOldMarkOop, but unless there is something I
omitted, I would like to defer it (and the necessary performance
checking work) to a later point.

> Reviewed.

Thanks,
  Thomas



From derek.white at oracle.com  Mon Jul 20 17:02:08 2015
From: derek.white at oracle.com (Derek White)
Date: Mon, 20 Jul 2015 13:02:08 -0400
Subject: RFR: 8066821(S) Enhance command line processing to manage
	deprecating and obsoleting -XX command line arguments
In-Reply-To: <55ACBF85.9080404@oracle.com>
References: <54B44661.8070007@oracle.com> <54B84E91.1050708@oracle.com>
	<F68EBF6B-38A5-4461-9ADD-41BB198FF943@oracle.com>
	<54D15A0B.6030501@oracle.com> <55A93BB3.4040402@oracle.com>
	<55ACBF85.9080404@oracle.com>
Message-ID: <55AD2990.8030000@oracle.com>

Hi David,

Thanks for looking this over.

On 7/20/15 5:29 AM, David Holmes wrote:
> Hi Derek,
>
> Sorry but I'm finding this a bit confused and inconsistent. Comments 
> below.
>
> On 18/07/2015 3:30 AM, Derek White wrote:
>> Request for review:
>>
>> [This updated webrev is being sent to wider audience, and has been
>> merged with Gerard's option constraints check-in. Also factored out
>> -XX:+AggressiveHeap processing into it's own chapter. I mean function 
>> :-)]
>>
>> http://cr.openjdk.java.net/~drwhite/8066821/webrev.06/
>> https://bugs.openjdk.java.net/browse/JDK-8066821
>
> argument.cpp:
>
>  258  *               been not scheduled).
>
> -> not been scheduled

OK.
>  260 *     OBSOLETE: An option may be removed (and deleted from 
> globals.hpp), but still be accepted on the command
>  261  *               line. A warning is printed to let the user know 
> that support may be removed in the future.
>
>
> Isn't this the stronger case that support has already been removed 
> (the option does nothing) and it will be removed at the next major 
> release. At the start of a major release we should be able to delete 
> all entries from the obsolete table - else it wasn't obsolete but 
> deprecated.
>
> Not sure "obsolete" is the right term here - obsolete things still 
> function. For us an obsolete option does nothing (it exists only in 
> the obsolete table).
It's not a great name, but that what the previous code called it. It's a 
"I'll pretend you didn't say that" option, like when a teenager hears an 
adult use out-of-date slang ("groovy!"). How about a "condescending" 
option :-)
>
>  276  * Tests:  Aliases are tested in VMAliasOptions.java.
>  277  *         Deprecated options are tested in 
> VMDeprecatedOptions.java.
>  278  *         Obsolete options are tested in various files.
>
> We don't normally document this kind of thing in the source.

I'm trying to head off unnecessary one-off test files for each alias and 
deprecated option. For example TestNoParNew.java and 
TestDefaultMaxRAMFraction.java. And I think that there should be one 
test file for obsolete options also (perhaps expanding 
ObsoleteFlagErrorMessage.java), instead of a bunch of separate test 
files, but that is not in this webrev.
>
>  281 // Obsolete or deprecated -XX flag.
>
> I can tell this is going to get confusing already.
>
>  284   JDK_Version obsoleted_in; // When the warning started (obsolete 
> or deprecated).
>
> But there is a difference: deprecated == warn but still process; 
> obsolete == warn and ignore.
Yes, but the SpecialFlag type is concerned with the common aspect of 
warning. The "ignore" or "process" aspects are handled by the different 
functions that look up the obsolete_jvm_flags and deprecated_jvm_flags 
arrays.
>
>  288 // When a flag is eliminated, it can be added to this list in 
> order to
>  289 // continue accepting this flag on the command-line, while 
> issuing a warning
>  290 // and ignoring the value.  Once the JDK version reaches the 
> 'accept_until'
>  291 // limit, we flatly refuse to admit the existence of the flag.
>  292 static SpecialFlag const obsolete_jvm_flags[] = {
>
> When a flag is eliminated it is gone from this table. As soon as the 
> accept_until version is the current version we wipe the table of all 
> such entries. This should be one of the first things done in a new 
> release.

I completely agree that this is a great plan. But until this April we 
still had obsolete flags listed for JDK 5! The obsolete_jvm_flags table 
did the right thing until the process caught up. In any case, this 
webrev doesn't really change the behavior of the obsolete_jvm_flags table.
>
> 320 // When a flag is deprecated, it can be added to this list in 
> order to issuing a warning when the flag is used.
>  321 // Once the JDK version reaches the 'accept_until' limit, we 
> flatly refuse to admit the existence of the flag.
>  322 static SpecialFlag const deprecated_jvm_flags[] = {
>
> A deprecated flag should be obsoleted before it reaches the 
> accept_until limit.
That's a policy that's under discussion on hotspot-runtime-dev. There 
are certainly option lifecycles that have been approved by our internal 
process that don't follow this proposed policy. The mechanism in this 
webrev was concerned about supporting the plethora of current 
lifecycles, for better or worse.
> Which suggests that when we start a new version we wipe the obsoleted 
> table and move the deprecated options into it.
I can add documentation to describe this case.

If we decide that we'll always treat a deprecated or aliased option as 
obsolete for one extra release, then it might make sense to have a 
combined option lifecycle table. But for now I like the fact that 
options in deprecated_jvm_flags should always have a real variable 
defined in globals.hpp (etc), and options in obsolete_jvm_flags should 
never have a global variable.

> 776 case 1: {
>  777       if (warn) {
>  778         char version[256];
>  779         since.to_string(version, sizeof(version));
>  780         if (real_name != arg) {
>  781           warning("Option %s was deprecated in version %s and 
> will likely be removed in a future release. Use option %s instead.",
>  782                   arg, version, real_name);
>  783         } else {
>  784           warning("Option %s was deprecated in version %s and 
> will likely be removed in a future release.",
>  785                   arg, version);
>  786         }
>
> This isn't distinguishing between deprecated and obsoleted ???

Yes, handle_aliases_and_deprecation() doesn't handle obsolete options 
(or it would have had a longer name :-) Maybe it should handle that 
case, but it would have complicated the control flow in the caller. I 
have another proposed change in the works that simplifies the caller 
quite a bit that would make the refactoring simpler.
>
> 997       warning("Ignoring option %s; support was removed in %s", 
> stripped_argname, version);
>
> You have changed a handful of warnings so they start with a capital 
> letter, and have changed the associated tests. But this seems a 
> pointless "convention" because we have dozens of warnings that don't 
> start with a capital.
The new deprecation and alias warnings replaces these various messages:
     warning("The UseParNewGC flag is deprecated and will likely be 
removed in a future release");
     warning("Using MaxGCMinorPauseMillis as minor pause goal is 
deprecated and will likely be removed in future release");
     warning("DefaultMaxRAMFraction is deprecated and will likely be 
removed in a future release. Use MaxRAMFraction instead.");
     jio_fprintf(defaultStream::error_stream(),
         "Please use -XX:MarkStackSize in place of -XX:CMSMarkStackSize 
or -XX:G1MarkStackSize in the future\n");
      jio_fprintf(defaultStream::error_stream(),
         "Please use -XX:ConcGCThreads in place of 
-XX:ParallelMarkingThreads or -XX:ParallelCMSThreads in the future\n");
      jio_fprintf(defaultStream::error_stream(),
          "Please use -XX:MarkStackSizeMax in place of 
-XX:CMSMarkStackSizeMax in the future\n");
      jio_fprintf(defaultStream::output_stream(),
           "CreateMinidumpOnCrash is replaced by CreateCoredumpOnCrash: 
CreateCoredumpOnCrash is on\n");

It made sense to have the case of the obsolete messages match the 
deprecation and aliases messages. Arguably I got carried away with 
changing the messages for UseAdaptiveSizePolicy, RequireSharedSpaces, 
and ScavengeRootsInCode warnings (but they didn't force changes to tests).

Thanks again for you comments!

  - Derek
>
> Thanks,
> David
> -----
>
>> This webrev adds support for handling "/deprecated/" -XX options
>> (options that still *do* something but are planned for removal) and
>> "/alias/" options (alternate names for other -XX options) by simply
>> adding entries to the deprecated_jvm_flags and/or aliased_jvm_flags
>> tables. This follows the example of the existing obsolete_jvm_flags 
>> table.
>>
>> This replaces a lot of ad-hoc and occasionally wrong code in
>> arguments.cpp as well as supporting automatically disabling options
>> after a certain version.
>>
>> Tests:
>> Deprecated and alias options can be tested by adding entries to tables
>> in new tests:
>>    VMAliasOptions.java
>>    VMDeprecatedOptions.java
>>
>> The new tests subsume these existing tests:
>>    test/gc/startup_warnings/TestDefaultMaxRAMFraction.java
>>    test/gc/startup_warnings/TestNoParNew.java
>>
>> Tests run:
>>      jprt
>>      jtreg (investigating errors in resource mgmt tests running on SE
>> embedded that seems unrelated to these changes.)
>>
>> Thanks,
>>
>>   - Derek
>>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150720/8c17e160/attachment.htm>

From kim.barrett at oracle.com  Mon Jul 20 21:35:08 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Mon, 20 Jul 2015 17:35:08 -0400
Subject: RFR: 8130931: refactor CardTableModRefBS[ForCTRS]
In-Reply-To: <1437381171.2272.10.camel@oracle.com>
References: <BF2A976B-4C2D-4AC9-9627-181D1B866735@oracle.com>
	<1436777224.2275.18.camel@oracle.com>
	<94C2AED9-51B2-4452-B38E-C778C93D0FB6@oracle.com>
	<1437381171.2272.10.camel@oracle.com>
Message-ID: <6503926D-8839-4210-88A7-CCF7C2CB077C@oracle.com>

On Jul 20, 2015, at 4:32 AM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
> 
> On Mon, 2015-07-20 at 02:27 -0400, Kim Barrett wrote:
>> On Jul 13, 2015, at 4:47 AM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
>>> 
>>> On Fri, 2015-07-10 at 20:23 -0400, Kim Barrett wrote:
>>>> Please review this refactoring of CardTableModRefBS and
>>>> CardTableModRefBSForCTRS.  The changes are in two stages, with
>>>> separate webrevs to make it easier to review.
>>>> 
>>>> The first stage eliminates some friend declarations from
>>>> CardTableModRefBS.
>>>> 
>>>> - BytecodeInterpreter doesn't need to be a friend.
>>>> - CheckForUnmarkedOops changed to use public byte_for_const().
>>> 
>>> Actually the whole member _unmarked_card can be removed.
>> 
>> Good point. Here?s the diff for that change.  I can produce a new webrev if needed.
> 
> Please do. You do need a second reviewer, not sure if he will want to
> dig through all the thread emails to get every bit of the change?



> 
> [...]
> 
>>> I would prefer if the parCardTableModRefBS.cpp file would be removed,
>>> its contents moved to the new CardTableModRefBSForCTRS.cpp as it only
>>> contains CardTableModRefBSForCTRS methods (and both seem only used for
>>> CMS).
>>> Iow, is there some reason to keep this file, or is this part of the
>>> "left for later" task?
>> 
>> Yes, this is part of the ?left for later? task.  I don?t want to move those functions
>> from one file to another, only to later have them moved (more or less) back.
> 
> Okay.
> 
> Thanks,
>  Thomas




From kim.barrett at oracle.com  Mon Jul 20 21:40:23 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Mon, 20 Jul 2015 17:40:23 -0400
Subject: RFR: 8130931: refactor CardTableModRefBS[ForCTRS]
In-Reply-To: <1437381171.2272.10.camel@oracle.com>
References: <BF2A976B-4C2D-4AC9-9627-181D1B866735@oracle.com>
	<1436777224.2275.18.camel@oracle.com>
	<94C2AED9-51B2-4452-B38E-C778C93D0FB6@oracle.com>
	<1437381171.2272.10.camel@oracle.com>
Message-ID: <D382FE5A-7F14-4891-B94E-B17B943DF9B7@oracle.com>

[Sorry about the earlier content-free reply.  Accidentally sent while still composing.]

On Jul 20, 2015, at 4:32 AM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
> 
> On Mon, 2015-07-20 at 02:27 -0400, Kim Barrett wrote:
>> On Jul 13, 2015, at 4:47 AM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
>>> 
>>> On Fri, 2015-07-10 at 20:23 -0400, Kim Barrett wrote:
>>>> Please review this refactoring of CardTableModRefBS and
>>>> CardTableModRefBSForCTRS.  The changes are in two stages, with
>>>> separate webrevs to make it easier to review.
>>>> 
>>>> The first stage eliminates some friend declarations from
>>>> CardTableModRefBS.
>>>> 
>>>> - BytecodeInterpreter doesn't need to be a friend.
>>>> - CheckForUnmarkedOops changed to use public byte_for_const().
>>> 
>>> Actually the whole member _unmarked_card can be removed.
>> 
>> Good point. Here?s the diff for that change.  I can produce a new webrev if needed.
> 
> Please do. You do need a second reviewer, not sure if he will want to
> dig through all the thread emails to get every bit of the change?

Incremental webrev for de-friending:
http://cr.openjdk.java.net/~kbarrett/8130931/remove-friends.01.incr/
This removes _unmarked_card.

Updated webrev for de-friending:
http://cr.openjdk.java.net/~kbarrett/8130931/remove-friends.01/

No changes to earlier refactoring webrev:
http://cr.openjdk.java.net/~kbarrett/8130931/refactor.00/

Updated full webrev:
http://cr.openjdk.java.net/~kbarrett/8130931/webrev.01/



From david.holmes at oracle.com  Tue Jul 21 02:03:44 2015
From: david.holmes at oracle.com (David Holmes)
Date: Tue, 21 Jul 2015 12:03:44 +1000
Subject: RFR: 8066821(S) Enhance command line processing to manage
	deprecating and obsoleting -XX command line arguments
In-Reply-To: <55AD2990.8030000@oracle.com>
References: <54B44661.8070007@oracle.com> <54B84E91.1050708@oracle.com>
	<F68EBF6B-38A5-4461-9ADD-41BB198FF943@oracle.com>
	<54D15A0B.6030501@oracle.com> <55A93BB3.4040402@oracle.com>
	<55ACBF85.9080404@oracle.com> <55AD2990.8030000@oracle.com>
Message-ID: <55ADA880.7090602@oracle.com>

On 21/07/2015 3:02 AM, Derek White wrote:
> Hi David,
>
> Thanks for looking this over.
>
> On 7/20/15 5:29 AM, David Holmes wrote:
>> Hi Derek,
>>
>> Sorry but I'm finding this a bit confused and inconsistent. Comments
>> below.
>>
>> On 18/07/2015 3:30 AM, Derek White wrote:
>>> Request for review:
>>>
>>> [This updated webrev is being sent to wider audience, and has been
>>> merged with Gerard's option constraints check-in. Also factored out
>>> -XX:+AggressiveHeap processing into it's own chapter. I mean function
>>> :-)]
>>>
>>> http://cr.openjdk.java.net/~drwhite/8066821/webrev.06/
>>> https://bugs.openjdk.java.net/browse/JDK-8066821
>>
>> argument.cpp:
>>
>>  258  *               been not scheduled).
>>
>> -> not been scheduled
>
> OK.
>>  260 *     OBSOLETE: An option may be removed (and deleted from
>> globals.hpp), but still be accepted on the command
>>  261  *               line. A warning is printed to let the user know
>> that support may be removed in the future.
>>
>>
>> Isn't this the stronger case that support has already been removed
>> (the option does nothing) and it will be removed at the next major
>> release. At the start of a major release we should be able to delete
>> all entries from the obsolete table - else it wasn't obsolete but
>> deprecated.
>>
>> Not sure "obsolete" is the right term here - obsolete things still
>> function. For us an obsolete option does nothing (it exists only in
>> the obsolete table).
> It's not a great name, but that what the previous code called it. It's a
> "I'll pretend you didn't say that" option, like when a teenager hears an
> adult use out-of-date slang ("groovy!"). How about a "condescending"
> option :-)

Name aside you didn't comment on my main comment here: an obsolete 
option has already been removed from globals etc and does nothing.

>>
>>  276  * Tests:  Aliases are tested in VMAliasOptions.java.
>>  277  *         Deprecated options are tested in
>> VMDeprecatedOptions.java.
>>  278  *         Obsolete options are tested in various files.
>>
>> We don't normally document this kind of thing in the source.
>
> I'm trying to head off unnecessary one-off test files for each alias and
> deprecated option. For example TestNoParNew.java and
> TestDefaultMaxRAMFraction.java. And I think that there should be one
> test file for obsolete options also (perhaps expanding
> ObsoleteFlagErrorMessage.java), instead of a bunch of separate test
> files, but that is not in this webrev.

Sounds fine but again we don't normally document test strategies in the 
source code.

>>
>>  281 // Obsolete or deprecated -XX flag.
>>
>> I can tell this is going to get confusing already.
>>
>>  284   JDK_Version obsoleted_in; // When the warning started (obsolete
>> or deprecated).
>>
>> But there is a difference: deprecated == warn but still process;
>> obsolete == warn and ignore.
> Yes, but the SpecialFlag type is concerned with the common aspect of
> warning. The "ignore" or "process" aspects are handled by the different
> functions that look up the obsolete_jvm_flags and deprecated_jvm_flags
> arrays.

So that variable should really be obsoleted_or_deprecated_in ?

>>
>>  288 // When a flag is eliminated, it can be added to this list in
>> order to
>>  289 // continue accepting this flag on the command-line, while
>> issuing a warning
>>  290 // and ignoring the value.  Once the JDK version reaches the
>> 'accept_until'
>>  291 // limit, we flatly refuse to admit the existence of the flag.
>>  292 static SpecialFlag const obsolete_jvm_flags[] = {
>>
>> When a flag is eliminated it is gone from this table. As soon as the
>> accept_until version is the current version we wipe the table of all
>> such entries. This should be one of the first things done in a new
>> release.
>
> I completely agree that this is a great plan. But until this April we
> still had obsolete flags listed for JDK 5! The obsolete_jvm_flags table
> did the right thing until the process caught up. In any case, this
> webrev doesn't really change the behavior of the obsolete_jvm_flags table.

But what you are seeing before April is the result of hotspot express 
(at least in a large part). Now that we don't have to support that we 
don't have legacy lists to maintain. The code could even be changed upon 
detecting that current JDK version == "until" version to report "Hey you 
forgot to update the table!" ;-)

My point is that the comments should reflect how we intend to use these, 
not give the impression that keeping eliminated options in the table is 
the expected thing to do.

>>
>> 320 // When a flag is deprecated, it can be added to this list in
>> order to issuing a warning when the flag is used.
>>  321 // Once the JDK version reaches the 'accept_until' limit, we
>> flatly refuse to admit the existence of the flag.
>>  322 static SpecialFlag const deprecated_jvm_flags[] = {
>>
>> A deprecated flag should be obsoleted before it reaches the
>> accept_until limit.
> That's a policy that's under discussion on hotspot-runtime-dev. There
> are certainly option lifecycles that have been approved by our internal
> process that don't follow this proposed policy. The mechanism in this
> webrev was concerned about supporting the plethora of current
> lifecycles, for better or worse.

But again comments should reflect expected usage - if we reach the 
"until" version of a deprecated option that wasn't obsoleted then 
something has probably gone wrong.

>> Which suggests that when we start a new version we wipe the obsoleted
>> table and move the deprecated options into it.
> I can add documentation to describe this case.
>
> If we decide that we'll always treat a deprecated or aliased option as
> obsolete for one extra release, then it might make sense to have a
> combined option lifecycle table. But for now I like the fact that
> options in deprecated_jvm_flags should always have a real variable
> defined in globals.hpp (etc), and options in obsolete_jvm_flags should
> never have a global variable.

Yes I like that too.
>
>> 776 case 1: {
>>  777       if (warn) {
>>  778         char version[256];
>>  779         since.to_string(version, sizeof(version));
>>  780         if (real_name != arg) {
>>  781           warning("Option %s was deprecated in version %s and
>> will likely be removed in a future release. Use option %s instead.",
>>  782                   arg, version, real_name);
>>  783         } else {
>>  784           warning("Option %s was deprecated in version %s and
>> will likely be removed in a future release.",
>>  785                   arg, version);
>>  786         }
>>
>> This isn't distinguishing between deprecated and obsoleted ???
>
> Yes, handle_aliases_and_deprecation() doesn't handle obsolete options
> (or it would have had a longer name :-) Maybe it should handle that
> case, but it would have complicated the control flow in the caller. I
> have another proposed change in the works that simplifies the caller
> quite a bit that would make the refactoring simpler.

I need to think more on that. It is hard to see the overall control flow 
for argument processing.

Thanks,
David

>>
>> 997       warning("Ignoring option %s; support was removed in %s",
>> stripped_argname, version);
>>
>> You have changed a handful of warnings so they start with a capital
>> letter, and have changed the associated tests. But this seems a
>> pointless "convention" because we have dozens of warnings that don't
>> start with a capital.
> The new deprecation and alias warnings replaces these various messages:
>      warning("The UseParNewGC flag is deprecated and will likely be
> removed in a future release");
>      warning("Using MaxGCMinorPauseMillis as minor pause goal is
> deprecated and will likely be removed in future release");
>      warning("DefaultMaxRAMFraction is deprecated and will likely be
> removed in a future release. Use MaxRAMFraction instead.");
>      jio_fprintf(defaultStream::error_stream(),
>          "Please use -XX:MarkStackSize in place of -XX:CMSMarkStackSize
> or -XX:G1MarkStackSize in the future\n");
>       jio_fprintf(defaultStream::error_stream(),
>          "Please use -XX:ConcGCThreads in place of
> -XX:ParallelMarkingThreads or -XX:ParallelCMSThreads in the future\n");
>       jio_fprintf(defaultStream::error_stream(),
>           "Please use -XX:MarkStackSizeMax in place of
> -XX:CMSMarkStackSizeMax in the future\n");
>       jio_fprintf(defaultStream::output_stream(),
>            "CreateMinidumpOnCrash is replaced by CreateCoredumpOnCrash:
> CreateCoredumpOnCrash is on\n");
>
> It made sense to have the case of the obsolete messages match the
> deprecation and aliases messages. Arguably I got carried away with
> changing the messages for UseAdaptiveSizePolicy, RequireSharedSpaces,
> and ScavengeRootsInCode warnings (but they didn't force changes to tests).
>
> Thanks again for you comments!
>
>   - Derek
>>
>> Thanks,
>> David
>> -----
>>
>>> This webrev adds support for handling "/deprecated/" -XX options
>>> (options that still *do* something but are planned for removal) and
>>> "/alias/" options (alternate names for other -XX options) by simply
>>> adding entries to the deprecated_jvm_flags and/or aliased_jvm_flags
>>> tables. This follows the example of the existing obsolete_jvm_flags
>>> table.
>>>
>>> This replaces a lot of ad-hoc and occasionally wrong code in
>>> arguments.cpp as well as supporting automatically disabling options
>>> after a certain version.
>>>
>>> Tests:
>>> Deprecated and alias options can be tested by adding entries to tables
>>> in new tests:
>>>    VMAliasOptions.java
>>>    VMDeprecatedOptions.java
>>>
>>> The new tests subsume these existing tests:
>>>    test/gc/startup_warnings/TestDefaultMaxRAMFraction.java
>>>    test/gc/startup_warnings/TestNoParNew.java
>>>
>>> Tests run:
>>>      jprt
>>>      jtreg (investigating errors in resource mgmt tests running on SE
>>> embedded that seems unrelated to these changes.)
>>>
>>> Thanks,
>>>
>>>   - Derek
>>>
>


From thomas.schatzl at oracle.com  Tue Jul 21 08:28:43 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Tue, 21 Jul 2015 10:28:43 +0200
Subject: RFR: 8131343: Remove unused imports from
	hotspot/test/testlibrary/jdk/test/lib/*.java
In-Reply-To: <55A677E7.3090405@oracle.com>
References: <55A677E7.3090405@oracle.com>
Message-ID: <1437467323.2070.0.camel@oracle.com>

Hi,

On Wed, 2015-07-15 at 18:10 +0300, Michail Chernov wrote:
> Hi,
> 
> Could you please review this small patch to 
> hotspot/test/testlibrary/jdk/test/lib/*.java which removes unused import 
> and fixes spelling.
> 
> http://cr.openjdk.java.net/~mchernov/8131343/webrev.00/
> https://bugs.openjdk.java.net/browse/JDK-8131343
> 

  looks good.

Thomas




From thomas.schatzl at oracle.com  Tue Jul 21 09:57:45 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Tue, 21 Jul 2015 11:57:45 +0200
Subject: RFR(XS): 8131761: Fix merge error adding code that was removed
	in 8077936.
In-Reply-To: <4295855A5C1DE049A61835A1887419CC2D006D99@DEWDFEMB12A.global.corp.sap>
References: <4295855A5C1DE049A61835A1887419CC2D006D99@DEWDFEMB12A.global.corp.sap>
Message-ID: <1437472665.2070.17.camel@oracle.com>

Hi Goetz,

On Fri, 2015-07-17 at 13:24 +0000, Lindenmaier, Goetz wrote:
> Hi,
>  
> 8077936 http://hg.openjdk.java.net/jdk9/dev/hotspot/rev/e57fce7b6ad2
> removed the function obj_field_volatile(). This was added again 
> in this merge
> change:http://hg.openjdk.java.net/jdk9/dev/hotspot/rev/5f2ef612ed74.
>  
> This change removes it once more.
>  
> Please review this change. I please need a sponsor.
> http://cr.openjdk.java.net/~goetz/webrevs/8131761-fixMerge/webrev.01/

  looks good. I will push it with the other change.

Thanks,
  Thomas





From mikael.gerdin at oracle.com  Tue Jul 21 12:09:08 2015
From: mikael.gerdin at oracle.com (Mikael Gerdin)
Date: Tue, 21 Jul 2015 14:09:08 +0200
Subject: RFR: 8130931: refactor CardTableModRefBS[ForCTRS]
In-Reply-To: <D382FE5A-7F14-4891-B94E-B17B943DF9B7@oracle.com>
References: <BF2A976B-4C2D-4AC9-9627-181D1B866735@oracle.com>	<1436777224.2275.18.camel@oracle.com>	<94C2AED9-51B2-4452-B38E-C778C93D0FB6@oracle.com>	<1437381171.2272.10.camel@oracle.com>
	<D382FE5A-7F14-4891-B94E-B17B943DF9B7@oracle.com>
Message-ID: <55AE3664.8020005@oracle.com>

Hi Kim,

On 2015-07-20 23:40, Kim Barrett wrote:
> [Sorry about the earlier content-free reply.  Accidentally sent while still composing.]
>
> On Jul 20, 2015, at 4:32 AM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
>>
>> On Mon, 2015-07-20 at 02:27 -0400, Kim Barrett wrote:
>>> On Jul 13, 2015, at 4:47 AM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
>>>>
>>>> On Fri, 2015-07-10 at 20:23 -0400, Kim Barrett wrote:
>>>>> Please review this refactoring of CardTableModRefBS and
>>>>> CardTableModRefBSForCTRS.  The changes are in two stages, with
>>>>> separate webrevs to make it easier to review.
>>>>>
>>>>> The first stage eliminates some friend declarations from
>>>>> CardTableModRefBS.
>>>>>
>>>>> - BytecodeInterpreter doesn't need to be a friend.
>>>>> - CheckForUnmarkedOops changed to use public byte_for_const().
>>>>
>>>> Actually the whole member _unmarked_card can be removed.
>>>
>>> Good point. Here?s the diff for that change.  I can produce a new webrev if needed.
>>
>> Please do. You do need a second reviewer, not sure if he will want to
>> dig through all the thread emails to get every bit of the change?
>
> Incremental webrev for de-friending:
> http://cr.openjdk.java.net/~kbarrett/8130931/remove-friends.01.incr/
> This removes _unmarked_card.
>
> Updated webrev for de-friending:
> http://cr.openjdk.java.net/~kbarrett/8130931/remove-friends.01/
>
> No changes to earlier refactoring webrev:
> http://cr.openjdk.java.net/~kbarrett/8130931/refactor.00/
>
> Updated full webrev:
> http://cr.openjdk.java.net/~kbarrett/8130931/webrev.01/

The change looks good to me.

/Mikael

>


From mikael.gerdin at oracle.com  Tue Jul 21 12:57:24 2015
From: mikael.gerdin at oracle.com (Mikael Gerdin)
Date: Tue, 21 Jul 2015 14:57:24 +0200
Subject: RFR (S): 8131319: Move G1Allocator::_summary_bytes_used back
	to G1CollectedHeap
In-Reply-To: <1436957266.2282.35.camel@oracle.com>
References: <1436957266.2282.35.camel@oracle.com>
Message-ID: <55AE41B4.9000802@oracle.com>

Hi Thomas,

On 2015-07-15 12:47, Thomas Schatzl wrote:
> Hi all,
>
>    can I have comments and reviews for the following change to move
> G1Allocator::_summary_bytes_used to a more fitting place?
>
> The problem from my POV is, that the member
> G1Allocator::_summary_bytes_used, that contains the number of bytes held
> by the heap outside of the allocators is located within G1Allocator.
>
> That sounds odd, particularly because G1Allocator actually never uses
> it, it's managed completely by G1CollectedHeap.
>
> Before the addition of the G1ArchiveAllocator one could have made the
> argument that G1Allocator holds all memory contained in the heap, but
> this is not true any more: the memory occupied by G1ArchiveAllocator
> needs to be added to the total sum of all managed space anyway, so
> moving _summary_bytes_used out of G1Allocator seems sensible (to me).
>
> I talked to StefanJ about this change who originally moved
> _summary_used_bytes into G1Allocator, and we could not find an argument
> for keeping _summary_used_bytes in G1Allocator.
>
> This allows us to also refine the responsibilities of G1Allocator a
> little more, to be the owner of the current allocation areas (regions)
> only.
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8131319
>
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8131319/webrev/

The change looks good to me.
Have you double-checked that the agent changes work (with jmap -heap)?

/Mikael

>
> Testing:
> jprt
>
> Thanks,
>    Thomas
>
>


From kim.barrett at oracle.com  Tue Jul 21 14:42:39 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Tue, 21 Jul 2015 10:42:39 -0400
Subject: RFR: 8130931: refactor CardTableModRefBS[ForCTRS]
In-Reply-To: <55AE3664.8020005@oracle.com>
References: <BF2A976B-4C2D-4AC9-9627-181D1B866735@oracle.com>	<1436777224.2275.18.camel@oracle.com>	<94C2AED9-51B2-4452-B38E-C778C93D0FB6@oracle.com>	<1437381171.2272.10.camel@oracle.com>
	<D382FE5A-7F14-4891-B94E-B17B943DF9B7@oracle.com>
	<55AE3664.8020005@oracle.com>
Message-ID: <96E82D3F-C3CE-41C7-B4B9-8A9DB35B89D3@oracle.com>

On Jul 21, 2015, at 8:09 AM, Mikael Gerdin <mikael.gerdin at oracle.com> wrote:
> 
> On 2015-07-20 23:40, Kim Barrett wrote:
>> Incremental webrev for de-friending:
>> http://cr.openjdk.java.net/~kbarrett/8130931/remove-friends.01.incr/
>> This removes _unmarked_card.
>> 
>> Updated webrev for de-friending:
>> http://cr.openjdk.java.net/~kbarrett/8130931/remove-friends.01/
>> 
>> No changes to earlier refactoring webrev:
>> http://cr.openjdk.java.net/~kbarrett/8130931/refactor.00/
>> 
>> Updated full webrev:
>> http://cr.openjdk.java.net/~kbarrett/8130931/webrev.01/
> 
> The change looks good to me.
> 
> /Mikael

Thanks.



From mikael.gerdin at oracle.com  Tue Jul 21 15:07:13 2015
From: mikael.gerdin at oracle.com (Mikael Gerdin)
Date: Tue, 21 Jul 2015 17:07:13 +0200
Subject: RFR (M): 8004687: G1: Parallelize object self-forwarding and
	scanning during an evacuation failure
In-Reply-To: <1437039612.2361.27.camel@oracle.com>
References: <1437039612.2361.27.camel@oracle.com>
Message-ID: <55AE6021.7040106@oracle.com>

Hi Thomas,

On 2015-07-16 11:40, Thomas Schatzl wrote:
> (resend with CR number in the subject)
> Hi all,
>
>    can I have reviews for the following change from a student (Walter
> Gugenberger; he has signed the OCA) who worked on this issue last
> semester?
>
> The change parallelizes object self-forwarding and scanning during an
> evacuation failure by:
> - reuse the entire existing evacuation copy closure and work queue
> mechanism in case of evacuation failure to store the work items.
> This allows evacuation failure to automatically benefit from work
> stealing etc.
> - remove the dedicate evacuation failure handling closure because it is
> not necessary any more.
>    - there is one subtle change in behavior: the old evacuation failure
> always had a NULL ReferenceProcessor, while now the standard reference
> processor for references is applied. Since the NULL ReferenceProcessor
> kept all references alive, now potentially less references will be kept
> alive after an evacuation failure. I do not see a problem here.

Right. Another difference is that the evac failure closure 
unconditionally called update_rs for each oop while the scanner closure 
only calls update_rs for non-cset oops and defers the cset updating to 
when a reference is popped from the work queue.

> - implementing per-thread preserved mark buffers
>
> As for actual performance improvements, there is none: the main problem
> is that in case of evacuation failure the code will simply serialize on
> the FreeList_lock when trying to get a new allocation region instead of
> the EvacFailureStack_lock.
>
> Evacuation failure and the early-exit for evac failure as suggested in
> the CR will only be really effective when the FreeList_lock can be
> avoided. This will be added in a follow-up change.
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8004687
>
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8004687/webrev/

Overall, a nice cleanup regardless of its lack of a general performance 
improvement.

I think that this change removes the last use of G1BarrierEvac, so the 
code for that in G1ParCopyClosure::do_oop_work can probably be removed.

/Mikael

>
> Testing:
> jprt, lots of internal evacuation failure testing, aurora runs with vm.quick/vm.gc nightly tests etc.
>
> Thanks,
>    THomas
>
>
>
>
>


From thomas.schatzl at oracle.com  Tue Jul 21 15:22:39 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Tue, 21 Jul 2015 17:22:39 +0200
Subject: RFR (S): 8131319: Move G1Allocator::_summary_bytes_used back to
	G1CollectedHeap
In-Reply-To: <55AE41B4.9000802@oracle.com>
References: <1436957266.2282.35.camel@oracle.com> <55AE41B4.9000802@oracle.com>
Message-ID: <1437492159.2278.9.camel@oracle.com>

Hi Mikael,

On Tue, 2015-07-21 at 14:57 +0200, Mikael Gerdin wrote:
> Hi Thomas,
> 
> On 2015-07-15 12:47, Thomas Schatzl wrote:
> > Hi all,
> >
> >    can I have comments and reviews for the following change to move
> > G1Allocator::_summary_bytes_used to a more fitting place?
> >
> > The problem from my POV is, that the member
> > G1Allocator::_summary_bytes_used, that contains the number of bytes held
> > by the heap outside of the allocators is located within G1Allocator.
> >
> > That sounds odd, particularly because G1Allocator actually never uses
> > it, it's managed completely by G1CollectedHeap.
> >
> > Before the addition of the G1ArchiveAllocator one could have made the
> > argument that G1Allocator holds all memory contained in the heap, but
> > this is not true any more: the memory occupied by G1ArchiveAllocator
> > needs to be added to the total sum of all managed space anyway, so
> > moving _summary_bytes_used out of G1Allocator seems sensible (to me).
> >
> > I talked to StefanJ about this change who originally moved
> > _summary_used_bytes into G1Allocator, and we could not find an argument
> > for keeping _summary_used_bytes in G1Allocator.
> >
> > This allows us to also refine the responsibilities of G1Allocator a
> > little more, to be the owner of the current allocation areas (regions)
> > only.
> >
> > CR:
> > https://bugs.openjdk.java.net/browse/JDK-8131319
> >
> > Webrev:
> > http://cr.openjdk.java.net/~tschatzl/8131319/webrev/
> 
> The change looks good to me.
> Have you double-checked that the agent changes work (with jmap -heap)?

Thanks for the review.

Apparently not :(. I checked now, and there needs to be some further tiny
change in the SA code:

http://cr.openjdk.java.net/~tschatzl/8131319/webrev.0_to_1 (diff)
http://cr.openjdk.java.net/~tschatzl/8131319/webrev.1 (full)

Thanks,
  Thomas




From thomas.schatzl at oracle.com  Tue Jul 21 17:13:23 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Tue, 21 Jul 2015 19:13:23 +0200
Subject: RFR (M): 8004687: G1: Parallelize object self-forwarding and
	scanning during an evacuation failure
In-Reply-To: <55AE6021.7040106@oracle.com>
References: <1437039612.2361.27.camel@oracle.com> <55AE6021.7040106@oracle.com>
Message-ID: <1437498803.1993.5.camel@oracle.com>

Hi Mikael,

On Tue, 2015-07-21 at 17:07 +0200, Mikael Gerdin wrote:
> Hi Thomas,
> 
> On 2015-07-16 11:40, Thomas Schatzl wrote:
> > (resend with CR number in the subject)
> > Hi all,
> >
> >    can I have reviews for the following change from a student (Walter
> > Gugenberger; he has signed the OCA) who worked on this issue last
> > semester?
> >
> > The change parallelizes object self-forwarding and scanning during an
> > evacuation failure by:
> > - reuse the entire existing evacuation copy closure and work queue
> > mechanism in case of evacuation failure to store the work items.
> > This allows evacuation failure to automatically benefit from work
> > stealing etc.
> > - remove the dedicate evacuation failure handling closure because it is
> > not necessary any more.
> >    - there is one subtle change in behavior: the old evacuation failure
> > always had a NULL ReferenceProcessor, while now the standard reference
> > processor for references is applied. Since the NULL ReferenceProcessor
> > kept all references alive, now potentially less references will be kept
> > alive after an evacuation failure. I do not see a problem here.
> 
> Right. Another difference is that the evac failure closure 
> unconditionally called update_rs for each oop while the scanner closure 
> only calls update_rs for non-cset oops and defers the cset updating to 
> when a reference is popped from the work queue.

I did not think they were worth mentioning, but you are right.

> > - implementing per-thread preserved mark buffers
> >
> > As for actual performance improvements, there is none: the main problem
> > is that in case of evacuation failure the code will simply serialize on
> > the FreeList_lock when trying to get a new allocation region instead of
> > the EvacFailureStack_lock.
> >
> > Evacuation failure and the early-exit for evac failure as suggested in
> > the CR will only be really effective when the FreeList_lock can be
> > avoided. This will be added in a follow-up change.
> >
> > CR:
> > https://bugs.openjdk.java.net/browse/JDK-8004687
> >
> > Webrev:
> > http://cr.openjdk.java.net/~tschatzl/8004687/webrev/
> 
> Overall, a nice cleanup regardless of its lack of a general performance 
> improvement.

Just waiting that this change gets reviewed, and then we are going look
at the next one :)

> I think that this change removes the last use of G1BarrierEvac, so the 
> code for that in G1ParCopyClosure::do_oop_work can probably be removed.

I prepared a webrev that removes G1BarrierEvac related code too at

 http://cr.openjdk.java.net/~tschatzl/8004687/webrev.1 (full)
 http://cr.openjdk.java.net/~tschatzl/8004687/webrev.0_to_1 (diff)

More deletions basically :)

Thanks,
  Thomas




From jon.masamitsu at oracle.com  Tue Jul 21 18:08:33 2015
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Tue, 21 Jul 2015 11:08:33 -0700
Subject: RFR (M): 8004687: G1: Parallelize object self-forwarding and
	scanning during an evacuation failure
In-Reply-To: <1437401771.2272.90.camel@oracle.com>
References: <1437039612.2361.27.camel@oracle.com>	
	<55A95E2D.1070500@oracle.com> <1437401771.2272.90.camel@oracle.com>
Message-ID: <55AE8AA1.8090909@oracle.com>



On 07/20/2015 07:16 AM, Thomas Schatzl wrote:
> Hi Jon,
>
>
> On Fri, 2015-07-17 at 12:57 -0700, Jon Masamitsu wrote:
>> Thomas,
>>
>> Changes look good.
>    thanks for the review.
>
>> Would it work to change
>>
>> OopAndMarkOop
>> to
>> OopAndOldMarkOop
>>
>> and change the constructor to
>>
>> 866     OopAndMarkOop(oop obj) : _o(obj), _m(markOop(obj))
>>
>> so that OopAndOldMarkOop specifically saves the oop and it's
>> markword rather than just an oop and (not necessarily related)
>> markword?
>    the only problem I see is that we earlier (8006025) changed the code
> to explicitly not re-read the markoop from the obj multiple times. The
> markoop is volatile, so it prevents some optimizations which we want
> here.
>
> It may not be a huge problem or one at all in this particular case,
> depending on where the OopAndOldMarkOop is instantiated, but requires
> checking.

If I understand this comment correctly, yes it does require
checking but just as it requires checking when the read of the
markoop was done (the one read of the markoop).  It seemed
to me to be modestly more clear with now that there was
an OopAndMarkOop (renamed OopAndOldMarkOop or
OopAndPreservedMarkOop which I like even better) but
sometimes less code is better (i.e., simpler not to use
OopAndMarkOop data structure) so your call.


>
> I put a webrev with pure renaming of the struct at
> http://cr.openjdk.java.net/~tschatzl/8004687/webrev.1/ (diff webrev:
> http://cr.openjdk.java.net/~tschatzl/8004687/webrev.0_to_1/).

The diff (0_to_1) seem to be only the deletion of some code,
not a renaming but maybe a moot point now if you still
like explicitly reading the value of the markoop that you
want to capture.

Jon

>
> I do not mind either version.
>
>> An enhancement to be considered now or later. You have
>> OopAndMarkOop now.  Could you pass an OopAndMarkOop
>> around instead of  an oop and a markword pair.  For
>> example, starting with
>>
>> 201 oop copy_to_survivor_space(InCSetState const state, oop const obj,
>> markOop const old_mark);
>>
>> substitute passing an OopAndMarkOop in place of the obj and old_mark.
> Prototyping this it does not seem to be huge gain. The use and passing
> of a struct is only really advantageous in a few places (in
> copy_to_survivor_spaces()). In many places the existing code manipulates
> the old object and old mark quite a bit, adding a "obj_and_mark" (if we
> call the parameter obj_and_mark) prefix in quite a few places only.
>
> I put that change at
> http://cr.openjdk.java.net/~tschatzl/8004687/webrev.refactor/.
> Particularly lines 250-265 in g1ParScanThreadState.cpp seem to be more
> obfuscated than before. There is opportunity to move the entire mark
> update method into OopAndOldMarkOop, but unless there is something I
> omitted, I would like to defer it (and the necessary performance
> checking work) to a later point.
>
>> Reviewed.
> Thanks,
>    Thomas
>



From thomas.schatzl at oracle.com  Tue Jul 21 18:28:17 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Tue, 21 Jul 2015 20:28:17 +0200
Subject: RFR (M): 8004687: G1: Parallelize object self-forwarding and
	scanning during an evacuation failure
In-Reply-To: <55AE8AA1.8090909@oracle.com>
References: <1437039612.2361.27.camel@oracle.com>
	<55A95E2D.1070500@oracle.com> <1437401771.2272.90.camel@oracle.com>
	<55AE8AA1.8090909@oracle.com>
Message-ID: <1437503297.1993.8.camel@oracle.com>

Hi Jon,

On Tue, 2015-07-21 at 11:08 -0700, Jon Masamitsu wrote:
> 
> On 07/20/2015 07:16 AM, Thomas Schatzl wrote:
> > Hi Jon,
> >
> >
> > On Fri, 2015-07-17 at 12:57 -0700, Jon Masamitsu wrote:
> >> Thomas,
> >>
> >> Changes look good.
> >    thanks for the review.
> >
> >> Would it work to change
> >>
> >> OopAndMarkOop
> >> to
> >> OopAndOldMarkOop
> >>
> >> and change the constructor to
> >>
> >> 866     OopAndMarkOop(oop obj) : _o(obj), _m(markOop(obj))
> >>
> >> so that OopAndOldMarkOop specifically saves the oop and it's
> >> markword rather than just an oop and (not necessarily related)
> >> markword?
> >    the only problem I see is that we earlier (8006025) changed the code
> > to explicitly not re-read the markoop from the obj multiple times. The
> > markoop is volatile, so it prevents some optimizations which we want
> > here.
> >
> > It may not be a huge problem or one at all in this particular case,
> > depending on where the OopAndOldMarkOop is instantiated, but requires
> > checking.
> 
> If I understand this comment correctly, yes it does require
> checking but just as it requires checking when the read of the
> markoop was done (the one read of the markoop).  It seemed

I meant checking as in "do performance checking by running benchmarks on
all systems and look through logs for changes". Sorry for being obtuse.

> to me to be modestly more clear with now that there was
> an OopAndMarkOop (renamed OopAndOldMarkOop or
> OopAndPreservedMarkOop which I like even better) but
> sometimes less code is better (i.e., simpler not to use
> OopAndMarkOop data structure) so your call.

I agree that less code is typically better. 

> > I put a webrev with pure renaming of the struct at
> > http://cr.openjdk.java.net/~tschatzl/8004687/webrev.1/ (diff webrev:
> > http://cr.openjdk.java.net/~tschatzl/8004687/webrev.0_to_1/).
> 
> The diff (0_to_1) seem to be only the deletion of some code,
> not a renaming but maybe a moot point now if you still
> like explicitly reading the value of the markoop that you
> want to capture.

  better link:
http://cr.openjdk.java.net/~tschatzl/8004687/webrev.refactor/

Sorry, I already overwrote the original with implementation of Mikael's
suggestion.

Thanks,
  Thomas




From kim.barrett at oracle.com  Tue Jul 21 22:21:43 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Tue, 21 Jul 2015 18:21:43 -0400
Subject: RFR: 8066821(S) Enhance command line processing to manage
	deprecating and obsoleting -XX command line arguments
In-Reply-To: <55A93BB3.4040402@oracle.com>
References: <54B44661.8070007@oracle.com> <54B84E91.1050708@oracle.com>
	<F68EBF6B-38A5-4461-9ADD-41BB198FF943@oracle.com>
	<54D15A0B.6030501@oracle.com> <55A93BB3.4040402@oracle.com>
Message-ID: <3CBAE8E6-244C-49E7-A5ED-AA2A3C2D3AAD@oracle.com>

On Jul 17, 2015, at 1:30 PM, Derek White <derek.white at oracle.com> wrote:
> 
> Request for review:
> 
> [This updated webrev is being sent to wider audience, and has been merged with Gerard's option constraints check-in. Also factored out -XX:+AggressiveHeap processing into it's own chapter. I mean function :-)]
> 
> http://cr.openjdk.java.net/~drwhite/8066821/webrev.06/
> https://bugs.openjdk.java.net/browse/JDK-8066821

------------------------------------------------------------------------------ 
src/share/vm/runtime/arguments.cpp
 284   JDK_Version obsoleted_in; // When the warning started (obsolete or deprecated).

"obsoleted_in" is confusingly named, given that it covers both the
"obsolete" and the "deprecated" states.  I think other reviewers
questioned whether "obsolete" was the proper term for that state.

At the risk of bikeshedding, I did a little exploring with a
thesaurus, and "discontinued" seems like a possibly better term for
that state.  The obsoleted_in field could retain that name, as
covering both deprecation and discontinuation.  Of course, that would
require some further updates, such as renaming is_newly_obsolete, and
updating various comments.

Some other possibilities include "defunct" and "disused", but I like
"discontinued" better than either of those.

------------------------------------------------------------------------------ 
src/share/vm/runtime/arguments.cpp
 288 // When a flag is eliminated, it can be added to this list in order to

I think "is eliminated" => "is made obsolete" or something like that.
I would expect "eliminated" == "removed", which is not what is being
described here.

------------------------------------------------------------------------------
src/share/vm/runtime/arguments.cpp
 320 // When a flag is deprecated, it can be added to this list in order to issuing a warning when the flag is used.

"issuing" => "issue"

------------------------------------------------------------------------------
src/share/vm/runtime/arguments.cpp
 808       return false; // "name" is a deprecated option that has expired.

I think the comment is wrong here.  I think it could just be a bogus
option name.  Later calls don't have any corresponding comment (which
is fine, just makes this one that I think might be wrong stand out
more).

------------------------------------------------------------------------------
src/share/vm/runtime/arguments.hpp
 420   // Returns true if the flag is obsolete and fits into the range specified
 421   // for being ignored.  In the case the 'version' buffer is filled in with 
 422   // the version number when the flag became obsolete. Otherwise the flag has 
 423   // expired and should be ignored.
 424   static bool is_newly_obsolete(const char* flag_name, JDK_Version* version);

The "otherwise" part of the description is not correct.  If this
returns false we don't actually know it has expired.  It could simply
not be present in the set of obsolete options.

Also, why the "newly" in the name?  "is_obsolete_flag" would be
consistent with "is_deprecated_flag".

------------------------------------------------------------------------------



From kim.barrett at oracle.com  Tue Jul 21 22:28:58 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Tue, 21 Jul 2015 18:28:58 -0400
Subject: RFR: 8066821(S) Enhance command line processing to manage
	deprecating and obsoleting -XX command line arguments
In-Reply-To: <3CBAE8E6-244C-49E7-A5ED-AA2A3C2D3AAD@oracle.com>
References: <54B44661.8070007@oracle.com> <54B84E91.1050708@oracle.com>
	<F68EBF6B-38A5-4461-9ADD-41BB198FF943@oracle.com>
	<54D15A0B.6030501@oracle.com> <55A93BB3.4040402@oracle.com>
	<3CBAE8E6-244C-49E7-A5ED-AA2A3C2D3AAD@oracle.com>
Message-ID: <CED52D43-2CEF-4C3A-B356-415F093F53DC@oracle.com>

On Jul 21, 2015, at 6:21 PM, Kim Barrett <kim.barrett at oracle.com> wrote:
> 
> On Jul 17, 2015, at 1:30 PM, Derek White <derek.white at oracle.com> wrote:
>> 
>> Request for review:
>> 
>> [This updated webrev is being sent to wider audience, and has been merged with Gerard's option constraints check-in. Also factored out -XX:+AggressiveHeap processing into it's own chapter. I mean function :-)]
>> 
>> http://cr.openjdk.java.net/~drwhite/8066821/webrev.06/
>> https://bugs.openjdk.java.net/browse/JDK-8066821
> [?]

I forgot to say, all of my suggestions are about comments / documentation / naming; the functionality looks good.




From bengt.rutisson at oracle.com  Wed Jul 22 12:12:32 2015
From: bengt.rutisson at oracle.com (Bengt Rutisson)
Date: Wed, 22 Jul 2015 14:12:32 +0200
Subject: RFR: JDK-8131600: heapdump/JMapHeap EXCEPTION_ACCESS_VIOLATION
Message-ID: <55AF88B0.40803@oracle.com>


Hi everyone,

Could I have a couple of reviews for this patch to fix one of the 
critical nightly issues we are currently having?

http://cr.openjdk.java.net/~brutisso/8131600/webrev.00/
https://bugs.openjdk.java.net/browse/JDK-8131600

Background

When we load a class we first create the InstanceKlass and then allocate 
the Java mirror on the heap. The allocation of the Java mirror is just 
like any other allocation, so it can be blocked by a safepoint since it 
will in some cases try to take the heap lock.

This means that when we hit a safepoint there can potentially exist 
InstanceKlasses where ik->java_mirror() == NULL.

The DumperSupport::dump_class_and_array_classes() currently does not 
handle that case. It blindly calls ik->signers() on the klass it is 
supposed to dump. signers() looks up the mirror (java_class()) and 
passes it on:


objArrayOop InstanceKlass::signers() const {
   // return the signers from the mirror
   return java_lang_Class::signers(java_mirror());
}

...to java_lang_Class::signers(), which will crash if it gets NULL as 
java_class:

objArrayOop java_lang_Class::signers(oop java_class) {
   assert(_signers_offset != 0, "must be set");
   return (objArrayOop)java_class->obj_field(_signers_offset);
}

The dump_class_and_array_classes() is called as part of dumping the heap 
from a jmap call. This is obviously done at a safepoint. So, if the jmap 
call comes in between the creation of the InstanceKlass and the 
allocation of the mirror we end up calling 
dump_class_and_array_classes() with an InstanceKlass that has NULL as 
its mirror. This is the reason for the crash in JDK-8131600.


Except for DumperSupport::dump_class_and_array_classes() there is only 
one user of InstanceKlass::signers(). That is 
VM_HeapWalkOperation::iterate_over_class(). This method has an early 
exit if the class has not been properly initialized yet:


     // ignore the class if it's has been initialized yet
     if (!ik->is_linked()) {
       return true;
     }


So, I think the proper fix for JDK-8131600 is to make sure that 
DumperSupport::dump_class_and_array_classes() has the same check.

Now, DumperSupport::dump_class_and_array_classes() actually calls 
InstanceKlass::signers() twice. The second time around we don't have an 
InstanceKlass but just a Klass. The Klass does not have the is_linked() 
method. I haven't tried to provoke that code to fail, but I assume it 
has the same problem. So, I added a java_class() != NULL check to that 
code to make it handle this case as well.

Thanks,
Bengt


From kim.barrett at oracle.com  Wed Jul 22 15:11:44 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Wed, 22 Jul 2015 11:11:44 -0400
Subject: RFR: JDK-8131600: heapdump/JMapHeap EXCEPTION_ACCESS_VIOLATION
In-Reply-To: <55AF88B0.40803@oracle.com>
References: <55AF88B0.40803@oracle.com>
Message-ID: <0A3E5ADA-5628-4123-A387-1354E239E64B@oracle.com>

On Jul 22, 2015, at 8:12 AM, Bengt Rutisson <bengt.rutisson at oracle.com> wrote:
> 
> 
> Hi everyone,
> 
> Could I have a couple of reviews for this patch to fix one of the critical nightly issues we are currently having?
> 
> http://cr.openjdk.java.net/~brutisso/8131600/webrev.00/
> https://bugs.openjdk.java.net/browse/JDK-8131600
> 
> Background
> 
> When we load a class we first create the InstanceKlass and then allocate the Java mirror on the heap. The allocation of the Java mirror is just like any other allocation, so it can be blocked by a safepoint since it will in some cases try to take the heap lock.
> 
> This means that when we hit a safepoint there can potentially exist InstanceKlasses where ik->java_mirror() == NULL.
> 
> The DumperSupport::dump_class_and_array_classes() currently does not handle that case. It blindly calls ik->signers() on the klass it is supposed to dump. signers() looks up the mirror (java_class()) and passes it on:
> 
> 
> objArrayOop InstanceKlass::signers() const {
>  // return the signers from the mirror
>  return java_lang_Class::signers(java_mirror());
> }
> 
> ...to java_lang_Class::signers(), which will crash if it gets NULL as java_class:
> 
> objArrayOop java_lang_Class::signers(oop java_class) {
>  assert(_signers_offset != 0, "must be set");
>  return (objArrayOop)java_class->obj_field(_signers_offset);
> }
> 
> The dump_class_and_array_classes() is called as part of dumping the heap from a jmap call. This is obviously done at a safepoint. So, if the jmap call comes in between the creation of the InstanceKlass and the allocation of the mirror we end up calling dump_class_and_array_classes() with an InstanceKlass that has NULL as its mirror. This is the reason for the crash in JDK-8131600.
> 
> 
> Except for DumperSupport::dump_class_and_array_classes() there is only one user of InstanceKlass::signers(). That is VM_HeapWalkOperation::iterate_over_class(). This method has an early exit if the class has not been properly initialized yet:
> 
> 
>    // ignore the class if it's has been initialized yet
>    if (!ik->is_linked()) {
>      return true;
>    }

Thanks for fixing the above comment.

> So, I think the proper fix for JDK-8131600 is to make sure that DumperSupport::dump_class_and_array_classes() has the same check.

This part looks good.

> 
> Now, DumperSupport::dump_class_and_array_classes() actually calls InstanceKlass::signers() twice. The second time around we don't have an InstanceKlass but just a Klass. The Klass does not have the is_linked() method. I haven't tried to provoke that code to fail, but I assume it has the same problem. So, I added a java_class() != NULL check to that code to make it handle this case as well.

I think this change shouldn't be needed.

The klass involved is the result of array_klass_or_null(). That
shouldn't be able to return an incompletely initialized (array) klass.



From kjkoster at java-monitor.com  Wed Jul 22 16:10:42 2015
From: kjkoster at java-monitor.com (Kees Jan Koster)
Date: Wed, 22 Jul 2015 16:10:42 +0000 (UTC)
Subject: G1 GC for 100GB+ heaps
Message-ID: <20150722161042.B985B1A981B@saffron.java-monitor.com>

Dear All,

Marcus Lagergren suggested I post this here.

Sorry if this message goes through twice, I had some trouble posting.

We are considering switching to G1 GC for a decently-sized HBase cluster. We run an HBase cluster on the bare metal with 128GB RAM in each machine. Today we use Java 7 with CMS, but we???d like to switch to Java 8 with G1.

* Are there practical limits to the heap size that we can use with G1 can have? Can we just allocate 100GB+ heaps and expect G1 to work untuned?

* Does the G1 GC have failure modes for edge cases that we need to keep in mind?

* What about region size? Should we keep some form of relation between the size of the HBase blocks and the G1 region size?

* What tunables do we have to look at and learn more about when we want to run with this size heap?

* Any gotcha???s or best practices to keep in mind when we switch over to G1?

--
Kees Jan

http://java-monitor.com/
kjkoster at kjkoster.org
+31651838192

Human beings make life so interesting. Do you know that in a universe so full of wonders,
they have managed to invent boredom. Quite astonishing... -- Terry Pratchett



From bengt.rutisson at oracle.com  Wed Jul 22 16:11:59 2015
From: bengt.rutisson at oracle.com (Bengt Rutisson)
Date: Wed, 22 Jul 2015 18:11:59 +0200
Subject: RFR: JDK-8131600: heapdump/JMapHeap EXCEPTION_ACCESS_VIOLATION
In-Reply-To: <0A3E5ADA-5628-4123-A387-1354E239E64B@oracle.com>
References: <55AF88B0.40803@oracle.com>
	<0A3E5ADA-5628-4123-A387-1354E239E64B@oracle.com>
Message-ID: <55AFC0CF.5000504@oracle.com>


Hi Kim,

Thanks for looking at this!

On 2015-07-22 17:11, Kim Barrett wrote:
> On Jul 22, 2015, at 8:12 AM, Bengt Rutisson <bengt.rutisson at oracle.com> wrote:
>>
>> Hi everyone,
>>
>> Could I have a couple of reviews for this patch to fix one of the critical nightly issues we are currently having?
>>
>> http://cr.openjdk.java.net/~brutisso/8131600/webrev.00/
>> https://bugs.openjdk.java.net/browse/JDK-8131600
>>
>> Background
>>
>> When we load a class we first create the InstanceKlass and then allocate the Java mirror on the heap. The allocation of the Java mirror is just like any other allocation, so it can be blocked by a safepoint since it will in some cases try to take the heap lock.
>>
>> This means that when we hit a safepoint there can potentially exist InstanceKlasses where ik->java_mirror() == NULL.
>>
>> The DumperSupport::dump_class_and_array_classes() currently does not handle that case. It blindly calls ik->signers() on the klass it is supposed to dump. signers() looks up the mirror (java_class()) and passes it on:
>>
>>
>> objArrayOop InstanceKlass::signers() const {
>>   // return the signers from the mirror
>>   return java_lang_Class::signers(java_mirror());
>> }
>>
>> ...to java_lang_Class::signers(), which will crash if it gets NULL as java_class:
>>
>> objArrayOop java_lang_Class::signers(oop java_class) {
>>   assert(_signers_offset != 0, "must be set");
>>   return (objArrayOop)java_class->obj_field(_signers_offset);
>> }
>>
>> The dump_class_and_array_classes() is called as part of dumping the heap from a jmap call. This is obviously done at a safepoint. So, if the jmap call comes in between the creation of the InstanceKlass and the allocation of the mirror we end up calling dump_class_and_array_classes() with an InstanceKlass that has NULL as its mirror. This is the reason for the crash in JDK-8131600.
>>
>>
>> Except for DumperSupport::dump_class_and_array_classes() there is only one user of InstanceKlass::signers(). That is VM_HeapWalkOperation::iterate_over_class(). This method has an early exit if the class has not been properly initialized yet:
>>
>>
>>     // ignore the class if it's has been initialized yet
>>     if (!ik->is_linked()) {
>>       return true;
>>     }
> Thanks for fixing the above comment.
>
>> So, I think the proper fix for JDK-8131600 is to make sure that DumperSupport::dump_class_and_array_classes() has the same check.
> This part looks good.

Good. Thanks.

>
>> Now, DumperSupport::dump_class_and_array_classes() actually calls InstanceKlass::signers() twice. The second time around we don't have an InstanceKlass but just a Klass. The Klass does not have the is_linked() method. I haven't tried to provoke that code to fail, but I assume it has the same problem. So, I added a java_class() != NULL check to that code to make it handle this case as well.
> I think this change shouldn't be needed.
>
> The klass involved is the result of array_klass_or_null(). That
> shouldn't be able to return an incompletely initialized (array) klass.

Yes, I was discussing this a bit with Mikael and we weren't really sure. 
I think you are right, I'm just a bit worried about how fragile the 
class initialization seems to be.

But I'll remove this part of the fix. Let's just keep our eyes open if a 
crash in this area comes up again.

Thanks,
Bengt

>



From kjkoster at java-monitor.com  Wed Jul 22 13:51:59 2015
From: kjkoster at java-monitor.com (Kees Jan Koster)
Date: Wed, 22 Jul 2015 13:51:59 +0000 (UTC)
Subject: G1 GC for 100GB+ heaps
Message-ID: <20150722135159.4CBEC1A981D@saffron.java-monitor.com>

Dear All,

Marcus Lagergren suggested I post these questions on this list. We are considering switching to using the G1 GC for a decently sized HBase cluster, and ran into some questions. Hope you can help me our, or point me to the place where I should ask.

First -> sizing: Our machines have 128GB of RAM. We run on the bare metal. Is there a practical limit to the heap size we should allocate to the JVM when using the G1 GC? What kind of region sizing should we use, or should we just let G1 do what it does?

Second -> failure modes. Does G1 have any failure or fall-back modes that it will use for edge cases? How do we monitor for those?

Finally: Are there any gotcha???s to keep in mind, or any tunables that we have to invest time into when we want to run smoothly with 100GB+ heap sizes?


--
Kees Jan

http://java-monitor.com/
kjkoster at kjkoster.org
+31651838192

Change is good. Granted, it is good in retrospect, but change is good.


From jon.masamitsu at oracle.com  Wed Jul 22 17:43:31 2015
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Wed, 22 Jul 2015 10:43:31 -0700
Subject: RFR (M): 8004687: G1: Parallelize object self-forwarding and
	scanning during an evacuation failure
In-Reply-To: <1437503297.1993.8.camel@oracle.com>
References: <1437039612.2361.27.camel@oracle.com>		
	<55A95E2D.1070500@oracle.com>
	<1437401771.2272.90.camel@oracle.com>	
	<55AE8AA1.8090909@oracle.com> <1437503297.1993.8.camel@oracle.com>
Message-ID: <55AFD643.5020902@oracle.com>

Thomas,

Still your call with regard to using OopAndMarkOop.  You made all
the changes.  If you don't like them, that's fine.

Jon

On 7/21/2015 11:28 AM, Thomas Schatzl wrote:
> Hi Jon,
>
> On Tue, 2015-07-21 at 11:08 -0700, Jon Masamitsu wrote:
>> On 07/20/2015 07:16 AM, Thomas Schatzl wrote:
>>> Hi Jon,
>>>
>>>
>>> On Fri, 2015-07-17 at 12:57 -0700, Jon Masamitsu wrote:
>>>> Thomas,
>>>>
>>>> Changes look good.
>>>     thanks for the review.
>>>
>>>> Would it work to change
>>>>
>>>> OopAndMarkOop
>>>> to
>>>> OopAndOldMarkOop
>>>>
>>>> and change the constructor to
>>>>
>>>> 866     OopAndMarkOop(oop obj) : _o(obj), _m(markOop(obj))
>>>>
>>>> so that OopAndOldMarkOop specifically saves the oop and it's
>>>> markword rather than just an oop and (not necessarily related)
>>>> markword?
>>>     the only problem I see is that we earlier (8006025) changed the code
>>> to explicitly not re-read the markoop from the obj multiple times. The
>>> markoop is volatile, so it prevents some optimizations which we want
>>> here.
>>>
>>> It may not be a huge problem or one at all in this particular case,
>>> depending on where the OopAndOldMarkOop is instantiated, but requires
>>> checking.
>> If I understand this comment correctly, yes it does require
>> checking but just as it requires checking when the read of the
>> markoop was done (the one read of the markoop).  It seemed
> I meant checking as in "do performance checking by running benchmarks on
> all systems and look through logs for changes". Sorry for being obtuse.
>
>> to me to be modestly more clear with now that there was
>> an OopAndMarkOop (renamed OopAndOldMarkOop or
>> OopAndPreservedMarkOop which I like even better) but
>> sometimes less code is better (i.e., simpler not to use
>> OopAndMarkOop data structure) so your call.
> I agree that less code is typically better.
>
>>> I put a webrev with pure renaming of the struct at
>>> http://cr.openjdk.java.net/~tschatzl/8004687/webrev.1/ (diff webrev:
>>> http://cr.openjdk.java.net/~tschatzl/8004687/webrev.0_to_1/).
>> The diff (0_to_1) seem to be only the deletion of some code,
>> not a renaming but maybe a moot point now if you still
>> like explicitly reading the value of the markoop that you
>> want to capture.
>    better link:
> http://cr.openjdk.java.net/~tschatzl/8004687/webrev.refactor/
>
> Sorry, I already overwrote the original with implementation of Mikael's
> suggestion.
>
> Thanks,
>    Thomas
>
>



From kim.barrett at oracle.com  Wed Jul 22 18:03:31 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Wed, 22 Jul 2015 14:03:31 -0400
Subject: RFR: JDK-8131600: heapdump/JMapHeap EXCEPTION_ACCESS_VIOLATION
In-Reply-To: <55AFC0CF.5000504@oracle.com>
References: <55AF88B0.40803@oracle.com>
	<0A3E5ADA-5628-4123-A387-1354E239E64B@oracle.com>
	<55AFC0CF.5000504@oracle.com>
Message-ID: <47092E7E-ADF4-4CF9-8CAC-54F2A1AB82F2@oracle.com>

On Jul 22, 2015, at 12:11 PM, Bengt Rutisson <bengt.rutisson at oracle.com> wrote:
> 
> On 2015-07-22 17:11, Kim Barrett wrote:
>> On Jul 22, 2015, at 8:12 AM, Bengt Rutisson <bengt.rutisson at oracle.com> wrote:
>>> 
>>> Hi everyone,
>>> 
>>> Could I have a couple of reviews for this patch to fix one of the critical nightly issues we are currently having?
>>> 
>>> http://cr.openjdk.java.net/~brutisso/8131600/webrev.00/
>>> https://bugs.openjdk.java.net/browse/JDK-8131600
>>> 
>>> Background
>>> [?]
>> 
>>> Now, DumperSupport::dump_class_and_array_classes() actually calls InstanceKlass::signers() twice. The second time around we don't have an InstanceKlass but just a Klass. The Klass does not have the is_linked() method. I haven't tried to provoke that code to fail, but I assume it has the same problem. So, I added a java_class() != NULL check to that code to make it handle this case as well.
>> I think this change shouldn't be needed.
>> 
>> The klass involved is the result of array_klass_or_null(). That
>> shouldn't be able to return an incompletely initialized (array) klass.
> 
> Yes, I was discussing this a bit with Mikael and we weren't really sure. I think you are right, I'm just a bit worried about how fragile the class initialization seems to be.
> 
> But I'll remove this part of the fix. Let's just keep our eyes open if a crash in this area comes up again.

With that part backed out, change looks good to me.



From bengt.rutisson at oracle.com  Thu Jul 23 06:37:38 2015
From: bengt.rutisson at oracle.com (Bengt Rutisson)
Date: Thu, 23 Jul 2015 08:37:38 +0200
Subject: RFR: JDK-8131600: heapdump/JMapHeap EXCEPTION_ACCESS_VIOLATION
In-Reply-To: <47092E7E-ADF4-4CF9-8CAC-54F2A1AB82F2@oracle.com>
References: <55AF88B0.40803@oracle.com>
	<0A3E5ADA-5628-4123-A387-1354E239E64B@oracle.com>
	<55AFC0CF.5000504@oracle.com>
	<47092E7E-ADF4-4CF9-8CAC-54F2A1AB82F2@oracle.com>
Message-ID: <55B08BB2.7090902@oracle.com>



On 2015-07-22 20:03, Kim Barrett wrote:
> On Jul 22, 2015, at 12:11 PM, Bengt Rutisson <bengt.rutisson at oracle.com> wrote:
>> On 2015-07-22 17:11, Kim Barrett wrote:
>>> On Jul 22, 2015, at 8:12 AM, Bengt Rutisson <bengt.rutisson at oracle.com> wrote:
>>>> Hi everyone,
>>>>
>>>> Could I have a couple of reviews for this patch to fix one of the critical nightly issues we are currently having?
>>>>
>>>> http://cr.openjdk.java.net/~brutisso/8131600/webrev.00/
>>>> https://bugs.openjdk.java.net/browse/JDK-8131600
>>>>
>>>> Background
>>>> [?]
>>>> Now, DumperSupport::dump_class_and_array_classes() actually calls InstanceKlass::signers() twice. The second time around we don't have an InstanceKlass but just a Klass. The Klass does not have the is_linked() method. I haven't tried to provoke that code to fail, but I assume it has the same problem. So, I added a java_class() != NULL check to that code to make it handle this case as well.
>>> I think this change shouldn't be needed.
>>>
>>> The klass involved is the result of array_klass_or_null(). That
>>> shouldn't be able to return an incompletely initialized (array) klass.
>> Yes, I was discussing this a bit with Mikael and we weren't really sure. I think you are right, I'm just a bit worried about how fragile the class initialization seems to be.
>>
>> But I'll remove this part of the fix. Let's just keep our eyes open if a crash in this area comes up again.
> With that part backed out, change looks good to me.

Thanks, Kim!

Here's an updated webrev:
http://cr.openjdk.java.net/~brutisso/8131600/webrev.01/

Bengt


>



From thomas.schatzl at oracle.com  Thu Jul 23 06:54:06 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 23 Jul 2015 08:54:06 +0200
Subject: RFR: JDK-8131600: heapdump/JMapHeap EXCEPTION_ACCESS_VIOLATION
In-Reply-To: <55B08BB2.7090902@oracle.com>
References: <55AF88B0.40803@oracle.com>
	<0A3E5ADA-5628-4123-A387-1354E239E64B@oracle.com>
	<55AFC0CF.5000504@oracle.com>
	<47092E7E-ADF4-4CF9-8CAC-54F2A1AB82F2@oracle.com>
	<55B08BB2.7090902@oracle.com>
Message-ID: <1437634446.2347.0.camel@oracle.com>

Hi Bengt,

On Thu, 2015-07-23 at 08:37 +0200, Bengt Rutisson wrote:
> 
> On 2015-07-22 20:03, Kim Barrett wrote:
> > On Jul 22, 2015, at 12:11 PM, Bengt Rutisson <bengt.rutisson at oracle.com> wrote:
> >> On 2015-07-22 17:11, Kim Barrett wrote:
> >>> On Jul 22, 2015, at 8:12 AM, Bengt Rutisson <bengt.rutisson at oracle.com> wrote:
> >>>> Hi everyone,
> >>>>
> >>>> Could I have a couple of reviews for this patch to fix one of the critical nightly issues we are currently having?
> >>>>
> >>>> http://cr.openjdk.java.net/~brutisso/8131600/webrev.00/
> >>>> https://bugs.openjdk.java.net/browse/JDK-8131600
> >>>>
> >>>> Background
> >>>> [?]
> >>>> Now, DumperSupport::dump_class_and_array_classes() actually calls InstanceKlass::signers() twice. The second time around we don't have an InstanceKlass but just a Klass. The Klass does not have the is_linked() method. I haven't tried to provoke that code to fail, but I assume it has the same problem. So, I added a java_class() != NULL check to that code to make it handle this case as well.
> >>> I think this change shouldn't be needed.
> >>>
> >>> The klass involved is the result of array_klass_or_null(). That
> >>> shouldn't be able to return an incompletely initialized (array) klass.
> >> Yes, I was discussing this a bit with Mikael and we weren't really sure. I think you are right, I'm just a bit worried about how fragile the class initialization seems to be.
> >>
> >> But I'll remove this part of the fix. Let's just keep our eyes open if a crash in this area comes up again.
> > With that part backed out, change looks good to me.
> 
> Thanks, Kim!
> 
> Here's an updated webrev:
> http://cr.openjdk.java.net/~brutisso/8131600/webrev.01/

  looks good.

Thomas




From bengt.rutisson at oracle.com  Thu Jul 23 06:51:35 2015
From: bengt.rutisson at oracle.com (Bengt Rutisson)
Date: Thu, 23 Jul 2015 08:51:35 +0200
Subject: RFR: JDK-8131600: heapdump/JMapHeap EXCEPTION_ACCESS_VIOLATION
In-Reply-To: <1437634446.2347.0.camel@oracle.com>
References: <55AF88B0.40803@oracle.com>	
	<0A3E5ADA-5628-4123-A387-1354E239E64B@oracle.com>	
	<55AFC0CF.5000504@oracle.com>	
	<47092E7E-ADF4-4CF9-8CAC-54F2A1AB82F2@oracle.com>	
	<55B08BB2.7090902@oracle.com> <1437634446.2347.0.camel@oracle.com>
Message-ID: <55B08EF7.8000405@oracle.com>


Hi Thomas,

On 2015-07-23 08:54, Thomas Schatzl wrote:
> Hi Bengt,
>
> On Thu, 2015-07-23 at 08:37 +0200, Bengt Rutisson wrote:
>> On 2015-07-22 20:03, Kim Barrett wrote:
>>> On Jul 22, 2015, at 12:11 PM, Bengt Rutisson <bengt.rutisson at oracle.com> wrote:
>>>> On 2015-07-22 17:11, Kim Barrett wrote:
>>>>> On Jul 22, 2015, at 8:12 AM, Bengt Rutisson <bengt.rutisson at oracle.com> wrote:
>>>>>> Hi everyone,
>>>>>>
>>>>>> Could I have a couple of reviews for this patch to fix one of the critical nightly issues we are currently having?
>>>>>>
>>>>>> http://cr.openjdk.java.net/~brutisso/8131600/webrev.00/
>>>>>> https://bugs.openjdk.java.net/browse/JDK-8131600
>>>>>>
>>>>>> Background
>>>>>> [?]
>>>>>> Now, DumperSupport::dump_class_and_array_classes() actually calls InstanceKlass::signers() twice. The second time around we don't have an InstanceKlass but just a Klass. The Klass does not have the is_linked() method. I haven't tried to provoke that code to fail, but I assume it has the same problem. So, I added a java_class() != NULL check to that code to make it handle this case as well.
>>>>> I think this change shouldn't be needed.
>>>>>
>>>>> The klass involved is the result of array_klass_or_null(). That
>>>>> shouldn't be able to return an incompletely initialized (array) klass.
>>>> Yes, I was discussing this a bit with Mikael and we weren't really sure. I think you are right, I'm just a bit worried about how fragile the class initialization seems to be.
>>>>
>>>> But I'll remove this part of the fix. Let's just keep our eyes open if a crash in this area comes up again.
>>> With that part backed out, change looks good to me.
>> Thanks, Kim!
>>
>> Here's an updated webrev:
>> http://cr.openjdk.java.net/~brutisso/8131600/webrev.01/
>    looks good.

Thanks for the review!

Bengt

>
> Thomas
>
>



From goetz.lindenmaier at sap.com  Thu Jul 23 07:08:26 2015
From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz)
Date: Thu, 23 Jul 2015 07:08:26 +0000
Subject: RFR(XS): 8131761: Fix merge error adding code that was removed
	in 8077936.
In-Reply-To: <1437472665.2070.17.camel@oracle.com>
References: <4295855A5C1DE049A61835A1887419CC2D006D99@DEWDFEMB12A.global.corp.sap>
	<1437472665.2070.17.camel@oracle.com>
Message-ID: <4295855A5C1DE049A61835A1887419CC2D008E47@DEWDFEMB12A.global.corp.sap>

Thomas,

thanks for helping out with these two changes!

Best regards,
  Goetz.

-----Original Message-----
From: Thomas Schatzl [mailto:thomas.schatzl at oracle.com] 
Sent: Dienstag, 21. Juli 2015 11:58
To: Lindenmaier, Goetz
Cc: hotspot-gc-dev at openjdk.java.net
Subject: Re: RFR(XS): 8131761: Fix merge error adding code that was removed in 8077936.

Hi Goetz,

On Fri, 2015-07-17 at 13:24 +0000, Lindenmaier, Goetz wrote:
> Hi,
>  
> 8077936 http://hg.openjdk.java.net/jdk9/dev/hotspot/rev/e57fce7b6ad2
> removed the function obj_field_volatile(). This was added again 
> in this merge
> change:http://hg.openjdk.java.net/jdk9/dev/hotspot/rev/5f2ef612ed74.
>  
> This change removes it once more.
>  
> Please review this change. I please need a sponsor.
> http://cr.openjdk.java.net/~goetz/webrevs/8131761-fixMerge/webrev.01/

  looks good. I will push it with the other change.

Thanks,
  Thomas




From thomas.schatzl at oracle.com  Thu Jul 23 07:50:07 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 23 Jul 2015 09:50:07 +0200
Subject: G1 GC for 100GB+ heaps
In-Reply-To: <20150722135159.4CBEC1A981D@saffron.java-monitor.com>
References: <20150722135159.4CBEC1A981D@saffron.java-monitor.com>
Message-ID: <1437637807.2347.37.camel@oracle.com>

Hi,

On Wed, 2015-07-22 at 13:51 +0000, Kees Jan Koster wrote:
> Dear All,
> 
> Marcus Lagergren suggested I post these questions on this list. We
> are considering switching to using the G1 GC for a decently sized
> HBase cluster, and ran into some questions. Hope you can help me
> our, or point me to the place where I should ask.

This place is fine, although hotspot-gc-use might be more appropriate.

> First -> sizing: Our machines have 128GB of RAM. We run on the bare
> metal. Is there a practical limit to the heap size we should
> allocate to the JVM when using the G1 GC?

100G heaps are fine with G1. We have heard of quite a few successful
reports of using G1 with big data setups. (E.g.
https://databricks.com/blog/2015/05/28/tuning-java-garbage-collection-for-spark-applications.html 100G heap, significantly beating Parallel GC in throughput, https://software.intel.com/en-us/blogs/2014/06/18/part-1-tuning-java-garbage-collection-for-hbase 100G heap, 100ms latency) or people moving to G1 as default (https://issues.apache.org/jira/browse/CASSANDRA-7486 ) or just being happy with using G1 on smaller setups (https://wiki.apache.org/solr/ShawnHeisey) for this kind of workloads.

However you do not mention what your goals are (throughput or latency or
a mix of that), so it is hard to say whether G1 can meet your
expectations.

The larger your heap gets (like TB heaps) and the more stringent the
requirements are, depending on the application, it may start failing to
meet your expectations.

> What kind of region sizing
> should we use, or should we just let G1 do what it does?

Initially we recommend just setting heap size (Xms/Xmx) and pause time
goals (-XX:MaxGCPauseMillis). Depending on your results decreasing
G1NewSizePercent and increasing the number of marking threads (see the
first few links above).

> Second -> failure modes. Does G1 have any failure or fall-back modes
> that it will use for edge cases? How do we monitor for those?

The obvious last-effort failure mode in case of going out of memory is
issuing a full GC, on these heaps taking minutes which is typically not
acceptable.

The GC log will tell you about it (-XX:+PrintGCTimeStamps -XX:
+PrintGCDetails, search for "Full GC"). This typically occurs because
marking could not finish before going out of memory. In these cases,
decreasing the heap occupancy of when marking starts
(-XX:InitiatingHeapOccupancyPercent=n) and/or increasing the number of
marking threads (-XX:ConcGCThreads=n) helps.

Again, the first link gives some further explanation.

I think HBase in particular needs -XX:+ParallelRefProcEnabled to work
well (independent of collector used), see the links given.

> Finally: Are there any gotcha?s to keep in mind, or any tunables that
> we have to invest time into when we want to run smoothly with 100GB+
> heap sizes?

Consider that the G1 needs some extra space for operation. So at 100G
Java heap, and 128G RAM, the system might start to swap/thrash,
particular if other stuff is running there. I.e. monitor that using e.g.
vmstat. Should be avoided :)

If you are running on Linux, completely disable Transparent Huge Pages
on Linux (use a search engine to get to know how it is done on your
particular distro). Always, we have found no exceptions.

Other than that the above recommendations should be okay. If there are
particular issues you may want to come back with a log of a problematic
run with at least -XX:+PrintGCTimeStamps -XX:+PrintGCDetails set.

Thanks,
  Thomas




From thomas.schatzl at oracle.com  Thu Jul 23 08:31:50 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 23 Jul 2015 10:31:50 +0200
Subject: RFR (M): 8004687: G1: Parallelize object self-forwarding and
	scanning during an evacuation failure
In-Reply-To: <55AFD643.5020902@oracle.com>
References: <1437039612.2361.27.camel@oracle.com>
	<55A95E2D.1070500@oracle.com> <1437401771.2272.90.camel@oracle.com>
	<55AE8AA1.8090909@oracle.com> <1437503297.1993.8.camel@oracle.com>
	<55AFD643.5020902@oracle.com>
Message-ID: <1437640310.2347.43.camel@oracle.com>

Hi Jon,

On Wed, 2015-07-22 at 10:43 -0700, Jon Masamitsu wrote:
> Thomas,
> 
> Still your call with regard to using OopAndMarkOop.  You made all
> the changes.  If you don't like them, that's fine.

  I will move these changes to later if you really do not mind. I
actually have a later patch that does some more encapsulation and moving
of that data structure in the works for this area anyway.

Thanks,
  Thomas




From mikael.gerdin at oracle.com  Thu Jul 23 08:47:05 2015
From: mikael.gerdin at oracle.com (Mikael Gerdin)
Date: Thu, 23 Jul 2015 10:47:05 +0200
Subject: RFR (M): 8004687: G1: Parallelize object self-forwarding and
	scanning during an evacuation failure
In-Reply-To: <1437498803.1993.5.camel@oracle.com>
References: <1437039612.2361.27.camel@oracle.com>	
	<55AE6021.7040106@oracle.com> <1437498803.1993.5.camel@oracle.com>
Message-ID: <55B0AA09.6060504@oracle.com>

Hi Thomas,

On 2015-07-21 19:13, Thomas Schatzl wrote:
> Hi Mikael,
>
> On Tue, 2015-07-21 at 17:07 +0200, Mikael Gerdin wrote:
>> Hi Thomas,
>>
>> On 2015-07-16 11:40, Thomas Schatzl wrote:
>>> (resend with CR number in the subject)
>>> Hi all,
>>>
>>>     can I have reviews for the following change from a student (Walter
>>> Gugenberger; he has signed the OCA) who worked on this issue last
>>> semester?
>>>
>>> The change parallelizes object self-forwarding and scanning during an
>>> evacuation failure by:
>>> - reuse the entire existing evacuation copy closure and work queue
>>> mechanism in case of evacuation failure to store the work items.
>>> This allows evacuation failure to automatically benefit from work
>>> stealing etc.
>>> - remove the dedicate evacuation failure handling closure because it is
>>> not necessary any more.
>>>     - there is one subtle change in behavior: the old evacuation failure
>>> always had a NULL ReferenceProcessor, while now the standard reference
>>> processor for references is applied. Since the NULL ReferenceProcessor
>>> kept all references alive, now potentially less references will be kept
>>> alive after an evacuation failure. I do not see a problem here.
>>
>> Right. Another difference is that the evac failure closure
>> unconditionally called update_rs for each oop while the scanner closure
>> only calls update_rs for non-cset oops and defers the cset updating to
>> when a reference is popped from the work queue.
>
> I did not think they were worth mentioning, but you are right.
>
>>> - implementing per-thread preserved mark buffers
>>>
>>> As for actual performance improvements, there is none: the main problem
>>> is that in case of evacuation failure the code will simply serialize on
>>> the FreeList_lock when trying to get a new allocation region instead of
>>> the EvacFailureStack_lock.
>>>
>>> Evacuation failure and the early-exit for evac failure as suggested in
>>> the CR will only be really effective when the FreeList_lock can be
>>> avoided. This will be added in a follow-up change.
>>>
>>> CR:
>>> https://bugs.openjdk.java.net/browse/JDK-8004687
>>>
>>> Webrev:
>>> http://cr.openjdk.java.net/~tschatzl/8004687/webrev/
>>
>> Overall, a nice cleanup regardless of its lack of a general performance
>> improvement.
>
> Just waiting that this change gets reviewed, and then we are going look
> at the next one :)
>
>> I think that this change removes the last use of G1BarrierEvac, so the
>> code for that in G1ParCopyClosure::do_oop_work can probably be removed.
>
> I prepared a webrev that removes G1BarrierEvac related code too at
>
>   http://cr.openjdk.java.net/~tschatzl/8004687/webrev.1 (full)
>   http://cr.openjdk.java.net/~tschatzl/8004687/webrev.0_to_1 (diff)

Looks good to me.

/Mikael

>
> More deletions basically :)
>
> Thanks,
>    Thomas
>
>


From mikael.gerdin at oracle.com  Thu Jul 23 08:47:50 2015
From: mikael.gerdin at oracle.com (Mikael Gerdin)
Date: Thu, 23 Jul 2015 10:47:50 +0200
Subject: RFR (S): 8131319: Move G1Allocator::_summary_bytes_used back
	to G1CollectedHeap
In-Reply-To: <1437492159.2278.9.camel@oracle.com>
References: <1436957266.2282.35.camel@oracle.com>	
	<55AE41B4.9000802@oracle.com> <1437492159.2278.9.camel@oracle.com>
Message-ID: <55B0AA36.5020203@oracle.com>



On 2015-07-21 17:22, Thomas Schatzl wrote:
> Hi Mikael,
>
> On Tue, 2015-07-21 at 14:57 +0200, Mikael Gerdin wrote:
>> Hi Thomas,
>>
>> On 2015-07-15 12:47, Thomas Schatzl wrote:
>>> Hi all,
>>>
>>>     can I have comments and reviews for the following change to move
>>> G1Allocator::_summary_bytes_used to a more fitting place?
>>>
>>> The problem from my POV is, that the member
>>> G1Allocator::_summary_bytes_used, that contains the number of bytes held
>>> by the heap outside of the allocators is located within G1Allocator.
>>>
>>> That sounds odd, particularly because G1Allocator actually never uses
>>> it, it's managed completely by G1CollectedHeap.
>>>
>>> Before the addition of the G1ArchiveAllocator one could have made the
>>> argument that G1Allocator holds all memory contained in the heap, but
>>> this is not true any more: the memory occupied by G1ArchiveAllocator
>>> needs to be added to the total sum of all managed space anyway, so
>>> moving _summary_bytes_used out of G1Allocator seems sensible (to me).
>>>
>>> I talked to StefanJ about this change who originally moved
>>> _summary_used_bytes into G1Allocator, and we could not find an argument
>>> for keeping _summary_used_bytes in G1Allocator.
>>>
>>> This allows us to also refine the responsibilities of G1Allocator a
>>> little more, to be the owner of the current allocation areas (regions)
>>> only.
>>>
>>> CR:
>>> https://bugs.openjdk.java.net/browse/JDK-8131319
>>>
>>> Webrev:
>>> http://cr.openjdk.java.net/~tschatzl/8131319/webrev/
>>
>> The change looks good to me.
>> Have you double-checked that the agent changes work (with jmap -heap)?
>
> Thanks for the review.
>
> Apparently not :(. I checked now, and there needs to be some further tiny
> change in the SA code:
>
> http://cr.openjdk.java.net/~tschatzl/8131319/webrev.0_to_1 (diff)
> http://cr.openjdk.java.net/~tschatzl/8131319/webrev.1 (full)

Great! Thanks for verifying the SA code.
Looks good now.

/Mikael

>
> Thanks,
>    Thomas
>
>


From thomas.schatzl at oracle.com  Thu Jul 23 09:17:56 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 23 Jul 2015 11:17:56 +0200
Subject: RFR (S): 8131319: Move G1Allocator::_summary_bytes_used back to
	G1CollectedHeap
In-Reply-To: <55B0AA36.5020203@oracle.com>
References: <1436957266.2282.35.camel@oracle.com>
	<55AE41B4.9000802@oracle.com> <1437492159.2278.9.camel@oracle.com>
	<55B0AA36.5020203@oracle.com>
Message-ID: <1437643076.2347.45.camel@oracle.com>

Hi Mikael,

On Thu, 2015-07-23 at 10:47 +0200, Mikael Gerdin wrote:
> 
> On 2015-07-21 17:22, Thomas Schatzl wrote:
> > Hi Mikael,
> >
> > On Tue, 2015-07-21 at 14:57 +0200, Mikael Gerdin wrote:
> >> Hi Thomas,
> >>
[...]
> > Apparently not :(. I checked now, and there needs to be some further tiny
> > change in the SA code:
> >
> > http://cr.openjdk.java.net/~tschatzl/8131319/webrev.0_to_1 (diff)
> > http://cr.openjdk.java.net/~tschatzl/8131319/webrev.1 (full)
> 
> Great! Thanks for verifying the SA code.
> Looks good now.
> 
> /Mikael

  thanks for the review.

Thomas




From thomas.schatzl at oracle.com  Thu Jul 23 09:18:25 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 23 Jul 2015 11:18:25 +0200
Subject: RFR (S): 8131319: Move G1Allocator::_summary_bytes_used back to
	G1CollectedHeap
In-Reply-To: <55A9394B.4040703@oracle.com>
References: <1436957266.2282.35.camel@oracle.com>
	<55A67F51.9010504@oracle.com> <1437038169.2361.26.camel@oracle.com>
	<1437042236.2361.36.camel@oracle.com> <55A90AEB.7090006@oracle.com>
	<1437148543.9068.25.camel@oracle.com> <55A9394B.4040703@oracle.com>
Message-ID: <1437643105.2347.46.camel@oracle.com>

Hi Tom,

On Fri, 2015-07-17 at 13:20 -0400, Tom Benson wrote:
> Hi Thomas,
> 
> On 7/17/2015 11:55 AM, Thomas Schatzl wrote:
> > Hi,
> [..]
> > The problem with immediate updates would obviously be concurrency, e.g.
> > if multiple threads were allocating into an archive region(s). Or
> > another allocator doing its updates directly like for pinned regions.
> >
> > Not sure about how MT safe the current archive allocator is actually.
> It is designed for single threaded use from the VM thread, not general 
> allocation.
> 
> Anyway,  your change for 8131319 still looks good to me.  Tnx,
> Tom

  thanks for your review :)

Thomas



From thomas.schatzl at oracle.com  Thu Jul 23 09:19:41 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 23 Jul 2015 11:19:41 +0200
Subject: RFR (M): 8004687: G1: Parallelize object self-forwarding and
	scanning during an evacuation failure
In-Reply-To: <55B0AA09.6060504@oracle.com>
References: <1437039612.2361.27.camel@oracle.com>
	<55AE6021.7040106@oracle.com> <1437498803.1993.5.camel@oracle.com>
	<55B0AA09.6060504@oracle.com>
Message-ID: <1437643181.2347.47.camel@oracle.com>

Hi Mikael,

On Thu, 2015-07-23 at 10:47 +0200, Mikael Gerdin wrote:
> Hi Thomas,
> 
> On 2015-07-21 19:13, Thomas Schatzl wrote:
> > Hi Mikael,
> [...]
> >> I think that this change removes the last use of G1BarrierEvac, so the
> >> code for that in G1ParCopyClosure::do_oop_work can probably be removed.
> >
> > I prepared a webrev that removes G1BarrierEvac related code too at
> >
> >   http://cr.openjdk.java.net/~tschatzl/8004687/webrev.1 (full)
> >   http://cr.openjdk.java.net/~tschatzl/8004687/webrev.0_to_1 (diff)
> 
> Looks good to me.
> 
> /Mikael

  thanks for the review,

  Thomas



From eric.caspole at oracle.com  Thu Jul 23 14:46:30 2015
From: eric.caspole at oracle.com (Eric Caspole)
Date: Thu, 23 Jul 2015 10:46:30 -0400
Subject: RFR: JDK-8132148 : G1 hs_err region dump legend out of sync with
	region values
Message-ID: <55B0FE46.5000204@oracle.com>

Hi everybody,
The other day we noticed the legend for the G1 regions in the hs_err 
file was out of sync with the actual values, here is a patch:

  http://cr.openjdk.java.net/~ecaspole/JDK-8132148/webrev/

For bug:
  https://bugs.openjdk.java.net/browse/JDK-8132148

Let me know what you think.
Thanks,
Eric


From mikael.gerdin at oracle.com  Thu Jul 23 15:03:59 2015
From: mikael.gerdin at oracle.com (Mikael Gerdin)
Date: Thu, 23 Jul 2015 17:03:59 +0200
Subject: RFR: JDK-8132148 : G1 hs_err region dump legend out of sync with
	region values
In-Reply-To: <55B0FE46.5000204@oracle.com>
References: <55B0FE46.5000204@oracle.com>
Message-ID: <55B1025F.3080305@oracle.com>

Hi Eric,

On 2015-07-23 16:46, Eric Caspole wrote:
> Hi everybody,
> The other day we noticed the legend for the G1 regions in the hs_err
> file was out of sync with the actual values, here is a patch:
>
>   http://cr.openjdk.java.net/~ecaspole/JDK-8132148/webrev/

Looks good.

/Mikael

>
> For bug:
>   https://bugs.openjdk.java.net/browse/JDK-8132148
>
> Let me know what you think.
> Thanks,
> Eric


From tom.benson at oracle.com  Thu Jul 23 15:06:38 2015
From: tom.benson at oracle.com (Tom Benson)
Date: Thu, 23 Jul 2015 11:06:38 -0400
Subject: RFR: JDK-8132148 : G1 hs_err region dump legend out of sync with
	region values
In-Reply-To: <55B1025F.3080305@oracle.com>
References: <55B0FE46.5000204@oracle.com> <55B1025F.3080305@oracle.com>
Message-ID: <55B102FE.2000502@oracle.com>

Looks good to me, too.
Tom

On 7/23/2015 11:03 AM, Mikael Gerdin wrote:
> Hi Eric,
>
> On 2015-07-23 16:46, Eric Caspole wrote:
>> Hi everybody,
>> The other day we noticed the legend for the G1 regions in the hs_err
>> file was out of sync with the actual values, here is a patch:
>>
>>   http://cr.openjdk.java.net/~ecaspole/JDK-8132148/webrev/
>
> Looks good.
>
> /Mikael
>
>>
>> For bug:
>>   https://bugs.openjdk.java.net/browse/JDK-8132148
>>
>> Let me know what you think.
>> Thanks,
>> Eric



From thomas.schatzl at oracle.com  Thu Jul 23 15:18:40 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 23 Jul 2015 17:18:40 +0200
Subject: RFR: JDK-8132148 : G1 hs_err region dump legend out of sync
	with region values
In-Reply-To: <55B0FE46.5000204@oracle.com>
References: <55B0FE46.5000204@oracle.com>
Message-ID: <1437664720.2347.85.camel@oracle.com>

Hi Eric,

On Thu, 2015-07-23 at 10:46 -0400, Eric Caspole wrote:
> Hi everybody,
> The other day we noticed the legend for the G1 regions in the hs_err 
> file was out of sync with the actual values, here is a patch:
> 
>   http://cr.openjdk.java.net/~ecaspole/JDK-8132148/webrev/
> 
> For bug:
>   https://bugs.openjdk.java.net/browse/JDK-8132148
> 
> Let me know what you think.

  looks good. If we want this to be more future-proof, we might want to
poke the results of HeapRegion::get_short_str() into the output string
instead of hardcoding them.

E.g. 

("Heap Regions: (%s=young(eden), %s=young(survivor), %s=old, "
               "%s=humongous(starts), %s=humongous(continues), "
[...]
    , HeapRegion::get_short_str(HeapRegionType::EdenTag), HeapRegion::get_short_str(HeapRegionType::SurvTage), ...

etc.

It's your call, as the others are fine with just hardcoding the values.

Thanks,
  Thomas




From derek.white at oracle.com  Thu Jul 23 16:09:01 2015
From: derek.white at oracle.com (Derek White)
Date: Thu, 23 Jul 2015 12:09:01 -0400
Subject: RFR: 8066821(S) Enhance command line processing to manage
	deprecating and obsoleting -XX command line arguments
In-Reply-To: <55ADA880.7090602@oracle.com>
References: <54B44661.8070007@oracle.com> <54B84E91.1050708@oracle.com>
	<F68EBF6B-38A5-4461-9ADD-41BB198FF943@oracle.com>
	<54D15A0B.6030501@oracle.com> <55A93BB3.4040402@oracle.com>
	<55ACBF85.9080404@oracle.com> <55AD2990.8030000@oracle.com>
	<55ADA880.7090602@oracle.com>
Message-ID: <55B1119D.4030802@oracle.com>

On 7/20/15 10:03 PM, David Holmes wrote:
> On 21/07/2015 3:02 AM, Derek White wrote:
>> Hi David,
>>
>> Thanks for looking this over.
>>
>> On 7/20/15 5:29 AM, David Holmes wrote:
>>> Hi Derek,
>>>
>>> Sorry but I'm finding this a bit confused and inconsistent. Comments
>>> below.
>>>
>>> On 18/07/2015 3:30 AM, Derek White wrote:
>>>> Request for review:
>>>>
>>>> [This updated webrev is being sent to wider audience, and has been
>>>> merged with Gerard's option constraints check-in. Also factored out
>>>> -XX:+AggressiveHeap processing into it's own chapter. I mean function
>>>> :-)]
>>>>
>>>> http://cr.openjdk.java.net/~drwhite/8066821/webrev.06/
>>>> https://bugs.openjdk.java.net/browse/JDK-8066821
>>>
>>> argument.cpp:
>>>
>>>  258  *               been not scheduled).
>>>
>>> -> not been scheduled
>>
>> OK.
>>>  260 *     OBSOLETE: An option may be removed (and deleted from
>>> globals.hpp), but still be accepted on the command
>>>  261  *               line. A warning is printed to let the user know
>>> that support may be removed in the future.
>>>
>>>
>>> Isn't this the stronger case that support has already been removed
>>> (the option does nothing) and it will be removed at the next major
>>> release. At the start of a major release we should be able to delete
>>> all entries from the obsolete table - else it wasn't obsolete but
>>> deprecated.
>>>
>>> Not sure "obsolete" is the right term here - obsolete things still
>>> function. For us an obsolete option does nothing (it exists only in
>>> the obsolete table).
>> It's not a great name, but that what the previous code called it. It's a
>> "I'll pretend you didn't say that" option, like when a teenager hears an
>> adult use out-of-date slang ("groovy!"). How about a "condescending"
>> option :-)
>
> Name aside you didn't comment on my main comment here: an obsolete 
> option has already been removed from globals etc and does nothing.

Ahh, OK. I must have been confusing in tense. I'll rewrite to be more 
direct.
>>>
>>>  276  * Tests:  Aliases are tested in VMAliasOptions.java.
>>>  277  *         Deprecated options are tested in
>>> VMDeprecatedOptions.java.
>>>  278  *         Obsolete options are tested in various files.
>>>
>>> We don't normally document this kind of thing in the source.
>>
>> I'm trying to head off unnecessary one-off test files for each alias and
>> deprecated option. For example TestNoParNew.java and
>> TestDefaultMaxRAMFraction.java. And I think that there should be one
>> test file for obsolete options also (perhaps expanding
>> ObsoleteFlagErrorMessage.java), instead of a bunch of separate test
>> files, but that is not in this webrev.
>
> Sounds fine but again we don't normally document test strategies in 
> the source code.
Normally I'd agree but I'm trying to change current practice of tests 
popping up on the corpses of dead or dying options like mushrooms in a 
forest.  I'm doubly adamant if I can add that last sentence to the 
comment :-)
>
>>>  281 // Obsolete or deprecated -XX flag.
>>>
>>> I can tell this is going to get confusing already.
>>>
>>>  284   JDK_Version obsoleted_in; // When the warning started (obsolete
>>> or deprecated).
>>>
>>> But there is a difference: deprecated == warn but still process;
>>> obsolete == warn and ignore.
>> Yes, but the SpecialFlag type is concerned with the common aspect of
>> warning. The "ignore" or "process" aspects are handled by the different
>> functions that look up the obsolete_jvm_flags and deprecated_jvm_flags
>> arrays.
>
> So that variable should really be obsoleted_or_deprecated_in ?

OK, I see now. It was right in front of me. Maybe "warning_started_in" 
is simpler.
>
>>>
>>>  288 // When a flag is eliminated, it can be added to this list in
>>> order to
>>>  289 // continue accepting this flag on the command-line, while
>>> issuing a warning
>>>  290 // and ignoring the value.  Once the JDK version reaches the
>>> 'accept_until'
>>>  291 // limit, we flatly refuse to admit the existence of the flag.
>>>  292 static SpecialFlag const obsolete_jvm_flags[] = {
>>>
>>> When a flag is eliminated it is gone from this table. As soon as the
>>> accept_until version is the current version we wipe the table of all
>>> such entries. This should be one of the first things done in a new
>>> release.
>>
>> I completely agree that this is a great plan. But until this April we
>> still had obsolete flags listed for JDK 5! The obsolete_jvm_flags table
>> did the right thing until the process caught up. In any case, this
>> webrev doesn't really change the behavior of the obsolete_jvm_flags 
>> table.
>
> But what you are seeing before April is the result of hotspot express 
> (at least in a large part). Now that we don't have to support that we 
> don't have legacy lists to maintain. The code could even be changed 
> upon detecting that current JDK version == "until" version to report 
> "Hey you forgot to update the table!" ;-)
OK, that history makes more sense.
> My point is that the comments should reflect how we intend to use 
> these, not give the impression that keeping eliminated options in the 
> table is the expected thing to do.

OK, I'll update the comments both here and up above to describe the 
presumably common "deprecated"->"obsolete" lifecycle.
>>>
>>> 320 // When a flag is deprecated, it can be added to this list in
>>> order to issuing a warning when the flag is used.
>>>  321 // Once the JDK version reaches the 'accept_until' limit, we
>>> flatly refuse to admit the existence of the flag.
>>>  322 static SpecialFlag const deprecated_jvm_flags[] = {
>>>
>>> A deprecated flag should be obsoleted before it reaches the
>>> accept_until limit.
>> That's a policy that's under discussion on hotspot-runtime-dev. There
>> are certainly option lifecycles that have been approved by our internal
>> process that don't follow this proposed policy. The mechanism in this
>> webrev was concerned about supporting the plethora of current
>> lifecycles, for better or worse.
>
> But again comments should reflect expected usage - if we reach the 
> "until" version of a deprecated option that wasn't obsoleted then 
> something has probably gone wrong.

OK.
>>> Which suggests that when we start a new version we wipe the obsoleted
>>> table and move the deprecated options into it.
>> I can add documentation to describe this case.
>>
>> If we decide that we'll always treat a deprecated or aliased option as
>> obsolete for one extra release, then it might make sense to have a
>> combined option lifecycle table. But for now I like the fact that
>> options in deprecated_jvm_flags should always have a real variable
>> defined in globals.hpp (etc), and options in obsolete_jvm_flags should
>> never have a global variable.
>
> Yes I like that too.
>>
>>> 776 case 1: {
>>>  777       if (warn) {
>>>  778         char version[256];
>>>  779         since.to_string(version, sizeof(version));
>>>  780         if (real_name != arg) {
>>>  781           warning("Option %s was deprecated in version %s and
>>> will likely be removed in a future release. Use option %s instead.",
>>>  782                   arg, version, real_name);
>>>  783         } else {
>>>  784           warning("Option %s was deprecated in version %s and
>>> will likely be removed in a future release.",
>>>  785                   arg, version);
>>>  786         }
>>>
>>> This isn't distinguishing between deprecated and obsoleted ???
>>
>> Yes, handle_aliases_and_deprecation() doesn't handle obsolete options
>> (or it would have had a longer name :-) Maybe it should handle that
>> case, but it would have complicated the control flow in the caller. I
>> have another proposed change in the works that simplifies the caller
>> quite a bit that would make the refactoring simpler.
>
> I need to think more on that. It is hard to see the overall control 
> flow for argument processing.

Yes it is hard to see. Currently options are parsed into tokens in a 
couple of places. It's roughly either:
   [+][-]arg
or:
   arg[:]=[value]

But not all of the code handles ":=". Furthermore option processing 
figures out the type of the argument by repeatedly trying to scanf 
"value" to see if it looks like floating point, or an integer, or maybe 
just text. I think we should first parsing out the arg name, get the 
Flag, and ask the Flag what type it is. This would make it much easier 
to handle aliases, deprecation, and obsoleting options in one place.

I have a workspace that does this but I suspect there could be slight 
differences in how errors would be reported.
>
> Thanks,
> David
Thanks for looking at this again. It's heading in the right direction!
  - Derek

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150723/8fb49ac3/attachment.htm>

From derek.white at oracle.com  Thu Jul 23 16:16:51 2015
From: derek.white at oracle.com (Derek White)
Date: Thu, 23 Jul 2015 12:16:51 -0400
Subject: RFR: 8066821(S) Enhance command line processing to manage
	deprecating and obsoleting -XX command line arguments
In-Reply-To: <3CBAE8E6-244C-49E7-A5ED-AA2A3C2D3AAD@oracle.com>
References: <54B44661.8070007@oracle.com> <54B84E91.1050708@oracle.com>
	<F68EBF6B-38A5-4461-9ADD-41BB198FF943@oracle.com>
	<54D15A0B.6030501@oracle.com> <55A93BB3.4040402@oracle.com>
	<3CBAE8E6-244C-49E7-A5ED-AA2A3C2D3AAD@oracle.com>
Message-ID: <55B11373.3010708@oracle.com>

Hi Kim,

On 7/21/15 6:21 PM, Kim Barrett wrote:
> On Jul 17, 2015, at 1:30 PM, Derek White <derek.white at oracle.com> wrote:
>> Request for review:
>>
>> [This updated webrev is being sent to wider audience, and has been merged with Gerard's option constraints check-in. Also factored out -XX:+AggressiveHeap processing into it's own chapter. I mean function :-)]
>>
>> http://cr.openjdk.java.net/~drwhite/8066821/webrev.06/
>> https://bugs.openjdk.java.net/browse/JDK-8066821
> ------------------------------------------------------------------------------
> src/share/vm/runtime/arguments.cpp
>   284   JDK_Version obsoleted_in; // When the warning started (obsolete or deprecated).
>
> "obsoleted_in" is confusingly named, given that it covers both the
> "obsolete" and the "deprecated" states.  I think other reviewers
> questioned whether "obsolete" was the proper term for that state.
>
> At the risk of bikeshedding, I did a little exploring with a
> thesaurus, and "discontinued" seems like a possibly better term for
> that state.  The obsoleted_in field could retain that name, as
> covering both deprecation and discontinuation.  Of course, that would
> require some further updates, such as renaming is_newly_obsolete, and
> updating various comments.
>
> Some other possibilities include "defunct" and "disused", but I like
> "discontinued" better than either of those.
I like "discontinued" better than "obsolete", but I think 
"warning_started_in" is more descriptive and doesn't define a new term.
> ------------------------------------------------------------------------------
> src/share/vm/runtime/arguments.cpp
>   288 // When a flag is eliminated, it can be added to this list in order to
>
> I think "is eliminated" => "is made obsolete" or something like that.
> I would expect "eliminated" == "removed", which is not what is being
> described here.

OK.
> ------------------------------------------------------------------------------
> src/share/vm/runtime/arguments.cpp
>   320 // When a flag is deprecated, it can be added to this list in order to issuing a warning when the flag is used.
>
> "issuing" => "issue"

OK
> ------------------------------------------------------------------------------
> src/share/vm/runtime/arguments.cpp
>   808       return false; // "name" is a deprecated option that has expired.
>
> I think the comment is wrong here.  I think it could just be a bogus
> option name.  Later calls don't have any corresponding comment (which
> is fine, just makes this one that I think might be wrong stand out
> more).

OK
> ------------------------------------------------------------------------------
> src/share/vm/runtime/arguments.hpp
>   420   // Returns true if the flag is obsolete and fits into the range specified
>   421   // for being ignored.  In the case the 'version' buffer is filled in with
>   422   // the version number when the flag became obsolete. Otherwise the flag has
>   423   // expired and should be ignored.
>   424   static bool is_newly_obsolete(const char* flag_name, JDK_Version* version);
>
> The "otherwise" part of the description is not correct.  If this
> returns false we don't actually know it has expired.  It could simply
> not be present in the set of obsolete options.

OK
> Also, why the "newly" in the name?  "is_obsolete_flag" would be
> consistent with "is_deprecated_flag".

That was pre-existing. I think they meant that "newly obsolete" options 
should get a warning, but "oldly obsolete" options don't. This webrev 
now defines "expired" to describe the later case though, so yes, maybe I 
should ditch the "newly".


Thanks Kim!

  - Derek


From kim.barrett at oracle.com  Thu Jul 23 17:00:51 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Thu, 23 Jul 2015 13:00:51 -0400
Subject: RFR: 8066821(S) Enhance command line processing to manage
	deprecating and obsoleting -XX command line arguments
In-Reply-To: <55B11373.3010708@oracle.com>
References: <54B44661.8070007@oracle.com> <54B84E91.1050708@oracle.com>
	<F68EBF6B-38A5-4461-9ADD-41BB198FF943@oracle.com>
	<54D15A0B.6030501@oracle.com> <55A93BB3.4040402@oracle.com>
	<3CBAE8E6-244C-49E7-A5ED-AA2A3C2D3AAD@oracle.com>
	<55B11373.3010708@oracle.com>
Message-ID: <7BF54F8F-6CBF-4454-A2AF-9215E356A373@oracle.com>

On Jul 23, 2015, at 12:16 PM, Derek White <derek.white at oracle.com> wrote:
> 
>> src/share/vm/runtime/arguments.cpp
>>  284   JDK_Version obsoleted_in; // When the warning started (obsolete or deprecated).
>> 
>> "obsoleted_in" is confusingly named, given that it covers both the
>> "obsolete" and the "deprecated" states.  I think other reviewers
>> questioned whether "obsolete" was the proper term for that state.
>> 
>> At the risk of bikeshedding, I did a little exploring with a
>> thesaurus, and "discontinued" seems like a possibly better term for
>> that state.  The obsoleted_in field could retain that name, as
>> covering both deprecation and discontinuation.  Of course, that would
>> require some further updates, such as renaming is_newly_obsolete, and
>> updating various comments.
>> 
>> Some other possibilities include "defunct" and "disused", but I like
>> "discontinued" better than either of those.
> I like "discontinued" better than "obsolete", but I think "warning_started_in" is more descriptive and doesn't define a new term.

That works for me.

>> src/share/vm/runtime/arguments.hpp
>>  420   // Returns true if the flag is obsolete and fits into the range specified
>>  421   // for being ignored.  In the case the 'version' buffer is filled in with
>>  422   // the version number when the flag became obsolete. Otherwise the flag has
>>  423   // expired and should be ignored.
>>  424   static bool is_newly_obsolete(const char* flag_name, JDK_Version* version);
>> 
>> The "otherwise" part of the description is not correct.  If this
>> returns false we don't actually know it has expired.  It could simply
>> not be present in the set of obsolete options.
> 
> OK
>> Also, why the "newly" in the name?  "is_obsolete_flag" would be
>> consistent with "is_deprecated_flag".
> 
> That was pre-existing. I think they meant that "newly obsolete" options should get a warning, but "oldly obsolete" options don't. This webrev now defines "expired" to describe the later case though, so yes, maybe I should ditch the "newly".

Please make the names for the deprecated and obsolete predicates consistent, whatever is done with ?newly?.

[And for the record, I don?t think we want an expired predicate, to distinguish that state from actually unknown options.]



From derek.white at oracle.com  Thu Jul 23 17:13:23 2015
From: derek.white at oracle.com (Derek White)
Date: Thu, 23 Jul 2015 13:13:23 -0400
Subject: RFR: 8066821(S) Enhance command line processing to manage
	deprecating and obsoleting -XX command line arguments
In-Reply-To: <7BF54F8F-6CBF-4454-A2AF-9215E356A373@oracle.com>
References: <54B44661.8070007@oracle.com> <54B84E91.1050708@oracle.com>
	<F68EBF6B-38A5-4461-9ADD-41BB198FF943@oracle.com>
	<54D15A0B.6030501@oracle.com> <55A93BB3.4040402@oracle.com>
	<3CBAE8E6-244C-49E7-A5ED-AA2A3C2D3AAD@oracle.com>
	<55B11373.3010708@oracle.com>
	<7BF54F8F-6CBF-4454-A2AF-9215E356A373@oracle.com>
Message-ID: <55B120B3.7020001@oracle.com>

On 7/23/15 1:00 PM, Kim Barrett wrote:
> On Jul 23, 2015, at 12:16 PM, Derek White <derek.white at oracle.com> wrote:
>>> src/share/vm/runtime/arguments.cpp
>>>   284   JDK_Version obsoleted_in; // When the warning started (obsolete or deprecated).
>>>
>>> "obsoleted_in" is confusingly named, given that it covers both the
>>> "obsolete" and the "deprecated" states.  I think other reviewers
>>> questioned whether "obsolete" was the proper term for that state.
>>>
>>> At the risk of bikeshedding, I did a little exploring with a
>>> thesaurus, and "discontinued" seems like a possibly better term for
>>> that state.  The obsoleted_in field could retain that name, as
>>> covering both deprecation and discontinuation.  Of course, that would
>>> require some further updates, such as renaming is_newly_obsolete, and
>>> updating various comments.
>>>
>>> Some other possibilities include "defunct" and "disused", but I like
>>> "discontinued" better than either of those.
>> I like "discontinued" better than "obsolete", but I think "warning_started_in" is more descriptive and doesn't define a new term.
> That works for me.
>
>>> src/share/vm/runtime/arguments.hpp
>>>   420   // Returns true if the flag is obsolete and fits into the range specified
>>>   421   // for being ignored.  In the case the 'version' buffer is filled in with
>>>   422   // the version number when the flag became obsolete. Otherwise the flag has
>>>   423   // expired and should be ignored.
>>>   424   static bool is_newly_obsolete(const char* flag_name, JDK_Version* version);
>>>
>>> The "otherwise" part of the description is not correct.  If this
>>> returns false we don't actually know it has expired.  It could simply
>>> not be present in the set of obsolete options.
>> OK
>>> Also, why the "newly" in the name?  "is_obsolete_flag" would be
>>> consistent with "is_deprecated_flag".
>> That was pre-existing. I think they meant that "newly obsolete" options should get a warning, but "oldly obsolete" options don't. This webrev now defines "expired" to describe the later case though, so yes, maybe I should ditch the "newly".
> Please make the names for the deprecated and obsolete predicates consistent, whatever is done with ?newly?.
>
> [And for the record, I don?t think we want an expired predicate, to distinguish that state from actually unknown options.]
>


OK. Thanks Kim!


From eric.caspole at oracle.com  Thu Jul 23 17:26:51 2015
From: eric.caspole at oracle.com (Eric Caspole)
Date: Thu, 23 Jul 2015 13:26:51 -0400
Subject: RFR: JDK-8132148 : G1 hs_err region dump legend out of sync with
	region values
In-Reply-To: <1437664720.2347.85.camel@oracle.com>
References: <55B0FE46.5000204@oracle.com> <1437664720.2347.85.camel@oracle.com>
Message-ID: <55B123DB.2030905@oracle.com>

Hi Thomas,
Since the Tag enum is private and not used anywhere else in the code, 
I'll leave it like this. Otherwise I'd call something in HeapRegionType 
for the legend but it seems like overkill.
Eric


On 07/23/2015 11:18 AM, Thomas Schatzl wrote:
> Hi Eric,
>
> On Thu, 2015-07-23 at 10:46 -0400, Eric Caspole wrote:
>> Hi everybody,
>> The other day we noticed the legend for the G1 regions in the hs_err
>> file was out of sync with the actual values, here is a patch:
>>
>>    http://cr.openjdk.java.net/~ecaspole/JDK-8132148/webrev/
>>
>> For bug:
>>    https://bugs.openjdk.java.net/browse/JDK-8132148
>>
>> Let me know what you think.
>
>    looks good. If we want this to be more future-proof, we might want to
> poke the results of HeapRegion::get_short_str() into the output string
> instead of hardcoding them.
>
> E.g.
>
> ("Heap Regions: (%s=young(eden), %s=young(survivor), %s=old, "
>                 "%s=humongous(starts), %s=humongous(continues), "
> [...]
>      , HeapRegion::get_short_str(HeapRegionType::EdenTag), HeapRegion::get_short_str(HeapRegionType::SurvTage), ...
>
> etc.
>
> It's your call, as the others are fine with just hardcoding the values.
>
> Thanks,
>    Thomas
>
>


From thomas.schatzl at oracle.com  Thu Jul 23 18:30:58 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 23 Jul 2015 20:30:58 +0200
Subject: RFR: JDK-8132148 : G1 hs_err region dump legend out of sync
	with region values
In-Reply-To: <55B123DB.2030905@oracle.com>
References: <55B0FE46.5000204@oracle.com>
	<1437664720.2347.85.camel@oracle.com> <55B123DB.2030905@oracle.com>
Message-ID: <1437676258.4047.1.camel@oracle.com>

Hi Eric,

On Thu, 2015-07-23 at 13:26 -0400, Eric Caspole wrote:
> Hi Thomas,
> Since the Tag enum is private and not used anywhere else in the code, 
> I'll leave it like this. Otherwise I'd call something in HeapRegionType 
> for the legend but it seems like overkill.
> Eric
> 

  that's okay with me.

Thanks,
  Thomas




From eric.caspole at oracle.com  Thu Jul 23 19:54:34 2015
From: eric.caspole at oracle.com (Eric Caspole)
Date: Thu, 23 Jul 2015 15:54:34 -0400
Subject: RFR: JDK-8132148 : G1 hs_err region dump legend out of sync with
	region values
In-Reply-To: <1437676258.4047.1.camel@oracle.com>
References: <55B0FE46.5000204@oracle.com>	
	<1437664720.2347.85.camel@oracle.com> <55B123DB.2030905@oracle.com>
	<1437676258.4047.1.camel@oracle.com>
Message-ID: <55B1467A.40505@oracle.com>

Thanks Thomas,
Eric

On 7/23/2015 2:30 PM, Thomas Schatzl wrote:
> Hi Eric,
>
> On Thu, 2015-07-23 at 13:26 -0400, Eric Caspole wrote:
>> Hi Thomas,
>> Since the Tag enum is private and not used anywhere else in the code,
>> I'll leave it like this. Otherwise I'd call something in HeapRegionType
>> for the legend but it seems like overkill.
>> Eric
>>
>    that's okay with me.
>
> Thanks,
>    Thomas
>
>



From dmitry.fazunenko at oracle.com  Fri Jul 24 16:23:05 2015
From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko)
Date: Fri, 24 Jul 2015 19:23:05 +0300
Subject: RFR (XXS): 8132334: [TESTBUG]
	gc/metaspace/TestCapacityUntilGCWrapAround.java Compilation failed
Message-ID: <55B26669.50302@oracle.com>

Hi everyone,

Can I have reviews for the trivial change that must fix the compilation 
problem?
https://bugs.openjdk.java.net/browse/JDK-8132334

webrev:
http://cr.openjdk.java.net/~dfazunen/8132334/webrev/

Comments: In some circumstances jtreg doesn't see wb classes if they 
weren't built explicitly.

Thanks,
Dima


From kim.barrett at oracle.com  Fri Jul 24 20:41:28 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Fri, 24 Jul 2015 16:41:28 -0400
Subject: RFR: 8079082: VerifyNoCSetOopsClosure is derived twice from Closure
Message-ID: <558F083C-B528-4356-B732-4B76624424E1@oracle.com>

Please review this cleanup to eliminate improper and unnecessary
multiple inheritance by a verification closure.

- Changed the closure to an ordinary function object, with some
simplifications, elimination of virtual functions, and renaming as a
result.

- Eliminated VerifyNoCSetOopsPhase enum, instead just making the calls
to setup the checking function pass in the appropriate string.  The
only thing the enum values were used for was to determine the desired
string.

- Replaced CMMarkStack::oops_do with iterate function template.

- Replaced GenericTaskQueue::oops_do with iterate function template.
This class really has no business having a hard-wired iterator over
oop elements.

- While I was in the neighborhood, eliminated an unnecessary nearby
cast.

CR:
https://bugs.openjdk.java.net/browse/JDK-8079082

Webrev:
http://cr.openjdk.java.net/~kbarrett/8079082/webrev.00/

Testing:
jprt, local jtreg



From abhijangda at hotmail.com  Sat Jul 25 20:30:37 2015
From: abhijangda at hotmail.com (Abhinav Jangda)
Date: Sun, 26 Jul 2015 02:00:37 +0530
Subject: Entry function of GC Collection
Message-ID: <BLU184-W47FBB70FEF6277610243ACA800@phx.gbl>

Hello all,

I have been working on a project which requires me to change GC Algorithm in Java Hotspot VM. I have been trying to understand the source code of GC in Hotspot VM. I know a couple of details about GC in Hotspot now. But I am not able to find the start function of GC Collection. I think VM_GenCollectFull::doit and VM_GenCollectForAllocation::doit() are the two functions called for GC. But if I add std::cout statements at the start of these functions and start the Dacapo Benchmarks I couldn't see any result produced by std::cout statements. I want to work with SerialGC.
Here is the command I used to run Hotspot:

./java -XX:+UseSerialGC -jar dacapo.jar jython

OpenJDK version is 8 and I am compiling building openjdk using following command

make CONF=linux-x86_64-normal-server-release

It will be great if anyone of you could help me.

Thank You,
Abhinav
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150726/1207c0cf/attachment.htm>

From aph at redhat.com  Mon Jul 27 07:56:20 2015
From: aph at redhat.com (Andrew Haley)
Date: Mon, 27 Jul 2015 08:56:20 +0100
Subject: Entry function of GC Collection
In-Reply-To: <BLU184-W47FBB70FEF6277610243ACA800@phx.gbl>
References: <BLU184-W47FBB70FEF6277610243ACA800@phx.gbl>
Message-ID: <55B5E424.3080501@redhat.com>

On 25/07/15 21:30, Abhinav Jangda wrote:
> It will be great if anyone of you could help me.

Please forgive me if I'm telling you things to already know, but:

This question is one you could answer very quickly if you were using a
debugger, so I am assuming that for some reason you don't use one.

Run the debug build of HotSpot inside a debugger.  Put a breakpoint on
the functions you care about.  Step through.  Trying to debug
"printf-style" doesn't help; learning to use a debugger will take some
time, but it will repay your effort many, many times over.

Andrew.


From mikael.gerdin at oracle.com  Mon Jul 27 12:25:36 2015
From: mikael.gerdin at oracle.com (Mikael Gerdin)
Date: Mon, 27 Jul 2015 14:25:36 +0200
Subject: RFR: 8079082: VerifyNoCSetOopsClosure is derived twice from
	Closure
In-Reply-To: <558F083C-B528-4356-B732-4B76624424E1@oracle.com>
References: <558F083C-B528-4356-B732-4B76624424E1@oracle.com>
Message-ID: <55B62340.6040501@oracle.com>

Hi Kim,

On 2015-07-24 22:41, Kim Barrett wrote:
> Please review this cleanup to eliminate improper and unnecessary
> multiple inheritance by a verification closure.
>
> - Changed the closure to an ordinary function object, with some
> simplifications, elimination of virtual functions, and renaming as a
> result.

Nice.

>
> - Eliminated VerifyNoCSetOopsPhase enum, instead just making the calls
> to setup the checking function pass in the appropriate string.  The
> only thing the enum values were used for was to determine the desired
> string.
>
> - Replaced CMMarkStack::oops_do with iterate function template.
>
> - Replaced GenericTaskQueue::oops_do with iterate function template.
> This class really has no business having a hard-wired iterator over
> oop elements.

It appears that you've adapted the assertion is_oop_or_null to an is_oop 
check in the function object.
I believe that the assertion that all oops on the global CM stack should 
pass the is_oop check but I guess we will find out if that's untrue :)



>
> - While I was in the neighborhood, eliminated an unnecessary nearby
> cast.
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8079082
>
> Webrev:
> http://cr.openjdk.java.net/~kbarrett/8079082/webrev.00/

Looks good to me.
/Mikael

>
> Testing:
> jprt, local jtreg
>


From eric.caspole at oracle.com  Mon Jul 27 13:55:03 2015
From: eric.caspole at oracle.com (Eric Caspole)
Date: Mon, 27 Jul 2015 09:55:03 -0400
Subject: RFR: 8079082: VerifyNoCSetOopsClosure is derived twice from
	Closure
In-Reply-To: <558F083C-B528-4356-B732-4B76624424E1@oracle.com>
References: <558F083C-B528-4356-B732-4B76624424E1@oracle.com>
Message-ID: <55B63837.9070408@oracle.com>

Hi Kim,
Looks good.
Eric


On 07/24/2015 04:41 PM, Kim Barrett wrote:
> Please review this cleanup to eliminate improper and unnecessary
> multiple inheritance by a verification closure.
>
> - Changed the closure to an ordinary function object, with some
> simplifications, elimination of virtual functions, and renaming as a
> result.
>
> - Eliminated VerifyNoCSetOopsPhase enum, instead just making the calls
> to setup the checking function pass in the appropriate string.  The
> only thing the enum values were used for was to determine the desired
> string.
>
> - Replaced CMMarkStack::oops_do with iterate function template.
>
> - Replaced GenericTaskQueue::oops_do with iterate function template.
> This class really has no business having a hard-wired iterator over
> oop elements.
>
> - While I was in the neighborhood, eliminated an unnecessary nearby
> cast.
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8079082
>
> Webrev:
> http://cr.openjdk.java.net/~kbarrett/8079082/webrev.00/
>
> Testing:
> jprt, local jtreg
>


From michail.chernov at oracle.com  Mon Jul 27 16:38:00 2015
From: michail.chernov at oracle.com (Michail Chernov)
Date: Mon, 27 Jul 2015 19:38:00 +0300
Subject: RFR: 8131343: Remove unused imports from
	hotspot/test/testlibrary/jdk/test/lib/*.java
In-Reply-To: <1437467323.2070.0.camel@oracle.com>
References: <55A677E7.3090405@oracle.com> <1437467323.2070.0.camel@oracle.com>
Message-ID: <55B65E68.7080900@oracle.com>

Thanks, Thomas!

Michail

On 21.07.2015 11:28, Thomas Schatzl wrote:
> Hi,
>
> On Wed, 2015-07-15 at 18:10 +0300, Michail Chernov wrote:
>> Hi,
>>
>> Could you please review this small patch to
>> hotspot/test/testlibrary/jdk/test/lib/*.java which removes unused import
>> and fixes spelling.
>>
>> http://cr.openjdk.java.net/~mchernov/8131343/webrev.00/
>> https://bugs.openjdk.java.net/browse/JDK-8131343
>>
>    looks good.
>
> Thomas
>
>
>



From kim.barrett at oracle.com  Mon Jul 27 18:14:50 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Mon, 27 Jul 2015 14:14:50 -0400
Subject: RFR: 8079082: VerifyNoCSetOopsClosure is derived twice from
	Closure
In-Reply-To: <55B62340.6040501@oracle.com>
References: <558F083C-B528-4356-B732-4B76624424E1@oracle.com>
	<55B62340.6040501@oracle.com>
Message-ID: <BA0CEA86-103C-4096-83C4-38E8F2CB132D@oracle.com>

On Jul 27, 2015, at 8:25 AM, Mikael Gerdin <mikael.gerdin at oracle.com> wrote:
> 
>> - Replaced GenericTaskQueue::oops_do with iterate function template.
>> This class really has no business having a hard-wired iterator over
>> oop elements.
> 
> It appears that you've adapted the assertion is_oop_or_null to an is_oop check in the function object.
> I believe that the assertion that all oops on the global CM stack should pass the is_oop check but I guess we will find out if that's untrue :)

Sorry, I should have mentioned this in the review description.

Yes, that change was made.  We (intentionally) never add NULLs to the mark stack / queues - see deal_with_reference.  They need to be filtered out at one end or the other; pre-filtering makes more sense, to eliminate unnecessary mark stack usage.

>> CR:
>> https://bugs.openjdk.java.net/browse/JDK-8079082
>> 
>> Webrev:
>> http://cr.openjdk.java.net/~kbarrett/8079082/webrev.00/
> 
> Looks good to me.
> /Mikael

Thanks for your review.



From kim.barrett at oracle.com  Mon Jul 27 18:15:22 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Mon, 27 Jul 2015 14:15:22 -0400
Subject: RFR: 8079082: VerifyNoCSetOopsClosure is derived twice from
	Closure
In-Reply-To: <55B63837.9070408@oracle.com>
References: <558F083C-B528-4356-B732-4B76624424E1@oracle.com>
	<55B63837.9070408@oracle.com>
Message-ID: <382F4587-50FD-4B44-998D-EAAC1ECDE859@oracle.com>

On Jul 27, 2015, at 9:55 AM, Eric Caspole <eric.caspole at oracle.com> wrote:
> 
> Hi Kim,
> Looks good.
> Eric

Thanks.



From jon.masamitsu at oracle.com  Mon Jul 27 18:28:46 2015
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Mon, 27 Jul 2015 11:28:46 -0700
Subject: Entry function of GC Collection
In-Reply-To: <BLU184-W47FBB70FEF6277610243ACA800@phx.gbl>
References: <BLU184-W47FBB70FEF6277610243ACA800@phx.gbl>
Message-ID: <55B6785E.5050303@oracle.com>

You're perhaps looking for the collect() methods.

For example,

void TenuredGeneration::collect()

and

void DefNewGeneration::collect()

Jon

On 07/25/2015 01:30 PM, Abhinav Jangda wrote:
> Hello all,
>
> I have been working on a project which requires me to change GC 
> Algorithm in Java Hotspot VM. I have been trying to understand the 
> source code of GC in Hotspot VM. I know a couple of details about GC 
> in Hotspot now. But I am not able to find the start function of GC 
> Collection. I think /VM_GenCollectFull::doit/ and 
> /VM_GenCollectForAllocation::doit() /are the two functions called for 
> GC. But if I add /std::cout/ statements at the start of these 
> functions and start the Dacapo Benchmarks I couldn't see any result 
> produced by /std::cout/ statements. I want to work with SerialGC.
> Here is the command I used to run Hotspot:
>
> ./java -XX:+UseSerialGC -jar dacapo.jar jython
>
> OpenJDK version is 8 and I am compiling building openjdk using 
> following command
>
> make CONF=linux-x86_64-normal-server-release
>
> It will be great if anyone of you could help me.
>
> Thank You,
> Abhinav

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150727/708dff86/attachment.htm>

From kim.barrett at oracle.com  Mon Jul 27 18:49:21 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Mon, 27 Jul 2015 14:49:21 -0400
Subject: RFR (XXS): 8132334: [TESTBUG]
	gc/metaspace/TestCapacityUntilGCWrapAround.java Compilation failed
In-Reply-To: <55B26669.50302@oracle.com>
References: <55B26669.50302@oracle.com>
Message-ID: <F4E156C3-967F-461B-AB99-D2D08AC970A8@oracle.com>

On Jul 24, 2015, at 12:23 PM, Dmitry Fazunenko <dmitry.fazunenko at oracle.com> wrote:
> 
> Hi everyone,
> 
> Can I have reviews for the trivial change that must fix the compilation problem?
> https://bugs.openjdk.java.net/browse/JDK-8132334
> 
> webrev:
> http://cr.openjdk.java.net/~dfazunen/8132334/webrev/
> 
> Comments: In some circumstances jtreg doesn't see wb classes if they weren't built explicitly.

Under what circumstances does this occur?  I see other tests that look very similar to the pre-change version of this test - what makes this test so special?



From dmitry.fazunenko at oracle.com  Tue Jul 28 01:12:49 2015
From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko)
Date: Tue, 28 Jul 2015 04:12:49 +0300
Subject: RFR (XXS): 8132334: [TESTBUG]
	gc/metaspace/TestCapacityUntilGCWrapAround.java Compilation failed
In-Reply-To: <F4E156C3-967F-461B-AB99-D2D08AC970A8@oracle.com>
References: <55B26669.50302@oracle.com>
	<F4E156C3-967F-461B-AB99-D2D08AC970A8@oracle.com>
Message-ID: <55B6D711.7030302@oracle.com>

Hi Kim,

On 27.07.2015 21:49, Kim Barrett wrote:
> On Jul 24, 2015, at 12:23 PM, Dmitry Fazunenko <dmitry.fazunenko at oracle.com> wrote:
>> Hi everyone,
>>
>> Can I have reviews for the trivial change that must fix the compilation problem?
>> https://bugs.openjdk.java.net/browse/JDK-8132334
>>
>> webrev:
>> http://cr.openjdk.java.net/~dfazunen/8132334/webrev/
>>
>> Comments: In some circumstances jtreg doesn't see wb classes if they weren't built explicitly.
> Under what circumstances does this occur?  I see other tests that look very similar to the pre-change version of this test - what makes this test so special?
>
I don't have an answer. I just heard about such problem, but I didn't 
find the bug. Might be because it's not filed yet. The hypothesis is the 
concurrency might cause such effect.

Thanks,
Dima





From kim.barrett at oracle.com  Tue Jul 28 01:40:10 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Mon, 27 Jul 2015 21:40:10 -0400
Subject: RFR (XXS): 8132334: [TESTBUG]
	gc/metaspace/TestCapacityUntilGCWrapAround.java Compilation failed
In-Reply-To: <55B6D711.7030302@oracle.com>
References: <55B26669.50302@oracle.com>
	<F4E156C3-967F-461B-AB99-D2D08AC970A8@oracle.com>
	<55B6D711.7030302@oracle.com>
Message-ID: <070EFF29-34DA-4BA8-BAF9-7473A0A53800@oracle.com>

On Jul 27, 2015, at 9:12 PM, Dmitry Fazunenko <dmitry.fazunenko at oracle.com> wrote:
> 
> On 27.07.2015 21:49, Kim Barrett wrote:
>> On Jul 24, 2015, at 12:23 PM, Dmitry Fazunenko <dmitry.fazunenko at oracle.com> wrote:
>>> Hi everyone,
>>> 
>>> Can I have reviews for the trivial change that must fix the compilation problem?
>>> https://bugs.openjdk.java.net/browse/JDK-8132334
>>> 
>>> webrev:
>>> http://cr.openjdk.java.net/~dfazunen/8132334/webrev/
>>> 
>>> Comments: In some circumstances jtreg doesn't see wb classes if they weren't built explicitly.
>> Under what circumstances does this occur?  I see other tests that look very similar to the pre-change version of this test - what makes this test so special?
>> 
> I don't have an answer. I just heard about such problem, but I didn't find the bug. Might be because it's not filed yet. The hypothesis is the concurrency might cause such effect.

Without an understanding of why this change is needed here but not
elsewhere, I'm concerned that we're either

1. papering over one instance of a more widespread problem, or
2. not really solving the problem with this test after all.



From thomas.schatzl at oracle.com  Tue Jul 28 08:46:14 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Tue, 28 Jul 2015 10:46:14 +0200
Subject: RFR (M): 8073052: Rename and clean up the allocation manager
	hierarchy in g1Allocator.?pp
Message-ID: <1438073174.2272.17.camel@oracle.com>

Hi all,

  can I have reviews for this change that cleans up G1Allocator related
code in preparation for actual PLAB sizing changes?

The change tries to decrease the number of dependencies between
g1allocator, g1collectedheap and g1allocregion.

It is motivated by existing circular calls between the classes,
particularly G1CollectedHeap and G1Allocator.

E.g. G1Allocator::allocate_direct_or_plab() calls
G1CollectedHeap::par_allocate_during_gc(), which after calling a few
helper functions calls G1Allocator::old/survivor_gc_alloc_region() and a
few G1AllocRegion methods.

This change makes the procedure much more straightforward by moving the
methods that deal with actual allocation (not policy) into G1Allocator,
allowing us to hide G1AllocRegion from G1CollectedHeap completely.
(G1AllocRegion is a helper class anyway).

Further, this moving around of methods showed that there is an include
cycle between g1CollectedHeap.inline.hpp, i.e.
G1CollectedHeap::obj_in_cs() referencing HeapRegion::in_collection_set()
in heapRegion.inline.hpp that references G1CollectedHeap::is_in_cset()
in g1CollectedHeap.inline.hpp again.

For whatever reason compilers do not complain, but moving methods made
that show up.

This has been fixed by moving more method implementatations into
the .inline.hpp files, and putting the implementation of
G1CollectedHeap::obj_in_cs() from the g1CollectedHeap.hpp into the .cpp
file.

Finally, the change aligns the naming of G1ParGCAllocator with other
classes that use "PLAB" in their names instead of "ParGC".

CR:
https://bugs.openjdk.java.net/browse/JDK-8073052

Webrev:
http://cr.openjdk.java.net/~tschatzl/8073052/webrev.3/

Testing:
jprt

Note that there has already been an RFR with that number earlier,
however at that time a lot of other changes in this area were done by
others, and the problems with these changes have only been fixed a few
weeks ago.

Thanks,
  Thomas



From mikael.gerdin at oracle.com  Tue Jul 28 12:42:12 2015
From: mikael.gerdin at oracle.com (Mikael Gerdin)
Date: Tue, 28 Jul 2015 14:42:12 +0200
Subject: RFR (M): 8073052: Rename and clean up the allocation manager
	hierarchy in g1Allocator.?pp
In-Reply-To: <1438073174.2272.17.camel@oracle.com>
References: <1438073174.2272.17.camel@oracle.com>
Message-ID: <55B778A4.4050602@oracle.com>

Hi Thomas,

On 2015-07-28 10:46, Thomas Schatzl wrote:
> Hi all,
>
>    can I have reviews for this change that cleans up G1Allocator related
> code in preparation for actual PLAB sizing changes?
>
> The change tries to decrease the number of dependencies between
> g1allocator, g1collectedheap and g1allocregion.
>
> It is motivated by existing circular calls between the classes,
> particularly G1CollectedHeap and G1Allocator.
>
> E.g. G1Allocator::allocate_direct_or_plab() calls
> G1CollectedHeap::par_allocate_during_gc(), which after calling a few
> helper functions calls G1Allocator::old/survivor_gc_alloc_region() and a
> few G1AllocRegion methods.
>
> This change makes the procedure much more straightforward by moving the
> methods that deal with actual allocation (not policy) into G1Allocator,
> allowing us to hide G1AllocRegion from G1CollectedHeap completely.
> (G1AllocRegion is a helper class anyway).
>
> Further, this moving around of methods showed that there is an include
> cycle between g1CollectedHeap.inline.hpp, i.e.
> G1CollectedHeap::obj_in_cs() referencing HeapRegion::in_collection_set()
> in heapRegion.inline.hpp that references G1CollectedHeap::is_in_cset()
> in g1CollectedHeap.inline.hpp again.
>
> For whatever reason compilers do not complain, but moving methods made
> that show up.
>
> This has been fixed by moving more method implementatations into
> the .inline.hpp files, and putting the implementation of
> G1CollectedHeap::obj_in_cs() from the g1CollectedHeap.hpp into the .cpp
> file.
>
> Finally, the change aligns the naming of G1ParGCAllocator with other
> classes that use "PLAB" in their names instead of "ParGC".
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8073052
>
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8073052/webrev.3/

Overall a nice cleanup!

g1Allocator.cpp:
In unsafe_max_tlab_alloc you get the heap via G1H::heap() instead of _g1h.

G1PLABAllocator seems to look up the G1H::heap() fairly often,
did you consider passing on the G1H* to the G1PLABAllocator to reduce 
the clutter from accessing through the static getter in multiple 
locations? Another option could be to expose the G1H* from G1Allocator, 
yielding _allocator->heap()->...
instead of _g1h->...

otherwise it looks good to me.

/Mikael


>
> Testing:
> jprt
>
> Note that there has already been an RFR with that number earlier,
> however at that time a lot of other changes in this area were done by
> others, and the problems with these changes have only been fixed a few
> weeks ago.
>
> Thanks,
>    Thomas
>



From dmitry.fazunenko at oracle.com  Tue Jul 28 18:07:50 2015
From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko)
Date: Tue, 28 Jul 2015 21:07:50 +0300
Subject: RFR (XXS): 8132334: [TESTBUG]
	gc/metaspace/TestCapacityUntilGCWrapAround.java Compilation failed
In-Reply-To: <070EFF29-34DA-4BA8-BAF9-7473A0A53800@oracle.com>
References: <55B26669.50302@oracle.com>
	<F4E156C3-967F-461B-AB99-D2D08AC970A8@oracle.com>
	<55B6D711.7030302@oracle.com>
	<070EFF29-34DA-4BA8-BAF9-7473A0A53800@oracle.com>
Message-ID: <55B7C4F6.9000100@oracle.com>



On 28.07.2015 4:40, Kim Barrett wrote:
> On Jul 27, 2015, at 9:12 PM, Dmitry Fazunenko <dmitry.fazunenko at oracle.com> wrote:
>> On 27.07.2015 21:49, Kim Barrett wrote:
>>> On Jul 24, 2015, at 12:23 PM, Dmitry Fazunenko <dmitry.fazunenko at oracle.com> wrote:
>>>> Hi everyone,
>>>>
>>>> Can I have reviews for the trivial change that must fix the compilation problem?
>>>> https://bugs.openjdk.java.net/browse/JDK-8132334
>>>>
>>>> webrev:
>>>> http://cr.openjdk.java.net/~dfazunen/8132334/webrev/
>>>>
>>>> Comments: In some circumstances jtreg doesn't see wb classes if they weren't built explicitly.
>>> Under what circumstances does this occur?  I see other tests that look very similar to the pre-change version of this test - what makes this test so special?
>>>
>> I don't have an answer. I just heard about such problem, but I didn't find the bug. Might be because it's not filed yet. The hypothesis is the concurrency might cause such effect.
> Without an understanding of why this change is needed here but not
> elsewhere, I'm concerned that we're either
>
> 1. papering over one instance of a more widespread problem, or
> 2. not really solving the problem with this test after all.
>
I found the jtreg bug: 
https://bugs.openjdk.java.net/browse/CODETOOLS-7900024
It's about 3 years old and still un-triaged, so I won't expect it will 
be fixed soon.
But the problem could be easily worked around and this change does it.

Thanks,
Dima





From thomas.schatzl at oracle.com  Wed Jul 29 07:51:21 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 29 Jul 2015 09:51:21 +0200
Subject: RFR (M): 8073052: Rename and clean up the allocation manager
	hierarchy in g1Allocator.?pp
In-Reply-To: <55B778A4.4050602@oracle.com>
References: <1438073174.2272.17.camel@oracle.com> <55B778A4.4050602@oracle.com>
Message-ID: <1438156281.2302.16.camel@oracle.com>

Hi Mikael,

  thanks for the review.

On Tue, 2015-07-28 at 14:42 +0200, Mikael Gerdin wrote:
> Hi Thomas,
> 
> On 2015-07-28 10:46, Thomas Schatzl wrote:
> > Hi all,
> [...]
> > Finally, the change aligns the naming of G1ParGCAllocator with other
> > classes that use "PLAB" in their names instead of "ParGC".
> >
> > CR:
> > https://bugs.openjdk.java.net/browse/JDK-8073052
> >
> > Webrev:
> > http://cr.openjdk.java.net/~tschatzl/8073052/webrev.3/
> 
> Overall a nice cleanup!
> 
> g1Allocator.cpp:
> In unsafe_max_tlab_alloc you get the heap via G1H::heap() instead of _g1h.

Fixed.

> G1PLABAllocator seems to look up the G1H::heap() fairly often,
> did you consider passing on the G1H* to the G1PLABAllocator to reduce 
> the clutter from accessing through the static getter in multiple 
> locations? Another option could be to expose the G1H* from G1Allocator, 
> yielding _allocator->heap()->...
> instead of _g1h->...

Fixed.

New webrevs:
http://cr.openjdk.java.net/~tschatzl/8073052/webrev.4/ (full)
http://cr.openjdk.java.net/~tschatzl/8073052/webrev.3_to_4/ (diff)

Thanks,
  Thomas




From kim.barrett at oracle.com  Wed Jul 29 07:57:28 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Wed, 29 Jul 2015 03:57:28 -0400
Subject: RFR: 8132306: java/lang/ref/ReferenceEnqueue.java fails with
	"RuntimeException: Error: poll() returned null; expected ref object"
Message-ID: <BA1C0BEB-F342-437A-87E7-029660BB5348@oracle.com>

Please review this fix of a race condition in
j.l.r.Reference/ReferenceQueue.  See comments in the bug report for a
description of the race.  The race is being fixed by reordering a pair
of volatile assignments.

CR:
https://bugs.openjdk.java.net/browse/JDK-8132306

Webrev:
http://cr.openjdk.java.net/~kbarrett/8132306/webrev.00/

Testing:
jprt with default and hotspot testsets

Locally tested race in original code by insertion of sleep in enqueue and
verifying ReferenceEnqueue regression test fails consistently as
described in the bug report.

Locally tested fixed code by insertion of sleeps at various points and
verifying ReferenceEnqueue regression test still passes.



From thomas.schatzl at oracle.com  Wed Jul 29 07:59:41 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 29 Jul 2015 09:59:41 +0200
Subject: RFR: 8132306: java/lang/ref/ReferenceEnqueue.java fails with
	"RuntimeException: Error: poll() returned null; expected ref object"
In-Reply-To: <BA1C0BEB-F342-437A-87E7-029660BB5348@oracle.com>
References: <BA1C0BEB-F342-437A-87E7-029660BB5348@oracle.com>
Message-ID: <1438156781.2302.17.camel@oracle.com>

Hi Kim,

On Wed, 2015-07-29 at 03:57 -0400, Kim Barrett wrote:
> Please review this fix of a race condition in
> j.l.r.Reference/ReferenceQueue.  See comments in the bug report for a
> description of the race.  The race is being fixed by reordering a pair
> of volatile assignments.
> 
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8132306
> 
> Webrev:
> http://cr.openjdk.java.net/~kbarrett/8132306/webrev.00/
> 
> Testing:
> jprt with default and hotspot testsets
> 
> Locally tested race in original code by insertion of sleep in enqueue and
> verifying ReferenceEnqueue regression test fails consistently as
> described in the bug report.
> 
> Locally tested fixed code by insertion of sleeps at various points and
> verifying ReferenceEnqueue regression test still passes.

  looks good.

Thanks for taking care of this so quickly,
  Thomas




From david.holmes at oracle.com  Wed Jul 29 08:32:43 2015
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 29 Jul 2015 18:32:43 +1000
Subject: RFR: 8132306: java/lang/ref/ReferenceEnqueue.java fails with
	"RuntimeException: Error: poll() returned null; expected ref object"
In-Reply-To: <BA1C0BEB-F342-437A-87E7-029660BB5348@oracle.com>
References: <BA1C0BEB-F342-437A-87E7-029660BB5348@oracle.com>
Message-ID: <55B88FAB.4080302@oracle.com>

Hi Kim,

On 29/07/2015 5:57 PM, Kim Barrett wrote:
> Please review this fix of a race condition in
> j.l.r.Reference/ReferenceQueue.  See comments in the bug report for a
> description of the race.  The race is being fixed by reordering a pair
> of volatile assignments.

While this seems logical for the failure at hand it isn't immediately 
obvious to me that setting next before setting the ENQUEUED state won't 
cause a problem for some other piece of code. Really these things need 
to be tightly synchronized - I think the real bug is the unsynchronized 
fast-path for the empty-queue case in poll(). While that change was 
deliberate in 6666739 this side-effect was not realized and would have 
affected the change. I hope Martin and/or Tom see this and chime in.

That aside the commentary is rather verbose, a simple:

// Only set ENQUEUED state after the reference is enqueued

would suffice (and add "volatiles ensure ordering" if you want that to 
be clearer).

Thanks,
David

> CR:
> https://bugs.openjdk.java.net/browse/JDK-8132306
>
> Webrev:
> http://cr.openjdk.java.net/~kbarrett/8132306/webrev.00/
>
> Testing:
> jprt with default and hotspot testsets
>
> Locally tested race in original code by insertion of sleep in enqueue and
> verifying ReferenceEnqueue regression test fails consistently as
> described in the bug report.
>
> Locally tested fixed code by insertion of sleeps at various points and
> verifying ReferenceEnqueue regression test still passes.
>


From mikael.gerdin at oracle.com  Wed Jul 29 09:04:36 2015
From: mikael.gerdin at oracle.com (Mikael Gerdin)
Date: Wed, 29 Jul 2015 11:04:36 +0200
Subject: RFR (M): 8073052: Rename and clean up the allocation manager
	hierarchy in g1Allocator.?pp
In-Reply-To: <1438156281.2302.16.camel@oracle.com>
References: <1438073174.2272.17.camel@oracle.com>	
	<55B778A4.4050602@oracle.com> <1438156281.2302.16.camel@oracle.com>
Message-ID: <55B89724.7030200@oracle.com>

Thomas,

On 2015-07-29 09:51, Thomas Schatzl wrote:
> Hi Mikael,
>
>    thanks for the review.
>
> On Tue, 2015-07-28 at 14:42 +0200, Mikael Gerdin wrote:
>> Hi Thomas,
>>
>> On 2015-07-28 10:46, Thomas Schatzl wrote:
>>> Hi all,
>> [...]
>>> Finally, the change aligns the naming of G1ParGCAllocator with other
>>> classes that use "PLAB" in their names instead of "ParGC".
>>>
>>> CR:
>>> https://bugs.openjdk.java.net/browse/JDK-8073052
>>>
>>> Webrev:
>>> http://cr.openjdk.java.net/~tschatzl/8073052/webrev.3/
>>
>> Overall a nice cleanup!
>>
>> g1Allocator.cpp:
>> In unsafe_max_tlab_alloc you get the heap via G1H::heap() instead of _g1h.
>
> Fixed.
>
>> G1PLABAllocator seems to look up the G1H::heap() fairly often,
>> did you consider passing on the G1H* to the G1PLABAllocator to reduce
>> the clutter from accessing through the static getter in multiple
>> locations? Another option could be to expose the G1H* from G1Allocator,
>> yielding _allocator->heap()->...
>> instead of _g1h->...
>
> Fixed.
>
> New webrevs:
> http://cr.openjdk.java.net/~tschatzl/8073052/webrev.4/ (full)
> http://cr.openjdk.java.net/~tschatzl/8073052/webrev.3_to_4/ (diff)

Looks good to me.
/Mikael

>
> Thanks,
>    Thomas
>
>



From eric.caspole at oracle.com  Wed Jul 29 22:12:54 2015
From: eric.caspole at oracle.com (Eric Caspole)
Date: Wed, 29 Jul 2015 18:12:54 -0400
Subject: RFR: JDK-8078904 : CMS: Assert failed: Ctl pt invariant
In-Reply-To: <1432035040.2555.25.camel@oracle.com>
References: <5553A32C.7030403@oracle.com> <1432035040.2555.25.camel@oracle.com>
Message-ID: <55B94FE6.3040000@oracle.com>

After a long time I finally got back to this problem JDK-8078904.

https://bugs.openjdk.java.net/browse/JDK-8078904

Webrev:

http://cr.openjdk.java.net/~ecaspole/JDK-8078904/03/webrev/

It is an assert in debug builds in CMS where the setup of the survivor 
chunk array used for setting up the CMS rescan did not completely scan 
all the per-thread plab arrays.

In product builds this would only result in uneven distribution of
parallel work where the last task might get 100x as much region to scan
as the others.

After the fix for 8079555 the problem would happen with different cmd 
line options but it was still there.

I think the fix for 8130459 somewhat narrowed the conditions where this 
problem would happen by checking the MinTLABSize against the 
YoungPLABSize, but it still happens.

My idea here is to stride over the survivor plab arrays where the stride 
length is based on the gc thread count, so the whole survivior plab 
arrays structure will be scanned no matter what MinTLABSize etc is set 
on the cmd line, and the parallel tasks are allocated more evenly sized 
chunks of work, without increasing the size of the _survivor_chunk_array.
Previously, very small MinTLABSize would cause so many recorded PLABs 
that there was not enough room in the _survivor_chunk_array and so there 
would be uneven work in the rescan tasks.

Thanks to Sangheon for a lot of side discussions about PLABs and option 
checking.

I tested this with the failing example using many combinations of gc 
thread counts and many MinTLABSize values from 64 up to 4M. And JPRT.
Thanks,
Eric


From daniel.fuchs at oracle.com  Wed Jul 29 08:25:35 2015
From: daniel.fuchs at oracle.com (Daniel Fuchs)
Date: Wed, 29 Jul 2015 10:25:35 +0200
Subject: RFR: 8132306: java/lang/ref/ReferenceEnqueue.java fails with
	"RuntimeException: Error: poll() returned null; expected ref object"
In-Reply-To: <BA1C0BEB-F342-437A-87E7-029660BB5348@oracle.com>
References: <BA1C0BEB-F342-437A-87E7-029660BB5348@oracle.com>
Message-ID: <55B88DFF.60204@oracle.com>

Hi Kim,

I agree with your proposed fix.

I see that you have added a comment for future maintainers.
Thanks for that - as the implication of the ordering between
the two volatile writes is not immediately perceptible to
the casual reader.

You have good eyes - I hadn't spotted the race condition, but
it is definitely there. I have tried to see if the proposed
reordering could have other side effects but couldn't find any:
The reference might now temporarily appear unenqueued though
it's already at the head of the queue - but since 'head' is
private in ReferenceQueue and access to the head reference
are adequately protected by 'lock' then it looks like this
is not going to be an issue.

+1

Thanks for finding the cause of the failure and the devising the fix!

best regards,

-- daniel

On 29/07/15 09:57, Kim Barrett wrote:
> Please review this fix of a race condition in
> j.l.r.Reference/ReferenceQueue.  See comments in the bug report for a
> description of the race.  The race is being fixed by reordering a pair
> of volatile assignments.
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8132306
>
> Webrev:
> http://cr.openjdk.java.net/~kbarrett/8132306/webrev.00/
>
> Testing:
> jprt with default and hotspot testsets
>
> Locally tested race in original code by insertion of sleep in enqueue and
> verifying ReferenceEnqueue regression test fails consistently as
> described in the bug report.
>
> Locally tested fixed code by insertion of sleeps at various points and
> verifying ReferenceEnqueue regression test still passes.
>



From kim.barrett at oracle.com  Wed Jul 29 23:46:39 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Wed, 29 Jul 2015 19:46:39 -0400
Subject: RFR: 8132306: java/lang/ref/ReferenceEnqueue.java fails with
	"RuntimeException: Error: poll() returned null; expected ref object"
In-Reply-To: <55B88FAB.4080302@oracle.com>
References: <BA1C0BEB-F342-437A-87E7-029660BB5348@oracle.com>
	<55B88FAB.4080302@oracle.com>
Message-ID: <929281CB-55E8-4A0C-A8D1-85ED37348E12@oracle.com>

On Jul 29, 2015, at 4:32 AM, David Holmes <david.holmes at oracle.com> wrote:
> 
> On 29/07/2015 5:57 PM, Kim Barrett wrote:
>> ...  The race is being fixed by reordering a pair
>> of volatile assignments.
> 
> While this seems logical for the failure at hand it isn't immediately obvious to me that setting next before setting the ENQUEUED state won't cause a problem for some other piece of code. Really these things need to be tightly synchronized - I think the real bug is the unsynchronized fast-path for the empty-queue case in poll(). While that change was deliberate in 6666739 this side-effect was not realized and would have affected the change. I hope Martin and/or Tom see this and chime in.

I thought the poll() fast-path from 6666739 was ok at the time it was
made, and that it was the later removal of synchronization on the
reference by 8014890 that lead to the race under discussion, and
reinstating that synchronization on the reference was another way to
fix this race.  Turns out I was wrong about that.

The poll() fast-path introduced the possibility of the following
unexpected behavior when another thread is in r.enqueue() and is in
the problematic window:

    !r.enqueue() && q.poll() == null => true

This can happen because the synchronization on the references is only
in ReferenceQueue.enqueue().  If it was instead in
Reference.enqueue(), or if ReferenceQueue.Null.enqueue() also
synchronized on the reference, this race would be prevented.

The removal of reference synchronization opened the door wider, also
allowing this unexpected behavior:

    r.isEnqueued() && q.poll() == null => true

The combination of those changes is needed for the ReferenceEnqueue
regression test to fail, since it requires

    r.isEnqueued() && !r.enqueue() && q.poll() == null => true

I wouldn't want to revert the poll() fast-path, since that was added
for specific performance reasons.

I don't think I'd want to add back synchronization on the reference,
but might be persuaded otherwise.  8029205 should be looked at in that
case.

I've looked carefully at uses of r.next and I don't think there is a
problem with reordering the r.next and r.queue assignments.  Of
course, the existing code was looked at by a number of people without
spotting the race under discussion.

A way to think about this that helped me (I think) understand the
locking structure used here is that q.head and r.next are part of the
queue associated with the reference.  We can manipulate those using
the queue's lock as the basis for protection, so long as the the
reference's queue isn't changed.  But when we change the reference's
queue, the queue (including r.next) must be in a consistent state.
[This suggests the r.queue = NULL assignment in reallyPoll() should be
moved later, though I think the assignment order in reallyPoll()
doesn't matter.]

> That aside the commentary is rather verbose, a simple:
> 
> // Only set ENQUEUED state after the reference is enqueued
> 
> would suffice (and add "volatiles ensure ordering" if you want that to be clearer).

I do get a bit wordy sometimes.  How about this:

            // Update r.queue *after* adding to queue's list, to avoid
            // race between enqueued checks and poll().  Volatiles
            // ensure ordering.


>> CR:
>> https://bugs.openjdk.java.net/browse/JDK-8132306
>> 
>> Webrev:
>> http://cr.openjdk.java.net/~kbarrett/8132306/webrev.00/
>> 
>> Testing:
>> jprt with default and hotspot testsets
>> 
>> Locally tested race in original code by insertion of sleep in enqueue and
>> verifying ReferenceEnqueue regression test fails consistently as
>> described in the bug report.
>> 
>> Locally tested fixed code by insertion of sleeps at various points and
>> verifying ReferenceEnqueue regression test still passes.




From david.holmes at oracle.com  Thu Jul 30 02:40:28 2015
From: david.holmes at oracle.com (David Holmes)
Date: Thu, 30 Jul 2015 12:40:28 +1000
Subject: RFR: 8132306: java/lang/ref/ReferenceEnqueue.java fails with
	"RuntimeException: Error: poll() returned null; expected ref object"
In-Reply-To: <929281CB-55E8-4A0C-A8D1-85ED37348E12@oracle.com>
References: <BA1C0BEB-F342-437A-87E7-029660BB5348@oracle.com>
	<55B88FAB.4080302@oracle.com>
	<929281CB-55E8-4A0C-A8D1-85ED37348E12@oracle.com>
Message-ID: <55B98E9C.10704@oracle.com>

Hi Kim,

tl;dr: Ship it! :)

On 30/07/2015 9:46 AM, Kim Barrett wrote:
> On Jul 29, 2015, at 4:32 AM, David Holmes <david.holmes at oracle.com> wrote:
>>
>> On 29/07/2015 5:57 PM, Kim Barrett wrote:
>>> ...  The race is being fixed by reordering a pair
>>> of volatile assignments.
>>
>> While this seems logical for the failure at hand it isn't immediately obvious to me that setting next before setting the ENQUEUED state won't cause a problem for some other piece of code. Really these things need to be tightly synchronized - I think the real bug is the unsynchronized fast-path for the empty-queue case in poll(). While that change was deliberate in 6666739 this side-effect was not realized and would have affected the change. I hope Martin and/or Tom see this and chime in.
>
> I thought the poll() fast-path from 6666739 was ok at the time it was
> made, and that it was the later removal of synchronization on the
> reference by 8014890 that lead to the race under discussion, and
> reinstating that synchronization on the reference was another way to
> fix this race.  Turns out I was wrong about that.

Seems there are multiple races present. :(

> The poll() fast-path introduced the possibility of the following
> unexpected behavior when another thread is in r.enqueue() and is in
> the problematic window:
>
>      !r.enqueue() && q.poll() == null => true
>
> This can happen because the synchronization on the references is only
> in ReferenceQueue.enqueue().  If it was instead in
> Reference.enqueue(), or if ReferenceQueue.Null.enqueue() also
> synchronized on the reference, this race would be prevented.

Right - the switching of the queues changes the lock in use and so we no 
longer serialize access to the reference state.

> The removal of reference synchronization opened the door wider, also
> allowing this unexpected behavior:
>
>      r.isEnqueued() && q.poll() == null => true
>
> The combination of those changes is needed for the ReferenceEnqueue
> regression test to fail, since it requires
>
>      r.isEnqueued() && !r.enqueue() && q.poll() == null => true

Yes - all of which can unfortunately run concurrently with the still 
in-progress r.enqueue().

> I wouldn't want to revert the poll() fast-path, since that was added
> for specific performance reasons.

But that was done under an assumption of correctness. If it isn't 
correct then its okay to regress performance. But of course if we can 
fix it some other way ...

> I don't think I'd want to add back synchronization on the reference,
> but might be persuaded otherwise.  8029205 should be looked at in that
> case.

It is a tricky one - as I wrote in the review thread for 8014890. Basic 
synchronization rules say you should lock the Reference whilst mutating 
its state - but that leads to nested locking issues when also trying to 
update the queue's state. So we add a volatile, drop a lock and create 
one or more races - and hope the we have correctly determined that the 
races are benign (which we didn't in this case).

> I've looked carefully at uses of r.next and I don't think there is a
> problem with reordering the r.next and r.queue assignments.  Of
> course, the existing code was looked at by a number of people without
> spotting the race under discussion.

Between your analysis and Daniel's I'm more confident about the change. 
The Java side of things seems fine, but trying to enumerate all the 
possibles races is difficult to say the least.

> A way to think about this that helped me (I think) understand the
> locking structure used here is that q.head and r.next are part of the
> queue associated with the reference.  We can manipulate those using
> the queue's lock as the basis for protection, so long as the the
> reference's queue isn't changed.  But when we change the reference's
> queue, the queue (including r.next) must be in a consistent state.

I'm not 100% sure it sufficiently covers the cases where we switch 
queues, but yes logically the Reference's queue field is guarded by the 
queue's lock.

> [This suggests the r.queue = NULL assignment in reallyPoll() should be
> moved later, though I think the assignment order in reallyPoll()
> doesn't matter.]

I don't see that it matters as the current queue is locked and the only 
interesting actions can happen with the current queue. For the NULL and 
ENQUEUED queues enqueuing is a no-op, and there can only be one "real" 
queue involved.

>> That aside the commentary is rather verbose, a simple:
>>
>> // Only set ENQUEUED state after the reference is enqueued
>>
>> would suffice (and add "volatiles ensure ordering" if you want that to be clearer).
>
> I do get a bit wordy sometimes.  How about this:
>
>              // Update r.queue *after* adding to queue's list, to avoid
>              // race between enqueued checks and poll().  Volatiles
>              // ensure ordering.

Sounds fine.

Thanks,
David
------

>
>>> CR:
>>> https://bugs.openjdk.java.net/browse/JDK-8132306
>>>
>>> Webrev:
>>> http://cr.openjdk.java.net/~kbarrett/8132306/webrev.00/
>>>
>>> Testing:
>>> jprt with default and hotspot testsets
>>>
>>> Locally tested race in original code by insertion of sleep in enqueue and
>>> verifying ReferenceEnqueue regression test fails consistently as
>>> described in the bug report.
>>>
>>> Locally tested fixed code by insertion of sleeps at various points and
>>> verifying ReferenceEnqueue regression test still passes.
>
>


From abhijangda at hotmail.com  Thu Jul 30 03:10:41 2015
From: abhijangda at hotmail.com (Abhinav Jangda)
Date: Thu, 30 Jul 2015 08:40:41 +0530
Subject: Entry function of GC Collection
In-Reply-To: <55B6785E.5050303@oracle.com>
References: <BLU184-W47FBB70FEF6277610243ACA800@phx.gbl>,
	<55B6785E.5050303@oracle.com>
Message-ID: <BLU184-W38810ADB86F5A9E3BEF74FCA8B0@phx.gbl>

Yes, I was looking for these methods. Thanks a lot for all of you.

I did some more digging and found out that The type of generation in Serial GC is TenuredGeneration. Now, TenuredGeneration::collect () calls OneContigSpaceCardGeneration::collect (). But in OneContigSpaceCardGeneration::collect () I couldn't see any code related to collect or relevant to collect. Please tell me if collection is happening somewhere else or here? If somewhere else then where?
 
I can see that TenuredGeneration::collect () is called from _gens[i]->collect(full, do_clear_all_soft_refs, size, is_tlab); in GenCollectedHeap::do_collection () function in file genCollectedHeap.cpp.

Thank You,

Abhinav
Date: Mon, 27 Jul 2015 11:28:46 -0700
From: jon.masamitsu at oracle.com
To: hotspot-gc-dev at openjdk.java.net
Subject: Re: Entry function of GC Collection


  
    
  
  
    You're perhaps looking for the collect() methods.

    

    For example,

    

    void TenuredGeneration::collect()

    

    and

    

    void DefNewGeneration::collect()

    

    Jon

    

    On 07/25/2015 01:30 PM, Abhinav Jangda
      wrote:

    
    
      
      Hello all,

        

        I have been working on a project which requires me to change GC
        Algorithm in Java Hotspot VM. I have been trying to understand
        the source code of GC in Hotspot VM. I know a couple of details
        about GC in Hotspot now. But I am not able to find the start
        function of GC Collection. I think VM_GenCollectFull::doit
        and VM_GenCollectForAllocation::doit() are the two
        functions called for GC. But if I add std::cout
        statements at the start of these functions and start the Dacapo
        Benchmarks I couldn't see any result produced by std::cout
        statements. I want to work with SerialGC.

        Here is the command I used to run Hotspot:

        

        ./java -XX:+UseSerialGC -jar dacapo.jar jython

        

        OpenJDK version is 8 and I am compiling building openjdk using
        following command

        

        make CONF=linux-x86_64-normal-server-release

        

        It will be great if anyone of you could help me.

        

        Thank You,

        Abhinav

      
    
    
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150730/94e5d2ad/attachment.htm>

From peter.levart at gmail.com  Thu Jul 30 08:38:43 2015
From: peter.levart at gmail.com (Peter Levart)
Date: Thu, 30 Jul 2015 10:38:43 +0200
Subject: RFR: 8132306: java/lang/ref/ReferenceEnqueue.java fails with
	"RuntimeException: Error: poll() returned null; expected ref object"
In-Reply-To: <929281CB-55E8-4A0C-A8D1-85ED37348E12@oracle.com>
References: <BA1C0BEB-F342-437A-87E7-029660BB5348@oracle.com>
	<55B88FAB.4080302@oracle.com>
	<929281CB-55E8-4A0C-A8D1-85ED37348E12@oracle.com>
Message-ID: <55B9E293.80102@gmail.com>

Hi,

Let me chime in and add some comments...

On 07/30/2015 01:46 AM, Kim Barrett wrote:
> On Jul 29, 2015, at 4:32 AM, David Holmes <david.holmes at oracle.com> wrote:
>> On 29/07/2015 5:57 PM, Kim Barrett wrote:
>>> ...  The race is being fixed by reordering a pair
>>> of volatile assignments.
>> While this seems logical for the failure at hand it isn't immediately obvious to me that setting next before setting the ENQUEUED state won't cause a problem for some other piece of code. Really these things need to be tightly synchronized - I think the real bug is the unsynchronized fast-path for the empty-queue case in poll(). While that change was deliberate in 6666739 this side-effect was not realized and would have affected the change. I hope Martin and/or Tom see this and chime in.
> I thought the poll() fast-path from 6666739 was ok at the time it was
> made, and that it was the later removal of synchronization on the
> reference by 8014890 that lead to the race under discussion, and
> reinstating that synchronization on the reference was another way to
> fix this race.  Turns out I was wrong about that.
>
> The poll() fast-path introduced the possibility of the following
> unexpected behavior when another thread is in r.enqueue() and is in
> the problematic window:
>
>      !r.enqueue() && q.poll() == null => true
>
> This can happen because the synchronization on the references is only
> in ReferenceQueue.enqueue().  If it was instead in
> Reference.enqueue(), or if ReferenceQueue.Null.enqueue() also
> synchronized on the reference, this race would be prevented.

Isn't above condition perfectly legal for references that have already 
been de-queued (and when the 'q' is otherwise empty)? But I see what you 
mean. If r.enqueue() grabs the ENQUEUED queue, the method returns false, 
but that does not mean that the reference has already been hooked on the 
'head' chain ('head' can still be null and q.poll() would return null 
because of fast-path).

reversing assignments of 'head' and 'queue' fix this situation too.

>
> The removal of reference synchronization opened the door wider, also
> allowing this unexpected behavior:
>
>      r.isEnqueued() && q.poll() == null => true

This condition is more representative, yes...

>
> The combination of those changes is needed for the ReferenceEnqueue
> regression test to fail, since it requires
>
>      r.isEnqueued() && !r.enqueue() && q.poll() == null => true
>
> I wouldn't want to revert the poll() fast-path, since that was added
> for specific performance reasons.
>
> I don't think I'd want to add back synchronization on the reference,
> but might be persuaded otherwise.  8029205 should be looked at in that
> case.
>
> I've looked carefully at uses of r.next and I don't think there is a
> problem with reordering the r.next and r.queue assignments.  Of
> course, the existing code was looked at by a number of people without
> spotting the race under discussion.
>
> A way to think about this that helped me (I think) understand the
> locking structure used here is that q.head and r.next are part of the
> queue associated with the reference.  We can manipulate those using
> the queue's lock as the basis for protection, so long as the the
> reference's queue isn't changed.  But when we change the reference's
> queue, the queue (including r.next) must be in a consistent state.
> [This suggests the r.queue = NULL assignment in reallyPoll() should be
> moved later, though I think the assignment order in reallyPoll()
> doesn't matter.]

I think the assignment to r.queue = NULL in realyPoll() should be moved 
*before* the assignment to 'head' (which might assign null if 'r' was 
the last element). Here's why:

Suppose we have a Reference 'r' and it's associated ReferenceQueue 'q'. 
Currently it can happen that the following evaluates to true, which is 
surprising:

q.poll() == null && r.isEnqueued()


Regards, Peter


>
>> That aside the commentary is rather verbose, a simple:
>>
>> // Only set ENQUEUED state after the reference is enqueued
>>
>> would suffice (and add "volatiles ensure ordering" if you want that to be clearer).
> I do get a bit wordy sometimes.  How about this:
>
>              // Update r.queue *after* adding to queue's list, to avoid
>              // race between enqueued checks and poll().  Volatiles
>              // ensure ordering.
>
>
>>> CR:
>>> https://bugs.openjdk.java.net/browse/JDK-8132306
>>>
>>> Webrev:
>>> http://cr.openjdk.java.net/~kbarrett/8132306/webrev.00/
>>>
>>> Testing:
>>> jprt with default and hotspot testsets
>>>
>>> Locally tested race in original code by insertion of sleep in enqueue and
>>> verifying ReferenceEnqueue regression test fails consistently as
>>> described in the bug report.
>>>
>>> Locally tested fixed code by insertion of sleeps at various points and
>>> verifying ReferenceEnqueue regression test still passes.
>



From peter.levart at gmail.com  Thu Jul 30 08:46:41 2015
From: peter.levart at gmail.com (Peter Levart)
Date: Thu, 30 Jul 2015 10:46:41 +0200
Subject: RFR: 8132306: java/lang/ref/ReferenceEnqueue.java fails with
	"RuntimeException: Error: poll() returned null; expected ref object"
In-Reply-To: <55B9E293.80102@gmail.com>
References: <BA1C0BEB-F342-437A-87E7-029660BB5348@oracle.com>
	<55B88FAB.4080302@oracle.com>
	<929281CB-55E8-4A0C-A8D1-85ED37348E12@oracle.com>
	<55B9E293.80102@gmail.com>
Message-ID: <55B9E471.1000209@gmail.com>



On 07/30/2015 10:38 AM, Peter Levart wrote:
>> [This suggests the r.queue = NULL assignment in reallyPoll() should be
>> moved later, though I think the assignment order in reallyPoll()
>> doesn't matter.]
>
> I think the assignment to r.queue = NULL in realyPoll() should be 
> moved *before* the assignment to 'head' (which might assign null if 
> 'r' was the last element). Here's why:
>
> Suppose we have a Reference 'r' and it's associated ReferenceQueue 
> 'q'. Currently it can happen that the following evaluates to true, 
> which is surprising:
>
> q.poll() == null && r.isEnqueued()
>
>
> Regards, Peter 

Well, the above condition is not very representative of the situation 
I'm trying to describe. It can be perfectly legal when racing with 
enqueueing of the Reference 'r'. But when racing with de-queueing (i.e. 
poll()) it is surprising. So let me re-phrase the expression which is 
always surprising when it evaluates to true:

r.isEnqueued() && q.poll() == null && r.isEnqueued()

Regards, Peter

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150730/2e31e166/attachment.htm>

From dmitry.fazunenko at oracle.com  Thu Jul 30 09:58:29 2015
From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko)
Date: Thu, 30 Jul 2015 12:58:29 +0300
Subject: Fwd: Re: RFR (XXS): 8132334: [TESTBUG]
	gc/metaspace/TestCapacityUntilGCWrapAround.java Compilation failed
In-Reply-To: <55B7C4F6.9000100@oracle.com>
References: <55B7C4F6.9000100@oracle.com>
Message-ID: <55B9F545.7000300@oracle.com>

Hi everyone,

A trivial fix is still waiting for reviewers...

Bug: https://bugs.openjdk.java.net/browse/JDK-8132334
Webrev: http://cr.openjdk.java.net/~dfazunen/8132334/webrev/

Comments: w/a  for known jtreg issue described: 
https://bugs.openjdk.java.net/browse/CODETOOLS-7900024

Thanks,
Dima
-------- Forwarded Message --------
Subject: 	Re: RFR (XXS): 8132334: [TESTBUG] 
gc/metaspace/TestCapacityUntilGCWrapAround.java Compilation failed
Date: 	Tue, 28 Jul 2015 21:07:50 +0300
From: 	Dmitry Fazunenko <dmitry.fazunenko at oracle.com>
To: 	Kim Barrett <kim.barrett at oracle.com>
CC: 	hotspot-gc-dev <hotspot-gc-dev at openjdk.java.net>



On 28.07.2015 4:40, Kim Barrett wrote:
> On Jul 27, 2015, at 9:12 PM, Dmitry Fazunenko <dmitry.fazunenko at oracle.com> wrote:
>> On 27.07.2015 21:49, Kim Barrett wrote:
>>> On Jul 24, 2015, at 12:23 PM, Dmitry Fazunenko <dmitry.fazunenko at oracle.com> wrote:
>>>> Hi everyone,
>>>>
>>>> Can I have reviews for the trivial change that must fix the compilation problem?
>>>> https://bugs.openjdk.java.net/browse/JDK-8132334
>>>>
>>>> webrev:
>>>> http://cr.openjdk.java.net/~dfazunen/8132334/webrev/
>>>>
>>>> Comments: In some circumstances jtreg doesn't see wb classes if they weren't built explicitly.
>>> Under what circumstances does this occur?  I see other tests that look very similar to the pre-change version of this test - what makes this test so special?
>>>
>> I don't have an answer. I just heard about such problem, but I didn't find the bug. Might be because it's not filed yet. The hypothesis is the concurrency might cause such effect.
> Without an understanding of why this change is needed here but not
> elsewhere, I'm concerned that we're either
>
> 1. papering over one instance of a more widespread problem, or
> 2. not really solving the problem with this test after all.
>
I found the jtreg bug:
https://bugs.openjdk.java.net/browse/CODETOOLS-7900024
It's about 3 years old and still un-triaged, so I won't expect it will
be fixed soon.
But the problem could be easily worked around and this change does it.

Thanks,
Dima





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150730/5efa46f4/attachment.htm>

From peter.levart at gmail.com  Thu Jul 30 10:20:04 2015
From: peter.levart at gmail.com (Peter Levart)
Date: Thu, 30 Jul 2015 12:20:04 +0200
Subject: RFR: 8132306: java/lang/ref/ReferenceEnqueue.java fails with
	"RuntimeException: Error: poll() returned null; expected ref object"
In-Reply-To: <55B9EA8A.1080102@oracle.com>
References: <BA1C0BEB-F342-437A-87E7-029660BB5348@oracle.com>
	<55B88FAB.4080302@oracle.com>
	<929281CB-55E8-4A0C-A8D1-85ED37348E12@oracle.com>
	<55B9E293.80102@gmail.com> <55B9EA8A.1080102@oracle.com>
Message-ID: <55B9FA54.5040902@gmail.com>



On 07/30/2015 11:12 AM, Daniel Fuchs wrote:
> Hi Peter,
>
> I'm glad you're looking at this too! We can't have too many eyes :-)
>
> On 30/07/15 10:38, Peter Levart wrote:
>> Suppose we have a Reference 'r' and it's associated ReferenceQueue 'q'.
>> Currently it can happen that the following evaluates to true, which is
>> surprising:
>>
>> q.poll() == null && r.isEnqueued()
>>
>
> But on the other hand this can only happen if two different
> threads are polling the queue - in which case only one of them
> will get the reference. In such a situation, the symmetric condition
> would not be surprising (as the other thread would
> get q.poll() != null):
>
>    r.isEnqueued() && q.poll() == null
>
> The original bug fixed by Kim is more surprising, because there's only
> one Thread doing the polling, and it does get:
>
>    r.isEnqueud() && q.poll() == null
>
> although nobody else is polling the queue.
> This should no longer occur in this situation with Kim's fix.
>
> cheers,
>
> -- daniel

Yes, these are two different issues. The one Kim has fixed is the race 
of above expressions with Queue.enqueue() (when the queue is changing 
from the associated instance to ENQUEUED). The one I'm pointing at and 
Kim has already identified as potential issue is the race of the 
following expression:

r.isEnqueued() && q.poll() == null && r.isEnqueued() ==> true

....with Queue.[realy]poll() (when the queue is changing from ENQUEUED 
to NULL).

Which only happens if at least two threads are polling the queue, but is 
equally surprising and prone to cause bugs, don't you think?

Regards, Peter




From david.holmes at oracle.com  Thu Jul 30 10:24:16 2015
From: david.holmes at oracle.com (David Holmes)
Date: Thu, 30 Jul 2015 20:24:16 +1000
Subject: RFR: 8132306: java/lang/ref/ReferenceEnqueue.java fails with
	"RuntimeException: Error: poll() returned null; expected ref object"
In-Reply-To: <55B9FA54.5040902@gmail.com>
References: <BA1C0BEB-F342-437A-87E7-029660BB5348@oracle.com>
	<55B88FAB.4080302@oracle.com>
	<929281CB-55E8-4A0C-A8D1-85ED37348E12@oracle.com>
	<55B9E293.80102@gmail.com>
	<55B9EA8A.1080102@oracle.com> <55B9FA54.5040902@gmail.com>
Message-ID: <55B9FB50.8050907@oracle.com>

On 30/07/2015 8:20 PM, Peter Levart wrote:
>
>
> On 07/30/2015 11:12 AM, Daniel Fuchs wrote:
>> Hi Peter,
>>
>> I'm glad you're looking at this too! We can't have too many eyes :-)
>>
>> On 30/07/15 10:38, Peter Levart wrote:
>>> Suppose we have a Reference 'r' and it's associated ReferenceQueue 'q'.
>>> Currently it can happen that the following evaluates to true, which is
>>> surprising:
>>>
>>> q.poll() == null && r.isEnqueued()
>>>
>>
>> But on the other hand this can only happen if two different
>> threads are polling the queue - in which case only one of them
>> will get the reference. In such a situation, the symmetric condition
>> would not be surprising (as the other thread would
>> get q.poll() != null):
>>
>>    r.isEnqueued() && q.poll() == null
>>
>> The original bug fixed by Kim is more surprising, because there's only
>> one Thread doing the polling, and it does get:
>>
>>    r.isEnqueud() && q.poll() == null
>>
>> although nobody else is polling the queue.
>> This should no longer occur in this situation with Kim's fix.
>>
>> cheers,
>>
>> -- daniel
>
> Yes, these are two different issues. The one Kim has fixed is the race
> of above expressions with Queue.enqueue() (when the queue is changing
> from the associated instance to ENQUEUED). The one I'm pointing at and
> Kim has already identified as potential issue is the race of the
> following expression:
>
> r.isEnqueued() && q.poll() == null && r.isEnqueued() ==> true
>
> ....with Queue.[realy]poll() (when the queue is changing from ENQUEUED
> to NULL).
>
> Which only happens if at least two threads are polling the queue, but is
> equally surprising and prone to cause bugs, don't you think?

Sorry I'm missing your point. The expression:

r.isEnqueued() && q.poll() == null

is exactly the race the current fix is addressing. Adding a second check 
of r.isEnqueued() which still returns true does not add anything that I 
can see. ??

David

> Regards, Peter
>
>


From peter.levart at gmail.com  Thu Jul 30 10:41:35 2015
From: peter.levart at gmail.com (Peter Levart)
Date: Thu, 30 Jul 2015 12:41:35 +0200
Subject: RFR: 8132306: java/lang/ref/ReferenceEnqueue.java fails with
	"RuntimeException: Error: poll() returned null; expected ref object"
In-Reply-To: <55B9FB50.8050907@oracle.com>
References: <BA1C0BEB-F342-437A-87E7-029660BB5348@oracle.com>
	<55B88FAB.4080302@oracle.com>
	<929281CB-55E8-4A0C-A8D1-85ED37348E12@oracle.com>
	<55B9E293.80102@gmail.com>
	<55B9EA8A.1080102@oracle.com> <55B9FA54.5040902@gmail.com>
	<55B9FB50.8050907@oracle.com>
Message-ID: <55B9FF5F.2020706@gmail.com>



On 07/30/2015 12:24 PM, David Holmes wrote:
> On 30/07/2015 8:20 PM, Peter Levart wrote:
>>
>>
>> On 07/30/2015 11:12 AM, Daniel Fuchs wrote:
>>> Hi Peter,
>>>
>>> I'm glad you're looking at this too! We can't have too many eyes :-)
>>>
>>> On 30/07/15 10:38, Peter Levart wrote:
>>>> Suppose we have a Reference 'r' and it's associated ReferenceQueue 
>>>> 'q'.
>>>> Currently it can happen that the following evaluates to true, which is
>>>> surprising:
>>>>
>>>> q.poll() == null && r.isEnqueued()
>>>>
>>>
>>> But on the other hand this can only happen if two different
>>> threads are polling the queue - in which case only one of them
>>> will get the reference. In such a situation, the symmetric condition
>>> would not be surprising (as the other thread would
>>> get q.poll() != null):
>>>
>>>    r.isEnqueued() && q.poll() == null
>>>
>>> The original bug fixed by Kim is more surprising, because there's only
>>> one Thread doing the polling, and it does get:
>>>
>>>    r.isEnqueud() && q.poll() == null
>>>
>>> although nobody else is polling the queue.
>>> This should no longer occur in this situation with Kim's fix.
>>>
>>> cheers,
>>>
>>> -- daniel
>>
>> Yes, these are two different issues. The one Kim has fixed is the race
>> of above expressions with Queue.enqueue() (when the queue is changing
>> from the associated instance to ENQUEUED). The one I'm pointing at and
>> Kim has already identified as potential issue is the race of the
>> following expression:
>>
>> r.isEnqueued() && q.poll() == null && r.isEnqueued() ==> true
>>
>> ....with Queue.[realy]poll() (when the queue is changing from ENQUEUED
>> to NULL).
>>
>> Which only happens if at least two threads are polling the queue, but is
>> equally surprising and prone to cause bugs, don't you think?
>
> Sorry I'm missing your point. The expression:
>
> r.isEnqueued() && q.poll() == null
>
> is exactly the race the current fix is addressing. Adding a second 
> check of r.isEnqueued() which still returns true does not add anything 
> that I can see. ??
>
> David

The expressions are similar, but the race is different. The one 
addressed by Kim is the race of:

r.isEnqueued() && q.poll() == null ==> true

with q.enqueue(r)

There, the reversal of assignments to q.head and r.queue in 
ReferenceQueue.enqueue() fixes the issue.


The one I'm pointing at is the race of:

r.isEnqueued() && q.poll() == null && r.isEnqueued() ==> true

with q.poll()

Here, the fix would be to also revert the assignments to q.head and 
r.queue in ReferenceQueue.reallyPoll() this time.


The 1st race is the race with enqueue-ing and the 2nd race is the race 
with de-queueing. Initially, my "surprising" expression was:

q.poll() == null && r.isEnqueued() ==> true


...but this is not representative, as it is also an expected outcome 
when racing with enqueue-ing. So I added a pre-condition to express the 
fact that it happens when racing with de-queueing only:

r.isEnqueued() && q.poll() == null && r.isEnqueued() ==> true


What is surprising in the above expression evaluating to true is the 
fact that 'r' appears to be enqueued before and after the q.poll() 
returns null. I can easily imagine code that would fail because it never 
imagined above expression to evaluate to true. For example:

if (r.isEnqueued()) {
     Reference s = q.poll();
     if (s == null) {
         // somebody else already dequeued 'r'
         assert !r.isEnqueued();
     }
}



Regards, Peter

>
>> Regards, Peter
>>
>>



From peter.levart at gmail.com  Thu Jul 30 11:37:41 2015
From: peter.levart at gmail.com (Peter Levart)
Date: Thu, 30 Jul 2015 13:37:41 +0200
Subject: RFR: 8132306: java/lang/ref/ReferenceEnqueue.java fails with
	"RuntimeException: Error: poll() returned null; expected ref object"
In-Reply-To: <55BA0117.6040203@oracle.com>
References: <BA1C0BEB-F342-437A-87E7-029660BB5348@oracle.com>
	<55B88FAB.4080302@oracle.com>
	<929281CB-55E8-4A0C-A8D1-85ED37348E12@oracle.com>
	<55B9E293.80102@gmail.com>
	<55B9EA8A.1080102@oracle.com> <55B9FA54.5040902@gmail.com>
	<55BA0117.6040203@oracle.com>
Message-ID: <55BA0C85.7090802@gmail.com>



On 07/30/2015 12:48 PM, Daniel Fuchs wrote:
> On 30/07/15 12:20, Peter Levart wrote:
>>
>>
>> On 07/30/2015 11:12 AM, Daniel Fuchs wrote:
>>> Hi Peter,
>>>
>>> I'm glad you're looking at this too! We can't have too many eyes :-)
>>>
>>> On 30/07/15 10:38, Peter Levart wrote:
>>>> Suppose we have a Reference 'r' and it's associated ReferenceQueue 
>>>> 'q'.
>>>> Currently it can happen that the following evaluates to true, which is
>>>> surprising:
>>>>
>>>> q.poll() == null && r.isEnqueued()
>>>>
>>>
>>> But on the other hand this can only happen if two different
>>> threads are polling the queue - in which case only one of them
>>> will get the reference. In such a situation, the symmetric condition
>>> would not be surprising (as the other thread would
>>> get q.poll() != null):
>>>
>>>    r.isEnqueued() && q.poll() == null
>>>
>>> The original bug fixed by Kim is more surprising, because there's only
>>> one Thread doing the polling, and it does get:
>>>
>>>    r.isEnqueud() && q.poll() == null
>>>
>>> although nobody else is polling the queue.
>>> This should no longer occur in this situation with Kim's fix.
>>>
>>> cheers,
>>>
>>> -- daniel
>>
>> Yes, these are two different issues. The one Kim has fixed is the race
>> of above expressions with Queue.enqueue() (when the queue is changing
>> from the associated instance to ENQUEUED). The one I'm pointing at and
>> Kim has already identified as potential issue is the race of the
>> following expression:
>>
>> r.isEnqueued() && q.poll() == null && r.isEnqueued() ==> true
>>
>> ....with Queue.[realy]poll() (when the queue is changing from ENQUEUED
>> to NULL).
>>
>> Which only happens if at least two threads are polling the queue, but is
>> equally surprising and prone to cause bugs, don't you think?
>
> Hi Peter,
>
> Yes - this is also surprising. Is it prone to cause bugs?
> possibly - but how serious I'm not sure.
> Is it 'equally' surprising - well - that was the point of my argument:
> there are two threads polling the same queue - so one of them should
> get null... Though I agree that in this case - 'r' should be seen
> has having changed state...

poll() returning null by itself is not surprising, but if 'r' appears to 
be "enqueued" before and after the fact, this is surprising.

>
> The question for me is whether this should be fixed in the same
> changeset - or whether we should make it in another changeset...

It's a different issue, though very similar.

Regards, Peter

>
> cheers, and thanks for pointing it out!
>
> -- daniel
>
>>
>> Regards, Peter
>>
>>
>



From david.holmes at oracle.com  Thu Jul 30 11:44:44 2015
From: david.holmes at oracle.com (David Holmes)
Date: Thu, 30 Jul 2015 21:44:44 +1000
Subject: RFR: 8132306: java/lang/ref/ReferenceEnqueue.java fails with
	"RuntimeException: Error: poll() returned null; expected ref object"
In-Reply-To: <55B9FF5F.2020706@gmail.com>
References: <BA1C0BEB-F342-437A-87E7-029660BB5348@oracle.com>
	<55B88FAB.4080302@oracle.com>
	<929281CB-55E8-4A0C-A8D1-85ED37348E12@oracle.com>
	<55B9E293.80102@gmail.com>
	<55B9EA8A.1080102@oracle.com> <55B9FA54.5040902@gmail.com>
	<55B9FB50.8050907@oracle.com> <55B9FF5F.2020706@gmail.com>
Message-ID: <55BA0E2B.9050101@oracle.com>



On 30/07/2015 8:41 PM, Peter Levart wrote:
>
>
> On 07/30/2015 12:24 PM, David Holmes wrote:
>> On 30/07/2015 8:20 PM, Peter Levart wrote:
>>>
>>>
>>> On 07/30/2015 11:12 AM, Daniel Fuchs wrote:
>>>> Hi Peter,
>>>>
>>>> I'm glad you're looking at this too! We can't have too many eyes :-)
>>>>
>>>> On 30/07/15 10:38, Peter Levart wrote:
>>>>> Suppose we have a Reference 'r' and it's associated ReferenceQueue
>>>>> 'q'.
>>>>> Currently it can happen that the following evaluates to true, which is
>>>>> surprising:
>>>>>
>>>>> q.poll() == null && r.isEnqueued()
>>>>>
>>>>
>>>> But on the other hand this can only happen if two different
>>>> threads are polling the queue - in which case only one of them
>>>> will get the reference. In such a situation, the symmetric condition
>>>> would not be surprising (as the other thread would
>>>> get q.poll() != null):
>>>>
>>>>    r.isEnqueued() && q.poll() == null
>>>>
>>>> The original bug fixed by Kim is more surprising, because there's only
>>>> one Thread doing the polling, and it does get:
>>>>
>>>>    r.isEnqueud() && q.poll() == null
>>>>
>>>> although nobody else is polling the queue.
>>>> This should no longer occur in this situation with Kim's fix.
>>>>
>>>> cheers,
>>>>
>>>> -- daniel
>>>
>>> Yes, these are two different issues. The one Kim has fixed is the race
>>> of above expressions with Queue.enqueue() (when the queue is changing
>>> from the associated instance to ENQUEUED). The one I'm pointing at and
>>> Kim has already identified as potential issue is the race of the
>>> following expression:
>>>
>>> r.isEnqueued() && q.poll() == null && r.isEnqueued() ==> true
>>>
>>> ....with Queue.[realy]poll() (when the queue is changing from ENQUEUED
>>> to NULL).
>>>
>>> Which only happens if at least two threads are polling the queue, but is
>>> equally surprising and prone to cause bugs, don't you think?
>>
>> Sorry I'm missing your point. The expression:
>>
>> r.isEnqueued() && q.poll() == null
>>
>> is exactly the race the current fix is addressing. Adding a second
>> check of r.isEnqueued() which still returns true does not add anything
>> that I can see. ??
>>
>> David
>
> The expressions are similar, but the race is different. The one
> addressed by Kim is the race of:
>
> r.isEnqueued() && q.poll() == null ==> true
>
> with q.enqueue(r)
>
> There, the reversal of assignments to q.head and r.queue in
> ReferenceQueue.enqueue() fixes the issue.
>
>
> The one I'm pointing at is the race of:
>
> r.isEnqueued() && q.poll() == null && r.isEnqueued() ==> true
>
> with q.poll()
>
> Here, the fix would be to also revert the assignments to q.head and
> r.queue in ReferenceQueue.reallyPoll() this time.
>
>
> The 1st race is the race with enqueue-ing and the 2nd race is the race
> with de-queueing. Initially, my "surprising" expression was:
>
> q.poll() == null && r.isEnqueued() ==> true
>
>
> ...but this is not representative, as it is also an expected outcome
> when racing with enqueue-ing. So I added a pre-condition to express the
> fact that it happens when racing with de-queueing only:
>
> r.isEnqueued() && q.poll() == null && r.isEnqueued() ==> true
>
>
> What is surprising in the above expression evaluating to true is the
> fact that 'r' appears to be enqueued before and after the q.poll()
> returns null. I can easily imagine code that would fail because it never
> imagined above expression to evaluate to true. For example:

So r has been enqueued and one poll() removes it, so the second poll() 
returns NULL, but r still claims to be enqueued. Sorry I'm not seeing 
how that is possible.

David

>
> if (r.isEnqueued()) {
>      Reference s = q.poll();
>      if (s == null) {
>          // somebody else already dequeued 'r'
>          assert !r.isEnqueued();
>      }
> }
>
>
>
> Regards, Peter
>
>>
>>> Regards, Peter
>>>
>>>
>


From peter.levart at gmail.com  Thu Jul 30 11:57:03 2015
From: peter.levart at gmail.com (Peter Levart)
Date: Thu, 30 Jul 2015 13:57:03 +0200
Subject: RFR: 8132306: java/lang/ref/ReferenceEnqueue.java fails with
	"RuntimeException: Error: poll() returned null; expected ref object"
In-Reply-To: <55BA0E2B.9050101@oracle.com>
References: <BA1C0BEB-F342-437A-87E7-029660BB5348@oracle.com>
	<55B88FAB.4080302@oracle.com>
	<929281CB-55E8-4A0C-A8D1-85ED37348E12@oracle.com>
	<55B9E293.80102@gmail.com>
	<55B9EA8A.1080102@oracle.com> <55B9FA54.5040902@gmail.com>
	<55B9FB50.8050907@oracle.com> <55B9FF5F.2020706@gmail.com>
	<55BA0E2B.9050101@oracle.com>
Message-ID: <55BA110F.2010101@gmail.com>



On 07/30/2015 01:44 PM, David Holmes wrote:
>> r.isEnqueued() && q.poll() == null && r.isEnqueued() ==> true
>>
>>
>> What is surprising in the above expression evaluating to true is the
>> fact that 'r' appears to be enqueued before and after the q.poll()
>> returns null. I can easily imagine code that would fail because it never
>> imagined above expression to evaluate to true. For example:
>
> So r has been enqueued and one poll() removes it, so the second poll() 
> returns NULL, but r still claims to be enqueued. Sorry I'm not seeing 
> how that is possible.
>
> David 

'r' has been enqueued.

Thread-1:

r.isEnqueued() &&
q.poll() == null &&
r.isEnqueued()

Thread-2:

q.poll();


Sequence of actions:

T1: r.isEnqueued() ==> true

T2: q.poll() executed to the following point (see HERE) and 'r' was the 
last element in the queue ('head' has been assigned to null):

     public Reference<? extends T> poll() {
         if (head == null)
             return null;
         synchronized (lock) {
             return reallyPoll();
         }
     }

     private Reference<? extends T> reallyPoll() {       /* Must hold 
lock */
         Reference<? extends T> r = head;
         if (r != null) {
             head = (r.next == r) ?
                 null :
                 r.next; // Unchecked due to the next field having a raw 
type in Reference

             // >> HERE <<<

             r.queue = NULL;
             r.next = r;
             queueLength--;
             if (r instanceof FinalReference) {
                 sun.misc.VM.addFinalRefCount(-1);
             }
             return r;
         }
         return null;
     }

T1: q.poll() finds head == null and returns null;

T1: r.isEnqueued() ==> true since r.queue is still ENQUEUED


Regards, Peter



From david.holmes at oracle.com  Thu Jul 30 12:09:12 2015
From: david.holmes at oracle.com (David Holmes)
Date: Thu, 30 Jul 2015 22:09:12 +1000
Subject: RFR: 8132306: java/lang/ref/ReferenceEnqueue.java fails with
	"RuntimeException: Error: poll() returned null; expected ref object"
In-Reply-To: <55BA110F.2010101@gmail.com>
References: <BA1C0BEB-F342-437A-87E7-029660BB5348@oracle.com>
	<55B88FAB.4080302@oracle.com>
	<929281CB-55E8-4A0C-A8D1-85ED37348E12@oracle.com>
	<55B9E293.80102@gmail.com>
	<55B9EA8A.1080102@oracle.com> <55B9FA54.5040902@gmail.com>
	<55B9FB50.8050907@oracle.com> <55B9FF5F.2020706@gmail.com>
	<55BA0E2B.9050101@oracle.com> <55BA110F.2010101@gmail.com>
Message-ID: <55BA13E8.1050606@oracle.com>

On 30/07/2015 9:57 PM, Peter Levart wrote:
>
>
> On 07/30/2015 01:44 PM, David Holmes wrote:
>>> r.isEnqueued() && q.poll() == null && r.isEnqueued() ==> true
>>>
>>>
>>> What is surprising in the above expression evaluating to true is the
>>> fact that 'r' appears to be enqueued before and after the q.poll()
>>> returns null. I can easily imagine code that would fail because it never
>>> imagined above expression to evaluate to true. For example:
>>
>> So r has been enqueued and one poll() removes it, so the second poll()
>> returns NULL, but r still claims to be enqueued. Sorry I'm not seeing
>> how that is possible.
>>
>> David
>
> 'r' has been enqueued.
>
> Thread-1:
>
> r.isEnqueued() &&
> q.poll() == null &&
> r.isEnqueued()
>
> Thread-2:
>
> q.poll();
>
>
> Sequence of actions:
>
> T1: r.isEnqueued() ==> true
>
> T2: q.poll() executed to the following point (see HERE) and 'r' was the
> last element in the queue ('head' has been assigned to null):

Yeah thanks - just realized it is that darned unsynchronized "fast-path" 
again. What a mess.

It a kind of inverse of the original problem.

Original: don't update reference state to enqueued before the queue is 
updated
This one: don't update the queue state to empty before the reference 
state shows it is de-queued.

So yes the fix here is to move "r.queue = null" to before the assignment 
to head.

Bring on the next race ;-)

Thanks,
David

>      public Reference<? extends T> poll() {
>          if (head == null)
>              return null;
>          synchronized (lock) {
>              return reallyPoll();
>          }
>      }
>
>      private Reference<? extends T> reallyPoll() {       /* Must hold
> lock */
>          Reference<? extends T> r = head;
>          if (r != null) {
>              head = (r.next == r) ?
>                  null :
>                  r.next; // Unchecked due to the next field having a raw
> type in Reference
>
>              // >> HERE <<<
>
>              r.queue = NULL;
>              r.next = r;
>              queueLength--;
>              if (r instanceof FinalReference) {
>                  sun.misc.VM.addFinalRefCount(-1);
>              }
>              return r;
>          }
>          return null;
>      }
>
> T1: q.poll() finds head == null and returns null;
>
> T1: r.isEnqueued() ==> true since r.queue is still ENQUEUED
>
>
> Regards, Peter
>


From kim.barrett at oracle.com  Thu Jul 30 19:54:11 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Thu, 30 Jul 2015 15:54:11 -0400
Subject: RFR: 8132306: java/lang/ref/ReferenceEnqueue.java fails with
	"RuntimeException: Error: poll() returned null; expected ref object"
In-Reply-To: <55BA13E8.1050606@oracle.com>
References: <BA1C0BEB-F342-437A-87E7-029660BB5348@oracle.com>
	<55B88FAB.4080302@oracle.com>
	<929281CB-55E8-4A0C-A8D1-85ED37348E12@oracle.com>
	<55B9E293.80102@gmail.com> <55B9EA8A.1080102@oracle.com>
	<55B9FA54.5040902@gmail.com> <55B9FB50.8050907@oracle.com>
	<55B9FF5F.2020706@gmail.com> <55BA0E2B.9050101@oracle.com>
	<55BA110F.2010101@gmail.com> <55BA13E8.1050606@oracle.com>
Message-ID: <CE7CA993-BB20-44E6-B378-419560CFFCCC@oracle.com>

On Jul 30, 2015, at 8:09 AM, David Holmes <david.holmes at oracle.com> wrote:
> 
> On 30/07/2015 9:57 PM, Peter Levart wrote:
>> 'r' has been enqueued.
>> 
>> Thread-1:
>> 
>> r.isEnqueued() &&
>> q.poll() == null &&
>> r.isEnqueued()
>> 
>> Thread-2:
>> 
>> q.poll();
>> 
>> 
>> Sequence of actions:
>> 
>> T1: r.isEnqueued() ==> true
>> 
>> T2: q.poll() executed to the following point (see HERE) and 'r' was the
>> last element in the queue ('head' has been assigned to null):
> 
> Yeah thanks - just realized it is that darned unsynchronized "fast-path" again. What a mess.
> 
> It a kind of inverse of the original problem.
> 
> Original: don't update reference state to enqueued before the queue is updated
> This one: don't update the queue state to empty before the reference state shows it is de-queued.
> 
> So yes the fix here is to move "r.queue = null" to before the assignment to head.
> 
> Bring on the next race ;-)

I agree with everything David said above.  Bleh!

So I think I can either:

1. Go ahead with my change + Peter's change.

2. Give this back to core-libs while I step carefully away :-)

I *think* option (1) is at least an improvement.  But I completely
missed Peter's race, despite having specifically looked for problems
there, so take my opinion with an appropriate quantity of salt.




From david.holmes at oracle.com  Thu Jul 30 21:33:36 2015
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 31 Jul 2015 07:33:36 +1000
Subject: RFR: 8132306: java/lang/ref/ReferenceEnqueue.java fails with
	"RuntimeException: Error: poll() returned null; expected ref object"
In-Reply-To: <CE7CA993-BB20-44E6-B378-419560CFFCCC@oracle.com>
References: <BA1C0BEB-F342-437A-87E7-029660BB5348@oracle.com>
	<55B88FAB.4080302@oracle.com>
	<929281CB-55E8-4A0C-A8D1-85ED37348E12@oracle.com>
	<55B9E293.80102@gmail.com>
	<55B9EA8A.1080102@oracle.com> <55B9FA54.5040902@gmail.com>
	<55B9FB50.8050907@oracle.com> <55B9FF5F.2020706@gmail.com>
	<55BA0E2B.9050101@oracle.com> <55BA110F.2010101@gmail.com>
	<55BA13E8.1050606@oracle.com>
	<CE7CA993-BB20-44E6-B378-419560CFFCCC@oracle.com>
Message-ID: <55BA9830.8030308@oracle.com>

On 31/07/2015 5:54 AM, Kim Barrett wrote:
> On Jul 30, 2015, at 8:09 AM, David Holmes <david.holmes at oracle.com> wrote:
>>
>> On 30/07/2015 9:57 PM, Peter Levart wrote:
>>> 'r' has been enqueued.
>>>
>>> Thread-1:
>>>
>>> r.isEnqueued() &&
>>> q.poll() == null &&
>>> r.isEnqueued()
>>>
>>> Thread-2:
>>>
>>> q.poll();
>>>
>>>
>>> Sequence of actions:
>>>
>>> T1: r.isEnqueued() ==> true
>>>
>>> T2: q.poll() executed to the following point (see HERE) and 'r' was the
>>> last element in the queue ('head' has been assigned to null):
>>
>> Yeah thanks - just realized it is that darned unsynchronized "fast-path" again. What a mess.
>>
>> It a kind of inverse of the original problem.
>>
>> Original: don't update reference state to enqueued before the queue is updated
>> This one: don't update the queue state to empty before the reference state shows it is de-queued.
>>
>> So yes the fix here is to move "r.queue = null" to before the assignment to head.
>>
>> Bring on the next race ;-)
>
> I agree with everything David said above.  Bleh!
>
> So I think I can either:
>
> 1. Go ahead with my change + Peter's change.
>
> 2. Give this back to core-libs while I step carefully away :-)
>
> I *think* option (1) is at least an improvement.  But I completely
> missed Peter's race, despite having specifically looked for problems
> there, so take my opinion with an appropriate quantity of salt.

I vote for 1 as well. I think we have now given good coverage to the two 
sources of problems (the two lock-free regions):
- reference queue state can be seen while queue state is in transition
- queue empty state can be seen while reference state is in transition

Oh for a tool that would do this analysis for us :(

Thanks,
David

>


From kim.barrett at oracle.com  Thu Jul 30 22:56:25 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Thu, 30 Jul 2015 18:56:25 -0400
Subject: RFR: 8132306: java/lang/ref/ReferenceEnqueue.java fails with
	"RuntimeException: Error: poll() returned null; expected ref object"
In-Reply-To: <55BA9830.8030308@oracle.com>
References: <BA1C0BEB-F342-437A-87E7-029660BB5348@oracle.com>
	<55B88FAB.4080302@oracle.com>
	<929281CB-55E8-4A0C-A8D1-85ED37348E12@oracle.com>
	<55B9E293.80102@gmail.com> <55B9EA8A.1080102@oracle.com>
	<55B9FA54.5040902@gmail.com> <55B9FB50.8050907@oracle.com>
	<55B9FF5F.2020706@gmail.com> <55BA0E2B.9050101@oracle.com>
	<55BA110F.2010101@gmail.com> <55BA13E8.1050606@oracle.com>
	<CE7CA993-BB20-44E6-B378-419560CFFCCC@oracle.com>
	<55BA9830.8030308@oracle.com>
Message-ID: <C25B3711-5931-4F4D-8B8C-4F83FAF83A51@oracle.com>

On Jul 30, 2015, at 5:33 PM, David Holmes <david.holmes at oracle.com> wrote:
> 
>> So I think I can either:
>> 
>> 1. Go ahead with my change + Peter's change.
>> 
>> 2. Give this back to core-libs while I step carefully away :-)
>> 
>> I *think* option (1) is at least an improvement.  But I completely
>> missed Peter's race, despite having specifically looked for problems
>> there, so take my opinion with an appropriate quantity of salt.
> 
> I vote for 1 as well. I think we have now given good coverage to the two sources of problems (the two lock-free regions):
> - reference queue state can be seen while queue state is in transition
> - queue empty state can be seen while reference state is in transition

New webrev, with both changes:

http://cr.openjdk.java.net/~kbarrett/8132306/webrev.01/



From david.holmes at oracle.com  Fri Jul 31 02:22:54 2015
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 31 Jul 2015 12:22:54 +1000
Subject: RFR: 8132306: java/lang/ref/ReferenceEnqueue.java fails with
	"RuntimeException: Error: poll() returned null; expected ref object"
In-Reply-To: <C25B3711-5931-4F4D-8B8C-4F83FAF83A51@oracle.com>
References: <BA1C0BEB-F342-437A-87E7-029660BB5348@oracle.com>
	<55B88FAB.4080302@oracle.com>
	<929281CB-55E8-4A0C-A8D1-85ED37348E12@oracle.com>
	<55B9E293.80102@gmail.com>
	<55B9EA8A.1080102@oracle.com> <55B9FA54.5040902@gmail.com>
	<55B9FB50.8050907@oracle.com> <55B9FF5F.2020706@gmail.com>
	<55BA0E2B.9050101@oracle.com> <55BA110F.2010101@gmail.com>
	<55BA13E8.1050606@oracle.com>
	<CE7CA993-BB20-44E6-B378-419560CFFCCC@oracle.com>
	<55BA9830.8030308@oracle.com>
	<C25B3711-5931-4F4D-8B8C-4F83FAF83A51@oracle.com>
Message-ID: <55BADBFE.2000808@oracle.com>

Looks good!

Thanks,
David

On 31/07/2015 8:56 AM, Kim Barrett wrote:
> On Jul 30, 2015, at 5:33 PM, David Holmes <david.holmes at oracle.com> wrote:
>>
>>> So I think I can either:
>>>
>>> 1. Go ahead with my change + Peter's change.
>>>
>>> 2. Give this back to core-libs while I step carefully away :-)
>>>
>>> I *think* option (1) is at least an improvement.  But I completely
>>> missed Peter's race, despite having specifically looked for problems
>>> there, so take my opinion with an appropriate quantity of salt.
>>
>> I vote for 1 as well. I think we have now given good coverage to the two sources of problems (the two lock-free regions):
>> - reference queue state can be seen while queue state is in transition
>> - queue empty state can be seen while reference state is in transition
>
> New webrev, with both changes:
>
> http://cr.openjdk.java.net/~kbarrett/8132306/webrev.01/
>


From peter.levart at gmail.com  Fri Jul 31 13:23:58 2015
From: peter.levart at gmail.com (Peter Levart)
Date: Fri, 31 Jul 2015 15:23:58 +0200
Subject: RFR: 8132306: java/lang/ref/ReferenceEnqueue.java fails with
	"RuntimeException: Error: poll() returned null; expected ref object"
In-Reply-To: <C25B3711-5931-4F4D-8B8C-4F83FAF83A51@oracle.com>
References: <BA1C0BEB-F342-437A-87E7-029660BB5348@oracle.com>
	<55B88FAB.4080302@oracle.com>
	<929281CB-55E8-4A0C-A8D1-85ED37348E12@oracle.com>
	<55B9E293.80102@gmail.com>
	<55B9EA8A.1080102@oracle.com> <55B9FA54.5040902@gmail.com>
	<55B9FB50.8050907@oracle.com> <55B9FF5F.2020706@gmail.com>
	<55BA0E2B.9050101@oracle.com> <55BA110F.2010101@gmail.com>
	<55BA13E8.1050606@oracle.com>
	<CE7CA993-BB20-44E6-B378-419560CFFCCC@oracle.com>
	<55BA9830.8030308@oracle.com>
	<C25B3711-5931-4F4D-8B8C-4F83FAF83A51@oracle.com>
Message-ID: <55BB76EE.2070306@gmail.com>



On 07/31/2015 12:56 AM, Kim Barrett wrote:
> On Jul 30, 2015, at 5:33 PM, David Holmes <david.holmes at oracle.com> wrote:
>>> So I think I can either:
>>>
>>> 1. Go ahead with my change + Peter's change.
>>>
>>> 2. Give this back to core-libs while I step carefully away :-)
>>>
>>> I *think* option (1) is at least an improvement.  But I completely
>>> missed Peter's race, despite having specifically looked for problems
>>> there, so take my opinion with an appropriate quantity of salt.
>> I vote for 1 as well. I think we have now given good coverage to the two sources of problems (the two lock-free regions):
>> - reference queue state can be seen while queue state is in transition
>> - queue empty state can be seen while reference state is in transition
> New webrev, with both changes:
>
> http://cr.openjdk.java.net/~kbarrett/8132306/webrev.01/
>

This looks good now. Thanks for taking both of the issues together. I 
wonder if a similar racy test could be devised for the 2nd race. I doubt 
the ReferenceEnqueue test was meant to catch invalid races specifically. 
It was more a coincidence that it failed.

I tried to create a reproducer for 2nd race, but it is not reliable and 
sometimes needs several minutes to fail.

Now I have to check if my prototype for improved Reference handling:

http://cr.openjdk.java.net/~plevart/misc/JEP132/ReferenceHandling/webrev.06/

suffers from these or similar races too. From 1st look it seems so.

Regards, Peter



From michail.chernov at oracle.com  Fri Jul 31 17:30:38 2015
From: michail.chernov at oracle.com (Michail Chernov)
Date: Fri, 31 Jul 2015 20:30:38 +0300
Subject: RFR: 8081317: [NEWTEST] documented GC ratio tuning and new size
	options should be covered by regression tests
Message-ID: <55BBB0BE.6080506@oracle.com>

Hi,

Please review these changes for JDK-8081317. There are new tests to 
cover GC ratio tuning and new size options.

Webrev:  http://cr.openjdk.java.net/~mchernov/8081317/
Bug: https://bugs.openjdk.java.net/browse/JDK-8081317

TestMaxMinHeapFreeRatioFlags.java - Verify that heap size changes 
according to max and min heap free ratios.

TestMinAndInitialSurvivorRatioFlags.java - Test verifies that VM can 
start with any GC when MinSurvivorRatio and InitialSurvivorRatio flags 
passed and for Parallel GC it verifies that after start up survivor 
ratio is equal to InitialSurvivorRatio value and that actual survivor 
ratio will never be less than MinSurvivorRatio.

TestNewRatioFlag.java - Verify that heap divided among generations 
according to NewRatio.

TestNewSizeFlags.java - Verify that young gen size conforms values 
specified by NewSize, MaxNewSize and Xmn options.

TestSurvivorRatioFlag - Verify that actual survivor ratio is equal to 
specified SurvivorRatio value.

TestTargetSurvivorRatioFlag.java - Test if objects size is less than 
(survivor_size * TargetSurvivorRatio / 100) then objects are stayed in 
survivor space until MaxTenuringThreshold minor GC cycles. If more than 
(survivor_size * TargetSurvivorRatio / 100) objects were allocated, then 
test verifies that after MaxTenuringThreshold minor GC cycles survivor 
space is almost empty.

New changes were tested on different platforms including embedded jdk on 
Linux x86.

Thanks,
Michail


From ysr1729 at gmail.com  Fri Jul 31 18:19:02 2015
From: ysr1729 at gmail.com (Srinivas Ramakrishna)
Date: Fri, 31 Jul 2015 11:19:02 -0700
Subject: Cost of single-threaded nmethod hotness updates at each safepoint (in
	JDK 8)
Message-ID: <CABzyjy=GHQ7aDyZyJg_4QgpqWuFkEPmwErq2pkLmFr7CRL8X6Q@mail.gmail.com>

Hello GC and Compiler teams!

One of our services that runs with several thousand threads recently
noticed an increase
in safepoint stop times, but not gc times, upon transitioning to JDK 8.

Further investigation revealed that most of the delta was related to the
so-called
pre-gc/vmop "cleanup" phase when various book-keeping activities are
performed,
and more specifically in the portion that walks java thread stacks
single-threaded (!)
and updates the hotness counters for the active nmethods. This code appears
to
be new to JDK 8 (in jdk 7 one would walk the stacks only during code cache
sweeps).

I have two questions:
(1) has anyone else (typically, I'd expect applications with many hundreds
or thousands of threads)
noticed this regression?
(2) Can we do better, for example, by:
      (a) doing these updates by walking thread stacks in multiple worker
threads in parallel, or best of all:
      (b) doing these updates when we walk the thread stacks during GC, and
skipping this phase entirely
            for non-GC safepoints (with attendant loss in frequency of this
update in low GC frequency
            scenarios).

It seems kind of silly to do GC's with many multiple worker threads, but do
these thread stack
walks single-threaded when it is embarrasingly parallel (one could
predicate the parallelization
based on the measured stack sizes and thread population, if there was
concern on the ovrhead of
activating and deactivating the thread gangs for the work).

A followup question: Any guesses as to how code cache sweep/eviction
quality might be compromised if one
were to dispense with these hotness updates entirely (or at a much reduced
frequency), as a temporary
workaround to the performance problem?

Thoughts/Comments? In particular, has this issue been addressed perhaps in
newer JVMs?

Thanks for any comments, feedback, pointers!
-- ramki

PS: for comparison, here's data with +TraceSafepointCleanup from JDK 7
(first, where this isn't done)
vs JDK 8 (where this is done) with a program that has a few thousands of
threads:



JDK 7:
..
2827.308: [sweeping nmethods, 0.0000020 secs]
2828.679: [sweeping nmethods, 0.0000030 secs]
2829.984: [sweeping nmethods, 0.0000030 secs]
2830.956: [sweeping nmethods, 0.0000030 secs]
..

JDK 8:
..
7368.634: [mark nmethods, 0.0177030 secs]
7369.587: [mark nmethods, 0.0178305 secs]
7370.479: [mark nmethods, 0.0180260 secs]
7371.503: [mark nmethods, 0.0186494 secs]
..
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150731/e1a5487f/attachment.htm>

From vitalyd at gmail.com  Fri Jul 31 18:31:48 2015
From: vitalyd at gmail.com (Vitaly Davidovich)
Date: Fri, 31 Jul 2015 14:31:48 -0400
Subject: Cost of single-threaded nmethod hotness updates at each safepoint
	(in JDK 8)
In-Reply-To: <CABzyjy=GHQ7aDyZyJg_4QgpqWuFkEPmwErq2pkLmFr7CRL8X6Q@mail.gmail.com>
References: <CABzyjy=GHQ7aDyZyJg_4QgpqWuFkEPmwErq2pkLmFr7CRL8X6Q@mail.gmail.com>
Message-ID: <CAHjP37FHJq_wEy3jak7qPnR38C2xTXHi+BgA+xk56aV+M=dpsQ@mail.gmail.com>

Ramki, are you running tiered compilation?

sent from my phone
On Jul 31, 2015 2:19 PM, "Srinivas Ramakrishna" <ysr1729 at gmail.com> wrote:

>
> Hello GC and Compiler teams!
>
> One of our services that runs with several thousand threads recently
> noticed an increase
> in safepoint stop times, but not gc times, upon transitioning to JDK 8.
>
> Further investigation revealed that most of the delta was related to the
> so-called
> pre-gc/vmop "cleanup" phase when various book-keeping activities are
> performed,
> and more specifically in the portion that walks java thread stacks
> single-threaded (!)
> and updates the hotness counters for the active nmethods. This code
> appears to
> be new to JDK 8 (in jdk 7 one would walk the stacks only during code cache
> sweeps).
>
> I have two questions:
> (1) has anyone else (typically, I'd expect applications with many hundreds
> or thousands of threads)
> noticed this regression?
> (2) Can we do better, for example, by:
>       (a) doing these updates by walking thread stacks in multiple worker
> threads in parallel, or best of all:
>       (b) doing these updates when we walk the thread stacks during GC,
> and skipping this phase entirely
>             for non-GC safepoints (with attendant loss in frequency of
> this update in low GC frequency
>             scenarios).
>
> It seems kind of silly to do GC's with many multiple worker threads, but
> do these thread stack
> walks single-threaded when it is embarrasingly parallel (one could
> predicate the parallelization
> based on the measured stack sizes and thread population, if there was
> concern on the ovrhead of
> activating and deactivating the thread gangs for the work).
>
> A followup question: Any guesses as to how code cache sweep/eviction
> quality might be compromised if one
> were to dispense with these hotness updates entirely (or at a much reduced
> frequency), as a temporary
> workaround to the performance problem?
>
> Thoughts/Comments? In particular, has this issue been addressed perhaps in
> newer JVMs?
>
> Thanks for any comments, feedback, pointers!
> -- ramki
>
> PS: for comparison, here's data with +TraceSafepointCleanup from JDK 7
> (first, where this isn't done)
> vs JDK 8 (where this is done) with a program that has a few thousands of
> threads:
>
>
>
> JDK 7:
> ..
> 2827.308: [sweeping nmethods, 0.0000020 secs]
> 2828.679: [sweeping nmethods, 0.0000030 secs]
> 2829.984: [sweeping nmethods, 0.0000030 secs]
> 2830.956: [sweeping nmethods, 0.0000030 secs]
> ..
>
> JDK 8:
> ..
> 7368.634: [mark nmethods, 0.0177030 secs]
> 7369.587: [mark nmethods, 0.0178305 secs]
> 7370.479: [mark nmethods, 0.0180260 secs]
> 7371.503: [mark nmethods, 0.0186494 secs]
> ..
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150731/a0865aef/attachment.htm>

From ysr1729 at gmail.com  Fri Jul 31 18:33:14 2015
From: ysr1729 at gmail.com (Srinivas Ramakrishna)
Date: Fri, 31 Jul 2015 11:33:14 -0700
Subject: Cost of single-threaded nmethod hotness updates at each safepoint
	(in JDK 8)
In-Reply-To: <CAHjP37FHJq_wEy3jak7qPnR38C2xTXHi+BgA+xk56aV+M=dpsQ@mail.gmail.com>
References: <CABzyjy=GHQ7aDyZyJg_4QgpqWuFkEPmwErq2pkLmFr7CRL8X6Q@mail.gmail.com>
	<CAHjP37FHJq_wEy3jak7qPnR38C2xTXHi+BgA+xk56aV+M=dpsQ@mail.gmail.com>
Message-ID: <CABzyjyn1GaD7VKj-Lskn4UTZcJdckt_iqaSrOgWMxWh-7ehEOQ@mail.gmail.com>

Yes.


On Fri, Jul 31, 2015 at 11:31 AM, Vitaly Davidovich <vitalyd at gmail.com>
wrote:

> Ramki, are you running tiered compilation?
>
> sent from my phone
> On Jul 31, 2015 2:19 PM, "Srinivas Ramakrishna" <ysr1729 at gmail.com> wrote:
>
>>
>> Hello GC and Compiler teams!
>>
>> One of our services that runs with several thousand threads recently
>> noticed an increase
>> in safepoint stop times, but not gc times, upon transitioning to JDK 8.
>>
>> Further investigation revealed that most of the delta was related to the
>> so-called
>> pre-gc/vmop "cleanup" phase when various book-keeping activities are
>> performed,
>> and more specifically in the portion that walks java thread stacks
>> single-threaded (!)
>> and updates the hotness counters for the active nmethods. This code
>> appears to
>> be new to JDK 8 (in jdk 7 one would walk the stacks only during code
>> cache sweeps).
>>
>> I have two questions:
>> (1) has anyone else (typically, I'd expect applications with many
>> hundreds or thousands of threads)
>> noticed this regression?
>> (2) Can we do better, for example, by:
>>       (a) doing these updates by walking thread stacks in multiple worker
>> threads in parallel, or best of all:
>>       (b) doing these updates when we walk the thread stacks during GC,
>> and skipping this phase entirely
>>             for non-GC safepoints (with attendant loss in frequency of
>> this update in low GC frequency
>>             scenarios).
>>
>> It seems kind of silly to do GC's with many multiple worker threads, but
>> do these thread stack
>> walks single-threaded when it is embarrasingly parallel (one could
>> predicate the parallelization
>> based on the measured stack sizes and thread population, if there was
>> concern on the ovrhead of
>> activating and deactivating the thread gangs for the work).
>>
>> A followup question: Any guesses as to how code cache sweep/eviction
>> quality might be compromised if one
>> were to dispense with these hotness updates entirely (or at a much
>> reduced frequency), as a temporary
>> workaround to the performance problem?
>>
>> Thoughts/Comments? In particular, has this issue been addressed perhaps
>> in newer JVMs?
>>
>> Thanks for any comments, feedback, pointers!
>> -- ramki
>>
>> PS: for comparison, here's data with +TraceSafepointCleanup from JDK 7
>> (first, where this isn't done)
>> vs JDK 8 (where this is done) with a program that has a few thousands of
>> threads:
>>
>>
>>
>> JDK 7:
>> ..
>> 2827.308: [sweeping nmethods, 0.0000020 secs]
>> 2828.679: [sweeping nmethods, 0.0000030 secs]
>> 2829.984: [sweeping nmethods, 0.0000030 secs]
>> 2830.956: [sweeping nmethods, 0.0000030 secs]
>> ..
>>
>> JDK 8:
>> ..
>> 7368.634: [mark nmethods, 0.0177030 secs]
>> 7369.587: [mark nmethods, 0.0178305 secs]
>> 7370.479: [mark nmethods, 0.0180260 secs]
>> 7371.503: [mark nmethods, 0.0186494 secs]
>> ..
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150731/dba01e2c/attachment.htm>

From vladimir.kozlov at oracle.com  Fri Jul 31 18:43:12 2015
From: vladimir.kozlov at oracle.com (Vladimir Kozlov)
Date: Fri, 31 Jul 2015 11:43:12 -0700
Subject: Cost of single-threaded nmethod hotness updates at each safepoint
	(in JDK 8)
In-Reply-To: <CABzyjyn1GaD7VKj-Lskn4UTZcJdckt_iqaSrOgWMxWh-7ehEOQ@mail.gmail.com>
References: <CABzyjy=GHQ7aDyZyJg_4QgpqWuFkEPmwErq2pkLmFr7CRL8X6Q@mail.gmail.com>
	<CAHjP37FHJq_wEy3jak7qPnR38C2xTXHi+BgA+xk56aV+M=dpsQ@mail.gmail.com>
	<CABzyjyn1GaD7VKj-Lskn4UTZcJdckt_iqaSrOgWMxWh-7ehEOQ@mail.gmail.com>
Message-ID: <55BBC1C0.3030709@oracle.com>

Hi Ramki,

Did you fill up CodeCache? It start scanning aggressive only with full 
CodeCache:

   // Force stack scanning if there is only 10% free space in the code 
cache.
   // We force stack scanning only non-profiled code heap gets full, 
since critical
   // allocation go to the non-profiled heap and we must be make sure 
that there is
   // enough space.
   double free_percent = 1 / 
CodeCache::reverse_free_ratio(CodeBlobType::MethodNonProfiled) * 100;
   if (free_percent <= StartAggressiveSweepingAt) {
     do_stack_scanning();
   }

Vladimir

On 7/31/15 11:33 AM, Srinivas Ramakrishna wrote:
>
> Yes.
>
>
> On Fri, Jul 31, 2015 at 11:31 AM, Vitaly Davidovich <vitalyd at gmail.com
> <mailto:vitalyd at gmail.com>> wrote:
>
>     Ramki, are you running tiered compilation?
>
>     sent from my phone
>
>     On Jul 31, 2015 2:19 PM, "Srinivas Ramakrishna" <ysr1729 at gmail.com
>     <mailto:ysr1729 at gmail.com>> wrote:
>
>
>         Hello GC and Compiler teams!
>
>         One of our services that runs with several thousand threads
>         recently noticed an increase
>         in safepoint stop times, but not gc times, upon transitioning to
>         JDK 8.
>
>         Further investigation revealed that most of the delta was
>         related to the so-called
>         pre-gc/vmop "cleanup" phase when various book-keeping activities
>         are performed,
>         and more specifically in the portion that walks java thread
>         stacks single-threaded (!)
>         and updates the hotness counters for the active nmethods. This
>         code appears to
>         be new to JDK 8 (in jdk 7 one would walk the stacks only during
>         code cache sweeps).
>
>         I have two questions:
>         (1) has anyone else (typically, I'd expect applications with
>         many hundreds or thousands of threads)
>         noticed this regression?
>         (2) Can we do better, for example, by:
>                (a) doing these updates by walking thread stacks in
>         multiple worker threads in parallel, or best of all:
>                (b) doing these updates when we walk the thread stacks
>         during GC, and skipping this phase entirely
>                      for non-GC safepoints (with attendant loss in
>         frequency of this update in low GC frequency
>                      scenarios).
>
>         It seems kind of silly to do GC's with many multiple worker
>         threads, but do these thread stack
>         walks single-threaded when it is embarrasingly parallel (one
>         could predicate the parallelization
>         based on the measured stack sizes and thread population, if
>         there was concern on the ovrhead of
>         activating and deactivating the thread gangs for the work).
>
>         A followup question: Any guesses as to how code cache
>         sweep/eviction quality might be compromised if one
>         were to dispense with these hotness updates entirely (or at a
>         much reduced frequency), as a temporary
>         workaround to the performance problem?
>
>         Thoughts/Comments? In particular, has this issue been addressed
>         perhaps in newer JVMs?
>
>         Thanks for any comments, feedback, pointers!
>         -- ramki
>
>         PS: for comparison, here's data with +TraceSafepointCleanup from
>         JDK 7 (first, where this isn't done)
>         vs JDK 8 (where this is done) with a program that has a few
>         thousands of threads:
>
>
>
>         JDK 7:
>         ..
>         2827.308: [sweeping nmethods, 0.0000020 secs]
>         2828.679: [sweeping nmethods, 0.0000030 secs]
>         2829.984: [sweeping nmethods, 0.0000030 secs]
>         2830.956: [sweeping nmethods, 0.0000030 secs]
>         ..
>
>         JDK 8:
>         ..
>         7368.634: [mark nmethods, 0.0177030 secs]
>         7369.587: [mark nmethods, 0.0178305 secs]
>         7370.479: [mark nmethods, 0.0180260 secs]
>         7371.503: [mark nmethods, 0.0186494 secs]
>         ..
>
>


From ysr1729 at gmail.com  Fri Jul 31 18:48:53 2015
From: ysr1729 at gmail.com (Srinivas Ramakrishna)
Date: Fri, 31 Jul 2015 11:48:53 -0700
Subject: Cost of single-threaded nmethod hotness updates at each safepoint
	(in JDK 8)
In-Reply-To: <55BBC1C0.3030709@oracle.com>
References: <CABzyjy=GHQ7aDyZyJg_4QgpqWuFkEPmwErq2pkLmFr7CRL8X6Q@mail.gmail.com>
	<CAHjP37FHJq_wEy3jak7qPnR38C2xTXHi+BgA+xk56aV+M=dpsQ@mail.gmail.com>
	<CABzyjyn1GaD7VKj-Lskn4UTZcJdckt_iqaSrOgWMxWh-7ehEOQ@mail.gmail.com>
	<55BBC1C0.3030709@oracle.com>
Message-ID: <CABzyjy=UmTx9+HCTAxZ_R27fWiOcBqAxk_xLRioBJN=ZSJmGyQ@mail.gmail.com>

Hi Vladimir --

I noticed the increase even with Initial and Reserved set to the default of
240 MB, but actual usage much lower (less than a quarter).

Look at this code path. Note that this is invoked at every safepoint
(although it says "periodically" in the comment).
In the mark_active_nmethods() method, there's a thread iteration in both
branches of the if. I haven't checked to
see which of the two was the culprit here, yet (if either).

// Various cleaning tasks that should be done periodically at safepoints

void SafepointSynchronize::do_cleanup_tasks() {

....

  {

    TraceTime t4("mark nmethods", TraceSafepointCleanupTime);

    NMethodSweeper::mark_active_nmethods();

  }

..

}


void NMethodSweeper::mark_active_nmethods() {

 ...

  if (!sweep_in_progress()) {

    _seen = 0;

    _sweep_fractions_left = NmethodSweepFraction;

    _current = CodeCache::first_nmethod();

    _traversals += 1;

    _total_time_this_sweep = Tickspan();


    if (PrintMethodFlushing) {

      tty->print_cr("### Sweep: stack traversal %d", _traversals);

    }

    Threads::nmethods_do(&mark_activation_closure);


  } else {

    // Only set hotness counter

    Threads::nmethods_do(&set_hotness_closure);

  }


  OrderAccess::storestore();

}

On Fri, Jul 31, 2015 at 11:43 AM, Vladimir Kozlov <
vladimir.kozlov at oracle.com> wrote:

> Hi Ramki,
>
> Did you fill up CodeCache? It start scanning aggressive only with full
> CodeCache:
>
>   // Force stack scanning if there is only 10% free space in the code
> cache.
>   // We force stack scanning only non-profiled code heap gets full, since
> critical
>   // allocation go to the non-profiled heap and we must be make sure that
> there is
>   // enough space.
>   double free_percent = 1 /
> CodeCache::reverse_free_ratio(CodeBlobType::MethodNonProfiled) * 100;
>   if (free_percent <= StartAggressiveSweepingAt) {
>     do_stack_scanning();
>   }
>
> Vladimir
>
> On 7/31/15 11:33 AM, Srinivas Ramakrishna wrote:
>
>>
>> Yes.
>>
>>
>> On Fri, Jul 31, 2015 at 11:31 AM, Vitaly Davidovich <vitalyd at gmail.com
>> <mailto:vitalyd at gmail.com>> wrote:
>>
>>     Ramki, are you running tiered compilation?
>>
>>     sent from my phone
>>
>>     On Jul 31, 2015 2:19 PM, "Srinivas Ramakrishna" <ysr1729 at gmail.com
>>     <mailto:ysr1729 at gmail.com>> wrote:
>>
>>
>>         Hello GC and Compiler teams!
>>
>>         One of our services that runs with several thousand threads
>>         recently noticed an increase
>>         in safepoint stop times, but not gc times, upon transitioning to
>>         JDK 8.
>>
>>         Further investigation revealed that most of the delta was
>>         related to the so-called
>>         pre-gc/vmop "cleanup" phase when various book-keeping activities
>>         are performed,
>>         and more specifically in the portion that walks java thread
>>         stacks single-threaded (!)
>>         and updates the hotness counters for the active nmethods. This
>>         code appears to
>>         be new to JDK 8 (in jdk 7 one would walk the stacks only during
>>         code cache sweeps).
>>
>>         I have two questions:
>>         (1) has anyone else (typically, I'd expect applications with
>>         many hundreds or thousands of threads)
>>         noticed this regression?
>>         (2) Can we do better, for example, by:
>>                (a) doing these updates by walking thread stacks in
>>         multiple worker threads in parallel, or best of all:
>>                (b) doing these updates when we walk the thread stacks
>>         during GC, and skipping this phase entirely
>>                      for non-GC safepoints (with attendant loss in
>>         frequency of this update in low GC frequency
>>                      scenarios).
>>
>>         It seems kind of silly to do GC's with many multiple worker
>>         threads, but do these thread stack
>>         walks single-threaded when it is embarrasingly parallel (one
>>         could predicate the parallelization
>>         based on the measured stack sizes and thread population, if
>>         there was concern on the ovrhead of
>>         activating and deactivating the thread gangs for the work).
>>
>>         A followup question: Any guesses as to how code cache
>>         sweep/eviction quality might be compromised if one
>>         were to dispense with these hotness updates entirely (or at a
>>         much reduced frequency), as a temporary
>>         workaround to the performance problem?
>>
>>         Thoughts/Comments? In particular, has this issue been addressed
>>         perhaps in newer JVMs?
>>
>>         Thanks for any comments, feedback, pointers!
>>         -- ramki
>>
>>         PS: for comparison, here's data with +TraceSafepointCleanup from
>>         JDK 7 (first, where this isn't done)
>>         vs JDK 8 (where this is done) with a program that has a few
>>         thousands of threads:
>>
>>
>>
>>         JDK 7:
>>         ..
>>         2827.308: [sweeping nmethods, 0.0000020 secs]
>>         2828.679: [sweeping nmethods, 0.0000030 secs]
>>         2829.984: [sweeping nmethods, 0.0000030 secs]
>>         2830.956: [sweeping nmethods, 0.0000030 secs]
>>         ..
>>
>>         JDK 8:
>>         ..
>>         7368.634: [mark nmethods, 0.0177030 secs]
>>         7369.587: [mark nmethods, 0.0178305 secs]
>>         7370.479: [mark nmethods, 0.0180260 secs]
>>         7371.503: [mark nmethods, 0.0186494 secs]
>>         ..
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150731/1f262687/attachment.htm>

From ysr1729 at gmail.com  Fri Jul 31 19:07:18 2015
From: ysr1729 at gmail.com (Srinivas Ramakrishna)
Date: Fri, 31 Jul 2015 12:07:18 -0700
Subject: Cost of single-threaded nmethod hotness updates at each safepoint
	(in JDK 8)
In-Reply-To: <CABzyjy=UmTx9+HCTAxZ_R27fWiOcBqAxk_xLRioBJN=ZSJmGyQ@mail.gmail.com>
References: <CABzyjy=GHQ7aDyZyJg_4QgpqWuFkEPmwErq2pkLmFr7CRL8X6Q@mail.gmail.com>
	<CAHjP37FHJq_wEy3jak7qPnR38C2xTXHi+BgA+xk56aV+M=dpsQ@mail.gmail.com>
	<CABzyjyn1GaD7VKj-Lskn4UTZcJdckt_iqaSrOgWMxWh-7ehEOQ@mail.gmail.com>
	<55BBC1C0.3030709@oracle.com>
	<CABzyjy=UmTx9+HCTAxZ_R27fWiOcBqAxk_xLRioBJN=ZSJmGyQ@mail.gmail.com>
Message-ID: <CABzyjy==w=riDMkGWk3FEmct-WaX91z=W2+fZk2eYAhHKCZ0gw@mail.gmail.com>

Hi Vladimir --


Here's a snapshot of the counters:

sun.ci.codeCacheCapacity=251658240

sun.ci.codeCacheMaxCapacity=251658240

sun.ci.codeCacheMethodsReclaimedNum=3450

sun.ci.codeCacheSweepsTotalNum=58

sun.ci.codeCacheSweepsTotalTimeMillis=1111

sun.ci.codeCacheUsed=35888704


Notice that the code cache usage is less that 35 MB, for the 240 MB
capacity, yet it seems we have had 58 sweeps already, and safepoint cleanup
says:

[mark nmethods, 0.0165062 secs]

Even if the two closures do little or no work, the single-threaded walk
over deep stacks of a thousand threads will cost time for applications with
many threads, and this is now done at each safepoint irrespective of the
sweeper activity as far as I can tell. It seems as if this work should be
somehow rolled up (via a suitable injection) into GC's thread walks that
are done in parallel, rather than doing this in a pre-GC phase (unless I am
mssing some reason that the sequencing is necessary, which it doesn't seem
to be here).

-- ramki

On Fri, Jul 31, 2015 at 11:48 AM, Srinivas Ramakrishna <ysr1729 at gmail.com>
wrote:

> Hi Vladimir --
>
> I noticed the increase even with Initial and Reserved set to the default
> of 240 MB, but actual usage much lower (less than a quarter).
>
> Look at this code path. Note that this is invoked at every safepoint
> (although it says "periodically" in the comment).
> In the mark_active_nmethods() method, there's a thread iteration in both
> branches of the if. I haven't checked to
> see which of the two was the culprit here, yet (if either).
>
> // Various cleaning tasks that should be done periodically at safepoints
>
> void SafepointSynchronize::do_cleanup_tasks() {
>
> ....
>
>   {
>
>     TraceTime t4("mark nmethods", TraceSafepointCleanupTime);
>
>     NMethodSweeper::mark_active_nmethods();
>
>   }
>
> ..
>
> }
>
>
> void NMethodSweeper::mark_active_nmethods() {
>
>  ...
>
>   if (!sweep_in_progress()) {
>
>     _seen = 0;
>
>     _sweep_fractions_left = NmethodSweepFraction;
>
>     _current = CodeCache::first_nmethod();
>
>     _traversals += 1;
>
>     _total_time_this_sweep = Tickspan();
>
>
>     if (PrintMethodFlushing) {
>
>       tty->print_cr("### Sweep: stack traversal %d", _traversals);
>
>     }
>
>     Threads::nmethods_do(&mark_activation_closure);
>
>
>   } else {
>
>     // Only set hotness counter
>
>     Threads::nmethods_do(&set_hotness_closure);
>
>   }
>
>
>   OrderAccess::storestore();
>
> }
>
> On Fri, Jul 31, 2015 at 11:43 AM, Vladimir Kozlov <
> vladimir.kozlov at oracle.com> wrote:
>
>> Hi Ramki,
>>
>> Did you fill up CodeCache? It start scanning aggressive only with full
>> CodeCache:
>>
>>   // Force stack scanning if there is only 10% free space in the code
>> cache.
>>   // We force stack scanning only non-profiled code heap gets full, since
>> critical
>>   // allocation go to the non-profiled heap and we must be make sure that
>> there is
>>   // enough space.
>>   double free_percent = 1 /
>> CodeCache::reverse_free_ratio(CodeBlobType::MethodNonProfiled) * 100;
>>   if (free_percent <= StartAggressiveSweepingAt) {
>>     do_stack_scanning();
>>   }
>>
>> Vladimir
>>
>> On 7/31/15 11:33 AM, Srinivas Ramakrishna wrote:
>>
>>>
>>> Yes.
>>>
>>>
>>> On Fri, Jul 31, 2015 at 11:31 AM, Vitaly Davidovich <vitalyd at gmail.com
>>> <mailto:vitalyd at gmail.com>> wrote:
>>>
>>>     Ramki, are you running tiered compilation?
>>>
>>>     sent from my phone
>>>
>>>     On Jul 31, 2015 2:19 PM, "Srinivas Ramakrishna" <ysr1729 at gmail.com
>>>     <mailto:ysr1729 at gmail.com>> wrote:
>>>
>>>
>>>         Hello GC and Compiler teams!
>>>
>>>         One of our services that runs with several thousand threads
>>>         recently noticed an increase
>>>         in safepoint stop times, but not gc times, upon transitioning to
>>>         JDK 8.
>>>
>>>         Further investigation revealed that most of the delta was
>>>         related to the so-called
>>>         pre-gc/vmop "cleanup" phase when various book-keeping activities
>>>         are performed,
>>>         and more specifically in the portion that walks java thread
>>>         stacks single-threaded (!)
>>>         and updates the hotness counters for the active nmethods. This
>>>         code appears to
>>>         be new to JDK 8 (in jdk 7 one would walk the stacks only during
>>>         code cache sweeps).
>>>
>>>         I have two questions:
>>>         (1) has anyone else (typically, I'd expect applications with
>>>         many hundreds or thousands of threads)
>>>         noticed this regression?
>>>         (2) Can we do better, for example, by:
>>>                (a) doing these updates by walking thread stacks in
>>>         multiple worker threads in parallel, or best of all:
>>>                (b) doing these updates when we walk the thread stacks
>>>         during GC, and skipping this phase entirely
>>>                      for non-GC safepoints (with attendant loss in
>>>         frequency of this update in low GC frequency
>>>                      scenarios).
>>>
>>>         It seems kind of silly to do GC's with many multiple worker
>>>         threads, but do these thread stack
>>>         walks single-threaded when it is embarrasingly parallel (one
>>>         could predicate the parallelization
>>>         based on the measured stack sizes and thread population, if
>>>         there was concern on the ovrhead of
>>>         activating and deactivating the thread gangs for the work).
>>>
>>>         A followup question: Any guesses as to how code cache
>>>         sweep/eviction quality might be compromised if one
>>>         were to dispense with these hotness updates entirely (or at a
>>>         much reduced frequency), as a temporary
>>>         workaround to the performance problem?
>>>
>>>         Thoughts/Comments? In particular, has this issue been addressed
>>>         perhaps in newer JVMs?
>>>
>>>         Thanks for any comments, feedback, pointers!
>>>         -- ramki
>>>
>>>         PS: for comparison, here's data with +TraceSafepointCleanup from
>>>         JDK 7 (first, where this isn't done)
>>>         vs JDK 8 (where this is done) with a program that has a few
>>>         thousands of threads:
>>>
>>>
>>>
>>>         JDK 7:
>>>         ..
>>>         2827.308: [sweeping nmethods, 0.0000020 secs]
>>>         2828.679: [sweeping nmethods, 0.0000030 secs]
>>>         2829.984: [sweeping nmethods, 0.0000030 secs]
>>>         2830.956: [sweeping nmethods, 0.0000030 secs]
>>>         ..
>>>
>>>         JDK 8:
>>>         ..
>>>         7368.634: [mark nmethods, 0.0177030 secs]
>>>         7369.587: [mark nmethods, 0.0178305 secs]
>>>         7370.479: [mark nmethods, 0.0180260 secs]
>>>         7371.503: [mark nmethods, 0.0186494 secs]
>>>         ..
>>>
>>>
>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150731/f72802b9/attachment.htm>

From ysr1729 at gmail.com  Fri Jul 31 19:08:50 2015
From: ysr1729 at gmail.com (Srinivas Ramakrishna)
Date: Fri, 31 Jul 2015 12:08:50 -0700
Subject: Cost of single-threaded nmethod hotness updates at each safepoint
	(in JDK 8)
In-Reply-To: <CABzyjy==w=riDMkGWk3FEmct-WaX91z=W2+fZk2eYAhHKCZ0gw@mail.gmail.com>
References: <CABzyjy=GHQ7aDyZyJg_4QgpqWuFkEPmwErq2pkLmFr7CRL8X6Q@mail.gmail.com>
	<CAHjP37FHJq_wEy3jak7qPnR38C2xTXHi+BgA+xk56aV+M=dpsQ@mail.gmail.com>
	<CABzyjyn1GaD7VKj-Lskn4UTZcJdckt_iqaSrOgWMxWh-7ehEOQ@mail.gmail.com>
	<55BBC1C0.3030709@oracle.com>
	<CABzyjy=UmTx9+HCTAxZ_R27fWiOcBqAxk_xLRioBJN=ZSJmGyQ@mail.gmail.com>
	<CABzyjy==w=riDMkGWk3FEmct-WaX91z=W2+fZk2eYAhHKCZ0gw@mail.gmail.com>
Message-ID: <CABzyjy=y1312mk+WP1260giQAmimnt2=PyJhP2+H_poxO6QyEw@mail.gmail.com>

BTW, i think some of those are counters that we added (a patch for which is
attached to the openjdk ticket i opened a few months ago, i think)...

-- ramki

On Fri, Jul 31, 2015 at 12:07 PM, Srinivas Ramakrishna <ysr1729 at gmail.com>
wrote:

> Hi Vladimir --
>
>
> Here's a snapshot of the counters:
>
> sun.ci.codeCacheCapacity=251658240
>
> sun.ci.codeCacheMaxCapacity=251658240
>
> sun.ci.codeCacheMethodsReclaimedNum=3450
>
> sun.ci.codeCacheSweepsTotalNum=58
>
> sun.ci.codeCacheSweepsTotalTimeMillis=1111
>
> sun.ci.codeCacheUsed=35888704
>
>
> Notice that the code cache usage is less that 35 MB, for the 240 MB
> capacity, yet it seems we have had 58 sweeps already, and safepoint cleanup
> says:
>
> [mark nmethods, 0.0165062 secs]
>
> Even if the two closures do little or no work, the single-threaded walk
> over deep stacks of a thousand threads will cost time for applications with
> many threads, and this is now done at each safepoint irrespective of the
> sweeper activity as far as I can tell. It seems as if this work should be
> somehow rolled up (via a suitable injection) into GC's thread walks that
> are done in parallel, rather than doing this in a pre-GC phase (unless I am
> mssing some reason that the sequencing is necessary, which it doesn't seem
> to be here).
>
> -- ramki
>
> On Fri, Jul 31, 2015 at 11:48 AM, Srinivas Ramakrishna <ysr1729 at gmail.com>
> wrote:
>
>> Hi Vladimir --
>>
>> I noticed the increase even with Initial and Reserved set to the default
>> of 240 MB, but actual usage much lower (less than a quarter).
>>
>> Look at this code path. Note that this is invoked at every safepoint
>> (although it says "periodically" in the comment).
>> In the mark_active_nmethods() method, there's a thread iteration in both
>> branches of the if. I haven't checked to
>> see which of the two was the culprit here, yet (if either).
>>
>> // Various cleaning tasks that should be done periodically at safepoints
>>
>> void SafepointSynchronize::do_cleanup_tasks() {
>>
>> ....
>>
>>   {
>>
>>     TraceTime t4("mark nmethods", TraceSafepointCleanupTime);
>>
>>     NMethodSweeper::mark_active_nmethods();
>>
>>   }
>>
>> ..
>>
>> }
>>
>>
>> void NMethodSweeper::mark_active_nmethods() {
>>
>>  ...
>>
>>   if (!sweep_in_progress()) {
>>
>>     _seen = 0;
>>
>>     _sweep_fractions_left = NmethodSweepFraction;
>>
>>     _current = CodeCache::first_nmethod();
>>
>>     _traversals += 1;
>>
>>     _total_time_this_sweep = Tickspan();
>>
>>
>>     if (PrintMethodFlushing) {
>>
>>       tty->print_cr("### Sweep: stack traversal %d", _traversals);
>>
>>     }
>>
>>     Threads::nmethods_do(&mark_activation_closure);
>>
>>
>>   } else {
>>
>>     // Only set hotness counter
>>
>>     Threads::nmethods_do(&set_hotness_closure);
>>
>>   }
>>
>>
>>   OrderAccess::storestore();
>>
>> }
>>
>> On Fri, Jul 31, 2015 at 11:43 AM, Vladimir Kozlov <
>> vladimir.kozlov at oracle.com> wrote:
>>
>>> Hi Ramki,
>>>
>>> Did you fill up CodeCache? It start scanning aggressive only with full
>>> CodeCache:
>>>
>>>   // Force stack scanning if there is only 10% free space in the code
>>> cache.
>>>   // We force stack scanning only non-profiled code heap gets full,
>>> since critical
>>>   // allocation go to the non-profiled heap and we must be make sure
>>> that there is
>>>   // enough space.
>>>   double free_percent = 1 /
>>> CodeCache::reverse_free_ratio(CodeBlobType::MethodNonProfiled) * 100;
>>>   if (free_percent <= StartAggressiveSweepingAt) {
>>>     do_stack_scanning();
>>>   }
>>>
>>> Vladimir
>>>
>>> On 7/31/15 11:33 AM, Srinivas Ramakrishna wrote:
>>>
>>>>
>>>> Yes.
>>>>
>>>>
>>>> On Fri, Jul 31, 2015 at 11:31 AM, Vitaly Davidovich <vitalyd at gmail.com
>>>> <mailto:vitalyd at gmail.com>> wrote:
>>>>
>>>>     Ramki, are you running tiered compilation?
>>>>
>>>>     sent from my phone
>>>>
>>>>     On Jul 31, 2015 2:19 PM, "Srinivas Ramakrishna" <ysr1729 at gmail.com
>>>>     <mailto:ysr1729 at gmail.com>> wrote:
>>>>
>>>>
>>>>         Hello GC and Compiler teams!
>>>>
>>>>         One of our services that runs with several thousand threads
>>>>         recently noticed an increase
>>>>         in safepoint stop times, but not gc times, upon transitioning to
>>>>         JDK 8.
>>>>
>>>>         Further investigation revealed that most of the delta was
>>>>         related to the so-called
>>>>         pre-gc/vmop "cleanup" phase when various book-keeping activities
>>>>         are performed,
>>>>         and more specifically in the portion that walks java thread
>>>>         stacks single-threaded (!)
>>>>         and updates the hotness counters for the active nmethods. This
>>>>         code appears to
>>>>         be new to JDK 8 (in jdk 7 one would walk the stacks only during
>>>>         code cache sweeps).
>>>>
>>>>         I have two questions:
>>>>         (1) has anyone else (typically, I'd expect applications with
>>>>         many hundreds or thousands of threads)
>>>>         noticed this regression?
>>>>         (2) Can we do better, for example, by:
>>>>                (a) doing these updates by walking thread stacks in
>>>>         multiple worker threads in parallel, or best of all:
>>>>                (b) doing these updates when we walk the thread stacks
>>>>         during GC, and skipping this phase entirely
>>>>                      for non-GC safepoints (with attendant loss in
>>>>         frequency of this update in low GC frequency
>>>>                      scenarios).
>>>>
>>>>         It seems kind of silly to do GC's with many multiple worker
>>>>         threads, but do these thread stack
>>>>         walks single-threaded when it is embarrasingly parallel (one
>>>>         could predicate the parallelization
>>>>         based on the measured stack sizes and thread population, if
>>>>         there was concern on the ovrhead of
>>>>         activating and deactivating the thread gangs for the work).
>>>>
>>>>         A followup question: Any guesses as to how code cache
>>>>         sweep/eviction quality might be compromised if one
>>>>         were to dispense with these hotness updates entirely (or at a
>>>>         much reduced frequency), as a temporary
>>>>         workaround to the performance problem?
>>>>
>>>>         Thoughts/Comments? In particular, has this issue been addressed
>>>>         perhaps in newer JVMs?
>>>>
>>>>         Thanks for any comments, feedback, pointers!
>>>>         -- ramki
>>>>
>>>>         PS: for comparison, here's data with +TraceSafepointCleanup from
>>>>         JDK 7 (first, where this isn't done)
>>>>         vs JDK 8 (where this is done) with a program that has a few
>>>>         thousands of threads:
>>>>
>>>>
>>>>
>>>>         JDK 7:
>>>>         ..
>>>>         2827.308: [sweeping nmethods, 0.0000020 secs]
>>>>         2828.679: [sweeping nmethods, 0.0000030 secs]
>>>>         2829.984: [sweeping nmethods, 0.0000030 secs]
>>>>         2830.956: [sweeping nmethods, 0.0000030 secs]
>>>>         ..
>>>>
>>>>         JDK 8:
>>>>         ..
>>>>         7368.634: [mark nmethods, 0.0177030 secs]
>>>>         7369.587: [mark nmethods, 0.0178305 secs]
>>>>         7370.479: [mark nmethods, 0.0180260 secs]
>>>>         7371.503: [mark nmethods, 0.0186494 secs]
>>>>         ..
>>>>
>>>>
>>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150731/65b269d0/attachment.htm>

From vladimir.kozlov at oracle.com  Fri Jul 31 21:28:35 2015
From: vladimir.kozlov at oracle.com (Vladimir Kozlov)
Date: Fri, 31 Jul 2015 14:28:35 -0700
Subject: Cost of single-threaded nmethod hotness updates at each safepoint
	(in JDK 8)
In-Reply-To: <CABzyjy=UmTx9+HCTAxZ_R27fWiOcBqAxk_xLRioBJN=ZSJmGyQ@mail.gmail.com>
References: <CABzyjy=GHQ7aDyZyJg_4QgpqWuFkEPmwErq2pkLmFr7CRL8X6Q@mail.gmail.com>
	<CAHjP37FHJq_wEy3jak7qPnR38C2xTXHi+BgA+xk56aV+M=dpsQ@mail.gmail.com>
	<CABzyjyn1GaD7VKj-Lskn4UTZcJdckt_iqaSrOgWMxWh-7ehEOQ@mail.gmail.com>
	<55BBC1C0.3030709@oracle.com>
	<CABzyjy=UmTx9+HCTAxZ_R27fWiOcBqAxk_xLRioBJN=ZSJmGyQ@mail.gmail.com>
Message-ID: <55BBE883.1080308@oracle.com>

Got it. Yes, it is issue with thousands java threads.
You are the first pointing this problem. File bug on compiler. We will 
look what we can do. Most likely we need parallelize this work.

Method's hotness is used only for UseCodeCacheFlushing. You can try to 
guard Threads::nmethods_do(&set_hotness_closure); with this flag and 
switch it off.

We need mark_as_seen_on_stack so leave it.

Thanks,
Vladimir

On 7/31/15 11:48 AM, Srinivas Ramakrishna wrote:
> Hi Vladimir --
>
> I noticed the increase even with Initial and Reserved set to the default
> of 240 MB, but actual usage much lower (less than a quarter).
>
> Look at this code path. Note that this is invoked at every safepoint
> (although it says "periodically" in the comment).
> In the mark_active_nmethods() method, there's a thread iteration in both
> branches of the if. I haven't checked to
> see which of the two was the culprit here, yet (if either).
>
> // Various cleaning tasks that should be done periodically at safepoints
>
> void SafepointSynchronize::do_cleanup_tasks() {
>
> ....
>
>    {
>
>      TraceTime t4("mark nmethods", TraceSafepointCleanupTime);
>
>      NMethodSweeper::mark_active_nmethods();
>
>    }
>
> ..
>
> }
>
>
> void NMethodSweeper::mark_active_nmethods() {
>
>   ...
>
>    if (!sweep_in_progress()) {
>
>      _seen = 0;
>
>      _sweep_fractions_left = NmethodSweepFraction;
>
>      _current = CodeCache::first_nmethod();
>
>      _traversals += 1;
>
>      _total_time_this_sweep = Tickspan();
>
>
>      if (PrintMethodFlushing) {
>
>        tty->print_cr("### Sweep: stack traversal %d", _traversals);
>
>      }
>
>      Threads::nmethods_do(&mark_activation_closure);
>
>
>    } else {
>
>      // Only set hotness counter
>
>      Threads::nmethods_do(&set_hotness_closure);
>
>    }
>
>
>    OrderAccess::storestore();
>
> }
>
>
> On Fri, Jul 31, 2015 at 11:43 AM, Vladimir Kozlov
> <vladimir.kozlov at oracle.com <mailto:vladimir.kozlov at oracle.com>> wrote:
>
>     Hi Ramki,
>
>     Did you fill up CodeCache? It start scanning aggressive only with
>     full CodeCache:
>
>        // Force stack scanning if there is only 10% free space in the
>     code cache.
>        // We force stack scanning only non-profiled code heap gets full,
>     since critical
>        // allocation go to the non-profiled heap and we must be make
>     sure that there is
>        // enough space.
>        double free_percent = 1 /
>     CodeCache::reverse_free_ratio(CodeBlobType::MethodNonProfiled) * 100;
>        if (free_percent <= StartAggressiveSweepingAt) {
>          do_stack_scanning();
>        }
>
>     Vladimir
>
>     On 7/31/15 11:33 AM, Srinivas Ramakrishna wrote:
>
>
>         Yes.
>
>
>         On Fri, Jul 31, 2015 at 11:31 AM, Vitaly Davidovich
>         <vitalyd at gmail.com <mailto:vitalyd at gmail.com>
>         <mailto:vitalyd at gmail.com <mailto:vitalyd at gmail.com>>> wrote:
>
>              Ramki, are you running tiered compilation?
>
>              sent from my phone
>
>              On Jul 31, 2015 2:19 PM, "Srinivas Ramakrishna"
>         <ysr1729 at gmail.com <mailto:ysr1729 at gmail.com>
>              <mailto:ysr1729 at gmail.com <mailto:ysr1729 at gmail.com>>> wrote:
>
>
>                  Hello GC and Compiler teams!
>
>                  One of our services that runs with several thousand threads
>                  recently noticed an increase
>                  in safepoint stop times, but not gc times, upon
>         transitioning to
>                  JDK 8.
>
>                  Further investigation revealed that most of the delta was
>                  related to the so-called
>                  pre-gc/vmop "cleanup" phase when various book-keeping
>         activities
>                  are performed,
>                  and more specifically in the portion that walks java thread
>                  stacks single-threaded (!)
>                  and updates the hotness counters for the active
>         nmethods. This
>                  code appears to
>                  be new to JDK 8 (in jdk 7 one would walk the stacks
>         only during
>                  code cache sweeps).
>
>                  I have two questions:
>                  (1) has anyone else (typically, I'd expect applications
>         with
>                  many hundreds or thousands of threads)
>                  noticed this regression?
>                  (2) Can we do better, for example, by:
>                         (a) doing these updates by walking thread stacks in
>                  multiple worker threads in parallel, or best of all:
>                         (b) doing these updates when we walk the thread
>         stacks
>                  during GC, and skipping this phase entirely
>                               for non-GC safepoints (with attendant loss in
>                  frequency of this update in low GC frequency
>                               scenarios).
>
>                  It seems kind of silly to do GC's with many multiple worker
>                  threads, but do these thread stack
>                  walks single-threaded when it is embarrasingly parallel
>         (one
>                  could predicate the parallelization
>                  based on the measured stack sizes and thread population, if
>                  there was concern on the ovrhead of
>                  activating and deactivating the thread gangs for the work).
>
>                  A followup question: Any guesses as to how code cache
>                  sweep/eviction quality might be compromised if one
>                  were to dispense with these hotness updates entirely
>         (or at a
>                  much reduced frequency), as a temporary
>                  workaround to the performance problem?
>
>                  Thoughts/Comments? In particular, has this issue been
>         addressed
>                  perhaps in newer JVMs?
>
>                  Thanks for any comments, feedback, pointers!
>                  -- ramki
>
>                  PS: for comparison, here's data with
>         +TraceSafepointCleanup from
>                  JDK 7 (first, where this isn't done)
>                  vs JDK 8 (where this is done) with a program that has a few
>                  thousands of threads:
>
>
>
>                  JDK 7:
>                  ..
>                  2827.308: [sweeping nmethods, 0.0000020 secs]
>                  2828.679: [sweeping nmethods, 0.0000030 secs]
>                  2829.984: [sweeping nmethods, 0.0000030 secs]
>                  2830.956: [sweeping nmethods, 0.0000030 secs]
>                  ..
>
>                  JDK 8:
>                  ..
>                  7368.634: [mark nmethods, 0.0177030 secs]
>                  7369.587: [mark nmethods, 0.0178305 secs]
>                  7370.479: [mark nmethods, 0.0180260 secs]
>                  7371.503: [mark nmethods, 0.0186494 secs]
>                  ..
>
>
>


From ecki at zusammenkunft.net  Fri Jul 31 21:44:45 2015
From: ecki at zusammenkunft.net (Bernd Eckenfels)
Date: Fri, 31 Jul 2015 23:44:45 +0200
Subject: Cost of single-threaded nmethod hotness updates at each
	safepoint (in JDK 8)
In-Reply-To: <CABzyjy==w=riDMkGWk3FEmct-WaX91z=W2+fZk2eYAhHKCZ0gw@mail.gmail.com>
References: <CABzyjy=GHQ7aDyZyJg_4QgpqWuFkEPmwErq2pkLmFr7CRL8X6Q@mail.gmail.com>
	<CAHjP37FHJq_wEy3jak7qPnR38C2xTXHi+BgA+xk56aV+M=dpsQ@mail.gmail.com>
	<CABzyjyn1GaD7VKj-Lskn4UTZcJdckt_iqaSrOgWMxWh-7ehEOQ@mail.gmail.com>
	<55BBC1C0.3030709@oracle.com>
	<CABzyjy=UmTx9+HCTAxZ_R27fWiOcBqAxk_xLRioBJN=ZSJmGyQ@mail.gmail.com>
	<CABzyjy==w=riDMkGWk3FEmct-WaX91z=W2+fZk2eYAhHKCZ0gw@mail.gmail.com>
Message-ID: <20150731234445.0000459c.ecki@zusammenkunft.net>

Am Fri, 31 Jul 2015 12:07:18 -0700
schrieb Srinivas Ramakrishna <ysr1729 at gmail.com>:

> sun.ci.codeCacheSweepsTotalNum=58
...
> Notice that the code cache usage is less that 35 MB, for the 240 MB
> capacity, yet it seems we have had 58 sweeps already

I would also be interested in what causes this. Is this caused by
System.gc maybe? (we do see sweeps and decreasing code cache usage on
systems where no pressure should exist).

Gruss
Bernd


From ysr1729 at gmail.com  Fri Jul 31 22:02:06 2015
From: ysr1729 at gmail.com (Srinivas Ramakrishna)
Date: Fri, 31 Jul 2015 15:02:06 -0700
Subject: Cost of single-threaded nmethod hotness updates at each safepoint
	(in JDK 8)
In-Reply-To: <55BBE883.1080308@oracle.com>
References: <CABzyjy=GHQ7aDyZyJg_4QgpqWuFkEPmwErq2pkLmFr7CRL8X6Q@mail.gmail.com>
	<CAHjP37FHJq_wEy3jak7qPnR38C2xTXHi+BgA+xk56aV+M=dpsQ@mail.gmail.com>
	<CABzyjyn1GaD7VKj-Lskn4UTZcJdckt_iqaSrOgWMxWh-7ehEOQ@mail.gmail.com>
	<55BBC1C0.3030709@oracle.com>
	<CABzyjy=UmTx9+HCTAxZ_R27fWiOcBqAxk_xLRioBJN=ZSJmGyQ@mail.gmail.com>
	<55BBE883.1080308@oracle.com>
Message-ID: <CABzyjykMwmdWLqa9=J6uiF6Lji51r_zHX_TDFxWKdTgrdQP4tA@mail.gmail.com>

OK, will do and add you as watcher; thanks Vladimir! (don't yet know if
with tiered and a necessarily bounded, if large, code cache whether
flushing will in fact eventually become necessary, wrt yr suggested
temporary workaround.)

Have a good weekend!
-- ramki

On Fri, Jul 31, 2015 at 2:28 PM, Vladimir Kozlov <vladimir.kozlov at oracle.com
> wrote:

> Got it. Yes, it is issue with thousands java threads.
> You are the first pointing this problem. File bug on compiler. We will
> look what we can do. Most likely we need parallelize this work.
>
> Method's hotness is used only for UseCodeCacheFlushing. You can try to
> guard Threads::nmethods_do(&set_hotness_closure); with this flag and switch
> it off.
>
> We need mark_as_seen_on_stack so leave it.
>
> Thanks,
> Vladimir
>
>
> On 7/31/15 11:48 AM, Srinivas Ramakrishna wrote:
>
>> Hi Vladimir --
>>
>> I noticed the increase even with Initial and Reserved set to the default
>> of 240 MB, but actual usage much lower (less than a quarter).
>>
>> Look at this code path. Note that this is invoked at every safepoint
>> (although it says "periodically" in the comment).
>> In the mark_active_nmethods() method, there's a thread iteration in both
>> branches of the if. I haven't checked to
>> see which of the two was the culprit here, yet (if either).
>>
>> // Various cleaning tasks that should be done periodically at safepoints
>>
>> void SafepointSynchronize::do_cleanup_tasks() {
>>
>> ....
>>
>>    {
>>
>>      TraceTime t4("mark nmethods", TraceSafepointCleanupTime);
>>
>>      NMethodSweeper::mark_active_nmethods();
>>
>>    }
>>
>> ..
>>
>> }
>>
>>
>> void NMethodSweeper::mark_active_nmethods() {
>>
>>   ...
>>
>>    if (!sweep_in_progress()) {
>>
>>      _seen = 0;
>>
>>      _sweep_fractions_left = NmethodSweepFraction;
>>
>>      _current = CodeCache::first_nmethod();
>>
>>      _traversals += 1;
>>
>>      _total_time_this_sweep = Tickspan();
>>
>>
>>      if (PrintMethodFlushing) {
>>
>>        tty->print_cr("### Sweep: stack traversal %d", _traversals);
>>
>>      }
>>
>>      Threads::nmethods_do(&mark_activation_closure);
>>
>>
>>    } else {
>>
>>      // Only set hotness counter
>>
>>      Threads::nmethods_do(&set_hotness_closure);
>>
>>    }
>>
>>
>>    OrderAccess::storestore();
>>
>> }
>>
>>
>> On Fri, Jul 31, 2015 at 11:43 AM, Vladimir Kozlov
>> <vladimir.kozlov at oracle.com <mailto:vladimir.kozlov at oracle.com>> wrote:
>>
>>     Hi Ramki,
>>
>>     Did you fill up CodeCache? It start scanning aggressive only with
>>     full CodeCache:
>>
>>        // Force stack scanning if there is only 10% free space in the
>>     code cache.
>>        // We force stack scanning only non-profiled code heap gets full,
>>     since critical
>>        // allocation go to the non-profiled heap and we must be make
>>     sure that there is
>>        // enough space.
>>        double free_percent = 1 /
>>     CodeCache::reverse_free_ratio(CodeBlobType::MethodNonProfiled) * 100;
>>        if (free_percent <= StartAggressiveSweepingAt) {
>>          do_stack_scanning();
>>        }
>>
>>     Vladimir
>>
>>     On 7/31/15 11:33 AM, Srinivas Ramakrishna wrote:
>>
>>
>>         Yes.
>>
>>
>>         On Fri, Jul 31, 2015 at 11:31 AM, Vitaly Davidovich
>>         <vitalyd at gmail.com <mailto:vitalyd at gmail.com>
>>         <mailto:vitalyd at gmail.com <mailto:vitalyd at gmail.com>>> wrote:
>>
>>              Ramki, are you running tiered compilation?
>>
>>              sent from my phone
>>
>>              On Jul 31, 2015 2:19 PM, "Srinivas Ramakrishna"
>>         <ysr1729 at gmail.com <mailto:ysr1729 at gmail.com>
>>              <mailto:ysr1729 at gmail.com <mailto:ysr1729 at gmail.com>>>
>> wrote:
>>
>>
>>                  Hello GC and Compiler teams!
>>
>>                  One of our services that runs with several thousand
>> threads
>>                  recently noticed an increase
>>                  in safepoint stop times, but not gc times, upon
>>         transitioning to
>>                  JDK 8.
>>
>>                  Further investigation revealed that most of the delta was
>>                  related to the so-called
>>                  pre-gc/vmop "cleanup" phase when various book-keeping
>>         activities
>>                  are performed,
>>                  and more specifically in the portion that walks java
>> thread
>>                  stacks single-threaded (!)
>>                  and updates the hotness counters for the active
>>         nmethods. This
>>                  code appears to
>>                  be new to JDK 8 (in jdk 7 one would walk the stacks
>>         only during
>>                  code cache sweeps).
>>
>>                  I have two questions:
>>                  (1) has anyone else (typically, I'd expect applications
>>         with
>>                  many hundreds or thousands of threads)
>>                  noticed this regression?
>>                  (2) Can we do better, for example, by:
>>                         (a) doing these updates by walking thread stacks
>> in
>>                  multiple worker threads in parallel, or best of all:
>>                         (b) doing these updates when we walk the thread
>>         stacks
>>                  during GC, and skipping this phase entirely
>>                               for non-GC safepoints (with attendant loss
>> in
>>                  frequency of this update in low GC frequency
>>                               scenarios).
>>
>>                  It seems kind of silly to do GC's with many multiple
>> worker
>>                  threads, but do these thread stack
>>                  walks single-threaded when it is embarrasingly parallel
>>         (one
>>                  could predicate the parallelization
>>                  based on the measured stack sizes and thread population,
>> if
>>                  there was concern on the ovrhead of
>>                  activating and deactivating the thread gangs for the
>> work).
>>
>>                  A followup question: Any guesses as to how code cache
>>                  sweep/eviction quality might be compromised if one
>>                  were to dispense with these hotness updates entirely
>>         (or at a
>>                  much reduced frequency), as a temporary
>>                  workaround to the performance problem?
>>
>>                  Thoughts/Comments? In particular, has this issue been
>>         addressed
>>                  perhaps in newer JVMs?
>>
>>                  Thanks for any comments, feedback, pointers!
>>                  -- ramki
>>
>>                  PS: for comparison, here's data with
>>         +TraceSafepointCleanup from
>>                  JDK 7 (first, where this isn't done)
>>                  vs JDK 8 (where this is done) with a program that has a
>> few
>>                  thousands of threads:
>>
>>
>>
>>                  JDK 7:
>>                  ..
>>                  2827.308: [sweeping nmethods, 0.0000020 secs]
>>                  2828.679: [sweeping nmethods, 0.0000030 secs]
>>                  2829.984: [sweeping nmethods, 0.0000030 secs]
>>                  2830.956: [sweeping nmethods, 0.0000030 secs]
>>                  ..
>>
>>                  JDK 8:
>>                  ..
>>                  7368.634: [mark nmethods, 0.0177030 secs]
>>                  7369.587: [mark nmethods, 0.0178305 secs]
>>                  7370.479: [mark nmethods, 0.0180260 secs]
>>                  7371.503: [mark nmethods, 0.0186494 secs]
>>                  ..
>>
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150731/22be048c/attachment.htm>

From ysr1729 at gmail.com  Fri Jul 31 22:04:28 2015
From: ysr1729 at gmail.com (Srinivas Ramakrishna)
Date: Fri, 31 Jul 2015 15:04:28 -0700
Subject: Cost of single-threaded nmethod hotness updates at each safepoint
	(in JDK 8)
In-Reply-To: <20150731234445.0000459c.ecki@zusammenkunft.net>
References: <CABzyjy=GHQ7aDyZyJg_4QgpqWuFkEPmwErq2pkLmFr7CRL8X6Q@mail.gmail.com>
	<CAHjP37FHJq_wEy3jak7qPnR38C2xTXHi+BgA+xk56aV+M=dpsQ@mail.gmail.com>
	<CABzyjyn1GaD7VKj-Lskn4UTZcJdckt_iqaSrOgWMxWh-7ehEOQ@mail.gmail.com>
	<55BBC1C0.3030709@oracle.com>
	<CABzyjy=UmTx9+HCTAxZ_R27fWiOcBqAxk_xLRioBJN=ZSJmGyQ@mail.gmail.com>
	<CABzyjy==w=riDMkGWk3FEmct-WaX91z=W2+fZk2eYAhHKCZ0gw@mail.gmail.com>
	<20150731234445.0000459c.ecki@zusammenkunft.net>
Message-ID: <CABzyjymUoEDLOfF74nEzVOOPg7ETSXYpRQ=SO3suzVWFW=sDkQ@mail.gmail.com>

Hi Bernd --

It doesn't seem coupled to GC; here's an example snapshot:

sun.ci.codeCacheCapacity=13041664
sun.ci.codeCacheMaxCapacity=50331648
sun.ci.codeCacheMethodsReclaimedNum=4281
sun.ci.codeCacheSweepsTotalNum=409
sun.ci.codeCacheSweepsTotalTimeMillis=1541
sun.ci.codeCacheUsed=10864704

sun.gc.collector.0.invocations=6319
sun.gc.collector.1.invocations=6

BTW, to a question of Vitaly's on this thread earlier, this code executes
irrespective of whether you are running tiered or not (the above is from
tiered explictly off  -- earlier ones were with tiered on --, yet note the
excessive time in the stack walk as part of this code in JDK 8):

[mark nmethods, 0.0171828 secs]

On a similar note (and to Vladimir's earlier note of turning off code cache
flushing) and with the understanding that there were lots of changes
related to tiered compilation and code cache flushing between 7 ad 8,
turning on Tiered in 7 leads to the occasional (rare) long safepoint for
the stack walks, but else not.

And finally the same issue must exist in 9 as well, albeit based on code
inspection, not running with JDK 9 yet.
Have a good weekend!
-- ramki

On Fri, Jul 31, 2015 at 2:44 PM, Bernd Eckenfels <ecki at zusammenkunft.net>
wrote:

> Am Fri, 31 Jul 2015 12:07:18 -0700
> schrieb Srinivas Ramakrishna <ysr1729 at gmail.com>:
>
> > sun.ci.codeCacheSweepsTotalNum=58
> ...
> > Notice that the code cache usage is less that 35 MB, for the 240 MB
> > capacity, yet it seems we have had 58 sweeps already
>
> I would also be interested in what causes this. Is this caused by
> System.gc maybe? (we do see sweeps and decreasing code cache usage on
> systems where no pressure should exist).
>
> Gruss
> Bernd
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150731/2d27ac13/attachment.htm>

From vitalyd at gmail.com  Fri Jul 31 22:08:14 2015
From: vitalyd at gmail.com (Vitaly Davidovich)
Date: Fri, 31 Jul 2015 18:08:14 -0400
Subject: Cost of single-threaded nmethod hotness updates at each safepoint
	(in JDK 8)
In-Reply-To: <CABzyjykMwmdWLqa9=J6uiF6Lji51r_zHX_TDFxWKdTgrdQP4tA@mail.gmail.com>
References: <CABzyjy=GHQ7aDyZyJg_4QgpqWuFkEPmwErq2pkLmFr7CRL8X6Q@mail.gmail.com>
	<CAHjP37FHJq_wEy3jak7qPnR38C2xTXHi+BgA+xk56aV+M=dpsQ@mail.gmail.com>
	<CABzyjyn1GaD7VKj-Lskn4UTZcJdckt_iqaSrOgWMxWh-7ehEOQ@mail.gmail.com>
	<55BBC1C0.3030709@oracle.com>
	<CABzyjy=UmTx9+HCTAxZ_R27fWiOcBqAxk_xLRioBJN=ZSJmGyQ@mail.gmail.com>
	<55BBE883.1080308@oracle.com>
	<CABzyjykMwmdWLqa9=J6uiF6Lji51r_zHX_TDFxWKdTgrdQP4tA@mail.gmail.com>
Message-ID: <CAHjP37HRYF-kB-qMGeK3tAL4A9cvzHu_P0rCYPwbnAtn6dViTQ@mail.gmail.com>

Ramki, are you actually seeing better peak perf with tiered than C2? I
experimented with it on a real workload and it was a net loss for peak perf
(anywhere from 8-20% worse than C2, but also quite unstable); this was with
a very large code cache to play it safe, but no other tuning.

sent from my phone
On Jul 31, 2015 6:02 PM, "Srinivas Ramakrishna" <ysr1729 at gmail.com> wrote:

> OK, will do and add you as watcher; thanks Vladimir! (don't yet know if
> with tiered and a necessarily bounded, if large, code cache whether
> flushing will in fact eventually become necessary, wrt yr suggested
> temporary workaround.)
>
> Have a good weekend!
> -- ramki
>
> On Fri, Jul 31, 2015 at 2:28 PM, Vladimir Kozlov <
> vladimir.kozlov at oracle.com> wrote:
>
>> Got it. Yes, it is issue with thousands java threads.
>> You are the first pointing this problem. File bug on compiler. We will
>> look what we can do. Most likely we need parallelize this work.
>>
>> Method's hotness is used only for UseCodeCacheFlushing. You can try to
>> guard Threads::nmethods_do(&set_hotness_closure); with this flag and switch
>> it off.
>>
>> We need mark_as_seen_on_stack so leave it.
>>
>> Thanks,
>> Vladimir
>>
>>
>> On 7/31/15 11:48 AM, Srinivas Ramakrishna wrote:
>>
>>> Hi Vladimir --
>>>
>>> I noticed the increase even with Initial and Reserved set to the default
>>> of 240 MB, but actual usage much lower (less than a quarter).
>>>
>>> Look at this code path. Note that this is invoked at every safepoint
>>> (although it says "periodically" in the comment).
>>> In the mark_active_nmethods() method, there's a thread iteration in both
>>> branches of the if. I haven't checked to
>>> see which of the two was the culprit here, yet (if either).
>>>
>>> // Various cleaning tasks that should be done periodically at safepoints
>>>
>>> void SafepointSynchronize::do_cleanup_tasks() {
>>>
>>> ....
>>>
>>>    {
>>>
>>>      TraceTime t4("mark nmethods", TraceSafepointCleanupTime);
>>>
>>>      NMethodSweeper::mark_active_nmethods();
>>>
>>>    }
>>>
>>> ..
>>>
>>> }
>>>
>>>
>>> void NMethodSweeper::mark_active_nmethods() {
>>>
>>>   ...
>>>
>>>    if (!sweep_in_progress()) {
>>>
>>>      _seen = 0;
>>>
>>>      _sweep_fractions_left = NmethodSweepFraction;
>>>
>>>      _current = CodeCache::first_nmethod();
>>>
>>>      _traversals += 1;
>>>
>>>      _total_time_this_sweep = Tickspan();
>>>
>>>
>>>      if (PrintMethodFlushing) {
>>>
>>>        tty->print_cr("### Sweep: stack traversal %d", _traversals);
>>>
>>>      }
>>>
>>>      Threads::nmethods_do(&mark_activation_closure);
>>>
>>>
>>>    } else {
>>>
>>>      // Only set hotness counter
>>>
>>>      Threads::nmethods_do(&set_hotness_closure);
>>>
>>>    }
>>>
>>>
>>>    OrderAccess::storestore();
>>>
>>> }
>>>
>>>
>>> On Fri, Jul 31, 2015 at 11:43 AM, Vladimir Kozlov
>>> <vladimir.kozlov at oracle.com <mailto:vladimir.kozlov at oracle.com>> wrote:
>>>
>>>     Hi Ramki,
>>>
>>>     Did you fill up CodeCache? It start scanning aggressive only with
>>>     full CodeCache:
>>>
>>>        // Force stack scanning if there is only 10% free space in the
>>>     code cache.
>>>        // We force stack scanning only non-profiled code heap gets full,
>>>     since critical
>>>        // allocation go to the non-profiled heap and we must be make
>>>     sure that there is
>>>        // enough space.
>>>        double free_percent = 1 /
>>>     CodeCache::reverse_free_ratio(CodeBlobType::MethodNonProfiled) * 100;
>>>        if (free_percent <= StartAggressiveSweepingAt) {
>>>          do_stack_scanning();
>>>        }
>>>
>>>     Vladimir
>>>
>>>     On 7/31/15 11:33 AM, Srinivas Ramakrishna wrote:
>>>
>>>
>>>         Yes.
>>>
>>>
>>>         On Fri, Jul 31, 2015 at 11:31 AM, Vitaly Davidovich
>>>         <vitalyd at gmail.com <mailto:vitalyd at gmail.com>
>>>         <mailto:vitalyd at gmail.com <mailto:vitalyd at gmail.com>>> wrote:
>>>
>>>              Ramki, are you running tiered compilation?
>>>
>>>              sent from my phone
>>>
>>>              On Jul 31, 2015 2:19 PM, "Srinivas Ramakrishna"
>>>         <ysr1729 at gmail.com <mailto:ysr1729 at gmail.com>
>>>              <mailto:ysr1729 at gmail.com <mailto:ysr1729 at gmail.com>>>
>>> wrote:
>>>
>>>
>>>                  Hello GC and Compiler teams!
>>>
>>>                  One of our services that runs with several thousand
>>> threads
>>>                  recently noticed an increase
>>>                  in safepoint stop times, but not gc times, upon
>>>         transitioning to
>>>                  JDK 8.
>>>
>>>                  Further investigation revealed that most of the delta
>>> was
>>>                  related to the so-called
>>>                  pre-gc/vmop "cleanup" phase when various book-keeping
>>>         activities
>>>                  are performed,
>>>                  and more specifically in the portion that walks java
>>> thread
>>>                  stacks single-threaded (!)
>>>                  and updates the hotness counters for the active
>>>         nmethods. This
>>>                  code appears to
>>>                  be new to JDK 8 (in jdk 7 one would walk the stacks
>>>         only during
>>>                  code cache sweeps).
>>>
>>>                  I have two questions:
>>>                  (1) has anyone else (typically, I'd expect applications
>>>         with
>>>                  many hundreds or thousands of threads)
>>>                  noticed this regression?
>>>                  (2) Can we do better, for example, by:
>>>                         (a) doing these updates by walking thread stacks
>>> in
>>>                  multiple worker threads in parallel, or best of all:
>>>                         (b) doing these updates when we walk the thread
>>>         stacks
>>>                  during GC, and skipping this phase entirely
>>>                               for non-GC safepoints (with attendant loss
>>> in
>>>                  frequency of this update in low GC frequency
>>>                               scenarios).
>>>
>>>                  It seems kind of silly to do GC's with many multiple
>>> worker
>>>                  threads, but do these thread stack
>>>                  walks single-threaded when it is embarrasingly parallel
>>>         (one
>>>                  could predicate the parallelization
>>>                  based on the measured stack sizes and thread
>>> population, if
>>>                  there was concern on the ovrhead of
>>>                  activating and deactivating the thread gangs for the
>>> work).
>>>
>>>                  A followup question: Any guesses as to how code cache
>>>                  sweep/eviction quality might be compromised if one
>>>                  were to dispense with these hotness updates entirely
>>>         (or at a
>>>                  much reduced frequency), as a temporary
>>>                  workaround to the performance problem?
>>>
>>>                  Thoughts/Comments? In particular, has this issue been
>>>         addressed
>>>                  perhaps in newer JVMs?
>>>
>>>                  Thanks for any comments, feedback, pointers!
>>>                  -- ramki
>>>
>>>                  PS: for comparison, here's data with
>>>         +TraceSafepointCleanup from
>>>                  JDK 7 (first, where this isn't done)
>>>                  vs JDK 8 (where this is done) with a program that has a
>>> few
>>>                  thousands of threads:
>>>
>>>
>>>
>>>                  JDK 7:
>>>                  ..
>>>                  2827.308: [sweeping nmethods, 0.0000020 secs]
>>>                  2828.679: [sweeping nmethods, 0.0000030 secs]
>>>                  2829.984: [sweeping nmethods, 0.0000030 secs]
>>>                  2830.956: [sweeping nmethods, 0.0000030 secs]
>>>                  ..
>>>
>>>                  JDK 8:
>>>                  ..
>>>                  7368.634: [mark nmethods, 0.0177030 secs]
>>>                  7369.587: [mark nmethods, 0.0178305 secs]
>>>                  7370.479: [mark nmethods, 0.0180260 secs]
>>>                  7371.503: [mark nmethods, 0.0186494 secs]
>>>                  ..
>>>
>>>
>>>
>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150731/79dac5cc/attachment.htm>

From vitalyd at gmail.com  Fri Jul 31 22:10:51 2015
From: vitalyd at gmail.com (Vitaly Davidovich)
Date: Fri, 31 Jul 2015 18:10:51 -0400
Subject: Cost of single-threaded nmethod hotness updates at each safepoint
	(in JDK 8)
In-Reply-To: <CABzyjymUoEDLOfF74nEzVOOPg7ETSXYpRQ=SO3suzVWFW=sDkQ@mail.gmail.com>
References: <CABzyjy=GHQ7aDyZyJg_4QgpqWuFkEPmwErq2pkLmFr7CRL8X6Q@mail.gmail.com>
	<CAHjP37FHJq_wEy3jak7qPnR38C2xTXHi+BgA+xk56aV+M=dpsQ@mail.gmail.com>
	<CABzyjyn1GaD7VKj-Lskn4UTZcJdckt_iqaSrOgWMxWh-7ehEOQ@mail.gmail.com>
	<55BBC1C0.3030709@oracle.com>
	<CABzyjy=UmTx9+HCTAxZ_R27fWiOcBqAxk_xLRioBJN=ZSJmGyQ@mail.gmail.com>
	<CABzyjy==w=riDMkGWk3FEmct-WaX91z=W2+fZk2eYAhHKCZ0gw@mail.gmail.com>
	<20150731234445.0000459c.ecki@zusammenkunft.net>
	<CABzyjymUoEDLOfF74nEzVOOPg7ETSXYpRQ=SO3suzVWFW=sDkQ@mail.gmail.com>
Message-ID: <CAHjP37Ev=jmtfJxVfnBeYdv0Gz5bDPC2i2a7KJ++SNnMFEuRaQ@mail.gmail.com>

Yes, I misremembered thinking turning off tiered disabled sweeps at
safepoints; sorry for the noise.

sent from my phone
On Jul 31, 2015 6:04 PM, "Srinivas Ramakrishna" <ysr1729 at gmail.com> wrote:

>
> Hi Bernd --
>
> It doesn't seem coupled to GC; here's an example snapshot:
>
> sun.ci.codeCacheCapacity=13041664
> sun.ci.codeCacheMaxCapacity=50331648
> sun.ci.codeCacheMethodsReclaimedNum=4281
> sun.ci.codeCacheSweepsTotalNum=409
> sun.ci.codeCacheSweepsTotalTimeMillis=1541
> sun.ci.codeCacheUsed=10864704
>
> sun.gc.collector.0.invocations=6319
> sun.gc.collector.1.invocations=6
>
> BTW, to a question of Vitaly's on this thread earlier, this code executes
> irrespective of whether you are running tiered or not (the above is from
> tiered explictly off  -- earlier ones were with tiered on --, yet note the
> excessive time in the stack walk as part of this code in JDK 8):
>
> [mark nmethods, 0.0171828 secs]
>
> On a similar note (and to Vladimir's earlier note of turning off code
> cache flushing) and with the understanding that there were lots of changes
> related to tiered compilation and code cache flushing between 7 ad 8,
> turning on Tiered in 7 leads to the occasional (rare) long safepoint for
> the stack walks, but else not.
>
> And finally the same issue must exist in 9 as well, albeit based on code
> inspection, not running with JDK 9 yet.
> Have a good weekend!
> -- ramki
>
> On Fri, Jul 31, 2015 at 2:44 PM, Bernd Eckenfels <ecki at zusammenkunft.net>
> wrote:
>
>> Am Fri, 31 Jul 2015 12:07:18 -0700
>> schrieb Srinivas Ramakrishna <ysr1729 at gmail.com>:
>>
>> > sun.ci.codeCacheSweepsTotalNum=58
>> ...
>> > Notice that the code cache usage is less that 35 MB, for the 240 MB
>> > capacity, yet it seems we have had 58 sweeps already
>>
>> I would also be interested in what causes this. Is this caused by
>> System.gc maybe? (we do see sweeps and decreasing code cache usage on
>> systems where no pressure should exist).
>>
>> Gruss
>> Bernd
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150731/3f5bf99a/attachment.htm>

From daniel.fuchs at oracle.com  Thu Jul 30 09:12:42 2015
From: daniel.fuchs at oracle.com (Daniel Fuchs)
Date: Thu, 30 Jul 2015 11:12:42 +0200
Subject: RFR: 8132306: java/lang/ref/ReferenceEnqueue.java fails with
	"RuntimeException: Error: poll() returned null; expected ref object"
In-Reply-To: <55B9E293.80102@gmail.com>
References: <BA1C0BEB-F342-437A-87E7-029660BB5348@oracle.com>	<55B88FAB.4080302@oracle.com>	<929281CB-55E8-4A0C-A8D1-85ED37348E12@oracle.com>
	<55B9E293.80102@gmail.com>
Message-ID: <55B9EA8A.1080102@oracle.com>

Hi Peter,

I'm glad you're looking at this too! We can't have too many eyes :-)

On 30/07/15 10:38, Peter Levart wrote:
> Suppose we have a Reference 'r' and it's associated ReferenceQueue 'q'.
> Currently it can happen that the following evaluates to true, which is
> surprising:
>
> q.poll() == null && r.isEnqueued()
>

But on the other hand this can only happen if two different
threads are polling the queue - in which case only one of them
will get the reference. In such a situation, the symmetric condition
would not be surprising (as the other thread would
get q.poll() != null):

    r.isEnqueued() && q.poll() == null

The original bug fixed by Kim is more surprising, because there's only
one Thread doing the polling, and it does get:

    r.isEnqueud() && q.poll() == null

although nobody else is polling the queue.
This should no longer occur in this situation with Kim's fix.

cheers,

-- daniel


From daniel.fuchs at oracle.com  Thu Jul 30 10:48:55 2015
From: daniel.fuchs at oracle.com (Daniel Fuchs)
Date: Thu, 30 Jul 2015 12:48:55 +0200
Subject: RFR: 8132306: java/lang/ref/ReferenceEnqueue.java fails with
	"RuntimeException: Error: poll() returned null; expected ref object"
In-Reply-To: <55B9FA54.5040902@gmail.com>
References: <BA1C0BEB-F342-437A-87E7-029660BB5348@oracle.com>
	<55B88FAB.4080302@oracle.com>
	<929281CB-55E8-4A0C-A8D1-85ED37348E12@oracle.com>
	<55B9E293.80102@gmail.com> <55B9EA8A.1080102@oracle.com>
	<55B9FA54.5040902@gmail.com>
Message-ID: <55BA0117.6040203@oracle.com>

On 30/07/15 12:20, Peter Levart wrote:
>
>
> On 07/30/2015 11:12 AM, Daniel Fuchs wrote:
>> Hi Peter,
>>
>> I'm glad you're looking at this too! We can't have too many eyes :-)
>>
>> On 30/07/15 10:38, Peter Levart wrote:
>>> Suppose we have a Reference 'r' and it's associated ReferenceQueue 'q'.
>>> Currently it can happen that the following evaluates to true, which is
>>> surprising:
>>>
>>> q.poll() == null && r.isEnqueued()
>>>
>>
>> But on the other hand this can only happen if two different
>> threads are polling the queue - in which case only one of them
>> will get the reference. In such a situation, the symmetric condition
>> would not be surprising (as the other thread would
>> get q.poll() != null):
>>
>>    r.isEnqueued() && q.poll() == null
>>
>> The original bug fixed by Kim is more surprising, because there's only
>> one Thread doing the polling, and it does get:
>>
>>    r.isEnqueud() && q.poll() == null
>>
>> although nobody else is polling the queue.
>> This should no longer occur in this situation with Kim's fix.
>>
>> cheers,
>>
>> -- daniel
>
> Yes, these are two different issues. The one Kim has fixed is the race
> of above expressions with Queue.enqueue() (when the queue is changing
> from the associated instance to ENQUEUED). The one I'm pointing at and
> Kim has already identified as potential issue is the race of the
> following expression:
>
> r.isEnqueued() && q.poll() == null && r.isEnqueued() ==> true
>
> ....with Queue.[realy]poll() (when the queue is changing from ENQUEUED
> to NULL).
>
> Which only happens if at least two threads are polling the queue, but is
> equally surprising and prone to cause bugs, don't you think?

Hi Peter,

Yes - this is also surprising. Is it prone to cause bugs?
possibly - but how serious I'm not sure.
Is it 'equally' surprising - well - that was the point of my argument:
there are two threads polling the same queue - so one of them should
get null... Though I agree that in this case - 'r' should be seen
has having changed state...

The question for me is whether this should be fixed in the same
changeset - or whether we should make it in another changeset...

cheers, and thanks for pointing it out!

-- daniel

>
> Regards, Peter
>
>



From daniel.fuchs at oracle.com  Thu Jul 30 12:38:18 2015
From: daniel.fuchs at oracle.com (Daniel Fuchs)
Date: Thu, 30 Jul 2015 14:38:18 +0200
Subject: RFR: 8132306: java/lang/ref/ReferenceEnqueue.java fails with
	"RuntimeException: Error: poll() returned null; expected ref object"
In-Reply-To: <55BA0C85.7090802@gmail.com>
References: <BA1C0BEB-F342-437A-87E7-029660BB5348@oracle.com>
	<55B88FAB.4080302@oracle.com>
	<929281CB-55E8-4A0C-A8D1-85ED37348E12@oracle.com>
	<55B9E293.80102@gmail.com> <55B9EA8A.1080102@oracle.com>
	<55B9FA54.5040902@gmail.com> <55BA0117.6040203@oracle.com>
	<55BA0C85.7090802@gmail.com>
Message-ID: <55BA1ABA.9080104@oracle.com>

On 30/07/15 13:37, Peter Levart wrote:
> poll() returning null by itself is not surprising, but if 'r' appears to
> be "enqueued" before and after the fact, this is surprising.

Agreed - not disputing this.

>> The question for me is whether this should be fixed in the same
>> changeset - or whether we should make it in another changeset...
>
> It's a different issue, though very similar.

Agreed on both counts :-)

-- daniel


From daniel.fuchs at oracle.com  Thu Jul 30 20:12:12 2015
From: daniel.fuchs at oracle.com (Daniel Fuchs)
Date: Thu, 30 Jul 2015 22:12:12 +0200
Subject: RFR: 8132306: java/lang/ref/ReferenceEnqueue.java fails with
	"RuntimeException: Error: poll() returned null; expected ref object"
In-Reply-To: <CE7CA993-BB20-44E6-B378-419560CFFCCC@oracle.com>
References: <BA1C0BEB-F342-437A-87E7-029660BB5348@oracle.com>
	<55B88FAB.4080302@oracle.com>
	<929281CB-55E8-4A0C-A8D1-85ED37348E12@oracle.com>
	<55B9E293.80102@gmail.com>
	<55B9EA8A.1080102@oracle.com> <55B9FA54.5040902@gmail.com>
	<55B9FB50.8050907@oracle.com> <55B9FF5F.2020706@gmail.com>
	<55BA0E2B.9050101@oracle.com> <55BA110F.2010101@gmail.com>
	<55BA13E8.1050606@oracle.com>
	<CE7CA993-BB20-44E6-B378-419560CFFCCC@oracle.com>
Message-ID: <55BA851C.5080706@oracle.com>

I vote for 1.

:-)

cheers,

-- daniel

On 7/30/15 9:54 PM, Kim Barrett wrote:
> On Jul 30, 2015, at 8:09 AM, David Holmes <david.holmes at oracle.com> wrote:
>> On 30/07/2015 9:57 PM, Peter Levart wrote:
>>> 'r' has been enqueued.
>>>
>>> Thread-1:
>>>
>>> r.isEnqueued() &&
>>> q.poll() == null &&
>>> r.isEnqueued()
>>>
>>> Thread-2:
>>>
>>> q.poll();
>>>
>>>
>>> Sequence of actions:
>>>
>>> T1: r.isEnqueued() ==> true
>>>
>>> T2: q.poll() executed to the following point (see HERE) and 'r' was the
>>> last element in the queue ('head' has been assigned to null):
>> Yeah thanks - just realized it is that darned unsynchronized "fast-path" again. What a mess.
>>
>> It a kind of inverse of the original problem.
>>
>> Original: don't update reference state to enqueued before the queue is updated
>> This one: don't update the queue state to empty before the reference state shows it is de-queued.
>>
>> So yes the fix here is to move "r.queue = null" to before the assignment to head.
>>
>> Bring on the next race ;-)
> I agree with everything David said above.  Bleh!
>
> So I think I can either:
>
> 1. Go ahead with my change + Peter's change.
>
> 2. Give this back to core-libs while I step carefully away :-)
>
> I *think* option (1) is at least an improvement.  But I completely
> missed Peter's race, despite having specifically looked for problems
> there, so take my opinion with an appropriate quantity of salt.
>
>



From daniel.fuchs at oracle.com  Fri Jul 31 09:10:32 2015
From: daniel.fuchs at oracle.com (Daniel Fuchs)
Date: Fri, 31 Jul 2015 11:10:32 +0200
Subject: RFR: 8132306: java/lang/ref/ReferenceEnqueue.java fails with
	"RuntimeException: Error: poll() returned null; expected ref object"
In-Reply-To: <C25B3711-5931-4F4D-8B8C-4F83FAF83A51@oracle.com>
References: <BA1C0BEB-F342-437A-87E7-029660BB5348@oracle.com>
	<55B88FAB.4080302@oracle.com>
	<929281CB-55E8-4A0C-A8D1-85ED37348E12@oracle.com>
	<55B9E293.80102@gmail.com> <55B9EA8A.1080102@oracle.com>
	<55B9FA54.5040902@gmail.com> <55B9FB50.8050907@oracle.com>
	<55B9FF5F.2020706@gmail.com> <55BA0E2B.9050101@oracle.com>
	<55BA110F.2010101@gmail.com> <55BA13E8.1050606@oracle.com>
	<CE7CA993-BB20-44E6-B378-419560CFFCCC@oracle.com>
	<55BA9830.8030308@oracle.com>
	<C25B3711-5931-4F4D-8B8C-4F83FAF83A51@oracle.com>
Message-ID: <55BB3B88.8070902@oracle.com>

On 31/07/15 00:56, Kim Barrett wrote:
> On Jul 30, 2015, at 5:33 PM, David Holmes <david.holmes at oracle.com> wrote:
>>
>>> So I think I can either:
>>>
>>> 1. Go ahead with my change + Peter's change.
>>>
>>> 2. Give this back to core-libs while I step carefully away :-)
>>>
>>> I *think* option (1) is at least an improvement.  But I completely
>>> missed Peter's race, despite having specifically looked for problems
>>> there, so take my opinion with an appropriate quantity of salt.
>>
>> I vote for 1 as well. I think we have now given good coverage to the two sources of problems (the two lock-free regions):
>> - reference queue state can be seen while queue state is in transition
>> - queue empty state can be seen while reference state is in transition
>
> New webrev, with both changes:
>
> http://cr.openjdk.java.net/~kbarrett/8132306/webrev.01/
>

+ 1

Thanks for taking care of this one!

PS: you might want to add @bug 8132306 to
     jdk/test/java/lang/ref/ReferenceEnqueue.java

best regards,

-- daniel