From ysr1729 at gmail.com  Sat Aug  1 00:39:03 2015
From: ysr1729 at gmail.com (ysr1729 at gmail.com)
Date: Fri, 31 Jul 2015 17:39:03 -0700
Subject: Cost of single-threaded nmethod hotness updates at each safepoint
	(in JDK 8)
In-Reply-To: <CAHjP37HRYF-kB-qMGeK3tAL4A9cvzHu_P0rCYPwbnAtn6dViTQ@mail.gmail.com>
References: <CABzyjy=GHQ7aDyZyJg_4QgpqWuFkEPmwErq2pkLmFr7CRL8X6Q@mail.gmail.com>
	<CAHjP37FHJq_wEy3jak7qPnR38C2xTXHi+BgA+xk56aV+M=dpsQ@mail.gmail.com>
	<CABzyjyn1GaD7VKj-Lskn4UTZcJdckt_iqaSrOgWMxWh-7ehEOQ@mail.gmail.com>
	<55BBC1C0.3030709@oracle.com>
	<CABzyjy=UmTx9+HCTAxZ_R27fWiOcBqAxk_xLRioBJN=ZSJmGyQ@mail.gmail.com>
	<55BBE883.1080308@oracle.com>
	<CABzyjykMwmdWLqa9=J6uiF6Lji51r_zHX_TDFxWKdTgrdQP4tA@mail.gmail.com>
	<CAHjP37HRYF-kB-qMGeK3tAL4A9cvzHu_P0rCYPwbnAtn6dViTQ@mail.gmail.com>
Message-ID: <9B8024CB-7044-4E97-96B4-C44147C1FE1B@gmail.com>

Hi Vitaly -- Which jdk 8 version were you testing? It's a bit of the proverbial curate's egg at the moment (albeit not in the original sense, i assure you!)  but if i may be allowed to mix my metaphors, I would be inclined not to throw out the baby with the bath water, yet. There are services that have seen benefits and some that haven't, and the picture overall is still a bit fuzzy. May be someone out there has done a more disciplined epidemiological study...

PS: a couple of services were running tiered when it wasn't the default (in jdk 7)...

-- ramki

Sent from my iPhone

> On Jul 31, 2015, at 3:08 PM, Vitaly Davidovich <vitalyd at gmail.com> wrote:
> 
> Ramki, are you actually seeing better peak perf with tiered than C2? I experimented with it on a real workload and it was a net loss for peak perf (anywhere from 8-20% worse than C2, but also quite unstable); this was with a very large code cache to play it safe, but no other tuning.
> 
> sent from my phone
> 
>> On Jul 31, 2015 6:02 PM, "Srinivas Ramakrishna" <ysr1729 at gmail.com> wrote:
>> OK, will do and add you as watcher; thanks Vladimir! (don't yet know if with tiered and a necessarily bounded, if large, code cache whether flushing will in fact eventually become necessary, wrt yr suggested temporary workaround.)
>> 
>> Have a good weekend!
>> -- ramki
>> 
>>> On Fri, Jul 31, 2015 at 2:28 PM, Vladimir Kozlov <vladimir.kozlov at oracle.com> wrote:
>>> Got it. Yes, it is issue with thousands java threads.
>>> You are the first pointing this problem. File bug on compiler. We will look what we can do. Most likely we need parallelize this work.
>>> 
>>> Method's hotness is used only for UseCodeCacheFlushing. You can try to guard Threads::nmethods_do(&set_hotness_closure); with this flag and switch it off.
>>> 
>>> We need mark_as_seen_on_stack so leave it.
>>> 
>>> Thanks,
>>> Vladimir
>>> 
>>> 
>>>> On 7/31/15 11:48 AM, Srinivas Ramakrishna wrote:
>>>> Hi Vladimir --
>>>> 
>>>> I noticed the increase even with Initial and Reserved set to the default
>>>> of 240 MB, but actual usage much lower (less than a quarter).
>>>> 
>>>> Look at this code path. Note that this is invoked at every safepoint
>>>> (although it says "periodically" in the comment).
>>>> In the mark_active_nmethods() method, there's a thread iteration in both
>>>> branches of the if. I haven't checked to
>>>> see which of the two was the culprit here, yet (if either).
>>>> 
>>>> // Various cleaning tasks that should be done periodically at safepoints
>>>> 
>>>> void SafepointSynchronize::do_cleanup_tasks() {
>>>> 
>>>> ....
>>>> 
>>>>    {
>>>> 
>>>>      TraceTime t4("mark nmethods", TraceSafepointCleanupTime);
>>>> 
>>>>      NMethodSweeper::mark_active_nmethods();
>>>> 
>>>>    }
>>>> 
>>>> ..
>>>> 
>>>> }
>>>> 
>>>> 
>>>> void NMethodSweeper::mark_active_nmethods() {
>>>> 
>>>>   ...
>>>> 
>>>>    if (!sweep_in_progress()) {
>>>> 
>>>>      _seen = 0;
>>>> 
>>>>      _sweep_fractions_left = NmethodSweepFraction;
>>>> 
>>>>      _current = CodeCache::first_nmethod();
>>>> 
>>>>      _traversals += 1;
>>>> 
>>>>      _total_time_this_sweep = Tickspan();
>>>> 
>>>> 
>>>>      if (PrintMethodFlushing) {
>>>> 
>>>>        tty->print_cr("### Sweep: stack traversal %d", _traversals);
>>>> 
>>>>      }
>>>> 
>>>>      Threads::nmethods_do(&mark_activation_closure);
>>>> 
>>>> 
>>>>    } else {
>>>> 
>>>>      // Only set hotness counter
>>>> 
>>>>      Threads::nmethods_do(&set_hotness_closure);
>>>> 
>>>>    }
>>>> 
>>>> 
>>>>    OrderAccess::storestore();
>>>> 
>>>> }
>>>> 
>>>> 
>>>> On Fri, Jul 31, 2015 at 11:43 AM, Vladimir Kozlov
>>>> <vladimir.kozlov at oracle.com <mailto:vladimir.kozlov at oracle.com>> wrote:
>>>> 
>>>>     Hi Ramki,
>>>> 
>>>>     Did you fill up CodeCache? It start scanning aggressive only with
>>>>     full CodeCache:
>>>> 
>>>>        // Force stack scanning if there is only 10% free space in the
>>>>     code cache.
>>>>        // We force stack scanning only non-profiled code heap gets full,
>>>>     since critical
>>>>        // allocation go to the non-profiled heap and we must be make
>>>>     sure that there is
>>>>        // enough space.
>>>>        double free_percent = 1 /
>>>>     CodeCache::reverse_free_ratio(CodeBlobType::MethodNonProfiled) * 100;
>>>>        if (free_percent <= StartAggressiveSweepingAt) {
>>>>          do_stack_scanning();
>>>>        }
>>>> 
>>>>     Vladimir
>>>> 
>>>>     On 7/31/15 11:33 AM, Srinivas Ramakrishna wrote:
>>>> 
>>>> 
>>>>         Yes.
>>>> 
>>>> 
>>>>         On Fri, Jul 31, 2015 at 11:31 AM, Vitaly Davidovich
>>>>         <vitalyd at gmail.com <mailto:vitalyd at gmail.com>
>>>>         <mailto:vitalyd at gmail.com <mailto:vitalyd at gmail.com>>> wrote:
>>>> 
>>>>              Ramki, are you running tiered compilation?
>>>> 
>>>>              sent from my phone
>>>> 
>>>>              On Jul 31, 2015 2:19 PM, "Srinivas Ramakrishna"
>>>>         <ysr1729 at gmail.com <mailto:ysr1729 at gmail.com>
>>>>              <mailto:ysr1729 at gmail.com <mailto:ysr1729 at gmail.com>>> wrote:
>>>> 
>>>> 
>>>>                  Hello GC and Compiler teams!
>>>> 
>>>>                  One of our services that runs with several thousand threads
>>>>                  recently noticed an increase
>>>>                  in safepoint stop times, but not gc times, upon
>>>>         transitioning to
>>>>                  JDK 8.
>>>> 
>>>>                  Further investigation revealed that most of the delta was
>>>>                  related to the so-called
>>>>                  pre-gc/vmop "cleanup" phase when various book-keeping
>>>>         activities
>>>>                  are performed,
>>>>                  and more specifically in the portion that walks java thread
>>>>                  stacks single-threaded (!)
>>>>                  and updates the hotness counters for the active
>>>>         nmethods. This
>>>>                  code appears to
>>>>                  be new to JDK 8 (in jdk 7 one would walk the stacks
>>>>         only during
>>>>                  code cache sweeps).
>>>> 
>>>>                  I have two questions:
>>>>                  (1) has anyone else (typically, I'd expect applications
>>>>         with
>>>>                  many hundreds or thousands of threads)
>>>>                  noticed this regression?
>>>>                  (2) Can we do better, for example, by:
>>>>                         (a) doing these updates by walking thread stacks in
>>>>                  multiple worker threads in parallel, or best of all:
>>>>                         (b) doing these updates when we walk the thread
>>>>         stacks
>>>>                  during GC, and skipping this phase entirely
>>>>                               for non-GC safepoints (with attendant loss in
>>>>                  frequency of this update in low GC frequency
>>>>                               scenarios).
>>>> 
>>>>                  It seems kind of silly to do GC's with many multiple worker
>>>>                  threads, but do these thread stack
>>>>                  walks single-threaded when it is embarrasingly parallel
>>>>         (one
>>>>                  could predicate the parallelization
>>>>                  based on the measured stack sizes and thread population, if
>>>>                  there was concern on the ovrhead of
>>>>                  activating and deactivating the thread gangs for the work).
>>>> 
>>>>                  A followup question: Any guesses as to how code cache
>>>>                  sweep/eviction quality might be compromised if one
>>>>                  were to dispense with these hotness updates entirely
>>>>         (or at a
>>>>                  much reduced frequency), as a temporary
>>>>                  workaround to the performance problem?
>>>> 
>>>>                  Thoughts/Comments? In particular, has this issue been
>>>>         addressed
>>>>                  perhaps in newer JVMs?
>>>> 
>>>>                  Thanks for any comments, feedback, pointers!
>>>>                  -- ramki
>>>> 
>>>>                  PS: for comparison, here's data with
>>>>         +TraceSafepointCleanup from
>>>>                  JDK 7 (first, where this isn't done)
>>>>                  vs JDK 8 (where this is done) with a program that has a few
>>>>                  thousands of threads:
>>>> 
>>>> 
>>>> 
>>>>                  JDK 7:
>>>>                  ..
>>>>                  2827.308: [sweeping nmethods, 0.0000020 secs]
>>>>                  2828.679: [sweeping nmethods, 0.0000030 secs]
>>>>                  2829.984: [sweeping nmethods, 0.0000030 secs]
>>>>                  2830.956: [sweeping nmethods, 0.0000030 secs]
>>>>                  ..
>>>> 
>>>>                  JDK 8:
>>>>                  ..
>>>>                  7368.634: [mark nmethods, 0.0177030 secs]
>>>>                  7369.587: [mark nmethods, 0.0178305 secs]
>>>>                  7370.479: [mark nmethods, 0.0180260 secs]
>>>>                  7371.503: [mark nmethods, 0.0186494 secs]
>>>>                  ..
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150731/d1b06f18/attachment.htm>

From vitalyd at gmail.com  Sat Aug  1 02:17:29 2015
From: vitalyd at gmail.com (Vitaly Davidovich)
Date: Fri, 31 Jul 2015 22:17:29 -0400
Subject: Cost of single-threaded nmethod hotness updates at each safepoint
	(in JDK 8)
In-Reply-To: <9B8024CB-7044-4E97-96B4-C44147C1FE1B@gmail.com>
References: <CABzyjy=GHQ7aDyZyJg_4QgpqWuFkEPmwErq2pkLmFr7CRL8X6Q@mail.gmail.com>
	<CAHjP37FHJq_wEy3jak7qPnR38C2xTXHi+BgA+xk56aV+M=dpsQ@mail.gmail.com>
	<CABzyjyn1GaD7VKj-Lskn4UTZcJdckt_iqaSrOgWMxWh-7ehEOQ@mail.gmail.com>
	<55BBC1C0.3030709@oracle.com>
	<CABzyjy=UmTx9+HCTAxZ_R27fWiOcBqAxk_xLRioBJN=ZSJmGyQ@mail.gmail.com>
	<55BBE883.1080308@oracle.com>
	<CABzyjykMwmdWLqa9=J6uiF6Lji51r_zHX_TDFxWKdTgrdQP4tA@mail.gmail.com>
	<CAHjP37HRYF-kB-qMGeK3tAL4A9cvzHu_P0rCYPwbnAtn6dViTQ@mail.gmail.com>
	<9B8024CB-7044-4E97-96B4-C44147C1FE1B@gmail.com>
Message-ID: <CAHjP37GZHyme6ZtAFPJ1U2yADHhLRhSGuVqUmZ-iovZ2ELWJRg@mail.gmail.com>

Hi Ramki,

That experiment was performed on 7u60, not 8; I may revisit this with 8 or
perhaps wait for segregated code cache to be available before trying
again.  One thing that worried me was the tuning aspect of tiered, which is
a bit opaque as compared to, say, GC logs - it's a bit too black boxey for
me.  Also, the servers I was running this on have tightly chosen cpu
affinity masks and there aren't many spare cores to dedicate to C1 and C2
compiler threads.  But, I may look at this again in the near future.

sent from my phone
On Jul 31, 2015 8:39 PM, <ysr1729 at gmail.com> wrote:

> Hi Vitaly -- Which jdk 8 version were you testing? It's a bit of the
> proverbial curate's egg at the moment (albeit not in the original sense, i
> assure you!)  but if i may be allowed to mix my metaphors, I would be
> inclined not to throw out the baby with the bath water, yet. There are
> services that have seen benefits and some that haven't, and the picture
> overall is still a bit fuzzy. May be someone out there has done a more
> disciplined epidemiological study...
>
> PS: a couple of services were running tiered when it wasn't the default
> (in jdk 7)...
>
> -- ramki
>
> Sent from my iPhone
>
> On Jul 31, 2015, at 3:08 PM, Vitaly Davidovich <vitalyd at gmail.com> wrote:
>
> Ramki, are you actually seeing better peak perf with tiered than C2? I
> experimented with it on a real workload and it was a net loss for peak perf
> (anywhere from 8-20% worse than C2, but also quite unstable); this was with
> a very large code cache to play it safe, but no other tuning.
>
> sent from my phone
> On Jul 31, 2015 6:02 PM, "Srinivas Ramakrishna" <ysr1729 at gmail.com> wrote:
>
>> OK, will do and add you as watcher; thanks Vladimir! (don't yet know if
>> with tiered and a necessarily bounded, if large, code cache whether
>> flushing will in fact eventually become necessary, wrt yr suggested
>> temporary workaround.)
>>
>> Have a good weekend!
>> -- ramki
>>
>> On Fri, Jul 31, 2015 at 2:28 PM, Vladimir Kozlov <
>> vladimir.kozlov at oracle.com> wrote:
>>
>>> Got it. Yes, it is issue with thousands java threads.
>>> You are the first pointing this problem. File bug on compiler. We will
>>> look what we can do. Most likely we need parallelize this work.
>>>
>>> Method's hotness is used only for UseCodeCacheFlushing. You can try to
>>> guard Threads::nmethods_do(&set_hotness_closure); with this flag and switch
>>> it off.
>>>
>>> We need mark_as_seen_on_stack so leave it.
>>>
>>> Thanks,
>>> Vladimir
>>>
>>>
>>> On 7/31/15 11:48 AM, Srinivas Ramakrishna wrote:
>>>
>>>> Hi Vladimir --
>>>>
>>>> I noticed the increase even with Initial and Reserved set to the default
>>>> of 240 MB, but actual usage much lower (less than a quarter).
>>>>
>>>> Look at this code path. Note that this is invoked at every safepoint
>>>> (although it says "periodically" in the comment).
>>>> In the mark_active_nmethods() method, there's a thread iteration in both
>>>> branches of the if. I haven't checked to
>>>> see which of the two was the culprit here, yet (if either).
>>>>
>>>> // Various cleaning tasks that should be done periodically at safepoints
>>>>
>>>> void SafepointSynchronize::do_cleanup_tasks() {
>>>>
>>>> ....
>>>>
>>>>    {
>>>>
>>>>      TraceTime t4("mark nmethods", TraceSafepointCleanupTime);
>>>>
>>>>      NMethodSweeper::mark_active_nmethods();
>>>>
>>>>    }
>>>>
>>>> ..
>>>>
>>>> }
>>>>
>>>>
>>>> void NMethodSweeper::mark_active_nmethods() {
>>>>
>>>>   ...
>>>>
>>>>    if (!sweep_in_progress()) {
>>>>
>>>>      _seen = 0;
>>>>
>>>>      _sweep_fractions_left = NmethodSweepFraction;
>>>>
>>>>      _current = CodeCache::first_nmethod();
>>>>
>>>>      _traversals += 1;
>>>>
>>>>      _total_time_this_sweep = Tickspan();
>>>>
>>>>
>>>>      if (PrintMethodFlushing) {
>>>>
>>>>        tty->print_cr("### Sweep: stack traversal %d", _traversals);
>>>>
>>>>      }
>>>>
>>>>      Threads::nmethods_do(&mark_activation_closure);
>>>>
>>>>
>>>>    } else {
>>>>
>>>>      // Only set hotness counter
>>>>
>>>>      Threads::nmethods_do(&set_hotness_closure);
>>>>
>>>>    }
>>>>
>>>>
>>>>    OrderAccess::storestore();
>>>>
>>>> }
>>>>
>>>>
>>>> On Fri, Jul 31, 2015 at 11:43 AM, Vladimir Kozlov
>>>> <vladimir.kozlov at oracle.com <mailto:vladimir.kozlov at oracle.com>> wrote:
>>>>
>>>>     Hi Ramki,
>>>>
>>>>     Did you fill up CodeCache? It start scanning aggressive only with
>>>>     full CodeCache:
>>>>
>>>>        // Force stack scanning if there is only 10% free space in the
>>>>     code cache.
>>>>        // We force stack scanning only non-profiled code heap gets full,
>>>>     since critical
>>>>        // allocation go to the non-profiled heap and we must be make
>>>>     sure that there is
>>>>        // enough space.
>>>>        double free_percent = 1 /
>>>>     CodeCache::reverse_free_ratio(CodeBlobType::MethodNonProfiled) *
>>>> 100;
>>>>        if (free_percent <= StartAggressiveSweepingAt) {
>>>>          do_stack_scanning();
>>>>        }
>>>>
>>>>     Vladimir
>>>>
>>>>     On 7/31/15 11:33 AM, Srinivas Ramakrishna wrote:
>>>>
>>>>
>>>>         Yes.
>>>>
>>>>
>>>>         On Fri, Jul 31, 2015 at 11:31 AM, Vitaly Davidovich
>>>>         <vitalyd at gmail.com <mailto:vitalyd at gmail.com>
>>>>         <mailto:vitalyd at gmail.com <mailto:vitalyd at gmail.com>>> wrote:
>>>>
>>>>              Ramki, are you running tiered compilation?
>>>>
>>>>              sent from my phone
>>>>
>>>>              On Jul 31, 2015 2:19 PM, "Srinivas Ramakrishna"
>>>>         <ysr1729 at gmail.com <mailto:ysr1729 at gmail.com>
>>>>              <mailto:ysr1729 at gmail.com <mailto:ysr1729 at gmail.com>>>
>>>> wrote:
>>>>
>>>>
>>>>                  Hello GC and Compiler teams!
>>>>
>>>>                  One of our services that runs with several thousand
>>>> threads
>>>>                  recently noticed an increase
>>>>                  in safepoint stop times, but not gc times, upon
>>>>         transitioning to
>>>>                  JDK 8.
>>>>
>>>>                  Further investigation revealed that most of the delta
>>>> was
>>>>                  related to the so-called
>>>>                  pre-gc/vmop "cleanup" phase when various book-keeping
>>>>         activities
>>>>                  are performed,
>>>>                  and more specifically in the portion that walks java
>>>> thread
>>>>                  stacks single-threaded (!)
>>>>                  and updates the hotness counters for the active
>>>>         nmethods. This
>>>>                  code appears to
>>>>                  be new to JDK 8 (in jdk 7 one would walk the stacks
>>>>         only during
>>>>                  code cache sweeps).
>>>>
>>>>                  I have two questions:
>>>>                  (1) has anyone else (typically, I'd expect applications
>>>>         with
>>>>                  many hundreds or thousands of threads)
>>>>                  noticed this regression?
>>>>                  (2) Can we do better, for example, by:
>>>>                         (a) doing these updates by walking thread
>>>> stacks in
>>>>                  multiple worker threads in parallel, or best of all:
>>>>                         (b) doing these updates when we walk the thread
>>>>         stacks
>>>>                  during GC, and skipping this phase entirely
>>>>                               for non-GC safepoints (with attendant
>>>> loss in
>>>>                  frequency of this update in low GC frequency
>>>>                               scenarios).
>>>>
>>>>                  It seems kind of silly to do GC's with many multiple
>>>> worker
>>>>                  threads, but do these thread stack
>>>>                  walks single-threaded when it is embarrasingly parallel
>>>>         (one
>>>>                  could predicate the parallelization
>>>>                  based on the measured stack sizes and thread
>>>> population, if
>>>>                  there was concern on the ovrhead of
>>>>                  activating and deactivating the thread gangs for the
>>>> work).
>>>>
>>>>                  A followup question: Any guesses as to how code cache
>>>>                  sweep/eviction quality might be compromised if one
>>>>                  were to dispense with these hotness updates entirely
>>>>         (or at a
>>>>                  much reduced frequency), as a temporary
>>>>                  workaround to the performance problem?
>>>>
>>>>                  Thoughts/Comments? In particular, has this issue been
>>>>         addressed
>>>>                  perhaps in newer JVMs?
>>>>
>>>>                  Thanks for any comments, feedback, pointers!
>>>>                  -- ramki
>>>>
>>>>                  PS: for comparison, here's data with
>>>>         +TraceSafepointCleanup from
>>>>                  JDK 7 (first, where this isn't done)
>>>>                  vs JDK 8 (where this is done) with a program that has
>>>> a few
>>>>                  thousands of threads:
>>>>
>>>>
>>>>
>>>>                  JDK 7:
>>>>                  ..
>>>>                  2827.308: [sweeping nmethods, 0.0000020 secs]
>>>>                  2828.679: [sweeping nmethods, 0.0000030 secs]
>>>>                  2829.984: [sweeping nmethods, 0.0000030 secs]
>>>>                  2830.956: [sweeping nmethods, 0.0000030 secs]
>>>>                  ..
>>>>
>>>>                  JDK 8:
>>>>                  ..
>>>>                  7368.634: [mark nmethods, 0.0177030 secs]
>>>>                  7369.587: [mark nmethods, 0.0178305 secs]
>>>>                  7370.479: [mark nmethods, 0.0180260 secs]
>>>>                  7371.503: [mark nmethods, 0.0186494 secs]
>>>>                  ..
>>>>
>>>>
>>>>
>>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150731/0a3666cf/attachment.htm>

From ysr1729 at gmail.com  Sun Aug  2 18:11:31 2015
From: ysr1729 at gmail.com (Srinivas Ramakrishna)
Date: Sun, 2 Aug 2015 11:11:31 -0700
Subject: Cost of single-threaded nmethod hotness updates at each safepoint
	(in JDK 8)
In-Reply-To: <55BBE883.1080308@oracle.com>
References: <CABzyjy=GHQ7aDyZyJg_4QgpqWuFkEPmwErq2pkLmFr7CRL8X6Q@mail.gmail.com>
	<CAHjP37FHJq_wEy3jak7qPnR38C2xTXHi+BgA+xk56aV+M=dpsQ@mail.gmail.com>
	<CABzyjyn1GaD7VKj-Lskn4UTZcJdckt_iqaSrOgWMxWh-7ehEOQ@mail.gmail.com>
	<55BBC1C0.3030709@oracle.com>
	<CABzyjy=UmTx9+HCTAxZ_R27fWiOcBqAxk_xLRioBJN=ZSJmGyQ@mail.gmail.com>
	<55BBE883.1080308@oracle.com>
Message-ID: <CABzyjynnOFuu_e0CTafW3VZR64e3mZeSVxouKxdmAwdf0FUVYw@mail.gmail.com>

I filed:  https://bugs.openjdk.java.net/browse/JDK-8132849

thanks!
-- ramki

On Fri, Jul 31, 2015 at 2:28 PM, Vladimir Kozlov <vladimir.kozlov at oracle.com
> wrote:

> Got it. Yes, it is issue with thousands java threads.
> You are the first pointing this problem. File bug on compiler. We will
> look what we can do. Most likely we need parallelize this work.
>
> Method's hotness is used only for UseCodeCacheFlushing. You can try to
> guard Threads::nmethods_do(&set_hotness_closure); with this flag and switch
> it off.
>
> We need mark_as_seen_on_stack so leave it.
>
> Thanks,
> Vladimir
>
>
> On 7/31/15 11:48 AM, Srinivas Ramakrishna wrote:
>
>> Hi Vladimir --
>>
>> I noticed the increase even with Initial and Reserved set to the default
>> of 240 MB, but actual usage much lower (less than a quarter).
>>
>> Look at this code path. Note that this is invoked at every safepoint
>> (although it says "periodically" in the comment).
>> In the mark_active_nmethods() method, there's a thread iteration in both
>> branches of the if. I haven't checked to
>> see which of the two was the culprit here, yet (if either).
>>
>> // Various cleaning tasks that should be done periodically at safepoints
>>
>> void SafepointSynchronize::do_cleanup_tasks() {
>>
>> ....
>>
>>    {
>>
>>      TraceTime t4("mark nmethods", TraceSafepointCleanupTime);
>>
>>      NMethodSweeper::mark_active_nmethods();
>>
>>    }
>>
>> ..
>>
>> }
>>
>>
>> void NMethodSweeper::mark_active_nmethods() {
>>
>>   ...
>>
>>    if (!sweep_in_progress()) {
>>
>>      _seen = 0;
>>
>>      _sweep_fractions_left = NmethodSweepFraction;
>>
>>      _current = CodeCache::first_nmethod();
>>
>>      _traversals += 1;
>>
>>      _total_time_this_sweep = Tickspan();
>>
>>
>>      if (PrintMethodFlushing) {
>>
>>        tty->print_cr("### Sweep: stack traversal %d", _traversals);
>>
>>      }
>>
>>      Threads::nmethods_do(&mark_activation_closure);
>>
>>
>>    } else {
>>
>>      // Only set hotness counter
>>
>>      Threads::nmethods_do(&set_hotness_closure);
>>
>>    }
>>
>>
>>    OrderAccess::storestore();
>>
>> }
>>
>>
>> On Fri, Jul 31, 2015 at 11:43 AM, Vladimir Kozlov
>> <vladimir.kozlov at oracle.com <mailto:vladimir.kozlov at oracle.com>> wrote:
>>
>>     Hi Ramki,
>>
>>     Did you fill up CodeCache? It start scanning aggressive only with
>>     full CodeCache:
>>
>>        // Force stack scanning if there is only 10% free space in the
>>     code cache.
>>        // We force stack scanning only non-profiled code heap gets full,
>>     since critical
>>        // allocation go to the non-profiled heap and we must be make
>>     sure that there is
>>        // enough space.
>>        double free_percent = 1 /
>>     CodeCache::reverse_free_ratio(CodeBlobType::MethodNonProfiled) * 100;
>>        if (free_percent <= StartAggressiveSweepingAt) {
>>          do_stack_scanning();
>>        }
>>
>>     Vladimir
>>
>>     On 7/31/15 11:33 AM, Srinivas Ramakrishna wrote:
>>
>>
>>         Yes.
>>
>>
>>         On Fri, Jul 31, 2015 at 11:31 AM, Vitaly Davidovich
>>         <vitalyd at gmail.com <mailto:vitalyd at gmail.com>
>>         <mailto:vitalyd at gmail.com <mailto:vitalyd at gmail.com>>> wrote:
>>
>>              Ramki, are you running tiered compilation?
>>
>>              sent from my phone
>>
>>              On Jul 31, 2015 2:19 PM, "Srinivas Ramakrishna"
>>         <ysr1729 at gmail.com <mailto:ysr1729 at gmail.com>
>>              <mailto:ysr1729 at gmail.com <mailto:ysr1729 at gmail.com>>>
>> wrote:
>>
>>
>>                  Hello GC and Compiler teams!
>>
>>                  One of our services that runs with several thousand
>> threads
>>                  recently noticed an increase
>>                  in safepoint stop times, but not gc times, upon
>>         transitioning to
>>                  JDK 8.
>>
>>                  Further investigation revealed that most of the delta was
>>                  related to the so-called
>>                  pre-gc/vmop "cleanup" phase when various book-keeping
>>         activities
>>                  are performed,
>>                  and more specifically in the portion that walks java
>> thread
>>                  stacks single-threaded (!)
>>                  and updates the hotness counters for the active
>>         nmethods. This
>>                  code appears to
>>                  be new to JDK 8 (in jdk 7 one would walk the stacks
>>         only during
>>                  code cache sweeps).
>>
>>                  I have two questions:
>>                  (1) has anyone else (typically, I'd expect applications
>>         with
>>                  many hundreds or thousands of threads)
>>                  noticed this regression?
>>                  (2) Can we do better, for example, by:
>>                         (a) doing these updates by walking thread stacks
>> in
>>                  multiple worker threads in parallel, or best of all:
>>                         (b) doing these updates when we walk the thread
>>         stacks
>>                  during GC, and skipping this phase entirely
>>                               for non-GC safepoints (with attendant loss
>> in
>>                  frequency of this update in low GC frequency
>>                               scenarios).
>>
>>                  It seems kind of silly to do GC's with many multiple
>> worker
>>                  threads, but do these thread stack
>>                  walks single-threaded when it is embarrasingly parallel
>>         (one
>>                  could predicate the parallelization
>>                  based on the measured stack sizes and thread population,
>> if
>>                  there was concern on the ovrhead of
>>                  activating and deactivating the thread gangs for the
>> work).
>>
>>                  A followup question: Any guesses as to how code cache
>>                  sweep/eviction quality might be compromised if one
>>                  were to dispense with these hotness updates entirely
>>         (or at a
>>                  much reduced frequency), as a temporary
>>                  workaround to the performance problem?
>>
>>                  Thoughts/Comments? In particular, has this issue been
>>         addressed
>>                  perhaps in newer JVMs?
>>
>>                  Thanks for any comments, feedback, pointers!
>>                  -- ramki
>>
>>                  PS: for comparison, here's data with
>>         +TraceSafepointCleanup from
>>                  JDK 7 (first, where this isn't done)
>>                  vs JDK 8 (where this is done) with a program that has a
>> few
>>                  thousands of threads:
>>
>>
>>
>>                  JDK 7:
>>                  ..
>>                  2827.308: [sweeping nmethods, 0.0000020 secs]
>>                  2828.679: [sweeping nmethods, 0.0000030 secs]
>>                  2829.984: [sweeping nmethods, 0.0000030 secs]
>>                  2830.956: [sweeping nmethods, 0.0000030 secs]
>>                  ..
>>
>>                  JDK 8:
>>                  ..
>>                  7368.634: [mark nmethods, 0.0177030 secs]
>>                  7369.587: [mark nmethods, 0.0178305 secs]
>>                  7370.479: [mark nmethods, 0.0180260 secs]
>>                  7371.503: [mark nmethods, 0.0186494 secs]
>>                  ..
>>
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150802/419d5450/attachment.htm>

From thomas.schatzl at oracle.com  Mon Aug  3 12:35:20 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 03 Aug 2015 14:35:20 +0200
Subject: RFR (S): 8003237: G1: Reduce unnecessary (and failing) allocation
	attempts when handling an evacuation failure
Message-ID: <1438605320.2292.47.camel@oracle.com>

Hi all,

  can I have reviews for this small change that improves evacuation
failure handling by short-cutting copy_to_survivor_space() when there is
no more space left?

In effect this makes the parallel phase during GC have the same
performance as if there is no evacuation failure, and I could not
measure regressions in the normal case either.

The change implements two things: first, notice when there is no more
space left in the last gen (i.e. old), and if so, short-cut
ParScanThreadState::copy_to_survivor_space(). The other change is to not
try to ask the free list for new regions if it's empty to avoid taking
the lock in the time window between the first few evac failures and when
space is completely empty.

There is still the somewhat heavyweight cleanup phases after evacuation
failure.

CR:
https://bugs.openjdk.java.net/browse/JDK-8003237
Webrev:
http://cr.openjdk.java.net/~tschatzl/8003237/webrev/
Testing:
jprt, lots of performance measurements with and without this change
compiled in

Thanks,
  Thomas




From thomas.schatzl at oracle.com  Mon Aug  3 12:43:47 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 03 Aug 2015 14:43:47 +0200
Subject: RFR (S): 8003237: G1: Reduce unnecessary (and failing)
	allocation attempts when handling an evacuation failure
In-Reply-To: <1438605320.2292.47.camel@oracle.com>
References: <1438605320.2292.47.camel@oracle.com>
Message-ID: <1438605827.2292.49.camel@oracle.com>

Hi,

  this change is based on the latest version of the change for 8073052
that is out for review, and is still longing for another reviewer :)

(I am okay with pushing it with only a single "R"eviewer since it is
only refactoring, but if somebody can find the time...)

Thanks,
  Thomas

On Mon, 2015-08-03 at 14:35 +0200, Thomas Schatzl wrote:
> Hi all,
> 
>   can I have reviews for this small change that improves evacuation
> failure handling by short-cutting copy_to_survivor_space() when there is
> no more space left?
> 
> In effect this makes the parallel phase during GC have the same
> performance as if there is no evacuation failure, and I could not
> measure regressions in the normal case either.
> 
> The change implements two things: first, notice when there is no more
> space left in the last gen (i.e. old), and if so, short-cut
> ParScanThreadState::copy_to_survivor_space(). The other change is to not
> try to ask the free list for new regions if it's empty to avoid taking
> the lock in the time window between the first few evac failures and when
> space is completely empty.
> 
> There is still the somewhat heavyweight cleanup phases after evacuation
> failure.
> 
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8003237
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8003237/webrev/
> Testing:
> jprt, lots of performance measurements with and without this change
> compiled in
> 
> Thanks,
>   Thomas
> 
> 




From jon.masamitsu at oracle.com  Mon Aug  3 18:53:51 2015
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Mon, 03 Aug 2015 11:53:51 -0700
Subject: RFR (XL): JDK-8086706 Parallel Full GC for CMS
In-Reply-To: <CA+n_jhjRd3_rm9v7n5qMGWptDqPcQ-ax4w4eHT91R=r1Tyj6mA@mail.gmail.com>
References: <CA+n_jhjw=T2O3pquZU++N5miGdXxwh=Kp_TBDTmk2ud79SJCgg@mail.gmail.com>	<5579B1DB.9090804@oracle.com>	<CA+3eh10-MyRSJ8gcievY_=hTXUjshGnUfwmZFGMsd9ZMD-xBuw@mail.gmail.com>	<C9B58FD5-C1E4-4143-A85B-4AB322246822@oracle.com>
	<CA+n_jhjRd3_rm9v7n5qMGWptDqPcQ-ax4w4eHT91R=r1Tyj6mA@mail.gmail.com>
Message-ID: <55BFB8BF.7000002@oracle.com>

Jungwoo,

As you can see no one has stepped up to do your review.
We?ve talked about it amongst the active GC committers.
It?s a large change and would require a very significant
commitment of anyone?s time.  As is obvious, our focus
is on G1.  And right now we?re in the position of having to
push tasks we?ve long planned for JDK 9 out into a JDK 9
update.  No one I?ve talked with is willing to push off more
of our G1 work to get this patch in.  And as maintainers
of the HotSpot GC's due diligence is required of us before
we can accept this change.

Jon

PS.  I saw that you said you had started another thread for this
discussion (or maybe the discussion of the JEP) but I could not
it.    Please feel free to repost this under the new subject.

On 06/27/2015 05:48 PM, Jungwoo Ha wrote:
> Sorry for the long wait. So many things have cleaned up on JDK9.
> Here is the webrev against jdk9/hs-rt.
> http://cr.openjdk.java.net/~jwha/8086706/webrev.jdk9.00 
> <http://cr.openjdk.java.net/%7Ejwha/8086706/webrev.jdk9.00>
>
> I am preparing for the JEP doc.
>
>
> On Thu, Jun 11, 2015 at 12:31 PM, Kim Barrett <kim.barrett at oracle.com 
> <mailto:kim.barrett at oracle.com>> wrote:
>
>     On Jun 11, 2015, at 12:33 PM, Volker Simonis
>     <volker.simonis at gmail.com <mailto:volker.simonis at gmail.com>> wrote:
>     >
>     > Hi,
>     >
>     > if nobody has already started doing it, I'll try to forward port the
>     > patch to jdk9/hs-gc/hotspot.
>
>     You?ll want to use jdk9/hs-rt rather than jdk9/hs-gc.
>
>
>
>
>
> -- 
> Jungwoo Ha | Java Platform Team | jwha at google.com <mailto:jwha at google.com>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150803/e4bacb47/attachment.htm>

From kim.barrett at oracle.com  Mon Aug  3 20:49:43 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Mon, 3 Aug 2015 16:49:43 -0400
Subject: RFR: 8132306: java/lang/ref/ReferenceEnqueue.java fails with
	"RuntimeException: Error: poll() returned null; expected ref object"
In-Reply-To: <55BB76EE.2070306@gmail.com>
References: <BA1C0BEB-F342-437A-87E7-029660BB5348@oracle.com>
	<55B88FAB.4080302@oracle.com>
	<929281CB-55E8-4A0C-A8D1-85ED37348E12@oracle.com>
	<55B9E293.80102@gmail.com> <55B9EA8A.1080102@oracle.com>
	<55B9FA54.5040902@gmail.com> <55B9FB50.8050907@oracle.com>
	<55B9FF5F.2020706@gmail.com> <55BA0E2B.9050101@oracle.com>
	<55BA110F.2010101@gmail.com> <55BA13E8.1050606@oracle.com>
	<CE7CA993-BB20-44E6-B378-419560CFFCCC@oracle.com>
	<55BA9830.8030308@oracle.com>
	<C25B3711-5931-4F4D-8B8C-4F83FAF83A51@oracle.com>
	<55BB76EE.2070306@gmail.com>
Message-ID: <92395C30-A636-437B-8D60-98A133C63768@oracle.com>

On Jul 31, 2015, at 9:23 AM, Peter Levart <peter.levart at gmail.com> wrote:
> 
> This looks good now. Thanks for taking both of the issues together. I wonder if a similar racy test could be devised for the 2nd race. I doubt the ReferenceEnqueue test was meant to catch invalid races specifically. It was more a coincidence that it failed.
> 
> I tried to create a reproducer for 2nd race, but it is not reliable and sometimes needs several minutes to fail.

I would expect it to be hard to reproduce without patching in a sleep to expand the window of vulnerability, as I did to test for the first race.




From kim.barrett at oracle.com  Mon Aug  3 20:59:58 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Mon, 3 Aug 2015 16:59:58 -0400
Subject: RFR: 8132306: java/lang/ref/ReferenceEnqueue.java fails with
	"RuntimeException: Error: poll() returned null; expected ref object"
In-Reply-To: <55BADBFE.2000808@oracle.com>
References: <BA1C0BEB-F342-437A-87E7-029660BB5348@oracle.com>
	<55B88FAB.4080302@oracle.com>
	<929281CB-55E8-4A0C-A8D1-85ED37348E12@oracle.com>
	<55B9E293.80102@gmail.com> <55B9EA8A.1080102@oracle.com>
	<55B9FA54.5040902@gmail.com> <55B9FB50.8050907@oracle.com>
	<55B9FF5F.2020706@gmail.com> <55BA0E2B.9050101@oracle.com>
	<55BA110F.2010101@gmail.com> <55BA13E8.1050606@oracle.com>
	<CE7CA993-BB20-44E6-B378-419560CFFCCC@oracle.com>
	<55BA9830.8030308@oracle.com>
	<C25B3711-5931-4F4D-8B8C-4F83FAF83A51@oracle.com>
	<55BADBFE.2000808@oracle.com>
Message-ID: <71B8E573-0BA0-4AAB-A069-AD0B4556CFF3@oracle.com>

On Jul 30, 2015, at 10:22 PM, David Holmes <david.holmes at oracle.com> wrote:
> 
> Looks good!

On Jul 31, 2015, at 5:10 AM, Daniel Fuchs <daniel.fuchs at oracle.com> wrote:
> Thanks for taking care of this one!
> 
> PS: you might want to add @bug 8132306 to
>    jdk/test/java/lang/ref/ReferenceEnqueue.java

Done.

On Jul 31, 2015, at 9:23 AM, Peter Levart <peter.levart at gmail.com> wrote:
> This looks good now. Thanks for taking both of the issues together.

Thanks for your reviews and all the discussion.

There might be a delay in pushing though; further testing produced an unexpected (and not (easily) reproducible) failure, which I?m studying to determine whether it might have anything to do with this change set.  I don?t think it does, and we?ve had some flaky infrastructure problems in the last few days, but I?m following due diligence.  I?ll either push or come back for more discussion.




From martijnverburg at gmail.com  Mon Aug  3 21:06:14 2015
From: martijnverburg at gmail.com (Martijn Verburg)
Date: Mon, 3 Aug 2015 22:06:14 +0100
Subject: RFR (XL): JDK-8086706 Parallel Full GC for CMS
In-Reply-To: <55BFB8BF.7000002@oracle.com>
References: <CA+n_jhjw=T2O3pquZU++N5miGdXxwh=Kp_TBDTmk2ud79SJCgg@mail.gmail.com>
	<5579B1DB.9090804@oracle.com>
	<CA+3eh10-MyRSJ8gcievY_=hTXUjshGnUfwmZFGMsd9ZMD-xBuw@mail.gmail.com>
	<C9B58FD5-C1E4-4143-A85B-4AB322246822@oracle.com>
	<CA+n_jhjRd3_rm9v7n5qMGWptDqPcQ-ax4w4eHT91R=r1Tyj6mA@mail.gmail.com>
	<55BFB8BF.7000002@oracle.com>
Message-ID: <CAP7YuASeh08VopSxzvR=L1m3NcLOBYr5UqNgYHyRnzL2J9UQ8g@mail.gmail.com>

We'd certainly like to see a JEP for this.  There are use cases where CMS
is the right collector and this proposed patch could (I'll stress could)
significantly improve its performance.

Cheers,
Martijn

On 3 August 2015 at 19:53, Jon Masamitsu <jon.masamitsu at oracle.com> wrote:

> Jungwoo,
>
> As you can see no one has stepped up to do your review.
> We?ve talked about it amongst the active GC committers.
> It?s a large change and would require a very significant
> commitment of anyone?s time.  As is obvious, our focus
> is on G1.  And right now we?re in the position of having to
> push tasks we?ve long planned for JDK 9 out into a JDK 9
> update.  No one I?ve talked with is willing to push off more
> of our G1 work to get this patch in.  And as maintainers
> of the HotSpot GC's due diligence is required of us before
> we can accept this change.
>
> Jon
>
> PS.  I saw that you said you had started another thread for this
> discussion (or maybe the discussion of the JEP) but I could not
> it.    Please feel free to repost this under the new subject.
>
> On 06/27/2015 05:48 PM, Jungwoo Ha wrote:
>
> Sorry for the long wait. So many things have cleaned up on JDK9.
> Here is the webrev against jdk9/hs-rt.
> http://cr.openjdk.java.net/~jwha/8086706/webrev.jdk9.00
>
> I am preparing for the JEP doc.
>
>
> On Thu, Jun 11, 2015 at 12:31 PM, Kim Barrett <kim.barrett at oracle.com>
> wrote:
>
>> On Jun 11, 2015, at 12:33 PM, Volker Simonis <volker.simonis at gmail.com>
>> wrote:
>> >
>> > Hi,
>> >
>> > if nobody has already started doing it, I'll try to forward port the
>> > patch to jdk9/hs-gc/hotspot.
>>
>> You?ll want to use jdk9/hs-rt rather than jdk9/hs-gc.
>>
>>
>>
>
>
> --
> Jungwoo Ha | Java Platform Team | jwha at google.com
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150803/a272484d/attachment.htm>

From jeremymanson at google.com  Mon Aug  3 22:20:48 2015
From: jeremymanson at google.com (Jeremy Manson)
Date: Mon, 3 Aug 2015 15:20:48 -0700
Subject: RFR (XL): JDK-8086706 Parallel Full GC for CMS
In-Reply-To: <CAP7YuASeh08VopSxzvR=L1m3NcLOBYr5UqNgYHyRnzL2J9UQ8g@mail.gmail.com>
References: <CA+n_jhjw=T2O3pquZU++N5miGdXxwh=Kp_TBDTmk2ud79SJCgg@mail.gmail.com>
	<5579B1DB.9090804@oracle.com>
	<CA+3eh10-MyRSJ8gcievY_=hTXUjshGnUfwmZFGMsd9ZMD-xBuw@mail.gmail.com>
	<C9B58FD5-C1E4-4143-A85B-4AB322246822@oracle.com>
	<CA+n_jhjRd3_rm9v7n5qMGWptDqPcQ-ax4w4eHT91R=r1Tyj6mA@mail.gmail.com>
	<55BFB8BF.7000002@oracle.com>
	<CAP7YuASeh08VopSxzvR=L1m3NcLOBYr5UqNgYHyRnzL2J9UQ8g@mail.gmail.com>
Message-ID: <CAPYFHW1pBh85HA1X6qxz3t2pP3kciRiNeBaMm5PiYENimohowg@mail.gmail.com>

What's the matter with the JEP that Jungwoo sent?

http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2015-July/014092.html

Jeremy

On Mon, Aug 3, 2015 at 2:06 PM, Martijn Verburg <martijnverburg at gmail.com>
wrote:

> We'd certainly like to see a JEP for this.  There are use cases where CMS
> is the right collector and this proposed patch could (I'll stress could)
> significantly improve its performance.
>
> Cheers,
> Martijn
>
> On 3 August 2015 at 19:53, Jon Masamitsu <jon.masamitsu at oracle.com> wrote:
>
>> Jungwoo,
>>
>> As you can see no one has stepped up to do your review.
>> We?ve talked about it amongst the active GC committers.
>> It?s a large change and would require a very significant
>> commitment of anyone?s time.  As is obvious, our focus
>> is on G1.  And right now we?re in the position of having to
>> push tasks we?ve long planned for JDK 9 out into a JDK 9
>> update.  No one I?ve talked with is willing to push off more
>> of our G1 work to get this patch in.  And as maintainers
>> of the HotSpot GC's due diligence is required of us before
>> we can accept this change.
>>
>> Jon
>>
>> PS.  I saw that you said you had started another thread for this
>> discussion (or maybe the discussion of the JEP) but I could not
>> it.    Please feel free to repost this under the new subject.
>>
>> On 06/27/2015 05:48 PM, Jungwoo Ha wrote:
>>
>> Sorry for the long wait. So many things have cleaned up on JDK9.
>> Here is the webrev against jdk9/hs-rt.
>> http://cr.openjdk.java.net/~jwha/8086706/webrev.jdk9.00
>>
>> I am preparing for the JEP doc.
>>
>>
>> On Thu, Jun 11, 2015 at 12:31 PM, Kim Barrett <kim.barrett at oracle.com>
>> wrote:
>>
>>> On Jun 11, 2015, at 12:33 PM, Volker Simonis <volker.simonis at gmail.com>
>>> wrote:
>>> >
>>> > Hi,
>>> >
>>> > if nobody has already started doing it, I'll try to forward port the
>>> > patch to jdk9/hs-gc/hotspot.
>>>
>>> You?ll want to use jdk9/hs-rt rather than jdk9/hs-gc.
>>>
>>>
>>>
>>
>>
>> --
>> Jungwoo Ha | Java Platform Team | jwha at google.com
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150803/9efe39e2/attachment.htm>

From kim.barrett at oracle.com  Mon Aug  3 22:43:27 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Mon, 3 Aug 2015 18:43:27 -0400
Subject: RFR (M): 8073052: Rename and clean up the allocation manager
	hierarchy in g1Allocator.?pp
In-Reply-To: <1438156281.2302.16.camel@oracle.com>
References: <1438073174.2272.17.camel@oracle.com> <55B778A4.4050602@oracle.com>
	<1438156281.2302.16.camel@oracle.com>
Message-ID: <ACF8A0BA-3821-4A49-BB4A-6D87AF033FE9@oracle.com>

On Jul 29, 2015, at 3:51 AM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
> 
> On Tue, 2015-07-28 at 14:42 +0200, Mikael Gerdin wrote:
>> G1PLABAllocator seems to look up the G1H::heap() fairly often,
>> did you consider passing on the G1H* to the G1PLABAllocator to reduce 
>> the clutter from accessing through the static getter in multiple 
>> locations? Another option could be to expose the G1H* from G1Allocator, 
>> yielding _allocator->heap()->...
>> instead of _g1h->...
> 
> Fixed.
> 
> New webrevs:
> http://cr.openjdk.java.net/~tschatzl/8073052/webrev.4/ (full)
> http://cr.openjdk.java.net/~tschatzl/8073052/webrev.3_to_4/ (diff)

------------------------------------------------------------------------------ 
src/share/vm/gc/g1/g1Allocator.cpp
 186   size_t gclab_word_size = _allocator->heap()->desired_plab_sz(dest);

[here and in other places]
I was expecting to see _g1h->desired_plab_sz(dest) here, in response
to Mikael's suggestion, but I see you took his "Another option"
instead.  Any reason for that, rather than giving the G1PLABAllocator
direct access to the heap object?

------------------------------------------------------------------------------

Other than that one question, looks good.



From martijnverburg at gmail.com  Tue Aug  4 07:36:58 2015
From: martijnverburg at gmail.com (Martijn Verburg)
Date: Tue, 4 Aug 2015 08:36:58 +0100
Subject: RFR (XL): JDK-8086706 Parallel Full GC for CMS
In-Reply-To: <CAPYFHW1pBh85HA1X6qxz3t2pP3kciRiNeBaMm5PiYENimohowg@mail.gmail.com>
References: <CA+n_jhjw=T2O3pquZU++N5miGdXxwh=Kp_TBDTmk2ud79SJCgg@mail.gmail.com>
	<5579B1DB.9090804@oracle.com>
	<CA+3eh10-MyRSJ8gcievY_=hTXUjshGnUfwmZFGMsd9ZMD-xBuw@mail.gmail.com>
	<C9B58FD5-C1E4-4143-A85B-4AB322246822@oracle.com>
	<CA+n_jhjRd3_rm9v7n5qMGWptDqPcQ-ax4w4eHT91R=r1Tyj6mA@mail.gmail.com>
	<55BFB8BF.7000002@oracle.com>
	<CAP7YuASeh08VopSxzvR=L1m3NcLOBYr5UqNgYHyRnzL2J9UQ8g@mail.gmail.com>
	<CAPYFHW1pBh85HA1X6qxz3t2pP3kciRiNeBaMm5PiYENimohowg@mail.gmail.com>
Message-ID: <CAP7YuAQ-bF8_B+tDv8UuaW2AjtbCd_nB-+00F1md9jz4CARr7Q@mail.gmail.com>

Nothing except for the fact I'm an idiot and forgot it was already posted!

Cheers,
Martijn

On 3 August 2015 at 23:20, Jeremy Manson <jeremymanson at google.com> wrote:

> What's the matter with the JEP that Jungwoo sent?
>
> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2015-July/014092.html
>
> Jeremy
>
> On Mon, Aug 3, 2015 at 2:06 PM, Martijn Verburg <martijnverburg at gmail.com>
> wrote:
>
>> We'd certainly like to see a JEP for this.  There are use cases where CMS
>> is the right collector and this proposed patch could (I'll stress could)
>> significantly improve its performance.
>>
>> Cheers,
>> Martijn
>>
>> On 3 August 2015 at 19:53, Jon Masamitsu <jon.masamitsu at oracle.com>
>> wrote:
>>
>>> Jungwoo,
>>>
>>> As you can see no one has stepped up to do your review.
>>> We?ve talked about it amongst the active GC committers.
>>> It?s a large change and would require a very significant
>>> commitment of anyone?s time.  As is obvious, our focus
>>> is on G1.  And right now we?re in the position of having to
>>> push tasks we?ve long planned for JDK 9 out into a JDK 9
>>> update.  No one I?ve talked with is willing to push off more
>>> of our G1 work to get this patch in.  And as maintainers
>>> of the HotSpot GC's due diligence is required of us before
>>> we can accept this change.
>>>
>>> Jon
>>>
>>> PS.  I saw that you said you had started another thread for this
>>> discussion (or maybe the discussion of the JEP) but I could not
>>> it.    Please feel free to repost this under the new subject.
>>>
>>> On 06/27/2015 05:48 PM, Jungwoo Ha wrote:
>>>
>>> Sorry for the long wait. So many things have cleaned up on JDK9.
>>> Here is the webrev against jdk9/hs-rt.
>>> http://cr.openjdk.java.net/~jwha/8086706/webrev.jdk9.00
>>>
>>> I am preparing for the JEP doc.
>>>
>>>
>>> On Thu, Jun 11, 2015 at 12:31 PM, Kim Barrett <kim.barrett at oracle.com>
>>> wrote:
>>>
>>>> On Jun 11, 2015, at 12:33 PM, Volker Simonis <volker.simonis at gmail.com>
>>>> wrote:
>>>> >
>>>> > Hi,
>>>> >
>>>> > if nobody has already started doing it, I'll try to forward port the
>>>> > patch to jdk9/hs-gc/hotspot.
>>>>
>>>> You?ll want to use jdk9/hs-rt rather than jdk9/hs-gc.
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Jungwoo Ha | Java Platform Team | jwha at google.com
>>>
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150804/c2b03a50/attachment.htm>

From thomas.schatzl at oracle.com  Tue Aug  4 07:48:59 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Tue, 04 Aug 2015 09:48:59 +0200
Subject: RFR (M): 8073052: Rename and clean up the allocation manager
	hierarchy in g1Allocator.?pp
In-Reply-To: <ACF8A0BA-3821-4A49-BB4A-6D87AF033FE9@oracle.com>
References: <1438073174.2272.17.camel@oracle.com>
	<55B778A4.4050602@oracle.com> <1438156281.2302.16.camel@oracle.com>
	<ACF8A0BA-3821-4A49-BB4A-6D87AF033FE9@oracle.com>
Message-ID: <1438674539.2146.5.camel@oracle.com>

Hi,

On Mon, 2015-08-03 at 18:43 -0400, Kim Barrett wrote:
> On Jul 29, 2015, at 3:51 AM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
> > 
> > On Tue, 2015-07-28 at 14:42 +0200, Mikael Gerdin wrote:
> >> G1PLABAllocator seems to look up the G1H::heap() fairly often,
> >> did you consider passing on the G1H* to the G1PLABAllocator to reduce 
> >> the clutter from accessing through the static getter in multiple 
> >> locations? Another option could be to expose the G1H* from G1Allocator, 
> >> yielding _allocator->heap()->...
> >> instead of _g1h->...
> > 
> > Fixed.
> > 
> > New webrevs:
> > http://cr.openjdk.java.net/~tschatzl/8073052/webrev.4/ (full)
> > http://cr.openjdk.java.net/~tschatzl/8073052/webrev.3_to_4/ (diff)
> 
> ------------------------------------------------------------------------------ 
> src/share/vm/gc/g1/g1Allocator.cpp
>  186   size_t gclab_word_size = _allocator->heap()->desired_plab_sz(dest);
> 
> [here and in other places]
> I was expecting to see _g1h->desired_plab_sz(dest) here, in response
> to Mikael's suggestion, but I see you took his "Another option"
> instead.  Any reason for that, rather than giving the G1PLABAllocator
> direct access to the heap object?
> 

PLABAllocator does not have a local reference to G1CollectedHeap. So I
could have either added one, or used the one from the G1Allocator.

I somewhat tend to dislike the thousands of local copies of
G1CollectedHeap for no reason.

If you strongly prefer making another copy, I can change this likewise.

Thanks for the review,
  Thomas




From staffan.larsen at oracle.com  Tue Aug  4 11:04:00 2015
From: staffan.larsen at oracle.com (Staffan Larsen)
Date: Tue, 4 Aug 2015 13:04:00 +0200
Subject: Low-Overhead Heap Profiling
In-Reply-To: <CAPYFHW2ryBf7DyAsEyAPW46atiWvFQpvYV-9HROaB=yp1=2p-A@mail.gmail.com>
References: <etPan.5589bc61.5358f81b.1215a@tw-mbp-tprintezis>
	<CAPYFHW3fVjDfc1j6du5dbAvaaKZsErx7vL8njx7hFhe-Ybvk6Q@mail.gmail.com>
	<etPan.558aef95.6b2c6c0b.1215a@tw-mbp-tprintezis>
	<etPan.558c716e.236c4c7e.1215a@tw-mbp-tprintezis>
	<CAPYFHW2ryBf7DyAsEyAPW46atiWvFQpvYV-9HROaB=yp1=2p-A@mail.gmail.com>
Message-ID: <B3482D30-E883-4002-A8D6-8651A1D3349E@oracle.com>

Hi,

Sorry for being away for so long.

If memory serves me right then the current AllocObjectInNewTLAB JFR event was written as a way to quickly get some allocation profiling information with a minimum of VM changes. It probably also carried over to Hotspot from JRockit. I agree that we can and should collect better information than what that event does. 

I like the approach of instrumenting the interpreter and compiler to do the sampling. I had thought it would be a lot of work to do it and was reluctant to go down that path. If the work is already done and Jeremy has maintained it for a few years without great problems, I think we should look at bringing it in. I am not worried about the overhead of a decrement and a compare in the allocation path, but of course we should benchmark that.

It wasn?t clear to me from the discussion how (or if) this was being exposed to end users. Was the proposal to just have the native entry points in the VM and have these be used by various agents? Would this be part of JVMTI?

I think a draft JEP would be the logical next step and make it easier for us all to discuss what exactly the proposal should contain. Also, let?s not forget the need for tests for the feature.

Thanks,
/Staffan




From michail.chernov at oracle.com  Tue Aug  4 14:35:01 2015
From: michail.chernov at oracle.com (Michail Chernov)
Date: Tue, 4 Aug 2015 17:35:01 +0300
Subject: RFR: 8081317: [NEWTEST] documented GC ratio tuning and new size
	options should be covered by regression tests
In-Reply-To: <55BBB0BE.6080506@oracle.com>
References: <55BBB0BE.6080506@oracle.com>
Message-ID: <55C0CD95.1080305@oracle.com>

Hi,

Could someone please take a look at this review?

Thanks,
Michail

On 31.07.2015 20:30, Michail Chernov wrote:
> Hi,
>
> Please review these changes for JDK-8081317. There are new tests to 
> cover GC ratio tuning and new size options.
>
> Webrev:  http://cr.openjdk.java.net/~mchernov/8081317/
> Bug: https://bugs.openjdk.java.net/browse/JDK-8081317
>
> TestMaxMinHeapFreeRatioFlags.java - Verify that heap size changes 
> according to max and min heap free ratios.
>
> TestMinAndInitialSurvivorRatioFlags.java - Test verifies that VM can 
> start with any GC when MinSurvivorRatio and InitialSurvivorRatio flags 
> passed and for Parallel GC it verifies that after start up survivor 
> ratio is equal to InitialSurvivorRatio value and that actual survivor 
> ratio will never be less than MinSurvivorRatio.
>
> TestNewRatioFlag.java - Verify that heap divided among generations 
> according to NewRatio.
>
> TestNewSizeFlags.java - Verify that young gen size conforms values 
> specified by NewSize, MaxNewSize and Xmn options.
>
> TestSurvivorRatioFlag - Verify that actual survivor ratio is equal to 
> specified SurvivorRatio value.
>
> TestTargetSurvivorRatioFlag.java - Test if objects size is less than 
> (survivor_size * TargetSurvivorRatio / 100) then objects are stayed in 
> survivor space until MaxTenuringThreshold minor GC cycles. If more 
> than (survivor_size * TargetSurvivorRatio / 100) objects were 
> allocated, then test verifies that after MaxTenuringThreshold minor GC 
> cycles survivor space is almost empty.
>
> New changes were tested on different platforms including embedded jdk 
> on Linux x86.
>
> Thanks,
> Michail
>



From kim.barrett at oracle.com  Tue Aug  4 16:59:58 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Tue, 4 Aug 2015 12:59:58 -0400
Subject: RFR (M): 8073052: Rename and clean up the allocation manager
	hierarchy in g1Allocator.?pp
In-Reply-To: <1438674539.2146.5.camel@oracle.com>
References: <1438073174.2272.17.camel@oracle.com> <55B778A4.4050602@oracle.com>
	<1438156281.2302.16.camel@oracle.com>
	<ACF8A0BA-3821-4A49-BB4A-6D87AF033FE9@oracle.com>
	<1438674539.2146.5.camel@oracle.com>
Message-ID: <6D1848C8-9A54-43A0-A7EF-2B079A40D126@oracle.com>

On Aug 4, 2015, at 3:48 AM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
> 
> PLABAllocator does not have a local reference to G1CollectedHeap. So I
> could have either added one, or used the one from the G1Allocator.
> 
> I somewhat tend to dislike the thousands of local copies of
> G1CollectedHeap for no reason.
> 
> If you strongly prefer making another copy, I can change this likewise.

Having a local reference to the heap object is pretty idiomatic in
this code base.  I'd rather follow existing practice, and dislike a
mix of mechanisms.  So yes, I'd prefer that form.



From tom.benson at oracle.com  Tue Aug  4 18:25:40 2015
From: tom.benson at oracle.com (Tom Benson)
Date: Tue, 04 Aug 2015 14:25:40 -0400
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
Message-ID: <55C103A4.1060505@oracle.com>

Hi,
Please review this change for JDK-8131734, an addition to the Archive 
Region support in G1 to handle -Xshared:auto behavior.  With 
Xshared:auto, if the shared string region file mapping fails, sharing is 
disabled and execution continues.  To allow this, the archive region 
space which has already been allocated needs to be freed.   A 
free_archive_regions() entry is added to g1CollectedHeap, to be called 
if the file mapping performed by CDS fails.  Its arguments are the same 
as those given to alloc_archive_regions (and that would have been given 
to fill_archive_regions if the mapping had succeeded).

The CDS code change will be posted separately by Jiangli Zhou.

JBS: https://bugs.openjdk.java.net/browse/JDK-8131734
Webrev: http://cr.openjdk.java.net/~tbenson/8131734/webrev/
Tested: JPRT, and benchmarks run with alloc/free_archive_regions calls 
forced at init time and heap verification enabled.
Jiangli also tried this with the failing Xshared:auto test and the 
corresponding CDS code change to call free_archive_regions.

Thanks,
Tom




From jon.masamitsu at oracle.com  Tue Aug  4 18:30:55 2015
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Tue, 04 Aug 2015 11:30:55 -0700
Subject: RFR (M): 8073052: Rename and clean up the allocation manager
	hierarchy in g1Allocator.?pp
In-Reply-To: <55B89724.7030200@oracle.com>
References: <1438073174.2272.17.camel@oracle.com>		<55B778A4.4050602@oracle.com>
	<1438156281.2302.16.camel@oracle.com> <55B89724.7030200@oracle.com>
Message-ID: <55C104DF.3060403@oracle.com>

Thomas,

I reviewed webrev.4 and it looks good.

Jon


On 07/29/2015 02:04 AM, Mikael Gerdin wrote:
> Thomas,
>
> On 2015-07-29 09:51, Thomas Schatzl wrote:
>> Hi Mikael,
>>
>>    thanks for the review.
>>
>> On Tue, 2015-07-28 at 14:42 +0200, Mikael Gerdin wrote:
>>> Hi Thomas,
>>>
>>> On 2015-07-28 10:46, Thomas Schatzl wrote:
>>>> Hi all,
>>> [...]
>>>> Finally, the change aligns the naming of G1ParGCAllocator with other
>>>> classes that use "PLAB" in their names instead of "ParGC".
>>>>
>>>> CR:
>>>> https://bugs.openjdk.java.net/browse/JDK-8073052
>>>>
>>>> Webrev:
>>>> http://cr.openjdk.java.net/~tschatzl/8073052/webrev.3/
>>>
>>> Overall a nice cleanup!
>>>
>>> g1Allocator.cpp:
>>> In unsafe_max_tlab_alloc you get the heap via G1H::heap() instead of 
>>> _g1h.
>>
>> Fixed.
>>
>>> G1PLABAllocator seems to look up the G1H::heap() fairly often,
>>> did you consider passing on the G1H* to the G1PLABAllocator to reduce
>>> the clutter from accessing through the static getter in multiple
>>> locations? Another option could be to expose the G1H* from G1Allocator,
>>> yielding _allocator->heap()->...
>>> instead of _g1h->...
>>
>> Fixed.
>>
>> New webrevs:
>> http://cr.openjdk.java.net/~tschatzl/8073052/webrev.4/ (full)
>> http://cr.openjdk.java.net/~tschatzl/8073052/webrev.3_to_4/ (diff)
>
> Looks good to me.
> /Mikael
>
>>
>> Thanks,
>>    Thomas
>>
>>
>



From jeremymanson at google.com  Tue Aug  4 21:22:52 2015
From: jeremymanson at google.com (Jeremy Manson)
Date: Tue, 4 Aug 2015 14:22:52 -0700
Subject: Low-Overhead Heap Profiling
In-Reply-To: <B3482D30-E883-4002-A8D6-8651A1D3349E@oracle.com>
References: <etPan.5589bc61.5358f81b.1215a@tw-mbp-tprintezis>
	<CAPYFHW3fVjDfc1j6du5dbAvaaKZsErx7vL8njx7hFhe-Ybvk6Q@mail.gmail.com>
	<etPan.558aef95.6b2c6c0b.1215a@tw-mbp-tprintezis>
	<etPan.558c716e.236c4c7e.1215a@tw-mbp-tprintezis>
	<CAPYFHW2ryBf7DyAsEyAPW46atiWvFQpvYV-9HROaB=yp1=2p-A@mail.gmail.com>
	<B3482D30-E883-4002-A8D6-8651A1D3349E@oracle.com>
Message-ID: <CAPYFHW00zGByRRTz3m_Xx=2OndU9WybK6=vQt_-cchDqe3fuZg@mail.gmail.com>

Thanks, Staffan.  I've been tinkering with a draft JEP, but it has been
going slowly because of other craziness in my life.  Some responses:

1) It was a fair bit of work to do it, mostly because it was the first time
I had tinkered with c2.  This was 5 years ago.  Most of the work since then
has been dealing with merge conflicts.

2) I did envision a JVMTI interface.  More in the JEP.

3) We have some internal tests, but obviously, we'd have to figure out how
to externalize them.  For native code, it's much easier only to have to
worry about building and running on Linux.  Presumably, there's already
code in there for JVMTI.

I'll try to get a JEP going for real, although it might not happen in the
next week or so, because I'm moving next week (+the JVMLS).

Jeremy

On Tue, Aug 4, 2015 at 4:04 AM, Staffan Larsen <staffan.larsen at oracle.com>
wrote:

> Hi,
>
> Sorry for being away for so long.
>
> If memory serves me right then the current AllocObjectInNewTLAB JFR event
> was written as a way to quickly get some allocation profiling information
> with a minimum of VM changes. It probably also carried over to Hotspot from
> JRockit. I agree that we can and should collect better information than
> what that event does.
>
> I like the approach of instrumenting the interpreter and compiler to do
> the sampling. I had thought it would be a lot of work to do it and was
> reluctant to go down that path. If the work is already done and Jeremy has
> maintained it for a few years without great problems, I think we should
> look at bringing it in. I am not worried about the overhead of a decrement
> and a compare in the allocation path, but of course we should benchmark
> that.
>
> It wasn?t clear to me from the discussion how (or if) this was being
> exposed to end users. Was the proposal to just have the native entry points
> in the VM and have these be used by various agents? Would this be part of
> JVMTI?
>
> I think a draft JEP would be the logical next step and make it easier for
> us all to discuss what exactly the proposal should contain. Also, let?s not
> forget the need for tests for the feature.
>
> Thanks,
> /Staffan
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150804/f9d9e072/attachment.htm>

From kim.barrett at oracle.com  Tue Aug  4 22:06:00 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Tue, 4 Aug 2015 18:06:00 -0400
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <55C103A4.1060505@oracle.com>
References: <55C103A4.1060505@oracle.com>
Message-ID: <A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>

On Aug 4, 2015, at 2:25 PM, Tom Benson <tom.benson at oracle.com> wrote:
> 
> Hi,
> Please review this change for JDK-8131734, an addition to the Archive Region support in G1 to handle -Xshared:auto behavior.  With Xshared:auto, if the shared string region file mapping fails, sharing is disabled and execution continues.  To allow this, the archive region space which has already been allocated needs to be freed.   A free_archive_regions() entry is added to g1CollectedHeap, to be called if the file mapping performed by CDS fails.  Its arguments are the same as those given to alloc_archive_regions (and that would have been given to fill_archive_regions if the mapping had succeeded).
> 
> The CDS code change will be posted separately by Jiangli Zhou.
> 
> JBS: https://bugs.openjdk.java.net/browse/JDK-8131734
> Webrev: http://cr.openjdk.java.net/~tbenson/8131734/webrev/
> Tested: JPRT, and benchmarks run with alloc/free_archive_regions calls forced at init time and heap verification enabled.
> Jiangli also tried this with the failing Xshared:auto test and the corresponding CDS code change to call free_archive_regions.

------------------------------------------------------------------------------ 
src/share/vm/gc/g1/g1CollectedHeap.cpp  
1134     size_used += ranges[i].word_size() * HeapWordSize;

Why not ranges[i].byte_size() ?

------------------------------------------------------------------------------ 
src/share/vm/gc/g1/g1CollectedHeap.cpp  
1144     if ((prev_last_region != NULL) && (start_region == prev_last_region)) {

I think (prev_last_region != NULL) is unnecessary here.

------------------------------------------------------------------------------
src/share/vm/gc/g1/g1CollectedHeap.cpp 
1155     HeapRegion* curr_region = start_region;
1156     while (curr_region != NULL) {
...
1162       if (curr_region != last_region) {
1163         curr_region = _hrm.next_region_in_heap(curr_region);
1164       } else {
1165         curr_region = NULL;
1166       }
1167     }

I think I would have found this easier to read if structured something
like:

  while (true) {
    ...
    if (curr_region == last_region) {
      break;
    } else {
      curr_region = _hrm.next_region_in_heap(curr_region);
      assert(curr_region != NULL, ...);
    }
  }

I?ll leave that up to you.

But, is next_region_in_heap really the right stepper function?  It
skips over regions that are not "is_available".  Are the regions we're
dealing with in this function all guaranteed to be available?
Clearly, bad things happen if last_region is not available while using
next_region_in_heap here.

------------------------------------------------------------------------------ 
src/share/vm/gc/g1/g1CollectedHeap.hpp
 789   // For each of the specified MemRegions, free the containing G1 regions 
 790   // which had been allocated by alloc_archive_regions. This should be called
 791   // rather than fill_archive_regions at JVM init time if the archive file 
 792   // mapping failed.
 793   void free_archive_regions(MemRegion* range, size_t count);

The implementation presently requires the ranges to be non-overlapping
and sorted in increasing order.  That ought to be said in the
description.

------------------------------------------------------------------------------
src/share/vm/gc/g1/g1MarkSweep.hpp
  61   // Mark or un-mark the regions containing the specified address range as archives.
  62   static void mark_range_archive(MemRegion range, bool is_archive);

With the functionality change, maybe the name should be changed.
Perhaps set_range_archive?

------------------------------------------------------------------------------



From thomas.schatzl at oracle.com  Wed Aug  5 08:13:29 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 05 Aug 2015 10:13:29 +0200
Subject: RFR (M): 8073052: Rename and clean up the allocation manager
	hierarchy in g1Allocator.?pp
In-Reply-To: <6D1848C8-9A54-43A0-A7EF-2B079A40D126@oracle.com>
References: <1438073174.2272.17.camel@oracle.com>
	<55B778A4.4050602@oracle.com> <1438156281.2302.16.camel@oracle.com>
	<ACF8A0BA-3821-4A49-BB4A-6D87AF033FE9@oracle.com>
	<1438674539.2146.5.camel@oracle.com>
	<6D1848C8-9A54-43A0-A7EF-2B079A40D126@oracle.com>
Message-ID: <1438762409.2378.2.camel@oracle.com>

Hi,

On Tue, 2015-08-04 at 12:59 -0400, Kim Barrett wrote:
> On Aug 4, 2015, at 3:48 AM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
> > 
> > PLABAllocator does not have a local reference to G1CollectedHeap. So I
> > could have either added one, or used the one from the G1Allocator.
> > 
> > I somewhat tend to dislike the thousands of local copies of
> > G1CollectedHeap for no reason.
> > 
> > If you strongly prefer making another copy, I can change this likewise.
> 
> Having a local reference to the heap object is pretty idiomatic in
> this code base.  I'd rather follow existing practice, and dislike a
> mix of mechanisms.  So yes, I'd prefer that form.

  new webrev:
http://cr.openjdk.java.net/~tschatzl/8073052/webrev.5
http://cr.openjdk.java.net/~tschatzl/8073052/webrev.4_to_5

Thanks,
  Thomas




From thomas.schatzl at oracle.com  Wed Aug  5 08:25:08 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 05 Aug 2015 10:25:08 +0200
Subject: RFR (S): 8003237: G1: Reduce unnecessary (and failing)
	allocation attempts when handling an evacuation failure
In-Reply-To: <1438605827.2292.49.camel@oracle.com>
References: <1438605320.2292.47.camel@oracle.com>
	<1438605827.2292.49.camel@oracle.com>
Message-ID: <1438763108.2378.9.camel@oracle.com>

Hi all,

  just fyi, I had to update the webrev because of changes in the 8073052
patch.
Since there were no reviews of these changes yet, I updated in-place.

There are no functional changes.

Also, if somebody already had a look at the changes, and the question
why G1ParScanThreadState::allocate_in_next_plab() only uses failed PLAB
allocation as indicator that heap is full came up, the later patch
JDK-8067336 "Allow that PLAB allocations at the end of regions are
flexible" will allow that PLABs can get quite small as they are sized
flexibly.

I wanted to avoid introducing some even more arbitrary heuristic to
determine when the heap is full now which will only a bit later be
superseded again anyway.

In any case current G1ParScanThreadState::allocate_in_next_plab() should
try harder than the previous one by always trying an inline allocation
even if it cannot allocate the object (supposedly smaller than a PLAB)
using a new PLAB.

Thanks,
  Thomas

On Mon, 2015-08-03 at 14:43 +0200, Thomas Schatzl wrote:
> Hi,
> 
>   this change is based on the latest version of the change for 8073052
> that is out for review, and is still longing for another reviewer :)
> 
> (I am okay with pushing it with only a single "R"eviewer since it is
> only refactoring, but if somebody can find the time...)
> 
> Thanks,
>   Thomas
> 
> On Mon, 2015-08-03 at 14:35 +0200, Thomas Schatzl wrote:
> > Hi all,
> > 
> >   can I have reviews for this small change that improves evacuation
> > failure handling by short-cutting copy_to_survivor_space() when there is
> > no more space left?
> > 
> > In effect this makes the parallel phase during GC have the same
> > performance as if there is no evacuation failure, and I could not
> > measure regressions in the normal case either.
> > 
> > The change implements two things: first, notice when there is no more
> > space left in the last gen (i.e. old), and if so, short-cut
> > ParScanThreadState::copy_to_survivor_space(). The other change is to not
> > try to ask the free list for new regions if it's empty to avoid taking
> > the lock in the time window between the first few evac failures and when
> > space is completely empty.
> > 
> > There is still the somewhat heavyweight cleanup phases after evacuation
> > failure.
> > 
> > CR:
> > https://bugs.openjdk.java.net/browse/JDK-8003237
> > Webrev:
> > http://cr.openjdk.java.net/~tschatzl/8003237/webrev/
> > Testing:
> > jprt, lots of performance measurements with and without this change
> > compiled in
> > 
> > Thanks,
> >   Thomas
> > 
> > 
> 
> 




From thomas.schatzl at oracle.com  Wed Aug  5 11:59:53 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 05 Aug 2015 13:59:53 +0200
Subject: RFR (XXS): 8133043: Clean up code related to termination stats
	printing
Message-ID: <1438775993.2378.31.camel@oracle.com>

Hi all,

  can I have some reviews for the following change that cleans up and
reformats the code and usage related to G1 termination stats?

Stuff like not using braces around if-clauses, dropping an extra
temporary lock, linebreaks in string constants, line breaks in the
definition of the methods, superfluous additional parameters.

I generally do not do reformatting changes, but every time I browse
through that code it gives me an urge to fix that.

CR:
https://bugs.openjdk.java.net/browse/JDK-8133043
Webrev:
http://cr.openjdk.java.net/~tschatzl/8133043/webrev/
Testing:
compilation, gcbasher run with -XX:+PrintTerminationStats, jprt

Thanks,
  Thomas



From thomas.schatzl at oracle.com  Wed Aug  5 11:59:59 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 05 Aug 2015 13:59:59 +0200
Subject: RFR (XXS): 8133042: Remove some direct accesses of
	G1Allocator::reuse_retained_old_region() to G1CollectedHeap members
Message-ID: <1438775999.2378.32.camel@oracle.com>

Hi all,

  could I have reviews of this tiny change that changes some direct
private member accesses of G1Allocator to G1CollectedHeap to some
already available public method calls?

While this is more a cosmetic change, I thought it would be good to
decrease the amount of "friendliness" between those two classes. I could
not find a good other CR to add them, so here we are....

CR:
https://bugs.openjdk.java.net/browse/JDK-8133042
Webrev:
http://cr.openjdk.java.net/~tschatzl/8133042/webrev/
Testing:
local compilation

Thanks,
  Thomas



From thomas.schatzl at oracle.com  Wed Aug  5 12:40:15 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 05 Aug 2015 14:40:15 +0200
Subject: RFR (S): 8133047: Rename G1ParScanThreadState::_queue_num to
	_worker_id
Message-ID: <1438778415.2378.45.camel@oracle.com>

Hi all,

  can I have reviews for this small cleanup that renames the
G1ParScanThreadState::_queue_num member to worker_id like it is called
for all members that have the same functionality.

G1ParScanThreadState::_queue_num is actually compared to a worker_id
member of another class, and assigned to a _worker_id member somewhere
else in the code.

It hopefully makes the code slightly easier to read in the future by not
using different names for the same thing throughput the code.

CR:
https://bugs.openjdk.java.net/browse/JDK-8133047
Webrev:
http://cr.openjdk.java.net/~tschatzl/8133047/webrev/
testing:
jprt

Thanks,
  Thomas




From thomas.schatzl at oracle.com  Wed Aug  5 13:37:24 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 05 Aug 2015 15:37:24 +0200
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>
References: <55C103A4.1060505@oracle.com>
	<A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>
Message-ID: <1438781844.2378.60.camel@oracle.com>

Hi,

On Tue, 2015-08-04 at 18:06 -0400, Kim Barrett wrote:
> On Aug 4, 2015, at 2:25 PM, Tom Benson <tom.benson at oracle.com> wrote:
> > 
> > Hi,
> > Please review this change for JDK-8131734, an addition to the Archive Region support in G1 to handle -Xshared:auto behavior.  With Xshared:auto, if the shared string region file mapping fails, sharing is disabled and execution continues.  To allow this, the archive region space which has already been allocated needs to be freed.   A free_archive_regions() entry is added to g1CollectedHeap, to be called if the file mapping performed by CDS fails.  Its arguments are the same as those given to alloc_archive_regions (and that would have been given to fill_archive_regions if the mapping had succeeded).
> > 
> > The CDS code change will be posted separately by Jiangli Zhou.
> > 
> > JBS: https://bugs.openjdk.java.net/browse/JDK-8131734
> > Webrev: http://cr.openjdk.java.net/~tbenson/8131734/webrev/
> > Tested: JPRT, and benchmarks run with alloc/free_archive_regions calls forced at init time and heap verification enabled.
> > Jiangli also tried this with the failing Xshared:auto test and the corresponding CDS code change to call free_archive_regions.
> 
> ----------------------------------

[...]
> 
> But, is next_region_in_heap really the right stepper function?  It
> skips over regions that are not "is_available".  Are the regions we're
> dealing with in this function all guaranteed to be available?

Yes. We assume that in the earlier call in allocate_containing_regions()
G1 made them available (committed them).

I assume that when mmap'ing it, the  

> Clearly, bad things happen if last_region is not available while using
> next_region_in_heap here.

One could do regular pointer arithmetic *within* the MemRegion (which is
always a guaranteed contiguous range) and then map the address back to
the HeapRegion*.

- I have some question about this code, particularly about the comment:

1137     HeapRegion* start_region = _hrm.addr_to_region(start_address);
1138     HeapRegion* last_region = _hrm.addr_to_region(last_address);
1139 
1140     // Check for ranges that start in the same G1 region in which the previous
1141     // range ended, and adjust the start address so we don't try to free
1142     // the same region again. If the current range is entirely within that
1143     // region, skip it.
1144     if ((prev_last_region != NULL) && (start_region == prev_last_region)) {
1145       start_address = start_region->end();
1146       if (start_address > last_address) {
1147         continue;
1148       }
1149       start_region = _hrm.addr_to_region(start_address);
1150     }

How could the situation mentioned in line 1140 happen? Are the given
memory regions not overlapping already, and the start addresses of these
MemRegions at the start of these regions?

Since last_region is the region containing the last address within the
memory range, wouldn't that mean given above preconditions, this could
not happen?

- I think the method should add a assert_at_safepoint(true) at the top
(and possibly all other archival methods if they are not yet through the
call chain), or decrease_used() made safe against concurrent
modification using the ParGC_Rare_Event_lock.

I would prefer just making sure the code is only run at a safepoint.

Thanks,
  Thomas





From thomas.schatzl at oracle.com  Wed Aug  5 13:52:37 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 05 Aug 2015 15:52:37 +0200
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <1438781844.2378.60.camel@oracle.com>
References: <55C103A4.1060505@oracle.com>
	<A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>
	<1438781844.2378.60.camel@oracle.com>
Message-ID: <1438782757.2378.62.camel@oracle.com>

Hi again,

[...]
> - I have some question about this code, particularly about the comment:
> 
> 1137     HeapRegion* start_region = _hrm.addr_to_region(start_address);
> 1138     HeapRegion* last_region = _hrm.addr_to_region(last_address);
> 1139 
> 1140     // Check for ranges that start in the same G1 region in which the previous
> 1141     // range ended, and adjust the start address so we don't try to free
> 1142     // the same region again. If the current range is entirely within that
> 1143     // region, skip it.
> 1144     if ((prev_last_region != NULL) && (start_region == prev_last_region)) {
> 1145       start_address = start_region->end();
> 1146       if (start_address > last_address) {
> 1147         continue;
> 1148       }
> 1149       start_region = _hrm.addr_to_region(start_address);
> 1150     }
> 
> How could the situation mentioned in line 1140 happen? Are the given
> memory regions not overlapping already, and the start addresses of these
> MemRegions at the start of these regions?

Probably because of using the same memory mapped file created from a VM
with different (smaller) heap region size?

Thanks,
  Thomas




From tom.benson at oracle.com  Wed Aug  5 14:12:41 2015
From: tom.benson at oracle.com (Tom Benson)
Date: Wed, 05 Aug 2015 10:12:41 -0400
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>
References: <55C103A4.1060505@oracle.com>
	<A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>
Message-ID: <55C219D9.90500@oracle.com>

Hi Kim,
Thanks very much for the review.

On 8/4/2015 6:06 PM, Kim Barrett wrote:
> ------------------------------------------------------------------------------
> src/share/vm/gc/g1/g1CollectedHeap.cpp
> 1134     size_used += ranges[i].word_size() * HeapWordSize;
>
> Why not ranges[i].byte_size() ?

Indeed, why not?  I'll change it.

>
> ------------------------------------------------------------------------------
> src/share/vm/gc/g1/g1CollectedHeap.cpp
> 1144     if ((prev_last_region != NULL) && (start_region == prev_last_region)) {
>
> I think (prev_last_region != NULL) is unnecessary here.

I agree.  Leftover from the alloc_archive_regions code I started with.

>
> ------------------------------------------------------------------------------
> src/share/vm/gc/g1/g1CollectedHeap.cpp
> 1155     HeapRegion* curr_region = start_region;
> 1156     while (curr_region != NULL) {
> ...
> 1162       if (curr_region != last_region) {
> 1163         curr_region = _hrm.next_region_in_heap(curr_region);
> 1164       } else {
> 1165         curr_region = NULL;
> 1166       }
> 1167     }
>
> I think I would have found this easier to read if structured something
> like:
>
>    while (true) {
>      ...
>      if (curr_region == last_region) {
>        break;
>      } else {
>        curr_region = _hrm.next_region_in_heap(curr_region);
>        assert(curr_region != NULL, ...);
>      }
>    }
>
> I?ll leave that up to you.

Both alloc_archive_regions and fill_archive_regions are written this way 
(the former for a slightly better reason), so I think I'll leave them 
all as-is.

>
> But, is next_region_in_heap really the right stepper function?

I see Thomas talked about this in his reply, but, yes.
>   It
> skips over regions that are not "is_available".  Are the regions we're
> dealing with in this function all guaranteed to be available?
This is basically part of an error path in mapping shared string space.  
We know the alloc_ succeeded, but the file mapping subsequently failed.
> Clearly, bad things happen if last_region is not available while using
> next_region_in_heap here.
>
> ------------------------------------------------------------------------------
> src/share/vm/gc/g1/g1CollectedHeap.hpp
>   789   // For each of the specified MemRegions, free the containing G1 regions
>   790   // which had been allocated by alloc_archive_regions. This should be called
>   791   // rather than fill_archive_regions at JVM init time if the archive file
>   792   // mapping failed.
>   793   void free_archive_regions(MemRegion* range, size_t count);
>
> The implementation presently requires the ranges to be non-overlapping
> and sorted in increasing order.  That ought to be said in the
> description.

OK.  I'll add a comment that the args should be the same as passed to 
alloc_archive_regions.
>
> ------------------------------------------------------------------------------
> src/share/vm/gc/g1/g1MarkSweep.hpp
>    61   // Mark or un-mark the regions containing the specified address range as archives.
>    62   static void mark_range_archive(MemRegion range, bool is_archive);
>
> With the functionality change, maybe the name should be changed.
> Perhaps set_range_archive?
>
> ------------------------------------------------------------------------------
>
I considered this, but didn't see much difference between "marking" a 
range as archive/non-archive and "setting" one.  However, I'll change it 
to "set" to avoid any possible confusion with "marking" in the normal GC 
sense.
Thanks,
Tom


From jesper.wilhelmsson at oracle.com  Wed Aug  5 14:18:14 2015
From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson)
Date: Wed, 05 Aug 2015 16:18:14 +0200
Subject: RFR (XXS): 8133043: Clean up code related to termination stats
	printing
In-Reply-To: <1438775993.2378.31.camel@oracle.com>
References: <1438775993.2378.31.camel@oracle.com>
Message-ID: <55C21B26.1040800@oracle.com>

Looks good! Thanks for cleaning this up!

Do you mind adding an empty line between the method declarations in 
g1ParScanThreadState.hpp?

  146   // Print the header for the per-thread termination statistics.
  147   static void print_termination_stats_hdr(outputStream* const st = 
gclog_or_tty);
-------- New empty line :)
  148   // Print actual per-thread termination statistics.
  149   void print_termination_stats(outputStream* const st = gclog_or_tty) const;

Thanks,
/Jesper


Thomas Schatzl skrev den 5/8/15 13:59:
> Hi all,
>
>    can I have some reviews for the following change that cleans up and
> reformats the code and usage related to G1 termination stats?
>
> Stuff like not using braces around if-clauses, dropping an extra
> temporary lock, linebreaks in string constants, line breaks in the
> definition of the methods, superfluous additional parameters.
>
> I generally do not do reformatting changes, but every time I browse
> through that code it gives me an urge to fix that.
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8133043
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8133043/webrev/
> Testing:
> compilation, gcbasher run with -XX:+PrintTerminationStats, jprt
>
> Thanks,
>    Thomas
>


From jesper.wilhelmsson at oracle.com  Wed Aug  5 14:19:50 2015
From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson)
Date: Wed, 05 Aug 2015 16:19:50 +0200
Subject: RFR (XXS): 8133042: Remove some direct accesses of
	G1Allocator::reuse_retained_old_region() to G1CollectedHeap members
In-Reply-To: <1438775999.2378.32.camel@oracle.com>
References: <1438775999.2378.32.camel@oracle.com>
Message-ID: <55C21B86.6000406@oracle.com>

Looks good!
/Jesper

Thomas Schatzl skrev den 5/8/15 13:59:
> Hi all,
>
>    could I have reviews of this tiny change that changes some direct
> private member accesses of G1Allocator to G1CollectedHeap to some
> already available public method calls?
>
> While this is more a cosmetic change, I thought it would be good to
> decrease the amount of "friendliness" between those two classes. I could
> not find a good other CR to add them, so here we are....
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8133042
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8133042/webrev/
> Testing:
> local compilation
>
> Thanks,
>    Thomas
>


From thomas.schatzl at oracle.com  Wed Aug  5 14:20:52 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 05 Aug 2015 16:20:52 +0200
Subject: RFR (XXS): 8133042: Remove some direct accesses of
	G1Allocator::reuse_retained_old_region() to G1CollectedHeap members
In-Reply-To: <55C21B86.6000406@oracle.com>
References: <1438775999.2378.32.camel@oracle.com> <55C21B86.6000406@oracle.com>
Message-ID: <1438784452.2378.63.camel@oracle.com>

Hi Jesper,

On Wed, 2015-08-05 at 16:19 +0200, Jesper Wilhelmsson wrote:
> Looks good!
> /Jesper

  thanks for the review.

Thomas




From thomas.schatzl at oracle.com  Wed Aug  5 14:25:01 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 05 Aug 2015 16:25:01 +0200
Subject: RFR (XXS): 8133043: Clean up code related to termination stats
	printing
In-Reply-To: <55C21B26.1040800@oracle.com>
References: <1438775993.2378.31.camel@oracle.com> <55C21B26.1040800@oracle.com>
Message-ID: <1438784701.2378.64.camel@oracle.com>

Hi Jesper,


On Wed, 2015-08-05 at 16:18 +0200, Jesper Wilhelmsson wrote:
> Looks good! Thanks for cleaning this up!

  thanks for the review.

> 
> Do you mind adding an empty line between the method declarations in 
> g1ParScanThreadState.hpp?
> 
>   146   // Print the header for the per-thread termination statistics.
>   147   static void print_termination_stats_hdr(outputStream* const st = 
> gclog_or_tty);
> -------- New empty line :)
>   148   // Print actual per-thread termination statistics.
>   149   void print_termination_stats(outputStream* const st = gclog_or_tty) const;
> 

New webrevs at
http://cr.openjdk.java.net/~tschatzl/8133043/webrev.1 (full)
http://cr.openjdk.java.net/~tschatzl/8133043/webrev.0_to_1/ (diff)

Thanks,
  Thomas




From jesper.wilhelmsson at oracle.com  Wed Aug  5 14:26:00 2015
From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson)
Date: Wed, 05 Aug 2015 16:26:00 +0200
Subject: RFR (S): 8133047: Rename G1ParScanThreadState::_queue_num to
	_worker_id
In-Reply-To: <1438778415.2378.45.camel@oracle.com>
References: <1438778415.2378.45.camel@oracle.com>
Message-ID: <55C21CF8.5020908@oracle.com>

Looks good.

There is another use of queue_num in 
G1CollectedHeap::preserve_mark_during_evac_failure(). Was that one intentionally 
left?

Thanks,
/Jesper


Thomas Schatzl skrev den 5/8/15 14:40:
> Hi all,
>
>    can I have reviews for this small cleanup that renames the
> G1ParScanThreadState::_queue_num member to worker_id like it is called
> for all members that have the same functionality.
>
> G1ParScanThreadState::_queue_num is actually compared to a worker_id
> member of another class, and assigned to a _worker_id member somewhere
> else in the code.
>
> It hopefully makes the code slightly easier to read in the future by not
> using different names for the same thing throughput the code.
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8133047
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8133047/webrev/
> testing:
> jprt
>
> Thanks,
>    Thomas
>
>


From tom.benson at oracle.com  Wed Aug  5 14:29:23 2015
From: tom.benson at oracle.com (Tom Benson)
Date: Wed, 05 Aug 2015 10:29:23 -0400
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <1438781844.2378.60.camel@oracle.com>
References: <55C103A4.1060505@oracle.com>	
	<A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>
	<1438781844.2378.60.camel@oracle.com>
Message-ID: <55C21DC3.5090305@oracle.com>

Hi Thomas,
Thanks for reviewing.

On 8/5/2015 9:37 AM, Thomas Schatzl wrote:
> Hi,
>
> On Tue, 2015-08-04 at 18:06 -0400, Kim Barrett wrote:
>> But, is next_region_in_heap really the right stepper function?  It
>> skips over regions that are not "is_available".  Are the regions we're
>> dealing with in this function all guaranteed to be available?
> Yes. We assume that in the earlier call in allocate_containing_regions()
> G1 made them available (committed them).
>
> I assume that when mmap'ing it, the

missing end of sentence?   If you were going to say....  the mapping 
fails, so we need to free the archive regions that were just 
allocated..., then I agree.  8^)

>
>> Clearly, bad things happen if last_region is not available while using
>> next_region_in_heap here.
> One could do regular pointer arithmetic *within* the MemRegion (which is
> always a guaranteed contiguous range) and then map the address back to
> the HeapRegion*.

I don't think this was a suggestion...   was it?

>
> - I have some question about this code, particularly about the comment:
>
> 1137     HeapRegion* start_region = _hrm.addr_to_region(start_address);
> 1138     HeapRegion* last_region = _hrm.addr_to_region(last_address);
> 1139
> 1140     // Check for ranges that start in the same G1 region in which the previous
> 1141     // range ended, and adjust the start address so we don't try to free
> 1142     // the same region again. If the current range is entirely within that
> 1143     // region, skip it.
> 1144     if ((prev_last_region != NULL) && (start_region == prev_last_region)) {
> 1145       start_address = start_region->end();
> 1146       if (start_address > last_address) {
> 1147         continue;
> 1148       }
> 1149       start_region = _hrm.addr_to_region(start_address);
> 1150     }
>
> How could the situation mentioned in line 1140 happen? Are the given
> memory regions not overlapping already, and the start addresses of these
> MemRegions at the start of these regions?

As you said in a subsequent message, this can happen if the G1 region 
size at dump time is smaller than region size at restore time.
> Since last_region is the region containing the last address within the
> memory range, wouldn't that mean given above preconditions, this could
> not happen?
>
> - I think the method should add a assert_at_safepoint(true) at the top
> (and possibly all other archival methods if they are not yet through the
> call chain), or decrease_used() made safe against concurrent
> modification using the ParGC_Rare_Event_lock.
>
> I would prefer just making sure the code is only run at a safepoint.

These restore-time routines (alloc_/free_ archive regions) are called at 
the beginning of JVM init, not at a safepoint.  The dump-time archive 
alloc routines (such as archive_mem_allocate) do check for safepoint 
context.
Thanks,
Tom

>
> Thanks,
>    Thomas
>
>
>



From jesper.wilhelmsson at oracle.com  Wed Aug  5 14:31:57 2015
From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson)
Date: Wed, 05 Aug 2015 16:31:57 +0200
Subject: RFR (XXS): 8133043: Clean up code related to termination stats
	printing
In-Reply-To: <1438784701.2378.64.camel@oracle.com>
References: <1438775993.2378.31.camel@oracle.com>	
	<55C21B26.1040800@oracle.com> <1438784701.2378.64.camel@oracle.com>
Message-ID: <55C21E5D.8070605@oracle.com>

Ship it!
/Jesper

Thomas Schatzl skrev den 5/8/15 16:25:
> Hi Jesper,
>
>
> On Wed, 2015-08-05 at 16:18 +0200, Jesper Wilhelmsson wrote:
>> Looks good! Thanks for cleaning this up!
>
>    thanks for the review.
>
>>
>> Do you mind adding an empty line between the method declarations in
>> g1ParScanThreadState.hpp?
>>
>>    146   // Print the header for the per-thread termination statistics.
>>    147   static void print_termination_stats_hdr(outputStream* const st =
>> gclog_or_tty);
>> -------- New empty line :)
>>    148   // Print actual per-thread termination statistics.
>>    149   void print_termination_stats(outputStream* const st = gclog_or_tty) const;
>>
>
> New webrevs at
> http://cr.openjdk.java.net/~tschatzl/8133043/webrev.1 (full)
> http://cr.openjdk.java.net/~tschatzl/8133043/webrev.0_to_1/ (diff)
>
> Thanks,
>    Thomas
>
>


From tom.benson at oracle.com  Wed Aug  5 14:31:36 2015
From: tom.benson at oracle.com (Tom Benson)
Date: Wed, 05 Aug 2015 10:31:36 -0400
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <1438782757.2378.62.camel@oracle.com>
References: <55C103A4.1060505@oracle.com>	<A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>	<1438781844.2378.60.camel@oracle.com>
	<1438782757.2378.62.camel@oracle.com>
Message-ID: <55C21E48.5060602@oracle.com>

Hi again,
On 8/5/2015 9:52 AM, Thomas Schatzl wrote:
> Hi again,
>
> [...]
>> - I have some question about this code, particularly about the comment:
>>
>> 1137     HeapRegion* start_region = _hrm.addr_to_region(start_address);
>> 1138     HeapRegion* last_region = _hrm.addr_to_region(last_address);
>> 1139
>> 1140     // Check for ranges that start in the same G1 region in which the previous
>> 1141     // range ended, and adjust the start address so we don't try to free
>> 1142     // the same region again. If the current range is entirely within that
>> 1143     // region, skip it.
>> 1144     if ((prev_last_region != NULL) && (start_region == prev_last_region)) {
>> 1145       start_address = start_region->end();
>> 1146       if (start_address > last_address) {
>> 1147         continue;
>> 1148       }
>> 1149       start_region = _hrm.addr_to_region(start_address);
>> 1150     }
>>
>> How could the situation mentioned in line 1140 happen? Are the given
>> memory regions not overlapping already, and the start addresses of these
>> MemRegions at the start of these regions?
> Probably because of using the same memory mapped file created from a VM
> with different (smaller) heap region size?

Right.
Tom
>
> Thanks,
>    Thomas
>
>



From thomas.schatzl at oracle.com  Wed Aug  5 14:35:06 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 05 Aug 2015 16:35:06 +0200
Subject: RFR (S): 8133047: Rename G1ParScanThreadState::_queue_num to
	_worker_id
In-Reply-To: <55C21CF8.5020908@oracle.com>
References: <1438778415.2378.45.camel@oracle.com> <55C21CF8.5020908@oracle.com>
Message-ID: <1438785306.2378.69.camel@oracle.com>

Hi,

On Wed, 2015-08-05 at 16:26 +0200, Jesper Wilhelmsson wrote:
> Looks good.
> 
> There is another use of queue_num in 
> G1CollectedHeap::preserve_mark_during_evac_failure(). Was that one intentionally 
> left?
> 

Overlooked, the code was not in G1ParScanThreadState so I did not look.
I do think that again worker id is expected here.

New webrevs:
http://cr.openjdk.java.net/~tschatzl/8133047/webrev.1 (full)
http://cr.openjdk.java.net/~tschatzl/8133047/webrev.0_to_1 (diff)

Thanks,
  Thomas





From thomas.schatzl at oracle.com  Wed Aug  5 14:38:52 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 05 Aug 2015 16:38:52 +0200
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <55C21DC3.5090305@oracle.com>
References: <55C103A4.1060505@oracle.com>
	<A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>
	<1438781844.2378.60.camel@oracle.com> <55C21DC3.5090305@oracle.com>
Message-ID: <1438785532.2378.73.camel@oracle.com>

Hi,

On Wed, 2015-08-05 at 10:29 -0400, Tom Benson wrote:
> Hi Thomas,
> Thanks for reviewing.
> 
> On 8/5/2015 9:37 AM, Thomas Schatzl wrote:
> > Hi,
> >
> > On Tue, 2015-08-04 at 18:06 -0400, Kim Barrett wrote:
> >> But, is next_region_in_heap really the right stepper function?  It
> >> skips over regions that are not "is_available".  Are the regions we're
> >> dealing with in this function all guaranteed to be available?
> > Yes. We assume that in the earlier call in allocate_containing_regions()
> > G1 made them available (committed them).
> >
> > I assume that when mmap'ing it, the
> 
> missing end of sentence?   If you were going to say....  the mapping 
> fails, so we need to free the archive regions that were just 
> allocated..., then I agree.  8^)

  committed memory will be automatically released.

> 
> >
> >> Clearly, bad things happen if last_region is not available while using
> >> next_region_in_heap here.
> > One could do regular pointer arithmetic *within* the MemRegion (which is
> > always a guaranteed contiguous range) and then map the address back to
> > the HeapRegion*.
> 
> I don't think this was a suggestion...   was it?
> 

An option, maybe Kim likes that more.

> > Since last_region is the region containing the last address within the
> > memory range, wouldn't that mean given above preconditions, this could
> > not happen?
> >
> > - I think the method should add a assert_at_safepoint(true) at the top
> > (and possibly all other archival methods if they are not yet through the
> > call chain), or decrease_used() made safe against concurrent
> > modification using the ParGC_Rare_Event_lock.
> >
> > I would prefer just making sure the code is only run at a safepoint.
> 
> These restore-time routines (alloc_/free_ archive regions) are called at 
> the beginning of JVM init, not at a safepoint.  The dump-time archive 
> alloc routines (such as archive_mem_allocate) do check for safepoint 
> context.

Sigh. Is there a way to check this reliably?

As mentioned, at least the decrement_used() call is not guarded against
concurrent modification. It is probably easier for now to guard against
using these methods at the wrong time (in case somebody wants to try
them if some other requirement comes up) instead of trying to make them
MT safe.

Thanks,
  Thomas




From jesper.wilhelmsson at oracle.com  Wed Aug  5 14:39:31 2015
From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson)
Date: Wed, 05 Aug 2015 16:39:31 +0200
Subject: RFR (S): 8133047: Rename G1ParScanThreadState::_queue_num to
	_worker_id
In-Reply-To: <1438785306.2378.69.camel@oracle.com>
References: <1438778415.2378.45.camel@oracle.com>	
	<55C21CF8.5020908@oracle.com> <1438785306.2378.69.camel@oracle.com>
Message-ID: <55C22023.40208@oracle.com>

Looks good!
/Jesper

Thomas Schatzl skrev den 5/8/15 16:35:
> Hi,
>
> On Wed, 2015-08-05 at 16:26 +0200, Jesper Wilhelmsson wrote:
>> Looks good.
>>
>> There is another use of queue_num in
>> G1CollectedHeap::preserve_mark_during_evac_failure(). Was that one intentionally
>> left?
>>
>
> Overlooked, the code was not in G1ParScanThreadState so I did not look.
> I do think that again worker id is expected here.
>
> New webrevs:
> http://cr.openjdk.java.net/~tschatzl/8133047/webrev.1 (full)
> http://cr.openjdk.java.net/~tschatzl/8133047/webrev.0_to_1 (diff)
>
> Thanks,
>    Thomas
>
>
>


From jon.masamitsu at oracle.com  Wed Aug  5 15:52:02 2015
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Wed, 05 Aug 2015 08:52:02 -0700
Subject: RFR (XXS): 8133043: Clean up code related to termination stats
	printing
In-Reply-To: <1438775993.2378.31.camel@oracle.com>
References: <1438775993.2378.31.camel@oracle.com>
Message-ID: <55C23122.9030901@oracle.com>

Thomas,

You substituted _queue_num for i (worker_id) in

http://cr.openjdk.java.net/~tschatzl/8133043/webrev/src/share/vm/gc/g1/g1ParScanThreadState.cpp.udiff.html

-               i, elapsed_ms, s_roots_ms, s_roots_ms * 100 / elapsed_ms,
+               _queue_num, elapsed_ms, s_roots_ms, s_roots_ms * 100 / elapsed_ms,


I can see how _queue_num would be more interesting  but is worker_id just
of no value so it is not worth printing?   I don't have any perspective 
on this
so I'm not objecting to dropping the worker_id.    Just wondering if 
printing
less is better.  And you can push it as is.

Rest looks good.

Reviewed.

Jon


On 8/5/2015 4:59 AM, Thomas Schatzl wrote:
> Hi all,
>
>    can I have some reviews for the following change that cleans up and
> reformats the code and usage related to G1 termination stats?
>
> Stuff like not using braces around if-clauses, dropping an extra
> temporary lock, linebreaks in string constants, line breaks in the
> definition of the methods, superfluous additional parameters.
>
> I generally do not do reformatting changes, but every time I browse
> through that code it gives me an urge to fix that.
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8133043
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8133043/webrev/
> Testing:
> compilation, gcbasher run with -XX:+PrintTerminationStats, jprt
>
> Thanks,
>    Thomas
>



From thomas.schatzl at oracle.com  Wed Aug  5 15:55:13 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 05 Aug 2015 17:55:13 +0200
Subject: RFR (XXS): 8133043: Clean up code related to termination stats
	printing
In-Reply-To: <55C23122.9030901@oracle.com>
References: <1438775993.2378.31.camel@oracle.com> <55C23122.9030901@oracle.com>
Message-ID: <1438790113.2378.83.camel@oracle.com>

Hi Jon,

On Wed, 2015-08-05 at 08:52 -0700, Jon Masamitsu wrote:
> Thomas,
> 
> You substituted _queue_num for i (worker_id) in
> 
> http://cr.openjdk.java.net/~tschatzl/8133043/webrev/src/share/vm/gc/g1/g1ParScanThreadState.cpp.udiff.html
> 
> -               i, elapsed_ms, s_roots_ms, s_roots_ms * 100 / elapsed_ms,
> +               _queue_num, elapsed_ms, s_roots_ms, s_roots_ms * 100 / elapsed_ms,
> 
> 
> I can see how _queue_num would be more interesting  but is worker_id just
> of no value so it is not worth printing?   I don't have any perspective 
> on this
> so I'm not objecting to dropping the worker_id.    Just wondering if 
> printing
> less is better.  And you can push it as is.

The original code passed worker_id as "i" here, however, _queue_num ==
worker_id.

If you look at g1CollectedHeap.cpp line 4514, the code passes worker_id
to the ParScanThreadState, that is _queue_num.

A later patch also for review changes _queue_num to _worker_id, and then
it becomes clear again.

> 
> Rest looks good.
> 
> Reviewed.

Thanks for the review,
  Thomas




From joseph.provino at oracle.com  Wed Aug  5 16:28:06 2015
From: joseph.provino at oracle.com (Joseph Provino)
Date: Wed, 05 Aug 2015 12:28:06 -0400
Subject: RFR (XXS): JDK-8085983
	G1CollectedHeap::collection_set_iterate_from()
	has unused code and can be simplified
In-Reply-To: <5582DB1A.6070609@oracle.com>
References: <5575EA21.4010803@oracle.com> <557869E2.4080905@oracle.com>
	<5582DB1A.6070609@oracle.com>
Message-ID: <55C23996.3020406@oracle.com>

Latest webrev is here: 
http://cr.openjdk.java.net/~jprovino/8085983/webrev.01

joe

On 6/18/2015 10:52 AM, Joseph Provino wrote:
>
> On 6/10/2015 12:46 PM, Stefan Karlsson wrote:
>> Hi Joe,
>>
>> On 2015-06-08 21:16, Joseph Provino wrote:
>>> One file.
>>>
>>> CR: https://bugs.openjdk.java.net/browse/JDK-8085983
>>>
>>> webrev: http://cr.openjdk.java.net/~jprovino/8085983/webrev.00
>>
>> Some comments:
>>
>> *+      guarantee(!cl->doHeapRegion(cur), err_msg("doHeapRegion() returned true"));*
>>
>> - You don't need err_msg if you don't have any format specifiers in 
>> the error message.
>>
> I'll fix that.
>
>> - I would prefer if you didn't put the cl->doHeapReagion() calls 
>> inside guarantees. It's too easy to incorrectly assume that the code 
>> in the guarantee is only verification code, IMHO.
>>
> That's a good point.  Something like this:
>
>     bool ret = cl->doHeapRegion(r);
>     guarantee(ret == false, "doHeapRegion() returned true");
>
> Or do you prefer !ret?
>
>> - Shouldn't this be an assert instead of a guarantee?
> I don't know the rules there.  Seems like assert would be better
> so it doesn't slow down the product build.
>
> thanks.
>
> joe
>
>>
>> Thanks,
>> StefanK
>>
>>>
>>> passes jprt.
>>>
>>> thanks.
>>>
>>> joe
>>>
>>>
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150805/3e535309/attachment.htm>

From kim.barrett at oracle.com  Wed Aug  5 16:29:51 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Wed, 5 Aug 2015 12:29:51 -0400
Subject: RFR (M): 8073052: Rename and clean up the allocation manager
	hierarchy in g1Allocator.?pp
In-Reply-To: <1438762409.2378.2.camel@oracle.com>
References: <1438073174.2272.17.camel@oracle.com> <55B778A4.4050602@oracle.com>
	<1438156281.2302.16.camel@oracle.com>
	<ACF8A0BA-3821-4A49-BB4A-6D87AF033FE9@oracle.com>
	<1438674539.2146.5.camel@oracle.com>
	<6D1848C8-9A54-43A0-A7EF-2B079A40D126@oracle.com>
	<1438762409.2378.2.camel@oracle.com>
Message-ID: <42F42137-F7A1-45B4-9528-FAA2458D356C@oracle.com>

On Aug 5, 2015, at 4:13 AM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
> 
>  new webrev:
> http://cr.openjdk.java.net/~tschatzl/8073052/webrev.5
> http://cr.openjdk.java.net/~tschatzl/8073052/webrev.4_to_5

Looks good.



From jon.masamitsu at oracle.com  Wed Aug  5 16:36:19 2015
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Wed, 05 Aug 2015 09:36:19 -0700
Subject: RFR (S): 8003237: G1: Reduce unnecessary (and failing) allocation
	attempts when handling an evacuation failure
In-Reply-To: <1438763108.2378.9.camel@oracle.com>
References: <1438605320.2292.47.camel@oracle.com>	<1438605827.2292.49.camel@oracle.com>
	<1438763108.2378.9.camel@oracle.com>
Message-ID: <55C23B83.90405@oracle.com>

Thomas,

I think that this method gets called if a new region is allocated
via allocate_inline_or_new_plab()

"src/share/vm/gc/g1/g1CollectedHeap.cpp"

6436 HeapRegion* G1CollectedHeap::new_gc_alloc_region(size_t word_size,
6437                                                  uint count,
6438                                                  InCSetState dest) {
6439   assert(FreeList_lock->owned_by_self(), "pre-condition");
6440
6441   if (count < g1_policy()->max_regions(dest)) {

Might the g1_policy()->max_regions() conflict with the test
in has_more_free_regions() which considers all free regions
(even uncommitted).   That is, has_more_free_regions() will
allow allocation attempts that will not success because of
max_regions().

http://cr.openjdk.java.net/~tschatzl/8003237/webrev/src/share/vm/gc/g1/g1ParScanThreadState.hpp.frames.html

>    74   // Indicates whether in the last generation (old) there is no more space
>    75   // available for allocation.
>    76   bool _no_more_space_in_old;

Would a name like _old_is_full suit you here in place of 
_no_more_space_in_old?  I like positives instead
negatives.

Jon

On 8/5/2015 1:25 AM, Thomas Schatzl wrote:
> Hi all,
>
>    just fyi, I had to update the webrev because of changes in the 8073052
> patch.
> Since there were no reviews of these changes yet, I updated in-place.
>
> There are no functional changes.
>
> Also, if somebody already had a look at the changes, and the question
> why G1ParScanThreadState::allocate_in_next_plab() only uses failed PLAB
> allocation as indicator that heap is full came up, the later patch
> JDK-8067336 "Allow that PLAB allocations at the end of regions are
> flexible" will allow that PLABs can get quite small as they are sized
> flexibly.
>
> I wanted to avoid introducing some even more arbitrary heuristic to
> determine when the heap is full now which will only a bit later be
> superseded again anyway.
>
> In any case current G1ParScanThreadState::allocate_in_next_plab() should
> try harder than the previous one by always trying an inline allocation
> even if it cannot allocate the object (supposedly smaller than a PLAB)
> using a new PLAB.
>
> Thanks,
>    Thomas
>
> On Mon, 2015-08-03 at 14:43 +0200, Thomas Schatzl wrote:
>> Hi,
>>
>>    this change is based on the latest version of the change for 8073052
>> that is out for review, and is still longing for another reviewer :)
>>
>> (I am okay with pushing it with only a single "R"eviewer since it is
>> only refactoring, but if somebody can find the time...)
>>
>> Thanks,
>>    Thomas
>>
>> On Mon, 2015-08-03 at 14:35 +0200, Thomas Schatzl wrote:
>>> Hi all,
>>>
>>>    can I have reviews for this small change that improves evacuation
>>> failure handling by short-cutting copy_to_survivor_space() when there is
>>> no more space left?
>>>
>>> In effect this makes the parallel phase during GC have the same
>>> performance as if there is no evacuation failure, and I could not
>>> measure regressions in the normal case either.
>>>
>>> The change implements two things: first, notice when there is no more
>>> space left in the last gen (i.e. old), and if so, short-cut
>>> ParScanThreadState::copy_to_survivor_space(). The other change is to not
>>> try to ask the free list for new regions if it's empty to avoid taking
>>> the lock in the time window between the first few evac failures and when
>>> space is completely empty.
>>>
>>> There is still the somewhat heavyweight cleanup phases after evacuation
>>> failure.
>>>
>>> CR:
>>> https://bugs.openjdk.java.net/browse/JDK-8003237
>>> Webrev:
>>> http://cr.openjdk.java.net/~tschatzl/8003237/webrev/
>>> Testing:
>>> jprt, lots of performance measurements with and without this change
>>> compiled in
>>>
>>> Thanks,
>>>    Thomas
>>>
>>>
>>
>



From jon.masamitsu at oracle.com  Wed Aug  5 16:39:51 2015
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Wed, 05 Aug 2015 09:39:51 -0700
Subject: RFR (XXS): 8133042: Remove some direct accesses of
	G1Allocator::reuse_retained_old_region() to G1CollectedHeap members
In-Reply-To: <1438775999.2378.32.camel@oracle.com>
References: <1438775999.2378.32.camel@oracle.com>
Message-ID: <55C23C57.6000002@oracle.com>

Looks good.

Jon


On 8/5/2015 4:59 AM, Thomas Schatzl wrote:
> Hi all,
>
>    could I have reviews of this tiny change that changes some direct
> private member accesses of G1Allocator to G1CollectedHeap to some
> already available public method calls?
>
> While this is more a cosmetic change, I thought it would be good to
> decrease the amount of "friendliness" between those two classes. I could
> not find a good other CR to add them, so here we are....
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8133042
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8133042/webrev/
> Testing:
> local compilation
>
> Thanks,
>    Thomas
>



From jon.masamitsu at oracle.com  Wed Aug  5 16:42:02 2015
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Wed, 05 Aug 2015 09:42:02 -0700
Subject: RFR (XXS): 8133043: Clean up code related to termination stats
	printing
In-Reply-To: <1438790113.2378.83.camel@oracle.com>
References: <1438775993.2378.31.camel@oracle.com>	
	<55C23122.9030901@oracle.com> <1438790113.2378.83.camel@oracle.com>
Message-ID: <55C23CD9.8050606@oracle.com>



On 8/5/2015 8:55 AM, Thomas Schatzl wrote:
> Hi Jon,
>
> On Wed, 2015-08-05 at 08:52 -0700, Jon Masamitsu wrote:
>> Thomas,
>>
>> You substituted _queue_num for i (worker_id) in
>>
>> http://cr.openjdk.java.net/~tschatzl/8133043/webrev/src/share/vm/gc/g1/g1ParScanThreadState.cpp.udiff.html
>>
>> -               i, elapsed_ms, s_roots_ms, s_roots_ms * 100 / elapsed_ms,
>> +               _queue_num, elapsed_ms, s_roots_ms, s_roots_ms * 100 / elapsed_ms,
>>
>>
>> I can see how _queue_num would be more interesting  but is worker_id just
>> of no value so it is not worth printing?   I don't have any perspective
>> on this
>> so I'm not objecting to dropping the worker_id.    Just wondering if
>> printing
>> less is better.  And you can push it as is.
> The original code passed worker_id as "i" here, however, _queue_num ==
> worker_id.
>
> If you look at g1CollectedHeap.cpp line 4514, the code passes worker_id
> to the ParScanThreadState, that is _queue_num.
>
> A later patch also for review changes _queue_num to _worker_id, and then
> it becomes clear again.
Yes.  Thanks.

Jon

>
>> Rest looks good.
>>
>> Reviewed.
> Thanks for the review,
>    Thomas
>
>



From kim.barrett at oracle.com  Wed Aug  5 16:54:12 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Wed, 5 Aug 2015 12:54:12 -0400
Subject: RFR (XXS): 8133042: Remove some direct accesses of
	G1Allocator::reuse_retained_old_region() to G1CollectedHeap members
In-Reply-To: <1438775999.2378.32.camel@oracle.com>
References: <1438775999.2378.32.camel@oracle.com>
Message-ID: <FD29FE78-EB89-4630-A5E9-16DECC9B8846@oracle.com>

On Aug 5, 2015, at 7:59 AM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
> 
> Hi all,
> 
>  could I have reviews of this tiny change that changes some direct
> private member accesses of G1Allocator to G1CollectedHeap to some
> already available public method calls?
> 
> While this is more a cosmetic change, I thought it would be good to
> decrease the amount of "friendliness" between those two classes. I could
> not find a good other CR to add them, so here we are....
> 
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8133042
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8133042/webrev/
> Testing:
> local compilation

There is very similar code to what is being changed here in G1ArchiveAllocator::alloc_new_region,
later in the same file (g1Allocator.cpp).

Would the proposed change or the additional change to alloc_new_region permit the elimination
of any friend declarations from G1CollectedHeap?



From jon.masamitsu at oracle.com  Wed Aug  5 17:03:06 2015
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Wed, 05 Aug 2015 10:03:06 -0700
Subject: RFR (S): 8133047: Rename G1ParScanThreadState::_queue_num to
	_worker_id
In-Reply-To: <1438785306.2378.69.camel@oracle.com>
References: <1438778415.2378.45.camel@oracle.com> <55C21CF8.5020908@oracle.com>
	<1438785306.2378.69.camel@oracle.com>
Message-ID: <55C241CA.5030705@oracle.com>

http://cr.openjdk.java.net/~tschatzl/8133047/webrev.1/src/share/vm/gc/g1/g1CollectedHeap.cpp.udiff.html

-  assert(_worker_id == _par_scan_state->queue_num(), "sanity");
+  assert(_worker_id == _par_scan_state->worker_id(), "sanity");



Kind of makes you wonder why G1ParClosureSuper needs its own _worker_id
when it has _par_scan_state.  Or is removal of _worker_id from 
G1ParClosureSuper
a change that coming soon?  If not, I'll file a CR.

class G1ParClosureSuper : public OopsInHeapRegionClosure {
protected:
   G1CollectedHeap* _g1;
   G1ParScanThreadState* _par_scan_state;
   uint _worker_id;

Looks good.

Jon




On 8/5/2015 7:35 AM, Thomas Schatzl wrote:
> Hi,
>
> On Wed, 2015-08-05 at 16:26 +0200, Jesper Wilhelmsson wrote:
>> Looks good.
>>
>> There is another use of queue_num in
>> G1CollectedHeap::preserve_mark_during_evac_failure(). Was that one intentionally
>> left?
>>
> Overlooked, the code was not in G1ParScanThreadState so I did not look.
> I do think that again worker id is expected here.
>
> New webrevs:
> http://cr.openjdk.java.net/~tschatzl/8133047/webrev.1 (full)
> http://cr.openjdk.java.net/~tschatzl/8133047/webrev.0_to_1 (diff)
>
> Thanks,
>    Thomas
>
>
>



From tom.benson at oracle.com  Wed Aug  5 17:33:51 2015
From: tom.benson at oracle.com (Tom Benson)
Date: Wed, 05 Aug 2015 13:33:51 -0400
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <1438785532.2378.73.camel@oracle.com>
References: <55C103A4.1060505@oracle.com>		
	<A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>	
	<1438781844.2378.60.camel@oracle.com> <55C21DC3.5090305@oracle.com>
	<1438785532.2378.73.camel@oracle.com>
Message-ID: <55C248FF.6010308@oracle.com>

Hi,

On 8/5/2015 10:38 AM, Thomas Schatzl wrote:
> Hi,
>
> On Wed, 2015-08-05 at 10:29 -0400, Tom Benson wrote:
>
>> These restore-time routines (alloc_/free_ archive regions) are called at
>> the beginning of JVM init, not at a safepoint.  The dump-time archive
>> alloc routines (such as archive_mem_allocate) do check for safepoint
>> context.
> Sigh. Is there a way to check this reliably?
I notice that cms/compactibleFreeListSpace.cpp contains the assertion:

   assert(SafepointSynchronize::is_at_safepoint() || !is_init_completed(),
          "Else races are possible");

... so perhaps that's a viable option.  I'll try it to ensure the CDS 
use does indeed occur while is_init_completed() is false.
Tom

>
> As mentioned, at least the decrement_used() call is not guarded against
> concurrent modification. It is probably easier for now to guard against
> using these methods at the wrong time (in case somebody wants to try
> them if some other requirement comes up) instead of trying to make them
> MT safe.
>
> Thanks,
>    Thomas
>
>



From jiangli.zhou at oracle.com  Wed Aug  5 18:05:18 2015
From: jiangli.zhou at oracle.com (Jiangli Zhou)
Date: Wed, 5 Aug 2015 11:05:18 -0700
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <55C248FF.6010308@oracle.com>
References: <55C103A4.1060505@oracle.com>		
	<A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>	
	<1438781844.2378.60.camel@oracle.com> <55C21DC3.5090305@oracle.com>
	<1438785532.2378.73.camel@oracle.com> <55C248FF.6010308@oracle.com>
Message-ID: <F73C4B9A-4DBA-47A8-8715-0988FF6D393D@oracle.com>

Hi Tom,

On Aug 5, 2015, at 10:33 AM, Tom Benson <tom.benson at oracle.com> wrote:

> Hi,
> 
> On 8/5/2015 10:38 AM, Thomas Schatzl wrote:
>> Hi,
>> 
>> On Wed, 2015-08-05 at 10:29 -0400, Tom Benson wrote:
>> 
>>> These restore-time routines (alloc_/free_ archive regions) are called at
>>> the beginning of JVM init, not at a safepoint.  The dump-time archive
>>> alloc routines (such as archive_mem_allocate) do check for safepoint
>>> context.
>> Sigh. Is there a way to check this reliably?
> I notice that cms/compactibleFreeListSpace.cpp contains the assertion:
> 
>  assert(SafepointSynchronize::is_at_safepoint() || !is_init_completed(),
>         "Else races are possible");
> 
> ... so perhaps that's a viable option.  I'll try it to ensure the CDS use does indeed occur while is_init_completed() is false.

The mapping and initialization of the CDS string data happens during VM initialization and before 'set_init_completed()? is called. Just verified that in gdb as well.

Thanks,
Jiangli

> Tom
> 
>> 
>> As mentioned, at least the decrement_used() call is not guarded against
>> concurrent modification. It is probably easier for now to guard against
>> using these methods at the wrong time (in case somebody wants to try
>> them if some other requirement comes up) instead of trying to make them
>> MT safe.
>> 
>> Thanks,
>>   Thomas
>> 
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150805/6a40ca2c/attachment.htm>

From tom.benson at oracle.com  Wed Aug  5 19:21:45 2015
From: tom.benson at oracle.com (Tom Benson)
Date: Wed, 05 Aug 2015 15:21:45 -0400
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <F73C4B9A-4DBA-47A8-8715-0988FF6D393D@oracle.com>
References: <55C103A4.1060505@oracle.com>		
	<A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>	
	<1438781844.2378.60.camel@oracle.com> <55C21DC3.5090305@oracle.com>
	<1438785532.2378.73.camel@oracle.com> <55C248FF.6010308@oracle.com>
	<F73C4B9A-4DBA-47A8-8715-0988FF6D393D@oracle.com>
Message-ID: <55C26249.1090902@oracle.com>

Hi,

Updated full and incremental webrevs are at:
   http://cr.openjdk.java.net/~tbenson/8131734/webrev.01/
   http://cr.openjdk.java.net/~tbenson/8131734/webrev.01.vs.00/

On 8/5/2015 2:05 PM, Jiangli Zhou wrote:
> Hi Tom,
>
> On Aug 5, 2015, at 10:33 AM, Tom Benson <tom.benson at oracle.com 
> <mailto:tom.benson at oracle.com>> wrote:
>
>> ... so perhaps that's a viable option.  I'll try it to ensure the CDS 
>> use does indeed occur while is_init_completed() is false.
>
> The mapping and initialization of the CDS string data happens during 
> VM initialization and before 'set_init_completed()? is called. Just 
> verified that in gdb as well.
>
> Thanks,
> Jiangli
>

Yes, I checked as well.  I've added this to each of 
alloc_/fill_/free_archive_regions:

    assert(!is_init_completed(), "Expect to be called at JVM init time");

Tom
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150805/8b8670d3/attachment.htm>

From kim.barrett at oracle.com  Wed Aug  5 19:27:04 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Wed, 5 Aug 2015 15:27:04 -0400
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <55C26249.1090902@oracle.com>
References: <55C103A4.1060505@oracle.com>		
	<A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>	
	<1438781844.2378.60.camel@oracle.com> <55C21DC3.5090305@oracle.com>
	<1438785532.2378.73.camel@oracle.com> <55C248FF.6010308@oracle.com>
	<F73C4B9A-4DBA-47A8-8715-0988FF6D393D@oracle.com>
	<55C26249.1090902@oracle.com>
Message-ID: <C149705F-696B-46CD-8FB8-CAD06C511244@oracle.com>

On Aug 5, 2015, at 3:21 PM, Tom Benson <tom.benson at oracle.com> wrote:
> 
> Hi,
> 
> Updated full and incremental webrevs are at:
>   http://cr.openjdk.java.net/~tbenson/8131734/webrev.01/
>   http://cr.openjdk.java.net/~tbenson/8131734/webrev.01.vs.00/

Looks good.



From tom.benson at oracle.com  Wed Aug  5 20:42:03 2015
From: tom.benson at oracle.com (Tom Benson)
Date: Wed, 05 Aug 2015 16:42:03 -0400
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <C149705F-696B-46CD-8FB8-CAD06C511244@oracle.com>
References: <55C103A4.1060505@oracle.com>		
	<A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>	
	<1438781844.2378.60.camel@oracle.com> <55C21DC3.5090305@oracle.com>
	<1438785532.2378.73.camel@oracle.com> <55C248FF.6010308@oracle.com>
	<F73C4B9A-4DBA-47A8-8715-0988FF6D393D@oracle.com>
	<55C26249.1090902@oracle.com>
	<C149705F-696B-46CD-8FB8-CAD06C511244@oracle.com>
Message-ID: <55C2751B.8070400@oracle.com>

Thanks, Kim.
Tom

On 8/5/2015 3:27 PM, Kim Barrett wrote:
> On Aug 5, 2015, at 3:21 PM, Tom Benson <tom.benson at oracle.com> wrote:
>> Hi,
>>
>> Updated full and incremental webrevs are at:
>>    http://cr.openjdk.java.net/~tbenson/8131734/webrev.01/
>>    http://cr.openjdk.java.net/~tbenson/8131734/webrev.01.vs.00/
> Looks good.
>



From thomas.schatzl at oracle.com  Thu Aug  6 07:48:49 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 06 Aug 2015 09:48:49 +0200
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <55C2751B.8070400@oracle.com>
References: <55C103A4.1060505@oracle.com>
	<A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>
	<1438781844.2378.60.camel@oracle.com> <55C21DC3.5090305@oracle.com>
	<1438785532.2378.73.camel@oracle.com> <55C248FF.6010308@oracle.com>
	<F73C4B9A-4DBA-47A8-8715-0988FF6D393D@oracle.com>
	<55C26249.1090902@oracle.com>
	<C149705F-696B-46CD-8FB8-CAD06C511244@oracle.com>
	<55C2751B.8070400@oracle.com>
Message-ID: <1438847329.2009.9.camel@oracle.com>

Hi,

On Wed, 2015-08-05 at 16:42 -0400, Tom Benson wrote:
> Thanks, Kim.
> Tom
> 
> On 8/5/2015 3:27 PM, Kim Barrett wrote:
> > On Aug 5, 2015, at 3:21 PM, Tom Benson <tom.benson at oracle.com> wrote:
> >> Hi,
> >>
> >> Updated full and incremental webrevs are at:
> >>    http://cr.openjdk.java.net/~tbenson/8131734/webrev.01/
> >>    http://cr.openjdk.java.net/~tbenson/8131734/webrev.01.vs.00/
> > Looks good.
> >

Looking again at the code starting from line 1144 with the comment,
there may still be an issue here: consider a VM A having 8M regions 1,
2, and 3, each spanning 8M (see figure below; each letter represents 1M
of space), and a mapping from a VM B with 2M region size with the
following layout that is loaded into VM A and failed.

    1        2       3       region#
AAAAAAAA AAAAAAAA AAAAAAAA   VM A regions (8M region size)
  BB  BB BBBBBBBB BBBB       VM B mapping (2M region size)

E.g. the mapping is from 2-3M, and another mapping from 6-20M. E.g. the
ranges array contains

  (2, 3), (6, 20)

now, after freeing region 1 from the first mapping (2, 3), start_region
of the second mapping (=1) equals prev_last_region (=1); now the cursor
is advanced to the region of the end of the second mapping (=3),
forgetting to free region 2.

What am I missing here?

Thanks for adding the initialization checks.

Thanks,
  Thomas





From thomas.schatzl at oracle.com  Thu Aug  6 07:59:36 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 06 Aug 2015 09:59:36 +0200
Subject: RFR (S): 8133047: Rename G1ParScanThreadState::_queue_num to
	_worker_id
In-Reply-To: <55C241CA.5030705@oracle.com>
References: <1438778415.2378.45.camel@oracle.com>
	<55C21CF8.5020908@oracle.com> <1438785306.2378.69.camel@oracle.com>
	<55C241CA.5030705@oracle.com>
Message-ID: <1438847976.2009.12.camel@oracle.com>

Hi Jon,

On Wed, 2015-08-05 at 10:03 -0700, Jon Masamitsu wrote:
> http://cr.openjdk.java.net/~tschatzl/8133047/webrev.1/src/share/vm/gc/g1/g1CollectedHeap.cpp.udiff.html
> 
> -  assert(_worker_id == _par_scan_state->queue_num(), "sanity");
> +  assert(_worker_id == _par_scan_state->worker_id(), "sanity");
> 
> 
> 
> Kind of makes you wonder why G1ParClosureSuper needs its own _worker_id
> when it has _par_scan_state.  Or is removal of _worker_id from 
> G1ParClosureSuper
> a change that coming soon?  If not, I'll file a CR.

No. The renaming was mostly because I have been working in the
G1ParScanState/G1Allocator area and noticed these issues, I did not
perform some sort of general cleanup.

Also, this change is not merely a renaming, but changes code, so would
imo need more scrutiny regarding performance impact (I am guessing the
extra _worker_id is because of this reason).

I filed JDK-8133096.

> class G1ParClosureSuper : public OopsInHeapRegionClosure {
> protected:
>    G1CollectedHeap* _g1;
>    G1ParScanThreadState* _par_scan_state;
>    uint _worker_id;
> 
> Looks good.

Thanks,
  Thomas




From thomas.schatzl at oracle.com  Thu Aug  6 08:10:37 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 06 Aug 2015 10:10:37 +0200
Subject: RFR (M): 8073052: Rename and clean up the allocation manager
	hierarchy in g1Allocator.?pp
In-Reply-To: <42F42137-F7A1-45B4-9528-FAA2458D356C@oracle.com>
References: <1438073174.2272.17.camel@oracle.com>
	<55B778A4.4050602@oracle.com> <1438156281.2302.16.camel@oracle.com>
	<ACF8A0BA-3821-4A49-BB4A-6D87AF033FE9@oracle.com>
	<1438674539.2146.5.camel@oracle.com>
	<6D1848C8-9A54-43A0-A7EF-2B079A40D126@oracle.com>
	<1438762409.2378.2.camel@oracle.com>
	<42F42137-F7A1-45B4-9528-FAA2458D356C@oracle.com>
Message-ID: <1438848637.2009.14.camel@oracle.com>

Hi,

On Wed, 2015-08-05 at 12:29 -0400, Kim Barrett wrote:
> On Aug 5, 2015, at 4:13 AM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
> > 
> >  new webrev:
> > http://cr.openjdk.java.net/~tschatzl/8073052/webrev.5
> > http://cr.openjdk.java.net/~tschatzl/8073052/webrev.4_to_5
> 
> Looks good.
> 

  thanks for the review Kim.

  Thomas



From thomas.schatzl at oracle.com  Thu Aug  6 08:15:59 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 06 Aug 2015 10:15:59 +0200
Subject: RFR (XXS): JDK-8085983
	G1CollectedHeap::collection_set_iterate_from() has unused code and can
	be simplified
In-Reply-To: <55C23996.3020406@oracle.com>
References: <5575EA21.4010803@oracle.com> <557869E2.4080905@oracle.com>
	<5582DB1A.6070609@oracle.com> <55C23996.3020406@oracle.com>
Message-ID: <1438848959.2009.19.camel@oracle.com>

Hi Joe,

On Wed, 2015-08-05 at 12:28 -0400, Joseph Provino wrote:
> Latest webrev is here:
> http://cr.openjdk.java.net/~jprovino/8085983/webrev.01

I assume this is the latest version of the change; see below for
comments.
> 
> joe
> 
> On 6/18/2015 10:52 AM, Joseph Provino wrote:
> 
> > 
> > On 6/10/2015 12:46 PM, Stefan Karlsson wrote:
> > 
> > > Hi Joe,
> > > 
> > > On 2015-06-08 21:16, Joseph Provino wrote:
> > > 
> > > > One file. 
> > > > 
> > > > CR:  https://bugs.openjdk.java.net/browse/JDK-8085983 
> > > > 
> > > > webrev:  http://cr.openjdk.java.net/~jprovino/8085983/webrev.00 
> > > 
> > > Some comments:
> > > 
> > > +     guarantee(!cl->doHeapRegion(cur), err_msg("doHeapRegion() returned true"));
> > > 
> > > - You don't need err_msg if you don't have any format specifiers
> > > in the error message.
> > > 
> > I'll fix that.
> > 
> > > - I would prefer if you didn't put the cl->doHeapReagion() calls
> > > inside guarantees. It's too easy to incorrectly assume that the
> > > code in the guarantee is only verification code, IMHO.

That should also be true for the other guarantees imho, not only the one
in G1CollectedHeap::collection_set_iterate().
> > > 
> > That's a good point.  Something like this:
> > 
> >     bool ret = cl->doHeapRegion(r);
> >     guarantee(ret == false, "doHeapRegion() returned true");
> > 
> > Or do you prefer !ret?

We do not compare to true/false directly, i.e. !ret is the choice here.
> > 
> > > - Shouldn't this be an assert instead of a guarantee?
> > I don't know the rules there.  Seems like assert would be better
> > so it doesn't slow down the product build.

Either is fine with me. Since this is work distribution on a per-region
basis, so an additional check very likely is unnoticeable.

Thanks,
  Thomas




From thomas.schatzl at oracle.com  Thu Aug  6 08:45:36 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 06 Aug 2015 10:45:36 +0200
Subject: RFR (M): 8073052: Rename and clean up the allocation manager
	hierarchy in g1Allocator.?pp
In-Reply-To: <55C104DF.3060403@oracle.com>
References: <1438073174.2272.17.camel@oracle.com>
	<55B778A4.4050602@oracle.com> <1438156281.2302.16.camel@oracle.com>
	<55B89724.7030200@oracle.com> <55C104DF.3060403@oracle.com>
Message-ID: <1438850736.2009.22.camel@oracle.com>

Hi Jon,

On Tue, 2015-08-04 at 11:30 -0700, Jon Masamitsu wrote:
> Thomas,
> 
> I reviewed webrev.4 and it looks good.

  thanks for the review,

  Thomas



From eric.caspole at oracle.com  Thu Aug  6 13:28:45 2015
From: eric.caspole at oracle.com (Eric Caspole)
Date: Thu, 06 Aug 2015 09:28:45 -0400
Subject: RFR: JDK-8078904 : CMS: Assert failed: Ctl pt invariant
In-Reply-To: <55B94FE6.3040000@oracle.com>
References: <5553A32C.7030403@oracle.com> <1432035040.2555.25.camel@oracle.com>
	<55B94FE6.3040000@oracle.com>
Message-ID: <55C3610D.4060809@oracle.com>

Thomas and I had a IM about this the other day and in the end decided to 
allow the main rescan chunk array to grow if the user explicitly sets a 
very small MinTLABSize on the command line. There doesn't seem to be any 
good reason to set a very small MinTLABSize except maybe stress testing, 
and so it is simpler to grow this structure for that unusual case rather 
than use the stride I had in v03.

The new one is here:

  http://cr.openjdk.java.net/~ecaspole/JDK-8078904/04/webrev/

Bug:
  https://bugs.openjdk.java.net/browse/JDK-8078904

I tested this with the failing example using many combinations of gc
thread counts and many MinTLABSize values from 64 up to 4M. And JPRT.
Thanks,
Eric


From thomas.schatzl at oracle.com  Thu Aug  6 13:35:58 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 06 Aug 2015 15:35:58 +0200
Subject: RFR (XXS): 8133042: Remove some direct accesses of
	G1Allocator::reuse_retained_old_region() to G1CollectedHeap members
In-Reply-To: <55C23C57.6000002@oracle.com>
References: <1438775999.2378.32.camel@oracle.com> <55C23C57.6000002@oracle.com>
Message-ID: <1438868158.2474.19.camel@oracle.com>

Hi,

On Wed, 2015-08-05 at 09:39 -0700, Jon Masamitsu wrote:
> Looks good.
> 
> Jon

  thanks for the review.

Thomas



From axel.siebenborn at sap.com  Thu Aug  6 13:49:50 2015
From: axel.siebenborn at sap.com (Siebenborn, Axel)
Date: Thu, 6 Aug 2015 13:49:50 +0000
Subject: RFR (XS):  Move implementation of process_grey_object to
	concurrentMark.inline.hpp 
Message-ID: <02D5D45C1F8DB848A7AE20E80EE61A5C39962EC5@DEWDFEMB20C.global.corp.sap>

Hi,
Could I have reviews and a sponsor for this small change 

We had a linker problem with a C++ - compiler that inlined process_grey_object at all uses but removed the symbol and the actual implementation.
As there are explicit instantiations of this method in concurrentMark.cpp, this probably should not happen and can be considered of a bug of the C++ compiler. Though, it seems to be pointless to mark that method inline if there are explicit instantiations.

Considering performance, it would be better to move the implementation of process_grey_object to concurrentMark.inline.hpp in order to inline the method in other compilation units.

CR:
https://bugs.openjdk.java.net/browse/JDK-8133121

Webrev:
http://cr.openjdk.java.net/~asiebenborn/8133121/webrev/


Thanks,
Axel


From tom.benson at oracle.com  Thu Aug  6 14:12:01 2015
From: tom.benson at oracle.com (Tom Benson)
Date: Thu, 06 Aug 2015 10:12:01 -0400
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <1438847329.2009.9.camel@oracle.com>
References: <55C103A4.1060505@oracle.com>			
	<A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>		
	<1438781844.2378.60.camel@oracle.com>
	<55C21DC3.5090305@oracle.com>	
	<1438785532.2378.73.camel@oracle.com>
	<55C248FF.6010308@oracle.com>	
	<F73C4B9A-4DBA-47A8-8715-0988FF6D393D@oracle.com>	
	<55C26249.1090902@oracle.com>	
	<C149705F-696B-46CD-8FB8-CAD06C511244@oracle.com>	
	<55C2751B.8070400@oracle.com> <1438847329.2009.9.camel@oracle.com>
Message-ID: <55C36B31.7000908@oracle.com>

Hi Thomas,

On 8/6/2015 3:48 AM, Thomas Schatzl wrote:
> Looking again at the code starting from line 1144 with the comment,
> there may still be an issue here: consider a VM A having 8M regions 1,
> 2, and 3, each spanning 8M (see figure below; each letter represents 1M
> of space), and a mapping from a VM B with 2M region size with the
> following layout that is loaded into VM A and failed.
>
>      1        2       3       region#
> AAAAAAAA AAAAAAAA AAAAAAAA   VM A regions (8M region size)
>    BB  BB BBBBBBBB BBBB       VM B mapping (2M region size)
>
> E.g. the mapping is from 2-3M, and another mapping from 6-20M. E.g. the
> ranges array contains
>
>    (2, 3), (6, 20)
>
> now, after freeing region 1 from the first mapping (2, 3), start_region
> of the second mapping (=1) equals prev_last_region (=1); now the cursor
> is advanced to the region of the end of the second mapping (=3),
> forgetting to free region 2.
>
> What am I missing here?
The cursor would not be advanced to to the end of the second mapping, 
but rather to the end of region A 1.  So the current range being freed 
would be reduced from (6,20) to (8,20).   Line 1148:

     if (start_region == prev_last_region) {
       start_address = start_region->end();     // Resets start address 
to end of A1, in your example

Tom

>
> Thanks for adding the initialization checks.
>
> Thanks,
>    Thomas
>
>
>



From thomas.schatzl at oracle.com  Thu Aug  6 14:35:23 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 06 Aug 2015 16:35:23 +0200
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <55C36B31.7000908@oracle.com>
References: <55C103A4.1060505@oracle.com>
	<A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>
	<1438781844.2378.60.camel@oracle.com> <55C21DC3.5090305@oracle.com>
	<1438785532.2378.73.camel@oracle.com> <55C248FF.6010308@oracle.com>
	<F73C4B9A-4DBA-47A8-8715-0988FF6D393D@oracle.com>
	<55C26249.1090902@oracle.com>
	<C149705F-696B-46CD-8FB8-CAD06C511244@oracle.com>
	<55C2751B.8070400@oracle.com> <1438847329.2009.9.camel@oracle.com>
	<55C36B31.7000908@oracle.com>
Message-ID: <1438871723.2474.37.camel@oracle.com>

Hi Tom,

On Thu, 2015-08-06 at 10:12 -0400, Tom
 Benson wrote:
Hi Thomas,
> 
> On 8/6/2015 3:48 AM, Thomas Schatzl wrote:
> > Looking again at the code starting from line 1144 with the comment,
> > there may still be an issue here: consider a VM A having 8M regions
1,
> > 2, and 3, each spanning 8M (see figure below; each letter represents
1M
> > of space), and a mapping from a VM B with 2M region size with the
> > following layout that is loaded into VM A and failed.
> >
> >      1        2       3       region#
> > AAAAAAAA AAAAAAAA AAAAAAAA   VM A regions (8M region size)
> >    BB  BB BBBBBBBB BBBB       VM B mapping (2M region size)
> >
> > E.g. the mapping is from 2-3M, and another mapping from 6-20M. E.g.
the
> > ranges array contains
> >
> >    (2, 3), (6, 20)
> >
> > now, after freeing region 1 from the first mapping (2, 3),
start_region
> > of the second mapping (=1) equals prev_last_region (=1); now the
cursor
> > is advanced to the region of the end of the second mapping (=3),
> > forgetting to free region 2.
> >
> > What am I missing here?
> The cursor would not be advanced to to the end of the second mapping, 
> but rather to the end of region A 1.  So the current range being
> freed 
> would be reduced from (6,20) to (8,20).   Line 1148:
> 
>      if (start_region == prev_last_region) {
>        start_address = start_region->end();     // Resets start
address 
> to end of A1, in your example
> 

Okay, looks good. :)

Thomas




From thomas.schatzl at oracle.com  Thu Aug  6 14:39:00 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 06 Aug 2015 16:39:00 +0200
Subject: RFR (S): 8003237: G1: Reduce unnecessary (and failing)
	allocation attempts when handling an evacuation failure
In-Reply-To: <55C23B83.90405@oracle.com>
References: <1438605320.2292.47.camel@oracle.com>
	<1438605827.2292.49.camel@oracle.com>
	<1438763108.2378.9.camel@oracle.com> <55C23B83.90405@oracle.com>
Message-ID: <1438871940.2474.38.camel@oracle.com>

(sorry if that mail is received multiple times, getting send error just
for this mail)

Hi Jon,

  thanks for the review...

On Wed, 2015-08-05 at 09:36 -0700, Jon Masamitsu wrote:
> Thomas,
> 
> I think that this method gets called if a new region is allocated
> via allocate_inline_or_new_plab()
> 
> "src/share/vm/gc/g1/g1CollectedHeap.cpp"
> 
> 6436 HeapRegion* G1CollectedHeap::new_gc_alloc_region(size_t
word_size,
> 6437                                                  uint count,
> 6438                                                  InCSetState
dest) {
> 6439   assert(FreeList_lock->owned_by_self(), "pre-condition");
> 6440
> 6441   if (count < g1_policy()->max_regions(dest)) {
> 
> Might the g1_policy()->max_regions() conflict with the test
> in has_more_free_regions() which considers all free regions
> (even uncommitted).   That is, has_more_free_regions() will
> allow allocation attempts that will not success because of
> max_regions().

No, because the worst case is that we go into the take-the-Mutex case
too often (which is bad).

However, thinking about this again, I found another option (which is
much clearer to see after recent refactoring of g1allocator classes): if
G1AllocRegion::attempt_allocation_locked() fails, we know that there is
really no more space in the heap left.

In this case, just set an internal flag, instead of relying on the
has_more_free_regions() from G1CollectedHeap.

> 
> http://cr.openjdk.java.net/~tschatzl/8003237/webrev/src/share/vm/gc/g1/g1ParScanThreadState.hpp.frames.html
> 
> >    74   // Indicates whether in the last generation (old) there is
no more space
> >    75   // available for allocation.
> >    76   bool _no_more_space_in_old;
> 
> Would a name like _old_is_full suit you here in place of 
> _no_more_space_in_old?  I like positives instead
> negatives.

Done.

New webrevs:
http://cr.openjdk.java.net/~tschatzl/8003237/webrev.1 (full)
http://cr.openjdk.java.net/~tschatzl/8003237/webrev.0_to_1 (partial)

I think this one looks better, performs the same (tested) and does not
introduce the additional dependency on G1CollectedHeap.

This also automatically takes into consideration restrictions like in
G1CollectedHeap::new_gc_alloc_region() without the need to repeat them.

The addition of the new methods as virtual has been intentional as other
Allocators may have different views on when the heap for the given
context is full.

Thanks,
  Thomas



From tom.benson at oracle.com  Thu Aug  6 14:49:26 2015
From: tom.benson at oracle.com (Tom Benson)
Date: Thu, 06 Aug 2015 10:49:26 -0400
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <1438871723.2474.37.camel@oracle.com>
References: <55C103A4.1060505@oracle.com>				
	<A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>			
	<1438781844.2378.60.camel@oracle.com>
	<55C21DC3.5090305@oracle.com>		
	<1438785532.2378.73.camel@oracle.com>
	<55C248FF.6010308@oracle.com>		
	<F73C4B9A-4DBA-47A8-8715-0988FF6D393D@oracle.com>		
	<55C26249.1090902@oracle.com>		
	<C149705F-696B-46CD-8FB8-CAD06C511244@oracle.com>		
	<55C2751B.8070400@oracle.com> <1438847329.2009.9.camel@oracle.com>	
	<55C36B31.7000908@oracle.com> <1438871723.2474.37.camel@oracle.com>
Message-ID: <55C373F6.9060609@oracle.com>

Thanks, Thomas.
Tom

On 8/6/2015 10:35 AM, Thomas Schatzl wrote:
> Hi Tom,
>
> [.....]
> Okay, looks good. :)
>
> Thomas
>
>



From thomas.schatzl at oracle.com  Thu Aug  6 14:50:49 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 06 Aug 2015 16:50:49 +0200
Subject: RFR (XXS): 8133042: Remove some direct accesses of
	G1Allocator::reuse_retained_old_region() to G1CollectedHeap members
In-Reply-To: <FD29FE78-EB89-4630-A5E9-16DECC9B8846@oracle.com>
References: <1438775999.2378.32.camel@oracle.com>
	<FD29FE78-EB89-4630-A5E9-16DECC9B8846@oracle.com>
Message-ID: <1438872649.2474.40.camel@oracle.com>

Hi Kim,

  thanks for looking at this...

On Wed, 2015-08-05 at 12:54 -0400, Kim Barrett wrote:
> On Aug 5, 2015, at 7:59 AM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
> > 
> > Hi all,
> > 
> >  could I have reviews of this tiny change that changes some direct
> > private member accesses of G1Allocator to G1CollectedHeap to some
> > already available public method calls?
> > 
> > While this is more a cosmetic change, I thought it would be good to
> > decrease the amount of "friendliness" between those two classes. I could
> > not find a good other CR to add them, so here we are....
> > 
> > CR:
> > https://bugs.openjdk.java.net/browse/JDK-8133042
> > Webrev:
> > http://cr.openjdk.java.net/~tschatzl/8133042/webrev/
> > Testing:
> > local compilation
> 
> There is very similar code to what is being changed here in G1ArchiveAllocator::alloc_new_region,
> later in the same file (g1Allocator.cpp).
> 
> Would the proposed change or the additional change to alloc_new_region permit the elimination
> of any friend declarations from G1CollectedHeap?
> 

  only if some additional methods from G1CollectedHeap were made public.

Here is a webrev doing that:
http://cr.openjdk.java.net/~tschatzl/8133042/webrev.1/ (full only
because the original change has been so tiny)

I am fine with either version.

Thanks,
  Thomas




From thomas.schatzl at oracle.com  Thu Aug  6 15:08:20 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 06 Aug 2015 17:08:20 +0200
Subject: RFR: JDK-8078904 : CMS: Assert failed: Ctl pt invariant
In-Reply-To: <55C3610D.4060809@oracle.com>
References: <5553A32C.7030403@oracle.com>
	<1432035040.2555.25.camel@oracle.com> <55B94FE6.3040000@oracle.com>
	<55C3610D.4060809@oracle.com>
Message-ID: <1438873700.2474.41.camel@oracle.com>

Hi,

On Thu, 2015-08-06 at 09:28 -0400, Eric Caspole wrote:
> Thomas and I had a IM about this the other day and in the end decided to 
> allow the main rescan chunk array to grow if the user explicitly sets a 
> very small MinTLABSize on the command line. There doesn't seem to be any 
> good reason to set a very small MinTLABSize except maybe stress testing, 
> and so it is simpler to grow this structure for that unusual case rather 
> than use the stride I had in v03.
> 
> The new one is here:
> 
>   http://cr.openjdk.java.net/~ecaspole/JDK-8078904/04/webrev/
> 
> Bug:
>   https://bugs.openjdk.java.net/browse/JDK-8078904
> 
> I tested this with the failing example using many combinations of gc
> thread counts and many MinTLABSize values from 64 up to 4M. And JPRT.

  looks good.

Thanks,
  Thomas




From kim.barrett at oracle.com  Thu Aug  6 15:24:34 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Thu, 6 Aug 2015 11:24:34 -0400
Subject: RFR (XXS): 8133042: Remove some direct accesses of
	G1Allocator::reuse_retained_old_region() to G1CollectedHeap members
In-Reply-To: <1438872649.2474.40.camel@oracle.com>
References: <1438775999.2378.32.camel@oracle.com>
	<FD29FE78-EB89-4630-A5E9-16DECC9B8846@oracle.com>
	<1438872649.2474.40.camel@oracle.com>
Message-ID: <D5B26462-A6D3-4446-ABC2-54B3C4168713@oracle.com>

On Aug 6, 2015, at 10:50 AM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
> 
>> There is very similar code to what is being changed here in G1ArchiveAllocator::alloc_new_region,
>> later in the same file (g1Allocator.cpp).
>> 
>> Would the proposed change or the additional change to alloc_new_region permit the elimination
>> of any friend declarations from G1CollectedHeap?
>> 
> 
>  only if some additional methods from G1CollectedHeap were made public.
> 
> Here is a webrev doing that:
> http://cr.openjdk.java.net/~tschatzl/8133042/webrev.1/ (full only
> because the original change has been so tiny)
> 
> I am fine with either version.

Looks good.

One tiny nit:

------------------------------------------------------------------------------ 
src/share/vm/gc/g1/g1CollectedHeap.hpp
 700   // It dirties the cards that cover the block so that so that the post

[pre-existing]
"so that so that" => "so that"

------------------------------------------------------------------------------



From tom.benson at oracle.com  Thu Aug  6 15:40:46 2015
From: tom.benson at oracle.com (Tom Benson)
Date: Thu, 06 Aug 2015 11:40:46 -0400
Subject: RFR: JDK-8078904 : CMS: Assert failed: Ctl pt invariant
In-Reply-To: <55C3610D.4060809@oracle.com>
References: <5553A32C.7030403@oracle.com>
	<1432035040.2555.25.camel@oracle.com>	<55B94FE6.3040000@oracle.com>
	<55C3610D.4060809@oracle.com>
Message-ID: <55C37FFE.6020601@oracle.com>

Hi Eric,
I understand the comment below resulting in no longer preventing 
max_plab_samples from getting smaller than 2k.  But I see the factor of 
2 in computing _survivor_chunk_capacity and array size was also removed, 
which would have affected any size.  Was that just viewed as an 
arbitrary inflation, or ?
Tnx,
Tom

On 8/6/2015 9:28 AM, Eric Caspole wrote:
> Thomas and I had a IM about this the other day and in the end decided 
> to allow the main rescan chunk array to grow if the user explicitly 
> sets a very small MinTLABSize on the command line. There doesn't seem 
> to be any good reason to set a very small MinTLABSize except maybe 
> stress testing, and so it is simpler to grow this structure for that 
> unusual case rather than use the stride I had in v03.
>
> The new one is here:
>
>  http://cr.openjdk.java.net/~ecaspole/JDK-8078904/04/webrev/
>
> Bug:
>  https://bugs.openjdk.java.net/browse/JDK-8078904
>
> I tested this with the failing example using many combinations of gc
> thread counts and many MinTLABSize values from 64 up to 4M. And JPRT.
> Thanks,
> Eric



From kim.barrett at oracle.com  Thu Aug  6 15:49:02 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Thu, 6 Aug 2015 11:49:02 -0400
Subject: RFR: JDK-8078904 : CMS: Assert failed: Ctl pt invariant
In-Reply-To: <55C3610D.4060809@oracle.com>
References: <5553A32C.7030403@oracle.com> <1432035040.2555.25.camel@oracle.com>
	<55B94FE6.3040000@oracle.com> <55C3610D.4060809@oracle.com>
Message-ID: <75994E95-700A-402E-B7FA-0D971453DF59@oracle.com>

On Aug 6, 2015, at 9:28 AM, Eric Caspole <eric.caspole at oracle.com> wrote:
> 
> Thomas and I had a IM about this the other day and in the end decided to allow the main rescan chunk array to grow if the user explicitly sets a very small MinTLABSize on the command line. There doesn't seem to be any good reason to set a very small MinTLABSize except maybe stress testing, and so it is simpler to grow this structure for that unusual case rather than use the stride I had in v03.
> 
> The new one is here:
> 
> http://cr.openjdk.java.net/~ecaspole/JDK-8078904/04/webrev/
> 
> Bug:
> https://bugs.openjdk.java.net/browse/JDK-8078904

Looks good.

One pre-existing issue. I don't need a new webrev if you fix this.

------------------------------------------------------------------------------
 623       ((DefNewGeneration*)_young_gen)->max_survivor_size() / (ThreadLocalAllocBuffer::min_size() * HeapWordSize);

[pre-existing]
Unnecessary explicit up-cast of _young_gen to DefNewGeneration*.

------------------------------------------------------------------------------



From thomas.schatzl at oracle.com  Thu Aug  6 15:49:11 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 06 Aug 2015 17:49:11 +0200
Subject: RFR (L): 8073013: Add detailed information about PLAB memory usage
Message-ID: <1438876151.2474.51.camel@oracle.com>

Hi all,

  can I have reviews for this change that prepares for a lot of changes
related to 8030849 "Investigate high fragmentation/waste in some
situations during allocation during GC in G1"?

Before investigating and fixing fragmentation during PLAB, we need to
track that information.

For this reason, PLABStats has been extended in a G1EvacStats class with
the necessary information that is needed to do better PLAB sizing, in
particular G1EvacStats::_region_end_waste that tracks the waste at the
end of regions during refill, G1EvacStats::_regions_filled that tracks
the number of region refills, G1EvacStats::_inline_allocated that tracks
the amount of inline allocation (excluding PLAB allocation),
G1EvacStats::_failure_used and G1EvacStats::_failure_waste that track
memory usage in regions where evacuation failure occurred.

There had to be somewhat extensive changes particularly to get the
_region_end_waste in the allocation classes, which is what makes this
change a bit large.

Overall there is no change in the sizing policy except for accounting
_region_end_waste.

As you may notice, there is a new JFR event, but no real additional
logging for the log file (the -XX:+PrintPLAB message has been extended
though). That will be added in an extra CR, as soon as the new logging
framework has been added.

CR:
https://bugs.openjdk.java.net/browse/JDK-8073013

Webrev:
http://cr.openjdk.java.net/~tschatzl/8073013/webrev

Testing:
jprt, a few aurora runs, local testing, lots of benchmarks mainly in
conjunction with the following changes.

Thanks,
  Thomas




From thomas.schatzl at oracle.com  Thu Aug  6 15:58:00 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 06 Aug 2015 17:58:00 +0200
Subject: RFR (XXS): 8133042: Remove some direct accesses of
	G1Allocator::reuse_retained_old_region() to G1CollectedHeap members
In-Reply-To: <D5B26462-A6D3-4446-ABC2-54B3C4168713@oracle.com>
References: <1438775999.2378.32.camel@oracle.com>
	<FD29FE78-EB89-4630-A5E9-16DECC9B8846@oracle.com>
	<1438872649.2474.40.camel@oracle.com>
	<D5B26462-A6D3-4446-ABC2-54B3C4168713@oracle.com>
Message-ID: <1438876680.2474.55.camel@oracle.com>

Hi Kim,

  thanks for the review.

On Thu, 2015-08-06 at 11:24 -0400, Kim Barrett wrote:
> On Aug 6, 2015, at 10:50 AM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
> > 
> >> There is very similar code to what is being changed here in G1ArchiveAllocator::alloc_new_region,
> >> later in the same file (g1Allocator.cpp).
> >> 
> >> Would the proposed change or the additional change to alloc_new_region permit the elimination
> >> of any friend declarations from G1CollectedHeap? 
> > 
> >  only if some additional methods from G1CollectedHeap were made public.
> > 
> > Here is a webrev doing that:
> > http://cr.openjdk.java.net/~tschatzl/8133042/webrev.1/ (full only
> > because the original change has been so tiny)
> > 
> > I am fine with either version.
> 
> Looks good.
> 
> One tiny nit:

> src/share/vm/gc/g1/g1CollectedHeap.hpp
>  700   // It dirties the cards that cover the block so that so that the post
> 
> [pre-existing]
> "so that so that" => "so that"

  I allowed myselves to just update the webrevs in place for this tiny
change.

http://cr.openjdk.java.net/~tschatzl/8133042/webrev.1/

Thanks,
  Thomas





From thomas.schatzl at oracle.com  Thu Aug  6 16:01:19 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 06 Aug 2015 18:01:19 +0200
Subject: RFR (L): 8073013: Add detailed information about PLAB memory usage
In-Reply-To: <1438876151.2474.51.camel@oracle.com>
References: <1438876151.2474.51.camel@oracle.com>
Message-ID: <1438876879.2474.58.camel@oracle.com>

Hi all,

  just one minor addition: this change is based on all recent RFRs
including 8003237 (the evacuation failure speedup change) and 8133042
(remove some friends in g1collectedheap).

For the others a push job is on its way.

Thanks,
  Thomas

On Thu, 2015-08-06 at 17:49 +0200, Thomas Schatzl wrote:
> Hi all,
> 
>   can I have reviews for this change that prepares for a lot of changes
> related to 8030849 "Investigate high fragmentation/waste in some
> situations during allocation during GC in G1"?
> 
> Before investigating and fixing fragmentation during PLAB, we need to
> track that information.
> 
> For this reason, PLABStats has been extended in a G1EvacStats class with
> the necessary information that is needed to do better PLAB sizing, in
> particular G1EvacStats::_region_end_waste that tracks the waste at the
> end of regions during refill, G1EvacStats::_regions_filled that tracks
> the number of region refills, G1EvacStats::_inline_allocated that tracks
> the amount of inline allocation (excluding PLAB allocation),
> G1EvacStats::_failure_used and G1EvacStats::_failure_waste that track
> memory usage in regions where evacuation failure occurred.
> 
> There had to be somewhat extensive changes particularly to get the
> _region_end_waste in the allocation classes, which is what makes this
> change a bit large.
> 
> Overall there is no change in the sizing policy except for accounting
> _region_end_waste.
> 
> As you may notice, there is a new JFR event, but no real additional
> logging for the log file (the -XX:+PrintPLAB message has been extended
> though). That will be added in an extra CR, as soon as the new logging
> framework has been added.
> 
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8073013
> 
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8073013/webrev
> 
> Testing:
> jprt, a few aurora runs, local testing, lots of benchmarks mainly in
> conjunction with the following changes.
> 
> Thanks,
>   Thomas
> 
> 




From eric.caspole at oracle.com  Thu Aug  6 18:20:10 2015
From: eric.caspole at oracle.com (Eric Caspole)
Date: Thu, 06 Aug 2015 14:20:10 -0400
Subject: RFR: JDK-8078904 : CMS: Assert failed: Ctl pt invariant
In-Reply-To: <55C37FFE.6020601@oracle.com>
References: <5553A32C.7030403@oracle.com>
	<1432035040.2555.25.camel@oracle.com>	<55B94FE6.3040000@oracle.com>
	<55C3610D.4060809@oracle.com> <55C37FFE.6020601@oracle.com>
Message-ID: <55C3A55A.6040109@oracle.com>

Hi Tom,
I am not sure about the theory of the existing algorithm, it does not 
seem too robust. The 2* still did not leave enough room to hold all the 
possible entries from the per-thread arrays. It was not that hard to 
find a combo of thread count and MinTLABSize that would trigger the 
assert in the original code. However, the only downside of the existing 
code was bad distribution of work in the rescan tasks.
Now it is assured there will be room in the main array with this new change.
Eric


On 8/6/2015 11:40 AM, Tom Benson wrote:
> Hi Eric,
> I understand the comment below resulting in no longer preventing 
> max_plab_samples from getting smaller than 2k.  But I see the factor 
> of 2 in computing _survivor_chunk_capacity and array size was also 
> removed, which would have affected any size.  Was that just viewed as 
> an arbitrary inflation, or ?
> Tnx,
> Tom
>
> On 8/6/2015 9:28 AM, Eric Caspole wrote:
>> Thomas and I had a IM about this the other day and in the end decided 
>> to allow the main rescan chunk array to grow if the user explicitly 
>> sets a very small MinTLABSize on the command line. There doesn't seem 
>> to be any good reason to set a very small MinTLABSize except maybe 
>> stress testing, and so it is simpler to grow this structure for that 
>> unusual case rather than use the stride I had in v03.
>>
>> The new one is here:
>>
>>  http://cr.openjdk.java.net/~ecaspole/JDK-8078904/04/webrev/
>>
>> Bug:
>>  https://bugs.openjdk.java.net/browse/JDK-8078904
>>
>> I tested this with the failing example using many combinations of gc
>> thread counts and many MinTLABSize values from 64 up to 4M. And JPRT.
>> Thanks,
>> Eric
>



From eric.caspole at oracle.com  Thu Aug  6 18:39:18 2015
From: eric.caspole at oracle.com (Eric Caspole)
Date: Thu, 06 Aug 2015 14:39:18 -0400
Subject: RFR: JDK-8078904 : CMS: Assert failed: Ctl pt invariant
In-Reply-To: <75994E95-700A-402E-B7FA-0D971453DF59@oracle.com>
References: <5553A32C.7030403@oracle.com> <1432035040.2555.25.camel@oracle.com>
	<55B94FE6.3040000@oracle.com> <55C3610D.4060809@oracle.com>
	<75994E95-700A-402E-B7FA-0D971453DF59@oracle.com>
Message-ID: <55C3A9D6.9000201@oracle.com>

Good catch, I'll fix that now.
Eric

On 8/6/2015 11:49 AM, Kim Barrett wrote:
> On Aug 6, 2015, at 9:28 AM, Eric Caspole <eric.caspole at oracle.com> wrote:
>> Thomas and I had a IM about this the other day and in the end decided to allow the main rescan chunk array to grow if the user explicitly sets a very small MinTLABSize on the command line. There doesn't seem to be any good reason to set a very small MinTLABSize except maybe stress testing, and so it is simpler to grow this structure for that unusual case rather than use the stride I had in v03.
>>
>> The new one is here:
>>
>> http://cr.openjdk.java.net/~ecaspole/JDK-8078904/04/webrev/
>>
>> Bug:
>> https://bugs.openjdk.java.net/browse/JDK-8078904
> Looks good.
>
> One pre-existing issue. I don't need a new webrev if you fix this.
>
> ------------------------------------------------------------------------------
>   623       ((DefNewGeneration*)_young_gen)->max_survivor_size() / (ThreadLocalAllocBuffer::min_size() * HeapWordSize);
>
> [pre-existing]
> Unnecessary explicit up-cast of _young_gen to DefNewGeneration*.
>
> ------------------------------------------------------------------------------
>



From jiangli.zhou at oracle.com  Thu Aug  6 21:32:56 2015
From: jiangli.zhou at oracle.com (Jiangli Zhou)
Date: Thu, 6 Aug 2015 14:32:56 -0700
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <55C373F6.9060609@oracle.com>
References: <55C103A4.1060505@oracle.com>				
	<A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>			
	<1438781844.2378.60.camel@oracle.com>
	<55C21DC3.5090305@oracle.com>		
	<1438785532.2378.73.camel@oracle.com>
	<55C248FF.6010308@oracle.com>		
	<F73C4B9A-4DBA-47A8-8715-0988FF6D393D@oracle.com>		
	<55C26249.1090902@oracle.com>		
	<C149705F-696B-46CD-8FB8-CAD06C511244@oracle.com>		
	<55C2751B.8070400@oracle.com> <1438847329.2009.9.camel@oracle.com>	
	<55C36B31.7000908@oracle.com> <1438871723.2474.37.camel@oracle.com>
	<55C373F6.9060609@oracle.com>
Message-ID: <AD45178E-A442-4A0D-80D6-50E91608E1FE@oracle.com>

Hi,

Here is the runtime part of the bug fix that calls the new free_archive_regions() when shared string mapping fails. I also added a jtreg test to test shared strings with -Xshare:auto.

  http://cr.openjdk.java.net/~jiangli/8131734/webrev.00/

Test:
- Tested by explicitly making the shared string mapping fail on linux-x64, -Xshare:auto runs without crash with the fix
- Tested with the new SharedStringsRunAuto test
- Tested with XX:+PrintNMTStatistics -XX:NativeMemoryTracking=detail

Thanks,
Jiangli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150806/eadde7ab/attachment.htm>

From dmitry.dmitriev at oracle.com  Thu Aug  6 21:59:23 2015
From: dmitry.dmitriev at oracle.com (Dmitry Dmitriev)
Date: Fri, 7 Aug 2015 00:59:23 +0300
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <AD45178E-A442-4A0D-80D6-50E91608E1FE@oracle.com>
References: <55C103A4.1060505@oracle.com>
	<A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>
	<1438781844.2378.60.camel@oracle.com> <55C21DC3.5090305@oracle.com>
	<1438785532.2378.73.camel@oracle.com> <55C248FF.6010308@oracle.com>
	<F73C4B9A-4DBA-47A8-8715-0988FF6D393D@oracle.com>
	<55C26249.1090902@oracle.com>
	<C149705F-696B-46CD-8FB8-CAD06C511244@oracle.com>
	<55C2751B.8070400@oracle.com> <1438847329.2009.9.camel@oracle.com>
	<55C36B31.7000908@oracle.com> <1438871723.2474.37.camel@oracle.com>
	<55C373F6.9060609@oracle.com>
	<AD45178E-A442-4A0D-80D6-50E91608E1FE@oracle.com>
Message-ID: <55C3D8BB.9000603@oracle.com>

Hello Jiangli,

I have few comments/questions.

src/share/vm/memory/filemap.cpp module:
1) Should free_archive_regions also called when verify_string_regions() 
returns false on line 717?
2) The same question about unmap_string_regions(). Region must be freed 
when unmap_string_regions() is called?
3) Extra space at the end of the line 711.

  test/runtime/SharedArchiveFile/SharedStringsRunAuto.java
1) Unneeded second creating of OutputAnalyzer on line 61. Also, probably 
will be better to use same scheme for OutputAnalyzer? On lines 46-49 you 
not use local variable, but on lines 61-63 use local variable.
   59         OutputAnalyzer output = new OutputAnalyzer(pb.start());
   60
   61         output = new OutputAnalyzer(pb.start());
2) Extra space at the end of the lines 25,26,30, 51

Thanks,
Dmitry

On 07.08.2015 0:32, Jiangli Zhou wrote:
> Hi,
>
> Here is the runtime part of the bug fix that calls the new 
> free_archive_regions() when shared string mapping fails. I also added 
> a jtreg test to test shared strings with -Xshare:auto.
>
> http://cr.openjdk.java.net/~jiangli/8131734/webrev.00/ 
> <http://cr.openjdk.java.net/%7Ejiangli/8131734/webrev.00/>
>
> Test:
> - Tested by explicitly making the shared string mapping fail on 
> linux-x64, -Xshare:auto runs without crash with the fix
> - Tested with the new SharedStringsRunAuto test
> - Tested with XX:+PrintNMTStatistics -XX:NativeMemoryTracking=detail
>
> Thanks,
> Jiangli

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150807/fc052a2e/attachment.htm>

From jiangli.zhou at oracle.com  Thu Aug  6 22:19:33 2015
From: jiangli.zhou at oracle.com (Jiangli Zhou)
Date: Thu, 6 Aug 2015 15:19:33 -0700
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <55C3D8BB.9000603@oracle.com>
References: <55C103A4.1060505@oracle.com>
	<A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>
	<1438781844.2378.60.camel@oracle.com> <55C21DC3.5090305@oracle.com>
	<1438785532.2378.73.camel@oracle.com> <55C248FF.6010308@oracle.com>
	<F73C4B9A-4DBA-47A8-8715-0988FF6D393D@oracle.com>
	<55C26249.1090902@oracle.com>
	<C149705F-696B-46CD-8FB8-CAD06C511244@oracle.com>
	<55C2751B.8070400@oracle.com> <1438847329.2009.9.camel@oracle.com>
	<55C36B31.7000908@oracle.com> <1438871723.2474.37.camel@oracle.com>
	<55C373F6.9060609@oracle.com>
	<AD45178E-A442-4A0D-80D6-50E91608E1FE@oracle.com>
	<55C3D8BB.9000603@oracle.com>
Message-ID: <48C0E571-0D7B-4D30-85DF-E31D99092A7C@oracle.com>

Hi Dmitry,

Thank you for the feedback! You are definitely right about freeing the archived regions in those failure cases. I should have thought it more thoroughly, instead of just fixing the case that I ran into. Will exam all cases.

I?ll change the test to incorporate your suggestion. Will fix the extra spaces before committing. I have one question, how do you detect the extra spaces from the webrev? :)

Thanks,
Jiangli

On Aug 6, 2015, at 2:59 PM, Dmitry Dmitriev <dmitry.dmitriev at oracle.com> wrote:

> Hello Jiangli,
> 
> I have few comments/questions.
> 
> src/share/vm/memory/filemap.cpp module:
> 1) Should free_archive_regions also called when verify_string_regions() returns false on line 717?
> 2) The same question about unmap_string_regions(). Region must be freed when unmap_string_regions() is called?
> 3) Extra space at the end of the line 711.
> 
>  test/runtime/SharedArchiveFile/SharedStringsRunAuto.java
> 1) Unneeded second creating of OutputAnalyzer on line 61. Also, probably will be better to use same scheme for OutputAnalyzer? On lines 46-49 you not use local variable, but on lines 61-63 use local variable.
>   59         OutputAnalyzer output = new OutputAnalyzer(pb.start());
>   60 
>   61         output = new OutputAnalyzer(pb.start());
> 2) Extra space at the end of the lines 25,26,30, 51
> 
> Thanks,
> Dmitry
> 
> On 07.08.2015 0:32, Jiangli Zhou wrote:
>> Hi,
>> 
>> Here is the runtime part of the bug fix that calls the new free_archive_regions() when shared string mapping fails. I also added a jtreg test to test shared strings with -Xshare:auto.
>> 
>>   http://cr.openjdk.java.net/~jiangli/8131734/webrev.00/
>> 
>> Test:
>> - Tested by explicitly making the shared string mapping fail on linux-x64, -Xshare:auto runs without crash with the fix
>> - Tested with the new SharedStringsRunAuto test
>> - Tested with XX:+PrintNMTStatistics -XX:NativeMemoryTracking=detail
>> 
>> Thanks,
>> Jiangli
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150806/76da59a9/attachment.htm>

From jiangli.zhou at oracle.com  Fri Aug  7 00:40:32 2015
From: jiangli.zhou at oracle.com (Jiangli Zhou)
Date: Thu, 6 Aug 2015 17:40:32 -0700
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <48C0E571-0D7B-4D30-85DF-E31D99092A7C@oracle.com>
References: <55C103A4.1060505@oracle.com>
	<A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>
	<1438781844.2378.60.camel@oracle.com> <55C21DC3.5090305@oracle.com>
	<1438785532.2378.73.camel@oracle.com> <55C248FF.6010308@oracle.com>
	<F73C4B9A-4DBA-47A8-8715-0988FF6D393D@oracle.com>
	<55C26249.1090902@oracle.com>
	<C149705F-696B-46CD-8FB8-CAD06C511244@oracle.com>
	<55C2751B.8070400@oracle.com> <1438847329.2009.9.camel@oracle.com>
	<55C36B31.7000908@oracle.com> <1438871723.2474.37.camel@oracle.com>
	<55C373F6.9060609@oracle.com>
	<AD45178E-A442-4A0D-80D6-50E91608E1FE@oracle.com>
	<55C3D8BB.9000603@oracle.com>
	<48C0E571-0D7B-4D30-85DF-E31D99092A7C@oracle.com>
Message-ID: <30B0248C-3BEA-4FAE-861F-DAAE19F56B45@oracle.com>

Hi Dmitry,

I?ve added shared string unmapping to handle the string data verification failure. I also added 'free_archive_regions()? call to FileMapInfo::unmap_string_regions(). My testing revealed some issues with the new archive region free code when I forced string verification failure. I?m working with Tom on the issues.

Thanks,
Jiangli


On Aug 6, 2015, at 3:19 PM, Jiangli Zhou <jiangli.zhou at oracle.com> wrote:

> Hi Dmitry,
> 
> Thank you for the feedback! You are definitely right about freeing the archived regions in those failure cases. I should have thought it more thoroughly, instead of just fixing the case that I ran into. Will exam all cases.
> 
> I?ll change the test to incorporate your suggestion. Will fix the extra spaces before committing. I have one question, how do you detect the extra spaces from the webrev? :)
> 
> Thanks,
> Jiangli
> 
> On Aug 6, 2015, at 2:59 PM, Dmitry Dmitriev <dmitry.dmitriev at oracle.com> wrote:
> 
>> Hello Jiangli,
>> 
>> I have few comments/questions.
>> 
>> src/share/vm/memory/filemap.cpp module:
>> 1) Should free_archive_regions also called when verify_string_regions() returns false on line 717?
>> 2) The same question about unmap_string_regions(). Region must be freed when unmap_string_regions() is called?
>> 3) Extra space at the end of the line 711.
>> 
>> test/runtime/SharedArchiveFile/SharedStringsRunAuto.java
>> 1) Unneeded second creating of OutputAnalyzer on line 61. Also, probably will be better to use same scheme for OutputAnalyzer? On lines 46-49 you not use local variable, but on lines 61-63 use local variable.
>>  59         OutputAnalyzer output = new OutputAnalyzer(pb.start());
>>  60 
>>  61         output = new OutputAnalyzer(pb.start());
>> 2) Extra space at the end of the lines 25,26,30, 51
>> 
>> Thanks,
>> Dmitry
>> 
>> On 07.08.2015 0:32, Jiangli Zhou wrote:
>>> Hi,
>>> 
>>> Here is the runtime part of the bug fix that calls the new free_archive_regions() when shared string mapping fails. I also added a jtreg test to test shared strings with -Xshare:auto.
>>> 
>>>  http://cr.openjdk.java.net/~jiangli/8131734/webrev.00/
>>> 
>>> Test:
>>> - Tested by explicitly making the shared string mapping fail on linux-x64, -Xshare:auto runs without crash with the fix
>>> - Tested with the new SharedStringsRunAuto test
>>> - Tested with XX:+PrintNMTStatistics -XX:NativeMemoryTracking=detail
>>> 
>>> Thanks,
>>> Jiangli
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150806/a789acdb/attachment.htm>

From jon.masamitsu at oracle.com  Fri Aug  7 04:44:07 2015
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Thu, 06 Aug 2015 21:44:07 -0700
Subject: RFR (S): 8003237: G1: Reduce unnecessary (and failing) allocation
	attempts when handling an evacuation failure
In-Reply-To: <1438871266.2474.31.camel@gmx.at>
References: <1438871266.2474.31.camel@gmx.at>
Message-ID: <55C43797.5000403@oracle.com>



On 8/6/2015 7:27 AM, Thomas Schatzl wrote:
> Hi Jon,
>
>    thanks for the review...
>
> On Wed, 2015-08-05 at 09:36 -0700, Jon Masamitsu wrote:
>> Thomas,
>>
>> I think that this method gets called if a new region is allocated
>> via allocate_inline_or_new_plab()
>>
>> "src/share/vm/gc/g1/g1CollectedHeap.cpp"
>>
>> 6436 HeapRegion* G1CollectedHeap::new_gc_alloc_region(size_t word_size,
>> 6437                                                  uint count,
>> 6438                                                  InCSetState dest) {
>> 6439   assert(FreeList_lock->owned_by_self(), "pre-condition");
>> 6440
>> 6441   if (count < g1_policy()->max_regions(dest)) {
>>
>> Might the g1_policy()->max_regions() conflict with the test
>> in has_more_free_regions() which considers all free regions
>> (even uncommitted).   That is, has_more_free_regions() will
>> allow allocation attempts that will not success because of
>> max_regions().
> No, because the worst case is that we go into the take-the-Mutex case
> too often (which is bad).
>
> However, thinking about this again, I found another option (which is
> much clearer to see after recent refactoring of g1allocator classes): if
> G1AllocRegion::attempt_allocation_locked() fails, we know that there is
> really no more space in the heap left.
>
> In this case, just set an internal flag, instead of relying on the
> has_more_free_regions() from G1CollectedHeap.
>
>> http://cr.openjdk.java.net/~tschatzl/8003237/webrev/src/share/vm/gc/g1/g1ParScanThreadState.hpp.frames.html
>>
>>>     74   // Indicates whether in the last generation (old) there is no more space
>>>     75   // available for allocation.
>>>     76   bool _no_more_space_in_old;
>> Would a name like _old_is_full suit you here in place of
>> _no_more_space_in_old?  I like positives instead
>> negatives.
> Done.
>
> New webrevs:
> http://cr.openjdk.java.net/~tschatzl/8003237/webrev.1 (full)
> http://cr.openjdk.java.net/~tschatzl/8003237/webrev.0_to_1 (partial)
>
> I think this one looks better, performs the same (tested) and does not
> introduce the additional dependency on G1CollectedHeap.
>
> This also automatically takes into consideration restrictions like in
> G1CollectedHeap::new_gc_alloc_region() without the need to repeat them.
>
> The addition of the new methods as virtual has been intentional as other
> Allocators may have different views on when the heap for the given
> context is full.

I like the changes.  Looks good.  Thanks for the name change to 
old_is_full like
names.

Jon

>
> Thanks,
>    Thomas
>
>
>
>



From dmitry.dmitriev at oracle.com  Fri Aug  7 07:58:33 2015
From: dmitry.dmitriev at oracle.com (Dmitry Dmitriev)
Date: Fri, 7 Aug 2015 10:58:33 +0300
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <30B0248C-3BEA-4FAE-861F-DAAE19F56B45@oracle.com>
References: <55C103A4.1060505@oracle.com>
	<A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>
	<1438781844.2378.60.camel@oracle.com> <55C21DC3.5090305@oracle.com>
	<1438785532.2378.73.camel@oracle.com> <55C248FF.6010308@oracle.com>
	<F73C4B9A-4DBA-47A8-8715-0988FF6D393D@oracle.com>
	<55C26249.1090902@oracle.com>
	<C149705F-696B-46CD-8FB8-CAD06C511244@oracle.com>
	<55C2751B.8070400@oracle.com> <1438847329.2009.9.camel@oracle.com>
	<55C36B31.7000908@oracle.com> <1438871723.2474.37.camel@oracle.com>
	<55C373F6.9060609@oracle.com>
	<AD45178E-A442-4A0D-80D6-50E91608E1FE@oracle.com>
	<55C3D8BB.9000603@oracle.com>
	<48C0E571-0D7B-4D30-85DF-E31D99092A7C@oracle.com>
	<30B0248C-3BEA-4FAE-861F-DAAE19F56B45@oracle.com>
Message-ID: <55C46529.4030601@oracle.com>

Hello Jiangli,

Thank you. About extra spaces - I just highlight changed lines in webrev 
and see that some lines have additional spaces at the end. :)

Dmitry

On 07.08.2015 3:40, Jiangli Zhou wrote:
> Hi Dmitry,
>
> I?ve added shared string unmapping to handle the string data 
> verification failure. I also added 'free_archive_regions()? call 
> to FileMapInfo::unmap_string_regions(). My testing revealed some 
> issues with the new archive region free code when I forced string 
> verification failure. I?m working with Tom on the issues.
>
> Thanks,
> Jiangli
>
>
> On Aug 6, 2015, at 3:19 PM, Jiangli Zhou <jiangli.zhou at oracle.com 
> <mailto:jiangli.zhou at oracle.com>> wrote:
>
>> Hi Dmitry,
>>
>> Thank you for the feedback! You are definitely right about freeing 
>> the archived regions in those failure cases. I should have thought it 
>> more thoroughly, instead of just fixing the case that I ran into. 
>> Will exam all cases.
>>
>> I?ll change the test to incorporate your suggestion. Will fix the 
>> extra spaces before committing. I have one question, how do you 
>> detect the extra spaces from the webrev? :)
>>
>> Thanks,
>> Jiangli
>>
>> On Aug 6, 2015, at 2:59 PM, Dmitry Dmitriev 
>> <dmitry.dmitriev at oracle.com <mailto:dmitry.dmitriev at oracle.com>> wrote:
>>
>>> Hello Jiangli,
>>>
>>> I have few comments/questions.
>>>
>>> src/share/vm/memory/filemap.cpp module:
>>> 1) Should free_archive_regions also called when 
>>> verify_string_regions() returns false on line 717?
>>> 2) The same question about unmap_string_regions(). Region must be 
>>> freed when unmap_string_regions() is called?
>>> 3) Extra space at the end of the line 711.
>>>
>>> test/runtime/SharedArchiveFile/SharedStringsRunAuto.java
>>> 1) Unneeded second creating of OutputAnalyzer on line 61. Also, 
>>> probably will be better to use same scheme for OutputAnalyzer? On 
>>> lines 46-49 you not use local variable, but on lines 61-63 use local 
>>> variable.
>>>  59         OutputAnalyzer output = new OutputAnalyzer(pb.start());
>>>  60
>>>  61         output = new OutputAnalyzer(pb.start());
>>> 2) Extra space at the end of the lines 25,26,30, 51
>>>
>>> Thanks,
>>> Dmitry
>>>
>>> On 07.08.2015 0:32, Jiangli Zhou wrote:
>>>> Hi,
>>>>
>>>> Here is the runtime part of the bug fix that calls the new 
>>>> free_archive_regions() when shared string mapping fails. I also 
>>>> added a jtreg test to test shared strings with -Xshare:auto.
>>>>
>>>> http://cr.openjdk.java.net/~jiangli/8131734/webrev.00/ 
>>>> <http://cr.openjdk.java.net/%7Ejiangli/8131734/webrev.00/>
>>>>
>>>> Test:
>>>> - Tested by explicitly making the shared string mapping fail on 
>>>> linux-x64, -Xshare:auto runs without crash with the fix
>>>> - Tested with the new SharedStringsRunAuto test
>>>> - Tested with XX:+PrintNMTStatistics -XX:NativeMemoryTracking=detail
>>>>
>>>> Thanks,
>>>> Jiangli
>>>
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150807/89adf47a/attachment.htm>

From thomas.schatzl at oracle.com  Fri Aug  7 12:05:02 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Fri, 07 Aug 2015 14:05:02 +0200
Subject: RFR (M): 8040162: Avoid reallocating PLABs between GC phases in G1
Message-ID: <1438949102.2305.42.camel@oracle.com>

Hi all,

  can I have reviews for this significant improvement in memory use
during GC by changing how G1 manages G1ParScanThreadState instances?

So until now G1 uses separate G1ParScanThreadState instances with their
separate PLABs for every phase of the GC. This means that the
G1ParScanThreadState/PLAB is thrown away after every phase. This means
that even PLABs which only have a single object in it, but are
potentially hundreds of kB in size are filled with dummy objects and
reloaded.

This change instantiates a single G1ParScanThreadState for the entire
GC. PLABs are reused as long as possible. This decreases memory use
during GC significantly in some cases, which can significantly reduce
the number of GCs, improves throughput, and so on.

Care has been taken in this change to not change the values for the
termination statistics during gc, so a few members of
G1ParScanThreadState were relocated (making it a lot nicer too imo).

There is also some bug where the age table and the per-region survivor
statistics are only taken from the initial parallel collection phase
into account.

This will be fixed as JDK-8073146, but if wanted I can merge them.

This change is based on the change for JDK-8073013 which is currently
out for review.

CR:
https://bugs.openjdk.java.net/browse/JDK-8040162
Webrev:
http://cr.openjdk.java.net/~tschatzl/8040162/webrev/
Testing:
lots of runs, jprt, a few aurora nightly runs

Thanks,
  Thomas




From thomas.schatzl at oracle.com  Fri Aug  7 12:05:16 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Fri, 07 Aug 2015 14:05:16 +0200
Subject: RFR (S): 8073146: G1 merges thread local age tables too early with
	global age table
Message-ID: <1438949116.2305.43.camel@oracle.com>

Hi,

  here is the mentioned fix for 8073146 which resolves a day-one issue
with age table statistics and per-region survivor statistics.

Basically G1 only ever merged the age table/region survivor statistics
at the end of the first parallel phase, forgetting any objects copied
later.

This is based on the change for 8040162 that is currently out for
review.

CR:
https://bugs.openjdk.java.net/browse/JDK-8073146
Webrev:
http://cr.openjdk.java.net/~tschatzl/8073146/webrev/
Testing:
jprt, lots of testing with various benchmarks

Thanks,
  Thomas




From bengt.rutisson at oracle.com  Fri Aug  7 12:17:34 2015
From: bengt.rutisson at oracle.com (Bengt Rutisson)
Date: Fri, 07 Aug 2015 14:17:34 +0200
Subject: RFR (S): 8003237: G1: Reduce unnecessary (and failing) allocation
	attempts when handling an evacuation failure
In-Reply-To: <1438871940.2474.38.camel@oracle.com>
References: <1438605320.2292.47.camel@oracle.com>	<1438605827.2292.49.camel@oracle.com>	<1438763108.2378.9.camel@oracle.com>
	<55C23B83.90405@oracle.com> <1438871940.2474.38.camel@oracle.com>
Message-ID: <55C4A1DE.4040500@oracle.com>


Hi Thomas,

On 2015-08-06 16:39, Thomas Schatzl wrote:
> (sorry if that mail is received multiple times, getting send error just
> for this mail)
>
> Hi Jon,
>
>    thanks for the review...
>
> On Wed, 2015-08-05 at 09:36 -0700, Jon Masamitsu wrote:
>> Thomas,
>>
>> I think that this method gets called if a new region is allocated
>> via allocate_inline_or_new_plab()
>>
>> "src/share/vm/gc/g1/g1CollectedHeap.cpp"
>>
>> 6436 HeapRegion* G1CollectedHeap::new_gc_alloc_region(size_t
> word_size,
>> 6437                                                  uint count,
>> 6438                                                  InCSetState
> dest) {
>> 6439   assert(FreeList_lock->owned_by_self(), "pre-condition");
>> 6440
>> 6441   if (count < g1_policy()->max_regions(dest)) {
>>
>> Might the g1_policy()->max_regions() conflict with the test
>> in has_more_free_regions() which considers all free regions
>> (even uncommitted).   That is, has_more_free_regions() will
>> allow allocation attempts that will not success because of
>> max_regions().
> No, because the worst case is that we go into the take-the-Mutex case
> too often (which is bad).
>
> However, thinking about this again, I found another option (which is
> much clearer to see after recent refactoring of g1allocator classes): if
> G1AllocRegion::attempt_allocation_locked() fails, we know that there is
> really no more space in the heap left.
>
> In this case, just set an internal flag, instead of relying on the
> has_more_free_regions() from G1CollectedHeap.
>
>> http://cr.openjdk.java.net/~tschatzl/8003237/webrev/src/share/vm/gc/g1/g1ParScanThreadState.hpp.frames.html
>>
>>>     74   // Indicates whether in the last generation (old) there is
> no more space
>>>     75   // available for allocation.
>>>     76   bool _no_more_space_in_old;
>> Would a name like _old_is_full suit you here in place of
>> _no_more_space_in_old?  I like positives instead
>> negatives.
> Done.
>
> New webrevs:
> http://cr.openjdk.java.net/~tschatzl/8003237/webrev.1 (full)
> http://cr.openjdk.java.net/~tschatzl/8003237/webrev.0_to_1 (partial)
>
> I think this one looks better, performs the same (tested) and does not
> introduce the additional dependency on G1CollectedHeap.
>
> This also automatically takes into consideration restrictions like in
> G1CollectedHeap::new_gc_alloc_region() without the need to repeat them.
>
> The addition of the new methods as virtual has been intentional as other
> Allocators may have different views on when the heap for the given
> context is full.


This looks good to me. A couple of minor comments:

What do you think of splitting 
G1PLABAllocator::allocate_inline_or_new_plab() up into two methods? One 
that allocates in plab and one that does the inline allocation. That way 
G1ParScanThreadState::copy_to_survivor_space() does not have to pass 
around the plab_refill_failed state. Instead it just first tries PLAB 
and then inline and it knows what happened and can pass the correct 
value to allocate_in_next_plab().

Would you mind calling _last_gen_is_full _old_gen_is_full instead? I 
think it is more clear and almost everywhere where _last_gen_is_full is 
being used there is a comment saying "old gen".

I'm on vacation next week, but I am fine with you push the changes with 
me as a reviewer if the changes contain more or less what you have now 
plus potentiality the updates I proposed.

Thanks,
Bengt

>
> Thanks,
>    Thomas
>



From thomas.schatzl at oracle.com  Fri Aug  7 12:42:30 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Fri, 07 Aug 2015 14:42:30 +0200
Subject: Fwd: Re: RFR (XXS): 8132334: [TESTBUG]
	gc/metaspace/TestCapacityUntilGCWrapAround.java Compilation failed
In-Reply-To: <55B9F545.7000300@oracle.com>
References: <55B7C4F6.9000100@oracle.com> <55B9F545.7000300@oracle.com>
Message-ID: <1438951350.10875.6.camel@oracle.com>

Hi,

On Thu, 2015-07-30 at 12:58 +0300, Dmitry Fazunenko wrote:
> Hi everyone, 
> 
> A trivial fix is still waiting for reviewers...
> 
> Bug: https://bugs.openjdk.java.net/browse/JDK-8132334 
> Webrev:  http://cr.openjdk.java.net/~dfazunen/8132334/webrev/ 
> 
> Comments: w/a  for known jtreg issue described:
> https://bugs.openjdk.java.net/browse/CODETOOLS-7900024 

  the workaround looks good to me.

Thanks,
  Thomas





From jesper.wilhelmsson at oracle.com  Fri Aug  7 13:03:58 2015
From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson)
Date: Fri, 07 Aug 2015 15:03:58 +0200
Subject: RFR (XXS): 8133042: Remove some direct accesses of
	G1Allocator::reuse_retained_old_region() to G1CollectedHeap members
In-Reply-To: <1438876680.2474.55.camel@oracle.com>
References: <1438775999.2378.32.camel@oracle.com>	<FD29FE78-EB89-4630-A5E9-16DECC9B8846@oracle.com>	<1438872649.2474.40.camel@oracle.com>	<D5B26462-A6D3-4446-ABC2-54B3C4168713@oracle.com>
	<1438876680.2474.55.camel@oracle.com>
Message-ID: <55C4ACBE.8030307@oracle.com>

Still looks good!
/Jesper

Thomas Schatzl skrev den 6/8/15 17:58:
> Hi Kim,
>
>    thanks for the review.
>
> On Thu, 2015-08-06 at 11:24 -0400, Kim Barrett wrote:
>> On Aug 6, 2015, at 10:50 AM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
>>>
>>>> There is very similar code to what is being changed here in G1ArchiveAllocator::alloc_new_region,
>>>> later in the same file (g1Allocator.cpp).
>>>>
>>>> Would the proposed change or the additional change to alloc_new_region permit the elimination
>>>> of any friend declarations from G1CollectedHeap?
>>>
>>>   only if some additional methods from G1CollectedHeap were made public.
>>>
>>> Here is a webrev doing that:
>>> http://cr.openjdk.java.net/~tschatzl/8133042/webrev.1/ (full only
>>> because the original change has been so tiny)
>>>
>>> I am fine with either version.
>>
>> Looks good.
>>
>> One tiny nit:
>
>> src/share/vm/gc/g1/g1CollectedHeap.hpp
>>   700   // It dirties the cards that cover the block so that so that the post
>>
>> [pre-existing]
>> "so that so that" => "so that"
>
>    I allowed myselves to just update the webrevs in place for this tiny
> change.
>
> http://cr.openjdk.java.net/~tschatzl/8133042/webrev.1/
>
> Thanks,
>    Thomas
>
>
>


From thomas.schatzl at oracle.com  Fri Aug  7 13:48:16 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Fri, 07 Aug 2015 15:48:16 +0200
Subject: RFR (XXS): 8133042: Remove some direct accesses of
	G1Allocator::reuse_retained_old_region() to G1CollectedHeap members
In-Reply-To: <55C4ACBE.8030307@oracle.com>
References: <1438775999.2378.32.camel@oracle.com>
	<FD29FE78-EB89-4630-A5E9-16DECC9B8846@oracle.com>
	<1438872649.2474.40.camel@oracle.com>
	<D5B26462-A6D3-4446-ABC2-54B3C4168713@oracle.com>
	<1438876680.2474.55.camel@oracle.com> <55C4ACBE.8030307@oracle.com>
Message-ID: <1438955296.1967.0.camel@oracle.com>

Hi Jesper,

On Fri, 2015-08-07 at 15:03 +0200, Jesper Wilhelmsson wrote:
> Still looks good!
> /Jesper

  thanks a lot.

Thomas



From tom.benson at oracle.com  Fri Aug  7 13:52:06 2015
From: tom.benson at oracle.com (Tom Benson)
Date: Fri, 07 Aug 2015 09:52:06 -0400
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <30B0248C-3BEA-4FAE-861F-DAAE19F56B45@oracle.com>
References: <55C103A4.1060505@oracle.com>	<A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>	<1438781844.2378.60.camel@oracle.com>
	<55C21DC3.5090305@oracle.com>	<1438785532.2378.73.camel@oracle.com>
	<55C248FF.6010308@oracle.com>	<F73C4B9A-4DBA-47A8-8715-0988FF6D393D@oracle.com>	<55C26249.1090902@oracle.com>	<C149705F-696B-46CD-8FB8-CAD06C511244@oracle.com>	<55C2751B.8070400@oracle.com>
	<1438847329.2009.9.camel@oracle.com>	<55C36B31.7000908@oracle.com>
	<1438871723.2474.37.camel@oracle.com>	<55C373F6.9060609@oracle.com>	<AD45178E-A442-4A0D-80D6-50E91608E1FE@oracle.com>	<55C3D8BB.9000603@oracle.com>	<48C0E571-0D7B-4D30-85DF-E31D99092A7C@oracle.com>
	<30B0248C-3BEA-4FAE-861F-DAAE19F56B45@oracle.com>
Message-ID: <55C4B806.7050504@oracle.com>

Hi,
On 8/6/2015 8:40 PM, Jiangli Zhou wrote:
> Hi Dmitry,
>
> I?ve added shared string unmapping to handle the string data verification failure. I also added 'free_archive_regions()? call to FileMapInfo::unmap_string_regions(). My testing revealed some issues with the new archive region free code when I forced string verification failure. I?m working with Tom on the issues.

The free_archive_regions code was intended to handle the failed mmap 
case, and thus the regions would have been empty/unused.  So perhaps not 
surprising this requires a small enhancement.
Tom


From igor.ignatyev at oracle.com  Fri Aug  7 13:52:16 2015
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Fri, 07 Aug 2015 16:52:16 +0300
Subject: RFR: 8081317: [NEWTEST] documented GC ratio tuning and new size
	options should be covered by regression tests
In-Reply-To: <55C0CD95.1080305@oracle.com>
References: <55BBB0BE.6080506@oracle.com> <55C0CD95.1080305@oracle.com>
Message-ID: <55C4B810.8020209@oracle.com>

Hi Michail,

looks good to me.

Igor

On 08/04/2015 05:35 PM, Michail Chernov wrote:
> Hi,
>
> Could someone please take a look at this review?
>
> Thanks,
> Michail
>
> On 31.07.2015 20:30, Michail Chernov wrote:
>> Hi,
>>
>> Please review these changes for JDK-8081317. There are new tests to
>> cover GC ratio tuning and new size options.
>>
>> Webrev:  http://cr.openjdk.java.net/~mchernov/8081317/
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8081317
>>
>> TestMaxMinHeapFreeRatioFlags.java - Verify that heap size changes
>> according to max and min heap free ratios.
>>
>> TestMinAndInitialSurvivorRatioFlags.java - Test verifies that VM can
>> start with any GC when MinSurvivorRatio and InitialSurvivorRatio flags
>> passed and for Parallel GC it verifies that after start up survivor
>> ratio is equal to InitialSurvivorRatio value and that actual survivor
>> ratio will never be less than MinSurvivorRatio.
>>
>> TestNewRatioFlag.java - Verify that heap divided among generations
>> according to NewRatio.
>>
>> TestNewSizeFlags.java - Verify that young gen size conforms values
>> specified by NewSize, MaxNewSize and Xmn options.
>>
>> TestSurvivorRatioFlag - Verify that actual survivor ratio is equal to
>> specified SurvivorRatio value.
>>
>> TestTargetSurvivorRatioFlag.java - Test if objects size is less than
>> (survivor_size * TargetSurvivorRatio / 100) then objects are stayed in
>> survivor space until MaxTenuringThreshold minor GC cycles. If more
>> than (survivor_size * TargetSurvivorRatio / 100) objects were
>> allocated, then test verifies that after MaxTenuringThreshold minor GC
>> cycles survivor space is almost empty.
>>
>> New changes were tested on different platforms including embedded jdk
>> on Linux x86.
>>
>> Thanks,
>> Michail
>>
>


From tom.benson at oracle.com  Fri Aug  7 14:56:32 2015
From: tom.benson at oracle.com (Tom Benson)
Date: Fri, 07 Aug 2015 10:56:32 -0400
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <55C4B806.7050504@oracle.com>
References: <55C103A4.1060505@oracle.com>	<A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>	<1438781844.2378.60.camel@oracle.com>	<55C21DC3.5090305@oracle.com>	<1438785532.2378.73.camel@oracle.com>	<55C248FF.6010308@oracle.com>	<F73C4B9A-4DBA-47A8-8715-0988FF6D393D@oracle.com>	<55C26249.1090902@oracle.com>	<C149705F-696B-46CD-8FB8-CAD06C511244@oracle.com>	<55C2751B.8070400@oracle.com>	<1438847329.2009.9.camel@oracle.com>	<55C36B31.7000908@oracle.com>	<1438871723.2474.37.camel@oracle.com>	<55C373F6.9060609@oracle.com>	<AD45178E-A442-4A0D-80D6-50E91608E1FE@oracle.com>	<55C3D8BB.9000603@oracle.com>	<48C0E571-0D7B-4D30-85DF-E31D99092A7C@oracle.com>	<30B0248C-3BEA-4FAE-861F-DAAE19F56B45@oracle.com>
	<55C4B806.7050504@oracle.com>
Message-ID: <55C4C720.5030903@oracle.com>

Hi,

On 8/7/2015 9:52 AM, Tom Benson wrote:
> Hi,
> On 8/6/2015 8:40 PM, Jiangli Zhou wrote:
>> Hi Dmitry,
>>
>> I?ve added shared string unmapping to handle the string data 
>> verification failure. I also added 'free_archive_regions()? call to 
>> FileMapInfo::unmap_string_regions(). My testing revealed some issues 
>> with the new archive region free code when I forced string 
>> verification failure. I?m working with Tom on the issues.
>

The problem is simply that in addition to calling free_archive_regions, 
FileMapInfo::unmap_string_regions also unmaps the memory.... so there's 
a segv when GC tries to re-use it. Let's talk directly about the best 
way to handle it.
Tom


From jiangli.zhou at oracle.com  Fri Aug  7 15:55:53 2015
From: jiangli.zhou at oracle.com (Jiangli Zhou)
Date: Fri, 7 Aug 2015 08:55:53 -0700
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <55C46529.4030601@oracle.com>
References: <55C103A4.1060505@oracle.com>
	<A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>
	<1438781844.2378.60.camel@oracle.com> <55C21DC3.5090305@oracle.com>
	<1438785532.2378.73.camel@oracle.com> <55C248FF.6010308@oracle.com>
	<F73C4B9A-4DBA-47A8-8715-0988FF6D393D@oracle.com>
	<55C26249.1090902@oracle.com>
	<C149705F-696B-46CD-8FB8-CAD06C511244@oracle.com>
	<55C2751B.8070400@oracle.com> <1438847329.2009.9.camel@oracle.com>
	<55C36B31.7000908@oracle.com> <1438871723.2474.37.camel@oracle.com>
	<55C373F6.9060609@oracle.com>
	<AD45178E-A442-4A0D-80D6-50E91608E1FE@oracle.com>
	<55C3D8BB.9000603@oracle.com>
	<48C0E571-0D7B-4D30-85DF-E31D99092A7C@oracle.com>
	<30B0248C-3BEA-4FAE-861F-DAAE19F56B45@oracle.com>
	<55C46529.4030601@oracle.com>
Message-ID: <21CC8572-5BFF-47AD-ADA9-CCE1F7762EDE@oracle.com>


On Aug 7, 2015, at 12:58 AM, Dmitry Dmitriev <dmitry.dmitriev at oracle.com> wrote:

> Hello Jiangli,
> 
> Thank you. About extra spaces - I just highlight changed lines in webrev and see that some lines have additional spaces at the end. :)

I see. :) Thanks, Dmitry.

Jiangli

> 
> Dmitry
> 
> On 07.08.2015 3:40, Jiangli Zhou wrote:
>> Hi Dmitry,
>> 
>> I?ve added shared string unmapping to handle the string data verification failure. I also added 'free_archive_regions()? call to FileMapInfo::unmap_string_regions(). My testing revealed some issues with the new archive region free code when I forced string verification failure. I?m working with Tom on the issues.
>> 
>> Thanks,
>> Jiangli
>> 
>> 
>> On Aug 6, 2015, at 3:19 PM, Jiangli Zhou <jiangli.zhou at oracle.com> wrote:
>> 
>>> Hi Dmitry,
>>> 
>>> Thank you for the feedback! You are definitely right about freeing the archived regions in those failure cases. I should have thought it more thoroughly, instead of just fixing the case that I ran into. Will exam all cases.
>>> 
>>> I?ll change the test to incorporate your suggestion. Will fix the extra spaces before committing. I have one question, how do you detect the extra spaces from the webrev? :)
>>> 
>>> Thanks,
>>> Jiangli
>>> 
>>> On Aug 6, 2015, at 2:59 PM, Dmitry Dmitriev <dmitry.dmitriev at oracle.com> wrote:
>>> 
>>>> Hello Jiangli,
>>>> 
>>>> I have few comments/questions.
>>>> 
>>>> src/share/vm/memory/filemap.cpp module:
>>>> 1) Should free_archive_regions also called when verify_string_regions() returns false on line 717?
>>>> 2) The same question about unmap_string_regions(). Region must be freed when unmap_string_regions() is called?
>>>> 3) Extra space at the end of the line 711.
>>>> 
>>>> test/runtime/SharedArchiveFile/SharedStringsRunAuto.java
>>>> 1) Unneeded second creating of OutputAnalyzer on line 61. Also, probably will be better to use same scheme for OutputAnalyzer? On lines 46-49 you not use local variable, but on lines 61-63 use local variable.
>>>>  59         OutputAnalyzer output = new OutputAnalyzer(pb.start());
>>>>  60 
>>>>  61         output = new OutputAnalyzer(pb.start());
>>>> 2) Extra space at the end of the lines 25,26,30, 51
>>>> 
>>>> Thanks,
>>>> Dmitry
>>>> 
>>>> On 07.08.2015 0:32, Jiangli Zhou wrote:
>>>>> Hi,
>>>>> 
>>>>> Here is the runtime part of the bug fix that calls the new free_archive_regions() when shared string mapping fails. I also added a jtreg test to test shared strings                 with -Xshare:auto.
>>>>> 
>>>>>  http://cr.openjdk.java.net/~jiangli/8131734/webrev.00/
>>>>> 
>>>>> Test:
>>>>> - Tested by explicitly making the shared string mapping fail on linux-x64, -Xshare:auto runs without crash with the fix
>>>>> - Tested with the new SharedStringsRunAuto test
>>>>> - Tested with XX:+PrintNMTStatistics -XX:NativeMemoryTracking=detail
>>>>> 
>>>>> Thanks,
>>>>> Jiangli
>>>> 
>>> 
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150807/bdf9e066/attachment.htm>

From michail.chernov at oracle.com  Fri Aug  7 17:34:06 2015
From: michail.chernov at oracle.com (Michail Chernov)
Date: Fri, 7 Aug 2015 20:34:06 +0300
Subject: RFR: 8081317: [NEWTEST] documented GC ratio tuning and new size
	options should be covered by regression tests
In-Reply-To: <55C4B810.8020209@oracle.com>
References: <55BBB0BE.6080506@oracle.com> <55C0CD95.1080305@oracle.com>
	<55C4B810.8020209@oracle.com>
Message-ID: <55C4EC0E.3060705@oracle.com>

Hi Igor,

Thanks a lot!

Michail


On 07.08.2015 16:52, Igor Ignatyev wrote:
> Hi Michail,
>
> looks good to me.
>
> Igor
>
> On 08/04/2015 05:35 PM, Michail Chernov wrote:
>> Hi,
>>
>> Could someone please take a look at this review?
>>
>> Thanks,
>> Michail
>>
>> On 31.07.2015 20:30, Michail Chernov wrote:
>>> Hi,
>>>
>>> Please review these changes for JDK-8081317. There are new tests to
>>> cover GC ratio tuning and new size options.
>>>
>>> Webrev:  http://cr.openjdk.java.net/~mchernov/8081317/
>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8081317
>>>
>>> TestMaxMinHeapFreeRatioFlags.java - Verify that heap size changes
>>> according to max and min heap free ratios.
>>>
>>> TestMinAndInitialSurvivorRatioFlags.java - Test verifies that VM can
>>> start with any GC when MinSurvivorRatio and InitialSurvivorRatio flags
>>> passed and for Parallel GC it verifies that after start up survivor
>>> ratio is equal to InitialSurvivorRatio value and that actual survivor
>>> ratio will never be less than MinSurvivorRatio.
>>>
>>> TestNewRatioFlag.java - Verify that heap divided among generations
>>> according to NewRatio.
>>>
>>> TestNewSizeFlags.java - Verify that young gen size conforms values
>>> specified by NewSize, MaxNewSize and Xmn options.
>>>
>>> TestSurvivorRatioFlag - Verify that actual survivor ratio is equal to
>>> specified SurvivorRatio value.
>>>
>>> TestTargetSurvivorRatioFlag.java - Test if objects size is less than
>>> (survivor_size * TargetSurvivorRatio / 100) then objects are stayed in
>>> survivor space until MaxTenuringThreshold minor GC cycles. If more
>>> than (survivor_size * TargetSurvivorRatio / 100) objects were
>>> allocated, then test verifies that after MaxTenuringThreshold minor GC
>>> cycles survivor space is almost empty.
>>>
>>> New changes were tested on different platforms including embedded jdk
>>> on Linux x86.
>>>
>>> Thanks,
>>> Michail
>>>
>>
>



From jon.masamitsu at oracle.com  Fri Aug  7 18:29:29 2015
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Fri, 07 Aug 2015 11:29:29 -0700
Subject: RFR (L): 8073013: Add detailed information about PLAB memory usage
In-Reply-To: <1438876151.2474.51.camel@oracle.com>
References: <1438876151.2474.51.camel@oracle.com>
Message-ID: <55C4F909.7050601@oracle.com>

Thomas,

Basically looks good.  Some questions around the edges.

http://cr.openjdk.java.net/~tschatzl/8073013/webrev/src/share/vm/gc/g1/g1CollectedHeap.hpp.frames.html

>   248   // Manages all kinds of allocations within regions. This excludes only
>   249   // humongous object allocations.
>   250   G1Allocator* _allocator;

Would this comment be more exact?

// Manages all allocations with regions except humongous object allocations.

http://cr.openjdk.java.net/~tschatzl/8073013/webrev/src/share/vm/gc/g1/g1EvacStats.cpp.html

   85 void G1EvacStats::send_obj_copy_mem_event(int for_gen) {
   86   EventGCG1EvacuationMemoryStatistics e;
   87   if (e.should_commit()) {
   88     e.set_gcId(GCId::peek().id());
   89     e.set_gen(for_gen);

Can the conversion of  InCSetState values to generation numbers be put into
the send_obj_copy_mem_event()?

http://cr.openjdk.java.net/~tschatzl/8073013/webrev/src/share/vm/trace/trace.xml.frames.html

  318       <value type="UINT" field="gen" label="Generation" description="Generation these PLAB statistics are for"/>


Can the "gen" field be a character string with "young" or "old" instead 
of a number?

http://cr.openjdk.java.net/~tschatzl/8073013/webrev/src/share/vm/gc/g1/g1EvacStats.hpp.html

Are these alternative comments correct?   Perhaps better for the less G1 
knowledgeable?

   38   // Number of words allocated during evacuation failure in the regions that failed evacuation.

// Number of words in live objects remaining in regions that ultimately suffered an
// evacuation failure.  This is used space in the regions when the regions are made old regions.


   39   size_t _failure_used;

   40   // Number of words wasted during evacuation failure in the regions that failed evacuation.

// Number of words wasted in regions which failed evacuation.  This is the sum of space for
// objects successfully copied out of the regions (now dead space) plus waste at the end of
// regions.

   41   size_t _failure_waste;


Not sure how ingrained the term "inline" allocation, but why "inline"?  It means that
the allocation was directly into the region and not in a LAB in the region, right?
Would _directly_allocated or _direct_allocations work for you?

>    36   size_t _inline_allocated; // Number of words allocated directly into the regions.

That's all.

Jon



On 8/6/2015 8:49 AM, Thomas Schatzl wrote:
> Hi all,
>
>    can I have reviews for this change that prepares for a lot of changes
> related to 8030849 "Investigate high fragmentation/waste in some
> situations during allocation during GC in G1"?
>
> Before investigating and fixing fragmentation during PLAB, we need to
> track that information.
>
> For this reason, PLABStats has been extended in a G1EvacStats class with
> the necessary information that is needed to do better PLAB sizing, in
> particular G1EvacStats::_region_end_waste that tracks the waste at the
> end of regions during refill, G1EvacStats::_regions_filled that tracks
> the number of region refills, G1EvacStats::_inline_allocated that tracks
> the amount of inline allocation (excluding PLAB allocation),
> G1EvacStats::_failure_used and G1EvacStats::_failure_waste that track
> memory usage in regions where evacuation failure occurred.
>
> There had to be somewhat extensive changes particularly to get the
> _region_end_waste in the allocation classes, which is what makes this
> change a bit large.
>
> Overall there is no change in the sizing policy except for accounting
> _region_end_waste.
>
> As you may notice, there is a new JFR event, but no real additional
> logging for the log file (the -XX:+PrintPLAB message has been extended
> though). That will be added in an extra CR, as soon as the new logging
> framework has been added.
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8073013
>
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8073013/webrev
>
> Testing:
> jprt, a few aurora runs, local testing, lots of benchmarks mainly in
> conjunction with the following changes.
>
> Thanks,
>    Thomas
>
>



From abhijangda at hotmail.com  Sat Aug  8 16:50:59 2015
From: abhijangda at hotmail.com (Abhinav Jangda)
Date: Sat, 8 Aug 2015 22:20:59 +0530
Subject: Collection of DefNewGeneration
Message-ID: <BLU184-W650DC0BF5226F685A6179ECA720@phx.gbl>

Hello everyone,

I was trying to understand how serial generational gc works in Hotspot. I was able to understand how TenuredGeneration is marked using different OopClosures. I can see how beautifully the concept of Visitor Design Pattern has been used.
But I am having a slight difficulty in understanding how DefNewGeneration works.
I do understand that DefNewGeneration::collect method is called which then calls GenCollectedHeap::gen_process_strong_roots. This is same as the TenuredGeneration::collect calls GenCollectedHeap::gen_process_strong_roots but in both of these cases the OopClosure are different. In TenuredGeneration::collect, FollowRootClosure (There are 3 closures but let us focus on only one of them) is used and in   DefNewGeneration::collect, FastScanClosure is used. Well, the problem starts here. 
In FollowRootClosure::do_oop method, the oop to be traversed is pushed on the marking stack which then is able to follow the contents of this oop. In FastScanClosure there is no such thing. In FastScanClosure::do_oop_work method, the oop is first checked whether it is in this generation and only if it is then oop is transferred to another space/generation. oop's fields are not traversed like they are in FollowRootClosure. Let us suppose the root object is not in the DefNewGeneration but its fields are. So, in this case the fields wouldn't be traversed? 
How, the fields of root objects are traversed in the case of FastScanClosure? If they are traversed by some other way and FastScanClosure use the result of it, then please tell me that.
Please help me, a small function name or hint should be enough for me.

Thank You,

Abhinav
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150808/d69cffbb/attachment.htm>

From jon.masamitsu at oracle.com  Mon Aug 10 03:03:06 2015
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Sun, 09 Aug 2015 20:03:06 -0700
Subject: Collection of DefNewGeneration
In-Reply-To: <BLU184-W650DC0BF5226F685A6179ECA720@phx.gbl>
References: <BLU184-W650DC0BF5226F685A6179ECA720@phx.gbl>
Message-ID: <55C8146A.1030009@oracle.com>

Look at the code in GenCollectedHeap::gen_process_roots()

  704     older_gens->set_generation(_old_gen);
  705     rem_set()->younger_refs_iterate(_old_gen, older_gens, 
scope->n_threads(     ));
  706     older_gens->reset_generation();

The defNew collect() copies live objects that it finds in the defNew 
(young) generation.
For objects in the young gen that are pointed to from the old 
generation, collect()
uses a remembered set (in the form of a card table).  The card table 
tells the defNew
collect() where to look for objects in the young generation referenced 
from the old
generation.

Jon

On 8/8/2015 9:50 AM, Abhinav Jangda wrote:
> Hello everyone,
>
> I was trying to understand how serial generational gc works in 
> Hotspot. I was able to understand how TenuredGeneration is marked 
> using different OopClosures. I can see how beautifully the concept of 
> Visitor Design Pattern has been used.
> But I am having a slight difficulty in understanding how 
> DefNewGeneration works.
> I do understand that DefNewGeneration::collect method is called which 
> then calls GenCollectedHeap::gen_process_strong_roots. This is same as 
> the TenuredGeneration::collect calls 
> GenCollectedHeap::gen_process_strong_roots but in both of these cases 
> the OopClosure are different. In TenuredGeneration::collect, 
> FollowRootClosure (There are 3 closures but let us focus on only one 
> of them) is used and in DefNewGeneration::collect, FastScanClosure is 
> used. Well, the problem starts here.
> In FollowRootClosure::do_oop method, the oop to be traversed is pushed 
> on the marking stack which then is able to follow the contents of this 
> oop. In FastScanClosure there is no such thing. In 
> FastScanClosure::do_oop_work method, the oop is first checked whether 
> it is in this generation and only if it is then oop is transferred to 
> another space/generation. oop's fields are not traversed like they are 
> in FollowRootClosure. Let us suppose the root object is not in the 
> DefNewGeneration but its fields are. So, in this case the fields 
> wouldn't be traversed?
> How, the fields of root objects are traversed in the case of 
> FastScanClosure? If they are traversed by some other way and 
> FastScanClosure use the result of it, then please tell me that.
> Please help me, a small function name or hint should be enough for me.
>
> Thank You,
>
> Abhinav

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150809/e8381adc/attachment.htm>

From thomas.schatzl at oracle.com  Mon Aug 10 10:15:57 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 10 Aug 2015 12:15:57 +0200
Subject: RFR (S): 8003237: G1: Reduce unnecessary (and failing)
	allocation attempts when handling an evacuation failure
In-Reply-To: <55C4A1DE.4040500@oracle.com>
References: <1438605320.2292.47.camel@oracle.com>
	<1438605827.2292.49.camel@oracle.com>	<1438763108.2378.9.camel@oracle.com>
	<55C23B83.90405@oracle.com> <1438871940.2474.38.camel@oracle.com>
	<55C4A1DE.4040500@oracle.com>
Message-ID: <1439201757.1970.13.camel@oracle.com>

Hi Bengt,

  thanks for the review :)

On Fri, 2015-08-07 at 14:17 +0200, Bengt Rutisson wrote:
> Hi Thomas,
> 
> On 2015-08-06 16:39, Thomas Schatzl wrote:
[...]
> 
> 
> This looks good to me. A couple of minor comments:
> 
> What do you think of splitting 
> G1PLABAllocator::allocate_inline_or_new_plab() up into two methods? One 
> that allocates in plab and one that does the inline allocation. That way 
> G1ParScanThreadState::copy_to_survivor_space() does not have to pass 
> around the plab_refill_failed state. Instead it just first tries PLAB 
> and then inline and it knows what happened and can pass the correct 
> value to allocate_in_next_plab().

The problem is that all callers of the method need the parameter, and
need to first check whether it should throw away the current plab and
then try direct allocation.
So a split here would require the change to duplicate its code in all
its callers, which seems against coding practices to me. I do think
there is an opportunity here to refactor the method into its simpler
parts.

Since I am going to revisit that method in JDK-8067339 I would like to
defer this change there. This would also give me less headache applying
that change :)

> 
> Would you mind calling _last_gen_is_full _old_gen_is_full instead? I 
> think it is more clear and almost everywhere where _last_gen_is_full is 
> being used there is a comment saying "old gen".

Done.

I also undid the rename of the method
G1PLABAllocator::allocate_inline_or_new_plab() to its original name
G1PLABAllocator::allocate_direct_or_new_plab() to meet Jon's comment
about direct vs. inline naming the review for JDK-8073013.

An additional "const" that has been required for the now-removed
has_more_free_regions() method in in the original patch in
heapRegionSet.hpp. I removed that change too.

New changes at:
http://cr.openjdk.java.net/~tschatzl/8003237/webrev.2/ (full)
http://cr.openjdk.java.net/~tschatzl/8003237/webrev.1_to_2/ (diff)

Thanks,
  Thomas




From thomas.schatzl at oracle.com  Mon Aug 10 13:01:47 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 10 Aug 2015 15:01:47 +0200
Subject: RFR (L): 8073013: Add detailed information about PLAB memory usage
In-Reply-To: <55C4F909.7050601@oracle.com>
References: <1438876151.2474.51.camel@oracle.com> <55C4F909.7050601@oracle.com>
Message-ID: <1439211707.2330.11.camel@oracle.com>

Hi Jon,

  thanks for the review. Comments inline.

On Fri, 2015-08-07 at 11:29 -0700, Jon Masamitsu wrote:
> Thomas,
> 
> Basically looks good.  Some questions around the edges.
> 
> http://cr.openjdk.java.net/~tschatzl/8073013/webrev/src/share/vm/gc/g1/g1CollectedHeap.hpp.frames.html
> 
> >   248   // Manages all kinds of allocations within regions. This excludes only
> >   249   // humongous object allocations.
> >   250   G1Allocator* _allocator;
> 
> Would this comment be more exact?
> 
> // Manages all allocations with regions except humongous object allocations.
> 

Done.

> http://cr.openjdk.java.net/~tschatzl/8073013/webrev/src/share/vm/gc/g1/g1EvacStats.cpp.html
> 
>    85 void G1EvacStats::send_obj_copy_mem_event(int for_gen) {
>    86   EventGCG1EvacuationMemoryStatistics e;
>    87   if (e.should_commit()) {
>    88     e.set_gcId(GCId::peek().id());
>    89     e.set_gen(for_gen);
> 
> Can the conversion of  InCSetState values to generation numbers be put into
> the send_obj_copy_mem_event()?
> 
> http://cr.openjdk.java.net/~tschatzl/8073013/webrev/src/share/vm/trace/trace.xml.frames.html
> 
>   318       <value type="UINT" field="gen" label="Generation" description="Generation these PLAB statistics are for"/>
> 
> 
> Can the "gen" field be a character string with "young" or "old" instead 
> of a number?

Did so in the new change. This obviates the need for a dedicated
conversion function, but adds a function to get the string.

Another option would be to have two separate events, not sure if this is
better.

> 
> http://cr.openjdk.java.net/~tschatzl/8073013/webrev/src/share/vm/gc/g1/g1EvacStats.hpp.html
> 
> Are these alternative comments correct?   Perhaps better for the less G1 
> knowledgeable?
> 
>    38   // Number of words allocated during evacuation failure in the regions that failed evacuation.
> 
> // Number of words in live objects remaining in regions that ultimately suffered an
> // evacuation failure.  This is used space in the regions when the regions are made old regions.
> 
> 
>    39   size_t _failure_used;
> 
>    40   // Number of words wasted during evacuation failure in the regions that failed evacuation.
> 
> // Number of words wasted in regions which failed evacuation.  This is the sum of space for
> // objects successfully copied out of the regions (now dead space) plus waste at the end of
> // regions.

Done.

> 
>    41   size_t _failure_waste;
> 
> 
> Not sure how ingrained the term "inline" allocation, but why "inline"?  It means that
> the allocation was directly into the region and not in a LAB in the region, right?
> Would _directly_allocated or _direct_allocations work for you?

Would be fine with me. In the most recent change for 8003237 I also
undid the renaming of that method from "inline" to "direct" so that
everything fits again.

I hope this makes the statistic more self-explaining, or at least less
confusing with other uses of "inline allocation".

I.e. afaik the term is used for allocation outside of PLAB using
additional inlined code.

Interestingly existing PLAB jfr events do not use the term inline
allocation either, the events have the suffix "OutsidePLAB" for object
promotion statistics.

> >    36   size_t _inline_allocated; // Number of words allocated directly into the regions.
> 
> That's all.

Thanks, your reviews help me a lot.

> 
> 
> On 8/6/2015 8:49 AM, Thomas Schatzl wrote:
> > Hi all,
> >
[...]
> > CR:
> > https://bugs.openjdk.java.net/browse/JDK-8073013
> >
> > Webrev:
> > http://cr.openjdk.java.net/~tschatzl/8073013/webrev
> >
> > Testing:
> > jprt, a few aurora runs, local testing, lots of benchmarks mainly in
> > conjunction with the following changes.
> >

New webrev at:
http://cr.openjdk.java.net/~tschatzl/8073013/webrev.1 (full)
http://cr.openjdk.java.net/~tschatzl/8073013/webrev.0_to_1 (diff)

Thanks,
  Thomas




From jon.masamitsu at oracle.com  Mon Aug 10 15:20:15 2015
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Mon, 10 Aug 2015 08:20:15 -0700
Subject: RFR (L): 8073013: Add detailed information about PLAB memory usage
In-Reply-To: <1439211707.2330.11.camel@oracle.com>
References: <1438876151.2474.51.camel@oracle.com>	
	<55C4F909.7050601@oracle.com> <1439211707.2330.11.camel@oracle.com>
Message-ID: <55C8C12F.1000809@oracle.com>



On 08/10/2015 06:01 AM, Thomas Schatzl wrote:
> Hi Jon,
>
>    thanks for the review. Comments inline.
>
> On Fri, 2015-08-07 at 11:29 -0700, Jon Masamitsu wrote:
>> Thomas,
>>
>> Basically looks good.  Some questions around the edges.
>>
>> http://cr.openjdk.java.net/~tschatzl/8073013/webrev/src/share/vm/gc/g1/g1CollectedHeap.hpp.frames.html
>>
>>>    248   // Manages all kinds of allocations within regions. This excludes only
>>>    249   // humongous object allocations.
>>>    250   G1Allocator* _allocator;
>> Would this comment be more exact?
>>
>> // Manages all allocations with regions except humongous object allocations.
>>
> Done.

Thanks.

>
>> http://cr.openjdk.java.net/~tschatzl/8073013/webrev/src/share/vm/gc/g1/g1EvacStats.cpp.html
>>
>>     85 void G1EvacStats::send_obj_copy_mem_event(int for_gen) {
>>     86   EventGCG1EvacuationMemoryStatistics e;
>>     87   if (e.should_commit()) {
>>     88     e.set_gcId(GCId::peek().id());
>>     89     e.set_gen(for_gen);
>>
>> Can the conversion of  InCSetState values to generation numbers be put into
>> the send_obj_copy_mem_event()?
>>
>> http://cr.openjdk.java.net/~tschatzl/8073013/webrev/src/share/vm/trace/trace.xml.frames.html
>>
>>    318       <value type="UINT" field="gen" label="Generation" description="Generation these PLAB statistics are for"/>
>>
>>
>> Can the "gen" field be a character string with "young" or "old" instead
>> of a number?
> Did so in the new change. This obviates the need for a dedicated
> conversion function, but adds a function to get the string.

Yes, I expected the function to get the string.  Thanks.

>
> Another option would be to have two separate events, not sure if this is
> better.

The eternal question for which there is no one good answer.  Your
call.

>
>> http://cr.openjdk.java.net/~tschatzl/8073013/webrev/src/share/vm/gc/g1/g1EvacStats.hpp.html
>>
>> Are these alternative comments correct?   Perhaps better for the less G1
>> knowledgeable?
>>
>>     38   // Number of words allocated during evacuation failure in the regions that failed evacuation.
>>
>> // Number of words in live objects remaining in regions that ultimately suffered an
>> // evacuation failure.  This is used space in the regions when the regions are made old regions.
>>
>>
>>     39   size_t _failure_used;
>>
>>     40   // Number of words wasted during evacuation failure in the regions that failed evacuation.
>>
>> // Number of words wasted in regions which failed evacuation.  This is the sum of space for
>> // objects successfully copied out of the regions (now dead space) plus waste at the end of
>> // regions.
> Done.

Thanks.
>
>>     41   size_t _failure_waste;
>>
>>
>> Not sure how ingrained the term "inline" allocation, but why "inline"?  It means that
>> the allocation was directly into the region and not in a LAB in the region, right?
>> Would _directly_allocated or _direct_allocations work for you?
> Would be fine with me. In the most recent change for 8003237 I also
> undid the renaming of that method from "inline" to "direct" so that
> everything fits again.
>
> I hope this makes the statistic more self-explaining, or at least less
> confusing with other uses of "inline allocation".
>
> I.e. afaik the term is used for allocation outside of PLAB using
> additional inlined code.
>
> Interestingly existing PLAB jfr events do not use the term inline
> allocation either, the events have the suffix "OutsidePLAB" for object
> promotion statistics.

Thanks for the change.  I hesitated a bit about using "direct" because
CMS uses "direct" to mean objects allocated out of the CMS gen
as opposed to promoted to the CMS gen but  I think I like direct better
still.

Jon
>
>>>     36   size_t _inline_allocated; // Number of words allocated directly into the regions.
>> That's all.
> Thanks, your reviews help me a lot.
>
>>
>> On 8/6/2015 8:49 AM, Thomas Schatzl wrote:
>>> Hi all,
>>>
> [...]
>>> CR:
>>> https://bugs.openjdk.java.net/browse/JDK-8073013
>>>
>>> Webrev:
>>> http://cr.openjdk.java.net/~tschatzl/8073013/webrev
>>>
>>> Testing:
>>> jprt, a few aurora runs, local testing, lots of benchmarks mainly in
>>> conjunction with the following changes.
>>>
> New webrev at:
> http://cr.openjdk.java.net/~tschatzl/8073013/webrev.1 (full)
> http://cr.openjdk.java.net/~tschatzl/8073013/webrev.0_to_1 (diff)
>
> Thanks,
>    Thomas
>
>



From volker.simonis at gmail.com  Tue Aug 11 12:21:08 2015
From: volker.simonis at gmail.com (Volker Simonis)
Date: Tue, 11 Aug 2015 14:21:08 +0200
Subject: RFR (XS): Move implementation of process_grey_object to
	concurrentMark.inline.hpp
In-Reply-To: <02D5D45C1F8DB848A7AE20E80EE61A5C39962EC5@DEWDFEMB20C.global.corp.sap>
References: <02D5D45C1F8DB848A7AE20E80EE61A5C39962EC5@DEWDFEMB20C.global.corp.sap>
Message-ID: <CA+3eh12qqoE06vjee8HPsno4gLAo_4PnR71LjPWewr6LQ+cypA@mail.gmail.com>

Hi Axel,

the change looks good!
I think we should always define template functions in .inline.hpp files.

Can somebody please sponsor this change?

Thank you and best regards,
Volker


On Thu, Aug 6, 2015 at 3:49 PM, Siebenborn, Axel
<axel.siebenborn at sap.com> wrote:
> Hi,
> Could I have reviews and a sponsor for this small change
>
> We had a linker problem with a C++ - compiler that inlined process_grey_object at all uses but removed the symbol and the actual implementation.
> As there are explicit instantiations of this method in concurrentMark.cpp, this probably should not happen and can be considered of a bug of the C++ compiler. Though, it seems to be pointless to mark that method inline if there are explicit instantiations.
>
> Considering performance, it would be better to move the implementation of process_grey_object to concurrentMark.inline.hpp in order to inline the method in other compilation units.
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8133121
>
> Webrev:
> http://cr.openjdk.java.net/~asiebenborn/8133121/webrev/
>
>
> Thanks,
> Axel


From david.lindholm at oracle.com  Tue Aug 11 13:07:33 2015
From: david.lindholm at oracle.com (David Lindholm)
Date: Tue, 11 Aug 2015 15:07:33 +0200
Subject: RFR (L): 8073013: Add detailed information about PLAB memory usage
In-Reply-To: <1439211707.2330.11.camel@oracle.com>
References: <1438876151.2474.51.camel@oracle.com> <55C4F909.7050601@oracle.com>
	<1439211707.2330.11.camel@oracle.com>
Message-ID: <55C9F395.8050008@oracle.com>

Hi Thomas.

This looks good, except for the changes in trace.xml. All fields should 
follow Java naming conventions, for example regionEndWaste, not 
region_end_waste. Also, 2 different event types are preferred according 
to the Servicability folks (instead of the "gen" field). Suggested names 
are GCG1EvacuationYoungStatistics and GCG1EvacuationOldStatistics.

The other parts looks good.


Thanks,
David

On 2015-08-10 15:01, Thomas Schatzl wrote:
> Hi Jon,
>
>    thanks for the review. Comments inline.
>
> On Fri, 2015-08-07 at 11:29 -0700, Jon Masamitsu wrote:
>> Thomas,
>>
>> Basically looks good.  Some questions around the edges.
>>
>> http://cr.openjdk.java.net/~tschatzl/8073013/webrev/src/share/vm/gc/g1/g1CollectedHeap.hpp.frames.html
>>
>>>    248   // Manages all kinds of allocations within regions. This excludes only
>>>    249   // humongous object allocations.
>>>    250   G1Allocator* _allocator;
>> Would this comment be more exact?
>>
>> // Manages all allocations with regions except humongous object allocations.
>>
> Done.
>
>> http://cr.openjdk.java.net/~tschatzl/8073013/webrev/src/share/vm/gc/g1/g1EvacStats.cpp.html
>>
>>     85 void G1EvacStats::send_obj_copy_mem_event(int for_gen) {
>>     86   EventGCG1EvacuationMemoryStatistics e;
>>     87   if (e.should_commit()) {
>>     88     e.set_gcId(GCId::peek().id());
>>     89     e.set_gen(for_gen);
>>
>> Can the conversion of  InCSetState values to generation numbers be put into
>> the send_obj_copy_mem_event()?
>>
>> http://cr.openjdk.java.net/~tschatzl/8073013/webrev/src/share/vm/trace/trace.xml.frames.html
>>
>>    318       <value type="UINT" field="gen" label="Generation" description="Generation these PLAB statistics are for"/>
>>
>>
>> Can the "gen" field be a character string with "young" or "old" instead
>> of a number?
> Did so in the new change. This obviates the need for a dedicated
> conversion function, but adds a function to get the string.
>
> Another option would be to have two separate events, not sure if this is
> better.
>
>> http://cr.openjdk.java.net/~tschatzl/8073013/webrev/src/share/vm/gc/g1/g1EvacStats.hpp.html
>>
>> Are these alternative comments correct?   Perhaps better for the less G1
>> knowledgeable?
>>
>>     38   // Number of words allocated during evacuation failure in the regions that failed evacuation.
>>
>> // Number of words in live objects remaining in regions that ultimately suffered an
>> // evacuation failure.  This is used space in the regions when the regions are made old regions.
>>
>>
>>     39   size_t _failure_used;
>>
>>     40   // Number of words wasted during evacuation failure in the regions that failed evacuation.
>>
>> // Number of words wasted in regions which failed evacuation.  This is the sum of space for
>> // objects successfully copied out of the regions (now dead space) plus waste at the end of
>> // regions.
> Done.
>
>>     41   size_t _failure_waste;
>>
>>
>> Not sure how ingrained the term "inline" allocation, but why "inline"?  It means that
>> the allocation was directly into the region and not in a LAB in the region, right?
>> Would _directly_allocated or _direct_allocations work for you?
> Would be fine with me. In the most recent change for 8003237 I also
> undid the renaming of that method from "inline" to "direct" so that
> everything fits again.
>
> I hope this makes the statistic more self-explaining, or at least less
> confusing with other uses of "inline allocation".
>
> I.e. afaik the term is used for allocation outside of PLAB using
> additional inlined code.
>
> Interestingly existing PLAB jfr events do not use the term inline
> allocation either, the events have the suffix "OutsidePLAB" for object
> promotion statistics.
>
>>>     36   size_t _inline_allocated; // Number of words allocated directly into the regions.
>> That's all.
> Thanks, your reviews help me a lot.
>
>>
>> On 8/6/2015 8:49 AM, Thomas Schatzl wrote:
>>> Hi all,
>>>
> [...]
>>> CR:
>>> https://bugs.openjdk.java.net/browse/JDK-8073013
>>>
>>> Webrev:
>>> http://cr.openjdk.java.net/~tschatzl/8073013/webrev
>>>
>>> Testing:
>>> jprt, a few aurora runs, local testing, lots of benchmarks mainly in
>>> conjunction with the following changes.
>>>
> New webrev at:
> http://cr.openjdk.java.net/~tschatzl/8073013/webrev.1 (full)
> http://cr.openjdk.java.net/~tschatzl/8073013/webrev.0_to_1 (diff)
>
> Thanks,
>    Thomas
>
>



From david.lindholm at oracle.com  Tue Aug 11 13:17:03 2015
From: david.lindholm at oracle.com (David Lindholm)
Date: Tue, 11 Aug 2015 15:17:03 +0200
Subject: RFR (M): 8040162: Avoid reallocating PLABs between GC phases in G1
In-Reply-To: <1438949102.2305.42.camel@oracle.com>
References: <1438949102.2305.42.camel@oracle.com>
Message-ID: <55C9F5CF.8040006@oracle.com>

Hi Thomas,

Great work. Looks good, reviewed.


Thanks,
David

On 2015-08-07 14:05, Thomas Schatzl wrote:
> Hi all,
>
>    can I have reviews for this significant improvement in memory use
> during GC by changing how G1 manages G1ParScanThreadState instances?
>
> So until now G1 uses separate G1ParScanThreadState instances with their
> separate PLABs for every phase of the GC. This means that the
> G1ParScanThreadState/PLAB is thrown away after every phase. This means
> that even PLABs which only have a single object in it, but are
> potentially hundreds of kB in size are filled with dummy objects and
> reloaded.
>
> This change instantiates a single G1ParScanThreadState for the entire
> GC. PLABs are reused as long as possible. This decreases memory use
> during GC significantly in some cases, which can significantly reduce
> the number of GCs, improves throughput, and so on.
>
> Care has been taken in this change to not change the values for the
> termination statistics during gc, so a few members of
> G1ParScanThreadState were relocated (making it a lot nicer too imo).
>
> There is also some bug where the age table and the per-region survivor
> statistics are only taken from the initial parallel collection phase
> into account.
>
> This will be fixed as JDK-8073146, but if wanted I can merge them.
>
> This change is based on the change for JDK-8073013 which is currently
> out for review.
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8040162
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8040162/webrev/
> Testing:
> lots of runs, jprt, a few aurora nightly runs
>
> Thanks,
>    Thomas
>
>



From thomas.schatzl at oracle.com  Tue Aug 11 14:56:52 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Tue, 11 Aug 2015 16:56:52 +0200
Subject: RFR (M): 8040162: Avoid reallocating PLABs between GC phases in G1
In-Reply-To: <55C9F5CF.8040006@oracle.com>
References: <1438949102.2305.42.camel@oracle.com> <55C9F5CF.8040006@oracle.com>
Message-ID: <1439305012.2327.1.camel@oracle.com>

Hi David,

On Tue, 2015-08-11 at 15:17 +0200, David Lindholm wrote:
> Hi Thomas,
> 
> Great work. Looks good, reviewed.
> 

  thanks for the review.

Thomas




From tom.benson at oracle.com  Tue Aug 11 17:43:12 2015
From: tom.benson at oracle.com (Tom Benson)
Date: Tue, 11 Aug 2015 13:43:12 -0400
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <55C4C720.5030903@oracle.com>
References: <55C103A4.1060505@oracle.com>	<A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>	<1438781844.2378.60.camel@oracle.com>	<55C21DC3.5090305@oracle.com>	<1438785532.2378.73.camel@oracle.com>	<55C248FF.6010308@oracle.com>	<F73C4B9A-4DBA-47A8-8715-0988FF6D393D@oracle.com>	<55C26249.1090902@oracle.com>	<C149705F-696B-46CD-8FB8-CAD06C511244@oracle.com>	<55C2751B.8070400@oracle.com>	<1438847329.2009.9.camel@oracle.com>	<55C36B31.7000908@oracle.com>	<1438871723.2474.37.camel@oracle.com>	<55C373F6.9060609@oracle.com>	<AD45178E-A442-4A0D-80D6-50E91608E1FE@oracle.com>	<55C3D8BB.9000603@oracle.com>	<48C0E571-0D7B-4D30-85DF-E31D99092A7C@oracle.com>	<30B0248C-3BEA-4FAE-861F-DAAE19F56B45@oracle.com>	<55C4B806.7050504@oracle.com>
	<55C4C720.5030903@oracle.com>
Message-ID: <55CA3430.3070300@oracle.com>

Hi,
On 8/7/2015 10:56 AM, Tom Benson wrote:
>
> The problem is simply that in addition to calling 
> free_archive_regions, FileMapInfo::unmap_string_regions also unmaps 
> the memory.... so there's a segv when GC tries to re-use it. Let's 
> talk directly about the best way to handle it.
> Tom

After some discussion, I've changed the definition and name of 
free_archive_regions.  Now called dealloc_archive_regions, it uncommits 
the specified regions, unmapping the memory, rather than adding them to 
the free list.  This means the CDS code will no longer do the unmapping 
on verification failures.

Updated full and incremental webrevs of the GC code are at:
http://cr.openjdk.java.net/~tbenson/8131734/webrev.02/
http://cr.openjdk.java.net/~tbenson/8131734/webrev.02.vs.01/

Tested with JPRT and running benchmarks with the dealloc_ performed 
explicitly.  Jiangli also tested the original failing cases, and will be 
posting an updated webrev.

Tom

On 8/7/2015 10:56 AM, Tom Benson wrote:
> Hi,
>
> On 8/7/2015 9:52 AM, Tom Benson wrote:
>> Hi,
>> On 8/6/2015 8:40 PM, Jiangli Zhou wrote:
>>> Hi Dmitry,
>>>
>>> I?ve added shared string unmapping to handle the string data 
>>> verification failure. I also added 'free_archive_regions()? call to 
>>> FileMapInfo::unmap_string_regions(). My testing revealed some issues 
>>> with the new archive region free code when I forced string 
>>> verification failure. I?m working with Tom on the issues.
>>
>
> The problem is simply that in addition to calling 
> free_archive_regions, FileMapInfo::unmap_string_regions also unmaps 
> the memory.... so there's a segv when GC tries to re-use it. Let's 
> talk directly about the best way to handle it.
> Tom



From kim.barrett at oracle.com  Tue Aug 11 21:24:01 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Tue, 11 Aug 2015 17:24:01 -0400
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <55CA3430.3070300@oracle.com>
References: <55C103A4.1060505@oracle.com>	<A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>	<1438781844.2378.60.camel@oracle.com>	<55C21DC3.5090305@oracle.com>	<1438785532.2378.73.camel@oracle.com>	<55C248FF.6010308@oracle.com>	<F73C4B9A-4DBA-47A8-8715-0988FF6D393D@oracle.com>	<55C26249.1090902@oracle.com>	<C149705F-696B-46CD-8FB8-CAD06C511244@oracle.com>	<55C2751B.8070400@oracle.com>	<1438847329.2009.9.camel@oracle.com>	<55C36B31.7000908@oracle.com>	<1438871723.2474.37.camel@oracle.com>	<55C373F6.9060609@oracle.com>	<AD45178E-A442-4A0D-80D6-50E91608E1FE@oracle.com>	<55C3D8BB.9000603@oracle.com>	<48C0E571-0D7B-4D30-85DF-E31D99092A7C@oracle.com>	<30B0248C-3BEA-4FAE-861F-DAAE19F56B45@oracle.com>	<55C4B806.7050504@oracle.com>
	<55C4C720.5030903@oracle.com> <55CA3430.3070300@oracle.com>
Message-ID: <9255850A-182F-4741-BE2A-C7BF72057603@oracle.com>

On Aug 11, 2015, at 1:43 PM, Tom Benson <tom.benson at oracle.com> wrote:
> 
> Hi,
> On 8/7/2015 10:56 AM, Tom Benson wrote:
>> 
>> The problem is simply that in addition to calling free_archive_regions, FileMapInfo::unmap_string_regions also unmaps the memory.... so there's a segv when GC tries to re-use it. Let's talk directly about the best way to handle it.
>> Tom
> 
> After some discussion, I've changed the definition and name of free_archive_regions.  Now called dealloc_archive_regions, it uncommits the specified regions, unmapping the memory, rather than adding them to the free list.  This means the CDS code will no longer do the unmapping on verification failures.
> 
> Updated full and incremental webrevs of the GC code are at:
> http://cr.openjdk.java.net/~tbenson/8131734/webrev.02/
> http://cr.openjdk.java.net/~tbenson/8131734/webrev.02.vs.01/
> 
> Tested with JPRT and running benchmarks with the dealloc_ performed explicitly.  Jiangli also tested the original failing cases, and will be posting an updated webrev.

Can this introduce uncommitted "holes" in committed space?  It seems
like it might.  Otherwise, why introduce shrink_at, rather than just
using shrink_by.  I'm not sure such holes are presently possible, and
I'm not sure they are handled properly everywhere.

For example, I think such a hole might confuse
HeapRegionManager::shrink_by.  I haven't looked carefully for other
code that might be confused by uncommitted holes.



From tom.benson at oracle.com  Tue Aug 11 21:40:12 2015
From: tom.benson at oracle.com (Tom Benson)
Date: Tue, 11 Aug 2015 17:40:12 -0400
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <9255850A-182F-4741-BE2A-C7BF72057603@oracle.com>
References: <55C103A4.1060505@oracle.com>	<A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>	<1438781844.2378.60.camel@oracle.com>	<55C21DC3.5090305@oracle.com>	<1438785532.2378.73.camel@oracle.com>	<55C248FF.6010308@oracle.com>	<F73C4B9A-4DBA-47A8-8715-0988FF6D393D@oracle.com>	<55C26249.1090902@oracle.com>	<C149705F-696B-46CD-8FB8-CAD06C511244@oracle.com>	<55C2751B.8070400@oracle.com>	<1438847329.2009.9.camel@oracle.com>	<55C36B31.7000908@oracle.com>	<1438871723.2474.37.camel@oracle.com>	<55C373F6.9060609@oracle.com>	<AD45178E-A442-4A0D-80D6-50E91608E1FE@oracle.com>	<55C3D8BB.9000603@oracle.com>	<48C0E571-0D7B-4D30-85DF-E31D99092A7C@oracle.com>	<30B0248C-3BEA-4FAE-861F-DAAE19F56B45@oracle.com>	<55C4B806.7050504@oracle.com>
	<55C4C720.5030903@oracle.com> <55CA3430.3070300@oracle.com>
	<9255850A-182F-4741-BE2A-C7BF72057603@oracle.com>
Message-ID: <55CA6BBC.8080605@oracle.com>

Hi Kim,

On 8/11/2015 5:24 PM, Kim Barrett wrote:
> On Aug 11, 2015, at 1:43 PM, Tom Benson <tom.benson at oracle.com> wrote:
>> Hi,
>> On 8/7/2015 10:56 AM, Tom Benson wrote:
>>> The problem is simply that in addition to calling free_archive_regions, FileMapInfo::unmap_string_regions also unmaps the memory.... so there's a segv when GC tries to re-use it. Let's talk directly about the best way to handle it.
>>> Tom
>> After some discussion, I've changed the definition and name of free_archive_regions.  Now called dealloc_archive_regions, it uncommits the specified regions, unmapping the memory, rather than adding them to the free list.  This means the CDS code will no longer do the unmapping on verification failures.
>>
>> Updated full and incremental webrevs of the GC code are at:
>> http://cr.openjdk.java.net/~tbenson/8131734/webrev.02/
>> http://cr.openjdk.java.net/~tbenson/8131734/webrev.02.vs.01/
>>
>> Tested with JPRT and running benchmarks with the dealloc_ performed explicitly.  Jiangli also tested the original failing cases, and will be posting an updated webrev.
> Can this introduce uncommitted "holes" in committed space?  It seems
> like it might.  Otherwise, why introduce shrink_at, rather than just
> using shrink_by.  I'm not sure such holes are presently possible, and
> I'm not sure they are handled properly everywhere.
shrink_by looks for free ranges it can uncommit.  Here, we want to 
uncommit the specific region(s) allocated with alloc_archive_regions.  
There is an expand_at that takes a specific index, which shrink_at is 
intended to be analogous to.

G1 is designed to allow uncommitted holes.   Even without this change, 
there is a big 'hole' in committed space:    All the regions between the 
low end of the heap where mutator allocation is occurring (initially 
sized by Xms)  and these archive regions which are at the highest end of 
the (Xmx-sized) heap.

In a way,  as used by CDS, calling dealloc_archive_regions is *removing* 
that hole. 8^)     The entire upper portion of the heap (above Xms) will 
again be uncommitted, as if alloc_archive_regions had never been called.

Tom

>
> For example, I think such a hole might confuse
> HeapRegionManager::shrink_by.  I haven't looked carefully for other
> code that might be confused by uncommitted holes.



From mandy.chung at oracle.com  Tue Aug 11 22:47:48 2015
From: mandy.chung at oracle.com (Mandy Chung)
Date: Tue, 11 Aug 2015 15:47:48 -0700
Subject: RFR: 8132306: java/lang/ref/ReferenceEnqueue.java fails with
	"RuntimeException: Error: poll() returned null; expected ref object"
In-Reply-To: <C25B3711-5931-4F4D-8B8C-4F83FAF83A51@oracle.com>
References: <BA1C0BEB-F342-437A-87E7-029660BB5348@oracle.com>	<55B88FAB.4080302@oracle.com>	<929281CB-55E8-4A0C-A8D1-85ED37348E12@oracle.com>	<55B9E293.80102@gmail.com>
	<55B9EA8A.1080102@oracle.com>	<55B9FA54.5040902@gmail.com>
	<55B9FB50.8050907@oracle.com>	<55B9FF5F.2020706@gmail.com>
	<55BA0E2B.9050101@oracle.com>	<55BA110F.2010101@gmail.com>
	<55BA13E8.1050606@oracle.com>	<CE7CA993-BB20-44E6-B378-419560CFFCCC@oracle.com>	<55BA9830.8030308@oracle.com>
	<C25B3711-5931-4F4D-8B8C-4F83FAF83A51@oracle.com>
Message-ID: <55CA7B94.5030502@oracle.com>



On 07/30/2015 03:56 PM, Kim Barrett wrote:
> New webrev, with both changes: 
> http://cr.openjdk.java.net/~kbarrett/8132306/webrev.01/ 

Thanks for fixing this.  The change looks fine to me.  Sorry for the 
belated reply as I just got back from vacation.

I agree that this patch includes both of your change and Peter's change.

Mandy


From kim.barrett at oracle.com  Wed Aug 12 00:02:11 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Tue, 11 Aug 2015 20:02:11 -0400
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <55CA6BBC.8080605@oracle.com>
References: <55C103A4.1060505@oracle.com>	<A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>	<1438781844.2378.60.camel@oracle.com>	<55C21DC3.5090305@oracle.com>	<1438785532.2378.73.camel@oracle.com>	<55C248FF.6010308@oracle.com>	<F73C4B9A-4DBA-47A8-8715-0988FF6D393D@oracle.com>	<55C26249.1090902@oracle.com>	<C149705F-696B-46CD-8FB8-CAD06C511244@oracle.com>	<55C2751B.8070400@oracle.com>	<1438847329.2009.9.camel@oracle.com>	<55C36B31.7000908@oracle.com>	<1438871723.2474.37.camel@oracle.com>	<55C373F6.9060609@oracle.com>	<AD45178E-A442-4A0D-80D6-50E91608E1FE@oracle.com>	<55C3D8BB.9000603@oracle.com>	<48C0E571-0D7B-4D30-85DF-E31D99092A7C@oracle.com>	<30B0248C-3BEA-4FAE-861F-DAAE19F56B45@oracle.com>	<55C4B806.7050504@oracle.com>
	<55C4C720.5030903@oracle.com> <55CA3430.3070300@oracle.com>
	<9255850A-182F-4741-BE2A-C7BF72057603@oracle.com>
	<55CA6BBC.8080605@oracle.com>
Message-ID: <CE5338A0-89AC-4701-AD8E-2D6D9349A5D0@oracle.com>

On Aug 11, 2015, at 5:40 PM, Tom Benson <tom.benson at oracle.com> wrote:
> 
> On 8/11/2015 5:24 PM, Kim Barrett wrote:
>> On Aug 11, 2015, at 1:43 PM, Tom Benson <tom.benson at oracle.com> wrote:
>>> Updated full and incremental webrevs of the GC code are at:
>>> http://cr.openjdk.java.net/~tbenson/8131734/webrev.02/
>>> http://cr.openjdk.java.net/~tbenson/8131734/webrev.02.vs.01/
>> Can this introduce uncommitted "holes" in committed space?  It seems
>> like it might.  Otherwise, why introduce shrink_at, rather than just
>> using shrink_by.  I'm not sure such holes are presently possible, and
>> I'm not sure they are handled properly everywhere.
> shrink_by looks for free ranges it can uncommit.  Here, we want to uncommit the specific region(s) allocated with alloc_archive_regions.  There is an expand_at that takes a specific index, which shrink_at is intended to be analogous to.

I seem to have earlier convinced myself that shrink_by would be
confused by a hole; after staring at it some more, I now think it's
ok.  Sorry for the noise.

But I think there might be an unrelated pre-existing performance bug
in shrink_by.  Line 431 is "cur -= num_last_found;".  If the recent
scan had to skip over a block of !available || !empty regions to find
the regions that were just removed, that skipped block isn't accounted
for by that decrement.  I think a better iteration step would be "cur
= idx_last_found;".  [This is part of what confused me about holes.]

> G1 is designed to allow uncommitted holes.   Even without this change, there is a big 'hole' in committed space:    All the regions between the low end of the heap where mutator allocation is occurring (initially sized by Xms)  and these archive regions which are at the highest end of the (Xmx-sized) heap.
> 
> In a way,  as used by CDS, calling dealloc_archive_regions is *removing* that hole. 8^)     The entire upper portion of the heap (above Xms) will again be uncommitted, as if alloc_archive_regions had never been called.

I do see code that seems to be intended to cope with such holes.

I wonder though, were there ever uncommitted holes in practice before
the introduction of this new archive region feature?  Previously, the
only time a region gets uncommitted is after a full gc.  I've not
studied G1's full gc enough to even guess whether it could leave such
holes.  If the answer is no, there could be lurking bugs waiting to be
uncovered.  I guess we'll deal with those if/when we find them.



From kim.barrett at oracle.com  Wed Aug 12 00:10:01 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Tue, 11 Aug 2015 20:10:01 -0400
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <CE5338A0-89AC-4701-AD8E-2D6D9349A5D0@oracle.com>
References: <55C103A4.1060505@oracle.com>	<A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>	<1438781844.2378.60.camel@oracle.com>	<55C21DC3.5090305@oracle.com>	<1438785532.2378.73.camel@oracle.com>	<55C248FF.6010308@oracle.com>	<F73C4B9A-4DBA-47A8-8715-0988FF6D393D@oracle.com>	<55C26249.1090902@oracle.com>	<C149705F-696B-46CD-8FB8-CAD06C511244@oracle.com>	<55C2751B.8070400@oracle.com>	<1438847329.2009.9.camel@oracle.com>	<55C36B31.7000908@oracle.com>	<1438871723.2474.37.camel@oracle.com>	<55C373F6.9060609@oracle.com>	<AD45178E-A442-4A0D-80D6-50E91608E1FE@oracle.com>	<55C3D8BB.9000603@oracle.com>	<48C0E571-0D7B-4D30-85DF-E31D99092A7C@oracle.com>	<30B0248C-3BEA-4FAE-861F-DAAE19F56B45@oracle.com>	<55C4B806.7050504@oracle.com>
	<55C4C720.5030903@oracle.com> <55CA3430.3070300@oracle.com>
	<9255850A-182F-4741-BE2A-C7BF72057603@oracle.com>
	<55CA6BBC.8080605@oracle.com>
	<CE5338A0-89AC-4701-AD8E-2D6D9349A5D0@oracle.com>
Message-ID: <CEC0388F-B4EC-4851-8E29-CC3B4CDA7C19@oracle.com>

On Aug 11, 2015, at 8:02 PM, Kim Barrett <kim.barrett at oracle.com> wrote:
> 
> On Aug 11, 2015, at 5:40 PM, Tom Benson <tom.benson at oracle.com> wrote:
>> 
>> On 8/11/2015 5:24 PM, Kim Barrett wrote:
>>> On Aug 11, 2015, at 1:43 PM, Tom Benson <tom.benson at oracle.com> wrote:
>>>> Updated full and incremental webrevs of the GC code are at:
>>>> http://cr.openjdk.java.net/~tbenson/8131734/webrev.02/
>>>> http://cr.openjdk.java.net/~tbenson/8131734/webrev.02.vs.01/
>>> Can this introduce uncommitted "holes" in committed space?  It seems
>>> like it might.  Otherwise, why introduce shrink_at, rather than just
>>> using shrink_by.  I'm not sure such holes are presently possible, and
>>> I'm not sure they are handled properly everywhere.
>> shrink_by looks for free ranges it can uncommit.  Here, we want to uncommit the specific region(s) allocated with alloc_archive_regions.  There is an expand_at that takes a specific index, which shrink_at is intended to be analogous to.
> 
> I seem to have earlier convinced myself that shrink_by would be
> confused by a hole; after staring at it some more, I now think it's
> ok.  Sorry for the noise.
> 
> But I think there might be an unrelated pre-existing performance bug
> in shrink_by.  Line 431 is "cur -= num_last_found;".  If the recent
> scan had to skip over a block of !available || !empty regions to find
> the regions that were just removed, that skipped block isn't accounted
> for by that decrement.  I think a better iteration step would be "cur
> = idx_last_found;".  [This is part of what confused me about holes.]
> 
>> G1 is designed to allow uncommitted holes.   Even without this change, there is a big 'hole' in committed space:    All the regions between the low end of the heap where mutator allocation is occurring (initially sized by Xms)  and these archive regions which are at the highest end of the (Xmx-sized) heap.
>> 
>> In a way,  as used by CDS, calling dealloc_archive_regions is *removing* that hole. 8^)     The entire upper portion of the heap (above Xms) will again be uncommitted, as if alloc_archive_regions had never been called.
> 
> I do see code that seems to be intended to cope with such holes.
> 
> I wonder though, were there ever uncommitted holes in practice before
> the introduction of this new archive region feature?  Previously, the
> only time a region gets uncommitted is after a full gc.  I've not
> studied G1's full gc enough to even guess whether it could leave such
> holes.  If the answer is no, there could be lurking bugs waiting to be
> uncovered.  I guess we'll deal with those if/when we find them.

Oops, forgot the important part of the reply.

Change looks good.



From david.lindholm at oracle.com  Wed Aug 12 08:21:32 2015
From: david.lindholm at oracle.com (David Lindholm)
Date: Wed, 12 Aug 2015 10:21:32 +0200
Subject: RFR (S): 8073146: G1 merges thread local age tables too early
	with global age table
In-Reply-To: <1438949116.2305.43.camel@oracle.com>
References: <1438949116.2305.43.camel@oracle.com>
Message-ID: <55CB020C.8010803@oracle.com>

Thomas,

This change looks good. Reviewed.


Thanks,
David

On 2015-08-07 14:05, Thomas Schatzl wrote:
> Hi,
>
>    here is the mentioned fix for 8073146 which resolves a day-one issue
> with age table statistics and per-region survivor statistics.
>
> Basically G1 only ever merged the age table/region survivor statistics
> at the end of the first parallel phase, forgetting any objects copied
> later.
>
> This is based on the change for 8040162 that is currently out for
> review.
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8073146
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8073146/webrev/
> Testing:
> jprt, lots of testing with various benchmarks
>
> Thanks,
>    Thomas
>
>



From thomas.schatzl at oracle.com  Wed Aug 12 08:44:07 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 12 Aug 2015 10:44:07 +0200
Subject: RFR (L): 8073013: Add detailed information about PLAB memory usage
In-Reply-To: <55C9F395.8050008@oracle.com>
References: <1438876151.2474.51.camel@oracle.com>
	<55C4F909.7050601@oracle.com> <1439211707.2330.11.camel@oracle.com>
	<55C9F395.8050008@oracle.com>
Message-ID: <1439369047.2324.1.camel@oracle.com>

Hi David,

  thanks for the review:

On Tue, 2015-08-11 at 15:07 +0200, David Lindholm wrote:
> Hi Thomas.
> 
> This looks good, except for the changes in trace.xml. All fields should 
> follow Java naming conventions, for example regionEndWaste, not 
> region_end_waste. Also, 2 different event types are preferred according 
> to the Servicability folks (instead of the "gen" field). Suggested names 
> are GCG1EvacuationYoungStatistics and GCG1EvacuationOldStatistics.
> 
> The other parts looks good.

I think all fixed in the new webrevs at:

http://cr.openjdk.java.net/~tschatzl/8073013/webrev.2 (full)
http://cr.openjdk.java.net/~tschatzl/8073013/webrev.1_to_2 (diff)

Thanks,
  Thomas




From david.lindholm at oracle.com  Wed Aug 12 08:50:37 2015
From: david.lindholm at oracle.com (David Lindholm)
Date: Wed, 12 Aug 2015 10:50:37 +0200
Subject: RFR (L): 8073013: Add detailed information about PLAB memory usage
In-Reply-To: <1439369047.2324.1.camel@oracle.com>
References: <1438876151.2474.51.camel@oracle.com>	
	<55C4F909.7050601@oracle.com>
	<1439211707.2330.11.camel@oracle.com>	
	<55C9F395.8050008@oracle.com> <1439369047.2324.1.camel@oracle.com>
Message-ID: <55CB08DD.9070906@oracle.com>

Hi Thomas,

This looks good now. Reviewed.


Thanks,
David

On 2015-08-12 10:44, Thomas Schatzl wrote:
> Hi David,
>
>    thanks for the review:
>
> On Tue, 2015-08-11 at 15:07 +0200, David Lindholm wrote:
>> Hi Thomas.
>>
>> This looks good, except for the changes in trace.xml. All fields should
>> follow Java naming conventions, for example regionEndWaste, not
>> region_end_waste. Also, 2 different event types are preferred according
>> to the Servicability folks (instead of the "gen" field). Suggested names
>> are GCG1EvacuationYoungStatistics and GCG1EvacuationOldStatistics.
>>
>> The other parts looks good.
> I think all fixed in the new webrevs at:
>
> http://cr.openjdk.java.net/~tschatzl/8073013/webrev.2 (full)
> http://cr.openjdk.java.net/~tschatzl/8073013/webrev.1_to_2 (diff)
>
> Thanks,
>    Thomas
>
>



From thomas.schatzl at oracle.com  Wed Aug 12 09:41:22 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 12 Aug 2015 11:41:22 +0200
Subject: RFR (L): 8073013: Add detailed information about PLAB memory usage
In-Reply-To: <55CB08DD.9070906@oracle.com>
References: <1438876151.2474.51.camel@oracle.com>
	<55C4F909.7050601@oracle.com> <1439211707.2330.11.camel@oracle.com>
	<55C9F395.8050008@oracle.com> <1439369047.2324.1.camel@oracle.com>
	<55CB08DD.9070906@oracle.com>
Message-ID: <1439372482.2324.6.camel@oracle.com>

Hi David,

On Wed, 2015-08-12 at 10:50 +0200, David Lindholm wrote:
> Hi Thomas,
> 
> This looks good now. Reviewed.

  thanks for the review.

Thomas



From thomas.schatzl at oracle.com  Wed Aug 12 10:37:43 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 12 Aug 2015 12:37:43 +0200
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <CE5338A0-89AC-4701-AD8E-2D6D9349A5D0@oracle.com>
References: <55C103A4.1060505@oracle.com>
	<A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>
	<1438781844.2378.60.camel@oracle.com>	<55C21DC3.5090305@oracle.com>
	<1438785532.2378.73.camel@oracle.com>	<55C248FF.6010308@oracle.com>
	<F73C4B9A-4DBA-47A8-8715-0988FF6D393D@oracle.com>
	<55C26249.1090902@oracle.com>
	<C149705F-696B-46CD-8FB8-CAD06C511244@oracle.com>
	<55C2751B.8070400@oracle.com>	<1438847329.2009.9.camel@oracle.com>
	<55C36B31.7000908@oracle.com>	<1438871723.2474.37.camel@oracle.com>
	<55C373F6.9060609@oracle.com>
	<AD45178E-A442-4A0D-80D6-50E91608E1FE@oracle.com>
	<55C3D8BB.9000603@oracle.com>
	<48C0E571-0D7B-4D30-85DF-E31D99092A7C@oracle.com>
	<30B0248C-3BEA-4FAE-861F-DAAE19F56B45@oracle.com>
	<55C4B806.7050504@oracle.com> <55C4C720.5030903@oracle.com>
	<55CA3430.3070300@oracle.com>
	<9255850A-182F-4741-BE2A-C7BF72057603@oracle.com>
	<55CA6BBC.8080605@oracle.com>
	<CE5338A0-89AC-4701-AD8E-2D6D9349A5D0@oracle.com>
Message-ID: <1439375863.2324.15.camel@oracle.com>

Hi,

On Tue, 2015-08-11 at 20:02 -0400, Kim Barrett wrote:
> On Aug 11, 2015, at 5:40 PM, Tom Benson <tom.benson at oracle.com> wrote:
> > 
> > On 8/11/2015 5:24 PM, Kim Barrett wrote:
> >> On Aug 11, 2015, at 1:43 PM, Tom Benson <tom.benson at oracle.com> wrote:
> >>> Updated full and incremental webrevs of the GC code are at:
> >>> http://cr.openjdk.java.net/~tbenson/8131734/webrev.02/
> >>> http://cr.openjdk.java.net/~tbenson/8131734/webrev.02.vs.01/
> >> Can this introduce uncommitted "holes" in committed space?  It seems
> >> like it might.  Otherwise, why introduce shrink_at, rather than just
> >> using shrink_by.  I'm not sure such holes are presently possible, and
> >> I'm not sure they are handled properly everywhere.
> > shrink_by looks for free ranges it can uncommit.  Here, we want to
> > uncommit the specific region(s) allocated with alloc_archive_regions.
> > There is an expand_at that takes a specific index, which shrink_at is
> > intended to be analogous to.
> 
> I seem to have earlier convinced myself that shrink_by would be
> confused by a hole; after staring at it some more, I now think it's
> ok.  Sorry for the noise.
> 
> But I think there might be an unrelated pre-existing performance bug
> in shrink_by.  Line 431 is "cur -= num_last_found;".  If the recent
> scan had to skip over a block of !available || !empty regions to find
> the regions that were just removed, that skipped block isn't accounted
> for by that decrement.  I think a better iteration step would be "cur
> = idx_last_found;".  [This is part of what confused me about holes.]

That's true. I filed https://bugs.openjdk.java.net/browse/JDK-8133456.

> > G1 is designed to allow uncommitted holes.   Even without this
> >change, there is a big 'hole' in committed space:    All the regions
> >between the low end of the heap where mutator allocation is occurring
> >(initially sized by Xms)  and these archive regions which are at the
>>highest end of the (Xmx-sized) heap.
> > 
> > In a way,  as used by CDS, calling dealloc_archive_regions is
>>*removing* that hole. 8^)     The entire upper portion of the heap
>>(above Xms) will again be uncommitted, as if alloc_archive_regions had
>>never been called.
> 
> I do see code that seems to be intended to cope with such holes.
> 
> I wonder though, were there ever uncommitted holes in practice before
> the introduction of this new archive region feature?  Previously, the
> only time a region gets uncommitted is after a full gc.  I've not
> studied G1's full gc enough to even guess whether it could leave such
> holes. 

Yes, when there are humongous objects in the heap.

> If the answer is no, there could be lurking bugs waiting to be
> uncovered.  I guess we'll deal with those if/when we find them.

There are explicit tests in the jtreg tests (gc/g1/TestShrink*) that
test shrinking, and TestShrinkDefragmentedHeap in particular tests this.

Also there are a few stress tests (ArrayJuggle*) that repeatedly create
holes using large objects and do full collections that cause heap
shrinking.

Thanks,
  Thomas




From thomas.schatzl at oracle.com  Wed Aug 12 10:38:24 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 12 Aug 2015 12:38:24 +0200
Subject: RFR (M): 8040162: Avoid reallocating PLABs between GC phases in G1
In-Reply-To: <1439305012.2327.1.camel@oracle.com>
References: <1438949102.2305.42.camel@oracle.com>
	<55C9F5CF.8040006@oracle.com> <1439305012.2327.1.camel@oracle.com>
Message-ID: <1439375904.2324.16.camel@oracle.com>

Hi,

On Tue, 2015-08-11 at 16:56 +0200, Thomas Schatzl wrote:
> Hi David,
> 
> On Tue, 2015-08-11 at 15:17 +0200, David Lindholm wrote:
> > Hi Thomas,
> > 
> > Great work. Looks good, reviewed. 
> 
>   thanks for the review.

  while testing the change a little more an issue with -XX:
+ParallelRefProcEnabled has been found.

Here is a new webrev:
http://cr.openjdk.java.net/~tschatzl/8040162/webrev.0_to_1/ (diff)
http://cr.openjdk.java.net/~tschatzl/8040162/webrev.1/ (full)

Thanks,
  Thomas




From thomas.schatzl at oracle.com  Wed Aug 12 11:00:59 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 12 Aug 2015 13:00:59 +0200
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <55CA3430.3070300@oracle.com>
References: <55C103A4.1060505@oracle.com>
	<A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>
	<1438781844.2378.60.camel@oracle.com>	<55C21DC3.5090305@oracle.com>
	<1438785532.2378.73.camel@oracle.com>	<55C248FF.6010308@oracle.com>
	<F73C4B9A-4DBA-47A8-8715-0988FF6D393D@oracle.com>
	<55C26249.1090902@oracle.com>
	<C149705F-696B-46CD-8FB8-CAD06C511244@oracle.com>
	<55C2751B.8070400@oracle.com>	<1438847329.2009.9.camel@oracle.com>
	<55C36B31.7000908@oracle.com>	<1438871723.2474.37.camel@oracle.com>
	<55C373F6.9060609@oracle.com>
	<AD45178E-A442-4A0D-80D6-50E91608E1FE@oracle.com>
	<55C3D8BB.9000603@oracle.com>
	<48C0E571-0D7B-4D30-85DF-E31D99092A7C@oracle.com>
	<30B0248C-3BEA-4FAE-861F-DAAE19F56B45@oracle.com>
	<55C4B806.7050504@oracle.com> <55C4C720.5030903@oracle.com>
	<55CA3430.3070300@oracle.com>
Message-ID: <1439377259.2324.27.camel@oracle.com>

Hi,

On Tue, 2015-08-11 at 13:43 -0400, Tom Benson wrote:
> Hi,
> On 8/7/2015 10:56 AM, Tom Benson wrote:
> >
> > The problem is simply that in addition to calling 
> > free_archive_regions, FileMapInfo::unmap_string_regions also unmaps 
> > the memory.... so there's a segv when GC tries to re-use it. Let's 
> > talk directly about the best way to handle it.
> > Tom
> 
> After some discussion, I've changed the definition and name of 
> free_archive_regions.  Now called dealloc_archive_regions, it uncommits 
> the specified regions, unmapping the memory, rather than adding them to 
> the free list.  This means the CDS code will no longer do the unmapping 
> on verification failures.
> 
> Updated full and incremental webrevs of the GC code are at:
> http://cr.openjdk.java.net/~tbenson/8131734/webrev.02/
> http://cr.openjdk.java.net/~tbenson/8131734/webrev.02.vs.01/
> 
> Tested with JPRT and running benchmarks with the dealloc_ performed 
> explicitly.  Jiangli also tested the original failing cases, and will be 
> posting an updated webrev.

- is it possible that shrink_by() uses shrink_at()? This would avoid two
paths that uncommit regions like expand_by()/expand_at()?

- I think the change should call at least HeapRegion::hr_clear() on the
region to remove or reset any auxiliary data structures, if not
G1CollectedHeap::free_region() (without adding the region to the free
list).
Since the HeapRegion* is not deallocated by the uncommit, this may cause
strange behavior later when the region is reused.

Other than that it looks okay.

Thanks,
  Thomas




From thomas.schatzl at oracle.com  Wed Aug 12 11:02:37 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 12 Aug 2015 13:02:37 +0200
Subject: RFR (S): 8073146: G1 merges thread local age tables too early
	with global age table
In-Reply-To: <55CB020C.8010803@oracle.com>
References: <1438949116.2305.43.camel@oracle.com> <55CB020C.8010803@oracle.com>
Message-ID: <1439377357.2324.28.camel@oracle.com>

Hi David,

On Wed, 2015-08-12 at 10:21 +0200, David Lindholm wrote:
> Thomas,
> 
> This change looks good. Reviewed.
> 

  thanks for the review.

Thomas



From kjkoster at gmail.com  Wed Aug 12 11:28:10 2015
From: kjkoster at gmail.com (Kees Jan Koster)
Date: Wed, 12 Aug 2015 13:28:10 +0200
Subject: Low-Overhead Heap Profiling
In-Reply-To: <CAPYFHW00zGByRRTz3m_Xx=2OndU9WybK6=vQt_-cchDqe3fuZg@mail.gmail.com>
References: <etPan.5589bc61.5358f81b.1215a@tw-mbp-tprintezis>
	<CAPYFHW3fVjDfc1j6du5dbAvaaKZsErx7vL8njx7hFhe-Ybvk6Q@mail.gmail.com>
	<etPan.558aef95.6b2c6c0b.1215a@tw-mbp-tprintezis>
	<etPan.558c716e.236c4c7e.1215a@tw-mbp-tprintezis>
	<CAPYFHW2ryBf7DyAsEyAPW46atiWvFQpvYV-9HROaB=yp1=2p-A@mail.gmail.com>
	<B3482D30-E883-4002-A8D6-8651A1D3349E@oracle.com>
	<CAPYFHW00zGByRRTz3m_Xx=2OndU9WybK6=vQt_-cchDqe3fuZg@mail.gmail.com>
Message-ID: <035570A0-5A63-4A99-96EE-D8A067A33B14@gmail.com>

Guys,

This piqued my interest. Where can I find the draft JEP and the sources for this, please?

Kees Jan


> On 4 Aug 2015, at 23:22, Jeremy Manson <jeremymanson at google.com> wrote:
> 
> Thanks, Staffan.  I've been tinkering with a draft JEP, but it has been going slowly because of other craziness in my life.  Some responses:
> 
> 1) It was a fair bit of work to do it, mostly because it was the first time I had tinkered with c2.  This was 5 years ago.  Most of the work since then has been dealing with merge conflicts.
> 
> 2) I did envision a JVMTI interface.  More in the JEP.
> 
> 3) We have some internal tests, but obviously, we'd have to figure out how to externalize them.  For native code, it's much easier only to have to worry about building and running on Linux.  Presumably, there's already code in there for JVMTI.
> 
> I'll try to get a JEP going for real, although it might not happen in the next week or so, because I'm moving next week (+the JVMLS).
> 
> Jeremy
> 
> On Tue, Aug 4, 2015 at 4:04 AM, Staffan Larsen <staffan.larsen at oracle.com> wrote:
> Hi,
> 
> Sorry for being away for so long.
> 
> If memory serves me right then the current AllocObjectInNewTLAB JFR event was written as a way to quickly get some allocation profiling information with a minimum of VM changes. It probably also carried over to Hotspot from JRockit. I agree that we can and should collect better information than what that event does.
> 
> I like the approach of instrumenting the interpreter and compiler to do the sampling. I had thought it would be a lot of work to do it and was reluctant to go down that path. If the work is already done and Jeremy has maintained it for a few years without great problems, I think we should look at bringing it in. I am not worried about the overhead of a decrement and a compare in the allocation path, but of course we should benchmark that.
> 
> It wasn?t clear to me from the discussion how (or if) this was being exposed to end users. Was the proposal to just have the native entry points in the VM and have these be used by various agents? Would this be part of JVMTI?
> 
> I think a draft JEP would be the logical next step and make it easier for us all to discuss what exactly the proposal should contain. Also, let?s not forget the need for tests for the feature.
> 
> Thanks,
> /Staffan
> 
> 
> 


--
Kees Jan

http://java-monitor.com/
kjkoster at kjkoster.org
+31651838192

The secret of success lies in the stability of the goal. -- Benjamin Disraeli

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 455 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150812/8adf0387/signature.asc>

From tom.benson at oracle.com  Wed Aug 12 12:31:46 2015
From: tom.benson at oracle.com (Tom Benson)
Date: Wed, 12 Aug 2015 08:31:46 -0400
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <CEC0388F-B4EC-4851-8E29-CC3B4CDA7C19@oracle.com>
References: <55C103A4.1060505@oracle.com>	<A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>	<1438781844.2378.60.camel@oracle.com>	<55C21DC3.5090305@oracle.com>	<1438785532.2378.73.camel@oracle.com>	<55C248FF.6010308@oracle.com>	<F73C4B9A-4DBA-47A8-8715-0988FF6D393D@oracle.com>	<55C26249.1090902@oracle.com>	<C149705F-696B-46CD-8FB8-CAD06C511244@oracle.com>	<55C2751B.8070400@oracle.com>	<1438847329.2009.9.camel@oracle.com>	<55C36B31.7000908@oracle.com>	<1438871723.2474.37.camel@oracle.com>	<55C373F6.9060609@oracle.com>	<AD45178E-A442-4A0D-80D6-50E91608E1FE@oracle.com>	<55C3D8BB.9000603@oracle.com>	<48C0E571-0D7B-4D30-85DF-E31D99092A7C@oracle.com>	<30B0248C-3BEA-4FAE-861F-DAAE19F56B45@oracle.com>	<55C4B806.7050504@oracle.com>
	<55C4C720.5030903@oracle.com> <55CA3430.3070300@oracle.com>
	<9255850A-182F-4741-BE2A-C7BF72057603@oracle.com>
	<55CA6BBC.8080605@oracle.com>
	<CE5338A0-89AC-4701-AD8E-2D6D9349A5D0@oracle.com>
	<CEC0388F-B4EC-4851-8E29-CC3B4CDA7C19@oracle.com>
Message-ID: <55CB3CB2.2000307@oracle.com>

Hi Kim -

On 8/11/2015 8:10 PM, Kim Barrett wrote:
> On Aug 11, 2015, at 8:02 PM, Kim Barrett <kim.barrett at oracle.com> wrote:
>> .....
> Oops, forgot the important part of the reply.
>
> Change looks good.
>
Thanks.   I see Thomas addressed your other (outside this change) comments.
Tom


From mikael.gerdin at oracle.com  Wed Aug 12 13:13:23 2015
From: mikael.gerdin at oracle.com (Mikael Gerdin)
Date: Wed, 12 Aug 2015 15:13:23 +0200
Subject: RFR (L): 8073013: Add detailed information about PLAB memory usage
In-Reply-To: <1439369047.2324.1.camel@oracle.com>
References: <1438876151.2474.51.camel@oracle.com>	<55C4F909.7050601@oracle.com>
	<1439211707.2330.11.camel@oracle.com>	<55C9F395.8050008@oracle.com>
	<1439369047.2324.1.camel@oracle.com>
Message-ID: <55CB4673.6070402@oracle.com>

Hi Thomas,

On 2015-08-12 10:44, Thomas Schatzl wrote:
> Hi David,
>
>    thanks for the review:
>
> On Tue, 2015-08-11 at 15:07 +0200, David Lindholm wrote:
>> Hi Thomas.
>>
>> This looks good, except for the changes in trace.xml. All fields should
>> follow Java naming conventions, for example regionEndWaste, not
>> region_end_waste. Also, 2 different event types are preferred according
>> to the Servicability folks (instead of the "gen" field). Suggested names
>> are GCG1EvacuationYoungStatistics and GCG1EvacuationOldStatistics.
>>
>> The other parts looks good.
>
> I think all fixed in the new webrevs at:
>
> http://cr.openjdk.java.net/~tschatzl/8073013/webrev.2 (full)
> http://cr.openjdk.java.net/~tschatzl/8073013/webrev.1_to_2 (diff)

I only had a cursory look at the change, overall it seems good and I 
know there's a large patch series to manage here.
However, in plab.hpp you've added includes of allocation.inline.hpp and 
atomic.inline.hpp. In general we should always avoid including inline 
headers in normal headers, it's not obvious to me why your other changes 
to plab.hpp would require the include change.

/Mikael

>
> Thanks,
>    Thomas
>
>



From mikael.gerdin at oracle.com  Wed Aug 12 13:27:01 2015
From: mikael.gerdin at oracle.com (Mikael Gerdin)
Date: Wed, 12 Aug 2015 15:27:01 +0200
Subject: RFR (S): 8073146: G1 merges thread local age tables too early
	with global age table
In-Reply-To: <1439377357.2324.28.camel@oracle.com>
References: <1438949116.2305.43.camel@oracle.com> <55CB020C.8010803@oracle.com>
	<1439377357.2324.28.camel@oracle.com>
Message-ID: <55CB49A5.40804@oracle.com>

On 2015-08-12 13:02, Thomas Schatzl wrote:
> Hi David,
>
> On Wed, 2015-08-12 at 10:21 +0200, David Lindholm wrote:
>> Thomas,
>>
>> This change looks good. Reviewed.

Looks good to me as well.
/Mikael

>>
>
>    thanks for the review.
>
> Thomas
>



From stefan.karlsson at oracle.com  Wed Aug 12 13:29:04 2015
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Wed, 12 Aug 2015 15:29:04 +0200
Subject: RFR (XXS): JDK-8085983
	G1CollectedHeap::collection_set_iterate_from()
	has unused code and can be simplified
In-Reply-To: <55C23996.3020406@oracle.com>
References: <5575EA21.4010803@oracle.com> <557869E2.4080905@oracle.com>
	<5582DB1A.6070609@oracle.com> <55C23996.3020406@oracle.com>
Message-ID: <55CB4A20.3010106@oracle.com>

Hi Joe,

On 2015-08-05 18:28, Joseph Provino wrote:
> Latest webrev is here: 
> http://cr.openjdk.java.net/~jprovino/8085983/webrev.01

Is this the latest webrev? My comments have not been addressed in this 
patch.

Thanks,
StefanK

>
> joe
>
> On 6/18/2015 10:52 AM, Joseph Provino wrote:
>>
>> On 6/10/2015 12:46 PM, Stefan Karlsson wrote:
>>> Hi Joe,
>>>
>>> On 2015-06-08 21:16, Joseph Provino wrote:
>>>> One file.
>>>>
>>>> CR: https://bugs.openjdk.java.net/browse/JDK-8085983
>>>>
>>>> webrev: http://cr.openjdk.java.net/~jprovino/8085983/webrev.00
>>>
>>> Some comments:
>>>
>>> *+      guarantee(!cl->doHeapRegion(cur), err_msg("doHeapRegion() returned true"));*
>>>
>>> - You don't need err_msg if you don't have any format specifiers in 
>>> the error message.
>>>
>> I'll fix that.
>>
>>> - I would prefer if you didn't put the cl->doHeapReagion() calls 
>>> inside guarantees. It's too easy to incorrectly assume that the code 
>>> in the guarantee is only verification code, IMHO.
>>>
>> That's a good point.  Something like this:
>>
>>     bool ret = cl->doHeapRegion(r);
>>     guarantee(ret == false, "doHeapRegion() returned true");
>>
>> Or do you prefer !ret?
>>
>>> - Shouldn't this be an assert instead of a guarantee?
>> I don't know the rules there.  Seems like assert would be better
>> so it doesn't slow down the product build.
>>
>> thanks.
>>
>> joe
>>
>>>
>>> Thanks,
>>> StefanK
>>>
>>>>
>>>> passes jprt.
>>>>
>>>> thanks.
>>>>
>>>> joe
>>>>
>>>>
>>>
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150812/534dedf1/attachment.htm>

From mikael.gerdin at oracle.com  Wed Aug 12 13:57:17 2015
From: mikael.gerdin at oracle.com (Mikael Gerdin)
Date: Wed, 12 Aug 2015 15:57:17 +0200
Subject: RFR (M): 8040162: Avoid reallocating PLABs between GC phases in G1
In-Reply-To: <1439375904.2324.16.camel@oracle.com>
References: <1438949102.2305.42.camel@oracle.com>	<55C9F5CF.8040006@oracle.com>
	<1439305012.2327.1.camel@oracle.com>
	<1439375904.2324.16.camel@oracle.com>
Message-ID: <55CB50BD.6040507@oracle.com>

Hi Thomas,

On 2015-08-12 12:38, Thomas Schatzl wrote:
> Hi,
>
> On Tue, 2015-08-11 at 16:56 +0200, Thomas Schatzl wrote:
>> Hi David,
>>
>> On Tue, 2015-08-11 at 15:17 +0200, David Lindholm wrote:
>>> Hi Thomas,
>>>
>>> Great work. Looks good, reviewed.
>>
>>    thanks for the review.
>
>    while testing the change a little more an issue with -XX:
> +ParallelRefProcEnabled has been found.
>
> Here is a new webrev:
> http://cr.openjdk.java.net/~tschatzl/8040162/webrev.0_to_1/ (diff)
> http://cr.openjdk.java.net/~tschatzl/8040162/webrev.1/ (full)

in g1CollectedHeap.cpp at
4571       }
4572
4573       _root_processor->evacuate_roots(strong_root_cl,
4574                                       weak_root_cl,
4575                                       strong_cld_cl,
4576                                       weak_cld_cl,
4577                                       trace_metadata,
4578                                       worker_id);
4579
4580       G1ParPushHeapRSClosure push_heap_rs_cl(_g1h, pss);
4581       double start_strong_roots_sec = os::elapsedTime();
4582       _root_processor->scan_remembered_sets(&push_heap_rs_cl,
4583                                             weak_root_cl,
4584                                             worker_id);
4585       double strong_roots_sec = os::elapsedTime() - 
start_strong_roots_sec;
4586

Shouldn't start_strong_roots_sec be taken before the call to 
evacuate_roots? Otherwise it just measures the time it takes to scan the 
remembered sets.

5400
5401 // Weak Reference processing during an evacuation pause (part 1).
5402 void 
G1CollectedHeap::process_discovered_references(G1ParScanThreadState** 
pss_) {
5403   double ref_proc_start = os::elapsedTime();
5404

If you wan the local to be called pss, can you name the ** 
per_thread_states as it's named in the caller?

in g1OopClosures.hpp
Would you consider moving set_ref_processor to G1ParScanClosure since 
it's only used for that particular concrete type?
As I understand it it's only needed for the _scanner G1ParScanClosure 
instance in the G1PSS that you need to be able to set the ref processor.

Otherwise the change looks good to me.

/Mikael

>
> Thanks,
>    Thomas
>
>



From david.lindholm at oracle.com  Wed Aug 12 14:15:24 2015
From: david.lindholm at oracle.com (David Lindholm)
Date: Wed, 12 Aug 2015 16:15:24 +0200
Subject: RFR: 8133193: Memory leak in G1 because G1RootProcessor doesn't have
	desctructor
Message-ID: <55CB54FC.5090204@oracle.com>

Hi,

Please review this small patch that prevents a memory leak in G1.

Bug: https://bugs.openjdk.java.net/browse/JDK-8133193
Webrev: http://cr.openjdk.java.net/~david/JDK-8133193/webrev.00/

Testing: JPRT.


Thanks,
David


From mikael.gerdin at oracle.com  Wed Aug 12 14:30:04 2015
From: mikael.gerdin at oracle.com (Mikael Gerdin)
Date: Wed, 12 Aug 2015 16:30:04 +0200
Subject: RFR: 8133193: Memory leak in G1 because G1RootProcessor doesn't
	have desctructor
In-Reply-To: <55CB54FC.5090204@oracle.com>
References: <55CB54FC.5090204@oracle.com>
Message-ID: <55CB586C.3050804@oracle.com>

Hi David,

On 2015-08-12 16:15, David Lindholm wrote:
> Hi,
>
> Please review this small patch that prevents a memory leak in G1.
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8133193
> Webrev: http://cr.openjdk.java.net/~david/JDK-8133193/webrev.00/

Looks good to me.

/Mikael

>
> Testing: JPRT.
>
>
> Thanks,
> David



From thomas.schatzl at oracle.com  Wed Aug 12 14:31:32 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 12 Aug 2015 16:31:32 +0200
Subject: RFR: 8133193: Memory leak in G1 because G1RootProcessor doesn't
	have desctructor
In-Reply-To: <55CB54FC.5090204@oracle.com>
References: <55CB54FC.5090204@oracle.com>
Message-ID: <1439389892.2324.62.camel@oracle.com>

Hi,

On Wed, 2015-08-12 at 16:15 +0200, David Lindholm wrote:
> Hi,
> 
> Please review this small patch that prevents a memory leak in G1.
> 
> Bug: https://bugs.openjdk.java.net/browse/JDK-8133193
> Webrev: http://cr.openjdk.java.net/~david/JDK-8133193/webrev.00/
> 
> Testing: JPRT.

  looks good.

Thomas




From staffan.friberg at oracle.com  Wed Aug 12 17:23:45 2015
From: staffan.friberg at oracle.com (Staffan Friberg)
Date: Wed, 12 Aug 2015 10:23:45 -0700
Subject: RFR JDK-8066443: Add G1 support for promotion event
Message-ID: <55CB8121.1080502@oracle.com>

Hi,

This webrev adds support for the G1 collector and the promotion event 
that collects statistics about the promoted object types.

Once reviewed I would need a sponsor to push the patch.

Bug: https://bugs.openjdk.java.net/browse/JDK-8066443
Webrev: http://cr.openjdk.java.net/~sfriberg/JDK-8066443/webrev.01/

//Staffan


From kim.barrett at oracle.com  Wed Aug 12 17:45:59 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Wed, 12 Aug 2015 13:45:59 -0400
Subject: RFR (XS): Move implementation of process_grey_object to
	concurrentMark.inline.hpp
In-Reply-To: <CA+3eh12qqoE06vjee8HPsno4gLAo_4PnR71LjPWewr6LQ+cypA@mail.gmail.com>
References: <02D5D45C1F8DB848A7AE20E80EE61A5C39962EC5@DEWDFEMB20C.global.corp.sap>
	<CA+3eh12qqoE06vjee8HPsno4gLAo_4PnR71LjPWewr6LQ+cypA@mail.gmail.com>
Message-ID: <35C5EA3B-DEE0-4DBB-972E-5DC0E58C04F2@oracle.com>

On Aug 11, 2015, at 8:21 AM, Volker Simonis <volker.simonis at gmail.com> wrote:
> 
> Hi Axel,
> 
> the change looks good!
> I think we should always define template functions in .inline.hpp files.
> 
> Can somebody please sponsor this change?
> 
> Thank you and best regards,
> Volker
> 
> 
> On Thu, Aug 6, 2015 at 3:49 PM, Siebenborn, Axel
> <axel.siebenborn at sap.com> wrote:
>> Hi,
>> Could I have reviews and a sponsor for this small change
>> 
>> We had a linker problem with a C++ - compiler that inlined process_grey_object at all uses but removed the symbol and the actual implementation.
>> As there are explicit instantiations of this method in concurrentMark.cpp, this probably should not happen and can be considered of a bug of the C++ compiler. Though, it seems to be pointless to mark that method inline if there are explicit instantiations.
>> 
>> Considering performance, it would be better to move the implementation of process_grey_object to concurrentMark.inline.hpp in order to inline the method in other compilation units.
>> 
>> CR:
>> https://bugs.openjdk.java.net/browse/JDK-8133121
>> 
>> Webrev:
>> http://cr.openjdk.java.net/~asiebenborn/8133121/webrev/
>> 
>> 
>> Thanks,
>> Axel

I can sponsor this, once I?ve reviewed it.  I authored the relevant code, and want to look it and the change over and try to recall why it was written the way it was.  The existing code is slightly odd, and I wouldn?t have written it that way without some reason?



From volker.simonis at gmail.com  Wed Aug 12 18:26:34 2015
From: volker.simonis at gmail.com (Volker Simonis)
Date: Wed, 12 Aug 2015 20:26:34 +0200
Subject: RFR (XS): Move implementation of process_grey_object to
	concurrentMark.inline.hpp
In-Reply-To: <35C5EA3B-DEE0-4DBB-972E-5DC0E58C04F2@oracle.com>
References: <02D5D45C1F8DB848A7AE20E80EE61A5C39962EC5@DEWDFEMB20C.global.corp.sap>
	<CA+3eh12qqoE06vjee8HPsno4gLAo_4PnR71LjPWewr6LQ+cypA@mail.gmail.com>
	<35C5EA3B-DEE0-4DBB-972E-5DC0E58C04F2@oracle.com>
Message-ID: <CA+3eh11R4O=wgspNr=BvkCTeGGKYL5dOV89Ey7a5Lkk4sO0SWA@mail.gmail.com>

Hi Kim,

thanks for looking into this issue.

We've mulled over the "slightly odd" coding for a while as well and
our explanation was that you may have got unsatisfied references to
the instantiations of process_grey_object() as well because they are
only defined in the .cpp file and that may have been the reason why
you have added the implicit instantiations.

Regards,
Volker


On Wed, Aug 12, 2015 at 7:45 PM, Kim Barrett <kim.barrett at oracle.com> wrote:
> On Aug 11, 2015, at 8:21 AM, Volker Simonis <volker.simonis at gmail.com> wrote:
>>
>> Hi Axel,
>>
>> the change looks good!
>> I think we should always define template functions in .inline.hpp files.
>>
>> Can somebody please sponsor this change?
>>
>> Thank you and best regards,
>> Volker
>>
>>
>> On Thu, Aug 6, 2015 at 3:49 PM, Siebenborn, Axel
>> <axel.siebenborn at sap.com> wrote:
>>> Hi,
>>> Could I have reviews and a sponsor for this small change
>>>
>>> We had a linker problem with a C++ - compiler that inlined process_grey_object at all uses but removed the symbol and the actual implementation.
>>> As there are explicit instantiations of this method in concurrentMark.cpp, this probably should not happen and can be considered of a bug of the C++ compiler. Though, it seems to be pointless to mark that method inline if there are explicit instantiations.
>>>
>>> Considering performance, it would be better to move the implementation of process_grey_object to concurrentMark.inline.hpp in order to inline the method in other compilation units.
>>>
>>> CR:
>>> https://bugs.openjdk.java.net/browse/JDK-8133121
>>>
>>> Webrev:
>>> http://cr.openjdk.java.net/~asiebenborn/8133121/webrev/
>>>
>>>
>>> Thanks,
>>> Axel
>
> I can sponsor this, once I?ve reviewed it.  I authored the relevant code, and want to look it and the change over and try to recall why it was written the way it was.  The existing code is slightly odd, and I wouldn?t have written it that way without some reason?
>


From kim.barrett at oracle.com  Wed Aug 12 19:57:48 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Wed, 12 Aug 2015 15:57:48 -0400
Subject: RFR (XS): Move implementation of process_grey_object to
	concurrentMark.inline.hpp
In-Reply-To: <CA+3eh11R4O=wgspNr=BvkCTeGGKYL5dOV89Ey7a5Lkk4sO0SWA@mail.gmail.com>
References: <02D5D45C1F8DB848A7AE20E80EE61A5C39962EC5@DEWDFEMB20C.global.corp.sap>
	<CA+3eh12qqoE06vjee8HPsno4gLAo_4PnR71LjPWewr6LQ+cypA@mail.gmail.com>
	<35C5EA3B-DEE0-4DBB-972E-5DC0E58C04F2@oracle.com>
	<CA+3eh11R4O=wgspNr=BvkCTeGGKYL5dOV89Ey7a5Lkk4sO0SWA@mail.gmail.com>
Message-ID: <16192E09-14D6-4263-832B-36C6CAE61513@oracle.com>

On Aug 12, 2015, at 2:26 PM, Volker Simonis <volker.simonis at gmail.com> wrote:
> 
> Hi Kim,
> 
> thanks for looking into this issue.
> 
> We've mulled over the "slightly odd" coding for a while as well and
> our explanation was that you may have got unsatisfied references to
> the instantiations of process_grey_object() as well because they are
> only defined in the .cpp file and that may have been the reason why
> you have added the implicit instantiations.
> 
> Regards,
> Volker

That's part of it.  I wrote a much longer reply, but decided it was
tl;dr.  The short version is that I think the proposed change is an
improvement.  That it dodges what appears to be a compiler bug is a
happy bonus.  (If Axel has time, I'm mildly curious whether just
removing the inline qualifier also works around the problem.)

So I think the change looks good.

I have one optional change to suggest, which is to move the inline
definition of scan_object from the .hpp file to the .inline.hpp file.
I think putting it in the .hpp file was a mistake on my part.  Axel,
let me know whether you want to make that additional change.


>>> On Thu, Aug 6, 2015 at 3:49 PM, Siebenborn, Axel
>>> <axel.siebenborn at sap.com> wrote:
>>>> Hi,
>>>> Could I have reviews and a sponsor for this small change
>>>> 
>>>> We had a linker problem with a C++ - compiler that inlined process_grey_object at all uses but removed the symbol and the actual implementation.
>>>> As there are explicit instantiations of this method in concurrentMark.cpp, this probably should not happen and can be considered of a bug of the C++ compiler. Though, it seems to be pointless to mark that method inline if there are explicit instantiations.
>>>> 
>>>> Considering performance, it would be better to move the implementation of process_grey_object to concurrentMark.inline.hpp in order to inline the method in other compilation units.
>>>> 
>>>> CR:
>>>> https://bugs.openjdk.java.net/browse/JDK-8133121
>>>> 
>>>> Webrev:
>>>> http://cr.openjdk.java.net/~asiebenborn/8133121/webrev/
>>>> 
>>>> 
>>>> Thanks,
>>>> Axel




From kim.barrett at oracle.com  Wed Aug 12 20:01:32 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Wed, 12 Aug 2015 16:01:32 -0400
Subject: RFR (XS): Move implementation of process_grey_object to
	concurrentMark.inline.hpp 
In-Reply-To: <02D5D45C1F8DB848A7AE20E80EE61A5C39962EC5@DEWDFEMB20C.global.corp.sap>
References: <02D5D45C1F8DB848A7AE20E80EE61A5C39962EC5@DEWDFEMB20C.global.corp.sap>
Message-ID: <3199F070-071D-4A00-9DBA-2E8D56353F53@oracle.com>

On Aug 6, 2015, at 9:49 AM, Siebenborn, Axel <axel.siebenborn at sap.com> wrote:
> 
> We had a linker problem with a C++ - compiler that inlined process_grey_object at all uses but removed the symbol and the actual implementation.
> As there are explicit instantiations of this method in concurrentMark.cpp, this probably should not happen and can be considered of a bug of the C++ compiler.

Did you mean ?did not inline all uses??  Otherwise, I?m not sure what the problem might be.




From volker.simonis at gmail.com  Thu Aug 13 06:45:08 2015
From: volker.simonis at gmail.com (Volker Simonis)
Date: Thu, 13 Aug 2015 08:45:08 +0200
Subject: RFR (XS): Move implementation of process_grey_object to
	concurrentMark.inline.hpp
In-Reply-To: <3199F070-071D-4A00-9DBA-2E8D56353F53@oracle.com>
References: <02D5D45C1F8DB848A7AE20E80EE61A5C39962EC5@DEWDFEMB20C.global.corp.sap>
	<3199F070-071D-4A00-9DBA-2E8D56353F53@oracle.com>
Message-ID: <CA+3eh10Z7bMdcSQrej2BCQ_4gnryzJfpAD1T_rDers7O8OX0uQ@mail.gmail.com>

On Wednesday, August 12, 2015, Kim Barrett <kim.barrett at oracle.com> wrote:

> On Aug 6, 2015, at 9:49 AM, Siebenborn, Axel <axel.siebenborn at sap.com
> <javascript:;>> wrote:
> >
> > We had a linker problem with a C++ - compiler that inlined
> process_grey_object at all uses but removed the symbol and the actual
> implementation.
> > As there are explicit instantiations of this method in
> concurrentMark.cpp, this probably should not happen and can be considered
> of a bug of the C++ compiler.
>
> Did you mean ?did not inline all uses??  Otherwise, I?m not sure what the
> problem might be.
>
>  "Inline all uses" when compiling concurrentMark.cpp. That's why the
explicit instantiations were removed from concurrentMark.o. But there are
other compilation units which reference process_grey_objec(), hence the
undefined symbols during linking.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150813/5cb74a7f/attachment.htm>

From axel.siebenborn at sap.com  Thu Aug 13 09:55:07 2015
From: axel.siebenborn at sap.com (Siebenborn, Axel)
Date: Thu, 13 Aug 2015 11:55:07 +0200
Subject: RFR (XS): Move implementation of process_grey_object to
	concurrentMark.inline.hpp
In-Reply-To: <16192E09-14D6-4263-832B-36C6CAE61513@oracle.com>
References: <02D5D45C1F8DB848A7AE20E80EE61A5C39962EC5@DEWDFEMB20C.global.corp.sap>
	<CA+3eh12qqoE06vjee8HPsno4gLAo_4PnR71LjPWewr6LQ+cypA@mail.gmail.com>
	<35C5EA3B-DEE0-4DBB-972E-5DC0E58C04F2@oracle.com>
	<CA+3eh11R4O=wgspNr=BvkCTeGGKYL5dOV89Ey7a5Lkk4sO0SWA@mail.gmail.com>
	<16192E09-14D6-4263-832B-36C6CAE61513@oracle.com>
Message-ID: <55CC697B.5020202@sap.com>

Hi Kim,

On 12.08.2015, at 21:57, Kim Barrett wrote:
> On Aug 12, 2015, at 2:26 PM, Volker Simonis <volker.simonis at gmail.com> wrote:
>> Hi Kim,
>>
>> thanks for looking into this issue.
>>
>> We've mulled over the "slightly odd" coding for a while as well and
>> our explanation was that you may have got unsatisfied references to
>> the instantiations of process_grey_object() as well because they are
>> only defined in the .cpp file and that may have been the reason why
>> you have added the implicit instantiations.
>>
>> Regards,
>> Volker
> That's part of it.  I wrote a much longer reply, but decided it was
> tl;dr.  The short version is that I think the proposed change is an
> improvement.  That it dodges what appears to be a compiler bug is a
> happy bonus.  (If Axel has time, I'm mildly curious whether just
> removing the inline qualifier also works around the problem.)
We were curious too and  indeed, removing the inline qualifier would 
work around the problem.
> So I think the change looks good.
>
> I have one optional change to suggest, which is to move the inline
> definition of scan_object from the .hpp file to the .inline.hpp file.
> I think putting it in the .hpp file was a mistake on my part.  Axel,
> let me know whether you want to make that additional change.
Yes, there is the new webrev:

http://cr.openjdk.java.net/~asiebenborn/8133121_0/webrev/

>
>
>>>> On Thu, Aug 6, 2015 at 3:49 PM, Siebenborn, Axel
>>>> <axel.siebenborn at sap.com> wrote:
>>>>> Hi,
>>>>> Could I have reviews and a sponsor for this small change
>>>>>
>>>>> We had a linker problem with a C++ - compiler that inlined process_grey_object at all uses but removed the symbol and the actual implementation.
>>>>> As there are explicit instantiations of this method in concurrentMark.cpp, this probably should not happen and can be considered of a bug of the C++ compiler. Though, it seems to be pointless to mark that method inline if there are explicit instantiations.
>>>>>
>>>>> Considering performance, it would be better to move the implementation of process_grey_object to concurrentMark.inline.hpp in order to inline the method in other compilation units.
>>>>>
>>>>> CR:
>>>>> https://bugs.openjdk.java.net/browse/JDK-8133121
>>>>>
>>>>> Webrev:
>>>>> http://cr.openjdk.java.net/~asiebenborn/8133121/webrev/
>>>>>
>>>>>
>>>>> Thanks,
>>>>> Axel
Thanks for reviewing and sponsoring,
Axel



From erik.helin at oracle.com  Thu Aug 13 10:33:38 2015
From: erik.helin at oracle.com (Erik Helin)
Date: Thu, 13 Aug 2015 12:33:38 +0200
Subject: RFR (L): 8073013: Add detailed information about PLAB memory usage
In-Reply-To: <1439369047.2324.1.camel@oracle.com>
References: <1438876151.2474.51.camel@oracle.com> <55C4F909.7050601@oracle.com>
	<1439211707.2330.11.camel@oracle.com> <55C9F395.8050008@oracle.com>
	<1439369047.2324.1.camel@oracle.com>
Message-ID: <20150813103338.GA13718@ehelin.jrpg.bea.com>

Hi Thomas,

On 2015-08-12, Thomas Schatzl wrote:
> Hi David,
> 
>   thanks for the review:
> 
> On Tue, 2015-08-11 at 15:07 +0200, David Lindholm wrote:
> > Hi Thomas.
> > 
> > This looks good, except for the changes in trace.xml. All fields should 
> > follow Java naming conventions, for example regionEndWaste, not 
> > region_end_waste. Also, 2 different event types are preferred according 
> > to the Servicability folks (instead of the "gen" field). Suggested names 
> > are GCG1EvacuationYoungStatistics and GCG1EvacuationOldStatistics.
> > 
> > The other parts looks good.
> 
> I think all fixed in the new webrevs at:
> 
> http://cr.openjdk.java.net/~tschatzl/8073013/webrev.2 (full)
> http://cr.openjdk.java.net/~tschatzl/8073013/webrev.1_to_2 (diff)

just two comments:
- could you split out the trace event into another patch to make this
  patch smaller?
- I have not reviewed the algorithm for
  G1EvacStats::adjust_desired_plab_sz in any more depth than veryfing
  that the copy from plab.cpp seems correct (I also know that there are
  more patches coming with changes to this logic).

Other than splitting out the trace event and Mikael's comment, I think
the patch looks good.

Thanks,
Erik

> Thanks,
>   Thomas
> 
> 


From thomas.schatzl at oracle.com  Thu Aug 13 10:39:58 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 13 Aug 2015 12:39:58 +0200
Subject: RFR (M): 8040162: Avoid reallocating PLABs between GC phases in G1
In-Reply-To: <55CB50BD.6040507@oracle.com>
References: <1438949102.2305.42.camel@oracle.com>
	<55C9F5CF.8040006@oracle.com> <1439305012.2327.1.camel@oracle.com>
	<1439375904.2324.16.camel@oracle.com> <55CB50BD.6040507@oracle.com>
Message-ID: <1439462398.2271.3.camel@oracle.com>

Hi,

On Wed, 2015-08-12 at 15:57 +0200, Mikael Gerdin wrote:
> Hi Thomas,
> 
> On 2015-08-12 12:38, Thomas Schatzl wrote:
> > Hi,
> >
> > On Tue, 2015-08-11 at 16:56 +0200, Thomas Schatzl wrote:
> >> Hi David,
> >>
> >> On Tue, 2015-08-11 at 15:17 +0200, David Lindholm wrote:
> >>> Hi Thomas,
> >>>
> >>> Great work. Looks good, reviewed.
> >>
> >>    thanks for the review.
> >
> >    while testing the change a little more an issue with -XX:
> > +ParallelRefProcEnabled has been found.
> >
> > Here is a new webrev:
> > http://cr.openjdk.java.net/~tschatzl/8040162/webrev.0_to_1/ (diff)
> > http://cr.openjdk.java.net/~tschatzl/8040162/webrev.1/ (full)
> 
> in g1CollectedHeap.cpp at
> 4571       }
> 4572
> 4573       _root_processor->evacuate_roots(strong_root_cl,
> 4574                                       weak_root_cl,
> 4575                                       strong_cld_cl,
> 4576                                       weak_cld_cl,
> 4577                                       trace_metadata,
> 4578                                       worker_id);
> 4579
> 4580       G1ParPushHeapRSClosure push_heap_rs_cl(_g1h, pss);
> 4581       double start_strong_roots_sec = os::elapsedTime();
> 4582       _root_processor->scan_remembered_sets(&push_heap_rs_cl,
> 4583                                             weak_root_cl,
> 4584                                             worker_id);
> 4585       double strong_roots_sec = os::elapsedTime() - 
> start_strong_roots_sec;
> 4586
> 
> Shouldn't start_strong_roots_sec be taken before the call to 
> evacuate_roots? Otherwise it just measures the time it takes to scan the 
> remembered sets.
> 
> 5400
> 5401 // Weak Reference processing during an evacuation pause (part 1).
> 5402 void 
> G1CollectedHeap::process_discovered_references(G1ParScanThreadState** 
> pss_) {
> 5403   double ref_proc_start = os::elapsedTime();
> 5404
> 
> If you wan the local to be called pss, can you name the ** 
> per_thread_states as it's named in the caller?
> 
> in g1OopClosures.hpp
> Would you consider moving set_ref_processor to G1ParScanClosure since 
> it's only used for that particular concrete type?
> As I understand it it's only needed for the _scanner G1ParScanClosure 
> instance in the G1PSS that you need to be able to set the ref processor.
> 
> Otherwise the change looks good to me.

All fixed. Thanks particularly for catching the issue with the
start_strong_roots_sec.

Thanks for the review.

Here are new webrevs.

http://cr.openjdk.java.net/~tschatzl/8040162/webrev.1_to_2/ (diff)
http://cr.openjdk.java.net/~tschatzl/8040162/webrev.2/ (full)

Thanks,
  Thomas




From mikael.gerdin at oracle.com  Thu Aug 13 10:48:09 2015
From: mikael.gerdin at oracle.com (Mikael Gerdin)
Date: Thu, 13 Aug 2015 12:48:09 +0200
Subject: RFR (M): 8040162: Avoid reallocating PLABs between GC phases in G1
In-Reply-To: <1439462398.2271.3.camel@oracle.com>
References: <1438949102.2305.42.camel@oracle.com>		<55C9F5CF.8040006@oracle.com>
	<1439305012.2327.1.camel@oracle.com>	
	<1439375904.2324.16.camel@oracle.com> <55CB50BD.6040507@oracle.com>
	<1439462398.2271.3.camel@oracle.com>
Message-ID: <55CC75E9.1060104@oracle.com>

Thomas,

On 2015-08-13 12:39, Thomas Schatzl wrote:
> Hi,
>
> On Wed, 2015-08-12 at 15:57 +0200, Mikael Gerdin wrote:
>> Hi Thomas,
>>
>> On 2015-08-12 12:38, Thomas Schatzl wrote:
>>> Hi,
>>>
>>> On Tue, 2015-08-11 at 16:56 +0200, Thomas Schatzl wrote:
>>>> Hi David,
>>>>
>>>> On Tue, 2015-08-11 at 15:17 +0200, David Lindholm wrote:
>>>>> Hi Thomas,
>>>>>
>>>>> Great work. Looks good, reviewed.
>>>>
>>>>     thanks for the review.
>>>
>>>     while testing the change a little more an issue with -XX:
>>> +ParallelRefProcEnabled has been found.
>>>
>>> Here is a new webrev:
>>> http://cr.openjdk.java.net/~tschatzl/8040162/webrev.0_to_1/ (diff)
>>> http://cr.openjdk.java.net/~tschatzl/8040162/webrev.1/ (full)
>>
>> in g1CollectedHeap.cpp at
>> 4571       }
>> 4572
>> 4573       _root_processor->evacuate_roots(strong_root_cl,
>> 4574                                       weak_root_cl,
>> 4575                                       strong_cld_cl,
>> 4576                                       weak_cld_cl,
>> 4577                                       trace_metadata,
>> 4578                                       worker_id);
>> 4579
>> 4580       G1ParPushHeapRSClosure push_heap_rs_cl(_g1h, pss);
>> 4581       double start_strong_roots_sec = os::elapsedTime();
>> 4582       _root_processor->scan_remembered_sets(&push_heap_rs_cl,
>> 4583                                             weak_root_cl,
>> 4584                                             worker_id);
>> 4585       double strong_roots_sec = os::elapsedTime() -
>> start_strong_roots_sec;
>> 4586
>>
>> Shouldn't start_strong_roots_sec be taken before the call to
>> evacuate_roots? Otherwise it just measures the time it takes to scan the
>> remembered sets.
>>
>> 5400
>> 5401 // Weak Reference processing during an evacuation pause (part 1).
>> 5402 void
>> G1CollectedHeap::process_discovered_references(G1ParScanThreadState**
>> pss_) {
>> 5403   double ref_proc_start = os::elapsedTime();
>> 5404
>>
>> If you wan the local to be called pss, can you name the **
>> per_thread_states as it's named in the caller?
>>
>> in g1OopClosures.hpp
>> Would you consider moving set_ref_processor to G1ParScanClosure since
>> it's only used for that particular concrete type?
>> As I understand it it's only needed for the _scanner G1ParScanClosure
>> instance in the G1PSS that you need to be able to set the ref processor.
>>
>> Otherwise the change looks good to me.
>
> All fixed. Thanks particularly for catching the issue with the
> start_strong_roots_sec.
>
> Thanks for the review.
>
> Here are new webrevs.
>
> http://cr.openjdk.java.net/~tschatzl/8040162/webrev.1_to_2/ (diff)
> http://cr.openjdk.java.net/~tschatzl/8040162/webrev.2/ (full)

Looks good.
/Mikael

>
> Thanks,
>    Thomas
>
>



From thomas.schatzl at oracle.com  Thu Aug 13 10:49:39 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 13 Aug 2015 12:49:39 +0200
Subject: RFR (S): 8133470: Uses of Atomic methods in plab.hpp should be
	moved to .inline.hpp file
Message-ID: <1439462979.2271.11.camel@oracle.com>

Hi all,

  in the review for 8073013 one of the review questions was why plab.hpp
added atomic.inline.hpp, and that that is not good practice.

This has been a pre-existing bug, i.e. plab.hpp already used the Atomic
functions that reside in .inline.hpp.

This CR fixes this issue by moving the problematic calls into
plab.inline.hpp.

The only problem is that changing includes from plab.hpp to
plab.inline.hpp in cpp files and adding plab.hpp in header files opens
up dependency issues and other missing includes.

That's why this change is larger than it should.

Webrev:
https://bugs.openjdk.java.net/browse/JDK-8133470
CR:
http://cr.openjdk.java.net/~tschatzl/8133470/webrev/

Testing:
jprt

Thanks,
  Thomas



From thomas.schatzl at oracle.com  Thu Aug 13 10:54:38 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 13 Aug 2015 12:54:38 +0200
Subject: RFR (L): 8073013: Add detailed information about PLAB memory usage
In-Reply-To: <20150813103338.GA13718@ehelin.jrpg.bea.com>
References: <1438876151.2474.51.camel@oracle.com>
	<55C4F909.7050601@oracle.com> <1439211707.2330.11.camel@oracle.com>
	<55C9F395.8050008@oracle.com> <1439369047.2324.1.camel@oracle.com>
	<20150813103338.GA13718@ehelin.jrpg.bea.com>
Message-ID: <1439463278.2271.14.camel@oracle.com>

Hi Erik,

On Thu, 2015-08-13 at 12:33 +0200, Erik Helin wrote:
> Hi Thomas,
> 
> On 2015-08-12, Thomas Schatzl wrote:
> > Hi David,
> > 
> >   thanks for the review:
> > 
> > On Tue, 2015-08-11 at 15:07 +0200, David Lindholm wrote:
> > > Hi Thomas.
> > > 
> > > This looks good, except for the changes in trace.xml. All fields should 
> > > follow Java naming conventions, for example regionEndWaste, not 
> > > region_end_waste. Also, 2 different event types are preferred according 
> > > to the Servicability folks (instead of the "gen" field). Suggested names 
> > > are GCG1EvacuationYoungStatistics and GCG1EvacuationOldStatistics.
> > > 
> > > The other parts looks good.
> > 
> > I think all fixed in the new webrevs at:
> > 
> > http://cr.openjdk.java.net/~tschatzl/8073013/webrev.2 (full)
> > http://cr.openjdk.java.net/~tschatzl/8073013/webrev.1_to_2 (diff)
> 
> just two comments:
> - could you split out the trace event into another patch to make this
>   patch smaller?
> - I have not reviewed the algorithm for
>   G1EvacStats::adjust_desired_plab_sz in any more depth than veryfing
>   that the copy from plab.cpp seems correct (I also know that there are
>   more patches coming with changes to this logic).
> 
> Other than splitting out the trace event and Mikael's comment, I think
> the patch looks good.
> 

  thanks for your review. Please look at my response to Mikael and the
new RFR for the split out JFR events.

Thanks,
  Thomas




From kim.barrett at oracle.com  Thu Aug 13 13:52:38 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Thu, 13 Aug 2015 09:52:38 -0400
Subject: RFR (XS): Move implementation of process_grey_object to
	concurrentMark.inline.hpp
In-Reply-To: <55CC697B.5020202@sap.com>
References: <02D5D45C1F8DB848A7AE20E80EE61A5C39962EC5@DEWDFEMB20C.global.corp.sap>
	<CA+3eh12qqoE06vjee8HPsno4gLAo_4PnR71LjPWewr6LQ+cypA@mail.gmail.com>
	<35C5EA3B-DEE0-4DBB-972E-5DC0E58C04F2@oracle.com>
	<CA+3eh11R4O=wgspNr=BvkCTeGGKYL5dOV89Ey7a5Lkk4sO0SWA@mail.gmail.com>
	<16192E09-14D6-4263-832B-36C6CAE61513@oracle.com>
	<55CC697B.5020202@sap.com>
Message-ID: <6F435898-9E0E-4A5D-8FEB-8FF97D11A7CA@oracle.com>

On Aug 13, 2015, at 5:55 AM, Siebenborn, Axel <axel.siebenborn at sap.com> wrote:
> 
> On 12.08.2015, at 21:57, Kim Barrett wrote:
>>  (If Axel has time, I'm mildly curious whether just
>> removing the inline qualifier also works around the problem.)
> We were curious too and  indeed, removing the inline qualifier would work around the problem.

Interesting.  I may spend some time this afternoon re-reading bits of the standard,
but I think you should report this as a bug to your compiler vendor.

>> I have one optional change to suggest, which is to move the inline
>> definition of scan_object from the .hpp file to the .inline.hpp file.
>> I think putting it in the .hpp file was a mistake on my part.  Axel,
>> let me know whether you want to make that additional change.
> Yes, there is the new webrev:
> 
> http://cr.openjdk.java.net/~asiebenborn/8133121_0/webrev/

Looks good.

> Thanks for reviewing and sponsoring,
> Axel

No problem.  I?m guessing Volker likes this version too, but I?d like an email ACK before pushing.



From volker.simonis at gmail.com  Thu Aug 13 14:42:55 2015
From: volker.simonis at gmail.com (Volker Simonis)
Date: Thu, 13 Aug 2015 16:42:55 +0200
Subject: RFR (XS): Move implementation of process_grey_object to
	concurrentMark.inline.hpp
In-Reply-To: <6F435898-9E0E-4A5D-8FEB-8FF97D11A7CA@oracle.com>
References: <02D5D45C1F8DB848A7AE20E80EE61A5C39962EC5@DEWDFEMB20C.global.corp.sap>
	<CA+3eh12qqoE06vjee8HPsno4gLAo_4PnR71LjPWewr6LQ+cypA@mail.gmail.com>
	<35C5EA3B-DEE0-4DBB-972E-5DC0E58C04F2@oracle.com>
	<CA+3eh11R4O=wgspNr=BvkCTeGGKYL5dOV89Ey7a5Lkk4sO0SWA@mail.gmail.com>
	<16192E09-14D6-4263-832B-36C6CAE61513@oracle.com>
	<55CC697B.5020202@sap.com>
	<6F435898-9E0E-4A5D-8FEB-8FF97D11A7CA@oracle.com>
Message-ID: <CA+3eh12yns6D_3HzVBnzYHfkXfcwyP4NRgCh-yDp_maCdimUww@mail.gmail.com>

On Thu, Aug 13, 2015 at 3:52 PM, Kim Barrett <kim.barrett at oracle.com> wrote:
> On Aug 13, 2015, at 5:55 AM, Siebenborn, Axel <axel.siebenborn at sap.com> wrote:
>>
>> On 12.08.2015, at 21:57, Kim Barrett wrote:
>>>  (If Axel has time, I'm mildly curious whether just
>>> removing the inline qualifier also works around the problem.)
>> We were curious too and  indeed, removing the inline qualifier would work around the problem.
>
> Interesting.  I may spend some time this afternoon re-reading bits of the standard,
> but I think you should report this as a bug to your compiler vendor.
>
>>> I have one optional change to suggest, which is to move the inline
>>> definition of scan_object from the .hpp file to the .inline.hpp file.
>>> I think putting it in the .hpp file was a mistake on my part.  Axel,
>>> let me know whether you want to make that additional change.
>> Yes, there is the new webrev:
>>
>> http://cr.openjdk.java.net/~asiebenborn/8133121_0/webrev/
>
> Looks good.
>
>> Thanks for reviewing and sponsoring,
>> Axel
>
> No problem.  I?m guessing Volker likes this version too, but I?d like an email ACK before pushing.
>

Hi Kim,

yes, I like the new version:)
Would be great if you could push it.

Thanks,
Volker


From thomas.schatzl at oracle.com  Thu Aug 13 15:32:06 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 13 Aug 2015 17:32:06 +0200
Subject: RFR (S): 8133470: Uses of Atomic methods in plab.hpp should be
	moved to .inline.hpp file
In-Reply-To: <1439462979.2271.11.camel@oracle.com>
References: <1439462979.2271.11.camel@oracle.com>
Message-ID: <1439479926.2271.54.camel@oracle.com>

Hi all,

On Thu, 2015-08-13 at 12:49 +0200, Thomas Schatzl wrote:
> Hi all,
> 
>   in the review for 8073013 one of the review questions was why plab.hpp
> added atomic.inline.hpp, and that that is not good practice.
> 
> This has been a pre-existing bug, i.e. plab.hpp already used the Atomic
> functions that reside in .inline.hpp.
> 
> This CR fixes this issue by moving the problematic calls into
> plab.inline.hpp.
> 
> The only problem is that changing includes from plab.hpp to
> plab.inline.hpp in cpp files and adding plab.hpp in header files opens
> up dependency issues and other missing includes.
> 
> That's why this change is larger than it should.
> 
> Webrev:
> https://bugs.openjdk.java.net/browse/JDK-8133470
> CR:
> http://cr.openjdk.java.net/~tschatzl/8133470/webrev/
> 
> Testing:
> jprt

  Mikael and Erik reminded me that it is not a good idea to move
PLABAllocator::allocate_aligned() into the .cpp file because CMS uses
only allocate_aligned(). This may result in performance degradation for
CMS, which is unwanted.

This required the introduction of a parNewGeneration.inline.hpp for the
method calling it. And some other code shuffling.

Here are new webrevs:
http://cr.openjdk.java.net/~tschatzl/8133470/webrev.0_to_1
http://cr.openjdk.java.net/~tschatzl/8133470/webrev.1

Testing:
jprt

Thanks,
  Thomas




From thomas.schatzl at oracle.com  Thu Aug 13 15:32:28 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 13 Aug 2015 17:32:28 +0200
Subject: RFR (L): 8073013: Add detailed information about PLAB memory usage
In-Reply-To: <55CB4673.6070402@oracle.com>
References: <1438876151.2474.51.camel@oracle.com>
	<55C4F909.7050601@oracle.com> <1439211707.2330.11.camel@oracle.com>
	<55C9F395.8050008@oracle.com> <1439369047.2324.1.camel@oracle.com>
	<55CB4673.6070402@oracle.com>
Message-ID: <1439479948.2271.55.camel@oracle.com>

Hi,

On Wed, 2015-08-12 at 15:13 +0200, Mikael Gerdin wrote:
> Hi Thomas,
> 
> On 2015-08-12 10:44, Thomas Schatzl wrote:
> > Hi David,
> >
> >    thanks for the review:
> >
> > On Tue, 2015-08-11 at 15:07 +0200, David Lindholm wrote:
> >> Hi Thomas.
> >>
> >> This looks good, except for the changes in trace.xml. All fields should
> >> follow Java naming conventions, for example regionEndWaste, not
> >> region_end_waste. Also, 2 different event types are preferred according
> >> to the Servicability folks (instead of the "gen" field). Suggested names
> >> are GCG1EvacuationYoungStatistics and GCG1EvacuationOldStatistics.
> >>
> >> The other parts looks good.
> >
> > I think all fixed in the new webrevs at:
> >
> > http://cr.openjdk.java.net/~tschatzl/8073013/webrev.2 (full)
> > http://cr.openjdk.java.net/~tschatzl/8073013/webrev.1_to_2 (diff)
> 
> I only had a cursory look at the change, overall it seems good and I 
> know there's a large patch series to manage here.
> However, in plab.hpp you've added includes of allocation.inline.hpp and 
> atomic.inline.hpp. In general we should always avoid including inline 
> headers in normal headers, it's not obvious to me why your other changes 
> to plab.hpp would require the include change.

  I moved out these changes into an extra CR, this is a fix for an
existing issue acually, PLABStats referencing members of Atomic only in
atomic.inline.hpp.

Please have a look at the RFR for JDK-8133470.

Following Erik's suggestion I also moved out the JFR event, see the
review for JDK-8133530.

This change includes any adaptations caused by JDK-8133470.

New webrev:
http://cr.openjdk.java.net/~tschatzl/8073013/webrev.3 (full)
http://cr.openjdk.java.net/~tschatzl/8073013/webrev.2_to_3 (diff)

Testing:
jprt

Thanks,
  Thomas





From thomas.schatzl at oracle.com  Thu Aug 13 15:43:25 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 13 Aug 2015 17:43:25 +0200
Subject: RFR (S): 8133530: Add JFR event for evacuation statistics
Message-ID: <1439480605.2271.65.camel@oracle.com>

Hi all,

  can I have reviews for this split-out from 8073013 that adds JFR
support for evacuation statistics.

The code has been changed to (hopefully) correspond better to the way
code that sends JFR events should look like.

CR:
https://bugs.openjdk.java.net/browse/JDK-8133530
Webrev:
http://cr.openjdk.java.net/~tschatzl/8133530/webrev/

Testing:
jprt

Thanks,
  Thomas



From thomas.schatzl at oracle.com  Thu Aug 13 15:46:54 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 13 Aug 2015 17:46:54 +0200
Subject: [closed] Re: RFR (S): 8133530: Add JFR event for evacuation statistics
In-Reply-To: <1439480605.2271.65.camel@oracle.com>
References: <1439480605.2271.65.camel@oracle.com>
Message-ID: <1439480814.2271.68.camel@oracle.com>

Hi all,

  here is the closed part of this change. It contains the latest code
(without changes) from the original review request for JDK-8073013.

Unless you looked at that original review request, there is no reason to
look at the provided diff webrevs. I added the links for convenience.

jdk/src/closed:

http://wikifiles.se.oracle.com/dev/tschatzl/8133530/jdk-src-closed/webrev.1_to_2 (diff)
http://wikifiles.se.oracle.com/dev/tschatzl/8133530/jdk-src-closed/webrev.2 (full)

jdk/test/closed:

http://wikifiles.se.oracle.com/dev/tschatzl/8133530/jdk-test-closed/webrev.1_to_2 (diff)
http://wikifiles.se.oracle.com/dev/tschatzl/8133530/jdk-test-closed/webrev.2 (full)

Thanks,
  Thomas

On Thu, 2015-08-13 at 17:43 +0200, Thomas Schatzl wrote:
> Hi all,
> 
>   can I have reviews for this split-out from 8073013 that adds JFR
> support for evacuation statistics.
> 
> The code has been changed to (hopefully) correspond better to the way
> code that sends JFR events should look like.
> 
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8133530
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8133530/webrev/
> 
> Testing:
> jprt
> 
> Thanks,
>   Thomas
> 




From tom_at_work at gmx.at  Thu Aug  6 14:27:46 2015
From: tom_at_work at gmx.at (Thomas Schatzl)
Date: Thu, 06 Aug 2015 16:27:46 +0200
Subject: RFR (S): 8003237: G1: Reduce unnecessary (and failing)
	allocation attempts when handling an evacuation failure
Message-ID: <1438871266.2474.31.camel@gmx.at>

Hi Jon,

  thanks for the review...

On Wed, 2015-08-05 at 09:36 -0700, Jon Masamitsu wrote:
> Thomas,
> 
> I think that this method gets called if a new region is allocated
> via allocate_inline_or_new_plab()
> 
> "src/share/vm/gc/g1/g1CollectedHeap.cpp"
> 
> 6436 HeapRegion* G1CollectedHeap::new_gc_alloc_region(size_t word_size,
> 6437                                                  uint count,
> 6438                                                  InCSetState dest) {
> 6439   assert(FreeList_lock->owned_by_self(), "pre-condition");
> 6440
> 6441   if (count < g1_policy()->max_regions(dest)) {
> 
> Might the g1_policy()->max_regions() conflict with the test
> in has_more_free_regions() which considers all free regions
> (even uncommitted).   That is, has_more_free_regions() will
> allow allocation attempts that will not success because of
> max_regions().

No, because the worst case is that we go into the take-the-Mutex case
too often (which is bad).

However, thinking about this again, I found another option (which is
much clearer to see after recent refactoring of g1allocator classes): if
G1AllocRegion::attempt_allocation_locked() fails, we know that there is
really no more space in the heap left.

In this case, just set an internal flag, instead of relying on the
has_more_free_regions() from G1CollectedHeap.

> 
> http://cr.openjdk.java.net/~tschatzl/8003237/webrev/src/share/vm/gc/g1/g1ParScanThreadState.hpp.frames.html
> 
> >    74   // Indicates whether in the last generation (old) there is no more space
> >    75   // available for allocation.
> >    76   bool _no_more_space_in_old;
> 
> Would a name like _old_is_full suit you here in place of 
> _no_more_space_in_old?  I like positives instead
> negatives.

Done.

New webrevs:
http://cr.openjdk.java.net/~tschatzl/8003237/webrev.1 (full)
http://cr.openjdk.java.net/~tschatzl/8003237/webrev.0_to_1 (partial)

I think this one looks better, performs the same (tested) and does not
introduce the additional dependency on G1CollectedHeap.

This also automatically takes into consideration restrictions like in
G1CollectedHeap::new_gc_alloc_region() without the need to repeat them.

The addition of the new methods as virtual has been intentional as other
Allocators may have different views on when the heap for the given
context is full.

Thanks,
  Thomas






From stefan.karlsson at oracle.com  Thu Aug 13 17:50:13 2015
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Thu, 13 Aug 2015 19:50:13 +0200
Subject: RFR: 8087324: Use semaphores when starting and stopping GC task
	threads
In-Reply-To: <55957BCD.9020604@oracle.com>
References: <557AF21B.2090102@oracle.com>
	<5591122F.2000704@oracle.com>	<5595784A.9050903@oracle.com>
	<55957BCD.9020604@oracle.com>
Message-ID: <55CCD8D5.1070509@oracle.com>

Hi all,

Could I get a second review for the patch below (plus the suggested 
removal)?

Thanks,
StefanK

On 2015-07-02 19:58, Stefan Karlsson wrote:
> On 2015-07-02 19:43, Jon Masamitsu wrote:
>>
>>
>> On 6/29/2015 2:38 AM, Stefan Karlsson wrote:
>>> Hi all,
>>>
>>> "8087322: Implement a Semaphore utility class" has now been pushed, 
>>> so I've updated the patch to reflect the changes.
>>>
>>> http://cr.openjdk.java.net/~stefank/8087324/webrev.01.delta
>>> http://cr.openjdk.java.net/~stefank/8087324/webrev.01
>> http://cr.openjdk.java.net/~stefank/8087324/webrev.01/src/share/vm/gc/shared/workgroup.hpp.frames.html
>>
>> Are these used?
>>
>>   194   void print_worker_started_task(AbstractGangTask* task, uint worker_id);
>>   195   void print_worker_finished_task(AbstractGangTask* task, uint worker_id);
>
> No. I'll remove them.
>
>>
>>
>> Rest looks good.  Just one question (just for my education).
>>   
>> http://cr.openjdk.java.net/~stefank/8087324/webrev.01/src/share/vm/gc/shared/workgroup.cpp.frames.html
>>
>>   336 void GangWorker::loop() {
>>   337   while (true) {
>>   338     WorkData data = wait_for_task();
>>   339
>>   340     run_task(data);
>>   341
>>   342     signal_task_done();
>>   343   }
>>   344 }
>>
>> Does this allow the same thread to execute more than 1
>> task ("data" here)?  Meaning if 2 threads are  requested but
>> 1 thread is not scheduled to a cpu, will the other thread
>> do both chunks of work?
>
> Yes, that's correct.
>
> Thanks for reviewing!
> StefanK
>
>> Jon
>>
>>
>>>
>>> - The IMPLEMENTS_SEMAPHORE_CLASS define was removed, since all 
>>> platforms need to provide a Semaphore implementation.
>>>
>>> - Removed the need to pass down "max number of workers" to the 
>>> Semaphore constructor.
>>>
>>> - Updated semaphore.hpp include path
>>>
>>> Thanks,
>>> StefanK
>>>
>>>
>>> On 2015-06-12 16:52, Stefan Karlsson wrote:
>>>> Hi all,
>>>>
>>>> The current implementation to distribute tasks to GC worker threads 
>>>> often cause long latencies (multiple milliseconds) when the threads 
>>>> are started and stopped.
>>>>
>>>> The main reason is that the worker threads have to fight over the 
>>>> Monitor lock when they are woken up from the call to Monitor::wait. 
>>>> Another reason is that all worker threads call notify_all when they 
>>>> finish a task and there wakes all all sleeping worker threads, 
>>>> which will yet again force the worker threads to fight over the lock.
>>>>
>>>> I propose that we use semaphores instead, so that the worker 
>>>> threads don't have to fight over a lock when they are woken up.
>>>>
>>>>
>>>> The patches build upon the following patch which introduces a 
>>>> Semaphore utility class. This patch will sent out for review on the 
>>>> hotspot-dev, since it affects non-GC parts of the code:
>>>> http://cr.openjdk.java.net/~stefank/8087322/webrev.00/
>>>> https://bugs.openjdk.java.net/browse/JDK-8087322
>>>>
>>>>
>>>> The first patch that I would like to get reviewed is:
>>>> http://cr.openjdk.java.net/~stefank/8087323/webrev.00/
>>>> https://bugs.openjdk.java.net/browse/JDK-8087323 - Unify and split 
>>>> the work gang classes
>>>>
>>>> It prepares for JDK-8087324, by separating the generic WorkGang 
>>>> implementation from the more elaborate YieldingFlexibleWorkGang 
>>>> (CMS) implementation. By having this part as a separate patch, I 
>>>> hope it will be easier to review JDK-8087324. The patch changes the 
>>>> work gang inheritance from:
>>>>
>>>> AbstractWorkGang
>>>>  WorkGang
>>>>   FlexibleWorkGang
>>>>    YieldingFlexibleWorkGang
>>>>
>>>> to:
>>>>
>>>> AbstractWorkGang
>>>>  WorkGang
>>>>  YieldingFlexibleWorkGang
>>>>
>>>> Parts of the FlexibleWorkGang and WorkGang code that is going to be 
>>>> used by both concrete work gang classes, has been moved into 
>>>> AbstractWorkGang. I've duplicated some code in WorkGang and 
>>>> YieldingFlexibleWorkGang, but that code will be removed from 
>>>> WorkGang in the following patch.
>>>>
>>>>
>>>> The second patch I'd like to get reviewed is:
>>>> http://cr.openjdk.java.net/~stefank/8087324/webrev.00/
>>>> https://bugs.openjdk.java.net/browse/JDK-8087324 - Use semaphores 
>>>> when starting and stopping GC task threads
>>>>
>>>> It first simplifies the way we distribute the tasks to the GC 
>>>> worker threads. For example, the coordinator thread dispatches a 
>>>> task to a specific number of workers, and then waits for all work 
>>>> to be completed. There's no risk that multiple tasks will be 
>>>> scheduled simultaneously, so there's no need for the sequences 
>>>> number that is used in the current implementation.
>>>>
>>>> The patch contains two task dispatch / thread synchronization 
>>>> implementations:
>>>>
>>>> The first implementation uses Monitors, similar to what we did 
>>>> before the patch, but with a slightly lower overhead since the code 
>>>> calls notify_all less often. It still suffers from the "thundering 
>>>> heard" problem. When the coordinator thread signals that the worker 
>>>> threads should start, they all wake up from Monitor::wait and they 
>>>> all try to lock the Monitor.
>>>>
>>>> The second, and the more interesting, implementation uses 
>>>> semaphores. When the worker threads wake up from the semaphore 
>>>> wait, they don't have to serialize the execution by taking a lock. 
>>>> This greatly decreases the time it takes to start and stop the 
>>>> worker threads.
>>>>
>>>> The semaphore implementation is used on all platforms where the 
>>>> Semaphore class has been implemented in JDK-8087322. So, on some 
>>>> OS:es the code will revert to the Monitor-based solution until a 
>>>> Semaphore class has been implemented for that OS. So, porters might 
>>>> want to consider implementing the Sempahore class.
>>>>
>>>> There's also a diagnostic vm option 
>>>> (-XX:+/-UseSemaphoreGCThreadsSynchronization) to turn off the 
>>>> Semaphore-based implementation, which can be used to debug this new 
>>>> code. It's mainly targeted towards support and sustaining engineering.
>>>>
>>>>
>>>> The patches have been performance tested on Linux, Solaris, OSX, 
>>>> and Windows.
>>>>
>>>> The effects of the patch can be seen by running benchmarks with 
>>>> small young gen sizes, which triggers frequent and short GCs.
>>>>
>>>> For example, here are runs from the SPECjvm2008 xml.transform 
>>>> benchmark with:
>>>> -Xmx1g -Xms1g -Xmn64m -XX:+PrintGC -XX:+UseG1GC -jar 
>>>> SPECjvm2008.jar -ikv xml.transform -it 30 -wt 30
>>>>
>>>> I got the following GC times:
>>>>
>>>>             Average    Median    99.9 percentile   Max
>>>> Baseline: 8.76ms    8.44 ms   25.9 ms 34.7 ms
>>>> Monitor:   6.17 ms 5.88 ms   26.0 ms 49.1 ms
>>>> Semaphore: 3.43 ms 3.26 ms   13.4 ms           33.4 ms
>>>>
>>>> If I run an empty GC task 10 times per GC, by running the following 
>>>> code:
>>>> http://cr.openjdk.java.net/~stefank/8087324/timedTask/
>>>>
>>>> I get the following numbers to complete the empty GC tasks:
>>>>
>>>>             Average    Median    99.9 percentile   Max
>>>> Baseline: 1.43 ms    0.92 ms   3.43 ms           9.30ms
>>>> Monitor:    0.75ms 0.72 ms   1.74 ms           2.78ms
>>>> Semaphore: 0.07 ms 0.07 ms   0.17 ms           0.26 ms
>>>>
>>>>
>>>>
>>>> The code has been tested with JPRT and our nightly testing suites.
>>>>
>>>> I've created a unit test to run a small test with both the 
>>>> semaphore implementation and the monitor implementation:
>>>> http://cr.openjdk.java.net/~stefank/8087324/workgangTest/
>>>>
>>>> But since we currently don't have code to shutdown worker threads 
>>>> after they have been started, I don't want to push this test (or 
>>>> clean it up) until we have that in place. I created this bug for that:
>>>> https://bugs.openjdk.java.net/browse/JDK-8087340
>>>>
>>>> Thanks,
>>>> StefanK
>>>>
>>>>
>>>>
>>>
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150813/7f0ddb79/attachment.htm>

From tom.benson at oracle.com  Thu Aug 13 19:32:44 2015
From: tom.benson at oracle.com (Tom Benson)
Date: Thu, 13 Aug 2015 15:32:44 -0400
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <1439377259.2324.27.camel@oracle.com>
References: <55C103A4.1060505@oracle.com>		<A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>		<1438781844.2378.60.camel@oracle.com>	<55C21DC3.5090305@oracle.com>		<1438785532.2378.73.camel@oracle.com>	<55C248FF.6010308@oracle.com>		<F73C4B9A-4DBA-47A8-8715-0988FF6D393D@oracle.com>		<55C26249.1090902@oracle.com>		<C149705F-696B-46CD-8FB8-CAD06C511244@oracle.com>		<55C2751B.8070400@oracle.com>	<1438847329.2009.9.camel@oracle.com>		<55C36B31.7000908@oracle.com>	<1438871723.2474.37.camel@oracle.com>		<55C373F6.9060609@oracle.com>		<AD45178E-A442-4A0D-80D6-50E91608E1FE@oracle.com>		<55C3D8BB.9000603@oracle.com>		<48C0E571-0D7B-4D30-85DF-E31D99092A7C@oracle.com>		<30B0248C-3BEA-4FAE-861F-DAAE19F56B45@oracle.com>		<55C4B806.7050504@oracle.com>
	<55C4C720.5030903@oracle.com>	 <55CA3430.3070300@oracle.com>
	<1439377259.2324.27.camel@oracle.com>
Message-ID: <55CCF0DC.2000800@oracle.com>

Hi Thomas,

On 8/12/2015 7:00 AM, Thomas Schatzl wrote:
> Hi,
>
> On Tue, 2015-08-11 at 13:43 -0400, Tom Benson wrote:
>> Hi,
>> On 8/7/2015 10:56 AM, Tom Benson wrote:
>> After some discussion, I've changed the definition and name of
>> free_archive_regions.  Now called dealloc_archive_regions, it uncommits
>> the specified regions, unmapping the memory, rather than adding them to
>> the free list.  This means the CDS code will no longer do the unmapping
>> on verification failures.
>>
>> Updated full and incremental webrevs of the GC code are at:
>> http://cr.openjdk.java.net/~tbenson/8131734/webrev.02/
>> http://cr.openjdk.java.net/~tbenson/8131734/webrev.02.vs.01/
>>
>> Tested with JPRT and running benchmarks with the dealloc_ performed
>> explicitly.  Jiangli also tested the original failing cases, and will be
>> posting an updated webrev.
> - is it possible that shrink_by() uses shrink_at()? This would avoid two
> paths that uncommit regions like expand_by()/expand_at()?

OK, I made the change.  I didn't do it originally because the asserts I 
wanted to add for the call from g1CollectedHeap seemed superfluous for 
the other call, and shrink_at was so small.    Now shrink_at takes a 
region count as well.

Updated full and incremental webrevs are at:
   http://cr.openjdk.java.net/~tbenson/8131734/webrev.03/
   http://cr.openjdk.java.net/~tbenson/8131734/webrev.03.vs.02/


>
> - I think the change should call at least HeapRegion::hr_clear() on the
> region to remove or reset any auxiliary data structures, if not
> G1CollectedHeap::free_region() (without adding the region to the free
> list).
> Since the HeapRegion* is not deallocated by the uncommit, this may cause
> strange behavior later when the region is reused.

I don't think calling hr_clear should be necessary...  If it is, we 
should be doing it in shrink_by as well, and I don't think we are. I 
don't see how a HeapRegion can be 'reused' without having gone through 
the constructor when expand_ asks (indirectly) for 'new HeapRegion', and 
that does an hr_clear() as well as the rest of init.     Or am I missing 
something there?

Thanks,
Tom


>
> Other than that it looks okay.
>
> Thanks,
>    Thomas
>
>



From david.lindholm at oracle.com  Fri Aug 14 07:30:53 2015
From: david.lindholm at oracle.com (David Lindholm)
Date: Fri, 14 Aug 2015 09:30:53 +0200
Subject: RFR: 8133553: Running with -XX:+UseParallelGC -XX:OldSize=30k crashes
	jvm
Message-ID: <55CD992D.6010200@oracle.com>

Hi,

Please review this patch that prevents the jvm to crash (or assert) when 
run with an extremely small value for the -XX:OldSize flag and a 
collector that uses GenCollectorPolicy.

Bug: https://bugs.openjdk.java.net/browse/JDK-8133553
Webrev: http://cr.openjdk.java.net/~david/JDK-8133553/webrev.00/

Testing: Passed JPRT.


Thanks,
David


From thomas.schatzl at oracle.com  Fri Aug 14 08:29:32 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Fri, 14 Aug 2015 10:29:32 +0200
Subject: RFR (S): 8067339: PLAB reallocation might result in failure to
	allocate object in that recently allocated PLAB
Message-ID: <1439540972.2264.12.camel@oracle.com>

Hi all,

  can I have reviews for the following small fix: if G1 reallocates a
PLAB for an object of size X, it allocates a new PLAB of size Y. 

PLABs are required to contain enough space for the object plus some
alignment reserve; so if X >= Y - alignment-reserve, G1 failed to
allocate this object it just allocated the PLAB for into that PLAB,
resulting in an evacuation failure although there is actually enough
space.

As far as I know this has been a rare occurrence until now because
current PLAB sizing tended to maximize PLAB sizes, and it is rare to
have such large objects, but with future changes that actually try to
size the PLABs so that the expected waste is kept, I noticed a few cases
of that issue occurring. So this needs to be fixed.

The fix is to consider the required alignment reserve in the PLAB
allocation request.

This is a day one G1 bug.

CR:
https://bugs.openjdk.java.net/browse/JDK-8067339
Webrev:
http://cr.openjdk.java.net/~tschatzl/8067339/webrev/
Testing:
JPRT, vm.gc testlist, lots of testing.

Thanks,
  Thomas




From mikael.gerdin at oracle.com  Fri Aug 14 08:42:59 2015
From: mikael.gerdin at oracle.com (Mikael Gerdin)
Date: Fri, 14 Aug 2015 10:42:59 +0200
Subject: RFR (S): 8067339: PLAB reallocation might result in failure to
	allocate object in that recently allocated PLAB
In-Reply-To: <1439540972.2264.12.camel@oracle.com>
References: <1439540972.2264.12.camel@oracle.com>
Message-ID: <55CDAA13.4080307@oracle.com>

Hi Thomas,

On 2015-08-14 10:29, Thomas Schatzl wrote:
> Hi all,
>
>    can I have reviews for the following small fix: if G1 reallocates a
> PLAB for an object of size X, it allocates a new PLAB of size Y.
>
> PLABs are required to contain enough space for the object plus some
> alignment reserve; so if X >= Y - alignment-reserve, G1 failed to
> allocate this object it just allocated the PLAB for into that PLAB,
> resulting in an evacuation failure although there is actually enough
> space.
>
> As far as I know this has been a rare occurrence until now because
> current PLAB sizing tended to maximize PLAB sizes, and it is rare to
> have such large objects, but with future changes that actually try to
> size the PLABs so that the expected waste is kept, I noticed a few cases
> of that issue occurring. So this needs to be fixed.
>
> The fix is to consider the required alignment reserve in the PLAB
> allocation request.
>
> This is a day one G1 bug.
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8067339
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8067339/webrev/

Looks good to me.

> Testing:
> JPRT, vm.gc testlist, lots of testing.

Lots of testing sounds excellent :)

/Mikael

>
> Thanks,
>    Thomas
>
>



From thomas.schatzl at oracle.com  Fri Aug 14 09:10:51 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Fri, 14 Aug 2015 11:10:51 +0200
Subject: RFR (S): 8067339: PLAB reallocation might result in failure to
	allocate object in that recently allocated PLAB
In-Reply-To: <55CDAA13.4080307@oracle.com>
References: <1439540972.2264.12.camel@oracle.com> <55CDAA13.4080307@oracle.com>
Message-ID: <1439543451.2264.16.camel@oracle.com>

Hi Mikael,

On Fri, 2015-08-14 at 10:42 +0200, Mikael Gerdin wrote:
> Hi Thomas,
> 
> On 2015-08-14 10:29, Thomas Schatzl wrote:
> > Hi all,
> >
> >    can I have reviews for the following small fix: if G1 reallocates a
> > PLAB for an object of size X, it allocates a new PLAB of size Y.
> >
> > PLABs are required to contain enough space for the object plus some
> > alignment reserve; so if X >= Y - alignment-reserve, G1 failed to
> > allocate this object it just allocated the PLAB for into that PLAB,
> > resulting in an evacuation failure although there is actually enough
> > space.
> >
> > As far as I know this has been a rare occurrence until now because
> > current PLAB sizing tended to maximize PLAB sizes, and it is rare to
> > have such large objects, but with future changes that actually try to
> > size the PLABs so that the expected waste is kept, I noticed a few cases
> > of that issue occurring. So this needs to be fixed.
> >
> > The fix is to consider the required alignment reserve in the PLAB
> > allocation request.
> >
> > This is a day one G1 bug.
> >
> > CR:
> > https://bugs.openjdk.java.net/browse/JDK-8067339
> > Webrev:
> > http://cr.openjdk.java.net/~tschatzl/8067339/webrev/
> 
> Looks good to me.

Thanks for the review.

Thanks,
  Thomas




From erik.helin at oracle.com  Fri Aug 14 09:34:12 2015
From: erik.helin at oracle.com (Erik Helin)
Date: Fri, 14 Aug 2015 11:34:12 +0200
Subject: RFR (S): 8133470: Uses of Atomic methods in plab.hpp should be
	moved to .inline.hpp file
In-Reply-To: <1439479926.2271.54.camel@oracle.com>
References: <1439462979.2271.11.camel@oracle.com>
	<1439479926.2271.54.camel@oracle.com>
Message-ID: <20150814093412.GA31725@ehelin.jrpg.bea.com>

On 2015-08-13, Thomas Schatzl wrote:
> Hi all,
> 
> On Thu, 2015-08-13 at 12:49 +0200, Thomas Schatzl wrote:
> > Hi all,
> > 
> >   in the review for 8073013 one of the review questions was why plab.hpp
> > added atomic.inline.hpp, and that that is not good practice.
> > 
> > This has been a pre-existing bug, i.e. plab.hpp already used the Atomic
> > functions that reside in .inline.hpp.
> > 
> > This CR fixes this issue by moving the problematic calls into
> > plab.inline.hpp.
> > 
> > The only problem is that changing includes from plab.hpp to
> > plab.inline.hpp in cpp files and adding plab.hpp in header files opens
> > up dependency issues and other missing includes.
> > 
> > That's why this change is larger than it should.
> > 
> > Webrev:
> > https://bugs.openjdk.java.net/browse/JDK-8133470
> > CR:
> > http://cr.openjdk.java.net/~tschatzl/8133470/webrev/
> > 
> > Testing:
> > jprt
> 
>   Mikael and Erik reminded me that it is not a good idea to move
> PLABAllocator::allocate_aligned() into the .cpp file because CMS uses
> only allocate_aligned(). This may result in performance degradation for
> CMS, which is unwanted.
> 
> This required the introduction of a parNewGeneration.inline.hpp for the
> method calling it. And some other code shuffling.
> 
> Here are new webrevs:
> http://cr.openjdk.java.net/~tschatzl/8133470/webrev.0_to_1
> http://cr.openjdk.java.net/~tschatzl/8133470/webrev.1

Thanks for fixing this! Looks good, Reviewed.

Thanks,
Erik

> Testing:
> jprt
> 
> Thanks,
>   Thomas
> 
> 


From mikael.gerdin at oracle.com  Fri Aug 14 09:53:27 2015
From: mikael.gerdin at oracle.com (Mikael Gerdin)
Date: Fri, 14 Aug 2015 11:53:27 +0200
Subject: RFR (S): 8133470: Uses of Atomic methods in plab.hpp should be
	moved to .inline.hpp file
In-Reply-To: <20150814093412.GA31725@ehelin.jrpg.bea.com>
References: <1439462979.2271.11.camel@oracle.com>	<1439479926.2271.54.camel@oracle.com>
	<20150814093412.GA31725@ehelin.jrpg.bea.com>
Message-ID: <55CDBA97.3050909@oracle.com>

On 2015-08-14 11:34, Erik Helin wrote:
> On 2015-08-13, Thomas Schatzl wrote:
>> Hi all,
>>
>> On Thu, 2015-08-13 at 12:49 +0200, Thomas Schatzl wrote:
>>> Hi all,
>>>
>>>    in the review for 8073013 one of the review questions was why plab.hpp
>>> added atomic.inline.hpp, and that that is not good practice.
>>>
>>> This has been a pre-existing bug, i.e. plab.hpp already used the Atomic
>>> functions that reside in .inline.hpp.
>>>
>>> This CR fixes this issue by moving the problematic calls into
>>> plab.inline.hpp.
>>>
>>> The only problem is that changing includes from plab.hpp to
>>> plab.inline.hpp in cpp files and adding plab.hpp in header files opens
>>> up dependency issues and other missing includes.
>>>
>>> That's why this change is larger than it should.
>>>
>>> Webrev:
>>> https://bugs.openjdk.java.net/browse/JDK-8133470
>>> CR:
>>> http://cr.openjdk.java.net/~tschatzl/8133470/webrev/
>>>
>>> Testing:
>>> jprt
>>
>>    Mikael and Erik reminded me that it is not a good idea to move
>> PLABAllocator::allocate_aligned() into the .cpp file because CMS uses
>> only allocate_aligned(). This may result in performance degradation for
>> CMS, which is unwanted.
>>
>> This required the introduction of a parNewGeneration.inline.hpp for the
>> method calling it. And some other code shuffling.
>>
>> Here are new webrevs:
>> http://cr.openjdk.java.net/~tschatzl/8133470/webrev.0_to_1
>> http://cr.openjdk.java.net/~tschatzl/8133470/webrev.1
>
> Thanks for fixing this! Looks good, Reviewed.

Looks good to me too.
/Mikael

>
> Thanks,
> Erik
>
>> Testing:
>> jprt
>>
>> Thanks,
>>    Thomas
>>
>>



From mikael.gerdin at oracle.com  Fri Aug 14 11:30:20 2015
From: mikael.gerdin at oracle.com (Mikael Gerdin)
Date: Fri, 14 Aug 2015 13:30:20 +0200
Subject: RFR (S): 8133530: Add JFR event for evacuation statistics
In-Reply-To: <1439480605.2271.65.camel@oracle.com>
References: <1439480605.2271.65.camel@oracle.com>
Message-ID: <55CDD14C.2050600@oracle.com>

Hi Thomas,

On 2015-08-13 17:43, Thomas Schatzl wrote:
> Hi all,
>
>    can I have reviews for this split-out from 8073013 that adds JFR
> support for evacuation statistics.
>
> The code has been changed to (hopefully) correspond better to the way
> code that sends JFR events should look like.
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8133530
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8133530/webrev/

in gcTrace.hpp
39 #include "gc/g1/g1YCTypes.hpp"
40 #include "gcHeapSummary.hpp"
41 #endif
42

you should use the full path to gcHeapSummary.hpp

in gcTraceSend.cpp
240   if (surv_evt.should_commit()) {
241     surv_evt.set_gcId(GCId::peek().id());
242     surv_evt.set_allocated(summary.allocated() * HeapWordSize);

The other events use
232     e.set_gcId(_shared_gc_info.gc_id().id());

When you assert_set_gc_id you actually verify that the shared gc info id 
is set.

/Mikael

>
> Testing:
> jprt
>
> Thanks,
>    Thomas
>



From stefan.johansson at oracle.com  Fri Aug 14 11:27:03 2015
From: stefan.johansson at oracle.com (Stefan Johansson)
Date: Fri, 14 Aug 2015 13:27:03 +0200
Subject: RFR: 8087324: Use semaphores when starting and stopping GC task
	threads
In-Reply-To: <559552A9.8030008@oracle.com>
References: <557AF21B.2090102@oracle.com>
	<559415DD.7030604@oracle.com>	<55952520.6030609@oracle.com>
	<559552A9.8030008@oracle.com>
Message-ID: <55CDD087.3050301@oracle.com>

Hi Stefan,

On 2015-07-02 17:03, Jon Masamitsu wrote:
>
>
> On 07/02/2015 04:48 AM, Stefan Karlsson wrote:
>>
>>
>> On 2015-07-01 18:31, Jon Masamitsu wrote:
>>>
>>>
>>> On 6/12/2015 7:52 AM, Stefan Karlsson wrote:
>>>> Hi all,
>>>>
>>>> The current implementation to distribute tasks to GC worker threads 
>>>> often cause long latencies (multiple milliseconds) when the threads 
>>>> are started and stopped.
>>>>
>>>> The main reason is that the worker threads have to fight over the 
>>>> Monitor lock when they are woken up from the call to Monitor::wait. 
>>>> Another reason is that all worker threads call notify_all when they 
>>>> finish a task and there wakes all all sleeping worker threads, 
>>>> which will yet again force the worker threads to fight over the lock.
>>>>
>>>> I propose that we use semaphores instead, so that the worker 
>>>> threads don't have to fight over a lock when they are woken up.
>>>>
>>>>
>>>> The patches build upon the following patch which introduces a 
>>>> Semaphore utility class. This patch will sent out for review on the 
>>>> hotspot-dev, since it affects non-GC parts of the code:
>>>> http://cr.openjdk.java.net/~stefank/8087322/webrev.00/
>>>> https://bugs.openjdk.java.net/browse/JDK-8087322
>>>>
>>>>
>>>> The first patch that I would like to get reviewed is:
>>>> http://cr.openjdk.java.net/~stefank/8087323/webrev.00/
>>>> https://bugs.openjdk.java.net/browse/JDK-8087323 - Unify and split 
>>>> the work gang classes
>>>>
>>>> It prepares for JDK-8087324, by separating the generic WorkGang 
>>>> implementation from the more elaborate YieldingFlexibleWorkGang 
>>>> (CMS) implementation. By having this part as a separate patch, I 
>>>> hope it will be easier to review JDK-8087324. The patch changes the 
>>>> work gang inheritance from:
>>>>
>>>> AbstractWorkGang
>>>>  WorkGang
>>>>   FlexibleWorkGang
>>>>    YieldingFlexibleWorkGang
>>>>
>>>> to:
>>>>
>>>> AbstractWorkGang
>>>>  WorkGang
>>>>  YieldingFlexibleWorkGang
>>>>
>>>> Parts of the FlexibleWorkGang and WorkGang code that is going to be 
>>>> used by both concrete work gang classes, has been moved into 
>>>> AbstractWorkGang. I've duplicated some code in WorkGang and 
>>>> YieldingFlexibleWorkGang, but that code will be removed from 
>>>> WorkGang in the following patch.
The changes for 8087323 looks good. I agree with Jons comments below 
about removing the function is_Yielding... and the cast in 
YieldingFlexibleGangTask.

Thanks,
Stefan

>>>
>>> http://cr.openjdk.java.net/~stefank/8087323/webrev.00/src/share/vm/gc/cms/yieldingWorkgroup.hpp.frames.html
>>>
>>> There seems to be only one definition of 
>>> is_YieldingFlexibleGang_task() now.  Is that right?  Is that useful?
>>>
>>>   131   NOT_PRODUCT(virtual bool is_YieldingFlexibleGang_task() const {
>>>   132     return true;
>>>   133   })
>>
>> I agree. I don't think we need it anymore.
>>
> Thanks.
>
>>
>>>
>>> Not a change  in your patch but
>>>
>>>    86   AbstractWorkGang(const char* name, uint workers, bool are_GC_task_threads, bool are_ConcurrentGC_threads) :
>>>    87       _name(name),
>>>    88       _total_workers(workers),
>>>    89       _active_workers(UseDynamicNumberOfGCThreads ? 1U : workers),
>>>    90       _are_GC_task_threads(are_GC_task_threads),
>>>    91       _are_Concurren
>>>
>>> _active_workers is always calculated as >= 2 unless _total_workers 
>>> is only 1.
>>> So line 89 should be
>>>
>>> _active_workers(UseDynamicNumberOfGCThreads ? MIN2(2, workers) : 
>>> workers)
>>>
>>> Should I file a CR for that?  Or do you want to include it.
>>
>> I'm not sure that what is proposed above is correct. I see that 
>> AdaptiveSizePolicy::calc_active_workers returns 2 as a minimum, but 
>> both ConcurrentMark::calc_parallel_marking_threads and 
>> AdaptiveSizePolicy::calc_active_conc_workers can return 1.
>>
>> I also don't think it should be AbstractWorkGang's responsibility to 
>> have the knowledge about the minimum number of worker threads that 
>> are used when UseDynamicNumberOfGCThreads are turned on. Maybe we 
>> should set it to 0, and let the calc_*_active_workers setup the 
>> default value.
>
> I think that at one time I had tried to set the default to 0 and 
> something failed.  I can see the point though.
>
>>
>> I would prefer to handle any changes, to this part of the code, as 
>> separate RFEs.
>
> Fair enough.  If I think it's worth doing, I'll file and RFE. Probably 
> something more than just setting
> it to 2 (maybe picking a default value with the help of 
> AdaptiveSizePolicy).
>
>
>>
>>> Have you considered (maybe for a later patch) changing 
>>> YieldingFlexibleWorkGang to
>>> simply YieldingWorkGang?  The "Flexible" attribute of 
>>> YieldingFlexibleWorkGang having
>>> been moved into AbstractWorkGang.
>>
>> I thought about it, but didn't think it was important enough to 
>> warrant that change in this patch. I wouldn't mind if a RFE was 
>> created to change the name.
>
> I'll file the RFE  if you agree with the point that "Flexible" 
> describes what AbstractWorkGang does now.
>
>>
>>>
>>> http://cr.openjdk.java.net/~stefank/8087323/webrev.00/src/share/vm/gc/cms/yieldingWorkgroup.cpp.frames.html
>>>
>>> Is the cast at 53 necessary? I see it in the original code too.
>>>
>>>    50 AbstractGangWorker* YieldingFlexibleWorkGang::allocate_worker(uint which) {
>>>    51   YieldingFlexibleGangWorker* new_member =
>>>    52       new YieldingFlexibleGangWorker(this, which);
>>>    53   return (YieldingFlexibleGangWorker*) new_member;
>>>    54 }
>>
>> Yes, this is unnecessary.
>
> Thanks.
>
> Jon
>
>>
>>>
>>> The rest looks good.
>>
>> Thanks.
>>
>>>
>>> I'll do the second patch next.
>>
>> Great.
>>
>> StefanK
>>
>>>
>>> Jon
>>>
>>>>
>>>>
>>>> The second patch I'd like to get reviewed is:
>>>> http://cr.openjdk.java.net/~stefank/8087324/webrev.00/
>>>> https://bugs.openjdk.java.net/browse/JDK-8087324 - Use semaphores 
>>>> when starting and stopping GC task threads
>>>>
>>>> It first simplifies the way we distribute the tasks to the GC 
>>>> worker threads. For example, the coordinator thread dispatches a 
>>>> task to a specific number of workers, and then waits for all work 
>>>> to be completed. There's no risk that multiple tasks will be 
>>>> scheduled simultaneously, so there's no need for the sequences 
>>>> number that is used in the current implementation.
>>>>
>>>> The patch contains two task dispatch / thread synchronization 
>>>> implementations:
>>>>
>>>> The first implementation uses Monitors, similar to what we did 
>>>> before the patch, but with a slightly lower overhead since the code 
>>>> calls notify_all less often. It still suffers from the "thundering 
>>>> heard" problem. When the coordinator thread signals that the worker 
>>>> threads should start, they all wake up from Monitor::wait and they 
>>>> all try to lock the Monitor.
>>>>
>>>> The second, and the more interesting, implementation uses 
>>>> semaphores. When the worker threads wake up from the semaphore 
>>>> wait, they don't have to serialize the execution by taking a lock. 
>>>> This greatly decreases the time it takes to start and stop the 
>>>> worker threads.
>>>>
>>>> The semaphore implementation is used on all platforms where the 
>>>> Semaphore class has been implemented in JDK-8087322. So, on some 
>>>> OS:es the code will revert to the Monitor-based solution until a 
>>>> Semaphore class has been implemented for that OS. So, porters might 
>>>> want to consider implementing the Sempahore class.
>>>>
>>>> There's also a diagnostic vm option 
>>>> (-XX:+/-UseSemaphoreGCThreadsSynchronization) to turn off the 
>>>> Semaphore-based implementation, which can be used to debug this new 
>>>> code. It's mainly targeted towards support and sustaining engineering.
>>>>
>>>>
>>>> The patches have been performance tested on Linux, Solaris, OSX, 
>>>> and Windows.
>>>>
>>>> The effects of the patch can be seen by running benchmarks with 
>>>> small young gen sizes, which triggers frequent and short GCs.
>>>>
>>>> For example, here are runs from the SPECjvm2008 xml.transform 
>>>> benchmark with:
>>>> -Xmx1g -Xms1g -Xmn64m -XX:+PrintGC -XX:+UseG1GC -jar 
>>>> SPECjvm2008.jar -ikv xml.transform -it 30 -wt 30
>>>>
>>>> I got the following GC times:
>>>>
>>>>             Average    Median    99.9 percentile   Max
>>>> Baseline: 8.76ms    8.44 ms   25.9 ms 34.7 ms
>>>> Monitor:   6.17 ms 5.88 ms   26.0 ms 49.1 ms
>>>> Semaphore: 3.43 ms 3.26 ms   13.4 ms           33.4 ms
>>>>
>>>> If I run an empty GC task 10 times per GC, by running the following 
>>>> code:
>>>> http://cr.openjdk.java.net/~stefank/8087324/timedTask/
>>>>
>>>> I get the following numbers to complete the empty GC tasks:
>>>>
>>>>             Average    Median    99.9 percentile   Max
>>>> Baseline: 1.43 ms    0.92 ms   3.43 ms           9.30ms
>>>> Monitor:    0.75ms 0.72 ms   1.74 ms           2.78ms
>>>> Semaphore: 0.07 ms 0.07 ms   0.17 ms           0.26 ms
>>>>
>>>>
>>>>
>>>> The code has been tested with JPRT and our nightly testing suites.
>>>>
>>>> I've created a unit test to run a small test with both the 
>>>> semaphore implementation and the monitor implementation:
>>>> http://cr.openjdk.java.net/~stefank/8087324/workgangTest/
>>>>
>>>> But since we currently don't have code to shutdown worker threads 
>>>> after they have been started, I don't want to push this test (or 
>>>> clean it up) until we have that in place. I created this bug for that:
>>>> https://bugs.openjdk.java.net/browse/JDK-8087340
>>>>
>>>> Thanks,
>>>> StefanK
>>>>
>>>>
>>>>
>>>
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150814/e0cfff33/attachment.htm>

From mikael.gerdin at oracle.com  Fri Aug 14 11:53:04 2015
From: mikael.gerdin at oracle.com (Mikael Gerdin)
Date: Fri, 14 Aug 2015 13:53:04 +0200
Subject: RFR (L): 8073013: Add detailed information about PLAB memory usage
In-Reply-To: <1439479948.2271.55.camel@oracle.com>
References: <1438876151.2474.51.camel@oracle.com>		<55C4F909.7050601@oracle.com>
	<1439211707.2330.11.camel@oracle.com>		<55C9F395.8050008@oracle.com>
	<1439369047.2324.1.camel@oracle.com>	 <55CB4673.6070402@oracle.com>
	<1439479948.2271.55.camel@oracle.com>
Message-ID: <55CDD6A0.60405@oracle.com>

Hi Thomas,

On 2015-08-13 17:32, Thomas Schatzl wrote:
> Hi,
>
> On Wed, 2015-08-12 at 15:13 +0200, Mikael Gerdin wrote:
>> Hi Thomas,
>>
>> On 2015-08-12 10:44, Thomas Schatzl wrote:
>>> Hi David,
>>>
>>>     thanks for the review:
>>>
>>> On Tue, 2015-08-11 at 15:07 +0200, David Lindholm wrote:
>>>> Hi Thomas.
>>>>
>>>> This looks good, except for the changes in trace.xml. All fields should
>>>> follow Java naming conventions, for example regionEndWaste, not
>>>> region_end_waste. Also, 2 different event types are preferred according
>>>> to the Servicability folks (instead of the "gen" field). Suggested names
>>>> are GCG1EvacuationYoungStatistics and GCG1EvacuationOldStatistics.
>>>>
>>>> The other parts looks good.
>>>
>>> I think all fixed in the new webrevs at:
>>>
>>> http://cr.openjdk.java.net/~tschatzl/8073013/webrev.2 (full)
>>> http://cr.openjdk.java.net/~tschatzl/8073013/webrev.1_to_2 (diff)
>>
>> I only had a cursory look at the change, overall it seems good and I
>> know there's a large patch series to manage here.
>> However, in plab.hpp you've added includes of allocation.inline.hpp and
>> atomic.inline.hpp. In general we should always avoid including inline
>> headers in normal headers, it's not obvious to me why your other changes
>> to plab.hpp would require the include change.
>
>    I moved out these changes into an extra CR, this is a fix for an
> existing issue acually, PLABStats referencing members of Atomic only in
> atomic.inline.hpp.
>
> Please have a look at the RFR for JDK-8133470.
>
> Following Erik's suggestion I also moved out the JFR event, see the
> review for JDK-8133530.
>
> This change includes any adaptations caused by JDK-8133470.
>
> New webrev:
> http://cr.openjdk.java.net/~tschatzl/8073013/webrev.3 (full)
> http://cr.openjdk.java.net/~tschatzl/8073013/webrev.2_to_3 (diff)

I think this is ready to go now, Reviewed.

/Mikael

>
> Testing:
> jprt
>
> Thanks,
>    Thomas
>
>
>



From stefan.johansson at oracle.com  Fri Aug 14 12:38:27 2015
From: stefan.johansson at oracle.com (Stefan Johansson)
Date: Fri, 14 Aug 2015 14:38:27 +0200
Subject: RFR: 8087324: Use semaphores when starting and stopping GC task
	threads
In-Reply-To: <55CCD8D5.1070509@oracle.com>
References: <557AF21B.2090102@oracle.com>	<5591122F.2000704@oracle.com>	<5595784A.9050903@oracle.com>	<55957BCD.9020604@oracle.com>
	<55CCD8D5.1070509@oracle.com>
Message-ID: <55CDE143.20808@oracle.com>

Hi Stefan,

Nice change, looks really good, but I have a few small comments.

In workgroup.hpp:
  177   GangTaskDispatcher* dispatcher()const {
Add a space before const.
---

In workgroup.cpp:
  122       // Limit the semaphore value to the number of workers.
  123       _start_semaphore(new Semaphore()),
  124       _end_semaphore(new Semaphore())
Seems like the limit has been lost. I would prefer to have it added, but 
if that's somehow problematic just remove the comment.

  140     // Wait for the last worker to signal the coordinator.
  141     _end_semaphore->wait();
  142
  143     // No workers are allowed to read the state variables after 
the coordinator has been signaled.
  144     _task         = NULL;
  145     _started      = 0;
  146     _not_finished = 0;
Do we need to set _not_finished to 0 or could we instead assert that it 
already is 0 since that's when _end_semaphore should have been signaled.
---

I don't need to see a new webrev for the above changes. Reviewed.

Thanks,
Stefan

On 2015-08-13 19:50, Stefan Karlsson wrote:
> Hi all,
>
> Could I get a second review for the patch below (plus the suggested 
> removal)?
>
> Thanks,
> StefanK
>
> On 2015-07-02 19:58, Stefan Karlsson wrote:
>> On 2015-07-02 19:43, Jon Masamitsu wrote:
>>>
>>>
>>> On 6/29/2015 2:38 AM, Stefan Karlsson wrote:
>>>> Hi all,
>>>>
>>>> "8087322: Implement a Semaphore utility class" has now been pushed, 
>>>> so I've updated the patch to reflect the changes.
>>>>
>>>> http://cr.openjdk.java.net/~stefank/8087324/webrev.01.delta
>>>> http://cr.openjdk.java.net/~stefank/8087324/webrev.01
>>> http://cr.openjdk.java.net/~stefank/8087324/webrev.01/src/share/vm/gc/shared/workgroup.hpp.frames.html
>>>
>>> Are these used?
>>>
>>>   194   void print_worker_started_task(AbstractGangTask* task, uint worker_id);
>>>   195   void print_worker_finished_task(AbstractGangTask* task, uint worker_id);
>>
>> No. I'll remove them.
>>
>>>
>>>
>>> Rest looks good.  Just one question (just for my education).
>>>   
>>> http://cr.openjdk.java.net/~stefank/8087324/webrev.01/src/share/vm/gc/shared/workgroup.cpp.frames.html
>>>
>>>   336 void GangWorker::loop() {
>>>   337   while (true) {
>>>   338     WorkData data = wait_for_task();
>>>   339
>>>   340     run_task(data);
>>>   341
>>>   342     signal_task_done();
>>>   343   }
>>>   344 }
>>>
>>> Does this allow the same thread to execute more than 1
>>> task ("data" here)?  Meaning if 2 threads are  requested but
>>> 1 thread is not scheduled to a cpu, will the other thread
>>> do both chunks of work?
>>
>> Yes, that's correct.
>>
>> Thanks for reviewing!
>> StefanK
>>
>>> Jon
>>>
>>>
>>>>
>>>> - The IMPLEMENTS_SEMAPHORE_CLASS define was removed, since all 
>>>> platforms need to provide a Semaphore implementation.
>>>>
>>>> - Removed the need to pass down "max number of workers" to the 
>>>> Semaphore constructor.
>>>>
>>>> - Updated semaphore.hpp include path
>>>>
>>>> Thanks,
>>>> StefanK
>>>>
>>>>
>>>> On 2015-06-12 16:52, Stefan Karlsson wrote:
>>>>> Hi all,
>>>>>
>>>>> The current implementation to distribute tasks to GC worker 
>>>>> threads often cause long latencies (multiple milliseconds) when 
>>>>> the threads are started and stopped.
>>>>>
>>>>> The main reason is that the worker threads have to fight over the 
>>>>> Monitor lock when they are woken up from the call to 
>>>>> Monitor::wait. Another reason is that all worker threads call 
>>>>> notify_all when they finish a task and there wakes all all 
>>>>> sleeping worker threads, which will yet again force the worker 
>>>>> threads to fight over the lock.
>>>>>
>>>>> I propose that we use semaphores instead, so that the worker 
>>>>> threads don't have to fight over a lock when they are woken up.
>>>>>
>>>>>
>>>>> The patches build upon the following patch which introduces a 
>>>>> Semaphore utility class. This patch will sent out for review on 
>>>>> the hotspot-dev, since it affects non-GC parts of the code:
>>>>> http://cr.openjdk.java.net/~stefank/8087322/webrev.00/
>>>>> https://bugs.openjdk.java.net/browse/JDK-8087322
>>>>>
>>>>>
>>>>> The first patch that I would like to get reviewed is:
>>>>> http://cr.openjdk.java.net/~stefank/8087323/webrev.00/
>>>>> https://bugs.openjdk.java.net/browse/JDK-8087323 - Unify and split 
>>>>> the work gang classes
>>>>>
>>>>> It prepares for JDK-8087324, by separating the generic WorkGang 
>>>>> implementation from the more elaborate YieldingFlexibleWorkGang 
>>>>> (CMS) implementation. By having this part as a separate patch, I 
>>>>> hope it will be easier to review JDK-8087324. The patch changes 
>>>>> the work gang inheritance from:
>>>>>
>>>>> AbstractWorkGang
>>>>>  WorkGang
>>>>>   FlexibleWorkGang
>>>>>    YieldingFlexibleWorkGang
>>>>>
>>>>> to:
>>>>>
>>>>> AbstractWorkGang
>>>>>  WorkGang
>>>>>  YieldingFlexibleWorkGang
>>>>>
>>>>> Parts of the FlexibleWorkGang and WorkGang code that is going to 
>>>>> be used by both concrete work gang classes, has been moved into 
>>>>> AbstractWorkGang. I've duplicated some code in WorkGang and 
>>>>> YieldingFlexibleWorkGang, but that code will be removed from 
>>>>> WorkGang in the following patch.
>>>>>
>>>>>
>>>>> The second patch I'd like to get reviewed is:
>>>>> http://cr.openjdk.java.net/~stefank/8087324/webrev.00/
>>>>> https://bugs.openjdk.java.net/browse/JDK-8087324 - Use semaphores 
>>>>> when starting and stopping GC task threads
>>>>>
>>>>> It first simplifies the way we distribute the tasks to the GC 
>>>>> worker threads. For example, the coordinator thread dispatches a 
>>>>> task to a specific number of workers, and then waits for all work 
>>>>> to be completed. There's no risk that multiple tasks will be 
>>>>> scheduled simultaneously, so there's no need for the sequences 
>>>>> number that is used in the current implementation.
>>>>>
>>>>> The patch contains two task dispatch / thread synchronization 
>>>>> implementations:
>>>>>
>>>>> The first implementation uses Monitors, similar to what we did 
>>>>> before the patch, but with a slightly lower overhead since the 
>>>>> code calls notify_all less often. It still suffers from the 
>>>>> "thundering heard" problem. When the coordinator thread signals 
>>>>> that the worker threads should start, they all wake up from 
>>>>> Monitor::wait and they all try to lock the Monitor.
>>>>>
>>>>> The second, and the more interesting, implementation uses 
>>>>> semaphores. When the worker threads wake up from the semaphore 
>>>>> wait, they don't have to serialize the execution by taking a lock. 
>>>>> This greatly decreases the time it takes to start and stop the 
>>>>> worker threads.
>>>>>
>>>>> The semaphore implementation is used on all platforms where the 
>>>>> Semaphore class has been implemented in JDK-8087322. So, on some 
>>>>> OS:es the code will revert to the Monitor-based solution until a 
>>>>> Semaphore class has been implemented for that OS. So, porters 
>>>>> might want to consider implementing the Sempahore class.
>>>>>
>>>>> There's also a diagnostic vm option 
>>>>> (-XX:+/-UseSemaphoreGCThreadsSynchronization) to turn off the 
>>>>> Semaphore-based implementation, which can be used to debug this 
>>>>> new code. It's mainly targeted towards support and sustaining 
>>>>> engineering.
>>>>>
>>>>>
>>>>> The patches have been performance tested on Linux, Solaris, OSX, 
>>>>> and Windows.
>>>>>
>>>>> The effects of the patch can be seen by running benchmarks with 
>>>>> small young gen sizes, which triggers frequent and short GCs.
>>>>>
>>>>> For example, here are runs from the SPECjvm2008 xml.transform 
>>>>> benchmark with:
>>>>> -Xmx1g -Xms1g -Xmn64m -XX:+PrintGC -XX:+UseG1GC -jar 
>>>>> SPECjvm2008.jar -ikv xml.transform -it 30 -wt 30
>>>>>
>>>>> I got the following GC times:
>>>>>
>>>>>             Average    Median    99.9 percentile   Max
>>>>> Baseline: 8.76ms    8.44 ms   25.9 ms 34.7 ms
>>>>> Monitor:   6.17 ms 5.88 ms   26.0 ms 49.1 ms
>>>>> Semaphore: 3.43 ms 3.26 ms   13.4 ms           33.4 ms
>>>>>
>>>>> If I run an empty GC task 10 times per GC, by running the 
>>>>> following code:
>>>>> http://cr.openjdk.java.net/~stefank/8087324/timedTask/
>>>>>
>>>>> I get the following numbers to complete the empty GC tasks:
>>>>>
>>>>>             Average    Median    99.9 percentile   Max
>>>>> Baseline: 1.43 ms    0.92 ms   3.43 ms           9.30ms
>>>>> Monitor:    0.75ms 0.72 ms   1.74 ms           2.78ms
>>>>> Semaphore: 0.07 ms 0.07 ms   0.17 ms 0.26 ms
>>>>>
>>>>>
>>>>>
>>>>> The code has been tested with JPRT and our nightly testing suites.
>>>>>
>>>>> I've created a unit test to run a small test with both the 
>>>>> semaphore implementation and the monitor implementation:
>>>>> http://cr.openjdk.java.net/~stefank/8087324/workgangTest/
>>>>>
>>>>> But since we currently don't have code to shutdown worker threads 
>>>>> after they have been started, I don't want to push this test (or 
>>>>> clean it up) until we have that in place. I created this bug for that:
>>>>> https://bugs.openjdk.java.net/browse/JDK-8087340
>>>>>
>>>>> Thanks,
>>>>> StefanK
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150814/8cdb4a9f/attachment.htm>

From jesper.wilhelmsson at oracle.com  Fri Aug 14 14:16:16 2015
From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson)
Date: Fri, 14 Aug 2015 16:16:16 +0200
Subject: RFR: 8133553: Running with -XX:+UseParallelGC -XX:OldSize=30k
	crashes jvm
In-Reply-To: <55CD992D.6010200@oracle.com>
References: <55CD992D.6010200@oracle.com>
Message-ID: <55CDF830.1010904@oracle.com>

Looks good!
/Jesper

David Lindholm skrev den 14/8/15 09:30:
> Hi,
>
> Please review this patch that prevents the jvm to crash (or assert) when run
> with an extremely small value for the -XX:OldSize flag and a collector that uses
> GenCollectorPolicy.
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8133553
> Webrev: http://cr.openjdk.java.net/~david/JDK-8133553/webrev.00/
>
> Testing: Passed JPRT.
>
>
> Thanks,
> David


From david.lindholm at oracle.com  Fri Aug 14 14:16:11 2015
From: david.lindholm at oracle.com (David Lindholm)
Date: Fri, 14 Aug 2015 16:16:11 +0200
Subject: RFR: 8133553: Running with -XX:+UseParallelGC -XX:OldSize=30k
	crashes jvm
In-Reply-To: <55CDF830.1010904@oracle.com>
References: <55CD992D.6010200@oracle.com> <55CDF830.1010904@oracle.com>
Message-ID: <55CDF82B.1080102@oracle.com>

Jesper,

Thanks for the review!


/David

On 2015-08-14 16:16, Jesper Wilhelmsson wrote:
> Looks good!
> /Jesper
>
> David Lindholm skrev den 14/8/15 09:30:
>> Hi,
>>
>> Please review this patch that prevents the jvm to crash (or assert) 
>> when run
>> with an extremely small value for the -XX:OldSize flag and a 
>> collector that uses
>> GenCollectorPolicy.
>>
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8133553
>> Webrev: http://cr.openjdk.java.net/~david/JDK-8133553/webrev.00/
>>
>> Testing: Passed JPRT.
>>
>>
>> Thanks,
>> David



From staffan.friberg at oracle.com  Fri Aug 14 16:41:05 2015
From: staffan.friberg at oracle.com (Staffan Friberg)
Date: Fri, 14 Aug 2015 09:41:05 -0700
Subject: RFR (S): 8133530: Add JFR event for evacuation statistics
In-Reply-To: <55CDD14C.2050600@oracle.com>
References: <1439480605.2271.65.camel@oracle.com> <55CDD14C.2050600@oracle.com>
Message-ID: <55CE1A21.9030506@oracle.com>

Hi Thomas,

Any reason to go via the G1EvacSummary class and not just send the stats 
(G1EvacStats) straight through to the event code?

It seems like it just adds a very short live object and a lot of copies 
of different fields, but I might miss some bigger picture thing.

Thanks,
Staffan


On 08/14/2015 04:30 AM, Mikael Gerdin wrote:
> Hi Thomas,
>
> On 2015-08-13 17:43, Thomas Schatzl wrote:
>> Hi all,
>>
>>    can I have reviews for this split-out from 8073013 that adds JFR
>> support for evacuation statistics.
>>
>> The code has been changed to (hopefully) correspond better to the way
>> code that sends JFR events should look like.
>>
>> CR:
>> https://bugs.openjdk.java.net/browse/JDK-8133530
>> Webrev:
>> http://cr.openjdk.java.net/~tschatzl/8133530/webrev/
>
> in gcTrace.hpp
> 39 #include "gc/g1/g1YCTypes.hpp"
> 40 #include "gcHeapSummary.hpp"
> 41 #endif
> 42
>
> you should use the full path to gcHeapSummary.hpp
>
> in gcTraceSend.cpp
> 240   if (surv_evt.should_commit()) {
> 241     surv_evt.set_gcId(GCId::peek().id());
> 242     surv_evt.set_allocated(summary.allocated() * HeapWordSize);
>
> The other events use
> 232     e.set_gcId(_shared_gc_info.gc_id().id());
>
> When you assert_set_gc_id you actually verify that the shared gc info 
> id is set.
>
> /Mikael
>
>>
>> Testing:
>> jprt
>>
>> Thanks,
>>    Thomas
>>
>



From jeremymanson at google.com  Mon Aug 17 05:03:39 2015
From: jeremymanson at google.com (Jeremy Manson)
Date: Sun, 16 Aug 2015 22:03:39 -0700
Subject: Low-Overhead Heap Profiling
In-Reply-To: <035570A0-5A63-4A99-96EE-D8A067A33B14@gmail.com>
References: <etPan.5589bc61.5358f81b.1215a@tw-mbp-tprintezis>
	<CAPYFHW3fVjDfc1j6du5dbAvaaKZsErx7vL8njx7hFhe-Ybvk6Q@mail.gmail.com>
	<etPan.558aef95.6b2c6c0b.1215a@tw-mbp-tprintezis>
	<etPan.558c716e.236c4c7e.1215a@tw-mbp-tprintezis>
	<CAPYFHW2ryBf7DyAsEyAPW46atiWvFQpvYV-9HROaB=yp1=2p-A@mail.gmail.com>
	<B3482D30-E883-4002-A8D6-8651A1D3349E@oracle.com>
	<CAPYFHW00zGByRRTz3m_Xx=2OndU9WybK6=vQt_-cchDqe3fuZg@mail.gmail.com>
	<035570A0-5A63-4A99-96EE-D8A067A33B14@gmail.com>
Message-ID: <CAPYFHW2Eq696O1bm0JxemhFwoQjhWhQxxHD+t8e3iAP0ES1oDA@mail.gmail.com>

Working on it.  I hope to have something to share in the next couple of
weeks.

Jeremy

On Wed, Aug 12, 2015 at 4:28 AM, Kees Jan Koster <kjkoster at gmail.com> wrote:

> Guys,
>
> This piqued my interest. Where can I find the draft JEP and the sources
> for this, please?
>
> Kees Jan
>
>
> > On 4 Aug 2015, at 23:22, Jeremy Manson <jeremymanson at google.com> wrote:
> >
> > Thanks, Staffan.  I've been tinkering with a draft JEP, but it has been
> going slowly because of other craziness in my life.  Some responses:
> >
> > 1) It was a fair bit of work to do it, mostly because it was the first
> time I had tinkered with c2.  This was 5 years ago.  Most of the work since
> then has been dealing with merge conflicts.
> >
> > 2) I did envision a JVMTI interface.  More in the JEP.
> >
> > 3) We have some internal tests, but obviously, we'd have to figure out
> how to externalize them.  For native code, it's much easier only to have to
> worry about building and running on Linux.  Presumably, there's already
> code in there for JVMTI.
> >
> > I'll try to get a JEP going for real, although it might not happen in
> the next week or so, because I'm moving next week (+the JVMLS).
> >
> > Jeremy
> >
> > On Tue, Aug 4, 2015 at 4:04 AM, Staffan Larsen <
> staffan.larsen at oracle.com> wrote:
> > Hi,
> >
> > Sorry for being away for so long.
> >
> > If memory serves me right then the current AllocObjectInNewTLAB JFR
> event was written as a way to quickly get some allocation profiling
> information with a minimum of VM changes. It probably also carried over to
> Hotspot from JRockit. I agree that we can and should collect better
> information than what that event does.
> >
> > I like the approach of instrumenting the interpreter and compiler to do
> the sampling. I had thought it would be a lot of work to do it and was
> reluctant to go down that path. If the work is already done and Jeremy has
> maintained it for a few years without great problems, I think we should
> look at bringing it in. I am not worried about the overhead of a decrement
> and a compare in the allocation path, but of course we should benchmark
> that.
> >
> > It wasn?t clear to me from the discussion how (or if) this was being
> exposed to end users. Was the proposal to just have the native entry points
> in the VM and have these be used by various agents? Would this be part of
> JVMTI?
> >
> > I think a draft JEP would be the logical next step and make it easier
> for us all to discuss what exactly the proposal should contain. Also, let?s
> not forget the need for tests for the feature.
> >
> > Thanks,
> > /Staffan
> >
> >
> >
>
>
> --
> Kees Jan
>
> http://java-monitor.com/
> kjkoster at kjkoster.org
> +31651838192
>
> The secret of success lies in the stability of the goal. -- Benjamin
> Disraeli
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150816/8e664c82/attachment.htm>

From stefan.karlsson at oracle.com  Mon Aug 17 07:10:04 2015
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Mon, 17 Aug 2015 09:10:04 +0200
Subject: RFR: 8087324: Use semaphores when starting and stopping GC task
	threads
In-Reply-To: <55CDD087.3050301@oracle.com>
References: <557AF21B.2090102@oracle.com>	<559415DD.7030604@oracle.com>	<55952520.6030609@oracle.com>	<559552A9.8030008@oracle.com>
	<55CDD087.3050301@oracle.com>
Message-ID: <55D188CC.1040101@oracle.com>

On 2015-08-14 13:27, Stefan Johansson wrote:
> Hi Stefan,
>
> On 2015-07-02 17:03, Jon Masamitsu wrote:
>>
>>
>> On 07/02/2015 04:48 AM, Stefan Karlsson wrote:
>>>
>>>
>>> On 2015-07-01 18:31, Jon Masamitsu wrote:
>>>>
>>>>
>>>> On 6/12/2015 7:52 AM, Stefan Karlsson wrote:
>>>>> Hi all,
>>>>>
>>>>> The current implementation to distribute tasks to GC worker 
>>>>> threads often cause long latencies (multiple milliseconds) when 
>>>>> the threads are started and stopped.
>>>>>
>>>>> The main reason is that the worker threads have to fight over the 
>>>>> Monitor lock when they are woken up from the call to 
>>>>> Monitor::wait. Another reason is that all worker threads call 
>>>>> notify_all when they finish a task and there wakes all all 
>>>>> sleeping worker threads, which will yet again force the worker 
>>>>> threads to fight over the lock.
>>>>>
>>>>> I propose that we use semaphores instead, so that the worker 
>>>>> threads don't have to fight over a lock when they are woken up.
>>>>>
>>>>>
>>>>> The patches build upon the following patch which introduces a 
>>>>> Semaphore utility class. This patch will sent out for review on 
>>>>> the hotspot-dev, since it affects non-GC parts of the code:
>>>>> http://cr.openjdk.java.net/~stefank/8087322/webrev.00/
>>>>> https://bugs.openjdk.java.net/browse/JDK-8087322
>>>>>
>>>>>
>>>>> The first patch that I would like to get reviewed is:
>>>>> http://cr.openjdk.java.net/~stefank/8087323/webrev.00/
>>>>> https://bugs.openjdk.java.net/browse/JDK-8087323 - Unify and split 
>>>>> the work gang classes
>>>>>
>>>>> It prepares for JDK-8087324, by separating the generic WorkGang 
>>>>> implementation from the more elaborate YieldingFlexibleWorkGang 
>>>>> (CMS) implementation. By having this part as a separate patch, I 
>>>>> hope it will be easier to review JDK-8087324. The patch changes 
>>>>> the work gang inheritance from:
>>>>>
>>>>> AbstractWorkGang
>>>>>  WorkGang
>>>>>   FlexibleWorkGang
>>>>>    YieldingFlexibleWorkGang
>>>>>
>>>>> to:
>>>>>
>>>>> AbstractWorkGang
>>>>>  WorkGang
>>>>>  YieldingFlexibleWorkGang
>>>>>
>>>>> Parts of the FlexibleWorkGang and WorkGang code that is going to 
>>>>> be used by both concrete work gang classes, has been moved into 
>>>>> AbstractWorkGang. I've duplicated some code in WorkGang and 
>>>>> YieldingFlexibleWorkGang, but that code will be removed from 
>>>>> WorkGang in the following patch.
> The changes for 8087323 looks good. I agree with Jons comments below 
> about removing the function is_Yielding... and the cast in 
> YieldingFlexibleGangTask.

Removed.

Thanks,
StefanK

>
> Thanks,
> Stefan
>
>>>>
>>>> http://cr.openjdk.java.net/~stefank/8087323/webrev.00/src/share/vm/gc/cms/yieldingWorkgroup.hpp.frames.html
>>>>
>>>> There seems to be only one definition of 
>>>> is_YieldingFlexibleGang_task() now.  Is that right?  Is that useful?
>>>>
>>>>   131   NOT_PRODUCT(virtual bool is_YieldingFlexibleGang_task() const {
>>>>   132     return true;
>>>>   133   })
>>>
>>> I agree. I don't think we need it anymore.
>>>
>> Thanks.
>>
>>>
>>>>
>>>> Not a change  in your patch but
>>>>
>>>>    86   AbstractWorkGang(const char* name, uint workers, bool are_GC_task_threads, bool are_ConcurrentGC_threads) :
>>>>    87       _name(name),
>>>>    88       _total_workers(workers),
>>>>    89       _active_workers(UseDynamicNumberOfGCThreads ? 1U : workers),
>>>>    90       _are_GC_task_threads(are_GC_task_threads),
>>>>    91       _are_Concurren
>>>>
>>>> _active_workers is always calculated as >= 2 unless _total_workers 
>>>> is only 1.
>>>> So line 89 should be
>>>>
>>>> _active_workers(UseDynamicNumberOfGCThreads ? MIN2(2, workers) : 
>>>> workers)
>>>>
>>>> Should I file a CR for that?  Or do you want to include it.
>>>
>>> I'm not sure that what is proposed above is correct. I see that 
>>> AdaptiveSizePolicy::calc_active_workers returns 2 as a minimum, but 
>>> both ConcurrentMark::calc_parallel_marking_threads and 
>>> AdaptiveSizePolicy::calc_active_conc_workers can return 1.
>>>
>>> I also don't think it should be AbstractWorkGang's responsibility to 
>>> have the knowledge about the minimum number of worker threads that 
>>> are used when UseDynamicNumberOfGCThreads are turned on. Maybe we 
>>> should set it to 0, and let the calc_*_active_workers setup the 
>>> default value.
>>
>> I think that at one time I had tried to set the default to 0 and 
>> something failed.  I can see the point though.
>>
>>>
>>> I would prefer to handle any changes, to this part of the code, as 
>>> separate RFEs.
>>
>> Fair enough.  If I think it's worth doing, I'll file and RFE. 
>> Probably something more than just setting
>> it to 2 (maybe picking a default value with the help of 
>> AdaptiveSizePolicy).
>>
>>
>>>
>>>> Have you considered (maybe for a later patch) changing 
>>>> YieldingFlexibleWorkGang to
>>>> simply YieldingWorkGang?  The "Flexible" attribute of 
>>>> YieldingFlexibleWorkGang having
>>>> been moved into AbstractWorkGang.
>>>
>>> I thought about it, but didn't think it was important enough to 
>>> warrant that change in this patch. I wouldn't mind if a RFE was 
>>> created to change the name.
>>
>> I'll file the RFE  if you agree with the point that "Flexible" 
>> describes what AbstractWorkGang does now.
>>
>>>
>>>>
>>>> http://cr.openjdk.java.net/~stefank/8087323/webrev.00/src/share/vm/gc/cms/yieldingWorkgroup.cpp.frames.html
>>>>
>>>> Is the cast at 53 necessary? I see it in the original code too.
>>>>
>>>>    50 AbstractGangWorker* YieldingFlexibleWorkGang::allocate_worker(uint which) {
>>>>    51   YieldingFlexibleGangWorker* new_member =
>>>>    52       new YieldingFlexibleGangWorker(this, which);
>>>>    53   return (YieldingFlexibleGangWorker*) new_member;
>>>>    54 }
>>>
>>> Yes, this is unnecessary.
>>
>> Thanks.
>>
>> Jon
>>
>>>
>>>>
>>>> The rest looks good.
>>>
>>> Thanks.
>>>
>>>>
>>>> I'll do the second patch next.
>>>
>>> Great.
>>>
>>> StefanK
>>>
>>>>
>>>> Jon
>>>>
>>>>>
>>>>>
>>>>> The second patch I'd like to get reviewed is:
>>>>> http://cr.openjdk.java.net/~stefank/8087324/webrev.00/
>>>>> https://bugs.openjdk.java.net/browse/JDK-8087324 - Use semaphores 
>>>>> when starting and stopping GC task threads
>>>>>
>>>>> It first simplifies the way we distribute the tasks to the GC 
>>>>> worker threads. For example, the coordinator thread dispatches a 
>>>>> task to a specific number of workers, and then waits for all work 
>>>>> to be completed. There's no risk that multiple tasks will be 
>>>>> scheduled simultaneously, so there's no need for the sequences 
>>>>> number that is used in the current implementation.
>>>>>
>>>>> The patch contains two task dispatch / thread synchronization 
>>>>> implementations:
>>>>>
>>>>> The first implementation uses Monitors, similar to what we did 
>>>>> before the patch, but with a slightly lower overhead since the 
>>>>> code calls notify_all less often. It still suffers from the 
>>>>> "thundering heard" problem. When the coordinator thread signals 
>>>>> that the worker threads should start, they all wake up from 
>>>>> Monitor::wait and they all try to lock the Monitor.
>>>>>
>>>>> The second, and the more interesting, implementation uses 
>>>>> semaphores. When the worker threads wake up from the semaphore 
>>>>> wait, they don't have to serialize the execution by taking a lock. 
>>>>> This greatly decreases the time it takes to start and stop the 
>>>>> worker threads.
>>>>>
>>>>> The semaphore implementation is used on all platforms where the 
>>>>> Semaphore class has been implemented in JDK-8087322. So, on some 
>>>>> OS:es the code will revert to the Monitor-based solution until a 
>>>>> Semaphore class has been implemented for that OS. So, porters 
>>>>> might want to consider implementing the Sempahore class.
>>>>>
>>>>> There's also a diagnostic vm option 
>>>>> (-XX:+/-UseSemaphoreGCThreadsSynchronization) to turn off the 
>>>>> Semaphore-based implementation, which can be used to debug this 
>>>>> new code. It's mainly targeted towards support and sustaining 
>>>>> engineering.
>>>>>
>>>>>
>>>>> The patches have been performance tested on Linux, Solaris, OSX, 
>>>>> and Windows.
>>>>>
>>>>> The effects of the patch can be seen by running benchmarks with 
>>>>> small young gen sizes, which triggers frequent and short GCs.
>>>>>
>>>>> For example, here are runs from the SPECjvm2008 xml.transform 
>>>>> benchmark with:
>>>>> -Xmx1g -Xms1g -Xmn64m -XX:+PrintGC -XX:+UseG1GC -jar 
>>>>> SPECjvm2008.jar -ikv xml.transform -it 30 -wt 30
>>>>>
>>>>> I got the following GC times:
>>>>>
>>>>>             Average    Median    99.9 percentile   Max
>>>>> Baseline: 8.76ms    8.44 ms   25.9 ms 34.7 ms
>>>>> Monitor:   6.17 ms 5.88 ms   26.0 ms 49.1 ms
>>>>> Semaphore: 3.43 ms 3.26 ms   13.4 ms           33.4 ms
>>>>>
>>>>> If I run an empty GC task 10 times per GC, by running the 
>>>>> following code:
>>>>> http://cr.openjdk.java.net/~stefank/8087324/timedTask/
>>>>>
>>>>> I get the following numbers to complete the empty GC tasks:
>>>>>
>>>>>             Average    Median    99.9 percentile   Max
>>>>> Baseline: 1.43 ms    0.92 ms   3.43 ms           9.30ms
>>>>> Monitor:    0.75ms 0.72 ms   1.74 ms           2.78ms
>>>>> Semaphore: 0.07 ms 0.07 ms   0.17 ms 0.26 ms
>>>>>
>>>>>
>>>>>
>>>>> The code has been tested with JPRT and our nightly testing suites.
>>>>>
>>>>> I've created a unit test to run a small test with both the 
>>>>> semaphore implementation and the monitor implementation:
>>>>> http://cr.openjdk.java.net/~stefank/8087324/workgangTest/
>>>>>
>>>>> But since we currently don't have code to shutdown worker threads 
>>>>> after they have been started, I don't want to push this test (or 
>>>>> clean it up) until we have that in place. I created this bug for that:
>>>>> https://bugs.openjdk.java.net/browse/JDK-8087340
>>>>>
>>>>> Thanks,
>>>>> StefanK
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150817/01417efd/attachment.htm>

From stefan.karlsson at oracle.com  Mon Aug 17 07:12:25 2015
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Mon, 17 Aug 2015 09:12:25 +0200
Subject: RFR: 8087324: Use semaphores when starting and stopping GC task
	threads
In-Reply-To: <55CDE143.20808@oracle.com>
References: <557AF21B.2090102@oracle.com>	<5591122F.2000704@oracle.com>	<5595784A.9050903@oracle.com>	<55957BCD.9020604@oracle.com>	<55CCD8D5.1070509@oracle.com>
	<55CDE143.20808@oracle.com>
Message-ID: <55D18959.8000603@oracle.com>

On 2015-08-14 14:38, Stefan Johansson wrote:
> Hi Stefan,
>
> Nice change, looks really good,

Thanks.

> but I have a few small comments.
>
> In workgroup.hpp:
>  177   GangTaskDispatcher* dispatcher()const {
> Add a space before const.

Fixed.

> ---
>
> In workgroup.cpp:
>  122       // Limit the semaphore value to the number of workers.
>  123       _start_semaphore(new Semaphore()),
>  124       _end_semaphore(new Semaphore())
> Seems like the limit has been lost. I would prefer to have it added, 
> but if that's somehow problematic just remove the comment.

Removed the comment.

>
>  140     // Wait for the last worker to signal the coordinator.
>  141     _end_semaphore->wait();
>  142
>  143     // No workers are allowed to read the state variables after 
> the coordinator has been signaled.
>  144     _task         = NULL;
>  145     _started      = 0;
>  146     _not_finished = 0;
> Do we need to set _not_finished to 0 or could we instead assert that 
> it already is 0 since that's when _end_semaphore should have been 
> signaled.

Changed into an assert.

> ---
>
> I don't need to see a new webrev for the above changes. Reviewed.

Thanks,
StefanK

>
> Thanks,
> Stefan
>
> On 2015-08-13 19:50, Stefan Karlsson wrote:
>> Hi all,
>>
>> Could I get a second review for the patch below (plus the suggested 
>> removal)?
>>
>> Thanks,
>> StefanK
>>
>> On 2015-07-02 19:58, Stefan Karlsson wrote:
>>> On 2015-07-02 19:43, Jon Masamitsu wrote:
>>>>
>>>>
>>>> On 6/29/2015 2:38 AM, Stefan Karlsson wrote:
>>>>> Hi all,
>>>>>
>>>>> "8087322: Implement a Semaphore utility class" has now been 
>>>>> pushed, so I've updated the patch to reflect the changes.
>>>>>
>>>>> http://cr.openjdk.java.net/~stefank/8087324/webrev.01.delta
>>>>> http://cr.openjdk.java.net/~stefank/8087324/webrev.01
>>>> http://cr.openjdk.java.net/~stefank/8087324/webrev.01/src/share/vm/gc/shared/workgroup.hpp.frames.html
>>>>
>>>> Are these used?
>>>>
>>>>   194   void print_worker_started_task(AbstractGangTask* task, uint worker_id);
>>>>   195   void print_worker_finished_task(AbstractGangTask* task, uint worker_id);
>>>
>>> No. I'll remove them.
>>>
>>>>
>>>>
>>>> Rest looks good.  Just one question (just for my education).
>>>>   
>>>> http://cr.openjdk.java.net/~stefank/8087324/webrev.01/src/share/vm/gc/shared/workgroup.cpp.frames.html
>>>>
>>>>   336 void GangWorker::loop() {
>>>>   337   while (true) {
>>>>   338     WorkData data = wait_for_task();
>>>>   339
>>>>   340     run_task(data);
>>>>   341
>>>>   342     signal_task_done();
>>>>   343   }
>>>>   344 }
>>>>
>>>> Does this allow the same thread to execute more than 1
>>>> task ("data" here)?  Meaning if 2 threads are  requested but
>>>> 1 thread is not scheduled to a cpu, will the other thread
>>>> do both chunks of work?
>>>
>>> Yes, that's correct.
>>>
>>> Thanks for reviewing!
>>> StefanK
>>>
>>>> Jon
>>>>
>>>>
>>>>>
>>>>> - The IMPLEMENTS_SEMAPHORE_CLASS define was removed, since all 
>>>>> platforms need to provide a Semaphore implementation.
>>>>>
>>>>> - Removed the need to pass down "max number of workers" to the 
>>>>> Semaphore constructor.
>>>>>
>>>>> - Updated semaphore.hpp include path
>>>>>
>>>>> Thanks,
>>>>> StefanK
>>>>>
>>>>>
>>>>> On 2015-06-12 16:52, Stefan Karlsson wrote:
>>>>>> Hi all,
>>>>>>
>>>>>> The current implementation to distribute tasks to GC worker 
>>>>>> threads often cause long latencies (multiple milliseconds) when 
>>>>>> the threads are started and stopped.
>>>>>>
>>>>>> The main reason is that the worker threads have to fight over the 
>>>>>> Monitor lock when they are woken up from the call to 
>>>>>> Monitor::wait. Another reason is that all worker threads call 
>>>>>> notify_all when they finish a task and there wakes all all 
>>>>>> sleeping worker threads, which will yet again force the worker 
>>>>>> threads to fight over the lock.
>>>>>>
>>>>>> I propose that we use semaphores instead, so that the worker 
>>>>>> threads don't have to fight over a lock when they are woken up.
>>>>>>
>>>>>>
>>>>>> The patches build upon the following patch which introduces a 
>>>>>> Semaphore utility class. This patch will sent out for review on 
>>>>>> the hotspot-dev, since it affects non-GC parts of the code:
>>>>>> http://cr.openjdk.java.net/~stefank/8087322/webrev.00/
>>>>>> https://bugs.openjdk.java.net/browse/JDK-8087322
>>>>>>
>>>>>>
>>>>>> The first patch that I would like to get reviewed is:
>>>>>> http://cr.openjdk.java.net/~stefank/8087323/webrev.00/
>>>>>> https://bugs.openjdk.java.net/browse/JDK-8087323 - Unify and 
>>>>>> split the work gang classes
>>>>>>
>>>>>> It prepares for JDK-8087324, by separating the generic WorkGang 
>>>>>> implementation from the more elaborate YieldingFlexibleWorkGang 
>>>>>> (CMS) implementation. By having this part as a separate patch, I 
>>>>>> hope it will be easier to review JDK-8087324. The patch changes 
>>>>>> the work gang inheritance from:
>>>>>>
>>>>>> AbstractWorkGang
>>>>>>  WorkGang
>>>>>>   FlexibleWorkGang
>>>>>>    YieldingFlexibleWorkGang
>>>>>>
>>>>>> to:
>>>>>>
>>>>>> AbstractWorkGang
>>>>>>  WorkGang
>>>>>>  YieldingFlexibleWorkGang
>>>>>>
>>>>>> Parts of the FlexibleWorkGang and WorkGang code that is going to 
>>>>>> be used by both concrete work gang classes, has been moved into 
>>>>>> AbstractWorkGang. I've duplicated some code in WorkGang and 
>>>>>> YieldingFlexibleWorkGang, but that code will be removed from 
>>>>>> WorkGang in the following patch.
>>>>>>
>>>>>>
>>>>>> The second patch I'd like to get reviewed is:
>>>>>> http://cr.openjdk.java.net/~stefank/8087324/webrev.00/
>>>>>> https://bugs.openjdk.java.net/browse/JDK-8087324 - Use semaphores 
>>>>>> when starting and stopping GC task threads
>>>>>>
>>>>>> It first simplifies the way we distribute the tasks to the GC 
>>>>>> worker threads. For example, the coordinator thread dispatches a 
>>>>>> task to a specific number of workers, and then waits for all work 
>>>>>> to be completed. There's no risk that multiple tasks will be 
>>>>>> scheduled simultaneously, so there's no need for the sequences 
>>>>>> number that is used in the current implementation.
>>>>>>
>>>>>> The patch contains two task dispatch / thread synchronization 
>>>>>> implementations:
>>>>>>
>>>>>> The first implementation uses Monitors, similar to what we did 
>>>>>> before the patch, but with a slightly lower overhead since the 
>>>>>> code calls notify_all less often. It still suffers from the 
>>>>>> "thundering heard" problem. When the coordinator thread signals 
>>>>>> that the worker threads should start, they all wake up from 
>>>>>> Monitor::wait and they all try to lock the Monitor.
>>>>>>
>>>>>> The second, and the more interesting, implementation uses 
>>>>>> semaphores. When the worker threads wake up from the semaphore 
>>>>>> wait, they don't have to serialize the execution by taking a 
>>>>>> lock. This greatly decreases the time it takes to start and stop 
>>>>>> the worker threads.
>>>>>>
>>>>>> The semaphore implementation is used on all platforms where the 
>>>>>> Semaphore class has been implemented in JDK-8087322. So, on some 
>>>>>> OS:es the code will revert to the Monitor-based solution until a 
>>>>>> Semaphore class has been implemented for that OS. So, porters 
>>>>>> might want to consider implementing the Sempahore class.
>>>>>>
>>>>>> There's also a diagnostic vm option 
>>>>>> (-XX:+/-UseSemaphoreGCThreadsSynchronization) to turn off the 
>>>>>> Semaphore-based implementation, which can be used to debug this 
>>>>>> new code. It's mainly targeted towards support and sustaining 
>>>>>> engineering.
>>>>>>
>>>>>>
>>>>>> The patches have been performance tested on Linux, Solaris, OSX, 
>>>>>> and Windows.
>>>>>>
>>>>>> The effects of the patch can be seen by running benchmarks with 
>>>>>> small young gen sizes, which triggers frequent and short GCs.
>>>>>>
>>>>>> For example, here are runs from the SPECjvm2008 xml.transform 
>>>>>> benchmark with:
>>>>>> -Xmx1g -Xms1g -Xmn64m -XX:+PrintGC -XX:+UseG1GC -jar 
>>>>>> SPECjvm2008.jar -ikv xml.transform -it 30 -wt 30
>>>>>>
>>>>>> I got the following GC times:
>>>>>>
>>>>>>             Average    Median    99.9 percentile Max
>>>>>> Baseline: 8.76ms    8.44 ms   25.9 ms 34.7 ms
>>>>>> Monitor:   6.17 ms 5.88 ms   26.0 ms 49.1 ms
>>>>>> Semaphore: 3.43 ms 3.26 ms   13.4 ms           33.4 ms
>>>>>>
>>>>>> If I run an empty GC task 10 times per GC, by running the 
>>>>>> following code:
>>>>>> http://cr.openjdk.java.net/~stefank/8087324/timedTask/
>>>>>>
>>>>>> I get the following numbers to complete the empty GC tasks:
>>>>>>
>>>>>>             Average    Median    99.9 percentile Max
>>>>>> Baseline: 1.43 ms    0.92 ms   3.43 ms           9.30ms
>>>>>> Monitor:    0.75ms 0.72 ms   1.74 ms           2.78ms
>>>>>> Semaphore: 0.07 ms 0.07 ms   0.17 ms 0.26 ms
>>>>>>
>>>>>>
>>>>>>
>>>>>> The code has been tested with JPRT and our nightly testing suites.
>>>>>>
>>>>>> I've created a unit test to run a small test with both the 
>>>>>> semaphore implementation and the monitor implementation:
>>>>>> http://cr.openjdk.java.net/~stefank/8087324/workgangTest/
>>>>>>
>>>>>> But since we currently don't have code to shutdown worker threads 
>>>>>> after they have been started, I don't want to push this test (or 
>>>>>> clean it up) until we have that in place. I created this bug for 
>>>>>> that:
>>>>>> https://bugs.openjdk.java.net/browse/JDK-8087340
>>>>>>
>>>>>> Thanks,
>>>>>> StefanK
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150817/35dc02ca/attachment.htm>

From stefan.karlsson at oracle.com  Mon Aug 17 07:12:33 2015
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Mon, 17 Aug 2015 09:12:33 +0200
Subject: RFR: 8087324: Use semaphores when starting and stopping GC task
	threads
In-Reply-To: <55CDE143.20808@oracle.com>
References: <557AF21B.2090102@oracle.com>	<5591122F.2000704@oracle.com>	<5595784A.9050903@oracle.com>	<55957BCD.9020604@oracle.com>	<55CCD8D5.1070509@oracle.com>
	<55CDE143.20808@oracle.com>
Message-ID: <55D18961.80902@oracle.com>

On 2015-08-14 14:38, Stefan Johansson wrote:
> Hi Stefan,
>
> Nice change, looks really good,

Thanks.

> but I have a few small comments.
>
> In workgroup.hpp:
>  177   GangTaskDispatcher* dispatcher()const {
> Add a space before const.

Fixed.

> ---
>
> In workgroup.cpp:
>  122       // Limit the semaphore value to the number of workers.
>  123       _start_semaphore(new Semaphore()),
>  124       _end_semaphore(new Semaphore())
> Seems like the limit has been lost. I would prefer to have it added, 
> but if that's somehow problematic just remove the comment.

Removed the comment.

>
>  140     // Wait for the last worker to signal the coordinator.
>  141     _end_semaphore->wait();
>  142
>  143     // No workers are allowed to read the state variables after 
> the coordinator has been signaled.
>  144     _task         = NULL;
>  145     _started      = 0;
>  146     _not_finished = 0;
> Do we need to set _not_finished to 0 or could we instead assert that 
> it already is 0 since that's when _end_semaphore should have been 
> signaled.

Changed into an assert.

> ---
>
> I don't need to see a new webrev for the above changes. Reviewed.

Thanks,
StefanK

>
> Thanks,
> Stefan
>
> On 2015-08-13 19:50, Stefan Karlsson wrote:
>> Hi all,
>>
>> Could I get a second review for the patch below (plus the suggested 
>> removal)?
>>
>> Thanks,
>> StefanK
>>
>> On 2015-07-02 19:58, Stefan Karlsson wrote:
>>> On 2015-07-02 19:43, Jon Masamitsu wrote:
>>>>
>>>>
>>>> On 6/29/2015 2:38 AM, Stefan Karlsson wrote:
>>>>> Hi all,
>>>>>
>>>>> "8087322: Implement a Semaphore utility class" has now been 
>>>>> pushed, so I've updated the patch to reflect the changes.
>>>>>
>>>>> http://cr.openjdk.java.net/~stefank/8087324/webrev.01.delta
>>>>> http://cr.openjdk.java.net/~stefank/8087324/webrev.01
>>>> http://cr.openjdk.java.net/~stefank/8087324/webrev.01/src/share/vm/gc/shared/workgroup.hpp.frames.html
>>>>
>>>> Are these used?
>>>>
>>>>   194   void print_worker_started_task(AbstractGangTask* task, uint worker_id);
>>>>   195   void print_worker_finished_task(AbstractGangTask* task, uint worker_id);
>>>
>>> No. I'll remove them.
>>>
>>>>
>>>>
>>>> Rest looks good.  Just one question (just for my education).
>>>>   
>>>> http://cr.openjdk.java.net/~stefank/8087324/webrev.01/src/share/vm/gc/shared/workgroup.cpp.frames.html
>>>>
>>>>   336 void GangWorker::loop() {
>>>>   337   while (true) {
>>>>   338     WorkData data = wait_for_task();
>>>>   339
>>>>   340     run_task(data);
>>>>   341
>>>>   342     signal_task_done();
>>>>   343   }
>>>>   344 }
>>>>
>>>> Does this allow the same thread to execute more than 1
>>>> task ("data" here)?  Meaning if 2 threads are  requested but
>>>> 1 thread is not scheduled to a cpu, will the other thread
>>>> do both chunks of work?
>>>
>>> Yes, that's correct.
>>>
>>> Thanks for reviewing!
>>> StefanK
>>>
>>>> Jon
>>>>
>>>>
>>>>>
>>>>> - The IMPLEMENTS_SEMAPHORE_CLASS define was removed, since all 
>>>>> platforms need to provide a Semaphore implementation.
>>>>>
>>>>> - Removed the need to pass down "max number of workers" to the 
>>>>> Semaphore constructor.
>>>>>
>>>>> - Updated semaphore.hpp include path
>>>>>
>>>>> Thanks,
>>>>> StefanK
>>>>>
>>>>>
>>>>> On 2015-06-12 16:52, Stefan Karlsson wrote:
>>>>>> Hi all,
>>>>>>
>>>>>> The current implementation to distribute tasks to GC worker 
>>>>>> threads often cause long latencies (multiple milliseconds) when 
>>>>>> the threads are started and stopped.
>>>>>>
>>>>>> The main reason is that the worker threads have to fight over the 
>>>>>> Monitor lock when they are woken up from the call to 
>>>>>> Monitor::wait. Another reason is that all worker threads call 
>>>>>> notify_all when they finish a task and there wakes all all 
>>>>>> sleeping worker threads, which will yet again force the worker 
>>>>>> threads to fight over the lock.
>>>>>>
>>>>>> I propose that we use semaphores instead, so that the worker 
>>>>>> threads don't have to fight over a lock when they are woken up.
>>>>>>
>>>>>>
>>>>>> The patches build upon the following patch which introduces a 
>>>>>> Semaphore utility class. This patch will sent out for review on 
>>>>>> the hotspot-dev, since it affects non-GC parts of the code:
>>>>>> http://cr.openjdk.java.net/~stefank/8087322/webrev.00/
>>>>>> https://bugs.openjdk.java.net/browse/JDK-8087322
>>>>>>
>>>>>>
>>>>>> The first patch that I would like to get reviewed is:
>>>>>> http://cr.openjdk.java.net/~stefank/8087323/webrev.00/
>>>>>> https://bugs.openjdk.java.net/browse/JDK-8087323 - Unify and 
>>>>>> split the work gang classes
>>>>>>
>>>>>> It prepares for JDK-8087324, by separating the generic WorkGang 
>>>>>> implementation from the more elaborate YieldingFlexibleWorkGang 
>>>>>> (CMS) implementation. By having this part as a separate patch, I 
>>>>>> hope it will be easier to review JDK-8087324. The patch changes 
>>>>>> the work gang inheritance from:
>>>>>>
>>>>>> AbstractWorkGang
>>>>>>  WorkGang
>>>>>>   FlexibleWorkGang
>>>>>>    YieldingFlexibleWorkGang
>>>>>>
>>>>>> to:
>>>>>>
>>>>>> AbstractWorkGang
>>>>>>  WorkGang
>>>>>>  YieldingFlexibleWorkGang
>>>>>>
>>>>>> Parts of the FlexibleWorkGang and WorkGang code that is going to 
>>>>>> be used by both concrete work gang classes, has been moved into 
>>>>>> AbstractWorkGang. I've duplicated some code in WorkGang and 
>>>>>> YieldingFlexibleWorkGang, but that code will be removed from 
>>>>>> WorkGang in the following patch.
>>>>>>
>>>>>>
>>>>>> The second patch I'd like to get reviewed is:
>>>>>> http://cr.openjdk.java.net/~stefank/8087324/webrev.00/
>>>>>> https://bugs.openjdk.java.net/browse/JDK-8087324 - Use semaphores 
>>>>>> when starting and stopping GC task threads
>>>>>>
>>>>>> It first simplifies the way we distribute the tasks to the GC 
>>>>>> worker threads. For example, the coordinator thread dispatches a 
>>>>>> task to a specific number of workers, and then waits for all work 
>>>>>> to be completed. There's no risk that multiple tasks will be 
>>>>>> scheduled simultaneously, so there's no need for the sequences 
>>>>>> number that is used in the current implementation.
>>>>>>
>>>>>> The patch contains two task dispatch / thread synchronization 
>>>>>> implementations:
>>>>>>
>>>>>> The first implementation uses Monitors, similar to what we did 
>>>>>> before the patch, but with a slightly lower overhead since the 
>>>>>> code calls notify_all less often. It still suffers from the 
>>>>>> "thundering heard" problem. When the coordinator thread signals 
>>>>>> that the worker threads should start, they all wake up from 
>>>>>> Monitor::wait and they all try to lock the Monitor.
>>>>>>
>>>>>> The second, and the more interesting, implementation uses 
>>>>>> semaphores. When the worker threads wake up from the semaphore 
>>>>>> wait, they don't have to serialize the execution by taking a 
>>>>>> lock. This greatly decreases the time it takes to start and stop 
>>>>>> the worker threads.
>>>>>>
>>>>>> The semaphore implementation is used on all platforms where the 
>>>>>> Semaphore class has been implemented in JDK-8087322. So, on some 
>>>>>> OS:es the code will revert to the Monitor-based solution until a 
>>>>>> Semaphore class has been implemented for that OS. So, porters 
>>>>>> might want to consider implementing the Sempahore class.
>>>>>>
>>>>>> There's also a diagnostic vm option 
>>>>>> (-XX:+/-UseSemaphoreGCThreadsSynchronization) to turn off the 
>>>>>> Semaphore-based implementation, which can be used to debug this 
>>>>>> new code. It's mainly targeted towards support and sustaining 
>>>>>> engineering.
>>>>>>
>>>>>>
>>>>>> The patches have been performance tested on Linux, Solaris, OSX, 
>>>>>> and Windows.
>>>>>>
>>>>>> The effects of the patch can be seen by running benchmarks with 
>>>>>> small young gen sizes, which triggers frequent and short GCs.
>>>>>>
>>>>>> For example, here are runs from the SPECjvm2008 xml.transform 
>>>>>> benchmark with:
>>>>>> -Xmx1g -Xms1g -Xmn64m -XX:+PrintGC -XX:+UseG1GC -jar 
>>>>>> SPECjvm2008.jar -ikv xml.transform -it 30 -wt 30
>>>>>>
>>>>>> I got the following GC times:
>>>>>>
>>>>>>             Average    Median    99.9 percentile Max
>>>>>> Baseline: 8.76ms    8.44 ms   25.9 ms 34.7 ms
>>>>>> Monitor:   6.17 ms 5.88 ms   26.0 ms 49.1 ms
>>>>>> Semaphore: 3.43 ms 3.26 ms   13.4 ms           33.4 ms
>>>>>>
>>>>>> If I run an empty GC task 10 times per GC, by running the 
>>>>>> following code:
>>>>>> http://cr.openjdk.java.net/~stefank/8087324/timedTask/
>>>>>>
>>>>>> I get the following numbers to complete the empty GC tasks:
>>>>>>
>>>>>>             Average    Median    99.9 percentile Max
>>>>>> Baseline: 1.43 ms    0.92 ms   3.43 ms           9.30ms
>>>>>> Monitor:    0.75ms 0.72 ms   1.74 ms           2.78ms
>>>>>> Semaphore: 0.07 ms 0.07 ms   0.17 ms 0.26 ms
>>>>>>
>>>>>>
>>>>>>
>>>>>> The code has been tested with JPRT and our nightly testing suites.
>>>>>>
>>>>>> I've created a unit test to run a small test with both the 
>>>>>> semaphore implementation and the monitor implementation:
>>>>>> http://cr.openjdk.java.net/~stefank/8087324/workgangTest/
>>>>>>
>>>>>> But since we currently don't have code to shutdown worker threads 
>>>>>> after they have been started, I don't want to push this test (or 
>>>>>> clean it up) until we have that in place. I created this bug for 
>>>>>> that:
>>>>>> https://bugs.openjdk.java.net/browse/JDK-8087340
>>>>>>
>>>>>> Thanks,
>>>>>> StefanK
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150817/044b8cbe/attachment.htm>

From stefan.johansson at oracle.com  Mon Aug 17 07:48:25 2015
From: stefan.johansson at oracle.com (Stefan Johansson)
Date: Mon, 17 Aug 2015 09:48:25 +0200
Subject: RFR: 8133553: Running with -XX:+UseParallelGC -XX:OldSize=30k
	crashes jvm
In-Reply-To: <55CDF82B.1080102@oracle.com>
References: <55CD992D.6010200@oracle.com> <55CDF830.1010904@oracle.com>
	<55CDF82B.1080102@oracle.com>
Message-ID: <55D191C9.9060903@oracle.com>

Looks good to me too.

Stefan

On 2015-08-14 16:16, David Lindholm wrote:
> Jesper,
>
> Thanks for the review!
>
>
> /David
>
> On 2015-08-14 16:16, Jesper Wilhelmsson wrote:
>> Looks good!
>> /Jesper
>>
>> David Lindholm skrev den 14/8/15 09:30:
>>> Hi,
>>>
>>> Please review this patch that prevents the jvm to crash (or assert) 
>>> when run
>>> with an extremely small value for the -XX:OldSize flag and a 
>>> collector that uses
>>> GenCollectorPolicy.
>>>
>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8133553
>>> Webrev: http://cr.openjdk.java.net/~david/JDK-8133553/webrev.00/
>>>
>>> Testing: Passed JPRT.
>>>
>>>
>>> Thanks,
>>> David
>



From bengt.rutisson at oracle.com  Mon Aug 17 08:50:17 2015
From: bengt.rutisson at oracle.com (Bengt Rutisson)
Date: Mon, 17 Aug 2015 10:50:17 +0200
Subject: RFR (S): 8067339: PLAB reallocation might result in failure to
	allocate object in that recently allocated PLAB
In-Reply-To: <1439540972.2264.12.camel@oracle.com>
References: <1439540972.2264.12.camel@oracle.com>
Message-ID: <55D1A049.1060303@oracle.com>


Hi Thomas,

On 2015-08-14 10:29, Thomas Schatzl wrote:
> Hi all,
>
>    can I have reviews for the following small fix: if G1 reallocates a
> PLAB for an object of size X, it allocates a new PLAB of size Y.
>
> PLABs are required to contain enough space for the object plus some
> alignment reserve; so if X >= Y - alignment-reserve, G1 failed to
> allocate this object it just allocated the PLAB for into that PLAB,
> resulting in an evacuation failure although there is actually enough
> space.
>
> As far as I know this has been a rare occurrence until now because
> current PLAB sizing tended to maximize PLAB sizes, and it is rare to
> have such large objects, but with future changes that actually try to
> size the PLABs so that the expected waste is kept, I noticed a few cases
> of that issue occurring. So this needs to be fixed.
>
> The fix is to consider the required alignment reserve in the PLAB
> allocation request.
>
> This is a day one G1 bug.
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8067339
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8067339/webrev/
> Testing:
> JPRT, vm.gc testlist, lots of testing.

Interesting bug. Nice find!

I agree that you fix solves the problem. I'm just wondering if we could 
make the dependency on the alignment reserve between G1PLABAllocator and 
PLAB more explicit. With your fix, 
G1PLABAllocator::allocate_direct_or_new_plab() has to know exactly what 
PLAB::set_buf() does to fix the sizes correctly. It would be nice if the 
logic could be in two places instead of one.

Not sure exactly how to do that. One way would be to move the alignment 
reserve fixup up to G1PLABAllocator::allocate_direct_or_new_plab(). It 
could add the alignment reserve to the size it passes to 
par_allocate_during_gc(). Then PLAB::set_buf() could just assert that 
the size if the PLAB is at least the size of the alignment reserve. 
Potentially this would look nicer if  PLAB::set_buf() and 
PLAB::set_word_size() were combined into one method that also takes the 
alignment reserve.

I realize that this would affect CMS also, unless the new changes are 
contained in G1PLAB. But looking at 
ParScanThreadState::alloc_in_to_space_slow(), I think that CMS may have 
the same issue, right? So, maybe it is good to fix it for both collectors?

Thanks,
Bengt

>
> Thanks,
>    Thomas
>
>



From erik.helin at oracle.com  Mon Aug 17 09:00:35 2015
From: erik.helin at oracle.com (Erik Helin)
Date: Mon, 17 Aug 2015 11:00:35 +0200
Subject: RFR (L): 8073013: Add detailed information about PLAB memory usage
In-Reply-To: <1439479948.2271.55.camel@oracle.com>
References: <1438876151.2474.51.camel@oracle.com> <55C4F909.7050601@oracle.com>
	<1439211707.2330.11.camel@oracle.com> <55C9F395.8050008@oracle.com>
	<1439369047.2324.1.camel@oracle.com> <55CB4673.6070402@oracle.com>
	<1439479948.2271.55.camel@oracle.com>
Message-ID: <20150817090035.GC31725@ehelin.jrpg.bea.com>

On 2015-08-13, Thomas Schatzl wrote:
> Hi,
> 
> On Wed, 2015-08-12 at 15:13 +0200, Mikael Gerdin wrote:
> > Hi Thomas,
> > 
> > On 2015-08-12 10:44, Thomas Schatzl wrote:
> > > Hi David,
> > >
> > >    thanks for the review:
> > >
> > > On Tue, 2015-08-11 at 15:07 +0200, David Lindholm wrote:
> > >> Hi Thomas.
> > >>
> > >> This looks good, except for the changes in trace.xml. All fields should
> > >> follow Java naming conventions, for example regionEndWaste, not
> > >> region_end_waste. Also, 2 different event types are preferred according
> > >> to the Servicability folks (instead of the "gen" field). Suggested names
> > >> are GCG1EvacuationYoungStatistics and GCG1EvacuationOldStatistics.
> > >>
> > >> The other parts looks good.
> > >
> > > I think all fixed in the new webrevs at:
> > >
> > > http://cr.openjdk.java.net/~tschatzl/8073013/webrev.2 (full)
> > > http://cr.openjdk.java.net/~tschatzl/8073013/webrev.1_to_2 (diff)
> > 
> > I only had a cursory look at the change, overall it seems good and I 
> > know there's a large patch series to manage here.
> > However, in plab.hpp you've added includes of allocation.inline.hpp and 
> > atomic.inline.hpp. In general we should always avoid including inline 
> > headers in normal headers, it's not obvious to me why your other changes 
> > to plab.hpp would require the include change.
> 
>   I moved out these changes into an extra CR, this is a fix for an
> existing issue acually, PLABStats referencing members of Atomic only in
> atomic.inline.hpp.
> 
> Please have a look at the RFR for JDK-8133470.
> 
> Following Erik's suggestion I also moved out the JFR event, see the
> review for JDK-8133530.
> 
> This change includes any adaptations caused by JDK-8133470.
> 
> New webrev:
> http://cr.openjdk.java.net/~tschatzl/8073013/webrev.3 (full)

Looks good, Reviewed.

Thanks,
Erik

> http://cr.openjdk.java.net/~tschatzl/8073013/webrev.2_to_3 (diff)
> 
> Testing:
> jprt
> 
> Thanks,
>   Thomas
> 
> 
> 


From bengt.rutisson at oracle.com  Mon Aug 17 09:10:24 2015
From: bengt.rutisson at oracle.com (Bengt Rutisson)
Date: Mon, 17 Aug 2015 11:10:24 +0200
Subject: RFR (S): 8067339: PLAB reallocation might result in failure to
	allocate object in that recently allocated PLAB
In-Reply-To: <55D1A049.1060303@oracle.com>
References: <1439540972.2264.12.camel@oracle.com> <55D1A049.1060303@oracle.com>
Message-ID: <55D1A500.8000407@oracle.com>



"It would be nice if the logic could be in two places instead of one. "

I claim 'vacation brain' since I just got back from vacation. I of 
course meant that the logic should be in one place. :)

Bengt

On 2015-08-17 10:50, Bengt Rutisson wrote:
>
> Hi Thomas,
>
> On 2015-08-14 10:29, Thomas Schatzl wrote:
>> Hi all,
>>
>>    can I have reviews for the following small fix: if G1 reallocates a
>> PLAB for an object of size X, it allocates a new PLAB of size Y.
>>
>> PLABs are required to contain enough space for the object plus some
>> alignment reserve; so if X >= Y - alignment-reserve, G1 failed to
>> allocate this object it just allocated the PLAB for into that PLAB,
>> resulting in an evacuation failure although there is actually enough
>> space.
>>
>> As far as I know this has been a rare occurrence until now because
>> current PLAB sizing tended to maximize PLAB sizes, and it is rare to
>> have such large objects, but with future changes that actually try to
>> size the PLABs so that the expected waste is kept, I noticed a few cases
>> of that issue occurring. So this needs to be fixed.
>>
>> The fix is to consider the required alignment reserve in the PLAB
>> allocation request.
>>
>> This is a day one G1 bug.
>>
>> CR:
>> https://bugs.openjdk.java.net/browse/JDK-8067339
>> Webrev:
>> http://cr.openjdk.java.net/~tschatzl/8067339/webrev/
>> Testing:
>> JPRT, vm.gc testlist, lots of testing.
>
> Interesting bug. Nice find!
>
> I agree that you fix solves the problem. I'm just wondering if we 
> could make the dependency on the alignment reserve between 
> G1PLABAllocator and PLAB more explicit. With your fix, 
> G1PLABAllocator::allocate_direct_or_new_plab() has to know exactly 
> what PLAB::set_buf() does to fix the sizes correctly. It would be nice 
> if the logic could be in two places instead of one.
>
> Not sure exactly how to do that. One way would be to move the 
> alignment reserve fixup up to 
> G1PLABAllocator::allocate_direct_or_new_plab(). It could add the 
> alignment reserve to the size it passes to par_allocate_during_gc(). 
> Then PLAB::set_buf() could just assert that the size if the PLAB is at 
> least the size of the alignment reserve. Potentially this would look 
> nicer if  PLAB::set_buf() and PLAB::set_word_size() were combined into 
> one method that also takes the alignment reserve.
>
> I realize that this would affect CMS also, unless the new changes are 
> contained in G1PLAB. But looking at 
> ParScanThreadState::alloc_in_to_space_slow(), I think that CMS may 
> have the same issue, right? So, maybe it is good to fix it for both 
> collectors?
>
> Thanks,
> Bengt
>
>>
>> Thanks,
>>    Thomas
>>
>>
>



From david.lindholm at oracle.com  Mon Aug 17 09:20:14 2015
From: david.lindholm at oracle.com (David Lindholm)
Date: Mon, 17 Aug 2015 11:20:14 +0200
Subject: RFR: 8133553: Running with -XX:+UseParallelGC -XX:OldSize=30k
	crashes jvm
In-Reply-To: <55D191C9.9060903@oracle.com>
References: <55CD992D.6010200@oracle.com>
	<55CDF830.1010904@oracle.com>	<55CDF82B.1080102@oracle.com>
	<55D191C9.9060903@oracle.com>
Message-ID: <55D1A74E.1000608@oracle.com>

Stefan,

Thanks for the review!

/David

On 2015-08-17 09:48, Stefan Johansson wrote:
> Looks good to me too.
>
> Stefan
>
> On 2015-08-14 16:16, David Lindholm wrote:
>> Jesper,
>>
>> Thanks for the review!
>>
>>
>> /David
>>
>> On 2015-08-14 16:16, Jesper Wilhelmsson wrote:
>>> Looks good!
>>> /Jesper
>>>
>>> David Lindholm skrev den 14/8/15 09:30:
>>>> Hi,
>>>>
>>>> Please review this patch that prevents the jvm to crash (or assert) 
>>>> when run
>>>> with an extremely small value for the -XX:OldSize flag and a 
>>>> collector that uses
>>>> GenCollectorPolicy.
>>>>
>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8133553
>>>> Webrev: http://cr.openjdk.java.net/~david/JDK-8133553/webrev.00/
>>>>
>>>> Testing: Passed JPRT.
>>>>
>>>>
>>>> Thanks,
>>>> David
>>
>



From thomas.schatzl at oracle.com  Mon Aug 17 10:21:03 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 17 Aug 2015 12:21:03 +0200
Subject: RFR (S): 8133470: Uses of Atomic methods in plab.hpp should be
	moved to .inline.hpp file
In-Reply-To: <55CDBA97.3050909@oracle.com>
References: <1439462979.2271.11.camel@oracle.com>
	<1439479926.2271.54.camel@oracle.com>
	<20150814093412.GA31725@ehelin.jrpg.bea.com>
	<55CDBA97.3050909@oracle.com>
Message-ID: <1439806863.2351.0.camel@oracle.com>

Hi Erik, Mikael,

  thanks for your reviews.

Thanks,
  Thomas




From thomas.schatzl at oracle.com  Mon Aug 17 10:21:48 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 17 Aug 2015 12:21:48 +0200
Subject: RFR (L): 8073013: Add detailed information about PLAB memory usage
In-Reply-To: <20150817090035.GC31725@ehelin.jrpg.bea.com>
References: <1438876151.2474.51.camel@oracle.com>
	<55C4F909.7050601@oracle.com> <1439211707.2330.11.camel@oracle.com>
	<55C9F395.8050008@oracle.com> <1439369047.2324.1.camel@oracle.com>
	<55CB4673.6070402@oracle.com> <1439479948.2271.55.camel@oracle.com>
	<20150817090035.GC31725@ehelin.jrpg.bea.com>
Message-ID: <1439806908.2351.1.camel@oracle.com>

Hi Erik,

On Mon, 2015-08-17 at 11:00 +0200, Erik Helin wrote:
> On 2015-08-13, Thomas Schatzl wrote:
> > Hi,

[...]
> > Please have a look at the RFR for JDK-8133470.
> > 
> > Following Erik's suggestion I also moved out the JFR event, see the
> > review for JDK-8133530.
> > 
> > This change includes any adaptations caused by JDK-8133470.
> > 
> > New webrev:
> > http://cr.openjdk.java.net/~tschatzl/8073013/webrev.3 (full)
> 
> Looks good, Reviewed.

  thanks for the review.

Thomas




From thomas.schatzl at oracle.com  Mon Aug 17 10:22:18 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 17 Aug 2015 12:22:18 +0200
Subject: RFR (L): 8073013: Add detailed information about PLAB memory usage
In-Reply-To: <55CDD6A0.60405@oracle.com>
References: <1438876151.2474.51.camel@oracle.com>
	<55C4F909.7050601@oracle.com> <1439211707.2330.11.camel@oracle.com>
	<55C9F395.8050008@oracle.com> <1439369047.2324.1.camel@oracle.com>
	<55CB4673.6070402@oracle.com> <1439479948.2271.55.camel@oracle.com>
	<55CDD6A0.60405@oracle.com>
Message-ID: <1439806938.2351.2.camel@oracle.com>

Hi Mikael,

On Fri, 2015-08-14 at 13:53 +0200, Mikael Gerdin wrote:
> Hi Thomas,
> 
[...]
> > Please have a look at the RFR for JDK-8133470.
> >
> > Following Erik's suggestion I also moved out the JFR event, see the
> > review for JDK-8133530.
> >
> > This change includes any adaptations caused by JDK-8133470.
> >
> > New webrev:
> > http://cr.openjdk.java.net/~tschatzl/8073013/webrev.3 (full)
> > http://cr.openjdk.java.net/~tschatzl/8073013/webrev.2_to_3 (diff)
> 
> I think this is ready to go now, Reviewed.

  thanks for your review.

Thanks,
  Thomas




From thomas.schatzl at oracle.com  Mon Aug 17 13:30:18 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 17 Aug 2015 15:30:18 +0200
Subject: RFR (S): 8133530: Add JFR event for evacuation statistics
In-Reply-To: <55CDD14C.2050600@oracle.com>
References: <1439480605.2271.65.camel@oracle.com> <55CDD14C.2050600@oracle.com>
Message-ID: <1439818218.2351.12.camel@oracle.com>

Hi Mikael,

On Fri, 2015-08-14 at 13:30 +0200, Mikael Gerdin wrote:
> Hi Thomas,
> 
> On 2015-08-13 17:43, Thomas Schatzl wrote:
> > Hi all,
> >
> >    can I have reviews for this split-out from 8073013 that adds JFR
> > support for evacuation statistics.
> >
> > The code has been changed to (hopefully) correspond better to the way
> > code that sends JFR events should look like.
> >
> > CR:
> > https://bugs.openjdk.java.net/browse/JDK-8133530
> > Webrev:
> > http://cr.openjdk.java.net/~tschatzl/8133530/webrev/
> 
> in gcTrace.hpp
> 39 #include "gc/g1/g1YCTypes.hpp"
> 40 #include "gcHeapSummary.hpp"
> 41 #endif
> 42
> 
> you should use the full path to gcHeapSummary.hpp
> 
> in gcTraceSend.cpp
> 240   if (surv_evt.should_commit()) {
> 241     surv_evt.set_gcId(GCId::peek().id());
> 242     surv_evt.set_allocated(summary.allocated() * HeapWordSize);
> 
> The other events use
> 232     e.set_gcId(_shared_gc_info.gc_id().id());
> 
> When you assert_set_gc_id you actually verify that the shared gc info id 
> is set.

  all fixed in 
http://cr.openjdk.java.net/~tschatzl/8133530/webrev.0_to_1 (diff)
http://cr.openjdk.java.net/~tschatzl/8133530/webrev.1 (full)

I also incorporated a suggestion by Erik that extracted the common
members of the two new events into a single (JFR) struct to benefit a
little from refactoring.

Thanks,
  Thomas




From thomas.schatzl at oracle.com  Mon Aug 17 15:09:22 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 17 Aug 2015 17:09:22 +0200
Subject: RFR (S): 8067339: PLAB reallocation might result in failure to
	allocate object in that recently allocated PLAB
In-Reply-To: <55D1A049.1060303@oracle.com>
References: <1439540972.2264.12.camel@oracle.com> <55D1A049.1060303@oracle.com>
Message-ID: <1439824162.2351.28.camel@oracle.com>

Hi Bengt,

On Mon, 2015-08-17 at 10:50 +0200, Bengt Rutisson wrote:
> Hi Thomas,
> 
> On 2015-08-14 10:29, Thomas Schatzl wrote:
> > Hi all,
> >
> >    can I have reviews for the following small fix: if G1 reallocates a
> > PLAB for an object of size X, it allocates a new PLAB of size Y.
> >
> > PLABs are required to contain enough space for the object plus some
> > alignment reserve; so if X >= Y - alignment-reserve, G1 failed to
> > allocate this object it just allocated the PLAB for into that PLAB,
> > resulting in an evacuation failure although there is actually enough
> > space.
> >
> > As far as I know this has been a rare occurrence until now because
> > current PLAB sizing tended to maximize PLAB sizes, and it is rare to
> > have such large objects, but with future changes that actually try to
> > size the PLABs so that the expected waste is kept, I noticed a few cases
> > of that issue occurring. So this needs to be fixed.
> >
> > The fix is to consider the required alignment reserve in the PLAB
> > allocation request.
> >
> > This is a day one G1 bug.
> >
> > CR:
> > https://bugs.openjdk.java.net/browse/JDK-8067339
> > Webrev:
> > http://cr.openjdk.java.net/~tschatzl/8067339/webrev/
> > Testing:
> > JPRT, vm.gc testlist, lots of testing.
> 
> Interesting bug. Nice find!
> 
> I agree that you fix solves the problem. I'm just wondering if we could 
> make the dependency on the alignment reserve between G1PLABAllocator and 
> PLAB more explicit. With your fix, 
> G1PLABAllocator::allocate_direct_or_new_plab() has to know exactly what 
> PLAB::set_buf() does to fix the sizes correctly. It would be nice if the 
> logic could be in two places instead of one.

We could ask the PLAB for a given allocation, how much space you need.
I.e. basically move line 236 in g1Allocator.cpp to the PLAB class.

New changes at
http://cr.openjdk.java.net/~tschatzl/8067339/webrev.0_to_1 (diff)
http://cr.openjdk.java.net/~tschatzl/8067339/webrev.1 (full)

> Not sure exactly how to do that. One way would be to move the alignment 
> reserve fixup up to G1PLABAllocator::allocate_direct_or_new_plab(). It 
> could add the alignment reserve to the size it passes to 
> par_allocate_during_gc(). Then PLAB::set_buf() could just assert that 
> the size if the PLAB is at least the size of the alignment reserve. 
> Potentially this would look nicer if  PLAB::set_buf() and 
> PLAB::set_word_size() were combined into one method that also takes the 
> alignment reserve.
> 
> I realize that this would affect CMS also, unless the new changes are 
> contained in G1PLAB. But looking at 
> ParScanThreadState::alloc_in_to_space_slow(), I think that CMS may have 
> the same issue, right? So, maybe it is good to fix it for both collectors?

CMS has the same issue, but it simply promotes the object, having a more
loose assert in that place. That would be an option in G1 too.

Parallel GC does not get into this issue because it simply inline
allocates any object less than half of the PLAB size. Parallel just
assumes that (PLAB-size / 2) > PLAB::alignment_reserve; besides, it has
a completely different PLAB implementation.

To me it seems what Parallel GC does is the best way of action. The only
problem is that in G1 this would almost certainly increase waste at the
end of regions significantly.
There will be some change in the future that fixes this, so for now I
would like to postpone this problem.

I created JDK-8133724 to look at this in more detail if you do not mind.

Thanks,
  Thomas




From bengt.rutisson at oracle.com  Mon Aug 17 15:22:31 2015
From: bengt.rutisson at oracle.com (Bengt Rutisson)
Date: Mon, 17 Aug 2015 17:22:31 +0200
Subject: RFR (S): 8067339: PLAB reallocation might result in failure to
	allocate object in that recently allocated PLAB
In-Reply-To: <1439824162.2351.28.camel@oracle.com>
References: <1439540972.2264.12.camel@oracle.com>	
	<55D1A049.1060303@oracle.com> <1439824162.2351.28.camel@oracle.com>
Message-ID: <55D1FC37.2000501@oracle.com>


Hi Thomas,

On 17/08/15 17:09, Thomas Schatzl wrote:
> Hi Bengt,
>
> On Mon, 2015-08-17 at 10:50 +0200, Bengt Rutisson wrote:
>> Hi Thomas,
>>
>> On 2015-08-14 10:29, Thomas Schatzl wrote:
>>> Hi all,
>>>
>>>     can I have reviews for the following small fix: if G1 reallocates a
>>> PLAB for an object of size X, it allocates a new PLAB of size Y.
>>>
>>> PLABs are required to contain enough space for the object plus some
>>> alignment reserve; so if X >= Y - alignment-reserve, G1 failed to
>>> allocate this object it just allocated the PLAB for into that PLAB,
>>> resulting in an evacuation failure although there is actually enough
>>> space.
>>>
>>> As far as I know this has been a rare occurrence until now because
>>> current PLAB sizing tended to maximize PLAB sizes, and it is rare to
>>> have such large objects, but with future changes that actually try to
>>> size the PLABs so that the expected waste is kept, I noticed a few cases
>>> of that issue occurring. So this needs to be fixed.
>>>
>>> The fix is to consider the required alignment reserve in the PLAB
>>> allocation request.
>>>
>>> This is a day one G1 bug.
>>>
>>> CR:
>>> https://bugs.openjdk.java.net/browse/JDK-8067339
>>> Webrev:
>>> http://cr.openjdk.java.net/~tschatzl/8067339/webrev/
>>> Testing:
>>> JPRT, vm.gc testlist, lots of testing.
>> Interesting bug. Nice find!
>>
>> I agree that you fix solves the problem. I'm just wondering if we could
>> make the dependency on the alignment reserve between G1PLABAllocator and
>> PLAB more explicit. With your fix,
>> G1PLABAllocator::allocate_direct_or_new_plab() has to know exactly what
>> PLAB::set_buf() does to fix the sizes correctly. It would be nice if the
>> logic could be in two places instead of one.
> We could ask the PLAB for a given allocation, how much space you need.
> I.e. basically move line 236 in g1Allocator.cpp to the PLAB class.
>
> New changes at
> http://cr.openjdk.java.net/~tschatzl/8067339/webrev.0_to_1 (diff)
> http://cr.openjdk.java.net/~tschatzl/8067339/webrev.1 (full)

I like this better. Thanks for fixing it!

>
>> Not sure exactly how to do that. One way would be to move the alignment
>> reserve fixup up to G1PLABAllocator::allocate_direct_or_new_plab(). It
>> could add the alignment reserve to the size it passes to
>> par_allocate_during_gc(). Then PLAB::set_buf() could just assert that
>> the size if the PLAB is at least the size of the alignment reserve.
>> Potentially this would look nicer if  PLAB::set_buf() and
>> PLAB::set_word_size() were combined into one method that also takes the
>> alignment reserve.
>>
>> I realize that this would affect CMS also, unless the new changes are
>> contained in G1PLAB. But looking at
>> ParScanThreadState::alloc_in_to_space_slow(), I think that CMS may have
>> the same issue, right? So, maybe it is good to fix it for both collectors?
> CMS has the same issue, but it simply promotes the object, having a more
> loose assert in that place. That would be an option in G1 too.
>
> Parallel GC does not get into this issue because it simply inline
> allocates any object less than half of the PLAB size. Parallel just
> assumes that (PLAB-size / 2) > PLAB::alignment_reserve; besides, it has
> a completely different PLAB implementation.
>
> To me it seems what Parallel GC does is the best way of action. The only
> problem is that in G1 this would almost certainly increase waste at the
> end of regions significantly.
> There will be some change in the future that fixes this, so for now I
> would like to postpone this problem.
>
> I created JDK-8133724 to look at this in more detail if you do not mind.

Yes, that sounds like a good approach.

Ship it!
Bengt

>
> Thanks,
>    Thomas
>
>



From dmitry.fazunenko at oracle.com  Mon Aug 17 17:35:26 2015
From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko)
Date: Mon, 17 Aug 2015 20:35:26 +0300
Subject: RFR: 8081317: [NEWTEST] documented GC ratio tuning and new size
	options should be covered by regression tests
In-Reply-To: <55C0CD95.1080305@oracle.com>
References: <55BBB0BE.6080506@oracle.com> <55C0CD95.1080305@oracle.com>
Message-ID: <55D21B5E.7020100@oracle.com>

Hi Michail,

the fix looks good to me.

Thanks,
Dima

On 04.08.2015 17:35, Michail Chernov wrote:
> Hi,
>
> Could someone please take a look at this review?
>
> Thanks,
> Michail
>
> On 31.07.2015 20:30, Michail Chernov wrote:
>> Hi,
>>
>> Please review these changes for JDK-8081317. There are new tests to 
>> cover GC ratio tuning and new size options.
>>
>> Webrev:  http://cr.openjdk.java.net/~mchernov/8081317/
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8081317
>>
>> TestMaxMinHeapFreeRatioFlags.java - Verify that heap size changes 
>> according to max and min heap free ratios.
>>
>> TestMinAndInitialSurvivorRatioFlags.java - Test verifies that VM can 
>> start with any GC when MinSurvivorRatio and InitialSurvivorRatio 
>> flags passed and for Parallel GC it verifies that after start up 
>> survivor ratio is equal to InitialSurvivorRatio value and that actual 
>> survivor ratio will never be less than MinSurvivorRatio.
>>
>> TestNewRatioFlag.java - Verify that heap divided among generations 
>> according to NewRatio.
>>
>> TestNewSizeFlags.java - Verify that young gen size conforms values 
>> specified by NewSize, MaxNewSize and Xmn options.
>>
>> TestSurvivorRatioFlag - Verify that actual survivor ratio is equal to 
>> specified SurvivorRatio value.
>>
>> TestTargetSurvivorRatioFlag.java - Test if objects size is less than 
>> (survivor_size * TargetSurvivorRatio / 100) then objects are stayed 
>> in survivor space until MaxTenuringThreshold minor GC cycles. If more 
>> than (survivor_size * TargetSurvivorRatio / 100) objects were 
>> allocated, then test verifies that after MaxTenuringThreshold minor 
>> GC cycles survivor space is almost empty.
>>
>> New changes were tested on different platforms including embedded jdk 
>> on Linux x86.
>>
>> Thanks,
>> Michail
>>
>



From jesper.wilhelmsson at oracle.com  Mon Aug 17 19:37:54 2015
From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson)
Date: Mon, 17 Aug 2015 21:37:54 +0200
Subject: RFR: 8130308 - Too low memory usage in
	TestPromotionFromSurvivorToTenuredAfterMinorGC.java
Message-ID: <55D23812.1030107@oracle.com>

Hi,

Please review this fix for TestPromotionFromSurvivorToTenuredAfterMinorGC.java

Basically what happened was:

1. The test stores the amount of used heap
2. Some allocation is done and a series of GCs are called
3. The test keeps track of anything allocated during the test by other threads
4. Verify that the total amount of used old gen is what we expect based on how 
much was allocated.

The problem was that if data was allocated and was still in the survivor space 
it was included in the expected size, but not included in the verification.

Since the test is all about tenured space and verifying that, I decided to 
reduce the expected amount by what's used in the survivor area rather than 
including the survivor area in the verification.

Bug: https://bugs.openjdk.java.net/browse/JDK-8130308
Webrev: http://cr.openjdk.java.net/~jwilhelm/8130308/webrev.00/

Thanks,
/Jesper


From dmitry.fazunenko at oracle.com  Mon Aug 17 19:57:40 2015
From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko)
Date: Mon, 17 Aug 2015 22:57:40 +0300
Subject: RFR: 8130308 - Too low memory usage in
	TestPromotionFromSurvivorToTenuredAfterMinorGC.java
In-Reply-To: <55D23812.1030107@oracle.com>
References: <55D23812.1030107@oracle.com>
Message-ID: <55D23CB4.305@oracle.com>

Hi Jesper,

The fix looks good to me.

Thanks,
Dima

On 17.08.2015 22:37, Jesper Wilhelmsson wrote:
> Hi,
>
> Please review this fix for 
> TestPromotionFromSurvivorToTenuredAfterMinorGC.java
>
> Basically what happened was:
>
> 1. The test stores the amount of used heap
> 2. Some allocation is done and a series of GCs are called
> 3. The test keeps track of anything allocated during the test by other 
> threads
> 4. Verify that the total amount of used old gen is what we expect 
> based on how much was allocated.
>
> The problem was that if data was allocated and was still in the 
> survivor space it was included in the expected size, but not included 
> in the verification.
>
> Since the test is all about tenured space and verifying that, I 
> decided to reduce the expected amount by what's used in the survivor 
> area rather than including the survivor area in the verification.
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8130308
> Webrev: http://cr.openjdk.java.net/~jwilhelm/8130308/webrev.00/
>
> Thanks,
> /Jesper



From jesper.wilhelmsson at oracle.com  Mon Aug 17 21:04:21 2015
From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson)
Date: Mon, 17 Aug 2015 23:04:21 +0200
Subject: RFR: 8130308 - Too low memory usage in
	TestPromotionFromSurvivorToTenuredAfterMinorGC.java
In-Reply-To: <55D23CB4.305@oracle.com>
References: <55D23812.1030107@oracle.com> <55D23CB4.305@oracle.com>
Message-ID: <55D24C55.8080606@oracle.com>

Thanks!
/Jesper

Dmitry Fazunenko skrev den 17/8/15 21:57:
> Hi Jesper,
>
> The fix looks good to me.
>
> Thanks,
> Dima
>
> On 17.08.2015 22:37, Jesper Wilhelmsson wrote:
>> Hi,
>>
>> Please review this fix for TestPromotionFromSurvivorToTenuredAfterMinorGC.java
>>
>> Basically what happened was:
>>
>> 1. The test stores the amount of used heap
>> 2. Some allocation is done and a series of GCs are called
>> 3. The test keeps track of anything allocated during the test by other threads
>> 4. Verify that the total amount of used old gen is what we expect based on how
>> much was allocated.
>>
>> The problem was that if data was allocated and was still in the survivor space
>> it was included in the expected size, but not included in the verification.
>>
>> Since the test is all about tenured space and verifying that, I decided to
>> reduce the expected amount by what's used in the survivor area rather than
>> including the survivor area in the verification.
>>
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8130308
>> Webrev: http://cr.openjdk.java.net/~jwilhelm/8130308/webrev.00/
>>
>> Thanks,
>> /Jesper
>


From eric.caspole at oracle.com  Tue Aug 18 01:29:21 2015
From: eric.caspole at oracle.com (Eric Caspole)
Date: Mon, 17 Aug 2015 21:29:21 -0400
Subject: RFR: JDK-8133349 - CMS: Assert failed: Ctl pt invariant
Message-ID: <55D28A71.30905@oracle.com>

Hi everybody,
Could I get reviews for this bug 8133349.
This is assert is happening because the _survivor_chunk_array size is 
based on a calculation from ThreadLocalAllocBuffer::min_size(), but 
during a GC the PLAB::min_size() is used as the lower limit.

These are different because the TLAB alignment_reserve() == 40 words, 
but the PLAB AlignmentReserve == 2 words which is from 
oopDesc::header_size().

So that means the _survivor_chunk_array is a little bit smaller than 
will fit all the PLABs that could be allocated during a GC.

Before the fix for JDK-8078904 
<https://bugs.openjdk.java.net/browse/JDK-8078904>, the 
_survivor_chunk_array was padded by 2x, not sure why, so this situation 
never caused a problem. With the fix for JDK-8078904 
<https://bugs.openjdk.java.net/browse/JDK-8078904>, the 
_survivor_chunk_array is exactly sized to hold the PLABs.

Bug:
https://bugs.openjdk.java.net/browse/JDK-8133349
Webrev:
http://cr.openjdk.java.net/~ecaspole/JDK-8133349/01/webrev/

Passes JPRT and many  counts of PGC threads in the failing test, which 
only was exposed in a certain Solaris situation.
Thanks,
Eric

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150817/3a3d41cb/attachment.htm>

From thomas.schatzl at oracle.com  Tue Aug 18 08:29:34 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Tue, 18 Aug 2015 10:29:34 +0200
Subject: RFR: JDK-8133349 - CMS: Assert failed: Ctl pt invariant
In-Reply-To: <55D28A71.30905@oracle.com>
References: <55D28A71.30905@oracle.com>
Message-ID: <1439886574.2316.1.camel@oracle.com>

Hi Eric,

On Mon, 2015-08-17 at 21:29 -0400, Eric Caspole wrote:
> Hi everybody,
> Could I get reviews for this bug 8133349. 
> This is assert is happening because the _survivor_chunk_array size is
> based on a calculation from ThreadLocalAllocBuffer::min_size(), but
> during a GC the PLAB::min_size() is used as the lower limit.
> 
> These are different because the TLAB alignment_reserve() == 40 words,
> but the PLAB AlignmentReserve == 2 words which is from
> oopDesc::header_size().

Any idea why the TLAB alignment reserve is 40 words? (Just curious).
> 
> So that means the _survivor_chunk_array is a little bit smaller than
> will fit all the PLABs that could be allocated during a GC. 
> 
> Before the fix for JDK-8078904, the _survivor_chunk_array was padded
> by 2x, not sure why, so this situation never caused a problem. With
> the fix for JDK-8078904, the _survivor_chunk_array is exactly sized to
> hold the PLABs.
> 
> Bug:
> https://bugs.openjdk.java.net/browse/JDK-8133349
> Webrev:
> http://cr.openjdk.java.net/~ecaspole/JDK-8133349/01/webrev/
> 
> Passes JPRT and many  counts of PGC threads in the failing test, which
> only was exposed in a certain Solaris situation.

  thanks, looks good. Is it possible to recreate that Solaris test using
a simple JPRT test? Or is this a 100% failure on the existing test?

Thanks,
  Thomas




From thomas.schatzl at oracle.com  Tue Aug 18 09:42:30 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Tue, 18 Aug 2015 11:42:30 +0200
Subject: RFR (S): 8133530: Add JFR event for evacuation statistics
In-Reply-To: <1439818218.2351.12.camel@oracle.com>
References: <1439480605.2271.65.camel@oracle.com>
	<55CDD14C.2050600@oracle.com> <1439818218.2351.12.camel@oracle.com>
Message-ID: <1439890950.2316.4.camel@oracle.com>

Hi all,

  Erik had some further comments in private regarding style and better
encapsulation of the code.

Here are the new webrevs:
http://cr.openjdk.java.net/~tschatzl/8133530/webrev.2 (full)
http://cr.openjdk.java.net/~tschatzl/8133530/webrev.1_to_2 (full)

Main changes are about moving the G1NewTracer::create_g1_evacstats()
methods into the cpp file as a static method.

Thanks,
  Thomas

On Mon, 2015-08-17 at 15:30 +0200, Thomas Schatzl wrote:
> Hi Mikael,
> 
> On Fri, 2015-08-14 at 13:30 +0200, Mikael Gerdin wrote:
> > Hi Thomas,
> > 
> > On 2015-08-13 17:43, Thomas Schatzl wrote:
> > > Hi all,
> > >
> > >    can I have reviews for this split-out from 8073013 that adds JFR
> > > support for evacuation statistics.
> > >
> > > The code has been changed to (hopefully) correspond better to the way
> > > code that sends JFR events should look like.
> > >
> > > CR:
> > > https://bugs.openjdk.java.net/browse/JDK-8133530
> > > Webrev:
> > > http://cr.openjdk.java.net/~tschatzl/8133530/webrev/
> > 
> > in gcTrace.hpp
> > 39 #include "gc/g1/g1YCTypes.hpp"
> > 40 #include "gcHeapSummary.hpp"
> > 41 #endif
> > 42
> > 
> > you should use the full path to gcHeapSummary.hpp
> > 
> > in gcTraceSend.cpp
> > 240   if (surv_evt.should_commit()) {
> > 241     surv_evt.set_gcId(GCId::peek().id());
> > 242     surv_evt.set_allocated(summary.allocated() * HeapWordSize);
> > 
> > The other events use
> > 232     e.set_gcId(_shared_gc_info.gc_id().id());
> > 
> > When you assert_set_gc_id you actually verify that the shared gc info id 
> > is set.
> 
>   all fixed in 
> http://cr.openjdk.java.net/~tschatzl/8133530/webrev.0_to_1 (diff)
> http://cr.openjdk.java.net/~tschatzl/8133530/webrev.1 (full)
> 
> I also incorporated a suggestion by Erik that extracted the common
> members of the two new events into a single (JFR) struct to benefit a
> little from refactoring.
> 
> Thanks,
>   Thomas
> 
> 




From thomas.schatzl at oracle.com  Tue Aug 18 10:09:13 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Tue, 18 Aug 2015 12:09:13 +0200
Subject: RFR (S): 8133530: Add JFR event for evacuation statistics
In-Reply-To: <55CE1A21.9030506@oracle.com>
References: <1439480605.2271.65.camel@oracle.com>
	<55CDD14C.2050600@oracle.com> <55CE1A21.9030506@oracle.com>
Message-ID: <1439892553.2316.6.camel@oracle.com>

Hi,

On Fri, 2015-08-14 at 09:41 -0700, Staffan Friberg wrote:
> Hi Thomas,
> 
> Any reason to go via the G1EvacSummary class and not just send the stats 
> (G1EvacStats) straight through to the event code?
> 
> It seems like it just adds a very short live object and a lot of copies 
> of different fields, but I might miss some bigger picture thing.

  other similar code that wants to send more than a few variables in a
given event also introduces this kind of summary classes. The intention
is to decouple the gc_tracer a little from the collector
implementations.

I am not sure if it is the best way to do this, but it would somewhat
break the "style" of the other code. So for now I would like to keep the
code as is; I talked with others offline without real conclusion except
that it is probably best to implement the event dispatching uniformly at
this time.

The latest revision of the change does away a little with some more code
duplication too.

Thanks,
  Thomas





From per.liden at oracle.com  Tue Aug 18 11:11:59 2015
From: per.liden at oracle.com (Per Liden)
Date: Tue, 18 Aug 2015 13:11:59 +0200
Subject: RFR: 8129417: Oop iteration clean-up to remove
	oop_ms_follow_contents
In-Reply-To: <559BCD11.3010801@oracle.com>
References: <5588217F.9090201@oracle.com>
	<20150706151537.GP3266@ehelin.jrpg.bea.com>
	<559BCD11.3010801@oracle.com>
Message-ID: <55D312FF.3070105@oracle.com>

Hi Stefan,

On 2015-07-07 14:58, Stefan Johansson wrote:
> Thanks Erik for reviewing,
>
> New webrevs:
> Part 2 - full:
> http://cr.openjdk.java.net/~sjohanss/8129417/2-add-oop_iterate_size/hotspot.01/
>
> Part 2 - inc:
> http://cr.openjdk.java.net/~sjohanss/8129417/2-add-oop_iterate_size/hotspot.00-01/
>
>
> Part 3 - full:
> http://cr.openjdk.java.net/~sjohanss/8129417/3-re-structure-mark-sweep/hotspot.02/
>
> Part 3 - inc:
> http://cr.openjdk.java.net/~sjohanss/8129417/3-re-structure-mark-sweep/hotspot.00-02/
>
> Also includes an updated assert, which was missed in previous webrev.
>
> Comments inline.

Just one thing that looks odd to me, but maybe you can clarify. 
Shouldn't MarkSweep::follow_object() call follow_array() instead of 
calling push_objarray()? That way the change made to 
ObjArrayTask::is_valid() wouldn't be needed.

/Per

>
> On 2015-07-06 17:15, Erik Helin wrote:
>> Hi Stefan,
>>
>> thanks a lot for taking on this work. Please see comments inline.
>>
>> On 2015-06-22, Stefan Johansson wrote:
>>> Hi,
>>>
>>> Please review these changes for RFE:
>>> https://bugs.openjdk.java.net/browse/JDK-8129417
>>>
>>> Webrev for the full change:
>>> http://cr.openjdk.java.net/~sjohanss/8129417/hotspot.00/
>>>
>>> Summary:
>>> To allow further cleanups and later remove G1 specific code from
>>> mark-sweep,
>>> we want to remove the mark-sweep specific visitor
>>> oop_ms_follow_contents.
>>> Doing this cleanup has proven to be a little more complicated than I
>>> first
>>> anticipated and to make the change easier to review I've split it
>>> into four
>>> different parts.
>>>
>>> Part 1 - removing oop_ms_follow_contents:
>>> http://cr.openjdk.java.net/~sjohanss/8129417/1-remove-oop_ms_follow_contents/hotspot.00/
>>>
>> - Looking in specialized_oop_closures.hpp, the comment
>>     71 // This is split into several because of a Visual C++ 6.0
>> compiler bug
>>     72 // where very long macros cause the compiler to crash
>>    seems a bit dated, given that Visual C++ 6.0 was released in 1998. I
>>    think we should try to merge ALL_OOP_OOP_ITERATE_CLOSURES_1 and 2 (and
>>    their corresponding macros). Do you think like doing that in this
>>    patch or do you want to file a follow-up bug?
> I would like to do/investigate this as a separate RFE, I have a list of
> things that should follow this clean up and I plan to file those before
> pushing this change.
>
>>
>> - The part of the patch that changes instanceMirrorKlass.inline.hpp
>>    might impact more GCs than just any of the marksweep ones. Have you
>>    seen any performance improvements/regressions with G1 or CMS?
> No, but I have not done as many runs and done as much analysis of this
> as I've done on the serial full gc. I've not seen any regression in
> aurora.se performance runs.
>>> Part 2 - introducing oop_iterate_size:
>>> http://cr.openjdk.java.net/~sjohanss/8129417/2-add-oop_iterate_size/hotspot.00/
>>>
>> - In arrayKlass.hpp, why add a space to:
>>    +#define OOP_OOP_ITERATE_DECL_NO_BACKWARDS(OopClosureType,
>> nv_suffix)         \
> I've followed '\' alignment "rule" for these macros, which seems to be
> two spaces after the longest line. I just realized that I should also
> update OOP_OOP_ITERATE_DECL_RANGE with one extra space to be consistent.
>
>> - In arrayKlass.hpp, the following lines seems aligned strangely:
>>    +void KlassType::oop_oop_iterate_range##nv_suffix(oop obj,
>> OopClosureType* closure, int start, int end) {  \
>>    +  oop_oop_iterate_range<nvs_to_bool(nv_suffix)>(obj, closure,
>> start, end);                               \
> Nice catch, missed fixing the alignment when making it void.
>
>> - In oop.inline.hpp, why must the size be computed before applying the
>>    closure? In the world of perm gen, this might have been important in
>>    order to not follow stale klass pointers, but with Metaspace the
>> Klass*
>>    won't move.
> As we've discussed offline, this is most likely safe. I've done an
> ad-hoc RBT run with some random testing when I assert on the size being
> the same both before and after the call to oop_oop_iterate.
>
> I would like to do this change as a separate change as well, so it's
> been added to my list of RFEs to file.
>
>>
>> - Can we remove the method MutableSpace::oop_iterate?
> Looks like it, no one seems to be using it. I'll remove it instead of
> updating it to use oop_iterate_size().
>
>>
>>> Part 3 - restructure mark-sweep:
>>> http://cr.openjdk.java.net/~sjohanss/8129417/3-re-structure-mark-sweep/hotspot.00/
>>>
>> - Can you please remove the commented assert in mark_and_push:
>>     +//  assert(Universe::heap()->is_in_reserved(p), "should be in
>> object space");
>>    or should it be enabled again?
> Removed.
>
>> - I agree with Per about moving the check for object arrays into
>>    follow_object.
> Already done, but gives a slight regression. I still think we can live
> with this small regression for now, to avoid people doing wrong and call
> follow_object on objArrays.
>
>>> Part 4 - compiler hints for inlining:
>>> http://cr.openjdk.java.net/~sjohanss/8129417/4-compiler-hints-for-inlining/hotspot.00/
>>>
>> - I would like to see the definiton of FORCE_INLINE in the compiler
>>    specific globalDefinitions file. Then the code in
>>    instanceKlass.inline.hpp can look like:
>>     #if defined(TARGET_COMPILER_visCPP) ||
>> defined(TARGET_COMPILER_sparcWorks)
>>       #define INLINE FORCE_INLINE
>>     #else
>>       #define INLINE inline
>>     #endif
>>
>> - The same comment for stack.inline.hpp, but with NO_INLINE:
>>    #if defined(TARGET_COMPILER_sparcWorks)
>>      #define STACK_NO_INLINE NO_INLINE
>>    #else
>>      #define STACK_NO_INLINE
>>    #endif
> I agree, we should come up with a better and cleaner way to do this. I
> will file a follow up RFE.
>
>> - I agree with Per about adding an #undef in stack.inline.hpp
> Already fixed.
>
> Thanks again Erik,
> Stefan
>
>> Thanks!
>> Erik
>>
>>> Testing:
>>> * Adhoc RBT run for functionality - no new failures.
>>> * Adhoc aurora.se run for performance - no obvious regression.
>>> * Many local runs tuned to do Full GCs a lot to verify that there is no
>>> obvious regression.
>>>
>>> Thanks,
>>> Stefan
>


From thomas.schatzl at oracle.com  Tue Aug 18 11:18:40 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Tue, 18 Aug 2015 13:18:40 +0200
Subject: RFR (M): 8067336: Allow that PLAB allocations at the end of regions
	are flexible
Message-ID: <1439896720.2316.7.camel@oracle.com>

Hi all,

  can I have reviews for this change that greatly reduces the amount of
PLAB waste caused by blocks of memory that cannot be used at the end of
regions?

The idea is simple: instead of allocating a PLAB that does not fit the
current region in a new region, look if it is possible to reuse the
remainder of the current region for the current PLAB.

E.g. if the current allocation region during GC has 19k words left, but
PLAB size is 20k, and the next allocation is just a few words, return a
19k PLAB to the caller instead of wasting the 19k.

There were no performance issues noticed here, the number of GCs tends
to decrease significantly though.

The main implementation idea is that in many allocation calls the single
"word_size" parameter is replaced by minimum, desired and actual word
size parameters which represent the minimum and desired (input)
allocation sizes, and the actual word size an output parameter that
indicates the actual number of words allocated.

CR:
https://bugs.openjdk.java.net/browse/JDK-8067336
Webrev:
http://cr.openjdk.java.net/~tschatzl/8067336/webrev/
Testing:
jprt, vm.gc test list

Thanks,
  Thomas




From jesper.wilhelmsson at oracle.com  Tue Aug 18 11:53:45 2015
From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson)
Date: Tue, 18 Aug 2015 13:53:45 +0200
Subject: RFR: JDK-8133349 - CMS: Assert failed: Ctl pt invariant
In-Reply-To: <55D28A71.30905@oracle.com>
References: <55D28A71.30905@oracle.com>
Message-ID: <F7A142BE-893E-48BB-8F16-D35C1EB4FABA@oracle.com>

Looks good!
/Jesper



> 18 aug 2015 kl. 03:29 skrev Eric Caspole <eric.caspole at oracle.com>:
> 
> Hi everybody,
> Could I get reviews for this bug 8133349. 
> This is assert is happening because the _survivor_chunk_array size is based on a calculation from ThreadLocalAllocBuffer::min_size(), but during a GC the PLAB::min_size() is used as the lower limit.
> 
> These are different because the TLAB alignment_reserve() == 40 words, but the PLAB AlignmentReserve == 2 words which is from oopDesc::header_size().
> 
> So that means the _survivor_chunk_array is a little bit smaller than will fit all the PLABs that could be allocated during a GC. 
> 
> Before the fix for JDK-8078904, the _survivor_chunk_array was padded by 2x, not sure why, so this situation never caused a problem. With the fix for JDK-8078904, the _survivor_chunk_array is exactly sized to hold the PLABs.
> 
> Bug:
> https://bugs.openjdk.java.net/browse/JDK-8133349
> Webrev:
> http://cr.openjdk.java.net/~ecaspole/JDK-8133349/01/webrev/
> 
> Passes JPRT and many  counts of PGC threads in the failing test, which only was exposed in a certain Solaris situation.
> Thanks,
> Eric
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150818/6b57eb4a/attachment.htm>

From eric.caspole at oracle.com  Tue Aug 18 13:53:41 2015
From: eric.caspole at oracle.com (Eric Caspole)
Date: Tue, 18 Aug 2015 09:53:41 -0400
Subject: RFR: JDK-8133349 - CMS: Assert failed: Ctl pt invariant
In-Reply-To: <1439886574.2316.1.camel@oracle.com>
References: <55D28A71.30905@oracle.com> <1439886574.2316.1.camel@oracle.com>
Message-ID: <55D338E5.6030208@oracle.com>

Hi Thomas,

In threadLocalAllocBuffer.hpp line 133 the TLAB leaves end_reserve space 
for prefetch with is probably different on various architectures.

On SPARC 4 core VMs like in dev ops, in that test with that object 
allocation pattern/heap layout it happened that it only failed with 4 or 
more PGC threads, but passed with 3 or less.

I'll try to make a specific test related to this situation now that I 
see what was going on.
Eric


On 08/18/2015 04:29 AM, Thomas Schatzl wrote:
> Hi Eric,
>
> On Mon, 2015-08-17 at 21:29 -0400, Eric Caspole wrote:
>> Hi everybody,
>> Could I get reviews for this bug 8133349.
>> This is assert is happening because the _survivor_chunk_array size is
>> based on a calculation from ThreadLocalAllocBuffer::min_size(), but
>> during a GC the PLAB::min_size() is used as the lower limit.
>>
>> These are different because the TLAB alignment_reserve() == 40 words,
>> but the PLAB AlignmentReserve == 2 words which is from
>> oopDesc::header_size().
>
> Any idea why the TLAB alignment reserve is 40 words? (Just curious).
>>
>> So that means the _survivor_chunk_array is a little bit smaller than
>> will fit all the PLABs that could be allocated during a GC.
>>
>> Before the fix for JDK-8078904, the _survivor_chunk_array was padded
>> by 2x, not sure why, so this situation never caused a problem. With
>> the fix for JDK-8078904, the _survivor_chunk_array is exactly sized to
>> hold the PLABs.
>>
>> Bug:
>> https://bugs.openjdk.java.net/browse/JDK-8133349
>> Webrev:
>> http://cr.openjdk.java.net/~ecaspole/JDK-8133349/01/webrev/
>>
>> Passes JPRT and many  counts of PGC threads in the failing test, which
>> only was exposed in a certain Solaris situation.
>
>    thanks, looks good. Is it possible to recreate that Solaris test using
> a simple JPRT test? Or is this a 100% failure on the existing test?
>
> Thanks,
>    Thomas
>
>


From tom.benson at oracle.com  Tue Aug 18 15:03:49 2015
From: tom.benson at oracle.com (Tom Benson)
Date: Tue, 18 Aug 2015 11:03:49 -0400
Subject: RFR (M): 8067336: Allow that PLAB allocations at the end of
	regions are flexible
In-Reply-To: <1439896720.2316.7.camel@oracle.com>
References: <1439896720.2316.7.camel@oracle.com>
Message-ID: <55D34955.10006@oracle.com>

Hi Thomas,
It looks like the actual plab allocation changes are missing from 
g1Allocator.cpp - only an include was added...   ?

Aside from that, just a couple of minor comments.  I think the comment 
at line 193-194 in g1AllocRegion.hpp should say "...maximum word size of 
the allocation in desired_word_size" rather than "... in word_size".

In g1AllocRegion.inline.hpp, attempt_allocation and 
attempt_allocation_locked both have an error path that includes:
trace("alloc failed", *actual_word_size);

In the old version, the value handed to trace was the requested size, so 
here it should probably be min_word_size, since *actual_word_size is 
likely to be uninitialized.

Tom

On 8/18/2015 7:18 AM, Thomas Schatzl wrote:
> Hi all,
>
>    can I have reviews for this change that greatly reduces the amount of
> PLAB waste caused by blocks of memory that cannot be used at the end of
> regions?
>
> The idea is simple: instead of allocating a PLAB that does not fit the
> current region in a new region, look if it is possible to reuse the
> remainder of the current region for the current PLAB.
>
> E.g. if the current allocation region during GC has 19k words left, but
> PLAB size is 20k, and the next allocation is just a few words, return a
> 19k PLAB to the caller instead of wasting the 19k.
>
> There were no performance issues noticed here, the number of GCs tends
> to decrease significantly though.
>
> The main implementation idea is that in many allocation calls the single
> "word_size" parameter is replaced by minimum, desired and actual word
> size parameters which represent the minimum and desired (input)
> allocation sizes, and the actual word size an output parameter that
> indicates the actual number of words allocated.
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8067336
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8067336/webrev/
> Testing:
> jprt, vm.gc test list
>
> Thanks,
>    Thomas
>
>



From erik.helin at oracle.com  Tue Aug 18 15:18:04 2015
From: erik.helin at oracle.com (Erik Helin)
Date: Tue, 18 Aug 2015 17:18:04 +0200
Subject: RFR (S): 8133530: Add JFR event for evacuation statistics
In-Reply-To: <1439890950.2316.4.camel@oracle.com>
References: <1439480605.2271.65.camel@oracle.com> <55CDD14C.2050600@oracle.com>
	<1439818218.2351.12.camel@oracle.com>
	<1439890950.2316.4.camel@oracle.com>
Message-ID: <20150818151804.GB19856@ehelin.jrpg.bea.com>

On 2015-08-18, Thomas Schatzl wrote:
> Hi all,
> 
>   Erik had some further comments in private regarding style and better
> encapsulation of the code.
> 
> Here are the new webrevs:
> http://cr.openjdk.java.net/~tschatzl/8133530/webrev.2 (full)
> http://cr.openjdk.java.net/~tschatzl/8133530/webrev.1_to_2 (full)

Looks good, Reviewed.

Thanks,
Erik

> Main changes are about moving the G1NewTracer::create_g1_evacstats()
> methods into the cpp file as a static method.
> 
> Thanks,
>   Thomas
> 
> On Mon, 2015-08-17 at 15:30 +0200, Thomas Schatzl wrote:
> > Hi Mikael,
> > 
> > On Fri, 2015-08-14 at 13:30 +0200, Mikael Gerdin wrote:
> > > Hi Thomas,
> > > 
> > > On 2015-08-13 17:43, Thomas Schatzl wrote:
> > > > Hi all,
> > > >
> > > >    can I have reviews for this split-out from 8073013 that adds JFR
> > > > support for evacuation statistics.
> > > >
> > > > The code has been changed to (hopefully) correspond better to the way
> > > > code that sends JFR events should look like.
> > > >
> > > > CR:
> > > > https://bugs.openjdk.java.net/browse/JDK-8133530
> > > > Webrev:
> > > > http://cr.openjdk.java.net/~tschatzl/8133530/webrev/
> > > 
> > > in gcTrace.hpp
> > > 39 #include "gc/g1/g1YCTypes.hpp"
> > > 40 #include "gcHeapSummary.hpp"
> > > 41 #endif
> > > 42
> > > 
> > > you should use the full path to gcHeapSummary.hpp
> > > 
> > > in gcTraceSend.cpp
> > > 240   if (surv_evt.should_commit()) {
> > > 241     surv_evt.set_gcId(GCId::peek().id());
> > > 242     surv_evt.set_allocated(summary.allocated() * HeapWordSize);
> > > 
> > > The other events use
> > > 232     e.set_gcId(_shared_gc_info.gc_id().id());
> > > 
> > > When you assert_set_gc_id you actually verify that the shared gc info id 
> > > is set.
> > 
> >   all fixed in 
> > http://cr.openjdk.java.net/~tschatzl/8133530/webrev.0_to_1 (diff)
> > http://cr.openjdk.java.net/~tschatzl/8133530/webrev.1 (full)
> > 
> > I also incorporated a suggestion by Erik that extracted the common
> > members of the two new events into a single (JFR) struct to benefit a
> > little from refactoring.
> > 
> > Thanks,
> >   Thomas
> > 
> > 
> 
> 


From bengt.rutisson at oracle.com  Tue Aug 18 15:22:18 2015
From: bengt.rutisson at oracle.com (Bengt Rutisson)
Date: Tue, 18 Aug 2015 17:22:18 +0200
Subject: RFR: 8130308 - Too low memory usage in
	TestPromotionFromSurvivorToTenuredAfterMinorGC.java
In-Reply-To: <55D23812.1030107@oracle.com>
References: <55D23812.1030107@oracle.com>
Message-ID: <55D34DAA.4050203@oracle.com>


Hi Jesper,

On 2015-08-17 21:37, Jesper Wilhelmsson wrote:
> Hi,
>
> Please review this fix for 
> TestPromotionFromSurvivorToTenuredAfterMinorGC.java
>
> Basically what happened was:
>
> 1. The test stores the amount of used heap
> 2. Some allocation is done and a series of GCs are called
> 3. The test keeps track of anything allocated during the test by other 
> threads
> 4. Verify that the total amount of used old gen is what we expect 
> based on how much was allocated.
>
> The problem was that if data was allocated and was still in the 
> survivor space it was included in the expected size, but not included 
> in the verification.
>
> Since the test is all about tenured space and verifying that, I 
> decided to reduce the expected amount by what's used in the survivor 
> area rather than including the survivor area in the verification.
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8130308
> Webrev: http://cr.openjdk.java.net/~jwilhelm/8130308/webrev.00/

Looks good.

Thanks,
Bengt

>
> Thanks,
> /Jesper



From alexander.harlap at oracle.com  Tue Aug 18 16:02:03 2015
From: alexander.harlap at oracle.com (Alexander Harlap)
Date: Tue, 18 Aug 2015 12:02:03 -0400
Subject: Request for code review - 8130265: gctests/LargeObjects/large001
	fails with OutOfMemoryError: Java heap space
Message-ID: <55D356FB.1030603@oracle.com>

This bug was caused by  too aggressive heap shrinkage in G1.

JDK-8130265 <https://bugs.openjdk.java.net/browse/JDK-8130265>

Proposed fix:
http://cr.openjdk.java.net/~aharlap/8130265/webrev.00
<http://cr.openjdk.java.net/%7Eaharlap/8130265/webrev.00>
Testing: JPRT, tonga

Thank you,
Alex

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150818/966b82ab/attachment.htm>

From jesper.wilhelmsson at oracle.com  Tue Aug 18 16:15:14 2015
From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson)
Date: Tue, 18 Aug 2015 18:15:14 +0200
Subject: Request for code review - 8130265: gctests/LargeObjects/large001
	fails with OutOfMemoryError: Java heap space
In-Reply-To: <55D356FB.1030603@oracle.com>
References: <55D356FB.1030603@oracle.com>
Message-ID: <55D35A12.8020608@oracle.com>

Hi,

+  minimum_desired_capacity += aligned_bytes_to_allocate;
+  maximum_desired_capacity += aligned_bytes_to_allocate;

Is it desired to always increase the desired capacity rather than doing it only 
if it is too small? E.g.:

minimum_desired_capacity = MAX2(minimum_desired_capacity, 
aligned_bytes_to_allocate);
maximum_desired_capacity = MAX2(maximum_desired_capacity, 
aligned_bytes_to_allocate);

Thanks,
/Jesper


Alexander Harlap skrev den 18/8/15 18:02:
> This bug was caused by  too aggressive heap shrinkage in G1.
>
> JDK-8130265 <https://bugs.openjdk.java.net/browse/JDK-8130265>
>
> Proposed fix:
> http://cr.openjdk.java.net/~aharlap/8130265/webrev.00
> <http://cr.openjdk.java.net/%7Eaharlap/8130265/webrev.00>
> Testing: JPRT, tonga
>
> Thank you,
> Alex
>


From alexander.harlap at oracle.com  Tue Aug 18 16:36:47 2015
From: alexander.harlap at oracle.com (Alexander Harlap)
Date: Tue, 18 Aug 2015 12:36:47 -0400
Subject: Request for code review - 8130265: gctests/LargeObjects/large001
	fails with OutOfMemoryError: Java heap space
In-Reply-To: <55D35A12.8020608@oracle.com>
References: <55D356FB.1030603@oracle.com> <55D35A12.8020608@oracle.com>
Message-ID: <55D35F1F.3090700@oracle.com>

Hi Jesper,

Here minimum_desired_capacity and  maximum_desired_capacity are 
calculated to accommodate already allocated sstuff,
aligned_bytes_to_allocate - extra size needed to allocate new memory (gc 
was called when we failed to allocate this new memory).

Alex


On 8/18/2015 12:15 PM, Jesper Wilhelmsson wrote:
> Hi,
>
> +  minimum_desired_capacity += aligned_bytes_to_allocate;
> +  maximum_desired_capacity += aligned_bytes_to_allocate;
>
> Is it desired to always increase the desired capacity rather than 
> doing it only if it is too small? E.g.:
>
> minimum_desired_capacity = MAX2(minimum_desired_capacity, 
> aligned_bytes_to_allocate);
> maximum_desired_capacity = MAX2(maximum_desired_capacity, 
> aligned_bytes_to_allocate);
>
> Thanks,
> /Jesper
>
>
> Alexander Harlap skrev den 18/8/15 18:02:
>> This bug was caused by  too aggressive heap shrinkage in G1.
>>
>> JDK-8130265 <https://bugs.openjdk.java.net/browse/JDK-8130265>
>>
>> Proposed fix:
>> http://cr.openjdk.java.net/~aharlap/8130265/webrev.00
>> <http://cr.openjdk.java.net/%7Eaharlap/8130265/webrev.00>
>> Testing: JPRT, tonga
>>
>> Thank you,
>> Alex
>>



From jesper.wilhelmsson at oracle.com  Tue Aug 18 17:39:19 2015
From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson)
Date: Tue, 18 Aug 2015 19:39:19 +0200
Subject: Request for code review - 8130265: gctests/LargeObjects/large001
	fails with OutOfMemoryError: Java heap space
In-Reply-To: <55D35F1F.3090700@oracle.com>
References: <55D356FB.1030603@oracle.com> <55D35A12.8020608@oracle.com>
	<55D35F1F.3090700@oracle.com>
Message-ID: <55D36DC7.6010901@oracle.com>

Alexander Harlap skrev den 18/8/15 18:36:
> Hi Jesper,
>
> Here minimum_desired_capacity and  maximum_desired_capacity are calculated to
> accommodate already allocated sstuff,
> aligned_bytes_to_allocate - extra size needed to allocate new memory (gc was
> called when we failed to allocate this new memory).

Right. Would it make sense to put this new addition around line 1658 instead, 
before we start to adjust the values with respect to max_heap_size?
/Jesper

>
> Alex
>
>
> On 8/18/2015 12:15 PM, Jesper Wilhelmsson wrote:
>> Hi,
>>
>> +  minimum_desired_capacity += aligned_bytes_to_allocate;
>> +  maximum_desired_capacity += aligned_bytes_to_allocate;
>>
>> Is it desired to always increase the desired capacity rather than doing it
>> only if it is too small? E.g.:
>>
>> minimum_desired_capacity = MAX2(minimum_desired_capacity,
>> aligned_bytes_to_allocate);
>> maximum_desired_capacity = MAX2(maximum_desired_capacity,
>> aligned_bytes_to_allocate);
>>
>> Thanks,
>> /Jesper
>>
>>
>> Alexander Harlap skrev den 18/8/15 18:02:
>>> This bug was caused by  too aggressive heap shrinkage in G1.
>>>
>>> JDK-8130265 <https://bugs.openjdk.java.net/browse/JDK-8130265>
>>>
>>> Proposed fix:
>>> http://cr.openjdk.java.net/~aharlap/8130265/webrev.00
>>> <http://cr.openjdk.java.net/%7Eaharlap/8130265/webrev.00>
>>> Testing: JPRT, tonga
>>>
>>> Thank you,
>>> Alex
>>>
>


From eric.caspole at oracle.com  Tue Aug 18 18:09:28 2015
From: eric.caspole at oracle.com (Eric Caspole)
Date: Tue, 18 Aug 2015 14:09:28 -0400
Subject: RFR: JDK-8133349 - CMS: Assert failed: Ctl pt invariant
In-Reply-To: <1439886574.2316.1.camel@oracle.com>
References: <55D28A71.30905@oracle.com> <1439886574.2316.1.camel@oracle.com>
Message-ID: <55D374D8.4090109@oracle.com>

It appears that the TLAB alignment_reserve() on x64 is 2, the same as 
for PLAB, so that explains why this assert never happened there.
Eric


On 08/18/2015 04:29 AM, Thomas Schatzl wrote:
> Hi Eric,
>
> On Mon, 2015-08-17 at 21:29 -0400, Eric Caspole wrote:
>> Hi everybody,
>> Could I get reviews for this bug 8133349.
>> This is assert is happening because the _survivor_chunk_array size is
>> based on a calculation from ThreadLocalAllocBuffer::min_size(), but
>> during a GC the PLAB::min_size() is used as the lower limit.
>>
>> These are different because the TLAB alignment_reserve() == 40 words,
>> but the PLAB AlignmentReserve == 2 words which is from
>> oopDesc::header_size().
>
> Any idea why the TLAB alignment reserve is 40 words? (Just curious).
>>
>> So that means the _survivor_chunk_array is a little bit smaller than
>> will fit all the PLABs that could be allocated during a GC.
>>
>> Before the fix for JDK-8078904, the _survivor_chunk_array was padded
>> by 2x, not sure why, so this situation never caused a problem. With
>> the fix for JDK-8078904, the _survivor_chunk_array is exactly sized to
>> hold the PLABs.
>>
>> Bug:
>> https://bugs.openjdk.java.net/browse/JDK-8133349
>> Webrev:
>> http://cr.openjdk.java.net/~ecaspole/JDK-8133349/01/webrev/
>>
>> Passes JPRT and many  counts of PGC threads in the failing test, which
>> only was exposed in a certain Solaris situation.
>
>    thanks, looks good. Is it possible to recreate that Solaris test using
> a simple JPRT test? Or is this a 100% failure on the existing test?
>
> Thanks,
>    Thomas
>
>


From thomas.schatzl at oracle.com  Wed Aug 19 06:19:51 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 19 Aug 2015 08:19:51 +0200
Subject: RFR (M): 8067336: Allow that PLAB allocations at the end of
	regions are flexible
In-Reply-To: <1439896720.2316.7.camel@oracle.com>
References: <1439896720.2316.7.camel@oracle.com>
Message-ID: <1439965191.1943.2.camel@oracle.com>

Hi all,

  let me retract this RFR for a short while. As Tom noticed, a few
changes are missing which results in the changes having no actual
effect.

>From what I can see this is because of many additional splits and
refactorings of all the recent dependent changes.

I expect to need one or two days fixing this and running it through
testing again.

Sorry and thanks,
  Thomas

On Tue, 2015-08-18 at 13:18 +0200, Thomas Schatzl wrote:
> Hi all,
> 
>   can I have reviews for this change that greatly reduces the amount of
> PLAB waste caused by blocks of memory that cannot be used at the end of
> regions?
> 
> The idea is simple: instead of allocating a PLAB that does not fit the
> current region in a new region, look if it is possible to reuse the
> remainder of the current region for the current PLAB.
> 
> E.g. if the current allocation region during GC has 19k words left, but
> PLAB size is 20k, and the next allocation is just a few words, return a
> 19k PLAB to the caller instead of wasting the 19k.
> 
> There were no performance issues noticed here, the number of GCs tends
> to decrease significantly though.
> 
> The main implementation idea is that in many allocation calls the single
> "word_size" parameter is replaced by minimum, desired and actual word
> size parameters which represent the minimum and desired (input)
> allocation sizes, and the actual word size an output parameter that
> indicates the actual number of words allocated.
> 
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8067336
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8067336/webrev/
> Testing:
> jprt, vm.gc test list
> 
> Thanks,
>   Thomas
> 
> 




From erik.helin at oracle.com  Wed Aug 19 09:38:47 2015
From: erik.helin at oracle.com (Erik Helin)
Date: Wed, 19 Aug 2015 11:38:47 +0200
Subject: RFR: 8133821: Refactor initialization of the heap and the collector
	policy
Message-ID: <20150819093847.GC19856@ehelin.jrpg.bea.com>

Hi all,

this small patch refactors how the heap is set up to make the code a bit
more extensible.

Webrev:
http://cr.openjdk.java.net/~ehelin/8133821/webrev.00/

Bug:
https://bugs.openjdk.java.net/browse/JDK-8133821

Testing:
JPRT

Thanks,
Erik


From thomas.schatzl at oracle.com  Wed Aug 19 10:25:33 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 19 Aug 2015 12:25:33 +0200
Subject: RFR: 8133821: Refactor initialization of the heap and the
	collector policy
In-Reply-To: <20150819093847.GC19856@ehelin.jrpg.bea.com>
References: <20150819093847.GC19856@ehelin.jrpg.bea.com>
Message-ID: <1439979933.2313.0.camel@oracle.com>

Hi Erik,

On Wed, 2015-08-19 at 11:38 +0200, Erik Helin wrote:
> Hi all,
> 
> this small patch refactors how the heap is set up to make the code a bit
> more extensible.
> 
> Webrev:
> http://cr.openjdk.java.net/~ehelin/8133821/webrev.00/
> 
> Bug:
> https://bugs.openjdk.java.net/browse/JDK-8133821
> 
> Testing:
> JPRT
> 

  looks good.

Thomas




From per.liden at oracle.com  Wed Aug 19 11:42:06 2015
From: per.liden at oracle.com (Per Liden)
Date: Wed, 19 Aug 2015 13:42:06 +0200
Subject: RFR: 8133821: Refactor initialization of the heap and the
	collector policy
In-Reply-To: <20150819093847.GC19856@ehelin.jrpg.bea.com>
References: <20150819093847.GC19856@ehelin.jrpg.bea.com>
Message-ID: <55D46B8E.8030708@oracle.com>

Erik,

On 2015-08-19 11:38, Erik Helin wrote:
> Hi all,
>
> this small patch refactors how the heap is set up to make the code a bit
> more extensible.
>
> Webrev:
> http://cr.openjdk.java.net/~ehelin/8133821/webrev.00/

Looks good.

/Per

>
> Bug:
> https://bugs.openjdk.java.net/browse/JDK-8133821
>
> Testing:
> JPRT
>
> Thanks,
> Erik
>


From thomas.schatzl at oracle.com  Wed Aug 19 12:43:19 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 19 Aug 2015 14:43:19 +0200
Subject: RFR (S): 8073146: G1 merges thread local age tables too early
	with global age table
In-Reply-To: <55CB49A5.40804@oracle.com>
References: <1438949116.2305.43.camel@oracle.com>
	<55CB020C.8010803@oracle.com> <1439377357.2324.28.camel@oracle.com>
	<55CB49A5.40804@oracle.com>
Message-ID: <1439988199.2313.1.camel@oracle.com>

Hi Mikael,

On Wed, 2015-08-12 at 15:27 +0200, Mikael Gerdin wrote:
> On 2015-08-12 13:02, Thomas Schatzl wrote:
> > Hi David,
> >
> > On Wed, 2015-08-12 at 10:21 +0200, David Lindholm wrote:
> >> Thomas,
> >>
> >> This change looks good. Reviewed.
> 
> Looks good to me as well.
> /Mikael

  thanks for the review.

Thomas



From erik.helin at oracle.com  Wed Aug 19 15:09:58 2015
From: erik.helin at oracle.com (Erik Helin)
Date: Wed, 19 Aug 2015 17:09:58 +0200
Subject: RFR: 8133825: Remove the class G1CollectorPolicyExt
Message-ID: <20150819150958.GB31586@ehelin.jrpg.bea.com>

Hi all,

after the patch "8133821: Refactor initialization of the heap and the
collector policy", the class G1CollectorPolicyExt can be removed.

Webrev:
http://cr.openjdk.java.net/~ehelin/8133825/webrev.00/

Bug:
https://bugs.openjdk.java.net/browse/JDK-8133825

Testing:
JPRT

Thanks,
Erik


From jesper.wilhelmsson at oracle.com  Wed Aug 19 15:22:41 2015
From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson)
Date: Wed, 19 Aug 2015 17:22:41 +0200
Subject: RFR: 8133825: Remove the class G1CollectorPolicyExt
In-Reply-To: <20150819150958.GB31586@ehelin.jrpg.bea.com>
References: <20150819150958.GB31586@ehelin.jrpg.bea.com>
Message-ID: <55D49F41.50902@oracle.com>

Looks good!
/Jesper

Erik Helin skrev den 19/8/15 17:09:
> Hi all,
>
> after the patch "8133821: Refactor initialization of the heap and the
> collector policy", the class G1CollectorPolicyExt can be removed.
>
> Webrev:
> http://cr.openjdk.java.net/~ehelin/8133825/webrev.00/
>
> Bug:
> https://bugs.openjdk.java.net/browse/JDK-8133825
>
> Testing:
> JPRT
>
> Thanks,
> Erik
>


From eric.caspole at oracle.com  Wed Aug 19 15:28:57 2015
From: eric.caspole at oracle.com (Eric Caspole)
Date: Wed, 19 Aug 2015 11:28:57 -0400
Subject: RFR: 8133825: Remove the class G1CollectorPolicyExt
In-Reply-To: <20150819150958.GB31586@ehelin.jrpg.bea.com>
References: <20150819150958.GB31586@ehelin.jrpg.bea.com>
Message-ID: <55D4A0B9.4040900@oracle.com>

Looks good.
Eric

On 08/19/2015 11:09 AM, Erik Helin wrote:
> Hi all,
>
> after the patch "8133821: Refactor initialization of the heap and the
> collector policy", the class G1CollectorPolicyExt can be removed.
>
> Webrev:
> http://cr.openjdk.java.net/~ehelin/8133825/webrev.00/
>
> Bug:
> https://bugs.openjdk.java.net/browse/JDK-8133825
>
> Testing:
> JPRT
>
> Thanks,
> Erik
>


From kim.barrett at oracle.com  Wed Aug 19 17:10:49 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Wed, 19 Aug 2015 13:10:49 -0400
Subject: RFR: 8131330: G1CollectedHeap::verify_dirty_young_list fails with
	assert
Message-ID: <5D7EB2C6-6AC4-4B87-8FD1-FC487328087C@oracle.com>

Please review this fix for a recurrence of a problem with certain uses
of memset on SPARC platforms [1][2].

This change adds a helper function, memset_with_concurrent_readers,
for use by G1 and CMS in situations where a memory region is to be
filled while there may be concurrent readers of that region.  This
function is implemented by simply forwarding to memset on most
platforms.  On SPARC, this function avoids calling memset, because
that function may be implemented using block initialization stores,
which lead to the problems described in [1] and [2].

As part of this change we also eliminate the UseMemSetInBOT
experimental command line option, thereby resolving [3].

The SPARC implementation of memset_with_concurrent_readers is written
using inline assembly code rather than a C++ for-loop, because the
compiler may recognize the for-loop and transform it into a call to
memset, undoing our intentional avoidance of memset.  We see exactly
this happening with Solaris Studio 12.3, for example.  It is this
transformation that led to this recurrence of the problem from [1][2].

CR:
https://bugs.openjdk.java.net/browse/JDK-8131330

Webrev:
http://cr.openjdk.java.net/~kbarrett/8131330/webrev.00/

Testing:
jprt
aurora ad hoc GC/Runtime Nightly, quicktests, tonga, jtreg

[1] https://bugs.openjdk.java.net/browse/JDK-8039042
[2] https://bugs.openjdk.java.net/browse/JDK-6948537
[3] https://bugs.openjdk.java.net/browse/JDK-8042930



From kim.barrett at oracle.com  Wed Aug 19 17:42:04 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Wed, 19 Aug 2015 13:42:04 -0400
Subject: RFR: 8072817: CardTableExtension kind() should be
	BarrierSet::CardTableExtension
Message-ID: <DA95D846-B071-45B7-81C1-4DD4C9DC0D92@oracle.com>

Please review this change to finally properly integrate the
BarrierSet::CardTableExtension barrier set kind. This has been
deferred for a long time; there was, until recently, a comment in
CardTableExtension that having kind() return
BarrierSet::CardTableExtension was "too risky for the 4/10/02
putback".

Much of the work involved in supporting the CardTableExtension tag had
been done long ago.  The remaining work involved actually hooking it
up.  The key missing step was to give CardTableModRefBSForCTRS its own
tag (to distinguish from CardTableExtension) and switch to using that
tag where needed.  The needed tag (BarrierSet::CardTableForRS) was
defined in the fix for JDK-8069016.  This change makes use of it.

This involved the following steps:

1. Most occurrences of BarrierSet::CardTableModRef are now
BarrierSet::CardTableForRS. Occurrences of BarrierSet::CardTableForRS
and BarrierSet::CardTableExtension are now nearly always paired.
Exceptions are in the constructors of the corresponding classes and in
the associated barrier_set_cast support.

2. The tests for kind() == BarrierSet::CardTableModRef that were
controlling the devirtualization of write_ref_field_pre,
write_ref_field, and write_region now test for either
BarrierSet::CardTableForRS or BarrierSet::CardTableExtension.  The
FakeRttiSupport class was extended to improve the new test.

3. The remaining small number of tests for kind() ==
BarrierSet::CardTableModRef are now tests for
is_a(BarrierSet::CardTableModRef). These are places where we really do
allow a CardTableModRefBS and don't care which subclass; all of these
involve access to the card table. This now includes G1 barrier sets in
a couple of places where they were previously inappropriately or
unnecessarily excluded.

4. The CardTableModRefBSForCTRS class now specifies its kind tag be
BarrierSet::CardTableForRS. The CardTableExtension class now specifies
its kind tag be BarrierSet::CardTableExtension.  The extra
FakeRttiSupport constructor that existed to support the old kludge for
those classes has been removed.

Also removed unused CollectorPolicy::barrier_set_name.

CR:
https://bugs.openjdk.java.net/browse/JDK-8072817

Webrev:
http://cr.openjdk.java.net/~kbarrett/8072817/webrev.00/

Testing:
jprt
aurora ad hoc GC/Runtime Nightly, quicktests, tonga, jtreg



From eric.caspole at oracle.com  Wed Aug 19 17:51:43 2015
From: eric.caspole at oracle.com (Eric Caspole)
Date: Wed, 19 Aug 2015 13:51:43 -0400
Subject: RFR: 8131330: G1CollectedHeap::verify_dirty_young_list fails
	with assert
In-Reply-To: <5D7EB2C6-6AC4-4B87-8FD1-FC487328087C@oracle.com>
References: <5D7EB2C6-6AC4-4B87-8FD1-FC487328087C@oracle.com>
Message-ID: <55D4C22F.2000003@oracle.com>

Hi Kim,
Is it allowed to use a more descriptive label name than 1,2,3 in the 
sparc inline asm? I expect this works fine and no one will ever look at 
it again but just in case, I think it is easier to read and see the use 
of the labels if it is a descriptive word.
The rest of it looks good.
Thanks,
Eric


On 08/19/2015 01:10 PM, Kim Barrett wrote:
> Please review this fix for a recurrence of a problem with certain uses
> of memset on SPARC platforms [1][2].
>
> This change adds a helper function, memset_with_concurrent_readers,
> for use by G1 and CMS in situations where a memory region is to be
> filled while there may be concurrent readers of that region.  This
> function is implemented by simply forwarding to memset on most
> platforms.  On SPARC, this function avoids calling memset, because
> that function may be implemented using block initialization stores,
> which lead to the problems described in [1] and [2].
>
> As part of this change we also eliminate the UseMemSetInBOT
> experimental command line option, thereby resolving [3].
>
> The SPARC implementation of memset_with_concurrent_readers is written
> using inline assembly code rather than a C++ for-loop, because the
> compiler may recognize the for-loop and transform it into a call to
> memset, undoing our intentional avoidance of memset.  We see exactly
> this happening with Solaris Studio 12.3, for example.  It is this
> transformation that led to this recurrence of the problem from [1][2].
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8131330
>
> Webrev:
> http://cr.openjdk.java.net/~kbarrett/8131330/webrev.00/
>
> Testing:
> jprt
> aurora ad hoc GC/Runtime Nightly, quicktests, tonga, jtreg
>
> [1] https://bugs.openjdk.java.net/browse/JDK-8039042
> [2] https://bugs.openjdk.java.net/browse/JDK-6948537
> [3] https://bugs.openjdk.java.net/browse/JDK-8042930
>


From alexander.harlap at oracle.com  Wed Aug 19 20:14:21 2015
From: alexander.harlap at oracle.com (Alexander Harlap)
Date: Wed, 19 Aug 2015 16:14:21 -0400
Subject: Request for code review - 8130265: gctests/LargeObjects/large001
	fails with OutOfMemoryError: Java heap space
In-Reply-To: <55D394AD.6020104@oracle.com>
References: <55D356FB.1030603@oracle.com> <55D35A12.8020608@oracle.com>
	<55D35F1F.3090700@oracle.com> <55D36DC7.6010901@oracle.com>
	<55D38CBE.8070408@oracle.com> <55D394AD.6020104@oracle.com>
Message-ID: <55D4E39D.5070805@oracle.com>

Hi Jesper,

I agree that touching code  in 
resize_if_necessary_after_full_collection(size_t word_size) is not pretty.

Instead of doing  precise shrink, let it "over shrink" and  expand after.

Here is link to new version of change:

http://cr.openjdk.java.net/~aharlap/8130265/webrev.01/

Alex

On 8/18/2015 4:25 PM, Jesper Wilhelmsson wrote:
> Hi Alex,
>
> Alexander Harlap skrev den 18/8/15 21:51:
>> Hi Jesper,
>>
>> I need whole HeapRegion(s) in aligned_bytes_to_allocate,
>> so operations on doubles will result in rounding and will not work.
>> If you care about overflowing size_t by doing += 
>> aligned_bytes_to_allocate, it
>> can avoided with code like this:
>>
>> minimum_desired_capacity =  ((max_heap_size - 
>> minimum_desired_capacity) >
>> aligned_bytes_to_allocate) ? minimum_desired_capacity +
>> aligned_bytes_to_allocate : max_heap_size ;
>>
>> maximum_desired_capacity =  ((max_heap_size - 
>> maximum_desired_capacity) >
>> aligned_bytes_to_allocate) ? maximum_desired_capacity +
>> aligned_bytes_to_allocate : max_heap_size ;
>
> Yes, I think we need to do something like this. It seems odd to go 
> through all that trouble above to avoid overflowing and then just add 
> a random size afterwards.
>
> I never really formulated my original concern though. I may be wrong 
> here so feel free to interrupt at any point. The min and max values 
> represent a range of sizes that we consider desirable. By adding the 
> new size to both min and max we move that entire range. That seems 
> wrong to me. If the size we need is already below the max value, all 
> we need to do is to add to the min value. Or maybe we need to add a 
> little to max, but not the entire size. Or is it the case that the max 
> value will always be too small?
>
> Thanks,
> /Jesper
>
>>
>>
>> On 8/18/2015 1:39 PM, Jesper Wilhelmsson wrote:
>>> Alexander Harlap skrev den 18/8/15 18:36:
>>>> Hi Jesper,
>>>>
>>>> Here minimum_desired_capacity and  maximum_desired_capacity are 
>>>> calculated to
>>>> accommodate already allocated sstuff,
>>>> aligned_bytes_to_allocate - extra size needed to allocate new 
>>>> memory (gc was
>>>> called when we failed to allocate this new memory).
>>>
>>> Right. Would it make sense to put this new addition around line 1658 
>>> instead,
>>> before we start to adjust the values with respect to max_heap_size?
>>> /Jesper
>>>
>>>>
>>>> Alex
>>>>
>>>>
>>>> On 8/18/2015 12:15 PM, Jesper Wilhelmsson wrote:
>>>>> Hi,
>>>>>
>>>>> +  minimum_desired_capacity += aligned_bytes_to_allocate;
>>>>> +  maximum_desired_capacity += aligned_bytes_to_allocate;
>>>>>
>>>>> Is it desired to always increase the desired capacity rather than 
>>>>> doing it
>>>>> only if it is too small? E.g.:
>>>>>
>>>>> minimum_desired_capacity = MAX2(minimum_desired_capacity,
>>>>> aligned_bytes_to_allocate);
>>>>> maximum_desired_capacity = MAX2(maximum_desired_capacity,
>>>>> aligned_bytes_to_allocate);
>>>>>
>>>>> Thanks,
>>>>> /Jesper
>>>>>
>>>>>
>>>>> Alexander Harlap skrev den 18/8/15 18:02:
>>>>>> This bug was caused by  too aggressive heap shrinkage in G1.
>>>>>>
>>>>>> JDK-8130265 <https://bugs.openjdk.java.net/browse/JDK-8130265>
>>>>>>
>>>>>> Proposed fix:
>>>>>> http://cr.openjdk.java.net/~aharlap/8130265/webrev.00
>>>>>> <http://cr.openjdk.java.net/%7Eaharlap/8130265/webrev.00>
>>>>>> Testing: JPRT, tonga
>>>>>>
>>>>>> Thank you,
>>>>>> Alex
>>>>>>
>>>>
>>



From jesper.wilhelmsson at oracle.com  Wed Aug 19 20:36:23 2015
From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson)
Date: Wed, 19 Aug 2015 22:36:23 +0200
Subject: Request for code review - 8130265: gctests/LargeObjects/large001
	fails with OutOfMemoryError: Java heap space
In-Reply-To: <55D4E39D.5070805@oracle.com>
References: <55D356FB.1030603@oracle.com> <55D35A12.8020608@oracle.com>
	<55D35F1F.3090700@oracle.com> <55D36DC7.6010901@oracle.com>
	<55D38CBE.8070408@oracle.com> <55D394AD.6020104@oracle.com>
	<55D4E39D.5070805@oracle.com>
Message-ID: <55D4E8C7.1080703@oracle.com>

Hi Alex,

Yes, I like this version much better.
I'm fine with the change as is, but would it make sense to add the same code 
after the first GC as well? So that every attempt_allocation_at_safepoint is 
followed by an expand_and_allocate. We could avoid a GC in some cases.
/Jesper

Alexander Harlap skrev den 19/8/15 22:14:
> Hi Jesper,
>
> I agree that touching code  in resize_if_necessary_after_full_collection(size_t
> word_size) is not pretty.
>
> Instead of doing  precise shrink, let it "over shrink" and  expand after.
>
> Here is link to new version of change:
>
> http://cr.openjdk.java.net/~aharlap/8130265/webrev.01/
>
> Alex
>
> On 8/18/2015 4:25 PM, Jesper Wilhelmsson wrote:
>> Hi Alex,
>>
>> Alexander Harlap skrev den 18/8/15 21:51:
>>> Hi Jesper,
>>>
>>> I need whole HeapRegion(s) in aligned_bytes_to_allocate,
>>> so operations on doubles will result in rounding and will not work.
>>> If you care about overflowing size_t by doing += aligned_bytes_to_allocate, it
>>> can avoided with code like this:
>>>
>>> minimum_desired_capacity =  ((max_heap_size - minimum_desired_capacity) >
>>> aligned_bytes_to_allocate) ? minimum_desired_capacity +
>>> aligned_bytes_to_allocate : max_heap_size ;
>>>
>>> maximum_desired_capacity =  ((max_heap_size - maximum_desired_capacity) >
>>> aligned_bytes_to_allocate) ? maximum_desired_capacity +
>>> aligned_bytes_to_allocate : max_heap_size ;
>>
>> Yes, I think we need to do something like this. It seems odd to go through all
>> that trouble above to avoid overflowing and then just add a random size
>> afterwards.
>>
>> I never really formulated my original concern though. I may be wrong here so
>> feel free to interrupt at any point. The min and max values represent a range
>> of sizes that we consider desirable. By adding the new size to both min and
>> max we move that entire range. That seems wrong to me. If the size we need is
>> already below the max value, all we need to do is to add to the min value. Or
>> maybe we need to add a little to max, but not the entire size. Or is it the
>> case that the max value will always be too small?
>>
>> Thanks,
>> /Jesper
>>
>>>
>>>
>>> On 8/18/2015 1:39 PM, Jesper Wilhelmsson wrote:
>>>> Alexander Harlap skrev den 18/8/15 18:36:
>>>>> Hi Jesper,
>>>>>
>>>>> Here minimum_desired_capacity and  maximum_desired_capacity are calculated to
>>>>> accommodate already allocated sstuff,
>>>>> aligned_bytes_to_allocate - extra size needed to allocate new memory (gc was
>>>>> called when we failed to allocate this new memory).
>>>>
>>>> Right. Would it make sense to put this new addition around line 1658 instead,
>>>> before we start to adjust the values with respect to max_heap_size?
>>>> /Jesper
>>>>
>>>>>
>>>>> Alex
>>>>>
>>>>>
>>>>> On 8/18/2015 12:15 PM, Jesper Wilhelmsson wrote:
>>>>>> Hi,
>>>>>>
>>>>>> +  minimum_desired_capacity += aligned_bytes_to_allocate;
>>>>>> +  maximum_desired_capacity += aligned_bytes_to_allocate;
>>>>>>
>>>>>> Is it desired to always increase the desired capacity rather than doing it
>>>>>> only if it is too small? E.g.:
>>>>>>
>>>>>> minimum_desired_capacity = MAX2(minimum_desired_capacity,
>>>>>> aligned_bytes_to_allocate);
>>>>>> maximum_desired_capacity = MAX2(maximum_desired_capacity,
>>>>>> aligned_bytes_to_allocate);
>>>>>>
>>>>>> Thanks,
>>>>>> /Jesper
>>>>>>
>>>>>>
>>>>>> Alexander Harlap skrev den 18/8/15 18:02:
>>>>>>> This bug was caused by  too aggressive heap shrinkage in G1.
>>>>>>>
>>>>>>> JDK-8130265 <https://bugs.openjdk.java.net/browse/JDK-8130265>
>>>>>>>
>>>>>>> Proposed fix:
>>>>>>> http://cr.openjdk.java.net/~aharlap/8130265/webrev.00
>>>>>>> <http://cr.openjdk.java.net/%7Eaharlap/8130265/webrev.00>
>>>>>>> Testing: JPRT, tonga
>>>>>>>
>>>>>>> Thank you,
>>>>>>> Alex
>>>>>>>
>>>>>
>>>
>


From kim.barrett at oracle.com  Wed Aug 19 20:48:21 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Wed, 19 Aug 2015 16:48:21 -0400
Subject: RFR: 8131330: G1CollectedHeap::verify_dirty_young_list fails with
	assert
In-Reply-To: <55D4C22F.2000003@oracle.com>
References: <5D7EB2C6-6AC4-4B87-8FD1-FC487328087C@oracle.com>
	<55D4C22F.2000003@oracle.com>
Message-ID: <08169C1E-696E-44C1-AA98-3F9AA2895197@oracle.com>

On Aug 19, 2015, at 1:51 PM, Eric Caspole <eric.caspole at oracle.com> wrote:
> 
> Is it allowed to use a more descriptive label name than 1,2,3 in the sparc inline asm? I expect this works fine and no one will ever look at it again but just in case, I think it is easier to read and see the use of the labels if it is a descriptive word.
> The rest of it looks good.
> Thanks,

Thanks Eric.

gcc extended asm provides a syntax for unique named labels, using
"%=".  But I couldn't get that to work under Solaris Studio.  This
might be a limitation of Solaris Studio's implementation of gcc's
extended asm.  Or it might be just a failure on my part to figure out
the right syntax.

I added a few comments to make it easier to spot the branches and
labels.

New webrev:
http://cr.openjdk.java.net/~kbarrett/8131330/webrev.01/

Incremental webrev:
http://cr.openjdk.java.net/~kbarrett/8131330/webrev.01.inc1/



From alexander.harlap at oracle.com  Wed Aug 19 21:02:45 2015
From: alexander.harlap at oracle.com (Alexander Harlap)
Date: Wed, 19 Aug 2015 17:02:45 -0400
Subject: Request for code review - 8130265: gctests/LargeObjects/large001
	fails with OutOfMemoryError: Java heap space
In-Reply-To: <55D4E8C7.1080703@oracle.com>
References: <55D356FB.1030603@oracle.com> <55D35A12.8020608@oracle.com>
	<55D35F1F.3090700@oracle.com> <55D36DC7.6010901@oracle.com>
	<55D38CBE.8070408@oracle.com> <55D394AD.6020104@oracle.com>
	<55D4E39D.5070805@oracle.com> <55D4E8C7.1080703@oracle.com>
Message-ID: <55D4EEF5.9030504@oracle.com>

I agree.

Here is new revision:

http://cr.openjdk.java.net/~aharlap/8130265/webrev.02/
<http://cr.openjdk.java.net/%7Eaharlap/8130265/webrev.02/>
Alex

On 8/19/2015 4:36 PM, Jesper Wilhelmsson wrote:
> Hi Alex,
>
> Yes, I like this version much better.
> I'm fine with the change as is, but would it make sense to add the 
> same code after the first GC as well? So that every 
> attempt_allocation_at_safepoint is followed by an expand_and_allocate. 
> We could avoid a GC in some cases.
> /Jesper
>
> Alexander Harlap skrev den 19/8/15 22:14:
>> Hi Jesper,
>>
>> I agree that touching code  in 
>> resize_if_necessary_after_full_collection(size_t
>> word_size) is not pretty.
>>
>> Instead of doing  precise shrink, let it "over shrink" and expand after.
>>
>> Here is link to new version of change:
>>
>> http://cr.openjdk.java.net/~aharlap/8130265/webrev.01/
>>
>> Alex
>>
>> On 8/18/2015 4:25 PM, Jesper Wilhelmsson wrote:
>>> Hi Alex,
>>>
>>> Alexander Harlap skrev den 18/8/15 21:51:
>>>> Hi Jesper,
>>>>
>>>> I need whole HeapRegion(s) in aligned_bytes_to_allocate,
>>>> so operations on doubles will result in rounding and will not work.
>>>> If you care about overflowing size_t by doing += 
>>>> aligned_bytes_to_allocate, it
>>>> can avoided with code like this:
>>>>
>>>> minimum_desired_capacity =  ((max_heap_size - 
>>>> minimum_desired_capacity) >
>>>> aligned_bytes_to_allocate) ? minimum_desired_capacity +
>>>> aligned_bytes_to_allocate : max_heap_size ;
>>>>
>>>> maximum_desired_capacity =  ((max_heap_size - 
>>>> maximum_desired_capacity) >
>>>> aligned_bytes_to_allocate) ? maximum_desired_capacity +
>>>> aligned_bytes_to_allocate : max_heap_size ;
>>>
>>> Yes, I think we need to do something like this. It seems odd to go 
>>> through all
>>> that trouble above to avoid overflowing and then just add a random size
>>> afterwards.
>>>
>>> I never really formulated my original concern though. I may be wrong 
>>> here so
>>> feel free to interrupt at any point. The min and max values 
>>> represent a range
>>> of sizes that we consider desirable. By adding the new size to both 
>>> min and
>>> max we move that entire range. That seems wrong to me. If the size 
>>> we need is
>>> already below the max value, all we need to do is to add to the min 
>>> value. Or
>>> maybe we need to add a little to max, but not the entire size. Or is 
>>> it the
>>> case that the max value will always be too small?
>>>
>>> Thanks,
>>> /Jesper
>>>
>>>>
>>>>
>>>> On 8/18/2015 1:39 PM, Jesper Wilhelmsson wrote:
>>>>> Alexander Harlap skrev den 18/8/15 18:36:
>>>>>> Hi Jesper,
>>>>>>
>>>>>> Here minimum_desired_capacity and maximum_desired_capacity are 
>>>>>> calculated to
>>>>>> accommodate already allocated sstuff,
>>>>>> aligned_bytes_to_allocate - extra size needed to allocate new 
>>>>>> memory (gc was
>>>>>> called when we failed to allocate this new memory).
>>>>>
>>>>> Right. Would it make sense to put this new addition around line 
>>>>> 1658 instead,
>>>>> before we start to adjust the values with respect to max_heap_size?
>>>>> /Jesper
>>>>>
>>>>>>
>>>>>> Alex
>>>>>>
>>>>>>
>>>>>> On 8/18/2015 12:15 PM, Jesper Wilhelmsson wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> +  minimum_desired_capacity += aligned_bytes_to_allocate;
>>>>>>> +  maximum_desired_capacity += aligned_bytes_to_allocate;
>>>>>>>
>>>>>>> Is it desired to always increase the desired capacity rather 
>>>>>>> than doing it
>>>>>>> only if it is too small? E.g.:
>>>>>>>
>>>>>>> minimum_desired_capacity = MAX2(minimum_desired_capacity,
>>>>>>> aligned_bytes_to_allocate);
>>>>>>> maximum_desired_capacity = MAX2(maximum_desired_capacity,
>>>>>>> aligned_bytes_to_allocate);
>>>>>>>
>>>>>>> Thanks,
>>>>>>> /Jesper
>>>>>>>
>>>>>>>
>>>>>>> Alexander Harlap skrev den 18/8/15 18:02:
>>>>>>>> This bug was caused by  too aggressive heap shrinkage in G1.
>>>>>>>>
>>>>>>>> JDK-8130265 <https://bugs.openjdk.java.net/browse/JDK-8130265>
>>>>>>>>
>>>>>>>> Proposed fix:
>>>>>>>> http://cr.openjdk.java.net/~aharlap/8130265/webrev.00
>>>>>>>> <http://cr.openjdk.java.net/%7Eaharlap/8130265/webrev.00>
>>>>>>>> Testing: JPRT, tonga
>>>>>>>>
>>>>>>>> Thank you,
>>>>>>>> Alex
>>>>>>>>
>>>>>>
>>>>
>>



From thomas.schatzl at oracle.com  Wed Aug 19 21:18:29 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 19 Aug 2015 23:18:29 +0200
Subject: RFR (M): 8067336: Allow that PLAB allocations at the end of
	regions are flexible
In-Reply-To: <55D34955.10006@oracle.com>
References: <1439896720.2316.7.camel@oracle.com> <55D34955.10006@oracle.com>
Message-ID: <1440019109.1994.51.camel@oracle.com>

Hi Tom,

On Tue, 2015-08-18 at 11:03 -0400, Tom Benson wrote:
> Hi Thomas,
> It looks like the actual plab allocation changes are missing from 
> g1Allocator.cpp - only an include was added...   ?
> 
> Aside from that, just a couple of minor comments.  I think the comment 
> at line 193-194 in g1AllocRegion.hpp should say "...maximum word size of 
> the allocation in desired_word_size" rather than "... in word_size".
> 
> In g1AllocRegion.inline.hpp, attempt_allocation and 
> attempt_allocation_locked both have an error path that includes:
> trace("alloc failed", *actual_word_size);
> 
> In the old version, the value handed to trace was the requested size, so 
> here it should probably be min_word_size, since *actual_word_size is 
> likely to be uninitialized.

  thanks for the review, even if it was for an unintentionally broken
change. I seem to have removed the actual change, the use of the new
methods in G1PLABAllocator::allocate_direct_or_new_plab(), while merging
these series of PLAB changes and the evac failure changes.

I also fixed the G1AllocRegion::trace() method, adding the new sizes
(desired/actual size) as needed.

Then I decided to move making G1OffsetTableContigSpace::_top volatile
(this is a _potential_ pre-existing bug - I am sure there is some weird
compiler that notices that it could hoist this access out of the loop in
G1OffsetTableContigSpace::par_allocate_impl()) into this CR. I am sure
you would have noticed anyway.

I think I also covered the other problems you mentioned. Thanks again.

CR:
https://bugs.openjdk.java.net/browse/JDK-8067336
Webrev:
http://cr.openjdk.java.net/~tschatzl/8067336/webrev.1 (full)
http://cr.openjdk.java.net/~tschatzl/8067336/webrev.0_to_1 (diff)
Testing:
jprt, vm.gc testlist, perf testing benchmarks, locally checked
allocation tracing (enabled by define only)

Thanks,
  Thomas



From jesper.wilhelmsson at oracle.com  Wed Aug 19 21:30:33 2015
From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson)
Date: Wed, 19 Aug 2015 23:30:33 +0200
Subject: Request for code review - 8130265: gctests/LargeObjects/large001
	fails with OutOfMemoryError: Java heap space
In-Reply-To: <55D4EEF5.9030504@oracle.com>
References: <55D356FB.1030603@oracle.com> <55D35A12.8020608@oracle.com>
	<55D35F1F.3090700@oracle.com> <55D36DC7.6010901@oracle.com>
	<55D38CBE.8070408@oracle.com> <55D394AD.6020104@oracle.com>
	<55D4E39D.5070805@oracle.com> <55D4E8C7.1080703@oracle.com>
	<55D4EEF5.9030504@oracle.com>
Message-ID: <55D4F579.5050501@oracle.com>

Looks good!
/Jesper

Alexander Harlap skrev den 19/8/15 23:02:
> I agree.
>
> Here is new revision:
>
> http://cr.openjdk.java.net/~aharlap/8130265/webrev.02/
> <http://cr.openjdk.java.net/%7Eaharlap/8130265/webrev.02/>
> Alex
>
> On 8/19/2015 4:36 PM, Jesper Wilhelmsson wrote:
>> Hi Alex,
>>
>> Yes, I like this version much better.
>> I'm fine with the change as is, but would it make sense to add the same code
>> after the first GC as well? So that every attempt_allocation_at_safepoint is
>> followed by an expand_and_allocate. We could avoid a GC in some cases.
>> /Jesper
>>
>> Alexander Harlap skrev den 19/8/15 22:14:
>>> Hi Jesper,
>>>
>>> I agree that touching code  in resize_if_necessary_after_full_collection(size_t
>>> word_size) is not pretty.
>>>
>>> Instead of doing  precise shrink, let it "over shrink" and expand after.
>>>
>>> Here is link to new version of change:
>>>
>>> http://cr.openjdk.java.net/~aharlap/8130265/webrev.01/
>>>
>>> Alex
>>>
>>> On 8/18/2015 4:25 PM, Jesper Wilhelmsson wrote:
>>>> Hi Alex,
>>>>
>>>> Alexander Harlap skrev den 18/8/15 21:51:
>>>>> Hi Jesper,
>>>>>
>>>>> I need whole HeapRegion(s) in aligned_bytes_to_allocate,
>>>>> so operations on doubles will result in rounding and will not work.
>>>>> If you care about overflowing size_t by doing += aligned_bytes_to_allocate, it
>>>>> can avoided with code like this:
>>>>>
>>>>> minimum_desired_capacity =  ((max_heap_size - minimum_desired_capacity) >
>>>>> aligned_bytes_to_allocate) ? minimum_desired_capacity +
>>>>> aligned_bytes_to_allocate : max_heap_size ;
>>>>>
>>>>> maximum_desired_capacity =  ((max_heap_size - maximum_desired_capacity) >
>>>>> aligned_bytes_to_allocate) ? maximum_desired_capacity +
>>>>> aligned_bytes_to_allocate : max_heap_size ;
>>>>
>>>> Yes, I think we need to do something like this. It seems odd to go through all
>>>> that trouble above to avoid overflowing and then just add a random size
>>>> afterwards.
>>>>
>>>> I never really formulated my original concern though. I may be wrong here so
>>>> feel free to interrupt at any point. The min and max values represent a range
>>>> of sizes that we consider desirable. By adding the new size to both min and
>>>> max we move that entire range. That seems wrong to me. If the size we need is
>>>> already below the max value, all we need to do is to add to the min value. Or
>>>> maybe we need to add a little to max, but not the entire size. Or is it the
>>>> case that the max value will always be too small?
>>>>
>>>> Thanks,
>>>> /Jesper
>>>>
>>>>>
>>>>>
>>>>> On 8/18/2015 1:39 PM, Jesper Wilhelmsson wrote:
>>>>>> Alexander Harlap skrev den 18/8/15 18:36:
>>>>>>> Hi Jesper,
>>>>>>>
>>>>>>> Here minimum_desired_capacity and maximum_desired_capacity are calculated to
>>>>>>> accommodate already allocated sstuff,
>>>>>>> aligned_bytes_to_allocate - extra size needed to allocate new memory (gc was
>>>>>>> called when we failed to allocate this new memory).
>>>>>>
>>>>>> Right. Would it make sense to put this new addition around line 1658 instead,
>>>>>> before we start to adjust the values with respect to max_heap_size?
>>>>>> /Jesper
>>>>>>
>>>>>>>
>>>>>>> Alex
>>>>>>>
>>>>>>>
>>>>>>> On 8/18/2015 12:15 PM, Jesper Wilhelmsson wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> +  minimum_desired_capacity += aligned_bytes_to_allocate;
>>>>>>>> +  maximum_desired_capacity += aligned_bytes_to_allocate;
>>>>>>>>
>>>>>>>> Is it desired to always increase the desired capacity rather than doing it
>>>>>>>> only if it is too small? E.g.:
>>>>>>>>
>>>>>>>> minimum_desired_capacity = MAX2(minimum_desired_capacity,
>>>>>>>> aligned_bytes_to_allocate);
>>>>>>>> maximum_desired_capacity = MAX2(maximum_desired_capacity,
>>>>>>>> aligned_bytes_to_allocate);
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> /Jesper
>>>>>>>>
>>>>>>>>
>>>>>>>> Alexander Harlap skrev den 18/8/15 18:02:
>>>>>>>>> This bug was caused by  too aggressive heap shrinkage in G1.
>>>>>>>>>
>>>>>>>>> JDK-8130265 <https://bugs.openjdk.java.net/browse/JDK-8130265>
>>>>>>>>>
>>>>>>>>> Proposed fix:
>>>>>>>>> http://cr.openjdk.java.net/~aharlap/8130265/webrev.00
>>>>>>>>> <http://cr.openjdk.java.net/%7Eaharlap/8130265/webrev.00>
>>>>>>>>> Testing: JPRT, tonga
>>>>>>>>>
>>>>>>>>> Thank you,
>>>>>>>>> Alex
>>>>>>>>>
>>>>>>>
>>>>>
>>>
>


From tom.benson at oracle.com  Wed Aug 19 22:19:38 2015
From: tom.benson at oracle.com (Tom Benson)
Date: Wed, 19 Aug 2015 18:19:38 -0400
Subject: RFR (M): 8067336: Allow that PLAB allocations at the end of
	regions are flexible
In-Reply-To: <1440019109.1994.51.camel@oracle.com>
References: <1439896720.2316.7.camel@oracle.com> <55D34955.10006@oracle.com>
	<1440019109.1994.51.camel@oracle.com>
Message-ID: <55D500FA.3070105@oracle.com>

Hi Thomas -
Looks good to me.   Though there was one odd thing I noticed, not in the 
changes but adjacent to one - this comment in g1Allocator.cpp:

  273     if (buf != NULL) {
  274       // Otherwise.
  275       alloc_buf->set_buf(buf, actual_plab_size);

There is another "Otherwise." comment just afterward at line 283, where 
it makes sense, but the one at 274 looks left over from some past 
restructuring.  Your call.  8^)
Tom

On 8/19/2015 5:18 PM, Thomas Schatzl wrote:
> Hi Tom,
>
> On Tue, 2015-08-18 at 11:03 -0400, Tom Benson wrote:
>> Hi Thomas,
>> It looks like the actual plab allocation changes are missing from
>> g1Allocator.cpp - only an include was added...   ?
>>
>> Aside from that, just a couple of minor comments.  I think the comment
>> at line 193-194 in g1AllocRegion.hpp should say "...maximum word size of
>> the allocation in desired_word_size" rather than "... in word_size".
>>
>> In g1AllocRegion.inline.hpp, attempt_allocation and
>> attempt_allocation_locked both have an error path that includes:
>> trace("alloc failed", *actual_word_size);
>>
>> In the old version, the value handed to trace was the requested size, so
>> here it should probably be min_word_size, since *actual_word_size is
>> likely to be uninitialized.
>    thanks for the review, even if it was for an unintentionally broken
> change. I seem to have removed the actual change, the use of the new
> methods in G1PLABAllocator::allocate_direct_or_new_plab(), while merging
> these series of PLAB changes and the evac failure changes.
>
> I also fixed the G1AllocRegion::trace() method, adding the new sizes
> (desired/actual size) as needed.
>
> Then I decided to move making G1OffsetTableContigSpace::_top volatile
> (this is a _potential_ pre-existing bug - I am sure there is some weird
> compiler that notices that it could hoist this access out of the loop in
> G1OffsetTableContigSpace::par_allocate_impl()) into this CR. I am sure
> you would have noticed anyway.
>
> I think I also covered the other problems you mentioned. Thanks again.
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8067336
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8067336/webrev.1 (full)
> http://cr.openjdk.java.net/~tschatzl/8067336/webrev.0_to_1 (diff)
> Testing:
> jprt, vm.gc testlist, perf testing benchmarks, locally checked
> allocation tracing (enabled by define only)
>
> Thanks,
>    Thomas
>



From kim.barrett at oracle.com  Wed Aug 19 22:43:54 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Wed, 19 Aug 2015 18:43:54 -0400
Subject: Request for code review - 8130265: gctests/LargeObjects/large001
	fails with OutOfMemoryError: Java heap space
In-Reply-To: <55D4EEF5.9030504@oracle.com>
References: <55D356FB.1030603@oracle.com> <55D35A12.8020608@oracle.com>
	<55D35F1F.3090700@oracle.com> <55D36DC7.6010901@oracle.com>
	<55D38CBE.8070408@oracle.com> <55D394AD.6020104@oracle.com>
	<55D4E39D.5070805@oracle.com> <55D4E8C7.1080703@oracle.com>
	<55D4EEF5.9030504@oracle.com>
Message-ID: <2C141666-BDBC-428C-9E40-4FB78F076192@oracle.com>

On Aug 19, 2015, at 5:02 PM, Alexander Harlap <alexander.harlap at oracle.com> wrote:
> 
> I agree.
> 
> Here is new revision:
> 
> http://cr.openjdk.java.net/~aharlap/8130265/webrev.02/
> <http://cr.openjdk.java.net/%7Eaharlap/8130265/webrev.02/>
> Alex
> 
> On 8/19/2015 4:36 PM, Jesper Wilhelmsson wrote:
>> Hi Alex,
>> 
>> Yes, I like this version much better.
>> I'm fine with the change as is, but would it make sense to add the same code after the first GC as well? So that every attempt_allocation_at_safepoint is followed by an expand_and_allocate. We could avoid a GC in some cases.
>> /Jesper

Looks good.

One very minor nit: Hotspot coding style guide says "As a general rule don't add bug numbers to comments.?

1778   // Expand heap will help - JDK-8130265



From thomas.schatzl at oracle.com  Thu Aug 20 14:06:12 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 20 Aug 2015 16:06:12 +0200
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <55CCF0DC.2000800@oracle.com>
References: <55C103A4.1060505@oracle.com>
	<A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>
	<1438781844.2378.60.camel@oracle.com>	<55C21DC3.5090305@oracle.com>
	<1438785532.2378.73.camel@oracle.com>	<55C248FF.6010308@oracle.com>
	<F73C4B9A-4DBA-47A8-8715-0988FF6D393D@oracle.com>
	<55C26249.1090902@oracle.com>
	<C149705F-696B-46CD-8FB8-CAD06C511244@oracle.com>
	<55C2751B.8070400@oracle.com>	<1438847329.2009.9.camel@oracle.com>
	<55C36B31.7000908@oracle.com>	<1438871723.2474.37.camel@oracle.com>
	<55C373F6.9060609@oracle.com>
	<AD45178E-A442-4A0D-80D6-50E91608E1FE@oracle.com>
	<55C3D8BB.9000603@oracle.com>
	<48C0E571-0D7B-4D30-85DF-E31D99092A7C@oracle.com>
	<30B0248C-3BEA-4FAE-861F-DAAE19F56B45@oracle.com>
	<55C4B806.7050504@oracle.com> <55C4C720.5030903@oracle.com>
	<55CA3430.3070300@oracle.com> <1439377259.2324.27.camel@oracle.com>
	<55CCF0DC.2000800@oracle.com>
Message-ID: <1440079572.2347.11.camel@oracle.com>

Hi Tom,

  sorry for the delay...

On Thu, 2015-08-13 at 15:32 -0400, Tom Benson wrote:
> Hi Thomas,
> 
> On 8/12/2015 7:00 AM, Thomas Schatzl wrote:
> > Hi,
> >
> > On Tue, 2015-08-11 at 13:43 -0400, Tom Benson wrote:
> >> Hi,
> >> On 8/7/2015 10:56 AM, Tom Benson wrote:
> >> After some discussion, I've changed the definition and name of
> >> free_archive_regions.  Now called dealloc_archive_regions, it uncommits
> >> the specified regions, unmapping the memory, rather than adding them to
> >> the free list.  This means the CDS code will no longer do the unmapping
> >> on verification failures.
> >>
> >> Updated full and incremental webrevs of the GC code are at:
> >> http://cr.openjdk.java.net/~tbenson/8131734/webrev.02/
> >> http://cr.openjdk.java.net/~tbenson/8131734/webrev.02.vs.01/
> >>
> >> Tested with JPRT and running benchmarks with the dealloc_ performed
> >> explicitly.  Jiangli also tested the original failing cases, and will be
> >> posting an updated webrev.
> > - is it possible that shrink_by() uses shrink_at()? This would avoid two
> > paths that uncommit regions like expand_by()/expand_at()?
> 
> OK, I made the change.  I didn't do it originally because the asserts I 
> wanted to add for the call from g1CollectedHeap seemed superfluous for 
> the other call, and shrink_at was so small.    Now shrink_at takes a 
> region count as well.
> 


> Updated full and incremental webrevs are at:
>    http://cr.openjdk.java.net/~tbenson/8131734/webrev.03/
>    http://cr.openjdk.java.net/~tbenson/8131734/webrev.03.vs.02/
> 
> 

Looks good.

> >
> > - I think the change should call at least HeapRegion::hr_clear() on the
> > region to remove or reset any auxiliary data structures, if not
> > G1CollectedHeap::free_region() (without adding the region to the free
> > list).
> > Since the HeapRegion* is not deallocated by the uncommit, this may cause
> > strange behavior later when the region is reused.
> 
> I don't think calling hr_clear should be necessary...  If it is, we 
> should be doing it in shrink_by as well, and I don't think we are. I 
> don't see how a HeapRegion can be 'reused' without having gone through 
> the constructor when expand_ asks (indirectly) for 'new HeapRegion', and 
> that does an hr_clear() as well as the rest of init.     Or am I missing 
> something there?

Leave it as is. I thought that a full gc (which is the only case where
the heap shrinks at the moment) also clears the remset of these regions
at least.

It should, I filed  JDK-8134048 for looking in this issue.

Looks good.

Thanks,
  Thomas




From tom.benson at oracle.com  Thu Aug 20 14:12:24 2015
From: tom.benson at oracle.com (Tom Benson)
Date: Thu, 20 Aug 2015 10:12:24 -0400
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <1440079572.2347.11.camel@oracle.com>
References: <55C103A4.1060505@oracle.com>			<A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>			<1438781844.2378.60.camel@oracle.com>	<55C21DC3.5090305@oracle.com>			<1438785532.2378.73.camel@oracle.com>	<55C248FF.6010308@oracle.com>			<F73C4B9A-4DBA-47A8-8715-0988FF6D393D@oracle.com>			<55C26249.1090902@oracle.com>			<C149705F-696B-46CD-8FB8-CAD06C511244@oracle.com>			<55C2751B.8070400@oracle.com>	<1438847329.2009.9.camel@oracle.com>			<55C36B31.7000908@oracle.com>	<1438871723.2474.37.camel@oracle.com>			<55C373F6.9060609@oracle.com>			<AD45178E-A442-4A0D-80D6-50E91608E1FE@oracle.com>			<55C3D8BB.9000603@oracle.com>			<48C0E571-0D7B-4D30-85DF-E31D99092A7C@oracle.com>			<30B0248C-3BEA-4FAE-861F-DAAE19F56B45@oracle.com>			<55C4B806.7050504@oracle.com>
	<55C4C720.5030903@oracle.com>		 <55CA3430.3070300@oracle.com>
	<1439377259.2324.27.camel@oracle.com>	
	<55CCF0DC.2000800@oracle.com> <1440079572.2347.11.camel@oracle.com>
Message-ID: <55D5E048.5070404@oracle.com>

Hi Thomas,
OK, thanks!
Tom

On 8/20/2015 10:06 AM, Thomas Schatzl wrote:
> Hi Tom,
>
>    sorry for the delay...
>
> On Thu, 2015-08-13 at 15:32 -0400, Tom Benson wrote:
>> Hi Thomas,
>>
>> On 8/12/2015 7:00 AM, Thomas Schatzl wrote:
>>> Hi,
>>>
>>> On Tue, 2015-08-11 at 13:43 -0400, Tom Benson wrote:
>>>> Hi,
>>>> On 8/7/2015 10:56 AM, Tom Benson wrote:
>>>> After some discussion, I've changed the definition and name of
>>>> free_archive_regions.  Now called dealloc_archive_regions, it uncommits
>>>> the specified regions, unmapping the memory, rather than adding them to
>>>> the free list.  This means the CDS code will no longer do the unmapping
>>>> on verification failures.
>>>>
>>>> Updated full and incremental webrevs of the GC code are at:
>>>> http://cr.openjdk.java.net/~tbenson/8131734/webrev.02/
>>>> http://cr.openjdk.java.net/~tbenson/8131734/webrev.02.vs.01/
>>>>
>>>> Tested with JPRT and running benchmarks with the dealloc_ performed
>>>> explicitly.  Jiangli also tested the original failing cases, and will be
>>>> posting an updated webrev.
>>> - is it possible that shrink_by() uses shrink_at()? This would avoid two
>>> paths that uncommit regions like expand_by()/expand_at()?
>> OK, I made the change.  I didn't do it originally because the asserts I
>> wanted to add for the call from g1CollectedHeap seemed superfluous for
>> the other call, and shrink_at was so small.    Now shrink_at takes a
>> region count as well.
>>
>
>> Updated full and incremental webrevs are at:
>>     http://cr.openjdk.java.net/~tbenson/8131734/webrev.03/
>>     http://cr.openjdk.java.net/~tbenson/8131734/webrev.03.vs.02/
>>
>>
> Looks good.
>
>>> - I think the change should call at least HeapRegion::hr_clear() on the
>>> region to remove or reset any auxiliary data structures, if not
>>> G1CollectedHeap::free_region() (without adding the region to the free
>>> list).
>>> Since the HeapRegion* is not deallocated by the uncommit, this may cause
>>> strange behavior later when the region is reused.
>> I don't think calling hr_clear should be necessary...  If it is, we
>> should be doing it in shrink_by as well, and I don't think we are. I
>> don't see how a HeapRegion can be 'reused' without having gone through
>> the constructor when expand_ asks (indirectly) for 'new HeapRegion', and
>> that does an hr_clear() as well as the rest of init.     Or am I missing
>> something there?
> Leave it as is. I thought that a full gc (which is the only case where
> the heap shrinks at the moment) also clears the remset of these regions
> at least.
>
> It should, I filed  JDK-8134048 for looking in this issue.
>
> Looks good.
>
> Thanks,
>    Thomas
>
>



From kim.barrett at oracle.com  Thu Aug 20 15:00:54 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Thu, 20 Aug 2015 11:00:54 -0400
Subject: RFR: 8131330: G1CollectedHeap::verify_dirty_young_list fails with
	assert
In-Reply-To: <08169C1E-696E-44C1-AA98-3F9AA2895197@oracle.com>
References: <5D7EB2C6-6AC4-4B87-8FD1-FC487328087C@oracle.com>
	<55D4C22F.2000003@oracle.com>
	<08169C1E-696E-44C1-AA98-3F9AA2895197@oracle.com>
Message-ID: <AEE51CEF-0B38-4A43-BB8D-F51FEC74B9B8@oracle.com>

On Aug 19, 2015, at 4:48 PM, Kim Barrett <kim.barrett at oracle.com> wrote:
> 
> New webrev:
> http://cr.openjdk.java.net/~kbarrett/8131330/webrev.01/
> 
> Incremental webrev:
> http://cr.openjdk.java.net/~kbarrett/8131330/webrev.01.inc1/

So why isn't the fill_subword helper function declared inline, you
might ask?

Because during development it made sense to have it not inlined, and I
forgot to reconsider that later.  Sigh!  I'll fix that and post a new
webrev this afternoon.



From thomas.schatzl at oracle.com  Thu Aug 20 15:01:34 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 20 Aug 2015 17:01:34 +0200
Subject: Request for code review - 8130265:
	gctests/LargeObjects/large001 fails with OutOfMemoryError: Java heap
	space
In-Reply-To: <2C141666-BDBC-428C-9E40-4FB78F076192@oracle.com>
References: <55D356FB.1030603@oracle.com> <55D35A12.8020608@oracle.com>
	<55D35F1F.3090700@oracle.com> <55D36DC7.6010901@oracle.com>
	<55D38CBE.8070408@oracle.com> <55D394AD.6020104@oracle.com>
	<55D4E39D.5070805@oracle.com> <55D4E8C7.1080703@oracle.com>
	<55D4EEF5.9030504@oracle.com>
	<2C141666-BDBC-428C-9E40-4FB78F076192@oracle.com>
Message-ID: <1440082894.2347.14.camel@oracle.com>

Hi Alex,

On Wed, 2015-08-19 at 18:43 -0400, Kim Barrett wrote:
> On Aug 19, 2015, at 5:02 PM, Alexander Harlap <alexander.harlap at oracle.com> wrote:
> > 
> > I agree.
> > 
> > Here is new revision:
> > 
> > http://cr.openjdk.java.net/~aharlap/8130265/webrev.02/
> > <http://cr.openjdk.java.net/%7Eaharlap/8130265/webrev.02/>

  would it be possible to fix the code duplication between lines
1759-1784 and 1785 and 1812? The code seems to be completely the same
except the different flag value passed to the Full GC call and the
assert about that should_clear_all_soft_refs().

Otherwise it looks good.

Thanks,
  Thomas





From thomas.schatzl at oracle.com  Thu Aug 20 15:20:19 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 20 Aug 2015 17:20:19 +0200
Subject: RFR: 8131330: G1CollectedHeap::verify_dirty_young_list fails
	with assert
In-Reply-To: <AEE51CEF-0B38-4A43-BB8D-F51FEC74B9B8@oracle.com>
References: <5D7EB2C6-6AC4-4B87-8FD1-FC487328087C@oracle.com>
	<55D4C22F.2000003@oracle.com>
	<08169C1E-696E-44C1-AA98-3F9AA2895197@oracle.com>
	<AEE51CEF-0B38-4A43-BB8D-F51FEC74B9B8@oracle.com>
Message-ID: <1440084019.2347.20.camel@oracle.com>

Hi,

On Thu, 2015-08-20 at 11:00 -0400, Kim Barrett wrote:
> On Aug 19, 2015, at 4:48 PM, Kim Barrett <kim.barrett at oracle.com> wrote:
> > 
> > New webrev:
> > http://cr.openjdk.java.net/~kbarrett/8131330/webrev.01/
> > 
> > Incremental webrev:
> > http://cr.openjdk.java.net/~kbarrett/8131330/webrev.01.inc1/
> 
> So why isn't the fill_subword helper function declared inline, you
> might ask?
> 
> Because during development it made sense to have it not inlined, and I
> forgot to reconsider that later.  Sigh!  I'll fix that and post a new
> webrev this afternoon.
> 

some tiny comments:

- blockOffsetTable.hpp, fill_range(): not sure why this code suddenly
requires an assert that we are not currently running UseG1GC. The change
may keep it, it's just confusing because if we are in that code when
using G1, I doubt the code will get to this position.

- This is just how I would comment assembly, but I would prefer if the
"//" next to the code were aligned throughout a complete block.

Otherwise looks good. Feel free to ignore my suggestions.

Thanks,
  Thomas




From kirill.zhaldybin at oracle.com  Thu Aug 20 16:45:44 2015
From: kirill.zhaldybin at oracle.com (Kirill Zhaldybin)
Date: Thu, 20 Aug 2015 19:45:44 +0300
Subject: RFR (S): 8132708: Add tests for Humongous objects allocation threshold
Message-ID: <55D60438.4080004@oracle.com>

Hi!

Could you please review a new simple test checking that only objects 
allocating more than  a half regions are allocated as humongous?

CR:
https://bugs.openjdk.java.net/browse/JDK-8132708

Webrev:
hotspot:
http://cr.openjdk.java.net/~kzhaldyb/webrevs/JDK-8132708/webrev.03/

Tests:
jprt

Thank you.

Regards, Kirill


From alexander.harlap at oracle.com  Thu Aug 20 19:41:43 2015
From: alexander.harlap at oracle.com (Alexander Harlap)
Date: Thu, 20 Aug 2015 15:41:43 -0400
Subject: Request for code review - 8130265: gctests/LargeObjects/large001
	fails with OutOfMemoryError: Java heap space
In-Reply-To: <1440082894.2347.14.camel@oracle.com>
References: <55D356FB.1030603@oracle.com> <55D35A12.8020608@oracle.com>
	<55D35F1F.3090700@oracle.com> <55D36DC7.6010901@oracle.com>
	<55D38CBE.8070408@oracle.com> <55D394AD.6020104@oracle.com>
	<55D4E39D.5070805@oracle.com> <55D4E8C7.1080703@oracle.com>
	<55D4EEF5.9030504@oracle.com>
	<2C141666-BDBC-428C-9E40-4FB78F076192@oracle.com>
	<1440082894.2347.14.camel@oracle.com>
Message-ID: <55D62D77.8020504@oracle.com>

Kim and Thomas,

Here is new version with your suggestions in:

http://cr.openjdk.java.net/~aharlap/8130265/webrev.03/
<http://cr.openjdk.java.net/%7Eaharlap/8130265/webrev.03/>
Alex

On 8/20/2015 11:01 AM, Thomas Schatzl wrote:
> Hi Alex,
>
> On Wed, 2015-08-19 at 18:43 -0400, Kim Barrett wrote:
>> On Aug 19, 2015, at 5:02 PM, Alexander Harlap <alexander.harlap at oracle.com> wrote:
>>> I agree.
>>>
>>> Here is new revision:
>>>
>>> http://cr.openjdk.java.net/~aharlap/8130265/webrev.02/
>>> <http://cr.openjdk.java.net/%7Eaharlap/8130265/webrev.02/>
>    would it be possible to fix the code duplication between lines
> 1759-1784 and 1785 and 1812? The code seems to be completely the same
> except the different flag value passed to the Full GC call and the
> assert about that should_clear_all_soft_refs().
>
> Otherwise it looks good.
>
> Thanks,
>    Thomas
>
>
>



From kim.barrett at oracle.com  Thu Aug 20 19:45:41 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Thu, 20 Aug 2015 15:45:41 -0400
Subject: RFR: 8131330: G1CollectedHeap::verify_dirty_young_list fails with
	assert
In-Reply-To: <1440084019.2347.20.camel@oracle.com>
References: <5D7EB2C6-6AC4-4B87-8FD1-FC487328087C@oracle.com>
	<55D4C22F.2000003@oracle.com>
	<08169C1E-696E-44C1-AA98-3F9AA2895197@oracle.com>
	<AEE51CEF-0B38-4A43-BB8D-F51FEC74B9B8@oracle.com>
	<1440084019.2347.20.camel@oracle.com>
Message-ID: <0720CAE2-8CE2-4EDF-B72F-E8CB132DDEA2@oracle.com>

On Aug 20, 2015, at 11:20 AM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
> 
> Hi,
> 
> On Thu, 2015-08-20 at 11:00 -0400, Kim Barrett wrote:
>> On Aug 19, 2015, at 4:48 PM, Kim Barrett <kim.barrett at oracle.com> wrote:
>>> 
>>> New webrev:
>>> http://cr.openjdk.java.net/~kbarrett/8131330/webrev.01/
>>> 
>>> Incremental webrev:
>>> http://cr.openjdk.java.net/~kbarrett/8131330/webrev.01.inc1/
>> 
>> So why isn't the fill_subword helper function declared inline, you
>> might ask?
>> 
>> Because during development it made sense to have it not inlined, and I
>> forgot to reconsider that later.  Sigh!  I'll fix that and post a new
>> webrev this afternoon.
>> 
> 
> some tiny comments:
> 
> - blockOffsetTable.hpp, fill_range(): not sure why this code suddenly
> requires an assert that we are not currently running UseG1GC. The change
> may keep it, it's just confusing because if we are in that code when
> using G1, I doubt the code will get to this position.

We?re deciding whether to use the new memset_with_concurrent_readers,
depending on whether we?re running a concurrent collector.  We could

(1) Test for UseG1GC in the same manner as UseConcMarkSweepGC.
(2) Assert !UseG1GC, because G1 doesn?t use this code.
(3) Ignore UseG1GC and hope G1 continues to not use this code.

I chose (2).

> - This is just how I would comment assembly, but I would prefer if the
> "//" next to the code were aligned throughout a complete block.

I got tired of arguing with emacs auto-placement of comments and didn?t
want to mess with the default comment indent, which is how it came to where
it is.

> Otherwise looks good. Feel free to ignore my suggestions.

Thanks for looking at this.



From jiangli.zhou at oracle.com  Thu Aug 20 19:55:21 2015
From: jiangli.zhou at oracle.com (Jiangli Zhou)
Date: Thu, 20 Aug 2015 12:55:21 -0700
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <55D5E048.5070404@oracle.com>
References: <55C103A4.1060505@oracle.com>			<A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>			<1438781844.2378.60.camel@oracle.com>	<55C21DC3.5090305@oracle.com>			<1438785532.2378.73.camel@oracle.com>	<55C248FF.6010308@oracle.com>			<F73C4B9A-4DBA-47A8-8715-0988FF6D393D@oracle.com>			<55C26249.1090902@oracle.com>			<C149705F-696B-46CD-8FB8-CAD06C511244@oracle.com>			<55C2751B.8070400@oracle.com>	<1438847329.2009.9.camel@oracle.com>			<55C36B31.7000908@oracle.com>	<1438871723.2474.37.camel@oracle.com>			<55C373F6.9060609@oracle.com>			<AD45178E-A442-4A0D-80D6-50E91608E1FE@oracle.com>			<55C3D8BB.9000603@oracle.com>			<48C0E571-0D7B-4D30-85DF-E31D99092A7C@oracle.com>			<30B0248C-3BEA-4FAE-861F-DAAE19F56B45@oracle.com>			<55C4B806.7050504@oracle.com>
	<55C4C720.5030903@oracle.com>		 <55CA3430.3070300@oracle.com>
	<1439377259.2324.27.camel@oracle.com>	
	<55CCF0DC.2000800@oracle.com> <1440079572.2347.11.camel@oracle.com>
	<55D5E048.5070404@oracle.com>
Message-ID: <8566C008-5A60-4376-94C4-968E1A475769@oracle.com>

Hi Dmitry,

Here is the updated runtime webrev that reflects Tom?s latest GC changes.

http://cr.openjdk.java.net/~jiangli/8131734/webrev.01/

I renamed the FileMapInfo::unmap_string_regions() to FileMapInfo::dealloc_string_regions(), which only deallocates the archived string region from the java heap without unmapping. The unmapping is handled by the GC system as the archived string region is part of the java heap. I also added dealloc_string_regions() call to the case where the string region verification fails.

Thanks,
Jiangli

On Aug 20, 2015, at 7:12 AM, Tom Benson <tom.benson at oracle.com> wrote:

> Hi Thomas,
> OK, thanks!
> Tom
> 
> On 8/20/2015 10:06 AM, Thomas Schatzl wrote:
>> Hi Tom,
>> 
>>   sorry for the delay...
>> 
>> On Thu, 2015-08-13 at 15:32 -0400, Tom Benson wrote:
>>> Hi Thomas,
>>> 
>>> On 8/12/2015 7:00 AM, Thomas Schatzl wrote:
>>>> Hi,
>>>> 
>>>> On Tue, 2015-08-11 at 13:43 -0400, Tom Benson wrote:
>>>>> Hi,
>>>>> On 8/7/2015 10:56 AM, Tom Benson wrote:
>>>>> After some discussion, I've changed the definition and name of
>>>>> free_archive_regions.  Now called dealloc_archive_regions, it uncommits
>>>>> the specified regions, unmapping the memory, rather than adding them to
>>>>> the free list.  This means the CDS code will no longer do the unmapping
>>>>> on verification failures.
>>>>> 
>>>>> Updated full and incremental webrevs of the GC code are at:
>>>>> http://cr.openjdk.java.net/~tbenson/8131734/webrev.02/
>>>>> http://cr.openjdk.java.net/~tbenson/8131734/webrev.02.vs.01/
>>>>> 
>>>>> Tested with JPRT and running benchmarks with the dealloc_ performed
>>>>> explicitly.  Jiangli also tested the original failing cases, and will be
>>>>> posting an updated webrev.
>>>> - is it possible that shrink_by() uses shrink_at()? This would avoid two
>>>> paths that uncommit regions like expand_by()/expand_at()?
>>> OK, I made the change.  I didn't do it originally because the asserts I
>>> wanted to add for the call from g1CollectedHeap seemed superfluous for
>>> the other call, and shrink_at was so small.    Now shrink_at takes a
>>> region count as well.
>>> 
>> 
>>> Updated full and incremental webrevs are at:
>>>    http://cr.openjdk.java.net/~tbenson/8131734/webrev.03/
>>>    http://cr.openjdk.java.net/~tbenson/8131734/webrev.03.vs.02/
>>> 
>>> 
>> Looks good.
>> 
>>>> - I think the change should call at least HeapRegion::hr_clear() on the
>>>> region to remove or reset any auxiliary data structures, if not
>>>> G1CollectedHeap::free_region() (without adding the region to the free
>>>> list).
>>>> Since the HeapRegion* is not deallocated by the uncommit, this may cause
>>>> strange behavior later when the region is reused.
>>> I don't think calling hr_clear should be necessary...  If it is, we
>>> should be doing it in shrink_by as well, and I don't think we are. I
>>> don't see how a HeapRegion can be 'reused' without having gone through
>>> the constructor when expand_ asks (indirectly) for 'new HeapRegion', and
>>> that does an hr_clear() as well as the rest of init.     Or am I missing
>>> something there?
>> Leave it as is. I thought that a full gc (which is the only case where
>> the heap shrinks at the moment) also clears the remset of these regions
>> at least.
>> 
>> It should, I filed  JDK-8134048 for looking in this issue.
>> 
>> Looks good.
>> 
>> Thanks,
>>   Thomas
>> 
>> 
> 



From kim.barrett at oracle.com  Thu Aug 20 20:29:46 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Thu, 20 Aug 2015 16:29:46 -0400
Subject: RFR: 8131330: G1CollectedHeap::verify_dirty_young_list fails with
	assert
In-Reply-To: <AEE51CEF-0B38-4A43-BB8D-F51FEC74B9B8@oracle.com>
References: <5D7EB2C6-6AC4-4B87-8FD1-FC487328087C@oracle.com>
	<55D4C22F.2000003@oracle.com>
	<08169C1E-696E-44C1-AA98-3F9AA2895197@oracle.com>
	<AEE51CEF-0B38-4A43-BB8D-F51FEC74B9B8@oracle.com>
Message-ID: <B8713224-3063-4BDB-9FAA-7ECB5F9AF0DA@oracle.com>

On Aug 20, 2015, at 11:00 AM, Kim Barrett <kim.barrett at oracle.com> wrote:
> 
> So why isn't the fill_subword helper function declared inline, you
> might ask?
> 
> Because during development it made sense to have it not inlined, and I
> forgot to reconsider that later.  Sigh!  I'll fix that and post a new
> webrev this afternoon.

Here's the new version.  I moved the fill_subword helper before its
caller and changed it to inline rather than static.

That change provoked a compiler warning that reminded me that
temporaries for inline assembler should be declared in the outputs and
then just left unreferenced after the asm statement.  I'd mistakenly
had them in the inputs.  Fixed now.

Full webrev:
http://cr.openjdk.java.net/~kbarrett/8131330/webrev.02/

Incremental webrev:
http://cr.openjdk.java.net/~kbarrett/8131330/webrev.02.inc2/



From thomas.schatzl at oracle.com  Fri Aug 21 08:05:37 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Fri, 21 Aug 2015 10:05:37 +0200
Subject: RFR (XXS): 8133456: HeapRegionManager::shrink_by() iterates
	suboptimally across regions
Message-ID: <1440144337.2233.5.camel@oracle.com>

Hi all,

  can I have reviews for this one-line change that improves efficiency
during heap shrinking?

The problem was that the increment to skip over already scanned regions
has been too conservative. In some cases it would mean that regions in
the region table would be scanned twice.

CR:
https://bugs.openjdk.java.net/browse/JDK-8133456
Webrev:
http://cr.openjdk.java.net/~tschatzl/8133456/webrev/
Testing:
jprt, vm.gc 

Thanks,
  Thomas



From bengt.rutisson at oracle.com  Fri Aug 21 08:27:30 2015
From: bengt.rutisson at oracle.com (Bengt Rutisson)
Date: Fri, 21 Aug 2015 10:27:30 +0200
Subject: RFR (XXS): 8133456: HeapRegionManager::shrink_by() iterates
	suboptimally across regions
In-Reply-To: <1440144337.2233.5.camel@oracle.com>
References: <1440144337.2233.5.camel@oracle.com>
Message-ID: <55D6E0F2.5030401@oracle.com>


Hi Thomas,

On 2015-08-21 10:05, Thomas Schatzl wrote:
> Hi all,
>
>    can I have reviews for this one-line change that improves efficiency
> during heap shrinking?
>
> The problem was that the increment to skip over already scanned regions
> has been too conservative. In some cases it would mean that regions in
> the region table would be scanned twice.
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8133456
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8133456/webrev/

The fix looks good, but do we really need both cur and idx_last_found? 
After your fix we just set them to the same value. Technically it would 
be enough that find_empty_from_idx_reverse() takes one parameter that is 
both the in and out value. But it may be cleaner to have the two separated.

I'll leave it up to you if you want to change it. I'm fine with the 
proposed patch too.

Thanks,
Bengt

> Testing:
> jprt, vm.gc
>
> Thanks,
>    Thomas
>



From thomas.schatzl at oracle.com  Fri Aug 21 08:38:42 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Fri, 21 Aug 2015 10:38:42 +0200
Subject: RFR (XXS): 8133456: HeapRegionManager::shrink_by() iterates
	suboptimally across regions
In-Reply-To: <55D6E0F2.5030401@oracle.com>
References: <1440144337.2233.5.camel@oracle.com> <55D6E0F2.5030401@oracle.com>
Message-ID: <1440146322.2233.12.camel@oracle.com>

Hi,

On Fri, 2015-08-21 at 10:27 +0200, Bengt Rutisson wrote:
> Hi Thomas,
> 
> On 2015-08-21 10:05, Thomas Schatzl wrote:
> > Hi all,
> >
> >    can I have reviews for this one-line change that improves efficiency
> > during heap shrinking?
> >
> > The problem was that the increment to skip over already scanned regions
> > has been too conservative. In some cases it would mean that regions in
> > the region table would be scanned twice.
> >
> > CR:
> > https://bugs.openjdk.java.net/browse/JDK-8133456
> > Webrev:
> > http://cr.openjdk.java.net/~tschatzl/8133456/webrev/
> 
> The fix looks good, but do we really need both cur and idx_last_found? 
> After your fix we just set them to the same value. Technically it would 
> be enough that find_empty_from_idx_reverse() takes one parameter that is 
> both the in and out value. But it may be cleaner to have the two separated.
> 
> I'll leave it up to you if you want to change it. I'm fine with the 
> proposed patch too.
> 

  I would be fine with either way too, but I have received some review
comments in the past (recently about 8067336 where originally I had one
in and one inout parameter instead of three parameters (in/in/out) for
all methods) to avoid inout parameters since it is easier to understand.

Let's get another opinion.

Thanks,
  Thomas




From tom.benson at oracle.com  Fri Aug 21 12:03:41 2015
From: tom.benson at oracle.com (Tom Benson)
Date: Fri, 21 Aug 2015 08:03:41 -0400
Subject: RFR (XXS): 8133456: HeapRegionManager::shrink_by() iterates
	suboptimally across regions
In-Reply-To: <1440146322.2233.12.camel@oracle.com>
References: <1440144337.2233.5.camel@oracle.com> <55D6E0F2.5030401@oracle.com>
	<1440146322.2233.12.camel@oracle.com>
Message-ID: <55D7139D.10500@oracle.com>

Hi,
The change looks good to me.

On the in/out parameter question, I'd say I slightly favor the two 
parameters.   On a related trivial note, I'd also prefer it slightly if 
find_available_from_idx_reverse had the index as return value, with the 
count as the output parameter.  Same for find_unavailable_from_idx.  
Then they would be consistent with find_contiguous and 
find_highest_free, and there wouldn't be 2 index params to look odd.   
But that's another change.

Tom

On 8/21/2015 4:38 AM, Thomas Schatzl wrote:
> Hi,
>
> On Fri, 2015-08-21 at 10:27 +0200, Bengt Rutisson wrote:
>> Hi Thomas,
>>
>> On 2015-08-21 10:05, Thomas Schatzl wrote:
>>> Hi all,
>>>
>>>     can I have reviews for this one-line change that improves efficiency
>>> during heap shrinking?
>>>
>>> The problem was that the increment to skip over already scanned regions
>>> has been too conservative. In some cases it would mean that regions in
>>> the region table would be scanned twice.
>>>
>>> CR:
>>> https://bugs.openjdk.java.net/browse/JDK-8133456
>>> Webrev:
>>> http://cr.openjdk.java.net/~tschatzl/8133456/webrev/
>> The fix looks good, but do we really need both cur and idx_last_found?
>> After your fix we just set them to the same value. Technically it would
>> be enough that find_empty_from_idx_reverse() takes one parameter that is
>> both the in and out value. But it may be cleaner to have the two separated.
>>
>> I'll leave it up to you if you want to change it. I'm fine with the
>> proposed patch too.
>>
>    I would be fine with either way too, but I have received some review
> comments in the past (recently about 8067336 where originally I had one
> in and one inout parameter instead of three parameters (in/in/out) for
> all methods) to avoid inout parameters since it is easier to understand.
>
> Let's get another opinion.
>
> Thanks,
>    Thomas
>
>



From stefan.johansson at oracle.com  Fri Aug 21 12:04:29 2015
From: stefan.johansson at oracle.com (Stefan Johansson)
Date: Fri, 21 Aug 2015 14:04:29 +0200
Subject: RFR: 8129417: Oop iteration clean-up to remove
	oop_ms_follow_contents
In-Reply-To: <55D312FF.3070105@oracle.com>
References: <5588217F.9090201@oracle.com>
	<20150706151537.GP3266@ehelin.jrpg.bea.com>
	<559BCD11.3010801@oracle.com> <55D312FF.3070105@oracle.com>
Message-ID: <55D713CD.3080305@oracle.com>

Hi Per,

On 2015-08-18 13:11, Per Liden wrote:
> Hi Stefan,
>
> On 2015-07-07 14:58, Stefan Johansson wrote:
>> Thanks Erik for reviewing,
>>
>> New webrevs:
>> Part 2 - full:
>> http://cr.openjdk.java.net/~sjohanss/8129417/2-add-oop_iterate_size/hotspot.01/ 
>>
>>
>> Part 2 - inc:
>> http://cr.openjdk.java.net/~sjohanss/8129417/2-add-oop_iterate_size/hotspot.00-01/ 
>>
>>
>>
>> Part 3 - full:
>> http://cr.openjdk.java.net/~sjohanss/8129417/3-re-structure-mark-sweep/hotspot.02/ 
>>
>>
>> Part 3 - inc:
>> http://cr.openjdk.java.net/~sjohanss/8129417/3-re-structure-mark-sweep/hotspot.00-02/ 
>>
>>
>> Also includes an updated assert, which was missed in previous webrev.
>>
>> Comments inline.
>
> Just one thing that looks odd to me, but maybe you can clarify. 
> Shouldn't MarkSweep::follow_object() call follow_array() instead of 
> calling push_objarray()? That way the change made to 
> ObjArrayTask::is_valid() wouldn't be needed.
>
Thanks for looking at this again. I agree that your proposed change 
would make the code nicer and I think I had it structured that way once 
during this work but had to revert it back to this to keep performance.

I went ahead and did the changes and some new measurements, and this 
change leads to a small regression for some platforms. I haven't done 
any deep analysis but I guess it leads to worse inlining decisions on 
these platforms.

I prefer to leave the code as is. The other possible solution would be 
to do further analysis and give even more special inlining hints to the 
different compilers.

Thanks,
Stefan

> /Per
>
>>
>> On 2015-07-06 17:15, Erik Helin wrote:
>>> Hi Stefan,
>>>
>>> thanks a lot for taking on this work. Please see comments inline.
>>>
>>> On 2015-06-22, Stefan Johansson wrote:
>>>> Hi,
>>>>
>>>> Please review these changes for RFE:
>>>> https://bugs.openjdk.java.net/browse/JDK-8129417
>>>>
>>>> Webrev for the full change:
>>>> http://cr.openjdk.java.net/~sjohanss/8129417/hotspot.00/
>>>>
>>>> Summary:
>>>> To allow further cleanups and later remove G1 specific code from
>>>> mark-sweep,
>>>> we want to remove the mark-sweep specific visitor
>>>> oop_ms_follow_contents.
>>>> Doing this cleanup has proven to be a little more complicated than I
>>>> first
>>>> anticipated and to make the change easier to review I've split it
>>>> into four
>>>> different parts.
>>>>
>>>> Part 1 - removing oop_ms_follow_contents:
>>>> http://cr.openjdk.java.net/~sjohanss/8129417/1-remove-oop_ms_follow_contents/hotspot.00/ 
>>>>
>>>>
>>> - Looking in specialized_oop_closures.hpp, the comment
>>>     71 // This is split into several because of a Visual C++ 6.0
>>> compiler bug
>>>     72 // where very long macros cause the compiler to crash
>>>    seems a bit dated, given that Visual C++ 6.0 was released in 1998. I
>>>    think we should try to merge ALL_OOP_OOP_ITERATE_CLOSURES_1 and 2 
>>> (and
>>>    their corresponding macros). Do you think like doing that in this
>>>    patch or do you want to file a follow-up bug?
>> I would like to do/investigate this as a separate RFE, I have a list of
>> things that should follow this clean up and I plan to file those before
>> pushing this change.
>>
>>>
>>> - The part of the patch that changes instanceMirrorKlass.inline.hpp
>>>    might impact more GCs than just any of the marksweep ones. Have you
>>>    seen any performance improvements/regressions with G1 or CMS?
>> No, but I have not done as many runs and done as much analysis of this
>> as I've done on the serial full gc. I've not seen any regression in
>> aurora.se performance runs.
>>>> Part 2 - introducing oop_iterate_size:
>>>> http://cr.openjdk.java.net/~sjohanss/8129417/2-add-oop_iterate_size/hotspot.00/ 
>>>>
>>>>
>>> - In arrayKlass.hpp, why add a space to:
>>>    +#define OOP_OOP_ITERATE_DECL_NO_BACKWARDS(OopClosureType,
>>> nv_suffix)         \
>> I've followed '\' alignment "rule" for these macros, which seems to be
>> two spaces after the longest line. I just realized that I should also
>> update OOP_OOP_ITERATE_DECL_RANGE with one extra space to be consistent.
>>
>>> - In arrayKlass.hpp, the following lines seems aligned strangely:
>>>    +void KlassType::oop_oop_iterate_range##nv_suffix(oop obj,
>>> OopClosureType* closure, int start, int end) {  \
>>>    +  oop_oop_iterate_range<nvs_to_bool(nv_suffix)>(obj, closure,
>>> start, end);                               \
>> Nice catch, missed fixing the alignment when making it void.
>>
>>> - In oop.inline.hpp, why must the size be computed before applying the
>>>    closure? In the world of perm gen, this might have been important in
>>>    order to not follow stale klass pointers, but with Metaspace the
>>> Klass*
>>>    won't move.
>> As we've discussed offline, this is most likely safe. I've done an
>> ad-hoc RBT run with some random testing when I assert on the size being
>> the same both before and after the call to oop_oop_iterate.
>>
>> I would like to do this change as a separate change as well, so it's
>> been added to my list of RFEs to file.
>>
>>>
>>> - Can we remove the method MutableSpace::oop_iterate?
>> Looks like it, no one seems to be using it. I'll remove it instead of
>> updating it to use oop_iterate_size().
>>
>>>
>>>> Part 3 - restructure mark-sweep:
>>>> http://cr.openjdk.java.net/~sjohanss/8129417/3-re-structure-mark-sweep/hotspot.00/ 
>>>>
>>>>
>>> - Can you please remove the commented assert in mark_and_push:
>>>     +//  assert(Universe::heap()->is_in_reserved(p), "should be in
>>> object space");
>>>    or should it be enabled again?
>> Removed.
>>
>>> - I agree with Per about moving the check for object arrays into
>>>    follow_object.
>> Already done, but gives a slight regression. I still think we can live
>> with this small regression for now, to avoid people doing wrong and call
>> follow_object on objArrays.
>>
>>>> Part 4 - compiler hints for inlining:
>>>> http://cr.openjdk.java.net/~sjohanss/8129417/4-compiler-hints-for-inlining/hotspot.00/ 
>>>>
>>>>
>>> - I would like to see the definiton of FORCE_INLINE in the compiler
>>>    specific globalDefinitions file. Then the code in
>>>    instanceKlass.inline.hpp can look like:
>>>     #if defined(TARGET_COMPILER_visCPP) ||
>>> defined(TARGET_COMPILER_sparcWorks)
>>>       #define INLINE FORCE_INLINE
>>>     #else
>>>       #define INLINE inline
>>>     #endif
>>>
>>> - The same comment for stack.inline.hpp, but with NO_INLINE:
>>>    #if defined(TARGET_COMPILER_sparcWorks)
>>>      #define STACK_NO_INLINE NO_INLINE
>>>    #else
>>>      #define STACK_NO_INLINE
>>>    #endif
>> I agree, we should come up with a better and cleaner way to do this. I
>> will file a follow up RFE.
>>
>>> - I agree with Per about adding an #undef in stack.inline.hpp
>> Already fixed.
>>
>> Thanks again Erik,
>> Stefan
>>
>>> Thanks!
>>> Erik
>>>
>>>> Testing:
>>>> * Adhoc RBT run for functionality - no new failures.
>>>> * Adhoc aurora.se run for performance - no obvious regression.
>>>> * Many local runs tuned to do Full GCs a lot to verify that there 
>>>> is no
>>>> obvious regression.
>>>>
>>>> Thanks,
>>>> Stefan
>>



From jesper.wilhelmsson at oracle.com  Fri Aug 21 12:18:11 2015
From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson)
Date: Fri, 21 Aug 2015 14:18:11 +0200
Subject: Request for code review - 8130265: gctests/LargeObjects/large001
	fails with OutOfMemoryError: Java heap space
In-Reply-To: <55D62D77.8020504@oracle.com>
References: <55D356FB.1030603@oracle.com>
	<55D35A12.8020608@oracle.com>	<55D35F1F.3090700@oracle.com>
	<55D36DC7.6010901@oracle.com>	<55D38CBE.8070408@oracle.com>
	<55D394AD.6020104@oracle.com>	<55D4E39D.5070805@oracle.com>
	<55D4E8C7.1080703@oracle.com>	<55D4EEF5.9030504@oracle.com>	<2C141666-BDBC-428C-9E40-4FB78F076192@oracle.com>	<1440082894.2347.14.camel@oracle.com>
	<55D62D77.8020504@oracle.com>
Message-ID: <55D71703.4070505@oracle.com>

Hi Alex,

Alexander Harlap skrev den 20/8/15 21:41:
> Kim and Thomas,
>
> Here is new version with your suggestions in:
>
> http://cr.openjdk.java.net/~aharlap/8130265/webrev.03/
> <http://cr.openjdk.java.net/%7Eaharlap/8130265/webrev.03/>
> Alex
>

A few comments on this latest version.

* Why do you assert_at_safepoint and set succeeded to true in your helper? The 
original code did not do this at these places. I think you can remove these lines.

1736   assert_at_safepoint(true /* should_be_vm_thread */);
1737
1738   *succeeded = true;


* Please align the arguments in

1732 G1CollectedHeap::satisfy_failed_allocation_helper(size_t word_size,
1733                                            AllocationContext_t context,
1734                                            bool clear_all_soft_refs,
1735                                            bool* succeeded) {

and

1748   HeapWord* result = attempt_allocation_at_safepoint(word_size,
1749                                            context,
1750                                            true /* 
expect_null_mutator_alloc_region */);

and also in the header file.


* Your last version did:

attempt_allocation_at_safepoint(word_size, context, false);
expand_and_allocate(word_size, context);
do_collection(false, false, word_size);
attempt_allocation_at_safepoint(word_size, context, true);
expand_and_allocate(word_size, context);
do_collection(false, true, word_size);
attempt_allocation_at_safepoint(word_size, context, true);
assert(!collector_policy()->should_clear_all_soft_refs());
expand_and_allocate(word_size, context);

You have chosen to break this into:

attempt_allocation_at_safepoint(word_size, context, false);
expand_and_allocate(word_size, context);

satisfy_failed_allocation_helper(word_size, context, false, succeeded);
    do_collection(false, false, word_size);
    attempt_allocation_at_safepoint(word_size, context, true);
    expand_and_allocate(word_size, context);

satisfy_failed_allocation_helper(word_size, context, true, succeeded);
    do_collection(false, true, word_size);
    attempt_allocation_at_safepoint(word_size, context, true);
    expand_and_allocate(word_size, context);

assert(!collector_policy()->should_clear_all_soft_refs());

This is not entirely identical, depending on how important the assert is that 
could be an issue since the assert is now skipped in some cases where it was 
previously used.

My suggestion would be to break the code up like this:

satisfy_failed_allocation_helper(word_size, context, do_gc, clear_softrefs)
    attempt_allocation_at_safepoint(word_size, context, false);
    expand_and_allocate(word_size, context);
    do_collection(false, false, word_size);

satisfy_failed_allocation_helper(word_size, context, do_gc, clear_softrefs)
    attempt_allocation_at_safepoint(word_size, context, true);
    expand_and_allocate(word_size, context);
    do_collection(false, true, word_size);

satisfy_failed_allocation_helper(word_size, context, do_gc, clear_softrefs)
    attempt_allocation_at_safepoint(word_size, context, true);
    if (!do_gc)
       assert(!collector_policy()->should_clear_all_soft_refs());
    expand_and_allocate(word_size, context);

This would make the code in satisfy_failed_allocation() a lot more clean.


* Unless I'm missing some subtle detail here, the usage of the variable 
'succeeded' is unnecessary as 'result' will always contain the same information. 
I don't think we should pass on 'succeeded' to the helper, but simply set it 
afterwards before we return from satisfy_failed_allocation().

Thanks,
/Jesper


> On 8/20/2015 11:01 AM, Thomas Schatzl wrote:
>> Hi Alex,
>>
>> On Wed, 2015-08-19 at 18:43 -0400, Kim Barrett wrote:
>>> On Aug 19, 2015, at 5:02 PM, Alexander Harlap <alexander.harlap at oracle.com>
>>> wrote:
>>>> I agree.
>>>>
>>>> Here is new revision:
>>>>
>>>> http://cr.openjdk.java.net/~aharlap/8130265/webrev.02/
>>>> <http://cr.openjdk.java.net/%7Eaharlap/8130265/webrev.02/>
>>    would it be possible to fix the code duplication between lines
>> 1759-1784 and 1785 and 1812? The code seems to be completely the same
>> except the different flag value passed to the Full GC call and the
>> assert about that should_clear_all_soft_refs().
>>
>> Otherwise it looks good.
>>
>> Thanks,
>>    Thomas
>>
>>
>>
>


From dmitry.dmitriev at oracle.com  Fri Aug 21 15:25:27 2015
From: dmitry.dmitriev at oracle.com (Dmitry Dmitriev)
Date: Fri, 21 Aug 2015 18:25:27 +0300
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <8566C008-5A60-4376-94C4-968E1A475769@oracle.com>
References: <55C103A4.1060505@oracle.com>
	<A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>
	<1438781844.2378.60.camel@oracle.com> <55C21DC3.5090305@oracle.com>
	<1438785532.2378.73.camel@oracle.com> <55C248FF.6010308@oracle.com>
	<F73C4B9A-4DBA-47A8-8715-0988FF6D393D@oracle.com>
	<55C26249.1090902@oracle.com>
	<C149705F-696B-46CD-8FB8-CAD06C511244@oracle.com>
	<55C2751B.8070400@oracle.com> <1438847329.2009.9.camel@oracle.com>
	<55C36B31.7000908@oracle.com> <1438871723.2474.37.camel@oracle.com>
	<55C373F6.9060609@oracle.com>
	<AD45178E-A442-4A0D-80D6-50E91608E1FE@oracle.com>
	<55C3D8BB.9000603@oracle.com>
	<48C0E571-0D7B-4D30-85DF-E31D99092A7C@oracle.com>
	<30B0248C-3BEA-4FAE-861F-DAAE19F56B45@oracle.com>
	<55C4B806.7050504@oracle.com> <55C4C720.5030903@oracle.com>
	<55CA3430.3070300@oracle.com> <1439377259.2324.27.camel@oracle.com>
	<55CCF0DC.2000800@oracle.com> <1440079572.2347.11.camel@oracle.com>
	<55D5E048.5070404@oracle.com>
	<8566C008-5A60-4376-94C4-968E1A475769@oracle.com>
Message-ID: <55D742E7.9080208@oracle.com>

Hello Jiangli,

This looks good to me, but I'm not a reviewer.

Also, I have question to you. Probably you can clarify me one moment.
num_ranges and string_ranges are modified only in code under "#if 
INCLUDE_ALL_GCS", so it make sense to include all usage of these 
variables also under "#if INCLUDE_ALL_GCS"? I mean following functions:
FileMapInfo::fixup_string_regions() and new 
FileMapInfo::dealloc_string_regions() function.

Thank you,
Dmitry

On 20.08.2015 22:55, Jiangli Zhou wrote:
> Hi Dmitry,
>
> Here is the updated runtime webrev that reflects Tom?s latest GC changes.
>
> http://cr.openjdk.java.net/~jiangli/8131734/webrev.01/
>
> I renamed the FileMapInfo::unmap_string_regions() to FileMapInfo::dealloc_string_regions(), which only deallocates the archived string region from the java heap without unmapping. The unmapping is handled by the GC system as the archived string region is part of the java heap. I also added dealloc_string_regions() call to the case where the string region verification fails.
>
> Thanks,
> Jiangli
>
> On Aug 20, 2015, at 7:12 AM, Tom Benson <tom.benson at oracle.com> wrote:
>
>> Hi Thomas,
>> OK, thanks!
>> Tom
>>
>> On 8/20/2015 10:06 AM, Thomas Schatzl wrote:
>>> Hi Tom,
>>>
>>>    sorry for the delay...
>>>
>>> On Thu, 2015-08-13 at 15:32 -0400, Tom Benson wrote:
>>>> Hi Thomas,
>>>>
>>>> On 8/12/2015 7:00 AM, Thomas Schatzl wrote:
>>>>> Hi,
>>>>>
>>>>> On Tue, 2015-08-11 at 13:43 -0400, Tom Benson wrote:
>>>>>> Hi,
>>>>>> On 8/7/2015 10:56 AM, Tom Benson wrote:
>>>>>> After some discussion, I've changed the definition and name of
>>>>>> free_archive_regions.  Now called dealloc_archive_regions, it uncommits
>>>>>> the specified regions, unmapping the memory, rather than adding them to
>>>>>> the free list.  This means the CDS code will no longer do the unmapping
>>>>>> on verification failures.
>>>>>>
>>>>>> Updated full and incremental webrevs of the GC code are at:
>>>>>> http://cr.openjdk.java.net/~tbenson/8131734/webrev.02/
>>>>>> http://cr.openjdk.java.net/~tbenson/8131734/webrev.02.vs.01/
>>>>>>
>>>>>> Tested with JPRT and running benchmarks with the dealloc_ performed
>>>>>> explicitly.  Jiangli also tested the original failing cases, and will be
>>>>>> posting an updated webrev.
>>>>> - is it possible that shrink_by() uses shrink_at()? This would avoid two
>>>>> paths that uncommit regions like expand_by()/expand_at()?
>>>> OK, I made the change.  I didn't do it originally because the asserts I
>>>> wanted to add for the call from g1CollectedHeap seemed superfluous for
>>>> the other call, and shrink_at was so small.    Now shrink_at takes a
>>>> region count as well.
>>>>
>>>> Updated full and incremental webrevs are at:
>>>>     http://cr.openjdk.java.net/~tbenson/8131734/webrev.03/
>>>>     http://cr.openjdk.java.net/~tbenson/8131734/webrev.03.vs.02/
>>>>
>>>>
>>> Looks good.
>>>
>>>>> - I think the change should call at least HeapRegion::hr_clear() on the
>>>>> region to remove or reset any auxiliary data structures, if not
>>>>> G1CollectedHeap::free_region() (without adding the region to the free
>>>>> list).
>>>>> Since the HeapRegion* is not deallocated by the uncommit, this may cause
>>>>> strange behavior later when the region is reused.
>>>> I don't think calling hr_clear should be necessary...  If it is, we
>>>> should be doing it in shrink_by as well, and I don't think we are. I
>>>> don't see how a HeapRegion can be 'reused' without having gone through
>>>> the constructor when expand_ asks (indirectly) for 'new HeapRegion', and
>>>> that does an hr_clear() as well as the rest of init.     Or am I missing
>>>> something there?
>>> Leave it as is. I thought that a full gc (which is the only case where
>>> the heap shrinks at the moment) also clears the remset of these regions
>>> at least.
>>>
>>> It should, I filed  JDK-8134048 for looking in this issue.
>>>
>>> Looks good.
>>>
>>> Thanks,
>>>    Thomas
>>>
>>>



From dmitry.fazunenko at oracle.com  Fri Aug 21 15:26:57 2015
From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko)
Date: Fri, 21 Aug 2015 18:26:57 +0300
Subject: RFR (S): 8132708: Add tests for Humongous objects allocation
	threshold
In-Reply-To: <55D60438.4080004@oracle.com>
References: <55D60438.4080004@oracle.com>
Message-ID: <55D74341.7000706@oracle.com>

Hi Kirill,

the change looks good me.

Thanks,
Dima

On 20.08.2015 19:45, Kirill Zhaldybin wrote:
> Hi!
>
> Could you please review a new simple test checking that only objects 
> allocating more than  a half regions are allocated as humongous?
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8132708
>
> Webrev:
> hotspot:
> http://cr.openjdk.java.net/~kzhaldyb/webrevs/JDK-8132708/webrev.03/
>
> Tests:
> jprt
>
> Thank you.
>
> Regards, Kirill



From alexander.harlap at oracle.com  Fri Aug 21 15:44:38 2015
From: alexander.harlap at oracle.com (Alexander Harlap)
Date: Fri, 21 Aug 2015 11:44:38 -0400
Subject: Request for code review - 8130265: gctests/LargeObjects/large001
	fails with OutOfMemoryError: Java heap space
In-Reply-To: <55D71703.4070505@oracle.com>
References: <55D356FB.1030603@oracle.com> <55D35A12.8020608@oracle.com>
	<55D35F1F.3090700@oracle.com> <55D36DC7.6010901@oracle.com>
	<55D38CBE.8070408@oracle.com> <55D394AD.6020104@oracle.com>
	<55D4E39D.5070805@oracle.com> <55D4E8C7.1080703@oracle.com>
	<55D4EEF5.9030504@oracle.com>
	<2C141666-BDBC-428C-9E40-4FB78F076192@oracle.com>
	<1440082894.2347.14.camel@oracle.com> <55D62D77.8020504@oracle.com>
	<55D71703.4070505@oracle.com>
Message-ID: <55D74766.5020202@oracle.com>



On 8/21/2015 8:18 AM, Jesper Wilhelmsson wrote:
> Hi Alex,
>
> Alexander Harlap skrev den 20/8/15 21:41:
>> Kim and Thomas,
>>
>> Here is new version with your suggestions in:
>>
>> http://cr.openjdk.java.net/~aharlap/8130265/webrev.03/
>> <http://cr.openjdk.java.net/%7Eaharlap/8130265/webrev.03/>
>> Alex
>>
>
> A few comments on this latest version.
>
> * Why do you assert_at_safepoint and set succeeded to true in your 
> helper? The original code did not do this at these places. I think you 
> can remove these lines.
>
> 1736   assert_at_safepoint(true /* should_be_vm_thread */);
> 1737
> 1738   *succeeded = true;
>
Even method called satisfy_failed_allocation_helper , there is no way to 
make sure that it is called /only /from satisfy_failed_allocation.
satisfy_failed_allocation_helper is a separate  method and those lines 
are needed.
So I would like to keep
1736   assert_at_safepoint(true /* should_be_vm_thread */);
1737
1738   *succeeded = true;
>
> * Please align the arguments in
>
> 1732 G1CollectedHeap::satisfy_failed_allocation_helper(size_t word_size,
> 1733 AllocationContext_t context,
> 1734                                            bool clear_all_soft_refs,
> 1735                                            bool* succeeded) {
>
> and
>
> 1748   HeapWord* result = attempt_allocation_at_safepoint(word_size,
> 1749                                            context,
> 1750                                            true /* 
> expect_null_mutator_alloc_region */);
>
> and also in the header file.
>
Sure.

>
> * Your last version did:
>
> attempt_allocation_at_safepoint(word_size, context, false);
> expand_and_allocate(word_size, context);
> do_collection(false, false, word_size);
> attempt_allocation_at_safepoint(word_size, context, true);
> expand_and_allocate(word_size, context);
> do_collection(false, true, word_size);
> attempt_allocation_at_safepoint(word_size, context, true);
> assert(!collector_policy()->should_clear_all_soft_refs());
> expand_and_allocate(word_size, context);
>
> You have chosen to break this into:
>
> attempt_allocation_at_safepoint(word_size, context, false);
> expand_and_allocate(word_size, context);
>
> satisfy_failed_allocation_helper(word_size, context, false, succeeded);
>    do_collection(false, false, word_size);
>    attempt_allocation_at_safepoint(word_size, context, true);
>    expand_and_allocate(word_size, context);
>
> satisfy_failed_allocation_helper(word_size, context, true, succeeded);
>    do_collection(false, true, word_size);
>    attempt_allocation_at_safepoint(word_size, context, true);
>    expand_and_allocate(word_size, context);
>
> assert(!collector_policy()->should_clear_all_soft_refs());
>
> This is not entirely identical, depending on how important the assert 
> is that could be an issue since the assert is now skipped in some 
> cases where it was previously used.
>
> My suggestion would be to break the code up like this:
>
> satisfy_failed_allocation_helper(word_size, context, do_gc, 
> clear_softrefs)
>    attempt_allocation_at_safepoint(word_size, context, false);
>    expand_and_allocate(word_size, context);
>    do_collection(false, false, word_size);
>
> satisfy_failed_allocation_helper(word_size, context, do_gc, 
> clear_softrefs)
>    attempt_allocation_at_safepoint(word_size, context, true);
>    expand_and_allocate(word_size, context);
>    do_collection(false, true, word_size);
>
> satisfy_failed_allocation_helper(word_size, context, do_gc, 
> clear_softrefs)
>    attempt_allocation_at_safepoint(word_size, context, true);
>    if (!do_gc)
> assert(!collector_policy()->should_clear_all_soft_refs());
>    expand_and_allocate(word_size, context);
>
> This would make the code in satisfy_failed_allocation() a lot more clean.
>
In webrev.03 I decided to put same way as it was in the original code 
(before any of my changes),
at the very end of method just when we already exhausted all 
possibilities and about to return NULL.

/attempt_allocation_at_safepoint(word_size, context, false); //
//expand_and_allocate(word_size, context); //
//do_collection(false, false, word_size); //
//attempt_allocation_at_safepoint(word_size, context, true); //
////do_collection(false, true, word_size); //
//attempt_allocation_at_safepoint(word_size, context, true); //
//assert(!collector_policy()->should_clear_all_soft_refs()); ///

  // What else? We might try synchronous finalization later. If the 
total 1799 // space available is large enough for the allocation, then a 
more
  // complete compaction phase than we've tried so far might be
// appropriate.
assert(*succeeded, "sanity");
return NULL; 1804
}

webrev.03 has this assert at same position, and I prefer it this way.

>
> * Unless I'm missing some subtle detail here, the usage of the 
> variable 'succeeded' is unnecessary as 'result' will always contain 
> the same information. I don't think we should pass on 'succeeded' to 
> the helper, but simply set it afterwards before we return from 
> satisfy_failed_allocation().
>
No, helper may return NULL in two differnt cases:
1. Full GC failed, *succeeded = false, no attempts to allocate was made
2. Full GC succeeded, *succeeded = true, all attempts to allocate failed

So we need to pass 'succeeded' to the helper.

> Thanks,
> /Jesper
>
>
>> On 8/20/2015 11:01 AM, Thomas Schatzl wrote:
>>> Hi Alex,
>>>
>>> On Wed, 2015-08-19 at 18:43 -0400, Kim Barrett wrote:
>>>> On Aug 19, 2015, at 5:02 PM, Alexander Harlap 
>>>> <alexander.harlap at oracle.com>
>>>> wrote:
>>>>> I agree.
>>>>>
>>>>> Here is new revision:
>>>>>
>>>>> http://cr.openjdk.java.net/~aharlap/8130265/webrev.02/
>>>>> <http://cr.openjdk.java.net/%7Eaharlap/8130265/webrev.02/>
>>>    would it be possible to fix the code duplication between lines
>>> 1759-1784 and 1785 and 1812? The code seems to be completely the same
>>> except the different flag value passed to the Full GC call and the
>>> assert about that should_clear_all_soft_refs().
>>>
>>> Otherwise it looks good.
>>>
>>> Thanks,
>>>    Thomas
>>>
>>>
>>>
>>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150821/6d5b829d/attachment.htm>

From alexander.harlap at oracle.com  Fri Aug 21 16:02:36 2015
From: alexander.harlap at oracle.com (Alexander Harlap)
Date: Fri, 21 Aug 2015 12:02:36 -0400
Subject: Request for code review - 8130265: gctests/LargeObjects/large001
	fails with OutOfMemoryError: Java heap space
In-Reply-To: <55D74766.5020202@oracle.com>
References: <55D356FB.1030603@oracle.com> <55D35A12.8020608@oracle.com>
	<55D35F1F.3090700@oracle.com> <55D36DC7.6010901@oracle.com>
	<55D38CBE.8070408@oracle.com> <55D394AD.6020104@oracle.com>
	<55D4E39D.5070805@oracle.com> <55D4E8C7.1080703@oracle.com>
	<55D4EEF5.9030504@oracle.com>
	<2C141666-BDBC-428C-9E40-4FB78F076192@oracle.com>
	<1440082894.2347.14.camel@oracle.com> <55D62D77.8020504@oracle.com>
	<55D71703.4070505@oracle.com> <55D74766.5020202@oracle.com>
Message-ID: <55D74B9C.6010900@oracle.com>

Hi Thomas,

it looks to me that change in webrev.02 (without new method) is easier 
to read than in in webrev.03 even it has some code duplication
You could use Jesper comments as good indicator that change in webrev.03 
became more complicated than it supposed to be.

May be better continue  from webrev.02?

Alex

On 8/21/2015 11:44 AM, Alexander Harlap wrote:
>
>
> On 8/21/2015 8:18 AM, Jesper Wilhelmsson wrote:
>> Hi Alex,
>>
>> Alexander Harlap skrev den 20/8/15 21:41:
>>> Kim and Thomas,
>>>
>>> Here is new version with your suggestions in:
>>>
>>> http://cr.openjdk.java.net/~aharlap/8130265/webrev.03/
>>> <http://cr.openjdk.java.net/%7Eaharlap/8130265/webrev.03/>
>>> Alex
>>>
>>
>> A few comments on this latest version.
>>
>> * Why do you assert_at_safepoint and set succeeded to true in your 
>> helper? The original code did not do this at these places. I think 
>> you can remove these lines.
>>
>> 1736   assert_at_safepoint(true /* should_be_vm_thread */);
>> 1737
>> 1738   *succeeded = true;
>>
> Even method called satisfy_failed_allocation_helper , there is no way 
> to make sure that it is called /only /from satisfy_failed_allocation.
> satisfy_failed_allocation_helper is a separate  method and those lines 
> are needed.
> So I would like to keep
> 1736   assert_at_safepoint(true /* should_be_vm_thread */);
> 1737
> 1738   *succeeded = true;
>>
>> * Please align the arguments in
>>
>> 1732 G1CollectedHeap::satisfy_failed_allocation_helper(size_t word_size,
>> 1733 AllocationContext_t context,
>> 1734                                            bool 
>> clear_all_soft_refs,
>> 1735                                            bool* succeeded) {
>>
>> and
>>
>> 1748   HeapWord* result = attempt_allocation_at_safepoint(word_size,
>> 1749                                            context,
>> 1750                                            true /* 
>> expect_null_mutator_alloc_region */);
>>
>> and also in the header file.
>>
> Sure.
>
>>
>> * Your last version did:
>>
>> attempt_allocation_at_safepoint(word_size, context, false);
>> expand_and_allocate(word_size, context);
>> do_collection(false, false, word_size);
>> attempt_allocation_at_safepoint(word_size, context, true);
>> expand_and_allocate(word_size, context);
>> do_collection(false, true, word_size);
>> attempt_allocation_at_safepoint(word_size, context, true);
>> assert(!collector_policy()->should_clear_all_soft_refs());
>> expand_and_allocate(word_size, context);
>>
>> You have chosen to break this into:
>>
>> attempt_allocation_at_safepoint(word_size, context, false);
>> expand_and_allocate(word_size, context);
>>
>> satisfy_failed_allocation_helper(word_size, context, false, succeeded);
>>    do_collection(false, false, word_size);
>>    attempt_allocation_at_safepoint(word_size, context, true);
>>    expand_and_allocate(word_size, context);
>>
>> satisfy_failed_allocation_helper(word_size, context, true, succeeded);
>>    do_collection(false, true, word_size);
>>    attempt_allocation_at_safepoint(word_size, context, true);
>>    expand_and_allocate(word_size, context);
>>
>> assert(!collector_policy()->should_clear_all_soft_refs());
>>
>> This is not entirely identical, depending on how important the assert 
>> is that could be an issue since the assert is now skipped in some 
>> cases where it was previously used.
>>
>> My suggestion would be to break the code up like this:
>>
>> satisfy_failed_allocation_helper(word_size, context, do_gc, 
>> clear_softrefs)
>>    attempt_allocation_at_safepoint(word_size, context, false);
>>    expand_and_allocate(word_size, context);
>>    do_collection(false, false, word_size);
>>
>> satisfy_failed_allocation_helper(word_size, context, do_gc, 
>> clear_softrefs)
>>    attempt_allocation_at_safepoint(word_size, context, true);
>>    expand_and_allocate(word_size, context);
>>    do_collection(false, true, word_size);
>>
>> satisfy_failed_allocation_helper(word_size, context, do_gc, 
>> clear_softrefs)
>>    attempt_allocation_at_safepoint(word_size, context, true);
>>    if (!do_gc)
>> assert(!collector_policy()->should_clear_all_soft_refs());
>>    expand_and_allocate(word_size, context);
>>
>> This would make the code in satisfy_failed_allocation() a lot more 
>> clean.
>>
> In webrev.03 I decided to put same way as it was in the original code 
> (before any of my changes),
> at the very end of method just when we already exhausted all 
> possibilities and about to return NULL.
>
> /attempt_allocation_at_safepoint(word_size, context, false); //
> //expand_and_allocate(word_size, context); //
> //do_collection(false, false, word_size); //
> //attempt_allocation_at_safepoint(word_size, context, true); //
> ////do_collection(false, true, word_size); //
> //attempt_allocation_at_safepoint(word_size, context, true); //
> //assert(!collector_policy()->should_clear_all_soft_refs()); /
>
>  // What else? We might try synchronous finalization later. If the 
> total 1799 // space available is large enough for the allocation, then 
> a more
>  // complete compaction phase than we've tried so far might be
> // appropriate.
> assert(*succeeded, "sanity");
> return NULL; 1804
> }
>
> webrev.03 has this assert at same position, and I prefer it this way.
>
>>
>> * Unless I'm missing some subtle detail here, the usage of the 
>> variable 'succeeded' is unnecessary as 'result' will always contain 
>> the same information. I don't think we should pass on 'succeeded' to 
>> the helper, but simply set it afterwards before we return from 
>> satisfy_failed_allocation().
>>
> No, helper may return NULL in two differnt cases:
> 1. Full GC failed, *succeeded = false, no attempts to allocate was made
> 2. Full GC succeeded, *succeeded = true, all attempts to allocate failed
>
> So we need to pass 'succeeded' to the helper.
>
>> Thanks,
>> /Jesper
>>
>>
>>> On 8/20/2015 11:01 AM, Thomas Schatzl wrote:
>>>> Hi Alex,
>>>>
>>>> On Wed, 2015-08-19 at 18:43 -0400, Kim Barrett wrote:
>>>>> On Aug 19, 2015, at 5:02 PM, Alexander Harlap 
>>>>> <alexander.harlap at oracle.com>
>>>>> wrote:
>>>>>> I agree.
>>>>>>
>>>>>> Here is new revision:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~aharlap/8130265/webrev.02/
>>>>>> <http://cr.openjdk.java.net/%7Eaharlap/8130265/webrev.02/>
>>>>    would it be possible to fix the code duplication between lines
>>>> 1759-1784 and 1785 and 1812? The code seems to be completely the same
>>>> except the different flag value passed to the Full GC call and the
>>>> assert about that should_clear_all_soft_refs().
>>>>
>>>> Otherwise it looks good.
>>>>
>>>> Thanks,
>>>>    Thomas
>>>>
>>>>
>>>>
>>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150821/b6ec94f6/attachment.htm>

From jiangli.zhou at oracle.com  Sat Aug 22 01:47:28 2015
From: jiangli.zhou at oracle.com (Jiangli Zhou)
Date: Fri, 21 Aug 2015 18:47:28 -0700
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <55D742E7.9080208@oracle.com>
References: <55C103A4.1060505@oracle.com>
	<A5ACA792-C586-4E46-BD80-F979B3CDE571@oracle.com>
	<1438781844.2378.60.camel@oracle.com> <55C21DC3.5090305@oracle.com>
	<1438785532.2378.73.camel@oracle.com> <55C248FF.6010308@oracle.com>
	<F73C4B9A-4DBA-47A8-8715-0988FF6D393D@oracle.com>
	<55C26249.1090902@oracle.com>
	<C149705F-696B-46CD-8FB8-CAD06C511244@oracle.com>
	<55C2751B.8070400@oracle.com> <1438847329.2009.9.camel@oracle.com>
	<55C36B31.7000908@oracle.com> <1438871723.2474.37.camel@oracle.com>
	<55C373F6.9060609@oracle.com>
	<AD45178E-A442-4A0D-80D6-50E91608E1FE@oracle.com>
	<55C3D8BB.9000603@oracle.com>
	<48C0E571-0D7B-4D30-85DF-E31D99092A7C@oracle.com>
	<30B0248C-3BEA-4FAE-861F-DAAE19F56B45@oracle.com>
	<55C4B806.7050504@oracle.com> <55C4C720.5030903@oracle.com>
	<55CA3430.3070300@oracle.com> <1439377259.2324.27.camel@oracle.com>
	<55CCF0DC.2000800@oracle.com> <1440079572.2347.11.camel@oracle.com>
	<55D5E048.5070404@oracle.com>
	<8566C008-5A60-4376-94C4-968E1A475769@oracle.com> <55D742E7.!
	9080208@oracle.com>
Message-ID: <CAA5A1C9-88F6-4EFE-9C6B-E7CAA366589F@oracle.com>

Hi Dmitry,

On Aug 21, 2015, at 8:25 AM, Dmitry Dmitriev <dmitry.dmitriev at oracle.com> wrote:

> Hello Jiangli,
> 
> This looks good to me, but I'm not a reviewer.

Thanks.

> 
> Also, I have question to you. Probably you can clarify me one moment.
> num_ranges and string_ranges are modified only in code under "#if INCLUDE_ALL_GCS", so it make sense to include all usage of these variables also under "#if INCLUDE_ALL_GCS"? I mean following functions:
> FileMapInfo::fixup_string_regions() and new FileMapInfo::dealloc_string_regions() function.

Agreed. Here is the updated webrev: http://cr.openjdk.java.net/~jiangli/8131734/webrev.02/src/share/vm/memory/filemap.cpp.sdiff.html.

I also reverified JPRT builds with the new #ifdef changes.

Thanks for the detailed review!

Jiangli

> 
> Thank you,
> Dmitry
> 
> On 20.08.2015 22:55, Jiangli Zhou wrote:
>> Hi Dmitry,
>> 
>> Here is the updated runtime webrev that reflects Tom?s latest GC changes.
>> 
>> http://cr.openjdk.java.net/~jiangli/8131734/webrev.01/
>> 
>> I renamed the FileMapInfo::unmap_string_regions() to FileMapInfo::dealloc_string_regions(), which only deallocates the archived string region from the java heap without unmapping. The unmapping is handled by the GC system as the archived string region is part of the java heap. I also added dealloc_string_regions() call to the case where the string region verification fails.
>> 
>> Thanks,
>> Jiangli
>> 
>> On Aug 20, 2015, at 7:12 AM, Tom Benson <tom.benson at oracle.com> wrote:
>> 
>>> Hi Thomas,
>>> OK, thanks!
>>> Tom
>>> 
>>> On 8/20/2015 10:06 AM, Thomas Schatzl wrote:
>>>> Hi Tom,
>>>> 
>>>>   sorry for the delay...
>>>> 
>>>> On Thu, 2015-08-13 at 15:32 -0400, Tom Benson wrote:
>>>>> Hi Thomas,
>>>>> 
>>>>> On 8/12/2015 7:00 AM, Thomas Schatzl wrote:
>>>>>> Hi,
>>>>>> 
>>>>>> On Tue, 2015-08-11 at 13:43 -0400, Tom Benson wrote:
>>>>>>> Hi,
>>>>>>> On 8/7/2015 10:56 AM, Tom Benson wrote:
>>>>>>> After some discussion, I've changed the definition and name of
>>>>>>> free_archive_regions.  Now called dealloc_archive_regions, it uncommits
>>>>>>> the specified regions, unmapping the memory, rather than adding them to
>>>>>>> the free list.  This means the CDS code will no longer do the unmapping
>>>>>>> on verification failures.
>>>>>>> 
>>>>>>> Updated full and incremental webrevs of the GC code are at:
>>>>>>> http://cr.openjdk.java.net/~tbenson/8131734/webrev.02/
>>>>>>> http://cr.openjdk.java.net/~tbenson/8131734/webrev.02.vs.01/
>>>>>>> 
>>>>>>> Tested with JPRT and running benchmarks with the dealloc_ performed
>>>>>>> explicitly.  Jiangli also tested the original failing cases, and will be
>>>>>>> posting an updated webrev.
>>>>>> - is it possible that shrink_by() uses shrink_at()? This would avoid two
>>>>>> paths that uncommit regions like expand_by()/expand_at()?
>>>>> OK, I made the change.  I didn't do it originally because the asserts I
>>>>> wanted to add for the call from g1CollectedHeap seemed superfluous for
>>>>> the other call, and shrink_at was so small.    Now shrink_at takes a
>>>>> region count as well.
>>>>> 
>>>>> Updated full and incremental webrevs are at:
>>>>>    http://cr.openjdk.java.net/~tbenson/8131734/webrev.03/
>>>>>    http://cr.openjdk.java.net/~tbenson/8131734/webrev.03.vs.02/
>>>>> 
>>>>> 
>>>> Looks good.
>>>> 
>>>>>> - I think the change should call at least HeapRegion::hr_clear() on the
>>>>>> region to remove or reset any auxiliary data structures, if not
>>>>>> G1CollectedHeap::free_region() (without adding the region to the free
>>>>>> list).
>>>>>> Since the HeapRegion* is not deallocated by the uncommit, this may cause
>>>>>> strange behavior later when the region is reused.
>>>>> I don't think calling hr_clear should be necessary...  If it is, we
>>>>> should be doing it in shrink_by as well, and I don't think we are. I
>>>>> don't see how a HeapRegion can be 'reused' without having gone through
>>>>> the constructor when expand_ asks (indirectly) for 'new HeapRegion', and
>>>>> that does an hr_clear() as well as the rest of init.     Or am I missing
>>>>> something there?
>>>> Leave it as is. I thought that a full gc (which is the only case where
>>>> the heap shrinks at the moment) also clears the remset of these regions
>>>> at least.
>>>> 
>>>> It should, I filed  JDK-8134048 for looking in this issue.
>>>> 
>>>> Looks good.
>>>> 
>>>> Thanks,
>>>>   Thomas
>>>> 
>>>> 
> 



From dmitry.dmitriev at oracle.com  Sun Aug 23 20:54:20 2015
From: dmitry.dmitriev at oracle.com (Dmitry Dmitriev)
Date: Sun, 23 Aug 2015 23:54:20 +0300
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <CAA5A1C9-88F6-4EFE-9C6B-E7CAA366589F@oracle.com>
References: <55C103A4.1060505@oracle.com> <55C21DC3.5090305@oracle.com>
	<1438785532.2378.73.camel@oracle.com> <55C248FF.6010308@oracle.com>
	<F73C4B9A-4DBA-47A8-8715-0988FF6D393D@oracle.com>
	<55C26249.1090902@oracle.com>
	<C149705F-696B-46CD-8FB8-CAD06C511244@oracle.com>
	<55C2751B.8070400@oracle.com> <1438847329.2009.9.camel@oracle.com>
	<55C36B31.7000908@oracle.com> <1438871723.2474.37.camel@oracle.com>
	<55C373F6.9060609@oracle.com>
	<AD45178E-A442-4A0D-80D6-50E91608E1FE@oracle.com>
	<55C3D8BB.9000603@oracle.com>
	<48C0E571-0D7B-4D30-85DF-E31D99092A7C@oracle.com>
	<30B0248C-3BEA-4FAE-861F-DAAE19F56B45@oracle.com>
	<55C4B806.7050504@oracle.com> <55C4C720.5030903@oracle.com>
	<55CA3430.3070300@oracle.com> <1439377259.2324.27.camel@oracle.com>
	<55CCF0DC.2000800@oracle.com> <1440079572.2347.11.camel@oracle.com>
	<55D5E048.5070404@oracle.com>
	<8566C008-5A60-4376-94C4-968E1A475769@oracle.com>
	<55D742E7.9080208@oracle.com>
	<CAA5A1C9-88F6-4EFE-9C6B-E7CAA366589F@oracle.com>
Message-ID: <55DA32FC.8000405@oracle.com>

Hi Jiangli,

Looks good to me!

Thank you,
Dmitry

On 22.08.2015 4:47, Jiangli Zhou wrote:
> Hi Dmitry,
>
> On Aug 21, 2015, at 8:25 AM, Dmitry Dmitriev <dmitry.dmitriev at oracle.com> wrote:
>
>> Hello Jiangli,
>>
>> This looks good to me, but I'm not a reviewer.
> Thanks.
>
>> Also, I have question to you. Probably you can clarify me one moment.
>> num_ranges and string_ranges are modified only in code under "#if INCLUDE_ALL_GCS", so it make sense to include all usage of these variables also under "#if INCLUDE_ALL_GCS"? I mean following functions:
>> FileMapInfo::fixup_string_regions() and new FileMapInfo::dealloc_string_regions() function.
> Agreed. Here is the updated webrev: http://cr.openjdk.java.net/~jiangli/8131734/webrev.02/src/share/vm/memory/filemap.cpp.sdiff.html.
>
> I also reverified JPRT builds with the new #ifdef changes.
>
> Thanks for the detailed review!
>
> Jiangli
>
>> Thank you,
>> Dmitry
>>
>> On 20.08.2015 22:55, Jiangli Zhou wrote:
>>> Hi Dmitry,
>>>
>>> Here is the updated runtime webrev that reflects Tom?s latest GC changes.
>>>
>>> http://cr.openjdk.java.net/~jiangli/8131734/webrev.01/
>>>
>>> I renamed the FileMapInfo::unmap_string_regions() to FileMapInfo::dealloc_string_regions(), which only deallocates the archived string region from the java heap without unmapping. The unmapping is handled by the GC system as the archived string region is part of the java heap. I also added dealloc_string_regions() call to the case where the string region verification fails.
>>>
>>> Thanks,
>>> Jiangli
>>>
>>> On Aug 20, 2015, at 7:12 AM, Tom Benson <tom.benson at oracle.com> wrote:
>>>
>>>> Hi Thomas,
>>>> OK, thanks!
>>>> Tom
>>>>
>>>> On 8/20/2015 10:06 AM, Thomas Schatzl wrote:
>>>>> Hi Tom,
>>>>>
>>>>>    sorry for the delay...
>>>>>
>>>>> On Thu, 2015-08-13 at 15:32 -0400, Tom Benson wrote:
>>>>>> Hi Thomas,
>>>>>>
>>>>>> On 8/12/2015 7:00 AM, Thomas Schatzl wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> On Tue, 2015-08-11 at 13:43 -0400, Tom Benson wrote:
>>>>>>>> Hi,
>>>>>>>> On 8/7/2015 10:56 AM, Tom Benson wrote:
>>>>>>>> After some discussion, I've changed the definition and name of
>>>>>>>> free_archive_regions.  Now called dealloc_archive_regions, it uncommits
>>>>>>>> the specified regions, unmapping the memory, rather than adding them to
>>>>>>>> the free list.  This means the CDS code will no longer do the unmapping
>>>>>>>> on verification failures.
>>>>>>>>
>>>>>>>> Updated full and incremental webrevs of the GC code are at:
>>>>>>>> http://cr.openjdk.java.net/~tbenson/8131734/webrev.02/
>>>>>>>> http://cr.openjdk.java.net/~tbenson/8131734/webrev.02.vs.01/
>>>>>>>>
>>>>>>>> Tested with JPRT and running benchmarks with the dealloc_ performed
>>>>>>>> explicitly.  Jiangli also tested the original failing cases, and will be
>>>>>>>> posting an updated webrev.
>>>>>>> - is it possible that shrink_by() uses shrink_at()? This would avoid two
>>>>>>> paths that uncommit regions like expand_by()/expand_at()?
>>>>>> OK, I made the change.  I didn't do it originally because the asserts I
>>>>>> wanted to add for the call from g1CollectedHeap seemed superfluous for
>>>>>> the other call, and shrink_at was so small.    Now shrink_at takes a
>>>>>> region count as well.
>>>>>>
>>>>>> Updated full and incremental webrevs are at:
>>>>>>     http://cr.openjdk.java.net/~tbenson/8131734/webrev.03/
>>>>>>     http://cr.openjdk.java.net/~tbenson/8131734/webrev.03.vs.02/
>>>>>>
>>>>>>
>>>>> Looks good.
>>>>>
>>>>>>> - I think the change should call at least HeapRegion::hr_clear() on the
>>>>>>> region to remove or reset any auxiliary data structures, if not
>>>>>>> G1CollectedHeap::free_region() (without adding the region to the free
>>>>>>> list).
>>>>>>> Since the HeapRegion* is not deallocated by the uncommit, this may cause
>>>>>>> strange behavior later when the region is reused.
>>>>>> I don't think calling hr_clear should be necessary...  If it is, we
>>>>>> should be doing it in shrink_by as well, and I don't think we are. I
>>>>>> don't see how a HeapRegion can be 'reused' without having gone through
>>>>>> the constructor when expand_ asks (indirectly) for 'new HeapRegion', and
>>>>>> that does an hr_clear() as well as the rest of init.     Or am I missing
>>>>>> something there?
>>>>> Leave it as is. I thought that a full gc (which is the only case where
>>>>> the heap shrinks at the moment) also clears the remset of these regions
>>>>> at least.
>>>>>
>>>>> It should, I filed  JDK-8134048 for looking in this issue.
>>>>>
>>>>> Looks good.
>>>>>
>>>>> Thanks,
>>>>>    Thomas
>>>>>
>>>>>



From marcus.larsson at oracle.com  Mon Aug 24 07:39:04 2015
From: marcus.larsson at oracle.com (Marcus Larsson)
Date: Mon, 24 Aug 2015 09:39:04 +0200
Subject: RFR (XS): Forward port of 8065331: Add trace events for failed
	allocations
Message-ID: <55DACA18.2070605@oracle.com>

Hi,

Please review the following patch to forward port the JEP-242 event to 
JDK9. The patch was originally reviewed in April/March [1], but was only 
integrated in 8u60. The original patch no longer applies cleanly to 9, 
due to directory restructuring and changes to includes, so I need a 
quick re-review for this.

The new patch has updated #includes for the new directory structure but 
should otherwise be identical to the previous patch.

Webrev:
http://cr.openjdk.java.net/~mlarsson/8065331_to_9/webrev.00/

Original webrev:
http://cr.openjdk.java.net/~mlarsson/8065331/webrev.02/

Issue:
https://bugs.openjdk.java.net/browse/JDK-8065331

Thanks,
Marcus

[1]: 
http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2015-March/012408.html


From aph at redhat.com  Mon Aug 24 09:31:59 2015
From: aph at redhat.com (Andrew Haley)
Date: Mon, 24 Aug 2015 10:31:59 +0100
Subject: An interesting bug
Message-ID: <55DAE48F.2030403@redhat.com>

This is JDK8 on AArch64.

I have an interesting bug which causes a crash.  Occasionally, but
rarely, I get a crash when accessing an instance of ForkJoinTask.  The
instance is allocated in the old generation of the parallel scan GC.
However, the "this" pointer of the ForkJoinTask points into the middle
of another allocated object.

I'm trying to figure out how this might happen.  Perhaps the root scan
of the thread running the ForkJoinTask did not happen, or for some
reason the roots it found were not added to the root set.  Or perhaps
they were found, but were not rewritten in the thread.

Has anyone seen anything like this before?  Do you have any advice for
me?

Thanks,
Andrew.


From thomas.schatzl at oracle.com  Mon Aug 24 09:53:17 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 24 Aug 2015 11:53:17 +0200
Subject: RFR (S): 8132708: Add tests for Humongous objects allocation
	threshold
In-Reply-To: <55D60438.4080004@oracle.com>
References: <55D60438.4080004@oracle.com>
Message-ID: <1440409997.2348.8.camel@oracle.com>

Hi Kirill,

On Thu, 2015-08-20 at 19:45 +0300, Kirill Zhaldybin wrote:
> Hi!
> 
> Could you please review a new simple test checking that only objects 
> allocating more than  a half regions are allocated as humongous?
> 
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8132708
> 
> Webrev:
> hotspot:
> http://cr.openjdk.java.net/~kzhaldyb/webrevs/JDK-8132708/webrev.03/
> 
> Tests:
> jprt
> 

 - MAX_CONTINIUS_SIZE_CHECK: typo in "CONTINUOUS".

 - I am a bit puzzled about the verification whether the object should
be humongous. Shouldn't the test program ask the VM whether a given
object is humongous or not?
Currently it seems to me that the test program only checks itself, since
it on the one hand allocates objects of varying sizes, using
REGION_SIZE/2 and then in allocateAndCheck again uses this same formula
to determine that an object is humongous.

At most the test program seems to verify that
Helpers.detectByteArrayAllocationOverhead() is correct.

In this case I would expect a whitebox method to ask the VM whether the
given object is a humongous object.

Thanks,
  Thomas




From aph at redhat.com  Mon Aug 24 10:12:29 2015
From: aph at redhat.com (Andrew Haley)
Date: Mon, 24 Aug 2015 11:12:29 +0100
Subject: [aarch64-port-dev ] An interesting bug
In-Reply-To: <55DAE48F.2030403@redhat.com>
References: <55DAE48F.2030403@redhat.com>
Message-ID: <55DAEE0D.1020907@redhat.com>

I suppose another possibility might be that the worker threads promoting
to the old generation somehow (because of a failed compare-and-swap)
overwrote each other's data.

Andrew.


From thomas.schatzl at oracle.com  Mon Aug 24 10:24:32 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 24 Aug 2015 12:24:32 +0200
Subject: RFR (XXS): 8133456: HeapRegionManager::shrink_by() iterates
	suboptimally across regions
In-Reply-To: <55D7139D.10500@oracle.com>
References: <1440144337.2233.5.camel@oracle.com>
	<55D6E0F2.5030401@oracle.com> <1440146322.2233.12.camel@oracle.com>
	<55D7139D.10500@oracle.com>
Message-ID: <1440411872.2348.35.camel@oracle.com>

Hi Tom,

On Fri, 2015-08-21 at 08:03 -0400, Tom Benson wrote:
> Hi,
> The change looks good to me.
> 

  thanks for the review.

> On the in/out parameter question, I'd say I slightly favor the two 
> parameters.   On a related trivial note, I'd also prefer it slightly if 
> find_available_from_idx_reverse had the index as return value, with the 
> count as the output parameter.  Same for find_unavailable_from_idx.  
> Then they would be consistent with find_contiguous and 
> find_highest_free, and there wouldn't be 2 index params to look odd.   
> But that's another change.

I created https://bugs.openjdk.java.net/browse/JDK-8134299.

Thanks,
  Thomas




From thomas.schatzl at oracle.com  Mon Aug 24 10:36:28 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 24 Aug 2015 12:36:28 +0200
Subject: RFR (M): 8067336: Allow that PLAB allocations at the end of
	regions are flexible
In-Reply-To: <55D500FA.3070105@oracle.com>
References: <1439896720.2316.7.camel@oracle.com> <55D34955.10006@oracle.com>
	<1440019109.1994.51.camel@oracle.com> <55D500FA.3070105@oracle.com>
Message-ID: <1440412588.2348.36.camel@oracle.com>

Hi Tom,

  thanks for the review.

On Wed, 2015-08-19 at 18:19 -0400, Tom Benson wrote:
> Hi Thomas -
> Looks good to me.   Though there was one odd thing I noticed, not in the 
> changes but adjacent to one - this comment in g1Allocator.cpp:
> 
>   273     if (buf != NULL) {
>   274       // Otherwise.
>   275       alloc_buf->set_buf(buf, actual_plab_size);
> 
> There is another "Otherwise." comment just afterward at line 283, where 
> it makes sense, but the one at 274 looks left over from some past 
> restructuring.  Your call.  8^)
> Tom

I will remove the one in 274.

New webrevs:
http://cr.openjdk.java.net/~tschatzl/8067336/webrev.1_to_2/index.html
(diff)
http://cr.openjdk.java.net/~tschatzl/8067336/webrev.2/index.html (full)

Thanks,
  Thomas

> 
> On 8/19/2015 5:18 PM, Thomas Schatzl wrote:
> > Hi Tom,
> >
> > On Tue, 2015-08-18 at 11:03 -0400, Tom Benson wrote:
> >> Hi Thomas,
> >> It looks like the actual plab allocation changes are missing from
> >> g1Allocator.cpp - only an include was added...   ?
> >>
> >> Aside from that, just a couple of minor comments.  I think the comment
> >> at line 193-194 in g1AllocRegion.hpp should say "...maximum word size of
> >> the allocation in desired_word_size" rather than "... in word_size".
> >>
> >> In g1AllocRegion.inline.hpp, attempt_allocation and
> >> attempt_allocation_locked both have an error path that includes:
> >> trace("alloc failed", *actual_word_size);
> >>
> >> In the old version, the value handed to trace was the requested size, so
> >> here it should probably be min_word_size, since *actual_word_size is
> >> likely to be uninitialized.
> >    thanks for the review, even if it was for an unintentionally broken
> > change. I seem to have removed the actual change, the use of the new
> > methods in G1PLABAllocator::allocate_direct_or_new_plab(), while merging
> > these series of PLAB changes and the evac failure changes.
> >
> > I also fixed the G1AllocRegion::trace() method, adding the new sizes
> > (desired/actual size) as needed.
> >
> > Then I decided to move making G1OffsetTableContigSpace::_top volatile
> > (this is a _potential_ pre-existing bug - I am sure there is some weird
> > compiler that notices that it could hoist this access out of the loop in
> > G1OffsetTableContigSpace::par_allocate_impl()) into this CR. I am sure
> > you would have noticed anyway.
> >
> > I think I also covered the other problems you mentioned. Thanks again.
> >
> > CR:
> > https://bugs.openjdk.java.net/browse/JDK-8067336
> > Webrev:
> > http://cr.openjdk.java.net/~tschatzl/8067336/webrev.1 (full)
> > http://cr.openjdk.java.net/~tschatzl/8067336/webrev.0_to_1 (diff)
> > Testing:
> > jprt, vm.gc testlist, perf testing benchmarks, locally checked
> > allocation tracing (enabled by define only)
> >
> > Thanks,
> >    Thomas
> >
> 




From mikael.gerdin at oracle.com  Mon Aug 24 11:18:03 2015
From: mikael.gerdin at oracle.com (Mikael Gerdin)
Date: Mon, 24 Aug 2015 13:18:03 +0200
Subject: RFR (M): 8067336: Allow that PLAB allocations at the end of
	regions are flexible
In-Reply-To: <1440412588.2348.36.camel@oracle.com>
References: <1439896720.2316.7.camel@oracle.com>
	<55D34955.10006@oracle.com>	<1440019109.1994.51.camel@oracle.com>
	<55D500FA.3070105@oracle.com> <1440412588.2348.36.camel@oracle.com>
Message-ID: <55DAFD6B.8040304@oracle.com>

Hi Thomas,

On 2015-08-24 12:36, Thomas Schatzl wrote:
> Hi Tom,
>
>    thanks for the review.
>
> On Wed, 2015-08-19 at 18:19 -0400, Tom Benson wrote:
>> Hi Thomas -
>> Looks good to me.   Though there was one odd thing I noticed, not in the
>> changes but adjacent to one - this comment in g1Allocator.cpp:
>>
>>    273     if (buf != NULL) {
>>    274       // Otherwise.
>>    275       alloc_buf->set_buf(buf, actual_plab_size);
>>
>> There is another "Otherwise." comment just afterward at line 283, where
>> it makes sense, but the one at 274 looks left over from some past
>> restructuring.  Your call.  8^)
>> Tom
>
> I will remove the one in 274.
>
> New webrevs:
> http://cr.openjdk.java.net/~tschatzl/8067336/webrev.1_to_2/index.html
> (diff)
> http://cr.openjdk.java.net/~tschatzl/8067336/webrev.2/index.html (full)

Overall I think this change is good.
I'm somewhat paranoid about output parameters, can you please add code 
to initialize the value to be set by the out parameter to a dummy value 
(-1?) and assert that it was properly set on the code paths where the 
actual size is being used.

The alignment of the \ in vmStructs_g1.hpp is wrong, I don't need to see 
that in a new webrev though :)

/Mikael

>
> Thanks,
>    Thomas
>
>>
>> On 8/19/2015 5:18 PM, Thomas Schatzl wrote:
>>> Hi Tom,
>>>
>>> On Tue, 2015-08-18 at 11:03 -0400, Tom Benson wrote:
>>>> Hi Thomas,
>>>> It looks like the actual plab allocation changes are missing from
>>>> g1Allocator.cpp - only an include was added...   ?
>>>>
>>>> Aside from that, just a couple of minor comments.  I think the comment
>>>> at line 193-194 in g1AllocRegion.hpp should say "...maximum word size of
>>>> the allocation in desired_word_size" rather than "... in word_size".
>>>>
>>>> In g1AllocRegion.inline.hpp, attempt_allocation and
>>>> attempt_allocation_locked both have an error path that includes:
>>>> trace("alloc failed", *actual_word_size);
>>>>
>>>> In the old version, the value handed to trace was the requested size, so
>>>> here it should probably be min_word_size, since *actual_word_size is
>>>> likely to be uninitialized.
>>>     thanks for the review, even if it was for an unintentionally broken
>>> change. I seem to have removed the actual change, the use of the new
>>> methods in G1PLABAllocator::allocate_direct_or_new_plab(), while merging
>>> these series of PLAB changes and the evac failure changes.
>>>
>>> I also fixed the G1AllocRegion::trace() method, adding the new sizes
>>> (desired/actual size) as needed.
>>>
>>> Then I decided to move making G1OffsetTableContigSpace::_top volatile
>>> (this is a _potential_ pre-existing bug - I am sure there is some weird
>>> compiler that notices that it could hoist this access out of the loop in
>>> G1OffsetTableContigSpace::par_allocate_impl()) into this CR. I am sure
>>> you would have noticed anyway.
>>>
>>> I think I also covered the other problems you mentioned. Thanks again.
>>>
>>> CR:
>>> https://bugs.openjdk.java.net/browse/JDK-8067336
>>> Webrev:
>>> http://cr.openjdk.java.net/~tschatzl/8067336/webrev.1 (full)
>>> http://cr.openjdk.java.net/~tschatzl/8067336/webrev.0_to_1 (diff)
>>> Testing:
>>> jprt, vm.gc testlist, perf testing benchmarks, locally checked
>>> allocation tracing (enabled by define only)
>>>
>>> Thanks,
>>>     Thomas
>>>
>>
>
>


From thomas.schatzl at oracle.com  Mon Aug 24 12:55:22 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 24 Aug 2015 14:55:22 +0200
Subject: Request for code review - 8130265:
	gctests/LargeObjects/large001 fails with OutOfMemoryError: Java heap
	space
In-Reply-To: <55D74766.5020202@oracle.com>
References: <55D356FB.1030603@oracle.com> <55D35A12.8020608@oracle.com>
	<55D35F1F.3090700@oracle.com> <55D36DC7.6010901@oracle.com>
	<55D38CBE.8070408@oracle.com> <55D394AD.6020104@oracle.com>
	<55D4E39D.5070805@oracle.com> <55D4E8C7.1080703@oracle.com>
	<55D4EEF5.9030504@oracle.com>
	<2C141666-BDBC-428C-9E40-4FB78F076192@oracle.com>
	<1440082894.2347.14.camel@oracle.com> <55D62D77.8020504@oracle.com>
	<55D71703.4070505@oracle.com> <55D74766.5020202@oracle.com>
Message-ID: <1440420922.2348.82.camel@oracle.com>

Hi,

On Fri, 2015-08-21 at 11:44 -0400, Alexander Harlap wrote:
> 
> 
> On 8/21/2015 8:18 AM, Jesper Wilhelmsson wrote:
> 
> > Hi Alex, 
> > 
> > Alexander Harlap skrev den 20/8/15 21:41: 
> > > Kim and Thomas, 
> > > 
> > > Here is new version with your suggestions in: 
> > > 
> > > http://cr.openjdk.java.net/~aharlap/8130265/webrev.03/ 
> > > <http://cr.openjdk.java.net/%7Eaharlap/8130265/webrev.03/> 
> > > Alex 
> > A few comments on this latest version. 
> > 
> > * Why do you assert_at_safepoint and set succeeded to true in your
> > helper? The original code did not do this at these places. I think
> > you can remove these lines. 
> > 
> > 1736   assert_at_safepoint(true /* should_be_vm_thread */); 
> > 1737 
> > 1738   *succeeded = true; 
> > 
> Even method called satisfy_failed_allocation_helper , there is no way
> to make sure that it is called only  from satisfy_failed_allocation.

I do not think we guarantee that for any method, i.e. that a given
method somehow checks its call chain. This would require much better
encapsulation, and in some cases this does not help you either. Also it
would require you to state the allowed callers somehow.

The helper may be private too, not protected.

> satisfy_failed_allocation_helper is a separate method and those lines
> are needed.
> So I would like to keep
> 1736   assert_at_safepoint(true /* should_be_vm_thread */); 
> 1737 
> 1738   *succeeded = true; 
> > 
> > * Please align the arguments in 
> > 
> > 1732 G1CollectedHeap::satisfy_failed_allocation_helper(size_t
> > word_size, 
> > 1733                                            AllocationContext_t
> > context, 
> > 1734                                            bool
> > clear_all_soft_refs, 
> > 1735                                            bool* succeeded) { 
> > 
> > and 
> > 
> > 1748   HeapWord* result =
> > attempt_allocation_at_safepoint(word_size, 
> > 1749                                            context, 
> > 1750                                            true /*
> > expect_null_mutator_alloc_region */); 
> > 
> > and also in the header file. 

Please also avoid method definitions like

<return value>
<class name>::<method-name> ...

(or even worse

<return value>
<class-name>::
<method-name>

(this change does not do it that way)
)
> > 
> Sure.
>
> > * Your last version did: 
> > 
> > attempt_allocation_at_safepoint(word_size, context, false); 
> > expand_and_allocate(word_size, context); 
> > do_collection(false, false, word_size); 
> > attempt_allocation_at_safepoint(word_size, context, true); 
> > expand_and_allocate(word_size, context); 
> > do_collection(false, true, word_size); 
> > attempt_allocation_at_safepoint(word_size, context, true); 
> > assert(!collector_policy()->should_clear_all_soft_refs()); 
> > expand_and_allocate(word_size, context); 
> > 
> > You have chosen to break this into: 
> > 
> > attempt_allocation_at_safepoint(word_size, context, false); 
> > expand_and_allocate(word_size, context); 
> > 
> > satisfy_failed_allocation_helper(word_size, context, false,
> > succeeded); 
> >    do_collection(false, false, word_size); 
> >    attempt_allocation_at_safepoint(word_size, context, true); 
> >    expand_and_allocate(word_size, context); 
> > 
> > satisfy_failed_allocation_helper(word_size, context, true,
> > succeeded); 
> >    do_collection(false, true, word_size); 
> >    attempt_allocation_at_safepoint(word_size, context, true); 
> >    expand_and_allocate(word_size, context); 
> > 
> > assert(!collector_policy()->should_clear_all_soft_refs()); 
> > 
> > This is not entirely identical, depending on how important the
> > assert is that could be an issue since the assert is now skipped in
> > some cases where it was previously used. 

The assert just states that after setting clear_all_soft_refs to true in
the call to do_collection, it must be false after collection.

So actually one could change the assert to

assert(!clear_all_soft_refs || !
collector_policy()->should_clear_all_soft_refs(), "...") and put it
directly in the helper.

Actually I think CollectorPolicy::clear_all_soft_refs() must be false
after any full gc, so it should be placed after the do_collection call.
> > 
> > My suggestion would be to break the code up like this: 
> > 
> > satisfy_failed_allocation_helper(word_size, context, do_gc,
> > clear_softrefs) 
> >    attempt_allocation_at_safepoint(word_size, context, false); 
> >    expand_and_allocate(word_size, context); 
> >    do_collection(false, false, word_size); 
> > 
> > satisfy_failed_allocation_helper(word_size, context, do_gc,
> > clear_softrefs) 
> >    attempt_allocation_at_safepoint(word_size, context, true); 
> >    expand_and_allocate(word_size, context); 
> >    do_collection(false, true, word_size); 
> > 
> > satisfy_failed_allocation_helper(word_size, context, do_gc,
> > clear_softrefs) 
> >    attempt_allocation_at_safepoint(word_size, context, true); 
> >    if (!do_gc) 
> >       assert(!collector_policy()->should_clear_all_soft_refs()); 
> >    expand_and_allocate(word_size, context); 
> > 
> > This would make the code in satisfy_failed_allocation() a lot more
> > clean. 
> > 
> In webrev.03 I decided to put same way as it was in the original code
> (before any of my changes), 
> at the very end of method just when we already exhausted all
> possibilities and about to return NULL.
> 
> attempt_allocation_at_safepoint(word_size, context, false); 
> expand_and_allocate(word_size, context); 
> do_collection(false, false, word_size); 
> attempt_allocation_at_safepoint(word_size, context, true); 
> do_collection(false, true, word_size); 
> attempt_allocation_at_safepoint(word_size, context, true); 
> assert(!collector_policy()->should_clear_all_soft_refs()); 
> 
>  // What else? We might try synchronous finalization later. If the
> total 1799 // space available is large enough for the allocation, then
> a more 
>  // complete compaction phase than we've tried so far might be 
> // appropriate. 
> assert(*succeeded, "sanity");
> return NULL; 1804 
> }
> 
> webrev.03 has this assert at same position, and I prefer it this way.
> 
> > 
> > * Unless I'm missing some subtle detail here, the usage of the
> > variable 'succeeded' is unnecessary as 'result' will always contain
> > the same information. I don't think we should pass on 'succeeded' to
> > the helper, but simply set it afterwards before we return from
> > satisfy_failed_allocation(). 

> No, helper may return NULL in two differnt cases:
> 1. Full GC failed, *succeeded = false, no attempts to allocate was
> made
> 2. Full GC succeeded, *succeeded = true, all attempts to allocate
> failed 
> 
> So we need to pass 'succeeded' to the helper.

Can we rename this to something more descriptive? I had to look into
do_collection() what it means. Even the gc_succeeded name in the helper
is better.

Also, the only condition where do_collection() fails is if the gc locker
is active. This return value is actually ignored in
do_full_collection().

So the gc locker check could be moved out of do_collection() (adding an
assert there), and the value of gc_succeeded/succeeded determined using
this check immediately.

At least, in the helper, move the "assert(*succeeded, ...)" up.

Like:
--------------

*succeeded = !GCLocker::check_active_before_gc(); // maybe use a temp
variable here, that is not negated because we check the negated result
again just later

if (!*succeeded) {
  return NULL;
}

do_collection(....);
assert(
   !collector_policy()->should_clear_all_soft_refs(), ...); // see above

assert(*succeeded, "..."); // for the paranoid, but I would just skip
it.

result = attempt_allocation_at_safepoint(...);

if (result != NULL) {
  return result;
}

result = expand_and_allocate(...);

return result;
---------------

(And the bool return value of do_collection removed).

I also have a question about the order of the order of
attempt_allocation_at_safepoint() and the expand_and_allocate(), and
could not find this answered before: so full gc might shrink the heap
too much after full gc, and we should expand the heap.
If I were to do this change, there would always (unconditionally) be an
expansion after full gc, because the current change only expands if the
allocation after gc fails. Also the 02 change.

The review thread so far suggests that full gc might shrink too much,
but this still seems to be the case here in some cases.

Thanks,
  Thomas




From kirill.zhaldybin at oracle.com  Mon Aug 24 13:23:15 2015
From: kirill.zhaldybin at oracle.com (Kirill Zhaldybin)
Date: Mon, 24 Aug 2015 16:23:15 +0300
Subject: RFR (S): 8132708: Add tests for Humongous objects allocation
	threshold
In-Reply-To: <1440409997.2348.8.camel@oracle.com>
References: <55D60438.4080004@oracle.com> <1440409997.2348.8.camel@oracle.com>
Message-ID: <55DB1AC3.6010004@oracle.com>

Hi Thomas,

Thank you for reviewing this fix.

Could you please read comments inline?

Regards, Kirill

On 24.08.2015 12:53, Thomas Schatzl wrote:
> Hi Kirill,
>
> On Thu, 2015-08-20 at 19:45 +0300, Kirill Zhaldybin wrote:
>> Hi!
>>
>> Could you please review a new simple test checking that only objects
>> allocating more than  a half regions are allocated as humongous?
>>
>> CR:
>> https://bugs.openjdk.java.net/browse/JDK-8132708
>>
>> Webrev:
>> hotspot:
>> http://cr.openjdk.java.net/~kzhaldyb/webrevs/JDK-8132708/webrev.03/
>>
>> Tests:
>> jprt
>>
>
>   - MAX_CONTINIUS_SIZE_CHECK: typo in "CONTINUOUS".
Will be fixed in next iteration.

>
>   - I am a bit puzzled about the verification whether the object should
> be humongous. Shouldn't the test program ask the VM whether a given
> object is humongous or not?
> Currently it seems to me that the test program only checks itself, since
> it on the one hand allocates objects of varying sizes, using
> REGION_SIZE/2 and then in allocateAndCheck again uses this same formula
> to determine that an object is humongous.
>
> At most the test program seems to verify that
> Helpers.detectByteArrayAllocationOverhead() is correct.
>
> In this case I would expect a whitebox method to ask the VM whether the
> given object is a humongous object.

There are two asserts in allocateAndCheck:
1. We do some obscure calculations to find out exact byte array's length 
we need to allocate largest non-humongous/smallest humongous object.
I think it worth to double-check that we calculated right.
The first check gets real allocation size (using white box method 
getObjectSize(Object)) and verifies that allocation was as we expected. 
If it's not as we expected it's likely a test bug.

2. The second check uses White box method g1IsHumongous(Object) and 
verifies that objects larger that a half of region are humongous and 
other object are non-humongous (I believe that is definition for 
humongous objects).

The first check could be considered redundant but since object layout, 
padding etc could be changed in future I think it is worth to do it.
It will allow easily distinguish a test bug from a product bug.

>
> Thanks,
>    Thomas
>
>



From eric.caspole at oracle.com  Mon Aug 24 13:29:05 2015
From: eric.caspole at oracle.com (Eric Caspole)
Date: Mon, 24 Aug 2015 09:29:05 -0400
Subject: An interesting bug
In-Reply-To: <55DAE48F.2030403@redhat.com>
References: <55DAE48F.2030403@redhat.com>
Message-ID: <55DB1C21.5060100@oracle.com>

Hi Andrew,
If I understand you, the ForkJoinTask is located in the old generation. 
When you get to the crash point, where did the wrong "this" pointer (a 
ref to a ForkJoinTask, right?) come from? Presumably, from a field of 
another object? If you know what object that is, what generation is that 
object in? That might help narrow it down.
Regards,
Eric


On 08/24/2015 05:31 AM, Andrew Haley wrote:
> This is JDK8 on AArch64.
>
> I have an interesting bug which causes a crash.  Occasionally, but
> rarely, I get a crash when accessing an instance of ForkJoinTask.  The
> instance is allocated in the old generation of the parallel scan GC.
> However, the "this" pointer of the ForkJoinTask points into the middle
> of another allocated object.
>
> I'm trying to figure out how this might happen.  Perhaps the root scan
> of the thread running the ForkJoinTask did not happen, or for some
> reason the roots it found were not added to the root set.  Or perhaps
> they were found, but were not rewritten in the thread.
>
> Has anyone seen anything like this before?  Do you have any advice for
> me?
>
> Thanks,
> Andrew.
>


From thomas.schatzl at oracle.com  Mon Aug 24 13:31:54 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 24 Aug 2015 15:31:54 +0200
Subject: RFR (S): 8132708: Add tests for Humongous objects allocation
	threshold
In-Reply-To: <55DB1AC3.6010004@oracle.com>
References: <55D60438.4080004@oracle.com>
	<1440409997.2348.8.camel@oracle.com> <55DB1AC3.6010004@oracle.com>
Message-ID: <1440423114.2348.93.camel@oracle.com>

Hi Kirill,

On Mon, 2015-08-24 at 16:23 +0300, Kirill Zhaldybin wrote:
> Hi Thomas,
[...]
> >
> >   - MAX_CONTINIUS_SIZE_CHECK: typo in "CONTINUOUS".
> Will be fixed in next iteration.
> 
> >
> >   - I am a bit puzzled about the verification whether the object should
> > be humongous. Shouldn't the test program ask the VM whether a given
> > object is humongous or not?
> > Currently it seems to me that the test program only checks itself, since
> > it on the one hand allocates objects of varying sizes, using
> > REGION_SIZE/2 and then in allocateAndCheck again uses this same formula
> > to determine that an object is humongous.
> >
> > At most the test program seems to verify that
> > Helpers.detectByteArrayAllocationOverhead() is correct.
> >
> > In this case I would expect a whitebox method to ask the VM whether the
> > given object is a humongous object.
> 
> There are two asserts in allocateAndCheck:
> 1. We do some obscure calculations to find out exact byte array's length 
> we need to allocate largest non-humongous/smallest humongous object.
> I think it worth to double-check that we calculated right.
> The first check gets real allocation size (using white box method 
> getObjectSize(Object)) and verifies that allocation was as we expected. 
> If it's not as we expected it's likely a test bug.
> 
> 2. The second check uses White box method g1IsHumongous(Object) and 
> verifies that objects larger that a half of region are humongous and 
> other object are non-humongous (I believe that is definition for 
> humongous objects).
> 

Okay, I just overlooked that somehow. Then there is only the typo that
should be fixed.

Thanks,
  Thomas




From thomas.schatzl at oracle.com  Mon Aug 24 13:43:48 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 24 Aug 2015 15:43:48 +0200
Subject: RFR (M): 8067336: Allow that PLAB allocations at the end of
	regions are flexible
In-Reply-To: <55DAFD6B.8040304@oracle.com>
References: <1439896720.2316.7.camel@oracle.com> <55D34955.10006@oracle.com>
	<1440019109.1994.51.camel@oracle.com> <55D500FA.3070105@oracle.com>
	<1440412588.2348.36.camel@oracle.com> <55DAFD6B.8040304@oracle.com>
Message-ID: <1440423828.2348.99.camel@oracle.com>

Hi Mikael,

On Mon, 2015-08-24 at 13:18 +0200, Mikael Gerdin wrote:
> Hi Thomas,
> 
> On 2015-08-24 12:36, Thomas Schatzl wrote:
> > Hi Tom,
> >
> >    thanks for the review.
> >
> > On Wed, 2015-08-19 at 18:19 -0400, Tom Benson wrote:
> >> Hi Thomas -
> >> Looks good to me.   Though there was one odd thing I noticed, not in the
> >> changes but adjacent to one - this comment in g1Allocator.cpp:
> >>
> >>    273     if (buf != NULL) {
> >>    274       // Otherwise.
> >>    275       alloc_buf->set_buf(buf, actual_plab_size);
> >>
> >> There is another "Otherwise." comment just afterward at line 283, where
> >> it makes sense, but the one at 274 looks left over from some past
> >> restructuring.  Your call.  8^)
> >> Tom
> >
> > I will remove the one in 274.
> >
> > New webrevs:
> > http://cr.openjdk.java.net/~tschatzl/8067336/webrev.1_to_2/index.html
> > (diff)
> > http://cr.openjdk.java.net/~tschatzl/8067336/webrev.2/index.html (full)
> 
> Overall I think this change is good.
> I'm somewhat paranoid about output parameters, can you please add code 
> to initialize the value to be set by the out parameter to a dummy value 
> (-1?) and assert that it was properly set on the code paths where the 
> actual size is being used.
> 
> The alignment of the \ in vmStructs_g1.hpp is wrong, I don't need to see 
> that in a new webrev though :)

  thanks for the review. I fixed these issues in

http://cr.openjdk.java.net/~tschatzl/8067336/webrev.3 (full)
http://cr.openjdk.java.net/~tschatzl/8067336/webrev.2_to_3 (diff)

The code sets the out parameter to zero and then checks whether it is
within min/max after the call if the allocation had been successful.

Testing:
jprt

Thanks,
  Thomas




From tom.benson at oracle.com  Mon Aug 24 14:04:29 2015
From: tom.benson at oracle.com (Tom Benson)
Date: Mon, 24 Aug 2015 10:04:29 -0400
Subject: RFR (M): 8067336: Allow that PLAB allocations at the end of
	regions are flexible
In-Reply-To: <1440423828.2348.99.camel@oracle.com>
References: <1439896720.2316.7.camel@oracle.com>
	<55D34955.10006@oracle.com>		<1440019109.1994.51.camel@oracle.com>
	<55D500FA.3070105@oracle.com>	
	<1440412588.2348.36.camel@oracle.com> <55DAFD6B.8040304@oracle.com>
	<1440423828.2348.99.camel@oracle.com>
Message-ID: <55DB246D.8060108@oracle.com>

Hi Thomas,
The update looks good to me.
Tom

On 8/24/2015 9:43 AM, Thomas Schatzl wrote:
> Hi Mikael,
>
> On Mon, 2015-08-24 at 13:18 +0200, Mikael Gerdin wrote:
>> Hi Thomas,
>>
>> On 2015-08-24 12:36, Thomas Schatzl wrote:
>>> Hi Tom,
>>>
>>>     thanks for the review.
>>>
>>> On Wed, 2015-08-19 at 18:19 -0400, Tom Benson wrote:
>>>> Hi Thomas -
>>>> Looks good to me.   Though there was one odd thing I noticed, not in the
>>>> changes but adjacent to one - this comment in g1Allocator.cpp:
>>>>
>>>>     273     if (buf != NULL) {
>>>>     274       // Otherwise.
>>>>     275       alloc_buf->set_buf(buf, actual_plab_size);
>>>>
>>>> There is another "Otherwise." comment just afterward at line 283, where
>>>> it makes sense, but the one at 274 looks left over from some past
>>>> restructuring.  Your call.  8^)
>>>> Tom
>>> I will remove the one in 274.
>>>
>>> New webrevs:
>>> http://cr.openjdk.java.net/~tschatzl/8067336/webrev.1_to_2/index.html
>>> (diff)
>>> http://cr.openjdk.java.net/~tschatzl/8067336/webrev.2/index.html (full)
>> Overall I think this change is good.
>> I'm somewhat paranoid about output parameters, can you please add code
>> to initialize the value to be set by the out parameter to a dummy value
>> (-1?) and assert that it was properly set on the code paths where the
>> actual size is being used.
>>
>> The alignment of the \ in vmStructs_g1.hpp is wrong, I don't need to see
>> that in a new webrev though :)
>    thanks for the review. I fixed these issues in
>
> http://cr.openjdk.java.net/~tschatzl/8067336/webrev.3 (full)
> http://cr.openjdk.java.net/~tschatzl/8067336/webrev.2_to_3 (diff)
>
> The code sets the out parameter to zero and then checks whether it is
> within min/max after the call if the allocation had been successful.
>
> Testing:
> jprt
>
> Thanks,
>    Thomas
>
>



From mikael.gerdin at oracle.com  Mon Aug 24 14:20:09 2015
From: mikael.gerdin at oracle.com (Mikael Gerdin)
Date: Mon, 24 Aug 2015 16:20:09 +0200
Subject: RFR (M): 8067336: Allow that PLAB allocations at the end of
	regions are flexible
In-Reply-To: <1440423828.2348.99.camel@oracle.com>
References: <1439896720.2316.7.camel@oracle.com>
	<55D34955.10006@oracle.com>		<1440019109.1994.51.camel@oracle.com>
	<55D500FA.3070105@oracle.com>	
	<1440412588.2348.36.camel@oracle.com> <55DAFD6B.8040304@oracle.com>
	<1440423828.2348.99.camel@oracle.com>
Message-ID: <55DB2819.7030304@oracle.com>

Thomas,

On 2015-08-24 15:43, Thomas Schatzl wrote:
> Hi Mikael,
>
> On Mon, 2015-08-24 at 13:18 +0200, Mikael Gerdin wrote:
>> Hi Thomas,
>>
>> On 2015-08-24 12:36, Thomas Schatzl wrote:
>>> Hi Tom,
>>>
>>>     thanks for the review.
>>>
>>> On Wed, 2015-08-19 at 18:19 -0400, Tom Benson wrote:
>>>> Hi Thomas -
>>>> Looks good to me.   Though there was one odd thing I noticed, not in the
>>>> changes but adjacent to one - this comment in g1Allocator.cpp:
>>>>
>>>>     273     if (buf != NULL) {
>>>>     274       // Otherwise.
>>>>     275       alloc_buf->set_buf(buf, actual_plab_size);
>>>>
>>>> There is another "Otherwise." comment just afterward at line 283, where
>>>> it makes sense, but the one at 274 looks left over from some past
>>>> restructuring.  Your call.  8^)
>>>> Tom
>>>
>>> I will remove the one in 274.
>>>
>>> New webrevs:
>>> http://cr.openjdk.java.net/~tschatzl/8067336/webrev.1_to_2/index.html
>>> (diff)
>>> http://cr.openjdk.java.net/~tschatzl/8067336/webrev.2/index.html (full)
>>
>> Overall I think this change is good.
>> I'm somewhat paranoid about output parameters, can you please add code
>> to initialize the value to be set by the out parameter to a dummy value
>> (-1?) and assert that it was properly set on the code paths where the
>> actual size is being used.
>>
>> The alignment of the \ in vmStructs_g1.hpp is wrong, I don't need to see
>> that in a new webrev though :)
>
>    thanks for the review. I fixed these issues in
>
> http://cr.openjdk.java.net/~tschatzl/8067336/webrev.3 (full)
> http://cr.openjdk.java.net/~tschatzl/8067336/webrev.2_to_3 (diff)

Looks good to me.
/Mikael

>
> The code sets the out parameter to zero and then checks whether it is
> within min/max after the call if the allocation had been successful.
>
> Testing:
> jprt
>
> Thanks,
>    Thomas
>
>


From kirill.zhaldybin at oracle.com  Mon Aug 24 15:23:21 2015
From: kirill.zhaldybin at oracle.com (Kirill Zhaldybin)
Date: Mon, 24 Aug 2015 18:23:21 +0300
Subject: RFR (S): 8132708: Add tests for Humongous objects allocation
	threshold
In-Reply-To: <1440423114.2348.93.camel@oracle.com>
References: <55D60438.4080004@oracle.com>	 <1440409997.2348.8.camel@oracle.com>
	<55DB1AC3.6010004@oracle.com> <1440423114.2348.93.camel@oracle.com>
Message-ID: <55DB36E9.6020806@oracle.com>

Hi Thomas,

New webrev (with fixed typo) is here: 
http://cr.openjdk.java.net/~kzhaldyb/webrevs/JDK-8132708/webrev.04/

Thank you.

Regards, Kirill


On 24.08.2015 16:31, Thomas Schatzl wrote:
> Hi Kirill,
>
> On Mon, 2015-08-24 at 16:23 +0300, Kirill Zhaldybin wrote:
>> Hi Thomas,
> [...]
>>>
>>>    - MAX_CONTINIUS_SIZE_CHECK: typo in "CONTINUOUS".
>> Will be fixed in next iteration.
>>
>>>
>>>    - I am a bit puzzled about the verification whether the object should
>>> be humongous. Shouldn't the test program ask the VM whether a given
>>> object is humongous or not?
>>> Currently it seems to me that the test program only checks itself, since
>>> it on the one hand allocates objects of varying sizes, using
>>> REGION_SIZE/2 and then in allocateAndCheck again uses this same formula
>>> to determine that an object is humongous.
>>>
>>> At most the test program seems to verify that
>>> Helpers.detectByteArrayAllocationOverhead() is correct.
>>>
>>> In this case I would expect a whitebox method to ask the VM whether the
>>> given object is a humongous object.
>>
>> There are two asserts in allocateAndCheck:
>> 1. We do some obscure calculations to find out exact byte array's length
>> we need to allocate largest non-humongous/smallest humongous object.
>> I think it worth to double-check that we calculated right.
>> The first check gets real allocation size (using white box method
>> getObjectSize(Object)) and verifies that allocation was as we expected.
>> If it's not as we expected it's likely a test bug.
>>
>> 2. The second check uses White box method g1IsHumongous(Object) and
>> verifies that objects larger that a half of region are humongous and
>> other object are non-humongous (I believe that is definition for
>> humongous objects).
>>
>
> Okay, I just overlooked that somehow. Then there is only the typo that
> should be fixed.
>
> Thanks,
>    Thomas
>
>



From jiangli.zhou at oracle.com  Mon Aug 24 16:47:56 2015
From: jiangli.zhou at oracle.com (Jiangli Zhou)
Date: Mon, 24 Aug 2015 09:47:56 -0700
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <55DA32FC.8000405@oracle.com>
References: <55C103A4.1060505@oracle.com> <55C21DC3.5090305@oracle.com>
	<1438785532.2378.73.camel@oracle.com> <55C248FF.6010308@oracle.com>
	<F73C4B9A-4DBA-47A8-8715-0988FF6D393D@oracle.com>
	<55C26249.1090902@oracle.com>
	<C149705F-696B-46CD-8FB8-CAD06C511244@oracle.com>
	<55C2751B.8070400@oracle.com> <1438847329.2009.9.camel@oracle.com>
	<55C36B31.7000908@oracle.com> <1438871723.2474.37.camel@oracle.com>
	<55C373F6.9060609@oracle.com>
	<AD45178E-A442-4A0D-80D6-50E91608E1FE@oracle.com>
	<55C3D8BB.9000603@oracle.com>
	<48C0E571-0D7B-4D30-85DF-E31D99092A7C@oracle.com>
	<30B0248C-3BEA-4FAE-861F-DAAE19F56B45@oracle.com>
	<55C4B806.7050504@oracle.com> <55C4C720.5030903@oracle.com>
	<55CA3430.3070300@oracle.com> <1439377259.2324.27.camel@oracle.com>
	<55CCF0DC.2000800@oracle.com> <1440079572.2347.11.camel@oracle.com>
	<55D5E048.5070404@oracle.com>
	<8566C008-5A60-4376-94C4-968E1A475769@oracle.com>
	<55D742E7.9080208@oracle.com>
	<CAA5A1C9-88F6-4EFE-9C6B-E7CAA366589F@oracle.com>
	<55DA32FC.8000405@! oracle.com>
Message-ID: <0D4F4950-644A-4CBB-A4E4-F3EC16044A8C@oracle.com>

Thanks, Dmitry!

Jiangli

On Aug 23, 2015, at 1:54 PM, Dmitry Dmitriev <dmitry.dmitriev at oracle.com> wrote:

> Hi Jiangli,
> 
> Looks good to me!
> 
> Thank you,
> Dmitry
> 
> On 22.08.2015 4:47, Jiangli Zhou wrote:
>> Hi Dmitry,
>> 
>> On Aug 21, 2015, at 8:25 AM, Dmitry Dmitriev <dmitry.dmitriev at oracle.com> wrote:
>> 
>>> Hello Jiangli,
>>> 
>>> This looks good to me, but I'm not a reviewer.
>> Thanks.
>> 
>>> Also, I have question to you. Probably you can clarify me one moment.
>>> num_ranges and string_ranges are modified only in code under "#if INCLUDE_ALL_GCS", so it make sense to include all usage of these variables also under "#if INCLUDE_ALL_GCS"? I mean following functions:
>>> FileMapInfo::fixup_string_regions() and new FileMapInfo::dealloc_string_regions() function.
>> Agreed. Here is the updated webrev: http://cr.openjdk.java.net/~jiangli/8131734/webrev.02/src/share/vm/memory/filemap.cpp.sdiff.html.
>> 
>> I also reverified JPRT builds with the new #ifdef changes.
>> 
>> Thanks for the detailed review!
>> 
>> Jiangli
>> 
>>> Thank you,
>>> Dmitry
>>> 
>>> On 20.08.2015 22:55, Jiangli Zhou wrote:
>>>> Hi Dmitry,
>>>> 
>>>> Here is the updated runtime webrev that reflects Tom?s latest GC changes.
>>>> 
>>>> http://cr.openjdk.java.net/~jiangli/8131734/webrev.01/
>>>> 
>>>> I renamed the FileMapInfo::unmap_string_regions() to FileMapInfo::dealloc_string_regions(), which only deallocates the archived string region from the java heap without unmapping. The unmapping is handled by the GC system as the archived string region is part of the java heap. I also added dealloc_string_regions() call to the case where the string region verification fails.
>>>> 
>>>> Thanks,
>>>> Jiangli
>>>> 
>>>> On Aug 20, 2015, at 7:12 AM, Tom Benson <tom.benson at oracle.com> wrote:
>>>> 
>>>>> Hi Thomas,
>>>>> OK, thanks!
>>>>> Tom
>>>>> 
>>>>> On 8/20/2015 10:06 AM, Thomas Schatzl wrote:
>>>>>> Hi Tom,
>>>>>> 
>>>>>>   sorry for the delay...
>>>>>> 
>>>>>> On Thu, 2015-08-13 at 15:32 -0400, Tom Benson wrote:
>>>>>>> Hi Thomas,
>>>>>>> 
>>>>>>> On 8/12/2015 7:00 AM, Thomas Schatzl wrote:
>>>>>>>> Hi,
>>>>>>>> 
>>>>>>>> On Tue, 2015-08-11 at 13:43 -0400, Tom Benson wrote:
>>>>>>>>> Hi,
>>>>>>>>> On 8/7/2015 10:56 AM, Tom Benson wrote:
>>>>>>>>> After some discussion, I've changed the definition and name of
>>>>>>>>> free_archive_regions.  Now called dealloc_archive_regions, it uncommits
>>>>>>>>> the specified regions, unmapping the memory, rather than adding them to
>>>>>>>>> the free list.  This means the CDS code will no longer do the unmapping
>>>>>>>>> on verification failures.
>>>>>>>>> 
>>>>>>>>> Updated full and incremental webrevs of the GC code are at:
>>>>>>>>> http://cr.openjdk.java.net/~tbenson/8131734/webrev.02/
>>>>>>>>> http://cr.openjdk.java.net/~tbenson/8131734/webrev.02.vs.01/
>>>>>>>>> 
>>>>>>>>> Tested with JPRT and running benchmarks with the dealloc_ performed
>>>>>>>>> explicitly.  Jiangli also tested the original failing cases, and will be
>>>>>>>>> posting an updated webrev.
>>>>>>>> - is it possible that shrink_by() uses shrink_at()? This would avoid two
>>>>>>>> paths that uncommit regions like expand_by()/expand_at()?
>>>>>>> OK, I made the change.  I didn't do it originally because the asserts I
>>>>>>> wanted to add for the call from g1CollectedHeap seemed superfluous for
>>>>>>> the other call, and shrink_at was so small.    Now shrink_at takes a
>>>>>>> region count as well.
>>>>>>> 
>>>>>>> Updated full and incremental webrevs are at:
>>>>>>>    http://cr.openjdk.java.net/~tbenson/8131734/webrev.03/
>>>>>>>    http://cr.openjdk.java.net/~tbenson/8131734/webrev.03.vs.02/
>>>>>>> 
>>>>>>> 
>>>>>> Looks good.
>>>>>> 
>>>>>>>> - I think the change should call at least HeapRegion::hr_clear() on the
>>>>>>>> region to remove or reset any auxiliary data structures, if not
>>>>>>>> G1CollectedHeap::free_region() (without adding the region to the free
>>>>>>>> list).
>>>>>>>> Since the HeapRegion* is not deallocated by the uncommit, this may cause
>>>>>>>> strange behavior later when the region is reused.
>>>>>>> I don't think calling hr_clear should be necessary...  If it is, we
>>>>>>> should be doing it in shrink_by as well, and I don't think we are. I
>>>>>>> don't see how a HeapRegion can be 'reused' without having gone through
>>>>>>> the constructor when expand_ asks (indirectly) for 'new HeapRegion', and
>>>>>>> that does an hr_clear() as well as the rest of init.     Or am I missing
>>>>>>> something there?
>>>>>> Leave it as is. I thought that a full gc (which is the only case where
>>>>>> the heap shrinks at the moment) also clears the remset of these regions
>>>>>> at least.
>>>>>> 
>>>>>> It should, I filed  JDK-8134048 for looking in this issue.
>>>>>> 
>>>>>> Looks good.
>>>>>> 
>>>>>> Thanks,
>>>>>>   Thomas
>>>>>> 
>>>>>> 
> 



From michail.chernov at oracle.com  Mon Aug 24 18:53:43 2015
From: michail.chernov at oracle.com (Michail Chernov)
Date: Mon, 24 Aug 2015 21:53:43 +0300
Subject: RFR: 8081317: [NEWTEST] documented GC ratio tuning and new size
	options should be covered by regression tests
In-Reply-To: <55D21B5E.7020100@oracle.com>
References: <55BBB0BE.6080506@oracle.com> <55C0CD95.1080305@oracle.com>
	<55D21B5E.7020100@oracle.com>
Message-ID: <55DB6837.40003@oracle.com>

Hi Dmitry,

Thanks a lot!

Michail

On 17.08.2015 20:35, Dmitry Fazunenko wrote:
> Hi Michail,
>
> the fix looks good to me.
>
> Thanks,
> Dima
>
> On 04.08.2015 17:35, Michail Chernov wrote:
>> Hi,
>>
>> Could someone please take a look at this review?
>>
>> Thanks,
>> Michail
>>
>> On 31.07.2015 20:30, Michail Chernov wrote:
>>> Hi,
>>>
>>> Please review these changes for JDK-8081317. There are new tests to 
>>> cover GC ratio tuning and new size options.
>>>
>>> Webrev:  http://cr.openjdk.java.net/~mchernov/8081317/
>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8081317
>>>
>>> TestMaxMinHeapFreeRatioFlags.java - Verify that heap size changes 
>>> according to max and min heap free ratios.
>>>
>>> TestMinAndInitialSurvivorRatioFlags.java - Test verifies that VM can 
>>> start with any GC when MinSurvivorRatio and InitialSurvivorRatio 
>>> flags passed and for Parallel GC it verifies that after start up 
>>> survivor ratio is equal to InitialSurvivorRatio value and that 
>>> actual survivor ratio will never be less than MinSurvivorRatio.
>>>
>>> TestNewRatioFlag.java - Verify that heap divided among generations 
>>> according to NewRatio.
>>>
>>> TestNewSizeFlags.java - Verify that young gen size conforms values 
>>> specified by NewSize, MaxNewSize and Xmn options.
>>>
>>> TestSurvivorRatioFlag - Verify that actual survivor ratio is equal 
>>> to specified SurvivorRatio value.
>>>
>>> TestTargetSurvivorRatioFlag.java - Test if objects size is less than 
>>> (survivor_size * TargetSurvivorRatio / 100) then objects are stayed 
>>> in survivor space until MaxTenuringThreshold minor GC cycles. If 
>>> more than (survivor_size * TargetSurvivorRatio / 100) objects were 
>>> allocated, then test verifies that after MaxTenuringThreshold minor 
>>> GC cycles survivor space is almost empty.
>>>
>>> New changes were tested on different platforms including embedded 
>>> jdk on Linux x86.
>>>
>>> Thanks,
>>> Michail
>>>
>>
>
>



From alexander.harlap at oracle.com  Mon Aug 24 21:24:14 2015
From: alexander.harlap at oracle.com (Alexander Harlap)
Date: Mon, 24 Aug 2015 17:24:14 -0400
Subject: Request for code review - 8130265: gctests/LargeObjects/large001
	fails with OutOfMemoryError: Java heap space
In-Reply-To: <1440420922.2348.82.camel@oracle.com>
References: <55D356FB.1030603@oracle.com> <55D35A12.8020608@oracle.com>
	<55D35F1F.3090700@oracle.com> <55D36DC7.6010901@oracle.com>
	<55D38CBE.8070408@oracle.com> <55D394AD.6020104@oracle.com>
	<55D4E39D.5070805@oracle.com> <55D4E8C7.1080703@oracle.com>
	<55D4EEF5.9030504@oracle.com>
	<2C141666-BDBC-428C-9E40-4FB78F076192@oracle.com>
	<1440082894.2347.14.camel@oracle.com> <55D62D77.8020504@oracle.com>
	<55D71703.4070505@oracle.com> <55D74766.5020202@oracle.com>
	<1440420922.2348.82.camel@oracle.com>
Message-ID: <55DB8B7E.5010601@oracle.com>

Hi Thomas and Jesper,

Here is new version of change:

http://cr.openjdk.java.net/~aharlap/8130265/webrev.04/

I followed Jesper's advice to have
satisfy_failed_allocation_helper(..) :
   attempt_allocation_at_safepoint(...);
   expand_and_allocate(...);
   do_collection(...);

So I did not reorder  calls to attempt_allocation_at_safepoint(...) and 
expand_and_allocate(...)

-Alex


On 8/24/2015 8:55 AM, Thomas Schatzl wrote:
> Hi,
>
> On Fri, 2015-08-21 at 11:44 -0400, Alexander Harlap wrote:
>>
>> On 8/21/2015 8:18 AM, Jesper Wilhelmsson wrote:
>>
>>> Hi Alex,
>>>
>>> Alexander Harlap skrev den 20/8/15 21:41:
>>>> Kim and Thomas,
>>>>
>>>> Here is new version with your suggestions in:
>>>>
>>>> http://cr.openjdk.java.net/~aharlap/8130265/webrev.03/
>>>> <http://cr.openjdk.java.net/%7Eaharlap/8130265/webrev.03/>
>>>> Alex
>>> A few comments on this latest version.
>>>
>>> * Why do you assert_at_safepoint and set succeeded to true in your
>>> helper? The original code did not do this at these places. I think
>>> you can remove these lines.
>>>
>>> 1736   assert_at_safepoint(true /* should_be_vm_thread */);
>>> 1737
>>> 1738   *succeeded = true;
>>>
>> Even method called satisfy_failed_allocation_helper , there is no way
>> to make sure that it is called only  from satisfy_failed_allocation.
> I do not think we guarantee that for any method, i.e. that a given
> method somehow checks its call chain. This would require much better
> encapsulation, and in some cases this does not help you either. Also it
> would require you to state the allowed callers somehow.
>
> The helper may be private too, not protected.
>
>> satisfy_failed_allocation_helper is a separate method and those lines
>> are needed.
>> So I would like to keep
>> 1736   assert_at_safepoint(true /* should_be_vm_thread */);
>> 1737
>> 1738   *succeeded = true;
>>> * Please align the arguments in
>>>
>>> 1732 G1CollectedHeap::satisfy_failed_allocation_helper(size_t
>>> word_size,
>>> 1733                                            AllocationContext_t
>>> context,
>>> 1734                                            bool
>>> clear_all_soft_refs,
>>> 1735                                            bool* succeeded) {
>>>
>>> and
>>>
>>> 1748   HeapWord* result =
>>> attempt_allocation_at_safepoint(word_size,
>>> 1749                                            context,
>>> 1750                                            true /*
>>> expect_null_mutator_alloc_region */);
>>>
>>> and also in the header file.
> Please also avoid method definitions like
>
> <return value>
> <class name>::<method-name> ...
>
> (or even worse
>
> <return value>
> <class-name>::
> <method-name>
>
> (this change does not do it that way)
> )
>> Sure.
>>
>>> * Your last version did:
>>>
>>> attempt_allocation_at_safepoint(word_size, context, false);
>>> expand_and_allocate(word_size, context);
>>> do_collection(false, false, word_size);
>>> attempt_allocation_at_safepoint(word_size, context, true);
>>> expand_and_allocate(word_size, context);
>>> do_collection(false, true, word_size);
>>> attempt_allocation_at_safepoint(word_size, context, true);
>>> assert(!collector_policy()->should_clear_all_soft_refs());
>>> expand_and_allocate(word_size, context);
>>>
>>> You have chosen to break this into:
>>>
>>> attempt_allocation_at_safepoint(word_size, context, false);
>>> expand_and_allocate(word_size, context);
>>>
>>> satisfy_failed_allocation_helper(word_size, context, false,
>>> succeeded);
>>>     do_collection(false, false, word_size);
>>>     attempt_allocation_at_safepoint(word_size, context, true);
>>>     expand_and_allocate(word_size, context);
>>>
>>> satisfy_failed_allocation_helper(word_size, context, true,
>>> succeeded);
>>>     do_collection(false, true, word_size);
>>>     attempt_allocation_at_safepoint(word_size, context, true);
>>>     expand_and_allocate(word_size, context);
>>>
>>> assert(!collector_policy()->should_clear_all_soft_refs());
>>>
>>> This is not entirely identical, depending on how important the
>>> assert is that could be an issue since the assert is now skipped in
>>> some cases where it was previously used.
> The assert just states that after setting clear_all_soft_refs to true in
> the call to do_collection, it must be false after collection.
>
> So actually one could change the assert to
>
> assert(!clear_all_soft_refs || !
> collector_policy()->should_clear_all_soft_refs(), "...") and put it
> directly in the helper.
>
> Actually I think CollectorPolicy::clear_all_soft_refs() must be false
> after any full gc, so it should be placed after the do_collection call.
>>> My suggestion would be to break the code up like this:
>>>
>>> satisfy_failed_allocation_helper(word_size, context, do_gc,
>>> clear_softrefs)
>>>     attempt_allocation_at_safepoint(word_size, context, false);
>>>     expand_and_allocate(word_size, context);
>>>     do_collection(false, false, word_size);
>>>
>>> satisfy_failed_allocation_helper(word_size, context, do_gc,
>>> clear_softrefs)
>>>     attempt_allocation_at_safepoint(word_size, context, true);
>>>     expand_and_allocate(word_size, context);
>>>     do_collection(false, true, word_size);
>>>
>>> satisfy_failed_allocation_helper(word_size, context, do_gc,
>>> clear_softrefs)
>>>     attempt_allocation_at_safepoint(word_size, context, true);
>>>     if (!do_gc)
>>>        assert(!collector_policy()->should_clear_all_soft_refs());
>>>     expand_and_allocate(word_size, context);
>>>
>>> This would make the code in satisfy_failed_allocation() a lot more
>>> clean.
>>>
>> In webrev.03 I decided to put same way as it was in the original code
>> (before any of my changes),
>> at the very end of method just when we already exhausted all
>> possibilities and about to return NULL.
>>
>> attempt_allocation_at_safepoint(word_size, context, false);
>> expand_and_allocate(word_size, context);
>> do_collection(false, false, word_size);
>> attempt_allocation_at_safepoint(word_size, context, true);
>> do_collection(false, true, word_size);
>> attempt_allocation_at_safepoint(word_size, context, true);
>> assert(!collector_policy()->should_clear_all_soft_refs());
>>
>>   // What else? We might try synchronous finalization later. If the
>> total 1799 // space available is large enough for the allocation, then
>> a more
>>   // complete compaction phase than we've tried so far might be
>> // appropriate.
>> assert(*succeeded, "sanity");
>> return NULL; 1804
>> }
>>
>> webrev.03 has this assert at same position, and I prefer it this way.
>>
>>> * Unless I'm missing some subtle detail here, the usage of the
>>> variable 'succeeded' is unnecessary as 'result' will always contain
>>> the same information. I don't think we should pass on 'succeeded' to
>>> the helper, but simply set it afterwards before we return from
>>> satisfy_failed_allocation().
>> No, helper may return NULL in two differnt cases:
>> 1. Full GC failed, *succeeded = false, no attempts to allocate was
>> made
>> 2. Full GC succeeded, *succeeded = true, all attempts to allocate
>> failed
>>
>> So we need to pass 'succeeded' to the helper.
> Can we rename this to something more descriptive? I had to look into
> do_collection() what it means. Even the gc_succeeded name in the helper
> is better.
>
> Also, the only condition where do_collection() fails is if the gc locker
> is active. This return value is actually ignored in
> do_full_collection().
>
> So the gc locker check could be moved out of do_collection() (adding an
> assert there), and the value of gc_succeeded/succeeded determined using
> this check immediately.
>
> At least, in the helper, move the "assert(*succeeded, ...)" up.
>
> Like:
> --------------
>
> *succeeded = !GCLocker::check_active_before_gc(); // maybe use a temp
> variable here, that is not negated because we check the negated result
> again just later
>
> if (!*succeeded) {
>    return NULL;
> }
>
> do_collection(....);
> assert(
>     !collector_policy()->should_clear_all_soft_refs(), ...); // see above
>
> assert(*succeeded, "..."); // for the paranoid, but I would just skip
> it.
>
> result = attempt_allocation_at_safepoint(...);
>
> if (result != NULL) {
>    return result;
> }
>
> result = expand_and_allocate(...);
>
> return result;
> ---------------
>
> (And the bool return value of do_collection removed).
>
> I also have a question about the order of the order of
> attempt_allocation_at_safepoint() and the expand_and_allocate(), and
> could not find this answered before: so full gc might shrink the heap
> too much after full gc, and we should expand the heap.
> If I were to do this change, there would always (unconditionally) be an
> expansion after full gc, because the current change only expands if the
> allocation after gc fails. Also the 02 change.
>
> The review thread so far suggests that full gc might shrink too much,
> but this still seems to be the case here in some cases.
>
> Thanks,
>    Thomas
>
>



From ysr1729 at gmail.com  Mon Aug 24 22:05:02 2015
From: ysr1729 at gmail.com (Srinivas Ramakrishna)
Date: Mon, 24 Aug 2015 15:05:02 -0700
Subject: JDK-8071931 and JDK-8047125 ?
Message-ID: <CABzyjyk8K991uxH1m2Yp=Zy8fRujOcrnj5QdhZSGU3knQoRqgg@mail.gmail.com>

Hello GC team  --

Is there a reason these aren't visible on OpenJDK's
https://bugs.openjdk.java.net/browse/JDK-$$ ?
I was following up on a JDK 8 related reference processing oddity and ran
into changes related to
these two tickets, but the tickets themselves aren't visible.  Is it
possible to make these visible, or
is that not possible?

Any pointers/comments appreciated!
-- ramki
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150824/30e225bb/attachment.htm>

From thomas.schatzl at oracle.com  Tue Aug 25 09:58:20 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Tue, 25 Aug 2015 11:58:20 +0200
Subject: RFR (S): 8132708: Add tests for Humongous objects allocation
	threshold
In-Reply-To: <55DB36E9.6020806@oracle.com>
References: <55D60438.4080004@oracle.com>
	<1440409997.2348.8.camel@oracle.com> <55DB1AC3.6010004@oracle.com>
	<1440423114.2348.93.camel@oracle.com> <55DB36E9.6020806@oracle.com>
Message-ID: <1440496700.2335.3.camel@oracle.com>

Hi Kirill,

On Mon, 2015-08-24 at 18:23 +0300, Kirill Zhaldybin wrote:
> Hi Thomas,
> 
> New webrev (with fixed typo) is here: 
> http://cr.openjdk.java.net/~kzhaldyb/webrevs/JDK-8132708/webrev.04/
> 
> Thank you.
> 

  looks good.

Thanks,
  Thomas




From per.liden at oracle.com  Tue Aug 25 11:29:42 2015
From: per.liden at oracle.com (Per Liden)
Date: Tue, 25 Aug 2015 13:29:42 +0200
Subject: RFR: 8129417: Oop iteration clean-up to remove
	oop_ms_follow_contents
In-Reply-To: <55D713CD.3080305@oracle.com>
References: <5588217F.9090201@oracle.com>
	<20150706151537.GP3266@ehelin.jrpg.bea.com>
	<559BCD11.3010801@oracle.com> <55D312FF.3070105@oracle.com>
	<55D713CD.3080305@oracle.com>
Message-ID: <55DC51A6.1080503@oracle.com>

Hi Stefan,

On 2015-08-21 14:04, Stefan Johansson wrote:
> Hi Per,
>
> On 2015-08-18 13:11, Per Liden wrote:
>> Hi Stefan,
>>
>> On 2015-07-07 14:58, Stefan Johansson wrote:
>>> Thanks Erik for reviewing,
>>>
>>> New webrevs:
>>> Part 2 - full:
>>> http://cr.openjdk.java.net/~sjohanss/8129417/2-add-oop_iterate_size/hotspot.01/ 
>>>
>>>
>>> Part 2 - inc:
>>> http://cr.openjdk.java.net/~sjohanss/8129417/2-add-oop_iterate_size/hotspot.00-01/ 
>>>
>>>
>>>
>>> Part 3 - full:
>>> http://cr.openjdk.java.net/~sjohanss/8129417/3-re-structure-mark-sweep/hotspot.02/ 
>>>
>>>
>>> Part 3 - inc:
>>> http://cr.openjdk.java.net/~sjohanss/8129417/3-re-structure-mark-sweep/hotspot.00-02/ 
>>>
>>>
>>> Also includes an updated assert, which was missed in previous webrev.
>>>
>>> Comments inline.
>>
>> Just one thing that looks odd to me, but maybe you can clarify. 
>> Shouldn't MarkSweep::follow_object() call follow_array() instead of 
>> calling push_objarray()? That way the change made to 
>> ObjArrayTask::is_valid() wouldn't be needed.
>>
> Thanks for looking at this again. I agree that your proposed change 
> would make the code nicer and I think I had it structured that way 
> once during this work but had to revert it back to this to keep 
> performance.
>
> I went ahead and did the changes and some new measurements, and this 
> change leads to a small regression for some platforms. I haven't done 
> any deep analysis but I guess it leads to worse inlining decisions on 
> these platforms.
>
> I prefer to leave the code as is. The other possible solution would be 
> to do further analysis and give even more special inlining hints to 
> the different compilers.

Ok, I'm fine with leaving it as is if the other approach leads to 
regressions.

/Per

>
> Thanks,
> Stefan
>
>> /Per
>>
>>>
>>> On 2015-07-06 17:15, Erik Helin wrote:
>>>> Hi Stefan,
>>>>
>>>> thanks a lot for taking on this work. Please see comments inline.
>>>>
>>>> On 2015-06-22, Stefan Johansson wrote:
>>>>> Hi,
>>>>>
>>>>> Please review these changes for RFE:
>>>>> https://bugs.openjdk.java.net/browse/JDK-8129417
>>>>>
>>>>> Webrev for the full change:
>>>>> http://cr.openjdk.java.net/~sjohanss/8129417/hotspot.00/
>>>>>
>>>>> Summary:
>>>>> To allow further cleanups and later remove G1 specific code from
>>>>> mark-sweep,
>>>>> we want to remove the mark-sweep specific visitor
>>>>> oop_ms_follow_contents.
>>>>> Doing this cleanup has proven to be a little more complicated than I
>>>>> first
>>>>> anticipated and to make the change easier to review I've split it
>>>>> into four
>>>>> different parts.
>>>>>
>>>>> Part 1 - removing oop_ms_follow_contents:
>>>>> http://cr.openjdk.java.net/~sjohanss/8129417/1-remove-oop_ms_follow_contents/hotspot.00/ 
>>>>>
>>>>>
>>>> - Looking in specialized_oop_closures.hpp, the comment
>>>>     71 // This is split into several because of a Visual C++ 6.0
>>>> compiler bug
>>>>     72 // where very long macros cause the compiler to crash
>>>>    seems a bit dated, given that Visual C++ 6.0 was released in 
>>>> 1998. I
>>>>    think we should try to merge ALL_OOP_OOP_ITERATE_CLOSURES_1 and 
>>>> 2 (and
>>>>    their corresponding macros). Do you think like doing that in this
>>>>    patch or do you want to file a follow-up bug?
>>> I would like to do/investigate this as a separate RFE, I have a list of
>>> things that should follow this clean up and I plan to file those before
>>> pushing this change.
>>>
>>>>
>>>> - The part of the patch that changes instanceMirrorKlass.inline.hpp
>>>>    might impact more GCs than just any of the marksweep ones. Have you
>>>>    seen any performance improvements/regressions with G1 or CMS?
>>> No, but I have not done as many runs and done as much analysis of this
>>> as I've done on the serial full gc. I've not seen any regression in
>>> aurora.se performance runs.
>>>>> Part 2 - introducing oop_iterate_size:
>>>>> http://cr.openjdk.java.net/~sjohanss/8129417/2-add-oop_iterate_size/hotspot.00/ 
>>>>>
>>>>>
>>>> - In arrayKlass.hpp, why add a space to:
>>>>    +#define OOP_OOP_ITERATE_DECL_NO_BACKWARDS(OopClosureType,
>>>> nv_suffix)         \
>>> I've followed '\' alignment "rule" for these macros, which seems to be
>>> two spaces after the longest line. I just realized that I should also
>>> update OOP_OOP_ITERATE_DECL_RANGE with one extra space to be 
>>> consistent.
>>>
>>>> - In arrayKlass.hpp, the following lines seems aligned strangely:
>>>>    +void KlassType::oop_oop_iterate_range##nv_suffix(oop obj,
>>>> OopClosureType* closure, int start, int end) {  \
>>>>    + oop_oop_iterate_range<nvs_to_bool(nv_suffix)>(obj, closure,
>>>> start, end);                               \
>>> Nice catch, missed fixing the alignment when making it void.
>>>
>>>> - In oop.inline.hpp, why must the size be computed before applying the
>>>>    closure? In the world of perm gen, this might have been 
>>>> important in
>>>>    order to not follow stale klass pointers, but with Metaspace the
>>>> Klass*
>>>>    won't move.
>>> As we've discussed offline, this is most likely safe. I've done an
>>> ad-hoc RBT run with some random testing when I assert on the size being
>>> the same both before and after the call to oop_oop_iterate.
>>>
>>> I would like to do this change as a separate change as well, so it's
>>> been added to my list of RFEs to file.
>>>
>>>>
>>>> - Can we remove the method MutableSpace::oop_iterate?
>>> Looks like it, no one seems to be using it. I'll remove it instead of
>>> updating it to use oop_iterate_size().
>>>
>>>>
>>>>> Part 3 - restructure mark-sweep:
>>>>> http://cr.openjdk.java.net/~sjohanss/8129417/3-re-structure-mark-sweep/hotspot.00/ 
>>>>>
>>>>>
>>>> - Can you please remove the commented assert in mark_and_push:
>>>>     +//  assert(Universe::heap()->is_in_reserved(p), "should be in
>>>> object space");
>>>>    or should it be enabled again?
>>> Removed.
>>>
>>>> - I agree with Per about moving the check for object arrays into
>>>>    follow_object.
>>> Already done, but gives a slight regression. I still think we can live
>>> with this small regression for now, to avoid people doing wrong and 
>>> call
>>> follow_object on objArrays.
>>>
>>>>> Part 4 - compiler hints for inlining:
>>>>> http://cr.openjdk.java.net/~sjohanss/8129417/4-compiler-hints-for-inlining/hotspot.00/ 
>>>>>
>>>>>
>>>> - I would like to see the definiton of FORCE_INLINE in the compiler
>>>>    specific globalDefinitions file. Then the code in
>>>>    instanceKlass.inline.hpp can look like:
>>>>     #if defined(TARGET_COMPILER_visCPP) ||
>>>> defined(TARGET_COMPILER_sparcWorks)
>>>>       #define INLINE FORCE_INLINE
>>>>     #else
>>>>       #define INLINE inline
>>>>     #endif
>>>>
>>>> - The same comment for stack.inline.hpp, but with NO_INLINE:
>>>>    #if defined(TARGET_COMPILER_sparcWorks)
>>>>      #define STACK_NO_INLINE NO_INLINE
>>>>    #else
>>>>      #define STACK_NO_INLINE
>>>>    #endif
>>> I agree, we should come up with a better and cleaner way to do this. I
>>> will file a follow up RFE.
>>>
>>>> - I agree with Per about adding an #undef in stack.inline.hpp
>>> Already fixed.
>>>
>>> Thanks again Erik,
>>> Stefan
>>>
>>>> Thanks!
>>>> Erik
>>>>
>>>>> Testing:
>>>>> * Adhoc RBT run for functionality - no new failures.
>>>>> * Adhoc aurora.se run for performance - no obvious regression.
>>>>> * Many local runs tuned to do Full GCs a lot to verify that there 
>>>>> is no
>>>>> obvious regression.
>>>>>
>>>>> Thanks,
>>>>> Stefan
>>>
>



From jesper.wilhelmsson at oracle.com  Tue Aug 25 13:25:09 2015
From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson)
Date: Tue, 25 Aug 2015 15:25:09 +0200
Subject: Request for code review - 8130265: gctests/LargeObjects/large001
	fails with OutOfMemoryError: Java heap space
In-Reply-To: <55DB8B7E.5010601@oracle.com>
References: <55D356FB.1030603@oracle.com> <55D35A12.8020608@oracle.com>
	<55D35F1F.3090700@oracle.com> <55D36DC7.6010901@oracle.com>
	<55D38CBE.8070408@oracle.com> <55D394AD.6020104@oracle.com>
	<55D4E39D.5070805@oracle.com> <55D4E8C7.1080703@oracle.com>
	<55D4EEF5.9030504@oracle.com>
	<2C141666-BDBC-428C-9E40-4FB78F076192@oracle.com>
	<1440082894.2347.14.camel@oracle.com> <55D62D77.8020504@oracle.com>
	<55D71703.4070505@oracle.com> <55D74766.5020202@oracle.com>
	<1440420922.2348.82.camel@oracle.com> <55DB8B7E.5010601@oracle.com>
Message-ID: <55DC6CB5.4010108@oracle.com>

Hi Alex,

I like this approach :)

Just one comment:

After GC in the helper you put the assert that soft references should have been 
handled. The first call to this helper will not clear soft references. The 
original code only has this assert at the end of the function. I think it is 
enough to keep it there, at the end of satisfy_failed_allocation().

1764     assert(!collector_policy()->should_clear_all_soft_refs(),
1765            "Flag should have been handled and cleared prior to this point");

Besides that I'm fine with this version.
/Jesper


Den 24/8/15 kl. 23:24, skrev Alexander Harlap:
> Hi Thomas and Jesper,
>
> Here is new version of change:
>
> http://cr.openjdk.java.net/~aharlap/8130265/webrev.04/
>
> I followed Jesper's advice to have
> satisfy_failed_allocation_helper(..) :
>    attempt_allocation_at_safepoint(...);
>    expand_and_allocate(...);
>    do_collection(...);
>
> So I did not reorder  calls to attempt_allocation_at_safepoint(...) and
> expand_and_allocate(...)
>
> -Alex
>
>
> On 8/24/2015 8:55 AM, Thomas Schatzl wrote:
>> Hi,
>>
>> On Fri, 2015-08-21 at 11:44 -0400, Alexander Harlap wrote:
>>>
>>> On 8/21/2015 8:18 AM, Jesper Wilhelmsson wrote:
>>>
>>>> Hi Alex,
>>>>
>>>> Alexander Harlap skrev den 20/8/15 21:41:
>>>>> Kim and Thomas,
>>>>>
>>>>> Here is new version with your suggestions in:
>>>>>
>>>>> http://cr.openjdk.java.net/~aharlap/8130265/webrev.03/
>>>>> <http://cr.openjdk.java.net/%7Eaharlap/8130265/webrev.03/>
>>>>> Alex
>>>> A few comments on this latest version.
>>>>
>>>> * Why do you assert_at_safepoint and set succeeded to true in your
>>>> helper? The original code did not do this at these places. I think
>>>> you can remove these lines.
>>>>
>>>> 1736   assert_at_safepoint(true /* should_be_vm_thread */);
>>>> 1737
>>>> 1738   *succeeded = true;
>>>>
>>> Even method called satisfy_failed_allocation_helper , there is no way
>>> to make sure that it is called only  from satisfy_failed_allocation.
>> I do not think we guarantee that for any method, i.e. that a given
>> method somehow checks its call chain. This would require much better
>> encapsulation, and in some cases this does not help you either. Also it
>> would require you to state the allowed callers somehow.
>>
>> The helper may be private too, not protected.
>>
>>> satisfy_failed_allocation_helper is a separate method and those lines
>>> are needed.
>>> So I would like to keep
>>> 1736   assert_at_safepoint(true /* should_be_vm_thread */);
>>> 1737
>>> 1738   *succeeded = true;
>>>> * Please align the arguments in
>>>>
>>>> 1732 G1CollectedHeap::satisfy_failed_allocation_helper(size_t
>>>> word_size,
>>>> 1733                                            AllocationContext_t
>>>> context,
>>>> 1734                                            bool
>>>> clear_all_soft_refs,
>>>> 1735                                            bool* succeeded) {
>>>>
>>>> and
>>>>
>>>> 1748   HeapWord* result =
>>>> attempt_allocation_at_safepoint(word_size,
>>>> 1749                                            context,
>>>> 1750                                            true /*
>>>> expect_null_mutator_alloc_region */);
>>>>
>>>> and also in the header file.
>> Please also avoid method definitions like
>>
>> <return value>
>> <class name>::<method-name> ...
>>
>> (or even worse
>>
>> <return value>
>> <class-name>::
>> <method-name>
>>
>> (this change does not do it that way)
>> )
>>> Sure.
>>>
>>>> * Your last version did:
>>>>
>>>> attempt_allocation_at_safepoint(word_size, context, false);
>>>> expand_and_allocate(word_size, context);
>>>> do_collection(false, false, word_size);
>>>> attempt_allocation_at_safepoint(word_size, context, true);
>>>> expand_and_allocate(word_size, context);
>>>> do_collection(false, true, word_size);
>>>> attempt_allocation_at_safepoint(word_size, context, true);
>>>> assert(!collector_policy()->should_clear_all_soft_refs());
>>>> expand_and_allocate(word_size, context);
>>>>
>>>> You have chosen to break this into:
>>>>
>>>> attempt_allocation_at_safepoint(word_size, context, false);
>>>> expand_and_allocate(word_size, context);
>>>>
>>>> satisfy_failed_allocation_helper(word_size, context, false,
>>>> succeeded);
>>>>     do_collection(false, false, word_size);
>>>>     attempt_allocation_at_safepoint(word_size, context, true);
>>>>     expand_and_allocate(word_size, context);
>>>>
>>>> satisfy_failed_allocation_helper(word_size, context, true,
>>>> succeeded);
>>>>     do_collection(false, true, word_size);
>>>>     attempt_allocation_at_safepoint(word_size, context, true);
>>>>     expand_and_allocate(word_size, context);
>>>>
>>>> assert(!collector_policy()->should_clear_all_soft_refs());
>>>>
>>>> This is not entirely identical, depending on how important the
>>>> assert is that could be an issue since the assert is now skipped in
>>>> some cases where it was previously used.
>> The assert just states that after setting clear_all_soft_refs to true in
>> the call to do_collection, it must be false after collection.
>>
>> So actually one could change the assert to
>>
>> assert(!clear_all_soft_refs || !
>> collector_policy()->should_clear_all_soft_refs(), "...") and put it
>> directly in the helper.
>>
>> Actually I think CollectorPolicy::clear_all_soft_refs() must be false
>> after any full gc, so it should be placed after the do_collection call.
>>>> My suggestion would be to break the code up like this:
>>>>
>>>> satisfy_failed_allocation_helper(word_size, context, do_gc,
>>>> clear_softrefs)
>>>>     attempt_allocation_at_safepoint(word_size, context, false);
>>>>     expand_and_allocate(word_size, context);
>>>>     do_collection(false, false, word_size);
>>>>
>>>> satisfy_failed_allocation_helper(word_size, context, do_gc,
>>>> clear_softrefs)
>>>>     attempt_allocation_at_safepoint(word_size, context, true);
>>>>     expand_and_allocate(word_size, context);
>>>>     do_collection(false, true, word_size);
>>>>
>>>> satisfy_failed_allocation_helper(word_size, context, do_gc,
>>>> clear_softrefs)
>>>>     attempt_allocation_at_safepoint(word_size, context, true);
>>>>     if (!do_gc)
>>>>        assert(!collector_policy()->should_clear_all_soft_refs());
>>>>     expand_and_allocate(word_size, context);
>>>>
>>>> This would make the code in satisfy_failed_allocation() a lot more
>>>> clean.
>>>>
>>> In webrev.03 I decided to put same way as it was in the original code
>>> (before any of my changes),
>>> at the very end of method just when we already exhausted all
>>> possibilities and about to return NULL.
>>>
>>> attempt_allocation_at_safepoint(word_size, context, false);
>>> expand_and_allocate(word_size, context);
>>> do_collection(false, false, word_size);
>>> attempt_allocation_at_safepoint(word_size, context, true);
>>> do_collection(false, true, word_size);
>>> attempt_allocation_at_safepoint(word_size, context, true);
>>> assert(!collector_policy()->should_clear_all_soft_refs());
>>>
>>>   // What else? We might try synchronous finalization later. If the
>>> total 1799 // space available is large enough for the allocation, then
>>> a more
>>>   // complete compaction phase than we've tried so far might be
>>> // appropriate.
>>> assert(*succeeded, "sanity");
>>> return NULL; 1804
>>> }
>>>
>>> webrev.03 has this assert at same position, and I prefer it this way.
>>>
>>>> * Unless I'm missing some subtle detail here, the usage of the
>>>> variable 'succeeded' is unnecessary as 'result' will always contain
>>>> the same information. I don't think we should pass on 'succeeded' to
>>>> the helper, but simply set it afterwards before we return from
>>>> satisfy_failed_allocation().
>>> No, helper may return NULL in two differnt cases:
>>> 1. Full GC failed, *succeeded = false, no attempts to allocate was
>>> made
>>> 2. Full GC succeeded, *succeeded = true, all attempts to allocate
>>> failed
>>>
>>> So we need to pass 'succeeded' to the helper.
>> Can we rename this to something more descriptive? I had to look into
>> do_collection() what it means. Even the gc_succeeded name in the helper
>> is better.
>>
>> Also, the only condition where do_collection() fails is if the gc locker
>> is active. This return value is actually ignored in
>> do_full_collection().
>>
>> So the gc locker check could be moved out of do_collection() (adding an
>> assert there), and the value of gc_succeeded/succeeded determined using
>> this check immediately.
>>
>> At least, in the helper, move the "assert(*succeeded, ...)" up.
>>
>> Like:
>> --------------
>>
>> *succeeded = !GCLocker::check_active_before_gc(); // maybe use a temp
>> variable here, that is not negated because we check the negated result
>> again just later
>>
>> if (!*succeeded) {
>>    return NULL;
>> }
>>
>> do_collection(....);
>> assert(
>>     !collector_policy()->should_clear_all_soft_refs(), ...); // see above
>>
>> assert(*succeeded, "..."); // for the paranoid, but I would just skip
>> it.
>>
>> result = attempt_allocation_at_safepoint(...);
>>
>> if (result != NULL) {
>>    return result;
>> }
>>
>> result = expand_and_allocate(...);
>>
>> return result;
>> ---------------
>>
>> (And the bool return value of do_collection removed).
>>
>> I also have a question about the order of the order of
>> attempt_allocation_at_safepoint() and the expand_and_allocate(), and
>> could not find this answered before: so full gc might shrink the heap
>> too much after full gc, and we should expand the heap.
>> If I were to do this change, there would always (unconditionally) be an
>> expansion after full gc, because the current change only expands if the
>> allocation after gc fails. Also the 02 change.
>>
>> The review thread so far suggests that full gc might shrink too much,
>> but this still seems to be the case here in some cases.
>>
>> Thanks,
>>    Thomas
>>
>>
>


From thomas.schatzl at oracle.com  Tue Aug 25 16:06:57 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Tue, 25 Aug 2015 18:06:57 +0200
Subject: RFR (S): 8067341: Modify PLAB sizing algorithm to waste less
Message-ID: <1440518817.2210.21.camel@oracle.com>

Hi all,

  can I have reviews for the following change that modifies the PLAB
sizing algorithm of G1 (only) to be much more stable and robust to
varying sizes of memory allocation?

The old (current for CMS) algorithm tends to create very large sized
PLABs that mean a lot of waste at the end of PLABs and particularly at
the end of regions. The other reason is that we can't figure out how the
current algorithm actually is supposed to work. :)

The end result is that PLAB sizes bounce a lot between some very low
value and maximum value (which is or has been particularly bad for g1
because of region boundaries), causing lots of wasted space. We have
seen that particularly mixed gcs waste up to 50% of memory which caused
useless GCs and degrading throughput significantly.

This changeset replaces the current algorithm with something more robust
(and rather conservative), which decreases the waste in most cases
significantly. Still it typically sizes PLABs larger than default ones
you would get with -XX:-ResizePLAB, improving performance.

The change is commented in the changeset.

In general, with fixed heap size, the change generally gives the same
overall throughput on our throughput benchmarks. Total pause times
decrease quite a bit, but since typically pause time is not an issue
there is no end-to-end difference.

The change has much more impact on applications that have a large
variety of object sizes.

This (and all other previous changes) affect automatic heap sizing
though, so it may result in some at first glance unexpected results
(like decreased heap usage). There may be some re-tuning of defaults in
the future.

CR:
https://bugs.openjdk.java.net/browse/JDK-8067341
Webrev:
http://cr.openjdk.java.net/~tschatzl/8067341/webrev/
Testing:
jprt, perf benchmarks, tested internally in many applications for more
than a year

Thanks,
  Thomas



From jiangli.zhou at oracle.com  Tue Aug 25 17:09:10 2015
From: jiangli.zhou at oracle.com (Jiangli Zhou)
Date: Tue, 25 Aug 2015 10:09:10 -0700
Subject: RFR (S): 8131734: Add free_archive_regions support to G1 for
	-Xshared:auto
In-Reply-To: <55DA32FC.8000405@oracle.com>
References: <55C103A4.1060505@oracle.com> <55C21DC3.5090305@oracle.com>
	<1438785532.2378.73.camel@oracle.com> <55C248FF.6010308@oracle.com>
	<F73C4B9A-4DBA-47A8-8715-0988FF6D393D@oracle.com>
	<55C26249.1090902@oracle.com>
	<C149705F-696B-46CD-8FB8-CAD06C511244@oracle.com>
	<55C2751B.8070400@oracle.com> <1438847329.2009.9.camel@oracle.com>
	<55C36B31.7000908@oracle.com> <1438871723.2474.37.camel@oracle.com>
	<55C373F6.9060609@oracle.com>
	<AD45178E-A442-4A0D-80D6-50E91608E1FE@oracle.com>
	<55C3D8BB.9000603@oracle.com>
	<48C0E571-0D7B-4D30-85DF-E31D99092A7C@oracle.com>
	<30B0248C-3BEA-4FAE-861F-DAAE19F56B45@oracle.com>
	<55C4B806.7050504@oracle.com> <55C4C720.5030903@oracle.com>
	<55CA3430.3070300@oracle.com> <1439377259.2324.27.camel@oracle.com>
	<55CCF0DC.2000800@oracle.com> <1440079572.2347.11.camel@oracle.com>
	<55D5E048.5070404@oracle.com>
	<8566C008-5A60-4376-94C4-968E1A475769@oracle.com>
	<55D742E7.9080208@oracle.com>
	<CAA5A1C9-88F6-4EFE-9C6B-E7CAA366589F@oracle.com>
	<55DA32FC.8000405@! oracle.com>
Message-ID: <0359932D-CDA2-4FC7-9C38-E9C7770655C4@oracle.com>

Could someone also help review the runtime part? I need one more Reviewer.

  http://cr.openjdk.java.net/~jiangli/8131734/webrev.02/ <http://cr.openjdk.java.net/~jiangli/8131734/webrev.02/>

Thanks,
Jiangli


> On Aug 23, 2015, at 1:54 PM, Dmitry Dmitriev <dmitry.dmitriev at oracle.com> wrote:
> 
> Hi Jiangli,
> 
> Looks good to me!
> 
> Thank you,
> Dmitry
> 
> On 22.08.2015 4:47, Jiangli Zhou wrote:
>> Hi Dmitry,
>> 
>> On Aug 21, 2015, at 8:25 AM, Dmitry Dmitriev <dmitry.dmitriev at oracle.com> wrote:
>> 
>>> Hello Jiangli,
>>> 
>>> This looks good to me, but I'm not a reviewer.
>> Thanks.
>> 
>>> Also, I have question to you. Probably you can clarify me one moment.
>>> num_ranges and string_ranges are modified only in code under "#if INCLUDE_ALL_GCS", so it make sense to include all usage of these variables also under "#if INCLUDE_ALL_GCS"? I mean following functions:
>>> FileMapInfo::fixup_string_regions() and new FileMapInfo::dealloc_string_regions() function.
>> Agreed. Here is the updated webrev: http://cr.openjdk.java.net/~jiangli/8131734/webrev.02/src/share/vm/memory/filemap.cpp.sdiff.html.
>> 
>> I also reverified JPRT builds with the new #ifdef changes.
>> 
>> Thanks for the detailed review!
>> 
>> Jiangli
>> 
>>> Thank you,
>>> Dmitry
>>> 
>>> On 20.08.2015 22:55, Jiangli Zhou wrote:
>>>> Hi Dmitry,
>>>> 
>>>> Here is the updated runtime webrev that reflects Tom?s latest GC changes.
>>>> 
>>>> http://cr.openjdk.java.net/~jiangli/8131734/webrev.01/
>>>> 
>>>> I renamed the FileMapInfo::unmap_string_regions() to FileMapInfo::dealloc_string_regions(), which only deallocates the archived string region from the java heap without unmapping. The unmapping is handled by the GC system as the archived string region is part of the java heap. I also added dealloc_string_regions() call to the case where the string region verification fails.
>>>> 
>>>> Thanks,
>>>> Jiangli
>>>> 
>>>> On Aug 20, 2015, at 7:12 AM, Tom Benson <tom.benson at oracle.com> wrote:
>>>> 
>>>>> Hi Thomas,
>>>>> OK, thanks!
>>>>> Tom
>>>>> 
>>>>> On 8/20/2015 10:06 AM, Thomas Schatzl wrote:
>>>>>> Hi Tom,
>>>>>> 
>>>>>>   sorry for the delay...
>>>>>> 
>>>>>> On Thu, 2015-08-13 at 15:32 -0400, Tom Benson wrote:
>>>>>>> Hi Thomas,
>>>>>>> 
>>>>>>> On 8/12/2015 7:00 AM, Thomas Schatzl wrote:
>>>>>>>> Hi,
>>>>>>>> 
>>>>>>>> On Tue, 2015-08-11 at 13:43 -0400, Tom Benson wrote:
>>>>>>>>> Hi,
>>>>>>>>> On 8/7/2015 10:56 AM, Tom Benson wrote:
>>>>>>>>> After some discussion, I've changed the definition and name of
>>>>>>>>> free_archive_regions.  Now called dealloc_archive_regions, it uncommits
>>>>>>>>> the specified regions, unmapping the memory, rather than adding them to
>>>>>>>>> the free list.  This means the CDS code will no longer do the unmapping
>>>>>>>>> on verification failures.
>>>>>>>>> 
>>>>>>>>> Updated full and incremental webrevs of the GC code are at:
>>>>>>>>> http://cr.openjdk.java.net/~tbenson/8131734/webrev.02/
>>>>>>>>> http://cr.openjdk.java.net/~tbenson/8131734/webrev.02.vs.01/
>>>>>>>>> 
>>>>>>>>> Tested with JPRT and running benchmarks with the dealloc_ performed
>>>>>>>>> explicitly.  Jiangli also tested the original failing cases, and will be
>>>>>>>>> posting an updated webrev.
>>>>>>>> - is it possible that shrink_by() uses shrink_at()? This would avoid two
>>>>>>>> paths that uncommit regions like expand_by()/expand_at()?
>>>>>>> OK, I made the change.  I didn't do it originally because the asserts I
>>>>>>> wanted to add for the call from g1CollectedHeap seemed superfluous for
>>>>>>> the other call, and shrink_at was so small.    Now shrink_at takes a
>>>>>>> region count as well.
>>>>>>> 
>>>>>>> Updated full and incremental webrevs are at:
>>>>>>>    http://cr.openjdk.java.net/~tbenson/8131734/webrev.03/
>>>>>>>    http://cr.openjdk.java.net/~tbenson/8131734/webrev.03.vs.02/
>>>>>>> 
>>>>>>> 
>>>>>> Looks good.
>>>>>> 
>>>>>>>> - I think the change should call at least HeapRegion::hr_clear() on the
>>>>>>>> region to remove or reset any auxiliary data structures, if not
>>>>>>>> G1CollectedHeap::free_region() (without adding the region to the free
>>>>>>>> list).
>>>>>>>> Since the HeapRegion* is not deallocated by the uncommit, this may cause
>>>>>>>> strange behavior later when the region is reused.
>>>>>>> I don't think calling hr_clear should be necessary...  If it is, we
>>>>>>> should be doing it in shrink_by as well, and I don't think we are. I
>>>>>>> don't see how a HeapRegion can be 'reused' without having gone through
>>>>>>> the constructor when expand_ asks (indirectly) for 'new HeapRegion', and
>>>>>>> that does an hr_clear() as well as the rest of init.     Or am I missing
>>>>>>> something there?
>>>>>> Leave it as is. I thought that a full gc (which is the only case where
>>>>>> the heap shrinks at the moment) also clears the remset of these regions
>>>>>> at least.
>>>>>> 
>>>>>> It should, I filed  JDK-8134048 for looking in this issue.
>>>>>> 
>>>>>> Looks good.
>>>>>> 
>>>>>> Thanks,
>>>>>>   Thomas
>>>>>> 
>>>>>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150825/c6ae2f57/attachment.htm>

From eric.caspole at oracle.com  Tue Aug 25 17:43:09 2015
From: eric.caspole at oracle.com (Eric Caspole)
Date: Tue, 25 Aug 2015 13:43:09 -0400
Subject: RFR (S): 8067341: Modify PLAB sizing algorithm to waste less
In-Reply-To: <1440518817.2210.21.camel@oracle.com>
References: <1440518817.2210.21.camel@oracle.com>
Message-ID: <55DCA92D.6050108@oracle.com>

Hi Thomas,
I like the new simpler math but I found the comment kind of confusing -

   61     // E.g. assume that if we recently used 100 words and a 
TargetPLABWastePct of 10.
   62     // If we had one thread, we could waste up to 10 words to meet 
that percentage.
   63     // Given that we also assume that that buffer is typically 
half-full, the new
   64     // desired PLAB size is 20 words.


So you mean this GC cycle used 100 words of 1 thread's PLAB when we are 
doing this calculation? Does the previous PLAB size matter or only the 
amount actually used?

   66     // We account region end waste fully to PLAB allocation (in 
the calculation of
   67     // what we consider as "used" below). This is not completely 
fair, but is a
   68     // conservative assumption because PLABs may be sized flexibly 
while we cannot
   69     // adjust inline allocations.


I know this comment was there before but how do the direct allocations 
affect the PLAB size? Because an object that does not fit immediately 
causes the the current PLAB to be discarded? So that increases the waste 
but it shows up in used()?

Thanks,
Eric



On 08/25/2015 12:06 PM, Thomas Schatzl wrote:
> Hi all,
>
>    can I have reviews for the following change that modifies the PLAB
> sizing algorithm of G1 (only) to be much more stable and robust to
> varying sizes of memory allocation?
>
> The old (current for CMS) algorithm tends to create very large sized
> PLABs that mean a lot of waste at the end of PLABs and particularly at
> the end of regions. The other reason is that we can't figure out how the
> current algorithm actually is supposed to work. :)
>
> The end result is that PLAB sizes bounce a lot between some very low
> value and maximum value (which is or has been particularly bad for g1
> because of region boundaries), causing lots of wasted space. We have
> seen that particularly mixed gcs waste up to 50% of memory which caused
> useless GCs and degrading throughput significantly.
>
> This changeset replaces the current algorithm with something more robust
> (and rather conservative), which decreases the waste in most cases
> significantly. Still it typically sizes PLABs larger than default ones
> you would get with -XX:-ResizePLAB, improving performance.
>
> The change is commented in the changeset.
>
> In general, with fixed heap size, the change generally gives the same
> overall throughput on our throughput benchmarks. Total pause times
> decrease quite a bit, but since typically pause time is not an issue
> there is no end-to-end difference.
>
> The change has much more impact on applications that have a large
> variety of object sizes.
>
> This (and all other previous changes) affect automatic heap sizing
> though, so it may result in some at first glance unexpected results
> (like decreased heap usage). There may be some re-tuning of defaults in
> the future.
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8067341
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8067341/webrev/
> Testing:
> jprt, perf benchmarks, tested internally in many applications for more
> than a year
>
> Thanks,
>    Thomas
>


From thomas.schatzl at oracle.com  Tue Aug 25 18:24:38 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Tue, 25 Aug 2015 20:24:38 +0200
Subject: RFR (S): 8067341: Modify PLAB sizing algorithm to waste less
In-Reply-To: <55DCA92D.6050108@oracle.com>
References: <1440518817.2210.21.camel@oracle.com> <55DCA92D.6050108@oracle.com>
Message-ID: <1440527078.1985.33.camel@oracle.com>

Hi,

On Tue, 2015-08-25 at 13:43 -0400, Eric Caspole wrote:
> Hi Thomas,
> I like the new simpler math but I found the comment kind of confusing -
> 
>    61     // E.g. assume that if we recently used 100 words and a 
> TargetPLABWastePct of 10.
>    62     // If we had one thread, we could waste up to 10 words to meet 
> that percentage.
>    63     // Given that we also assume that that buffer is typically 
> half-full, the new
>    64     // desired PLAB size is 20 words.
> 
> 
> So you mean this GC cycle used 100 words of 1 thread's PLAB when we are 
> doing this calculation? 

This is total amount of allocation. The amount of allocation and (total)
waste G1 can spend to meet the threshold, is ideally the same whether
you use one or hundred threads.

So, if there were one thread to copy the same amount of objects, it
could waste the mentioned 10 words, so we set PLAB size to that size
because that's the maximum that can ever be wasted (barring only having
objects larger than 10; and actually in that case it would not waste
anything because 10 divides 100 evenly, but think 101 :).

[And expanding that, if you have n threads, each of them can waste <max
plab size for one thread> / n in a rough approximation. The number of
threads only comes into play later when actually trying to retrieve the
desired plab size]

> Does the previous PLAB size matter or only the 
> amount actually used?

This only calculates the "optimal" PLAB size for the current/last GC.
That value is fed into the usual exponential decaying average to remove
allocation/waste spikes to guess the next PLAB size.

>    66     // We account region end waste fully to PLAB allocation (in 
> the calculation of
>    67     // what we consider as "used" below). This is not completely 
> fair, but is a
>    68     // conservative assumption because PLABs may be sized flexibly 
> while we cannot
>    69     // adjust inline allocations.
> 
> 
> I know this comment was there before but how do the direct allocations 
> affect the PLAB size? Because an object that does not fit immediately 
> causes the the current PLAB to be discarded? So that increases the waste 
> but it shows up in used()?

Direct allocations affect PLAB size in how much waste is generated. This
potentially decreases PLAB size (hence "conservative").

The main problem here is region end waste. At the moment we still throw
away regions that cannot satisfy the current allocation, potentially
generating lots of that kind of waste.

G1 almost generates no region end waste when allocating new PLABs due to
JDK-8067336 at this time (consider that typically the minimum size
requested is << than what is available). Direct allocation is not
"flexible", so it still can cause lots of region end waste.

There is one more larger functionality change (JDK-8067433) that tries
to minimize that region end waste (quite successfully in my
measurements) by keeping around regions that still have space left for
allocating PLABs, and retrying.

I still need to do some refactoring and splitting of that change into
multiple changes for review as it is a bit large, so I decided to ask
for reviews for this one earlier than planned.

Everything else is just some logging, cleaning up or tweaking.

There are certainly better methods, but in real world (tm) it already
works much better than previously.

There is always the option to disable automatic PLAB resizing (and still
can try to set the "optimal" value for a particular application given
the PLAB statistics which are now always calculated).

Thanks,
  Thomas

[...]
> >
> > CR:
> > https://bugs.openjdk.java.net/browse/JDK-8067341
> > Webrev:
> > http://cr.openjdk.java.net/~tschatzl/8067341/webrev/
> > Testing:
> > jprt, perf benchmarks, tested internally in many applications for more
> > than a year
> >
> > Thanks,
> >    Thomas
> >




From alexander.harlap at oracle.com  Tue Aug 25 18:36:19 2015
From: alexander.harlap at oracle.com (Alexander Harlap)
Date: Tue, 25 Aug 2015 14:36:19 -0400
Subject: Request for code review - 8130265: gctests/LargeObjects/large001
	fails with OutOfMemoryError: Java heap space
In-Reply-To: <55DC6CB5.4010108@oracle.com>
References: <55D356FB.1030603@oracle.com> <55D35A12.8020608@oracle.com>
	<55D35F1F.3090700@oracle.com> <55D36DC7.6010901@oracle.com>
	<55D38CBE.8070408@oracle.com> <55D394AD.6020104@oracle.com>
	<55D4E39D.5070805@oracle.com> <55D4E8C7.1080703@oracle.com>
	<55D4EEF5.9030504@oracle.com>
	<2C141666-BDBC-428C-9E40-4FB78F076192@oracle.com>
	<1440082894.2347.14.camel@oracle.com> <55D62D77.8020504@oracle.com>
	<55D71703.4070505@oracle.com> <55D74766.5020202@oracle.com>
	<1440420922.2348.82.camel@oracle.com> <55DB8B7E.5010601@oracle.com>
	<55DC6CB5.4010108@oracle.com>
Message-ID: <55DCB5A2.3060805@oracle.com>

I placed assert( 
assert(!collector_policy()->should_clear_all_soft_refs(), "Flag should 
have been handled and cleared prior to this point");
according to Jesper's suggestion.

This assert was introduced by Jon in change 6b73e879f1c2.

New version is here:

http://cr.openjdk.java.net/~aharlap/8130265/webrev.05/

Alex



On 8/25/2015 9:25 AM, Jesper Wilhelmsson wrote:
> Hi Alex,
>
> I like this approach :)
>
> Just one comment:
>
> After GC in the helper you put the assert that soft references should 
> have been handled. The first call to this helper will not clear soft 
> references. The original code only has this assert at the end of the 
> function. I think it is enough to keep it there, at the end of 
> satisfy_failed_allocation().
>
> 1764 assert(!collector_policy()->should_clear_all_soft_refs(),
> 1765            "Flag should have been handled and cleared prior to 
> this point");
>
> Besides that I'm fine with this version.
> /Jesper
>
>
> Den 24/8/15 kl. 23:24, skrev Alexander Harlap:
>> Hi Thomas and Jesper,
>>
>> Here is new version of change:
>>
>> http://cr.openjdk.java.net/~aharlap/8130265/webrev.04/
>>
>> I followed Jesper's advice to have
>> satisfy_failed_allocation_helper(..) :
>>    attempt_allocation_at_safepoint(...);
>>    expand_and_allocate(...);
>>    do_collection(...);
>>
>> So I did not reorder  calls to attempt_allocation_at_safepoint(...) and
>> expand_and_allocate(...)
>>
>> -Alex
>>
>>
>> On 8/24/2015 8:55 AM, Thomas Schatzl wrote:
>>> Hi,
>>>
>>> On Fri, 2015-08-21 at 11:44 -0400, Alexander Harlap wrote:
>>>>
>>>> On 8/21/2015 8:18 AM, Jesper Wilhelmsson wrote:
>>>>
>>>>> Hi Alex,
>>>>>
>>>>> Alexander Harlap skrev den 20/8/15 21:41:
>>>>>> Kim and Thomas,
>>>>>>
>>>>>> Here is new version with your suggestions in:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~aharlap/8130265/webrev.03/
>>>>>> <http://cr.openjdk.java.net/%7Eaharlap/8130265/webrev.03/>
>>>>>> Alex
>>>>> A few comments on this latest version.
>>>>>
>>>>> * Why do you assert_at_safepoint and set succeeded to true in your
>>>>> helper? The original code did not do this at these places. I think
>>>>> you can remove these lines.
>>>>>
>>>>> 1736   assert_at_safepoint(true /* should_be_vm_thread */);
>>>>> 1737
>>>>> 1738   *succeeded = true;
>>>>>
>>>> Even method called satisfy_failed_allocation_helper , there is no way
>>>> to make sure that it is called only  from satisfy_failed_allocation.
>>> I do not think we guarantee that for any method, i.e. that a given
>>> method somehow checks its call chain. This would require much better
>>> encapsulation, and in some cases this does not help you either. Also it
>>> would require you to state the allowed callers somehow.
>>>
>>> The helper may be private too, not protected.
>>>
>>>> satisfy_failed_allocation_helper is a separate method and those lines
>>>> are needed.
>>>> So I would like to keep
>>>> 1736   assert_at_safepoint(true /* should_be_vm_thread */);
>>>> 1737
>>>> 1738   *succeeded = true;
>>>>> * Please align the arguments in
>>>>>
>>>>> 1732 G1CollectedHeap::satisfy_failed_allocation_helper(size_t
>>>>> word_size,
>>>>> 1733 AllocationContext_t
>>>>> context,
>>>>> 1734                                            bool
>>>>> clear_all_soft_refs,
>>>>> 1735                                            bool* succeeded) {
>>>>>
>>>>> and
>>>>>
>>>>> 1748   HeapWord* result =
>>>>> attempt_allocation_at_safepoint(word_size,
>>>>> 1749                                            context,
>>>>> 1750                                            true /*
>>>>> expect_null_mutator_alloc_region */);
>>>>>
>>>>> and also in the header file.
>>> Please also avoid method definitions like
>>>
>>> <return value>
>>> <class name>::<method-name> ...
>>>
>>> (or even worse
>>>
>>> <return value>
>>> <class-name>::
>>> <method-name>
>>>
>>> (this change does not do it that way)
>>> )
>>>> Sure.
>>>>
>>>>> * Your last version did:
>>>>>
>>>>> attempt_allocation_at_safepoint(word_size, context, false);
>>>>> expand_and_allocate(word_size, context);
>>>>> do_collection(false, false, word_size);
>>>>> attempt_allocation_at_safepoint(word_size, context, true);
>>>>> expand_and_allocate(word_size, context);
>>>>> do_collection(false, true, word_size);
>>>>> attempt_allocation_at_safepoint(word_size, context, true);
>>>>> assert(!collector_policy()->should_clear_all_soft_refs());
>>>>> expand_and_allocate(word_size, context);
>>>>>
>>>>> You have chosen to break this into:
>>>>>
>>>>> attempt_allocation_at_safepoint(word_size, context, false);
>>>>> expand_and_allocate(word_size, context);
>>>>>
>>>>> satisfy_failed_allocation_helper(word_size, context, false,
>>>>> succeeded);
>>>>>     do_collection(false, false, word_size);
>>>>>     attempt_allocation_at_safepoint(word_size, context, true);
>>>>>     expand_and_allocate(word_size, context);
>>>>>
>>>>> satisfy_failed_allocation_helper(word_size, context, true,
>>>>> succeeded);
>>>>>     do_collection(false, true, word_size);
>>>>>     attempt_allocation_at_safepoint(word_size, context, true);
>>>>>     expand_and_allocate(word_size, context);
>>>>>
>>>>> assert(!collector_policy()->should_clear_all_soft_refs());
>>>>>
>>>>> This is not entirely identical, depending on how important the
>>>>> assert is that could be an issue since the assert is now skipped in
>>>>> some cases where it was previously used.
>>> The assert just states that after setting clear_all_soft_refs to 
>>> true in
>>> the call to do_collection, it must be false after collection.
>>>
>>> So actually one could change the assert to
>>>
>>> assert(!clear_all_soft_refs || !
>>> collector_policy()->should_clear_all_soft_refs(), "...") and put it
>>> directly in the helper.
>>>
>>> Actually I think CollectorPolicy::clear_all_soft_refs() must be false
>>> after any full gc, so it should be placed after the do_collection call.
>>>>> My suggestion would be to break the code up like this:
>>>>>
>>>>> satisfy_failed_allocation_helper(word_size, context, do_gc,
>>>>> clear_softrefs)
>>>>>     attempt_allocation_at_safepoint(word_size, context, false);
>>>>>     expand_and_allocate(word_size, context);
>>>>>     do_collection(false, false, word_size);
>>>>>
>>>>> satisfy_failed_allocation_helper(word_size, context, do_gc,
>>>>> clear_softrefs)
>>>>>     attempt_allocation_at_safepoint(word_size, context, true);
>>>>>     expand_and_allocate(word_size, context);
>>>>>     do_collection(false, true, word_size);
>>>>>
>>>>> satisfy_failed_allocation_helper(word_size, context, do_gc,
>>>>> clear_softrefs)
>>>>>     attempt_allocation_at_safepoint(word_size, context, true);
>>>>>     if (!do_gc)
>>>>> assert(!collector_policy()->should_clear_all_soft_refs());
>>>>>     expand_and_allocate(word_size, context);
>>>>>
>>>>> This would make the code in satisfy_failed_allocation() a lot more
>>>>> clean.
>>>>>
>>>> In webrev.03 I decided to put same way as it was in the original code
>>>> (before any of my changes),
>>>> at the very end of method just when we already exhausted all
>>>> possibilities and about to return NULL.
>>>>
>>>> attempt_allocation_at_safepoint(word_size, context, false);
>>>> expand_and_allocate(word_size, context);
>>>> do_collection(false, false, word_size);
>>>> attempt_allocation_at_safepoint(word_size, context, true);
>>>> do_collection(false, true, word_size);
>>>> attempt_allocation_at_safepoint(word_size, context, true);
>>>> assert(!collector_policy()->should_clear_all_soft_refs());
>>>>
>>>>   // What else? We might try synchronous finalization later. If the
>>>> total 1799 // space available is large enough for the allocation, then
>>>> a more
>>>>   // complete compaction phase than we've tried so far might be
>>>> // appropriate.
>>>> assert(*succeeded, "sanity");
>>>> return NULL; 1804
>>>> }
>>>>
>>>> webrev.03 has this assert at same position, and I prefer it this way.
>>>>
>>>>> * Unless I'm missing some subtle detail here, the usage of the
>>>>> variable 'succeeded' is unnecessary as 'result' will always contain
>>>>> the same information. I don't think we should pass on 'succeeded' to
>>>>> the helper, but simply set it afterwards before we return from
>>>>> satisfy_failed_allocation().
>>>> No, helper may return NULL in two differnt cases:
>>>> 1. Full GC failed, *succeeded = false, no attempts to allocate was
>>>> made
>>>> 2. Full GC succeeded, *succeeded = true, all attempts to allocate
>>>> failed
>>>>
>>>> So we need to pass 'succeeded' to the helper.
>>> Can we rename this to something more descriptive? I had to look into
>>> do_collection() what it means. Even the gc_succeeded name in the helper
>>> is better.
>>>
>>> Also, the only condition where do_collection() fails is if the gc 
>>> locker
>>> is active. This return value is actually ignored in
>>> do_full_collection().
>>>
>>> So the gc locker check could be moved out of do_collection() (adding an
>>> assert there), and the value of gc_succeeded/succeeded determined using
>>> this check immediately.
>>>
>>> At least, in the helper, move the "assert(*succeeded, ...)" up.
>>>
>>> Like:
>>> --------------
>>>
>>> *succeeded = !GCLocker::check_active_before_gc(); // maybe use a temp
>>> variable here, that is not negated because we check the negated result
>>> again just later
>>>
>>> if (!*succeeded) {
>>>    return NULL;
>>> }
>>>
>>> do_collection(....);
>>> assert(
>>>     !collector_policy()->should_clear_all_soft_refs(), ...); // see 
>>> above
>>>
>>> assert(*succeeded, "..."); // for the paranoid, but I would just skip
>>> it.
>>>
>>> result = attempt_allocation_at_safepoint(...);
>>>
>>> if (result != NULL) {
>>>    return result;
>>> }
>>>
>>> result = expand_and_allocate(...);
>>>
>>> return result;
>>> ---------------
>>>
>>> (And the bool return value of do_collection removed).
>>>
>>> I also have a question about the order of the order of
>>> attempt_allocation_at_safepoint() and the expand_and_allocate(), and
>>> could not find this answered before: so full gc might shrink the heap
>>> too much after full gc, and we should expand the heap.
>>> If I were to do this change, there would always (unconditionally) be an
>>> expansion after full gc, because the current change only expands if the
>>> allocation after gc fails. Also the 02 change.
>>>
>>> The review thread so far suggests that full gc might shrink too much,
>>> but this still seems to be the case here in some cases.
>>>
>>> Thanks,
>>>    Thomas
>>>
>>>
>>



From eric.caspole at oracle.com  Tue Aug 25 18:56:03 2015
From: eric.caspole at oracle.com (Eric Caspole)
Date: Tue, 25 Aug 2015 14:56:03 -0400
Subject: RFR (S): 8067341: Modify PLAB sizing algorithm to waste less
In-Reply-To: <1440527078.1985.33.camel@oracle.com>
References: <1440518817.2210.21.camel@oracle.com>
	<55DCA92D.6050108@oracle.com> <1440527078.1985.33.camel@oracle.com>
Message-ID: <55DCBA43.5080704@oracle.com>

Could you massage this great explanation into those comments? Especially 
the part about 1 vs N threads makes it more clear to me.
Thanks,
Eric


On 8/25/2015 2:24 PM, Thomas Schatzl wrote:
> Hi,
>
> On Tue, 2015-08-25 at 13:43 -0400, Eric Caspole wrote:
>> Hi Thomas,
>> I like the new simpler math but I found the comment kind of confusing -
>>
>>     61     // E.g. assume that if we recently used 100 words and a
>> TargetPLABWastePct of 10.
>>     62     // If we had one thread, we could waste up to 10 words to meet
>> that percentage.
>>     63     // Given that we also assume that that buffer is typically
>> half-full, the new
>>     64     // desired PLAB size is 20 words.
>>
>>
>> So you mean this GC cycle used 100 words of 1 thread's PLAB when we are
>> doing this calculation?
> This is total amount of allocation. The amount of allocation and (total)
> waste G1 can spend to meet the threshold, is ideally the same whether
> you use one or hundred threads.
>
> So, if there were one thread to copy the same amount of objects, it
> could waste the mentioned 10 words, so we set PLAB size to that size
> because that's the maximum that can ever be wasted (barring only having
> objects larger than 10; and actually in that case it would not waste
> anything because 10 divides 100 evenly, but think 101 :).
>
> [And expanding that, if you have n threads, each of them can waste <max
> plab size for one thread> / n in a rough approximation. The number of
> threads only comes into play later when actually trying to retrieve the
> desired plab size]
>
>> Does the previous PLAB size matter or only the
>> amount actually used?
> This only calculates the "optimal" PLAB size for the current/last GC.
> That value is fed into the usual exponential decaying average to remove
> allocation/waste spikes to guess the next PLAB size.
>
>>     66     // We account region end waste fully to PLAB allocation (in
>> the calculation of
>>     67     // what we consider as "used" below). This is not completely
>> fair, but is a
>>     68     // conservative assumption because PLABs may be sized flexibly
>> while we cannot
>>     69     // adjust inline allocations.
>>
>>
>> I know this comment was there before but how do the direct allocations
>> affect the PLAB size? Because an object that does not fit immediately
>> causes the the current PLAB to be discarded? So that increases the waste
>> but it shows up in used()?
> Direct allocations affect PLAB size in how much waste is generated. This
> potentially decreases PLAB size (hence "conservative").
>
> The main problem here is region end waste. At the moment we still throw
> away regions that cannot satisfy the current allocation, potentially
> generating lots of that kind of waste.
>
> G1 almost generates no region end waste when allocating new PLABs due to
> JDK-8067336 at this time (consider that typically the minimum size
> requested is << than what is available). Direct allocation is not
> "flexible", so it still can cause lots of region end waste.
>
> There is one more larger functionality change (JDK-8067433) that tries
> to minimize that region end waste (quite successfully in my
> measurements) by keeping around regions that still have space left for
> allocating PLABs, and retrying.
>
> I still need to do some refactoring and splitting of that change into
> multiple changes for review as it is a bit large, so I decided to ask
> for reviews for this one earlier than planned.
>
> Everything else is just some logging, cleaning up or tweaking.
>
> There are certainly better methods, but in real world (tm) it already
> works much better than previously.
>
> There is always the option to disable automatic PLAB resizing (and still
> can try to set the "optimal" value for a particular application given
> the PLAB statistics which are now always calculated).
>
> Thanks,
>    Thomas
>
> [...]
>>> CR:
>>> https://bugs.openjdk.java.net/browse/JDK-8067341
>>> Webrev:
>>> http://cr.openjdk.java.net/~tschatzl/8067341/webrev/
>>> Testing:
>>> jprt, perf benchmarks, tested internally in many applications for more
>>> than a year
>>>
>>> Thanks,
>>>     Thomas
>>>
>



From ysr1729 at gmail.com  Tue Aug 25 19:56:15 2015
From: ysr1729 at gmail.com (Srinivas Ramakrishna)
Date: Tue, 25 Aug 2015 12:56:15 -0700
Subject: race condition in GC logs
In-Reply-To: <CA61B527-3D6E-46EF-AAAA-425C306ED14B@kodewerk.com>
References: <9E79C717-7C53-49F5-AD0F-C003A03F11E5@kodewerk.com>
	<55D30D50.8060302@oracle.com> <55D32774.2090903@oracle.com>
	<CA61B527-3D6E-46EF-AAAA-425C306ED14B@kodewerk.com>
Message-ID: <CABzyjymhn1DXaCfZ35zGC2tVah2SiMk1W-mbgvhYVjBY38se8w@mail.gmail.com>

-bcc/hotspot-dev
+cc/hotspot-gc-dev

I have a patch for JDK-8133818 (on JDK 8): as suggested here earlier,
besides splitting out Cleaners, it also reports JNI wk refs by counting
them on-the-fly much like for other Reference types.

I'll attach it to the ticket once my testing completes.

thanks.
-- ramki

On Wed, Aug 19, 2015 at 3:20 PM, Kirk Pepperdine <kirk at kodewerk.com> wrote:

> Hi Bengt,
>
> Thanks for filing. I really didn?t consider this a bug but it seems
> reasonable to add clarity in the log.
>
> ? Kirk
>
> > On Aug 18, 2015, at 8:39 AM, Bengt Rutisson <bengt.rutisson at oracle.com>
> wrote:
> >
> >
> >
> > On 2015-08-18 12:47, Bengt Rutisson wrote:
> >>
> >> Hi Kirk,
> >>
> >> On 2015-08-18 12:31, Kirk Pepperdine wrote:
> >>> Hi all,
> >>>
> >>> I found this.. in a few logs. Before I started trolling the source
> trying to sort out what the second PhantomReferences count was, I pinged
> Charlie and asked him. He suggested it was a (concurrency??) bug witnessed
> only once by someone else. I?ve now seen it in several logs where the
> numbers reported are different. Sorry for being lazy for not trolling the
> code before asking but is this a bug?
> >>
> >> The second ref count is for cleaner references. They used to be treated
> the same way as phantom references, but a recent improvement was to
> separate the handling of the cleaners to after all phantom references have
> been handled. So, the first count is the "real" phantom references and the
> second count is the cleaner references.
> >>
> >> It's confusing logging, I agree.
> >>
> >> This came in with this fix:
> >> http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/2194fdf3fbd9
> >>
> >> since this adds one more call to preclean_discovered_reflist(), which
> is the method that prints the refs count.
> >
> > Thomas was nice and created a bug report for this :)
> >
> > Additional number of processed references printed with
> -XX:+PrintReferenceGC after JDK-8047125
> > https://bugs.openjdk.java.net/browse/JDK-8133818
> >
> > Bengt
> >
> >>
> >> Thanks,
> >> Bengt
> >>
> >>>
> >>> Regards,
> >>> Kirk
> >>>
> >>> Java HotSpot(TM) 64-Bit Server VM (25.51-b03) for linux-amd64 JRE
> (1.8.0_51-b16), built on Jun  8 2015 19:28:07 by "java_re" with gcc 4.3.0
> 20080428 (Red Hat 4.3.0-8)
> >>> Memory: 4k page, physical 9570132k(9340292k free), swap
> 4095996k(4095996k free)
> >>> CommandLine flags: -XX:+AggressiveOpts -XX:G1HeapRegionSize=16777216
> -XX:InitialHeapSize=3221225472 -XX:InitiatingHeapOccupancyPercent=75
> -XX:+ManagementServer -XX:MaxGCPauseMillis=250 -XX:MaxHeapSize=3221225472
> -XX:+ParallelRefProcEnabled -XX:+PerfDisableSharedMem
> -XX:+PrintAdaptiveSizePolicy -XX:+PrintGC
> -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCDateStamps
> -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintReferenceGC
> -XX:+PrintTenuringDistribution -XX:+UseCompressedClassPointers
> -XX:+UseCompressedOops -XX:+UseG1GC -XX:+UseLargePages
> >>>  0.198: [G1Ergonomics (Heap Sizing) expand the heap, requested
> expansion amount: 3221225472 bytes, attempted expansion amount: 3221225472
> bytes]
> >>> 2015-08-11T08:23:02.219+0200: 7.630: [GC pause (G1 Evacuation Pause)
> (young)
> >>> Desired survivor size 16777216 bytes, new threshold 15 (max 15)
> >>>  7.631: [G1Ergonomics (CSet Construction) start choosing CSet,
> _pending_cards: 0, predicted base time: 10.00 ms, remaining time: 240.00
> ms, target pause time: 250.00 ms]
> >>>  7.631: [G1Ergonomics (CSet Construction) add young regions to CSet,
> eden: 9 regions, survivors: 0 regions, predicted young region time: 3626.58
> ms]
> >>>  7.631: [G1Ergonomics (CSet Construction) finish choosing CSet, eden:
> 9 regions, survivors: 0 regions, old: 0 regions, predicted pause time:
> 3636.58 ms, target pause time: 250.00 ms]
> >>> 7.642: [SoftReference, 0 refs, 0.0006965 secs]7.643: [WeakReference,
> 40 refs, 0.0003039 secs]7.643: [FinalReference, 3477 refs, 0.0036191
> secs]7.647: [PhantomReference, 0 refs, 0 refs, 0.0006649 secs]7.647: [JNI
> Weak Reference, 0.0000119 secs], 0.0173298 secs]
> >>
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150825/2d3181b9/attachment.htm>

From jesper.wilhelmsson at oracle.com  Tue Aug 25 20:15:46 2015
From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson)
Date: Tue, 25 Aug 2015 22:15:46 +0200
Subject: Request for code review - 8130265: gctests/LargeObjects/large001
	fails with OutOfMemoryError: Java heap space
In-Reply-To: <55DCB5A2.3060805@oracle.com>
References: <55D356FB.1030603@oracle.com> <55D35A12.8020608@oracle.com>
	<55D35F1F.3090700@oracle.com> <55D36DC7.6010901@oracle.com>
	<55D38CBE.8070408@oracle.com> <55D394AD.6020104@oracle.com>
	<55D4E39D.5070805@oracle.com> <55D4E8C7.1080703@oracle.com>
	<55D4EEF5.9030504@oracle.com>
	<2C141666-BDBC-428C-9E40-4FB78F076192@oracle.com>
	<1440082894.2347.14.camel@oracle.com> <55D62D77.8020504@oracle.com>
	<55D71703.4070505@oracle.com> <55D74766.5020202@oracle.com>
	<1440420922.2348.82.camel@oracle.com> <55DB8B7E.5010601@oracle.com>
	<55DC6CB5.4010108@oracle.com> <55DCB5A2.3060805@oracle.com>
Message-ID: <55DCCCF2.6010808@oracle.com>

Den 25/8/15 kl. 20:36, skrev Alexander Harlap:
> I placed assert( assert(!collector_policy()->should_clear_all_soft_refs(), "Flag
> should have been handled and cleared prior to this point");
> according to Jesper's suggestion.

Ah, I see.  I had a "if (!do_gc)" before the assert in my example so that it 
would only be executed in the last of the three calls.

The latest change is good as is. If you want to move the assert into the helper 
again it could be like:

    if (do_gc) {
      // Expansion didn't work, we'll try to do a Full GC.
      *gc_succeeded = do_collection(false, /* explicit_gc */
                                    clear_all_soft_refs,
                                    word_size);
    } else {
      assert(!collector_policy()->should_clear_all_soft_refs(),
             "Flag should have been handled and cleared prior to this point");
    }

But looking at this code snippet, I think I like your latest version better.

Thanks,
/Jesper

>
> This assert was introduced by Jon in change 6b73e879f1c2.
>
> New version is here:
>
> http://cr.openjdk.java.net/~aharlap/8130265/webrev.05/
>
> Alex
>
>
>
> On 8/25/2015 9:25 AM, Jesper Wilhelmsson wrote:
>> Hi Alex,
>>
>> I like this approach :)
>>
>> Just one comment:
>>
>> After GC in the helper you put the assert that soft references should have
>> been handled. The first call to this helper will not clear soft references.
>> The original code only has this assert at the end of the function. I think it
>> is enough to keep it there, at the end of satisfy_failed_allocation().
>>
>> 1764 assert(!collector_policy()->should_clear_all_soft_refs(),
>> 1765            "Flag should have been handled and cleared prior to this point");
>>
>> Besides that I'm fine with this version.
>> /Jesper
>>
>>
>> Den 24/8/15 kl. 23:24, skrev Alexander Harlap:
>>> Hi Thomas and Jesper,
>>>
>>> Here is new version of change:
>>>
>>> http://cr.openjdk.java.net/~aharlap/8130265/webrev.04/
>>>
>>> I followed Jesper's advice to have
>>> satisfy_failed_allocation_helper(..) :
>>>    attempt_allocation_at_safepoint(...);
>>>    expand_and_allocate(...);
>>>    do_collection(...);
>>>
>>> So I did not reorder  calls to attempt_allocation_at_safepoint(...) and
>>> expand_and_allocate(...)
>>>
>>> -Alex
>>>
>>>
>>> On 8/24/2015 8:55 AM, Thomas Schatzl wrote:
>>>> Hi,
>>>>
>>>> On Fri, 2015-08-21 at 11:44 -0400, Alexander Harlap wrote:
>>>>>
>>>>> On 8/21/2015 8:18 AM, Jesper Wilhelmsson wrote:
>>>>>
>>>>>> Hi Alex,
>>>>>>
>>>>>> Alexander Harlap skrev den 20/8/15 21:41:
>>>>>>> Kim and Thomas,
>>>>>>>
>>>>>>> Here is new version with your suggestions in:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~aharlap/8130265/webrev.03/
>>>>>>> <http://cr.openjdk.java.net/%7Eaharlap/8130265/webrev.03/>
>>>>>>> Alex
>>>>>> A few comments on this latest version.
>>>>>>
>>>>>> * Why do you assert_at_safepoint and set succeeded to true in your
>>>>>> helper? The original code did not do this at these places. I think
>>>>>> you can remove these lines.
>>>>>>
>>>>>> 1736   assert_at_safepoint(true /* should_be_vm_thread */);
>>>>>> 1737
>>>>>> 1738   *succeeded = true;
>>>>>>
>>>>> Even method called satisfy_failed_allocation_helper , there is no way
>>>>> to make sure that it is called only  from satisfy_failed_allocation.
>>>> I do not think we guarantee that for any method, i.e. that a given
>>>> method somehow checks its call chain. This would require much better
>>>> encapsulation, and in some cases this does not help you either. Also it
>>>> would require you to state the allowed callers somehow.
>>>>
>>>> The helper may be private too, not protected.
>>>>
>>>>> satisfy_failed_allocation_helper is a separate method and those lines
>>>>> are needed.
>>>>> So I would like to keep
>>>>> 1736   assert_at_safepoint(true /* should_be_vm_thread */);
>>>>> 1737
>>>>> 1738   *succeeded = true;
>>>>>> * Please align the arguments in
>>>>>>
>>>>>> 1732 G1CollectedHeap::satisfy_failed_allocation_helper(size_t
>>>>>> word_size,
>>>>>> 1733 AllocationContext_t
>>>>>> context,
>>>>>> 1734                                            bool
>>>>>> clear_all_soft_refs,
>>>>>> 1735                                            bool* succeeded) {
>>>>>>
>>>>>> and
>>>>>>
>>>>>> 1748   HeapWord* result =
>>>>>> attempt_allocation_at_safepoint(word_size,
>>>>>> 1749                                            context,
>>>>>> 1750                                            true /*
>>>>>> expect_null_mutator_alloc_region */);
>>>>>>
>>>>>> and also in the header file.
>>>> Please also avoid method definitions like
>>>>
>>>> <return value>
>>>> <class name>::<method-name> ...
>>>>
>>>> (or even worse
>>>>
>>>> <return value>
>>>> <class-name>::
>>>> <method-name>
>>>>
>>>> (this change does not do it that way)
>>>> )
>>>>> Sure.
>>>>>
>>>>>> * Your last version did:
>>>>>>
>>>>>> attempt_allocation_at_safepoint(word_size, context, false);
>>>>>> expand_and_allocate(word_size, context);
>>>>>> do_collection(false, false, word_size);
>>>>>> attempt_allocation_at_safepoint(word_size, context, true);
>>>>>> expand_and_allocate(word_size, context);
>>>>>> do_collection(false, true, word_size);
>>>>>> attempt_allocation_at_safepoint(word_size, context, true);
>>>>>> assert(!collector_policy()->should_clear_all_soft_refs());
>>>>>> expand_and_allocate(word_size, context);
>>>>>>
>>>>>> You have chosen to break this into:
>>>>>>
>>>>>> attempt_allocation_at_safepoint(word_size, context, false);
>>>>>> expand_and_allocate(word_size, context);
>>>>>>
>>>>>> satisfy_failed_allocation_helper(word_size, context, false,
>>>>>> succeeded);
>>>>>>     do_collection(false, false, word_size);
>>>>>>     attempt_allocation_at_safepoint(word_size, context, true);
>>>>>>     expand_and_allocate(word_size, context);
>>>>>>
>>>>>> satisfy_failed_allocation_helper(word_size, context, true,
>>>>>> succeeded);
>>>>>>     do_collection(false, true, word_size);
>>>>>>     attempt_allocation_at_safepoint(word_size, context, true);
>>>>>>     expand_and_allocate(word_size, context);
>>>>>>
>>>>>> assert(!collector_policy()->should_clear_all_soft_refs());
>>>>>>
>>>>>> This is not entirely identical, depending on how important the
>>>>>> assert is that could be an issue since the assert is now skipped in
>>>>>> some cases where it was previously used.
>>>> The assert just states that after setting clear_all_soft_refs to true in
>>>> the call to do_collection, it must be false after collection.
>>>>
>>>> So actually one could change the assert to
>>>>
>>>> assert(!clear_all_soft_refs || !
>>>> collector_policy()->should_clear_all_soft_refs(), "...") and put it
>>>> directly in the helper.
>>>>
>>>> Actually I think CollectorPolicy::clear_all_soft_refs() must be false
>>>> after any full gc, so it should be placed after the do_collection call.
>>>>>> My suggestion would be to break the code up like this:
>>>>>>
>>>>>> satisfy_failed_allocation_helper(word_size, context, do_gc,
>>>>>> clear_softrefs)
>>>>>>     attempt_allocation_at_safepoint(word_size, context, false);
>>>>>>     expand_and_allocate(word_size, context);
>>>>>>     do_collection(false, false, word_size);
>>>>>>
>>>>>> satisfy_failed_allocation_helper(word_size, context, do_gc,
>>>>>> clear_softrefs)
>>>>>>     attempt_allocation_at_safepoint(word_size, context, true);
>>>>>>     expand_and_allocate(word_size, context);
>>>>>>     do_collection(false, true, word_size);
>>>>>>
>>>>>> satisfy_failed_allocation_helper(word_size, context, do_gc,
>>>>>> clear_softrefs)
>>>>>>     attempt_allocation_at_safepoint(word_size, context, true);
>>>>>>     if (!do_gc)
>>>>>> assert(!collector_policy()->should_clear_all_soft_refs());
>>>>>>     expand_and_allocate(word_size, context);
>>>>>>
>>>>>> This would make the code in satisfy_failed_allocation() a lot more
>>>>>> clean.
>>>>>>
>>>>> In webrev.03 I decided to put same way as it was in the original code
>>>>> (before any of my changes),
>>>>> at the very end of method just when we already exhausted all
>>>>> possibilities and about to return NULL.
>>>>>
>>>>> attempt_allocation_at_safepoint(word_size, context, false);
>>>>> expand_and_allocate(word_size, context);
>>>>> do_collection(false, false, word_size);
>>>>> attempt_allocation_at_safepoint(word_size, context, true);
>>>>> do_collection(false, true, word_size);
>>>>> attempt_allocation_at_safepoint(word_size, context, true);
>>>>> assert(!collector_policy()->should_clear_all_soft_refs());
>>>>>
>>>>>   // What else? We might try synchronous finalization later. If the
>>>>> total 1799 // space available is large enough for the allocation, then
>>>>> a more
>>>>>   // complete compaction phase than we've tried so far might be
>>>>> // appropriate.
>>>>> assert(*succeeded, "sanity");
>>>>> return NULL; 1804
>>>>> }
>>>>>
>>>>> webrev.03 has this assert at same position, and I prefer it this way.
>>>>>
>>>>>> * Unless I'm missing some subtle detail here, the usage of the
>>>>>> variable 'succeeded' is unnecessary as 'result' will always contain
>>>>>> the same information. I don't think we should pass on 'succeeded' to
>>>>>> the helper, but simply set it afterwards before we return from
>>>>>> satisfy_failed_allocation().
>>>>> No, helper may return NULL in two differnt cases:
>>>>> 1. Full GC failed, *succeeded = false, no attempts to allocate was
>>>>> made
>>>>> 2. Full GC succeeded, *succeeded = true, all attempts to allocate
>>>>> failed
>>>>>
>>>>> So we need to pass 'succeeded' to the helper.
>>>> Can we rename this to something more descriptive? I had to look into
>>>> do_collection() what it means. Even the gc_succeeded name in the helper
>>>> is better.
>>>>
>>>> Also, the only condition where do_collection() fails is if the gc locker
>>>> is active. This return value is actually ignored in
>>>> do_full_collection().
>>>>
>>>> So the gc locker check could be moved out of do_collection() (adding an
>>>> assert there), and the value of gc_succeeded/succeeded determined using
>>>> this check immediately.
>>>>
>>>> At least, in the helper, move the "assert(*succeeded, ...)" up.
>>>>
>>>> Like:
>>>> --------------
>>>>
>>>> *succeeded = !GCLocker::check_active_before_gc(); // maybe use a temp
>>>> variable here, that is not negated because we check the negated result
>>>> again just later
>>>>
>>>> if (!*succeeded) {
>>>>    return NULL;
>>>> }
>>>>
>>>> do_collection(....);
>>>> assert(
>>>>     !collector_policy()->should_clear_all_soft_refs(), ...); // see above
>>>>
>>>> assert(*succeeded, "..."); // for the paranoid, but I would just skip
>>>> it.
>>>>
>>>> result = attempt_allocation_at_safepoint(...);
>>>>
>>>> if (result != NULL) {
>>>>    return result;
>>>> }
>>>>
>>>> result = expand_and_allocate(...);
>>>>
>>>> return result;
>>>> ---------------
>>>>
>>>> (And the bool return value of do_collection removed).
>>>>
>>>> I also have a question about the order of the order of
>>>> attempt_allocation_at_safepoint() and the expand_and_allocate(), and
>>>> could not find this answered before: so full gc might shrink the heap
>>>> too much after full gc, and we should expand the heap.
>>>> If I were to do this change, there would always (unconditionally) be an
>>>> expansion after full gc, because the current change only expands if the
>>>> allocation after gc fails. Also the 02 change.
>>>>
>>>> The review thread so far suggests that full gc might shrink too much,
>>>> but this still seems to be the case here in some cases.
>>>>
>>>> Thanks,
>>>>    Thomas
>>>>
>>>>
>>>
>


From jon.masamitsu at oracle.com  Tue Aug 25 23:09:30 2015
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Tue, 25 Aug 2015 16:09:30 -0700
Subject: RFR (S): 8067341: Modify PLAB sizing algorithm to waste less
In-Reply-To: <1440518817.2210.21.camel@oracle.com>
References: <1440518817.2210.21.camel@oracle.com>
Message-ID: <55DCF5AA.6030604@oracle.com>

http://cr.openjdk.java.net/~tschatzl/8067341/webrev/src/share/vm/gc/g1/g1_globals.hpp.frames.html

   85   experimental(size_t, G1ExpectedAveragePLABOccupancyPercent, 50,           \
   86                "Expected average occupancy of PLABs after GC in percent.")  \
   87                range(0, 100)                                                \

Range should be (1, 100) since you divide by 
G1ExpectedAveragePLABOccupancyPercent
in the calculations?

http://cr.openjdk.java.net/~tschatzl/8067341/webrev/src/share/vm/gc/g1/g1EvacStats.cpp.frames.html

77     size_t const waste_used_for_calculation = used() > _region_end_waste ? used() - _region_end_waste : 0;

"waste_used_for_calculation" is a measure of "used" during the GC
so maybe a better name would be "used_for_waste_calculation" or
"used_for_PLAB_calculation"?

When _region_end_waste is high, waste_used_for_calculation under 
estimates the
amount used so under estimates the amount of waste (since the waste
is a percentage of waste_used_for_calculation ) and so lowers the size
of the next PLAB.  I can see that it scales reasonably.

Why G1ExpectedAveragePLABOccupancyPercent that can be different than 50?

If it is so that the user can scale the PLAB's up or down, what about 
scale factor
called G1PLABPercent?

   80     size_t const total_waste_allowed = waste_used_for_calculation * (TargetPLABWastePct / 100);
   81     size_t const cur_plab_sz = total_waste_allowed * (G1PLABPercent / 100)


G1PLABPercent would be easier to explain.  It's effect would be linear 
with its
value (as opposed to inversely proportional) and is a shorter name :-).

Jon


On 08/25/2015 09:06 AM, Thomas Schatzl wrote:
> Hi all,
>
>    can I have reviews for the following change that modifies the PLAB
> sizing algorithm of G1 (only) to be much more stable and robust to
> varying sizes of memory allocation?
>
> The old (current for CMS) algorithm tends to create very large sized
> PLABs that mean a lot of waste at the end of PLABs and particularly at
> the end of regions. The other reason is that we can't figure out how the
> current algorithm actually is supposed to work. :)
>
> The end result is that PLAB sizes bounce a lot between some very low
> value and maximum value (which is or has been particularly bad for g1
> because of region boundaries), causing lots of wasted space. We have
> seen that particularly mixed gcs waste up to 50% of memory which caused
> useless GCs and degrading throughput significantly.
>
> This changeset replaces the current algorithm with something more robust
> (and rather conservative), which decreases the waste in most cases
> significantly. Still it typically sizes PLABs larger than default ones
> you would get with -XX:-ResizePLAB, improving performance.
>
> The change is commented in the changeset.
>
> In general, with fixed heap size, the change generally gives the same
> overall throughput on our throughput benchmarks. Total pause times
> decrease quite a bit, but since typically pause time is not an issue
> there is no end-to-end difference.
>
> The change has much more impact on applications that have a large
> variety of object sizes.
>
> This (and all other previous changes) affect automatic heap sizing
> though, so it may result in some at first glance unexpected results
> (like decreased heap usage). There may be some re-tuning of defaults in
> the future.
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8067341
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8067341/webrev/
> Testing:
> jprt, perf benchmarks, tested internally in many applications for more
> than a year
>
> Thanks,
>    Thomas
>



From thomas.schatzl at oracle.com  Wed Aug 26 09:54:57 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 26 Aug 2015 11:54:57 +0200
Subject: RFR (S): 8067341: Modify PLAB sizing algorithm to waste less
In-Reply-To: <55DCF5AA.6030604@oracle.com>
References: <1440518817.2210.21.camel@oracle.com> <55DCF5AA.6030604@oracle.com>
Message-ID: <1440582897.2315.5.camel@oracle.com>

Hi Jon (and Eric),

  thanks for your reviews.

On Tue, 2015-08-25 at 16:09 -0700, Jon Masamitsu wrote:
> http://cr.openjdk.java.net/~tschatzl/8067341/webrev/src/share/vm/gc/g1/g1_globals.hpp.frames.html
> 
>    85   experimental(size_t, G1ExpectedAveragePLABOccupancyPercent, 50,           \
>    86                "Expected average occupancy of PLABs after GC in percent.")  \
>    87                range(0, 100)                                                \
> 
> Range should be (1, 100) since you divide by 
> G1ExpectedAveragePLABOccupancyPercent
> in the calculations?
> 

Okay, good catch. I introduced that variable very late.

> http://cr.openjdk.java.net/~tschatzl/8067341/webrev/src/share/vm/gc/g1/g1EvacStats.cpp.frames.html
> 
> 77     size_t const waste_used_for_calculation = used() > _region_end_waste ? used() - _region_end_waste : 0;
> 
> "waste_used_for_calculation" is a measure of "used" during the GC
> so maybe a better name would be "used_for_waste_calculation" or
> "used_for_PLAB_calculation"?

Okay.

> When _region_end_waste is high, waste_used_for_calculation under 
> estimates the
> amount used so under estimates the amount of waste (since the waste
> is a percentage of waste_used_for_calculation ) and so lowers the size
> of the next PLAB.  I can see that it scales reasonably.
> 
> Why G1ExpectedAveragePLABOccupancyPercent that can be different than 50?
> 
> If it is so that the user can scale the PLAB's up or down, what about 

That is the intent.

> scale factor called G1PLABPercent?

That name seems to me a bit too unspecific, I chose
G1PLABOccupancyFactor for this revision. Feel free to comment on this.

> 
>    80     size_t const total_waste_allowed = waste_used_for_calculation * (TargetPLABWastePct / 100);
>    81     size_t const cur_plab_sz = total_waste_allowed * (G1PLABPercent / 100)
> 
> 
> G1PLABPercent would be easier to explain.  It's effect would be linear 
> with its
> value (as opposed to inversely proportional) and is a shorter name :-).

In that case to me a simple scale factor seems more appropriate - I
created a new webrev to look at:

http://cr.openjdk.java.net/~tschatzl/8067341/webrev.1
http://cr.openjdk.java.net/~tschatzl/8067341/webrev.0_to_1

I also tried to improve the documentation.

Thanks,
  Thomas




From marcus.larsson at oracle.com  Wed Aug 26 10:04:51 2015
From: marcus.larsson at oracle.com (Marcus Larsson)
Date: Wed, 26 Aug 2015 12:04:51 +0200
Subject: RFR (XS): Forward port of 8065331: Add trace events for failed
	allocations
In-Reply-To: <55DACA18.2070605@oracle.com>
References: <55DACA18.2070605@oracle.com>
Message-ID: <55DD8F43.7070003@oracle.com>

Ping!

On 2015-08-24 09:39, Marcus Larsson wrote:
> Hi,
>
> Please review the following patch to forward port the JEP-242 event to 
> JDK9. The patch was originally reviewed in April/March [1], but was 
> only integrated in 8u60. The original patch no longer applies cleanly 
> to 9, due to directory restructuring and changes to includes, so I 
> need a quick re-review for this.
>
> The new patch has updated #includes for the new directory structure 
> but should otherwise be identical to the previous patch.
>
> Webrev:
> http://cr.openjdk.java.net/~mlarsson/8065331_to_9/webrev.00/
>
> Original webrev:
> http://cr.openjdk.java.net/~mlarsson/8065331/webrev.02/
>
> Issue:
> https://bugs.openjdk.java.net/browse/JDK-8065331
>
> Thanks,
> Marcus
>
> [1]: 
> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2015-March/012408.html



From thomas.schatzl at oracle.com  Wed Aug 26 10:12:50 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 26 Aug 2015 12:12:50 +0200
Subject: RFR JDK-8066443: Add G1 support for promotion event
In-Reply-To: <55CB8121.1080502@oracle.com>
References: <55CB8121.1080502@oracle.com>
Message-ID: <1440583970.2315.6.camel@oracle.com>

Hi Staffan,

On Wed, 2015-08-12 at 10:23 -0700, Staffan Friberg wrote:
> Hi,
> 
> This webrev adds support for the G1 collector and the promotion event 
> that collects statistics about the promoted object types.
> 
> Once reviewed I would need a sponsor to push the patch.
> 
> Bug: https://bugs.openjdk.java.net/browse/JDK-8066443
> Webrev: http://cr.openjdk.java.net/~sfriberg/JDK-8066443/webrev.01/
> 

  looks good to me. I can sponsor it if required.

Thanks,
  Thomas




From thomas.schatzl at oracle.com  Wed Aug 26 10:26:54 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 26 Aug 2015 12:26:54 +0200
Subject: Request for code review - 8130265:
	gctests/LargeObjects/large001 fails with OutOfMemoryError: Java heap
	space
In-Reply-To: <55DC6CB5.4010108@oracle.com>
References: <55D356FB.1030603@oracle.com> <55D35A12.8020608@oracle.com>
	<55D35F1F.3090700@oracle.com> <55D36DC7.6010901@oracle.com>
	<55D38CBE.8070408@oracle.com> <55D394AD.6020104@oracle.com>
	<55D4E39D.5070805@oracle.com> <55D4E8C7.1080703@oracle.com>
	<55D4EEF5.9030504@oracle.com>
	<2C141666-BDBC-428C-9E40-4FB78F076192@oracle.com>
	<1440082894.2347.14.camel@oracle.com> <55D62D77.8020504@oracle.com>
	<55D71703.4070505@oracle.com> <55D74766.5020202@oracle.com>
	<1440420922.2348.82.camel@oracle.com> <55DB8B7E.5010601@oracle.com>
	<55DC6CB5.4010108@oracle.com>
Message-ID: <1440584814.2315.16.camel@oracle.com>

Hi,

  looking at webrev.05, but answering here.

On Tue, 2015-08-25 at 15:25 +0200, Jesper Wilhelmsson wrote:
> Hi Alex,
> 
> I like this approach :)
> 
> Just one comment:
> 
> After GC in the helper you put the assert that soft references should have been 
> handled. The first call to this helper will not clear soft references. The 
> original code only has this assert at the end of the function. I think it is 

Whatever the original code did, there is the general requirement that
after any full gc, collector_policy()->should_clear_all_soft_refs()
should be false.

> enough to keep it there, at the end of satisfy_failed_allocation().

It seems to be randomly placed now, checking only after everything else
fails. I do agree that the original code did the same, but why not
improve upon this?

If you want in a separate CR.

> 1764     assert(!collector_policy()->should_clear_all_soft_refs(),
> 1765            "Flag should have been handled and cleared prior to this point");
>
> Besides that I'm fine with this version.


- Some braces next to boolean parameters are not aligned properly. E.g.
in line 1777 and 1804.

>
> 
> Den 24/8/15 kl. 23:24, skrev Alexander Harlap:
> > Hi Thomas and Jesper,
> >
> > Here is new version of change:
> >
> > http://cr.openjdk.java.net/~aharlap/8130265/webrev.04/
> >
> > I followed Jesper's advice to have
> > satisfy_failed_allocation_helper(..) :
> >    attempt_allocation_at_safepoint(...);
> >    expand_and_allocate(...);
> >    do_collection(...);
> >
> > So I did not reorder  calls to attempt_allocation_at_safepoint(...) and
> > expand_and_allocate(...)
> >
> > -Alex
> >
> >
> > On 8/24/2015 8:55 AM, Thomas Schatzl wrote:
> >> Hi,
> >>
> >> On Fri, 2015-08-21 at 11:44 -0400, Alexander Harlap wrote:
> >>>
> >>> On 8/21/2015 8:18 AM, Jesper Wilhelmsson wrote:
> >>>
> >>>> Hi Alex,
> >>>>
> >>>> Alexander Harlap skrev den 20/8/15 21:41:
> >>>>> Kim and Thomas,
> >>>>>
> >>>>> Here is new version with your suggestions in:
> >>>>>
> >>>>> http://cr.openjdk.java.net/~aharlap/8130265/webrev.03/
> >>>>> <http://cr.openjdk.java.net/%7Eaharlap/8130265/webrev.03/>
> >>>>> Alex
> >>>> A few comments on this latest version.
> >>>>
> >>>> * Why do you assert_at_safepoint and set succeeded to true in your
> >>>> helper? The original code did not do this at these places. I think
> >>>> you can remove these lines.
> >>>>
> >>>> 1736   assert_at_safepoint(true /* should_be_vm_thread */);
> >>>> 1737
> >>>> 1738   *succeeded = true;
> >>>>
> >>> Even method called satisfy_failed_allocation_helper , there is no way
> >>> to make sure that it is called only  from satisfy_failed_allocation.
> >> I do not think we guarantee that for any method, i.e. that a given
> >> method somehow checks its call chain. This would require much better
> >> encapsulation, and in some cases this does not help you either. Also it
> >> would require you to state the allowed callers somehow.
> >>
> >> The helper may be private too, not protected.
> >>
> >>> satisfy_failed_allocation_helper is a separate method and those lines
> >>> are needed.
> >>> So I would like to keep
> >>> 1736   assert_at_safepoint(true /* should_be_vm_thread */);
> >>> 1737
> >>> 1738   *succeeded = true;
> >>>> * Please align the arguments in
> >>>>
> >>>> 1732 G1CollectedHeap::satisfy_failed_allocation_helper(size_t
> >>>> word_size,
> >>>> 1733                                            AllocationContext_t
> >>>> context,
> >>>> 1734                                            bool
> >>>> clear_all_soft_refs,
> >>>> 1735                                            bool* succeeded) {
> >>>>
> >>>> and
> >>>>
> >>>> 1748   HeapWord* result =
> >>>> attempt_allocation_at_safepoint(word_size,
> >>>> 1749                                            context,
> >>>> 1750                                            true /*
> >>>> expect_null_mutator_alloc_region */);
> >>>>
> >>>> and also in the header file.
> >> Please also avoid method definitions like
> >>
> >> <return value>
> >> <class name>::<method-name> ...
> >>
> >> (or even worse
> >>
> >> <return value>
> >> <class-name>::
> >> <method-name>
> >>
> >> (this change does not do it that way)
> >> )
> >>> Sure.
> >>>
> >>>> * Your last version did:
> >>>>
> >>>> attempt_allocation_at_safepoint(word_size, context, false);
> >>>> expand_and_allocate(word_size, context);
> >>>> do_collection(false, false, word_size);
> >>>> attempt_allocation_at_safepoint(word_size, context, true);
> >>>> expand_and_allocate(word_size, context);
> >>>> do_collection(false, true, word_size);
> >>>> attempt_allocation_at_safepoint(word_size, context, true);
> >>>> assert(!collector_policy()->should_clear_all_soft_refs());
> >>>> expand_and_allocate(word_size, context);
> >>>>
> >>>> You have chosen to break this into:
> >>>>
> >>>> attempt_allocation_at_safepoint(word_size, context, false);
> >>>> expand_and_allocate(word_size, context);
> >>>>
> >>>> satisfy_failed_allocation_helper(word_size, context, false,
> >>>> succeeded);
> >>>>     do_collection(false, false, word_size);
> >>>>     attempt_allocation_at_safepoint(word_size, context, true);
> >>>>     expand_and_allocate(word_size, context);
> >>>>
> >>>> satisfy_failed_allocation_helper(word_size, context, true,
> >>>> succeeded);
> >>>>     do_collection(false, true, word_size);
> >>>>     attempt_allocation_at_safepoint(word_size, context, true);
> >>>>     expand_and_allocate(word_size, context);
> >>>>
> >>>> assert(!collector_policy()->should_clear_all_soft_refs());
> >>>>
> >>>> This is not entirely identical, depending on how important the
> >>>> assert is that could be an issue since the assert is now skipped in
> >>>> some cases where it was previously used.
> >> The assert just states that after setting clear_all_soft_refs to true in
> >> the call to do_collection, it must be false after collection.
> >>
> >> So actually one could change the assert to
> >>
> >> assert(!clear_all_soft_refs || !
> >> collector_policy()->should_clear_all_soft_refs(), "...") and put it
> >> directly in the helper.
> >>
> >> Actually I think CollectorPolicy::clear_all_soft_refs() must be false
> >> after any full gc, so it should be placed after the do_collection call.
> >>>> My suggestion would be to break the code up like this:
> >>>>
> >>>> satisfy_failed_allocation_helper(word_size, context, do_gc,
> >>>> clear_softrefs)
> >>>>     attempt_allocation_at_safepoint(word_size, context, false);
> >>>>     expand_and_allocate(word_size, context);
> >>>>     do_collection(false, false, word_size);
> >>>>
> >>>> satisfy_failed_allocation_helper(word_size, context, do_gc,
> >>>> clear_softrefs)
> >>>>     attempt_allocation_at_safepoint(word_size, context, true);
> >>>>     expand_and_allocate(word_size, context);
> >>>>     do_collection(false, true, word_size);
> >>>>
> >>>> satisfy_failed_allocation_helper(word_size, context, do_gc,
> >>>> clear_softrefs)
> >>>>     attempt_allocation_at_safepoint(word_size, context, true);
> >>>>     if (!do_gc)
> >>>>        assert(!collector_policy()->should_clear_all_soft_refs());
> >>>>     expand_and_allocate(word_size, context);
> >>>>
> >>>> This would make the code in satisfy_failed_allocation() a lot more
> >>>> clean.
> >>>>
> >>> In webrev.03 I decided to put same way as it was in the original code
> >>> (before any of my changes),
> >>> at the very end of method just when we already exhausted all
> >>> possibilities and about to return NULL.
> >>>
> >>> attempt_allocation_at_safepoint(word_size, context, false);
> >>> expand_and_allocate(word_size, context);
> >>> do_collection(false, false, word_size);
> >>> attempt_allocation_at_safepoint(word_size, context, true);
> >>> do_collection(false, true, word_size);
> >>> attempt_allocation_at_safepoint(word_size, context, true);
> >>> assert(!collector_policy()->should_clear_all_soft_refs());
> >>>
> >>>   // What else? We might try synchronous finalization later. If the
> >>> total 1799 // space available is large enough for the allocation, then
> >>> a more
> >>>   // complete compaction phase than we've tried so far might be
> >>> // appropriate.
> >>> assert(*succeeded, "sanity");
> >>> return NULL; 1804
> >>> }
> >>>
> >>> webrev.03 has this assert at same position, and I prefer it this way.
> >>>

[...]

> >> At least, in the helper, move the "assert(*succeeded, ...)" up.
> >>
> >> Like:
> >> --------------
> >>
> >> *succeeded = !GCLocker::check_active_before_gc(); // maybe use a temp
> >> variable here, that is not negated because we check the negated result
> >> again just later
> >>
> >> if (!*succeeded) {
> >>    return NULL;
> >> }
> >>
> >> do_collection(....);
> >> assert(
> >>     !collector_policy()->should_clear_all_soft_refs(), ...); // see above
> >>
> >> assert(*succeeded, "..."); // for the paranoid, but I would just skip
> >> it.
> >>
> >> result = attempt_allocation_at_safepoint(...);
> >>
> >> if (result != NULL) {
> >>    return result;
> >> }
> >>
> >> result = expand_and_allocate(...);
> >>
> >> return result;
> >> ---------------
> >>
> >> (And the bool return value of do_collection removed).
> >>
> >> I also have a question about the order of
> >> attempt_allocation_at_safepoint() and the expand_and_allocate(), and
> >> could not find this answered before: so full gc might shrink the heap
> >> too much after full gc, and we should expand the heap.
> >> If I were to do this change, there would always (unconditionally) be an
> >> expansion after full gc, because the current change only expands if the
> >> allocation after gc fails. Also the 02 change.
> >>
> >> The review thread so far suggests that full gc might shrink too much,
> >> but this still seems to be the case here in some cases.

Could you clarify this question? To me, this looks like a potential bug
that has been overlooked so far in this change.

Thanks,
  Thomas



From thomas.schatzl at oracle.com  Wed Aug 26 12:29:02 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 26 Aug 2015 14:29:02 +0200
Subject: RFR (S): 8067341: Modify PLAB sizing algorithm to waste less
In-Reply-To: <1440582897.2315.5.camel@oracle.com>
References: <1440518817.2210.21.camel@oracle.com>
	<55DCF5AA.6030604@oracle.com> <1440582897.2315.5.camel@oracle.com>
Message-ID: <1440592142.2315.30.camel@oracle.com>

Hi all,

  I tried to get some more internal comments on the question of the
final change to improve PLAB sizes.

On Wed, 2015-08-26 at 11:54 +0200, Thomas Schatzl wrote:
> Hi Jon (and Eric),
[...]
> > When _region_end_waste is high, waste_used_for_calculation under 
> > estimates the
> > amount used so under estimates the amount of waste (since the waste
> > is a percentage of waste_used_for_calculation ) and so lowers the size
> > of the next PLAB.  I can see that it scales reasonably.
> > 
> > Why G1ExpectedAveragePLABOccupancyPercent that can be different than 50?
> > 
> > If it is so that the user can scale the PLAB's up or down, what about 
> 
> That is the intent.
> 
> > scale factor called G1PLABPercent?
> 
> That name seems to me a bit too unspecific, I chose
> G1PLABOccupancyFactor for this revision. Feel free to comment on this.

Others commented on the name and the value range; mainly that a
percentage from 1-100 (or something similar) seems more natural or at
least common, also the use as a divisor in the calculation.

Using a double instead of an integer allows more flexibility in the
choice of values.

As for the name, the argument came up that the G1PLABPercent suggestion
is somewhat too undescriptive, and it is better to use a longer, more
appropriate name like G1LastPLABAverageOccupancy (note that added "Last"
in the name). Since we do not expect it to be used a lot, except experts
that measured that there is an issue there previously, a longer name
does not seem to be a big issue.

http://cr.openjdk.java.net/~tschatzl/8067341/webrev.2
http://cr.openjdk.java.net/~tschatzl/8067341/webrev.0_to_2 (change from baseline!)

Thanks,
  Thomas



From erik.helin at oracle.com  Wed Aug 26 13:56:24 2015
From: erik.helin at oracle.com (Erik Helin)
Date: Wed, 26 Aug 2015 15:56:24 +0200
Subject: RFR: 8134504: Remove usage of EvacuationInfo from G1CollectorPolicy
Message-ID: <20150826135624.GD10460@ehelin.jrpg.bea.com>

Hi all,

this tiny patch cleans up the relationship between the classes
EvacuationInfo and G1CollectorPolicy. There is no need for the
G1CollectorPolicy to take EvacuationInfo objects as arguments in two
methods just to fill in some data, it just complicates the signature of
those methods. With the patch the data is retrieved from the
G1CollectorPolicy using getters instead.

Webrev:
http://cr.openjdk.java.net/~ehelin/8134504/webrev.00/

Bug:
https://bugs.openjdk.java.net/browse/JDK-8134504

Testing:
JPRT

Thanks,
Erik


From erik.helin at oracle.com  Wed Aug 26 14:19:00 2015
From: erik.helin at oracle.com (Erik Helin)
Date: Wed, 26 Aug 2015 16:19:00 +0200
Subject: RFR: 8134509: G1ParCopyClosure does not need a ReferenceProcessor
Message-ID: <20150826141900.GE10460@ehelin.jrpg.bea.com>

Hi all,

this small patch removes an unused argument to the constructor of
G1ParCopyClosure. The argument ReferenceProcessor* rp is never used and
can therefore be removed, the getter method rp() can therefore also be
removed.

Webrev:
http://cr.openjdk.java.net/~ehelin/8134509/webrev.00/

Bug:
https://bugs.openjdk.java.net/browse/JDK-8134509

Testing:
JPRT

Thanks,
Erik


From mikael.gerdin at oracle.com  Wed Aug 26 14:20:22 2015
From: mikael.gerdin at oracle.com (Mikael Gerdin)
Date: Wed, 26 Aug 2015 16:20:22 +0200
Subject: RFR: 8134504: Remove usage of EvacuationInfo from
	G1CollectorPolicy
In-Reply-To: <20150826135624.GD10460@ehelin.jrpg.bea.com>
References: <20150826135624.GD10460@ehelin.jrpg.bea.com>
Message-ID: <55DDCB26.2080703@oracle.com>

Hi Erik,

On 2015-08-26 15:56, Erik Helin wrote:
> Hi all,
>
> this tiny patch cleans up the relationship between the classes
> EvacuationInfo and G1CollectorPolicy. There is no need for the
> G1CollectorPolicy to take EvacuationInfo objects as arguments in two
> methods just to fill in some data, it just complicates the signature of
> those methods. With the patch the data is retrieved from the
> G1CollectorPolicy using getters instead.
>
> Webrev:
> http://cr.openjdk.java.net/~ehelin/8134504/webrev.00/

Looks good to me.
/Mikael

>
> Bug:
> https://bugs.openjdk.java.net/browse/JDK-8134504
>
> Testing:
> JPRT
>
> Thanks,
> Erik
>



From mikael.gerdin at oracle.com  Wed Aug 26 14:21:03 2015
From: mikael.gerdin at oracle.com (Mikael Gerdin)
Date: Wed, 26 Aug 2015 16:21:03 +0200
Subject: RFR: 8134509: G1ParCopyClosure does not need a ReferenceProcessor
In-Reply-To: <20150826141900.GE10460@ehelin.jrpg.bea.com>
References: <20150826141900.GE10460@ehelin.jrpg.bea.com>
Message-ID: <55DDCB4F.3080000@oracle.com>

Hi Erik,

On 2015-08-26 16:19, Erik Helin wrote:
> Hi all,
>
> this small patch removes an unused argument to the constructor of
> G1ParCopyClosure. The argument ReferenceProcessor* rp is never used and
> can therefore be removed, the getter method rp() can therefore also be
> removed.
>
> Webrev:
> http://cr.openjdk.java.net/~ehelin/8134509/webrev.00/

Looks good to me.
/Mikael

>
> Bug:
> https://bugs.openjdk.java.net/browse/JDK-8134509
>
> Testing:
> JPRT
>
> Thanks,
> Erik
>



From stefan.karlsson at oracle.com  Wed Aug 26 14:25:31 2015
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Wed, 26 Aug 2015 16:25:31 +0200
Subject: RFR: 8134509: G1ParCopyClosure does not need a ReferenceProcessor
In-Reply-To: <20150826141900.GE10460@ehelin.jrpg.bea.com>
References: <20150826141900.GE10460@ehelin.jrpg.bea.com>
Message-ID: <55DDCC5B.3030003@oracle.com>

On 2015-08-26 16:19, Erik Helin wrote:
> Hi all,
>
> this small patch removes an unused argument to the constructor of
> G1ParCopyClosure. The argument ReferenceProcessor* rp is never used and
> can therefore be removed, the getter method rp() can therefore also be
> removed.
>
> Webrev:
> http://cr.openjdk.java.net/~ehelin/8134509/webrev.00/

Looks good.

StefanK

>
> Bug:
> https://bugs.openjdk.java.net/browse/JDK-8134509
>
> Testing:
> JPRT
>
> Thanks,
> Erik



From jon.masamitsu at oracle.com  Wed Aug 26 14:36:57 2015
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Wed, 26 Aug 2015 07:36:57 -0700
Subject: RFR (S): 8067341: Modify PLAB sizing algorithm to waste less
In-Reply-To: <1440592142.2315.30.camel@oracle.com>
References: <1440518817.2210.21.camel@oracle.com>
	<55DCF5AA.6030604@oracle.com> <1440582897.2315.5.camel@oracle.com>
	<1440592142.2315.30.camel@oracle.com>
Message-ID: <55DDCF09.3090203@oracle.com>

Thomas,

Looks good.

After looking at the version with G1PLABOccupancyFactor, returning
to G1LastPLABAverageOccupancy does seem more natural.   It's easier
to understand what the flag represents.

One correction on the comment.

http://cr.openjdk.java.net/~tschatzl/8067341/webrev.0_to_2/src/share/vm/gc/g1/g1EvacStats.cpp.frames.html

68 // The amount of allocation performed is we can spend is independent 
of the
69 // number of threads, so is the maximum waste we can spend in total. 
So if we used Should be ? // The amount of allocation performed  is independent of the

And a suggested rewording.

   57     // Calculate the new PLAB size as the amount of space that could be wasted to
   58     // keep TargetPLABWastePct given latest memory usage and that the last buffer
59 // will be G1PLABPercent full. // The size of the PLAB caps the 
amount of space that can be wasted at the end of // the collection. 
Calculate the new PLAB size to achieve the TargetPLABWastePct
// given latest memory usage and that the last bufferwill be G1PLABPercent full.  


  Your call on the suggested rewording.  I don't need a new webrev.

Jon

On 8/26/2015 5:29 AM, Thomas Schatzl wrote:
> Hi all,
>
>    I tried to get some more internal comments on the question of the
> final change to improve PLAB sizes.
>
> On Wed, 2015-08-26 at 11:54 +0200, Thomas Schatzl wrote:
>> Hi Jon (and Eric),
> [...]
>>> When _region_end_waste is high, waste_used_for_calculation under
>>> estimates the
>>> amount used so under estimates the amount of waste (since the waste
>>> is a percentage of waste_used_for_calculation ) and so lowers the size
>>> of the next PLAB.  I can see that it scales reasonably.
>>>
>>> Why G1ExpectedAveragePLABOccupancyPercent that can be different than 50?
>>>
>>> If it is so that the user can scale the PLAB's up or down, what about
>> That is the intent.
>>
>>> scale factor called G1PLABPercent?
>> That name seems to me a bit too unspecific, I chose
>> G1PLABOccupancyFactor for this revision. Feel free to comment on this.
> Others commented on the name and the value range; mainly that a
> percentage from 1-100 (or something similar) seems more natural or at
> least common, also the use as a divisor in the calculation.
>
> Using a double instead of an integer allows more flexibility in the
> choice of values.
>
> As for the name, the argument came up that the G1PLABPercent suggestion
> is somewhat too undescriptive, and it is better to use a longer, more
> appropriate name like G1LastPLABAverageOccupancy (note that added "Last"
> in the name). Since we do not expect it to be used a lot, except experts
> that measured that there is an issue there previously, a longer name
> does not seem to be a big issue.
>
> http://cr.openjdk.java.net/~tschatzl/8067341/webrev.2
> http://cr.openjdk.java.net/~tschatzl/8067341/webrev.0_to_2 (change from baseline!)
>
> Thanks,
>    Thomas
>



From jesper.wilhelmsson at oracle.com  Wed Aug 26 16:27:44 2015
From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson)
Date: Wed, 26 Aug 2015 18:27:44 +0200
Subject: RFR: 8134504: Remove usage of EvacuationInfo from
	G1CollectorPolicy
In-Reply-To: <20150826135624.GD10460@ehelin.jrpg.bea.com>
References: <20150826135624.GD10460@ehelin.jrpg.bea.com>
Message-ID: <55DDE900.3080302@oracle.com>

Looks good!
/Jesper

Den 26/8/15 kl. 15:56, skrev Erik Helin:
> Hi all,
>
> this tiny patch cleans up the relationship between the classes
> EvacuationInfo and G1CollectorPolicy. There is no need for the
> G1CollectorPolicy to take EvacuationInfo objects as arguments in two
> methods just to fill in some data, it just complicates the signature of
> those methods. With the patch the data is retrieved from the
> G1CollectorPolicy using getters instead.
>
> Webrev:
> http://cr.openjdk.java.net/~ehelin/8134504/webrev.00/
>
> Bug:
> https://bugs.openjdk.java.net/browse/JDK-8134504
>
> Testing:
> JPRT
>
> Thanks,
> Erik
>


From alexander.harlap at oracle.com  Thu Aug 27 00:00:29 2015
From: alexander.harlap at oracle.com (Alexander Harlap)
Date: Wed, 26 Aug 2015 20:00:29 -0400
Subject: Request for code review - 8130265: gctests/LargeObjects/large001
	fails with OutOfMemoryError: Java heap space
In-Reply-To: <1440584814.2315.16.camel@oracle.com>
References: <55D356FB.1030603@oracle.com> <55D35A12.8020608@oracle.com>
	<55D35F1F.3090700@oracle.com> <55D36DC7.6010901@oracle.com>
	<55D38CBE.8070408@oracle.com> <55D394AD.6020104@oracle.com>
	<55D4E39D.5070805@oracle.com> <55D4E8C7.1080703@oracle.com>
	<55D4EEF5.9030504@oracle.com>
	<2C141666-BDBC-428C-9E40-4FB78F076192@oracle.com>
	<1440082894.2347.14.camel@oracle.com> <55D62D77.8020504@oracle.com>
	<55D71703.4070505@oracle.com> <55D74766.5020202@oracle.com>
	<1440420922.2348.82.camel@oracle.com> <55DB8B7E.5010601@oracle.com>
	<55DC6CB5.4010108@oracle.com> <1440584814.2315.16.camel@oracle.com>
Message-ID: <55DE531D.9090703@oracle.com>



On 8/26/2015 6:26 AM, Thomas Schatzl wrote:
> Hi,
>
>    looking at webrev.05, but answering here.
>
> On Tue, 2015-08-25 at 15:25 +0200, Jesper Wilhelmsson wrote:
>> Hi Alex,
>>
>> I like this approach :)
>>
>> Just one comment:
>>
>> After GC in the helper you put the assert that soft references should have been
>> handled. The first call to this helper will not clear soft references. The
>> original code only has this assert at the end of the function. I think it is
> Whatever the original code did, there is the general requirement that
> after any full gc, collector_policy()->should_clear_all_soft_refs()
> should be false.
>   
>> enough to keep it there, at the end of satisfy_failed_allocation().
> It seems to be randomly placed now, checking only after everything else
> fails. I do agree that the original code did the same, but why not
> improve upon this?
>
> If you want in a separate CR.
Yes, I would agree to make a separate CR. Only I will need help to 
specify issue.
>
>> 1764     assert(!collector_policy()->should_clear_all_soft_refs(),
>> 1765            "Flag should have been handled and cleared prior to this point");
>>
>> Besides that I'm fine with this version.
> - Some braces next to boolean parameters are not aligned properly. E.g.
> in line 1777 and 1804.
I am sorry, I can not locate those defects - did you mean lines 1777 and 
1804 in webrev.05?
Alex
>> Den 24/8/15 kl. 23:24, skrev Alexander Harlap:
>>> Hi Thomas and Jesper,
>>>
>>> Here is new version of change:
>>>
>>> http://cr.openjdk.java.net/~aharlap/8130265/webrev.04/
>>>
>>> I followed Jesper's advice to have
>>> satisfy_failed_allocation_helper(..) :
>>>     attempt_allocation_at_safepoint(...);
>>>     expand_and_allocate(...);
>>>     do_collection(...);
>>>
>>> So I did not reorder  calls to attempt_allocation_at_safepoint(...) and
>>> expand_and_allocate(...)
>>>
>>> -Alex
>>>
>>>
>>> On 8/24/2015 8:55 AM, Thomas Schatzl wrote:
>>>> Hi,
>>>>
>>>> On Fri, 2015-08-21 at 11:44 -0400, Alexander Harlap wrote:
>>>>> On 8/21/2015 8:18 AM, Jesper Wilhelmsson wrote:
>>>>>
>>>>>> Hi Alex,
>>>>>>
>>>>>> Alexander Harlap skrev den 20/8/15 21:41:
>>>>>>> Kim and Thomas,
>>>>>>>
>>>>>>> Here is new version with your suggestions in:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~aharlap/8130265/webrev.03/
>>>>>>> <http://cr.openjdk.java.net/%7Eaharlap/8130265/webrev.03/>
>>>>>>> Alex
>>>>>> A few comments on this latest version.
>>>>>>
>>>>>> * Why do you assert_at_safepoint and set succeeded to true in your
>>>>>> helper? The original code did not do this at these places. I think
>>>>>> you can remove these lines.
>>>>>>
>>>>>> 1736   assert_at_safepoint(true /* should_be_vm_thread */);
>>>>>> 1737
>>>>>> 1738   *succeeded = true;
>>>>>>
>>>>> Even method called satisfy_failed_allocation_helper , there is no way
>>>>> to make sure that it is called only  from satisfy_failed_allocation.
>>>> I do not think we guarantee that for any method, i.e. that a given
>>>> method somehow checks its call chain. This would require much better
>>>> encapsulation, and in some cases this does not help you either. Also it
>>>> would require you to state the allowed callers somehow.
>>>>
>>>> The helper may be private too, not protected.
>>>>
>>>>> satisfy_failed_allocation_helper is a separate method and those lines
>>>>> are needed.
>>>>> So I would like to keep
>>>>> 1736   assert_at_safepoint(true /* should_be_vm_thread */);
>>>>> 1737
>>>>> 1738   *succeeded = true;
>>>>>> * Please align the arguments in
>>>>>>
>>>>>> 1732 G1CollectedHeap::satisfy_failed_allocation_helper(size_t
>>>>>> word_size,
>>>>>> 1733                                            AllocationContext_t
>>>>>> context,
>>>>>> 1734                                            bool
>>>>>> clear_all_soft_refs,
>>>>>> 1735                                            bool* succeeded) {
>>>>>>
>>>>>> and
>>>>>>
>>>>>> 1748   HeapWord* result =
>>>>>> attempt_allocation_at_safepoint(word_size,
>>>>>> 1749                                            context,
>>>>>> 1750                                            true /*
>>>>>> expect_null_mutator_alloc_region */);
>>>>>>
>>>>>> and also in the header file.
>>>> Please also avoid method definitions like
>>>>
>>>> <return value>
>>>> <class name>::<method-name> ...
>>>>
>>>> (or even worse
>>>>
>>>> <return value>
>>>> <class-name>::
>>>> <method-name>
>>>>
>>>> (this change does not do it that way)
>>>> )
>>>>> Sure.
>>>>>
>>>>>> * Your last version did:
>>>>>>
>>>>>> attempt_allocation_at_safepoint(word_size, context, false);
>>>>>> expand_and_allocate(word_size, context);
>>>>>> do_collection(false, false, word_size);
>>>>>> attempt_allocation_at_safepoint(word_size, context, true);
>>>>>> expand_and_allocate(word_size, context);
>>>>>> do_collection(false, true, word_size);
>>>>>> attempt_allocation_at_safepoint(word_size, context, true);
>>>>>> assert(!collector_policy()->should_clear_all_soft_refs());
>>>>>> expand_and_allocate(word_size, context);
>>>>>>
>>>>>> You have chosen to break this into:
>>>>>>
>>>>>> attempt_allocation_at_safepoint(word_size, context, false);
>>>>>> expand_and_allocate(word_size, context);
>>>>>>
>>>>>> satisfy_failed_allocation_helper(word_size, context, false,
>>>>>> succeeded);
>>>>>>      do_collection(false, false, word_size);
>>>>>>      attempt_allocation_at_safepoint(word_size, context, true);
>>>>>>      expand_and_allocate(word_size, context);
>>>>>>
>>>>>> satisfy_failed_allocation_helper(word_size, context, true,
>>>>>> succeeded);
>>>>>>      do_collection(false, true, word_size);
>>>>>>      attempt_allocation_at_safepoint(word_size, context, true);
>>>>>>      expand_and_allocate(word_size, context);
>>>>>>
>>>>>> assert(!collector_policy()->should_clear_all_soft_refs());
>>>>>>
>>>>>> This is not entirely identical, depending on how important the
>>>>>> assert is that could be an issue since the assert is now skipped in
>>>>>> some cases where it was previously used.
>>>> The assert just states that after setting clear_all_soft_refs to true in
>>>> the call to do_collection, it must be false after collection.
>>>>
>>>> So actually one could change the assert to
>>>>
>>>> assert(!clear_all_soft_refs || !
>>>> collector_policy()->should_clear_all_soft_refs(), "...") and put it
>>>> directly in the helper.
>>>>
>>>> Actually I think CollectorPolicy::clear_all_soft_refs() must be false
>>>> after any full gc, so it should be placed after the do_collection call.
>>>>>> My suggestion would be to break the code up like this:
>>>>>>
>>>>>> satisfy_failed_allocation_helper(word_size, context, do_gc,
>>>>>> clear_softrefs)
>>>>>>      attempt_allocation_at_safepoint(word_size, context, false);
>>>>>>      expand_and_allocate(word_size, context);
>>>>>>      do_collection(false, false, word_size);
>>>>>>
>>>>>> satisfy_failed_allocation_helper(word_size, context, do_gc,
>>>>>> clear_softrefs)
>>>>>>      attempt_allocation_at_safepoint(word_size, context, true);
>>>>>>      expand_and_allocate(word_size, context);
>>>>>>      do_collection(false, true, word_size);
>>>>>>
>>>>>> satisfy_failed_allocation_helper(word_size, context, do_gc,
>>>>>> clear_softrefs)
>>>>>>      attempt_allocation_at_safepoint(word_size, context, true);
>>>>>>      if (!do_gc)
>>>>>>         assert(!collector_policy()->should_clear_all_soft_refs());
>>>>>>      expand_and_allocate(word_size, context);
>>>>>>
>>>>>> This would make the code in satisfy_failed_allocation() a lot more
>>>>>> clean.
>>>>>>
>>>>> In webrev.03 I decided to put same way as it was in the original code
>>>>> (before any of my changes),
>>>>> at the very end of method just when we already exhausted all
>>>>> possibilities and about to return NULL.
>>>>>
>>>>> attempt_allocation_at_safepoint(word_size, context, false);
>>>>> expand_and_allocate(word_size, context);
>>>>> do_collection(false, false, word_size);
>>>>> attempt_allocation_at_safepoint(word_size, context, true);
>>>>> do_collection(false, true, word_size);
>>>>> attempt_allocation_at_safepoint(word_size, context, true);
>>>>> assert(!collector_policy()->should_clear_all_soft_refs());
>>>>>
>>>>>    // What else? We might try synchronous finalization later. If the
>>>>> total 1799 // space available is large enough for the allocation, then
>>>>> a more
>>>>>    // complete compaction phase than we've tried so far might be
>>>>> // appropriate.
>>>>> assert(*succeeded, "sanity");
>>>>> return NULL; 1804
>>>>> }
>>>>>
>>>>> webrev.03 has this assert at same position, and I prefer it this way.
>>>>>
> [...]
>
>>>> At least, in the helper, move the "assert(*succeeded, ...)" up.
>>>>
>>>> Like:
>>>> --------------
>>>>
>>>> *succeeded = !GCLocker::check_active_before_gc(); // maybe use a temp
>>>> variable here, that is not negated because we check the negated result
>>>> again just later
>>>>
>>>> if (!*succeeded) {
>>>>     return NULL;
>>>> }
>>>>
>>>> do_collection(....);
>>>> assert(
>>>>      !collector_policy()->should_clear_all_soft_refs(), ...); // see above
>>>>
>>>> assert(*succeeded, "..."); // for the paranoid, but I would just skip
>>>> it.
>>>>
>>>> result = attempt_allocation_at_safepoint(...);
>>>>
>>>> if (result != NULL) {
>>>>     return result;
>>>> }
>>>>
>>>> result = expand_and_allocate(...);
>>>>
>>>> return result;
>>>> ---------------
>>>>
>>>> (And the bool return value of do_collection removed).
>>>>
>>>> I also have a question about the order of
>>>> attempt_allocation_at_safepoint() and the expand_and_allocate(), and
>>>> could not find this answered before: so full gc might shrink the heap
>>>> too much after full gc, and we should expand the heap.
>>>> If I were to do this change, there would always (unconditionally) be an
>>>> expansion after full gc, because the current change only expands if the
>>>> allocation after gc fails. Also the 02 change.
>>>>
>>>> The review thread so far suggests that full gc might shrink too much,
>>>> but this still seems to be the case here in some cases.
> Could you clarify this question? To me, this looks like a potential bug
> that has been overlooked so far in this change.
Yes, full GC in some situation may over-shrink heap such way that heap 
expansion will be needed. It was reason for 
gctests/LargeObjects/large001 failure.  Still I prefer to do expansion 
only after allocation attempt failed. Why should we do unconditional 
expansion ( even if  it is not needed )?  In the existing code we have 
call to expand_and_allocate() only after 
attempt_allocation_at_safepoint() failure.  What potential bug do you mean?

> Thanks,
>    Thomas
>



From ysr1729 at gmail.com  Thu Aug 27 02:16:00 2015
From: ysr1729 at gmail.com (Srinivas Ramakrishna)
Date: Wed, 26 Aug 2015 19:16:00 -0700
Subject: race condition in GC logs
In-Reply-To: <CABzyjymhn1DXaCfZ35zGC2tVah2SiMk1W-mbgvhYVjBY38se8w@mail.gmail.com>
References: <9E79C717-7C53-49F5-AD0F-C003A03F11E5@kodewerk.com>
	<55D30D50.8060302@oracle.com> <55D32774.2090903@oracle.com>
	<CA61B527-3D6E-46EF-AAAA-425C306ED14B@kodewerk.com>
	<CABzyjymhn1DXaCfZ35zGC2tVah2SiMk1W-mbgvhYVjBY38se8w@mail.gmail.com>
Message-ID: <CABzyjyknwSz-WwZyhmse=pEmNjziyNPqk-AvM=mNB_Q0icf=1Q@mail.gmail.com>

I've attached a webrev/patch to the ticket. Happy to offer it here for code
review if that's a better route.

thanks!
-- ramki

On Tue, Aug 25, 2015 at 12:56 PM, Srinivas Ramakrishna <ysr1729 at gmail.com>
wrote:

> -bcc/hotspot-dev
> +cc/hotspot-gc-dev
>
> I have a patch for JDK-8133818 (on JDK 8): as suggested here earlier,
> besides splitting out Cleaners, it also reports JNI wk refs by counting
> them on-the-fly much like for other Reference types.
>
> I'll attach it to the ticket once my testing completes.
>
> thanks.
> -- ramki
>
> On Wed, Aug 19, 2015 at 3:20 PM, Kirk Pepperdine <kirk at kodewerk.com>
> wrote:
>
>> Hi Bengt,
>>
>> Thanks for filing. I really didn?t consider this a bug but it seems
>> reasonable to add clarity in the log.
>>
>> ? Kirk
>>
>> > On Aug 18, 2015, at 8:39 AM, Bengt Rutisson <bengt.rutisson at oracle.com>
>> wrote:
>> >
>> >
>> >
>> > On 2015-08-18 12:47, Bengt Rutisson wrote:
>> >>
>> >> Hi Kirk,
>> >>
>> >> On 2015-08-18 12:31, Kirk Pepperdine wrote:
>> >>> Hi all,
>> >>>
>> >>> I found this.. in a few logs. Before I started trolling the source
>> trying to sort out what the second PhantomReferences count was, I pinged
>> Charlie and asked him. He suggested it was a (concurrency??) bug witnessed
>> only once by someone else. I?ve now seen it in several logs where the
>> numbers reported are different. Sorry for being lazy for not trolling the
>> code before asking but is this a bug?
>> >>
>> >> The second ref count is for cleaner references. They used to be
>> treated the same way as phantom references, but a recent improvement was to
>> separate the handling of the cleaners to after all phantom references have
>> been handled. So, the first count is the "real" phantom references and the
>> second count is the cleaner references.
>> >>
>> >> It's confusing logging, I agree.
>> >>
>> >> This came in with this fix:
>> >> http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/2194fdf3fbd9
>> >>
>> >> since this adds one more call to preclean_discovered_reflist(), which
>> is the method that prints the refs count.
>> >
>> > Thomas was nice and created a bug report for this :)
>> >
>> > Additional number of processed references printed with
>> -XX:+PrintReferenceGC after JDK-8047125
>> > https://bugs.openjdk.java.net/browse/JDK-8133818
>> >
>> > Bengt
>> >
>> >>
>> >> Thanks,
>> >> Bengt
>> >>
>> >>>
>> >>> Regards,
>> >>> Kirk
>> >>>
>> >>> Java HotSpot(TM) 64-Bit Server VM (25.51-b03) for linux-amd64 JRE
>> (1.8.0_51-b16), built on Jun  8 2015 19:28:07 by "java_re" with gcc 4.3.0
>> 20080428 (Red Hat 4.3.0-8)
>> >>> Memory: 4k page, physical 9570132k(9340292k free), swap
>> 4095996k(4095996k free)
>> >>> CommandLine flags: -XX:+AggressiveOpts -XX:G1HeapRegionSize=16777216
>> -XX:InitialHeapSize=3221225472 -XX:InitiatingHeapOccupancyPercent=75
>> -XX:+ManagementServer -XX:MaxGCPauseMillis=250 -XX:MaxHeapSize=3221225472
>> -XX:+ParallelRefProcEnabled -XX:+PerfDisableSharedMem
>> -XX:+PrintAdaptiveSizePolicy -XX:+PrintGC
>> -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCDateStamps
>> -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintReferenceGC
>> -XX:+PrintTenuringDistribution -XX:+UseCompressedClassPointers
>> -XX:+UseCompressedOops -XX:+UseG1GC -XX:+UseLargePages
>> >>>  0.198: [G1Ergonomics (Heap Sizing) expand the heap, requested
>> expansion amount: 3221225472 bytes, attempted expansion amount: 3221225472
>> bytes]
>> >>> 2015-08-11T08:23:02.219+0200: 7.630: [GC pause (G1 Evacuation Pause)
>> (young)
>> >>> Desired survivor size 16777216 bytes, new threshold 15 (max 15)
>> >>>  7.631: [G1Ergonomics (CSet Construction) start choosing CSet,
>> _pending_cards: 0, predicted base time: 10.00 ms, remaining time: 240.00
>> ms, target pause time: 250.00 ms]
>> >>>  7.631: [G1Ergonomics (CSet Construction) add young regions to CSet,
>> eden: 9 regions, survivors: 0 regions, predicted young region time: 3626.58
>> ms]
>> >>>  7.631: [G1Ergonomics (CSet Construction) finish choosing CSet, eden:
>> 9 regions, survivors: 0 regions, old: 0 regions, predicted pause time:
>> 3636.58 ms, target pause time: 250.00 ms]
>> >>> 7.642: [SoftReference, 0 refs, 0.0006965 secs]7.643: [WeakReference,
>> 40 refs, 0.0003039 secs]7.643: [FinalReference, 3477 refs, 0.0036191
>> secs]7.647: [PhantomReference, 0 refs, 0 refs, 0.0006649 secs]7.647: [JNI
>> Weak Reference, 0.0000119 secs], 0.0173298 secs]
>> >>
>> >
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150826/2ed3d06f/attachment.htm>

From erik.helin at oracle.com  Thu Aug 27 07:32:42 2015
From: erik.helin at oracle.com (Erik Helin)
Date: Thu, 27 Aug 2015 09:32:42 +0200
Subject: RFR: 8134504: Remove usage of EvacuationInfo from
	G1CollectorPolicy
In-Reply-To: <55DDE900.3080302@oracle.com>
References: <20150826135624.GD10460@ehelin.jrpg.bea.com>
	<55DDE900.3080302@oracle.com>
Message-ID: <20150827073242.GH10460@ehelin.jrpg.bea.com>

On 2015-08-26, Jesper Wilhelmsson wrote:
> Looks good!

Thanks!
Erik

> /Jesper
> 
> Den 26/8/15 kl. 15:56, skrev Erik Helin:
> >Hi all,
> >
> >this tiny patch cleans up the relationship between the classes
> >EvacuationInfo and G1CollectorPolicy. There is no need for the
> >G1CollectorPolicy to take EvacuationInfo objects as arguments in two
> >methods just to fill in some data, it just complicates the signature of
> >those methods. With the patch the data is retrieved from the
> >G1CollectorPolicy using getters instead.
> >
> >Webrev:
> >http://cr.openjdk.java.net/~ehelin/8134504/webrev.00/
> >
> >Bug:
> >https://bugs.openjdk.java.net/browse/JDK-8134504
> >
> >Testing:
> >JPRT
> >
> >Thanks,
> >Erik
> >


From erik.helin at oracle.com  Thu Aug 27 07:32:58 2015
From: erik.helin at oracle.com (Erik Helin)
Date: Thu, 27 Aug 2015 09:32:58 +0200
Subject: RFR: 8134504: Remove usage of EvacuationInfo from
	G1CollectorPolicy
In-Reply-To: <55DDCB26.2080703@oracle.com>
References: <20150826135624.GD10460@ehelin.jrpg.bea.com>
	<55DDCB26.2080703@oracle.com>
Message-ID: <20150827073258.GI10460@ehelin.jrpg.bea.com>

On 2015-08-26, Mikael Gerdin wrote:
> Hi Erik,
> 
> On 2015-08-26 15:56, Erik Helin wrote:
> >Hi all,
> >
> >this tiny patch cleans up the relationship between the classes
> >EvacuationInfo and G1CollectorPolicy. There is no need for the
> >G1CollectorPolicy to take EvacuationInfo objects as arguments in two
> >methods just to fill in some data, it just complicates the signature of
> >those methods. With the patch the data is retrieved from the
> >G1CollectorPolicy using getters instead.
> >
> >Webrev:
> >http://cr.openjdk.java.net/~ehelin/8134504/webrev.00/
> 
> Looks good to me.

Thanks for reviewing!
Erik

> /Mikael
> 
> >
> >Bug:
> >https://bugs.openjdk.java.net/browse/JDK-8134504
> >
> >Testing:
> >JPRT
> >
> >Thanks,
> >Erik
> >
> 


From erik.helin at oracle.com  Thu Aug 27 07:33:14 2015
From: erik.helin at oracle.com (Erik Helin)
Date: Thu, 27 Aug 2015 09:33:14 +0200
Subject: RFR: 8134509: G1ParCopyClosure does not need a ReferenceProcessor
In-Reply-To: <55DDCC5B.3030003@oracle.com>
References: <20150826141900.GE10460@ehelin.jrpg.bea.com>
	<55DDCC5B.3030003@oracle.com>
Message-ID: <20150827073314.GJ10460@ehelin.jrpg.bea.com>

On 2015-08-26, Stefan Karlsson wrote:
> On 2015-08-26 16:19, Erik Helin wrote:
> >Hi all,
> >
> >this small patch removes an unused argument to the constructor of
> >G1ParCopyClosure. The argument ReferenceProcessor* rp is never used and
> >can therefore be removed, the getter method rp() can therefore also be
> >removed.
> >
> >Webrev:
> >http://cr.openjdk.java.net/~ehelin/8134509/webrev.00/
> 
> Looks good.

Thanks!
Erik

> StefanK
> 
> >
> >Bug:
> >https://bugs.openjdk.java.net/browse/JDK-8134509
> >
> >Testing:
> >JPRT
> >
> >Thanks,
> >Erik
> 


From erik.helin at oracle.com  Thu Aug 27 07:33:30 2015
From: erik.helin at oracle.com (Erik Helin)
Date: Thu, 27 Aug 2015 09:33:30 +0200
Subject: RFR: 8134509: G1ParCopyClosure does not need a ReferenceProcessor
In-Reply-To: <55DDCB4F.3080000@oracle.com>
References: <20150826141900.GE10460@ehelin.jrpg.bea.com>
	<55DDCB4F.3080000@oracle.com>
Message-ID: <20150827073330.GK10460@ehelin.jrpg.bea.com>

On 2015-08-26, Mikael Gerdin wrote:
> Hi Erik,
> 
> On 2015-08-26 16:19, Erik Helin wrote:
> >Hi all,
> >
> >this small patch removes an unused argument to the constructor of
> >G1ParCopyClosure. The argument ReferenceProcessor* rp is never used and
> >can therefore be removed, the getter method rp() can therefore also be
> >removed.
> >
> >Webrev:
> >http://cr.openjdk.java.net/~ehelin/8134509/webrev.00/
> 
> Looks good to me.

Thanks!
Erik

> /Mikael
> 
> >
> >Bug:
> >https://bugs.openjdk.java.net/browse/JDK-8134509
> >
> >Testing:
> >JPRT
> >
> >Thanks,
> >Erik
> >
> 


From tobias.hartmann at oracle.com  Thu Aug 27 13:05:44 2015
From: tobias.hartmann at oracle.com (Tobias Hartmann)
Date: Thu, 27 Aug 2015 15:05:44 +0200
Subject: [9] RFR(S): 8134493: Cleaning inline caches of unloaded nmethods
	should be done in sweeper
In-Reply-To: <7C9B87B351A4BA4AA9EC95BB418116566AD5FAEB@DEWDFEMB19A.global.corp.sap>
References: <55DEB7E0.6040002@oracle.com>
	<7C9B87B351A4BA4AA9EC95BB418116566AD5FAEB@DEWDFEMB19A.global.corp.sap>
Message-ID: <55DF0B28.4000901@oracle.com>

Hi Martin,

thanks for looking at this.

On 27.08.2015 12:26, Doerr, Martin wrote:
> Hi,
> 
> one question about always avoiding transition stubs for unloaded nmethods in cleanup_inline_caches() came into my mind when taking a second look at the change.
> When we transition to zombie state, this should be safe because it's guaranteed that no activations exist.
> However, cleanup_inline_caches() is used at other places, too.
> 
> The comment about the unloaded state only says "// there should be no activations". Is it guaranteed that there is no activation of an nmethod which is in unloaded state?
> If not, the change may have undesired side effects.

I think it is guaranteed that unloaded nmethods are not on the stack and it should be save to clean their ICs without a transition stub. We also rely on this assumption in the sweeper by directly flushing unloaded OSR nmethods (see line 644 in sweeper.cpp [1]).

Maybe someone from the GC team (CC'ed) can clarify this.

Thanks,
Tobias

[1] http://cr.openjdk.java.net/~thartmann/8134493/webrev.00/src/share/vm/runtime/sweeper.cpp.html

> 
> Besides this, the change looks good to me.
> 
> Best regards,
> Martin
> 
> 
> -----Original Message-----
> From: Tobias Hartmann [mailto:tobias.hartmann at oracle.com] 
> Sent: Donnerstag, 27. August 2015 09:10
> To: hotspot-compiler-dev at openjdk.java.net
> Cc: Doerr, Martin
> Subject: [9] RFR(S): 8134493: Cleaning inline caches of unloaded nmethods should be done in sweeper
> 
> Hi,
> 
> please review the following patch.
> 
> https://bugs.openjdk.java.net/browse/JDK-8134493
> http://cr.openjdk.java.net/~thartmann/8134493/webrev.00/
> 
> Problem:
> This is a follow up on JDK-8075805 [1] which modified CodeCache::gc_epilogue() to clean the ICs of unloaded nmethods as well. The problem is that this code is executed at a safepoint and may affect safepoint duration. The other changes of JDK-8134493 are fine.
> 
> Solution:
> We do the cleaning of unloaded nmethods at the unloaded -> zombie transition in the sweeper. I also modified nmethod::cleanup_inline_caches() to not emit any transition stubs if the nmethod is already dead.
> 
> As Martin Doerr pointed out in another thread, we have to be careful with accessing CompiledIC::cached_metadata() of unloaded nmethods. For example, the following scenario may happen (IC of A references B):
> 
>     state of A       state of B
> -------------------------------
>     not-entrant
> S   [not-on-stack]      
> S   zombie           
> 	             unloaded
> 
> Now the IC of A still references the unloaded nmethod B and is_call_to_compiled() will access the unloaded metadata. I fixed this by checking caller->is_alive().
> 
> Thanks,
> Tobias
> 
> [1] https://bugs.openjdk.java.net/browse/JDK-8075805
> 
> 


From eric.caspole at oracle.com  Thu Aug 27 13:53:37 2015
From: eric.caspole at oracle.com (Eric Caspole)
Date: Thu, 27 Aug 2015 09:53:37 -0400
Subject: RFR (S): 8067341: Modify PLAB sizing algorithm to waste less
In-Reply-To: <1440592142.2315.30.camel@oracle.com>
References: <1440518817.2210.21.camel@oracle.com>	<55DCF5AA.6030604@oracle.com>
	<1440582897.2315.5.camel@oracle.com>
	<1440592142.2315.30.camel@oracle.com>
Message-ID: <55DF1661.4000902@oracle.com>

Hi Thomas,
Maybe you already got reviews for v2 but I noticed some typos:

In this part it should be the G1LastPLABAverageOccupancy, right?

   57     // Calculate the new PLAB size as the amount of space that 
could be wasted to
   58     // keep TargetPLABWastePct given latest memory usage and that 
the last buffer
   59     // will be G1PLABPercent full.


Mangled sentence:

   68     // The amount of allocation performed is we can spend is 
independent of the

Otherwise looks good.
Thanks,
Eric



On 08/26/2015 08:29 AM, Thomas Schatzl wrote:
> Hi all,
>
>    I tried to get some more internal comments on the question of the
> final change to improve PLAB sizes.
>
> On Wed, 2015-08-26 at 11:54 +0200, Thomas Schatzl wrote:
>> Hi Jon (and Eric),
> [...]
>>> When _region_end_waste is high, waste_used_for_calculation under
>>> estimates the
>>> amount used so under estimates the amount of waste (since the waste
>>> is a percentage of waste_used_for_calculation ) and so lowers the size
>>> of the next PLAB.  I can see that it scales reasonably.
>>>
>>> Why G1ExpectedAveragePLABOccupancyPercent that can be different than 50?
>>>
>>> If it is so that the user can scale the PLAB's up or down, what about
>>
>> That is the intent.
>>
>>> scale factor called G1PLABPercent?
>>
>> That name seems to me a bit too unspecific, I chose
>> G1PLABOccupancyFactor for this revision. Feel free to comment on this.
>
> Others commented on the name and the value range; mainly that a
> percentage from 1-100 (or something similar) seems more natural or at
> least common, also the use as a divisor in the calculation.
>
> Using a double instead of an integer allows more flexibility in the
> choice of values.
>
> As for the name, the argument came up that the G1PLABPercent suggestion
> is somewhat too undescriptive, and it is better to use a longer, more
> appropriate name like G1LastPLABAverageOccupancy (note that added "Last"
> in the name). Since we do not expect it to be used a lot, except experts
> that measured that there is an issue there previously, a longer name
> does not seem to be a big issue.
>
> http://cr.openjdk.java.net/~tschatzl/8067341/webrev.2
> http://cr.openjdk.java.net/~tschatzl/8067341/webrev.0_to_2 (change from baseline!)
>
> Thanks,
>    Thomas
>


From kim.barrett at oracle.com  Thu Aug 27 17:51:00 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Thu, 27 Aug 2015 13:51:00 -0400
Subject: RFR: 8072817: CardTableExtension kind() should be
	BarrierSet::CardTableExtension
In-Reply-To: <DA95D846-B071-45B7-81C1-4DD4C9DC0D92@oracle.com>
References: <DA95D846-B071-45B7-81C1-4DD4C9DC0D92@oracle.com>
Message-ID: <565A2547-9AE9-49C1-B14B-9B4160F3361A@oracle.com>

Still looking for reviews.

On Aug 19, 2015, at 1:42 PM, Kim Barrett <kim.barrett at oracle.com> wrote:
> 
> Please review this change to finally properly integrate the
> BarrierSet::CardTableExtension barrier set kind. This has been
> deferred for a long time; there was, until recently, a comment in
> CardTableExtension that having kind() return
> BarrierSet::CardTableExtension was "too risky for the 4/10/02
> putback".
> 
> Much of the work involved in supporting the CardTableExtension tag had
> been done long ago.  The remaining work involved actually hooking it
> up.  The key missing step was to give CardTableModRefBSForCTRS its own
> tag (to distinguish from CardTableExtension) and switch to using that
> tag where needed.  The needed tag (BarrierSet::CardTableForRS) was
> defined in the fix for JDK-8069016.  This change makes use of it.
> 
> This involved the following steps:
> 
> 1. Most occurrences of BarrierSet::CardTableModRef are now
> BarrierSet::CardTableForRS. Occurrences of BarrierSet::CardTableForRS
> and BarrierSet::CardTableExtension are now nearly always paired.
> Exceptions are in the constructors of the corresponding classes and in
> the associated barrier_set_cast support.
> 
> 2. The tests for kind() == BarrierSet::CardTableModRef that were
> controlling the devirtualization of write_ref_field_pre,
> write_ref_field, and write_region now test for either
> BarrierSet::CardTableForRS or BarrierSet::CardTableExtension.  The
> FakeRttiSupport class was extended to improve the new test.
> 
> 3. The remaining small number of tests for kind() ==
> BarrierSet::CardTableModRef are now tests for
> is_a(BarrierSet::CardTableModRef). These are places where we really do
> allow a CardTableModRefBS and don't care which subclass; all of these
> involve access to the card table. This now includes G1 barrier sets in
> a couple of places where they were previously inappropriately or
> unnecessarily excluded.
> 
> 4. The CardTableModRefBSForCTRS class now specifies its kind tag be
> BarrierSet::CardTableForRS. The CardTableExtension class now specifies
> its kind tag be BarrierSet::CardTableExtension.  The extra
> FakeRttiSupport constructor that existed to support the old kludge for
> those classes has been removed.
> 
> Also removed unused CollectorPolicy::barrier_set_name.
> 
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8072817
> 
> Webrev:
> http://cr.openjdk.java.net/~kbarrett/8072817/webrev.00/
> 
> Testing:
> jprt
> aurora ad hoc GC/Runtime Nightly, quicktests, tonga, jtreg




From jesper.wilhelmsson at oracle.com  Thu Aug 27 18:48:15 2015
From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson)
Date: Thu, 27 Aug 2015 20:48:15 +0200
Subject: RFR: 8072817: CardTableExtension kind() should be
	BarrierSet::CardTableExtension
In-Reply-To: <565A2547-9AE9-49C1-B14B-9B4160F3361A@oracle.com>
References: <DA95D846-B071-45B7-81C1-4DD4C9DC0D92@oracle.com>
	<565A2547-9AE9-49C1-B14B-9B4160F3361A@oracle.com>
Message-ID: <55DF5B6F.1050000@oracle.com>

Hi Kim,

Looks good in general.

In src/cpu/aarch64/vm/macroAssembler_aarch64.cpp there is a comma between the 
two checks:

+  assert(bs->kind() == BarrierSet::CardTableForRS,
+         bs->kind() == BarrierSet::CardTableExtension,
+         "Wrong barrier set kind");

I assume there should be a ||

Besides that the patch seems to do what you described below.

Thanks,
/Jesper


Den 27/8/15 kl. 19:51, skrev Kim Barrett:
> Still looking for reviews.
>
> On Aug 19, 2015, at 1:42 PM, Kim Barrett <kim.barrett at oracle.com> wrote:
>>
>> Please review this change to finally properly integrate the
>> BarrierSet::CardTableExtension barrier set kind. This has been
>> deferred for a long time; there was, until recently, a comment in
>> CardTableExtension that having kind() return
>> BarrierSet::CardTableExtension was "too risky for the 4/10/02
>> putback".
>>
>> Much of the work involved in supporting the CardTableExtension tag had
>> been done long ago.  The remaining work involved actually hooking it
>> up.  The key missing step was to give CardTableModRefBSForCTRS its own
>> tag (to distinguish from CardTableExtension) and switch to using that
>> tag where needed.  The needed tag (BarrierSet::CardTableForRS) was
>> defined in the fix for JDK-8069016.  This change makes use of it.
>>
>> This involved the following steps:
>>
>> 1. Most occurrences of BarrierSet::CardTableModRef are now
>> BarrierSet::CardTableForRS. Occurrences of BarrierSet::CardTableForRS
>> and BarrierSet::CardTableExtension are now nearly always paired.
>> Exceptions are in the constructors of the corresponding classes and in
>> the associated barrier_set_cast support.
>>
>> 2. The tests for kind() == BarrierSet::CardTableModRef that were
>> controlling the devirtualization of write_ref_field_pre,
>> write_ref_field, and write_region now test for either
>> BarrierSet::CardTableForRS or BarrierSet::CardTableExtension.  The
>> FakeRttiSupport class was extended to improve the new test.
>>
>> 3. The remaining small number of tests for kind() ==
>> BarrierSet::CardTableModRef are now tests for
>> is_a(BarrierSet::CardTableModRef). These are places where we really do
>> allow a CardTableModRefBS and don't care which subclass; all of these
>> involve access to the card table. This now includes G1 barrier sets in
>> a couple of places where they were previously inappropriately or
>> unnecessarily excluded.
>>
>> 4. The CardTableModRefBSForCTRS class now specifies its kind tag be
>> BarrierSet::CardTableForRS. The CardTableExtension class now specifies
>> its kind tag be BarrierSet::CardTableExtension.  The extra
>> FakeRttiSupport constructor that existed to support the old kludge for
>> those classes has been removed.
>>
>> Also removed unused CollectorPolicy::barrier_set_name.
>>
>> CR:
>> https://bugs.openjdk.java.net/browse/JDK-8072817
>>
>> Webrev:
>> http://cr.openjdk.java.net/~kbarrett/8072817/webrev.00/
>>
>> Testing:
>> jprt
>> aurora ad hoc GC/Runtime Nightly, quicktests, tonga, jtreg
>
>


From eric.caspole at oracle.com  Thu Aug 27 19:42:50 2015
From: eric.caspole at oracle.com (Eric Caspole)
Date: Thu, 27 Aug 2015 15:42:50 -0400
Subject: RFR: 8131330: G1CollectedHeap::verify_dirty_young_list fails with
	assert
In-Reply-To: <B8713224-3063-4BDB-9FAA-7ECB5F9AF0DA@oracle.com>
References: <5D7EB2C6-6AC4-4B87-8FD1-FC487328087C@oracle.com>
	<55D4C22F.2000003@oracle.com>
	<08169C1E-696E-44C1-AA98-3F9AA2895197@oracle.com>
	<AEE51CEF-0B38-4A43-BB8D-F51FEC74B9B8@oracle.com>
	<B8713224-3063-4BDB-9FAA-7ECB5F9AF0DA@oracle.com>
Message-ID: <55DF683A.1070402@oracle.com>

Hi Kim,
Looks good.
Eric


On 8/20/2015 4:29 PM, Kim Barrett wrote:
> On Aug 20, 2015, at 11:00 AM, Kim Barrett <kim.barrett at oracle.com> wrote:
>> So why isn't the fill_subword helper function declared inline, you
>> might ask?
>>
>> Because during development it made sense to have it not inlined, and I
>> forgot to reconsider that later.  Sigh!  I'll fix that and post a new
>> webrev this afternoon.
> Here's the new version.  I moved the fill_subword helper before its
> caller and changed it to inline rather than static.
>
> That change provoked a compiler warning that reminded me that
> temporaries for inline assembler should be declared in the outputs and
> then just left unreferenced after the asm statement.  I'd mistakenly
> had them in the inputs.  Fixed now.
>
> Full webrev:
> http://cr.openjdk.java.net/~kbarrett/8131330/webrev.02/
>
> Incremental webrev:
> http://cr.openjdk.java.net/~kbarrett/8131330/webrev.02.inc2/
>



From jon.masamitsu at oracle.com  Thu Aug 27 19:53:35 2015
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Thu, 27 Aug 2015 12:53:35 -0700
Subject: RFR: 8072817: CardTableExtension kind() should be
	BarrierSet::CardTableExtension
In-Reply-To: <55DF5B6F.1050000@oracle.com>
References: <DA95D846-B071-45B7-81C1-4DD4C9DC0D92@oracle.com>	<565A2547-9AE9-49C1-B14B-9B4160F3361A@oracle.com>
	<55DF5B6F.1050000@oracle.com>
Message-ID: <55DF6ABF.9030104@oracle.com>

Kim,

Looks good assuming, as Jesper suggests, you meant
an ||.

Jon


On 08/27/2015 11:48 AM, Jesper Wilhelmsson wrote:
> Hi Kim,
>
> Looks good in general.
>
> In src/cpu/aarch64/vm/macroAssembler_aarch64.cpp there is a comma 
> between the two checks:
>
> +  assert(bs->kind() == BarrierSet::CardTableForRS,
> +         bs->kind() == BarrierSet::CardTableExtension,
> +         "Wrong barrier set kind");
>
> I assume there should be a ||
>
> Besides that the patch seems to do what you described below.
>
> Thanks,
> /Jesper
>
>
> Den 27/8/15 kl. 19:51, skrev Kim Barrett:
>> Still looking for reviews.
>>
>> On Aug 19, 2015, at 1:42 PM, Kim Barrett <kim.barrett at oracle.com> wrote:
>>>
>>> Please review this change to finally properly integrate the
>>> BarrierSet::CardTableExtension barrier set kind. This has been
>>> deferred for a long time; there was, until recently, a comment in
>>> CardTableExtension that having kind() return
>>> BarrierSet::CardTableExtension was "too risky for the 4/10/02
>>> putback".
>>>
>>> Much of the work involved in supporting the CardTableExtension tag had
>>> been done long ago.  The remaining work involved actually hooking it
>>> up.  The key missing step was to give CardTableModRefBSForCTRS its own
>>> tag (to distinguish from CardTableExtension) and switch to using that
>>> tag where needed.  The needed tag (BarrierSet::CardTableForRS) was
>>> defined in the fix for JDK-8069016.  This change makes use of it.
>>>
>>> This involved the following steps:
>>>
>>> 1. Most occurrences of BarrierSet::CardTableModRef are now
>>> BarrierSet::CardTableForRS. Occurrences of BarrierSet::CardTableForRS
>>> and BarrierSet::CardTableExtension are now nearly always paired.
>>> Exceptions are in the constructors of the corresponding classes and in
>>> the associated barrier_set_cast support.
>>>
>>> 2. The tests for kind() == BarrierSet::CardTableModRef that were
>>> controlling the devirtualization of write_ref_field_pre,
>>> write_ref_field, and write_region now test for either
>>> BarrierSet::CardTableForRS or BarrierSet::CardTableExtension. The
>>> FakeRttiSupport class was extended to improve the new test.
>>>
>>> 3. The remaining small number of tests for kind() ==
>>> BarrierSet::CardTableModRef are now tests for
>>> is_a(BarrierSet::CardTableModRef). These are places where we really do
>>> allow a CardTableModRefBS and don't care which subclass; all of these
>>> involve access to the card table. This now includes G1 barrier sets in
>>> a couple of places where they were previously inappropriately or
>>> unnecessarily excluded.
>>>
>>> 4. The CardTableModRefBSForCTRS class now specifies its kind tag be
>>> BarrierSet::CardTableForRS. The CardTableExtension class now specifies
>>> its kind tag be BarrierSet::CardTableExtension.  The extra
>>> FakeRttiSupport constructor that existed to support the old kludge for
>>> those classes has been removed.
>>>
>>> Also removed unused CollectorPolicy::barrier_set_name.
>>>
>>> CR:
>>> https://bugs.openjdk.java.net/browse/JDK-8072817
>>>
>>> Webrev:
>>> http://cr.openjdk.java.net/~kbarrett/8072817/webrev.00/
>>>
>>> Testing:
>>> jprt
>>> aurora ad hoc GC/Runtime Nightly, quicktests, tonga, jtreg
>>
>>



From kim.barrett at oracle.com  Thu Aug 27 20:45:57 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Thu, 27 Aug 2015 16:45:57 -0400
Subject: RFR: 8072817: CardTableExtension kind() should be
	BarrierSet::CardTableExtension
In-Reply-To: <55DF5B6F.1050000@oracle.com>
References: <DA95D846-B071-45B7-81C1-4DD4C9DC0D92@oracle.com>
	<565A2547-9AE9-49C1-B14B-9B4160F3361A@oracle.com>
	<55DF5B6F.1050000@oracle.com>
Message-ID: <9D063A21-B906-44DA-816F-7B500B590158@oracle.com>

On Aug 27, 2015, at 2:48 PM, Jesper Wilhelmsson <jesper.wilhelmsson at oracle.com> wrote:
> 
> Hi Kim,
> 
> Looks good in general.
> 
> In src/cpu/aarch64/vm/macroAssembler_aarch64.cpp there is a comma between the two checks:
> 
> +  assert(bs->kind() == BarrierSet::CardTableForRS,
> +         bs->kind() == BarrierSet::CardTableExtension,
> +         "Wrong barrier set kind");
> 
> I assume there should be a ||
> 
> Besides that the patch seems to do what you described below.

You are correct.  Unfortunately, jprt doesn?t test the platform that uses that file.



From kim.barrett at oracle.com  Thu Aug 27 21:09:44 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Thu, 27 Aug 2015 17:09:44 -0400
Subject: RFR: 8072817: CardTableExtension kind() should be
	BarrierSet::CardTableExtension
In-Reply-To: <55DF6ABF.9030104@oracle.com>
References: <DA95D846-B071-45B7-81C1-4DD4C9DC0D92@oracle.com>	<565A2547-9AE9-49C1-B14B-9B4160F3361A@oracle.com>
	<55DF5B6F.1050000@oracle.com> <55DF6ABF.9030104@oracle.com>
Message-ID: <FBABD689-0606-466E-9804-A89428A5DE58@oracle.com>

On Aug 27, 2015, at 3:53 PM, Jon Masamitsu <jon.masamitsu at oracle.com> wrote:
> 
> Kim,
> 
> Looks good assuming, as Jesper suggests, you meant
> an ||.

Thanks.  I?ll fix that.



From jesper.wilhelmsson at oracle.com  Fri Aug 28 14:37:59 2015
From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson)
Date: Fri, 28 Aug 2015 16:37:59 +0200
Subject: RFR: 8134626 - Misc cleanups after generation array removal
Message-ID: <55E07247.8020304@oracle.com>

Hi,

Please review this rather large cleanup. While working with the generation array 
removal I did a bunch of unrelated cleanups in the code I passed by. I split 
this out into a separate patch to keep the other changes clean.

This change is split into three webrevs to make it easier to review. I intend to 
push all three as one change.

Bug: https://bugs.openjdk.java.net/browse/JDK-8134626


Increment 1: Renaming
http://cr.openjdk.java.net/~jwilhelm/8134626/webrev.00/inc1-renaming/

Changes the variable name gen to young_gen or old_gen depending on what the 
variable will contain. The few cases where the name gen is untouched are places 
where the variable actually can contain either generation.


Increment 2: Comments and indentation
http://cr.openjdk.java.net/~jwilhelm/8134626/webrev.00/inc2-comments_and_indentation/

* Fixes up alignment in lots of places.
* Inserts and removes empty lines where needed.
* Cleans up comments where we previously talked about older and younger 
generations. Fixes a few typos and clarifies some comments with respect to 
young/old generations and collections.
* Added spaces around some operators.


Increment 3: Code changes
http://cr.openjdk.java.net/~jwilhelm/8134626/webrev.00/inc3-code_changes/

* Merged strings that was split on several lines for no good reason.
* Added braces for if statements and for loops.
* Removed dead code.
* Moved variable initialization to initializer list in CollectedHeap constructor.
* Updated flag descriptions "youngest" -> "young".

Thanks,
/Jesper



From thomas.schatzl at oracle.com  Mon Aug 31 07:59:10 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 31 Aug 2015 09:59:10 +0200
Subject: RFR: 8131330: G1CollectedHeap::verify_dirty_young_list fails
	with assert
In-Reply-To: <55DF683A.1070402@oracle.com>
References: <5D7EB2C6-6AC4-4B87-8FD1-FC487328087C@oracle.com>
	<55D4C22F.2000003@oracle.com>
	<08169C1E-696E-44C1-AA98-3F9AA2895197@oracle.com>
	<AEE51CEF-0B38-4A43-BB8D-F51FEC74B9B8@oracle.com>
	<B8713224-3063-4BDB-9FAA-7ECB5F9AF0DA@oracle.com>
	<55DF683A.1070402@oracle.com>
Message-ID: <1441007950.2740.8.camel@oracle.com>

Hi,

On Thu, 2015-08-27 at 15:42 -0400, Eric Caspole wrote:
> Hi Kim,
> Looks good.
> Eric
> 
> 
> On 8/20/2015 4:29 PM, Kim Barrett wrote:
> > On Aug 20, 2015, at 11:00 AM, Kim Barrett <kim.barrett at oracle.com> wrote:
> >> So why isn't the fill_subword helper function declared inline, you
> >> might ask?
> >>
> >> Because during development it made sense to have it not inlined, and I
> >> forgot to reconsider that later.  Sigh!  I'll fix that and post a new
> >> webrev this afternoon.
> > Here's the new version.  I moved the fill_subword helper before its
> > caller and changed it to inline rather than static.
> >
> > That change provoked a compiler warning that reminded me that
> > temporaries for inline assembler should be declared in the outputs and
> > then just left unreferenced after the asm statement.  I'd mistakenly
> > had them in the inputs.  Fixed now.
> >
> > Full webrev:
> > http://cr.openjdk.java.net/~kbarrett/8131330/webrev.02/
> >
> > Incremental webrev:
> > http://cr.openjdk.java.net/~kbarrett/8131330/webrev.02.inc2/
> >

  looks good.

Thomas




From thomas.schatzl at oracle.com  Mon Aug 31 08:40:44 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 31 Aug 2015 10:40:44 +0200
Subject: Request for code review - 8130265:
	gctests/LargeObjects/large001 fails with OutOfMemoryError: Java heap
	space
In-Reply-To: <55DE531D.9090703@oracle.com>
References: <55D356FB.1030603@oracle.com> <55D35A12.8020608@oracle.com>
	<55D35F1F.3090700@oracle.com> <55D36DC7.6010901@oracle.com>
	<55D38CBE.8070408@oracle.com> <55D394AD.6020104@oracle.com>
	<55D4E39D.5070805@oracle.com> <55D4E8C7.1080703@oracle.com>
	<55D4EEF5.9030504@oracle.com>
	<2C141666-BDBC-428C-9E40-4FB78F076192@oracle.com>
	<1440082894.2347.14.camel@oracle.com> <55D62D77.8020504@oracle.com>
	<55D71703.4070505@oracle.com> <55D74766.5020202@oracle.com>
	<1440420922.2348.82.camel@oracle.com> <55DB8B7E.5010601@oracle.com>
	<55DC6CB5.4010108@oracle.com> <1440584814.2315.16.camel@oracle.com>
	<55DE531D.9090703@oracle.com>
Message-ID: <1441010444.2740.18.camel@oracle.com>

Hi,

On Wed, 2015-08-26 at 20:00 -0400, Alexander Harlap wrote:
> 
> On 8/26/2015 6:26 AM, Thomas Schatzl wrote:
> > Hi,
> >
> >    looking at webrev.05, but answering here.
> >
> > On Tue, 2015-08-25 at 15:25 +0200, Jesper Wilhelmsson wrote:
> >> Hi Alex,
> >>
> >> I like this approach :)
> >>
> >> Just one comment:
> >>
> >> After GC in the helper you put the assert that soft references should have been
> >> handled. The first call to this helper will not clear soft references. The
> >> original code only has this assert at the end of the function. I think it is
> > Whatever the original code did, there is the general requirement that
> > after any full gc, collector_policy()->should_clear_all_soft_refs()
> > should be false.
> >   
> >> enough to keep it there, at the end of satisfy_failed_allocation().
> > It seems to be randomly placed now, checking only after everything else
> > fails. I do agree that the original code did the same, but why not
> > improve upon this?
> >
> > If you want in a separate CR.
> Yes, I would agree to make a separate CR. Only I will need help to 
> specify issue.

Something like "Misplaced check for
CollectedPolicy::should_clear_all_soft_refs() in G1" should be fine.

> >
> >> 1764     assert(!collector_policy()->should_clear_all_soft_refs(),
> >> 1765            "Flag should have been handled and cleared prior to this point");
> >>
> >> Besides that I'm fine with this version.
> > - Some braces next to boolean parameters are not aligned properly. E.g.
> > in line 1777 and 1804.
> I am sorry, I can not locate those defects - did you mean lines 1777 and 
> 1804 in webrev.05?

In g1CollectedHeap.cpp, line 1777 and 1804 (sorry for not mentioning the
file):

1774   HeapWord* result =
1775     satisfy_failed_allocation_helper(word_size,
1776                                      context,
1777                                      true, /* do_gc */
1778                                      false, /* clear_all_soft_refs */
1779                                      false, /* expect_null_mutator_alloc_region */
1780                                      succeeded);
1781 

The opening comment bracket in line 1777 is not aligned to the ones
below afaik. Same in 1804.

[...]
> >>>>
> >>>> (And the bool return value of do_collection removed).
> >>>>
> >>>> I also have a question about the order of
> >>>> attempt_allocation_at_safepoint() and the expand_and_allocate(), and
> >>>> could not find this answered before: so full gc might shrink the heap
> >>>> too much after full gc, and we should expand the heap.
> >>>> If I were to do this change, there would always (unconditionally) be an
> >>>> expansion after full gc, because the current change only expands if the
> >>>> allocation after gc fails. Also the 02 change.
> >>>>
> >>>> The review thread so far suggests that full gc might shrink too much,
> >>>> but this still seems to be the case here in some cases.
>>
> > Could you clarify this question? To me, this looks like a potential bug
> > that has been overlooked so far in this change.
>
> Yes, full GC in some situation may over-shrink heap such way that heap 
> expansion will be needed. It was reason for 
> gctests/LargeObjects/large001 failure.  Still I prefer to do expansion 
> only after allocation attempt failed. Why should we do unconditional 
> expansion ( even if it is not needed )? 

To keep the Min/MaxHeapFreeRatio constraints the heap should observe.

>  In the existing code we have 
> call to expand_and_allocate() only after 
> attempt_allocation_at_safepoint() failure. What potential bug do you mean?

That the Min/MaxHeapFreeRatio values are not observed, causing
performance degradation, in the worst case like in this situation.

The descriptions of the flags do not indicate an exception like "do not
care if the GC thinks it is not required".

Thanks,
  Thomas




From bengt.rutisson at oracle.com  Mon Aug 31 08:41:03 2015
From: bengt.rutisson at oracle.com (Bengt Rutisson)
Date: Mon, 31 Aug 2015 10:41:03 +0200
Subject: race condition in GC logs
In-Reply-To: <CABzyjyknwSz-WwZyhmse=pEmNjziyNPqk-AvM=mNB_Q0icf=1Q@mail.gmail.com>
References: <9E79C717-7C53-49F5-AD0F-C003A03F11E5@kodewerk.com>	<55D30D50.8060302@oracle.com>	<55D32774.2090903@oracle.com>	<CA61B527-3D6E-46EF-AAAA-425C306ED14B@kodewerk.com>	<CABzyjymhn1DXaCfZ35zGC2tVah2SiMk1W-mbgvhYVjBY38se8w@mail.gmail.com>
	<CABzyjyknwSz-WwZyhmse=pEmNjziyNPqk-AvM=mNB_Q0icf=1Q@mail.gmail.com>
Message-ID: <55E4131F.1090805@oracle.com>


Hi Ramki,

On 2015-08-27 04:16, Srinivas Ramakrishna wrote:
> I've attached a webrev/patch to the ticket. Happy to offer it here for 
> code review if that's a better route.


Thanks for providing the patch! I just had a very quick look at it and I 
think it looks good.

I think you need to publish the webrev on cr.openjdk.java.net and send 
it out for review on this mailing list if we should be able to push it. 
I'm happy to help you if you need any assistance with publishing the 
webrev or so. Just let me know.

Thanks for doing this!
Bengt

>
> thanks!
> -- ramki
>
> On Tue, Aug 25, 2015 at 12:56 PM, Srinivas Ramakrishna 
> <ysr1729 at gmail.com <mailto:ysr1729 at gmail.com>> wrote:
>
>     -bcc/hotspot-dev
>     +cc/hotspot-gc-dev
>
>     I have a patch for JDK-8133818 (on JDK 8): as suggested here
>     earlier, besides splitting out Cleaners, it also reports JNI wk
>     refs by counting them on-the-fly much like for other Reference types.
>
>     I'll attach it to the ticket once my testing completes.
>
>     thanks.
>     -- ramki
>
>     On Wed, Aug 19, 2015 at 3:20 PM, Kirk Pepperdine
>     <kirk at kodewerk.com <mailto:kirk at kodewerk.com>> wrote:
>
>         Hi Bengt,
>
>         Thanks for filing. I really didn?t consider this a bug but it
>         seems reasonable to add clarity in the log.
>
>         ? Kirk
>
>         > On Aug 18, 2015, at 8:39 AM, Bengt Rutisson
>         <bengt.rutisson at oracle.com <mailto:bengt.rutisson at oracle.com>>
>         wrote:
>         >
>         >
>         >
>         > On 2015-08-18 12:47, Bengt Rutisson wrote:
>         >>
>         >> Hi Kirk,
>         >>
>         >> On 2015-08-18 12:31, Kirk Pepperdine wrote:
>         >>> Hi all,
>         >>>
>         >>> I found this.. in a few logs. Before I started trolling
>         the source trying to sort out what the second
>         PhantomReferences count was, I pinged Charlie and asked him.
>         He suggested it was a (concurrency??) bug witnessed only once
>         by someone else. I?ve now seen it in several logs where the
>         numbers reported are different. Sorry for being lazy for not
>         trolling the code before asking but is this a bug?
>         >>
>         >> The second ref count is for cleaner references. They used
>         to be treated the same way as phantom references, but a recent
>         improvement was to separate the handling of the cleaners to
>         after all phantom references have been handled. So, the first
>         count is the "real" phantom references and the second count is
>         the cleaner references.
>         >>
>         >> It's confusing logging, I agree.
>         >>
>         >> This came in with this fix:
>         >> http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/2194fdf3fbd9
>         >>
>         >> since this adds one more call to
>         preclean_discovered_reflist(), which is the method that prints
>         the refs count.
>         >
>         > Thomas was nice and created a bug report for this :)
>         >
>         > Additional number of processed references printed with
>         -XX:+PrintReferenceGC after JDK-8047125
>         > https://bugs.openjdk.java.net/browse/JDK-8133818
>         >
>         > Bengt
>         >
>         >>
>         >> Thanks,
>         >> Bengt
>         >>
>         >>>
>         >>> Regards,
>         >>> Kirk
>         >>>
>         >>> Java HotSpot(TM) 64-Bit Server VM (25.51-b03) for
>         linux-amd64 JRE (1.8.0_51-b16), built on Jun  8 2015 19:28:07
>         by "java_re" with gcc 4.3.0 20080428 (Red Hat 4.3.0-8)
>         >>> Memory: 4k page, physical 9570132k(9340292k free), swap
>         4095996k(4095996k free)
>         >>> CommandLine flags: -XX:+AggressiveOpts
>         -XX:G1HeapRegionSize=16777216 -XX:InitialHeapSize=3221225472
>         -XX:InitiatingHeapOccupancyPercent=75 -XX:+ManagementServer
>         -XX:MaxGCPauseMillis=250 -XX:MaxHeapSize=3221225472
>         -XX:+ParallelRefProcEnabled -XX:+PerfDisableSharedMem
>         -XX:+PrintAdaptiveSizePolicy -XX:+PrintGC
>         -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCDateStamps
>         -XX:+PrintGCDetails -XX:+PrintGCTimeStamps
>         -XX:+PrintReferenceGC -XX:+PrintTenuringDistribution
>         -XX:+UseCompressedClassPointers -XX:+UseCompressedOops
>         -XX:+UseG1GC -XX:+UseLargePages
>         >>>  0.198: [G1Ergonomics (Heap Sizing) expand the heap,
>         requested expansion amount: 3221225472 bytes, attempted
>         expansion amount: 3221225472 bytes]
>         >>> 2015-08-11T08:23:02.219+0200: 7.630: [GC pause (G1
>         Evacuation Pause) (young)
>         >>> Desired survivor size 16777216 bytes, new threshold 15
>         (max 15)
>         >>>  7.631: [G1Ergonomics (CSet Construction) start choosing
>         CSet, _pending_cards: 0, predicted base time: 10.00 ms,
>         remaining time: 240.00 ms, target pause time: 250.00 ms]
>         >>>  7.631: [G1Ergonomics (CSet Construction) add young
>         regions to CSet, eden: 9 regions, survivors: 0 regions,
>         predicted young region time: 3626.58 ms]
>         >>>  7.631: [G1Ergonomics (CSet Construction) finish choosing
>         CSet, eden: 9 regions, survivors: 0 regions, old: 0 regions,
>         predicted pause time: 3636.58 ms, target pause time: 250.00 ms]
>         >>> 7.642: [SoftReference, 0 refs, 0.0006965 secs]7.643:
>         [WeakReference, 40 refs, 0.0003039 secs]7.643:
>         [FinalReference, 3477 refs, 0.0036191 secs]7.647:
>         [PhantomReference, 0 refs, 0 refs, 0.0006649 secs]7.647: [JNI
>         Weak Reference, 0.0000119 secs], 0.0173298 secs]
>         >>
>         >
>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150831/f7470b89/attachment.htm>

From per.liden at oracle.com  Mon Aug 31 10:02:33 2015
From: per.liden at oracle.com (Per Liden)
Date: Mon, 31 Aug 2015 12:02:33 +0200
Subject: RFR(XS): 8134738: Remove CollectorPolicy::Name
Message-ID: <55E42639.5020904@oracle.com>

Hi,

Please review this small cleanup, which removes the unused 
CollectorPolicy::Name enum and the related kind() functions. They are 
not used by the SA either, so it's unaffected.

Webrev: http://cr.openjdk.java.net/~pliden/8134738/webrev.0/

Bug: https://bugs.openjdk.java.net/browse/JDK-8134738

cheers,
/Per



From mikael.gerdin at oracle.com  Mon Aug 31 10:11:12 2015
From: mikael.gerdin at oracle.com (Mikael Gerdin)
Date: Mon, 31 Aug 2015 12:11:12 +0200
Subject: RFR(XS): 8134738: Remove CollectorPolicy::Name
In-Reply-To: <55E42639.5020904@oracle.com>
References: <55E42639.5020904@oracle.com>
Message-ID: <55E42840.9010704@oracle.com>

Hi Per,

On 2015-08-31 12:02, Per Liden wrote:
> Hi,
>
> Please review this small cleanup, which removes the unused
> CollectorPolicy::Name enum and the related kind() functions. They are
> not used by the SA either, so it's unaffected.
>
> Webrev: http://cr.openjdk.java.net/~pliden/8134738/webrev.0/

Looks good.

/Mikael

>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8134738
>
> cheers,
> /Per
>



From bengt.rutisson at oracle.com  Mon Aug 31 10:05:24 2015
From: bengt.rutisson at oracle.com (Bengt Rutisson)
Date: Mon, 31 Aug 2015 12:05:24 +0200
Subject: RFR(XS): 8134738: Remove CollectorPolicy::Name
In-Reply-To: <55E42639.5020904@oracle.com>
References: <55E42639.5020904@oracle.com>
Message-ID: <55E426E4.8040407@oracle.com>


Hi Per,

On 2015-08-31 12:02, Per Liden wrote:
> Hi,
>
> Please review this small cleanup, which removes the unused 
> CollectorPolicy::Name enum and the related kind() functions. They are 
> not used by the SA either, so it's unaffected.
>
> Webrev: http://cr.openjdk.java.net/~pliden/8134738/webrev.0/
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8134738


Looks good.

Bengt

>
> cheers,
> /Per
>



From per.liden at oracle.com  Mon Aug 31 10:28:33 2015
From: per.liden at oracle.com (Per Liden)
Date: Mon, 31 Aug 2015 12:28:33 +0200
Subject: RFR(XS): 8134738: Remove CollectorPolicy::Name
In-Reply-To: <55E426E4.8040407@oracle.com>
References: <55E42639.5020904@oracle.com> <55E426E4.8040407@oracle.com>
Message-ID: <55E42C51.1070503@oracle.com>

Thanks Mikael and Bengt!

/Per

On 2015-08-31 12:05, Bengt Rutisson wrote:
>
> Hi Per,
>
> On 2015-08-31 12:02, Per Liden wrote:
>> Hi,
>>
>> Please review this small cleanup, which removes the unused
>> CollectorPolicy::Name enum and the related kind() functions. They are
>> not used by the SA either, so it's unaffected.
>>
>> Webrev: http://cr.openjdk.java.net/~pliden/8134738/webrev.0/
>>
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8134738
>
>
> Looks good.
>
> Bengt
>
>>
>> cheers,
>> /Per
>>
>


From thomas.schatzl at oracle.com  Mon Aug 31 11:38:16 2015
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 31 Aug 2015 13:38:16 +0200
Subject: RFR (S): 8067341: Modify PLAB sizing algorithm to waste less
In-Reply-To: <55DDCF09.3090203@oracle.com>
References: <1440518817.2210.21.camel@oracle.com>
	<55DCF5AA.6030604@oracle.com> <1440582897.2315.5.camel@oracle.com>
	<1440592142.2315.30.camel@oracle.com> <55DDCF09.3090203@oracle.com>
Message-ID: <1441021096.8174.5.camel@oracle.com>

Hi Jon, Eric,

  thanks for your reviews.

On Wed, 2015-08-26 at 07:36 -0700, Jon Masamitsu wrote:
> Thomas,
> 
> Looks good.
> 
> After looking at the version with G1PLABOccupancyFactor, returning
> to G1LastPLABAverageOccupancy does seem more natural.   It's easier
> to understand what the flag represents.
> 

Okay :)

> One correction on the comment.

I fixed the comment according to both of your suggestions.

http://cr.openjdk.java.net/~tschatzl/8067341/webrev.3
http://cr.openjdk.java.net/~tschatzl/8067341/webrev.2_to_3

Thanks,
  Thomas




From david.lindholm at oracle.com  Mon Aug 31 13:22:04 2015
From: david.lindholm at oracle.com (David Lindholm)
Date: Mon, 31 Aug 2015 15:22:04 +0200
Subject: RFR: 8134626 - Misc cleanups after generation array removal
In-Reply-To: <55E07247.8020304@oracle.com>
References: <55E07247.8020304@oracle.com>
Message-ID: <55E454FC.7060106@oracle.com>

Jesper,

I think these changes looks good. There are changes to quite a lot of 
files, especially in increment 2. I don't know if backporting fixes 
later on will be a problem? Also, some of these files are in runtime, 
for example vmStructs.cpp. Maybe you should let someone from runtime 
review as well?

That said, I think these are nice cleanups.


Thanks,
David

On 2015-08-28 16:37, Jesper Wilhelmsson wrote:
> Hi,
>
> Please review this rather large cleanup. While working with the 
> generation array removal I did a bunch of unrelated cleanups in the 
> code I passed by. I split this out into a separate patch to keep the 
> other changes clean.
>
> This change is split into three webrevs to make it easier to review. I 
> intend to push all three as one change.
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8134626
>
>
> Increment 1: Renaming
> http://cr.openjdk.java.net/~jwilhelm/8134626/webrev.00/inc1-renaming/
>
> Changes the variable name gen to young_gen or old_gen depending on 
> what the variable will contain. The few cases where the name gen is 
> untouched are places where the variable actually can contain either 
> generation.
>
>
> Increment 2: Comments and indentation
> http://cr.openjdk.java.net/~jwilhelm/8134626/webrev.00/inc2-comments_and_indentation/ 
>
>
> * Fixes up alignment in lots of places.
> * Inserts and removes empty lines where needed.
> * Cleans up comments where we previously talked about older and 
> younger generations. Fixes a few typos and clarifies some comments 
> with respect to young/old generations and collections.
> * Added spaces around some operators.
>
>
> Increment 3: Code changes
> http://cr.openjdk.java.net/~jwilhelm/8134626/webrev.00/inc3-code_changes/
>
> * Merged strings that was split on several lines for no good reason.
> * Added braces for if statements and for loops.
> * Removed dead code.
> * Moved variable initialization to initializer list in CollectedHeap 
> constructor.
> * Updated flag descriptions "youngest" -> "young".
>
> Thanks,
> /Jesper
>



From jesper.wilhelmsson at oracle.com  Mon Aug 31 13:51:14 2015
From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson)
Date: Mon, 31 Aug 2015 15:51:14 +0200
Subject: RFR: 8134626 - Misc cleanups after generation array removal
In-Reply-To: <55E454FC.7060106@oracle.com>
References: <55E07247.8020304@oracle.com> <55E454FC.7060106@oracle.com>
Message-ID: <55E45BD2.3010303@oracle.com>

Thanks David!

There doesn't seem to be a lot of backporting going on lately. I don't mind 
backporting this change if we want to ease future backporting. FYI, previous 
changes like this (e.g. JDK-8025856) was not backported due to low interest from 
other developers.

I'll forward the RFR to the runtime list as well.

Thanks,
/Jesper


Den 31/8/15 kl. 15:22, skrev David Lindholm:
> Jesper,
>
> I think these changes looks good. There are changes to quite a lot of files,
> especially in increment 2. I don't know if backporting fixes later on will be a
> problem? Also, some of these files are in runtime, for example vmStructs.cpp.
> Maybe you should let someone from runtime review as well?
>
> That said, I think these are nice cleanups.
>
>
> Thanks,
> David
>
> On 2015-08-28 16:37, Jesper Wilhelmsson wrote:
>> Hi,
>>
>> Please review this rather large cleanup. While working with the generation
>> array removal I did a bunch of unrelated cleanups in the code I passed by. I
>> split this out into a separate patch to keep the other changes clean.
>>
>> This change is split into three webrevs to make it easier to review. I intend
>> to push all three as one change.
>>
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8134626
>>
>>
>> Increment 1: Renaming
>> http://cr.openjdk.java.net/~jwilhelm/8134626/webrev.00/inc1-renaming/
>>
>> Changes the variable name gen to young_gen or old_gen depending on what the
>> variable will contain. The few cases where the name gen is untouched are
>> places where the variable actually can contain either generation.
>>
>>
>> Increment 2: Comments and indentation
>> http://cr.openjdk.java.net/~jwilhelm/8134626/webrev.00/inc2-comments_and_indentation/
>>
>>
>> * Fixes up alignment in lots of places.
>> * Inserts and removes empty lines where needed.
>> * Cleans up comments where we previously talked about older and younger
>> generations. Fixes a few typos and clarifies some comments with respect to
>> young/old generations and collections.
>> * Added spaces around some operators.
>>
>>
>> Increment 3: Code changes
>> http://cr.openjdk.java.net/~jwilhelm/8134626/webrev.00/inc3-code_changes/
>>
>> * Merged strings that was split on several lines for no good reason.
>> * Added braces for if statements and for loops.
>> * Removed dead code.
>> * Moved variable initialization to initializer list in CollectedHeap constructor.
>> * Updated flag descriptions "youngest" -> "young".
>>
>> Thanks,
>> /Jesper
>>
>


From jesper.wilhelmsson at oracle.com  Mon Aug 31 13:56:54 2015
From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson)
Date: Mon, 31 Aug 2015 15:56:54 +0200
Subject: FWD: RFR: 8134626 - Misc cleanups after generation array removal
In-Reply-To: <55E07247.8020304@oracle.com>
References: <55E07247.8020304@oracle.com>
Message-ID: <55E45D26.5040000@oracle.com>

Hi runtime,

While working with the generation array removal I did a bunch of unrelated 
cleanups in the code I passed by. I split this out into a separate patch to keep 
the other changes clean.

This is mostly GC code, but a few runtime files was touched as well and might 
cause merge conflicts with your favorite local repository, so I was advised to 
forward this RFR to the runtime list as well.


This change is split into three webrevs to make it easier to review. I intend to 
push all three as one change.

Bug: https://bugs.openjdk.java.net/browse/JDK-8134626


Increment 1: Renaming
http://cr.openjdk.java.net/~jwilhelm/8134626/webrev.00/inc1-renaming/

Changes the variable name gen to young_gen or old_gen depending on what the
variable will contain. The few cases where the name gen is untouched are places
where the variable actually can contain either generation.


Increment 2: Comments and indentation
http://cr.openjdk.java.net/~jwilhelm/8134626/webrev.00/inc2-comments_and_indentation/

* Fixes up alignment in lots of places.
* Inserts and removes empty lines where needed.
* Cleans up comments where we previously talked about older and younger
generations. Fixes a few typos and clarifies some comments with respect to
young/old generations and collections.
* Added spaces around some operators.


Increment 3: Code changes
http://cr.openjdk.java.net/~jwilhelm/8134626/webrev.00/inc3-code_changes/

* Merged strings that was split on several lines for no good reason.
* Added braces for if statements and for loops.
* Removed dead code.
* Moved variable initialization to initializer list in CollectedHeap constructor.
* Updated flag descriptions "youngest" -> "young".

Thanks,
/Jesper





From per.liden at oracle.com  Mon Aug 31 14:15:50 2015
From: per.liden at oracle.com (Per Liden)
Date: Mon, 31 Aug 2015 16:15:50 +0200
Subject: RFR(xs): 8134749: SoftReferences declared dead too early
Message-ID: <55E46196.4010305@oracle.com>


Hi,

While preparing a different patch I looked through the 
ReferenceProcessor and noticed what looks like a silent mistreatment of 
SoftReferences.

Summary: RefProcPhase1Task uses WorkerThread::id() to select the list to 
process. After the recent changes to how WorkGang works[1], this no 
longer guarantees that all reference lists are processed. This in turn 
means we might kill SoftReferences even when the policy says they should 
be kept alive.

The fix here would be to use the worker_id (named "i" in this code) 
passed into the Task's work() function instead of using WorkerThread::id().


[1] There's no longer any guarantee that all GangWorker threads will 
execute the Task's work() function, since a quick-to-wakeup worker 
thread might steal the work intended for a slow-to-wakeup worker thread.


Webrev: http://cr.openjdk.java.net/~pliden/8134749/webrev.0/

Bug: https://bugs.openjdk.java.net/browse/JDK-8134749

Testing: jprt


Btw, I'll to send out a follow-up patch (JDK-8134744) which makes sure 
WorkerThread::id() and the worker_id always match to avoid this type of 
bugs in the future.

/Per


From per.liden at oracle.com  Mon Aug 31 14:16:18 2015
From: per.liden at oracle.com (Per Liden)
Date: Mon, 31 Aug 2015 16:16:18 +0200
Subject: RFR(xs): 8134744: WorkerThread::id() should match GangWorker's
	worker_id
Message-ID: <55E461B2.2070802@oracle.com>

Hi,

This is a follow up patch to JDK-8134749 (SoftReferences declared dead 
too early), which makes sure WorkerThread::id() and match GangWorker's 
worker_id always match. This allows them to be used interchangeably, 
which is both convenient and removes a potential source for bugs (it's 
easy to assumed they are the same). This change should not affect any 
existing code.

Webrev: http://cr.openjdk.java.net/~pliden/8134744/webrev.0/

Bug: https://bugs.openjdk.java.net/browse/JDK-8134744

Testing: jprt

/Per



From eric.caspole at oracle.com  Mon Aug 31 14:31:09 2015
From: eric.caspole at oracle.com (Eric Caspole)
Date: Mon, 31 Aug 2015 10:31:09 -0400
Subject: RFR: JDK-8134761 - Add PrintGCDetails for debugging JDK-8134523
Message-ID: <55E4652D.3070702@oracle.com>

Hi everybody,
I'd like to add +PrintGCDetails and +PrintHeapAtGC to a rarely-failing test:

  http://cr.openjdk.java.net/~ecaspole/JDK-8134761/00/webrev/

ER:
  https://bugs.openjdk.java.net/browse/JDK-8134761


This is to help debug a failure in a new humongous test, see the 
original bug:

  https://bugs.openjdk.java.net/browse/JDK-8134523

I do see the PrintGCDetails output in the .jtr file when run locally so 
I think JPRT will capture it as well.
Thanks,
Eric


From alexander.harlap at oracle.com  Mon Aug 31 15:38:26 2015
From: alexander.harlap at oracle.com (Alexander Harlap)
Date: Mon, 31 Aug 2015 11:38:26 -0400
Subject: Request for code review - 8130265: gctests/LargeObjects/large001
	fails with OutOfMemoryError: Java heap space
In-Reply-To: <1441010444.2740.18.camel@oracle.com>
References: <55D356FB.1030603@oracle.com> <55D35A12.8020608@oracle.com>
	<55D35F1F.3090700@oracle.com> <55D36DC7.6010901@oracle.com>
	<55D38CBE.8070408@oracle.com> <55D394AD.6020104@oracle.com>
	<55D4E39D.5070805@oracle.com> <55D4E8C7.1080703@oracle.com>
	<55D4EEF5.9030504@oracle.com>
	<2C141666-BDBC-428C-9E40-4FB78F076192@oracle.com>
	<1440082894.2347.14.camel@oracle.com> <55D62D77.8020504@oracle.com>
	<55D71703.4070505@oracle.com> <55D74766.5020202@oracle.com>
	<1440420922.2348.82.camel@oracle.com> <55DB8B7E.5010601@oracle.com>
	<55DC6CB5.4010108@oracle.com> <1440584814.2315.16.camel@oracle.com>
	<55DE531D.9090703@oracle.com> <1441010444.2740.18.camel@oracle.com>
Message-ID: <55E474F2.8030404@oracle.com>



On 8/31/2015 4:40 AM, Thomas Schatzl wrote:
> Hi,
>
> On Wed, 2015-08-26 at 20:00 -0400, Alexander Harlap wrote:
>> On 8/26/2015 6:26 AM, Thomas Schatzl wrote:
>>> Hi,
>>>
>>>     looking at webrev.05, but answering here.
>>>
>>> On Tue, 2015-08-25 at 15:25 +0200, Jesper Wilhelmsson wrote:
>>>> Hi Alex,
>>>>
>>>> I like this approach :)
>>>>
>>>> Just one comment:
>>>>
>>>> After GC in the helper you put the assert that soft references should have been
>>>> handled. The first call to this helper will not clear soft references. The
>>>> original code only has this assert at the end of the function. I think it is
>>> Whatever the original code did, there is the general requirement that
>>> after any full gc, collector_policy()->should_clear_all_soft_refs()
>>> should be false.
>>>    
>>>> enough to keep it there, at the end of satisfy_failed_allocation().
>>> It seems to be randomly placed now, checking only after everything else
>>> fails. I do agree that the original code did the same, but why not
>>> improve upon this?
>>>
>>> If you want in a separate CR.
>> Yes, I would agree to make a separate CR. Only I will need help to
>> specify issue.
> Something like "Misplaced check for
> CollectedPolicy::should_clear_all_soft_refs() in G1" should be fine.
OK
>>>> 1764     assert(!collector_policy()->should_clear_all_soft_refs(),
>>>> 1765            "Flag should have been handled and cleared prior to this point");
>>>>
>>>> Besides that I'm fine with this version.
>>> - Some braces next to boolean parameters are not aligned properly. E.g.
>>> in line 1777 and 1804.
>> I am sorry, I can not locate those defects - did you mean lines 1777 and
>> 1804 in webrev.05?
> In g1CollectedHeap.cpp, line 1777 and 1804 (sorry for not mentioning the
> file):
>
> 1774   HeapWord* result =
> 1775     satisfy_failed_allocation_helper(word_size,
> 1776                                      context,
> 1777                                      true, /* do_gc */
> 1778                                      false, /* clear_all_soft_refs */
> 1779                                      false, /* expect_null_mutator_alloc_region */
> 1780                                      succeeded);
> 1781
>
> The opening comment bracket in line 1777 is not aligned to the ones
> below afaik. Same in 1804.
>
> [...]
Now I understand
>>>>>> (And the bool return value of do_collection removed).
>>>>>>
>>>>>> I also have a question about the order of
>>>>>> attempt_allocation_at_safepoint() and the expand_and_allocate(), and
>>>>>> could not find this answered before: so full gc might shrink the heap
>>>>>> too much after full gc, and we should expand the heap.
>>>>>> If I were to do this change, there would always (unconditionally) be an
>>>>>> expansion after full gc, because the current change only expands if the
>>>>>> allocation after gc fails. Also the 02 change.
>>>>>>
>>>>>> The review thread so far suggests that full gc might shrink too much,
>>>>>> but this still seems to be the case here in some cases.
>>> Could you clarify this question? To me, this looks like a potential bug
>>> that has been overlooked so far in this change.
>> Yes, full GC in some situation may over-shrink heap such way that heap
>> expansion will be needed. It was reason for
>> gctests/LargeObjects/large001 failure.  Still I prefer to do expansion
>> only after allocation attempt failed. Why should we do unconditional
>> expansion ( even if it is not needed )?
> To keep the Min/MaxHeapFreeRatio constraints the heap should observe.
>
>>   In the existing code we have
>> call to expand_and_allocate() only after
>> attempt_allocation_at_safepoint() failure. What potential bug do you mean?
> That the Min/MaxHeapFreeRatio values are not observed, causing
> performance degradation, in the worst case like in this situation.
>
> The descriptions of the flags do not indicate an exception like "do not
> care if the GC thinks it is not required".
>
> Thanks,
>    Thomas
>

Min/MaxHeapFreeRatio already were observed in calculation of shrink value. I do not see reason for expanding heap immediately after do_collection before doing attempt to allocate.

  





From stefan.johansson at oracle.com  Mon Aug 31 16:09:23 2015
From: stefan.johansson at oracle.com (Stefan Johansson)
Date: Mon, 31 Aug 2015 18:09:23 +0200
Subject: RFR: 8129417: Oop iteration clean-up to remove
	oop_ms_follow_contents
In-Reply-To: <55DC51A6.1080503@oracle.com>
References: <5588217F.9090201@oracle.com>
	<20150706151537.GP3266@ehelin.jrpg.bea.com>
	<559BCD11.3010801@oracle.com> <55D312FF.3070105@oracle.com>
	<55D713CD.3080305@oracle.com> <55DC51A6.1080503@oracle.com>
Message-ID: <55E47C33.5000609@oracle.com>

Hi,

After some offline discussions with Erik and Stefan I've done some small 
changes to avoid unnecessary calls to do_metadata from 
oop_iterate_range(). I also modified the handling of object arrays 
slightly and made sure not push empty arrays to avoid extra unnecessary 
calls into the iteration code.

Webrev of these changes:
http://cr.openjdk.java.net/~sjohanss/8129417/5-no-range-metadata/hotspot.00/

Full webrev:
http://cr.openjdk.java.net/~sjohanss/8129417/hotspot.03/

Did a sanity RBT run and also re-did some of the benchmarking to make 
sure these changes are on par with previous versions, and no problems 
were found.

Thanks,
Stefan

On 2015-08-25 13:29, Per Liden wrote:
> Hi Stefan,
>
> On 2015-08-21 14:04, Stefan Johansson wrote:
>> Hi Per,
>>
>> On 2015-08-18 13:11, Per Liden wrote:
>>> Hi Stefan,
>>>
>>> On 2015-07-07 14:58, Stefan Johansson wrote:
>>>> Thanks Erik for reviewing,
>>>>
>>>> New webrevs:
>>>> Part 2 - full:
>>>> http://cr.openjdk.java.net/~sjohanss/8129417/2-add-oop_iterate_size/hotspot.01/ 
>>>>
>>>>
>>>> Part 2 - inc:
>>>> http://cr.openjdk.java.net/~sjohanss/8129417/2-add-oop_iterate_size/hotspot.00-01/ 
>>>>
>>>>
>>>>
>>>> Part 3 - full:
>>>> http://cr.openjdk.java.net/~sjohanss/8129417/3-re-structure-mark-sweep/hotspot.02/ 
>>>>
>>>>
>>>> Part 3 - inc:
>>>> http://cr.openjdk.java.net/~sjohanss/8129417/3-re-structure-mark-sweep/hotspot.00-02/ 
>>>>
>>>>
>>>> Also includes an updated assert, which was missed in previous webrev.
>>>>
>>>> Comments inline.
>>>
>>> Just one thing that looks odd to me, but maybe you can clarify. 
>>> Shouldn't MarkSweep::follow_object() call follow_array() instead of 
>>> calling push_objarray()? That way the change made to 
>>> ObjArrayTask::is_valid() wouldn't be needed.
>>>
>> Thanks for looking at this again. I agree that your proposed change 
>> would make the code nicer and I think I had it structured that way 
>> once during this work but had to revert it back to this to keep 
>> performance.
>>
>> I went ahead and did the changes and some new measurements, and this 
>> change leads to a small regression for some platforms. I haven't done 
>> any deep analysis but I guess it leads to worse inlining decisions on 
>> these platforms.
>>
>> I prefer to leave the code as is. The other possible solution would 
>> be to do further analysis and give even more special inlining hints 
>> to the different compilers.
>
> Ok, I'm fine with leaving it as is if the other approach leads to 
> regressions.
>
> /Per
>
>>
>> Thanks,
>> Stefan
>>
>>> /Per
>>>
>>>>
>>>> On 2015-07-06 17:15, Erik Helin wrote:
>>>>> Hi Stefan,
>>>>>
>>>>> thanks a lot for taking on this work. Please see comments inline.
>>>>>
>>>>> On 2015-06-22, Stefan Johansson wrote:
>>>>>> Hi,
>>>>>>
>>>>>> Please review these changes for RFE:
>>>>>> https://bugs.openjdk.java.net/browse/JDK-8129417
>>>>>>
>>>>>> Webrev for the full change:
>>>>>> http://cr.openjdk.java.net/~sjohanss/8129417/hotspot.00/
>>>>>>
>>>>>> Summary:
>>>>>> To allow further cleanups and later remove G1 specific code from
>>>>>> mark-sweep,
>>>>>> we want to remove the mark-sweep specific visitor
>>>>>> oop_ms_follow_contents.
>>>>>> Doing this cleanup has proven to be a little more complicated than I
>>>>>> first
>>>>>> anticipated and to make the change easier to review I've split it
>>>>>> into four
>>>>>> different parts.
>>>>>>
>>>>>> Part 1 - removing oop_ms_follow_contents:
>>>>>> http://cr.openjdk.java.net/~sjohanss/8129417/1-remove-oop_ms_follow_contents/hotspot.00/ 
>>>>>>
>>>>>>
>>>>> - Looking in specialized_oop_closures.hpp, the comment
>>>>>     71 // This is split into several because of a Visual C++ 6.0
>>>>> compiler bug
>>>>>     72 // where very long macros cause the compiler to crash
>>>>>    seems a bit dated, given that Visual C++ 6.0 was released in 
>>>>> 1998. I
>>>>>    think we should try to merge ALL_OOP_OOP_ITERATE_CLOSURES_1 and 
>>>>> 2 (and
>>>>>    their corresponding macros). Do you think like doing that in this
>>>>>    patch or do you want to file a follow-up bug?
>>>> I would like to do/investigate this as a separate RFE, I have a 
>>>> list of
>>>> things that should follow this clean up and I plan to file those 
>>>> before
>>>> pushing this change.
>>>>
>>>>>
>>>>> - The part of the patch that changes instanceMirrorKlass.inline.hpp
>>>>>    might impact more GCs than just any of the marksweep ones. Have 
>>>>> you
>>>>>    seen any performance improvements/regressions with G1 or CMS?
>>>> No, but I have not done as many runs and done as much analysis of this
>>>> as I've done on the serial full gc. I've not seen any regression in
>>>> aurora.se performance runs.
>>>>>> Part 2 - introducing oop_iterate_size:
>>>>>> http://cr.openjdk.java.net/~sjohanss/8129417/2-add-oop_iterate_size/hotspot.00/ 
>>>>>>
>>>>>>
>>>>> - In arrayKlass.hpp, why add a space to:
>>>>>    +#define OOP_OOP_ITERATE_DECL_NO_BACKWARDS(OopClosureType,
>>>>> nv_suffix)         \
>>>> I've followed '\' alignment "rule" for these macros, which seems to be
>>>> two spaces after the longest line. I just realized that I should also
>>>> update OOP_OOP_ITERATE_DECL_RANGE with one extra space to be 
>>>> consistent.
>>>>
>>>>> - In arrayKlass.hpp, the following lines seems aligned strangely:
>>>>>    +void KlassType::oop_oop_iterate_range##nv_suffix(oop obj,
>>>>> OopClosureType* closure, int start, int end) {  \
>>>>>    + oop_oop_iterate_range<nvs_to_bool(nv_suffix)>(obj, closure,
>>>>> start, end);                               \
>>>> Nice catch, missed fixing the alignment when making it void.
>>>>
>>>>> - In oop.inline.hpp, why must the size be computed before applying 
>>>>> the
>>>>>    closure? In the world of perm gen, this might have been 
>>>>> important in
>>>>>    order to not follow stale klass pointers, but with Metaspace the
>>>>> Klass*
>>>>>    won't move.
>>>> As we've discussed offline, this is most likely safe. I've done an
>>>> ad-hoc RBT run with some random testing when I assert on the size 
>>>> being
>>>> the same both before and after the call to oop_oop_iterate.
>>>>
>>>> I would like to do this change as a separate change as well, so it's
>>>> been added to my list of RFEs to file.
>>>>
>>>>>
>>>>> - Can we remove the method MutableSpace::oop_iterate?
>>>> Looks like it, no one seems to be using it. I'll remove it instead of
>>>> updating it to use oop_iterate_size().
>>>>
>>>>>
>>>>>> Part 3 - restructure mark-sweep:
>>>>>> http://cr.openjdk.java.net/~sjohanss/8129417/3-re-structure-mark-sweep/hotspot.00/ 
>>>>>>
>>>>>>
>>>>> - Can you please remove the commented assert in mark_and_push:
>>>>>     +//  assert(Universe::heap()->is_in_reserved(p), "should be in
>>>>> object space");
>>>>>    or should it be enabled again?
>>>> Removed.
>>>>
>>>>> - I agree with Per about moving the check for object arrays into
>>>>>    follow_object.
>>>> Already done, but gives a slight regression. I still think we can live
>>>> with this small regression for now, to avoid people doing wrong and 
>>>> call
>>>> follow_object on objArrays.
>>>>
>>>>>> Part 4 - compiler hints for inlining:
>>>>>> http://cr.openjdk.java.net/~sjohanss/8129417/4-compiler-hints-for-inlining/hotspot.00/ 
>>>>>>
>>>>>>
>>>>> - I would like to see the definiton of FORCE_INLINE in the compiler
>>>>>    specific globalDefinitions file. Then the code in
>>>>>    instanceKlass.inline.hpp can look like:
>>>>>     #if defined(TARGET_COMPILER_visCPP) ||
>>>>> defined(TARGET_COMPILER_sparcWorks)
>>>>>       #define INLINE FORCE_INLINE
>>>>>     #else
>>>>>       #define INLINE inline
>>>>>     #endif
>>>>>
>>>>> - The same comment for stack.inline.hpp, but with NO_INLINE:
>>>>>    #if defined(TARGET_COMPILER_sparcWorks)
>>>>>      #define STACK_NO_INLINE NO_INLINE
>>>>>    #else
>>>>>      #define STACK_NO_INLINE
>>>>>    #endif
>>>> I agree, we should come up with a better and cleaner way to do this. I
>>>> will file a follow up RFE.
>>>>
>>>>> - I agree with Per about adding an #undef in stack.inline.hpp
>>>> Already fixed.
>>>>
>>>> Thanks again Erik,
>>>> Stefan
>>>>
>>>>> Thanks!
>>>>> Erik
>>>>>
>>>>>> Testing:
>>>>>> * Adhoc RBT run for functionality - no new failures.
>>>>>> * Adhoc aurora.se run for performance - no obvious regression.
>>>>>> * Many local runs tuned to do Full GCs a lot to verify that there 
>>>>>> is no
>>>>>> obvious regression.
>>>>>>
>>>>>> Thanks,
>>>>>> Stefan
>>>>
>>
>



From ysr1729 at gmail.com  Mon Aug 31 16:25:09 2015
From: ysr1729 at gmail.com (Srinivas Ramakrishna)
Date: Mon, 31 Aug 2015 09:25:09 -0700
Subject: Request for code review (M) - 8081629: CMS split_block() does not
	correctly fix up block-offset-table for large blocks
In-Reply-To: <558868A3.2070604@oracle.com>
References: <558868A3.2070604@oracle.com>
Message-ID: <CABzyjymWMaZOKG=fLRQFLzKmD7NH=bmFZRjfRDkAfTyjG8Gp7A@mail.gmail.com>

Hi Jon --

Sorry for the looong delay in getting to this and then of turning around
the webrev once I started looking at the code.
It's been a while since I was in this code, and the code is quite
complicated, so it took a while to swap back
the context and understand (the correctness of) the change.



On Mon, Jun 22, 2015 at 12:57 PM, Jon Masamitsu <jon.masamitsu at oracle.com>
wrote:

> This bug is causing large young GC times for ParNew when
> there are large objects in the heap and the logarithmic strides
> are being used (badly).
>
> https://bugs.openjdk.java.net/browse/JDK-8081629
>
> The change in webrev_min corrects the problem
> with the minimum code change (least risk).
>
> http://cr.openjdk.java.net/~jmasa/8081629/webrev_min.02/



The above changes look good to me. Thanks also for confirming that these
changes fix the assertion failure seen with +VerifyBlockOffsetArray. It
might be
worthwhile writing (in the future) a slightly more lightweight
VerifyBlockOffsetArray
so that it becomes practical to use.

I haven't had the opportunity to look at and understand the second version
of the
webrev below yet. But the one above looks good to me. I'll try and get to
the
version below in the next day or so. Sorry for the ongoing delay.

thanks!
-- ramki (openjdk: ysr)


>
> A rewriting of split_block() in included in a second webrev
>
> http://cr.openjdk.java.net/~jmasa/8081629/webrev.02/
>
> I've willing to go with the minimum change but could also
> be encouraged to push the rewrite of split_block().
>
> Vote for the one you like with your code review.
>
> The diff between webrev_min.02 and webrev.02 is
> here but I don't find it that useful.
>
> http://cr.openjdk.java.net/~jmasa/8081629/webrev_delta.02/
>
> Thanks.
>
> Jon
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150831/ab051f7d/attachment.htm>

From kim.barrett at oracle.com  Mon Aug 31 17:32:05 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Mon, 31 Aug 2015 13:32:05 -0400
Subject: RFR: 8131330: G1CollectedHeap::verify_dirty_young_list fails with
	assert
In-Reply-To: <55DF683A.1070402@oracle.com>
References: <5D7EB2C6-6AC4-4B87-8FD1-FC487328087C@oracle.com>
	<55D4C22F.2000003@oracle.com>
	<08169C1E-696E-44C1-AA98-3F9AA2895197@oracle.com>
	<AEE51CEF-0B38-4A43-BB8D-F51FEC74B9B8@oracle.com>
	<B8713224-3063-4BDB-9FAA-7ECB5F9AF0DA@oracle.com>
	<55DF683A.1070402@oracle.com>
Message-ID: <544B3C8E-8636-423F-9122-B4D9856492EB@oracle.com>

On Aug 27, 2015, at 3:42 PM, Eric Caspole <eric.caspole at oracle.com> wrote:
> 
> Hi Kim,
> Looks good.
> Eric

Thanks.



From kim.barrett at oracle.com  Mon Aug 31 17:32:50 2015
From: kim.barrett at oracle.com (Kim Barrett)
Date: Mon, 31 Aug 2015 13:32:50 -0400
Subject: RFR: 8131330: G1CollectedHeap::verify_dirty_young_list fails with
	assert
In-Reply-To: <1441007950.2740.8.camel@oracle.com>
References: <5D7EB2C6-6AC4-4B87-8FD1-FC487328087C@oracle.com>
	<55D4C22F.2000003@oracle.com>
	<08169C1E-696E-44C1-AA98-3F9AA2895197@oracle.com>
	<AEE51CEF-0B38-4A43-BB8D-F51FEC74B9B8@oracle.com>
	<B8713224-3063-4BDB-9FAA-7ECB5F9AF0DA@oracle.com>
	<55DF683A.1070402@oracle.com> <1441007950.2740.8.camel@oracle.com>
Message-ID: <AA008676-37FD-4574-A41F-4A0703007F2E@oracle.com>

On Aug 31, 2015, at 3:59 AM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
> 
> Hi,
> 
> On Thu, 2015-08-27 at 15:42 -0400, Eric Caspole wrote:
>> Hi Kim,
>> Looks good.
>> Eric
>> 
>> 
>> On 8/20/2015 4:29 PM, Kim Barrett wrote:
>>> [?]
>>> Full webrev:
>>> http://cr.openjdk.java.net/~kbarrett/8131330/webrev.02/
>>> 
>>> Incremental webrev:
>>> http://cr.openjdk.java.net/~kbarrett/8131330/webrev.02.inc2/
>>> 
> 
>  looks good.
> 
> Thomas

Thanks Thomas.



From jon.masamitsu at oracle.com  Mon Aug 31 18:20:58 2015
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Mon, 31 Aug 2015 11:20:58 -0700
Subject: RFR(xs): 8134749: SoftReferences declared dead too early
In-Reply-To: <55E46196.4010305@oracle.com>
References: <55E46196.4010305@oracle.com>
Message-ID: <55E49B0A.20707@oracle.com>

Per,

Change looks good.

Could you please run some tests with your fix and 
-XX:+UseDynamicNumberOfGCThreads and
-XX:+UseParallelGC threads?  UseDynamicNumberOfGCThreads sometimes moves
work among the queues to get them into queues that will be processed by 
an active
GC worker.  Thanks.

Jon


On 08/31/2015 07:15 AM, Per Liden wrote:
>
> Hi,
>
> While preparing a different patch I looked through the 
> ReferenceProcessor and noticed what looks like a silent mistreatment 
> of SoftReferences.
>
> Summary: RefProcPhase1Task uses WorkerThread::id() to select the list 
> to process. After the recent changes to how WorkGang works[1], this no 
> longer guarantees that all reference lists are processed. This in turn 
> means we might kill SoftReferences even when the policy says they 
> should be kept alive.
>
> The fix here would be to use the worker_id (named "i" in this code) 
> passed into the Task's work() function instead of using 
> WorkerThread::id().
>
>
> [1] There's no longer any guarantee that all GangWorker threads will 
> execute the Task's work() function, since a quick-to-wakeup worker 
> thread might steal the work intended for a slow-to-wakeup worker thread.
>
>
> Webrev: http://cr.openjdk.java.net/~pliden/8134749/webrev.0/
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8134749
>
> Testing: jprt
>
>
> Btw, I'll to send out a follow-up patch (JDK-8134744) which makes sure 
> WorkerThread::id() and the worker_id always match to avoid this type 
> of bugs in the future.
>
> /Per



From tom.benson at oracle.com  Mon Aug 31 18:27:13 2015
From: tom.benson at oracle.com (Tom Benson)
Date: Mon, 31 Aug 2015 14:27:13 -0400
Subject: RFR(xs): 8134749: SoftReferences declared dead too early
In-Reply-To: <55E49B0A.20707@oracle.com>
References: <55E46196.4010305@oracle.com> <55E49B0A.20707@oracle.com>
Message-ID: <55E49C81.4070003@oracle.com>

Per,
Also looks good to me.   I actually picked up both your changes, on the 
off chance it might help with 8130261.  (That appears it may be a 
problem with 'last ditch' GCs, which free all soft refs.  Yes, a 
longshot, but I tried it anyway.  No help.)

Tom

On 8/31/2015 2:20 PM, Jon Masamitsu wrote:
> Per,
>
> Change looks good.
>
> Could you please run some tests with your fix and 
> -XX:+UseDynamicNumberOfGCThreads and
> -XX:+UseParallelGC threads?  UseDynamicNumberOfGCThreads sometimes moves
> work among the queues to get them into queues that will be processed 
> by an active
> GC worker.  Thanks.
>
> Jon
>
>
> On 08/31/2015 07:15 AM, Per Liden wrote:
>>
>> Hi,
>>
>> While preparing a different patch I looked through the 
>> ReferenceProcessor and noticed what looks like a silent mistreatment 
>> of SoftReferences.
>>
>> Summary: RefProcPhase1Task uses WorkerThread::id() to select the list 
>> to process. After the recent changes to how WorkGang works[1], this 
>> no longer guarantees that all reference lists are processed. This in 
>> turn means we might kill SoftReferences even when the policy says 
>> they should be kept alive.
>>
>> The fix here would be to use the worker_id (named "i" in this code) 
>> passed into the Task's work() function instead of using 
>> WorkerThread::id().
>>
>>
>> [1] There's no longer any guarantee that all GangWorker threads will 
>> execute the Task's work() function, since a quick-to-wakeup worker 
>> thread might steal the work intended for a slow-to-wakeup worker thread.
>>
>>
>> Webrev: http://cr.openjdk.java.net/~pliden/8134749/webrev.0/
>>
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8134749
>>
>> Testing: jprt
>>
>>
>> Btw, I'll to send out a follow-up patch (JDK-8134744) which makes 
>> sure WorkerThread::id() and the worker_id always match to avoid this 
>> type of bugs in the future.
>>
>> /Per
>



From jon.masamitsu at oracle.com  Mon Aug 31 18:35:14 2015
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Mon, 31 Aug 2015 11:35:14 -0700
Subject: RFR(xs): 8134744: WorkerThread::id() should match GangWorker's
	worker_id
In-Reply-To: <55E461B2.2070802@oracle.com>
References: <55E461B2.2070802@oracle.com>
Message-ID: <55E49E62.4050100@oracle.com>

Per,

A GangWorker is given an "uint id" when it is
created.  If it is going to be reset the id whenever it
does work, should it have its own "id"?

Jon

On 08/31/2015 07:16 AM, Per Liden wrote:
> Hi,
>
> This is a follow up patch to JDK-8134749 (SoftReferences declared dead 
> too early), which makes sure WorkerThread::id() and match GangWorker's 
> worker_id always match. This allows them to be used interchangeably, 
> which is both convenient and removes a potential source for bugs (it's 
> easy to assumed they are the same). This change should not affect any 
> existing code.
>
> Webrev: http://cr.openjdk.java.net/~pliden/8134744/webrev.0/
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8134744
>
> Testing: jprt
>
> /Per
>



From eric.caspole at oracle.com  Mon Aug 31 21:54:12 2015
From: eric.caspole at oracle.com (Eric Caspole)
Date: Mon, 31 Aug 2015 17:54:12 -0400
Subject: RFR: JDK-8134761 - Add PrintGCDetails for debugging JDK-8134523
In-Reply-To: <55E4652D.3070702@oracle.com>
References: <55E4652D.3070702@oracle.com>
Message-ID: <55E4CD04.2030100@oracle.com>

I can reproduce this problem now, so for the time being I retract this 
review.
Thanks,
Eric


On 08/31/2015 10:31 AM, Eric Caspole wrote:
> Hi everybody,
> I'd like to add +PrintGCDetails and +PrintHeapAtGC to a rarely-failing
> test:
>
>   http://cr.openjdk.java.net/~ecaspole/JDK-8134761/00/webrev/
>
> ER:
>   https://bugs.openjdk.java.net/browse/JDK-8134761
>
>
> This is to help debug a failure in a new humongous test, see the
> original bug:
>
>   https://bugs.openjdk.java.net/browse/JDK-8134523
>
> I do see the PrintGCDetails output in the .jtr file when run locally so
> I think JPRT will capture it as well.
> Thanks,
> Eric


From derek.white at oracle.com  Mon Aug 31 22:12:50 2015
From: derek.white at oracle.com (Derek White)
Date: Mon, 31 Aug 2015 18:12:50 -0400
Subject: RFR(xs): 8134744: WorkerThread::id() should match GangWorker's
	worker_id
In-Reply-To: <55E49E62.4050100@oracle.com>
References: <55E461B2.2070802@oracle.com> <55E49E62.4050100@oracle.com>
Message-ID: <55E4D162.2030508@oracle.com>

Hi Per,

I'm confused on the same issue, or I've gotten lost between thinking 
about [Abstract] (WorkGang | GangWorker) :-)

AbstractWorkGang::initialize_workers() assigns ids from 
0..total_workers(). This id is also GangWorker's index into the 
AbstractWorkGang's_worker array. So changing GangWorker's id breaks 
implied consistency that AbstractWorkGang::worker(some_gang_worker._id) 
== some_gang_worker. Right? This seem more confusing than the WorkData's 
_worker_id != GangerWorkers._id.

Note that I didn't see how WorkerThread::id(), mentioned below, fits in 
this picture.

Since the WorkData's _worker_id is arbitrarily handed out, but the 
GangWorker's id is an index into an array, it seems safer to keep 
GangWorker's ID static, and setting WorkData's id when it gets assigned 
to a worker.

  - Derek

On 8/31/15 2:35 PM, Jon Masamitsu wrote:
> Per,
>
> A GangWorker is given an "uint id" when it is
> created.  If it is going to be reset the id whenever it
> does work, should it have its own "id"?
>
> Jon
>
> On 08/31/2015 07:16 AM, Per Liden wrote:
>> Hi,
>>
>> This is a follow up patch to JDK-8134749 (SoftReferences declared 
>> dead too early), which makes sure WorkerThread::id() and match 
>> GangWorker's worker_id always match. This allows them to be used 
>> interchangeably, which is both convenient and removes a potential 
>> source for bugs (it's easy to assumed they are the same). This change 
>> should not affect any existing code.
>>
>> Webrev: http://cr.openjdk.java.net/~pliden/8134744/webrev.0/
>>
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8134744
>>
>> Testing: jprt
>>
>> /Per
>>
>