From kim.barrett at oracle.com  Wed Nov  1 05:03:38 2017
From: kim.barrett at oracle.com (Kim Barrett)
Date: Wed, 1 Nov 2017 01:03:38 -0400
Subject: RFR: 8188102: [JVMCI] Convert special JVMCI oops in nmethod to
 jweak values
In-Reply-To: <BDD0189D-ECBF-42A8-BDDE-2014753B53C9@oracle.com>
References: <94B50A35-9898-47BA-8F20-A829FD7547AC@oracle.com>
 <E0D909B8-A115-4F83-9D0B-12E3FCF7EBEB@oracle.com>
 <BDD0189D-ECBF-42A8-BDDE-2014753B53C9@oracle.com>
Message-ID: <BABEA69B-939D-4863-9A5B-271F32F330EB@oracle.com>

> On Oct 30, 2017, at 7:14 AM, Doug Simon <doug.simon at oracle.com> wrote:
> 
> Hi Kim,
> 
> Thanks for the detailed review.
> 
>> On 29 Oct 2017, at 23:29, Kim Barrett <kim.barrett at oracle.com> wrote:
>> 
>> [added hotspot-gc-dev to cc list]
>> 
>>> On Oct 27, 2017, at 5:05 PM, Doug Simon <doug.simon at oracle.com> wrote:
>>> 
>>> Please review this change that converts the JVMCI-specific object references in nmethod from oops to weak values. This removes GC API extensions added purely for these fields (e.g. so that G1 can insert it into the right remembered set, and when unloading an nmethod, to go and remove the nmethod from that remembered set).
>>> 
>>> Testing: I've run the Graal unit tests (mx unittest --verbose --gc-after-test -Xlog:class+unload=trace) which trigger a lot of nmethod unloading.
>>> 
>>> https://bugs.openjdk.java.net/browse/JDK-8188102
>>> http://cr.openjdk.java.net/~dnsimon/8188102/
>> 
>> I didn't look at the .java, .py, or project files.
>> 
>> ------------------------------------------------------------------------------ 
>> src/hotspot/share/jvmci/jvmciCompilerToVM.cpp
>> 1061       nmethod* nm = cb->as_nmethod_or_null();
>> 
>> This appears to be dead code now.
> 
> Indeed.
> 
>> ------------------------------------------------------------------------------ 
>> src/hotspot/share/code/nmethod.cpp
>> 1023   assert(Universe::heap()->is_gc_active(), "should only be called during gc");
>> ...
>> 1036     if (!Universe::heap()->is_gc_active() && cause != NULL)
>> 1037       cause->klass()->print_on(&ls);
>> 
>> I was going to mention that lines 1036-1037 are missing braces around
>> the if-body.  However, those lines appear to be dead code, given the
>> assertion on line 1023.
> 
> Good catch. That problem pre-dates this webrev but I will clean it up here.
> 
>> ------------------------------------------------------------------------------ 
>> src/hotspot/share/code/nmethod.cpp
>> 1504 bool nmethod::do_unloading_jvmci(BoolObjectClosure* is_alive, bool unloading_occurred) {
>> ...
>> 1506     oop installed_code = JNIHandles::resolve(_jvmci_installed_code);
>> 
>> Resolving a weak reference can keep an otherwise dead referent alive.
>> See JDK-8188055 for a discussion of the corresponding problem for
>> j.l.r.Reference.
>> 
>> Right now, I think JNIHandles doesn't provide a (public) solution to
>> what I think is being attempted here that works for all collectors.
>> There is in-progress work toward a solution, but it's just that, "in
>> progress".
>> 
>> As a (possibly interim) solution, a function like the following might
>> be added to JNIHandles (put the definition near resolve_jweak).
>> 
>> bool JNIHandles::is_global_weak_cleared(jweak handle) {
>> assert(is_jweak(handle), "not a weak handle");
>> return guard_value<false>(jweak_ref(handle)) == NULL;
>> }
> 
> Adding JNIHandles::is_global_weak_cleared makes sense. I've put it the public section near destroy_weak_global instead of the private section where resolve_jweak is declared.
> 
>> (That's completely untested, and I haven't thought carefully about the
>> name.  And should get input from other GC folks on how to deal with
>> this.)  I *think* do_unloading_jvmci then becomes something like the
>> following (again, completely untested)
>> 
>> bool nmethod::do_unloading_jvmci(BoolObjectClosure* is_alive, bool unloading_occurred) {
>> if (_jvmci_installed_code != NULL) {
>>   if (JNIHandles::is_global_weak_cleared(_jvmci_installed_code)) {
>>     if (_jvmci_installed_code_triggers_unloading) {
>>       make_unloaded(is_alive, NULL);
>>       return true;
>>     } else {
>>       clear_jvmci_installed_code();
>>     }
>>   }
>> }
>> return false;
>> }
> 
> I think your change works but comes at the cost of potentially preventing nmethod unloading for 1 extra (full?) GC cycle. It assumes that jweak clearing occurs before nmethod scanning. Is that guaranteed? If not, then I think what we want is:
> 
> bool nmethod::do_unloading_jvmci(BoolObjectClosure* is_alive, bool unloading_occurred) {
>  if (_jvmci_installed_code != NULL) {
>    bool cleared = JNIHandles::is_global_weak_cleared(_jvmci_installed_code);
>    if (_jvmci_installed_code_triggers_unloading) {
>      if (cleared) {
>        // jweak reference processing has already cleared the referent
>        make_unloaded(is_alive, NULL);
>        return true;
>      } else {
>        oop installed_code = JNIHandles::resolve(_jvmci_installed_code);
>        if (can_unload(is_alive, (oop*)&installed_code, unloading_occurred)) {
>          return true;
>        }
>      }
>    } else {
>      if (cleared || !is_alive->do_object_b(JNIHandles::resolve(_jvmci_installed_code))) {
>        clear_jvmci_installed_code();
>      }
>    }
>  }
>  return false;
> }
> 
> I've created a new webrev at http://cr.openjdk.java.net/~dnsimon/8188102_2.
> 
> -Doug

The old code was looking for objects that are no longer alive; that
can't be determined until after reference processing, as finalization
could change the answer.  jweak clearing used to happen as part of
reference processing, but there's been some recent refactoring.  I
don't think that was intended to change the order of jweak processing
wrto other things, but I haven't looked at the code to verify I'm not
overlooking something.

In addition, I think

     } else {
       oop installed_code = JNIHandles::resolve(_jvmci_installed_code);
       if (can_unload(is_alive, (oop*)&installed_code, unloading_occurred)) {
         return true;
       }

doesn't work for some collectors (like G1), since the resolve will
force installed_code to be live (if jweak processing were performed
after nmethod processing), so can_unload will always return false.



From erik.osterlund at oracle.com  Wed Nov  1 09:44:53 2017
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Wed, 1 Nov 2017 10:44:53 +0100
Subject: RFR: 8188102: [JVMCI] Convert special JVMCI oops in nmethod to
 jweak values
In-Reply-To: <BABEA69B-939D-4863-9A5B-271F32F330EB@oracle.com>
References: <94B50A35-9898-47BA-8F20-A829FD7547AC@oracle.com>
 <E0D909B8-A115-4F83-9D0B-12E3FCF7EBEB@oracle.com>
 <BDD0189D-ECBF-42A8-BDDE-2014753B53C9@oracle.com>
 <BABEA69B-939D-4863-9A5B-271F32F330EB@oracle.com>
Message-ID: <59F99795.6090404@oracle.com>

Hi Kim,

On 2017-11-01 06:03, Kim Barrett wrote:
>> On Oct 30, 2017, at 7:14 AM, Doug Simon <doug.simon at oracle.com> wrote:
>>
>> Hi Kim,
>>
>> Thanks for the detailed review.
>>
>>> On 29 Oct 2017, at 23:29, Kim Barrett <kim.barrett at oracle.com> wrote:
>>>
>>> [added hotspot-gc-dev to cc list]
>>>
>>>> On Oct 27, 2017, at 5:05 PM, Doug Simon <doug.simon at oracle.com> wrote:
>>>>
>>>> Please review this change that converts the JVMCI-specific object references in nmethod from oops to weak values. This removes GC API extensions added purely for these fields (e.g. so that G1 can insert it into the right remembered set, and when unloading an nmethod, to go and remove the nmethod from that remembered set).
>>>>
>>>> Testing: I've run the Graal unit tests (mx unittest --verbose --gc-after-test -Xlog:class+unload=trace) which trigger a lot of nmethod unloading.
>>>>
>>>> https://bugs.openjdk.java.net/browse/JDK-8188102
>>>> http://cr.openjdk.java.net/~dnsimon/8188102/
>>> I didn't look at the .java, .py, or project files.
>>>
>>> ------------------------------------------------------------------------------
>>> src/hotspot/share/jvmci/jvmciCompilerToVM.cpp
>>> 1061       nmethod* nm = cb->as_nmethod_or_null();
>>>
>>> This appears to be dead code now.
>> Indeed.
>>
>>> ------------------------------------------------------------------------------
>>> src/hotspot/share/code/nmethod.cpp
>>> 1023   assert(Universe::heap()->is_gc_active(), "should only be called during gc");
>>> ...
>>> 1036     if (!Universe::heap()->is_gc_active() && cause != NULL)
>>> 1037       cause->klass()->print_on(&ls);
>>>
>>> I was going to mention that lines 1036-1037 are missing braces around
>>> the if-body.  However, those lines appear to be dead code, given the
>>> assertion on line 1023.
>> Good catch. That problem pre-dates this webrev but I will clean it up here.
>>
>>> ------------------------------------------------------------------------------
>>> src/hotspot/share/code/nmethod.cpp
>>> 1504 bool nmethod::do_unloading_jvmci(BoolObjectClosure* is_alive, bool unloading_occurred) {
>>> ...
>>> 1506     oop installed_code = JNIHandles::resolve(_jvmci_installed_code);
>>>
>>> Resolving a weak reference can keep an otherwise dead referent alive.
>>> See JDK-8188055 for a discussion of the corresponding problem for
>>> j.l.r.Reference.
>>>
>>> Right now, I think JNIHandles doesn't provide a (public) solution to
>>> what I think is being attempted here that works for all collectors.
>>> There is in-progress work toward a solution, but it's just that, "in
>>> progress".
>>>
>>> As a (possibly interim) solution, a function like the following might
>>> be added to JNIHandles (put the definition near resolve_jweak).
>>>
>>> bool JNIHandles::is_global_weak_cleared(jweak handle) {
>>> assert(is_jweak(handle), "not a weak handle");
>>> return guard_value<false>(jweak_ref(handle)) == NULL;
>>> }
>> Adding JNIHandles::is_global_weak_cleared makes sense. I've put it the public section near destroy_weak_global instead of the private section where resolve_jweak is declared.
>>
>>> (That's completely untested, and I haven't thought carefully about the
>>> name.  And should get input from other GC folks on how to deal with
>>> this.)  I *think* do_unloading_jvmci then becomes something like the
>>> following (again, completely untested)
>>>
>>> bool nmethod::do_unloading_jvmci(BoolObjectClosure* is_alive, bool unloading_occurred) {
>>> if (_jvmci_installed_code != NULL) {
>>>    if (JNIHandles::is_global_weak_cleared(_jvmci_installed_code)) {
>>>      if (_jvmci_installed_code_triggers_unloading) {
>>>        make_unloaded(is_alive, NULL);
>>>        return true;
>>>      } else {
>>>        clear_jvmci_installed_code();
>>>      }
>>>    }
>>> }
>>> return false;
>>> }
>> I think your change works but comes at the cost of potentially preventing nmethod unloading for 1 extra (full?) GC cycle. It assumes that jweak clearing occurs before nmethod scanning. Is that guaranteed? If not, then I think what we want is:
>>
>> bool nmethod::do_unloading_jvmci(BoolObjectClosure* is_alive, bool unloading_occurred) {
>>   if (_jvmci_installed_code != NULL) {
>>     bool cleared = JNIHandles::is_global_weak_cleared(_jvmci_installed_code);
>>     if (_jvmci_installed_code_triggers_unloading) {
>>       if (cleared) {
>>         // jweak reference processing has already cleared the referent
>>         make_unloaded(is_alive, NULL);
>>         return true;
>>       } else {
>>         oop installed_code = JNIHandles::resolve(_jvmci_installed_code);
>>         if (can_unload(is_alive, (oop*)&installed_code, unloading_occurred)) {
>>           return true;
>>         }
>>       }
>>     } else {
>>       if (cleared || !is_alive->do_object_b(JNIHandles::resolve(_jvmci_installed_code))) {
>>         clear_jvmci_installed_code();
>>       }
>>     }
>>   }
>>   return false;
>> }
>>
>> I've created a new webrev at http://cr.openjdk.java.net/~dnsimon/8188102_2.
>>
>> -Doug
> The old code was looking for objects that are no longer alive; that
> can't be determined until after reference processing, as finalization
> could change the answer.  jweak clearing used to happen as part of
> reference processing, but there's been some recent refactoring.  I
> don't think that was intended to change the order of jweak processing
> wrto other things, but I haven't looked at the code to verify I'm not
> overlooking something.
>
> In addition, I think
>
>       } else {
>         oop installed_code = JNIHandles::resolve(_jvmci_installed_code);
>         if (can_unload(is_alive, (oop*)&installed_code, unloading_occurred)) {
>           return true;
>         }
>
> doesn't work for some collectors (like G1), since the resolve will
> force installed_code to be live (if jweak processing were performed
> after nmethod processing), so can_unload will always return false.
>

JNIHandles::resolve with G1 should conditionally enqueue the oops if the 
SATB queue is activated, which it will not be after marking terminated. 
At this point, marking should have terminated already, and hence the 
resolve will not change the liveness of the oop. So I don't quite see 
how the resolve will change the oop to live here. Am I missing something?

Having said that, I agree this should be a "peek" resolve, which we may 
implement soon.

Thanks,
/Erik


From rs at jelastic.com  Wed Nov  1 10:30:50 2017
From: rs at jelastic.com (Ruslan Synytsky)
Date: Wed, 1 Nov 2017 11:30:50 +0100
Subject: Heap Size Ergonomics in docker containers
In-Reply-To: <1509463358.3199.2.camel@oracle.com>
References: <CA++bR4O9j=C5s2tJCH=JHRc51AdnCRiocvcUorJnngoG7i_R2A@mail.gmail.com>
 <1509463358.3199.2.camel@oracle.com>
Message-ID: <CA++bR4N6jKfx6Rhzt9xktwfsQt=4tgZ_qJ-m1R_fYeCZTSfpDQ@mail.gmail.com>

Hi Thomas, thank you for the feedback. Please see inline.

On 31 October 2017 at 16:22, Thomas Schatzl <thomas.schatzl at oracle.com>
wrote:

> Hi Ruslan,
>
> On Mon, 2017-10-30 at 19:06 +0100, Ruslan Synytsky wrote:
> > Hi guys, I'm interested in improvements in this direction as well.
> > Let me share the statistical experience from Jelastic customers who
> > run Java stacks in containers for a long time.
> >
> > We set the minimum Xmx to 32m by default as we believe majority wants
> > to start from the minimum, specifically in the "micro" world
> > (services, containers). G1 GC provides good enough elasticity of
> > memory allocation. You can grow up quickly, and important to scale
> > down later when the load goes away.
> >
> > By default, Xmx is 80% of available RAM with container limits. It
> > works great for the majority of our users. We track OOM Kill events
> > and send alerts to the users. So the numbers are based on the real
> > end users applications.
> >
> > We keep 80% from max by default always, even if users allocate a
> > bigger amount of RAM, for two reasons: 1 - it's easier to remember
> > the defaults and 2 - unused resources are not reserved with
> > containers, so if Java does not use it - no problem, they are free,
> > and at the same time additional memory available for native non-heap
> > data or for other processes, just like an insurance.
>
> Thanks for your insights into real-world usage. So maybe the suggested
> 50% as default for small heaps is indeed a bit too conservative.
>
Understanding of the situations when people rely on the default heuristic
will be really helpful to make a final decision... From my personal
experience, if you do not specify Xmx you always get troubles at the end.


> > Big memory and big data solutions - everything like this requires a
> > fine-tuning, the customers who run such stacks definitely do not rely
> > on the default heuristic. You can always find instructions and
> > recommended RAM usage configs for these stacks. Some solutions even
> > use direct memory access, a small heap size is preferable for them.
> > In general, it means you have to do a customization if you want to
> > get a stable solution.
> >
> > To make it simple, we came up with a supervisor script that
> > applies custom memory configs. As result, customers can define the
> > needed parameters with env variables in a container if they do not
> > want to rely on the heuristic.
> >
> > Btw, is there any guaranteed way to pass Xmx and Xms to java process
> > w/o any external scripts? I mean, it will be cool if java can check
> > env variable like XMX and if it exists apply.
> >
>
> There is JAVA_TOOL_OPTIONS, which are always prepended to the java VM
> command line. See the description [0] for more information.
>
Indeed this option is close to what I was looking for! But seems
JAVA_TOOL_OPTIONS does not work for Xmx. Just a quick test with Java
HotSpot(TM) 64-Bit Server VM (build 9+181, mixed mode)

*Default*
$ java -XX:+PrintFlagsFinal -version | grep MaxHeapSize
   size_t MaxHeapSize                              = 643825664
                   {product} {command line}

*Define Xmx = 2GB via JAVA_TOOL_OPTIONS *
$ export JAVA_TOOL_OPTIONS="-Xmx2g"
$ java -XX:+PrintFlagsFinal -version | grep MaxHeapSize
Picked up JAVA_TOOL_OPTIONS: -Xmx2g
   size_t MaxHeapSize                              = 643825664
                   {product} {command line}

It says that Xmx has been picked up, but in reality it has not been
applied.

*Define Xmx = 3GB via _JAVA_OPTIONS *
$ export _JAVA_OPTIONS="-Xmx3g"
$ java -XX:+PrintFlagsFinal -version | grep MaxHeapSize
Picked up JAVA_TOOL_OPTIONS: -Xmx2g
Picked up _JAVA_OPTIONS: -Xmx3g
   size_t MaxHeapSize                              = 3221225472
                  {product} {command line}

Now it works as expected. However, another issue is that _JAVA_OPTIONS is
undocumented and hotspot-specific?...

Also important to mention that _JAVA_OPTIONS trumps command-line arguments,
and command-line arguments trump JAVA_TOOL_OPTIONS. That gives a quite good
flexibility. Because sometimes you may want to override command line args
and sometimes you need just to provide better deafults, but if users
specify command line args use them. In other words, I believe, if we can a)
improve JAVA_TOOL_OPTIONS (or introduce a new name like
JAVA_OPTIONS_DEFAULT) for enabling people to specify Xmx via this env var
and b) make _JAVA_OPTIONS as a standard/documented option, then this will
solve the existig dificulties and provide enough flexibility.

Thanks


>
> I think this is what you were looking for.
>
> Thanks,
>   Thomas
>
> [0] https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot
> /envvars002.html
> <https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/envvars002.html>
>



-- 
Ruslan
CEO @ Jelastic <https://jelastic.com/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20171101/29842c5b/attachment.htm>

From yasuenag at gmail.com  Wed Nov  1 13:02:40 2017
From: yasuenag at gmail.com (Yasumasa Suenaga)
Date: Wed, 1 Nov 2017 22:02:40 +0900
Subject: PING: RFR: JDK-8153333: [REDO] STW phases at Concurrent GC should
 count in PerfCounter
In-Reply-To: <c8ffa7b0-64a0-2899-db6d-c82857139000@gmail.com>
References: <CAGFVN2CoZiXRsr3McK69PSNz1bYwDvnx+LkqEaJc25c4uuDovw@mail.gmail.com>
 <e8d8f095-80cf-40ca-1bf8-fd45eeb7611e@gmail.com>
 <2f4e0901-1602-6276-e7fd-84e168e7b317@gmail.com>
 <c8ffa7b0-64a0-2899-db6d-c82857139000@gmail.com>
Message-ID: <fc8505a4-05a7-e668-d782-088db41f6528@gmail.com>

PING: Could you review and sponsor it?

>    http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/

Also I need JPRT results of this change.
Could you cooperate?


Thanks,

Yasumasa


On 2017/09/27 0:08, Yasumasa Suenaga wrote:
> Hi all,
> 
> I uploaded new webrev to be adapted to jdk10/hs:
> 
>  ? http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/
> 
> I want to check this patch via JPRT, but I cannot access it.
> Could you cooperate?
> 
> 
> Thanks,
> 
> yasumasa
> 
> 
> On 2017/09/21 7:46, Yasumasa Suenaga wrote:
>> PING:
>>
>> Have you checked this issue?
>>
>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/hotspot/
>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/jdk/
>>
>>
>> Yasumasa
>>
>>
>> On 2017/07/01 23:44, Yasumasa Suenaga wrote:
>>> PING:
>>>
>>> Have you checked this issue?
>>>
>>>
>>> Yasumasa
>>>
>>>
>>> On 2017/06/14 13:22, Yasumasa Suenaga wrote:
>>>> Hi all,
>>>>
>>>> I changed PerfCounter to show CGC STW phase in jstat in JDK-8151674.
>>>> However, it occurred several jtreg test failure, so it was back-outed.
>>>>
>>>> I want to resume to work for this issue.
>>>>
>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/hotspot/
>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/jdk/
>>>>
>>>> These changes are work fine on jtreg test as below:
>>>>
>>>> ?? hotspot/test/serviceability/tmtools/jstat
>>>> ?? jdk/test/sun/tools
>>>>
>>>>
>>>> Since JDK 9, default GC algorithm is set to G1.
>>>> So I think this change is useful to watch GC behavior through jstat.
>>>>
>>>> I cannot access JPRT. Could you help?
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Yasumasa
>>>>


From kim.barrett at oracle.com  Wed Nov  1 17:57:10 2017
From: kim.barrett at oracle.com (Kim Barrett)
Date: Wed, 1 Nov 2017 13:57:10 -0400
Subject: RFR: 8188102: [JVMCI] Convert special JVMCI oops in nmethod to
 jweak values
In-Reply-To: <59F99795.6090404@oracle.com>
References: <94B50A35-9898-47BA-8F20-A829FD7547AC@oracle.com>
 <E0D909B8-A115-4F83-9D0B-12E3FCF7EBEB@oracle.com>
 <BDD0189D-ECBF-42A8-BDDE-2014753B53C9@oracle.com>
 <BABEA69B-939D-4863-9A5B-271F32F330EB@oracle.com>
 <59F99795.6090404@oracle.com>
Message-ID: <33980406-F96C-434E-AAB7-949B39D46026@oracle.com>

> On Nov 1, 2017, at 5:44 AM, Erik ?sterlund <erik.osterlund at oracle.com> wrote:
> 
> Hi Kim,
> 
> On 2017-11-01 06:03, Kim Barrett wrote:
>>> On Oct 30, 2017, at 7:14 AM, Doug Simon <doug.simon at oracle.com> wrote:
>>> 
>>> Hi Kim,
>>> 
>>> Thanks for the detailed review.
>>> 
>>>> On 29 Oct 2017, at 23:29, Kim Barrett <kim.barrett at oracle.com> wrote:
>>>> 
>>>> [added hotspot-gc-dev to cc list]
>>>> 
>>>>> On Oct 27, 2017, at 5:05 PM, Doug Simon <doug.simon at oracle.com> wrote:
>>>>> 
>>>>> Please review this change that converts the JVMCI-specific object references in nmethod from oops to weak values. This removes GC API extensions added purely for these fields (e.g. so that G1 can insert it into the right remembered set, and when unloading an nmethod, to go and remove the nmethod from that remembered set).
>>>>> 
>>>>> Testing: I've run the Graal unit tests (mx unittest --verbose --gc-after-test -Xlog:class+unload=trace) which trigger a lot of nmethod unloading.
>>>>> 
>>>>> https://bugs.openjdk.java.net/browse/JDK-8188102
>>>>> http://cr.openjdk.java.net/~dnsimon/8188102/
>>>> I didn't look at the .java, .py, or project files.
>>>> 
>>>> ------------------------------------------------------------------------------
>>>> src/hotspot/share/jvmci/jvmciCompilerToVM.cpp
>>>> 1061       nmethod* nm = cb->as_nmethod_or_null();
>>>> 
>>>> This appears to be dead code now.
>>> Indeed.
>>> 
>>>> ------------------------------------------------------------------------------
>>>> src/hotspot/share/code/nmethod.cpp
>>>> 1023   assert(Universe::heap()->is_gc_active(), "should only be called during gc");
>>>> ...
>>>> 1036     if (!Universe::heap()->is_gc_active() && cause != NULL)
>>>> 1037       cause->klass()->print_on(&ls);
>>>> 
>>>> I was going to mention that lines 1036-1037 are missing braces around
>>>> the if-body.  However, those lines appear to be dead code, given the
>>>> assertion on line 1023.
>>> Good catch. That problem pre-dates this webrev but I will clean it up here.
>>> 
>>>> ------------------------------------------------------------------------------
>>>> src/hotspot/share/code/nmethod.cpp
>>>> 1504 bool nmethod::do_unloading_jvmci(BoolObjectClosure* is_alive, bool unloading_occurred) {
>>>> ...
>>>> 1506     oop installed_code = JNIHandles::resolve(_jvmci_installed_code);
>>>> 
>>>> Resolving a weak reference can keep an otherwise dead referent alive.
>>>> See JDK-8188055 for a discussion of the corresponding problem for
>>>> j.l.r.Reference.
>>>> 
>>>> Right now, I think JNIHandles doesn't provide a (public) solution to
>>>> what I think is being attempted here that works for all collectors.
>>>> There is in-progress work toward a solution, but it's just that, "in
>>>> progress".
>>>> 
>>>> As a (possibly interim) solution, a function like the following might
>>>> be added to JNIHandles (put the definition near resolve_jweak).
>>>> 
>>>> bool JNIHandles::is_global_weak_cleared(jweak handle) {
>>>> assert(is_jweak(handle), "not a weak handle");
>>>> return guard_value<false>(jweak_ref(handle)) == NULL;
>>>> }
>>> Adding JNIHandles::is_global_weak_cleared makes sense. I've put it the public section near destroy_weak_global instead of the private section where resolve_jweak is declared.
>>> 
>>>> (That's completely untested, and I haven't thought carefully about the
>>>> name.  And should get input from other GC folks on how to deal with
>>>> this.)  I *think* do_unloading_jvmci then becomes something like the
>>>> following (again, completely untested)
>>>> 
>>>> bool nmethod::do_unloading_jvmci(BoolObjectClosure* is_alive, bool unloading_occurred) {
>>>> if (_jvmci_installed_code != NULL) {
>>>>   if (JNIHandles::is_global_weak_cleared(_jvmci_installed_code)) {
>>>>     if (_jvmci_installed_code_triggers_unloading) {
>>>>       make_unloaded(is_alive, NULL);
>>>>       return true;
>>>>     } else {
>>>>       clear_jvmci_installed_code();
>>>>     }
>>>>   }
>>>> }
>>>> return false;
>>>> }
>>> I think your change works but comes at the cost of potentially preventing nmethod unloading for 1 extra (full?) GC cycle. It assumes that jweak clearing occurs before nmethod scanning. Is that guaranteed? If not, then I think what we want is:
>>> 
>>> bool nmethod::do_unloading_jvmci(BoolObjectClosure* is_alive, bool unloading_occurred) {
>>>  if (_jvmci_installed_code != NULL) {
>>>    bool cleared = JNIHandles::is_global_weak_cleared(_jvmci_installed_code);
>>>    if (_jvmci_installed_code_triggers_unloading) {
>>>      if (cleared) {
>>>        // jweak reference processing has already cleared the referent
>>>        make_unloaded(is_alive, NULL);
>>>        return true;
>>>      } else {
>>>        oop installed_code = JNIHandles::resolve(_jvmci_installed_code);
>>>        if (can_unload(is_alive, (oop*)&installed_code, unloading_occurred)) {
>>>          return true;
>>>        }
>>>      }
>>>    } else {
>>>      if (cleared || !is_alive->do_object_b(JNIHandles::resolve(_jvmci_installed_code))) {
>>>        clear_jvmci_installed_code();
>>>      }
>>>    }
>>>  }
>>>  return false;
>>> }
>>> 
>>> I've created a new webrev at http://cr.openjdk.java.net/~dnsimon/8188102_2.
>>> 
>>> -Doug
>> The old code was looking for objects that are no longer alive; that
>> can't be determined until after reference processing, as finalization
>> could change the answer.  jweak clearing used to happen as part of
>> reference processing, but there's been some recent refactoring.  I
>> don't think that was intended to change the order of jweak processing
>> wrto other things, but I haven't looked at the code to verify I'm not
>> overlooking something.
>> 
>> In addition, I think
>> 
>>      } else {
>>        oop installed_code = JNIHandles::resolve(_jvmci_installed_code);
>>        if (can_unload(is_alive, (oop*)&installed_code, unloading_occurred)) {
>>          return true;
>>        }
>> 
>> doesn't work for some collectors (like G1), since the resolve will
>> force installed_code to be live (if jweak processing were performed
>> after nmethod processing), so can_unload will always return false.
>> 
> 
> JNIHandles::resolve with G1 should conditionally enqueue the oops if the SATB queue is activated, which it will not be after marking terminated. At this point, marking should have terminated already, and hence the resolve will not change the liveness of the oop. So I don't quite see how the resolve will change the oop to live here. Am I missing something?

Does this nmethod pass get run during young collections?  If not, then maybe you are right that it doesn?t matter.

But I still think that a cleanup pass that is based on using weak references like this should just be properly ordered
wrto weak reference processing, as it presumably already was.  In which case the resolve and can_unload is
effectively useless; either the jweak has been cleared and we won?t reach this clause, or the jweak has not been
cleared because the referent is still live, in which case can_unload will always return false and there was no point
in calling it.

> Having said that, I agree this should be a "peek" resolve, which we may implement soon.

I think there shouldn?t be a resolve at all, ?peek? or otherwise.



From kim.barrett at oracle.com  Thu Nov  2 03:47:00 2017
From: kim.barrett at oracle.com (Kim Barrett)
Date: Wed, 1 Nov 2017 23:47:00 -0400
Subject: RFR: 8188102: [JVMCI] Convert special JVMCI oops in nmethod to
 jweak values
In-Reply-To: <111F6116-A1A4-4308-BAEC-D471F9E974B9@oracle.com>
References: <94B50A35-9898-47BA-8F20-A829FD7547AC@oracle.com>
 <E0D909B8-A115-4F83-9D0B-12E3FCF7EBEB@oracle.com>
 <BDD0189D-ECBF-42A8-BDDE-2014753B53C9@oracle.com>
 <BABEA69B-939D-4863-9A5B-271F32F330EB@oracle.com>
 <59F99795.6090404@oracle.com>
 <33980406-F96C-434E-AAB7-949B39D46026@oracle.com>
 <111F6116-A1A4-4308-BAEC-D471F9E974B9@oracle.com>
Message-ID: <CD3B456E-8FB6-4295-ADAD-45BEDBD3E81F@oracle.com>

> On Nov 1, 2017, at 5:10 PM, Doug Simon <doug.simon at oracle.com> wrote:
> 
> 
> 
>> On 1 Nov 2017, at 18:57, Kim Barrett <kim.barrett at oracle.com> wrote:
>> 
>>> On Nov 1, 2017, at 5:44 AM, Erik ?sterlund <erik.osterlund at oracle.com> wrote:
>>> 
>>> Hi Kim,
>>> 
>>> On 2017-11-01 06:03, Kim Barrett wrote:
>>>>> On Oct 30, 2017, at 7:14 AM, Doug Simon <doug.simon at oracle.com> wrote:
>>> JNIHandles::resolve with G1 should conditionally enqueue the oops if the SATB queue is activated, which it will not be after marking terminated. At this point, marking should have terminated already, and hence the resolve will not change the liveness of the oop. So I don't quite see how the resolve will change the oop to live here. Am I missing something?
>> 
>> Does this nmethod pass get run during young collections?  If not, then maybe you are right that it doesn?t matter.
>> 
>> But I still think that a cleanup pass that is based on using weak references like this should just be properly ordered
>> wrto weak reference processing, as it presumably already was.  In which case the resolve and can_unload is
>> effectively useless; either the jweak has been cleared and we won?t reach this clause, or the jweak has not been
>> cleared because the referent is still live, in which case can_unload will always return false and there was no point
>> in calling it.
> 
> I agree. As stated previously, my (defensive) code is based on assuming that nmethod unloading might happen before reference processing. If it never/rarely happens, then your suggested change is the right one. I did some extra testing and never saw it happen. Based on that, I'd like to adopt your solution. In the worst case, nmethod unloading will be delayed one full GC cycle if nmethod unloading precedes reference processing.
> 
> I've created another rev based on this:
> 
> http://cr.openjdk.java.net/~dnsimon/8188102_3
> 
> -Doug

This looks good.

------------------------------------------------------------------------------ 
src/hotspot/share/runtime/jniHandles.cpp

I'd like is_global_weak_cleared moved up a few lines, immediately
following the specializations of resolve_jweak, which I think it is
closely related to.

I don't need another webrev for this.

------------------------------------------------------------------------------ 
src/hotspot/share/code/nmethod.cpp

2934 char* nmethod::jvmci_installed_code_name(char* buf, size_t buflen) {
2935   if (!this->is_compiled_by_jvmci()) {
2936     return NULL;
2937   }
2938   oop installed_code = JNIHandles::resolve(_jvmci_installed_code);

Are there any circumstances where it would be unfortunate to have
getting the name keep alive the object? I was thinking logging. Or
perhaps it's a feature that getting the name might artificially extend
the lifetime? Looking at the callers, it doesn't seem like a problem
right now.

This is one of those places where a "peek" access (coming soon from
the GC interface) might be appropriate.

------------------------------------------------------------------------------




From kim.barrett at oracle.com  Thu Nov  2 03:55:18 2017
From: kim.barrett at oracle.com (Kim Barrett)
Date: Wed, 1 Nov 2017 23:55:18 -0400
Subject: RFR: 8183542: Factor out serial GC specific code from
 GenCollectedHeap into its own subclass
In-Reply-To: <90d877fb-a406-e1e3-6d26-a5e00e5666eb@redhat.com>
References: <6dea5306-1ec9-d439-b498-b9e657d7daf0@redhat.com>
 <3979E865-1363-4B60-ABDF-63128BAF04CE@oracle.com>
 <352c558d-40ba-27c4-861b-fcdb99ef706d@redhat.com>
 <487881EC-2CCD-47C4-BA7A-1723A40C1F9F@oracle.com>
 <ac5bcbc7-87cd-b1c7-73a2-1de96c0fbae8@redhat.com>
 <11759C2F-6F17-49C0-9D68-5B5B057C14B7@oracle.com>
 <4f84ff16-3f69-d2cf-2eb3-2e820aad49af@redhat.com>
 <A3E6BAD8-82A2-43A8-84FE-C3202BBEC9B1@oracle.com>
 <90d877fb-a406-e1e3-6d26-a5e00e5666eb@redhat.com>
Message-ID: <2C530D1F-DAFA-428F-BC80-BF04EDB1DA98@oracle.com>

> On Oct 25, 2017, at 4:07 AM, Roman Kennke <rkennke at redhat.com> wrote:
> 
> Hi Kim, hi Jini,
> 
> thank you both for your reviews!
> 
> I think I need a sponsor now. The final webrev (same as before plus Reviewed-by line):
> http://cr.openjdk.java.net/~rkennke/8183542/webrev.03/ <http://cr.openjdk.java.net/%7Erkennke/8183542/webrev.03/>

Sorry, I overlooked the sponsorship request.  I?ll take care of it, but not today.



From sangheon.kim at oracle.com  Thu Nov  2 05:53:12 2017
From: sangheon.kim at oracle.com (sangheon.kim)
Date: Wed, 1 Nov 2017 22:53:12 -0700
Subject: RFR(M): 8190308: Supporting heap allocation on alternative memory
 devices and CSR review
In-Reply-To: <F89640DCD01A85489FCBA68183A6A0F3BCD4BDC5@ORSMSX116.amr.corp.intel.com>
References: <F89640DCD01A85489FCBA68183A6A0F3BCD4BDC5@ORSMSX116.amr.corp.intel.com>
Message-ID: <a31f2457-149b-f5fb-71c3-b0595ddfa87a@oracle.com>

Hi Kishor,

On 10/31/2017 04:53 PM, Kharbas, Kishor wrote:
>
> Greetings,
>
> I am continuing the earlier discussion [1] with a revised issue number 
> representing the implementation of the JEP [2].
>
> I have addressed the remaining unaddressed comments (copied below) in 
> these webrevs -
>
> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.11/ 
> <http://cr.openjdk.java.net/%7Ekkharbas/8190308/webrev.11/>
>
> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.11_to_10/ 
> <http://cr.openjdk.java.net/%7Ekkharbas/8190308/webrev.11_to_10/>
>
> Also, I would appreciate a review of the CSR for the new flag at 
> https://bugs.openjdk.java.net/browse/JDK-8190309.
>
CSR: Reviewed.


> > > ?- in that same thread there has also been the question about the
>
> > > need
>
> > > for blocking the signals for the process that has gone unanswered.
>
> > >
>
> Removed the blocking of signals.
>
> > > ?- Arguments::finalize_vm_init_args: maybe at the place where the
>
> > > change adds the warning/info message about NUMA support being
>
> > > specific
>
> > > to the file system; also add the same warning about UseLargePages
>
> > > that
>
> > > is located elsewhere.
>
> > >
>
> > > Like "UseXXXX may not be supported in some specific file system
>
> > > implementations." - or just ignore as Andrew said in the other
>
> > > email thread.
>
> > >
>
> > > Note that I am not sure that the selected place is the correct
>
> > > place
>
> > > for such warning about incompatible flags (and maybe
>
> > > UseNUMA/UseLargePages should be forcibly disabled here? But then
>
> > > again, it does not hurt just having it enabled?).
>
> > >
>
> > > I will let the runtime team comment as a lot of that argument
>
> > > processing changed in jdk9.
>
> > >
>
> > > Maybe Arguments::check_vm_args_consistency() is better.
>
> > >
>
> > > There is similar code in Arguments::adjust_after_os()...
>
> > >
>
> 1. Moved the check from finalize_vm_init_args() to 
> check_vm_args_consistency()
>
> 2. When UseNUMA flag is true, adaptive logical group chunk resizing is 
> used for Numa aware collectors such as ParallelOld. Just like UseNUMA 
> is disabled in os::inti_2() in Linux when UseLargePages is set, it 
> will be a good idea to disable UseNUMA when the new option is used.
>
> 3. I think its ok to leave UseNUMAInterleaving ON as it can be used 
> for other memory areas.
>
> 4. For the same reason I also do not disable UseLargePages.
>
> 5. About handling UseLargePages, I thought of handling it the same way 
> as when reserve_memory_special() fails to allocate large pages, i.e. 
> generating a log_debug message.
>
> /// failed; try to reserve regular memory below/
>
> /? if (UseLargePages && (!FLAG_IS_DEFAULT(UseLargePages) ||/
>
> /!FLAG_IS_DEFAULT(LargePageSizeInBytes))) {/
>
> /?log_debug(gc, heap, coops)("Reserve regular memory without large 
> pages");/
>
> > > ?- here I may probably be speaking wrongly on behalf of the
>
> > > runtime
>
> > > team, but os.hpp, as an abstraction of the OS should probably not
>
> > > have
>
> > > platform specific ifdefs in it.
>
> > >
>
> and
>
> > > > - You removed os::attempt_reserve_memory_at() from os.cpp and
>
> > > > split
>
> > > > into os_posix.cpp and os_windows.cpp.
>
> > > > ? But I think you should remain os::attempt_reserve_memory_at()
>
> > > > at
>
> > > > os.cpp and implement os specific stuffs at each os_xxx.cpp files
>
> > > > for
>
> > > > pd_xxx. Of cource move MemTracker function call as well.
>
> In the updated webrev, I move the implementation from os_posix.cpp and 
> os_window.cpp to respective pd_xxx functions. No AIX specific ifdef is 
> required now.
>
Thank you for all changes.

I have minor nits now:
==============================================
- os***.cpp has some function names which include *dax*. I would prefer 
not to include it. As you know, Thomas also pointed it.

==============================================
src/hotspot/share/runtime/arguments.cpp
2537???? if (!FLAG_IS_DEFAULT(UseNUMAInterleaving) || 
!FLAG_IS_DEFAULT(UseNUMA)) {
- Don't we need to check these 2 flags' value to be true? i.e. if user 
sets to false, below message will be printed.

==============================================
test/hotspot/jtreg/gc/TestAllocateHeapAt.java
- On other discussion, I mentioned to test only for Windows and Linux as 
the JEP described only about those 2. But without *dax* function names, 
it seems like not filtering OS seems okay too.

2? * Copyright (c) 2017, xxx Oracle and/or its affiliates. All rights 
reserved.
- Please remove 'xxx '.

47???? Collections.addAll(vmOpts, new String[] 
{"-XX:AllocateHeapAt="+test_dir,
- Add spaces. '+test_dir' -> ' + test_dir'

49????????????????????????????????????????????? "-Xmx50m",
50????????????????????????????????????????????? "-Xms50m",
- You said there were no special reason for 200m(heap size of webrev.10) 
on other discussion. I would recommend 32m.

FYI, I ran hs-tier1 and hs-tier2 with your patch and the result is good. 
i.e. no new failures.

Thanks,
Sangheon


> Thank you and looking forward for your feedback.
>
> - Kishor
>
> [1] 
> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2017-October/020682.html
>
> [2] https://bugs.openjdk.java.net/browse/JDK-8171181 
> <https://bugs.openjdk.java.net/browse/JDK-8171181>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20171101/f2735827/attachment.htm>

From igor.ignatyev at oracle.com  Thu Nov  2 06:27:05 2017
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Wed, 1 Nov 2017 23:27:05 -0700
Subject: RFR(L/S): 8190499 : convert vm.metaspace tonga tests to jtreg	
Message-ID: <5FA0C880-8EF2-4103-9803-5F754B343B21@oracle.com>

http://javaweb.us.oracle.com/~iignatye//8190499/webrev.01/index.html
> 946 lines changed: 490 ins; 216 del; 240 mod;

Hi all,

could you please review the patch which converts all tonga tests from vm.metaspace testlist to equivalent jtreg tests?

as usual, the huge part of the patch is done automatically, the manual part includes:
- creation of fromTonga_largepage test group which includes the tests from vm.metaspace.testlist which are also used in largepage.testlist
- updating PropertyResolvingWrapper to handle empty arguments
- mapping b/w jtreg and tonga env. variables in maxMetaspaceSize.sh
- updating ShrinkGrowMultiJVM to skip empty arguments, needed due to difference b/w tonga and jtreg in passing arguments
- propagation of CP in ShrinkGrowMultiJVM
- metaspace.stressHierarchy.common.GenClassesBuilder class which runs metaspace.stressHierarchy.common.HumongousClassGen and compile generated classes
- adding actions to run m.s.c.GenClassesBuilder to the test which use these generated classes -- stressHierarchy tests which have '-triggerUnloadingByFillingMetaspace' flag
- replacing exit(0) w/ exit(95) in StressHierarchyBaseClass
- vm/share/gc/TriggerUnloadingByFillingMetaspace which is used only by metaspace tests have been moved from tonga/src to jtreg/fromTonga directory 
- changes in task definitions
- clean up in test groups, problem list

PS metaspace.compiler classes are used by some other tonga tests, so I had to leave them in tonga/src for now

JBS: https://bugs.openjdk.java.net/browse/JDK-8190499

testing:
- all tonga testlists on linux-x64: http://java.se.oracle.com:10065/mdash/jobs/iignatye-tonga2jtreg-20171102-0054-3914
- converted tests: http://java.se.oracle.com:10065/mdash/jobs/iignatye-tonga2jtreg-20171102-0039-3913

webrevs:
- all changes(uploading): http://javaweb.us.oracle.com/~iignatye//8190499/webrev.01
- automatically generated part: http://javaweb.us.oracle.com/~iignatye/8190499/webrev.00
- manual part: http://javaweb.us.oracle.com/~iignatye/8190499/webrev.0-1

Thanks,
-- Igor

From kim.barrett at oracle.com  Thu Nov  2 03:47:00 2017
From: kim.barrett at oracle.com (Kim Barrett)
Date: Wed, 1 Nov 2017 23:47:00 -0400
Subject: RFR: 8188102: [JVMCI] Convert special JVMCI oops in nmethod to
 jweak values
In-Reply-To: <111F6116-A1A4-4308-BAEC-D471F9E974B9@oracle.com>
References: <94B50A35-9898-47BA-8F20-A829FD7547AC@oracle.com>
 <E0D909B8-A115-4F83-9D0B-12E3FCF7EBEB@oracle.com>
 <BDD0189D-ECBF-42A8-BDDE-2014753B53C9@oracle.com>
 <BABEA69B-939D-4863-9A5B-271F32F330EB@oracle.com>
 <59F99795.6090404@oracle.com>
 <33980406-F96C-434E-AAB7-949B39D46026@oracle.com>
 <111F6116-A1A4-4308-BAEC-D471F9E974B9@oracle.com>
Message-ID: <CD3B456E-8FB6-4295-ADAD-45BEDBD3E81F@oracle.com>

> On Nov 1, 2017, at 5:10 PM, Doug Simon <doug.simon at oracle.com> wrote:
> 
> 
> 
>> On 1 Nov 2017, at 18:57, Kim Barrett <kim.barrett at oracle.com> wrote:
>> 
>>> On Nov 1, 2017, at 5:44 AM, Erik ??sterlund <erik.osterlund at oracle.com> wrote:
>>> 
>>> Hi Kim,
>>> 
>>> On 2017-11-01 06:03, Kim Barrett wrote:
>>>>> On Oct 30, 2017, at 7:14 AM, Doug Simon <doug.simon at oracle.com> wrote:
>>> JNIHandles::resolve with G1 should conditionally enqueue the oops if the SATB queue is activated, which it will not be after marking terminated. At this point, marking should have terminated already, and hence the resolve will not change the liveness of the oop. So I don't quite see how the resolve will change the oop to live here. Am I missing something?
>> 
>> Does this nmethod pass get run during young collections?  If not, then maybe you are right that it doesn???t matter.
>> 
>> But I still think that a cleanup pass that is based on using weak references like this should just be properly ordered
>> wrto weak reference processing, as it presumably already was.  In which case the resolve and can_unload is
>> effectively useless; either the jweak has been cleared and we won???t reach this clause, or the jweak has not been
>> cleared because the referent is still live, in which case can_unload will always return false and there was no point
>> in calling it.
> 
> I agree. As stated previously, my (defensive) code is based on assuming that nmethod unloading might happen before reference processing. If it never/rarely happens, then your suggested change is the right one. I did some extra testing and never saw it happen. Based on that, I'd like to adopt your solution. In the worst case, nmethod unloading will be delayed one full GC cycle if nmethod unloading precedes reference processing.
> 
> I've created another rev based on this:
> 
> http://cr.openjdk.java.net/~dnsimon/8188102_3
> 
> -Doug

This looks good.

------------------------------------------------------------------------------ 
src/hotspot/share/runtime/jniHandles.cpp

I'd like is_global_weak_cleared moved up a few lines, immediately
following the specializations of resolve_jweak, which I think it is
closely related to.

I don't need another webrev for this.

------------------------------------------------------------------------------ 
src/hotspot/share/code/nmethod.cpp

2934 char* nmethod::jvmci_installed_code_name(char* buf, size_t buflen) {
2935   if (!this->is_compiled_by_jvmci()) {
2936     return NULL;
2937   }
2938   oop installed_code = JNIHandles::resolve(_jvmci_installed_code);

Are there any circumstances where it would be unfortunate to have
getting the name keep alive the object? I was thinking logging. Or
perhaps it's a feature that getting the name might artificially extend
the lifetime? Looking at the callers, it doesn't seem like a problem
right now.

This is one of those places where a "peek" access (coming soon from
the GC interface) might be appropriate.

------------------------------------------------------------------------------


X-sender: <hotspot-gc-dev-bounces at openjdk.java.net>
X-Receiver: <rene.schuenemann at sap.com>
X-CreatedBy: MSExchange15
X-HeloDomain: mx.expurgate.net
X-ExtendedProps: BQBjAAoAOtZ9Vtsd1QgFADcAAgAADwA8AAAATWljcm9zb2Z0LkV4Y2hhbmdlLlRyYW5zcG9ydC5NYWlsUmVjaXBpZW50Lk9yZ2FuaXphdGlvblNjb3BlEQAAAAAAAAAAAAAAAAAAAAAADwAJAAAAQ0lBdWRpdGVkAgABBQBJAAIAAQUAYgAKAN8nAAB1AAAABQBkAA8ABAAAAEVkZ2UX-Source: SMTP:External Receive Connector
X-SourceIPAddress: 194.145.224.110
X-EndOfInjectedXHeaders: 8913
Received: from mx.expurgate.net (194.145.224.110) by smtpgw03.sap-ag.de
 (155.56.66.98) with Microsoft SMTP Server (TLS) id 15.0.1210.3; Thu, 2 Nov
 2017 04:48:05 +0100
Received: from mx.expurgate.net (helo=localhost)
	by mx.expurgate.net with esmtp
	id 1eA6U9-0007UQ-7z
	for rene.schuenemann at sap.com; Thu, 02 Nov 2017 04:48:05 +0100
Received: from [141.146.126.229] (helo?sinet41.oracle.com)
	by mxtls.expurgate.net with ESMTPS (eXpurgate 4.3.1)
	(envelope-from <hotspot-gc-dev-bounces at openjdk.java.net>)
	id 59fa956e-036c-c0a8039809dd-8d927ee532db-3
	for <multiple-recipients>; Thu, 02 Nov 2017 04:48:04 +0100
Received: from aojmv0009 (unknown [137.254.59.6]) by acsinet41.oracle.com with smtp
	 id 645c_08a3_eef3c025_1258_41a3_ab9b_b459ba183827;
	Thu, 02 Nov 2017 03:47:40 +0000
Received: from aojmv0009.oracle.com (localhost [127.0.0.1])
	by aojmv0009 (Postfix) with ESMTP id 1EBDE229BAC;
	Thu,  2 Nov 2017 03:50:23 +0000 (UTC)
X-Original-To: hotspot-gc-dev at openjdk.java.net
Delivered-To: hotspot-gc-dev at openjdk.java.net
Received: from acsinet40.oracle.com (acsinet40.oracle.com [141.146.126.228])
 by aojmv0009 (Postfix) with ESMTP id 4ECB7229B7C;
 Thu,  2 Nov 2017 03:50:19 +0000 (UTC)
Received: from aserp1040.oracle.com (unknown [141.146.126.69]) by
 acsinet40.oracle.com with smtp
 (TLS: TLSv1/SSLv3,256bits,DHE-RSA-AES256-GCM-SHA384)
 id 4a75_ed4c_f697796b_06c7_46c7_9bfb_97aa86676c35;
 Thu, 02 Nov 2017 03:47:07 +0000
Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234])
 by aserp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id
 vA23l7mg021860
 (version=TLSv1.2 cipher?DHE-RSA-AES256-GCM-SHA384 bits%6 verify=OK);
 Thu, 2 Nov 2017 03:47:07 GMT
Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72])
 by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id vA23l7jf021541
 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits%6 verify=OK);
 Thu, 2 Nov 2017 03:47:07 GMT
Received: from abhmp0016.oracle.com (abhmp0016.oracle.com [141.146.116.22])
 by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id vA23l6ae016734;
 Thu, 2 Nov 2017 03:47:06 GMT
Received: from dhcp-10-152-15-141.usdhcp.oraclecorp.com (/10.152.15.141)
 by default (Oracle Beehive Gateway v4.0)
 with ESMTP ; Wed, 01 Nov 2017 20:47:06 -0700
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0 (Mac OS X Mail 10.3 \(3273\))
Subject: Re: RFR: 8188102: [JVMCI] Convert special JVMCI oops in nmethod to
 jweak values
From: Kim Barrett <kim.barrett at oracle.com>
In-Reply-To: <111F6116-A1A4-4308-BAEC-D471F9E974B9 at oracle.com>
Date: Wed, 1 Nov 2017 23:47:00 -0400
Content-Transfer-Encoding: quoted-printable
Message-ID: <CD3B456E-8FB6-4295-ADAD-45BEDBD3E81F at oracle.com>
References: <94B50A35-9898-47BA-8F20-A829FD7547AC at oracle.com>
 <E0D909B8-A115-4F83-9D0B-12E3FCF7EBEB at oracle.com>
 <BDD0189D-ECBF-42A8-BDDE-2014753B53C9 at oracle.com>
 <BABEA69B-939D-4863-9A5B-271F32F330EB at oracle.com>
 <59F99795.6090404 at oracle.com>
 <33980406-F96C-434E-AAB7-949B39D46026 at oracle.com>
 <111F6116-A1A4-4308-BAEC-D471F9E974B9 at oracle.com>
To: Doug Simon <doug.simon at oracle.com>
X-Mailer: Apple Mail (2.3273)
X-Source-IP: aserv0022.oracle.com [141.146.126.234]
CC: hotspot compiler <hotspot-compiler-dev at openjdk.java.net>, hotspot-gc-dev
 openjdk.java.net <hotspot-gc-dev at openjdk.java.net>, Tom Rodriguez
	<tom.rodriguez at oracle.com>
X-BeenThere: hotspot-gc-dev at openjdk.java.net
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: Technical discussion about the development of the HotSpot garbage
 collectors <hotspot-gc-dev.openjdk.java.net>
List-Unsubscribe: <http://mail.openjdk.java.net/mailman/options/hotspot-gc-dev>, 
 <mailto:hotspot-gc-dev-request at openjdk.java.net?subject=unsubscribe>
List-Archive: <http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/>
List-Post: <mailto:hotspot-gc-dev at openjdk.java.net>
List-Help: <mailto:hotspot-gc-dev-request at openjdk.java.net?subject=help>
List-Subscribe: <http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-dev>, 
 <mailto:hotspot-gc-dev-request at openjdk.java.net?subject=subscribe>
Errors-To: hotspot-gc-dev-bounces at openjdk.java.net
Sender: hotspot-gc-dev <hotspot-gc-dev-bounces at openjdk.java.net>
X-purgate-ID: tlsNG-9b2b54/1509594485-0000036C-FB75D426/9/51289
X-purgate-type: clean
X-purgate-size: 3238
X-purgate-Ad: Categorized by eleven eXpurgate (R) http://www.eleven.de
Comment: eleven transport token MTQxLjE0Ni4xMjYuMjI5AGhvdHNwb3QtZ2MtZGV2LWJvdW5jZXNAb3Blbmpkay5qYXZhLm5ldAByZW5lLnNjaHVlbmVtYW5uQHNhcC5jb20AYjM1ZDE3M2UyNjhjNDNjMDRmMjBhNzhiMDY1ZGU1ZTE1ZmM5M2I2ZQ=
X-purgate: This mail is considered clean (visit http://www.eleven.de for further information)
X-purgate: clean
X-SAP-SPAM-Status: clean
Return-Path: hotspot-gc-dev-bounces at openjdk.java.net
X-MS-Exchange-Organization-OriginalArrivalTime: 02 Nov 2017 03:48:05.5655
 (UTC)
X-MS-Exchange-Organization-OriginalClientIPAddress: 194.145.224.110
X-MS-Exchange-Organization-OriginalServerIPAddress: 155.56.66.98
X-MS-Exchange-Organization-AuthSource: DEWDFE13EDGE01.wdf.sap.corp
X-MS-Exchange-Organization-AuthAs: Anonymous
X-MS-Exchange-Organization-Cross-Premises-Headers-Processed: DEWDFE13EDGE01.wdf.sap.corp
X-MS-Exchange-Organization-Network-Message-Id: 179037f5-0281-4084-7800-08d521a48663
X-MS-Exchange-Organization-OriginalSize: 8593
X-MS-Exchange-Organization-HygienePolicy: Standard
X-MS-Exchange-Organization-MessageHighPrecisionLatencyInProgress:
	LSRV?WDFE13EDGE01.wdf.sap.corp:TOTAL-EDGE778.765|UTH=0.001|RST778.765|CATCM-McAfeeTxRoutingAgent=0.001|CATCM=0.002|CAT=0.003;2017-11-02T07:37:44.330Z

> On Nov 1, 2017, at 5:10 PM, Doug Simon <doug.simon at oracle.com> wrote:
> 
> 
> 
>> On 1 Nov 2017, at 18:57, Kim Barrett <kim.barrett at oracle.com> wrote:
>> 
>>> On Nov 1, 2017, at 5:44 AM, Erik ??sterlund <erik.osterlund at oracle.com> wrote:
>>> 
>>> Hi Kim,
>>> 
>>> On 2017-11-01 06:03, Kim Barrett wrote:
>>>>> On Oct 30, 2017, at 7:14 AM, Doug Simon <doug.simon at oracle.com> wrote:
>>> JNIHandles::resolve with G1 should conditionally enqueue the oops if the SATB queue is activated, which it will not be after marking terminated. At this point, marking should have terminated already, and hence the resolve will not change the liveness of the oop. So I don't quite see how the resolve will change the oop to live here. Am I missing something?
>> 
>> Does this nmethod pass get run during young collections?  If not, then maybe you are right that it doesn???t matter.
>> 
>> But I still think that a cleanup pass that is based on using weak references like this should just be properly ordered
>> wrto weak reference processing, as it presumably already was.  In which case the resolve and can_unload is
>> effectively useless; either the jweak has been cleared and we won???t reach this clause, or the jweak has not been
>> cleared because the referent is still live, in which case can_unload will always return false and there was no point
>> in calling it.
> 
> I agree. As stated previously, my (defensive) code is based on assuming that nmethod unloading might happen before reference processing. If it never/rarely happens, then your suggested change is the right one. I did some extra testing and never saw it happen. Based on that, I'd like to adopt your solution. In the worst case, nmethod unloading will be delayed one full GC cycle if nmethod unloading precedes reference processing.
> 
> I've created another rev based on this:
> 
> http://cr.openjdk.java.net/~dnsimon/8188102_3
> 
> -Doug

This looks good.

------------------------------------------------------------------------------ 
src/hotspot/share/runtime/jniHandles.cpp

I'd like is_global_weak_cleared moved up a few lines, immediately
following the specializations of resolve_jweak, which I think it is
closely related to.

I don't need another webrev for this.

------------------------------------------------------------------------------ 
src/hotspot/share/code/nmethod.cpp

2934 char* nmethod::jvmci_installed_code_name(char* buf, size_t buflen) {
2935   if (!this->is_compiled_by_jvmci()) {
2936     return NULL;
2937   }
2938   oop installed_code = JNIHandles::resolve(_jvmci_installed_code);

Are there any circumstances where it would be unfortunate to have
getting the name keep alive the object? I was thinking logging. Or
perhaps it's a feature that getting the name might artificially extend
the lifetime? Looking at the callers, it doesn't seem like a problem
right now.

This is one of those places where a "peek" access (coming soon from
the GC interface) might be appropriate.

------------------------------------------------------------------------------


X-sender: <hotspot-gc-dev-bounces at openjdk.java.net>
X-Receiver: <gunter.haug at sap.com>
X-CreatedBy: MSExchange15
X-HeloDomain: mx.expurgate.net
X-ExtendedProps: BQBjAAoAONZ9Vtsd1QgFADcAAgAADwA8AAAATWljcm9zb2Z0LkV4Y2hhbmdlLlRyYW5zcG9ydC5NYWlsUmVjaXBpZW50Lk9yZ2FuaXphdGlvblNjb3BlEQAAAAAAAAAAAAAAAAAAAAAADwAJAAAAQ0lBdWRpdGVkAgABBQBJAAIAAQUAYgAKAOUnAAB1AAAABQBkAA8ABAAAAEVkZ2UX-Source: SMTP:External Receive Connector
X-SourceIPAddress: 194.145.224.110
X-EndOfInjectedXHeaders: 8900
Received: from mx.expurgate.net (194.145.224.110) by smtpgw03.sap-ag.de
 (155.56.66.98) with Microsoft SMTP Server (TLS) id 15.0.1210.3; Thu, 2 Nov
 2017 04:48:07 +0100
Received: from mx.expurgate.net (helo=localhost)
	by mx.expurgate.net with esmtp
	id 1eA6U9-0007UQ-6v
	for gunter.haug at sap.com; Thu, 02 Nov 2017 04:48:05 +0100
Received: from [141.146.126.229] (helo?sinet41.oracle.com)
	by mxtls.expurgate.net with ESMTPS (eXpurgate 4.3.1)
	(envelope-from <hotspot-gc-dev-bounces at openjdk.java.net>)
	id 59fa956e-036c-c0a8039809dd-8d927ee532db-3
	for <multiple-recipients>; Thu, 02 Nov 2017 04:48:04 +0100
Received: from aojmv0009 (unknown [137.254.59.6]) by acsinet41.oracle.com with smtp
	 id 645c_08a3_eef3c025_1258_41a3_ab9b_b459ba183827;
	Thu, 02 Nov 2017 03:47:40 +0000
Received: from aojmv0009.oracle.com (localhost [127.0.0.1])
	by aojmv0009 (Postfix) with ESMTP id 1EBDE229BAC;
	Thu,  2 Nov 2017 03:50:23 +0000 (UTC)
X-Original-To: hotspot-gc-dev at openjdk.java.net
Delivered-To: hotspot-gc-dev at openjdk.java.net
Received: from acsinet40.oracle.com (acsinet40.oracle.com [141.146.126.228])
 by aojmv0009 (Postfix) with ESMTP id 4ECB7229B7C;
 Thu,  2 Nov 2017 03:50:19 +0000 (UTC)
Received: from aserp1040.oracle.com (unknown [141.146.126.69]) by
 acsinet40.oracle.com with smtp
 (TLS: TLSv1/SSLv3,256bits,DHE-RSA-AES256-GCM-SHA384)
 id 4a75_ed4c_f697796b_06c7_46c7_9bfb_97aa86676c35;
 Thu, 02 Nov 2017 03:47:07 +0000
Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234])
 by aserp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id
 vA23l7mg021860
 (version=TLSv1.2 cipher?DHE-RSA-AES256-GCM-SHA384 bits%6 verify=OK);
 Thu, 2 Nov 2017 03:47:07 GMT
Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72])
 by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id vA23l7jf021541
 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits%6 verify=OK);
 Thu, 2 Nov 2017 03:47:07 GMT
Received: from abhmp0016.oracle.com (abhmp0016.oracle.com [141.146.116.22])
 by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id vA23l6ae016734;
 Thu, 2 Nov 2017 03:47:06 GMT
Received: from dhcp-10-152-15-141.usdhcp.oraclecorp.com (/10.152.15.141)
 by default (Oracle Beehive Gateway v4.0)
 with ESMTP ; Wed, 01 Nov 2017 20:47:06 -0700
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0 (Mac OS X Mail 10.3 \(3273\))
Subject: Re: RFR: 8188102: [JVMCI] Convert special JVMCI oops in nmethod to
 jweak values
From: Kim Barrett <kim.barrett at oracle.com>
In-Reply-To: <111F6116-A1A4-4308-BAEC-D471F9E974B9 at oracle.com>
Date: Wed, 1 Nov 2017 23:47:00 -0400
Content-Transfer-Encoding: quoted-printable
Message-ID: <CD3B456E-8FB6-4295-ADAD-45BEDBD3E81F at oracle.com>
References: <94B50A35-9898-47BA-8F20-A829FD7547AC at oracle.com>
 <E0D909B8-A115-4F83-9D0B-12E3FCF7EBEB at oracle.com>
 <BDD0189D-ECBF-42A8-BDDE-2014753B53C9 at oracle.com>
 <BABEA69B-939D-4863-9A5B-271F32F330EB at oracle.com>
 <59F99795.6090404 at oracle.com>
 <33980406-F96C-434E-AAB7-949B39D46026 at oracle.com>
 <111F6116-A1A4-4308-BAEC-D471F9E974B9 at oracle.com>
To: Doug Simon <doug.simon at oracle.com>
X-Mailer: Apple Mail (2.3273)
X-Source-IP: aserv0022.oracle.com [141.146.126.234]
CC: hotspot compiler <hotspot-compiler-dev at openjdk.java.net>, hotspot-gc-dev
 openjdk.java.net <hotspot-gc-dev at openjdk.java.net>, Tom Rodriguez
	<tom.rodriguez at oracle.com>
X-BeenThere: hotspot-gc-dev at openjdk.java.net
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: Technical discussion about the development of the HotSpot garbage
 collectors <hotspot-gc-dev.openjdk.java.net>
List-Unsubscribe: <http://mail.openjdk.java.net/mailman/options/hotspot-gc-dev>, 
 <mailto:hotspot-gc-dev-request at openjdk.java.net?subject=unsubscribe>
List-Archive: <http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/>
List-Post: <mailto:hotspot-gc-dev at openjdk.java.net>
List-Help: <mailto:hotspot-gc-dev-request at openjdk.java.net?subject=help>
List-Subscribe: <http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-dev>, 
 <mailto:hotspot-gc-dev-request at openjdk.java.net?subject=subscribe>
Errors-To: hotspot-gc-dev-bounces at openjdk.java.net
Sender: hotspot-gc-dev <hotspot-gc-dev-bounces at openjdk.java.net>
X-purgate-ID: tlsNG-9b2b54/1509594485-0000036C-FB75D426/9/51289
X-purgate-type: clean
X-purgate-size: 3238
X-purgate-Ad: Categorized by eleven eXpurgate (R) http://www.eleven.de
Comment: eleven transport token MTQxLjE0Ni4xMjYuMjI5AGhvdHNwb3QtZ2MtZGV2LWJvdW5jZXNAb3Blbmpkay5qYXZhLm5ldABndW50ZXIuaGF1Z0BzYXAuY29tAGZlZTE1ZDY4ODdjMjM2Zjc5NDNiNDAwNzc4ZTdkODM2YzhjZGIzZGMX-purgate: This mail is considered clean (visit http://www.eleven.de for further information)
X-purgate: clean
X-SAP-SPAM-Status: clean
Return-Path: hotspot-gc-dev-bounces at openjdk.java.net
X-MS-Exchange-Organization-OriginalArrivalTime: 02 Nov 2017 03:48:07.7218
 (UTC)
X-MS-Exchange-Organization-OriginalClientIPAddress: 194.145.224.110
X-MS-Exchange-Organization-OriginalServerIPAddress: 155.56.66.98
X-MS-Exchange-Organization-AuthSource: DEWDFE13EDGE01.wdf.sap.corp
X-MS-Exchange-Organization-AuthAs: Anonymous
X-MS-Exchange-Organization-Cross-Premises-Headers-Processed: DEWDFE13EDGE01.wdf.sap.corp
X-MS-Exchange-Organization-Network-Message-Id: fd121385-9359-4a1d-7c94-08d521a487ac
X-MS-Exchange-Organization-OriginalSize: 8580
X-MS-Exchange-Organization-HygienePolicy: Standard
X-MS-Exchange-Organization-MessageHighPrecisionLatencyInProgress:
	LSRV?WDFE13EDGE01.wdf.sap.corp:TOTAL-EDGE776.609|UTH=0.001|RST776.609|CATCM-McAfeeTxRoutingAgent=0.001|CATCM=0.001|CAT=0.002;2017-11-02T07:37:44.330Z

> On Nov 1, 2017, at 5:10 PM, Doug Simon <doug.simon at oracle.com> wrote:
> 
> 
> 
>> On 1 Nov 2017, at 18:57, Kim Barrett <kim.barrett at oracle.com> wrote:
>> 
>>> On Nov 1, 2017, at 5:44 AM, Erik ??sterlund <erik.osterlund at oracle.com> wrote:
>>> 
>>> Hi Kim,
>>> 
>>> On 2017-11-01 06:03, Kim Barrett wrote:
>>>>> On Oct 30, 2017, at 7:14 AM, Doug Simon <doug.simon at oracle.com> wrote:
>>> JNIHandles::resolve with G1 should conditionally enqueue the oops if the SATB queue is activated, which it will not be after marking terminated. At this point, marking should have terminated already, and hence the resolve will not change the liveness of the oop. So I don't quite see how the resolve will change the oop to live here. Am I missing something?
>> 
>> Does this nmethod pass get run during young collections?  If not, then maybe you are right that it doesn???t matter.
>> 
>> But I still think that a cleanup pass that is based on using weak references like this should just be properly ordered
>> wrto weak reference processing, as it presumably already was.  In which case the resolve and can_unload is
>> effectively useless; either the jweak has been cleared and we won???t reach this clause, or the jweak has not been
>> cleared because the referent is still live, in which case can_unload will always return false and there was no point
>> in calling it.
> 
> I agree. As stated previously, my (defensive) code is based on assuming that nmethod unloading might happen before reference processing. If it never/rarely happens, then your suggested change is the right one. I did some extra testing and never saw it happen. Based on that, I'd like to adopt your solution. In the worst case, nmethod unloading will be delayed one full GC cycle if nmethod unloading precedes reference processing.
> 
> I've created another rev based on this:
> 
> http://cr.openjdk.java.net/~dnsimon/8188102_3
> 
> -Doug

This looks good.

------------------------------------------------------------------------------ 
src/hotspot/share/runtime/jniHandles.cpp

I'd like is_global_weak_cleared moved up a few lines, immediately
following the specializations of resolve_jweak, which I think it is
closely related to.

I don't need another webrev for this.

------------------------------------------------------------------------------ 
src/hotspot/share/code/nmethod.cpp

2934 char* nmethod::jvmci_installed_code_name(char* buf, size_t buflen) {
2935   if (!this->is_compiled_by_jvmci()) {
2936     return NULL;
2937   }
2938   oop installed_code = JNIHandles::resolve(_jvmci_installed_code);

Are there any circumstances where it would be unfortunate to have
getting the name keep alive the object? I was thinking logging. Or
perhaps it's a feature that getting the name might artificially extend
the lifetime? Looking at the callers, it doesn't seem like a problem
right now.

This is one of those places where a "peek" access (coming soon from
the GC interface) might be appropriate.

------------------------------------------------------------------------------




From kim.barrett at oracle.com  Thu Nov  2 03:47:00 2017
From: kim.barrett at oracle.com (Kim Barrett)
Date: Wed, 1 Nov 2017 23:47:00 -0400
Subject: RFR: 8188102: [JVMCI] Convert special JVMCI oops in nmethod to
 jweak values
In-Reply-To: <111F6116-A1A4-4308-BAEC-D471F9E974B9@oracle.com>
References: <94B50A35-9898-47BA-8F20-A829FD7547AC@oracle.com>
 <E0D909B8-A115-4F83-9D0B-12E3FCF7EBEB@oracle.com>
 <BDD0189D-ECBF-42A8-BDDE-2014753B53C9@oracle.com>
 <BABEA69B-939D-4863-9A5B-271F32F330EB@oracle.com>
 <59F99795.6090404@oracle.com>
 <33980406-F96C-434E-AAB7-949B39D46026@oracle.com>
 <111F6116-A1A4-4308-BAEC-D471F9E974B9@oracle.com>
Message-ID: <CD3B456E-8FB6-4295-ADAD-45BEDBD3E81F@oracle.com>

> On Nov 1, 2017, at 5:10 PM, Doug Simon <doug.simon at oracle.com> wrote:
> 
> 
> 
>> On 1 Nov 2017, at 18:57, Kim Barrett <kim.barrett at oracle.com> wrote:
>> 
>>> On Nov 1, 2017, at 5:44 AM, Erik ??sterlund <erik.osterlund at oracle.com> wrote:
>>> 
>>> Hi Kim,
>>> 
>>> On 2017-11-01 06:03, Kim Barrett wrote:
>>>>> On Oct 30, 2017, at 7:14 AM, Doug Simon <doug.simon at oracle.com> wrote:
>>> JNIHandles::resolve with G1 should conditionally enqueue the oops if the SATB queue is activated, which it will not be after marking terminated. At this point, marking should have terminated already, and hence the resolve will not change the liveness of the oop. So I don't quite see how the resolve will change the oop to live here. Am I missing something?
>> 
>> Does this nmethod pass get run during young collections?  If not, then maybe you are right that it doesn???t matter.
>> 
>> But I still think that a cleanup pass that is based on using weak references like this should just be properly ordered
>> wrto weak reference processing, as it presumably already was.  In which case the resolve and can_unload is
>> effectively useless; either the jweak has been cleared and we won???t reach this clause, or the jweak has not been
>> cleared because the referent is still live, in which case can_unload will always return false and there was no point
>> in calling it.
> 
> I agree. As stated previously, my (defensive) code is based on assuming that nmethod unloading might happen before reference processing. If it never/rarely happens, then your suggested change is the right one. I did some extra testing and never saw it happen. Based on that, I'd like to adopt your solution. In the worst case, nmethod unloading will be delayed one full GC cycle if nmethod unloading precedes reference processing.
> 
> I've created another rev based on this:
> 
> http://cr.openjdk.java.net/~dnsimon/8188102_3
> 
> -Doug

This looks good.

------------------------------------------------------------------------------ 
src/hotspot/share/runtime/jniHandles.cpp

I'd like is_global_weak_cleared moved up a few lines, immediately
following the specializations of resolve_jweak, which I think it is
closely related to.

I don't need another webrev for this.

------------------------------------------------------------------------------ 
src/hotspot/share/code/nmethod.cpp

2934 char* nmethod::jvmci_installed_code_name(char* buf, size_t buflen) {
2935   if (!this->is_compiled_by_jvmci()) {
2936     return NULL;
2937   }
2938   oop installed_code = JNIHandles::resolve(_jvmci_installed_code);

Are there any circumstances where it would be unfortunate to have
getting the name keep alive the object? I was thinking logging. Or
perhaps it's a feature that getting the name might artificially extend
the lifetime? Looking at the callers, it doesn't seem like a problem
right now.

This is one of those places where a "peek" access (coming soon from
the GC interface) might be appropriate.

------------------------------------------------------------------------------


X-sender: <hotspot-gc-dev-bounces at openjdk.java.net>
X-Receiver: <rene.schuenemann at sap.com>
X-CreatedBy: MSExchange15
X-HeloDomain: mx.expurgate.net
X-ExtendedProps: BQBjAAoAOtZ9Vtsd1QgFADcAAgAADwA8AAAATWljcm9zb2Z0LkV4Y2hhbmdlLlRyYW5zcG9ydC5NYWlsUmVjaXBpZW50Lk9yZ2FuaXphdGlvblNjb3BlEQAAAAAAAAAAAAAAAAAAAAAADwAJAAAAQ0lBdWRpdGVkAgABBQBJAAIAAQUAYgAKAN8nAAB1AAAABQBkAA8ABAAAAEVkZ2UX-Source: SMTP:External Receive Connector
X-SourceIPAddress: 194.145.224.110
X-EndOfInjectedXHeaders: 8915
Received: from mx.expurgate.net (194.145.224.110) by smtpgw03.sap-ag.de
 (155.56.66.98) with Microsoft SMTP Server (TLS) id 15.0.1210.3; Thu, 2 Nov
 2017 04:48:05 +0100
Received: from mx.expurgate.net (helo=localhost)
	by mx.expurgate.net with esmtp
	id 1eA6U9-0007UQ-7z
	for rene.schuenemann at sap.com; Thu, 02 Nov 2017 04:48:05 +0100
Received: from [141.146.126.229] (helo?sinet41.oracle.com)
	by mxtls.expurgate.net with ESMTPS (eXpurgate 4.3.1)
	(envelope-from <hotspot-gc-dev-bounces at openjdk.java.net>)
	id 59fa956e-036c-c0a8039809dd-8d927ee532db-3
	for <multiple-recipients>; Thu, 02 Nov 2017 04:48:04 +0100
Received: from aojmv0009 (unknown [137.254.59.6]) by acsinet41.oracle.com with smtp
	 id 645c_08a3_eef3c025_1258_41a3_ab9b_b459ba183827;
	Thu, 02 Nov 2017 03:47:40 +0000
Received: from aojmv0009.oracle.com (localhost [127.0.0.1])
	by aojmv0009 (Postfix) with ESMTP id 1EBDE229BAC;
	Thu,  2 Nov 2017 03:50:23 +0000 (UTC)
X-Original-To: hotspot-gc-dev at openjdk.java.net
Delivered-To: hotspot-gc-dev at openjdk.java.net
Received: from acsinet40.oracle.com (acsinet40.oracle.com [141.146.126.228])
 by aojmv0009 (Postfix) with ESMTP id 4ECB7229B7C;
 Thu,  2 Nov 2017 03:50:19 +0000 (UTC)
Received: from aserp1040.oracle.com (unknown [141.146.126.69]) by
 acsinet40.oracle.com with smtp
 (TLS: TLSv1/SSLv3,256bits,DHE-RSA-AES256-GCM-SHA384)
 id 4a75_ed4c_f697796b_06c7_46c7_9bfb_97aa86676c35;
 Thu, 02 Nov 2017 03:47:07 +0000
Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234])
 by aserp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id
 vA23l7mg021860
 (version=TLSv1.2 cipher?DHE-RSA-AES256-GCM-SHA384 bits%6 verify=OK);
 Thu, 2 Nov 2017 03:47:07 GMT
Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72])
 by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id vA23l7jf021541
 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits%6 verify=OK);
 Thu, 2 Nov 2017 03:47:07 GMT
Received: from abhmp0016.oracle.com (abhmp0016.oracle.com [141.146.116.22])
 by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id vA23l6ae016734;
 Thu, 2 Nov 2017 03:47:06 GMT
Received: from dhcp-10-152-15-141.usdhcp.oraclecorp.com (/10.152.15.141)
 by default (Oracle Beehive Gateway v4.0)
 with ESMTP ; Wed, 01 Nov 2017 20:47:06 -0700
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0 (Mac OS X Mail 10.3 \(3273\))
Subject: Re: RFR: 8188102: [JVMCI] Convert special JVMCI oops in nmethod to
 jweak values
From: Kim Barrett <kim.barrett at oracle.com>
In-Reply-To: <111F6116-A1A4-4308-BAEC-D471F9E974B9 at oracle.com>
Date: Wed, 1 Nov 2017 23:47:00 -0400
Content-Transfer-Encoding: quoted-printable
Message-ID: <CD3B456E-8FB6-4295-ADAD-45BEDBD3E81F at oracle.com>
References: <94B50A35-9898-47BA-8F20-A829FD7547AC at oracle.com>
 <E0D909B8-A115-4F83-9D0B-12E3FCF7EBEB at oracle.com>
 <BDD0189D-ECBF-42A8-BDDE-2014753B53C9 at oracle.com>
 <BABEA69B-939D-4863-9A5B-271F32F330EB at oracle.com>
 <59F99795.6090404 at oracle.com>
 <33980406-F96C-434E-AAB7-949B39D46026 at oracle.com>
 <111F6116-A1A4-4308-BAEC-D471F9E974B9 at oracle.com>
To: Doug Simon <doug.simon at oracle.com>
X-Mailer: Apple Mail (2.3273)
X-Source-IP: aserv0022.oracle.com [141.146.126.234]
CC: hotspot compiler <hotspot-compiler-dev at openjdk.java.net>, "hotspot-gc-dev
 openjdk.java.net" <hotspot-gc-dev at openjdk.java.net>, Tom Rodriguez
	<tom.rodriguez at oracle.com>
X-BeenThere: hotspot-gc-dev at openjdk.java.net
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: Technical discussion about the development of the HotSpot garbage
 collectors <hotspot-gc-dev.openjdk.java.net>
List-Unsubscribe: <http://mail.openjdk.java.net/mailman/options/hotspot-gc-dev>, 
 <mailto:hotspot-gc-dev-request at openjdk.java.net?subject=unsubscribe>
List-Archive: <http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/>
List-Post: <mailto:hotspot-gc-dev at openjdk.java.net>
List-Help: <mailto:hotspot-gc-dev-request at openjdk.java.net?subject=help>
List-Subscribe: <http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-dev>, 
 <mailto:hotspot-gc-dev-request at openjdk.java.net?subject=subscribe>
Errors-To: hotspot-gc-dev-bounces at openjdk.java.net
Sender: hotspot-gc-dev <hotspot-gc-dev-bounces at openjdk.java.net>
X-purgate-ID: tlsNG-9b2b54/1509594485-0000036C-FB75D426/9/51289
X-purgate-type: clean
X-purgate-size: 3238
X-purgate-Ad: Categorized by eleven eXpurgate (R) http://www.eleven.de
Comment: eleven transport token MTQxLjE0Ni4xMjYuMjI5AGhvdHNwb3QtZ2MtZGV2LWJvdW5jZXNAb3Blbmpkay5qYXZhLm5ldAByZW5lLnNjaHVlbmVtYW5uQHNhcC5jb20AYjM1ZDE3M2UyNjhjNDNjMDRmMjBhNzhiMDY1ZGU1ZTE1ZmM5M2I2ZQ=
X-purgate: This mail is considered clean (visit http://www.eleven.de for further information)
X-purgate: clean
X-SAP-SPAM-Status: clean
Return-Path: hotspot-gc-dev-bounces at openjdk.java.net
X-MS-Exchange-Organization-OriginalArrivalTime: 02 Nov 2017 03:48:05.5655
 (UTC)
X-MS-Exchange-Organization-OriginalClientIPAddress: 194.145.224.110
X-MS-Exchange-Organization-OriginalServerIPAddress: 155.56.66.98
X-MS-Exchange-Organization-AuthSource: DEWDFE13EDGE01.wdf.sap.corp
X-MS-Exchange-Organization-AuthAs: Anonymous
X-MS-Exchange-Organization-Cross-Premises-Headers-Processed: DEWDFE13EDGE01.wdf.sap.corp
X-MS-Exchange-Organization-Network-Message-Id: 179037f5-0281-4084-7800-08d521a48663
X-MS-Exchange-Organization-OriginalSize: 8593
X-MS-Exchange-Organization-HygienePolicy: Standard
X-MS-Exchange-Organization-MessageHighPrecisionLatencyInProgress:
	LSRV?WDFE13EDGE01.wdf.sap.corp:TOTAL-EDGE959.013|UTH=0.001|RST959.013|CATCM-McAfeeTxRoutingAgent=0.002|CATCM=0.003|CAT=0.004;2017-11-02T08:14:04.578Z

> On Nov 1, 2017, at 5:10 PM, Doug Simon <doug.simon at oracle.com> wrote:
> 
> 
> 
>> On 1 Nov 2017, at 18:57, Kim Barrett <kim.barrett at oracle.com> wrote:
>> 
>>> On Nov 1, 2017, at 5:44 AM, Erik ??sterlund <erik.osterlund at oracle.com> wrote:
>>> 
>>> Hi Kim,
>>> 
>>> On 2017-11-01 06:03, Kim Barrett wrote:
>>>>> On Oct 30, 2017, at 7:14 AM, Doug Simon <doug.simon at oracle.com> wrote:
>>> JNIHandles::resolve with G1 should conditionally enqueue the oops if the SATB queue is activated, which it will not be after marking terminated. At this point, marking should have terminated already, and hence the resolve will not change the liveness of the oop. So I don't quite see how the resolve will change the oop to live here. Am I missing something?
>> 
>> Does this nmethod pass get run during young collections?  If not, then maybe you are right that it doesn???t matter.
>> 
>> But I still think that a cleanup pass that is based on using weak references like this should just be properly ordered
>> wrto weak reference processing, as it presumably already was.  In which case the resolve and can_unload is
>> effectively useless; either the jweak has been cleared and we won???t reach this clause, or the jweak has not been
>> cleared because the referent is still live, in which case can_unload will always return false and there was no point
>> in calling it.
> 
> I agree. As stated previously, my (defensive) code is based on assuming that nmethod unloading might happen before reference processing. If it never/rarely happens, then your suggested change is the right one. I did some extra testing and never saw it happen. Based on that, I'd like to adopt your solution. In the worst case, nmethod unloading will be delayed one full GC cycle if nmethod unloading precedes reference processing.
> 
> I've created another rev based on this:
> 
> http://cr.openjdk.java.net/~dnsimon/8188102_3
> 
> -Doug

This looks good.

------------------------------------------------------------------------------ 
src/hotspot/share/runtime/jniHandles.cpp

I'd like is_global_weak_cleared moved up a few lines, immediately
following the specializations of resolve_jweak, which I think it is
closely related to.

I don't need another webrev for this.

------------------------------------------------------------------------------ 
src/hotspot/share/code/nmethod.cpp

2934 char* nmethod::jvmci_installed_code_name(char* buf, size_t buflen) {
2935   if (!this->is_compiled_by_jvmci()) {
2936     return NULL;
2937   }
2938   oop installed_code = JNIHandles::resolve(_jvmci_installed_code);

Are there any circumstances where it would be unfortunate to have
getting the name keep alive the object? I was thinking logging. Or
perhaps it's a feature that getting the name might artificially extend
the lifetime? Looking at the callers, it doesn't seem like a problem
right now.

This is one of those places where a "peek" access (coming soon from
the GC interface) might be appropriate.

------------------------------------------------------------------------------


X-sender: <hotspot-gc-dev-bounces at openjdk.java.net>
X-Receiver: <gunter.haug at sap.com>
X-CreatedBy: MSExchange15
X-HeloDomain: mx.expurgate.net
X-ExtendedProps: BQBjAAoAONZ9Vtsd1QgFADcAAgAADwA8AAAATWljcm9zb2Z0LkV4Y2hhbmdlLlRyYW5zcG9ydC5NYWlsUmVjaXBpZW50Lk9yZ2FuaXphdGlvblNjb3BlEQAAAAAAAAAAAAAAAAAAAAAADwAJAAAAQ0lBdWRpdGVkAgABBQBJAAIAAQUAYgAKAOUnAAB1AAAABQBkAA8ABAAAAEVkZ2UX-Source: SMTP:External Receive Connector
X-SourceIPAddress: 194.145.224.110
X-EndOfInjectedXHeaders: 8902
Received: from mx.expurgate.net (194.145.224.110) by smtpgw03.sap-ag.de
 (155.56.66.98) with Microsoft SMTP Server (TLS) id 15.0.1210.3; Thu, 2 Nov
 2017 04:48:07 +0100
Received: from mx.expurgate.net (helo=localhost)
	by mx.expurgate.net with esmtp
	id 1eA6U9-0007UQ-6v
	for gunter.haug at sap.com; Thu, 02 Nov 2017 04:48:05 +0100
Received: from [141.146.126.229] (helo?sinet41.oracle.com)
	by mxtls.expurgate.net with ESMTPS (eXpurgate 4.3.1)
	(envelope-from <hotspot-gc-dev-bounces at openjdk.java.net>)
	id 59fa956e-036c-c0a8039809dd-8d927ee532db-3
	for <multiple-recipients>; Thu, 02 Nov 2017 04:48:04 +0100
Received: from aojmv0009 (unknown [137.254.59.6]) by acsinet41.oracle.com with smtp
	 id 645c_08a3_eef3c025_1258_41a3_ab9b_b459ba183827;
	Thu, 02 Nov 2017 03:47:40 +0000
Received: from aojmv0009.oracle.com (localhost [127.0.0.1])
	by aojmv0009 (Postfix) with ESMTP id 1EBDE229BAC;
	Thu,  2 Nov 2017 03:50:23 +0000 (UTC)
X-Original-To: hotspot-gc-dev at openjdk.java.net
Delivered-To: hotspot-gc-dev at openjdk.java.net
Received: from acsinet40.oracle.com (acsinet40.oracle.com [141.146.126.228])
 by aojmv0009 (Postfix) with ESMTP id 4ECB7229B7C;
 Thu,  2 Nov 2017 03:50:19 +0000 (UTC)
Received: from aserp1040.oracle.com (unknown [141.146.126.69]) by
 acsinet40.oracle.com with smtp
 (TLS: TLSv1/SSLv3,256bits,DHE-RSA-AES256-GCM-SHA384)
 id 4a75_ed4c_f697796b_06c7_46c7_9bfb_97aa86676c35;
 Thu, 02 Nov 2017 03:47:07 +0000
Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234])
 by aserp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id
 vA23l7mg021860
 (version=TLSv1.2 cipher?DHE-RSA-AES256-GCM-SHA384 bits%6 verify=OK);
 Thu, 2 Nov 2017 03:47:07 GMT
Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72])
 by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id vA23l7jf021541
 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits%6 verify=OK);
 Thu, 2 Nov 2017 03:47:07 GMT
Received: from abhmp0016.oracle.com (abhmp0016.oracle.com [141.146.116.22])
 by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id vA23l6ae016734;
 Thu, 2 Nov 2017 03:47:06 GMT
Received: from dhcp-10-152-15-141.usdhcp.oraclecorp.com (/10.152.15.141)
 by default (Oracle Beehive Gateway v4.0)
 with ESMTP ; Wed, 01 Nov 2017 20:47:06 -0700
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0 (Mac OS X Mail 10.3 \(3273\))
Subject: Re: RFR: 8188102: [JVMCI] Convert special JVMCI oops in nmethod to
 jweak values
From: Kim Barrett <kim.barrett at oracle.com>
In-Reply-To: <111F6116-A1A4-4308-BAEC-D471F9E974B9 at oracle.com>
Date: Wed, 1 Nov 2017 23:47:00 -0400
Content-Transfer-Encoding: quoted-printable
Message-ID: <CD3B456E-8FB6-4295-ADAD-45BEDBD3E81F at oracle.com>
References: <94B50A35-9898-47BA-8F20-A829FD7547AC at oracle.com>
 <E0D909B8-A115-4F83-9D0B-12E3FCF7EBEB at oracle.com>
 <BDD0189D-ECBF-42A8-BDDE-2014753B53C9 at oracle.com>
 <BABEA69B-939D-4863-9A5B-271F32F330EB at oracle.com>
 <59F99795.6090404 at oracle.com>
 <33980406-F96C-434E-AAB7-949B39D46026 at oracle.com>
 <111F6116-A1A4-4308-BAEC-D471F9E974B9 at oracle.com>
To: Doug Simon <doug.simon at oracle.com>
X-Mailer: Apple Mail (2.3273)
X-Source-IP: aserv0022.oracle.com [141.146.126.234]
CC: hotspot compiler <hotspot-compiler-dev at openjdk.java.net>, "hotspot-gc-dev
 openjdk.java.net" <hotspot-gc-dev at openjdk.java.net>, Tom Rodriguez
	<tom.rodriguez at oracle.com>
X-BeenThere: hotspot-gc-dev at openjdk.java.net
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: Technical discussion about the development of the HotSpot garbage
 collectors <hotspot-gc-dev.openjdk.java.net>
List-Unsubscribe: <http://mail.openjdk.java.net/mailman/options/hotspot-gc-dev>, 
 <mailto:hotspot-gc-dev-request at openjdk.java.net?subject=unsubscribe>
List-Archive: <http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/>
List-Post: <mailto:hotspot-gc-dev at openjdk.java.net>
List-Help: <mailto:hotspot-gc-dev-request at openjdk.java.net?subject=help>
List-Subscribe: <http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-dev>, 
 <mailto:hotspot-gc-dev-request at openjdk.java.net?subject=subscribe>
Errors-To: hotspot-gc-dev-bounces at openjdk.java.net
Sender: hotspot-gc-dev <hotspot-gc-dev-bounces at openjdk.java.net>
X-purgate-ID: tlsNG-9b2b54/1509594485-0000036C-FB75D426/9/51289
X-purgate-type: clean
X-purgate-size: 3238
X-purgate-Ad: Categorized by eleven eXpurgate (R) http://www.eleven.de
Comment: eleven transport token MTQxLjE0Ni4xMjYuMjI5AGhvdHNwb3QtZ2MtZGV2LWJvdW5jZXNAb3Blbmpkay5qYXZhLm5ldABndW50ZXIuaGF1Z0BzYXAuY29tAGZlZTE1ZDY4ODdjMjM2Zjc5NDNiNDAwNzc4ZTdkODM2YzhjZGIzZGMX-purgate: This mail is considered clean (visit http://www.eleven.de for further information)
X-purgate: clean
X-SAP-SPAM-Status: clean
Return-Path: hotspot-gc-dev-bounces at openjdk.java.net
X-MS-Exchange-Organization-OriginalArrivalTime: 02 Nov 2017 03:48:07.7218
 (UTC)
X-MS-Exchange-Organization-OriginalClientIPAddress: 194.145.224.110
X-MS-Exchange-Organization-OriginalServerIPAddress: 155.56.66.98
X-MS-Exchange-Organization-AuthSource: DEWDFE13EDGE01.wdf.sap.corp
X-MS-Exchange-Organization-AuthAs: Anonymous
X-MS-Exchange-Organization-Cross-Premises-Headers-Processed: DEWDFE13EDGE01.wdf.sap.corp
X-MS-Exchange-Organization-Network-Message-Id: fd121385-9359-4a1d-7c94-08d521a487ac
X-MS-Exchange-Organization-OriginalSize: 8580
X-MS-Exchange-Organization-HygienePolicy: Standard
X-MS-Exchange-Organization-MessageHighPrecisionLatencyInProgress:
	LSRV?WDFE13EDGE01.wdf.sap.corp:TOTAL-EDGE956.873|UTH=0.001|RST956.857|CATCM-McAfeeTxRoutingAgent=0.002|CATCM=0.002|CAT=0.003;2017-11-02T08:14:04.594Z

> On Nov 1, 2017, at 5:10 PM, Doug Simon <doug.simon at oracle.com> wrote:
> 
> 
> 
>> On 1 Nov 2017, at 18:57, Kim Barrett <kim.barrett at oracle.com> wrote:
>> 
>>> On Nov 1, 2017, at 5:44 AM, Erik ??sterlund <erik.osterlund at oracle.com> wrote:
>>> 
>>> Hi Kim,
>>> 
>>> On 2017-11-01 06:03, Kim Barrett wrote:
>>>>> On Oct 30, 2017, at 7:14 AM, Doug Simon <doug.simon at oracle.com> wrote:
>>> JNIHandles::resolve with G1 should conditionally enqueue the oops if the SATB queue is activated, which it will not be after marking terminated. At this point, marking should have terminated already, and hence the resolve will not change the liveness of the oop. So I don't quite see how the resolve will change the oop to live here. Am I missing something?
>> 
>> Does this nmethod pass get run during young collections?  If not, then maybe you are right that it doesn???t matter.
>> 
>> But I still think that a cleanup pass that is based on using weak references like this should just be properly ordered
>> wrto weak reference processing, as it presumably already was.  In which case the resolve and can_unload is
>> effectively useless; either the jweak has been cleared and we won???t reach this clause, or the jweak has not been
>> cleared because the referent is still live, in which case can_unload will always return false and there was no point
>> in calling it.
> 
> I agree. As stated previously, my (defensive) code is based on assuming that nmethod unloading might happen before reference processing. If it never/rarely happens, then your suggested change is the right one. I did some extra testing and never saw it happen. Based on that, I'd like to adopt your solution. In the worst case, nmethod unloading will be delayed one full GC cycle if nmethod unloading precedes reference processing.
> 
> I've created another rev based on this:
> 
> http://cr.openjdk.java.net/~dnsimon/8188102_3
> 
> -Doug

This looks good.

------------------------------------------------------------------------------ 
src/hotspot/share/runtime/jniHandles.cpp

I'd like is_global_weak_cleared moved up a few lines, immediately
following the specializations of resolve_jweak, which I think it is
closely related to.

I don't need another webrev for this.

------------------------------------------------------------------------------ 
src/hotspot/share/code/nmethod.cpp

2934 char* nmethod::jvmci_installed_code_name(char* buf, size_t buflen) {
2935   if (!this->is_compiled_by_jvmci()) {
2936     return NULL;
2937   }
2938   oop installed_code = JNIHandles::resolve(_jvmci_installed_code);

Are there any circumstances where it would be unfortunate to have
getting the name keep alive the object? I was thinking logging. Or
perhaps it's a feature that getting the name might artificially extend
the lifetime? Looking at the callers, it doesn't seem like a problem
right now.

This is one of those places where a "peek" access (coming soon from
the GC interface) might be appropriate.

------------------------------------------------------------------------------




From thomas.schatzl at oracle.com  Thu Nov  2 09:09:13 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 02 Nov 2017 10:09:13 +0100
Subject: Passing command line options to Java [Was: Re: Heap Size Ergonomics
 in docker containers]
In-Reply-To: <CA++bR4N6jKfx6Rhzt9xktwfsQt=4tgZ_qJ-m1R_fYeCZTSfpDQ@mail.gmail.com>
References: <CA++bR4O9j=C5s2tJCH=JHRc51AdnCRiocvcUorJnngoG7i_R2A@mail.gmail.com>
 <1509463358.3199.2.camel@oracle.com>
 <CA++bR4N6jKfx6Rhzt9xktwfsQt=4tgZ_qJ-m1R_fYeCZTSfpDQ@mail.gmail.com>
Message-ID: <1509613753.3627.4.camel@oracle.com>

Hi Ruslan,

On Wed, 2017-11-01 at 11:30 +0100, Ruslan Synytsky wrote:
> Hi Thomas, thank you for the feedback. Please see inline.?
> 
> [...]
> 
> > There is JAVA_TOOL_OPTIONS, which are always prepended to the java
> > VM command line. See the description [0] for more information.
> 
> Indeed this option is close to what I was looking for! But seems
> JAVA_TOOL_OPTIONS does not work for Xmx. Just a quick test with Java
> HotSpot(TM) 64-Bit Server VM (build 9+181, mixed mode)
> 
> Default
> $ java -XX:+PrintFlagsFinal -version | grep MaxHeapSize
> ? ?size_t MaxHeapSize ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?= 643825664 ? ? ?
> ? ? ? ? ? ? ? ? ? ? ? ? ?{product} {command line}
> 
> Define Xmx = 2GB via?JAVA_TOOL_OPTIONS?
> $ export JAVA_TOOL_OPTIONS="-Xmx2g"?
> $ java -XX:+PrintFlagsFinal -version | grep MaxHeapSize
> Picked up JAVA_TOOL_OPTIONS: -Xmx2g
> ? ?size_t MaxHeapSize ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?= 643825664 ? ? ?
> ? ? ? ? ? ? ? ? ? ? ? ? ?{product} {command line}
> 
> It says that Xmx has been picked up, but in reality it has not been
> applied.??

Hmm, it seems to work here.

E.g.

$ java -XX:+PrintFlagsFinal HelloWorld | grep MaxHeap
???size_t MaxHeapSize?????                         = 3072327680
                                     {product} {ergonomic}
$ JAVA_TOOL_OPTIONS=-Xmx100m java -XX:+PrintFlagsFinal HelloWorld |
grep MaxHeap
Picked up JAVA_TOOL_OPTIONS: -Xmx100m
???size_t MaxHeapSize??????????????????????????????= 104857600
                                     {product} {command line}
$ echo $JAVA_TOOL_OPTIONS

$ export JAVA_TOOL_OPTIONS=-Xmx10m
$ java -XX:+PrintFlagsFinal HelloWorld | grep MaxHeap
Picked up JAVA_TOOL_OPTIONS: -Xmx10m
???size_t MaxHeapSize??????????????????????????????= 10485760
?????????????????????????????????{product} {command line}

(linux-x64, same hotspot build)
(Also tried with surrounding quotes)
(It does not matter whether you specify a java class on the command
line or not).

There is a note in the documentation that prevents use of it under
particular circumstances, but then according to the code the VM would
not print the "Picked up" line. 

I haven't seen an issue reported with that anywhere either.

> 
> Define Xmx = 3GB via _JAVA_OPTIONS?
> $ export _JAVA_OPTIONS="-Xmx3g"?
> $ java -XX:+PrintFlagsFinal -version | grep MaxHeapSize
> Picked up JAVA_TOOL_OPTIONS: -Xmx2g
> Picked up _JAVA_OPTIONS: -Xmx3g
> ? ?size_t MaxHeapSize ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?= 3221225472 ? ? ?
> ? ? ? ? ? ? ? ? ? ? ? ? {product} {command line}
> 
> Now it works as expected. However, another issue is that
> _JAVA_OPTIONS is undocumented and?hotspot-specific?...

I did not mention _JAVA_OPTIONS because I thought it wouldn't fit your
requirements and overwrites existing options. Also it's undocumented as
you noted.

There is another way to specify multiple java arguments via argument
files using  the "@" parameter which in some cases may be more useful
than the other options.

Thanks,
  Thomas


From rkennke at redhat.com  Thu Nov  2 12:04:05 2017
From: rkennke at redhat.com (Roman Kennke)
Date: Thu, 2 Nov 2017 13:04:05 +0100
Subject: RFR: 8183542: Factor out serial GC specific code from
 GenCollectedHeap into its own subclass
In-Reply-To: <2C530D1F-DAFA-428F-BC80-BF04EDB1DA98@oracle.com>
References: <6dea5306-1ec9-d439-b498-b9e657d7daf0@redhat.com>
 <3979E865-1363-4B60-ABDF-63128BAF04CE@oracle.com>
 <352c558d-40ba-27c4-861b-fcdb99ef706d@redhat.com>
 <487881EC-2CCD-47C4-BA7A-1723A40C1F9F@oracle.com>
 <ac5bcbc7-87cd-b1c7-73a2-1de96c0fbae8@redhat.com>
 <11759C2F-6F17-49C0-9D68-5B5B057C14B7@oracle.com>
 <4f84ff16-3f69-d2cf-2eb3-2e820aad49af@redhat.com>
 <A3E6BAD8-82A2-43A8-84FE-C3202BBEC9B1@oracle.com>
 <90d877fb-a406-e1e3-6d26-a5e00e5666eb@redhat.com>
 <2C530D1F-DAFA-428F-BC80-BF04EDB1DA98@oracle.com>
Message-ID: <842392bf-ed10-7aa5-08de-31bce3998bb1@redhat.com>

Am 02.11.2017 um 04:55 schrieb Kim Barrett:
>> On Oct 25, 2017, at 4:07 AM, Roman Kennke <rkennke at redhat.com> wrote:
>>
>> Hi Kim, hi Jini,
>>
>> thank you both for your reviews!
>>
>> I think I need a sponsor now. The final webrev (same as before plus Reviewed-by line):
>> http://cr.openjdk.java.net/~rkennke/8183542/webrev.03/ <http://cr.openjdk.java.net/%7Erkennke/8183542/webrev.03/>
> Sorry, I overlooked the sponsorship request.  I?ll take care of it, but not today.
>
Thanks, Kim!

Roman



From thomas.schatzl at oracle.com  Thu Nov  2 13:03:43 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 02 Nov 2017 14:03:43 +0100
Subject: RFR (S/M): 8149127: Rename g1/concurrentMarkThread.* to
 g1/g1ConcurrentMarkThread.*
Message-ID: <1509627823.3627.10.camel@oracle.com>

Hi all,

  can I have reviews for this renaming work as indicated by the
subject?

For easier review I provided three webrevs (although the "full" is not
that hard to review), the first renaming the files, the second renaming
the classes on top of that, and the third the complete webrev I intend
to push.

CR:
https://bugs.openjdk.java.net/browse/JDK-8149127
Webrev:
http://cr.openjdk.java.net/~tschatzl/8149127/webrev/ (full)
http://cr.openjdk.java.net/~tschatzl/8149127/webrev.a/ (file renaming)
http://cr.openjdk.java.net/~tschatzl/8149127/webrev.b/ (class renaming)
Testing:
jprt builds

Thanks,
  Thomas




From thomas.schatzl at oracle.com  Thu Nov  2 15:45:12 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 02 Nov 2017 16:45:12 +0100
Subject: RFR (): 8140255: Move the management of G1YoungRemSetSamplingThread
 from G1ConcurrentRefine
Message-ID: <1509637512.3627.17.camel@oracle.com>

Hi all,

  can I have reviews for this change that moves the ownership of the
young gen sampling thread from G1ConcurrentRefine to G1CollectedHeap,
in the process removing a few comments like "// The RS sampling thread
has nothing to do with refinement, but is here for now.".

CR:
https://bugs.openjdk.java.net/browse/JDK-8140255
Webrev:
http://cr.openjdk.java.net/~tschatzl/8140255/webrev/
Testing:
hs-tier1, hs-tier2, local testing

Thanks,
  Thomas


From doug.simon at oracle.com  Wed Nov  1 21:10:28 2017
From: doug.simon at oracle.com (Doug Simon)
Date: Wed, 1 Nov 2017 22:10:28 +0100
Subject: RFR: 8188102: [JVMCI] Convert special JVMCI oops in nmethod to
 jweak values
In-Reply-To: <33980406-F96C-434E-AAB7-949B39D46026@oracle.com>
References: <94B50A35-9898-47BA-8F20-A829FD7547AC@oracle.com>
 <E0D909B8-A115-4F83-9D0B-12E3FCF7EBEB@oracle.com>
 <BDD0189D-ECBF-42A8-BDDE-2014753B53C9@oracle.com>
 <BABEA69B-939D-4863-9A5B-271F32F330EB@oracle.com>
 <59F99795.6090404@oracle.com>
 <33980406-F96C-434E-AAB7-949B39D46026@oracle.com>
Message-ID: <111F6116-A1A4-4308-BAEC-D471F9E974B9@oracle.com>



> On 1 Nov 2017, at 18:57, Kim Barrett <kim.barrett at oracle.com> wrote:
> 
>> On Nov 1, 2017, at 5:44 AM, Erik ?sterlund <erik.osterlund at oracle.com> wrote:
>> 
>> Hi Kim,
>> 
>> On 2017-11-01 06:03, Kim Barrett wrote:
>>>> On Oct 30, 2017, at 7:14 AM, Doug Simon <doug.simon at oracle.com> wrote:
>>>> 
>>>> Hi Kim,
>>>> 
>>>> Thanks for the detailed review.
>>>> 
>>>>> On 29 Oct 2017, at 23:29, Kim Barrett <kim.barrett at oracle.com> wrote:
>>>>> 
>>>>> [added hotspot-gc-dev to cc list]
>>>>> 
>>>>>> On Oct 27, 2017, at 5:05 PM, Doug Simon <doug.simon at oracle.com> wrote:
>>>>>> 
>>>>>> Please review this change that converts the JVMCI-specific object references in nmethod from oops to weak values. This removes GC API extensions added purely for these fields (e.g. so that G1 can insert it into the right remembered set, and when unloading an nmethod, to go and remove the nmethod from that remembered set).
>>>>>> 
>>>>>> Testing: I've run the Graal unit tests (mx unittest --verbose --gc-after-test -Xlog:class+unload=trace) which trigger a lot of nmethod unloading.
>>>>>> 
>>>>>> https://bugs.openjdk.java.net/browse/JDK-8188102
>>>>>> http://cr.openjdk.java.net/~dnsimon/8188102/
>>>>> I didn't look at the .java, .py, or project files.
>>>>> 
>>>>> ------------------------------------------------------------------------------
>>>>> src/hotspot/share/jvmci/jvmciCompilerToVM.cpp
>>>>> 1061       nmethod* nm = cb->as_nmethod_or_null();
>>>>> 
>>>>> This appears to be dead code now.
>>>> Indeed.
>>>> 
>>>>> ------------------------------------------------------------------------------
>>>>> src/hotspot/share/code/nmethod.cpp
>>>>> 1023   assert(Universe::heap()->is_gc_active(), "should only be called during gc");
>>>>> ...
>>>>> 1036     if (!Universe::heap()->is_gc_active() && cause != NULL)
>>>>> 1037       cause->klass()->print_on(&ls);
>>>>> 
>>>>> I was going to mention that lines 1036-1037 are missing braces around
>>>>> the if-body.  However, those lines appear to be dead code, given the
>>>>> assertion on line 1023.
>>>> Good catch. That problem pre-dates this webrev but I will clean it up here.
>>>> 
>>>>> ------------------------------------------------------------------------------
>>>>> src/hotspot/share/code/nmethod.cpp
>>>>> 1504 bool nmethod::do_unloading_jvmci(BoolObjectClosure* is_alive, bool unloading_occurred) {
>>>>> ...
>>>>> 1506     oop installed_code = JNIHandles::resolve(_jvmci_installed_code);
>>>>> 
>>>>> Resolving a weak reference can keep an otherwise dead referent alive.
>>>>> See JDK-8188055 for a discussion of the corresponding problem for
>>>>> j.l.r.Reference.
>>>>> 
>>>>> Right now, I think JNIHandles doesn't provide a (public) solution to
>>>>> what I think is being attempted here that works for all collectors.
>>>>> There is in-progress work toward a solution, but it's just that, "in
>>>>> progress".
>>>>> 
>>>>> As a (possibly interim) solution, a function like the following might
>>>>> be added to JNIHandles (put the definition near resolve_jweak).
>>>>> 
>>>>> bool JNIHandles::is_global_weak_cleared(jweak handle) {
>>>>> assert(is_jweak(handle), "not a weak handle");
>>>>> return guard_value<false>(jweak_ref(handle)) == NULL;
>>>>> }
>>>> Adding JNIHandles::is_global_weak_cleared makes sense. I've put it the public section near destroy_weak_global instead of the private section where resolve_jweak is declared.
>>>> 
>>>>> (That's completely untested, and I haven't thought carefully about the
>>>>> name.  And should get input from other GC folks on how to deal with
>>>>> this.)  I *think* do_unloading_jvmci then becomes something like the
>>>>> following (again, completely untested)
>>>>> 
>>>>> bool nmethod::do_unloading_jvmci(BoolObjectClosure* is_alive, bool unloading_occurred) {
>>>>> if (_jvmci_installed_code != NULL) {
>>>>>  if (JNIHandles::is_global_weak_cleared(_jvmci_installed_code)) {
>>>>>    if (_jvmci_installed_code_triggers_unloading) {
>>>>>      make_unloaded(is_alive, NULL);
>>>>>      return true;
>>>>>    } else {
>>>>>      clear_jvmci_installed_code();
>>>>>    }
>>>>>  }
>>>>> }
>>>>> return false;
>>>>> }
>>>> I think your change works but comes at the cost of potentially preventing nmethod unloading for 1 extra (full?) GC cycle. It assumes that jweak clearing occurs before nmethod scanning. Is that guaranteed? If not, then I think what we want is:
>>>> 
>>>> bool nmethod::do_unloading_jvmci(BoolObjectClosure* is_alive, bool unloading_occurred) {
>>>> if (_jvmci_installed_code != NULL) {
>>>>   bool cleared = JNIHandles::is_global_weak_cleared(_jvmci_installed_code);
>>>>   if (_jvmci_installed_code_triggers_unloading) {
>>>>     if (cleared) {
>>>>       // jweak reference processing has already cleared the referent
>>>>       make_unloaded(is_alive, NULL);
>>>>       return true;
>>>>     } else {
>>>>       oop installed_code = JNIHandles::resolve(_jvmci_installed_code);
>>>>       if (can_unload(is_alive, (oop*)&installed_code, unloading_occurred)) {
>>>>         return true;
>>>>       }
>>>>     }
>>>>   } else {
>>>>     if (cleared || !is_alive->do_object_b(JNIHandles::resolve(_jvmci_installed_code))) {
>>>>       clear_jvmci_installed_code();
>>>>     }
>>>>   }
>>>> }
>>>> return false;
>>>> }
>>>> 
>>>> I've created a new webrev at http://cr.openjdk.java.net/~dnsimon/8188102_2.
>>>> 
>>>> -Doug
>>> The old code was looking for objects that are no longer alive; that
>>> can't be determined until after reference processing, as finalization
>>> could change the answer.  jweak clearing used to happen as part of
>>> reference processing, but there's been some recent refactoring.  I
>>> don't think that was intended to change the order of jweak processing
>>> wrto other things, but I haven't looked at the code to verify I'm not
>>> overlooking something.
>>> 
>>> In addition, I think
>>> 
>>>     } else {
>>>       oop installed_code = JNIHandles::resolve(_jvmci_installed_code);
>>>       if (can_unload(is_alive, (oop*)&installed_code, unloading_occurred)) {
>>>         return true;
>>>       }
>>> 
>>> doesn't work for some collectors (like G1), since the resolve will
>>> force installed_code to be live (if jweak processing were performed
>>> after nmethod processing), so can_unload will always return false.
>>> 
>> 
>> JNIHandles::resolve with G1 should conditionally enqueue the oops if the SATB queue is activated, which it will not be after marking terminated. At this point, marking should have terminated already, and hence the resolve will not change the liveness of the oop. So I don't quite see how the resolve will change the oop to live here. Am I missing something?
> 
> Does this nmethod pass get run during young collections?  If not, then maybe you are right that it doesn?t matter.
> 
> But I still think that a cleanup pass that is based on using weak references like this should just be properly ordered
> wrto weak reference processing, as it presumably already was.  In which case the resolve and can_unload is
> effectively useless; either the jweak has been cleared and we won?t reach this clause, or the jweak has not been
> cleared because the referent is still live, in which case can_unload will always return false and there was no point
> in calling it.

I agree. As stated previously, my (defensive) code is based on assuming that nmethod unloading might happen before reference processing. If it never/rarely happens, then your suggested change is the right one. I did some extra testing and never saw it happen. Based on that, I'd like to adopt your solution. In the worst case, nmethod unloading will be delayed one full GC cycle if nmethod unloading precedes reference processing.

I've created another rev based on this:

http://cr.openjdk.java.net/~dnsimon/8188102_3

-Doug

From doug.simon at oracle.com  Thu Nov  2 07:36:39 2017
From: doug.simon at oracle.com (Doug Simon)
Date: Thu, 2 Nov 2017 08:36:39 +0100
Subject: RFR: 8188102: [JVMCI] Convert special JVMCI oops in nmethod to
 jweak values
In-Reply-To: <CD3B456E-8FB6-4295-ADAD-45BEDBD3E81F@oracle.com>
References: <94B50A35-9898-47BA-8F20-A829FD7547AC@oracle.com>
 <E0D909B8-A115-4F83-9D0B-12E3FCF7EBEB@oracle.com>
 <BDD0189D-ECBF-42A8-BDDE-2014753B53C9@oracle.com>
 <BABEA69B-939D-4863-9A5B-271F32F330EB@oracle.com>
 <59F99795.6090404@oracle.com>
 <33980406-F96C-434E-AAB7-949B39D46026@oracle.com>
 <111F6116-A1A4-4308-BAEC-D471F9E974B9@oracle.com>
 <CD3B456E-8FB6-4295-ADAD-45BEDBD3E81F@oracle.com>
Message-ID: <79532B9B-F7B4-41F4-9156-8B03B9D6D30B@oracle.com>



> On 2 Nov 2017, at 04:47, Kim Barrett <kim.barrett at oracle.com> wrote:
> 
>> On Nov 1, 2017, at 5:10 PM, Doug Simon <doug.simon at oracle.com> wrote:
>> 
>> 
>> 
>>> On 1 Nov 2017, at 18:57, Kim Barrett <kim.barrett at oracle.com> wrote:
>>> 
>>>> On Nov 1, 2017, at 5:44 AM, Erik ?sterlund <erik.osterlund at oracle.com> wrote:
>>>> 
>>>> Hi Kim,
>>>> 
>>>> On 2017-11-01 06:03, Kim Barrett wrote:
>>>>>> On Oct 30, 2017, at 7:14 AM, Doug Simon <doug.simon at oracle.com> wrote:
>>>> JNIHandles::resolve with G1 should conditionally enqueue the oops if the SATB queue is activated, which it will not be after marking terminated. At this point, marking should have terminated already, and hence the resolve will not change the liveness of the oop. So I don't quite see how the resolve will change the oop to live here. Am I missing something?
>>> 
>>> Does this nmethod pass get run during young collections?  If not, then maybe you are right that it doesn?t matter.
>>> 
>>> But I still think that a cleanup pass that is based on using weak references like this should just be properly ordered
>>> wrto weak reference processing, as it presumably already was.  In which case the resolve and can_unload is
>>> effectively useless; either the jweak has been cleared and we won?t reach this clause, or the jweak has not been
>>> cleared because the referent is still live, in which case can_unload will always return false and there was no point
>>> in calling it.
>> 
>> I agree. As stated previously, my (defensive) code is based on assuming that nmethod unloading might happen before reference processing. If it never/rarely happens, then your suggested change is the right one. I did some extra testing and never saw it happen. Based on that, I'd like to adopt your solution. In the worst case, nmethod unloading will be delayed one full GC cycle if nmethod unloading precedes reference processing.
>> 
>> I've created another rev based on this:
>> 
>> http://cr.openjdk.java.net/~dnsimon/8188102_3
>> 
>> -Doug
> 
> This looks good.
> 
> ------------------------------------------------------------------------------ 
> src/hotspot/share/runtime/jniHandles.cpp
> 
> I'd like is_global_weak_cleared moved up a few lines, immediately
> following the specializations of resolve_jweak, which I think it is
> closely related to.
> 
> I don't need another webrev for this.

Will do. I updated the latest webrev in place.

> ------------------------------------------------------------------------------ 
> src/hotspot/share/code/nmethod.cpp
> 
> 2934 char* nmethod::jvmci_installed_code_name(char* buf, size_t buflen) {
> 2935   if (!this->is_compiled_by_jvmci()) {
> 2936     return NULL;
> 2937   }
> 2938   oop installed_code = JNIHandles::resolve(_jvmci_installed_code);
> 
> Are there any circumstances where it would be unfortunate to have
> getting the name keep alive the object? I was thinking logging. Or
> perhaps it's a feature that getting the name might artificially extend
> the lifetime? Looking at the callers, it doesn't seem like a problem
> right now.

Right, this is only called from tracing or debugging code.

> This is one of those places where a "peek" access (coming soon from
> the GC interface) might be appropriate.

This potential for resolving a jweak keeping the referent alive is unfortunate but I'm guessing it's also unavoidable.

Thanks once again for a very helpful review!

-Doug

From doug.simon at oracle.com  Thu Nov  2 07:37:31 2017
From: doug.simon at oracle.com (Doug Simon)
Date: Thu, 2 Nov 2017 08:37:31 +0100
Subject: RFR: 8188102: [JVMCI] Convert special JVMCI oops in nmethod to
 jweak values
In-Reply-To: <CD3B456E-8FB6-4295-ADAD-45BEDBD3E81F@oracle.com>
References: <94B50A35-9898-47BA-8F20-A829FD7547AC@oracle.com>
 <E0D909B8-A115-4F83-9D0B-12E3FCF7EBEB@oracle.com>
 <BDD0189D-ECBF-42A8-BDDE-2014753B53C9@oracle.com>
 <BABEA69B-939D-4863-9A5B-271F32F330EB@oracle.com>
 <59F99795.6090404@oracle.com>
 <33980406-F96C-434E-AAB7-949B39D46026@oracle.com>
 <111F6116-A1A4-4308-BAEC-D471F9E974B9@oracle.com>
 <CD3B456E-8FB6-4295-ADAD-45BEDBD3E81F@oracle.com>
Message-ID: <53561CBA-6032-46D1-A677-251872708A12@oracle.com>



> On 2 Nov 2017, at 04:47, Kim Barrett <kim.barrett at oracle.com> wrote:
> 
>> On Nov 1, 2017, at 5:10 PM, Doug Simon <doug.simon at oracle.com> wrote:
>> 
>> 
>> 
>>> On 1 Nov 2017, at 18:57, Kim Barrett <kim.barrett at oracle.com> wrote:
>>> 
>>>> On Nov 1, 2017, at 5:44 AM, Erik ?sterlund <erik.osterlund at oracle.com> wrote:
>>>> 
>>>> Hi Kim,
>>>> 
>>>> On 2017-11-01 06:03, Kim Barrett wrote:
>>>>>> On Oct 30, 2017, at 7:14 AM, Doug Simon <doug.simon at oracle.com> wrote:
>>>> JNIHandles::resolve with G1 should conditionally enqueue the oops if the SATB queue is activated, which it will not be after marking terminated. At this point, marking should have terminated already, and hence the resolve will not change the liveness of the oop. So I don't quite see how the resolve will change the oop to live here. Am I missing something?
>>> 
>>> Does this nmethod pass get run during young collections?  If not, then maybe you are right that it doesn?t matter.
>>> 
>>> But I still think that a cleanup pass that is based on using weak references like this should just be properly ordered
>>> wrto weak reference processing, as it presumably already was.  In which case the resolve and can_unload is
>>> effectively useless; either the jweak has been cleared and we won?t reach this clause, or the jweak has not been
>>> cleared because the referent is still live, in which case can_unload will always return false and there was no point
>>> in calling it.
>> 
>> I agree. As stated previously, my (defensive) code is based on assuming that nmethod unloading might happen before reference processing. If it never/rarely happens, then your suggested change is the right one. I did some extra testing and never saw it happen. Based on that, I'd like to adopt your solution. In the worst case, nmethod unloading will be delayed one full GC cycle if nmethod unloading precedes reference processing.
>> 
>> I've created another rev based on this:
>> 
>> http://cr.openjdk.java.net/~dnsimon/8188102_3
>> 
>> -Doug
> 
> This looks good.

Can I please get sign off from someone in the compiler team. Thanks.

-Doug

From mandy.chung at oracle.com  Thu Nov  2 17:16:47 2017
From: mandy.chung at oracle.com (mandy chung)
Date: Thu, 2 Nov 2017 10:16:47 -0700
Subject: Passing command line options to Java [Was: Re: Heap Size
 Ergonomics in docker containers]
In-Reply-To: <1509613753.3627.4.camel@oracle.com>
References: <CA++bR4O9j=C5s2tJCH=JHRc51AdnCRiocvcUorJnngoG7i_R2A@mail.gmail.com>
 <1509463358.3199.2.camel@oracle.com>
 <CA++bR4N6jKfx6Rhzt9xktwfsQt=4tgZ_qJ-m1R_fYeCZTSfpDQ@mail.gmail.com>
 <1509613753.3627.4.camel@oracle.com>
Message-ID: <c7524d27-ac43-6a0f-ecf4-d390eaddf904@oracle.com>



On 11/2/17 2:09 AM, Thomas Schatzl wrote:
>>> There is JAVA_TOOL_OPTIONS, which are always prepended to the java
>>> VM command line. See the description [0] for more information.
>>

You may want to check out the JDK_JAVA_OPTIONS environment variable and 
@argfile support introduced in JDK 9 java launcher [1].

Mandy

[1] 
https://docs.oracle.com/javase/9/tools/java.htm#GUID-3B1CE181-CD30-4178-9602-230B800D4FAE__USINGTHEJDK_JAVA_OPTIONSLAUNCHERENV-F3C0E3BA
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20171102/c2d5d3d5/attachment.htm>

From tom.rodriguez at oracle.com  Thu Nov  2 17:47:22 2017
From: tom.rodriguez at oracle.com (Tom Rodriguez)
Date: Thu, 02 Nov 2017 10:47:22 -0700
Subject: RFR: 8188102: [JVMCI] Convert special JVMCI oops in nmethod to
 jweak values
In-Reply-To: <79532B9B-F7B4-41F4-9156-8B03B9D6D30B@oracle.com>
References: <94B50A35-9898-47BA-8F20-A829FD7547AC@oracle.com>
 <E0D909B8-A115-4F83-9D0B-12E3FCF7EBEB@oracle.com>
 <BDD0189D-ECBF-42A8-BDDE-2014753B53C9@oracle.com>
 <BABEA69B-939D-4863-9A5B-271F32F330EB@oracle.com>
 <59F99795.6090404@oracle.com>
 <33980406-F96C-434E-AAB7-949B39D46026@oracle.com>
 <111F6116-A1A4-4308-BAEC-D471F9E974B9@oracle.com>
 <CD3B456E-8FB6-4295-ADAD-45BEDBD3E81F@oracle.com>
 <79532B9B-F7B4-41F4-9156-8B03B9D6D30B@oracle.com>
Message-ID: <59FB5A2A.3050407@oracle.com>

Looks good to me.

tom

Doug Simon wrote:
>
>> On 2 Nov 2017, at 04:47, Kim Barrett<kim.barrett at oracle.com>  wrote:
>>
>>> On Nov 1, 2017, at 5:10 PM, Doug Simon<doug.simon at oracle.com>  wrote:
>>>
>>>
>>>
>>>> On 1 Nov 2017, at 18:57, Kim Barrett<kim.barrett at oracle.com>  wrote:
>>>>
>>>>> On Nov 1, 2017, at 5:44 AM, Erik ?sterlund<erik.osterlund at oracle.com>  wrote:
>>>>>
>>>>> Hi Kim,
>>>>>
>>>>> On 2017-11-01 06:03, Kim Barrett wrote:
>>>>>>> On Oct 30, 2017, at 7:14 AM, Doug Simon<doug.simon at oracle.com>  wrote:
>>>>> JNIHandles::resolve with G1 should conditionally enqueue the oops if the SATB queue is activated, which it will not be after marking terminated. At this point, marking should have terminated already, and hence the resolve will not change the liveness of the oop. So I don't quite see how the resolve will change the oop to live here. Am I missing something?
>>>> Does this nmethod pass get run during young collections?  If not, then maybe you are right that it doesn?t matter.
>>>>
>>>> But I still think that a cleanup pass that is based on using weak references like this should just be properly ordered
>>>> wrto weak reference processing, as it presumably already was.  In which case the resolve and can_unload is
>>>> effectively useless; either the jweak has been cleared and we won?t reach this clause, or the jweak has not been
>>>> cleared because the referent is still live, in which case can_unload will always return false and there was no point
>>>> in calling it.
>>> I agree. As stated previously, my (defensive) code is based on assuming that nmethod unloading might happen before reference processing. If it never/rarely happens, then your suggested change is the right one. I did some extra testing and never saw it happen. Based on that, I'd like to adopt your solution. In the worst case, nmethod unloading will be delayed one full GC cycle if nmethod unloading precedes reference processing.
>>>
>>> I've created another rev based on this:
>>>
>>> http://cr.openjdk.java.net/~dnsimon/8188102_3
>>>
>>> -Doug
>> This looks good.
>>
>> ------------------------------------------------------------------------------
>> src/hotspot/share/runtime/jniHandles.cpp
>>
>> I'd like is_global_weak_cleared moved up a few lines, immediately
>> following the specializations of resolve_jweak, which I think it is
>> closely related to.
>>
>> I don't need another webrev for this.
>
> Will do. I updated the latest webrev in place.
>
>> ------------------------------------------------------------------------------
>> src/hotspot/share/code/nmethod.cpp
>>
>> 2934 char* nmethod::jvmci_installed_code_name(char* buf, size_t buflen) {
>> 2935   if (!this->is_compiled_by_jvmci()) {
>> 2936     return NULL;
>> 2937   }
>> 2938   oop installed_code = JNIHandles::resolve(_jvmci_installed_code);
>>
>> Are there any circumstances where it would be unfortunate to have
>> getting the name keep alive the object? I was thinking logging. Or
>> perhaps it's a feature that getting the name might artificially extend
>> the lifetime? Looking at the callers, it doesn't seem like a problem
>> right now.
>
> Right, this is only called from tracing or debugging code.
>
>> This is one of those places where a "peek" access (coming soon from
>> the GC interface) might be appropriate.
>
> This potential for resolving a jweak keeping the referent alive is unfortunate but I'm guessing it's also unavoidable.
>
> Thanks once again for a very helpful review!
>
> -Doug


From vladimir.kozlov at oracle.com  Thu Nov  2 22:03:37 2017
From: vladimir.kozlov at oracle.com (Vladimir Kozlov)
Date: Thu, 2 Nov 2017 15:03:37 -0700
Subject: RFR: 8188102: [JVMCI] Convert special JVMCI oops in nmethod to
 jweak values
In-Reply-To: <53561CBA-6032-46D1-A677-251872708A12@oracle.com>
References: <94B50A35-9898-47BA-8F20-A829FD7547AC@oracle.com>
 <E0D909B8-A115-4F83-9D0B-12E3FCF7EBEB@oracle.com>
 <BDD0189D-ECBF-42A8-BDDE-2014753B53C9@oracle.com>
 <BABEA69B-939D-4863-9A5B-271F32F330EB@oracle.com>
 <59F99795.6090404@oracle.com>
 <33980406-F96C-434E-AAB7-949B39D46026@oracle.com>
 <111F6116-A1A4-4308-BAEC-D471F9E974B9@oracle.com>
 <CD3B456E-8FB6-4295-ADAD-45BEDBD3E81F@oracle.com>
 <53561CBA-6032-46D1-A677-251872708A12@oracle.com>
Message-ID: <9168e4cc-aac3-ac2b-2229-11b766be0a2e@oracle.com>

Seems good to me.

Thanks,
Vladimir

On 11/2/17 12:37 AM, Doug Simon wrote:
> 
> 
>> On 2 Nov 2017, at 04:47, Kim Barrett <kim.barrett at oracle.com> wrote:
>>
>>> On Nov 1, 2017, at 5:10 PM, Doug Simon <doug.simon at oracle.com> wrote:
>>>
>>>
>>>
>>>> On 1 Nov 2017, at 18:57, Kim Barrett <kim.barrett at oracle.com> wrote:
>>>>
>>>>> On Nov 1, 2017, at 5:44 AM, Erik ?sterlund <erik.osterlund at oracle.com> wrote:
>>>>>
>>>>> Hi Kim,
>>>>>
>>>>> On 2017-11-01 06:03, Kim Barrett wrote:
>>>>>>> On Oct 30, 2017, at 7:14 AM, Doug Simon <doug.simon at oracle.com> wrote:
>>>>> JNIHandles::resolve with G1 should conditionally enqueue the oops if the SATB queue is activated, which it will not be after marking terminated. At this point, marking should have terminated already, and hence the resolve will not change the liveness of the oop. So I don't quite see how the resolve will change the oop to live here. Am I missing something?
>>>>
>>>> Does this nmethod pass get run during young collections?  If not, then maybe you are right that it doesn?t matter.
>>>>
>>>> But I still think that a cleanup pass that is based on using weak references like this should just be properly ordered
>>>> wrto weak reference processing, as it presumably already was.  In which case the resolve and can_unload is
>>>> effectively useless; either the jweak has been cleared and we won?t reach this clause, or the jweak has not been
>>>> cleared because the referent is still live, in which case can_unload will always return false and there was no point
>>>> in calling it.
>>>
>>> I agree. As stated previously, my (defensive) code is based on assuming that nmethod unloading might happen before reference processing. If it never/rarely happens, then your suggested change is the right one. I did some extra testing and never saw it happen. Based on that, I'd like to adopt your solution. In the worst case, nmethod unloading will be delayed one full GC cycle if nmethod unloading precedes reference processing.
>>>
>>> I've created another rev based on this:
>>>
>>> http://cr.openjdk.java.net/~dnsimon/8188102_3
>>>
>>> -Doug
>>
>> This looks good.
> 
> Can I please get sign off from someone in the compiler team. Thanks.
> 
> -Doug
> 


From dean.long at oracle.com  Fri Nov  3 02:02:59 2017
From: dean.long at oracle.com (dean.long at oracle.com)
Date: Thu, 2 Nov 2017 19:02:59 -0700
Subject: 8u144 hotspot fails to reach safepoint due to compiler thread -
 VM frozen
In-Reply-To: <CAHjP37F2Ey8z2j6m+i6NtONzvqfA5Svpi3P5gZ4ie9ampMLdsw@mail.gmail.com>
References: <CAHjP37F2Ey8z2j6m+i6NtONzvqfA5Svpi3P5gZ4ie9ampMLdsw@mail.gmail.com>
Message-ID: <5f39dfcf-64f9-7a9f-1ed3-c810eb6f1985@oracle.com>

 From a quick search, this sounds similar to 8085965.? I wonder if 
another circularity in the siblings list has been created somehow.? Can 
you check for that with gdb?

dl


On 10/31/17 11:08 AM, Vitaly Davidovich wrote:
> Hi guys,
>
> I have some colleagues who appear to be running into 
> https://bugs.openjdk.java.net/browse/JDK-8059128 on Oracle JDK 8u144 
> (Linux, x86-64).? Naturally, there's no reproducer but they've seen 
> this happen several times in the last couple of months.
>
> The symptom is the JVM becomes unresponsive - the application is not 
> servicing any traffic, and jstack doesn't work without the force 
> option. ?jstack output (with native frames) captured some time apart 
> shows the compiler thread either in Parse::do_all_blocks -> 
> do_one_block -> do_one_bytecode -> ... 
> InstanceKlass::has_finalizable_subclass -> 
> Dependencies::find_finalizable_subclass or <same as the previous one> 
> ... Dependencies::has_finalizable_subclass() -> Klass::next_sibling()
>
> I see that 8059128 was closed as Incomplete, but it does look like 
> there's a real issue here.? Has anyone looked into this further or has 
> any new thoughts/ideas?
>
> My understanding is the working theory is it's related to some data 
> race between class unloading and the compiler thread observing an 
> inconsistent (corrupt?) type hierarchy.? I see 
> https://bugs.openjdk.java.net/browse/JDK-8114823 is also noted as 
> possibly related - the app we're having trouble with is using G1, but 
> class unloading isn't disabled of course.? Is there some work around 
> to reduce the likelihood of having the compiler thread and GC cross 
> paths like this?
>
> Let me know if you need more info.
>
> Thanks



From kishor.kharbas at intel.com  Fri Nov  3 08:55:04 2017
From: kishor.kharbas at intel.com (Kharbas, Kishor)
Date: Fri, 3 Nov 2017 08:55:04 +0000
Subject: RFR(M): 8190308: Supporting heap allocation on alternative
 memory devices and CSR review
In-Reply-To: <a31f2457-149b-f5fb-71c3-b0595ddfa87a@oracle.com>
References: <F89640DCD01A85489FCBA68183A6A0F3BCD4BDC5@ORSMSX116.amr.corp.intel.com>
 <a31f2457-149b-f5fb-71c3-b0595ddfa87a@oracle.com>
Message-ID: <F89640DCD01A85489FCBA68183A6A0F3BCD4FCFF@ORSMSX116.amr.corp.intel.com>

Hi Sangheon,

Thanks for the review and comments. Here is an updated webrev-
http://cr.openjdk.java.net/~kkharbas/8190308/webrev.12
http://cr.openjdk.java.net/~kkharbas/8190308/webrev.12_to_11

In addition to your suggested corrections, I added code to set Linux core dump filter ensuring Heap is dumped correctly when this feature is used. This is follow-up to Jini George's comment (http://openjdk.5641.n7.nabble.com/RFR-M-8171181-Supporting-heap-allocation-on-alternative-memory-devices-td300109.html#a300450).

(some reply is inline)

Thanks
Kishor
From: sangheon.kim [mailto:sangheon.kim at oracle.com]
Sent: Wednesday, November 1, 2017 10:53 PM
To: Kharbas, Kishor <kishor.kharbas at intel.com>; 'hotspot-gc-dev at openjdk.java.net' <hotspot-gc-dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net
Cc: Thomas Schatzl <thomas.schatzl at oracle.com>
Subject: Re: RFR(M): 8190308: Supporting heap allocation on alternative memory devices and CSR review

Hi Kishor,
On 10/31/2017 04:53 PM, Kharbas, Kishor wrote:

Greetings,

I am continuing the earlier discussion [1] with a revised issue number representing the implementation of the JEP [2].

I have addressed the remaining unaddressed comments (copied below) in these webrevs -
http://cr.openjdk.java.net/~kkharbas/8190308/webrev.11/<http://cr.openjdk.java.net/%7Ekkharbas/8190308/webrev.11/>
http://cr.openjdk.java.net/~kkharbas/8190308/webrev.11_to_10/<http://cr.openjdk.java.net/%7Ekkharbas/8190308/webrev.11_to_10/>

Also, I would appreciate a review of the CSR for the new flag at https://bugs.openjdk.java.net/browse/JDK-8190309.
CSR: Reviewed.





> >  - in that same thread there has also been the question about the

> > need

> > for blocking the signals for the process that has gone unanswered.

> >



Removed the blocking of signals.



> >  - Arguments::finalize_vm_init_args: maybe at the place where the

> > change adds the warning/info message about NUMA support being

> > specific

> > to the file system; also add the same warning about UseLargePages

> > that

> > is located elsewhere.

> >

> > Like "UseXXXX may not be supported in some specific file system

> > implementations." - or just ignore as Andrew said in the other

> > email thread.

> >

> > Note that I am not sure that the selected place is the correct

> > place

> > for such warning about incompatible flags (and maybe

> > UseNUMA/UseLargePages should be forcibly disabled here? But then

> > again, it does not hurt just having it enabled?).

> >

> > I will let the runtime team comment as a lot of that argument

> > processing changed in jdk9.

> >

> > Maybe Arguments::check_vm_args_consistency() is better.

> >

> > There is similar code in Arguments::adjust_after_os()...

> >



1. Moved the check from finalize_vm_init_args() to check_vm_args_consistency()

2. When UseNUMA flag is true, adaptive logical group chunk resizing is used for Numa aware collectors such as ParallelOld. Just like UseNUMA is disabled in os::inti_2() in Linux when UseLargePages is set, it will be a good idea to disable UseNUMA when the new option is used.

3. I think its ok to leave UseNUMAInterleaving ON as it can be used for other memory areas.

4. For the same reason I also do not disable UseLargePages.

5. About handling UseLargePages, I thought of handling it the same way as when reserve_memory_special() fails to allocate large pages, i.e. generating a log_debug message.

      // failed; try to reserve regular memory below

  if (UseLargePages && (!FLAG_IS_DEFAULT(UseLargePages) ||

                        !FLAG_IS_DEFAULT(LargePageSizeInBytes))) {

    log_debug(gc, heap, coops)("Reserve regular memory without large pages");




> >  - here I may probably be speaking wrongly on behalf of the

> > runtime

> > team, but os.hpp, as an abstraction of the OS should probably not

> > have

> > platform specific ifdefs in it.

> >
and

> > > - You removed os::attempt_reserve_memory_at() from os.cpp and

> > > split

> > > into os_posix.cpp and os_windows.cpp.

> > >   But I think you should remain os::attempt_reserve_memory_at()

> > > at

> > > os.cpp and implement os specific stuffs at each os_xxx.cpp files

> > > for

> > > pd_xxx. Of cource move MemTracker function call as well.


In the updated webrev, I move the implementation from os_posix.cpp and os_window.cpp to respective pd_xxx functions. No AIX specific ifdef is required now.
Thank you for all changes.

I have minor nits now:
==============================================
- os***.cpp has some function names which include *dax*. I would prefer not to include it. As you know, Thomas also pointed it.

[Kharbas, Kishor] Done.

==============================================
src/hotspot/share/runtime/arguments.cpp
2537     if (!FLAG_IS_DEFAULT(UseNUMAInterleaving) || !FLAG_IS_DEFAULT(UseNUMA)) {
- Don't we need to check these 2 flags' value to be true? i.e. if user sets to false, below message will be printed.

[Kharbas, Kishor] That's correct. I changed it such that you check the flag is true and it's not default. In future if these flags become 'true' by default, we may not want to print warning.

==============================================
test/hotspot/jtreg/gc/TestAllocateHeapAt.java
- On other discussion, I mentioned to test only for Windows and Linux as the JEP described only about those 2. But without *dax* function names, it seems like not filtering OS seems okay too.

2  * Copyright (c) 2017, xxx Oracle and/or its affiliates. All rights reserved.
- Please remove 'xxx '.

47     Collections.addAll(vmOpts, new String[] {"-XX:AllocateHeapAt="+test_dir,
- Add spaces. '+test_dir' -> ' + test_dir'

49                                              "-Xmx50m",
50                                              "-Xms50m",
- You said there were no special reason for 200m(heap size of webrev.10) on other discussion. I would recommend 32m.

[Kharbas, Kishor] All done.

FYI, I ran hs-tier1 and hs-tier2 with your patch and the result is good. i.e. no new failures.

[Kharbas, Kishor] That's great. Thanks!

Thanks,
Sangheon








Thank you and looking forward for your feedback.

- Kishor


[1] http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2017-October/020682.html
[2] https://bugs.openjdk.java.net/browse/JDK-8171181


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20171103/d474e7b5/attachment.htm>

From leo.korinth at oracle.com  Fri Nov  3 09:24:58 2017
From: leo.korinth at oracle.com (Leo Korinth)
Date: Fri, 3 Nov 2017 10:24:58 +0100
Subject: RFR: 8186502: Assert when range testing G1RefProcDrainInterval on
 64-bit systems
Message-ID: <00ad9ed1-8822-d696-cae3-6911190c4402@oracle.com>

Hi,

Fix range checking when using -XX:G1RefProcDrainInterval=n

On 64 bit machines, when issuing extremly large integer arguments
to -XX:G1RefProcDrainInterval=n, a 32 bit integer will overflow and
cause a crash.

First I have improved the tests by adding the flag
-XX:+ExplicitGCInvokesConcurrent when testing the flag
-XX:G1RefProcDrainInterval=n to trigger the fault in our
test suit.

Then I have fixed the fault by limiting the arguments on 64 bit
systems (using an int type instead of intx type and a range up to
only INT_MAX for argument parsing).

Bug:
https://bugs.openjdk.java.net/browse/JDK-8186502

Webrev:
http://cr.openjdk.java.net/~lkorinth/8186502/00/

Testing:
- JPRT

Thanks,
Leo


From stefan.johansson at oracle.com  Fri Nov  3 11:30:51 2017
From: stefan.johansson at oracle.com (Stefan Johansson)
Date: Fri, 3 Nov 2017 12:30:51 +0100
Subject: RFR (S/M): 8149127: Rename g1/concurrentMarkThread.* to
 g1/g1ConcurrentMarkThread.*
In-Reply-To: <1509627823.3627.10.camel@oracle.com>
References: <1509627823.3627.10.camel@oracle.com>
Message-ID: <2024d19a-2eee-efeb-0934-cdd4130f1ab5@oracle.com>

Hi Thomas,

On 2017-11-02 14:03, Thomas Schatzl wrote:
> Hi all,
>
>    can I have reviews for this renaming work as indicated by the
> subject?
>
> For easier review I provided three webrevs (although the "full" is not
> that hard to review), the first renaming the files, the second renaming
> the classes on top of that, and the third the complete webrev I intend
> to push.
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8149127
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8149127/webrev/ (full)
> http://cr.openjdk.java.net/~tschatzl/8149127/webrev.a/ (file renaming)
> http://cr.openjdk.java.net/~tschatzl/8149127/webrev.b/ (class renaming)
Thanks for doing this cleanup Thomas.

Looks really good. Just one small thing, could you rename the getter in 
G1CollectedHeap:
1392?? G1ConcurrentRefine* concurrent_g1_refine() const { return _cg1r; }

I think it would make sense to rename it to g1_concurrent_refine().

Thanks,
Stefan

> Testing:
> jprt builds
>
> Thanks,
>    Thomas
>
>



From stefan.johansson at oracle.com  Fri Nov  3 11:35:30 2017
From: stefan.johansson at oracle.com (Stefan Johansson)
Date: Fri, 3 Nov 2017 12:35:30 +0100
Subject: RFR (): 8140255: Move the management of
 G1YoungRemSetSamplingThread from G1ConcurrentRefine
In-Reply-To: <1509637512.3627.17.camel@oracle.com>
References: <1509637512.3627.17.camel@oracle.com>
Message-ID: <b5792050-3269-a887-3098-872881d9a16e@oracle.com>

Hi Thomas,

On 2017-11-02 16:45, Thomas Schatzl wrote:
> Hi all,
>
>    can I have reviews for this change that moves the ownership of the
> young gen sampling thread from G1ConcurrentRefine to G1CollectedHeap,
> in the process removing a few comments like "// The RS sampling thread
> has nothing to do with refinement, but is here for now.".
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8140255
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8140255/webrev/
Looks good,
Stefan

> Testing:
> hs-tier1, hs-tier2, local testing
>
> Thanks,
>    Thomas



From vitalyd at gmail.com  Fri Nov  3 13:05:04 2017
From: vitalyd at gmail.com (Vitaly Davidovich)
Date: Fri, 03 Nov 2017 13:05:04 +0000
Subject: 8u144 hotspot fails to reach safepoint due to compiler thread -
 VM frozen
In-Reply-To: <5f39dfcf-64f9-7a9f-1ed3-c810eb6f1985@oracle.com>
References: <CAHjP37F2Ey8z2j6m+i6NtONzvqfA5Svpi3P5gZ4ie9ampMLdsw@mail.gmail.com>
 <5f39dfcf-64f9-7a9f-1ed3-c810eb6f1985@oracle.com>
Message-ID: <CAHjP37Er=aXdcM0K87wP8G9Vpo4tPtXYCRkxKASMZhs0LZ0dmQ@mail.gmail.com>

On Thu, Nov 2, 2017 at 10:03 PM <dean.long at oracle.com> wrote:

>  From a quick search, this sounds similar to 8085965.  I wonder if
> another circularity in the siblings list has been created somehow.  Can
> you check for that with gdb?

Hi Dean,

Yes it does sound like that bug, although that?s marked as fixed in 8u72.
I guess the implication is that it?s not actually fixed or this is a
different codepath/circumstance that ends up in the same blackhole.

I?ll see if my colleagues can dig around in the core file.  It?s a product
build so not sure how easy it will be to make sense of it without debug
symbols.

Thanks

>
>
> dl
>
>
> On 10/31/17 11:08 AM, Vitaly Davidovich wrote:
> > Hi guys,
> >
> > I have some colleagues who appear to be running into
> > https://bugs.openjdk.java.net/browse/JDK-8059128 on Oracle JDK 8u144
> > (Linux, x86-64).  Naturally, there's no reproducer but they've seen
> > this happen several times in the last couple of months.
> >
> > The symptom is the JVM becomes unresponsive - the application is not
> > servicing any traffic, and jstack doesn't work without the force
> > option.  jstack output (with native frames) captured some time apart
> > shows the compiler thread either in Parse::do_all_blocks ->
> > do_one_block -> do_one_bytecode -> ...
> > InstanceKlass::has_finalizable_subclass ->
> > Dependencies::find_finalizable_subclass or <same as the previous one>
> > ... Dependencies::has_finalizable_subclass() -> Klass::next_sibling()
> >
> > I see that 8059128 was closed as Incomplete, but it does look like
> > there's a real issue here.  Has anyone looked into this further or has
> > any new thoughts/ideas?
> >
> > My understanding is the working theory is it's related to some data
> > race between class unloading and the compiler thread observing an
> > inconsistent (corrupt?) type hierarchy.  I see
> > https://bugs.openjdk.java.net/browse/JDK-8114823 is also noted as
> > possibly related - the app we're having trouble with is using G1, but
> > class unloading isn't disabled of course.  Is there some work around
> > to reduce the likelihood of having the compiler thread and GC cross
> > paths like this?
> >
> > Let me know if you need more info.
> >
> > Thanks
>
> --
Sent from my phone
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20171103/05e781d5/attachment.htm>

From thomas.schatzl at oracle.com  Fri Nov  3 13:47:14 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Fri, 03 Nov 2017 14:47:14 +0100
Subject: RFR (S/M): 8149127: Rename g1/concurrentMarkThread.* to
 g1/g1ConcurrentMarkThread.*
In-Reply-To: <2024d19a-2eee-efeb-0934-cdd4130f1ab5@oracle.com>
References: <1509627823.3627.10.camel@oracle.com>
 <2024d19a-2eee-efeb-0934-cdd4130f1ab5@oracle.com>
Message-ID: <1509716834.3207.7.camel@oracle.com>

Hi Stefan,

   thanks for your review.

On Fri, 2017-11-03 at 12:30 +0100, Stefan Johansson wrote:
> Hi Thomas,
> 
> On 2017-11-02 14:03, Thomas Schatzl wrote:
> > Hi all,
> > 
> > ???can I have reviews for this renaming work as indicated by the
> > subject?
> > 
> > For easier review I provided three webrevs (although the "full" is
> > not
> > that hard to review), the first renaming the files, the second
> > renaming
> > the classes on top of that, and the third the complete webrev I
> > intend
> > to push.
> > 
> > CR:
> > https://bugs.openjdk.java.net/browse/JDK-8149127
> > Webrev:
> > http://cr.openjdk.java.net/~tschatzl/8149127/webrev/ (full)
> > http://cr.openjdk.java.net/~tschatzl/8149127/webrev.a/ (file
> > renaming)
> > http://cr.openjdk.java.net/~tschatzl/8149127/webrev.b/ (class
> > renaming)
> 
> Thanks for doing this cleanup Thomas.
> 
> Looks really good. Just one small thing, could you rename the getter
> in?
> G1CollectedHeap:
> 1392?? G1ConcurrentRefine* concurrent_g1_refine() const { return
> _cg1r; }
> 
> I think it would make sense to rename it to g1_concurrent_refine().
> 

  renamed to concurrent_refine() - also renamed the few "cg1r" names
floating around for consistency.

http://cr.openjdk.java.net/~tschatzl/8149127/webrev.1/ (full)
http://cr.openjdk.java.net/~tschatzl/8149127/webrev.b.1 (which also
represents the 0_to_1 patch)

Thanks,
  Thomas



From thomas.schatzl at oracle.com  Fri Nov  3 14:31:17 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Fri, 03 Nov 2017 15:31:17 +0100
Subject: RFR(M): 8190308: Supporting heap allocation on alternative
 memory devices and CSR review
In-Reply-To: <F89640DCD01A85489FCBA68183A6A0F3BCD4FCFF@ORSMSX116.amr.corp.intel.com>
References: <F89640DCD01A85489FCBA68183A6A0F3BCD4BDC5@ORSMSX116.amr.corp.intel.com>
 <a31f2457-149b-f5fb-71c3-b0595ddfa87a@oracle.com>
 <F89640DCD01A85489FCBA68183A6A0F3BCD4FCFF@ORSMSX116.amr.corp.intel.com>
Message-ID: <1509719477.3207.10.camel@oracle.com>

Hi,

On Fri, 2017-11-03 at 08:55 +0000, Kharbas, Kishor wrote:
> Hi Sangheon,
> ?
> Thanks for the review and comments. Here is an updated webrev-
> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.12
> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.12_to_11
> ?
> In addition to your suggested corrections, I added code to set Linux
> core dump filter ensuring Heap is dumped correctly when this feature
> is used. This is follow-up to Jini George?s comment
> (http://openjdk.5641.n7.nabble.com/RFR-M-8171181-Supporting-heap-
> allocation-on-alternative-memory-devices-td300109.html#a300450).

Some minor nits:

 - os_posix.cpp:300: please move the else next to the brace
 - arguments.cpp:4624: please add a space between "if" and the bracket

I do not need to see a new webrev for these changes. Looks good.

Thanks,
  Thomas



From thomas.schatzl at oracle.com  Fri Nov  3 16:28:15 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Fri, 03 Nov 2017 17:28:15 +0100
Subject: RFR (M): 8190426: Lazily initialize refinement threads with
 UseDynamicNumberOfGCThreads
Message-ID: <1509726495.2630.2.camel@oracle.com>

Hi,

  can I have reviews for this change that makes refinement threads
behave the same as the other thread groups we have in G1?

I.e. with UseDynamicNumberOfGCThreads enabled (which is off by default)
they are created lazily.

This helps a little bit further with startup/footprint benchmarks (if
enabled).

CR:
https://bugs.openjdk.java.net/browse/JDK-8190426
Webrev:
http://cr.openjdk.java.net/~tschatzl/8190426/webrev/
Testing:
hs-tier1+2

Thanks,
  Thomas


From dean.long at oracle.com  Fri Nov  3 16:56:42 2017
From: dean.long at oracle.com (dean.long at oracle.com)
Date: Fri, 3 Nov 2017 09:56:42 -0700
Subject: 8u144 hotspot fails to reach safepoint due to compiler thread -
 VM frozen
In-Reply-To: <CAHjP37Er=aXdcM0K87wP8G9Vpo4tPtXYCRkxKASMZhs0LZ0dmQ@mail.gmail.com>
References: <CAHjP37F2Ey8z2j6m+i6NtONzvqfA5Svpi3P5gZ4ie9ampMLdsw@mail.gmail.com>
 <5f39dfcf-64f9-7a9f-1ed3-c810eb6f1985@oracle.com>
 <CAHjP37Er=aXdcM0K87wP8G9Vpo4tPtXYCRkxKASMZhs0LZ0dmQ@mail.gmail.com>
Message-ID: <6e0b34c7-177c-4148-20b8-d717a1bab55d@oracle.com>

On 11/3/17 6:05 AM, Vitaly Davidovich wrote:

>
> On Thu, Nov 2, 2017 at 10:03 PM <dean.long at oracle.com 
> <mailto:dean.long at oracle.com>> wrote:
>
>     ?From a quick search, this sounds similar to 8085965.? I wonder if
>     another circularity in the siblings list has been created
>     somehow.? Can
>     you check for that with gdb?
>
> Hi Dean,
>
> Yes it does sound like that bug, although that?s marked as fixed in 
> 8u72.? I guess the implication is that it?s not actually fixed or this 
> is a different codepath/circumstance that ends up in the same blackhole.
>
> I?ll see if my colleagues can dig around in the core file.? It?s a 
> product build so not sure how easy it will be to make sense of it 
> without debug symbols.
>

I'm not an expert with jhsdb/clhsdb/hsdb, but I believe it can attach to 
a core file and give you some information even with a product build.

dl

> Thanks
>
>
>
>     dl
>
>
>     On 10/31/17 11:08 AM, Vitaly Davidovich wrote:
>     > Hi guys,
>     >
>     > I have some colleagues who appear to be running into
>     > https://bugs.openjdk.java.net/browse/JDK-8059128 on Oracle JDK 8u144
>     > (Linux, x86-64).? Naturally, there's no reproducer but they've seen
>     > this happen several times in the last couple of months.
>     >
>     > The symptom is the JVM becomes unresponsive - the application is not
>     > servicing any traffic, and jstack doesn't work without the force
>     > option. ?jstack output (with native frames) captured some time apart
>     > shows the compiler thread either in Parse::do_all_blocks ->
>     > do_one_block -> do_one_bytecode -> ...
>     > InstanceKlass::has_finalizable_subclass ->
>     > Dependencies::find_finalizable_subclass or <same as the previous
>     one>
>     > ... Dependencies::has_finalizable_subclass() ->
>     Klass::next_sibling()
>     >
>     > I see that 8059128 was closed as Incomplete, but it does look like
>     > there's a real issue here.? Has anyone looked into this further
>     or has
>     > any new thoughts/ideas?
>     >
>     > My understanding is the working theory is it's related to some data
>     > race between class unloading and the compiler thread observing an
>     > inconsistent (corrupt?) type hierarchy.? I see
>     > https://bugs.openjdk.java.net/browse/JDK-8114823 is also noted as
>     > possibly related - the app we're having trouble with is using
>     G1, but
>     > class unloading isn't disabled of course.? Is there some work around
>     > to reduce the likelihood of having the compiler thread and GC cross
>     > paths like this?
>     >
>     > Let me know if you need more info.
>     >
>     > Thanks
>
> -- 
> Sent from my phone

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20171103/8ec6514d/attachment.htm>

From kishor.kharbas at intel.com  Fri Nov  3 18:39:19 2017
From: kishor.kharbas at intel.com (Kharbas, Kishor)
Date: Fri, 3 Nov 2017 18:39:19 +0000
Subject: RFR(M): 8190308: Supporting heap allocation on alternative
 memory devices and CSR review
In-Reply-To: <1509719477.3207.10.camel@oracle.com>
References: <F89640DCD01A85489FCBA68183A6A0F3BCD4BDC5@ORSMSX116.amr.corp.intel.com>
 <a31f2457-149b-f5fb-71c3-b0595ddfa87a@oracle.com>
 <F89640DCD01A85489FCBA68183A6A0F3BCD4FCFF@ORSMSX116.amr.corp.intel.com>
 <1509719477.3207.10.camel@oracle.com>
Message-ID: <F89640DCD01A85489FCBA68183A6A0F3BCD4FF76@ORSMSX116.amr.corp.intel.com>

Thanks a lot!

Link to updated webrevs -
http://cr.openjdk.java.net/~kkharbas/8190308/webrev.13/
http://cr.openjdk.java.net/~kkharbas/8190308/webrev.13_to_12/

@Sangheon: Please let me know if you see any corrections needed.

-Kishor

> -----Original Message-----
> From: Thomas Schatzl [mailto:thomas.schatzl at oracle.com]
> Sent: Friday, November 3, 2017 7:31 AM
> To: Kharbas, Kishor <kishor.kharbas at intel.com>; sangheon.kim
> <sangheon.kim at oracle.com>; 'hotspot-gc-dev at openjdk.java.net'
> <hotspot-gc-dev at openjdk.java.net>; hotspot-runtime-
> dev at openjdk.java.net
> Subject: Re: RFR(M): 8190308: Supporting heap allocation on alternative
> memory devices and CSR review
> 
> Hi,
> 
> On Fri, 2017-11-03 at 08:55 +0000, Kharbas, Kishor wrote:
> > Hi Sangheon,
> >
> > Thanks for the review and comments. Here is an updated webrev-
> > http://cr.openjdk.java.net/~kkharbas/8190308/webrev.12
> > http://cr.openjdk.java.net/~kkharbas/8190308/webrev.12_to_11
> >
> > In addition to your suggested corrections, I added code to set Linux
> > core dump filter ensuring Heap is dumped correctly when this feature
> > is used. This is follow-up to Jini George?s comment
> > (http://openjdk.5641.n7.nabble.com/RFR-M-8171181-Supporting-heap-
> > allocation-on-alternative-memory-devices-td300109.html#a300450).
> 
> Some minor nits:
> 
>  - os_posix.cpp:300: please move the else next to the brace
>  - arguments.cpp:4624: please add a space between "if" and the bracket
> 
> I do not need to see a new webrev for these changes. Looks good.
> 
> Thanks,
>   Thomas


From sangheon.kim at oracle.com  Fri Nov  3 21:38:12 2017
From: sangheon.kim at oracle.com (sangheon.kim)
Date: Fri, 3 Nov 2017 14:38:12 -0700
Subject: RFR(M): 8190308: Supporting heap allocation on alternative memory
 devices and CSR review
In-Reply-To: <F89640DCD01A85489FCBA68183A6A0F3BCD4FF76@ORSMSX116.amr.corp.intel.com>
References: <F89640DCD01A85489FCBA68183A6A0F3BCD4BDC5@ORSMSX116.amr.corp.intel.com>
 <a31f2457-149b-f5fb-71c3-b0595ddfa87a@oracle.com>
 <F89640DCD01A85489FCBA68183A6A0F3BCD4FCFF@ORSMSX116.amr.corp.intel.com>
 <1509719477.3207.10.camel@oracle.com>
 <F89640DCD01A85489FCBA68183A6A0F3BCD4FF76@ORSMSX116.amr.corp.intel.com>
Message-ID: <9b259cdd-111a-0634-80a1-0c6ba1a8d260@oracle.com>

Hi Kishor,

On 11/03/2017 11:39 AM, Kharbas, Kishor wrote:
> Thanks a lot!
>
> Link to updated webrevs -
> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.13/
> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.13_to_12/
Thank you for fixing all.
Looks good to me except below.

Could you update the copyright format in TestAllocateHeapAt.java?
2? * Copyright (c) 2017 Oracle and/or its affiliates. All rights reserved.
- Missing comma: * Copyright (c) 2017_*,*_ Oracle and/or its affiliates. 
All rights reserved.

Thanks,
Sangheon


>
> @Sangheon: Please let me know if you see any corrections needed.
>
> -Kishor
>
>> -----Original Message-----
>> From: Thomas Schatzl [mailto:thomas.schatzl at oracle.com]
>> Sent: Friday, November 3, 2017 7:31 AM
>> To: Kharbas, Kishor <kishor.kharbas at intel.com>; sangheon.kim
>> <sangheon.kim at oracle.com>; 'hotspot-gc-dev at openjdk.java.net'
>> <hotspot-gc-dev at openjdk.java.net>; hotspot-runtime-
>> dev at openjdk.java.net
>> Subject: Re: RFR(M): 8190308: Supporting heap allocation on alternative
>> memory devices and CSR review
>>
>> Hi,
>>
>> On Fri, 2017-11-03 at 08:55 +0000, Kharbas, Kishor wrote:
>>> Hi Sangheon,
>>>
>>> Thanks for the review and comments. Here is an updated webrev-
>>> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.12
>>> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.12_to_11
>>>
>>> In addition to your suggested corrections, I added code to set Linux
>>> core dump filter ensuring Heap is dumped correctly when this feature
>>> is used. This is follow-up to Jini George?s comment
>>> (http://openjdk.5641.n7.nabble.com/RFR-M-8171181-Supporting-heap-
>>> allocation-on-alternative-memory-devices-td300109.html#a300450).
>> Some minor nits:
>>
>>   - os_posix.cpp:300: please move the else next to the brace
>>   - arguments.cpp:4624: please add a space between "if" and the bracket
>>
>> I do not need to see a new webrev for these changes. Looks good.
>>
>> Thanks,
>>    Thomas

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20171103/870f2a2a/attachment.htm>

From kishor.kharbas at intel.com  Fri Nov  3 21:42:52 2017
From: kishor.kharbas at intel.com (Kharbas, Kishor)
Date: Fri, 3 Nov 2017 21:42:52 +0000
Subject: RFR(M): 8190308: Supporting heap allocation on alternative
 memory devices and CSR review
In-Reply-To: <9b259cdd-111a-0634-80a1-0c6ba1a8d260@oracle.com>
References: <F89640DCD01A85489FCBA68183A6A0F3BCD4BDC5@ORSMSX116.amr.corp.intel.com>
 <a31f2457-149b-f5fb-71c3-b0595ddfa87a@oracle.com>
 <F89640DCD01A85489FCBA68183A6A0F3BCD4FCFF@ORSMSX116.amr.corp.intel.com>
 <1509719477.3207.10.camel@oracle.com>
 <F89640DCD01A85489FCBA68183A6A0F3BCD4FF76@ORSMSX116.amr.corp.intel.com>
 <9b259cdd-111a-0634-80a1-0c6ba1a8d260@oracle.com>
Message-ID: <F89640DCD01A85489FCBA68183A6A0F3BCD5006E@ORSMSX116.amr.corp.intel.com>

Thanks Sangheon!

I will make the change.

-Kishor

From: sangheon.kim [mailto:sangheon.kim at oracle.com]
Sent: Friday, November 3, 2017 2:38 PM
To: Kharbas, Kishor <kishor.kharbas at intel.com>; Thomas Schatzl <thomas.schatzl at oracle.com>; 'hotspot-gc-dev at openjdk.java.net' <hotspot-gc-dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net
Subject: Re: RFR(M): 8190308: Supporting heap allocation on alternative memory devices and CSR review

Hi Kishor,
On 11/03/2017 11:39 AM, Kharbas, Kishor wrote:

Thanks a lot!



Link to updated webrevs -

http://cr.openjdk.java.net/~kkharbas/8190308/webrev.13/

http://cr.openjdk.java.net/~kkharbas/8190308/webrev.13_to_12/
Thank you for fixing all.
Looks good to me except below.

Could you update the copyright format in TestAllocateHeapAt.java?
2  * Copyright (c) 2017 Oracle and/or its affiliates. All rights reserved.
- Missing comma: * Copyright (c) 2017, Oracle and/or its affiliates. All rights reserved.

Thanks,
Sangheon








@Sangheon: Please let me know if you see any corrections needed.



-Kishor



-----Original Message-----

From: Thomas Schatzl [mailto:thomas.schatzl at oracle.com]

Sent: Friday, November 3, 2017 7:31 AM

To: Kharbas, Kishor <kishor.kharbas at intel.com><mailto:kishor.kharbas at intel.com>; sangheon.kim

<sangheon.kim at oracle.com><mailto:sangheon.kim at oracle.com>; 'hotspot-gc-dev at openjdk.java.net<mailto:hotspot-gc-dev at openjdk.java.net>'

<hotspot-gc-dev at openjdk.java.net><mailto:hotspot-gc-dev at openjdk.java.net>; hotspot-runtime-

dev at openjdk.java.net<mailto:dev at openjdk.java.net>

Subject: Re: RFR(M): 8190308: Supporting heap allocation on alternative

memory devices and CSR review



Hi,



On Fri, 2017-11-03 at 08:55 +0000, Kharbas, Kishor wrote:

Hi Sangheon,



Thanks for the review and comments. Here is an updated webrev-

http://cr.openjdk.java.net/~kkharbas/8190308/webrev.12

http://cr.openjdk.java.net/~kkharbas/8190308/webrev.12_to_11



In addition to your suggested corrections, I added code to set Linux

core dump filter ensuring Heap is dumped correctly when this feature

is used. This is follow-up to Jini George?s comment

(http://openjdk.5641.n7.nabble.com/RFR-M-8171181-Supporting-heap-

allocation-on-alternative-memory-devices-td300109.html#a300450).



Some minor nits:



 - os_posix.cpp:300: please move the else next to the brace

 - arguments.cpp:4624: please add a space between "if" and the bracket



I do not need to see a new webrev for these changes. Looks good.



Thanks,

  Thomas



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20171103/94686968/attachment.htm>

From kishor.kharbas at intel.com  Fri Nov  3 21:59:59 2017
From: kishor.kharbas at intel.com (Kharbas, Kishor)
Date: Fri, 3 Nov 2017 21:59:59 +0000
Subject: RFR(M): 8190308: Supporting heap allocation on alternative
 memory devices and CSR review
In-Reply-To: <9b259cdd-111a-0634-80a1-0c6ba1a8d260@oracle.com>
References: <F89640DCD01A85489FCBA68183A6A0F3BCD4BDC5@ORSMSX116.amr.corp.intel.com>
 <a31f2457-149b-f5fb-71c3-b0595ddfa87a@oracle.com>
 <F89640DCD01A85489FCBA68183A6A0F3BCD4FCFF@ORSMSX116.amr.corp.intel.com>
 <1509719477.3207.10.camel@oracle.com>
 <F89640DCD01A85489FCBA68183A6A0F3BCD4FF76@ORSMSX116.amr.corp.intel.com>
 <9b259cdd-111a-0634-80a1-0c6ba1a8d260@oracle.com>
Message-ID: <F89640DCD01A85489FCBA68183A6A0F3BCD5008C@ORSMSX116.amr.corp.intel.com>

Hi Sangheon,

Here is link to the updated webrev-
http://cr.openjdk.java.net/~kkharbas/8190308/webrev.14/
http://cr.openjdk.java.net/~kkharbas/8190308/webrev.14_to_13/

Thanks
Kishor

From: sangheon.kim [mailto:sangheon.kim at oracle.com]
Sent: Friday, November 3, 2017 2:38 PM
To: Kharbas, Kishor <kishor.kharbas at intel.com>; Thomas Schatzl <thomas.schatzl at oracle.com>; 'hotspot-gc-dev at openjdk.java.net' <hotspot-gc-dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net
Subject: Re: RFR(M): 8190308: Supporting heap allocation on alternative memory devices and CSR review

Hi Kishor,
On 11/03/2017 11:39 AM, Kharbas, Kishor wrote:

Thanks a lot!



Link to updated webrevs -

http://cr.openjdk.java.net/~kkharbas/8190308/webrev.13/

http://cr.openjdk.java.net/~kkharbas/8190308/webrev.13_to_12/
Thank you for fixing all.
Looks good to me except below.

Could you update the copyright format in TestAllocateHeapAt.java?
2  * Copyright (c) 2017 Oracle and/or its affiliates. All rights reserved.
- Missing comma: * Copyright (c) 2017, Oracle and/or its affiliates. All rights reserved.

Thanks,
Sangheon








@Sangheon: Please let me know if you see any corrections needed.



-Kishor



-----Original Message-----

From: Thomas Schatzl [mailto:thomas.schatzl at oracle.com]

Sent: Friday, November 3, 2017 7:31 AM

To: Kharbas, Kishor <kishor.kharbas at intel.com><mailto:kishor.kharbas at intel.com>; sangheon.kim

<sangheon.kim at oracle.com><mailto:sangheon.kim at oracle.com>; 'hotspot-gc-dev at openjdk.java.net<mailto:hotspot-gc-dev at openjdk.java.net>'

<hotspot-gc-dev at openjdk.java.net><mailto:hotspot-gc-dev at openjdk.java.net>; hotspot-runtime-

dev at openjdk.java.net<mailto:dev at openjdk.java.net>

Subject: Re: RFR(M): 8190308: Supporting heap allocation on alternative

memory devices and CSR review



Hi,



On Fri, 2017-11-03 at 08:55 +0000, Kharbas, Kishor wrote:

Hi Sangheon,



Thanks for the review and comments. Here is an updated webrev-

http://cr.openjdk.java.net/~kkharbas/8190308/webrev.12

http://cr.openjdk.java.net/~kkharbas/8190308/webrev.12_to_11



In addition to your suggested corrections, I added code to set Linux

core dump filter ensuring Heap is dumped correctly when this feature

is used. This is follow-up to Jini George?s comment

(http://openjdk.5641.n7.nabble.com/RFR-M-8171181-Supporting-heap-

allocation-on-alternative-memory-devices-td300109.html#a300450).



Some minor nits:



 - os_posix.cpp:300: please move the else next to the brace

 - arguments.cpp:4624: please add a space between "if" and the bracket



I do not need to see a new webrev for these changes. Looks good.



Thanks,

  Thomas



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20171103/0c07027d/attachment.htm>

From sangheon.kim at oracle.com  Fri Nov  3 23:07:16 2017
From: sangheon.kim at oracle.com (sangheon.kim)
Date: Fri, 3 Nov 2017 16:07:16 -0700
Subject: RFR(M): 8190308: Supporting heap allocation on alternative memory
 devices and CSR review
In-Reply-To: <F89640DCD01A85489FCBA68183A6A0F3BCD5008C@ORSMSX116.amr.corp.intel.com>
References: <F89640DCD01A85489FCBA68183A6A0F3BCD4BDC5@ORSMSX116.amr.corp.intel.com>
 <a31f2457-149b-f5fb-71c3-b0595ddfa87a@oracle.com>
 <F89640DCD01A85489FCBA68183A6A0F3BCD4FCFF@ORSMSX116.amr.corp.intel.com>
 <1509719477.3207.10.camel@oracle.com>
 <F89640DCD01A85489FCBA68183A6A0F3BCD4FF76@ORSMSX116.amr.corp.intel.com>
 <9b259cdd-111a-0634-80a1-0c6ba1a8d260@oracle.com>
 <F89640DCD01A85489FCBA68183A6A0F3BCD5008C@ORSMSX116.amr.corp.intel.com>
Message-ID: <4171dc3d-3a43-a455-0efb-ee5ff5640b93@oracle.com>

Hi Kishor,


On 11/03/2017 02:59 PM, Kharbas, Kishor wrote:
>
> Hi Sangheon,
>
> Here is link to the updated webrev-
>
> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.14/ 
> <http://cr.openjdk.java.net/%7Ekkharbas/8190308/webrev.14/>
>
> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.14_to_13/ 
> <http://cr.openjdk.java.net/%7Ekkharbas/8190308/webrev.14_to_13/>
>
Looks good to me.

Thanks,
Sangheon


> Thanks
>
> Kishor
>
> *From:*sangheon.kim [mailto:sangheon.kim at oracle.com]
> *Sent:* Friday, November 3, 2017 2:38 PM
> *To:* Kharbas, Kishor <kishor.kharbas at intel.com>; Thomas Schatzl 
> <thomas.schatzl at oracle.com>; 'hotspot-gc-dev at openjdk.java.net' 
> <hotspot-gc-dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net
> *Subject:* Re: RFR(M): 8190308: Supporting heap allocation on 
> alternative memory devices and CSR review
>
> Hi Kishor,
>
> On 11/03/2017 11:39 AM, Kharbas, Kishor wrote:
>
>     Thanks a lot!
>
>     Link to updated webrevs -
>
>     http://cr.openjdk.java.net/~kkharbas/8190308/webrev.13/
>     <http://cr.openjdk.java.net/%7Ekkharbas/8190308/webrev.13/>
>
>     http://cr.openjdk.java.net/~kkharbas/8190308/webrev.13_to_12/
>     <http://cr.openjdk.java.net/%7Ekkharbas/8190308/webrev.13_to_12/>
>
> Thank you for fixing all.
> Looks good to me except below.
>
> Could you update the copyright format in TestAllocateHeapAt.java?
> 2? * Copyright (c) 2017 Oracle and/or its affiliates. All rights reserved.
> - Missing comma: * Copyright (c) 2017*_,_* Oracle and/or its 
> affiliates. All rights reserved.
>
> Thanks,
> Sangheon
>
>
>
>     @Sangheon: Please let me know if you see any corrections needed.
>
>     -Kishor
>
>         -----Original Message-----
>
>         From: Thomas Schatzl [mailto:thomas.schatzl at oracle.com]
>
>         Sent: Friday, November 3, 2017 7:31 AM
>
>         To: Kharbas, Kishor<kishor.kharbas at intel.com> <mailto:kishor.kharbas at intel.com>; sangheon.kim
>
>         <sangheon.kim at oracle.com> <mailto:sangheon.kim at oracle.com>; 'hotspot-gc-dev at openjdk.java.net
>         <mailto:hotspot-gc-dev at openjdk.java.net>'
>
>         <hotspot-gc-dev at openjdk.java.net>
>         <mailto:hotspot-gc-dev at openjdk.java.net>; hotspot-runtime-
>
>         dev at openjdk.java.net <mailto:dev at openjdk.java.net>
>
>         Subject: Re: RFR(M): 8190308: Supporting heap allocation on alternative
>
>         memory devices and CSR review
>
>         Hi,
>
>         On Fri, 2017-11-03 at 08:55 +0000, Kharbas, Kishor wrote:
>
>             Hi Sangheon,
>
>             Thanks for the review and comments. Here is an updated webrev-
>
>             http://cr.openjdk.java.net/~kkharbas/8190308/webrev.12
>             <http://cr.openjdk.java.net/%7Ekkharbas/8190308/webrev.12>
>
>             http://cr.openjdk.java.net/~kkharbas/8190308/webrev.12_to_11
>             <http://cr.openjdk.java.net/%7Ekkharbas/8190308/webrev.12_to_11>
>
>             In addition to your suggested corrections, I added code to set Linux
>
>             core dump filter ensuring Heap is dumped correctly when this feature
>
>             is used. This is follow-up to Jini George?s comment
>
>             (http://openjdk.5641.n7.nabble.com/RFR-M-8171181-Supporting-heap-
>
>             allocation-on-alternative-memory-devices-td300109.html#a300450).
>
>         Some minor nits:
>
>           - os_posix.cpp:300: please move the else next to the brace
>
>           - arguments.cpp:4624: please add a space between "if" and the bracket
>
>         I do not need to see a new webrev for these changes. Looks good.
>
>         Thanks,
>
>          ? Thomas
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20171103/fb672934/attachment.htm>

From sangheon.kim at oracle.com  Fri Nov  3 23:20:04 2017
From: sangheon.kim at oracle.com (sangheon.kim)
Date: Fri, 3 Nov 2017 16:20:04 -0700
Subject: RFR (S/M): 8149127: Rename g1/concurrentMarkThread.* to
 g1/g1ConcurrentMarkThread.*
In-Reply-To: <1509716834.3207.7.camel@oracle.com>
References: <1509627823.3627.10.camel@oracle.com>
 <2024d19a-2eee-efeb-0934-cdd4130f1ab5@oracle.com>
 <1509716834.3207.7.camel@oracle.com>
Message-ID: <60997df7-f463-9267-5c69-3929f2fe7df6@oracle.com>

Hi Thomas,

On 11/03/2017 06:47 AM, Thomas Schatzl wrote:
> Hi Stefan,
>
>     thanks for your review.
>
> On Fri, 2017-11-03 at 12:30 +0100, Stefan Johansson wrote:
>> Hi Thomas,
>>
>> On 2017-11-02 14:03, Thomas Schatzl wrote:
>>> Hi all,
>>>
>>>  ???can I have reviews for this renaming work as indicated by the
>>> subject?
>>>
>>> For easier review I provided three webrevs (although the "full" is
>>> not
>>> that hard to review), the first renaming the files, the second
>>> renaming
>>> the classes on top of that, and the third the complete webrev I
>>> intend
>>> to push.
>>>
>>> CR:
>>> https://bugs.openjdk.java.net/browse/JDK-8149127
>>> Webrev:
>>> http://cr.openjdk.java.net/~tschatzl/8149127/webrev/ (full)
>>> http://cr.openjdk.java.net/~tschatzl/8149127/webrev.a/ (file
>>> renaming)
>>> http://cr.openjdk.java.net/~tschatzl/8149127/webrev.b/ (class
>>> renaming)
>> Thanks for doing this cleanup Thomas.
>>
>> Looks really good. Just one small thing, could you rename the getter
>> in
>> G1CollectedHeap:
>> 1392?? G1ConcurrentRefine* concurrent_g1_refine() const { return
>> _cg1r; }
>>
>> I think it would make sense to rename it to g1_concurrent_refine().
>>
>    renamed to concurrent_refine() - also renamed the few "cg1r" names
> floating around for consistency.
>
> http://cr.openjdk.java.net/~tschatzl/8149127/webrev.1/ (full)
> http://cr.openjdk.java.net/~tschatzl/8149127/webrev.b.1 (which also
> represents the 0_to_1 patch)
Looks good.
Nice cleanup!

Thanks,
Sangheon


>
> Thanks,
>    Thomas
>



From sangheon.kim at oracle.com  Fri Nov  3 23:36:43 2017
From: sangheon.kim at oracle.com (sangheon.kim)
Date: Fri, 3 Nov 2017 16:36:43 -0700
Subject: RFR: 8186502: Assert when range testing G1RefProcDrainInterval on
 64-bit systems
In-Reply-To: <00ad9ed1-8822-d696-cae3-6911190c4402@oracle.com>
References: <00ad9ed1-8822-d696-cae3-6911190c4402@oracle.com>
Message-ID: <532c6234-f3c3-b6c6-0cd2-95bbb8c6c439@oracle.com>

Hi Leo,

On 11/03/2017 02:24 AM, Leo Korinth wrote:
> Hi,
>
> Fix range checking when using -XX:G1RefProcDrainInterval=n
>
> On 64 bit machines, when issuing extremly large integer arguments
> to -XX:G1RefProcDrainInterval=n, a 32 bit integer will overflow and
> cause a crash.
>
> First I have improved the tests by adding the flag
> -XX:+ExplicitGCInvokesConcurrent when testing the flag
> -XX:G1RefProcDrainInterval=n to trigger the fault in our
> test suit.
>
> Then I have fixed the fault by limiting the arguments on 64 bit
> systems (using an int type instead of intx type and a range up to
> only INT_MAX for argument parsing).
>
> Bug:
> https://bugs.openjdk.java.net/browse/JDK-8186502
>
> Webrev:
> http://cr.openjdk.java.net/~lkorinth/8186502/00/
Looks good, thank you for fixing for this.
Just minor nit: you can update copyright year at g1_globals.hpp.

BTW, did you run TestOptionsWithRanges.java manually?
Probably this test is still excluded by default when you run JPRT.

Thanks,
Sangheon


>
> Testing:
> - JPRT
>
> Thanks,
> Leo



From sangheon.kim at oracle.com  Fri Nov  3 23:45:46 2017
From: sangheon.kim at oracle.com (sangheon.kim)
Date: Fri, 3 Nov 2017 16:45:46 -0700
Subject: RFR (): 8140255: Move the management of
 G1YoungRemSetSamplingThread from G1ConcurrentRefine
In-Reply-To: <1509637512.3627.17.camel@oracle.com>
References: <1509637512.3627.17.camel@oracle.com>
Message-ID: <6e6678c0-d87e-1437-2293-06db3ae4a1b8@oracle.com>

Hi Thomas,

On 11/02/2017 08:45 AM, Thomas Schatzl wrote:
> Hi all,
>
>    can I have reviews for this change that moves the ownership of the
> young gen sampling thread from G1ConcurrentRefine to G1CollectedHeap,
> in the process removing a few comments like "// The RS sampling thread
> has nothing to do with refinement, but is here for now.".
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8140255
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8140255/webrev/
> Testing:
> hs-tier1, hs-tier2, local testing
Looks good.

Thanks,
Sangheon


>
> Thanks,
>    Thomas



From thomas.schatzl at oracle.com  Mon Nov  6 07:45:27 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 06 Nov 2017 08:45:27 +0100
Subject: RFR (): 8140255: Move the management of
 G1YoungRemSetSamplingThread from G1ConcurrentRefine
In-Reply-To: <6e6678c0-d87e-1437-2293-06db3ae4a1b8@oracle.com>
References: <1509637512.3627.17.camel@oracle.com>
 <6e6678c0-d87e-1437-2293-06db3ae4a1b8@oracle.com>
Message-ID: <1509954327.2624.0.camel@oracle.com>

Hi Sangheon, Stefan,

On Fri, 2017-11-03 at 16:45 -0700, sangheon.kim wrote:
> Hi Thomas,
> 
> On 11/02/2017 08:45 AM, Thomas Schatzl wrote:
> > Hi all,
> > 
> > ???can I have reviews for this change that moves the ownership of
> > the
> > young gen sampling thread from G1ConcurrentRefine to
> > G1CollectedHeap,
> > in the process removing a few comments like "// The RS sampling
> > thread
> > has nothing to do with refinement, but is here for now.".
> > 
> > CR:
> > https://bugs.openjdk.java.net/browse/JDK-8140255
> > Webrev:
> > http://cr.openjdk.java.net/~tschatzl/8140255/webrev/
> > Testing:
> > hs-tier1, hs-tier2, local testing
> 
> Looks good.
> 
> Thanks,
> Sangheon

On Fri, 2017-11-03 at 12:35 +0100, Stefan Johansson wrote:
> Hi Thomas,
>?
[...]
>?
> Looks good,
> Stefan

  thanks for your reviews.

Thomas


From thomas.schatzl at oracle.com  Mon Nov  6 07:47:01 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 06 Nov 2017 08:47:01 +0100
Subject: RFR (S/M): 8149127: Rename g1/concurrentMarkThread.* to
 g1/g1ConcurrentMarkThread.*
In-Reply-To: <60997df7-f463-9267-5c69-3929f2fe7df6@oracle.com>
References: <1509627823.3627.10.camel@oracle.com>
 <2024d19a-2eee-efeb-0934-cdd4130f1ab5@oracle.com>
 <1509716834.3207.7.camel@oracle.com>
 <60997df7-f463-9267-5c69-3929f2fe7df6@oracle.com>
Message-ID: <1509954421.2624.4.camel@oracle.com>

Hi Sangheon, Stefan,

On Fri, 2017-11-03 at 16:20 -0700, sangheon.kim wrote:
> Hi Thomas,
> 
> On 11/03/2017 06:47 AM, Thomas Schatzl wrote:
> > 
[...]
> > > Looks really good. Just one small thing, could you rename the
> > > getter in G1CollectedHeap:
> > > 1392?? G1ConcurrentRefine* concurrent_g1_refine() const { return
> > > _cg1r; }
> > > 
> > > I think it would make sense to rename it to
> > > g1_concurrent_refine().
> > > 
> > 
> > ???renamed to concurrent_refine() - also renamed the few "cg1r"
> > names
> > floating around for consistency.
> > 
> > http://cr.openjdk.java.net/~tschatzl/8149127/webrev.1/ (full)
> > http://cr.openjdk.java.net/~tschatzl/8149127/webrev.b.1 (which also
> > represents the 0_to_1 patch)
> 
> Looks good.
> Nice cleanup!

  thanks for your reviews.

Thomas



From leo.korinth at oracle.com  Mon Nov  6 09:55:57 2017
From: leo.korinth at oracle.com (Leo Korinth)
Date: Mon, 6 Nov 2017 10:55:57 +0100
Subject: RFR: 8186502: Assert when range testing G1RefProcDrainInterval on
 64-bit systems
In-Reply-To: <532c6234-f3c3-b6c6-0cd2-95bbb8c6c439@oracle.com>
References: <00ad9ed1-8822-d696-cae3-6911190c4402@oracle.com>
 <532c6234-f3c3-b6c6-0cd2-95bbb8c6c439@oracle.com>
Message-ID: <b16d9282-4bf7-171e-136a-8b9df9350dff@oracle.com>

Hi Sangheon.

On 04/11/17 00:36, sangheon.kim wrote:
 > Hi Leo,
 >
 > On 11/03/2017 02:24 AM, Leo Korinth wrote:
 >> Hi,
 >>
 >> Fix range checking when using -XX:G1RefProcDrainInterval=n
 >>
 >> On 64 bit machines, when issuing extremly large integer arguments
 >> to -XX:G1RefProcDrainInterval=n, a 32 bit integer will overflow and
 >> cause a crash.
 >>
 >> First I have improved the tests by adding the flag
 >> -XX:+ExplicitGCInvokesConcurrent when testing the flag
 >> -XX:G1RefProcDrainInterval=n to trigger the fault in our
 >> test suit.
 >>
 >> Then I have fixed the fault by limiting the arguments on 64 bit
 >> systems (using an int type instead of intx type and a range up to
 >> only INT_MAX for argument parsing).
 >>
 >> Bug:
 >> https://bugs.openjdk.java.net/browse/JDK-8186502
 >>
 >> Webrev:
 >> http://cr.openjdk.java.net/~lkorinth/8186502/00/
 > Looks good, thank you for fixing for this.
 > Just minor nit: you can update copyright year at g1_globals.hpp.

Thanks, copyright year fixed in:
http://cr.openjdk.java.net/~lkorinth/8186502/01/

 > BTW, did you run TestOptionsWithRanges.java manually?
 > Probably this test is still excluded by default when you run JPRT.

Yes, first I updated the test so that it triggered the bug (and ran the 
test manually to verify that it indeed triggered the bug). Then I fixed 
the bug. After that I ran the test again (and the test passed).

I am unsure if the test is enabled when you run JPRT though, but if it 
is, it will pass.

/Leo

 > Thanks,
 > Sangheon
 >
 >
 >>
 >> Testing:
 >> - JPRT
 >>
 >> Thanks,
 >> Leo
 >


From stefan.johansson at oracle.com  Mon Nov  6 10:21:54 2017
From: stefan.johansson at oracle.com (Stefan Johansson)
Date: Mon, 6 Nov 2017 11:21:54 +0100
Subject: RFR: 8186502: Assert when range testing G1RefProcDrainInterval on
 64-bit systems
In-Reply-To: <b16d9282-4bf7-171e-136a-8b9df9350dff@oracle.com>
References: <00ad9ed1-8822-d696-cae3-6911190c4402@oracle.com>
 <532c6234-f3c3-b6c6-0cd2-95bbb8c6c439@oracle.com>
 <b16d9282-4bf7-171e-136a-8b9df9350dff@oracle.com>
Message-ID: <c27b49e8-39ae-92dc-86d0-10327e4041d6@oracle.com>

Thanks for fixing this Leo,

Change looks good and I can do the push for you,
StefanJ

On 2017-11-06 10:55, Leo Korinth wrote:
> Hi Sangheon.
>
> On 04/11/17 00:36, sangheon.kim wrote:
> > Hi Leo,
> >
> > On 11/03/2017 02:24 AM, Leo Korinth wrote:
> >> Hi,
> >>
> >> Fix range checking when using -XX:G1RefProcDrainInterval=n
> >>
> >> On 64 bit machines, when issuing extremly large integer arguments
> >> to -XX:G1RefProcDrainInterval=n, a 32 bit integer will overflow and
> >> cause a crash.
> >>
> >> First I have improved the tests by adding the flag
> >> -XX:+ExplicitGCInvokesConcurrent when testing the flag
> >> -XX:G1RefProcDrainInterval=n to trigger the fault in our
> >> test suit.
> >>
> >> Then I have fixed the fault by limiting the arguments on 64 bit
> >> systems (using an int type instead of intx type and a range up to
> >> only INT_MAX for argument parsing).
> >>
> >> Bug:
> >> https://bugs.openjdk.java.net/browse/JDK-8186502
> >>
> >> Webrev:
> >> http://cr.openjdk.java.net/~lkorinth/8186502/00/
> > Looks good, thank you for fixing for this.
> > Just minor nit: you can update copyright year at g1_globals.hpp.
>
> Thanks, copyright year fixed in:
> http://cr.openjdk.java.net/~lkorinth/8186502/01/
>
> > BTW, did you run TestOptionsWithRanges.java manually?
> > Probably this test is still excluded by default when you run JPRT.
>
> Yes, first I updated the test so that it triggered the bug (and ran 
> the test manually to verify that it indeed triggered the bug). Then I 
> fixed the bug. After that I ran the test again (and the test passed).
>
> I am unsure if the test is enabled when you run JPRT though, but if it 
> is, it will pass.
>
> /Leo
>
> > Thanks,
> > Sangheon
> >
> >
> >>
> >> Testing:
> >> - JPRT
> >>
> >> Thanks,
> >> Leo
> >



From rkennke at redhat.com  Mon Nov  6 10:55:07 2017
From: rkennke at redhat.com (Roman Kennke)
Date: Mon, 6 Nov 2017 11:55:07 +0100
Subject: RFR: 8183542: Factor out serial GC specific code from
 GenCollectedHeap into its own subclass
In-Reply-To: <2C530D1F-DAFA-428F-BC80-BF04EDB1DA98@oracle.com>
References: <6dea5306-1ec9-d439-b498-b9e657d7daf0@redhat.com>
 <3979E865-1363-4B60-ABDF-63128BAF04CE@oracle.com>
 <352c558d-40ba-27c4-861b-fcdb99ef706d@redhat.com>
 <487881EC-2CCD-47C4-BA7A-1723A40C1F9F@oracle.com>
 <ac5bcbc7-87cd-b1c7-73a2-1de96c0fbae8@redhat.com>
 <11759C2F-6F17-49C0-9D68-5B5B057C14B7@oracle.com>
 <4f84ff16-3f69-d2cf-2eb3-2e820aad49af@redhat.com>
 <A3E6BAD8-82A2-43A8-84FE-C3202BBEC9B1@oracle.com>
 <90d877fb-a406-e1e3-6d26-a5e00e5666eb@redhat.com>
 <2C530D1F-DAFA-428F-BC80-BF04EDB1DA98@oracle.com>
Message-ID: <ced5a014-2dba-2f01-adca-263c4a93c82a@redhat.com>

Am 02.11.2017 um 04:55 schrieb Kim Barrett:
>> On Oct 25, 2017, at 4:07 AM, Roman Kennke <rkennke at redhat.com> wrote:
>>
>> Hi Kim, hi Jini,
>>
>> thank you both for your reviews!
>>
>> I think I need a sponsor now. The final webrev (same as before plus Reviewed-by line):
>> http://cr.openjdk.java.net/~rkennke/8183542/webrev.03/ <http://cr.openjdk.java.net/%7Erkennke/8183542/webrev.03/>
> Sorry, I overlooked the sponsorship request.  I?ll take care of it, but not today.
>
Ping? :-)



From rkennke at redhat.com  Mon Nov  6 11:13:01 2017
From: rkennke at redhat.com (Roman Kennke)
Date: Mon, 6 Nov 2017 12:13:01 +0100
Subject: RFR: 8189171: Move GC argument processing into GC specific classes
In-Reply-To: <affd801a-d184-1bcd-7c58-65c1ecaa94da@oracle.com>
References: <e72385e3-88f1-9b05-8495-ab7f4c4cbd60@redhat.com>
 <ed0085ca-7384-05e7-4a2c-e2f048f64909@oracle.com>
 <5a7fecea-1a1b-16db-e366-4dd8d0c9ed63@redhat.com>
 <59E9E9A9.7020306@oracle.com>
 <511327b5-4632-0e27-6914-2a3d076a1dc3@redhat.com>
 <59EA002D.1050605@oracle.com>
 <2b37fed8-0383-eb19-3e45-076aabd74f22@redhat.com>
 <59F0AD40.3020307@oracle.com>
 <e8e953a1-06b3-648b-d5da-c87ee7c640ad@redhat.com>
 <B3E05F9C-5AC2-40D3-BBF3-A46ED9B12ED5@oracle.com>
 <affd801a-d184-1bcd-7c58-65c1ecaa94da@oracle.com>
Message-ID: <3c75a6c5-ffe5-948f-b2c3-b4950a4b1a9f@redhat.com>

Am 26.10.2017 um 11:43 schrieb Per Liden:
> Hi,
>
> On 2017-10-25 18:11, Erik Osterlund wrote:
> [...]
>>> So let me just put your changes up for review (again), if you don't 
>>> mind:
>>>
>>> Full webrev:
>>> http://cr.openjdk.java.net/~eosterlund/8189171/webrev.03/
>>> <http://cr.openjdk.java.net/%7Eeosterlund/8189171/webrev.03/>
>
> I like this. Just two naming suggestions:
>
> src/hotspot/share/gc/shared/gcArguments.hpp:
>
> ? 39?? static jint create_instance();
> ? 40?? static bool is_initialized();
> ? 41?? static GCArguments* instance();
>
> Could we call these:
>
> ? 39?? static jint initialize();
> ? 40?? static bool is_initialized();
> ? 41?? static GCArguments* arguments();
>
> Reasoning: initialize() maps better to its companion is_initialized(). 
> GCArguments::arguments() maps better to the existing patterns we have 
> with CollectedHeap::heap().
Ok, that sounds good to me. Actually, it's almost full-circle back to my 
original proposal ;-)

Differential:
http://cr.openjdk.java.net/~rkennke/8189171/webrev.04.diff/ 
<http://cr.openjdk.java.net/%7Erkennke/8189171/webrev.04.diff/>
Full:
http://cr.openjdk.java.net/~rkennke/8189171/webrev.04/ 
<http://cr.openjdk.java.net/%7Erkennke/8189171/webrev.04/>

Ok to push now?

Roman


From erik.osterlund at oracle.com  Mon Nov  6 11:16:17 2017
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Mon, 6 Nov 2017 12:16:17 +0100
Subject: RFR: 8189171: Move GC argument processing into GC specific classes
In-Reply-To: <3c75a6c5-ffe5-948f-b2c3-b4950a4b1a9f@redhat.com>
References: <e72385e3-88f1-9b05-8495-ab7f4c4cbd60@redhat.com>
 <ed0085ca-7384-05e7-4a2c-e2f048f64909@oracle.com>
 <5a7fecea-1a1b-16db-e366-4dd8d0c9ed63@redhat.com>
 <59E9E9A9.7020306@oracle.com>
 <511327b5-4632-0e27-6914-2a3d076a1dc3@redhat.com>
 <59EA002D.1050605@oracle.com>
 <2b37fed8-0383-eb19-3e45-076aabd74f22@redhat.com>
 <59F0AD40.3020307@oracle.com>
 <e8e953a1-06b3-648b-d5da-c87ee7c640ad@redhat.com>
 <B3E05F9C-5AC2-40D3-BBF3-A46ED9B12ED5@oracle.com>
 <affd801a-d184-1bcd-7c58-65c1ecaa94da@oracle.com>
 <3c75a6c5-ffe5-948f-b2c3-b4950a4b1a9f@redhat.com>
Message-ID: <5A004481.3050001@oracle.com>

Hi Roman,

Looks good.

Thanks,
/Erik

On 2017-11-06 12:13, Roman Kennke wrote:
> Am 26.10.2017 um 11:43 schrieb Per Liden:
>> Hi,
>>
>> On 2017-10-25 18:11, Erik Osterlund wrote:
>> [...]
>>>> So let me just put your changes up for review (again), if you don't 
>>>> mind:
>>>>
>>>> Full webrev:
>>>> http://cr.openjdk.java.net/~eosterlund/8189171/webrev.03/
>>>> <http://cr.openjdk.java.net/%7Eeosterlund/8189171/webrev.03/>
>>
>> I like this. Just two naming suggestions:
>>
>> src/hotspot/share/gc/shared/gcArguments.hpp:
>>
>>   39   static jint create_instance();
>>   40   static bool is_initialized();
>>   41   static GCArguments* instance();
>>
>> Could we call these:
>>
>>   39   static jint initialize();
>>   40   static bool is_initialized();
>>   41   static GCArguments* arguments();
>>
>> Reasoning: initialize() maps better to its companion 
>> is_initialized(). GCArguments::arguments() maps better to the 
>> existing patterns we have with CollectedHeap::heap().
> Ok, that sounds good to me. Actually, it's almost full-circle back to 
> my original proposal ;-)
>
> Differential:
> http://cr.openjdk.java.net/~rkennke/8189171/webrev.04.diff/ 
> <http://cr.openjdk.java.net/%7Erkennke/8189171/webrev.04.diff/>
> Full:
> http://cr.openjdk.java.net/~rkennke/8189171/webrev.04/ 
> <http://cr.openjdk.java.net/%7Erkennke/8189171/webrev.04/>
>
> Ok to push now?
>
> Roman



From thomas.schatzl at oracle.com  Mon Nov  6 11:27:03 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 06 Nov 2017 12:27:03 +0100
Subject: RFR (S): 8190703: TestSystemGCWith* infrequently times out on SPARC
Message-ID: <1509967623.2750.2.camel@oracle.com>

Hi all,

  can I have reviews for this change the fixes intermittent timeouts in
a stress test when run on slow machines and debug builds.

The fix is to simply let the test bail out early if time is up - this
allows the test run through as intended in the many configurations
where it does not take excessive amount of time.

CR:
https://bugs.openjdk.java.net/browse/JDK-8190703
Webrev:
http://cr.openjdk.java.net/~tschatzl/8190703/webrev
Testing:
Ran the specific tests in our test lab on slow configurations (SPARCv9,
debug build), checking manually that they do not exceed their specified
time out.

Thanks,
  Thomas



From shade at redhat.com  Mon Nov  6 19:41:09 2017
From: shade at redhat.com (Aleksey Shipilev)
Date: Mon, 6 Nov 2017 20:41:09 +0100
Subject: RFC: Epsilon GC JEP
In-Reply-To: <67f6d4a2-d129-1491-4906-473586dc6680@redhat.com>
References: <67f6d4a2-d129-1491-4906-473586dc6680@redhat.com>
Message-ID: <235b0c9f-af6e-a518-2e57-3a7ef4b38378@redhat.com>

On 07/10/2017 10:14 PM, Aleksey Shipilev wrote:
> I would like to solicit feedback on Epsilon GC JEP:
>   https://bugs.openjdk.java.net/browse/JDK-8174901
>   http://openjdk.java.net/jeps/8174901

Following up on this after the discussion, please add yourself to Reviewed-by (or Endorsed-by, if
you are the group lead) in JEP!

Erik Helin, Roman Kennke, Thomas Schatzl, and Erik Osterlund replied in this thread. More reviews
and endorsements are welcome.

Thanks,
-Aleksey



-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20171106/9c5e0f99/signature.asc>

From shade at redhat.com  Mon Nov  6 19:46:52 2017
From: shade at redhat.com (Aleksey Shipilev)
Date: Mon, 6 Nov 2017 20:46:52 +0100
Subject: RFR (S): 8190703: TestSystemGCWith* infrequently times out on
 SPARC
In-Reply-To: <1509967623.2750.2.camel@oracle.com>
References: <1509967623.2750.2.camel@oracle.com>
Message-ID: <8cb38009-4d26-2cdd-75a5-460857452aa5@redhat.com>

On 11/06/2017 12:27 PM, Thomas Schatzl wrote:
>   can I have reviews for this change the fixes intermittent timeouts in
> a stress test when run on slow machines and debug builds.
> 
> The fix is to simply let the test bail out early if time is up - this
> allows the test run through as intended in the many configurations
> where it does not take excessive amount of time.

Makes sense. Real timeouts would still surface.

> CR:
> https://bugs.openjdk.java.net/browse/JDK-8190703
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8190703/webrev

*) This change seems like a leftover:

  93 System.err.println("SystemGCTask with delay " + delayMS);

*) This change also does not look right -- should we instead increase delays in callers?

 101             ThreadUtils.sleep(delayMS * 10);


*) I think you can just say:

 for (int i = 0; (i < 4) && (System.currentTimeMillis() < endTime); i++)

 ...instead of rewriting the whole loop.

-Aleksey

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20171106/445f6dae/signature.asc>

From sangheon.kim at oracle.com  Mon Nov  6 20:46:22 2017
From: sangheon.kim at oracle.com (sangheon.kim)
Date: Mon, 6 Nov 2017 12:46:22 -0800
Subject: RFR: 8186502: Assert when range testing G1RefProcDrainInterval on
 64-bit systems
In-Reply-To: <b16d9282-4bf7-171e-136a-8b9df9350dff@oracle.com>
References: <00ad9ed1-8822-d696-cae3-6911190c4402@oracle.com>
 <532c6234-f3c3-b6c6-0cd2-95bbb8c6c439@oracle.com>
 <b16d9282-4bf7-171e-136a-8b9df9350dff@oracle.com>
Message-ID: <b96e499d-f0f8-389c-0b87-d8cbdc08c401@oracle.com>



On 11/06/2017 01:55 AM, Leo Korinth wrote:
> Hi Sangheon.
>
> On 04/11/17 00:36, sangheon.kim wrote:
> > Hi Leo,
> >
> > On 11/03/2017 02:24 AM, Leo Korinth wrote:
> >> Hi,
> >>
> >> Fix range checking when using -XX:G1RefProcDrainInterval=n
> >>
> >> On 64 bit machines, when issuing extremly large integer arguments
> >> to -XX:G1RefProcDrainInterval=n, a 32 bit integer will overflow and
> >> cause a crash.
> >>
> >> First I have improved the tests by adding the flag
> >> -XX:+ExplicitGCInvokesConcurrent when testing the flag
> >> -XX:G1RefProcDrainInterval=n to trigger the fault in our
> >> test suit.
> >>
> >> Then I have fixed the fault by limiting the arguments on 64 bit
> >> systems (using an int type instead of intx type and a range up to
> >> only INT_MAX for argument parsing).
> >>
> >> Bug:
> >> https://bugs.openjdk.java.net/browse/JDK-8186502
> >>
> >> Webrev:
> >> http://cr.openjdk.java.net/~lkorinth/8186502/00/
> > Looks good, thank you for fixing for this.
> > Just minor nit: you can update copyright year at g1_globals.hpp.
>
> Thanks, copyright year fixed in:
> http://cr.openjdk.java.net/~lkorinth/8186502/01/
Looks good.

>
> > BTW, did you run TestOptionsWithRanges.java manually?
> > Probably this test is still excluded by default when you run JPRT.
>
> Yes, first I updated the test so that it triggered the bug (and ran 
> the test manually to verify that it indeed triggered the bug). Then I 
> fixed the bug. After that I ran the test again (and the test passed).
>
> I am unsure if the test is enabled when you run JPRT though, but if it 
> is, it will pass.
I agree with you that the test will pass.

FYI, test/hotspot/jtreg/TEST.groups : line 153
-runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java \
: means this test will be excluded from 'hotspot_tier1_runtime' which is 
one of lists run via JPRT.
So to run via JPRT, you should remove that line.

Thanks,
Sangheon


>
> /Leo
>
> > Thanks,
> > Sangheon
> >
> >
> >>
> >> Testing:
> >> - JPRT
> >>
> >> Thanks,
> >> Leo
> >



From thomas.schatzl at oracle.com  Tue Nov  7 09:16:50 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Tue, 07 Nov 2017 10:16:50 +0100
Subject: RFR (S): 8190703: TestSystemGCWith* infrequently times out on
 SPARC
In-Reply-To: <8cb38009-4d26-2cdd-75a5-460857452aa5@redhat.com>
References: <1509967623.2750.2.camel@oracle.com>
 <8cb38009-4d26-2cdd-75a5-460857452aa5@redhat.com>
Message-ID: <1510046210.4175.2.camel@oracle.com>

Hi Alexey,

  thanks for your review, and sorry for the bad quality of the patch :(

On Mon, 2017-11-06 at 20:46 +0100, Aleksey Shipilev wrote:
> On 11/06/2017 12:27 PM, Thomas Schatzl wrote:
> > ? can I have reviews for this change the fixes intermittent
> > timeouts in
> > a stress test when run on slow machines and debug builds.
> > 
> > The fix is to simply let the test bail out early if time is up -
> > this
> > allows the test run through as intended in the many configurations
> > where it does not take excessive amount of time.
> 
> Makes sense. Real timeouts would still surface.
> 
> > CR:
> > https://bugs.openjdk.java.net/browse/JDK-8190703
> > Webrev:
> > http://cr.openjdk.java.net/~tschatzl/8190703/webrev
> 
> *) This change seems like a leftover:
> 
> ? 93 System.err.println("SystemGCTask with delay " + delayMS);

Yes.

> 
> *) This change also does not look right -- should we instead increase
> delays in callers?
> 
> ?101?????????????ThreadUtils.sleep(delayMS * 10);
> 

Another debugging leftover when I tried to come up with alternative
solutions.

> 
> *) I think you can just say:
> 
> ?for (int i = 0; (i < 4) && (System.currentTimeMillis() < endTime);
> i++)
> 
> ?...instead of rewriting the whole loop.

Done.

Also added some very basic check that a timeout is passed to the main
program.

http://cr.openjdk.java.net/~tschatzl/8190703/webrev.1/ (full patch)
http://cr.openjdk.java.net/~tschatzl/8190703/webrev.0_to_1/ (diff)

I rechecked that these changes do not change the execution times of the
test.

Thanks,
  Thomas



From per.liden at oracle.com  Tue Nov  7 09:49:39 2017
From: per.liden at oracle.com (Per Liden)
Date: Tue, 7 Nov 2017 10:49:39 +0100
Subject: RFR: 8189171: Move GC argument processing into GC specific classes
In-Reply-To: <3c75a6c5-ffe5-948f-b2c3-b4950a4b1a9f@redhat.com>
References: <e72385e3-88f1-9b05-8495-ab7f4c4cbd60@redhat.com>
 <ed0085ca-7384-05e7-4a2c-e2f048f64909@oracle.com>
 <5a7fecea-1a1b-16db-e366-4dd8d0c9ed63@redhat.com>
 <59E9E9A9.7020306@oracle.com>
 <511327b5-4632-0e27-6914-2a3d076a1dc3@redhat.com>
 <59EA002D.1050605@oracle.com>
 <2b37fed8-0383-eb19-3e45-076aabd74f22@redhat.com>
 <59F0AD40.3020307@oracle.com>
 <e8e953a1-06b3-648b-d5da-c87ee7c640ad@redhat.com>
 <B3E05F9C-5AC2-40D3-BBF3-A46ED9B12ED5@oracle.com>
 <affd801a-d184-1bcd-7c58-65c1ecaa94da@oracle.com>
 <3c75a6c5-ffe5-948f-b2c3-b4950a4b1a9f@redhat.com>
Message-ID: <3f188247-aea1-55d7-cbd4-8ddcd33aa4a8@oracle.com>

Looks good Roman!

/Per

On 2017-11-06 12:13, Roman Kennke wrote:
> Am 26.10.2017 um 11:43 schrieb Per Liden:
>> Hi,
>>
>> On 2017-10-25 18:11, Erik Osterlund wrote:
>> [...]
>>>> So let me just put your changes up for review (again), if you don't
>>>> mind:
>>>>
>>>> Full webrev:
>>>> http://cr.openjdk.java.net/~eosterlund/8189171/webrev.03/
>>>> <http://cr.openjdk.java.net/%7Eeosterlund/8189171/webrev.03/>
>>
>> I like this. Just two naming suggestions:
>>
>> src/hotspot/share/gc/shared/gcArguments.hpp:
>>
>>   39   static jint create_instance();
>>   40   static bool is_initialized();
>>   41   static GCArguments* instance();
>>
>> Could we call these:
>>
>>   39   static jint initialize();
>>   40   static bool is_initialized();
>>   41   static GCArguments* arguments();
>>
>> Reasoning: initialize() maps better to its companion is_initialized().
>> GCArguments::arguments() maps better to the existing patterns we have
>> with CollectedHeap::heap().
> Ok, that sounds good to me. Actually, it's almost full-circle back to my
> original proposal ;-)
>
> Differential:
> http://cr.openjdk.java.net/~rkennke/8189171/webrev.04.diff/
> <http://cr.openjdk.java.net/%7Erkennke/8189171/webrev.04.diff/>
> Full:
> http://cr.openjdk.java.net/~rkennke/8189171/webrev.04/
> <http://cr.openjdk.java.net/%7Erkennke/8189171/webrev.04/>
>
> Ok to push now?
>
> Roman


From rkennke at redhat.com  Tue Nov  7 11:01:15 2017
From: rkennke at redhat.com (Roman Kennke)
Date: Tue, 7 Nov 2017 12:01:15 +0100
Subject: RFR: 8189171: Move GC argument processing into GC specific classes
In-Reply-To: <3f188247-aea1-55d7-cbd4-8ddcd33aa4a8@oracle.com>
References: <e72385e3-88f1-9b05-8495-ab7f4c4cbd60@redhat.com>
 <ed0085ca-7384-05e7-4a2c-e2f048f64909@oracle.com>
 <5a7fecea-1a1b-16db-e366-4dd8d0c9ed63@redhat.com>
 <59E9E9A9.7020306@oracle.com>
 <511327b5-4632-0e27-6914-2a3d076a1dc3@redhat.com>
 <59EA002D.1050605@oracle.com>
 <2b37fed8-0383-eb19-3e45-076aabd74f22@redhat.com>
 <59F0AD40.3020307@oracle.com>
 <e8e953a1-06b3-648b-d5da-c87ee7c640ad@redhat.com>
 <B3E05F9C-5AC2-40D3-BBF3-A46ED9B12ED5@oracle.com>
 <affd801a-d184-1bcd-7c58-65c1ecaa94da@oracle.com>
 <3c75a6c5-ffe5-948f-b2c3-b4950a4b1a9f@redhat.com>
 <3f188247-aea1-55d7-cbd4-8ddcd33aa4a8@oracle.com>
Message-ID: <c57f6ad3-9629-214a-cc41-34ac0c3688ad@redhat.com>

Hi Per & Erik,

thanks for the reviews!

Now I need a sponsor.

Final webrev (same as before, including Reviewed-by line):
http://cr.openjdk.java.net/~rkennke/8189171/webrev.05/ 
<http://cr.openjdk.java.net/%7Erkennke/8189171/webrev.05/>

Thanks, Roman


> Looks good Roman!
>
> /Per
>
> On 2017-11-06 12:13, Roman Kennke wrote:
>> Am 26.10.2017 um 11:43 schrieb Per Liden:
>>> Hi,
>>>
>>> On 2017-10-25 18:11, Erik Osterlund wrote:
>>> [...]
>>>>> So let me just put your changes up for review (again), if you don't
>>>>> mind:
>>>>>
>>>>> Full webrev:
>>>>> http://cr.openjdk.java.net/~eosterlund/8189171/webrev.03/
>>>>> <http://cr.openjdk.java.net/%7Eeosterlund/8189171/webrev.03/>
>>>
>>> I like this. Just two naming suggestions:
>>>
>>> src/hotspot/share/gc/shared/gcArguments.hpp:
>>>
>>> ? 39?? static jint create_instance();
>>> ? 40?? static bool is_initialized();
>>> ? 41?? static GCArguments* instance();
>>>
>>> Could we call these:
>>>
>>> ? 39?? static jint initialize();
>>> ? 40?? static bool is_initialized();
>>> ? 41?? static GCArguments* arguments();
>>>
>>> Reasoning: initialize() maps better to its companion is_initialized().
>>> GCArguments::arguments() maps better to the existing patterns we have
>>> with CollectedHeap::heap().
>> Ok, that sounds good to me. Actually, it's almost full-circle back to my
>> original proposal ;-)
>>
>> Differential:
>> http://cr.openjdk.java.net/~rkennke/8189171/webrev.04.diff/
>> <http://cr.openjdk.java.net/%7Erkennke/8189171/webrev.04.diff/>
>> Full:
>> http://cr.openjdk.java.net/~rkennke/8189171/webrev.04/
>> <http://cr.openjdk.java.net/%7Erkennke/8189171/webrev.04/>
>>
>> Ok to push now?
>>
>> Roman




From stefan.johansson at oracle.com  Tue Nov  7 11:50:20 2017
From: stefan.johansson at oracle.com (Stefan Johansson)
Date: Tue, 7 Nov 2017 12:50:20 +0100
Subject: RFR (S): 8190703: TestSystemGCWith* infrequently times out on
 SPARC
In-Reply-To: <1510046210.4175.2.camel@oracle.com>
References: <1509967623.2750.2.camel@oracle.com>
 <8cb38009-4d26-2cdd-75a5-460857452aa5@redhat.com>
 <1510046210.4175.2.camel@oracle.com>
Message-ID: <f323d680-fe09-95e5-ce5c-766956fbb597@oracle.com>

Hi Thomas,

On 2017-11-07 10:16, Thomas Schatzl wrote:
>
> Also added some very basic check that a timeout is passed to the main
> program.
>
> http://cr.openjdk.java.net/~tschatzl/8190703/webrev.1/ (full patch)
> http://cr.openjdk.java.net/~tschatzl/8190703/webrev.0_to_1/ (diff)
These changes looks good, thanks for fixing this.
StefanJ
> I rechecked that these changes do not change the execution times of the
> test.
>
> Thanks,
>    Thomas
>



From shade at redhat.com  Tue Nov  7 12:22:33 2017
From: shade at redhat.com (Aleksey Shipilev)
Date: Tue, 7 Nov 2017 13:22:33 +0100
Subject: RFR (S): 8190703: TestSystemGCWith* infrequently times out on
 SPARC
In-Reply-To: <1510046210.4175.2.camel@oracle.com>
References: <1509967623.2750.2.camel@oracle.com>
 <8cb38009-4d26-2cdd-75a5-460857452aa5@redhat.com>
 <1510046210.4175.2.camel@oracle.com>
Message-ID: <fdf936fe-427b-447a-2ba9-4f3120a8e2d0@redhat.com>

On 11/07/2017 10:16 AM, Thomas Schatzl wrote:
> http://cr.openjdk.java.net/~tschatzl/8190703/webrev.1/ (full patch)
> http://cr.openjdk.java.net/~tschatzl/8190703/webrev.0_to_1/ (diff)

Looks good.

-Aleksey


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20171107/8c49c5b3/signature.asc>

From thomas.schatzl at oracle.com  Tue Nov  7 14:13:28 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Tue, 07 Nov 2017 15:13:28 +0100
Subject: RFR (S): 8190703: TestSystemGCWith* infrequently times out on
 SPARC
In-Reply-To: <fdf936fe-427b-447a-2ba9-4f3120a8e2d0@redhat.com>
References: <1509967623.2750.2.camel@oracle.com>
 <8cb38009-4d26-2cdd-75a5-460857452aa5@redhat.com>
 <1510046210.4175.2.camel@oracle.com>
 <fdf936fe-427b-447a-2ba9-4f3120a8e2d0@redhat.com>
Message-ID: <1510064008.4175.5.camel@oracle.com>

Hi Aleksey, Stefan,

On Tue, 2017-11-07 at 13:22 +0100, Aleksey Shipilev wrote:
> On 11/07/2017 10:16 AM, Thomas Schatzl wrote:
> > http://cr.openjdk.java.net/~tschatzl/8190703/webrev.1/ (full patch)
> > http://cr.openjdk.java.net/~tschatzl/8190703/webrev.0_to_1/ (diff)
> 
> Looks good.
> 
> -Aleksey

On Tue, 2017-11-07 at 12:50 +0100, Stefan Johansson wrote:
> Hi Thomas,
>?
[...]?
> These changes looks good, thanks for fixing this.
> StefanJ


  thanks for your reviews :) Is on its way...

Thomas



From kirk at kodewerk.com  Tue Nov  7 10:07:06 2017
From: kirk at kodewerk.com (Kirk Pepperdine)
Date: Tue, 7 Nov 2017 11:07:06 +0100
Subject: RFC: Epsilon GC JEP
In-Reply-To: <235b0c9f-af6e-a518-2e57-3a7ef4b38378@redhat.com>
References: <67f6d4a2-d129-1491-4906-473586dc6680@redhat.com>
 <235b0c9f-af6e-a518-2e57-3a7ef4b38378@redhat.com>
Message-ID: <FD3E29C3-F405-447F-9039-C3487D13E548@kodewerk.com>

Hi Aleksey,

After more thought about Epsilon GC I now believe I have run into production systems that would benefit from this work.

Kind regards,
Kirk Pepperdine

> On Nov 6, 2017, at 8:41 PM, Aleksey Shipilev <shade at redhat.com> wrote:
> 
> On 07/10/2017 10:14 PM, Aleksey Shipilev wrote:
>> I would like to solicit feedback on Epsilon GC JEP:
>>  https://bugs.openjdk.java.net/browse/JDK-8174901
>>  http://openjdk.java.net/jeps/8174901
> 
> Following up on this after the discussion, please add yourself to Reviewed-by (or Endorsed-by, if
> you are the group lead) in JEP!
> 
> Erik Helin, Roman Kennke, Thomas Schatzl, and Erik Osterlund replied in this thread. More reviews
> and endorsements are welcome.
> 
> Thanks,
> -Aleksey
> 
> 
> 



From stefan.johansson at oracle.com  Wed Nov  8 10:47:47 2017
From: stefan.johansson at oracle.com (Stefan Johansson)
Date: Wed, 8 Nov 2017 11:47:47 +0100
Subject: RFR: 8190711: Assert in G1MMUTracker due to concurrent modification
Message-ID: <b04d0032-9c84-cbdd-bc5f-6e846117d1a3@oracle.com>

Hi all,

Please review this fix for:
https://bugs.openjdk.java.net/browse/JDK-8190711

Webrev:
http://cr.openjdk.java.net/~sjohanss/8190711/00/

Summary:
The G1MMUTracker is used both by the concurrent gc threads and during 
STW by the VM thread. Up until this change we have had modifications 
done to the internal data structure from both sides. To synchronize we 
have used the SuspendibleThreadSet but this doesn't stop two different 
concurrent threads from modifying the data at the same time. We need to 
prevent this.

The reason to update the data structure (remove entires) in the 
concurrent phase is a possible performance improvement due to having 
fewer entries to iterate when calling when_sec(). Since there can never 
be more than 64 entries, the gain, if any, is very small. We still 
remove expired entries when adding new ones, done by add_pause() during 
STW. To avoid having more than one thread doing modification of the list 
this change removes the call to remove_expired_entries() in when_sec(). 
Having the SuspendibleThreadSet will ensure the the concurrent threads 
won't access the data while the VM thread is updating it.

Testing:
* Tier 1-2

Thanks,
Stefan


From thomas.schatzl at oracle.com  Wed Nov  8 11:37:37 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 08 Nov 2017 12:37:37 +0100
Subject: RFR: 8190711: Assert in G1MMUTracker due to concurrent
 modification
In-Reply-To: <b04d0032-9c84-cbdd-bc5f-6e846117d1a3@oracle.com>
References: <b04d0032-9c84-cbdd-bc5f-6e846117d1a3@oracle.com>
Message-ID: <1510141057.3155.5.camel@oracle.com>

Hi,

On Wed, 2017-11-08 at 11:47 +0100, Stefan Johansson wrote:
> Hi all,
> 
> Please review this fix for:
> https://bugs.openjdk.java.net/browse/JDK-8190711
> 
> Webrev:
> http://cr.openjdk.java.net/~sjohanss/8190711/00/
> 
> Summary:
> The G1MMUTracker is used both by the concurrent gc threads and
> during?STW by the VM thread. Up until this change we have had
> modifications?done to the internal data structure from both sides. To
> synchronize we have used the SuspendibleThreadSet but this doesn't
> stop two different?concurrent threads from modifying the data at the
> same time. We need to?prevent this.
> 
> The reason to update the data structure (remove entires) in the?
> concurrent phase is a possible performance improvement due to having?
> fewer entries to iterate when calling when_sec(). Since there can
> never?be more than 64 entries, the gain, if any, is very small. We
> still?remove expired entries when adding new ones, done by
> add_pause() during?STW. To avoid having more than one thread doing
> modification of the list?this change removes the call to
> remove_expired_entries() in when_sec().?
> Having the SuspendibleThreadSet will ensure the the concurrent
> threads?won't access the data while the VM thread is updating it.

  looks good.

There does not seem to be any difference in the number of entries we
can have in that list, as add_pause() is the only one adding entries
anyway.
As you note we might only get some more expired entries to work
through.

Thanks,
  Thomas



From robbin.ehn at oracle.com  Wed Nov  8 14:33:08 2017
From: robbin.ehn at oracle.com (Robbin Ehn)
Date: Wed, 8 Nov 2017 15:33:08 +0100
Subject: RFR: 8190711: Assert in G1MMUTracker due to concurrent
 modification
In-Reply-To: <b04d0032-9c84-cbdd-bc5f-6e846117d1a3@oracle.com>
References: <b04d0032-9c84-cbdd-bc5f-6e846117d1a3@oracle.com>
Message-ID: <eb25a445-52a4-2731-ac48-314b6e3f067d@oracle.com>

Thanks for fixing this fast!

 From what I can tell, looks good!

/Robbin

On 11/08/2017 11:47 AM, Stefan Johansson wrote:
> Hi all,
> 
> Please review this fix for:
> https://bugs.openjdk.java.net/browse/JDK-8190711
> 
> Webrev:
> http://cr.openjdk.java.net/~sjohanss/8190711/00/
> 
> Summary:
> The G1MMUTracker is used both by the concurrent gc threads and during STW by the 
> VM thread. Up until this change we have had modifications done to the internal 
> data structure from both sides. To synchronize we have used the 
> SuspendibleThreadSet but this doesn't stop two different concurrent threads from 
> modifying the data at the same time. We need to prevent this.
> 
> The reason to update the data structure (remove entires) in the concurrent phase 
> is a possible performance improvement due to having fewer entries to iterate 
> when calling when_sec(). Since there can never be more than 64 entries, the 
> gain, if any, is very small. We still remove expired entries when adding new 
> ones, done by add_pause() during STW. To avoid having more than one thread doing 
> modification of the list this change removes the call to 
> remove_expired_entries() in when_sec(). Having the SuspendibleThreadSet will 
> ensure the the concurrent threads won't access the data while the VM thread is 
> updating it.
> 
> Testing:
> * Tier 1-2
> 
> Thanks,
> Stefan


From aph at redhat.com  Wed Nov  8 15:03:42 2017
From: aph at redhat.com (Andrew Haley)
Date: Wed, 8 Nov 2017 15:03:42 +0000
Subject: RFC: Epsilon GC JEP
In-Reply-To: <FD3E29C3-F405-447F-9039-C3487D13E548@kodewerk.com>
References: <67f6d4a2-d129-1491-4906-473586dc6680@redhat.com>
 <235b0c9f-af6e-a518-2e57-3a7ef4b38378@redhat.com>
 <FD3E29C3-F405-447F-9039-C3487D13E548@kodewerk.com>
Message-ID: <74bad8d0-b038-0f8e-5c6b-66e5c230fa65@redhat.com>

On 07/11/17 10:07, Kirk Pepperdine wrote:
> After more thought about Epsilon GC I now believe I have run into production systems that would benefit from this work.

So have I.  It's a low-risk change and we should make it available to users.

-- 
Andrew Haley
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From sangheon.kim at oracle.com  Wed Nov  8 15:07:07 2017
From: sangheon.kim at oracle.com (sangheon.kim)
Date: Wed, 8 Nov 2017 07:07:07 -0800
Subject: RFR: 8190711: Assert in G1MMUTracker due to concurrent
 modification
In-Reply-To: <b04d0032-9c84-cbdd-bc5f-6e846117d1a3@oracle.com>
References: <b04d0032-9c84-cbdd-bc5f-6e846117d1a3@oracle.com>
Message-ID: <351826c5-47a6-ac91-9fa8-83d0316c57a4@oracle.com>

Hi Stefan,

Thank you for fixing this bug!

On 11/08/2017 02:47 AM, Stefan Johansson wrote:
> Hi all,
>
> Please review this fix for:
> https://bugs.openjdk.java.net/browse/JDK-8190711
>
> Webrev:
> http://cr.openjdk.java.net/~sjohanss/8190711/00/
>
> Summary:
> The G1MMUTracker is used both by the concurrent gc threads and during 
> STW by the VM thread. Up until this change we have had modifications 
> done to the internal data structure from both sides. To synchronize we 
> have used the SuspendibleThreadSet but this doesn't stop two different 
> concurrent threads from modifying the data at the same time. We need 
> to prevent this.
>
> The reason to update the data structure (remove entires) in the 
> concurrent phase is a possible performance improvement due to having 
> fewer entries to iterate when calling when_sec(). Since there can 
> never be more than 64 entries, the gain, if any, is very small. We 
> still remove expired entries when adding new ones, done by add_pause() 
> during STW. To avoid having more than one thread doing modification of 
> the list this change removes the call to remove_expired_entries() in 
> when_sec(). Having the SuspendibleThreadSet will ensure the the 
> concurrent threads won't access the data while the VM thread is 
> updating it.
Looks good.

Thanks,
Sangheon


>
> Testing:
> * Tier 1-2
>
> Thanks,
> Stefan



From daniel.daugherty at oracle.com  Wed Nov  8 17:49:43 2017
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Wed, 8 Nov 2017 12:49:43 -0500
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <dd56dd01-0bc1-8a94-157a-e3020e42eebc@oracle.com>
References: <d09b6490-b681-5a08-f348-1649ba44eda4@oracle.com>
 <dd56dd01-0bc1-8a94-157a-e3020e42eebc@oracle.com>
Message-ID: <543b5bd6-1f5e-37df-d4fc-199b4e67fca3@oracle.com>

Greetings,

This code review from Stefan Karlsson was originally posted on an Oracle
internal alias that we use for discussing HotSpot SMR development issues.
That subject was: "Thread-SMR (8167108)(JDK10): CR round 0 changes". I did
not have time to address Stefan's review before I went on vacation and
Stefan graciously allowed me to defer his review to the OpenJDK review.

With Stefan's permission, I'm replying to his code review on the OpenJDK
aliases that we're using for the JDK-8167108 code review.


On 10/6/17 1:19 PM, Stefan Karlsson wrote:
> Hi Dan,
>
> Great that this is getting done!

Thanks! It has been an adventure for all three primary contributors...


> Erik already mentioned the file so I think that's on track, and that's 
> what I was most concerned about.
>
>
> I have not been involved in the review of this patch at all, but now 
> that I've been looking at it I have a few comments. I hope you don't 
> mind. I don't want them to block the open review request, but at the 
> same time I'd like to share my opinion and have it weighted in for the 
> future of this code.
>
> 1) ThreadsListHandle allows a safepoint to block and allow GCs to run 
> and restructure objects and metadata, iff the ThreadsListHandle is 
> nested. This forced me to review all usages of ThreadsListHandle in 
> great detail to visually verify that it didn't break the surrounding code.
>
> If ThreadsListHandle didn't ever block for safepoints, I wouldn't have 
> had to care about that aspect when reading and reviewing the code. 
> This also means that we in the future runs the risk of someone 
> accidentally adding a nested ThreadsListHandle somewhere where they 
> don't expect a safepoint to happen.

We included the following to automatically sanity check the placement
of the ThreadsListHandle:

src/hotspot/share/runtime/thread.cpp:

3596 ThreadsList *Threads::acquire_stable_list(Thread *self, bool 
is_ThreadsListSetter) {
3597?? assert(self != NULL, "sanity check");
3598?? // acquire_stable_list_nested_path() will grab the Threads_lock
3599?? // so let's make sure the ThreadsListHandle is in a safe place.
3600?? // ThreadsListSetter cannot make this check on this code path.
3601?? debug_only(if (!is_ThreadsListSetter && StrictSafepointChecks) 
self->check_for_valid_safepoint_state(/* potential_vm_operation */ false);)

So each ThreadsListHandle location is checked for being a valid
safepoint state.

We tested this idea with our work on JvmtiEnv::GetThreadLocalStorage().
GetThreadLocalStorage() is called from the 'in native' state so we
cannot place a common ThreadsListHandle for all code paths because
it would fail the assertion when called with 'thread == NULL', i.e.,
the thread is operating on itself.

Update: Erik is going to explore a lock free solution for the
nesting algorithm. That will likely mean that a ThreadsListHandle
will not require placement in a safepoint safe location...


> If the lock protecting the threads list could be taken with 
> no_safepoint_check, then the described problem above would completely 
> vanish. Unfortunately, we can't acquire the Threads_lock with 
> no_safepoint_check. A solution to this would be to introduced a 
> low-rank (rank == access) lock, say ThreadsList_lock, and always take 
> it with no_safepoint_check.

The problem with switching locks is that we are protecting the scanning
phase of the SMR algorithm with the Threads_lock so we would have to
switch that lock also. I believe we use the Threads_lock with the
scanning because we're using closures... Of course, Erik or Robbin
should probably jump in here... :-)

Update: Erik is going to explore a lock-free solution for the
nesting algorithm.

> That solution would also solve a potential lock-rank problem, we have 
> that ThreadsListHandle can't be used if a higher-rank lock is held.

So far we haven't run into that problem and we think that the
check_for_valid_safepoint_state() will catch that if the code
should evolve to introduce one.


Update: Erik is going to explore a lock-free solution for the
nesting algorithm.


> 2) The following one-liner:
> - for (JavaThread* t = Threads::first(); t; t = t->next()) {
> has been converted to a five-liner:
>
> + {
> + ThreadsListHandle tlh;
> + JavaThreadIterator jti(tlh.list());
> + for (JavaThread* t = jti.first(); t != NULL; t = jti.next()) {
> ... + } I find that unfortunate, and would prefer if this was 
> rewritten. For example: for (JavaThreadIterator jti; JavaThread* t = 
> jit.next();) { jti will implicitly hold the ThreadListHandle on the 
> stack. I know that the implicit conversion of pointer to bool is 
> non-conventional in the HotSpot code, but in this case I prefer that 
> over five lines of extra noise in the GC code.

Coleen made a similar comment in her OpenJDK review:

> Hi,? From my initial look at this, I really like new interface for
> walking the threads list, except every instance but 2 uses of
> JavaThreadIterator has a preceding ThreadsListHandle declaration.
>
> +? ThreadsListHandle tlh;
> +? JavaThreadIterator jti(tlh.list());
>
>
> Could there be a wrapper that embeds the ThreadsListHandle, like:
>
> class JavaThreadsListIterator {
> ??? ThreadsListHandle _tlh;
> ...
> } 


Yup a definite code noise problem. We originally didn't have an
iterator and used direct thread_at(foo) calls so we had a pretty
close correspondence between the old for-loop and the new for-loop.
Early reviewers asked for an iterator abstraction so we modeled
JavaThreadIterator after other existing iterators in the JVM/TI
area... (Yes, Dan stayed pretty close to "home" here...)

There are places where we still need the existing JavaThreadIterator
because a ThreadsList is passed down a call stack... So we've added
a JavaThreadIteratorWithHandle class.

Here's an example of the noisy code in the GC area:

src/hotspot/share/gc/g1/dirtyCardQueue.cpp:

@@ -319,8 +320,12 @@
 ?? clear();
 ?? // Since abandon is done only at safepoints, we can safely manipulate
 ?? // these queues.
-? for (JavaThread* t = Threads::first(); t; t = t->next()) {
-??? t->dirty_card_queue().reset();
+? {
+??? ThreadsListHandle tlh;
+??? JavaThreadIterator jti(tlh.list());
+??? for (JavaThread* t = jti.first(); t != NULL; t = jti.next()) {
+????? t->dirty_card_queue().reset();
+??? }
 ?? }
 ?? shared_dirty_card_queue()->reset();
 ?}

Here's an example of the revised code in the GC area:

src/hotspot/share/gc/g1/dirtyCardQueue.cpp:

@@ -319,7 +320,7 @@
 ?? clear();
 ?? // Since abandon is done only at safepoints, we can safely manipulate
 ?? // these queues.
-? for (JavaThread* t = Threads::first(); t; t = t->next()) {
+? for (JavaThreadIteratorWithHandle jtiwh; JavaThread *t = 
jtiwh.next(); ) {
 ???? t->dirty_card_queue().reset();
 ?? }
 ?? shared_dirty_card_queue()->reset();

This is definitely much less noisy and I'm hoping I don't catch too
much grief for the implied boolean usage...


> 3) cv_internal_thread_to_JavaThread Has one return value and two 
> output parameter. It forces the callers to setup stack lock variables 
> for the two new variables. --- Used to be ---
> - oop java_thread = JNIHandles::resolve_non_null(jthread); 
> java_lang_Thread::set_priority(java_thread, (ThreadPriority)prio); - 
> JavaThread* thr = java_lang_Thread::thread(java_thread); - if (thr != 
> NULL) { // Thread not yet started; priority pushed down when it is - 
> Thread::set_priority(thr, (ThreadPriority)prio);
> --- Now is ---
> + oop java_thread = NULL; + JavaThread* receiver = NULL; + bool 
> is_alive = tlh.cv_internal_thread_to_JavaThread(jthread, &receiver, 
> &java_thread); java_lang_Thread::set_priority(java_thread, 
> (ThreadPriority)prio); + + if (is_alive) { + // jthread refers to a 
> live JavaThread. + Thread::set_priority(receiver, 
> (ThreadPriority)prio); --- Maybe could be ---
> oop java_thread = JNIHandles::resolve_non_null(jthread); 
> java_lang_Thread::set_priority(java_thread, (ThreadPriority)prio); 
> JavaThread* thr = tlh.cv_internal_thread_to_JavaThread(java_thread); 
> if (thr != NULL) { // Thread not yet started; priority pushed down 
> when it is Thread::set_priority(thr, (ThreadPriority)prio); I don't 
> see the need to for the extra complexity of the two output parameters. 
> The return value is always true when thr is non-null, and always false 
> when thr is null.

There are three cv_*_to_JavaThread() functions and we had to craft them
very, very carefully to avoid changing some of the subtle semantics at
the original call sites. If you carefully examine the output parameters
and return value usages at all of the call sites, you'll see that each
was added to fit some existing semantic that we didn't want to risk
changing. Of course, not all of the call sites need all of the special
behaviors individually, but they do as a union.

In short, compatibility and risk management led us here.

> But the usage of cv_internal_thread_to_JavaThread is contained within 
> the Runtime code, so my opinion should probably not weigh as much as 
> other's opinions. :)

We definitely agree this is a mess, but not one we're willing to risk
changing at this time. Dan has made the observation that the JVM_*
entry points, like Y2K code, shows a variety of different coding
patterns, each with different (potential) issues...

Thanks for being flexible on accepting cv_internal_thread_to_JavaThread()
as is for now!


> 4) I'd prefer if abbreviations where expanded, so that the casual 
> reader would immediately understood the code. For example: t_list -> 
> thread_list cv_internal_thread_to_JavaThread -> 
> internal_thread_to_JavaThread

We've had requests to shorten names, spell out names, use different
names, etc. It seems that no one is going to be happy with all of
the names we used in this code.

Our guidelines:

- use the same consistently, e.g., t_list for ThreadsLists
- use a name that doesn't show up in an existing grep (if possible)

We hope you don't mind, but we're not planning to make any more name
changes...


> 5) This macro and the jesting is pretty bad.

Yup!


> I complained about it to Erik, and then he confessed that he wrote it :D
> +// Possibly the ugliest for loop the world has seen. C++ does not 
> allow +// multiple types in the declaration section of the for loop. 
> In this case +// we are only dealing with pointers and hence can cast 
> them. It looks ugly +// but macros are ugly and therefore it's fine to 
> make things absurdly ugly. +#define DO_JAVA_THREADS(LIST, X) \ + for 
> (JavaThread *MACRO_scan_interval = 
> (JavaThread*)(uintptr_t)PrefetchScanIntervalInBytes, \ + *MACRO_list = 
> (JavaThread*)(LIST), \ + **MACRO_end = 
> ((JavaThread**)((ThreadsList*)MACRO_list)->threads()) + 
> ((ThreadsList*)MACRO_list)->length(), \ + **MACRO_current_p = 
> (JavaThread**)((ThreadsList*)MACRO_list)->threads(), \ + *X = 
> (JavaThread*)prefetch_and_load_ptr((void**)MACRO_current_p, 
> (intx)MACRO_scan_interval); \ + MACRO_current_p != MACRO_end; \ + 
> MACRO_current_p++, \ + X = 
> (JavaThread*)prefetch_and_load_ptr((void**)MACRO_current_p, 
> (intx)MACRO_scan_interval)) + This can be rewritten without all these 
> cast, and minimal usage of the macros expansions: struct 
> JavaThreadPrefetchedIterator { ThreadList* _list; JavaThread** _end; 
> JavaThread** _current; JavaThreadIteror(ThreadList* list) : 
> _list(list), _end(list->threads() + list->length()), 
> _current(list->threads()) {} JavaThread* current() { return 
> context._current != context._end ?? 
> prefetch_and_load_ptr(context._current, PrefetchScanIntervalInBytes) 
> ?: NULL) // ^ prefetch would be rewritten to return JavaThread* and 
> not void* } void next() { _current++; }; #define DO_JAVA_THREADS(LIST, 
> X) \ for (JavaThreadPrefetchedIterator iter(LIST); JavaThread* X = 
> iter.current(); iter.next()) (I did some changes to the code above and 
> haven't compiled this exact version)

Erik hasn't chimed in on this comment so I (Dan) am not planning to
try to resolve this comment in this round.

Update: Erik is planning to look at cleaning up the ugly macro...


> 6) I think it would be nice if the SMR stuff in thread.hpp were 
> encapsulated into an class instead of added directly to Thread and 
> Threads. I sort-of expected the SMR variables to be moved to 
> threadSMR.hpp. For example:
> class Threads: AllStatic { friend class VMStructs; private: + // Safe 
> Memory Reclamation (SMR) support: + static Monitor* _smr_delete_lock; 
> + // The '_cnt', '_max' and '_times" fields are enabled via + // 
> -XX:+EnableThreadSMRStatistics: + static uint 
> _smr_delete_lock_wait_cnt; + static uint _smr_delete_lock_wait_max; + 
> static volatile int _smr_delete_notify; + static volatile jint 
> _smr_deleted_thread_cnt; + static volatile jint 
> _smr_deleted_thread_time_max; + static volatile jint 
> _smr_deleted_thread_times; + static ThreadsList* volatile 
> _smr_java_thread_list; + static ThreadsList* 
> get_smr_java_thread_list() { + return 
> (ThreadsList*)OrderAccess::load_ptr_acquire((void* 
> volatile*)&_smr_java_thread_list); + } + static ThreadsList* 
> xchg_smr_java_thread_list(ThreadsList* new_list) { + return 
> (ThreadsList*)Atomic::xchg_ptr((void*)new_list, (volatile 
> void*)&_smr_java_thread_list); + } + static long 
> _smr_java_thread_list_alloc_cnt; + static long 
> _smr_java_thread_list_free_cnt; + static uint 
> _smr_java_thread_list_max; + static uint _smr_nested_thread_list_max; 
> + static volatile jint _smr_tlh_cnt; + static volatile jint 
> _smr_tlh_time_max; + static volatile jint _smr_tlh_times; + static 
> ThreadsList* _smr_to_delete_list; + static uint 
> _smr_to_delete_list_cnt; + static uint _smr_to_delete_list_max; Could 
> be: class Threads: AllStatic { friend class VMStructs; private: // 
> Safe Memory Reclamation (SMR) support: SMRSupport _smr_support; And 
> SMRSupport could be moved to threadSMR.hpp. I haven't read all the 
> code in detail, so I'm not sure if this is feasible or not, but my 
> gut-feeling says that it would be worth testing. The above is just an 
> example, and the rest of the code could probably be encapsulated and 
> moved as well.

We already moved all of the SMR stuff that was "easy" to move based
on your earlier comment. (Of course, doing that move introduced a
number of compilation complications, but that's just Dan complaining :-))

We don't really want to try this migration at this time since we're
still hoping to get this code into 18.3. (Also Dan doesn't see the
value in moving the SMR fields... Threads is already an eclectic static
class so why does SMR have to encapsulate when no other project did so?)


> 7) Currently, threadSMR.hpp "only" contains the ThreadList. Unless, 
> you move the SMR stuff into threadSMR.hpp, maybe it should be renamed 
> to threadsList.hpp? Maybe it does make sense to have both 
> threadSMR.hpp and threadsList.hpp?

We're planning to stick with the threadSMR.hpp and threadSMR.cpp
file names. Currently they contain the more standalone pieces of
thread SMR (more than just ThreadsList) so we're planning to keep
those names.

Thanks for the detailed review!!

Dan, Erik and Robbin


> Cheers and thanks! StefanK

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20171108/ed5e72d7/attachment.htm>

From daniel.daugherty at oracle.com  Wed Nov  8 17:53:22 2017
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Wed, 8 Nov 2017 12:53:22 -0500
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <2b180329-9cf3-d83b-133b-ddd5d33b4f1f@oracle.com>
References: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
 <2b180329-9cf3-d83b-133b-ddd5d33b4f1f@oracle.com>
Message-ID: <10022dbf-74a6-5b38-265e-079ebaad22c8@oracle.com>

Added back the other three OpenJDK aliases being used for this review...

Coleen, thanks for chiming in on this review thread. Sorry for the
delay in responding... I was on vacation...


On 10/11/17 11:32 AM, coleen.phillimore at oracle.com wrote:
>
> Hi,? From my initial look at this, I really like new interface for 
> walking the threads list, except every instance but 2 uses of 
> JavaThreadIterator has a preceding ThreadsListHandle declaration.
>
> +? ThreadsListHandle tlh;
> +? JavaThreadIterator jti(tlh.list());
>
>
> Could there be a wrapper that embeds the ThreadsListHandle, like:
>
> class JavaThreadsListIterator {
> ??? ThreadsListHandle _tlh;
> ...
> }

This issue has been addressed as part of my response to Stefan K's
code review comments. I quoted the above comment in that reply so
it should be easy to find that resolution...

Dan


>
> Thanks,
> Coleen
>
> On 10/9/17 3:41 PM, Daniel D. Daugherty wrote:
>> Greetings,
>>
>> We have a (eXtra Large) fix for the following bug:
>>
>> 8167108 inconsistent handling of SR_lock can lead to crashes
>> https://bugs.openjdk.java.net/browse/JDK-8167108
>>
>> This fix adds a Safe Memory Reclamation (SMR) mechanism based on
>> Hazard Pointers to manage JavaThread lifecycle.
>>
>> Here's a PDF for the internal wiki that we've been using to describe
>> and track the work on this project:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/SMR_and_JavaThread_Lifecycle-JDK10-04.pdf 
>>
>>
>> Dan has noticed that the indenting is wrong in some of the code quotes
>> in the PDF that are not present in the internal wiki. We don't have a
>> solution for that problem yet.
>>
>> Here's the webrev for current JDK10 version of this fix:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-04-full
>>
>> This fix has been run through many rounds of JPRT and Mach5 tier[2-5]
>> testing, additional stress testing on Dan's Solaris X64 server, and
>> additional testing on Erik and Robbin's machines.
>>
>> We welcome comments, suggestions and feedback.
>>
>> Daniel Daugherty
>> Erik Osterlund
>> Robbin Ehn
>



From daniel.daugherty at oracle.com  Wed Nov  8 18:05:57 2017
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Wed, 8 Nov 2017 13:05:57 -0500
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
References: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
Message-ID: <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>

Greetings,

Resolving one of the code review comments (from both Stefan K and Coleen)
on jdk10-04-full required quite a few changes so it is being done as a
standalone patch and corresponding webrevs:

Here's the mq comment for the change:

 ? stefank, coleenp CR - refactor most JavaThreadIterator usage to use
 ??? JavaThreadIteratorWithHandle.

Here is the full webrev:

http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-full/

And here is the delta webrev:

http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-delta/

We welcome comments, suggestions and feedback.

Dan, Erik and Robbin


On 10/9/17 3:41 PM, Daniel D. Daugherty wrote:
> Greetings,
>
> We have a (eXtra Large) fix for the following bug:
>
> 8167108 inconsistent handling of SR_lock can lead to crashes
> https://bugs.openjdk.java.net/browse/JDK-8167108
>
> This fix adds a Safe Memory Reclamation (SMR) mechanism based on
> Hazard Pointers to manage JavaThread lifecycle.
>
> Here's a PDF for the internal wiki that we've been using to describe
> and track the work on this project:
>
> http://cr.openjdk.java.net/~dcubed/8167108-webrev/SMR_and_JavaThread_Lifecycle-JDK10-04.pdf 
>
>
> Dan has noticed that the indenting is wrong in some of the code quotes
> in the PDF that are not present in the internal wiki. We don't have a
> solution for that problem yet.
>
> Here's the webrev for current JDK10 version of this fix:
>
> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-04-full
>
> This fix has been run through many rounds of JPRT and Mach5 tier[2-5]
> testing, additional stress testing on Dan's Solaris X64 server, and
> additional testing on Erik and Robbin's machines.
>
> We welcome comments, suggestions and feedback.
>
> Daniel Daugherty
> Erik Osterlund
> Robbin Ehn
>



From daniel.daugherty at oracle.com  Wed Nov  8 23:09:41 2017
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Wed, 8 Nov 2017 18:09:41 -0500
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>
References: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
 <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>
Message-ID: <1c0edbe2-1b9c-dc7e-c610-4ea053831790@oracle.com>

The jdk10-05-full bits have passed JPRT testing (hs-tier1 testing) and
hs-tier[2-5] testing via Mach 5. No unexpected test failures.

Dan



On 11/8/17 1:05 PM, Daniel D. Daugherty wrote:
> Greetings,
>
> Resolving one of the code review comments (from both Stefan K and Coleen)
> on jdk10-04-full required quite a few changes so it is being done as a
> standalone patch and corresponding webrevs:
>
> Here's the mq comment for the change:
>
> ? stefank, coleenp CR - refactor most JavaThreadIterator usage to use
> ??? JavaThreadIteratorWithHandle.
>
> Here is the full webrev:
>
> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-full/
>
> And here is the delta webrev:
>
> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-delta/
>
> We welcome comments, suggestions and feedback.
>
> Dan, Erik and Robbin
>
>
> On 10/9/17 3:41 PM, Daniel D. Daugherty wrote:
>> Greetings,
>>
>> We have a (eXtra Large) fix for the following bug:
>>
>> 8167108 inconsistent handling of SR_lock can lead to crashes
>> https://bugs.openjdk.java.net/browse/JDK-8167108
>>
>> This fix adds a Safe Memory Reclamation (SMR) mechanism based on
>> Hazard Pointers to manage JavaThread lifecycle.
>>
>> Here's a PDF for the internal wiki that we've been using to describe
>> and track the work on this project:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/SMR_and_JavaThread_Lifecycle-JDK10-04.pdf 
>>
>>
>> Dan has noticed that the indenting is wrong in some of the code quotes
>> in the PDF that are not present in the internal wiki. We don't have a
>> solution for that problem yet.
>>
>> Here's the webrev for current JDK10 version of this fix:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-04-full
>>
>> This fix has been run through many rounds of JPRT and Mach5 tier[2-5]
>> testing, additional stress testing on Dan's Solaris X64 server, and
>> additional testing on Erik and Robbin's machines.
>>
>> We welcome comments, suggestions and feedback.
>>
>> Daniel Daugherty
>> Erik Osterlund
>> Robbin Ehn
>>
>
>



From thomas.schatzl at oracle.com  Thu Nov  9 12:17:37 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 09 Nov 2017 13:17:37 +0100
Subject: RFR (XS): 8186480: Remove warning when AssumeMP is true and
 ergonomics determines to use one worker thread
Message-ID: <1510229857.2808.7.camel@oracle.com>

Hi all,

  can I have reviews for this change that removes some imho useless
warning that confuses some tests?

In particular, if AssumeMP is true, and ergonomics determines that it
only wants to use one thread (e.g. in a container/VM), it gives a
*warning * about that AssumeMP and the ergonomics decision seems to
contradict each other.

The problem is that, some time ago AssumeMP has been made true default,
and already deprecated in 10 (JDK-8185062) and the plan is to obsolete
it, so this message shows up in some tests if run on e.g. a single-
thread VM.

I suggest to remove this message because of this and it does not seem
to be terribly useful: to get only one thread by ergonomics you need to
explicitly run the VM in an environment that only provides a single
thread. It seems logical that in this case the user can expect (given
he relies on ergonomics) that the GC will only use one thread.

And alternative could be to only show this message if the user
explicitly sets AssumeMP to true, but given that AssumeMP is on its way
out and the above reasoning about the usefulness of the message I think
it is okay to just remove that message.

CR:
https://bugs.openjdk.java.net/browse/JDK-8186480
Webrev:
http://cr.openjdk.java.net/~tschatzl/8186480/webrev/
Testing:
compilation

Thanks,
  Thomas



From shade at redhat.com  Thu Nov  9 13:01:48 2017
From: shade at redhat.com (Aleksey Shipilev)
Date: Thu, 9 Nov 2017 14:01:48 +0100
Subject: RFR (XS): 8186480: Remove warning when AssumeMP is true and
 ergonomics determines to use one worker thread
In-Reply-To: <1510229857.2808.7.camel@oracle.com>
References: <1510229857.2808.7.camel@oracle.com>
Message-ID: <96b3e06d-5eb1-b792-8eba-b9a54b8449a3@redhat.com>

On 11/09/2017 01:17 PM, Thomas Schatzl wrote:
> The problem is that, some time ago AssumeMP has been made true default,
> and already deprecated in 10 (JDK-8185062) and the plan is to obsolete
> it, so this message shows up in some tests if run on e.g. a single-
> thread VM.
> 
> I suggest to remove this message because of this and it does not seem
> to be terribly useful: to get only one thread by ergonomics you need to
> explicitly run the VM in an environment that only provides a single
> thread. It seems logical that in this case the user can expect (given
> he relies on ergonomics) that the GC will only use one thread.
> 
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8186480
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8186480/webrev/

That makes sense. Looks good to me.

-Aleksey


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20171109/769be375/signature.asc>

From thomas.schatzl at oracle.com  Thu Nov  9 14:29:32 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 09 Nov 2017 15:29:32 +0100
Subject: RFR (XS): 8186480: Remove warning when AssumeMP is true and
 ergonomics determines to use one worker thread
In-Reply-To: <96b3e06d-5eb1-b792-8eba-b9a54b8449a3@redhat.com>
References: <1510229857.2808.7.camel@oracle.com>
 <96b3e06d-5eb1-b792-8eba-b9a54b8449a3@redhat.com>
Message-ID: <1510237772.4685.0.camel@oracle.com>

Hi Aleksey,

On Thu, 2017-11-09 at 14:01 +0100, Aleksey Shipilev wrote:
> On 11/09/2017 01:17 PM, Thomas Schatzl wrote:
> > The problem is that, some time ago AssumeMP has been made true
> > default,
> > 
> > [...]
> > CR:
> > https://bugs.openjdk.java.net/browse/JDK-8186480
> > Webrev:
> > http://cr.openjdk.java.net/~tschatzl/8186480/webrev/
> 
> That makes sense. Looks good to me.
> 

 thanks for your review.

Thomas



From sangheon.kim at oracle.com  Thu Nov  9 23:00:08 2017
From: sangheon.kim at oracle.com (sangheon.kim)
Date: Thu, 9 Nov 2017 15:00:08 -0800
Subject: RFR (M): 8190426: Lazily initialize refinement threads with
 UseDynamicNumberOfGCThreads
In-Reply-To: <1509726495.2630.2.camel@oracle.com>
References: <1509726495.2630.2.camel@oracle.com>
Message-ID: <1f3e5ef0-60f3-770f-f581-8c7764e44f2e@oracle.com>

Hi Thomas,

On 11/03/2017 09:28 AM, Thomas Schatzl wrote:
> Hi,
>
>    can I have reviews for this change that makes refinement threads
> behave the same as the other thread groups we have in G1?
>
> I.e. with UseDynamicNumberOfGCThreads enabled (which is off by default)
> they are created lazily.
>
> This helps a little bit further with startup/footprint benchmarks (if
> enabled).
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8190426
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8190426/webrev/
> Testing:
> hs-tier1+2
I like this idea, thank you for improving this.

I have some comments.
------------------------------------------------
src/hotspot/share/gc/g1/g1ConcurrentRefine.cpp
37?? _cg1r(NULL),
- Probably _cr or _g1cr.

54?? assert(cg1r != NULL, "Passed g1ConcurrentRefine must not be NULL");
- G1ConcurrentRefine not g1ConcurrentRefine.

57?? _threads = NEW_C_HEAP_ARRAY(G1ConcurrentRefineThread*, 
num_max_threads, mtGC);
58?? for (uint i = 0; i < num_max_threads; i++) {
- Don't we need NULL check at line 58?

67 void G1ConcurrentRefineThreadControl::maybe_activate_next(uint 
cur_worker_id) {
- without 'next' seems better or 'maybe_activate_thread()'.

68?? assert(cur_worker_id < _num_max_threads, "Tried to activate from 
impossible thread %u", cur_worker_
- I would prefer more detailed logging rather than 'impossible'. e.g. 
Activating current worker id exceeds maximum number of threads.

78???? _threads[worker_id] = new G1ConcurrentRefineThread(_cg1r, worker_id);
79???? thread_to_activate = _threads[worker_id];
- NULL check at line 79?
- If we fail to allocate and G1ConcRefinementThreads is set, printing 
warning message would needed.

81?? thread_to_activate->activate();
- Probably NULL check is needed for 'thread_to_activate'.

198?? _thread_control.initialize(this, max_num_threads());
- G1ConcurrentRefine::create() seems proper location to treat allocation 
failure.

389 void G1ConcurrentRefine::maybe_activate_more_threads(uint worker_id, 
size_t num_cur_buffers) {
- maybe_activate_more_thread() as this will activate only 1 thread.

------------------------------------------------
src/hotspot/share/gc/g1/g1ConcurrentRefineThread.hpp
62?? G1ConcurrentRefineThread(G1ConcurrentRefine* cg1r, uint worker_id);
- cg1r -> cr or g1cr?

Thanks,
Sangheon


> Thanks,
>    Thomas



From daniel.daugherty at oracle.com  Mon Nov 13 17:30:57 2017
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Mon, 13 Nov 2017 12:30:57 -0500
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>
References: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
 <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>
Message-ID: <836b5c6c-5eef-9245-c16a-e71372093d9a@oracle.com>

Greetings,

I rebased the project to the 2017.10.26 jdk10/hs PIT snapshot.

Here are the updated webrevs:

Here's the mq comment for the change:

 ? Rebase to 2017.10.25 PIT snapshot.

Here is the full webrev:

http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-full/

And here is the delta webrev:

http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-delta/

I ran the above bits throught Mach5 tier[1-5] testing over the holiday
weekend. Didn't see any issues in a quick look. Going to take a closer
look today.

We welcome comments, suggestions and feedback.

Dan, Erik and Robbin


On 11/8/17 1:05 PM, Daniel D. Daugherty wrote:
> Greetings,
>
> Resolving one of the code review comments (from both Stefan K and Coleen)
> on jdk10-04-full required quite a few changes so it is being done as a
> standalone patch and corresponding webrevs:
>
> Here's the mq comment for the change:
>
> ? stefank, coleenp CR - refactor most JavaThreadIterator usage to use
> ??? JavaThreadIteratorWithHandle.
>
> Here is the full webrev:
>
> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-full/
>
> And here is the delta webrev:
>
> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-delta/
>
> We welcome comments, suggestions and feedback.
>
> Dan, Erik and Robbin
>
>
> On 10/9/17 3:41 PM, Daniel D. Daugherty wrote:
>> Greetings,
>>
>> We have a (eXtra Large) fix for the following bug:
>>
>> 8167108 inconsistent handling of SR_lock can lead to crashes
>> https://bugs.openjdk.java.net/browse/JDK-8167108
>>
>> This fix adds a Safe Memory Reclamation (SMR) mechanism based on
>> Hazard Pointers to manage JavaThread lifecycle.
>>
>> Here's a PDF for the internal wiki that we've been using to describe
>> and track the work on this project:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/SMR_and_JavaThread_Lifecycle-JDK10-04.pdf 
>>
>>
>> Dan has noticed that the indenting is wrong in some of the code quotes
>> in the PDF that are not present in the internal wiki. We don't have a
>> solution for that problem yet.
>>
>> Here's the webrev for current JDK10 version of this fix:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-04-full
>>
>> This fix has been run through many rounds of JPRT and Mach5 tier[2-5]
>> testing, additional stress testing on Dan's Solaris X64 server, and
>> additional testing on Erik and Robbin's machines.
>>
>> We welcome comments, suggestions and feedback.
>>
>> Daniel Daugherty
>> Erik Osterlund
>> Robbin Ehn
>>
>
>



From kishor.kharbas at intel.com  Mon Nov 13 19:40:56 2017
From: kishor.kharbas at intel.com (Kharbas, Kishor)
Date: Mon, 13 Nov 2017 19:40:56 +0000
Subject: RFR(M): 8190308: Supporting heap allocation on alternative
 memory devices and CSR review
In-Reply-To: <4171dc3d-3a43-a455-0efb-ee5ff5640b93@oracle.com>
References: <F89640DCD01A85489FCBA68183A6A0F3BCD4BDC5@ORSMSX116.amr.corp.intel.com>
 <a31f2457-149b-f5fb-71c3-b0595ddfa87a@oracle.com>
 <F89640DCD01A85489FCBA68183A6A0F3BCD4FCFF@ORSMSX116.amr.corp.intel.com>
 <1509719477.3207.10.camel@oracle.com>
 <F89640DCD01A85489FCBA68183A6A0F3BCD4FF76@ORSMSX116.amr.corp.intel.com>
 <9b259cdd-111a-0634-80a1-0c6ba1a8d260@oracle.com>
 <F89640DCD01A85489FCBA68183A6A0F3BCD5008C@ORSMSX116.amr.corp.intel.com>
 <4171dc3d-3a43-a455-0efb-ee5ff5640b93@oracle.com>
Message-ID: <F89640DCD01A85489FCBA68183A6A0F3BCD54AC2@ORSMSX116.amr.corp.intel.com>

Greetings,

I have an updated webrev to remove compilation warning on Windows 32-bit.
http://cr.openjdk.java.net/~kkharbas/8190308/webrev.15/
http://cr.openjdk.java.net/~kkharbas/8190308/webrev.15_to_14/

Sorry missed this earlier. I request for a review on this update.

Thanks
Kishor

From: sangheon.kim [mailto:sangheon.kim at oracle.com]
Sent: Friday, November 3, 2017 4:07 PM
To: Kharbas, Kishor <kishor.kharbas at intel.com>; Thomas Schatzl <thomas.schatzl at oracle.com>; 'hotspot-gc-dev at openjdk.java.net' <hotspot-gc-dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net
Subject: Re: RFR(M): 8190308: Supporting heap allocation on alternative memory devices and CSR review

Hi Kishor,

On 11/03/2017 02:59 PM, Kharbas, Kishor wrote:
Hi Sangheon,

Here is link to the updated webrev-
http://cr.openjdk.java.net/~kkharbas/8190308/webrev.14/<http://cr.openjdk.java.net/%7Ekkharbas/8190308/webrev.14/>
http://cr.openjdk.java.net/~kkharbas/8190308/webrev.14_to_13/<http://cr.openjdk.java.net/%7Ekkharbas/8190308/webrev.14_to_13/>
Looks good to me.

Thanks,
Sangheon




Thanks
Kishor

From: sangheon.kim [mailto:sangheon.kim at oracle.com]
Sent: Friday, November 3, 2017 2:38 PM
To: Kharbas, Kishor <kishor.kharbas at intel.com><mailto:kishor.kharbas at intel.com>; Thomas Schatzl <thomas.schatzl at oracle.com><mailto:thomas.schatzl at oracle.com>; 'hotspot-gc-dev at openjdk.java.net<mailto:hotspot-gc-dev at openjdk.java.net>' <hotspot-gc-dev at openjdk.java.net><mailto:hotspot-gc-dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net<mailto:hotspot-runtime-dev at openjdk.java.net>
Subject: Re: RFR(M): 8190308: Supporting heap allocation on alternative memory devices and CSR review

Hi Kishor,
On 11/03/2017 11:39 AM, Kharbas, Kishor wrote:

Thanks a lot!



Link to updated webrevs -

http://cr.openjdk.java.net/~kkharbas/8190308/webrev.13/<http://cr.openjdk.java.net/%7Ekkharbas/8190308/webrev.13/>

http://cr.openjdk.java.net/~kkharbas/8190308/webrev.13_to_12/<http://cr.openjdk.java.net/%7Ekkharbas/8190308/webrev.13_to_12/>
Thank you for fixing all.
Looks good to me except below.

Could you update the copyright format in TestAllocateHeapAt.java?
2  * Copyright (c) 2017 Oracle and/or its affiliates. All rights reserved.
- Missing comma: * Copyright (c) 2017, Oracle and/or its affiliates. All rights reserved.

Thanks,
Sangheon









@Sangheon: Please let me know if you see any corrections needed.



-Kishor



-----Original Message-----

From: Thomas Schatzl [mailto:thomas.schatzl at oracle.com]

Sent: Friday, November 3, 2017 7:31 AM

To: Kharbas, Kishor <kishor.kharbas at intel.com><mailto:kishor.kharbas at intel.com>; sangheon.kim

<sangheon.kim at oracle.com><mailto:sangheon.kim at oracle.com>; 'hotspot-gc-dev at openjdk.java.net<mailto:hotspot-gc-dev at openjdk.java.net>'

<hotspot-gc-dev at openjdk.java.net><mailto:hotspot-gc-dev at openjdk.java.net>; hotspot-runtime-

dev at openjdk.java.net<mailto:dev at openjdk.java.net>

Subject: Re: RFR(M): 8190308: Supporting heap allocation on alternative

memory devices and CSR review



Hi,



On Fri, 2017-11-03 at 08:55 +0000, Kharbas, Kishor wrote:

Hi Sangheon,



Thanks for the review and comments. Here is an updated webrev-

http://cr.openjdk.java.net/~kkharbas/8190308/webrev.12<http://cr.openjdk.java.net/%7Ekkharbas/8190308/webrev.12>

http://cr.openjdk.java.net/~kkharbas/8190308/webrev.12_to_11<http://cr.openjdk.java.net/%7Ekkharbas/8190308/webrev.12_to_11>



In addition to your suggested corrections, I added code to set Linux

core dump filter ensuring Heap is dumped correctly when this feature

is used. This is follow-up to Jini George?s comment

(http://openjdk.5641.n7.nabble.com/RFR-M-8171181-Supporting-heap-

allocation-on-alternative-memory-devices-td300109.html#a300450).



Some minor nits:



 - os_posix.cpp:300: please move the else next to the brace

 - arguments.cpp:4624: please add a space between "if" and the bracket



I do not need to see a new webrev for these changes. Looks good.



Thanks,

  Thomas




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20171113/ece6e25f/attachment.htm>

From thomas.schatzl at oracle.com  Mon Nov 13 20:40:10 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 13 Nov 2017 21:40:10 +0100
Subject: RFR(M): 8190308: Supporting heap allocation on alternative
 memory devices and CSR review
In-Reply-To: <F89640DCD01A85489FCBA68183A6A0F3BCD54AC2@ORSMSX116.amr.corp.intel.com>
References: <F89640DCD01A85489FCBA68183A6A0F3BCD4BDC5@ORSMSX116.amr.corp.intel.com>
 <a31f2457-149b-f5fb-71c3-b0595ddfa87a@oracle.com>
 <F89640DCD01A85489FCBA68183A6A0F3BCD4FCFF@ORSMSX116.amr.corp.intel.com>
 <1509719477.3207.10.camel@oracle.com>
 <F89640DCD01A85489FCBA68183A6A0F3BCD4FF76@ORSMSX116.amr.corp.intel.com>
 <9b259cdd-111a-0634-80a1-0c6ba1a8d260@oracle.com>
 <F89640DCD01A85489FCBA68183A6A0F3BCD5008C@ORSMSX116.amr.corp.intel.com>
 <4171dc3d-3a43-a455-0efb-ee5ff5640b93@oracle.com>
 <F89640DCD01A85489FCBA68183A6A0F3BCD54AC2@ORSMSX116.amr.corp.intel.com>
Message-ID: <1510605610.7132.3.camel@oracle.com>

Hi Kishor,

On Mon, 2017-11-13 at 19:40 +0000, Kharbas, Kishor wrote:
> Greetings,
> ?
> I have an updated webrev to remove compilation warning on Windows 32-
> bit.
> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.15/
> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.15_to_14/
> ?
> Sorry missed this earlier. I request for a review on this update.

  looks good. The other changes from webrev.13 on also look good.

Thanks,
  Thomas

> ?
> Thanks
> Kishor
> ?
> From: sangheon.kim [mailto:sangheon.kim at oracle.com]?
> Sent: Friday, November 3, 2017 4:07 PM
> To: Kharbas, Kishor <kishor.kharbas at intel.com>; Thomas Schatzl <thoma
> s.schatzl at oracle.com>; 'hotspot-gc-dev at openjdk.java.net' <hotspot-gc-
> dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net
> Subject: Re: RFR(M): 8190308: Supporting heap allocation on
> alternative memory devices and CSR review
> ?
> Hi Kishor,
> 
> 
> On 11/03/2017 02:59 PM, Kharbas, Kishor wrote:
> Hi Sangheon,
> ?
> Here is link to the updated webrev-
> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.14/
> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.14_to_13/
> Looks good to me.
> 
> Thanks,
> Sangheon
> 
> 
> 
> ?
> Thanks
> Kishor
> ?
> From: sangheon.kim [mailto:sangheon.kim at oracle.com]?
> Sent: Friday, November 3, 2017 2:38 PM
> To: Kharbas, Kishor <kishor.kharbas at intel.com>; Thomas Schatzl <thoma
> s.schatzl at oracle.com>; 'hotspot-gc-dev at openjdk.java.net' <hotspot-gc-
> dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net
> Subject: Re: RFR(M): 8190308: Supporting heap allocation on
> alternative memory devices and CSR review
> ?
> Hi Kishor,
> 
> On 11/03/2017 11:39 AM, Kharbas, Kishor wrote:
> Thanks a lot!
> ?
> Link to updated webrevs -
> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.13/
> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.13_to_12/
> Thank you for fixing all.
> Looks good to me except below.
> 
> Could you update the copyright format in TestAllocateHeapAt.java?
> 2? * Copyright (c) 2017 Oracle and/or its affiliates. All rights
> reserved.
> - Missing comma: * Copyright (c) 2017, Oracle and/or its affiliates.
> All rights reserved.
> 
> Thanks,
> Sangheon
> 
> 
> 
> 
> ?
> ?
> @Sangheon: Please let me know if you see any corrections needed.
> ?
> -Kishor
> ?
> -----Original Message-----
> From: Thomas Schatzl [mailto:thomas.schatzl at oracle.com]
> Sent: Friday, November 3, 2017 7:31 AM
> To: Kharbas, Kishor <kishor.kharbas at intel.com>; sangheon.kim
> <sangheon.kim at oracle.com>; 'hotspot-gc-dev at openjdk.java.net'
> <hotspot-gc-dev at openjdk.java.net>; hotspot-runtime-
> dev at openjdk.java.net
> Subject: Re: RFR(M): 8190308: Supporting heap allocation on
> alternative
> memory devices and CSR review
> ?
> Hi,
> ?
> On Fri, 2017-11-03 at 08:55 +0000, Kharbas, Kishor wrote:
> Hi Sangheon,
> ?
> Thanks for the review and comments. Here is an updated webrev-
> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.12
> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.12_to_11
> ?
> In addition to your suggested corrections, I added code to set Linux
> core dump filter ensuring Heap is dumped correctly when this feature
> is used. This is follow-up to Jini George?s comment
> (http://openjdk.5641.n7.nabble.com/RFR-M-8171181-Supporting-heap-
> allocation-on-alternative-memory-devices-td300109.html#a300450).
> ?
> Some minor nits:
> ?
> ?- os_posix.cpp:300: please move the else next to the brace
> ?- arguments.cpp:4624: please add a space between "if" and the
> bracket
> ?
> I do not need to see a new webrev for these changes. Looks good.
> ?
> Thanks,
> ? Thomas
> ?
> ?
> ?



From sangheon.kim at oracle.com  Mon Nov 13 20:41:09 2017
From: sangheon.kim at oracle.com (sangheon.kim)
Date: Mon, 13 Nov 2017 12:41:09 -0800
Subject: RFR(M): 8190308: Supporting heap allocation on alternative memory
 devices and CSR review
In-Reply-To: <1510605610.7132.3.camel@oracle.com>
References: <F89640DCD01A85489FCBA68183A6A0F3BCD4BDC5@ORSMSX116.amr.corp.intel.com>
 <a31f2457-149b-f5fb-71c3-b0595ddfa87a@oracle.com>
 <F89640DCD01A85489FCBA68183A6A0F3BCD4FCFF@ORSMSX116.amr.corp.intel.com>
 <1509719477.3207.10.camel@oracle.com>
 <F89640DCD01A85489FCBA68183A6A0F3BCD4FF76@ORSMSX116.amr.corp.intel.com>
 <9b259cdd-111a-0634-80a1-0c6ba1a8d260@oracle.com>
 <F89640DCD01A85489FCBA68183A6A0F3BCD5008C@ORSMSX116.amr.corp.intel.com>
 <4171dc3d-3a43-a455-0efb-ee5ff5640b93@oracle.com>
 <F89640DCD01A85489FCBA68183A6A0F3BCD54AC2@ORSMSX116.amr.corp.intel.com>
 <1510605610.7132.3.camel@oracle.com>
Message-ID: <3e6131cd-bac3-c8db-971f-297bb13b087d@oracle.com>

Hi Kishor,

On 11/13/2017 12:40 PM, Thomas Schatzl wrote:
> Hi Kishor,
>
> On Mon, 2017-11-13 at 19:40 +0000, Kharbas, Kishor wrote:
>> Greetings,
>>   
>> I have an updated webrev to remove compilation warning on Windows 32-
>> bit.
>> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.15/
>> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.15_to_14/
>>   
>> Sorry missed this earlier. I request for a review on this update.
>    looks good. The other changes from webrev.13 on also look good.
+1

Thanks,
Sangheon


>
> Thanks,
>    Thomas
>
>>   
>> Thanks
>> Kishor
>>   
>> From: sangheon.kim [mailto:sangheon.kim at oracle.com]
>> Sent: Friday, November 3, 2017 4:07 PM
>> To: Kharbas, Kishor <kishor.kharbas at intel.com>; Thomas Schatzl <thoma
>> s.schatzl at oracle.com>; 'hotspot-gc-dev at openjdk.java.net' <hotspot-gc-
>> dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net
>> Subject: Re: RFR(M): 8190308: Supporting heap allocation on
>> alternative memory devices and CSR review
>>   
>> Hi Kishor,
>>
>>
>> On 11/03/2017 02:59 PM, Kharbas, Kishor wrote:
>> Hi Sangheon,
>>   
>> Here is link to the updated webrev-
>> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.14/
>> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.14_to_13/
>> Looks good to me.
>>
>> Thanks,
>> Sangheon
>>
>>
>>
>>   
>> Thanks
>> Kishor
>>   
>> From: sangheon.kim [mailto:sangheon.kim at oracle.com]
>> Sent: Friday, November 3, 2017 2:38 PM
>> To: Kharbas, Kishor <kishor.kharbas at intel.com>; Thomas Schatzl <thoma
>> s.schatzl at oracle.com>; 'hotspot-gc-dev at openjdk.java.net' <hotspot-gc-
>> dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net
>> Subject: Re: RFR(M): 8190308: Supporting heap allocation on
>> alternative memory devices and CSR review
>>   
>> Hi Kishor,
>>
>> On 11/03/2017 11:39 AM, Kharbas, Kishor wrote:
>> Thanks a lot!
>>   
>> Link to updated webrevs -
>> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.13/
>> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.13_to_12/
>> Thank you for fixing all.
>> Looks good to me except below.
>>
>> Could you update the copyright format in TestAllocateHeapAt.java?
>> 2? * Copyright (c) 2017 Oracle and/or its affiliates. All rights
>> reserved.
>> - Missing comma: * Copyright (c) 2017, Oracle and/or its affiliates.
>> All rights reserved.
>>
>> Thanks,
>> Sangheon
>>
>>
>>
>>
>>   
>>   
>> @Sangheon: Please let me know if you see any corrections needed.
>>   
>> -Kishor
>>   
>> -----Original Message-----
>> From: Thomas Schatzl [mailto:thomas.schatzl at oracle.com]
>> Sent: Friday, November 3, 2017 7:31 AM
>> To: Kharbas, Kishor <kishor.kharbas at intel.com>; sangheon.kim
>> <sangheon.kim at oracle.com>; 'hotspot-gc-dev at openjdk.java.net'
>> <hotspot-gc-dev at openjdk.java.net>; hotspot-runtime-
>> dev at openjdk.java.net
>> Subject: Re: RFR(M): 8190308: Supporting heap allocation on
>> alternative
>> memory devices and CSR review
>>   
>> Hi,
>>   
>> On Fri, 2017-11-03 at 08:55 +0000, Kharbas, Kishor wrote:
>> Hi Sangheon,
>>   
>> Thanks for the review and comments. Here is an updated webrev-
>> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.12
>> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.12_to_11
>>   
>> In addition to your suggested corrections, I added code to set Linux
>> core dump filter ensuring Heap is dumped correctly when this feature
>> is used. This is follow-up to Jini George?s comment
>> (http://openjdk.5641.n7.nabble.com/RFR-M-8171181-Supporting-heap-
>> allocation-on-alternative-memory-devices-td300109.html#a300450).
>>   
>> Some minor nits:
>>   
>>  ?- os_posix.cpp:300: please move the else next to the brace
>>  ?- arguments.cpp:4624: please add a space between "if" and the
>> bracket
>>   
>> I do not need to see a new webrev for these changes. Looks good.
>>   
>> Thanks,
>>  ? Thomas
>>   
>>   
>>   



From kishor.kharbas at intel.com  Mon Nov 13 21:33:21 2017
From: kishor.kharbas at intel.com (Kharbas, Kishor)
Date: Mon, 13 Nov 2017 21:33:21 +0000
Subject: RFR(M): 8190308: Supporting heap allocation on alternative
 memory devices and CSR review
In-Reply-To: <3e6131cd-bac3-c8db-971f-297bb13b087d@oracle.com>
References: <F89640DCD01A85489FCBA68183A6A0F3BCD4BDC5@ORSMSX116.amr.corp.intel.com>
 <a31f2457-149b-f5fb-71c3-b0595ddfa87a@oracle.com>
 <F89640DCD01A85489FCBA68183A6A0F3BCD4FCFF@ORSMSX116.amr.corp.intel.com>
 <1509719477.3207.10.camel@oracle.com>
 <F89640DCD01A85489FCBA68183A6A0F3BCD4FF76@ORSMSX116.amr.corp.intel.com>
 <9b259cdd-111a-0634-80a1-0c6ba1a8d260@oracle.com>
 <F89640DCD01A85489FCBA68183A6A0F3BCD5008C@ORSMSX116.amr.corp.intel.com>
 <4171dc3d-3a43-a455-0efb-ee5ff5640b93@oracle.com>
 <F89640DCD01A85489FCBA68183A6A0F3BCD54AC2@ORSMSX116.amr.corp.intel.com>
 <1510605610.7132.3.camel@oracle.com>
 <3e6131cd-bac3-c8db-971f-297bb13b087d@oracle.com>
Message-ID: <F89640DCD01A85489FCBA68183A6A0F3BCD54C80@ORSMSX116.amr.corp.intel.com>

Thanks Sangheon and Thomas!

-----Original Message-----
From: sangheon.kim [mailto:sangheon.kim at oracle.com] 
Sent: Monday, November 13, 2017 12:41 PM
To: Thomas Schatzl <thomas.schatzl at oracle.com>; Kharbas, Kishor <kishor.kharbas at intel.com>; 'hotspot-gc-dev at openjdk.java.net' <hotspot-gc-dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net
Subject: Re: RFR(M): 8190308: Supporting heap allocation on alternative memory devices and CSR review

Hi Kishor,

On 11/13/2017 12:40 PM, Thomas Schatzl wrote:
> Hi Kishor,
>
> On Mon, 2017-11-13 at 19:40 +0000, Kharbas, Kishor wrote:
>> Greetings,
>>   
>> I have an updated webrev to remove compilation warning on Windows 32- 
>> bit.
>> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.15/
>> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.15_to_14/
>>   
>> Sorry missed this earlier. I request for a review on this update.
>    looks good. The other changes from webrev.13 on also look good.
+1

Thanks,
Sangheon


>
> Thanks,
>    Thomas
>
>>   
>> Thanks
>> Kishor
>>   
>> From: sangheon.kim [mailto:sangheon.kim at oracle.com]
>> Sent: Friday, November 3, 2017 4:07 PM
>> To: Kharbas, Kishor <kishor.kharbas at intel.com>; Thomas Schatzl <thoma 
>> s.schatzl at oracle.com>; 'hotspot-gc-dev at openjdk.java.net' <hotspot-gc- 
>> dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net
>> Subject: Re: RFR(M): 8190308: Supporting heap allocation on 
>> alternative memory devices and CSR review
>>   
>> Hi Kishor,
>>
>>
>> On 11/03/2017 02:59 PM, Kharbas, Kishor wrote:
>> Hi Sangheon,
>>   
>> Here is link to the updated webrev-
>> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.14/
>> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.14_to_13/
>> Looks good to me.
>>
>> Thanks,
>> Sangheon
>>
>>
>>
>>   
>> Thanks
>> Kishor
>>   
>> From: sangheon.kim [mailto:sangheon.kim at oracle.com]
>> Sent: Friday, November 3, 2017 2:38 PM
>> To: Kharbas, Kishor <kishor.kharbas at intel.com>; Thomas Schatzl <thoma 
>> s.schatzl at oracle.com>; 'hotspot-gc-dev at openjdk.java.net' <hotspot-gc- 
>> dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net
>> Subject: Re: RFR(M): 8190308: Supporting heap allocation on 
>> alternative memory devices and CSR review
>>   
>> Hi Kishor,
>>
>> On 11/03/2017 11:39 AM, Kharbas, Kishor wrote:
>> Thanks a lot!
>>   
>> Link to updated webrevs -
>> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.13/
>> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.13_to_12/
>> Thank you for fixing all.
>> Looks good to me except below.
>>
>> Could you update the copyright format in TestAllocateHeapAt.java?
>> 2? * Copyright (c) 2017 Oracle and/or its affiliates. All rights 
>> reserved.
>> - Missing comma: * Copyright (c) 2017, Oracle and/or its affiliates.
>> All rights reserved.
>>
>> Thanks,
>> Sangheon
>>
>>
>>
>>
>>   
>>   
>> @Sangheon: Please let me know if you see any corrections needed.
>>   
>> -Kishor
>>   
>> -----Original Message-----
>> From: Thomas Schatzl [mailto:thomas.schatzl at oracle.com]
>> Sent: Friday, November 3, 2017 7:31 AM
>> To: Kharbas, Kishor <kishor.kharbas at intel.com>; sangheon.kim 
>> <sangheon.kim at oracle.com>; 'hotspot-gc-dev at openjdk.java.net'
>> <hotspot-gc-dev at openjdk.java.net>; hotspot-runtime- 
>> dev at openjdk.java.net
>> Subject: Re: RFR(M): 8190308: Supporting heap allocation on 
>> alternative memory devices and CSR review
>>   
>> Hi,
>>   
>> On Fri, 2017-11-03 at 08:55 +0000, Kharbas, Kishor wrote:
>> Hi Sangheon,
>>   
>> Thanks for the review and comments. Here is an updated webrev-
>> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.12
>> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.12_to_11
>>   
>> In addition to your suggested corrections, I added code to set Linux 
>> core dump filter ensuring Heap is dumped correctly when this feature 
>> is used. This is follow-up to Jini George?s comment
>> (http://openjdk.5641.n7.nabble.com/RFR-M-8171181-Supporting-heap-
>> allocation-on-alternative-memory-devices-td300109.html#a300450).
>>   
>> Some minor nits:
>>   
>>  ?- os_posix.cpp:300: please move the else next to the brace
>>  ?- arguments.cpp:4624: please add a space between "if" and the 
>> bracket
>>   
>> I do not need to see a new webrev for these changes. Looks good.
>>   
>> Thanks,
>>  ? Thomas
>>   
>>   
>>   


From kishor.kharbas at intel.com  Mon Nov 13 23:51:57 2017
From: kishor.kharbas at intel.com (Kharbas, Kishor)
Date: Mon, 13 Nov 2017 23:51:57 +0000
Subject: RFR(M): 8190828: Test plan: JEP 8171181: Support heap allocation on
 alternative memory devices
Message-ID: <F89640DCD01A85489FCBA68183A6A0F3BCD54D7C@ORSMSX116.amr.corp.intel.com>

Hi!

I have developed a test plan for the implementation of 8171181.
I would appreciate a review and further guidance from the gc-dev members. I am hoping to get everything done well before 18.3 code freeze (have a vacation planned during that time).

Test plan: https://bugs.openjdk.java.net/browse/JDK-8190828
Test webrev: http://cr.openjdk.java.net/~kkharbas/8190980/webrev.01/
JEP: https://bugs.openjdk.java.net/browse/JDK-8171181
Implementation webrev : http://cr.openjdk.java.net/~kkharbas/8190308/webrev.15/

Thank you!
Kishor
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20171113/94f9cbb5/attachment.htm>

From stefan.johansson at oracle.com  Tue Nov 14 08:47:22 2017
From: stefan.johansson at oracle.com (Stefan Johansson)
Date: Tue, 14 Nov 2017 09:47:22 +0100
Subject: RFR: 8186571: Implementation: JEP 307: Parallel Full GC for G1
In-Reply-To: <1383da75-ffbc-2282-9d6a-0710f7087aca@oracle.com>
References: <bb896ae0-83e6-4ff9-07d0-40de53ad125b@oracle.com>
 <c4c62c03-2f0e-0b06-e6f3-85ea78252f6f@oracle.com>
 <bef96f7f-d569-7182-bc5e-65ccbc90924e@oracle.com>
 <1383da75-ffbc-2282-9d6a-0710f7087aca@oracle.com>
Message-ID: <c55538cb-abf0-07bc-cbe8-e242a320e727@oracle.com>

The JEP is now targeted and I will push it shortly.

There were some small conflicting changes made in HS and here are the 
changes I've done to match the current state of the repository:
http://cr.openjdk.java.net/~sjohanss/8186571/hotspot.04-rebase-fixes/

The final change set will also include a small change to make reference 
discovery more efficient:
http://cr.openjdk.java.net/~sjohanss/8186571/hotspot.04-discovery-fix/

I've discussed both these changes with Thomas, Erik and Sangheon over 
chat and consider them reviewed as well.

Thanks for the reviews,
Stefan


On 2017-10-18 15:49, sangheon.kim wrote:
> Looks good to me too.
>
> Thanks,
> Sangheon
>
>
> On 10/18/2017 02:45 AM, Stefan Johansson wrote:
>> Hi again,
>>
>> A lot more internal review has been done and here are the latest 
>> webrevs:
>> Full: http://cr.openjdk.java.net/~sjohanss/8186571/hotspot.04/
>> Incremental: http://cr.openjdk.java.net/~sjohanss/8186571/hotspot.03-04/
>> Incremental (from previous mail): 
>> http://cr.openjdk.java.net/~sjohanss/8186571/hotspot.01-04/
>>
>> Summary of changes:
>> * Updated calculations for heap sizing after gc.
>> * Removed G1FullGCWorkerData and moved data into G1FullCollector 
>> instead.
>> * Renamed G1MarkStack to G1FullGCMarker.
>> * Renamed G1CompactionPoint to G1FullGCCompactionPoint.
>> * Removed now unused RebuildRSOopClosure and par_write_ref from 
>> G1RemSet.
>> * Updated comments to be more informative.
>> * Better naming of functions and variables.
>> * Updated copyright for a lot of files.
>>
>> Big thanks to Erik D, Thomas S and Sangheon K for working your way 
>> through this big change.
>>
>> Cheers,
>> Stefan
>>
>> On 2017-09-19 17:32, Stefan Johansson wrote:
>>> Hi,
>>>
>>> We're moving forward with the review internally and doing some 
>>> performance enhancements as well. Here are updated webrevs:
>>> Full: http://cr.openjdk.java.net/~sjohanss/8186571/hotspot.01/
>>> Incremental: 
>>> http://cr.openjdk.java.net/~sjohanss/8186571/hotspot.00-01/
>>>
>>> Note that the full webrev is based on the new consolidated repo, but 
>>> the incremental was generated with the old structure.
>>>
>>> Highlight in this update:
>>> * Cleaned out unused code in PreservedMarks.
>>> * Fixed memory leak in GenericTaskQueueSet.
>>> * HeapRegionClaimerBase has been removed and instead we now have two 
>>> functions to iterate through all heap regions.
>>> * General cleanups and renames to ease understanding the code.
>>> * G1 Hot Card Cache cleanup made parallel and moved into appropriate 
>>> phase.
>>> * Updated HeapRegion::apply_to_marked_objects to be a template 
>>> function to avoid virtual call.
>>>
>>> Thanks Erik D and Thomas S for all comments so far.
>>>
>>> Cheers,
>>> Stefan
>>>
>>>
>>> On 2017-09-04 17:36, Stefan Johansson wrote:
>>>> Hi,
>>>>
>>>> Please review the implementation of JEP-307:
>>>> https://bugs.openjdk.java.net/browse/JDK-8172890
>>>>
>>>> Webrev:
>>>> http://cr.openjdk.java.net/~sjohanss/8186571/hotspot.00/
>>>>
>>>> Summary:
>>>> As communicated late last year [1], I've been working on 
>>>> parallelizing the Full GC for G1. The implementation is now ready 
>>>> for review.
>>>>
>>>> The approach I chose was to redo marking at the start of the Full 
>>>> GC and not reuse the marking information from the concurrent mark 
>>>> cycle. The main reason behind this is to maximize the chance of 
>>>> freeing up memory. I reused the marking bitmap from the concurrent 
>>>> mark code though, so instead of marking in the mark word a bitmap 
>>>> is used. The mark word is still used for forwarding pointers, so 
>>>> marks will still have to be preserved for some objects.
>>>>
>>>> The algorithm is still a four phased mark-compact but each phase is 
>>>> handled by parallel workers. Marking and reference processing is 
>>>> done in phase 1. In phase 2 all worker threads work through the 
>>>> heap claiming regions which they prepare for compaction. This is 
>>>> done by installing forwarding pointers into the mark word of the 
>>>> live objects that will move. The regions claimed by a worker in 
>>>> this phase will be the same regions that the worker will compact in 
>>>> phase 4. This ensures that objects are not overwritten before 
>>>> compacted.
>>>>
>>>> In phase 3, all pointers to other objects are updated by looking at 
>>>> the forwarding pointers. At this point all information needed to 
>>>> create new remembered sets is available and this rebuilding has 
>>>> been added to phase 3. In the old version remembered set rebuilding 
>>>> was done separately after the compaction, but this is more efficient.
>>>>
>>>> As mentioned phase 4 is when the compaction is done. In this first 
>>>> version, to avoid some complexity, there is no work stealing in 
>>>> this phase. This will lead to some imbalance between the workers, 
>>>> but this can be treated as a separate RFE in the future.
>>>>
>>>> The part of this work that has generated the most questions during 
>>>> internal discussions are the serial parts of phase 2 and 4. They 
>>>> are executed if no regions are to be freed up by the parallel 
>>>> workers. It is kind of a safety mechanism to avoid throwing a 
>>>> premature OOM. In the case of no regions being freed by the 
>>>> parallel code path a single threaded pass over the last region of 
>>>> each worker is done (at most number-of-workers regions are handled) 
>>>> to further compact these regions and hopefully free up some regions.
>>>>
>>>> Testing:
>>>> * A lot of local sanity testing, both functional and performance.
>>>> * Passed tier 1-5 of internal testing on supported platforms.
>>>> * No regressions in performance testing.
>>>>
>>>> Cheers,
>>>> Stefan
>>>>
>>>> [1] 
>>>> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2016-November/019216.html
>>>
>>
>



From rkennke at redhat.com  Tue Nov 14 11:53:17 2017
From: rkennke at redhat.com (Roman Kennke)
Date: Tue, 14 Nov 2017 12:53:17 +0100
Subject: RFR: 8189389: Move heap creation into GC interface
Message-ID: <66c8c65f-9dc6-8b78-59d8-da1efa36f6c5@redhat.com>

This moves the heap creation code that is currently inside universe.cpp 
to the GC specific GCArguments subclasses, thus reducing GC specific 
stuff in shared code. It's based on top of 8189171 which should arrive 
in the repo today or very soon.

http://cr.openjdk.java.net/~rkennke/8189389/webrev.00/ 
<http://cr.openjdk.java.net/%7Erkennke/8189389/webrev.00/>

Test: hotspot_gc

Ok?

Roman



From daniel.daugherty at oracle.com  Tue Nov 14 21:48:42 2017
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Tue, 14 Nov 2017 16:48:42 -0500
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <836b5c6c-5eef-9245-c16a-e71372093d9a@oracle.com>
References: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
 <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>
 <836b5c6c-5eef-9245-c16a-e71372093d9a@oracle.com>
Message-ID: <82d0e473-a24f-9feb-82e9-7d04c4d7a91d@oracle.com>

Greetings,

I rebased the project to the 2017.11.10 jdk/hs PIT snapshot.
(Yes, we're playing chase-the-repo...)

Here is the updated full webrev:

http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-07-full/

Unlike the previous rebase, there were no changes required to the
open code to get the rebased bits to build so there is no delta
webrev.

These bits have been run through JPRT and I've submitted the usual
Mach5 tier[1-5] test run...

We welcome comments, suggestions and feedback.

Dan, Erik and Robbin


On 11/13/17 12:30 PM, Daniel D. Daugherty wrote:
> Greetings,
>
> I rebased the project to the 2017.10.26 jdk10/hs PIT snapshot.
>
> Here are the updated webrevs:
>
> Here's the mq comment for the change:
>
> ? Rebase to 2017.10.25 PIT snapshot.
>
> Here is the full webrev:
>
> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-full/
>
> And here is the delta webrev:
>
> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-delta/
>
> I ran the above bits throught Mach5 tier[1-5] testing over the holiday
> weekend. Didn't see any issues in a quick look. Going to take a closer
> look today.
>
> We welcome comments, suggestions and feedback.
>
> Dan, Erik and Robbin
>
>
> On 11/8/17 1:05 PM, Daniel D. Daugherty wrote:
>> Greetings,
>>
>> Resolving one of the code review comments (from both Stefan K and 
>> Coleen)
>> on jdk10-04-full required quite a few changes so it is being done as a
>> standalone patch and corresponding webrevs:
>>
>> Here's the mq comment for the change:
>>
>> ? stefank, coleenp CR - refactor most JavaThreadIterator usage to use
>> ??? JavaThreadIteratorWithHandle.
>>
>> Here is the full webrev:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-full/
>>
>> And here is the delta webrev:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-delta/
>>
>> We welcome comments, suggestions and feedback.
>>
>> Dan, Erik and Robbin
>>
>>
>> On 10/9/17 3:41 PM, Daniel D. Daugherty wrote:
>>> Greetings,
>>>
>>> We have a (eXtra Large) fix for the following bug:
>>>
>>> 8167108 inconsistent handling of SR_lock can lead to crashes
>>> https://bugs.openjdk.java.net/browse/JDK-8167108
>>>
>>> This fix adds a Safe Memory Reclamation (SMR) mechanism based on
>>> Hazard Pointers to manage JavaThread lifecycle.
>>>
>>> Here's a PDF for the internal wiki that we've been using to describe
>>> and track the work on this project:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/SMR_and_JavaThread_Lifecycle-JDK10-04.pdf 
>>>
>>>
>>> Dan has noticed that the indenting is wrong in some of the code quotes
>>> in the PDF that are not present in the internal wiki. We don't have a
>>> solution for that problem yet.
>>>
>>> Here's the webrev for current JDK10 version of this fix:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-04-full
>>>
>>> This fix has been run through many rounds of JPRT and Mach5 tier[2-5]
>>> testing, additional stress testing on Dan's Solaris X64 server, and
>>> additional testing on Erik and Robbin's machines.
>>>
>>> We welcome comments, suggestions and feedback.
>>>
>>> Daniel Daugherty
>>> Erik Osterlund
>>> Robbin Ehn
>>>
>>
>>
>
>



From martin.doerr at sap.com  Wed Nov 15 15:44:37 2017
From: martin.doerr at sap.com (Doerr, Martin)
Date: Wed, 15 Nov 2017 15:44:37 +0000
Subject: RFR: 8186571: Implementation: JEP 307: Parallel Full GC for G1
In-Reply-To: <c55538cb-abf0-07bc-cbe8-e242a320e727@oracle.com>
References: <bb896ae0-83e6-4ff9-07d0-40de53ad125b@oracle.com>
 <c4c62c03-2f0e-0b06-e6f3-85ea78252f6f@oracle.com>
 <bef96f7f-d569-7182-bc5e-65ccbc90924e@oracle.com>
 <1383da75-ffbc-2282-9d6a-0710f7087aca@oracle.com>
 <c55538cb-abf0-07bc-cbe8-e242a320e727@oracle.com>
Message-ID: <9f4ec5a7b52c470e8cca06feda894dae@sap.com>

Hi Stefan,

thanks for contributing this change. We basically like it. But is has the little drawback that it doesn't build with older GCC (e.g. 4.8).
We got the following error:
g1FullGCOopClosures.hpp:140: undefined reference to `void G1VerifyOopClosure::do_oop_nv<unsigned int>(unsigned int*)'

It can be fixed by moving the do_oop implementations into the cpp file (see diffs below).

I can open a bug for it and provide a webrev. Or would you prefer to put it into another follow-up change?

Best regards,
Martin


diff -r 7092940fbaff src/hotspot/share/gc/g1/g1FullGCOopClosures.cpp
--- a/src/hotspot/share/gc/g1/g1FullGCOopClosures.cpp   Wed Nov 15 08:25:28 2017 -0500
+++ b/src/hotspot/share/gc/g1/g1FullGCOopClosures.cpp   Wed Nov 15 15:50:07 2017 +0100
@@ -144,5 +144,8 @@
   }
 }

+void G1VerifyOopClosure::do_oop(oop* p)       { do_oop_nv(p); }
+void G1VerifyOopClosure::do_oop(narrowOop* p) { do_oop_nv(p); }
+
 // Generate G1 full GC specialized oop_oop_iterate functions.
 SPECIALIZED_OOP_OOP_ITERATE_CLOSURES_G1FULL(ALL_KLASS_OOP_OOP_ITERATE_DEFN)
diff -r 7092940fbaff src/hotspot/share/gc/g1/g1FullGCOopClosures.hpp
--- a/src/hotspot/share/gc/g1/g1FullGCOopClosures.hpp   Wed Nov 15 08:25:28 2017 -0500
+++ b/src/hotspot/share/gc/g1/g1FullGCOopClosures.hpp   Wed Nov 15 15:50:07 2017 +0100
@@ -136,8 +136,8 @@

   template <class T> void do_oop_nv(T* p);

-  void do_oop(oop* p)       { do_oop_nv(p); }
-  void do_oop(narrowOop* p) { do_oop_nv(p); }
+  void do_oop(oop* p)      ;// { do_oop_nv(p); }
+  void do_oop(narrowOop* p);// { do_oop_nv(p); }
 };

 class G1FollowStackClosure: public VoidClosure {



-----Original Message-----
From: hotspot-gc-dev [mailto:hotspot-gc-dev-bounces at openjdk.java.net] On Behalf Of Stefan Johansson
Sent: Dienstag, 14. November 2017 09:47
To: sangheon.kim <sangheon.kim at oracle.com>; hotspot-gc-dev at openjdk.java.net
Subject: Re: RFR: 8186571: Implementation: JEP 307: Parallel Full GC for G1

The JEP is now targeted and I will push it shortly.

There were some small conflicting changes made in HS and here are the 
changes I've done to match the current state of the repository:
http://cr.openjdk.java.net/~sjohanss/8186571/hotspot.04-rebase-fixes/

The final change set will also include a small change to make reference 
discovery more efficient:
http://cr.openjdk.java.net/~sjohanss/8186571/hotspot.04-discovery-fix/

I've discussed both these changes with Thomas, Erik and Sangheon over 
chat and consider them reviewed as well.

Thanks for the reviews,
Stefan


On 2017-10-18 15:49, sangheon.kim wrote:
> Looks good to me too.
>
> Thanks,
> Sangheon
>
>
> On 10/18/2017 02:45 AM, Stefan Johansson wrote:
>> Hi again,
>>
>> A lot more internal review has been done and here are the latest 
>> webrevs:
>> Full: http://cr.openjdk.java.net/~sjohanss/8186571/hotspot.04/
>> Incremental: http://cr.openjdk.java.net/~sjohanss/8186571/hotspot.03-04/
>> Incremental (from previous mail): 
>> http://cr.openjdk.java.net/~sjohanss/8186571/hotspot.01-04/
>>
>> Summary of changes:
>> * Updated calculations for heap sizing after gc.
>> * Removed G1FullGCWorkerData and moved data into G1FullCollector 
>> instead.
>> * Renamed G1MarkStack to G1FullGCMarker.
>> * Renamed G1CompactionPoint to G1FullGCCompactionPoint.
>> * Removed now unused RebuildRSOopClosure and par_write_ref from 
>> G1RemSet.
>> * Updated comments to be more informative.
>> * Better naming of functions and variables.
>> * Updated copyright for a lot of files.
>>
>> Big thanks to Erik D, Thomas S and Sangheon K for working your way 
>> through this big change.
>>
>> Cheers,
>> Stefan
>>
>> On 2017-09-19 17:32, Stefan Johansson wrote:
>>> Hi,
>>>
>>> We're moving forward with the review internally and doing some 
>>> performance enhancements as well. Here are updated webrevs:
>>> Full: http://cr.openjdk.java.net/~sjohanss/8186571/hotspot.01/
>>> Incremental: 
>>> http://cr.openjdk.java.net/~sjohanss/8186571/hotspot.00-01/
>>>
>>> Note that the full webrev is based on the new consolidated repo, but 
>>> the incremental was generated with the old structure.
>>>
>>> Highlight in this update:
>>> * Cleaned out unused code in PreservedMarks.
>>> * Fixed memory leak in GenericTaskQueueSet.
>>> * HeapRegionClaimerBase has been removed and instead we now have two 
>>> functions to iterate through all heap regions.
>>> * General cleanups and renames to ease understanding the code.
>>> * G1 Hot Card Cache cleanup made parallel and moved into appropriate 
>>> phase.
>>> * Updated HeapRegion::apply_to_marked_objects to be a template 
>>> function to avoid virtual call.
>>>
>>> Thanks Erik D and Thomas S for all comments so far.
>>>
>>> Cheers,
>>> Stefan
>>>
>>>
>>> On 2017-09-04 17:36, Stefan Johansson wrote:
>>>> Hi,
>>>>
>>>> Please review the implementation of JEP-307:
>>>> https://bugs.openjdk.java.net/browse/JDK-8172890
>>>>
>>>> Webrev:
>>>> http://cr.openjdk.java.net/~sjohanss/8186571/hotspot.00/
>>>>
>>>> Summary:
>>>> As communicated late last year [1], I've been working on 
>>>> parallelizing the Full GC for G1. The implementation is now ready 
>>>> for review.
>>>>
>>>> The approach I chose was to redo marking at the start of the Full 
>>>> GC and not reuse the marking information from the concurrent mark 
>>>> cycle. The main reason behind this is to maximize the chance of 
>>>> freeing up memory. I reused the marking bitmap from the concurrent 
>>>> mark code though, so instead of marking in the mark word a bitmap 
>>>> is used. The mark word is still used for forwarding pointers, so 
>>>> marks will still have to be preserved for some objects.
>>>>
>>>> The algorithm is still a four phased mark-compact but each phase is 
>>>> handled by parallel workers. Marking and reference processing is 
>>>> done in phase 1. In phase 2 all worker threads work through the 
>>>> heap claiming regions which they prepare for compaction. This is 
>>>> done by installing forwarding pointers into the mark word of the 
>>>> live objects that will move. The regions claimed by a worker in 
>>>> this phase will be the same regions that the worker will compact in 
>>>> phase 4. This ensures that objects are not overwritten before 
>>>> compacted.
>>>>
>>>> In phase 3, all pointers to other objects are updated by looking at 
>>>> the forwarding pointers. At this point all information needed to 
>>>> create new remembered sets is available and this rebuilding has 
>>>> been added to phase 3. In the old version remembered set rebuilding 
>>>> was done separately after the compaction, but this is more efficient.
>>>>
>>>> As mentioned phase 4 is when the compaction is done. In this first 
>>>> version, to avoid some complexity, there is no work stealing in 
>>>> this phase. This will lead to some imbalance between the workers, 
>>>> but this can be treated as a separate RFE in the future.
>>>>
>>>> The part of this work that has generated the most questions during 
>>>> internal discussions are the serial parts of phase 2 and 4. They 
>>>> are executed if no regions are to be freed up by the parallel 
>>>> workers. It is kind of a safety mechanism to avoid throwing a 
>>>> premature OOM. In the case of no regions being freed by the 
>>>> parallel code path a single threaded pass over the last region of 
>>>> each worker is done (at most number-of-workers regions are handled) 
>>>> to further compact these regions and hopefully free up some regions.
>>>>
>>>> Testing:
>>>> * A lot of local sanity testing, both functional and performance.
>>>> * Passed tier 1-5 of internal testing on supported platforms.
>>>> * No regressions in performance testing.
>>>>
>>>> Cheers,
>>>> Stefan
>>>>
>>>> [1] 
>>>> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2016-November/019216.html
>>>
>>
>


From stefan.johansson at oracle.com  Wed Nov 15 15:57:34 2017
From: stefan.johansson at oracle.com (Stefan Johansson)
Date: Wed, 15 Nov 2017 16:57:34 +0100
Subject: RFR: 8186571: Implementation: JEP 307: Parallel Full GC for G1
In-Reply-To: <9f4ec5a7b52c470e8cca06feda894dae@sap.com>
References: <bb896ae0-83e6-4ff9-07d0-40de53ad125b@oracle.com>
 <c4c62c03-2f0e-0b06-e6f3-85ea78252f6f@oracle.com>
 <bef96f7f-d569-7182-bc5e-65ccbc90924e@oracle.com>
 <1383da75-ffbc-2282-9d6a-0710f7087aca@oracle.com>
 <c55538cb-abf0-07bc-cbe8-e242a320e727@oracle.com>
 <9f4ec5a7b52c470e8cca06feda894dae@sap.com>
Message-ID: <de502fcc-e708-ce7a-36d4-ab06c918b1df@oracle.com>

Hi Martin,

Sorry for breaking the build.

Please open a bug for this and provide a webrev for the fix, it is 
probably easier since you can make sure the fix really works. I'm not 
sure moving the do_oop implementation is what we should do, maybe 
instead move the do_oop_nv() implementation into the inline.hpp-file and 
make sure it is included where needed.

Thanks,
Stefan

On 2017-11-15 16:44, Doerr, Martin wrote:
> Hi Stefan,
>
> thanks for contributing this change. We basically like it. But is has the little drawback that it doesn't build with older GCC (e.g. 4.8).
> We got the following error:
> g1FullGCOopClosures.hpp:140: undefined reference to `void G1VerifyOopClosure::do_oop_nv<unsigned int>(unsigned int*)'
>
> It can be fixed by moving the do_oop implementations into the cpp file (see diffs below).
>
> I can open a bug for it and provide a webrev. Or would you prefer to put it into another follow-up change?
>
> Best regards,
> Martin
>
>
> diff -r 7092940fbaff src/hotspot/share/gc/g1/g1FullGCOopClosures.cpp
> --- a/src/hotspot/share/gc/g1/g1FullGCOopClosures.cpp   Wed Nov 15 08:25:28 2017 -0500
> +++ b/src/hotspot/share/gc/g1/g1FullGCOopClosures.cpp   Wed Nov 15 15:50:07 2017 +0100
> @@ -144,5 +144,8 @@
>     }
>   }
>
> +void G1VerifyOopClosure::do_oop(oop* p)       { do_oop_nv(p); }
> +void G1VerifyOopClosure::do_oop(narrowOop* p) { do_oop_nv(p); }
> +
>   // Generate G1 full GC specialized oop_oop_iterate functions.
>   SPECIALIZED_OOP_OOP_ITERATE_CLOSURES_G1FULL(ALL_KLASS_OOP_OOP_ITERATE_DEFN)
> diff -r 7092940fbaff src/hotspot/share/gc/g1/g1FullGCOopClosures.hpp
> --- a/src/hotspot/share/gc/g1/g1FullGCOopClosures.hpp   Wed Nov 15 08:25:28 2017 -0500
> +++ b/src/hotspot/share/gc/g1/g1FullGCOopClosures.hpp   Wed Nov 15 15:50:07 2017 +0100
> @@ -136,8 +136,8 @@
>
>     template <class T> void do_oop_nv(T* p);
>
> -  void do_oop(oop* p)       { do_oop_nv(p); }
> -  void do_oop(narrowOop* p) { do_oop_nv(p); }
> +  void do_oop(oop* p)      ;// { do_oop_nv(p); }
> +  void do_oop(narrowOop* p);// { do_oop_nv(p); }
>   };
>
>   class G1FollowStackClosure: public VoidClosure {
>
>
>
> -----Original Message-----
> From: hotspot-gc-dev [mailto:hotspot-gc-dev-bounces at openjdk.java.net] On Behalf Of Stefan Johansson
> Sent: Dienstag, 14. November 2017 09:47
> To: sangheon.kim <sangheon.kim at oracle.com>; hotspot-gc-dev at openjdk.java.net
> Subject: Re: RFR: 8186571: Implementation: JEP 307: Parallel Full GC for G1
>
> The JEP is now targeted and I will push it shortly.
>
> There were some small conflicting changes made in HS and here are the
> changes I've done to match the current state of the repository:
> http://cr.openjdk.java.net/~sjohanss/8186571/hotspot.04-rebase-fixes/
>
> The final change set will also include a small change to make reference
> discovery more efficient:
> http://cr.openjdk.java.net/~sjohanss/8186571/hotspot.04-discovery-fix/
>
> I've discussed both these changes with Thomas, Erik and Sangheon over
> chat and consider them reviewed as well.
>
> Thanks for the reviews,
> Stefan
>
>
> On 2017-10-18 15:49, sangheon.kim wrote:
>> Looks good to me too.
>>
>> Thanks,
>> Sangheon
>>
>>
>> On 10/18/2017 02:45 AM, Stefan Johansson wrote:
>>> Hi again,
>>>
>>> A lot more internal review has been done and here are the latest
>>> webrevs:
>>> Full: http://cr.openjdk.java.net/~sjohanss/8186571/hotspot.04/
>>> Incremental: http://cr.openjdk.java.net/~sjohanss/8186571/hotspot.03-04/
>>> Incremental (from previous mail):
>>> http://cr.openjdk.java.net/~sjohanss/8186571/hotspot.01-04/
>>>
>>> Summary of changes:
>>> * Updated calculations for heap sizing after gc.
>>> * Removed G1FullGCWorkerData and moved data into G1FullCollector
>>> instead.
>>> * Renamed G1MarkStack to G1FullGCMarker.
>>> * Renamed G1CompactionPoint to G1FullGCCompactionPoint.
>>> * Removed now unused RebuildRSOopClosure and par_write_ref from
>>> G1RemSet.
>>> * Updated comments to be more informative.
>>> * Better naming of functions and variables.
>>> * Updated copyright for a lot of files.
>>>
>>> Big thanks to Erik D, Thomas S and Sangheon K for working your way
>>> through this big change.
>>>
>>> Cheers,
>>> Stefan
>>>
>>> On 2017-09-19 17:32, Stefan Johansson wrote:
>>>> Hi,
>>>>
>>>> We're moving forward with the review internally and doing some
>>>> performance enhancements as well. Here are updated webrevs:
>>>> Full: http://cr.openjdk.java.net/~sjohanss/8186571/hotspot.01/
>>>> Incremental:
>>>> http://cr.openjdk.java.net/~sjohanss/8186571/hotspot.00-01/
>>>>
>>>> Note that the full webrev is based on the new consolidated repo, but
>>>> the incremental was generated with the old structure.
>>>>
>>>> Highlight in this update:
>>>> * Cleaned out unused code in PreservedMarks.
>>>> * Fixed memory leak in GenericTaskQueueSet.
>>>> * HeapRegionClaimerBase has been removed and instead we now have two
>>>> functions to iterate through all heap regions.
>>>> * General cleanups and renames to ease understanding the code.
>>>> * G1 Hot Card Cache cleanup made parallel and moved into appropriate
>>>> phase.
>>>> * Updated HeapRegion::apply_to_marked_objects to be a template
>>>> function to avoid virtual call.
>>>>
>>>> Thanks Erik D and Thomas S for all comments so far.
>>>>
>>>> Cheers,
>>>> Stefan
>>>>
>>>>
>>>> On 2017-09-04 17:36, Stefan Johansson wrote:
>>>>> Hi,
>>>>>
>>>>> Please review the implementation of JEP-307:
>>>>> https://bugs.openjdk.java.net/browse/JDK-8172890
>>>>>
>>>>> Webrev:
>>>>> http://cr.openjdk.java.net/~sjohanss/8186571/hotspot.00/
>>>>>
>>>>> Summary:
>>>>> As communicated late last year [1], I've been working on
>>>>> parallelizing the Full GC for G1. The implementation is now ready
>>>>> for review.
>>>>>
>>>>> The approach I chose was to redo marking at the start of the Full
>>>>> GC and not reuse the marking information from the concurrent mark
>>>>> cycle. The main reason behind this is to maximize the chance of
>>>>> freeing up memory. I reused the marking bitmap from the concurrent
>>>>> mark code though, so instead of marking in the mark word a bitmap
>>>>> is used. The mark word is still used for forwarding pointers, so
>>>>> marks will still have to be preserved for some objects.
>>>>>
>>>>> The algorithm is still a four phased mark-compact but each phase is
>>>>> handled by parallel workers. Marking and reference processing is
>>>>> done in phase 1. In phase 2 all worker threads work through the
>>>>> heap claiming regions which they prepare for compaction. This is
>>>>> done by installing forwarding pointers into the mark word of the
>>>>> live objects that will move. The regions claimed by a worker in
>>>>> this phase will be the same regions that the worker will compact in
>>>>> phase 4. This ensures that objects are not overwritten before
>>>>> compacted.
>>>>>
>>>>> In phase 3, all pointers to other objects are updated by looking at
>>>>> the forwarding pointers. At this point all information needed to
>>>>> create new remembered sets is available and this rebuilding has
>>>>> been added to phase 3. In the old version remembered set rebuilding
>>>>> was done separately after the compaction, but this is more efficient.
>>>>>
>>>>> As mentioned phase 4 is when the compaction is done. In this first
>>>>> version, to avoid some complexity, there is no work stealing in
>>>>> this phase. This will lead to some imbalance between the workers,
>>>>> but this can be treated as a separate RFE in the future.
>>>>>
>>>>> The part of this work that has generated the most questions during
>>>>> internal discussions are the serial parts of phase 2 and 4. They
>>>>> are executed if no regions are to be freed up by the parallel
>>>>> workers. It is kind of a safety mechanism to avoid throwing a
>>>>> premature OOM. In the case of no regions being freed by the
>>>>> parallel code path a single threaded pass over the last region of
>>>>> each worker is done (at most number-of-workers regions are handled)
>>>>> to further compact these regions and hopefully free up some regions.
>>>>>
>>>>> Testing:
>>>>> * A lot of local sanity testing, both functional and performance.
>>>>> * Passed tier 1-5 of internal testing on supported platforms.
>>>>> * No regressions in performance testing.
>>>>>
>>>>> Cheers,
>>>>> Stefan
>>>>>
>>>>> [1]
>>>>> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2016-November/019216.html



From martin.doerr at sap.com  Wed Nov 15 16:23:18 2017
From: martin.doerr at sap.com (Doerr, Martin)
Date: Wed, 15 Nov 2017 16:23:18 +0000
Subject: RFR(XS): 8191337: GCC 4.8 build broken after 8186571
Message-ID: <3ddb39f79c244679928670bb8d56bb78@sap.com>

Hi,

I'd like to build the VM with GCC 4.8.5, but I got the following error while linking:
g1FullGCOopClosures.hpp:140: undefined reference to `void G1VerifyOopClosure::do_oop_nv<unsigned int>(unsigned int*)'

It can be fixed by:
http://cr.openjdk.java.net/~mdoerr/8191337_fix_linux_build/webrev/

Please review. I will also need a sponsor.

Thanks and best regards,
Martin

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20171115/c72e9838/attachment.htm>

From stefan.johansson at oracle.com  Wed Nov 15 16:24:23 2017
From: stefan.johansson at oracle.com (Stefan Johansson)
Date: Wed, 15 Nov 2017 17:24:23 +0100
Subject: RFR: 8189733: Cleanup Full GC setup and tear down
Message-ID: <3b9604e1-9d2e-c0d9-7b2e-e5d9232c6888@oracle.com>

Hi,

Please review this enhancement:
https://bugs.openjdk.java.net/browse/JDK-8189733

Webrev:
http://cr.openjdk.java.net/~sjohanss//8189733/00/index.html

Summary:
After the G1FullCollector has been introduced as part of JEP 307 it 
makes sense to move more of the setup and tear down for the Full GC into 
this class. The G1FullGCScope is also moved into the G1FullCollector 
since it is now only used here.

Testing:
* Tier 1-2

Thanks,
Stefan




From stefan.johansson at oracle.com  Wed Nov 15 16:45:00 2017
From: stefan.johansson at oracle.com (Stefan Johansson)
Date: Wed, 15 Nov 2017 17:45:00 +0100
Subject: RFR(XS): 8191337: GCC 4.8 build broken after 8186571
In-Reply-To: <3ddb39f79c244679928670bb8d56bb78@sap.com>
References: <3ddb39f79c244679928670bb8d56bb78@sap.com>
Message-ID: <c333138f-7dc4-b641-7530-f5bd4194f5bf@oracle.com>

Hi Martin,

On 2017-11-15 17:23, Doerr, Martin wrote:
>
> Hi,
>
> I?d like to build the VM with GCC 4.8.5, but I got the following error 
> while linking:
>
> g1FullGCOopClosures.hpp:140: undefined reference to `void 
> G1VerifyOopClosure::do_oop_nv<unsigned int>(unsigned int*)'
>
> It can be fixed by:
>
> http://cr.openjdk.java.net/~mdoerr/8191337_fix_linux_build/webrev/ 
> <http://cr.openjdk.java.net/%7Emdoerr/8191337_fix_linux_build/webrev/>
>
> Please review. I will also need a sponsor.
>
Thanks for fixing this, I can sponsor the change.

Cheers,
Stefan
>
> Thanks and best regards,
>
> Martin
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20171115/622e0535/attachment.htm>

From daniel.daugherty at oracle.com  Wed Nov 15 20:32:13 2017
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Wed, 15 Nov 2017 15:32:13 -0500
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <82d0e473-a24f-9feb-82e9-7d04c4d7a91d@oracle.com>
References: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
 <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>
 <836b5c6c-5eef-9245-c16a-e71372093d9a@oracle.com>
 <82d0e473-a24f-9feb-82e9-7d04c4d7a91d@oracle.com>
Message-ID: <354f7ece-66bc-c065-9ee8-f55abcc1c23e@oracle.com>

Greetings,

Robbin rebased the project last night/this morning to merge with Thread
Local Handshakes (TLH) and also picked up additional changesets up thru:

> Changeset: fa736014cf28
> Author:    cjplummer
> Date:      2017-11-14 18:08 -0800
> URL:http://hg.openjdk.java.net/jdk/hs/rev/fa736014cf28
>
> 8191049: Add alternate version of pns() that is callable from within hotspot source
> Summary: added pns2() to debug.cpp
> Reviewed-by: stuefe, gthornbr

This is the first time we've rebased the project to bits that are this
fresh (< 12 hours old at rebase time). We've done this because we think
we're done with this project and are in the final review-change-retest
cycle(s)... In other words, we're not planning on making any more major
changes for this project... :-)

*** Time for code reviewers to chime in on this thread! ***

Here is the updated full webrev:

http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-full/

Here's is the delta webrev from the 2017.11.10 rebase:

http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-delta/

Dan has submitted the bits for the usual Mach5 tier[1-5] testing
(and the new baseline also)...

We're expecting this round to be a little noisier than usual because
we did not rebase on a PIT snapshot so the new baseline has not been
through Jesper's usual care-and-feeding of integration_blockers, etc.

We welcome comments, suggestions and feedback.

Dan, Erik and Robbin


On 11/14/17 4:48 PM, Daniel D. Daugherty wrote:
> Greetings,
>
> I rebased the project to the 2017.11.10 jdk/hs PIT snapshot.
> (Yes, we're playing chase-the-repo...)
>
> Here is the updated full webrev:
>
> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-07-full/
>
> Unlike the previous rebase, there were no changes required to the
> open code to get the rebased bits to build so there is no delta
> webrev.
>
> These bits have been run through JPRT and I've submitted the usual
> Mach5 tier[1-5] test run...
>
> We welcome comments, suggestions and feedback.
>
> Dan, Erik and Robbin
>
>
> On 11/13/17 12:30 PM, Daniel D. Daugherty wrote:
>> Greetings,
>>
>> I rebased the project to the 2017.10.26 jdk10/hs PIT snapshot.
>>
>> Here are the updated webrevs:
>>
>> Here's the mq comment for the change:
>>
>> ? Rebase to 2017.10.25 PIT snapshot.
>>
>> Here is the full webrev:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-full/
>>
>> And here is the delta webrev:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-delta/
>>
>> I ran the above bits throught Mach5 tier[1-5] testing over the holiday
>> weekend. Didn't see any issues in a quick look. Going to take a closer
>> look today.
>>
>> We welcome comments, suggestions and feedback.
>>
>> Dan, Erik and Robbin
>>
>>
>> On 11/8/17 1:05 PM, Daniel D. Daugherty wrote:
>>> Greetings,
>>>
>>> Resolving one of the code review comments (from both Stefan K and 
>>> Coleen)
>>> on jdk10-04-full required quite a few changes so it is being done as a
>>> standalone patch and corresponding webrevs:
>>>
>>> Here's the mq comment for the change:
>>>
>>> ? stefank, coleenp CR - refactor most JavaThreadIterator usage to use
>>> ??? JavaThreadIteratorWithHandle.
>>>
>>> Here is the full webrev:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-full/
>>>
>>> And here is the delta webrev:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-delta/
>>>
>>> We welcome comments, suggestions and feedback.
>>>
>>> Dan, Erik and Robbin
>>>
>>>
>>> On 10/9/17 3:41 PM, Daniel D. Daugherty wrote:
>>>> Greetings,
>>>>
>>>> We have a (eXtra Large) fix for the following bug:
>>>>
>>>> 8167108 inconsistent handling of SR_lock can lead to crashes
>>>> https://bugs.openjdk.java.net/browse/JDK-8167108
>>>>
>>>> This fix adds a Safe Memory Reclamation (SMR) mechanism based on
>>>> Hazard Pointers to manage JavaThread lifecycle.
>>>>
>>>> Here's a PDF for the internal wiki that we've been using to describe
>>>> and track the work on this project:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/SMR_and_JavaThread_Lifecycle-JDK10-04.pdf 
>>>>
>>>>
>>>> Dan has noticed that the indenting is wrong in some of the code quotes
>>>> in the PDF that are not present in the internal wiki. We don't have a
>>>> solution for that problem yet.
>>>>
>>>> Here's the webrev for current JDK10 version of this fix:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-04-full
>>>>
>>>> This fix has been run through many rounds of JPRT and Mach5 tier[2-5]
>>>> testing, additional stress testing on Dan's Solaris X64 server, and
>>>> additional testing on Erik and Robbin's machines.
>>>>
>>>> We welcome comments, suggestions and feedback.
>>>>
>>>> Daniel Daugherty
>>>> Erik Osterlund
>>>> Robbin Ehn
>>>>
>>>
>>>
>>
>>
>
>



From kim.barrett at oracle.com  Wed Nov 15 20:59:54 2017
From: kim.barrett at oracle.com (Kim Barrett)
Date: Wed, 15 Nov 2017 15:59:54 -0500
Subject: RFR(XS): 8191337: GCC 4.8 build broken after 8186571
In-Reply-To: <3ddb39f79c244679928670bb8d56bb78@sap.com>
References: <3ddb39f79c244679928670bb8d56bb78@sap.com>
Message-ID: <3399BF1D-6C71-4787-ADD9-F5F4BC1205B2@oracle.com>

> On Nov 15, 2017, at 11:23 AM, Doerr, Martin <martin.doerr at sap.com> wrote:
> 
> Hi,
>  
> I?d like to build the VM with GCC 4.8.5, but I got the following error while linking:
> g1FullGCOopClosures.hpp:140: undefined reference to `void G1VerifyOopClosure::do_oop_nv<unsigned int>(unsigned int*)'
>  
> It can be fixed by:
> http://cr.openjdk.java.net/~mdoerr/8191337_fix_linux_build/webrev/
>  
> Please review. I will also need a sponsor.
>  
> Thanks and best regards,
> Martin

Rather than moving the do_oop definitions, maybe instead add explicit
instantiations of the template in the .cpp file, e.g.

template void G1VerifyOopClosure::do_oop_nv(oop*); 
template void G1VerifyOopClosure::do_oop_nv(narrowOop*); 

I'm not sure what's different about gcc4.9 (what we test with at
Oracle) that would have allowed the existing code to work at all.



From thomas.schatzl at oracle.com  Thu Nov 16 08:31:49 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 16 Nov 2017 09:31:49 +0100
Subject: RFR: 8189733: Cleanup Full GC setup and tear down
In-Reply-To: <3b9604e1-9d2e-c0d9-7b2e-e5d9232c6888@oracle.com>
References: <3b9604e1-9d2e-c0d9-7b2e-e5d9232c6888@oracle.com>
Message-ID: <1510821109.5332.1.camel@oracle.com>

Hi,

On Wed, 2017-11-15 at 17:24 +0100, Stefan Johansson wrote:
> Hi,
> 
> Please review this enhancement:
> https://bugs.openjdk.java.net/browse/JDK-8189733
> 
> Webrev:
> http://cr.openjdk.java.net/~sjohanss//8189733/00/index.html
> 
> Summary:
> After the G1FullCollector has been introduced as part of JEP 307 it?
> makes sense to move more of the setup and tear down for the Full GC
> into?
> this class. The G1FullGCScope is also moved into the G1FullCollector?
> since it is now only used here.

  looks good.

Thomas



From thomas.schatzl at oracle.com  Thu Nov 16 08:41:51 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 16 Nov 2017 09:41:51 +0100
Subject: RFR (XS): 8188877: Improper synchronization in offer_termination
In-Reply-To: <4D9972BA-B76C-4FB7-B24D-31D507DB1A6A@oracle.com>
References: <CY1PR0701MB16328948D4FDF19FBC30F65684710@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <3229BC22-BB8A-4D3D-8440-AB799AB5394F@oracle.com>
 <CY1PR0701MB16328E4C6563BE60CF8E99D084750@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <4D9972BA-B76C-4FB7-B24D-31D507DB1A6A@oracle.com>
Message-ID: <1510821711.5332.3.camel@oracle.com>

On Mon, 2017-10-09 at 21:44 -0400, Kim Barrett wrote:
> > On Oct 9, 2017, at 8:20 PM, White, Derek <Derek.White at cavium.com>
> > wrote:
> > > src/hotspot/share/gc/shared/taskqueue.cpp
> > > 
> > > The change to make _offered_termination volatile prevents the
> > > compiler
> > > from hoisting it out of the loop, but I'm not sure it is enough
> > > to prevent
> > > unnecessary delays in recognizing the termination condition.
> > > In particular, while offer_termination is spinning, it's not
> > > clear there's
> > > anything to ensure an update by some other thread will become
> > > visible
> > > during the repeated spin cycles.??yield and sleep seem likely to
> > > involve fences
> > > that will make such updates visible, but not so much for
> > > SpinPause.
> > 
> > [Derek]?
> > The concurrent access around _offered_termination is (semi*)-
> > independent of other data accesses. So I'm not sure if more
> > ordering constraints will help much here?
> > 
> > The atomic incr/dec operations provide a fence to publish the
> > change. On aarch64 this is a DMB instruction. A full sync (DSB)
> > will also publish the change, but blocks until the publish has been
> > acked. But it doesn't publish any faster.
> > 
> > The only extra thing we could do is read _offered_termination using
> > OrderAccess::load_acquire(). I think the only thing this adds in
> > this case is ordering between the loads of _offered_termination in
> > different??loop iterations. Is this required or helpful??
> 
> Consider the case where there is no further work to do, and all
> threads are heading toward offer_termination.??Until the last thread
> makes the offer, the others will be in the spin1...spinN/yield/sleep
> cycle, as expected.??But because there appear to be no memory
> barriers in the spin part of that cycle, a thread in the spin part
> might not see the final offer until it completes the full spin cycle
> and then performs the yield (assuming yield does have sufficient
> memory barriers to acquire the published value).
> 
> So I think the read of _offered_termination for comparison with
> _n_threads probably ought to be an acquire.

  I agree with Derek that adding an acquire for the load of
_offered_termination does not seem to improve termination properties,
and is imho unnecessary (but the volatile declaration of course is).

Thanks,
  Thomas



From rkennke at redhat.com  Thu Nov 16 09:23:03 2017
From: rkennke at redhat.com (Roman Kennke)
Date: Thu, 16 Nov 2017 10:23:03 +0100
Subject: RFR: 8189389: Move heap creation into GC interface
In-Reply-To: <66c8c65f-9dc6-8b78-59d8-da1efa36f6c5@redhat.com>
References: <66c8c65f-9dc6-8b78-59d8-da1efa36f6c5@redhat.com>
Message-ID: <7b33a506-e62d-dba4-3513-942586b6c0d7@redhat.com>


> This moves the heap creation code that is currently inside 
> universe.cpp to the GC specific GCArguments subclasses, thus reducing 
> GC specific stuff in shared code. It's based on top of 8189171 which 
> should arrive in the repo today or very soon.
>
> http://cr.openjdk.java.net/~rkennke/8189389/webrev.00/ 
> <http://cr.openjdk.java.net/%7Erkennke/8189389/webrev.00/>
>
> Test: hotspot_gc
>
> Ok?
>
> Roman
>
Ping?



From erik.osterlund at oracle.com  Thu Nov 16 09:56:15 2017
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Thu, 16 Nov 2017 10:56:15 +0100
Subject: RFR: 8189389: Move heap creation into GC interface
In-Reply-To: <7b33a506-e62d-dba4-3513-942586b6c0d7@redhat.com>
References: <66c8c65f-9dc6-8b78-59d8-da1efa36f6c5@redhat.com>
 <7b33a506-e62d-dba4-3513-942586b6c0d7@redhat.com>
Message-ID: <5A0D60BF.1020309@oracle.com>

Hi Roman,

Sorry for the delay. Am I correct to assume a gcArguments.inline.hpp is 
missing from this changeset, with a create_heap_with_policy() member 
function?
Otherwise, the approach looks good to me.

Thanks,
/Erik

On 2017-11-16 10:23, Roman Kennke wrote:
>
>> This moves the heap creation code that is currently inside 
>> universe.cpp to the GC specific GCArguments subclasses, thus reducing 
>> GC specific stuff in shared code. It's based on top of 8189171 which 
>> should arrive in the repo today or very soon.
>>
>> http://cr.openjdk.java.net/~rkennke/8189389/webrev.00/ 
>> <http://cr.openjdk.java.net/%7Erkennke/8189389/webrev.00/>
>>
>> Test: hotspot_gc
>>
>> Ok?
>>
>> Roman
>>
> Ping?
>



From rkennke at redhat.com  Thu Nov 16 12:07:26 2017
From: rkennke at redhat.com (Roman Kennke)
Date: Thu, 16 Nov 2017 13:07:26 +0100
Subject: RFR: 8189389: Move heap creation into GC interface
In-Reply-To: <5A0D60BF.1020309@oracle.com>
References: <66c8c65f-9dc6-8b78-59d8-da1efa36f6c5@redhat.com>
 <7b33a506-e62d-dba4-3513-942586b6c0d7@redhat.com>
 <5A0D60BF.1020309@oracle.com>
Message-ID: <745fd14b-b579-de5e-708a-c814147bd308@redhat.com>

Oops. Forgot hg add. Full webrev:

http://cr.openjdk.java.net/~rkennke/8189389/webrev.01/ 
<http://cr.openjdk.java.net/%7Erkennke/8189389/webrev.01/>

Ok now?

Roman

> Hi Roman,
>
> Sorry for the delay. Am I correct to assume a gcArguments.inline.hpp 
> is missing from this changeset, with a create_heap_with_policy() 
> member function?
> Otherwise, the approach looks good to me.
>
> Thanks,
> /Erik
>
> On 2017-11-16 10:23, Roman Kennke wrote:
>>
>>> This moves the heap creation code that is currently inside 
>>> universe.cpp to the GC specific GCArguments subclasses, thus 
>>> reducing GC specific stuff in shared code. It's based on top of 
>>> 8189171 which should arrive in the repo today or very soon.
>>>
>>> http://cr.openjdk.java.net/~rkennke/8189389/webrev.00/ 
>>> <http://cr.openjdk.java.net/%7Erkennke/8189389/webrev.00/>
>>>
>>> Test: hotspot_gc
>>>
>>> Ok?
>>>
>>> Roman
>>>
>> Ping?
>>
>



From per.liden at oracle.com  Thu Nov 16 13:56:36 2017
From: per.liden at oracle.com (Per Liden)
Date: Thu, 16 Nov 2017 14:56:36 +0100
Subject: RFR: 8189389: Move heap creation into GC interface
In-Reply-To: <745fd14b-b579-de5e-708a-c814147bd308@redhat.com>
References: <66c8c65f-9dc6-8b78-59d8-da1efa36f6c5@redhat.com>
 <7b33a506-e62d-dba4-3513-942586b6c0d7@redhat.com>
 <5A0D60BF.1020309@oracle.com>
 <745fd14b-b579-de5e-708a-c814147bd308@redhat.com>
Message-ID: <6f8fa87b-11cd-7d1d-fcaf-55e898cb666d@oracle.com>

Looks good!

cheers,
Per

On 2017-11-16 13:07, Roman Kennke wrote:
> Oops. Forgot hg add. Full webrev:
>
> http://cr.openjdk.java.net/~rkennke/8189389/webrev.01/
> <http://cr.openjdk.java.net/%7Erkennke/8189389/webrev.01/>
>
> Ok now?
>
> Roman
>
>> Hi Roman,
>>
>> Sorry for the delay. Am I correct to assume a gcArguments.inline.hpp
>> is missing from this changeset, with a create_heap_with_policy()
>> member function?
>> Otherwise, the approach looks good to me.
>>
>> Thanks,
>> /Erik
>>
>> On 2017-11-16 10:23, Roman Kennke wrote:
>>>
>>>> This moves the heap creation code that is currently inside
>>>> universe.cpp to the GC specific GCArguments subclasses, thus
>>>> reducing GC specific stuff in shared code. It's based on top of
>>>> 8189171 which should arrive in the repo today or very soon.
>>>>
>>>> http://cr.openjdk.java.net/~rkennke/8189389/webrev.00/
>>>> <http://cr.openjdk.java.net/%7Erkennke/8189389/webrev.00/>
>>>>
>>>> Test: hotspot_gc
>>>>
>>>> Ok?
>>>>
>>>> Roman
>>>>
>>> Ping?
>>>
>>
>


From bob.vandette at oracle.com  Thu Nov 16 15:16:06 2017
From: bob.vandette at oracle.com (Bob Vandette)
Date: Thu, 16 Nov 2017 10:16:06 -0500
Subject: RFR: 8189171: Move GC argument processing into GC specific classes
In-Reply-To: <c57f6ad3-9629-214a-cc41-34ac0c3688ad@redhat.com>
References: <e72385e3-88f1-9b05-8495-ab7f4c4cbd60@redhat.com>
 <ed0085ca-7384-05e7-4a2c-e2f048f64909@oracle.com>
 <5a7fecea-1a1b-16db-e366-4dd8d0c9ed63@redhat.com>
 <59E9E9A9.7020306@oracle.com>
 <511327b5-4632-0e27-6914-2a3d076a1dc3@redhat.com>
 <59EA002D.1050605@oracle.com>
 <2b37fed8-0383-eb19-3e45-076aabd74f22@redhat.com>
 <59F0AD40.3020307@oracle.com>
 <e8e953a1-06b3-648b-d5da-c87ee7c640ad@redhat.com>
 <B3E05F9C-5AC2-40D3-BBF3-A46ED9B12ED5@oracle.com>
 <affd801a-d184-1bcd-7c58-65c1ecaa94da@oracle.com>
 <3c75a6c5-ffe5-948f-b2c3-b4950a4b1a9f@redhat.com>
 <3f188247-aea1-55d7-cbd4-8ddcd33aa4a8@oracle.com>
 <c57f6ad3-9629-214a-cc41-34ac0c3688ad@redhat.com>
Message-ID: <B11BD63B-6459-486A-92C3-2C6D045740AF@oracle.com>

Roman,

It looks like this change may have caused the build of the minimal VM to
fail.  The minimal VM is no longer a configuration that we regularly build and test
but it is still a build option that can be selected via "--with-jvm-variants=minimal1"


/scratch/users/bobv/jdk10hs/open/src/hotspot/share/gc/shared/gcArguments.cpp: In static member function 'static jint GCArguments::initialize()':
/scratch/users/bobv/jdk10hs/open/src/hotspot/share/gc/shared/gcArguments.cpp:113:17: error: 'defaultStream' has not been declared
     jio_fprintf(defaultStream::error_stream(), "UseParallelGC not supported in this VM.\n");
                 ^
/scratch/users/bobv/jdk10hs/open/src/hotspot/share/gc/shared/gcArguments.cpp:116:17: error: 'defaultStream' has not been declared
     jio_fprintf(defaultStream::error_stream(), "UseG1GC not supported in this VM.\n");
                 ^
/scratch/users/bobv/jdk10hs/open/src/hotspot/share/gc/shared/gcArguments.cpp:119:17: error: 'defaultStream' has not been declared
     jio_fprintf(defaultStream::error_stream(), "UseConcMarkSweepGC not supported in this VM.\n");
                 ^
gmake[3]: *** [/scratch/users/bobv/jdk10hs/build/b00/linux-x64-minimal1/hotspot/variant-minimal/libjvm/objs/gcArguments.o] Error 1

Bob.


> On Nov 7, 2017, at 6:01 AM, Roman Kennke <rkennke at redhat.com> wrote:
> 
> Hi Per & Erik,
> 
> thanks for the reviews!
> 
> Now I need a sponsor.
> 
> Final webrev (same as before, including Reviewed-by line):
> http://cr.openjdk.java.net/~rkennke/8189171/webrev.05/ <http://cr.openjdk.java.net/%7Erkennke/8189171/webrev.05/>
> 
> Thanks, Roman
> 
> 
>> Looks good Roman!
>> 
>> /Per
>> 
>> On 2017-11-06 12:13, Roman Kennke wrote:
>>> Am 26.10.2017 um 11:43 schrieb Per Liden:
>>>> Hi,
>>>> 
>>>> On 2017-10-25 18:11, Erik Osterlund wrote:
>>>> [...]
>>>>>> So let me just put your changes up for review (again), if you don't
>>>>>> mind:
>>>>>> 
>>>>>> Full webrev:
>>>>>> http://cr.openjdk.java.net/~eosterlund/8189171/webrev.03/
>>>>>> <http://cr.openjdk.java.net/%7Eeosterlund/8189171/webrev.03/>
>>>> 
>>>> I like this. Just two naming suggestions:
>>>> 
>>>> src/hotspot/share/gc/shared/gcArguments.hpp:
>>>> 
>>>>   39   static jint create_instance();
>>>>   40   static bool is_initialized();
>>>>   41   static GCArguments* instance();
>>>> 
>>>> Could we call these:
>>>> 
>>>>   39   static jint initialize();
>>>>   40   static bool is_initialized();
>>>>   41   static GCArguments* arguments();
>>>> 
>>>> Reasoning: initialize() maps better to its companion is_initialized().
>>>> GCArguments::arguments() maps better to the existing patterns we have
>>>> with CollectedHeap::heap().
>>> Ok, that sounds good to me. Actually, it's almost full-circle back to my
>>> original proposal ;-)
>>> 
>>> Differential:
>>> http://cr.openjdk.java.net/~rkennke/8189171/webrev.04.diff/
>>> <http://cr.openjdk.java.net/%7Erkennke/8189171/webrev.04.diff/>
>>> Full:
>>> http://cr.openjdk.java.net/~rkennke/8189171/webrev.04/
>>> <http://cr.openjdk.java.net/%7Erkennke/8189171/webrev.04/>
>>> 
>>> Ok to push now?
>>> 
>>> Roman
> 
> 



From rkennke at redhat.com  Thu Nov 16 16:16:18 2017
From: rkennke at redhat.com (Roman Kennke)
Date: Thu, 16 Nov 2017 17:16:18 +0100
Subject: RFR: 8191424: Missing include in gcArguments.cpp
Message-ID: <c9e65b4e-52ec-62a8-93ad-3b9b74b32ba4@redhat.com>

I forgot to include "utilities/defaultStream.hpp" in gcArguments.cpp. 
This breaks the minimal JVM build.

http://cr.openjdk.java.net/~rkennke/8191424/webrev.00/ 
<http://cr.openjdk.java.net/%7Erkennke/8191424/webrev.00/>

OK?

Roman



From rkennke at redhat.com  Thu Nov 16 16:17:30 2017
From: rkennke at redhat.com (Roman Kennke)
Date: Thu, 16 Nov 2017 17:17:30 +0100
Subject: RFR: 8189171: Move GC argument processing into GC specific classes
In-Reply-To: <B11BD63B-6459-486A-92C3-2C6D045740AF@oracle.com>
References: <e72385e3-88f1-9b05-8495-ab7f4c4cbd60@redhat.com>
 <ed0085ca-7384-05e7-4a2c-e2f048f64909@oracle.com>
 <5a7fecea-1a1b-16db-e366-4dd8d0c9ed63@redhat.com>
 <59E9E9A9.7020306@oracle.com>
 <511327b5-4632-0e27-6914-2a3d076a1dc3@redhat.com>
 <59EA002D.1050605@oracle.com>
 <2b37fed8-0383-eb19-3e45-076aabd74f22@redhat.com>
 <59F0AD40.3020307@oracle.com>
 <e8e953a1-06b3-648b-d5da-c87ee7c640ad@redhat.com>
 <B3E05F9C-5AC2-40D3-BBF3-A46ED9B12ED5@oracle.com>
 <affd801a-d184-1bcd-7c58-65c1ecaa94da@oracle.com>
 <3c75a6c5-ffe5-948f-b2c3-b4950a4b1a9f@redhat.com>
 <3f188247-aea1-55d7-cbd4-8ddcd33aa4a8@oracle.com>
 <c57f6ad3-9629-214a-cc41-34ac0c3688ad@redhat.com>
 <B11BD63B-6459-486A-92C3-2C6D045740AF@oracle.com>
Message-ID: <411d8f0f-82e7-88e3-a1a4-12515b59452a@redhat.com>

Hi Bob,
thanks for letting me know.

I filed: https://bugs.openjdk.java.net/browse/JDK-8191424
and posted for review: 
http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2017-November/020784.html

Thanks, Roman

> Roman,
>
> It looks like this change may have caused the build of the minimal VM to
> fail.  The minimal VM is no longer a configuration that we regularly build and test
> but it is still a build option that can be selected via "--with-jvm-variants=minimal1"
>
>
> /scratch/users/bobv/jdk10hs/open/src/hotspot/share/gc/shared/gcArguments.cpp: In static member function 'static jint GCArguments::initialize()':
> /scratch/users/bobv/jdk10hs/open/src/hotspot/share/gc/shared/gcArguments.cpp:113:17: error: 'defaultStream' has not been declared
>       jio_fprintf(defaultStream::error_stream(), "UseParallelGC not supported in this VM.\n");
>                   ^
> /scratch/users/bobv/jdk10hs/open/src/hotspot/share/gc/shared/gcArguments.cpp:116:17: error: 'defaultStream' has not been declared
>       jio_fprintf(defaultStream::error_stream(), "UseG1GC not supported in this VM.\n");
>                   ^
> /scratch/users/bobv/jdk10hs/open/src/hotspot/share/gc/shared/gcArguments.cpp:119:17: error: 'defaultStream' has not been declared
>       jio_fprintf(defaultStream::error_stream(), "UseConcMarkSweepGC not supported in this VM.\n");
>                   ^
> gmake[3]: *** [/scratch/users/bobv/jdk10hs/build/b00/linux-x64-minimal1/hotspot/variant-minimal/libjvm/objs/gcArguments.o] Error 1
>
> Bob.
>
>
>> On Nov 7, 2017, at 6:01 AM, Roman Kennke <rkennke at redhat.com> wrote:
>>
>> Hi Per & Erik,
>>
>> thanks for the reviews!
>>
>> Now I need a sponsor.
>>
>> Final webrev (same as before, including Reviewed-by line):
>> http://cr.openjdk.java.net/~rkennke/8189171/webrev.05/ <http://cr.openjdk.java.net/%7Erkennke/8189171/webrev.05/>
>>
>> Thanks, Roman
>>
>>
>>> Looks good Roman!
>>>
>>> /Per
>>>
>>> On 2017-11-06 12:13, Roman Kennke wrote:
>>>> Am 26.10.2017 um 11:43 schrieb Per Liden:
>>>>> Hi,
>>>>>
>>>>> On 2017-10-25 18:11, Erik Osterlund wrote:
>>>>> [...]
>>>>>>> So let me just put your changes up for review (again), if you don't
>>>>>>> mind:
>>>>>>>
>>>>>>> Full webrev:
>>>>>>> http://cr.openjdk.java.net/~eosterlund/8189171/webrev.03/
>>>>>>> <http://cr.openjdk.java.net/%7Eeosterlund/8189171/webrev.03/>
>>>>> I like this. Just two naming suggestions:
>>>>>
>>>>> src/hotspot/share/gc/shared/gcArguments.hpp:
>>>>>
>>>>>    39   static jint create_instance();
>>>>>    40   static bool is_initialized();
>>>>>    41   static GCArguments* instance();
>>>>>
>>>>> Could we call these:
>>>>>
>>>>>    39   static jint initialize();
>>>>>    40   static bool is_initialized();
>>>>>    41   static GCArguments* arguments();
>>>>>
>>>>> Reasoning: initialize() maps better to its companion is_initialized().
>>>>> GCArguments::arguments() maps better to the existing patterns we have
>>>>> with CollectedHeap::heap().
>>>> Ok, that sounds good to me. Actually, it's almost full-circle back to my
>>>> original proposal ;-)
>>>>
>>>> Differential:
>>>> http://cr.openjdk.java.net/~rkennke/8189171/webrev.04.diff/
>>>> <http://cr.openjdk.java.net/%7Erkennke/8189171/webrev.04.diff/>
>>>> Full:
>>>> http://cr.openjdk.java.net/~rkennke/8189171/webrev.04/
>>>> <http://cr.openjdk.java.net/%7Erkennke/8189171/webrev.04/>
>>>>
>>>> Ok to push now?
>>>>
>>>> Roman
>>



From bob.vandette at oracle.com  Thu Nov 16 16:33:12 2017
From: bob.vandette at oracle.com (Bob Vandette)
Date: Thu, 16 Nov 2017 11:33:12 -0500
Subject: RFR: 8189171: Move GC argument processing into GC specific classes
In-Reply-To: <411d8f0f-82e7-88e3-a1a4-12515b59452a@redhat.com>
References: <e72385e3-88f1-9b05-8495-ab7f4c4cbd60@redhat.com>
 <ed0085ca-7384-05e7-4a2c-e2f048f64909@oracle.com>
 <5a7fecea-1a1b-16db-e366-4dd8d0c9ed63@redhat.com>
 <59E9E9A9.7020306@oracle.com>
 <511327b5-4632-0e27-6914-2a3d076a1dc3@redhat.com>
 <59EA002D.1050605@oracle.com>
 <2b37fed8-0383-eb19-3e45-076aabd74f22@redhat.com>
 <59F0AD40.3020307@oracle.com>
 <e8e953a1-06b3-648b-d5da-c87ee7c640ad@redhat.com>
 <B3E05F9C-5AC2-40D3-BBF3-A46ED9B12ED5@oracle.com>
 <affd801a-d184-1bcd-7c58-65c1ecaa94da@oracle.com>
 <3c75a6c5-ffe5-948f-b2c3-b4950a4b1a9f@redhat.com>
 <3f188247-aea1-55d7-cbd4-8ddcd33aa4a8@oracle.com>
 <c57f6ad3-9629-214a-cc41-34ac0c3688ad@redhat.com>
 <B11BD63B-6459-486A-92C3-2C6D045740AF@oracle.com>
 <411d8f0f-82e7-88e3-a1a4-12515b59452a@redhat.com>
Message-ID: <A8796C24-E60A-484F-B2CA-00C00F4AD908@oracle.com>

Yes, that?s the fix.  I built it correctly with that added include.

There are still a few other issues with building and using the minimal VM but those 
existed before your change.

1. You cannot build the minimal VM without also building the server VM.  There
is a Makefile dependency on the server VM.
2. The jvm.cfg produced in the jvm and jre images does not add the minimalvm
as one of the possible VM to select.  As a work-around jlink produces a workable jvm.cfg.

Thanks,
Bob.


> On Nov 16, 2017, at 11:17 AM, Roman Kennke <rkennke at redhat.com> wrote:
> 
> Hi Bob,
> thanks for letting me know.
> 
> I filed: https://bugs.openjdk.java.net/browse/JDK-8191424
> and posted for review: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2017-November/020784.html
> 
> Thanks, Roman
> 
>> Roman,
>> 
>> It looks like this change may have caused the build of the minimal VM to
>> fail.  The minimal VM is no longer a configuration that we regularly build and test
>> but it is still a build option that can be selected via "--with-jvm-variants=minimal1"
>> 
>> 
>> /scratch/users/bobv/jdk10hs/open/src/hotspot/share/gc/shared/gcArguments.cpp: In static member function 'static jint GCArguments::initialize()':
>> /scratch/users/bobv/jdk10hs/open/src/hotspot/share/gc/shared/gcArguments.cpp:113:17: error: 'defaultStream' has not been declared
>>      jio_fprintf(defaultStream::error_stream(), "UseParallelGC not supported in this VM.\n");
>>                  ^
>> /scratch/users/bobv/jdk10hs/open/src/hotspot/share/gc/shared/gcArguments.cpp:116:17: error: 'defaultStream' has not been declared
>>      jio_fprintf(defaultStream::error_stream(), "UseG1GC not supported in this VM.\n");
>>                  ^
>> /scratch/users/bobv/jdk10hs/open/src/hotspot/share/gc/shared/gcArguments.cpp:119:17: error: 'defaultStream' has not been declared
>>      jio_fprintf(defaultStream::error_stream(), "UseConcMarkSweepGC not supported in this VM.\n");
>>                  ^
>> gmake[3]: *** [/scratch/users/bobv/jdk10hs/build/b00/linux-x64-minimal1/hotspot/variant-minimal/libjvm/objs/gcArguments.o] Error 1
>> 
>> Bob.
>> 
>> 
>>> On Nov 7, 2017, at 6:01 AM, Roman Kennke <rkennke at redhat.com> wrote:
>>> 
>>> Hi Per & Erik,
>>> 
>>> thanks for the reviews!
>>> 
>>> Now I need a sponsor.
>>> 
>>> Final webrev (same as before, including Reviewed-by line):
>>> http://cr.openjdk.java.net/~rkennke/8189171/webrev.05/ <http://cr.openjdk.java.net/%7Erkennke/8189171/webrev.05/>
>>> 
>>> Thanks, Roman
>>> 
>>> 
>>>> Looks good Roman!
>>>> 
>>>> /Per
>>>> 
>>>> On 2017-11-06 12:13, Roman Kennke wrote:
>>>>> Am 26.10.2017 um 11:43 schrieb Per Liden:
>>>>>> Hi,
>>>>>> 
>>>>>> On 2017-10-25 18:11, Erik Osterlund wrote:
>>>>>> [...]
>>>>>>>> So let me just put your changes up for review (again), if you don't
>>>>>>>> mind:
>>>>>>>> 
>>>>>>>> Full webrev:
>>>>>>>> http://cr.openjdk.java.net/~eosterlund/8189171/webrev.03/
>>>>>>>> <http://cr.openjdk.java.net/%7Eeosterlund/8189171/webrev.03/>
>>>>>> I like this. Just two naming suggestions:
>>>>>> 
>>>>>> src/hotspot/share/gc/shared/gcArguments.hpp:
>>>>>> 
>>>>>>   39   static jint create_instance();
>>>>>>   40   static bool is_initialized();
>>>>>>   41   static GCArguments* instance();
>>>>>> 
>>>>>> Could we call these:
>>>>>> 
>>>>>>   39   static jint initialize();
>>>>>>   40   static bool is_initialized();
>>>>>>   41   static GCArguments* arguments();
>>>>>> 
>>>>>> Reasoning: initialize() maps better to its companion is_initialized().
>>>>>> GCArguments::arguments() maps better to the existing patterns we have
>>>>>> with CollectedHeap::heap().
>>>>> Ok, that sounds good to me. Actually, it's almost full-circle back to my
>>>>> original proposal ;-)
>>>>> 
>>>>> Differential:
>>>>> http://cr.openjdk.java.net/~rkennke/8189171/webrev.04.diff/
>>>>> <http://cr.openjdk.java.net/%7Erkennke/8189171/webrev.04.diff/>
>>>>> Full:
>>>>> http://cr.openjdk.java.net/~rkennke/8189171/webrev.04/
>>>>> <http://cr.openjdk.java.net/%7Erkennke/8189171/webrev.04/>
>>>>> 
>>>>> Ok to push now?
>>>>> 
>>>>> Roman
>>> 
> 



From erik.osterlund at oracle.com  Thu Nov 16 16:41:44 2017
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Thu, 16 Nov 2017 17:41:44 +0100
Subject: RFR: 8191424: Missing include in gcArguments.cpp
In-Reply-To: <c9e65b4e-52ec-62a8-93ad-3b9b74b32ba4@redhat.com>
References: <c9e65b4e-52ec-62a8-93ad-3b9b74b32ba4@redhat.com>
Message-ID: <5A0DBFC8.2060403@oracle.com>

Hi Roman,

Looks good.

Thanks,
/Erik

On 2017-11-16 17:16, Roman Kennke wrote:
> I forgot to include "utilities/defaultStream.hpp" in gcArguments.cpp. 
> This breaks the minimal JVM build.
>
> http://cr.openjdk.java.net/~rkennke/8191424/webrev.00/ 
> <http://cr.openjdk.java.net/%7Erkennke/8191424/webrev.00/>
>
> OK?
>
> Roman
>



From rkennke at redhat.com  Thu Nov 16 16:41:11 2017
From: rkennke at redhat.com (Roman Kennke)
Date: Thu, 16 Nov 2017 17:41:11 +0100
Subject: RFR: 8191424: Missing include in gcArguments.cpp
In-Reply-To: <5A0DBFC8.2060403@oracle.com>
References: <c9e65b4e-52ec-62a8-93ad-3b9b74b32ba4@redhat.com>
 <5A0DBFC8.2060403@oracle.com>
Message-ID: <ecc35952-e68b-5653-cae8-0a02ffd7f986@redhat.com>

Hi Erik,
thanks for reviewing!

Want to sponsor too?

(And while you're at it, sponsor 8189389 too?)

Roman

> Hi Roman,
>
> Looks good.
>
> Thanks,
> /Erik
>
> On 2017-11-16 17:16, Roman Kennke wrote:
>> I forgot to include "utilities/defaultStream.hpp" in gcArguments.cpp. 
>> This breaks the minimal JVM build.
>>
>> http://cr.openjdk.java.net/~rkennke/8191424/webrev.00/ 
>> <http://cr.openjdk.java.net/%7Erkennke/8191424/webrev.00/>
>>
>> OK?
>>
>> Roman
>>
>



From alexander.harlap at oracle.com  Thu Nov 16 19:46:43 2017
From: alexander.harlap at oracle.com (Alexander Harlap)
Date: Thu, 16 Nov 2017 14:46:43 -0500
Subject: Need sponsor to push attached 8187819 into
 http://hg.openjdk.java.net/jdk10/hs
Message-ID: <7c02edb4-1a18-e750-3c6b-28e303fb2b82@oracle.com>

I need a sponsor to push attached8187819.patch - gc/TestFullGCALot.java 
fails on jdk10 started with "-XX:-UseCompressedOops" option

Patch should go into jdk10/hs

Reviewed by Thomas Schatzl? and Stefan Johansson.

Thank you,

Alex

-------------- next part --------------
# HG changeset patch
# User aharlap
# Date 1510859204 18000
# Node ID b84e87a8ac77f4798614ccf72eee50cfc88f004c
# Parent  d85284ccd1bd865ebb1391d921be7a8ebfc5f2c9
8187819: gc/TestFullGCALot.java fails on jdk10 started with "-XX:-UseCompressedOops" option
Summary: Need to initialized metaspace performance counters before their potential use
Reviewed-by: tschatzl, sjohanss

diff -r d85284ccd1bd -r b84e87a8ac77 src/hotspot/share/memory/universe.cpp
--- a/src/hotspot/share/memory/universe.cpp	Fri Nov 03 17:09:25 2017 -0700
+++ b/src/hotspot/share/memory/universe.cpp	Thu Nov 16 14:06:44 2017 -0500
@@ -695,6 +695,10 @@
 
   Metaspace::global_initialize();
 
+  // Initialize performance counters for metaspaces
+  MetaspaceCounters::initialize_performance_counters();
+  CompressedClassSpaceCounters::initialize_performance_counters();
+
   AOTLoader::universe_init();
 
   // Checks 'AfterMemoryInit' constraints.
@@ -1112,10 +1116,6 @@
   // ("weak") refs processing infrastructure initialization
   Universe::heap()->post_initialize();
 
-  // Initialize performance counters for metaspaces
-  MetaspaceCounters::initialize_performance_counters();
-  CompressedClassSpaceCounters::initialize_performance_counters();
-
   MemoryService::add_metaspace_memory_pools();
 
   MemoryService::set_universe_heap(Universe::heap());
diff -r d85284ccd1bd -r b84e87a8ac77 test/hotspot/jtreg/gc/TestFullGCALot.java
--- a/test/hotspot/jtreg/gc/TestFullGCALot.java	Fri Nov 03 17:09:25 2017 -0700
+++ b/test/hotspot/jtreg/gc/TestFullGCALot.java	Thu Nov 16 14:06:44 2017 -0500
@@ -24,7 +24,7 @@
 /*
  * @test TestFullGCALot
  * @key gc
- * @bug 4187687
+ * @bug 4187687 8187819
  * @summary Ensure no access violation when using FullGCALot
  * @requires vm.debug
  * @run main/othervm -XX:NewSize=10m -XX:+FullGCALot -XX:FullGCALotInterval=120 TestFullGCALot

From kim.barrett at oracle.com  Thu Nov 16 23:01:57 2017
From: kim.barrett at oracle.com (Kim Barrett)
Date: Thu, 16 Nov 2017 18:01:57 -0500
Subject: RFR: 8191424: Missing include in gcArguments.cpp
In-Reply-To: <c9e65b4e-52ec-62a8-93ad-3b9b74b32ba4@redhat.com>
References: <c9e65b4e-52ec-62a8-93ad-3b9b74b32ba4@redhat.com>
Message-ID: <57784779-31B6-4CEF-BC2E-0C91F836CC00@oracle.com>

> On Nov 16, 2017, at 11:16 AM, Roman Kennke <rkennke at redhat.com> wrote:
> 
> I forgot to include "utilities/defaultStream.hpp" in gcArguments.cpp. This breaks the minimal JVM build.
> 
> http://cr.openjdk.java.net/~rkennke/8191424/webrev.00/ <http://cr.openjdk.java.net/%7Erkennke/8191424/webrev.00/>
> 
> OK?
> 
> Roman

Looks good.



From sangheon.kim at oracle.com  Fri Nov 17 06:27:36 2017
From: sangheon.kim at oracle.com (sangheon.kim)
Date: Thu, 16 Nov 2017 22:27:36 -0800
Subject: RFR(M): 8190828: Test plan: JEP 8171181: Support heap allocation
 on alternative memory devices
In-Reply-To: <F89640DCD01A85489FCBA68183A6A0F3BCD54D7C@ORSMSX116.amr.corp.intel.com>
References: <F89640DCD01A85489FCBA68183A6A0F3BCD54D7C@ORSMSX116.amr.corp.intel.com>
Message-ID: <08b72b19-9624-8bee-aab4-b456f3b89497@oracle.com>

Hi Kishor,

On 11/13/2017 03:51 PM, Kharbas, Kishor wrote:
>
> Hi!
>
> I have developed a test plan for the implementation of 8171181.
>
> I would appreciate a review and further guidance from the gc-dev 
> members. I am hoping to get everything done well before 18.3 code 
> freeze (have a vacation planned during that time).
>
> Test plan: https://bugs.openjdk.java.net/browse/JDK-8190828 
> <https://bugs.openjdk.java.net/browse/JDK-8190828>
>
> Test webrev: http://cr.openjdk.java.net/~kkharbas/8190980/webrev.01/ 
> <http://cr.openjdk.java.net/%7Ekkharbas/8190980/webrev.01/>
>
Looking at the comment at 8190980, webrev.2 seems the latest one, so my 
comments are for the webrev.2.

--------------------------------------
test/hotspot/jtreg/gc/TestAllocateHeapAtMultiple.java

 ? 51???? String[] extraOptsList = new String[] {
 ? 52?????? "-Xmx32m -Xms32m -XX:+UseCompressedOops",???? // 1. With 
compressedoops enabled.
 ? 53?????? "-Xmx32m -Xms32m -XX:-UseCompressedOops",???? // 2. With 
compressedoops disabled.
 ? 54?????? "-Xmx32m -Xms32m -XX:HeapBaseMinAddress=3g",? // 3. With 
user specified HeapBaseMinAddress.
 ? 55?????? "-Xmx4g -Xms4g",????????????????????????????? // 4. With 
larger heap size (UnscaledNarrowOop not possible).
 ? 56?????? "-Xmx4g -Xms4g -XX:+UseLargePages",?????????? // 5. Set 
UseLargePages.
 ? 57?????? "-Xmx4g -Xms4g -XX:+UseNUMA"????????????????? // 6. Set UseNUMA.
 ? 58???? };
- I think we do differently to run sub-tests. Maybe SQE folks would give 
better comment on this.
e.g. TestAllocationInEden.java
 ?* @run main/othervm -Xbootclasspath/a:. -XX:+UnlockDiagnosticVMOptions
...
 ?*?????????????????? TestAllocationInEden 10m 9 EDEN
 ?* @run main/othervm -Xbootclasspath/a:. -XX:+UnlockDiagnosticVMOptions
...
 ?*?????????????????? TestAllocationInEden 10m 47 EDEN
 ?* @run main/othervm -Xbootclasspath/a:. -XX:+UnlockDiagnosticVMOptions

52?????? "-Xmx32m -Xms32m -XX:+UseCompressedOops",???? // 1. With 
compressedoops enabled.
54?????? "-Xmx32m -Xms32m -XX:HeapBaseMinAddress=3g",? // 3. With user 
specified HeapBaseMinAddress.
55?????? "-Xmx4g -Xms4g",????????????????????????????? // 4. With larger 
heap size (UnscaledNarrowOop not possible).
- I think these 3 sub-tests are testing different compressed oop modes. 
I would recommend to include other 1 type(non-zero based) as well. In 
addition, adding the comment also would help increase the readability.

--------------------------------------
test/hotspot/jtreg/gc/stress/gcbasher/TestGCBasherWithAllocateHeapAt.java

1) This seems identical to TestGCBasherWithG1.java, how about just 
adding another '@run'?
i.e.? adding "* @run main/othervm/timeout=500 -Xlog:gc*=info -Xmx256m 
-server -XX:+UseG1GC _*-XX:AllocateHeapAt=.*_ TestGCBasherWithG1 120000"

2) Don't we need testing for other GC types as well? i.e. Serial, 
Parallel and CMS.

2? * Copyright (c) 2016, 2017, Oracle and/or its affiliates. All rights 
reserved.
- Is this intended to start from 2016 as this seems to be copied from 
TestGCBasherWithXXX.java?

34? * @run main/othervm/timeout=500 -Xlog:gc*=info -Xmx256m -server 
-XX:+UseG1GC -XX:AllocateHeapAt=. TestGCBasherWithAllocateHeapAt 120000
- Are there any reason to use timeout of 500? TestGCBasherWithG1 is 
using 200ms.

Thanks,
Sangheon



> JEP: https://bugs.openjdk.java.net/browse/JDK-8171181
>
> Implementation webrev : 
> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.15/ 
> <http://cr.openjdk.java.net/%7Ekkharbas/8190308/webrev.15/>
>
> Thank you!
>
> Kishor
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20171116/e1b1813f/attachment.htm>

From stefan.johansson at oracle.com  Fri Nov 17 09:06:49 2017
From: stefan.johansson at oracle.com (Stefan Johansson)
Date: Fri, 17 Nov 2017 10:06:49 +0100
Subject: RFR(XS): 8191337: GCC 4.8 build broken after 8186571
In-Reply-To: <3399BF1D-6C71-4787-ADD9-F5F4BC1205B2@oracle.com>
References: <3ddb39f79c244679928670bb8d56bb78@sap.com>
 <3399BF1D-6C71-4787-ADD9-F5F4BC1205B2@oracle.com>
Message-ID: <014ae4e0-37ab-eb0b-a7d4-4b5a97288cd6@oracle.com>



On 2017-11-15 21:59, Kim Barrett wrote:
>> On Nov 15, 2017, at 11:23 AM, Doerr, Martin <martin.doerr at sap.com> wrote:
>>
>> Hi,
>>   
>> I?d like to build the VM with GCC 4.8.5, but I got the following error while linking:
>> g1FullGCOopClosures.hpp:140: undefined reference to `void G1VerifyOopClosure::do_oop_nv<unsigned int>(unsigned int*)'
>>   
>> It can be fixed by:
>> http://cr.openjdk.java.net/~mdoerr/8191337_fix_linux_build/webrev/
>>   
>> Please review. I will also need a sponsor.
>>   
>> Thanks and best regards,
>> Martin
> Rather than moving the do_oop definitions, maybe instead add explicit
> instantiations of the template in the .cpp file, e.g.
>
> template void G1VerifyOopClosure::do_oop_nv(oop*);
> template void G1VerifyOopClosure::do_oop_nv(narrowOop*);
Thanks Kim,

I spoke to Martin and we agreed on using this solution. Martin created a 
new webrev and I will put myself as the second reviewer and push:
http://cr.openjdk.java.net/~mdoerr/8191337_fix_linux_build/webrev.01/

Cheers,
Stefan

>
> I'm not sure what's different about gcc4.9 (what we test with at
> Oracle) that would have allowed the existing code to work at all.
>



From martin.doerr at sap.com  Fri Nov 17 09:17:16 2017
From: martin.doerr at sap.com (Doerr, Martin)
Date: Fri, 17 Nov 2017 09:17:16 +0000
Subject: RFR(XS): 8191337: GCC 4.8 build broken after 8186571
In-Reply-To: <014ae4e0-37ab-eb0b-a7d4-4b5a97288cd6@oracle.com>
References: <3ddb39f79c244679928670bb8d56bb78@sap.com>
 <3399BF1D-6C71-4787-ADD9-F5F4BC1205B2@oracle.com>
 <014ae4e0-37ab-eb0b-a7d4-4b5a97288cd6@oracle.com>
Message-ID: <29891eaa0eb943c19dabd8820c6c1de9@sap.com>

Hi Stefan and Kim,

thanks for sponsoring and reviewing.

Best regards,
Martin


-----Original Message-----
From: hotspot-gc-dev [mailto:hotspot-gc-dev-bounces at openjdk.java.net] On Behalf Of Stefan Johansson
Sent: Freitag, 17. November 2017 10:07
To: hotspot-gc-dev at openjdk.java.net
Subject: Re: RFR(XS): 8191337: GCC 4.8 build broken after 8186571



On 2017-11-15 21:59, Kim Barrett wrote:
>> On Nov 15, 2017, at 11:23 AM, Doerr, Martin <martin.doerr at sap.com> wrote:
>>
>> Hi,
>>   
>> I?d like to build the VM with GCC 4.8.5, but I got the following error while linking:
>> g1FullGCOopClosures.hpp:140: undefined reference to `void G1VerifyOopClosure::do_oop_nv<unsigned int>(unsigned int*)'
>>   
>> It can be fixed by:
>> http://cr.openjdk.java.net/~mdoerr/8191337_fix_linux_build/webrev/
>>   
>> Please review. I will also need a sponsor.
>>   
>> Thanks and best regards,
>> Martin
> Rather than moving the do_oop definitions, maybe instead add explicit
> instantiations of the template in the .cpp file, e.g.
>
> template void G1VerifyOopClosure::do_oop_nv(oop*);
> template void G1VerifyOopClosure::do_oop_nv(narrowOop*);
Thanks Kim,

I spoke to Martin and we agreed on using this solution. Martin created a 
new webrev and I will put myself as the second reviewer and push:
http://cr.openjdk.java.net/~mdoerr/8191337_fix_linux_build/webrev.01/

Cheers,
Stefan

>
> I'm not sure what's different about gcc4.9 (what we test with at
> Oracle) that would have allowed the existing code to work at all.
>


From erik.osterlund at oracle.com  Fri Nov 17 09:25:08 2017
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Fri, 17 Nov 2017 10:25:08 +0100
Subject: RFR: 8191424: Missing include in gcArguments.cpp
In-Reply-To: <ecc35952-e68b-5653-cae8-0a02ffd7f986@redhat.com>
References: <c9e65b4e-52ec-62a8-93ad-3b9b74b32ba4@redhat.com>
 <5A0DBFC8.2060403@oracle.com>
 <ecc35952-e68b-5653-cae8-0a02ffd7f986@redhat.com>
Message-ID: <5A0EAAF4.6070902@oracle.com>

Hi Roman,

Sure, I am on it.

Thanks,
/Erik

On 2017-11-16 17:41, Roman Kennke wrote:
> Hi Erik,
> thanks for reviewing!
>
> Want to sponsor too?
>
> (And while you're at it, sponsor 8189389 too?)
>
> Roman
>
>> Hi Roman,
>>
>> Looks good.
>>
>> Thanks,
>> /Erik
>>
>> On 2017-11-16 17:16, Roman Kennke wrote:
>>> I forgot to include "utilities/defaultStream.hpp" in 
>>> gcArguments.cpp. This breaks the minimal JVM build.
>>>
>>> http://cr.openjdk.java.net/~rkennke/8191424/webrev.00/ 
>>> <http://cr.openjdk.java.net/%7Erkennke/8191424/webrev.00/>
>>>
>>> OK?
>>>
>>> Roman
>>>
>>
>



From erik.osterlund at oracle.com  Fri Nov 17 09:26:07 2017
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Fri, 17 Nov 2017 10:26:07 +0100
Subject: RFR: 8189389: Move heap creation into GC interface
In-Reply-To: <745fd14b-b579-de5e-708a-c814147bd308@redhat.com>
References: <66c8c65f-9dc6-8b78-59d8-da1efa36f6c5@redhat.com>
 <7b33a506-e62d-dba4-3513-942586b6c0d7@redhat.com>
 <5A0D60BF.1020309@oracle.com>
 <745fd14b-b579-de5e-708a-c814147bd308@redhat.com>
Message-ID: <5A0EAB2F.4010103@oracle.com>

Hi Roman,

New version looks good.

Thanks,
/Erik

On 2017-11-16 13:07, Roman Kennke wrote:
> Oops. Forgot hg add. Full webrev:
>
> http://cr.openjdk.java.net/~rkennke/8189389/webrev.01/ 
> <http://cr.openjdk.java.net/%7Erkennke/8189389/webrev.01/>
>
> Ok now?
>
> Roman
>
>> Hi Roman,
>>
>> Sorry for the delay. Am I correct to assume a gcArguments.inline.hpp 
>> is missing from this changeset, with a create_heap_with_policy() 
>> member function?
>> Otherwise, the approach looks good to me.
>>
>> Thanks,
>> /Erik
>>
>> On 2017-11-16 10:23, Roman Kennke wrote:
>>>
>>>> This moves the heap creation code that is currently inside 
>>>> universe.cpp to the GC specific GCArguments subclasses, thus 
>>>> reducing GC specific stuff in shared code. It's based on top of 
>>>> 8189171 which should arrive in the repo today or very soon.
>>>>
>>>> http://cr.openjdk.java.net/~rkennke/8189389/webrev.00/ 
>>>> <http://cr.openjdk.java.net/%7Erkennke/8189389/webrev.00/>
>>>>
>>>> Test: hotspot_gc
>>>>
>>>> Ok?
>>>>
>>>> Roman
>>>>
>>> Ping?
>>>
>>
>



From kim.barrett at oracle.com  Fri Nov 17 16:50:12 2017
From: kim.barrett at oracle.com (Kim Barrett)
Date: Fri, 17 Nov 2017 11:50:12 -0500
Subject: RFR(XS): 8191337: GCC 4.8 build broken after 8186571
In-Reply-To: <014ae4e0-37ab-eb0b-a7d4-4b5a97288cd6@oracle.com>
References: <3ddb39f79c244679928670bb8d56bb78@sap.com>
 <3399BF1D-6C71-4787-ADD9-F5F4BC1205B2@oracle.com>
 <014ae4e0-37ab-eb0b-a7d4-4b5a97288cd6@oracle.com>
Message-ID: <AD6D3CE3-BC48-4817-87CE-090890E0E3C8@oracle.com>

> On Nov 17, 2017, at 4:06 AM, Stefan Johansson <stefan.johansson at oracle.com> wrote:
> 
> 
> 
> On 2017-11-15 21:59, Kim Barrett wrote:
>>> On Nov 15, 2017, at 11:23 AM, Doerr, Martin <martin.doerr at sap.com> wrote:
>>> 
>>> Hi,
>>>  I?d like to build the VM with GCC 4.8.5, but I got the following error while linking:
>>> g1FullGCOopClosures.hpp:140: undefined reference to `void G1VerifyOopClosure::do_oop_nv<unsigned int>(unsigned int*)'
>>>  It can be fixed by:
>>> http://cr.openjdk.java.net/~mdoerr/8191337_fix_linux_build/webrev/
>>>  Please review. I will also need a sponsor.
>>>  Thanks and best regards,
>>> Martin
>> Rather than moving the do_oop definitions, maybe instead add explicit
>> instantiations of the template in the .cpp file, e.g.
>> 
>> template void G1VerifyOopClosure::do_oop_nv(oop*);
>> template void G1VerifyOopClosure::do_oop_nv(narrowOop*);
> Thanks Kim,
> 
> I spoke to Martin and we agreed on using this solution. Martin created a new webrev and I will put myself as the second reviewer and push:
> http://cr.openjdk.java.net/~mdoerr/8191337_fix_linux_build/webrev.01/

That looks good.

> 
> Cheers,
> Stefan
> 
>> 
>> I'm not sure what's different about gcc4.9 (what we test with at
>> Oracle) that would have allowed the existing code to work at all.




From mark.reinhold at oracle.com  Fri Nov 17 22:39:37 2017
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Fri, 17 Nov 2017 14:39:37 -0800 (PST)
Subject: JEP 318: Epsilon: An Arbitrarily Low-Overhead Garbage Collector
Message-ID: <20171117223937.1B72EFE35B@eggemoggin.niobe.net>

New JEP Candidate: http://openjdk.java.net/jeps/318

- Mark


From kishor.kharbas at intel.com  Fri Nov 17 22:59:06 2017
From: kishor.kharbas at intel.com (Kharbas, Kishor)
Date: Fri, 17 Nov 2017 22:59:06 +0000
Subject: RFR(M): 8190828: Test plan: JEP 8171181: Support heap
 allocation on alternative memory devices
In-Reply-To: <08b72b19-9624-8bee-aab4-b456f3b89497@oracle.com>
References: <F89640DCD01A85489FCBA68183A6A0F3BCD54D7C@ORSMSX116.amr.corp.intel.com>
 <08b72b19-9624-8bee-aab4-b456f3b89497@oracle.com>
Message-ID: <F89640DCD01A85489FCBA68183A6A0F3BCD56806@ORSMSX116.amr.corp.intel.com>

Hi Sangheon!
Thanks for the reply. Please find my reply inline.

From: sangheon.kim [mailto:sangheon.kim at oracle.com]
Sent: Thursday, November 16, 2017 10:28 PM
To: Kharbas, Kishor <kishor.kharbas at intel.com>; hotspot-gc-dev at openjdk.java.net
Subject: Re: RFR(M): 8190828: Test plan: JEP 8171181: Support heap allocation on alternative memory devices

Hi Kishor,
On 11/13/2017 03:51 PM, Kharbas, Kishor wrote:
Hi!

I have developed a test plan for the implementation of 8171181.
I would appreciate a review and further guidance from the gc-dev members. I am hoping to get everything done well before 18.3 code freeze (have a vacation planned during that time).

Test plan: https://bugs.openjdk.java.net/browse/JDK-8190828
Test webrev: http://cr.openjdk.java.net/~kkharbas/8190980/webrev.01/<http://cr.openjdk.java.net/%7Ekkharbas/8190980/webrev.01/>
Looking at the comment at 8190980, webrev.2 seems the latest one, so my comments are for the webrev.2.

--------------------------------------
test/hotspot/jtreg/gc/TestAllocateHeapAtMultiple.java

  51     String[] extraOptsList = new String[] {
  52       "-Xmx32m -Xms32m -XX:+UseCompressedOops",     // 1. With compressedoops enabled.
  53       "-Xmx32m -Xms32m -XX:-UseCompressedOops",     // 2. With compressedoops disabled.
  54       "-Xmx32m -Xms32m -XX:HeapBaseMinAddress=3g",  // 3. With user specified HeapBaseMinAddress.
  55       "-Xmx4g -Xms4g",                              // 4. With larger heap size (UnscaledNarrowOop not possible).
  56       "-Xmx4g -Xms4g -XX:+UseLargePages",           // 5. Set UseLargePages.
  57       "-Xmx4g -Xms4g -XX:+UseNUMA"                  // 6. Set UseNUMA.
  58     };
- I think we do differently to run sub-tests. Maybe SQE folks would give better comment on this.
e.g. TestAllocationInEden.java
 * @run main/othervm -Xbootclasspath/a:. -XX:+UnlockDiagnosticVMOptions
...
 *                   TestAllocationInEden 10m 9 EDEN
 * @run main/othervm -Xbootclasspath/a:. -XX:+UnlockDiagnosticVMOptions
...
 *                   TestAllocationInEden 10m 47 EDEN
 * @run main/othervm -Xbootclasspath/a:. -XX:+UnlockDiagnosticVMOptions

[Kharbas, Kishor] In my tests I use stdout to check for log generated by -Xlog:heap+gc=info to test correct allocation of Heap. That's why I need to spawn a process from within this test to get a handle on stdout, which (as per my limited knowledge) is not possible if I use @run. Please correct me if I am wrong.

52       "-Xmx32m -Xms32m -XX:+UseCompressedOops",     // 1. With compressedoops enabled.
54       "-Xmx32m -Xms32m -XX:HeapBaseMinAddress=3g",  // 3. With user specified HeapBaseMinAddress.
55       "-Xmx4g -Xms4g",                              // 4. With larger heap size (UnscaledNarrowOop not possible).
- I think these 3 sub-tests are testing different compressed oop modes. I would recommend to include other 1 type(non-zero based) as well. In addition, adding the comment also would help increase the readability.

[Kharbas, Kishor] Yes I can do that. I need to specify heap size close to 32 GB right? There should be that much disk space available in the test environment, do you think that would be problem?

--------------------------------------
test/hotspot/jtreg/gc/stress/gcbasher/TestGCBasherWithAllocateHeapAt.java

1) This seems identical to TestGCBasherWithG1.java, how about just adding another '@run'?
i.e.  adding "* @run main/othervm/timeout=500 -Xlog:gc*=info -Xmx256m -server -XX:+UseG1GC  -XX:AllocateHeapAt=. TestGCBasherWithG1 120000"

2) Don't we need testing for other GC types as well? i.e. Serial, Parallel and CMS.

[Kharbas, Kishor] The idea was to exercise the Java heap (allocated on a file) by some stress test. I thought one GC option would suffice, do you think we need more?

2  * Copyright (c) 2016, 2017, Oracle and/or its affiliates. All rights reserved.
- Is this intended to start from 2016 as this seems to be copied from TestGCBasherWithXXX.java?

[Kharbas, Kishor] I will change this.

34  * @run main/othervm/timeout=500 -Xlog:gc*=info -Xmx256m -server -XX:+UseG1GC -XX:AllocateHeapAt=. TestGCBasherWithAllocateHeapAt 120000
- Are there any reason to use timeout of 500? TestGCBasherWithG1 is using 200ms.
[Kharbas, Kishor] I increased the timeout since heap is mapped to a file on disk (either HDD or SSD depending on test environment) which makes the test run slower.

Thanks,
Sangheon




JEP: https://bugs.openjdk.java.net/browse/JDK-8171181
Implementation webrev : http://cr.openjdk.java.net/~kkharbas/8190308/webrev.15/<http://cr.openjdk.java.net/%7Ekkharbas/8190308/webrev.15/>

Thank you!
Kishor

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20171117/37db5f72/attachment.htm>

From sangheon.kim at oracle.com  Fri Nov 17 23:41:16 2017
From: sangheon.kim at oracle.com (sangheon.kim)
Date: Fri, 17 Nov 2017 15:41:16 -0800
Subject: RFR(M): 8190828: Test plan: JEP 8171181: Support heap allocation
 on alternative memory devices
In-Reply-To: <F89640DCD01A85489FCBA68183A6A0F3BCD56806@ORSMSX116.amr.corp.intel.com>
References: <F89640DCD01A85489FCBA68183A6A0F3BCD54D7C@ORSMSX116.amr.corp.intel.com>
 <08b72b19-9624-8bee-aab4-b456f3b89497@oracle.com>
 <F89640DCD01A85489FCBA68183A6A0F3BCD56806@ORSMSX116.amr.corp.intel.com>
Message-ID: <b20f67ff-b713-6599-d718-cf11415634ff@oracle.com>

Hi Kishor,


On 11/17/2017 02:59 PM, Kharbas, Kishor wrote:
>
> Hi Sangheon!
>
> Thanks for the reply. Please find my reply inline.
>
> *From:*sangheon.kim [mailto:sangheon.kim at oracle.com]
> *Sent:* Thursday, November 16, 2017 10:28 PM
> *To:* Kharbas, Kishor <kishor.kharbas at intel.com>; 
> hotspot-gc-dev at openjdk.java.net
> *Subject:* Re: RFR(M): 8190828: Test plan: JEP 8171181: Support heap 
> allocation on alternative memory devices
>
> Hi Kishor,
>
> On 11/13/2017 03:51 PM, Kharbas, Kishor wrote:
>
>     Hi!
>
>     I have developed a test plan for the implementation of 8171181.
>
>     I would appreciate a review and further guidance from the gc-dev
>     members. I am hoping to get everything done well before 18.3 code
>     freeze (have a vacation planned during that time).
>
>     Test plan: https://bugs.openjdk.java.net/browse/JDK-8190828
>     <https://bugs.openjdk.java.net/browse/JDK-8190828>
>
>     Test webrev:
>     http://cr.openjdk.java.net/~kkharbas/8190980/webrev.01/
>     <http://cr.openjdk.java.net/%7Ekkharbas/8190980/webrev.01/>
>
> Looking at the comment at 8190980, webrev.2 seems the latest one, so 
> my comments are for the webrev.2.
>
> --------------------------------------
> test/hotspot/jtreg/gc/TestAllocateHeapAtMultiple.java
>
> ? 51???? String[] extraOptsList = new String[] {
> ? 52?????? "-Xmx32m -Xms32m -XX:+UseCompressedOops", // 1. With 
> compressedoops enabled.
> ? 53?????? "-Xmx32m -Xms32m -XX:-UseCompressedOops", // 2. With 
> compressedoops disabled.
> ? 54?????? "-Xmx32m -Xms32m -XX:HeapBaseMinAddress=3g", // 3. With 
> user specified HeapBaseMinAddress.
> ? 55?????? "-Xmx4g -Xms4g", // 4. With larger heap size 
> (UnscaledNarrowOop not possible).
> ? 56?????? "-Xmx4g -Xms4g -XX:+UseLargePages", // 5. Set UseLargePages.
> ? 57?????? "-Xmx4g -Xms4g -XX:+UseNUMA" // 6. Set UseNUMA.
> ? 58???? };
> - I think we do differently to run sub-tests. Maybe SQE folks would 
> give better comment on this.
> e.g. TestAllocationInEden.java
> ?* @run main/othervm -Xbootclasspath/a:. -XX:+UnlockDiagnosticVMOptions
> ...
> ?*?????????????????? TestAllocationInEden 10m 9 EDEN
> ?* @run main/othervm -Xbootclasspath/a:. -XX:+UnlockDiagnosticVMOptions
> ...
> ?*?????????????????? TestAllocationInEden 10m 47 EDEN
> ?* @run main/othervm -Xbootclasspath/a:. -XX:+UnlockDiagnosticVMOptions
>
> */[Kharbas, Kishor] In my tests I use stdout to check for log 
> generated by ?Xlog:heap+gc=info to test correct allocation of Heap. 
> That?s why I need to spawn a process from within this test to get a 
> handle on stdout, which (as per my limited knowledge) is not possible 
> if I use @run. Please correct me if I am wrong./*
>
Please keep as is.
I think handling stdout seems okay with @run approach but @run doesn't 
allow to add more vm options. In this test, we need 'test_dir', so we 
can't use @run.


> *//*
>
>
> 52?????? "-Xmx32m -Xms32m -XX:+UseCompressedOops",???? // 1. With 
> compressedoops enabled.
> 54?????? "-Xmx32m -Xms32m -XX:HeapBaseMinAddress=3g",? // 3. With user 
> specified HeapBaseMinAddress.
> 55?????? "-Xmx4g -Xms4g",????????????????????????????? // 4. With 
> larger heap size (UnscaledNarrowOop not possible).
> - I think these 3 sub-tests are testing different compressed oop 
> modes. I would recommend to include other 1 type(non-zero based) as 
> well. In addition, adding the comment also would help increase the 
> readability.
>
> */[Kharbas, Kishor] Yes I can do that. I need to specify heap size 
> close to 32 GB right? There should be that much disk space available 
> in the test environment, do you think that would be problem?/*
>
No need so huge heap. Please refer UseComporessedOops.java for an 
example of each cases.
universe.hpp:378 also has good explanation.
e.g. -Xmx32m -XX:HeapBaseMinAddress=0x800000008 also uses non-zero based.

/*
*/
>
> *//*
>
>
> --------------------------------------
> test/hotspot/jtreg/gc/stress/gcbasher/TestGCBasherWithAllocateHeapAt.java
>
> 1) This seems identical to TestGCBasherWithG1.java, how about just 
> adding another '@run'?
> i.e.? adding "* @run main/othervm/timeout=500 -Xlog:gc*=info -Xmx256m 
> -server -XX:+UseG1GC *_-XX:AllocateHeapAt=._* TestGCBasherWithG1 120000"
>
> 2) Don't we need testing for other GC types as well? i.e. Serial, 
> Parallel and CMS.
>
> */[Kharbas, Kishor] The idea was to exercise the Java heap (allocated 
> on a file) by some stress test. I thought one GC option would suffice, 
> do you think we need more?/*
>
>
Let's leave as is.


> 2? * Copyright (c) 2016, 2017, Oracle and/or its affiliates. All 
> rights reserved.
> - Is this intended to start from 2016 as this seems to be copied from 
> TestGCBasherWithXXX.java?
>
> */[Kharbas, Kishor] I will change this./*
>
OK.
But, if you agree to go 1) option(adding @run, instead of making 
TestGCBasherWithAllocateHeapAt.java, this comment can be ignored./*


*/
>
> *//*
>
>
> 34? * @run main/othervm/timeout=500 -Xlog:gc*=info -Xmx256m -server 
> -XX:+UseG1GC -XX:AllocateHeapAt=. TestGCBasherWithAllocateHeapAt 120000
> - Are there any reason to use timeout of 500? TestGCBasherWithG1 is 
> using 200ms.
>
> */[Kharbas, Kishor] I increased the timeout since heap is mapped to a 
> file on disk (either HDD or SSD depending on test environment) which 
> makes the test run slower./*
>
Make sense to have longer timeout, not sure additional 300ms is good 
enough though.

At least when I ran your 8190308(webrev.15) with this webrev.2, all 
tier1~tier5 tests were okay. There were some failures but those are 
known issues.

Thanks,
Sangheon


>
> Thanks,
> Sangheon
>
>
>
>
>     JEP: https://bugs.openjdk.java.net/browse/JDK-8171181
>     <https://bugs.openjdk.java.net/browse/JDK-8171181>
>
>     Implementation webrev :
>     http://cr.openjdk.java.net/~kkharbas/8190308/webrev.15/
>     <http://cr.openjdk.java.net/%7Ekkharbas/8190308/webrev.15/>
>
>     Thank you!
>
>     Kishor
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20171117/a0da4348/attachment.htm>

From kishor.kharbas at intel.com  Sat Nov 18 00:52:33 2017
From: kishor.kharbas at intel.com (Kharbas, Kishor)
Date: Sat, 18 Nov 2017 00:52:33 +0000
Subject: RFR(M): 8190828: Test plan: JEP 8171181: Support heap
 allocation on alternative memory devices
In-Reply-To: <b20f67ff-b713-6599-d718-cf11415634ff@oracle.com>
References: <F89640DCD01A85489FCBA68183A6A0F3BCD54D7C@ORSMSX116.amr.corp.intel.com>
 <08b72b19-9624-8bee-aab4-b456f3b89497@oracle.com>
 <F89640DCD01A85489FCBA68183A6A0F3BCD56806@ORSMSX116.amr.corp.intel.com>
 <b20f67ff-b713-6599-d718-cf11415634ff@oracle.com>
Message-ID: <F89640DCD01A85489FCBA68183A6A0F3BCD569ED@ORSMSX116.amr.corp.intel.com>

Hi Sagheon,

I have new webrev at - http://cr.openjdk.java.net/~kkharbas/8190980/webrev.03
and my reply inline.

Thank you!
From: sangheon.kim [mailto:sangheon.kim at oracle.com]
Sent: Friday, November 17, 2017 3:41 PM
To: Kharbas, Kishor <kishor.kharbas at intel.com>; hotspot-gc-dev at openjdk.java.net
Subject: Re: RFR(M): 8190828: Test plan: JEP 8171181: Support heap allocation on alternative memory devices

Hi Kishor,

On 11/17/2017 02:59 PM, Kharbas, Kishor wrote:
Hi Sangheon!
Thanks for the reply. Please find my reply inline.

From: sangheon.kim [mailto:sangheon.kim at oracle.com]
Sent: Thursday, November 16, 2017 10:28 PM
To: Kharbas, Kishor <kishor.kharbas at intel.com><mailto:kishor.kharbas at intel.com>; hotspot-gc-dev at openjdk.java.net<mailto:hotspot-gc-dev at openjdk.java.net>
Subject: Re: RFR(M): 8190828: Test plan: JEP 8171181: Support heap allocation on alternative memory devices

Hi Kishor,
On 11/13/2017 03:51 PM, Kharbas, Kishor wrote:
Hi!

I have developed a test plan for the implementation of 8171181.
I would appreciate a review and further guidance from the gc-dev members. I am hoping to get everything done well before 18.3 code freeze (have a vacation planned during that time).

Test plan: https://bugs.openjdk.java.net/browse/JDK-8190828
Test webrev: http://cr.openjdk.java.net/~kkharbas/8190980/webrev.01/<http://cr.openjdk.java.net/%7Ekkharbas/8190980/webrev.01/>
Looking at the comment at 8190980, webrev.2 seems the latest one, so my comments are for the webrev.2.

--------------------------------------
test/hotspot/jtreg/gc/TestAllocateHeapAtMultiple.java

  51     String[] extraOptsList = new String[] {
  52       "-Xmx32m -Xms32m -XX:+UseCompressedOops",     // 1. With compressedoops enabled.
  53       "-Xmx32m -Xms32m -XX:-UseCompressedOops",     // 2. With compressedoops disabled.
  54       "-Xmx32m -Xms32m -XX:HeapBaseMinAddress=3g",  // 3. With user specified HeapBaseMinAddress.
  55       "-Xmx4g -Xms4g",                              // 4. With larger heap size (UnscaledNarrowOop not possible).
  56       "-Xmx4g -Xms4g -XX:+UseLargePages",           // 5. Set UseLargePages.
  57       "-Xmx4g -Xms4g -XX:+UseNUMA"                  // 6. Set UseNUMA.
  58     };
- I think we do differently to run sub-tests. Maybe SQE folks would give better comment on this.
e.g. TestAllocationInEden.java
 * @run main/othervm -Xbootclasspath/a:. -XX:+UnlockDiagnosticVMOptions
...
 *                   TestAllocationInEden 10m 9 EDEN
 * @run main/othervm -Xbootclasspath/a:. -XX:+UnlockDiagnosticVMOptions
...
 *                   TestAllocationInEden 10m 47 EDEN
 * @run main/othervm -Xbootclasspath/a:. -XX:+UnlockDiagnosticVMOptions


[Kharbas, Kishor] In my tests I use stdout to check for log generated by -Xlog:heap+gc=info to test correct allocation of Heap. That's why I need to spawn a process from within this test to get a handle on stdout, which (as per my limited knowledge) is not possible if I use @run. Please correct me if I am wrong.
Please keep as is.
I think handling stdout seems okay with @run approach but @run doesn't allow to add more vm options. In this test, we need 'test_dir', so we can't use @run.

[Kharbas, Kishor] Ok. Thanks




52       "-Xmx32m -Xms32m -XX:+UseCompressedOops",     // 1. With compressedoops enabled.
54       "-Xmx32m -Xms32m -XX:HeapBaseMinAddress=3g",  // 3. With user specified HeapBaseMinAddress.
55       "-Xmx4g -Xms4g",                              // 4. With larger heap size (UnscaledNarrowOop not possible).
- I think these 3 sub-tests are testing different compressed oop modes. I would recommend to include other 1 type(non-zero based) as well. In addition, adding the comment also would help increase the readability.


[Kharbas, Kishor] Yes I can do that. I need to specify heap size close to 32 GB right? There should be that much disk space available in the test environment, do you think that would be problem?
No need so huge heap. Please refer UseComporessedOops.java for an example of each cases.
universe.hpp:378 also has good explanation.
e.g. -Xmx32m -XX:HeapBaseMinAddress=0x800000008 also uses non-zero based.

[Kharbas, Kishor] I added this sub-test, used the options from UseCompressedOops.java for same non-zero base, unscaled case.

--------------------------------------
test/hotspot/jtreg/gc/stress/gcbasher/TestGCBasherWithAllocateHeapAt.java

1) This seems identical to TestGCBasherWithG1.java, how about just adding another '@run'?
i.e.  adding "* @run main/othervm/timeout=500 -Xlog:gc*=info -Xmx256m -server -XX:+UseG1GC  -XX:AllocateHeapAt=. TestGCBasherWithG1 120000"

2) Don't we need testing for other GC types as well? i.e. Serial, Parallel and CMS.


[Kharbas, Kishor] The idea was to exercise the Java heap (allocated on a file) by some stress test. I thought one GC option would suffice, do you think we need more?

Let's leave as is.

[Kharbas, Kishor] Ok Thanks


2  * Copyright (c) 2016, 2017, Oracle and/or its affiliates. All rights reserved.
- Is this intended to start from 2016 as this seems to be copied from TestGCBasherWithXXX.java?


[Kharbas, Kishor] I will change this.
OK.
But, if you agree to go 1) option(adding @run, instead of making TestGCBasherWithAllocateHeapAt.java, this comment can be ignored.

[Kharbas, Kishor] I feel it's better to keep this separate so as to not give an impression that AllocateHeapAt is tied to G1GC. Change the copyright.


34  * @run main/othervm/timeout=500 -Xlog:gc*=info -Xmx256m -server -XX:+UseG1GC -XX:AllocateHeapAt=. TestGCBasherWithAllocateHeapAt 120000
- Are there any reason to use timeout of 500? TestGCBasherWithG1 is using 200ms.
[Kharbas, Kishor] I increased the timeout since heap is mapped to a file on disk (either HDD or SSD depending on test environment) which makes the test run slower.
Make sense to have longer timeout, not sure additional 300ms is good enough though.

[Kharbas, Kishor] I my environment even 200s did not fail, but I observed it taking longer. Whole or part of file is paged in memory so run time is not very bad.

At least when I ran your 8190308(webrev.15) with this webrev.2, all tier1~tier5 tests were okay. There were some failures but those are known issues.

Thanks,
Sangheon




Thanks,
Sangheon





JEP: https://bugs.openjdk.java.net/browse/JDK-8171181
Implementation webrev : http://cr.openjdk.java.net/~kkharbas/8190308/webrev.15/<http://cr.openjdk.java.net/%7Ekkharbas/8190308/webrev.15/>

Thank you!
Kishor


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20171118/4ef06486/attachment.htm>

From thomas.schatzl at oracle.com  Sat Nov 18 15:37:04 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Sat, 18 Nov 2017 16:37:04 +0100
Subject: OOM error caused by large array allocation in G1
In-Reply-To: <CAEyFhSdj4V4rRYVRnDRbg7Pw_-4_FLw4c4mbVVwXNr3q4VXOZw@mail.gmail.com>
References: <CAEyFhSdj4V4rRYVRnDRbg7Pw_-4_FLw4c4mbVVwXNr3q4VXOZw@mail.gmail.com>
Message-ID: <1511019424.3046.29.camel@oracle.com>

Hi,

On Sat, 2017-11-18 at 20:53 +0800, Lijie Xu wrote:
> Hi All,
> I recently encountered an OOM error in a Spark application using G1
> collector. This application launches multiple JVM instances to
> process the large data. Each JVM has 6.5GB heap size and uses G1
> collector. A JVM instance throws an OOM error during allocating a
> large (570MB) array. However, this JVM has about 3GB free heap space
> at that time. After analyzing the application logic, heap usage, and
> GC log, I guess the root cause may be the lack of consecutive space
> for holding this large array in G1. I want to know whether my guess
> is right ...

Very likely. This is a long-standing issue (actually I have once
investigated about it like 10 years ago on a different regional
collector), and given your findings it is very likely you are correct.
The issue also has an extra section in the tuning guide [0].

> ... and why G1 has this defect.

Nobody fixed it yet. :)

Reasons:
- workaround easy and typically "just works".
- no "real world" test setups where fixes could be tested available.
People tend to disappear after getting to know the workaround.
Unfortunately, Apache SPARK which is probably one of the more frequent
environmnet it happens with, but it still does not work on jdk9/10 and
soon 11 yet where development happens.
- it's not very interesting work for many. Not sure why, probably
because it involves implementing and evaluating longer term strategies
in the collector to minimize impact of fragmentation which is a complex
topic (at least if you are not satisfied with the last-ditch brute
force approach).
- there are more problematic issues to deal with that affect more
installations, have test setups, and no or no good workaround.

Actually I have been discussing this with colleagues just last week
again in context of work for students/interns. :)

If you want to look into this there are a bunch of CRs open that you
might want to start with (e.g. [1][2][3]) to get an idea of
possibilities - these CRs do not even mention the one brute force
solution other VMs probably apply in that situation: have the full gc
move large arrays too.

Feel free to start a discussion about this topic either here or
preferably in the hotspot-gc-dev mailing list.

> In the following sections, I will detail the JVM info, application,
> OOM phase, and heap usage. Any suggestions will be appreciated.

Simply either increase the heap size or increase region size via
-XX:HeapRegionSize. I think 16m regions will fix the issue in your case
without any other performance impact, and reduce the amount of
humongous objects significantly.

> [JVM info]
> java version "1.8.0_121"
> Oracle Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
> Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)

While it won't impact this issue, I recommend updating at least to the
latest 8u release. Not suggesting jdk 9 here because we know that SPARK
does not work there yet.

Thanks,
  Thomas

[0] https://docs.oracle.com/javase/9/gctuning/garbage-first-garbage-col
lector-tuning.htm#GUID-2428DA90-B93D-48E6-B336-A849ADF1C552
[1] https://bugs.openjdk.java.net/browse/JDK-8172713
[2] https://bugs.openjdk.java.net/browse/JDK-8038487
[3] https://bugs.openjdk.java.net/browse/JDK-8173627



From thomas.schatzl at oracle.com  Sat Nov 18 15:37:43 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Sat, 18 Nov 2017 16:37:43 +0100
Subject: OOM error caused by large array allocation in G1
In-Reply-To: <CAEyFhSdj4V4rRYVRnDRbg7Pw_-4_FLw4c4mbVVwXNr3q4VXOZw@mail.gmail.com>
References: <CAEyFhSdj4V4rRYVRnDRbg7Pw_-4_FLw4c4mbVVwXNr3q4VXOZw@mail.gmail.com>
Message-ID: <1511019463.3046.32.camel@oracle.com>

Hi,

On Sat, 2017-11-18 at 20:53 +0800, Lijie Xu wrote:
> Hi All,
> I recently encountered an OOM error in a Spark application using G1
> collector. This application launches multiple JVM instances to
> process the large data. Each JVM has 6.5GB heap size and uses G1
> collector. A JVM instance throws an OOM error during allocating a
> large (570MB) array. However, this JVM has about 3GB free heap space
> at that time. After analyzing the application logic, heap usage, and
> GC log, I guess the root cause may be the lack of consecutive space
> for holding this large array in G1. I want to know whether my guess
> is right ...

Very likely. This is a long-standing issue (actually I have once
investigated about it like 10 years ago on a different regional
collector), and given your findings it is very likely you are correct.
The issue also has an extra section in the tuning guide [0].

> ... and why G1 has this defect.

Nobody fixed it yet. :)

Reasons:
- workaround easy and typically "just works".
- no "real world" test setups where fixes could be tested available.
People tend to disappear after getting to know the workaround.
Unfortunately, Apache SPARK which is probably one of the more frequent
environmnet it happens with, but it still does not work on jdk9/10 and
soon 11 yet where development happens.
- it's not very interesting work for many. Not sure why, probably
because it involves implementing and evaluating longer term strategies
in the collector to minimize impact of fragmentation which is a complex
topic (at least if you are not satisfied with the last-ditch brute
force approach).
- there are more problematic issues to deal with that affect more
installations, have test setups, and no or no good workaround.

Actually I have been discussing this with colleagues just last week
again in context of work for students/interns. :)

If you want to look into this there are a bunch of CRs open that you
might want to start with (e.g. [1][2][3]) to get an idea of
possibilities - these CRs do not even mention the one brute force
solution other VMs probably apply in that situation: have the full gc
move large arrays too.

Feel free to start a discussion about this topic either here or
preferably in the hotspot-gc-dev mailing list.

> In the following sections, I will detail the JVM info, application,
> OOM phase, and heap usage. Any suggestions will be appreciated.

Simply either increase the heap size or increase region size via
-XX:HeapRegionSize. I think 16m regions will fix the issue in your case
without any other performance impact, and reduce the amount of
humongous objects significantly.

> [JVM info]
> java version "1.8.0_121"
> Oracle Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
> Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)

While it won't impact this issue, I recommend updating at least to the
latest 8u release. Not suggesting jdk 9 here because we know that SPARK
does not work there yet.

Thanks,
  Thomas

[0] https://docs.oracle.com/javase/9/gctuning/garbage-first-garbage-col
lector-tuning.htm#GUID-2428DA90-B93D-48E6-B336-A849ADF1C552
[1] https://bugs.openjdk.java.net/browse/JDK-8172713
[2] https://bugs.openjdk.java.net/browse/JDK-8038487
[3] https://bugs.openjdk.java.net/browse/JDK-8173627



From daniel.daugherty at oracle.com  Sun Nov 19 01:49:43 2017
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Sat, 18 Nov 2017 20:49:43 -0500
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <354f7ece-66bc-c065-9ee8-f55abcc1c23e@oracle.com>
References: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
 <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>
 <836b5c6c-5eef-9245-c16a-e71372093d9a@oracle.com>
 <82d0e473-a24f-9feb-82e9-7d04c4d7a91d@oracle.com>
 <354f7ece-66bc-c065-9ee8-f55abcc1c23e@oracle.com>
Message-ID: <0ff8df8e-e93c-02d2-cb97-b13b1ac0f53f@oracle.com>

Greetings,

Testing of the last round of changes revealed a hang in one of the new
TLH tests. Robbin has fixed the hang, updated the existing TLH test, and
added another TLH test for good measure.

Here is the updated full webrev:

http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/

Here is the updated delta webrev:

http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-delta/

Dan ran the bits thru the usual Mach5 tier[1-5] testing and there are
no unexpected failures.

We welcome comments, suggestions and feedback.

Dan, Erik and Robbin


On 11/15/17 3:32 PM, Daniel D. Daugherty wrote:
> Greetings,
>
> Robbin rebased the project last night/this morning to merge with Thread
> Local Handshakes (TLH) and also picked up additional changesets up thru:
>
>> Changeset: fa736014cf28
>> Author:??? cjplummer
>> Date:????? 2017-11-14 18:08 -0800
>> URL:http://hg.openjdk.java.net/jdk/hs/rev/fa736014cf28
>>
>> 8191049: Add alternate version of pns() that is callable from within 
>> hotspot source
>> Summary: added pns2() to debug.cpp
>> Reviewed-by: stuefe, gthornbr
>
> This is the first time we've rebased the project to bits that are this
> fresh (< 12 hours old at rebase time). We've done this because we think
> we're done with this project and are in the final review-change-retest
> cycle(s)... In other words, we're not planning on making any more major
> changes for this project... :-)
>
> *** Time for code reviewers to chime in on this thread! ***
>
> Here is the updated full webrev:
>
> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-full/
>
> Here's is the delta webrev from the 2017.11.10 rebase:
>
> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-delta/
>
> Dan has submitted the bits for the usual Mach5 tier[1-5] testing
> (and the new baseline also)...
>
> We're expecting this round to be a little noisier than usual because
> we did not rebase on a PIT snapshot so the new baseline has not been
> through Jesper's usual care-and-feeding of integration_blockers, etc.
>
> We welcome comments, suggestions and feedback.
>
> Dan, Erik and Robbin
>
>
> On 11/14/17 4:48 PM, Daniel D. Daugherty wrote:
>> Greetings,
>>
>> I rebased the project to the 2017.11.10 jdk/hs PIT snapshot.
>> (Yes, we're playing chase-the-repo...)
>>
>> Here is the updated full webrev:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-07-full/
>>
>> Unlike the previous rebase, there were no changes required to the
>> open code to get the rebased bits to build so there is no delta
>> webrev.
>>
>> These bits have been run through JPRT and I've submitted the usual
>> Mach5 tier[1-5] test run...
>>
>> We welcome comments, suggestions and feedback.
>>
>> Dan, Erik and Robbin
>>
>>
>> On 11/13/17 12:30 PM, Daniel D. Daugherty wrote:
>>> Greetings,
>>>
>>> I rebased the project to the 2017.10.26 jdk10/hs PIT snapshot.
>>>
>>> Here are the updated webrevs:
>>>
>>> Here's the mq comment for the change:
>>>
>>> ? Rebase to 2017.10.25 PIT snapshot.
>>>
>>> Here is the full webrev:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-full/
>>>
>>> And here is the delta webrev:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-delta/
>>>
>>> I ran the above bits throught Mach5 tier[1-5] testing over the holiday
>>> weekend. Didn't see any issues in a quick look. Going to take a closer
>>> look today.
>>>
>>> We welcome comments, suggestions and feedback.
>>>
>>> Dan, Erik and Robbin
>>>
>>>
>>> On 11/8/17 1:05 PM, Daniel D. Daugherty wrote:
>>>> Greetings,
>>>>
>>>> Resolving one of the code review comments (from both Stefan K and 
>>>> Coleen)
>>>> on jdk10-04-full required quite a few changes so it is being done as a
>>>> standalone patch and corresponding webrevs:
>>>>
>>>> Here's the mq comment for the change:
>>>>
>>>> ? stefank, coleenp CR - refactor most JavaThreadIterator usage to use
>>>> ??? JavaThreadIteratorWithHandle.
>>>>
>>>> Here is the full webrev:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-full/
>>>>
>>>> And here is the delta webrev:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-delta/
>>>>
>>>> We welcome comments, suggestions and feedback.
>>>>
>>>> Dan, Erik and Robbin
>>>>
>>>>
>>>> On 10/9/17 3:41 PM, Daniel D. Daugherty wrote:
>>>>> Greetings,
>>>>>
>>>>> We have a (eXtra Large) fix for the following bug:
>>>>>
>>>>> 8167108 inconsistent handling of SR_lock can lead to crashes
>>>>> https://bugs.openjdk.java.net/browse/JDK-8167108
>>>>>
>>>>> This fix adds a Safe Memory Reclamation (SMR) mechanism based on
>>>>> Hazard Pointers to manage JavaThread lifecycle.
>>>>>
>>>>> Here's a PDF for the internal wiki that we've been using to describe
>>>>> and track the work on this project:
>>>>>
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/SMR_and_JavaThread_Lifecycle-JDK10-04.pdf 
>>>>>
>>>>>
>>>>> Dan has noticed that the indenting is wrong in some of the code 
>>>>> quotes
>>>>> in the PDF that are not present in the internal wiki. We don't have a
>>>>> solution for that problem yet.
>>>>>
>>>>> Here's the webrev for current JDK10 version of this fix:
>>>>>
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-04-full
>>>>>
>>>>> This fix has been run through many rounds of JPRT and Mach5 tier[2-5]
>>>>> testing, additional stress testing on Dan's Solaris X64 server, and
>>>>> additional testing on Erik and Robbin's machines.
>>>>>
>>>>> We welcome comments, suggestions and feedback.
>>>>>
>>>>> Daniel Daugherty
>>>>> Erik Osterlund
>>>>> Robbin Ehn
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>



From david.holmes at oracle.com  Mon Nov 20 01:09:31 2017
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 20 Nov 2017 11:09:31 +1000
Subject: RFR: 8189171: Move GC argument processing into GC specific classes
In-Reply-To: <A8796C24-E60A-484F-B2CA-00C00F4AD908@oracle.com>
References: <e72385e3-88f1-9b05-8495-ab7f4c4cbd60@redhat.com>
 <ed0085ca-7384-05e7-4a2c-e2f048f64909@oracle.com>
 <5a7fecea-1a1b-16db-e366-4dd8d0c9ed63@redhat.com>
 <59E9E9A9.7020306@oracle.com>
 <511327b5-4632-0e27-6914-2a3d076a1dc3@redhat.com>
 <59EA002D.1050605@oracle.com>
 <2b37fed8-0383-eb19-3e45-076aabd74f22@redhat.com>
 <59F0AD40.3020307@oracle.com>
 <e8e953a1-06b3-648b-d5da-c87ee7c640ad@redhat.com>
 <B3E05F9C-5AC2-40D3-BBF3-A46ED9B12ED5@oracle.com>
 <affd801a-d184-1bcd-7c58-65c1ecaa94da@oracle.com>
 <3c75a6c5-ffe5-948f-b2c3-b4950a4b1a9f@redhat.com>
 <3f188247-aea1-55d7-cbd4-8ddcd33aa4a8@oracle.com>
 <c57f6ad3-9629-214a-cc41-34ac0c3688ad@redhat.com>
 <B11BD63B-6459-486A-92C3-2C6D045740AF@oracle.com>
 <411d8f0f-82e7-88e3-a1a4-12515b59452a@redhat.com>
 <A8796C24-E60A-484F-B2CA-00C00F4AD908@oracle.com>
Message-ID: <38c8c5c1-6231-76c2-0c33-7920101814cd@oracle.com>

On 17/11/2017 2:33 AM, Bob Vandette wrote:
> Yes, that?s the fix.  I built it correctly with that added include.

How? I can't build the MinimalVM in JPRT because the UNSUPPORTED_OPTION 
macro is only locally defined in arguments.cpp, but is now used in 
gcArguments.cpp:

/opt/jprt/T/P1/003543.daholme/s/open/src/hotspot/share/gc/shared/gcArguments.cpp:77:29: 
error: 'UNSUPPORTED_OPTION' was not declared in this scope
    UNSUPPORTED_OPTION(UseG1GC);
                              ^
make[3]: *** 
[/opt/jprt/T/P1/003543.daholme/s/build/linux-x86-debug/hotspot/variant-minimal/libjvm/objs/gcArguments.o] 
Error 1

---

Roman/Erik: If changing code that has #include guards you must ensure 
all the paths are tested!

Thanks,
David

> There are still a few other issues with building and using the minimal VM but those
> existed before your change.
> 
> 1. You cannot build the minimal VM without also building the server VM.  There
> is a Makefile dependency on the server VM.
> 2. The jvm.cfg produced in the jvm and jre images does not add the minimalvm
> as one of the possible VM to select.  As a work-around jlink produces a workable jvm.cfg.
> 
> Thanks,
> Bob.
> 
> 
>> On Nov 16, 2017, at 11:17 AM, Roman Kennke <rkennke at redhat.com> wrote:
>>
>> Hi Bob,
>> thanks for letting me know.
>>
>> I filed: https://bugs.openjdk.java.net/browse/JDK-8191424
>> and posted for review: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2017-November/020784.html
>>
>> Thanks, Roman
>>
>>> Roman,
>>>
>>> It looks like this change may have caused the build of the minimal VM to
>>> fail.  The minimal VM is no longer a configuration that we regularly build and test
>>> but it is still a build option that can be selected via "--with-jvm-variants=minimal1"
>>>
>>>
>>> /scratch/users/bobv/jdk10hs/open/src/hotspot/share/gc/shared/gcArguments.cpp: In static member function 'static jint GCArguments::initialize()':
>>> /scratch/users/bobv/jdk10hs/open/src/hotspot/share/gc/shared/gcArguments.cpp:113:17: error: 'defaultStream' has not been declared
>>>       jio_fprintf(defaultStream::error_stream(), "UseParallelGC not supported in this VM.\n");
>>>                   ^
>>> /scratch/users/bobv/jdk10hs/open/src/hotspot/share/gc/shared/gcArguments.cpp:116:17: error: 'defaultStream' has not been declared
>>>       jio_fprintf(defaultStream::error_stream(), "UseG1GC not supported in this VM.\n");
>>>                   ^
>>> /scratch/users/bobv/jdk10hs/open/src/hotspot/share/gc/shared/gcArguments.cpp:119:17: error: 'defaultStream' has not been declared
>>>       jio_fprintf(defaultStream::error_stream(), "UseConcMarkSweepGC not supported in this VM.\n");
>>>                   ^
>>> gmake[3]: *** [/scratch/users/bobv/jdk10hs/build/b00/linux-x64-minimal1/hotspot/variant-minimal/libjvm/objs/gcArguments.o] Error 1
>>>
>>> Bob.
>>>
>>>
>>>> On Nov 7, 2017, at 6:01 AM, Roman Kennke <rkennke at redhat.com> wrote:
>>>>
>>>> Hi Per & Erik,
>>>>
>>>> thanks for the reviews!
>>>>
>>>> Now I need a sponsor.
>>>>
>>>> Final webrev (same as before, including Reviewed-by line):
>>>> http://cr.openjdk.java.net/~rkennke/8189171/webrev.05/ <http://cr.openjdk.java.net/%7Erkennke/8189171/webrev.05/>
>>>>
>>>> Thanks, Roman
>>>>
>>>>
>>>>> Looks good Roman!
>>>>>
>>>>> /Per
>>>>>
>>>>> On 2017-11-06 12:13, Roman Kennke wrote:
>>>>>> Am 26.10.2017 um 11:43 schrieb Per Liden:
>>>>>>> Hi,
>>>>>>>
>>>>>>> On 2017-10-25 18:11, Erik Osterlund wrote:
>>>>>>> [...]
>>>>>>>>> So let me just put your changes up for review (again), if you don't
>>>>>>>>> mind:
>>>>>>>>>
>>>>>>>>> Full webrev:
>>>>>>>>> http://cr.openjdk.java.net/~eosterlund/8189171/webrev.03/
>>>>>>>>> <http://cr.openjdk.java.net/%7Eeosterlund/8189171/webrev.03/>
>>>>>>> I like this. Just two naming suggestions:
>>>>>>>
>>>>>>> src/hotspot/share/gc/shared/gcArguments.hpp:
>>>>>>>
>>>>>>>    39   static jint create_instance();
>>>>>>>    40   static bool is_initialized();
>>>>>>>    41   static GCArguments* instance();
>>>>>>>
>>>>>>> Could we call these:
>>>>>>>
>>>>>>>    39   static jint initialize();
>>>>>>>    40   static bool is_initialized();
>>>>>>>    41   static GCArguments* arguments();
>>>>>>>
>>>>>>> Reasoning: initialize() maps better to its companion is_initialized().
>>>>>>> GCArguments::arguments() maps better to the existing patterns we have
>>>>>>> with CollectedHeap::heap().
>>>>>> Ok, that sounds good to me. Actually, it's almost full-circle back to my
>>>>>> original proposal ;-)
>>>>>>
>>>>>> Differential:
>>>>>> http://cr.openjdk.java.net/~rkennke/8189171/webrev.04.diff/
>>>>>> <http://cr.openjdk.java.net/%7Erkennke/8189171/webrev.04.diff/>
>>>>>> Full:
>>>>>> http://cr.openjdk.java.net/~rkennke/8189171/webrev.04/
>>>>>> <http://cr.openjdk.java.net/%7Erkennke/8189171/webrev.04/>
>>>>>>
>>>>>> Ok to push now?
>>>>>>
>>>>>> Roman
>>>>
>>
> 


From david.holmes at oracle.com  Mon Nov 20 01:31:04 2017
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 20 Nov 2017 11:31:04 +1000
Subject: RFR: 8189171: Move GC argument processing into GC specific classes
In-Reply-To: <38c8c5c1-6231-76c2-0c33-7920101814cd@oracle.com>
References: <e72385e3-88f1-9b05-8495-ab7f4c4cbd60@redhat.com>
 <ed0085ca-7384-05e7-4a2c-e2f048f64909@oracle.com>
 <5a7fecea-1a1b-16db-e366-4dd8d0c9ed63@redhat.com>
 <59E9E9A9.7020306@oracle.com>
 <511327b5-4632-0e27-6914-2a3d076a1dc3@redhat.com>
 <59EA002D.1050605@oracle.com>
 <2b37fed8-0383-eb19-3e45-076aabd74f22@redhat.com>
 <59F0AD40.3020307@oracle.com>
 <e8e953a1-06b3-648b-d5da-c87ee7c640ad@redhat.com>
 <B3E05F9C-5AC2-40D3-BBF3-A46ED9B12ED5@oracle.com>
 <affd801a-d184-1bcd-7c58-65c1ecaa94da@oracle.com>
 <3c75a6c5-ffe5-948f-b2c3-b4950a4b1a9f@redhat.com>
 <3f188247-aea1-55d7-cbd4-8ddcd33aa4a8@oracle.com>
 <c57f6ad3-9629-214a-cc41-34ac0c3688ad@redhat.com>
 <B11BD63B-6459-486A-92C3-2C6D045740AF@oracle.com>
 <411d8f0f-82e7-88e3-a1a4-12515b59452a@redhat.com>
 <A8796C24-E60A-484F-B2CA-00C00F4AD908@oracle.com>
 <38c8c5c1-6231-76c2-0c33-7920101814cd@oracle.com>
Message-ID: <0944d8e4-1f00-8c3c-2c50-a598acd098ad@oracle.com>

Typo ...

On 20/11/2017 11:09 AM, David Holmes wrote:
> On 17/11/2017 2:33 AM, Bob Vandette wrote:
>> Yes, that?s the fix.? I built it correctly with that added include.
> 
> How? I can't build the MinimalVM in JPRT because the UNSUPPORTED_OPTION 
> macro is only locally defined in arguments.cpp, but is now used in 
> gcArguments.cpp:
> 
> /opt/jprt/T/P1/003543.daholme/s/open/src/hotspot/share/gc/shared/gcArguments.cpp:77:29: 
> error: 'UNSUPPORTED_OPTION' was not declared in this scope
>  ?? UNSUPPORTED_OPTION(UseG1GC);
>  ???????????????????????????? ^
> make[3]: *** 
> [/opt/jprt/T/P1/003543.daholme/s/build/linux-x86-debug/hotspot/variant-minimal/libjvm/objs/gcArguments.o] 
> Error 1
> 
> ---
> 
> Roman/Erik: If changing code that has #include guards you must ensure 
> all the paths are tested!

#include -> #ifdef

David

> Thanks,
> David
> 
>> There are still a few other issues with building and using the minimal 
>> VM but those
>> existed before your change.
>>
>> 1. You cannot build the minimal VM without also building the server 
>> VM.? There
>> is a Makefile dependency on the server VM.
>> 2. The jvm.cfg produced in the jvm and jre images does not add the 
>> minimalvm
>> as one of the possible VM to select.? As a work-around jlink produces 
>> a workable jvm.cfg.
>>
>> Thanks,
>> Bob.
>>
>>
>>> On Nov 16, 2017, at 11:17 AM, Roman Kennke <rkennke at redhat.com> wrote:
>>>
>>> Hi Bob,
>>> thanks for letting me know.
>>>
>>> I filed: https://bugs.openjdk.java.net/browse/JDK-8191424
>>> and posted for review: 
>>> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2017-November/020784.html 
>>>
>>>
>>> Thanks, Roman
>>>
>>>> Roman,
>>>>
>>>> It looks like this change may have caused the build of the minimal 
>>>> VM to
>>>> fail.? The minimal VM is no longer a configuration that we regularly 
>>>> build and test
>>>> but it is still a build option that can be selected via 
>>>> "--with-jvm-variants=minimal1"
>>>>
>>>>
>>>> /scratch/users/bobv/jdk10hs/open/src/hotspot/share/gc/shared/gcArguments.cpp: 
>>>> In static member function 'static jint GCArguments::initialize()':
>>>> /scratch/users/bobv/jdk10hs/open/src/hotspot/share/gc/shared/gcArguments.cpp:113:17: 
>>>> error: 'defaultStream' has not been declared
>>>> ????? jio_fprintf(defaultStream::error_stream(), "UseParallelGC not 
>>>> supported in this VM.\n");
>>>> ????????????????? ^
>>>> /scratch/users/bobv/jdk10hs/open/src/hotspot/share/gc/shared/gcArguments.cpp:116:17: 
>>>> error: 'defaultStream' has not been declared
>>>> ????? jio_fprintf(defaultStream::error_stream(), "UseG1GC not 
>>>> supported in this VM.\n");
>>>> ????????????????? ^
>>>> /scratch/users/bobv/jdk10hs/open/src/hotspot/share/gc/shared/gcArguments.cpp:119:17: 
>>>> error: 'defaultStream' has not been declared
>>>> ????? jio_fprintf(defaultStream::error_stream(), "UseConcMarkSweepGC 
>>>> not supported in this VM.\n");
>>>> ????????????????? ^
>>>> gmake[3]: *** 
>>>> [/scratch/users/bobv/jdk10hs/build/b00/linux-x64-minimal1/hotspot/variant-minimal/libjvm/objs/gcArguments.o] 
>>>> Error 1
>>>>
>>>> Bob.
>>>>
>>>>
>>>>> On Nov 7, 2017, at 6:01 AM, Roman Kennke <rkennke at redhat.com> wrote:
>>>>>
>>>>> Hi Per & Erik,
>>>>>
>>>>> thanks for the reviews!
>>>>>
>>>>> Now I need a sponsor.
>>>>>
>>>>> Final webrev (same as before, including Reviewed-by line):
>>>>> http://cr.openjdk.java.net/~rkennke/8189171/webrev.05/ 
>>>>> <http://cr.openjdk.java.net/%7Erkennke/8189171/webrev.05/>
>>>>>
>>>>> Thanks, Roman
>>>>>
>>>>>
>>>>>> Looks good Roman!
>>>>>>
>>>>>> /Per
>>>>>>
>>>>>> On 2017-11-06 12:13, Roman Kennke wrote:
>>>>>>> Am 26.10.2017 um 11:43 schrieb Per Liden:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> On 2017-10-25 18:11, Erik Osterlund wrote:
>>>>>>>> [...]
>>>>>>>>>> So let me just put your changes up for review (again), if you 
>>>>>>>>>> don't
>>>>>>>>>> mind:
>>>>>>>>>>
>>>>>>>>>> Full webrev:
>>>>>>>>>> http://cr.openjdk.java.net/~eosterlund/8189171/webrev.03/
>>>>>>>>>> <http://cr.openjdk.java.net/%7Eeosterlund/8189171/webrev.03/>
>>>>>>>> I like this. Just two naming suggestions:
>>>>>>>>
>>>>>>>> src/hotspot/share/gc/shared/gcArguments.hpp:
>>>>>>>>
>>>>>>>> ?? 39?? static jint create_instance();
>>>>>>>> ?? 40?? static bool is_initialized();
>>>>>>>> ?? 41?? static GCArguments* instance();
>>>>>>>>
>>>>>>>> Could we call these:
>>>>>>>>
>>>>>>>> ?? 39?? static jint initialize();
>>>>>>>> ?? 40?? static bool is_initialized();
>>>>>>>> ?? 41?? static GCArguments* arguments();
>>>>>>>>
>>>>>>>> Reasoning: initialize() maps better to its companion 
>>>>>>>> is_initialized().
>>>>>>>> GCArguments::arguments() maps better to the existing patterns we 
>>>>>>>> have
>>>>>>>> with CollectedHeap::heap().
>>>>>>> Ok, that sounds good to me. Actually, it's almost full-circle 
>>>>>>> back to my
>>>>>>> original proposal ;-)
>>>>>>>
>>>>>>> Differential:
>>>>>>> http://cr.openjdk.java.net/~rkennke/8189171/webrev.04.diff/
>>>>>>> <http://cr.openjdk.java.net/%7Erkennke/8189171/webrev.04.diff/>
>>>>>>> Full:
>>>>>>> http://cr.openjdk.java.net/~rkennke/8189171/webrev.04/
>>>>>>> <http://cr.openjdk.java.net/%7Erkennke/8189171/webrev.04/>
>>>>>>>
>>>>>>> Ok to push now?
>>>>>>>
>>>>>>> Roman
>>>>>
>>>
>>


From david.holmes at oracle.com  Mon Nov 20 05:51:06 2017
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 20 Nov 2017 15:51:06 +1000
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <0ff8df8e-e93c-02d2-cb97-b13b1ac0f53f@oracle.com>
References: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
 <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>
 <836b5c6c-5eef-9245-c16a-e71372093d9a@oracle.com>
 <82d0e473-a24f-9feb-82e9-7d04c4d7a91d@oracle.com>
 <354f7ece-66bc-c065-9ee8-f55abcc1c23e@oracle.com>
 <0ff8df8e-e93c-02d2-cb97-b13b1ac0f53f@oracle.com>
Message-ID: <43d2f342-3f4b-5c1a-a687-c24c4efedba9@oracle.com>

Hi Dan,

Figured I'd better cast an eye over this again before it gets pushed :)

Only one thing (repeated many times) jumped out at me and apologies if 
this has already been debated back and forth. I missed the exchange 
where the for loop iteration was rewritten into this unusual form:

for (JavaThreadIteratorWithHandle jtiwh; JavaThread *jt = jtiwh.next(); ) {

On first reading I expect most people would mistake the control 
expression for the iteration-expression due to the next(). I didn't even 
know the control expression could introduce a local variable! I find 
this really hard to read stylistically and far too cute/clever for its 
own good. It also violates the style rules regarding implicit boolean 
checks.

I know Stefan proposed this as he did not like the alternate (in a few 
places):

+  {
+    ThreadsListHandle tlh;
+    JavaThreadIterator jti(tlh.list());
+    for (JavaThread* t = jti.first(); t != NULL; t = jti.next()) {
...
+  }

but it seems to me the iterator code has evolved since then and only a 
couple of places need a distinct TLH separate from the actual iterator 
object. So I'm more in favour of the more classic for-loop, with the 
iterator declared before the loop. Or even convert to while loops, as 
this iterator seems more suited to that.

Other than that just a couple of comments on variable type choices, and 
a few typos spotted below.

Thanks,
David
---

src/hotspot/share/runtime/globals.hpp

2476   diagnostic(bool, EnableThreadSMRExtraValidityChecks, true, 
         \
2477           "Enable Thread SMR extra validity checks") 
         \
2478 
         \
2479   diagnostic(bool, EnableThreadSMRStatistics, true, 
         \
2480           "Enable Thread SMR Statistics") 
         \

Indent of strings is off by  3 spaces.

---

src/hotspot/share/runtime/handshake.cpp

  140       // There is assumption in code that Threads_lock should be lock
  200       // There is assumption in code that Threads_lock should be lock

lock -> locked

  146         // handshake_process_by_vmthread will check the semaphore 
for us again

Needs period at end.

  148         // should be okey since the new thread will not have an 
operation.

okey -> okay

  151         // We can't warn here is since the thread do 
cancel_handshake after it have been removed

I think:

// We can't warn here since the thread does cancel_handshake after it 
has been removed

  152         // from ThreadsList. So we should just keep looping here 
until while below return negative.

from -> from the

Needs period at end.

  204             // A new thread on the ThreadsList will not have an 
operation.

Change period to comma, and ...

  205             // Hence is skipped in handshake_process_by_vmthread.

Hence is -> hence it is

---

src/hotspot/share/runtime/thread.cpp

1482     // dcubed - Looks like the Threads_lock is for stable access
1483     // to _jvmci_old_thread_counters and _jvmci_counters.

Does this comment need revisiting?

3451 volatile jint ...

Why are you using jint for field types here? (Surprised Coleen hasn't 
spotted them! ;-) ).

3456 long                  Threads::_smr_java_thread_list_alloc_cnt = 1;
3457 long                  Threads::_smr_java_thread_list_free_cnt = 0;

long? If you really want 64-bit counters here you won't get them on 
Windows with that declaration. If you really want variable sized 
counters I suggest uintptr_t; otherwise uint64_t.

----

On 19/11/2017 11:49 AM, Daniel D. Daugherty wrote:
> Greetings,
> 
> Testing of the last round of changes revealed a hang in one of the new
> TLH tests. Robbin has fixed the hang, updated the existing TLH test, and
> added another TLH test for good measure.
> 
> Here is the updated full webrev:
> 
> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/
> 
> Here is the updated delta webrev:
> 
> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-delta/
> 
> Dan ran the bits thru the usual Mach5 tier[1-5] testing and there are
> no unexpected failures.
> 
> We welcome comments, suggestions and feedback.
> 
> Dan, Erik and Robbin
> 
> 
> On 11/15/17 3:32 PM, Daniel D. Daugherty wrote:
>> Greetings,
>>
>> Robbin rebased the project last night/this morning to merge with Thread
>> Local Handshakes (TLH) and also picked up additional changesets up thru:
>>
>>> Changeset: fa736014cf28
>>> Author:??? cjplummer
>>> Date:????? 2017-11-14 18:08 -0800
>>> URL:http://hg.openjdk.java.net/jdk/hs/rev/fa736014cf28
>>>
>>> 8191049: Add alternate version of pns() that is callable from within 
>>> hotspot source
>>> Summary: added pns2() to debug.cpp
>>> Reviewed-by: stuefe, gthornbr
>>
>> This is the first time we've rebased the project to bits that are this
>> fresh (< 12 hours old at rebase time). We've done this because we think
>> we're done with this project and are in the final review-change-retest
>> cycle(s)... In other words, we're not planning on making any more major
>> changes for this project... :-)
>>
>> *** Time for code reviewers to chime in on this thread! ***
>>
>> Here is the updated full webrev:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-full/
>>
>> Here's is the delta webrev from the 2017.11.10 rebase:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-delta/
>>
>> Dan has submitted the bits for the usual Mach5 tier[1-5] testing
>> (and the new baseline also)...
>>
>> We're expecting this round to be a little noisier than usual because
>> we did not rebase on a PIT snapshot so the new baseline has not been
>> through Jesper's usual care-and-feeding of integration_blockers, etc.
>>
>> We welcome comments, suggestions and feedback.
>>
>> Dan, Erik and Robbin
>>
>>
>> On 11/14/17 4:48 PM, Daniel D. Daugherty wrote:
>>> Greetings,
>>>
>>> I rebased the project to the 2017.11.10 jdk/hs PIT snapshot.
>>> (Yes, we're playing chase-the-repo...)
>>>
>>> Here is the updated full webrev:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-07-full/
>>>
>>> Unlike the previous rebase, there were no changes required to the
>>> open code to get the rebased bits to build so there is no delta
>>> webrev.
>>>
>>> These bits have been run through JPRT and I've submitted the usual
>>> Mach5 tier[1-5] test run...
>>>
>>> We welcome comments, suggestions and feedback.
>>>
>>> Dan, Erik and Robbin
>>>
>>>
>>> On 11/13/17 12:30 PM, Daniel D. Daugherty wrote:
>>>> Greetings,
>>>>
>>>> I rebased the project to the 2017.10.26 jdk10/hs PIT snapshot.
>>>>
>>>> Here are the updated webrevs:
>>>>
>>>> Here's the mq comment for the change:
>>>>
>>>> ? Rebase to 2017.10.25 PIT snapshot.
>>>>
>>>> Here is the full webrev:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-full/
>>>>
>>>> And here is the delta webrev:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-delta/
>>>>
>>>> I ran the above bits throught Mach5 tier[1-5] testing over the holiday
>>>> weekend. Didn't see any issues in a quick look. Going to take a closer
>>>> look today.
>>>>
>>>> We welcome comments, suggestions and feedback.
>>>>
>>>> Dan, Erik and Robbin
>>>>
>>>>
>>>> On 11/8/17 1:05 PM, Daniel D. Daugherty wrote:
>>>>> Greetings,
>>>>>
>>>>> Resolving one of the code review comments (from both Stefan K and 
>>>>> Coleen)
>>>>> on jdk10-04-full required quite a few changes so it is being done as a
>>>>> standalone patch and corresponding webrevs:
>>>>>
>>>>> Here's the mq comment for the change:
>>>>>
>>>>> ? stefank, coleenp CR - refactor most JavaThreadIterator usage to use
>>>>> ??? JavaThreadIteratorWithHandle.
>>>>>
>>>>> Here is the full webrev:
>>>>>
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-full/
>>>>>
>>>>> And here is the delta webrev:
>>>>>
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-delta/
>>>>>
>>>>> We welcome comments, suggestions and feedback.
>>>>>
>>>>> Dan, Erik and Robbin
>>>>>
>>>>>
>>>>> On 10/9/17 3:41 PM, Daniel D. Daugherty wrote:
>>>>>> Greetings,
>>>>>>
>>>>>> We have a (eXtra Large) fix for the following bug:
>>>>>>
>>>>>> 8167108 inconsistent handling of SR_lock can lead to crashes
>>>>>> https://bugs.openjdk.java.net/browse/JDK-8167108
>>>>>>
>>>>>> This fix adds a Safe Memory Reclamation (SMR) mechanism based on
>>>>>> Hazard Pointers to manage JavaThread lifecycle.
>>>>>>
>>>>>> Here's a PDF for the internal wiki that we've been using to describe
>>>>>> and track the work on this project:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/SMR_and_JavaThread_Lifecycle-JDK10-04.pdf 
>>>>>>
>>>>>>
>>>>>> Dan has noticed that the indenting is wrong in some of the code 
>>>>>> quotes
>>>>>> in the PDF that are not present in the internal wiki. We don't have a
>>>>>> solution for that problem yet.
>>>>>>
>>>>>> Here's the webrev for current JDK10 version of this fix:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-04-full
>>>>>>
>>>>>> This fix has been run through many rounds of JPRT and Mach5 tier[2-5]
>>>>>> testing, additional stress testing on Dan's Solaris X64 server, and
>>>>>> additional testing on Erik and Robbin's machines.
>>>>>>
>>>>>> We welcome comments, suggestions and feedback.
>>>>>>
>>>>>> Daniel Daugherty
>>>>>> Erik Osterlund
>>>>>> Robbin Ehn
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
> 


From robin.westberg at oracle.com  Mon Nov 20 09:48:15 2017
From: robin.westberg at oracle.com (Robin Westberg)
Date: Mon, 20 Nov 2017 10:48:15 +0100
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <0ff8df8e-e93c-02d2-cb97-b13b1ac0f53f@oracle.com>
References: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
 <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>
 <836b5c6c-5eef-9245-c16a-e71372093d9a@oracle.com>
 <82d0e473-a24f-9feb-82e9-7d04c4d7a91d@oracle.com>
 <354f7ece-66bc-c065-9ee8-f55abcc1c23e@oracle.com>
 <0ff8df8e-e93c-02d2-cb97-b13b1ac0f53f@oracle.com>
Message-ID: <D56C03FB-4A8B-48ED-9CBD-5F22C7C05F91@oracle.com>

Hi Dan,

Overall I must say this looks very nice, can?t wait to use it! (Also a disclaimer: not a reviewer, and have not looked at the gc or jmvti specific changes nor the tests (yet)). 

Didn?t spot any real issues, just a few small efficiency related things:

src/hotspot/share/runtime/biasedLocking.cpp

217     for (JavaThread* cur_thread = Threads::first(); cur_thread != NULL; cur_thread = cur_thread->next()) {
218       if (cur_thread == biased_thread) {
219         thread_is_alive = true;

This loop could be replaced with:

ThreadsListHandle tlh;
thread_is_alive = tlh.includes(biased_thread);


src/hotspot/share/runtime/memprofiler.cpp

55-56 grabs the Threads_lock, shouldn?t be needed anymore.


src/hotspot/share/runtime/thread.inline.hpp

198   TerminatedTypes l_terminated = (TerminatedTypes)
199       OrderAccess::load_acquire((volatile jint *) &_terminated);
200   return check_is_terminated(_terminated);

The variable used in the return statement was intended to be l_terminated I guess? 


The following are more minor suggestions / comments:

src/hotspot/share/runtime/thread.cpp

3432 // operations over all threads.  It is protected by its own Mutex
3433 // lock, which is also used in other contexts to protect thread

Should this comment perhaps be revised to mention SMR?

4745   hash_table_size--;
4746   hash_table_size |= hash_table_size >> 1;
...

This calculation is repeated around line 4922 as well, perhaps put it in a function?

4828   // If someone set a handshake on us just as we entered exit path, we simple cancel it.

Perhaps something like ?.. entered the exit path, we simply cancel it.?


src/hotspot/share/runtime/thread.hpp

1179   bool on_thread_list() { return _on_thread_list; }
1180   void set_on_thread_list() { _on_thread_list = true; }
1181 
1182   // thread has called JavaThread::exit() or is terminated
1183   bool is_exiting() const;
1184   // thread is terminated (no longer on the threads list); we compare
1185   // against the two non-terminated values so that a freed JavaThread
1186   // will also be considered terminated.
1187   bool check_is_terminated(TerminatedTypes l_terminated) const {
1188     return l_terminated != _not_terminated && l_terminated != _thread_exiting;
1189   }
1190   bool is_terminated();

Use of const is inconsistent here, it?s added to 1183, should it perhaps be added to 1179 and 1190 as well then?


src/hotspot/share/runtime/vm_operations.hpp

406   DeadlockCycle* result()               { return _deadlocks; };
407   VMOp_Type type() const                { return VMOp_FindDeadlocks; }

Whitespace only change that seems spurious.

Best regards,
Robin

> On 19 Nov 2017, at 02:49, Daniel D. Daugherty <daniel.daugherty at oracle.com> wrote:
> 
> Greetings,
> 
> Testing of the last round of changes revealed a hang in one of the new
> TLH tests. Robbin has fixed the hang, updated the existing TLH test, and
> added another TLH test for good measure.
> 
> Here is the updated full webrev:
> 
> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/
> 
> Here is the updated delta webrev:
> 
> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-delta/
> 
> Dan ran the bits thru the usual Mach5 tier[1-5] testing and there are
> no unexpected failures.
> 
> We welcome comments, suggestions and feedback.
> 
> Dan, Erik and Robbin
> 
> 
> On 11/15/17 3:32 PM, Daniel D. Daugherty wrote:
>> Greetings,
>> 
>> Robbin rebased the project last night/this morning to merge with Thread
>> Local Handshakes (TLH) and also picked up additional changesets up thru:
>> 
>>> Changeset: fa736014cf28
>>> Author:    cjplummer
>>> Date:      2017-11-14 18:08 -0800
>>> URL:http://hg.openjdk.java.net/jdk/hs/rev/fa736014cf28
>>> 
>>> 8191049: Add alternate version of pns() that is callable from within hotspot source
>>> Summary: added pns2() to debug.cpp
>>> Reviewed-by: stuefe, gthornbr
>> 
>> This is the first time we've rebased the project to bits that are this
>> fresh (< 12 hours old at rebase time). We've done this because we think
>> we're done with this project and are in the final review-change-retest
>> cycle(s)... In other words, we're not planning on making any more major
>> changes for this project... :-)
>> 
>> *** Time for code reviewers to chime in on this thread! ***
>> 
>> Here is the updated full webrev:
>> 
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-full/
>> 
>> Here's is the delta webrev from the 2017.11.10 rebase:
>> 
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-delta/
>> 
>> Dan has submitted the bits for the usual Mach5 tier[1-5] testing
>> (and the new baseline also)...
>> 
>> We're expecting this round to be a little noisier than usual because
>> we did not rebase on a PIT snapshot so the new baseline has not been
>> through Jesper's usual care-and-feeding of integration_blockers, etc.
>> 
>> We welcome comments, suggestions and feedback.
>> 
>> Dan, Erik and Robbin
>> 
>> 
>> On 11/14/17 4:48 PM, Daniel D. Daugherty wrote:
>>> Greetings,
>>> 
>>> I rebased the project to the 2017.11.10 jdk/hs PIT snapshot.
>>> (Yes, we're playing chase-the-repo...)
>>> 
>>> Here is the updated full webrev:
>>> 
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-07-full/
>>> 
>>> Unlike the previous rebase, there were no changes required to the
>>> open code to get the rebased bits to build so there is no delta
>>> webrev.
>>> 
>>> These bits have been run through JPRT and I've submitted the usual
>>> Mach5 tier[1-5] test run...
>>> 
>>> We welcome comments, suggestions and feedback.
>>> 
>>> Dan, Erik and Robbin
>>> 
>>> 
>>> On 11/13/17 12:30 PM, Daniel D. Daugherty wrote:
>>>> Greetings,
>>>> 
>>>> I rebased the project to the 2017.10.26 jdk10/hs PIT snapshot.
>>>> 
>>>> Here are the updated webrevs:
>>>> 
>>>> Here's the mq comment for the change:
>>>> 
>>>>   Rebase to 2017.10.25 PIT snapshot.
>>>> 
>>>> Here is the full webrev:
>>>> 
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-full/
>>>> 
>>>> And here is the delta webrev:
>>>> 
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-delta/
>>>> 
>>>> I ran the above bits throught Mach5 tier[1-5] testing over the holiday
>>>> weekend. Didn't see any issues in a quick look. Going to take a closer
>>>> look today.
>>>> 
>>>> We welcome comments, suggestions and feedback.
>>>> 
>>>> Dan, Erik and Robbin
>>>> 
>>>> 
>>>> On 11/8/17 1:05 PM, Daniel D. Daugherty wrote:
>>>>> Greetings,
>>>>> 
>>>>> Resolving one of the code review comments (from both Stefan K and Coleen)
>>>>> on jdk10-04-full required quite a few changes so it is being done as a
>>>>> standalone patch and corresponding webrevs:
>>>>> 
>>>>> Here's the mq comment for the change:
>>>>> 
>>>>>   stefank, coleenp CR - refactor most JavaThreadIterator usage to use
>>>>>     JavaThreadIteratorWithHandle.
>>>>> 
>>>>> Here is the full webrev:
>>>>> 
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-full/
>>>>> 
>>>>> And here is the delta webrev:
>>>>> 
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-delta/
>>>>> 
>>>>> We welcome comments, suggestions and feedback.
>>>>> 
>>>>> Dan, Erik and Robbin
>>>>> 
>>>>> 
>>>>> On 10/9/17 3:41 PM, Daniel D. Daugherty wrote:
>>>>>> Greetings,
>>>>>> 
>>>>>> We have a (eXtra Large) fix for the following bug:
>>>>>> 
>>>>>> 8167108 inconsistent handling of SR_lock can lead to crashes
>>>>>> https://bugs.openjdk.java.net/browse/JDK-8167108
>>>>>> 
>>>>>> This fix adds a Safe Memory Reclamation (SMR) mechanism based on
>>>>>> Hazard Pointers to manage JavaThread lifecycle.
>>>>>> 
>>>>>> Here's a PDF for the internal wiki that we've been using to describe
>>>>>> and track the work on this project:
>>>>>> 
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/SMR_and_JavaThread_Lifecycle-JDK10-04.pdf 
>>>>>> 
>>>>>> Dan has noticed that the indenting is wrong in some of the code quotes
>>>>>> in the PDF that are not present in the internal wiki. We don't have a
>>>>>> solution for that problem yet.
>>>>>> 
>>>>>> Here's the webrev for current JDK10 version of this fix:
>>>>>> 
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-04-full
>>>>>> 
>>>>>> This fix has been run through many rounds of JPRT and Mach5 tier[2-5]
>>>>>> testing, additional stress testing on Dan's Solaris X64 server, and
>>>>>> additional testing on Erik and Robbin's machines.
>>>>>> 
>>>>>> We welcome comments, suggestions and feedback.
>>>>>> 
>>>>>> Daniel Daugherty
>>>>>> Erik Osterlund
>>>>>> Robbin Ehn
>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>>> 
>>> 
>>> 
>> 
>> 
> 



From rkennke at redhat.com  Mon Nov 20 10:15:40 2017
From: rkennke at redhat.com (Roman Kennke)
Date: Mon, 20 Nov 2017 11:15:40 +0100
Subject: RFR: 8189171: Move GC argument processing into GC specific classes
In-Reply-To: <38c8c5c1-6231-76c2-0c33-7920101814cd@oracle.com>
References: <e72385e3-88f1-9b05-8495-ab7f4c4cbd60@redhat.com>
 <ed0085ca-7384-05e7-4a2c-e2f048f64909@oracle.com>
 <5a7fecea-1a1b-16db-e366-4dd8d0c9ed63@redhat.com>
 <59E9E9A9.7020306@oracle.com>
 <511327b5-4632-0e27-6914-2a3d076a1dc3@redhat.com>
 <59EA002D.1050605@oracle.com>
 <2b37fed8-0383-eb19-3e45-076aabd74f22@redhat.com>
 <59F0AD40.3020307@oracle.com>
 <e8e953a1-06b3-648b-d5da-c87ee7c640ad@redhat.com>
 <B3E05F9C-5AC2-40D3-BBF3-A46ED9B12ED5@oracle.com>
 <affd801a-d184-1bcd-7c58-65c1ecaa94da@oracle.com>
 <3c75a6c5-ffe5-948f-b2c3-b4950a4b1a9f@redhat.com>
 <3f188247-aea1-55d7-cbd4-8ddcd33aa4a8@oracle.com>
 <c57f6ad3-9629-214a-cc41-34ac0c3688ad@redhat.com>
 <B11BD63B-6459-486A-92C3-2C6D045740AF@oracle.com>
 <411d8f0f-82e7-88e3-a1a4-12515b59452a@redhat.com>
 <A8796C24-E60A-484F-B2CA-00C00F4AD908@oracle.com>
 <38c8c5c1-6231-76c2-0c33-7920101814cd@oracle.com>
Message-ID: <4ac6a8ef-a42c-f44b-7135-a9fc846b1848@redhat.com>

Hi David,

I just built linux-x86_64-normal-minimal-slowdebug and it does not fail. 
That's what I did when I tested the patch(es). It's not reasonable to 
expect devs to try and build all possible combinations of jvm variants, 
debug-levels, or even arches etc.

I'll file and fix a follow-up bug for it.

Roman

> On 17/11/2017 2:33 AM, Bob Vandette wrote:
>> Yes, that?s the fix.? I built it correctly with that added include.
>
> How? I can't build the MinimalVM in JPRT because the 
> UNSUPPORTED_OPTION macro is only locally defined in arguments.cpp, but 
> is now used in gcArguments.cpp:
>
> /opt/jprt/T/P1/003543.daholme/s/open/src/hotspot/share/gc/shared/gcArguments.cpp:77:29: 
> error: 'UNSUPPORTED_OPTION' was not declared in this scope
> ?? UNSUPPORTED_OPTION(UseG1GC);
> ???????????????????????????? ^
> make[3]: *** 
> [/opt/jprt/T/P1/003543.daholme/s/build/linux-x86-debug/hotspot/variant-minimal/libjvm/objs/gcArguments.o] 
> Error 1
>
> ---
>
> Roman/Erik: If changing code that has #include guards you must ensure 
> all the paths are tested!
>
> Thanks,
> David
>
>> There are still a few other issues with building and using the 
>> minimal VM but those
>> existed before your change.
>>
>> 1. You cannot build the minimal VM without also building the server 
>> VM.? There
>> is a Makefile dependency on the server VM.
>> 2. The jvm.cfg produced in the jvm and jre images does not add the 
>> minimalvm
>> as one of the possible VM to select.? As a work-around jlink produces 
>> a workable jvm.cfg.
>>
>> Thanks,
>> Bob.
>>
>>
>>> On Nov 16, 2017, at 11:17 AM, Roman Kennke <rkennke at redhat.com> wrote:
>>>
>>> Hi Bob,
>>> thanks for letting me know.
>>>
>>> I filed: https://bugs.openjdk.java.net/browse/JDK-8191424
>>> and posted for review: 
>>> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2017-November/020784.html
>>>
>>> Thanks, Roman
>>>
>>>> Roman,
>>>>
>>>> It looks like this change may have caused the build of the minimal 
>>>> VM to
>>>> fail.? The minimal VM is no longer a configuration that we 
>>>> regularly build and test
>>>> but it is still a build option that can be selected via 
>>>> "--with-jvm-variants=minimal1"
>>>>
>>>>
>>>> /scratch/users/bobv/jdk10hs/open/src/hotspot/share/gc/shared/gcArguments.cpp: 
>>>> In static member function 'static jint GCArguments::initialize()':
>>>> /scratch/users/bobv/jdk10hs/open/src/hotspot/share/gc/shared/gcArguments.cpp:113:17: 
>>>> error: 'defaultStream' has not been declared
>>>> ????? jio_fprintf(defaultStream::error_stream(), "UseParallelGC not 
>>>> supported in this VM.\n");
>>>> ????????????????? ^
>>>> /scratch/users/bobv/jdk10hs/open/src/hotspot/share/gc/shared/gcArguments.cpp:116:17: 
>>>> error: 'defaultStream' has not been declared
>>>> ????? jio_fprintf(defaultStream::error_stream(), "UseG1GC not 
>>>> supported in this VM.\n");
>>>> ????????????????? ^
>>>> /scratch/users/bobv/jdk10hs/open/src/hotspot/share/gc/shared/gcArguments.cpp:119:17: 
>>>> error: 'defaultStream' has not been declared
>>>> ????? jio_fprintf(defaultStream::error_stream(), 
>>>> "UseConcMarkSweepGC not supported in this VM.\n");
>>>> ????????????????? ^
>>>> gmake[3]: *** 
>>>> [/scratch/users/bobv/jdk10hs/build/b00/linux-x64-minimal1/hotspot/variant-minimal/libjvm/objs/gcArguments.o] 
>>>> Error 1
>>>>
>>>> Bob.
>>>>
>>>>
>>>>> On Nov 7, 2017, at 6:01 AM, Roman Kennke <rkennke at redhat.com> wrote:
>>>>>
>>>>> Hi Per & Erik,
>>>>>
>>>>> thanks for the reviews!
>>>>>
>>>>> Now I need a sponsor.
>>>>>
>>>>> Final webrev (same as before, including Reviewed-by line):
>>>>> http://cr.openjdk.java.net/~rkennke/8189171/webrev.05/ 
>>>>> <http://cr.openjdk.java.net/%7Erkennke/8189171/webrev.05/>
>>>>>
>>>>> Thanks, Roman
>>>>>
>>>>>
>>>>>> Looks good Roman!
>>>>>>
>>>>>> /Per
>>>>>>
>>>>>> On 2017-11-06 12:13, Roman Kennke wrote:
>>>>>>> Am 26.10.2017 um 11:43 schrieb Per Liden:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> On 2017-10-25 18:11, Erik Osterlund wrote:
>>>>>>>> [...]
>>>>>>>>>> So let me just put your changes up for review (again), if you 
>>>>>>>>>> don't
>>>>>>>>>> mind:
>>>>>>>>>>
>>>>>>>>>> Full webrev:
>>>>>>>>>> http://cr.openjdk.java.net/~eosterlund/8189171/webrev.03/
>>>>>>>>>> <http://cr.openjdk.java.net/%7Eeosterlund/8189171/webrev.03/>
>>>>>>>> I like this. Just two naming suggestions:
>>>>>>>>
>>>>>>>> src/hotspot/share/gc/shared/gcArguments.hpp:
>>>>>>>>
>>>>>>>> ?? 39?? static jint create_instance();
>>>>>>>> ?? 40?? static bool is_initialized();
>>>>>>>> ?? 41?? static GCArguments* instance();
>>>>>>>>
>>>>>>>> Could we call these:
>>>>>>>>
>>>>>>>> ?? 39?? static jint initialize();
>>>>>>>> ?? 40?? static bool is_initialized();
>>>>>>>> ?? 41?? static GCArguments* arguments();
>>>>>>>>
>>>>>>>> Reasoning: initialize() maps better to its companion 
>>>>>>>> is_initialized().
>>>>>>>> GCArguments::arguments() maps better to the existing patterns 
>>>>>>>> we have
>>>>>>>> with CollectedHeap::heap().
>>>>>>> Ok, that sounds good to me. Actually, it's almost full-circle 
>>>>>>> back to my
>>>>>>> original proposal ;-)
>>>>>>>
>>>>>>> Differential:
>>>>>>> http://cr.openjdk.java.net/~rkennke/8189171/webrev.04.diff/
>>>>>>> <http://cr.openjdk.java.net/%7Erkennke/8189171/webrev.04.diff/>
>>>>>>> Full:
>>>>>>> http://cr.openjdk.java.net/~rkennke/8189171/webrev.04/
>>>>>>> <http://cr.openjdk.java.net/%7Erkennke/8189171/webrev.04/>
>>>>>>>
>>>>>>> Ok to push now?
>>>>>>>
>>>>>>> Roman
>>>>>
>>>
>>



From rkennke at redhat.com  Mon Nov 20 10:17:37 2017
From: rkennke at redhat.com (Roman Kennke)
Date: Mon, 20 Nov 2017 11:17:37 +0100
Subject: RFR: 8189171: Move GC argument processing into GC specific classes
In-Reply-To: <38c8c5c1-6231-76c2-0c33-7920101814cd@oracle.com>
References: <e72385e3-88f1-9b05-8495-ab7f4c4cbd60@redhat.com>
 <ed0085ca-7384-05e7-4a2c-e2f048f64909@oracle.com>
 <5a7fecea-1a1b-16db-e366-4dd8d0c9ed63@redhat.com>
 <59E9E9A9.7020306@oracle.com>
 <511327b5-4632-0e27-6914-2a3d076a1dc3@redhat.com>
 <59EA002D.1050605@oracle.com>
 <2b37fed8-0383-eb19-3e45-076aabd74f22@redhat.com>
 <59F0AD40.3020307@oracle.com>
 <e8e953a1-06b3-648b-d5da-c87ee7c640ad@redhat.com>
 <B3E05F9C-5AC2-40D3-BBF3-A46ED9B12ED5@oracle.com>
 <affd801a-d184-1bcd-7c58-65c1ecaa94da@oracle.com>
 <3c75a6c5-ffe5-948f-b2c3-b4950a4b1a9f@redhat.com>
 <3f188247-aea1-55d7-cbd4-8ddcd33aa4a8@oracle.com>
 <c57f6ad3-9629-214a-cc41-34ac0c3688ad@redhat.com>
 <B11BD63B-6459-486A-92C3-2C6D045740AF@oracle.com>
 <411d8f0f-82e7-88e3-a1a4-12515b59452a@redhat.com>
 <A8796C24-E60A-484F-B2CA-00C00F4AD908@oracle.com>
 <38c8c5c1-6231-76c2-0c33-7920101814cd@oracle.com>
Message-ID: <7ad0f09d-c6ea-c4aa-eb1f-2e31e2a8a951@redhat.com>

... and minimal-fastdebug fails with something else:

/home/rkennke/src/openjdk/jdk-hs/src/hotspot/cpu/x86/c1_Runtime1_x86.cpp:136:10: 
error: 'call_offset' may be used uninitialized in this function 
[-Werror=maybe-uninitialized]
 ?? return call_offset;

I guess I can fix this blindly by including arguments.hpp in 
gcArguments.cpp and hope for the best?

Roman

> On 17/11/2017 2:33 AM, Bob Vandette wrote:
>> Yes, that?s the fix.? I built it correctly with that added include.
>
> How? I can't build the MinimalVM in JPRT because the 
> UNSUPPORTED_OPTION macro is only locally defined in arguments.cpp, but 
> is now used in gcArguments.cpp:
>
> /opt/jprt/T/P1/003543.daholme/s/open/src/hotspot/share/gc/shared/gcArguments.cpp:77:29: 
> error: 'UNSUPPORTED_OPTION' was not declared in this scope
> ?? UNSUPPORTED_OPTION(UseG1GC);
> ???????????????????????????? ^
> make[3]: *** 
> [/opt/jprt/T/P1/003543.daholme/s/build/linux-x86-debug/hotspot/variant-minimal/libjvm/objs/gcArguments.o] 
> Error 1
>
> ---
>
> Roman/Erik: If changing code that has #include guards you must ensure 
> all the paths are tested!
>
> Thanks,
> David
>
>> There are still a few other issues with building and using the 
>> minimal VM but those
>> existed before your change.
>>
>> 1. You cannot build the minimal VM without also building the server 
>> VM.? There
>> is a Makefile dependency on the server VM.
>> 2. The jvm.cfg produced in the jvm and jre images does not add the 
>> minimalvm
>> as one of the possible VM to select.? As a work-around jlink produces 
>> a workable jvm.cfg.
>>
>> Thanks,
>> Bob.
>>
>>
>>> On Nov 16, 2017, at 11:17 AM, Roman Kennke <rkennke at redhat.com> wrote:
>>>
>>> Hi Bob,
>>> thanks for letting me know.
>>>
>>> I filed: https://bugs.openjdk.java.net/browse/JDK-8191424
>>> and posted for review: 
>>> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2017-November/020784.html
>>>
>>> Thanks, Roman
>>>
>>>> Roman,
>>>>
>>>> It looks like this change may have caused the build of the minimal 
>>>> VM to
>>>> fail.? The minimal VM is no longer a configuration that we 
>>>> regularly build and test
>>>> but it is still a build option that can be selected via 
>>>> "--with-jvm-variants=minimal1"
>>>>
>>>>
>>>> /scratch/users/bobv/jdk10hs/open/src/hotspot/share/gc/shared/gcArguments.cpp: 
>>>> In static member function 'static jint GCArguments::initialize()':
>>>> /scratch/users/bobv/jdk10hs/open/src/hotspot/share/gc/shared/gcArguments.cpp:113:17: 
>>>> error: 'defaultStream' has not been declared
>>>> ????? jio_fprintf(defaultStream::error_stream(), "UseParallelGC not 
>>>> supported in this VM.\n");
>>>> ????????????????? ^
>>>> /scratch/users/bobv/jdk10hs/open/src/hotspot/share/gc/shared/gcArguments.cpp:116:17: 
>>>> error: 'defaultStream' has not been declared
>>>> ????? jio_fprintf(defaultStream::error_stream(), "UseG1GC not 
>>>> supported in this VM.\n");
>>>> ????????????????? ^
>>>> /scratch/users/bobv/jdk10hs/open/src/hotspot/share/gc/shared/gcArguments.cpp:119:17: 
>>>> error: 'defaultStream' has not been declared
>>>> ????? jio_fprintf(defaultStream::error_stream(), 
>>>> "UseConcMarkSweepGC not supported in this VM.\n");
>>>> ????????????????? ^
>>>> gmake[3]: *** 
>>>> [/scratch/users/bobv/jdk10hs/build/b00/linux-x64-minimal1/hotspot/variant-minimal/libjvm/objs/gcArguments.o] 
>>>> Error 1
>>>>
>>>> Bob.
>>>>
>>>>
>>>>> On Nov 7, 2017, at 6:01 AM, Roman Kennke <rkennke at redhat.com> wrote:
>>>>>
>>>>> Hi Per & Erik,
>>>>>
>>>>> thanks for the reviews!
>>>>>
>>>>> Now I need a sponsor.
>>>>>
>>>>> Final webrev (same as before, including Reviewed-by line):
>>>>> http://cr.openjdk.java.net/~rkennke/8189171/webrev.05/ 
>>>>> <http://cr.openjdk.java.net/%7Erkennke/8189171/webrev.05/>
>>>>>
>>>>> Thanks, Roman
>>>>>
>>>>>
>>>>>> Looks good Roman!
>>>>>>
>>>>>> /Per
>>>>>>
>>>>>> On 2017-11-06 12:13, Roman Kennke wrote:
>>>>>>> Am 26.10.2017 um 11:43 schrieb Per Liden:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> On 2017-10-25 18:11, Erik Osterlund wrote:
>>>>>>>> [...]
>>>>>>>>>> So let me just put your changes up for review (again), if you 
>>>>>>>>>> don't
>>>>>>>>>> mind:
>>>>>>>>>>
>>>>>>>>>> Full webrev:
>>>>>>>>>> http://cr.openjdk.java.net/~eosterlund/8189171/webrev.03/
>>>>>>>>>> <http://cr.openjdk.java.net/%7Eeosterlund/8189171/webrev.03/>
>>>>>>>> I like this. Just two naming suggestions:
>>>>>>>>
>>>>>>>> src/hotspot/share/gc/shared/gcArguments.hpp:
>>>>>>>>
>>>>>>>> ?? 39?? static jint create_instance();
>>>>>>>> ?? 40?? static bool is_initialized();
>>>>>>>> ?? 41?? static GCArguments* instance();
>>>>>>>>
>>>>>>>> Could we call these:
>>>>>>>>
>>>>>>>> ?? 39?? static jint initialize();
>>>>>>>> ?? 40?? static bool is_initialized();
>>>>>>>> ?? 41?? static GCArguments* arguments();
>>>>>>>>
>>>>>>>> Reasoning: initialize() maps better to its companion 
>>>>>>>> is_initialized().
>>>>>>>> GCArguments::arguments() maps better to the existing patterns 
>>>>>>>> we have
>>>>>>>> with CollectedHeap::heap().
>>>>>>> Ok, that sounds good to me. Actually, it's almost full-circle 
>>>>>>> back to my
>>>>>>> original proposal ;-)
>>>>>>>
>>>>>>> Differential:
>>>>>>> http://cr.openjdk.java.net/~rkennke/8189171/webrev.04.diff/
>>>>>>> <http://cr.openjdk.java.net/%7Erkennke/8189171/webrev.04.diff/>
>>>>>>> Full:
>>>>>>> http://cr.openjdk.java.net/~rkennke/8189171/webrev.04/
>>>>>>> <http://cr.openjdk.java.net/%7Erkennke/8189171/webrev.04/>
>>>>>>>
>>>>>>> Ok to push now?
>>>>>>>
>>>>>>> Roman
>>>>>
>>>
>>



From rkennke at redhat.com  Mon Nov 20 10:36:55 2017
From: rkennke at redhat.com (Roman Kennke)
Date: Mon, 20 Nov 2017 11:36:55 +0100
Subject: RFR: 8191562: Missing include in gcArguments.cpp breaks minimal JVM
 build
Message-ID: <5699db6f-5397-d144-956e-eaafe37cca69@redhat.com>

David Holmes reported build failures in minimal JVM due to missing 
include in gcArguments.cpp:

https://bugs.openjdk.java.net/browse/JDK-8191562

This should fix it:

http://cr.openjdk.java.net/~rkennke/8191562/webrev.00/ 
<http://cr.openjdk.java.net/%7Erkennke/8191562/webrev.00/>

Unfortunately, the minimal JVM doesn't build at all for me (*), so I am 
posting this blindly. However, it seems fairly obvious that it's 
correct: UNSUPPORTED_OPTION is defined in arguments.hpp, including it 
should fix the build failure.

Ok to go?

Roman

(*)

/home/rkennke/src/openjdk/jdk-hs/src/hotspot/cpu/x86/c1_Runtime1_x86.cpp: 
In member function 'int StubAssembler::call_RT(Register, Register, 
address, int)':
/home/rkennke/src/openjdk/jdk-hs/src/hotspot/cpu/x86/c1_Runtime1_x86.cpp:136:10: 
error: 'call_offset' may be used uninitialized in this function 
[-Werror=maybe-uninitialized]
 ?? return call_offset;
 ????????? ^
cc1plus: all warnings being treated as errors



From thomas.schatzl at oracle.com  Mon Nov 20 11:26:29 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 20 Nov 2017 12:26:29 +0100
Subject: RFR (M): 8190426: Lazily initialize refinement threads with
 UseDynamicNumberOfGCThreads
In-Reply-To: <1f3e5ef0-60f3-770f-f581-8c7764e44f2e@oracle.com>
References: <1509726495.2630.2.camel@oracle.com>
 <1f3e5ef0-60f3-770f-f581-8c7764e44f2e@oracle.com>
Message-ID: <1511177189.2337.39.camel@oracle.com>

Hi Sangheon,

  thanks for your review, and sorry for the late answer - I have been
travelling lately...

On Thu, 2017-11-09 at 15:00 -0800, sangheon.kim wrote:
> Hi Thomas,
> 
> On 11/03/2017 09:28 AM, Thomas Schatzl wrote:
> > Hi,
> > 
> >    can I have reviews for this change that makes refinement threads
> > behave the same as the other thread groups we have in G1?
> > 
> > I.e. with UseDynamicNumberOfGCThreads enabled (which is off by
> > default) they are created lazily.
> > 
> > This helps a little bit further with startup/footprint benchmarks
> > (if
> > enabled).
> > 
> > CR:
> > https://bugs.openjdk.java.net/browse/JDK-8190426
> > Webrev:
> > http://cr.openjdk.java.net/~tschatzl/8190426/webrev/
> > Testing:
> > hs-tier1+2
> 
> I like this idea, thank you for improving this.
> 
> I have some comments.

I think I addressed all your concerns. Some of them were due to me
being confused about which memory allocations cause VM exit, and which
don't. I hope I got that right now.

I also changed the behavior of the thread startup to be more similar to
the worker threads, namely:
- always start one refinement thread at startup
- support InjectGCWorkerCreationFailure (which is tested by our tests
btw)
- make error messages and failure modes more similar to the ones for
the work gangs.

Even with that we want to look at JDK-8191082 for better specifying the
expected behavior.

Webrevs:
http://cr.openjdk.java.net/~tschatzl/8190426/webrev.0_to_1/ (diff)
http://cr.openjdk.java.net/~tschatzl/8190426/webrev.1/ (full)
Testing:
hs-tier1+2

Thanks,
  Thomas



From david.holmes at oracle.com  Mon Nov 20 11:53:54 2017
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 20 Nov 2017 21:53:54 +1000
Subject: RFR: 8189171: Move GC argument processing into GC specific classes
In-Reply-To: <4ac6a8ef-a42c-f44b-7135-a9fc846b1848@redhat.com>
References: <e72385e3-88f1-9b05-8495-ab7f4c4cbd60@redhat.com>
 <ed0085ca-7384-05e7-4a2c-e2f048f64909@oracle.com>
 <5a7fecea-1a1b-16db-e366-4dd8d0c9ed63@redhat.com>
 <59E9E9A9.7020306@oracle.com>
 <511327b5-4632-0e27-6914-2a3d076a1dc3@redhat.com>
 <59EA002D.1050605@oracle.com>
 <2b37fed8-0383-eb19-3e45-076aabd74f22@redhat.com>
 <59F0AD40.3020307@oracle.com>
 <e8e953a1-06b3-648b-d5da-c87ee7c640ad@redhat.com>
 <B3E05F9C-5AC2-40D3-BBF3-A46ED9B12ED5@oracle.com>
 <affd801a-d184-1bcd-7c58-65c1ecaa94da@oracle.com>
 <3c75a6c5-ffe5-948f-b2c3-b4950a4b1a9f@redhat.com>
 <3f188247-aea1-55d7-cbd4-8ddcd33aa4a8@oracle.com>
 <c57f6ad3-9629-214a-cc41-34ac0c3688ad@redhat.com>
 <B11BD63B-6459-486A-92C3-2C6D045740AF@oracle.com>
 <411d8f0f-82e7-88e3-a1a4-12515b59452a@redhat.com>
 <A8796C24-E60A-484F-B2CA-00C00F4AD908@oracle.com>
 <38c8c5c1-6231-76c2-0c33-7920101814cd@oracle.com>
 <4ac6a8ef-a42c-f44b-7135-a9fc846b1848@redhat.com>
Message-ID: <04b02b4a-b208-846c-8dc3-a8081c474d86@oracle.com>

On 20/11/2017 8:15 PM, Roman Kennke wrote:
> Hi David,
> 
> I just built linux-x86_64-normal-minimal-slowdebug and it does not fail. 
> That's what I did when I tested the patch(es). It's not reasonable to 
> expect devs to try and build all possible combinations of jvm variants, 
> debug-levels, or even arches etc.

I couldn't understand how any minimal VM build could succeed**. I can 
only guess that it is due to precompiled headers pulling in 
arguments.hpp in one case but not another - though I can't say why 
fastdebug and slowdebug should behave differently ??

In JPRT we build product and fastdebug.

** I mistakenly thought the macro was declared in arguments.cpp and 
hence could never be available elsewhere.

> I'll file and fix a follow-up bug for it.

Thanks,
David
-----
> 
> Roman
> 
>> On 17/11/2017 2:33 AM, Bob Vandette wrote:
>>> Yes, that?s the fix.? I built it correctly with that added include.
>>
>> How? I can't build the MinimalVM in JPRT because the 
>> UNSUPPORTED_OPTION macro is only locally defined in arguments.cpp, but 
>> is now used in gcArguments.cpp:
>>
>> /opt/jprt/T/P1/003543.daholme/s/open/src/hotspot/share/gc/shared/gcArguments.cpp:77:29: 
>> error: 'UNSUPPORTED_OPTION' was not declared in this scope
>> ?? UNSUPPORTED_OPTION(UseG1GC);
>> ???????????????????????????? ^
>> make[3]: *** 
>> [/opt/jprt/T/P1/003543.daholme/s/build/linux-x86-debug/hotspot/variant-minimal/libjvm/objs/gcArguments.o] 
>> Error 1
>>
>> ---
>>
>> Roman/Erik: If changing code that has #include guards you must ensure 
>> all the paths are tested!
>>
>> Thanks,
>> David
>>
>>> There are still a few other issues with building and using the 
>>> minimal VM but those
>>> existed before your change.
>>>
>>> 1. You cannot build the minimal VM without also building the server 
>>> VM.? There
>>> is a Makefile dependency on the server VM.
>>> 2. The jvm.cfg produced in the jvm and jre images does not add the 
>>> minimalvm
>>> as one of the possible VM to select.? As a work-around jlink produces 
>>> a workable jvm.cfg.
>>>
>>> Thanks,
>>> Bob.
>>>
>>>
>>>> On Nov 16, 2017, at 11:17 AM, Roman Kennke <rkennke at redhat.com> wrote:
>>>>
>>>> Hi Bob,
>>>> thanks for letting me know.
>>>>
>>>> I filed: https://bugs.openjdk.java.net/browse/JDK-8191424
>>>> and posted for review: 
>>>> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2017-November/020784.html 
>>>>
>>>>
>>>> Thanks, Roman
>>>>
>>>>> Roman,
>>>>>
>>>>> It looks like this change may have caused the build of the minimal 
>>>>> VM to
>>>>> fail.? The minimal VM is no longer a configuration that we 
>>>>> regularly build and test
>>>>> but it is still a build option that can be selected via 
>>>>> "--with-jvm-variants=minimal1"
>>>>>
>>>>>
>>>>> /scratch/users/bobv/jdk10hs/open/src/hotspot/share/gc/shared/gcArguments.cpp: 
>>>>> In static member function 'static jint GCArguments::initialize()':
>>>>> /scratch/users/bobv/jdk10hs/open/src/hotspot/share/gc/shared/gcArguments.cpp:113:17: 
>>>>> error: 'defaultStream' has not been declared
>>>>> ????? jio_fprintf(defaultStream::error_stream(), "UseParallelGC not 
>>>>> supported in this VM.\n");
>>>>> ????????????????? ^
>>>>> /scratch/users/bobv/jdk10hs/open/src/hotspot/share/gc/shared/gcArguments.cpp:116:17: 
>>>>> error: 'defaultStream' has not been declared
>>>>> ????? jio_fprintf(defaultStream::error_stream(), "UseG1GC not 
>>>>> supported in this VM.\n");
>>>>> ????????????????? ^
>>>>> /scratch/users/bobv/jdk10hs/open/src/hotspot/share/gc/shared/gcArguments.cpp:119:17: 
>>>>> error: 'defaultStream' has not been declared
>>>>> ????? jio_fprintf(defaultStream::error_stream(), 
>>>>> "UseConcMarkSweepGC not supported in this VM.\n");
>>>>> ????????????????? ^
>>>>> gmake[3]: *** 
>>>>> [/scratch/users/bobv/jdk10hs/build/b00/linux-x64-minimal1/hotspot/variant-minimal/libjvm/objs/gcArguments.o] 
>>>>> Error 1
>>>>>
>>>>> Bob.
>>>>>
>>>>>
>>>>>> On Nov 7, 2017, at 6:01 AM, Roman Kennke <rkennke at redhat.com> wrote:
>>>>>>
>>>>>> Hi Per & Erik,
>>>>>>
>>>>>> thanks for the reviews!
>>>>>>
>>>>>> Now I need a sponsor.
>>>>>>
>>>>>> Final webrev (same as before, including Reviewed-by line):
>>>>>> http://cr.openjdk.java.net/~rkennke/8189171/webrev.05/ 
>>>>>> <http://cr.openjdk.java.net/%7Erkennke/8189171/webrev.05/>
>>>>>>
>>>>>> Thanks, Roman
>>>>>>
>>>>>>
>>>>>>> Looks good Roman!
>>>>>>>
>>>>>>> /Per
>>>>>>>
>>>>>>> On 2017-11-06 12:13, Roman Kennke wrote:
>>>>>>>> Am 26.10.2017 um 11:43 schrieb Per Liden:
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> On 2017-10-25 18:11, Erik Osterlund wrote:
>>>>>>>>> [...]
>>>>>>>>>>> So let me just put your changes up for review (again), if you 
>>>>>>>>>>> don't
>>>>>>>>>>> mind:
>>>>>>>>>>>
>>>>>>>>>>> Full webrev:
>>>>>>>>>>> http://cr.openjdk.java.net/~eosterlund/8189171/webrev.03/
>>>>>>>>>>> <http://cr.openjdk.java.net/%7Eeosterlund/8189171/webrev.03/>
>>>>>>>>> I like this. Just two naming suggestions:
>>>>>>>>>
>>>>>>>>> src/hotspot/share/gc/shared/gcArguments.hpp:
>>>>>>>>>
>>>>>>>>> ?? 39?? static jint create_instance();
>>>>>>>>> ?? 40?? static bool is_initialized();
>>>>>>>>> ?? 41?? static GCArguments* instance();
>>>>>>>>>
>>>>>>>>> Could we call these:
>>>>>>>>>
>>>>>>>>> ?? 39?? static jint initialize();
>>>>>>>>> ?? 40?? static bool is_initialized();
>>>>>>>>> ?? 41?? static GCArguments* arguments();
>>>>>>>>>
>>>>>>>>> Reasoning: initialize() maps better to its companion 
>>>>>>>>> is_initialized().
>>>>>>>>> GCArguments::arguments() maps better to the existing patterns 
>>>>>>>>> we have
>>>>>>>>> with CollectedHeap::heap().
>>>>>>>> Ok, that sounds good to me. Actually, it's almost full-circle 
>>>>>>>> back to my
>>>>>>>> original proposal ;-)
>>>>>>>>
>>>>>>>> Differential:
>>>>>>>> http://cr.openjdk.java.net/~rkennke/8189171/webrev.04.diff/
>>>>>>>> <http://cr.openjdk.java.net/%7Erkennke/8189171/webrev.04.diff/>
>>>>>>>> Full:
>>>>>>>> http://cr.openjdk.java.net/~rkennke/8189171/webrev.04/
>>>>>>>> <http://cr.openjdk.java.net/%7Erkennke/8189171/webrev.04/>
>>>>>>>>
>>>>>>>> Ok to push now?
>>>>>>>>
>>>>>>>> Roman
>>>>>>
>>>>
>>>
> 


From thomas.schatzl at oracle.com  Mon Nov 20 11:56:03 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 20 Nov 2017 12:56:03 +0100
Subject: RFR (XXS): 8179244: Assert failed in instanceMirrorKlass.inline.hpp
Message-ID: <1511178963.2337.46.camel@oracle.com>

Hi all,

  can I have reviews for this small comment update that clarifies that
the assert in that location is too strong and should be commented out.

The issue is that an assert fails during CMS precleaning if class
loading occurred where the mirror ended up in the old generation (in
some situations CMS allocates directly into old gen). This assert found
that a klass that has had a NULL value suddenly changed its value to
something else.

This is normal behavior in the situation described above.

Also checked that we do not miss iterating over a CLD for such oops -
there is actually explicit support in class loading for this, where
such CLDs (and their mirrors) are kept alive until remark and iterated
over again there. (Look for "CMS support" in classLoaderData.?pp). 

CR:
https://bugs.openjdk.java.net/browse/JDK-8179244
Webrev:
http://cr.openjdk.java.net/~tschatzl/8179244/webrev/
Testing:
local compilation (just a comment added)

Thanks,
  Thomas



From rkennke at redhat.com  Mon Nov 20 15:32:31 2017
From: rkennke at redhat.com (Roman Kennke)
Date: Mon, 20 Nov 2017 16:32:31 +0100
Subject: RFR: 8191564: Refactor GC related servicability code into GC specific
 subclasses
Message-ID: <387d2b93-cb8b-51be-a6e0-3b5f9d6cbb1c@redhat.com>

Currently, there's lots of GC specific code sprinkled over 
src/hotspot/share/services. This change introduces a GC interface for 
that, and moves all GC specific code to their respective 
src/hotspot/share/gc directory.

http://cr.openjdk.java.net/~rkennke/8191564/webrev.00/ 
<http://cr.openjdk.java.net/%7Erkennke/8191564/webrev.00/>

Testing: hotspot_gc and hotspot_serviceability, none showed regressions

Built minimal and server without regressions

What do you think?

Roman




From martin.doerr at sap.com  Mon Nov 20 17:00:24 2017
From: martin.doerr at sap.com (Doerr, Martin)
Date: Mon, 20 Nov 2017 17:00:24 +0000
Subject: 8191562: Missing include in gcArguments.cpp breaks minimal JVM
 build
In-Reply-To: <5699db6f-5397-d144-956e-eaafe37cca69@redhat.com>
References: <5699db6f-5397-d144-956e-eaafe37cca69@redhat.com>
Message-ID: <dc34b321d7a24e6e8a527836581c874f@sap.com>

Hi Roman,

unfortunately, I was checking AIX too late. There's another missing include.
We need to #include "memory/allocation.inline.hpp". Otherwise xlC doesn't find a definition for the new/delete operators like CHeapObj<F>::operator new.

Do you want me to open a new bug for it?

Best regards,
Martin


-----Original Message-----
From: hotspot-gc-dev [mailto:hotspot-gc-dev-bounces at openjdk.java.net] On Behalf Of Roman Kennke
Sent: Montag, 20. November 2017 11:37
To: hotspot-gc-dev at openjdk.java.net
Subject: RFR: 8191562: Missing include in gcArguments.cpp breaks minimal JVM build

David Holmes reported build failures in minimal JVM due to missing 
include in gcArguments.cpp:

https://bugs.openjdk.java.net/browse/JDK-8191562

This should fix it:

http://cr.openjdk.java.net/~rkennke/8191562/webrev.00/ 
<http://cr.openjdk.java.net/%7Erkennke/8191562/webrev.00/>

Unfortunately, the minimal JVM doesn't build at all for me (*), so I am 
posting this blindly. However, it seems fairly obvious that it's 
correct: UNSUPPORTED_OPTION is defined in arguments.hpp, including it 
should fix the build failure.

Ok to go?

Roman

(*)

/home/rkennke/src/openjdk/jdk-hs/src/hotspot/cpu/x86/c1_Runtime1_x86.cpp: 
In member function 'int StubAssembler::call_RT(Register, Register, 
address, int)':
/home/rkennke/src/openjdk/jdk-hs/src/hotspot/cpu/x86/c1_Runtime1_x86.cpp:136:10: 
error: 'call_offset' may be used uninitialized in this function 
[-Werror=maybe-uninitialized]
 ?? return call_offset;
 ????????? ^
cc1plus: all warnings being treated as errors


From rkennke at redhat.com  Mon Nov 20 17:14:17 2017
From: rkennke at redhat.com (Roman Kennke)
Date: Mon, 20 Nov 2017 18:14:17 +0100
Subject: 8191562: Missing include in gcArguments.cpp breaks minimal JVM
 build
In-Reply-To: <dc34b321d7a24e6e8a527836581c874f@sap.com>
References: <5699db6f-5397-d144-956e-eaafe37cca69@redhat.com>
 <dc34b321d7a24e6e8a527836581c874f@sap.com>
Message-ID: <6be81483-c3f6-0f8a-19db-1975036d1c2c@redhat.com>

Hi Martin,
no, I don't think this is necessary. We can sneak it into this change:

http://cr.openjdk.java.net/~rkennke/8191562/webrev.01/ 
<http://cr.openjdk.java.net/%7Erkennke/8191562/webrev.01/>

Is that what aix needs?

Thanks,
Roman

> Hi Roman,
>
> unfortunately, I was checking AIX too late. There's another missing include.
> We need to #include "memory/allocation.inline.hpp". Otherwise xlC doesn't find a definition for the new/delete operators like CHeapObj<F>::operator new.
>
> Do you want me to open a new bug for it?
>
> Best regards,
> Martin
>
>
> -----Original Message-----
> From: hotspot-gc-dev [mailto:hotspot-gc-dev-bounces at openjdk.java.net] On Behalf Of Roman Kennke
> Sent: Montag, 20. November 2017 11:37
> To: hotspot-gc-dev at openjdk.java.net
> Subject: RFR: 8191562: Missing include in gcArguments.cpp breaks minimal JVM build
>
> David Holmes reported build failures in minimal JVM due to missing
> include in gcArguments.cpp:
>
> https://bugs.openjdk.java.net/browse/JDK-8191562
>
> This should fix it:
>
> http://cr.openjdk.java.net/~rkennke/8191562/webrev.00/
> <http://cr.openjdk.java.net/%7Erkennke/8191562/webrev.00/>
>
> Unfortunately, the minimal JVM doesn't build at all for me (*), so I am
> posting this blindly. However, it seems fairly obvious that it's
> correct: UNSUPPORTED_OPTION is defined in arguments.hpp, including it
> should fix the build failure.
>
> Ok to go?
>
> Roman
>
> (*)
>
> /home/rkennke/src/openjdk/jdk-hs/src/hotspot/cpu/x86/c1_Runtime1_x86.cpp:
> In member function 'int StubAssembler::call_RT(Register, Register,
> address, int)':
> /home/rkennke/src/openjdk/jdk-hs/src/hotspot/cpu/x86/c1_Runtime1_x86.cpp:136:10:
> error: 'call_offset' may be used uninitialized in this function
> [-Werror=maybe-uninitialized]
>   ?? return call_offset;
>   ????????? ^
> cc1plus: all warnings being treated as errors
>



From martin.doerr at sap.com  Mon Nov 20 17:16:23 2017
From: martin.doerr at sap.com (Doerr, Martin)
Date: Mon, 20 Nov 2017 17:16:23 +0000
Subject: 8191562: Missing include in gcArguments.cpp breaks minimal JVM
 build
In-Reply-To: <6be81483-c3f6-0f8a-19db-1975036d1c2c@redhat.com>
References: <5699db6f-5397-d144-956e-eaafe37cca69@redhat.com>
 <dc34b321d7a24e6e8a527836581c874f@sap.com>
 <6be81483-c3f6-0f8a-19db-1975036d1c2c@redhat.com>
Message-ID: <58e569919c544a01a876909f9d2cb0c0@sap.com>

Hi Roman,

cool, that looks good. Thanks a lot.

Best regards,
Martin


-----Original Message-----
From: Roman Kennke [mailto:rkennke at redhat.com] 
Sent: Montag, 20. November 2017 18:14
To: Doerr, Martin <martin.doerr at sap.com>; hotspot-gc-dev at openjdk.java.net
Subject: Re: 8191562: Missing include in gcArguments.cpp breaks minimal JVM build

Hi Martin,
no, I don't think this is necessary. We can sneak it into this change:

http://cr.openjdk.java.net/~rkennke/8191562/webrev.01/ 
<http://cr.openjdk.java.net/%7Erkennke/8191562/webrev.01/>

Is that what aix needs?

Thanks,
Roman

> Hi Roman,
>
> unfortunately, I was checking AIX too late. There's another missing include.
> We need to #include "memory/allocation.inline.hpp". Otherwise xlC doesn't find a definition for the new/delete operators like CHeapObj<F>::operator new.
>
> Do you want me to open a new bug for it?
>
> Best regards,
> Martin
>
>
> -----Original Message-----
> From: hotspot-gc-dev [mailto:hotspot-gc-dev-bounces at openjdk.java.net] On Behalf Of Roman Kennke
> Sent: Montag, 20. November 2017 11:37
> To: hotspot-gc-dev at openjdk.java.net
> Subject: RFR: 8191562: Missing include in gcArguments.cpp breaks minimal JVM build
>
> David Holmes reported build failures in minimal JVM due to missing
> include in gcArguments.cpp:
>
> https://bugs.openjdk.java.net/browse/JDK-8191562
>
> This should fix it:
>
> http://cr.openjdk.java.net/~rkennke/8191562/webrev.00/
> <http://cr.openjdk.java.net/%7Erkennke/8191562/webrev.00/>
>
> Unfortunately, the minimal JVM doesn't build at all for me (*), so I am
> posting this blindly. However, it seems fairly obvious that it's
> correct: UNSUPPORTED_OPTION is defined in arguments.hpp, including it
> should fix the build failure.
>
> Ok to go?
>
> Roman
>
> (*)
>
> /home/rkennke/src/openjdk/jdk-hs/src/hotspot/cpu/x86/c1_Runtime1_x86.cpp:
> In member function 'int StubAssembler::call_RT(Register, Register,
> address, int)':
> /home/rkennke/src/openjdk/jdk-hs/src/hotspot/cpu/x86/c1_Runtime1_x86.cpp:136:10:
> error: 'call_offset' may be used uninitialized in this function
> [-Werror=maybe-uninitialized]
>   ?? return call_offset;
>   ????????? ^
> cc1plus: all warnings being treated as errors
>


From thomas.schatzl at oracle.com  Mon Nov 20 18:17:34 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 20 Nov 2017 19:17:34 +0100
Subject: RFR (M): 8190426: Lazily initialize refinement threads with
 UseDynamicNumberOfGCThreads
In-Reply-To: <1511177189.2337.39.camel@oracle.com>
References: <1509726495.2630.2.camel@oracle.com>
 <1f3e5ef0-60f3-770f-f581-8c7764e44f2e@oracle.com>
 <1511177189.2337.39.camel@oracle.com>
Message-ID: <1511201854.2229.4.camel@oracle.com>

Hi all,

  Stefan Johansson found another issue: in the original code, before
activating a thread it checked whether that thread had already been
activated. The changes did not do that.

Here's a new webrev:

http://cr.openjdk.java.net/~tschatzl/8190426/webrev.1_to_2/ (diff)
http://cr.openjdk.java.net/~tschatzl/8190426/webrev.2/ (full)

Testing:
hs tier1+2

Thanks,
  Thomas


On Mon, 2017-11-20 at 12:26 +0100, Thomas Schatzl wrote:
> Hi Sangheon,
> 
>   thanks for your review, and sorry for the late answer - I have been
> travelling lately...
> 
> On Thu, 2017-11-09 at 15:00 -0800, sangheon.kim wrote:
> > Hi Thomas,
> > 
> > On 11/03/2017 09:28 AM, Thomas Schatzl wrote:
> > > Hi,
> > > 
> > >    can I have reviews for this change that makes refinement
> > > threads
> > > behave the same as the other thread groups we have in G1?
> > > 
> > > I.e. with UseDynamicNumberOfGCThreads enabled (which is off by
> > > default) they are created lazily.
> > > 
> > > This helps a little bit further with startup/footprint benchmarks
> > > (if
> > > enabled).
> > > 
> > > CR:
> > > https://bugs.openjdk.java.net/browse/JDK-8190426
> > > Webrev:
> > > http://cr.openjdk.java.net/~tschatzl/8190426/webrev/
> > > Testing:
> > > hs-tier1+2
> > 
> > I like this idea, thank you for improving this.
> > 
> > I have some comments.
> 
> I think I addressed all your concerns. Some of them were due to me
> being confused about which memory allocations cause VM exit, and
> which
> don't. I hope I got that right now.
> 
> I also changed the behavior of the thread startup to be more similar
> to
> the worker threads, namely:
> - always start one refinement thread at startup
> - support InjectGCWorkerCreationFailure (which is tested by our tests
> btw)
> - make error messages and failure modes more similar to the ones for
> the work gangs.
> 
> Even with that we want to look at JDK-8191082 for better specifying
> the
> expected behavior.
> 
> Webrevs:
> http://cr.openjdk.java.net/~tschatzl/8190426/webrev.0_to_1/ (diff)
> http://cr.openjdk.java.net/~tschatzl/8190426/webrev.1/ (full)
> Testing:
> hs-tier1+2
> 
> Thanks,
>   Thomas
> 



From sangheon.kim at oracle.com  Mon Nov 20 19:05:59 2017
From: sangheon.kim at oracle.com (sangheon.kim)
Date: Mon, 20 Nov 2017 11:05:59 -0800
Subject: RFR(XS): 8178497: Bug in MutableNUMASpace::ensure_parsability
Message-ID: <8f8e7c5e-5de8-ee26-acb4-e90662dad0af@oracle.com>

Hi all,

Can I have some reviews for this tiny pointer arithmetic change?
Current code doesn't use pointer arithmetic, so the the resulting values 
are wrong(too small). i.e. adding two values first and then typecast to 
HeapWord* which is wrong here.
e.g.
intptr_t cur_top = X;
size_t touched_words = XX;
...
align_down((HeapWord*)(cur_top + touched_words), XXX);

This should be 'align_down( (HeapWord*)cur_top + touched_words, XXXX);'.

As I don't see any good reason of using 'intptr_t', changed to use 
'HeapWord*' and changed related stuff.

I didn't add regression test or some further investigation as this is 
clear that the calculation is wrong. And hard to provoke the problem 
outside.

CR: https://bugs.openjdk.java.net/browse/JDK-8178497
Webrev: http://cr.openjdk.java.net/~sangheki/8178497/webrev.0
Testing: local building

Thanks,
Sangheon


From daniel.daugherty at oracle.com  Tue Nov 21 01:50:29 2017
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Mon, 20 Nov 2017 20:50:29 -0500
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <43d2f342-3f4b-5c1a-a687-c24c4efedba9@oracle.com>
References: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
 <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>
 <836b5c6c-5eef-9245-c16a-e71372093d9a@oracle.com>
 <82d0e473-a24f-9feb-82e9-7d04c4d7a91d@oracle.com>
 <354f7ece-66bc-c065-9ee8-f55abcc1c23e@oracle.com>
 <0ff8df8e-e93c-02d2-cb97-b13b1ac0f53f@oracle.com>
 <43d2f342-3f4b-5c1a-a687-c24c4efedba9@oracle.com>
Message-ID: <92d1f572-a745-6f2b-1b5f-e68a1d252bab@oracle.com>

On 11/20/17 12:51 AM, David Holmes wrote:
> Hi Dan,
>
> Figured I'd better cast an eye over this again before it gets pushed :)

Thanks for reviewing again!!


> Only one thing (repeated many times) jumped out at me and apologies if 
> this has already been debated back and forth. I missed the exchange 
> where the for loop iteration was rewritten into this unusual form:
>
> for (JavaThreadIteratorWithHandle jtiwh; JavaThread *jt = 
> jtiwh.next(); ) {
>
> On first reading I expect most people would mistake the control 
> expression for the iteration-expression due to the next(). I didn't 
> even know the control expression could introduce a local variable! I 
> find this really hard to read stylistically and far too cute/clever 
> for its own good. It also violates the style rules regarding implicit 
> boolean checks.
>
> I know Stefan proposed this as he did not like the alternate (in a few 
> places):
>
> +? {
> +??? ThreadsListHandle tlh;
> +??? JavaThreadIterator jti(tlh.list());
> +??? for (JavaThread* t = jti.first(); t != NULL; t = jti.next()) {
> ...
> +? }
>
> but it seems to me the iterator code has evolved since then and only a 
> couple of places need a distinct TLH separate from the actual iterator 
> object. So I'm more in favour of the more classic for-loop, with the 
> iterator declared before the loop. Or even convert to while loops, as 
> this iterator seems more suited to that.

Actually both Coleen and Stefan had a problem with how much additional
code was added to support an iterator model. In some cases we went from
1-line to 3-lines and in some cases we went from 1-line to 5-lines. For
Coleen, she wanted 2 of the new lines compressed into 1 new line by using
an iterator with an embedded ThreadsListHandle. Stefan wanted us to try
and get back to 1-line where possible.

The advantages of the new style are:

- 1-line to 1-line in all but a few cases
- automatic limited scope of the embedded ThreadsListHandle so we were
 ? able to remove extra braces that were added earlier in most cases

The disadvantages of the new style are:

- it is an unusual for-loop style (in HotSpot)
- it breaks HotSpot's style rule about implied booleans


Stefan K is pretty adamant that the original iterator version where we
go from 1-line to 3-lines or 1-line to 5-lines is not acceptable for GC
code. The compiler guys haven't chimed in on this debate so I'm guessing
they don't have a strong opinion either way. One thing that we don't want
to do is have different styles for the different teams.

So I had to make a judgement call on whether I thought Runtime could live
with Stefan's idea. Originally I wasn't fond of it either and breaking the
implied boolean style rule is a problem for me (I post that comment a lot
in my code reviews). However, I have to say that I'm a lot happier about
the compactness of the code and the fact that limited ThreadsListHandle
scope comes for 'free' in most cases.

We're planning to keep the new iterator style for the following reasons:

- smaller change footprint
- consistent style used for all of the teams
- limited ThreadsListHandle scope comes for free in most cases

We're hoping that you can live with this decision (for now) and maybe
even grow to like it in the future. :-)


> Other than that just a couple of comments on variable type choices, 
> and a few typos spotted below.

Replies embedded below.


>
> Thanks,
> David
> ---
>
> src/hotspot/share/runtime/globals.hpp
>
> 2476?? diagnostic(bool, EnableThreadSMRExtraValidityChecks, true, 
> ??????? \
> 2477?????????? "Enable Thread SMR extra validity checks") \
> 2478 ??????? \
> 2479?? diagnostic(bool, EnableThreadSMRStatistics, true, ??????? \
> 2480?????????? "Enable Thread SMR Statistics") ??????? \
>
> Indent of strings is off by? 3 spaces.

Fixed.


> ---
>
> src/hotspot/share/runtime/handshake.cpp
>
> ?140?????? // There is assumption in code that Threads_lock should be 
> lock
> ?200?????? // There is assumption in code that Threads_lock should be 
> lock
>
> lock -> locked

Fixed.


> 146???????? // handshake_process_by_vmthread will check the semaphore 
> for us again
>
> Needs period at end.

Fixed.


> 148???????? // should be okey since the new thread will not have an 
> operation.
>
> okey -> okay

Fixed.


> 151???????? // We can't warn here is since the thread do 
> cancel_handshake after it have been removed
>
> I think:
>
> // We can't warn here since the thread does cancel_handshake after it 
> has been removed

Fixed.


> 152???????? // from ThreadsList. So we should just keep looping here 
> until while below return negative.
>
> from -> from the
>
> Needs period at end.

Fixed both.


>
> ?204???????????? // A new thread on the ThreadsList will not have an 
> operation.
>
> Change period to comma, and ...

Fixed.


> 205???????????? // Hence is skipped in handshake_process_by_vmthread.
>
> Hence is -> hence it is

Fixed.


> ---
>
> src/hotspot/share/runtime/thread.cpp
>
> 1482???? // dcubed - Looks like the Threads_lock is for stable access
> 1483???? // to _jvmci_old_thread_counters and _jvmci_counters.
>
> Does this comment need revisiting?

We've been trying to decide what to do about this comment. We'll be
filing a follow up bug for the Compiler team to take another look at
how _jvmci_old_thread_counters and _jvmci_counters are protected.
Threads_lock is a big hammer and there should be a less expensive
solution. Once that bug gets filed, this comment can go away.


> 3451 volatile jint ...
>
> Why are you using jint for field types here? (Surprised Coleen hasn't 
> spotted them! ;-) ).

volatile jint???????? Threads::_smr_delete_notify = 0;
volatile jint???????? Threads::_smr_deleted_thread_cnt = 0;
volatile jint???????? Threads::_smr_deleted_thread_time_max = 0;
volatile jint???????? Threads::_smr_deleted_thread_times = 0;
:
volatile jint???????? Threads::_smr_tlh_cnt = 0;
volatile jint???????? Threads::_smr_tlh_time_max = 0;
volatile jint???????? Threads::_smr_tlh_times = 0;

_smr_delete_notify is a "flag" that indicates when an _smr_delete_lock
notify() operation is needed. It counts up and down and should be a fairly
small number.

_smr_deleted_thread_cnt counts the number of Threads that have been
deleted over the life of the VM. It's an always increasing number so
it's size depends on how long the VM has been running.

_smr_deleted_thread_time_max is the maximum time in millis it has
taken to delete a thread. This field was originally a jlong, but
IIRC the Atomic::cmpxchg() code was not happy on ARM... (might have
just been Atomic::add() that was not happy)

_smr_deleted_thread_times accumulates the time in millis that it
has taken to delete threads. It's an always increasing number so
it's size depends on how long the VM has been running. This field
was originally a jlong, but IIRC the Atomic::add() code was not
happy on ARM... (might have just been Atomic::cmpxchg() that was
not happy)

_smr_tlh_cnt counts the number of ThreadsListHandles that have been
deleted over the life of the VM. It's an always increasing number so
it's size depends on how long the VM has been running.

_smr_tlh_time_max is the maximum time in millis it has taken to
delete a ThreadsListHandle. This field was originally a jlong, but
IIRC the Atomic::cmpxchg() code was not happy on ARM... (might have
just been Atomic::add() that was not happy)

_smr_tlh_times? accumulates the time in millis that it has taken to
delete ThreadsListHandles. It's an always increasing number so
it's size depends on how long the VM has been running.? This field
was originally a jlong, but IIRC the Atomic::add() code was not
happy on ARM... (might have just been Atomic::cmpxchg() that was
not happy)


It just dawned on me that I'm not sure whether you think the 'jint'
fields should be larger or smaller or the same size but a different
type... :-)

The fact that I had to write so much to explain what these fields
are for and how they're used indicates I need more comments. See
below...


> 3456 long Threads::_smr_java_thread_list_alloc_cnt = 1;
> 3457 long????????????????? Threads::_smr_java_thread_list_free_cnt = 0;
>
> long? If you really want 64-bit counters here you won't get them on 
> Windows with that declaration. If you really want variable sized 
> counters I suggest uintptr_t; otherwise uint64_t.

long????????????????? Threads::_smr_java_thread_list_alloc_cnt = 1;
long????????????????? Threads::_smr_java_thread_list_free_cnt = 0;

_smr_java_thread_list_alloc_cnt counts the number of ThreadsLists that
have been allocated over the life of the VM. It's an always increasing
number so it's size depends on how long the VM has been running and how
many Threads have been created and destroyed.

_smr_java_thread_list_free_cnt counts the number of ThreadsLists that
have been freed over the life of the VM. It's an always increasing
number so it's size depends on how long the VM has been running and how
many Threads have been created and destroyed.

I can't remember why we chose 'long' and I agree it's not a good choice
for Win*.


Okay so it dawns on me that we haven't written down a description for
the various new '_cnt', '_max' and '_times" fields so I'm adding these
comments to thread.hpp:

 ?private:
 ? // Safe Memory Reclamation (SMR) support:
 ? static Monitor*????????????? _smr_delete_lock;
 ? // The '_cnt', '_max' and '_times" fields are enabled via
 ? // -XX:+EnableThreadSMRStatistics:
 ?????????????????????????????? // # of parallel threads in 
_smr_delete_lock->wait().
 ? static uint????????????????? _smr_delete_lock_wait_cnt;
 ?????????????????????????????? // Max # of parallel threads in 
_smr_delete_lock->wait().
 ? static uint????????????????? _smr_delete_lock_wait_max;
 ?????????????????????????????? // Flag to indicate when an 
_smr_delete_lock->notify() is needed.
 ? static volatile uint???????? _smr_delete_notify;
 ?????????????????????????????? // # of threads deleted over VM lifetime.
 ? static volatile uint???????? _smr_deleted_thread_cnt;
 ?????????????????????????????? // Max time in millis to delete a thread.
 ? static volatile uint???????? _smr_deleted_thread_time_max;
 ?????????????????????????????? // Cumulative time in millis to delete 
threads.
 ? static volatile uint???????? _smr_deleted_thread_times;
 ? static ThreadsList* volatile _smr_java_thread_list;
 ? static ThreadsList*????????? get_smr_java_thread_list();
 ? static ThreadsList* xchg_smr_java_thread_list(ThreadsList* new_list);
 ?????????????????????????????? // # of ThreadsLists allocated over VM 
lifetime.
 ? static uint64_t????????????? _smr_java_thread_list_alloc_cnt;
 ?????????????????????????????? // # of ThreadsLists freed over VM lifetime.
 ? static uint64_t????????????? _smr_java_thread_list_free_cnt;
 ?????????????????????????????? // Max size ThreadsList allocated.
 ? static uint????????????????? _smr_java_thread_list_max;
 ?????????????????????????????? // Max # of nested ThreadsLists for a 
thread.
 ? static uint????????????????? _smr_nested_thread_list_max;
 ?????????????????????????????? // # of ThreadsListHandles deleted over 
VM lifetime.
 ? static volatile uint???????? _smr_tlh_cnt;
 ?????????????????????????????? // Max time in millis to delete a 
ThreadsListHandle.
 ? static volatile jint???????? _smr_tlh_time_max;
 ?????????????????????????????? // Cumulative time in millis to delete 
ThreadsListHandles.
 ? static volatile jint???????? _smr_tlh_times;
 ? static ThreadsList*????????? _smr_to_delete_list;
 ?????????????????????????????? // # of parallel ThreadsLists on the 
to-delete list.
 ? static uint????????????????? _smr_to_delete_list_cnt;
 ?????????????????????????????? // Max # of parallel ThreadsLists on the 
to-delete list.
 ? static uint????????????????? _smr_to_delete_list_max;


I've also gone through all the new '_cnt', '_max' and '_times" fields
in thread.cpp and added an impl note to explain the choice of type:

// Safe Memory Reclamation (SMR) support:
Monitor*????????????? Threads::_smr_delete_lock =
 ????????????????????????? new Monitor(Monitor::special, "smr_delete_lock",
 ????????????????????????????????????? false /* allow_vm_block */,
Monitor::_safepoint_check_never);
// The '_cnt', '_max' and '_times" fields are enabled via
// -XX:+EnableThreadSMRStatistics:
 ????????????????????? // # of parallel threads in _smr_delete_lock->wait().
 ????????????????????? // Impl note: Hard to imagine > 64K waiting 
threads so
 ????????????????????? // this could be 16-bit, but there is no nice 16-bit
 ????????????????????? // _FORMAT support.
uint????????????????? Threads::_smr_delete_lock_wait_cnt = 0;
 ????????????????????? // Max # of parallel threads in 
_smr_delete_lock->wait().
 ????????????????????? // Impl note: See _smr_delete_lock_wait_cnt note.
uint????????????????? Threads::_smr_delete_lock_wait_max = 0;
 ????????????????????? // Flag to indicate when an 
_smr_delete_lock->notify() is needed.
 ????????????????????? // Impl note: See _smr_delete_lock_wait_cnt note.
volatile uint???????? Threads::_smr_delete_notify = 0;
 ????????????????????? // # of threads deleted over VM lifetime.
 ????????????????????? // Impl note: Atomically incremented over VM 
lifetime so
 ????????????????????? // use unsigned for more range. Unsigned 64-bit would
 ????????????????????? // be more future proof, but 64-bit atomic inc isn't
 ????????????????????? // available everywhere (or is it?).
volatile uint???????? Threads::_smr_deleted_thread_cnt = 0;
 ????????????????????? // Max time in millis to delete a thread.
 ????????????????????? // Impl note: 16-bit might be too small on an 
overloaded
 ????????????????????? // machine. Use unsigned since this is a time 
value. Set
 ????????????????????? // via Atomic::cmpxchg() in a loop for correctness.
volatile uint???????? Threads::_smr_deleted_thread_time_max = 0;
 ????????????????????? // Cumulative time in millis to delete threads.
 ????????????????????? // Impl note: Atomically added to over VM 
lifetime so use
 ????????????????????? // unsigned for more range. Unsigned 64-bit would 
be more
 ????????????????????? // future proof, but 64-bit atomic inc isn't 
available
 ????????????????????? // everywhere (or is it?).
volatile uint???????? Threads::_smr_deleted_thread_times = 0;
ThreadsList* volatile Threads::_smr_java_thread_list = new ThreadsList(0);
 ????????????????????? // # of ThreadsLists allocated over VM lifetime.
 ????????????????????? // Impl note: We allocate a new ThreadsList for every
 ????????????????????? // thread create and every thread delete so we need a
 ????????????????????? // bigger type than the _smr_deleted_thread_cnt 
field.
uint64_t????????????? Threads::_smr_java_thread_list_alloc_cnt = 1;
 ????????????????????? // # of ThreadsLists freed over VM lifetime.
 ????????????????????? // Impl note: See _smr_java_thread_list_alloc_cnt 
note.
uint64_t????????????? Threads::_smr_java_thread_list_free_cnt = 0;
 ????????????????????? // Max size ThreadsList allocated.
 ????????????????????? // Impl note: Max # of threads alive at one time 
should
 ????????????????????? // fit in unsigned 32-bit.
uint????????????????? Threads::_smr_java_thread_list_max = 0;
 ????????????????????? // Max # of nested ThreadsLists for a thread.
 ????????????????????? // Impl note: Hard to imagine > 64K nested 
ThreadsLists
 ????????????????????? // so his could be 16-bit, but there is no nice 
16-bit
 ????????????????????? // _FORMAT support.
uint????????????????? Threads::_smr_nested_thread_list_max = 0;
 ????????????????????? // # of ThreadsListHandles deleted over VM lifetime.
 ????????????????????? // Impl note: Atomically incremented over VM 
lifetime so
 ????????????????????? // use unsigned for more range. There will be fewer
 ????????????????????? // ThreadsListHandles than threads so unsigned 32-bit
 ????????????????????? // should be fine.
volatile uint???????? Threads::_smr_tlh_cnt = 0;
 ????????????????????? // Max time in millis to delete a ThreadsListHandle.
 ????????????????????? // Impl note: 16-bit might be too small on an 
overloaded
 ????????????????????? // machine. Use unsigned since this is a time 
value. Set
 ????????????????????? // via Atomic::cmpxchg() in a loop for correctness.
volatile uint???????? Threads::_smr_tlh_time_max = 0;
 ????????????????????? // Cumulative time in millis to delete 
ThreadsListHandles.
 ????????????????????? // Impl note: Atomically added to over VM 
lifetime so use
 ????????????????????? // unsigned for more range. Unsigned 64-bit would 
be more
 ????????????????????? // future proof, but 64-bit atomic inc isn't 
available
 ????????????????????? // everywhere (or is it?).
volatile uint???????? Threads::_smr_tlh_times = 0;
ThreadsList*????????? Threads::_smr_to_delete_list = NULL;
 ????????????????????? // # of parallel ThreadsLists on the to-delete list.
 ????????????????????? // Impl note: Hard to imagine > 64K ThreadsLists 
needing
 ????????????????????? // to be deleted so this could be 16-bit, but 
there is no
 ????????????????????? // nice 16-bit _FORMAT support.
uint????????????????? Threads::_smr_to_delete_list_cnt = 0;
 ????????????????????? // Max # of parallel ThreadsLists on the 
to-delete list.
 ????????????????????? // Impl note: See _smr_to_delete_list_cnt note.
uint????????????????? Threads::_smr_to_delete_list_max = 0;


Yikes! With the additional comments, the additions to thread.hpp and
thread.cpp grew by a bunch... I've done a test build build on my Mac
with these changes and I'm about to kick off a Mach5 tier1 job...

Dan



>
> ----
>
> On 19/11/2017 11:49 AM, Daniel D. Daugherty wrote:
>> Greetings,
>>
>> Testing of the last round of changes revealed a hang in one of the new
>> TLH tests. Robbin has fixed the hang, updated the existing TLH test, and
>> added another TLH test for good measure.
>>
>> Here is the updated full webrev:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/
>>
>> Here is the updated delta webrev:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-delta/
>>
>> Dan ran the bits thru the usual Mach5 tier[1-5] testing and there are
>> no unexpected failures.
>>
>> We welcome comments, suggestions and feedback.
>>
>> Dan, Erik and Robbin
>>
>>
>> On 11/15/17 3:32 PM, Daniel D. Daugherty wrote:
>>> Greetings,
>>>
>>> Robbin rebased the project last night/this morning to merge with Thread
>>> Local Handshakes (TLH) and also picked up additional changesets up 
>>> thru:
>>>
>>>> Changeset: fa736014cf28
>>>> Author:??? cjplummer
>>>> Date:????? 2017-11-14 18:08 -0800
>>>> URL:http://hg.openjdk.java.net/jdk/hs/rev/fa736014cf28
>>>>
>>>> 8191049: Add alternate version of pns() that is callable from 
>>>> within hotspot source
>>>> Summary: added pns2() to debug.cpp
>>>> Reviewed-by: stuefe, gthornbr
>>>
>>> This is the first time we've rebased the project to bits that are this
>>> fresh (< 12 hours old at rebase time). We've done this because we think
>>> we're done with this project and are in the final review-change-retest
>>> cycle(s)... In other words, we're not planning on making any more major
>>> changes for this project... :-)
>>>
>>> *** Time for code reviewers to chime in on this thread! ***
>>>
>>> Here is the updated full webrev:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-full/
>>>
>>> Here's is the delta webrev from the 2017.11.10 rebase:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-delta/
>>>
>>> Dan has submitted the bits for the usual Mach5 tier[1-5] testing
>>> (and the new baseline also)...
>>>
>>> We're expecting this round to be a little noisier than usual because
>>> we did not rebase on a PIT snapshot so the new baseline has not been
>>> through Jesper's usual care-and-feeding of integration_blockers, etc.
>>>
>>> We welcome comments, suggestions and feedback.
>>>
>>> Dan, Erik and Robbin
>>>
>>>
>>> On 11/14/17 4:48 PM, Daniel D. Daugherty wrote:
>>>> Greetings,
>>>>
>>>> I rebased the project to the 2017.11.10 jdk/hs PIT snapshot.
>>>> (Yes, we're playing chase-the-repo...)
>>>>
>>>> Here is the updated full webrev:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-07-full/
>>>>
>>>> Unlike the previous rebase, there were no changes required to the
>>>> open code to get the rebased bits to build so there is no delta
>>>> webrev.
>>>>
>>>> These bits have been run through JPRT and I've submitted the usual
>>>> Mach5 tier[1-5] test run...
>>>>
>>>> We welcome comments, suggestions and feedback.
>>>>
>>>> Dan, Erik and Robbin
>>>>
>>>>
>>>> On 11/13/17 12:30 PM, Daniel D. Daugherty wrote:
>>>>> Greetings,
>>>>>
>>>>> I rebased the project to the 2017.10.26 jdk10/hs PIT snapshot.
>>>>>
>>>>> Here are the updated webrevs:
>>>>>
>>>>> Here's the mq comment for the change:
>>>>>
>>>>> ? Rebase to 2017.10.25 PIT snapshot.
>>>>>
>>>>> Here is the full webrev:
>>>>>
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-full/
>>>>>
>>>>> And here is the delta webrev:
>>>>>
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-delta/
>>>>>
>>>>> I ran the above bits throught Mach5 tier[1-5] testing over the 
>>>>> holiday
>>>>> weekend. Didn't see any issues in a quick look. Going to take a 
>>>>> closer
>>>>> look today.
>>>>>
>>>>> We welcome comments, suggestions and feedback.
>>>>>
>>>>> Dan, Erik and Robbin
>>>>>
>>>>>
>>>>> On 11/8/17 1:05 PM, Daniel D. Daugherty wrote:
>>>>>> Greetings,
>>>>>>
>>>>>> Resolving one of the code review comments (from both Stefan K and 
>>>>>> Coleen)
>>>>>> on jdk10-04-full required quite a few changes so it is being done 
>>>>>> as a
>>>>>> standalone patch and corresponding webrevs:
>>>>>>
>>>>>> Here's the mq comment for the change:
>>>>>>
>>>>>> ? stefank, coleenp CR - refactor most JavaThreadIterator usage to 
>>>>>> use
>>>>>> ??? JavaThreadIteratorWithHandle.
>>>>>>
>>>>>> Here is the full webrev:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-full/
>>>>>>
>>>>>> And here is the delta webrev:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-delta/
>>>>>>
>>>>>> We welcome comments, suggestions and feedback.
>>>>>>
>>>>>> Dan, Erik and Robbin
>>>>>>
>>>>>>
>>>>>> On 10/9/17 3:41 PM, Daniel D. Daugherty wrote:
>>>>>>> Greetings,
>>>>>>>
>>>>>>> We have a (eXtra Large) fix for the following bug:
>>>>>>>
>>>>>>> 8167108 inconsistent handling of SR_lock can lead to crashes
>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8167108
>>>>>>>
>>>>>>> This fix adds a Safe Memory Reclamation (SMR) mechanism based on
>>>>>>> Hazard Pointers to manage JavaThread lifecycle.
>>>>>>>
>>>>>>> Here's a PDF for the internal wiki that we've been using to 
>>>>>>> describe
>>>>>>> and track the work on this project:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/SMR_and_JavaThread_Lifecycle-JDK10-04.pdf 
>>>>>>>
>>>>>>>
>>>>>>> Dan has noticed that the indenting is wrong in some of the code 
>>>>>>> quotes
>>>>>>> in the PDF that are not present in the internal wiki. We don't 
>>>>>>> have a
>>>>>>> solution for that problem yet.
>>>>>>>
>>>>>>> Here's the webrev for current JDK10 version of this fix:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-04-full
>>>>>>>
>>>>>>> This fix has been run through many rounds of JPRT and Mach5 
>>>>>>> tier[2-5]
>>>>>>> testing, additional stress testing on Dan's Solaris X64 server, and
>>>>>>> additional testing on Erik and Robbin's machines.
>>>>>>>
>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>
>>>>>>> Daniel Daugherty
>>>>>>> Erik Osterlund
>>>>>>> Robbin Ehn
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>


From david.holmes at oracle.com  Tue Nov 21 02:13:40 2017
From: david.holmes at oracle.com (David Holmes)
Date: Tue, 21 Nov 2017 12:13:40 +1000
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <92d1f572-a745-6f2b-1b5f-e68a1d252bab@oracle.com>
References: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
 <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>
 <836b5c6c-5eef-9245-c16a-e71372093d9a@oracle.com>
 <82d0e473-a24f-9feb-82e9-7d04c4d7a91d@oracle.com>
 <354f7ece-66bc-c065-9ee8-f55abcc1c23e@oracle.com>
 <0ff8df8e-e93c-02d2-cb97-b13b1ac0f53f@oracle.com>
 <43d2f342-3f4b-5c1a-a687-c24c4efedba9@oracle.com>
 <92d1f572-a745-6f2b-1b5f-e68a1d252bab@oracle.com>
Message-ID: <fb046f8a-22e6-2be4-6fb0-319b51817958@oracle.com>

Hi Dan,

Just to be clear my comment about use of jint's was not about expected 
size but the fact you were using a j-type instead of a C++ type when the 
field is not a Java field. (Coleen has been on a crusade to remove 
j-types from where they don't belong - and they were removed from the 
Atomic API as part of that recent templatization effort.)

No further comments. :)

Thanks,
David

On 21/11/2017 11:50 AM, Daniel D. Daugherty wrote:
> On 11/20/17 12:51 AM, David Holmes wrote:
>> Hi Dan,
>>
>> Figured I'd better cast an eye over this again before it gets pushed :)
> 
> Thanks for reviewing again!!
> 
> 
>> Only one thing (repeated many times) jumped out at me and apologies if 
>> this has already been debated back and forth. I missed the exchange 
>> where the for loop iteration was rewritten into this unusual form:
>>
>> for (JavaThreadIteratorWithHandle jtiwh; JavaThread *jt = 
>> jtiwh.next(); ) {
>>
>> On first reading I expect most people would mistake the control 
>> expression for the iteration-expression due to the next(). I didn't 
>> even know the control expression could introduce a local variable! I 
>> find this really hard to read stylistically and far too cute/clever 
>> for its own good. It also violates the style rules regarding implicit 
>> boolean checks.
>>
>> I know Stefan proposed this as he did not like the alternate (in a few 
>> places):
>>
>> +? {
>> +??? ThreadsListHandle tlh;
>> +??? JavaThreadIterator jti(tlh.list());
>> +??? for (JavaThread* t = jti.first(); t != NULL; t = jti.next()) {
>> ...
>> +? }
>>
>> but it seems to me the iterator code has evolved since then and only a 
>> couple of places need a distinct TLH separate from the actual iterator 
>> object. So I'm more in favour of the more classic for-loop, with the 
>> iterator declared before the loop. Or even convert to while loops, as 
>> this iterator seems more suited to that.
> 
> Actually both Coleen and Stefan had a problem with how much additional
> code was added to support an iterator model. In some cases we went from
> 1-line to 3-lines and in some cases we went from 1-line to 5-lines. For
> Coleen, she wanted 2 of the new lines compressed into 1 new line by using
> an iterator with an embedded ThreadsListHandle. Stefan wanted us to try
> and get back to 1-line where possible.
> 
> The advantages of the new style are:
> 
> - 1-line to 1-line in all but a few cases
> - automatic limited scope of the embedded ThreadsListHandle so we were
>  ? able to remove extra braces that were added earlier in most cases
> 
> The disadvantages of the new style are:
> 
> - it is an unusual for-loop style (in HotSpot)
> - it breaks HotSpot's style rule about implied booleans
> 
> 
> Stefan K is pretty adamant that the original iterator version where we
> go from 1-line to 3-lines or 1-line to 5-lines is not acceptable for GC
> code. The compiler guys haven't chimed in on this debate so I'm guessing
> they don't have a strong opinion either way. One thing that we don't want
> to do is have different styles for the different teams.
> 
> So I had to make a judgement call on whether I thought Runtime could live
> with Stefan's idea. Originally I wasn't fond of it either and breaking the
> implied boolean style rule is a problem for me (I post that comment a lot
> in my code reviews). However, I have to say that I'm a lot happier about
> the compactness of the code and the fact that limited ThreadsListHandle
> scope comes for 'free' in most cases.
> 
> We're planning to keep the new iterator style for the following reasons:
> 
> - smaller change footprint
> - consistent style used for all of the teams
> - limited ThreadsListHandle scope comes for free in most cases
> 
> We're hoping that you can live with this decision (for now) and maybe
> even grow to like it in the future. :-)
> 
> 
>> Other than that just a couple of comments on variable type choices, 
>> and a few typos spotted below.
> 
> Replies embedded below.
> 
> 
>>
>> Thanks,
>> David
>> ---
>>
>> src/hotspot/share/runtime/globals.hpp
>>
>> 2476?? diagnostic(bool, EnableThreadSMRExtraValidityChecks, true, 
>> ??????? \
>> 2477?????????? "Enable Thread SMR extra validity checks") \
>> 2478 ??????? \
>> 2479?? diagnostic(bool, EnableThreadSMRStatistics, true, ??????? \
>> 2480?????????? "Enable Thread SMR Statistics") ??????? \
>>
>> Indent of strings is off by? 3 spaces.
> 
> Fixed.
> 
> 
>> ---
>>
>> src/hotspot/share/runtime/handshake.cpp
>>
>> ?140?????? // There is assumption in code that Threads_lock should be 
>> lock
>> ?200?????? // There is assumption in code that Threads_lock should be 
>> lock
>>
>> lock -> locked
> 
> Fixed.
> 
> 
>> 146???????? // handshake_process_by_vmthread will check the semaphore 
>> for us again
>>
>> Needs period at end.
> 
> Fixed.
> 
> 
>> 148???????? // should be okey since the new thread will not have an 
>> operation.
>>
>> okey -> okay
> 
> Fixed.
> 
> 
>> 151???????? // We can't warn here is since the thread do 
>> cancel_handshake after it have been removed
>>
>> I think:
>>
>> // We can't warn here since the thread does cancel_handshake after it 
>> has been removed
> 
> Fixed.
> 
> 
>> 152???????? // from ThreadsList. So we should just keep looping here 
>> until while below return negative.
>>
>> from -> from the
>>
>> Needs period at end.
> 
> Fixed both.
> 
> 
>>
>> ?204???????????? // A new thread on the ThreadsList will not have an 
>> operation.
>>
>> Change period to comma, and ...
> 
> Fixed.
> 
> 
>> 205???????????? // Hence is skipped in handshake_process_by_vmthread.
>>
>> Hence is -> hence it is
> 
> Fixed.
> 
> 
>> ---
>>
>> src/hotspot/share/runtime/thread.cpp
>>
>> 1482???? // dcubed - Looks like the Threads_lock is for stable access
>> 1483???? // to _jvmci_old_thread_counters and _jvmci_counters.
>>
>> Does this comment need revisiting?
> 
> We've been trying to decide what to do about this comment. We'll be
> filing a follow up bug for the Compiler team to take another look at
> how _jvmci_old_thread_counters and _jvmci_counters are protected.
> Threads_lock is a big hammer and there should be a less expensive
> solution. Once that bug gets filed, this comment can go away.
> 
> 
>> 3451 volatile jint ...
>>
>> Why are you using jint for field types here? (Surprised Coleen hasn't 
>> spotted them! ;-) ).
> 
> volatile jint???????? Threads::_smr_delete_notify = 0;
> volatile jint???????? Threads::_smr_deleted_thread_cnt = 0;
> volatile jint???????? Threads::_smr_deleted_thread_time_max = 0;
> volatile jint???????? Threads::_smr_deleted_thread_times = 0;
> :
> volatile jint???????? Threads::_smr_tlh_cnt = 0;
> volatile jint???????? Threads::_smr_tlh_time_max = 0;
> volatile jint???????? Threads::_smr_tlh_times = 0;
> 
> _smr_delete_notify is a "flag" that indicates when an _smr_delete_lock
> notify() operation is needed. It counts up and down and should be a fairly
> small number.
> 
> _smr_deleted_thread_cnt counts the number of Threads that have been
> deleted over the life of the VM. It's an always increasing number so
> it's size depends on how long the VM has been running.
> 
> _smr_deleted_thread_time_max is the maximum time in millis it has
> taken to delete a thread. This field was originally a jlong, but
> IIRC the Atomic::cmpxchg() code was not happy on ARM... (might have
> just been Atomic::add() that was not happy)
> 
> _smr_deleted_thread_times accumulates the time in millis that it
> has taken to delete threads. It's an always increasing number so
> it's size depends on how long the VM has been running. This field
> was originally a jlong, but IIRC the Atomic::add() code was not
> happy on ARM... (might have just been Atomic::cmpxchg() that was
> not happy)
> 
> _smr_tlh_cnt counts the number of ThreadsListHandles that have been
> deleted over the life of the VM. It's an always increasing number so
> it's size depends on how long the VM has been running.
> 
> _smr_tlh_time_max is the maximum time in millis it has taken to
> delete a ThreadsListHandle. This field was originally a jlong, but
> IIRC the Atomic::cmpxchg() code was not happy on ARM... (might have
> just been Atomic::add() that was not happy)
> 
> _smr_tlh_times? accumulates the time in millis that it has taken to
> delete ThreadsListHandles. It's an always increasing number so
> it's size depends on how long the VM has been running.? This field
> was originally a jlong, but IIRC the Atomic::add() code was not
> happy on ARM... (might have just been Atomic::cmpxchg() that was
> not happy)
> 
> 
> It just dawned on me that I'm not sure whether you think the 'jint'
> fields should be larger or smaller or the same size but a different
> type... :-)
> 
> The fact that I had to write so much to explain what these fields
> are for and how they're used indicates I need more comments. See
> below...
> 
> 
>> 3456 long Threads::_smr_java_thread_list_alloc_cnt = 1;
>> 3457 long????????????????? Threads::_smr_java_thread_list_free_cnt = 0;
>>
>> long? If you really want 64-bit counters here you won't get them on 
>> Windows with that declaration. If you really want variable sized 
>> counters I suggest uintptr_t; otherwise uint64_t.
> 
> long????????????????? Threads::_smr_java_thread_list_alloc_cnt = 1;
> long????????????????? Threads::_smr_java_thread_list_free_cnt = 0;
> 
> _smr_java_thread_list_alloc_cnt counts the number of ThreadsLists that
> have been allocated over the life of the VM. It's an always increasing
> number so it's size depends on how long the VM has been running and how
> many Threads have been created and destroyed.
> 
> _smr_java_thread_list_free_cnt counts the number of ThreadsLists that
> have been freed over the life of the VM. It's an always increasing
> number so it's size depends on how long the VM has been running and how
> many Threads have been created and destroyed.
> 
> I can't remember why we chose 'long' and I agree it's not a good choice
> for Win*.
> 
> 
> Okay so it dawns on me that we haven't written down a description for
> the various new '_cnt', '_max' and '_times" fields so I'm adding these
> comments to thread.hpp:
> 
>  ?private:
>  ? // Safe Memory Reclamation (SMR) support:
>  ? static Monitor*????????????? _smr_delete_lock;
>  ? // The '_cnt', '_max' and '_times" fields are enabled via
>  ? // -XX:+EnableThreadSMRStatistics:
>  ?????????????????????????????? // # of parallel threads in 
> _smr_delete_lock->wait().
>  ? static uint????????????????? _smr_delete_lock_wait_cnt;
>  ?????????????????????????????? // Max # of parallel threads in 
> _smr_delete_lock->wait().
>  ? static uint????????????????? _smr_delete_lock_wait_max;
>  ?????????????????????????????? // Flag to indicate when an 
> _smr_delete_lock->notify() is needed.
>  ? static volatile uint???????? _smr_delete_notify;
>  ?????????????????????????????? // # of threads deleted over VM lifetime.
>  ? static volatile uint???????? _smr_deleted_thread_cnt;
>  ?????????????????????????????? // Max time in millis to delete a thread.
>  ? static volatile uint???????? _smr_deleted_thread_time_max;
>  ?????????????????????????????? // Cumulative time in millis to delete 
> threads.
>  ? static volatile uint???????? _smr_deleted_thread_times;
>  ? static ThreadsList* volatile _smr_java_thread_list;
>  ? static ThreadsList*????????? get_smr_java_thread_list();
>  ? static ThreadsList* xchg_smr_java_thread_list(ThreadsList* new_list);
>  ?????????????????????????????? // # of ThreadsLists allocated over VM 
> lifetime.
>  ? static uint64_t????????????? _smr_java_thread_list_alloc_cnt;
>  ?????????????????????????????? // # of ThreadsLists freed over VM 
> lifetime.
>  ? static uint64_t????????????? _smr_java_thread_list_free_cnt;
>  ?????????????????????????????? // Max size ThreadsList allocated.
>  ? static uint????????????????? _smr_java_thread_list_max;
>  ?????????????????????????????? // Max # of nested ThreadsLists for a 
> thread.
>  ? static uint????????????????? _smr_nested_thread_list_max;
>  ?????????????????????????????? // # of ThreadsListHandles deleted over 
> VM lifetime.
>  ? static volatile uint???????? _smr_tlh_cnt;
>  ?????????????????????????????? // Max time in millis to delete a 
> ThreadsListHandle.
>  ? static volatile jint???????? _smr_tlh_time_max;
>  ?????????????????????????????? // Cumulative time in millis to delete 
> ThreadsListHandles.
>  ? static volatile jint???????? _smr_tlh_times;
>  ? static ThreadsList*????????? _smr_to_delete_list;
>  ?????????????????????????????? // # of parallel ThreadsLists on the 
> to-delete list.
>  ? static uint????????????????? _smr_to_delete_list_cnt;
>  ?????????????????????????????? // Max # of parallel ThreadsLists on the 
> to-delete list.
>  ? static uint????????????????? _smr_to_delete_list_max;
> 
> 
> I've also gone through all the new '_cnt', '_max' and '_times" fields
> in thread.cpp and added an impl note to explain the choice of type:
> 
> // Safe Memory Reclamation (SMR) support:
> Monitor*????????????? Threads::_smr_delete_lock =
>  ????????????????????????? new Monitor(Monitor::special, "smr_delete_lock",
>  ????????????????????????????????????? false /* allow_vm_block */,
> Monitor::_safepoint_check_never);
> // The '_cnt', '_max' and '_times" fields are enabled via
> // -XX:+EnableThreadSMRStatistics:
>  ????????????????????? // # of parallel threads in 
> _smr_delete_lock->wait().
>  ????????????????????? // Impl note: Hard to imagine > 64K waiting 
> threads so
>  ????????????????????? // this could be 16-bit, but there is no nice 16-bit
>  ????????????????????? // _FORMAT support.
> uint????????????????? Threads::_smr_delete_lock_wait_cnt = 0;
>  ????????????????????? // Max # of parallel threads in 
> _smr_delete_lock->wait().
>  ????????????????????? // Impl note: See _smr_delete_lock_wait_cnt note.
> uint????????????????? Threads::_smr_delete_lock_wait_max = 0;
>  ????????????????????? // Flag to indicate when an 
> _smr_delete_lock->notify() is needed.
>  ????????????????????? // Impl note: See _smr_delete_lock_wait_cnt note.
> volatile uint???????? Threads::_smr_delete_notify = 0;
>  ????????????????????? // # of threads deleted over VM lifetime.
>  ????????????????????? // Impl note: Atomically incremented over VM 
> lifetime so
>  ????????????????????? // use unsigned for more range. Unsigned 64-bit 
> would
>  ????????????????????? // be more future proof, but 64-bit atomic inc isn't
>  ????????????????????? // available everywhere (or is it?).
> volatile uint???????? Threads::_smr_deleted_thread_cnt = 0;
>  ????????????????????? // Max time in millis to delete a thread.
>  ????????????????????? // Impl note: 16-bit might be too small on an 
> overloaded
>  ????????????????????? // machine. Use unsigned since this is a time 
> value. Set
>  ????????????????????? // via Atomic::cmpxchg() in a loop for correctness.
> volatile uint???????? Threads::_smr_deleted_thread_time_max = 0;
>  ????????????????????? // Cumulative time in millis to delete threads.
>  ????????????????????? // Impl note: Atomically added to over VM 
> lifetime so use
>  ????????????????????? // unsigned for more range. Unsigned 64-bit would 
> be more
>  ????????????????????? // future proof, but 64-bit atomic inc isn't 
> available
>  ????????????????????? // everywhere (or is it?).
> volatile uint???????? Threads::_smr_deleted_thread_times = 0;
> ThreadsList* volatile Threads::_smr_java_thread_list = new ThreadsList(0);
>  ????????????????????? // # of ThreadsLists allocated over VM lifetime.
>  ????????????????????? // Impl note: We allocate a new ThreadsList for 
> every
>  ????????????????????? // thread create and every thread delete so we 
> need a
>  ????????????????????? // bigger type than the _smr_deleted_thread_cnt 
> field.
> uint64_t????????????? Threads::_smr_java_thread_list_alloc_cnt = 1;
>  ????????????????????? // # of ThreadsLists freed over VM lifetime.
>  ????????????????????? // Impl note: See _smr_java_thread_list_alloc_cnt 
> note.
> uint64_t????????????? Threads::_smr_java_thread_list_free_cnt = 0;
>  ????????????????????? // Max size ThreadsList allocated.
>  ????????????????????? // Impl note: Max # of threads alive at one time 
> should
>  ????????????????????? // fit in unsigned 32-bit.
> uint????????????????? Threads::_smr_java_thread_list_max = 0;
>  ????????????????????? // Max # of nested ThreadsLists for a thread.
>  ????????????????????? // Impl note: Hard to imagine > 64K nested 
> ThreadsLists
>  ????????????????????? // so his could be 16-bit, but there is no nice 
> 16-bit
>  ????????????????????? // _FORMAT support.
> uint????????????????? Threads::_smr_nested_thread_list_max = 0;
>  ????????????????????? // # of ThreadsListHandles deleted over VM lifetime.
>  ????????????????????? // Impl note: Atomically incremented over VM 
> lifetime so
>  ????????????????????? // use unsigned for more range. There will be fewer
>  ????????????????????? // ThreadsListHandles than threads so unsigned 
> 32-bit
>  ????????????????????? // should be fine.
> volatile uint???????? Threads::_smr_tlh_cnt = 0;
>  ????????????????????? // Max time in millis to delete a ThreadsListHandle.
>  ????????????????????? // Impl note: 16-bit might be too small on an 
> overloaded
>  ????????????????????? // machine. Use unsigned since this is a time 
> value. Set
>  ????????????????????? // via Atomic::cmpxchg() in a loop for correctness.
> volatile uint???????? Threads::_smr_tlh_time_max = 0;
>  ????????????????????? // Cumulative time in millis to delete 
> ThreadsListHandles.
>  ????????????????????? // Impl note: Atomically added to over VM 
> lifetime so use
>  ????????????????????? // unsigned for more range. Unsigned 64-bit would 
> be more
>  ????????????????????? // future proof, but 64-bit atomic inc isn't 
> available
>  ????????????????????? // everywhere (or is it?).
> volatile uint???????? Threads::_smr_tlh_times = 0;
> ThreadsList*????????? Threads::_smr_to_delete_list = NULL;
>  ????????????????????? // # of parallel ThreadsLists on the to-delete list.
>  ????????????????????? // Impl note: Hard to imagine > 64K ThreadsLists 
> needing
>  ????????????????????? // to be deleted so this could be 16-bit, but 
> there is no
>  ????????????????????? // nice 16-bit _FORMAT support.
> uint????????????????? Threads::_smr_to_delete_list_cnt = 0;
>  ????????????????????? // Max # of parallel ThreadsLists on the 
> to-delete list.
>  ????????????????????? // Impl note: See _smr_to_delete_list_cnt note.
> uint????????????????? Threads::_smr_to_delete_list_max = 0;
> 
> 
> Yikes! With the additional comments, the additions to thread.hpp and
> thread.cpp grew by a bunch... I've done a test build build on my Mac
> with these changes and I'm about to kick off a Mach5 tier1 job...
> 
> Dan
> 
> 
> 
>>
>> ----
>>
>> On 19/11/2017 11:49 AM, Daniel D. Daugherty wrote:
>>> Greetings,
>>>
>>> Testing of the last round of changes revealed a hang in one of the new
>>> TLH tests. Robbin has fixed the hang, updated the existing TLH test, and
>>> added another TLH test for good measure.
>>>
>>> Here is the updated full webrev:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/
>>>
>>> Here is the updated delta webrev:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-delta/
>>>
>>> Dan ran the bits thru the usual Mach5 tier[1-5] testing and there are
>>> no unexpected failures.
>>>
>>> We welcome comments, suggestions and feedback.
>>>
>>> Dan, Erik and Robbin
>>>
>>>
>>> On 11/15/17 3:32 PM, Daniel D. Daugherty wrote:
>>>> Greetings,
>>>>
>>>> Robbin rebased the project last night/this morning to merge with Thread
>>>> Local Handshakes (TLH) and also picked up additional changesets up 
>>>> thru:
>>>>
>>>>> Changeset: fa736014cf28
>>>>> Author:??? cjplummer
>>>>> Date:????? 2017-11-14 18:08 -0800
>>>>> URL:http://hg.openjdk.java.net/jdk/hs/rev/fa736014cf28
>>>>>
>>>>> 8191049: Add alternate version of pns() that is callable from 
>>>>> within hotspot source
>>>>> Summary: added pns2() to debug.cpp
>>>>> Reviewed-by: stuefe, gthornbr
>>>>
>>>> This is the first time we've rebased the project to bits that are this
>>>> fresh (< 12 hours old at rebase time). We've done this because we think
>>>> we're done with this project and are in the final review-change-retest
>>>> cycle(s)... In other words, we're not planning on making any more major
>>>> changes for this project... :-)
>>>>
>>>> *** Time for code reviewers to chime in on this thread! ***
>>>>
>>>> Here is the updated full webrev:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-full/
>>>>
>>>> Here's is the delta webrev from the 2017.11.10 rebase:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-delta/
>>>>
>>>> Dan has submitted the bits for the usual Mach5 tier[1-5] testing
>>>> (and the new baseline also)...
>>>>
>>>> We're expecting this round to be a little noisier than usual because
>>>> we did not rebase on a PIT snapshot so the new baseline has not been
>>>> through Jesper's usual care-and-feeding of integration_blockers, etc.
>>>>
>>>> We welcome comments, suggestions and feedback.
>>>>
>>>> Dan, Erik and Robbin
>>>>
>>>>
>>>> On 11/14/17 4:48 PM, Daniel D. Daugherty wrote:
>>>>> Greetings,
>>>>>
>>>>> I rebased the project to the 2017.11.10 jdk/hs PIT snapshot.
>>>>> (Yes, we're playing chase-the-repo...)
>>>>>
>>>>> Here is the updated full webrev:
>>>>>
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-07-full/
>>>>>
>>>>> Unlike the previous rebase, there were no changes required to the
>>>>> open code to get the rebased bits to build so there is no delta
>>>>> webrev.
>>>>>
>>>>> These bits have been run through JPRT and I've submitted the usual
>>>>> Mach5 tier[1-5] test run...
>>>>>
>>>>> We welcome comments, suggestions and feedback.
>>>>>
>>>>> Dan, Erik and Robbin
>>>>>
>>>>>
>>>>> On 11/13/17 12:30 PM, Daniel D. Daugherty wrote:
>>>>>> Greetings,
>>>>>>
>>>>>> I rebased the project to the 2017.10.26 jdk10/hs PIT snapshot.
>>>>>>
>>>>>> Here are the updated webrevs:
>>>>>>
>>>>>> Here's the mq comment for the change:
>>>>>>
>>>>>> ? Rebase to 2017.10.25 PIT snapshot.
>>>>>>
>>>>>> Here is the full webrev:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-full/
>>>>>>
>>>>>> And here is the delta webrev:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-delta/
>>>>>>
>>>>>> I ran the above bits throught Mach5 tier[1-5] testing over the 
>>>>>> holiday
>>>>>> weekend. Didn't see any issues in a quick look. Going to take a 
>>>>>> closer
>>>>>> look today.
>>>>>>
>>>>>> We welcome comments, suggestions and feedback.
>>>>>>
>>>>>> Dan, Erik and Robbin
>>>>>>
>>>>>>
>>>>>> On 11/8/17 1:05 PM, Daniel D. Daugherty wrote:
>>>>>>> Greetings,
>>>>>>>
>>>>>>> Resolving one of the code review comments (from both Stefan K and 
>>>>>>> Coleen)
>>>>>>> on jdk10-04-full required quite a few changes so it is being done 
>>>>>>> as a
>>>>>>> standalone patch and corresponding webrevs:
>>>>>>>
>>>>>>> Here's the mq comment for the change:
>>>>>>>
>>>>>>> ? stefank, coleenp CR - refactor most JavaThreadIterator usage to 
>>>>>>> use
>>>>>>> ??? JavaThreadIteratorWithHandle.
>>>>>>>
>>>>>>> Here is the full webrev:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-full/
>>>>>>>
>>>>>>> And here is the delta webrev:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-delta/
>>>>>>>
>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>
>>>>>>> Dan, Erik and Robbin
>>>>>>>
>>>>>>>
>>>>>>> On 10/9/17 3:41 PM, Daniel D. Daugherty wrote:
>>>>>>>> Greetings,
>>>>>>>>
>>>>>>>> We have a (eXtra Large) fix for the following bug:
>>>>>>>>
>>>>>>>> 8167108 inconsistent handling of SR_lock can lead to crashes
>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8167108
>>>>>>>>
>>>>>>>> This fix adds a Safe Memory Reclamation (SMR) mechanism based on
>>>>>>>> Hazard Pointers to manage JavaThread lifecycle.
>>>>>>>>
>>>>>>>> Here's a PDF for the internal wiki that we've been using to 
>>>>>>>> describe
>>>>>>>> and track the work on this project:
>>>>>>>>
>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/SMR_and_JavaThread_Lifecycle-JDK10-04.pdf 
>>>>>>>>
>>>>>>>>
>>>>>>>> Dan has noticed that the indenting is wrong in some of the code 
>>>>>>>> quotes
>>>>>>>> in the PDF that are not present in the internal wiki. We don't 
>>>>>>>> have a
>>>>>>>> solution for that problem yet.
>>>>>>>>
>>>>>>>> Here's the webrev for current JDK10 version of this fix:
>>>>>>>>
>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-04-full
>>>>>>>>
>>>>>>>> This fix has been run through many rounds of JPRT and Mach5 
>>>>>>>> tier[2-5]
>>>>>>>> testing, additional stress testing on Dan's Solaris X64 server, and
>>>>>>>> additional testing on Erik and Robbin's machines.
>>>>>>>>
>>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>>
>>>>>>>> Daniel Daugherty
>>>>>>>> Erik Osterlund
>>>>>>>> Robbin Ehn
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
> 

From daniel.daugherty at oracle.com  Tue Nov 21 03:51:16 2017
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Mon, 20 Nov 2017 22:51:16 -0500
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <fb046f8a-22e6-2be4-6fb0-319b51817958@oracle.com>
References: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
 <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>
 <836b5c6c-5eef-9245-c16a-e71372093d9a@oracle.com>
 <82d0e473-a24f-9feb-82e9-7d04c4d7a91d@oracle.com>
 <354f7ece-66bc-c065-9ee8-f55abcc1c23e@oracle.com>
 <0ff8df8e-e93c-02d2-cb97-b13b1ac0f53f@oracle.com>
 <43d2f342-3f4b-5c1a-a687-c24c4efedba9@oracle.com>
 <92d1f572-a745-6f2b-1b5f-e68a1d252bab@oracle.com>
 <fb046f8a-22e6-2be4-6fb0-319b51817958@oracle.com>
Message-ID: <0fc655c8-fffe-03cc-0f7b-a34a9893923d@oracle.com>

On 11/20/17 9:13 PM, David Holmes wrote:
> Hi Dan,
>
> Just to be clear my comment about use of jint's was not about expected 
> size but the fact you were using a j-type instead of a C++ type when 
> the field is not a Java field. (Coleen has been on a crusade to remove 
> j-types from where they don't belong - and they were removed from the 
> Atomic API as part of that recent templatization effort.)

Thanks for making that clear. I think I've gotten rid of all
the jint's at this point...


> No further comments. :)

Thanks. I'll be sending out more webrevs when I get through all
the code review comments...

Dan


>
> Thanks,
> David
>
> On 21/11/2017 11:50 AM, Daniel D. Daugherty wrote:
>> On 11/20/17 12:51 AM, David Holmes wrote:
>>> Hi Dan,
>>>
>>> Figured I'd better cast an eye over this again before it gets pushed :)
>>
>> Thanks for reviewing again!!
>>
>>
>>> Only one thing (repeated many times) jumped out at me and apologies 
>>> if this has already been debated back and forth. I missed the 
>>> exchange where the for loop iteration was rewritten into this 
>>> unusual form:
>>>
>>> for (JavaThreadIteratorWithHandle jtiwh; JavaThread *jt = 
>>> jtiwh.next(); ) {
>>>
>>> On first reading I expect most people would mistake the control 
>>> expression for the iteration-expression due to the next(). I didn't 
>>> even know the control expression could introduce a local variable! I 
>>> find this really hard to read stylistically and far too cute/clever 
>>> for its own good. It also violates the style rules regarding 
>>> implicit boolean checks.
>>>
>>> I know Stefan proposed this as he did not like the alternate (in a 
>>> few places):
>>>
>>> +? {
>>> +??? ThreadsListHandle tlh;
>>> +??? JavaThreadIterator jti(tlh.list());
>>> +??? for (JavaThread* t = jti.first(); t != NULL; t = jti.next()) {
>>> ...
>>> +? }
>>>
>>> but it seems to me the iterator code has evolved since then and only 
>>> a couple of places need a distinct TLH separate from the actual 
>>> iterator object. So I'm more in favour of the more classic for-loop, 
>>> with the iterator declared before the loop. Or even convert to while 
>>> loops, as this iterator seems more suited to that.
>>
>> Actually both Coleen and Stefan had a problem with how much additional
>> code was added to support an iterator model. In some cases we went from
>> 1-line to 3-lines and in some cases we went from 1-line to 5-lines. For
>> Coleen, she wanted 2 of the new lines compressed into 1 new line by 
>> using
>> an iterator with an embedded ThreadsListHandle. Stefan wanted us to try
>> and get back to 1-line where possible.
>>
>> The advantages of the new style are:
>>
>> - 1-line to 1-line in all but a few cases
>> - automatic limited scope of the embedded ThreadsListHandle so we were
>> ?? able to remove extra braces that were added earlier in most cases
>>
>> The disadvantages of the new style are:
>>
>> - it is an unusual for-loop style (in HotSpot)
>> - it breaks HotSpot's style rule about implied booleans
>>
>>
>> Stefan K is pretty adamant that the original iterator version where we
>> go from 1-line to 3-lines or 1-line to 5-lines is not acceptable for GC
>> code. The compiler guys haven't chimed in on this debate so I'm guessing
>> they don't have a strong opinion either way. One thing that we don't 
>> want
>> to do is have different styles for the different teams.
>>
>> So I had to make a judgement call on whether I thought Runtime could 
>> live
>> with Stefan's idea. Originally I wasn't fond of it either and 
>> breaking the
>> implied boolean style rule is a problem for me (I post that comment a 
>> lot
>> in my code reviews). However, I have to say that I'm a lot happier about
>> the compactness of the code and the fact that limited ThreadsListHandle
>> scope comes for 'free' in most cases.
>>
>> We're planning to keep the new iterator style for the following reasons:
>>
>> - smaller change footprint
>> - consistent style used for all of the teams
>> - limited ThreadsListHandle scope comes for free in most cases
>>
>> We're hoping that you can live with this decision (for now) and maybe
>> even grow to like it in the future. :-)
>>
>>
>>> Other than that just a couple of comments on variable type choices, 
>>> and a few typos spotted below.
>>
>> Replies embedded below.
>>
>>
>>>
>>> Thanks,
>>> David
>>> ---
>>>
>>> src/hotspot/share/runtime/globals.hpp
>>>
>>> 2476?? diagnostic(bool, EnableThreadSMRExtraValidityChecks, true, 
>>> ??????? \
>>> 2477?????????? "Enable Thread SMR extra validity checks") \
>>> 2478 ??????? \
>>> 2479?? diagnostic(bool, EnableThreadSMRStatistics, true, ??????? \
>>> 2480?????????? "Enable Thread SMR Statistics") ??????? \
>>>
>>> Indent of strings is off by? 3 spaces.
>>
>> Fixed.
>>
>>
>>> ---
>>>
>>> src/hotspot/share/runtime/handshake.cpp
>>>
>>> ?140?????? // There is assumption in code that Threads_lock should 
>>> be lock
>>> ?200?????? // There is assumption in code that Threads_lock should 
>>> be lock
>>>
>>> lock -> locked
>>
>> Fixed.
>>
>>
>>> 146???????? // handshake_process_by_vmthread will check the 
>>> semaphore for us again
>>>
>>> Needs period at end.
>>
>> Fixed.
>>
>>
>>> 148???????? // should be okey since the new thread will not have an 
>>> operation.
>>>
>>> okey -> okay
>>
>> Fixed.
>>
>>
>>> 151???????? // We can't warn here is since the thread do 
>>> cancel_handshake after it have been removed
>>>
>>> I think:
>>>
>>> // We can't warn here since the thread does cancel_handshake after 
>>> it has been removed
>>
>> Fixed.
>>
>>
>>> 152???????? // from ThreadsList. So we should just keep looping here 
>>> until while below return negative.
>>>
>>> from -> from the
>>>
>>> Needs period at end.
>>
>> Fixed both.
>>
>>
>>>
>>> ?204???????????? // A new thread on the ThreadsList will not have an 
>>> operation.
>>>
>>> Change period to comma, and ...
>>
>> Fixed.
>>
>>
>>> 205???????????? // Hence is skipped in handshake_process_by_vmthread.
>>>
>>> Hence is -> hence it is
>>
>> Fixed.
>>
>>
>>> ---
>>>
>>> src/hotspot/share/runtime/thread.cpp
>>>
>>> 1482???? // dcubed - Looks like the Threads_lock is for stable access
>>> 1483???? // to _jvmci_old_thread_counters and _jvmci_counters.
>>>
>>> Does this comment need revisiting?
>>
>> We've been trying to decide what to do about this comment. We'll be
>> filing a follow up bug for the Compiler team to take another look at
>> how _jvmci_old_thread_counters and _jvmci_counters are protected.
>> Threads_lock is a big hammer and there should be a less expensive
>> solution. Once that bug gets filed, this comment can go away.
>>
>>
>>> 3451 volatile jint ...
>>>
>>> Why are you using jint for field types here? (Surprised Coleen 
>>> hasn't spotted them! ;-) ).
>>
>> volatile jint???????? Threads::_smr_delete_notify = 0;
>> volatile jint???????? Threads::_smr_deleted_thread_cnt = 0;
>> volatile jint???????? Threads::_smr_deleted_thread_time_max = 0;
>> volatile jint???????? Threads::_smr_deleted_thread_times = 0;
>> :
>> volatile jint???????? Threads::_smr_tlh_cnt = 0;
>> volatile jint???????? Threads::_smr_tlh_time_max = 0;
>> volatile jint???????? Threads::_smr_tlh_times = 0;
>>
>> _smr_delete_notify is a "flag" that indicates when an _smr_delete_lock
>> notify() operation is needed. It counts up and down and should be a 
>> fairly
>> small number.
>>
>> _smr_deleted_thread_cnt counts the number of Threads that have been
>> deleted over the life of the VM. It's an always increasing number so
>> it's size depends on how long the VM has been running.
>>
>> _smr_deleted_thread_time_max is the maximum time in millis it has
>> taken to delete a thread. This field was originally a jlong, but
>> IIRC the Atomic::cmpxchg() code was not happy on ARM... (might have
>> just been Atomic::add() that was not happy)
>>
>> _smr_deleted_thread_times accumulates the time in millis that it
>> has taken to delete threads. It's an always increasing number so
>> it's size depends on how long the VM has been running. This field
>> was originally a jlong, but IIRC the Atomic::add() code was not
>> happy on ARM... (might have just been Atomic::cmpxchg() that was
>> not happy)
>>
>> _smr_tlh_cnt counts the number of ThreadsListHandles that have been
>> deleted over the life of the VM. It's an always increasing number so
>> it's size depends on how long the VM has been running.
>>
>> _smr_tlh_time_max is the maximum time in millis it has taken to
>> delete a ThreadsListHandle. This field was originally a jlong, but
>> IIRC the Atomic::cmpxchg() code was not happy on ARM... (might have
>> just been Atomic::add() that was not happy)
>>
>> _smr_tlh_times? accumulates the time in millis that it has taken to
>> delete ThreadsListHandles. It's an always increasing number so
>> it's size depends on how long the VM has been running.? This field
>> was originally a jlong, but IIRC the Atomic::add() code was not
>> happy on ARM... (might have just been Atomic::cmpxchg() that was
>> not happy)
>>
>>
>> It just dawned on me that I'm not sure whether you think the 'jint'
>> fields should be larger or smaller or the same size but a different
>> type... :-)
>>
>> The fact that I had to write so much to explain what these fields
>> are for and how they're used indicates I need more comments. See
>> below...
>>
>>
>>> 3456 long Threads::_smr_java_thread_list_alloc_cnt = 1;
>>> 3457 long Threads::_smr_java_thread_list_free_cnt = 0;
>>>
>>> long? If you really want 64-bit counters here you won't get them on 
>>> Windows with that declaration. If you really want variable sized 
>>> counters I suggest uintptr_t; otherwise uint64_t.
>>
>> long????????????????? Threads::_smr_java_thread_list_alloc_cnt = 1;
>> long????????????????? Threads::_smr_java_thread_list_free_cnt = 0;
>>
>> _smr_java_thread_list_alloc_cnt counts the number of ThreadsLists that
>> have been allocated over the life of the VM. It's an always increasing
>> number so it's size depends on how long the VM has been running and how
>> many Threads have been created and destroyed.
>>
>> _smr_java_thread_list_free_cnt counts the number of ThreadsLists that
>> have been freed over the life of the VM. It's an always increasing
>> number so it's size depends on how long the VM has been running and how
>> many Threads have been created and destroyed.
>>
>> I can't remember why we chose 'long' and I agree it's not a good choice
>> for Win*.
>>
>>
>> Okay so it dawns on me that we haven't written down a description for
>> the various new '_cnt', '_max' and '_times" fields so I'm adding these
>> comments to thread.hpp:
>>
>> ??private:
>> ?? // Safe Memory Reclamation (SMR) support:
>> ?? static Monitor*????????????? _smr_delete_lock;
>> ?? // The '_cnt', '_max' and '_times" fields are enabled via
>> ?? // -XX:+EnableThreadSMRStatistics:
>> ??????????????????????????????? // # of parallel threads in 
>> _smr_delete_lock->wait().
>> ?? static uint????????????????? _smr_delete_lock_wait_cnt;
>> ??????????????????????????????? // Max # of parallel threads in 
>> _smr_delete_lock->wait().
>> ?? static uint????????????????? _smr_delete_lock_wait_max;
>> ??????????????????????????????? // Flag to indicate when an 
>> _smr_delete_lock->notify() is needed.
>> ?? static volatile uint???????? _smr_delete_notify;
>> ??????????????????????????????? // # of threads deleted over VM 
>> lifetime.
>> ?? static volatile uint???????? _smr_deleted_thread_cnt;
>> ??????????????????????????????? // Max time in millis to delete a 
>> thread.
>> ?? static volatile uint???????? _smr_deleted_thread_time_max;
>> ??????????????????????????????? // Cumulative time in millis to 
>> delete threads.
>> ?? static volatile uint???????? _smr_deleted_thread_times;
>> ?? static ThreadsList* volatile _smr_java_thread_list;
>> ?? static ThreadsList*????????? get_smr_java_thread_list();
>> ?? static ThreadsList* xchg_smr_java_thread_list(ThreadsList* new_list);
>> ??????????????????????????????? // # of ThreadsLists allocated over 
>> VM lifetime.
>> ?? static uint64_t????????????? _smr_java_thread_list_alloc_cnt;
>> ??????????????????????????????? // # of ThreadsLists freed over VM 
>> lifetime.
>> ?? static uint64_t????????????? _smr_java_thread_list_free_cnt;
>> ??????????????????????????????? // Max size ThreadsList allocated.
>> ?? static uint????????????????? _smr_java_thread_list_max;
>> ??????????????????????????????? // Max # of nested ThreadsLists for a 
>> thread.
>> ?? static uint????????????????? _smr_nested_thread_list_max;
>> ??????????????????????????????? // # of ThreadsListHandles deleted 
>> over VM lifetime.
>> ?? static volatile uint???????? _smr_tlh_cnt;
>> ??????????????????????????????? // Max time in millis to delete a 
>> ThreadsListHandle.
>> ?? static volatile jint???????? _smr_tlh_time_max;
>> ??????????????????????????????? // Cumulative time in millis to 
>> delete ThreadsListHandles.
>> ?? static volatile jint???????? _smr_tlh_times;
>> ?? static ThreadsList*????????? _smr_to_delete_list;
>> ??????????????????????????????? // # of parallel ThreadsLists on the 
>> to-delete list.
>> ?? static uint????????????????? _smr_to_delete_list_cnt;
>> ??????????????????????????????? // Max # of parallel ThreadsLists on 
>> the to-delete list.
>> ?? static uint????????????????? _smr_to_delete_list_max;
>>
>>
>> I've also gone through all the new '_cnt', '_max' and '_times" fields
>> in thread.cpp and added an impl note to explain the choice of type:
>>
>> // Safe Memory Reclamation (SMR) support:
>> Monitor*????????????? Threads::_smr_delete_lock =
>> ?????????????????????????? new Monitor(Monitor::special, 
>> "smr_delete_lock",
>> ?????????????????????????????????????? false /* allow_vm_block */,
>> Monitor::_safepoint_check_never);
>> // The '_cnt', '_max' and '_times" fields are enabled via
>> // -XX:+EnableThreadSMRStatistics:
>> ?????????????????????? // # of parallel threads in 
>> _smr_delete_lock->wait().
>> ?????????????????????? // Impl note: Hard to imagine > 64K waiting 
>> threads so
>> ?????????????????????? // this could be 16-bit, but there is no nice 
>> 16-bit
>> ?????????????????????? // _FORMAT support.
>> uint????????????????? Threads::_smr_delete_lock_wait_cnt = 0;
>> ?????????????????????? // Max # of parallel threads in 
>> _smr_delete_lock->wait().
>> ?????????????????????? // Impl note: See _smr_delete_lock_wait_cnt note.
>> uint????????????????? Threads::_smr_delete_lock_wait_max = 0;
>> ?????????????????????? // Flag to indicate when an 
>> _smr_delete_lock->notify() is needed.
>> ?????????????????????? // Impl note: See _smr_delete_lock_wait_cnt note.
>> volatile uint???????? Threads::_smr_delete_notify = 0;
>> ?????????????????????? // # of threads deleted over VM lifetime.
>> ?????????????????????? // Impl note: Atomically incremented over VM 
>> lifetime so
>> ?????????????????????? // use unsigned for more range. Unsigned 
>> 64-bit would
>> ?????????????????????? // be more future proof, but 64-bit atomic inc 
>> isn't
>> ?????????????????????? // available everywhere (or is it?).
>> volatile uint???????? Threads::_smr_deleted_thread_cnt = 0;
>> ?????????????????????? // Max time in millis to delete a thread.
>> ?????????????????????? // Impl note: 16-bit might be too small on an 
>> overloaded
>> ?????????????????????? // machine. Use unsigned since this is a time 
>> value. Set
>> ?????????????????????? // via Atomic::cmpxchg() in a loop for 
>> correctness.
>> volatile uint???????? Threads::_smr_deleted_thread_time_max = 0;
>> ?????????????????????? // Cumulative time in millis to delete threads.
>> ?????????????????????? // Impl note: Atomically added to over VM 
>> lifetime so use
>> ?????????????????????? // unsigned for more range. Unsigned 64-bit 
>> would be more
>> ?????????????????????? // future proof, but 64-bit atomic inc isn't 
>> available
>> ?????????????????????? // everywhere (or is it?).
>> volatile uint???????? Threads::_smr_deleted_thread_times = 0;
>> ThreadsList* volatile Threads::_smr_java_thread_list = new 
>> ThreadsList(0);
>> ?????????????????????? // # of ThreadsLists allocated over VM lifetime.
>> ?????????????????????? // Impl note: We allocate a new ThreadsList 
>> for every
>> ?????????????????????? // thread create and every thread delete so we 
>> need a
>> ?????????????????????? // bigger type than the 
>> _smr_deleted_thread_cnt field.
>> uint64_t????????????? Threads::_smr_java_thread_list_alloc_cnt = 1;
>> ?????????????????????? // # of ThreadsLists freed over VM lifetime.
>> ?????????????????????? // Impl note: See 
>> _smr_java_thread_list_alloc_cnt note.
>> uint64_t????????????? Threads::_smr_java_thread_list_free_cnt = 0;
>> ?????????????????????? // Max size ThreadsList allocated.
>> ?????????????????????? // Impl note: Max # of threads alive at one 
>> time should
>> ?????????????????????? // fit in unsigned 32-bit.
>> uint????????????????? Threads::_smr_java_thread_list_max = 0;
>> ?????????????????????? // Max # of nested ThreadsLists for a thread.
>> ?????????????????????? // Impl note: Hard to imagine > 64K nested 
>> ThreadsLists
>> ?????????????????????? // so his could be 16-bit, but there is no 
>> nice 16-bit
>> ?????????????????????? // _FORMAT support.
>> uint????????????????? Threads::_smr_nested_thread_list_max = 0;
>> ?????????????????????? // # of ThreadsListHandles deleted over VM 
>> lifetime.
>> ?????????????????????? // Impl note: Atomically incremented over VM 
>> lifetime so
>> ?????????????????????? // use unsigned for more range. There will be 
>> fewer
>> ?????????????????????? // ThreadsListHandles than threads so unsigned 
>> 32-bit
>> ?????????????????????? // should be fine.
>> volatile uint???????? Threads::_smr_tlh_cnt = 0;
>> ?????????????????????? // Max time in millis to delete a 
>> ThreadsListHandle.
>> ?????????????????????? // Impl note: 16-bit might be too small on an 
>> overloaded
>> ?????????????????????? // machine. Use unsigned since this is a time 
>> value. Set
>> ?????????????????????? // via Atomic::cmpxchg() in a loop for 
>> correctness.
>> volatile uint???????? Threads::_smr_tlh_time_max = 0;
>> ?????????????????????? // Cumulative time in millis to delete 
>> ThreadsListHandles.
>> ?????????????????????? // Impl note: Atomically added to over VM 
>> lifetime so use
>> ?????????????????????? // unsigned for more range. Unsigned 64-bit 
>> would be more
>> ?????????????????????? // future proof, but 64-bit atomic inc isn't 
>> available
>> ?????????????????????? // everywhere (or is it?).
>> volatile uint???????? Threads::_smr_tlh_times = 0;
>> ThreadsList*????????? Threads::_smr_to_delete_list = NULL;
>> ?????????????????????? // # of parallel ThreadsLists on the to-delete 
>> list.
>> ?????????????????????? // Impl note: Hard to imagine > 64K 
>> ThreadsLists needing
>> ?????????????????????? // to be deleted so this could be 16-bit, but 
>> there is no
>> ?????????????????????? // nice 16-bit _FORMAT support.
>> uint????????????????? Threads::_smr_to_delete_list_cnt = 0;
>> ?????????????????????? // Max # of parallel ThreadsLists on the 
>> to-delete list.
>> ?????????????????????? // Impl note: See _smr_to_delete_list_cnt note.
>> uint????????????????? Threads::_smr_to_delete_list_max = 0;
>>
>>
>> Yikes! With the additional comments, the additions to thread.hpp and
>> thread.cpp grew by a bunch... I've done a test build build on my Mac
>> with these changes and I'm about to kick off a Mach5 tier1 job...
>>
>> Dan
>>
>>
>>
>>>
>>> ----
>>>
>>> On 19/11/2017 11:49 AM, Daniel D. Daugherty wrote:
>>>> Greetings,
>>>>
>>>> Testing of the last round of changes revealed a hang in one of the new
>>>> TLH tests. Robbin has fixed the hang, updated the existing TLH 
>>>> test, and
>>>> added another TLH test for good measure.
>>>>
>>>> Here is the updated full webrev:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/
>>>>
>>>> Here is the updated delta webrev:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-delta/
>>>>
>>>> Dan ran the bits thru the usual Mach5 tier[1-5] testing and there are
>>>> no unexpected failures.
>>>>
>>>> We welcome comments, suggestions and feedback.
>>>>
>>>> Dan, Erik and Robbin
>>>>
>>>>
>>>> On 11/15/17 3:32 PM, Daniel D. Daugherty wrote:
>>>>> Greetings,
>>>>>
>>>>> Robbin rebased the project last night/this morning to merge with 
>>>>> Thread
>>>>> Local Handshakes (TLH) and also picked up additional changesets up 
>>>>> thru:
>>>>>
>>>>>> Changeset: fa736014cf28
>>>>>> Author:??? cjplummer
>>>>>> Date:????? 2017-11-14 18:08 -0800
>>>>>> URL:http://hg.openjdk.java.net/jdk/hs/rev/fa736014cf28
>>>>>>
>>>>>> 8191049: Add alternate version of pns() that is callable from 
>>>>>> within hotspot source
>>>>>> Summary: added pns2() to debug.cpp
>>>>>> Reviewed-by: stuefe, gthornbr
>>>>>
>>>>> This is the first time we've rebased the project to bits that are 
>>>>> this
>>>>> fresh (< 12 hours old at rebase time). We've done this because we 
>>>>> think
>>>>> we're done with this project and are in the final 
>>>>> review-change-retest
>>>>> cycle(s)... In other words, we're not planning on making any more 
>>>>> major
>>>>> changes for this project... :-)
>>>>>
>>>>> *** Time for code reviewers to chime in on this thread! ***
>>>>>
>>>>> Here is the updated full webrev:
>>>>>
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-full/
>>>>>
>>>>> Here's is the delta webrev from the 2017.11.10 rebase:
>>>>>
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-delta/
>>>>>
>>>>> Dan has submitted the bits for the usual Mach5 tier[1-5] testing
>>>>> (and the new baseline also)...
>>>>>
>>>>> We're expecting this round to be a little noisier than usual because
>>>>> we did not rebase on a PIT snapshot so the new baseline has not been
>>>>> through Jesper's usual care-and-feeding of integration_blockers, etc.
>>>>>
>>>>> We welcome comments, suggestions and feedback.
>>>>>
>>>>> Dan, Erik and Robbin
>>>>>
>>>>>
>>>>> On 11/14/17 4:48 PM, Daniel D. Daugherty wrote:
>>>>>> Greetings,
>>>>>>
>>>>>> I rebased the project to the 2017.11.10 jdk/hs PIT snapshot.
>>>>>> (Yes, we're playing chase-the-repo...)
>>>>>>
>>>>>> Here is the updated full webrev:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-07-full/
>>>>>>
>>>>>> Unlike the previous rebase, there were no changes required to the
>>>>>> open code to get the rebased bits to build so there is no delta
>>>>>> webrev.
>>>>>>
>>>>>> These bits have been run through JPRT and I've submitted the usual
>>>>>> Mach5 tier[1-5] test run...
>>>>>>
>>>>>> We welcome comments, suggestions and feedback.
>>>>>>
>>>>>> Dan, Erik and Robbin
>>>>>>
>>>>>>
>>>>>> On 11/13/17 12:30 PM, Daniel D. Daugherty wrote:
>>>>>>> Greetings,
>>>>>>>
>>>>>>> I rebased the project to the 2017.10.26 jdk10/hs PIT snapshot.
>>>>>>>
>>>>>>> Here are the updated webrevs:
>>>>>>>
>>>>>>> Here's the mq comment for the change:
>>>>>>>
>>>>>>> ? Rebase to 2017.10.25 PIT snapshot.
>>>>>>>
>>>>>>> Here is the full webrev:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-full/
>>>>>>>
>>>>>>> And here is the delta webrev:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-delta/
>>>>>>>
>>>>>>> I ran the above bits throught Mach5 tier[1-5] testing over the 
>>>>>>> holiday
>>>>>>> weekend. Didn't see any issues in a quick look. Going to take a 
>>>>>>> closer
>>>>>>> look today.
>>>>>>>
>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>
>>>>>>> Dan, Erik and Robbin
>>>>>>>
>>>>>>>
>>>>>>> On 11/8/17 1:05 PM, Daniel D. Daugherty wrote:
>>>>>>>> Greetings,
>>>>>>>>
>>>>>>>> Resolving one of the code review comments (from both Stefan K 
>>>>>>>> and Coleen)
>>>>>>>> on jdk10-04-full required quite a few changes so it is being 
>>>>>>>> done as a
>>>>>>>> standalone patch and corresponding webrevs:
>>>>>>>>
>>>>>>>> Here's the mq comment for the change:
>>>>>>>>
>>>>>>>> ? stefank, coleenp CR - refactor most JavaThreadIterator usage 
>>>>>>>> to use
>>>>>>>> ??? JavaThreadIteratorWithHandle.
>>>>>>>>
>>>>>>>> Here is the full webrev:
>>>>>>>>
>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-full/
>>>>>>>>
>>>>>>>> And here is the delta webrev:
>>>>>>>>
>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-delta/
>>>>>>>>
>>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>>
>>>>>>>> Dan, Erik and Robbin
>>>>>>>>
>>>>>>>>
>>>>>>>> On 10/9/17 3:41 PM, Daniel D. Daugherty wrote:
>>>>>>>>> Greetings,
>>>>>>>>>
>>>>>>>>> We have a (eXtra Large) fix for the following bug:
>>>>>>>>>
>>>>>>>>> 8167108 inconsistent handling of SR_lock can lead to crashes
>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8167108
>>>>>>>>>
>>>>>>>>> This fix adds a Safe Memory Reclamation (SMR) mechanism based on
>>>>>>>>> Hazard Pointers to manage JavaThread lifecycle.
>>>>>>>>>
>>>>>>>>> Here's a PDF for the internal wiki that we've been using to 
>>>>>>>>> describe
>>>>>>>>> and track the work on this project:
>>>>>>>>>
>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/SMR_and_JavaThread_Lifecycle-JDK10-04.pdf 
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Dan has noticed that the indenting is wrong in some of the 
>>>>>>>>> code quotes
>>>>>>>>> in the PDF that are not present in the internal wiki. We don't 
>>>>>>>>> have a
>>>>>>>>> solution for that problem yet.
>>>>>>>>>
>>>>>>>>> Here's the webrev for current JDK10 version of this fix:
>>>>>>>>>
>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-04-full
>>>>>>>>>
>>>>>>>>> This fix has been run through many rounds of JPRT and Mach5 
>>>>>>>>> tier[2-5]
>>>>>>>>> testing, additional stress testing on Dan's Solaris X64 
>>>>>>>>> server, and
>>>>>>>>> additional testing on Erik and Robbin's machines.
>>>>>>>>>
>>>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>>>
>>>>>>>>> Daniel Daugherty
>>>>>>>>> Erik Osterlund
>>>>>>>>> Robbin Ehn
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>


From erik.helin at oracle.com  Tue Nov 21 08:02:05 2017
From: erik.helin at oracle.com (Erik Helin)
Date: Tue, 21 Nov 2017 09:02:05 +0100
Subject: RFR: 8182284: G1Analytics uses uninitialized fields
Message-ID: <c7b55bda-ee23-6503-92b6-04854a659660@oracle.com>

Hi all,

this small patch just initializes two uninitialized fields in 
G1Analytics. 0.0 is a good enough as starting value, the fields are not 
meant to be read before a GC has been run (but if this would happen, 
observing 0.0 fits together with the calculations that the values 
represents).

Bug:
https://bugs.openjdk.java.net/browse/JDK-8182284

Webrev:
http://cr.openjdk.java.net/~ehelin/8182284/00/

Testing:
- Mach5 hs-tier1,hs-tier2
- Newly added unit test

Thanks,
Erik


From thomas.schatzl at oracle.com  Tue Nov 21 08:06:02 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Tue, 21 Nov 2017 09:06:02 +0100
Subject: RFR: 8182284: G1Analytics uses uninitialized fields
In-Reply-To: <c7b55bda-ee23-6503-92b6-04854a659660@oracle.com>
References: <c7b55bda-ee23-6503-92b6-04854a659660@oracle.com>
Message-ID: <1511251562.9073.0.camel@oracle.com>

Hi,

On Tue, 2017-11-21 at 09:02 +0100, Erik Helin wrote:
> Hi all,
> 
> this small patch just initializes two uninitialized fields in 
> G1Analytics. 0.0 is a good enough as starting value, the fields are
> not 
> meant to be read before a GC has been run (but if this would happen, 
> observing 0.0 fits together with the calculations that the values 
> represents).
> 
> Bug:
> https://bugs.openjdk.java.net/browse/JDK-8182284
> 
> Webrev:
> http://cr.openjdk.java.net/~ehelin/8182284/00/
> 

  looks good.

Thomas



From stefan.johansson at oracle.com  Tue Nov 21 10:07:15 2017
From: stefan.johansson at oracle.com (Stefan Johansson)
Date: Tue, 21 Nov 2017 11:07:15 +0100
Subject: RFR: 8182284: G1Analytics uses uninitialized fields
In-Reply-To: <c7b55bda-ee23-6503-92b6-04854a659660@oracle.com>
References: <c7b55bda-ee23-6503-92b6-04854a659660@oracle.com>
Message-ID: <aea5a827-2022-8bac-fb37-78d7dd982ac5@oracle.com>



On 2017-11-21 09:02, Erik Helin wrote:
> Hi all,
>
> this small patch just initializes two uninitialized fields in 
> G1Analytics. 0.0 is a good enough as starting value, the fields are 
> not meant to be read before a GC has been run (but if this would 
> happen, observing 0.0 fits together with the calculations that the 
> values represents).
>
> Bug:
> https://bugs.openjdk.java.net/browse/JDK-8182284
>
> Webrev:
> http://cr.openjdk.java.net/~ehelin/8182284/00/
>
Looks good,
Stefan
> Testing:
> - Mach5 hs-tier1,hs-tier2
> - Newly added unit test
>
> Thanks,
> Erik



From thomas.schatzl at oracle.com  Tue Nov 21 11:27:51 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Tue, 21 Nov 2017 12:27:51 +0100
Subject: RFR(XS): 8178497: Bug in MutableNUMASpace::ensure_parsability
In-Reply-To: <8f8e7c5e-5de8-ee26-acb4-e90662dad0af@oracle.com>
References: <8f8e7c5e-5de8-ee26-acb4-e90662dad0af@oracle.com>
Message-ID: <1511263671.9073.7.camel@oracle.com>

Hi Sangheon,

On Mon, 2017-11-20 at 11:05 -0800, sangheon.kim wrote:
> Hi all,
> 
> Can I have some reviews for this tiny pointer arithmetic change?
> Current code doesn't use pointer arithmetic, so the the resulting
> values are wrong(too small). i.e. adding two values first and then
> typecast to HeapWord* which is wrong here.
> e.g.
> intptr_t cur_top = X;
> size_t touched_words = XX;
> ...
> align_down((HeapWord*)(cur_top + touched_words), XXX);
> 
> This should be 'align_down( (HeapWord*)cur_top + touched_words,
> XXXX);'.
> 
> As I don't see any good reason of using 'intptr_t', changed to use 
> 'HeapWord*' and changed related stuff.
> 
> I didn't add regression test or some further investigation as this
> is clear that the calculation is wrong. And hard to provoke the
> problem outside.
> 
> CR: https://bugs.openjdk.java.net/browse/JDK-8178497
> Webrev: http://cr.openjdk.java.net/~sangheki/8178497/webrev.0
> Testing: local building

  actually after reading this code a bit in more detail I think the
failure mode is "only" performance loss. I.e. the "invalid" part of a
MutableNUMASpace is freed and reallocated for better NUMA locality.

With a too small invalid size what happens is that this locality
improvement will not happen for that part of the MutableNUMASpace. I do
not think there is a useful way of creating a reproducer.

Looks good.

Thanks,
  Thomas



From rkennke at redhat.com  Tue Nov 21 11:49:51 2017
From: rkennke at redhat.com (Roman Kennke)
Date: Tue, 21 Nov 2017 12:49:51 +0100
Subject: 8191562: Missing include in gcArguments.cpp breaks minimal JVM
 build
In-Reply-To: <6be81483-c3f6-0f8a-19db-1975036d1c2c@redhat.com>
References: <5699db6f-5397-d144-956e-eaafe37cca69@redhat.com>
 <dc34b321d7a24e6e8a527836581c874f@sap.com>
 <6be81483-c3f6-0f8a-19db-1975036d1c2c@redhat.com>
Message-ID: <6923281c-dbb8-1840-cf76-c93cc343c666@redhat.com>

I think this change is trivial and can be pushed:

Final webrev (incl. Reviewed-by: line):
http://cr.openjdk.java.net/~rkennke/8191562/webrev.02/

Can somebody please sponsor it?

Thanks,
Roman

> Hi Martin,
> no, I don't think this is necessary. We can sneak it into this change:
> 
> http://cr.openjdk.java.net/~rkennke/8191562/webrev.01/ 
> <http://cr.openjdk.java.net/%7Erkennke/8191562/webrev.01/>


> 
> Is that what aix needs?
> 
> Thanks,
> Roman
> 
>> Hi Roman,
>>
>> unfortunately, I was checking AIX too late. There's another missing 
>> include.
>> We need to #include "memory/allocation.inline.hpp". Otherwise xlC 
>> doesn't find a definition for the new/delete operators like 
>> CHeapObj<F>::operator new.
>>
>> Do you want me to open a new bug for it?
>>
>> Best regards,
>> Martin
>>
>>
>> -----Original Message-----
>> From: hotspot-gc-dev [mailto:hotspot-gc-dev-bounces at openjdk.java.net] 
>> On Behalf Of Roman Kennke
>> Sent: Montag, 20. November 2017 11:37
>> To: hotspot-gc-dev at openjdk.java.net
>> Subject: RFR: 8191562: Missing include in gcArguments.cpp breaks 
>> minimal JVM build
>>
>> David Holmes reported build failures in minimal JVM due to missing
>> include in gcArguments.cpp:
>>
>> https://bugs.openjdk.java.net/browse/JDK-8191562
>>
>> This should fix it:
>>
>> http://cr.openjdk.java.net/~rkennke/8191562/webrev.00/
>> <http://cr.openjdk.java.net/%7Erkennke/8191562/webrev.00/>
>>
>> Unfortunately, the minimal JVM doesn't build at all for me (*), so I am
>> posting this blindly. However, it seems fairly obvious that it's
>> correct: UNSUPPORTED_OPTION is defined in arguments.hpp, including it
>> should fix the build failure.
>>
>> Ok to go?
>>
>> Roman
>>
>> (*)
>>
>> /home/rkennke/src/openjdk/jdk-hs/src/hotspot/cpu/x86/c1_Runtime1_x86.cpp:
>> In member function 'int StubAssembler::call_RT(Register, Register,
>> address, int)':
>> /home/rkennke/src/openjdk/jdk-hs/src/hotspot/cpu/x86/c1_Runtime1_x86.cpp:136:10: 
>>
>> error: 'call_offset' may be used uninitialized in this function
>> [-Werror=maybe-uninitialized]
>> ? ?? return call_offset;
>> ? ????????? ^
>> cc1plus: all warnings being treated as errors
>>
> 



From rkennke at redhat.com  Tue Nov 21 11:50:14 2017
From: rkennke at redhat.com (Roman Kennke)
Date: Tue, 21 Nov 2017 12:50:14 +0100
Subject: RFR: 8191564: Refactor GC related servicability code into GC
 specific subclasses
In-Reply-To: <387d2b93-cb8b-51be-a6e0-3b5f9d6cbb1c@redhat.com>
References: <387d2b93-cb8b-51be-a6e0-3b5f9d6cbb1c@redhat.com>
Message-ID: <211c2faf-c6fc-5d7b-0426-1d96b4e6627e@redhat.com>


> Currently, there's lots of GC specific code sprinkled over 
> src/hotspot/share/services. This change introduces a GC interface for 
> that, and moves all GC specific code to their respective 
> src/hotspot/share/gc directory.
> 
> http://cr.openjdk.java.net/~rkennke/8191564/webrev.00/ 
> <http://cr.openjdk.java.net/%7Erkennke/8191564/webrev.00/>
> 
> Testing: hotspot_gc and hotspot_serviceability, none showed regressions
> 
> Built minimal and server without regressions
> 
> What do you think?
> 

Ping?

Roman



From stefan.johansson at oracle.com  Tue Nov 21 13:29:09 2017
From: stefan.johansson at oracle.com (Stefan Johansson)
Date: Tue, 21 Nov 2017 14:29:09 +0100
Subject: RFR(XS): 8178497: Bug in MutableNUMASpace::ensure_parsability
In-Reply-To: <8f8e7c5e-5de8-ee26-acb4-e90662dad0af@oracle.com>
References: <8f8e7c5e-5de8-ee26-acb4-e90662dad0af@oracle.com>
Message-ID: <e6739b0a-dd9c-77bc-cfa3-d5b411aa40c9@oracle.com>

Hi Sangheon,

On 2017-11-20 20:05, sangheon.kim wrote:
> Hi all,
>
> Can I have some reviews for this tiny pointer arithmetic change?
> Current code doesn't use pointer arithmetic, so the the resulting 
> values are wrong(too small). i.e. adding two values first and then 
> typecast to HeapWord* which is wrong here.
> e.g.
> intptr_t cur_top = X;
> size_t touched_words = XX;
> ...
> align_down((HeapWord*)(cur_top + touched_words), XXX);
>
> This should be 'align_down( (HeapWord*)cur_top + touched_words, XXXX);'.
>
> As I don't see any good reason of using 'intptr_t', changed to use 
> 'HeapWord*' and changed related stuff.
>
> I didn't add regression test or some further investigation as this is 
> clear that the calculation is wrong. And hard to provoke the problem 
> outside.
>
> CR: https://bugs.openjdk.java.net/browse/JDK-8178497
> Webrev: http://cr.openjdk.java.net/~sangheki/8178497/webrev.0
Thanks for taking care of this. I think the changes is good, but I would 
feel even more certain if you ran some testing with NUMA and Parallel to 
make sure we haven't overlooked anything.

Thanks,
StefanJ
> Testing: local building
>
> Thanks,
> Sangheon



From erik.helin at oracle.com  Tue Nov 21 13:29:27 2017
From: erik.helin at oracle.com (Erik Helin)
Date: Tue, 21 Nov 2017 14:29:27 +0100
Subject: 8191562: Missing include in gcArguments.cpp breaks minimal JVM
 build
In-Reply-To: <6923281c-dbb8-1840-cf76-c93cc343c666@redhat.com>
References: <5699db6f-5397-d144-956e-eaafe37cca69@redhat.com>
 <dc34b321d7a24e6e8a527836581c874f@sap.com>
 <6be81483-c3f6-0f8a-19db-1975036d1c2c@redhat.com>
 <6923281c-dbb8-1840-cf76-c93cc343c666@redhat.com>
Message-ID: <8fd3bb02-b0fa-39dc-074a-3d1102de3238@oracle.com>

On 11/21/2017 12:49 PM, Roman Kennke wrote:
> I think this change is trivial and can be pushed:
> 
> Final webrev (incl. Reviewed-by: line):
> http://cr.openjdk.java.net/~rkennke/8191562/webrev.02/
> 
> Can somebody please sponsor it?

Sure, I can sponsor.

Thanks,
Erik

> Thanks,
> Roman
> 
>> Hi Martin,
>> no, I don't think this is necessary. We can sneak it into this change:
>>
>> http://cr.openjdk.java.net/~rkennke/8191562/webrev.01/ 
>> <http://cr.openjdk.java.net/%7Erkennke/8191562/webrev.01/>
> 
> 
>>
>> Is that what aix needs?
>>
>> Thanks,
>> Roman
>>
>>> Hi Roman,
>>>
>>> unfortunately, I was checking AIX too late. There's another missing 
>>> include.
>>> We need to #include "memory/allocation.inline.hpp". Otherwise xlC 
>>> doesn't find a definition for the new/delete operators like 
>>> CHeapObj<F>::operator new.
>>>
>>> Do you want me to open a new bug for it?
>>>
>>> Best regards,
>>> Martin
>>>
>>>
>>> -----Original Message-----
>>> From: hotspot-gc-dev [mailto:hotspot-gc-dev-bounces at openjdk.java.net] 
>>> On Behalf Of Roman Kennke
>>> Sent: Montag, 20. November 2017 11:37
>>> To: hotspot-gc-dev at openjdk.java.net
>>> Subject: RFR: 8191562: Missing include in gcArguments.cpp breaks 
>>> minimal JVM build
>>>
>>> David Holmes reported build failures in minimal JVM due to missing
>>> include in gcArguments.cpp:
>>>
>>> https://bugs.openjdk.java.net/browse/JDK-8191562
>>>
>>> This should fix it:
>>>
>>> http://cr.openjdk.java.net/~rkennke/8191562/webrev.00/
>>> <http://cr.openjdk.java.net/%7Erkennke/8191562/webrev.00/>
>>>
>>> Unfortunately, the minimal JVM doesn't build at all for me (*), so I am
>>> posting this blindly. However, it seems fairly obvious that it's
>>> correct: UNSUPPORTED_OPTION is defined in arguments.hpp, including it
>>> should fix the build failure.
>>>
>>> Ok to go?
>>>
>>> Roman
>>>
>>> (*)
>>>
>>> /home/rkennke/src/openjdk/jdk-hs/src/hotspot/cpu/x86/c1_Runtime1_x86.cpp: 
>>>
>>> In member function 'int StubAssembler::call_RT(Register, Register,
>>> address, int)':
>>> /home/rkennke/src/openjdk/jdk-hs/src/hotspot/cpu/x86/c1_Runtime1_x86.cpp:136:10: 
>>>
>>> error: 'call_offset' may be used uninitialized in this function
>>> [-Werror=maybe-uninitialized]
>>> ? ?? return call_offset;
>>> ? ????????? ^
>>> cc1plus: all warnings being treated as errors
>>>
>>
> 


From rkennke at redhat.com  Tue Nov 21 13:43:49 2017
From: rkennke at redhat.com (Roman Kennke)
Date: Tue, 21 Nov 2017 14:43:49 +0100
Subject: 8191562: Missing include in gcArguments.cpp breaks minimal JVM
 build
In-Reply-To: <8fd3bb02-b0fa-39dc-074a-3d1102de3238@oracle.com>
References: <5699db6f-5397-d144-956e-eaafe37cca69@redhat.com>
 <dc34b321d7a24e6e8a527836581c874f@sap.com>
 <6be81483-c3f6-0f8a-19db-1975036d1c2c@redhat.com>
 <6923281c-dbb8-1840-cf76-c93cc343c666@redhat.com>
 <8fd3bb02-b0fa-39dc-074a-3d1102de3238@oracle.com>
Message-ID: <21116a35-b3db-04d8-5fe0-c7bf78630b7a@redhat.com>


> On 11/21/2017 12:49 PM, Roman Kennke wrote:
>> I think this change is trivial and can be pushed:
>>
>> Final webrev (incl. Reviewed-by: line):
>> http://cr.openjdk.java.net/~rkennke/8191562/webrev.02/
>>
>> Can somebody please sponsor it?
> 
> Sure, I can sponsor.

Thanks!!



From thomas.schatzl at oracle.com  Tue Nov 21 13:59:43 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Tue, 21 Nov 2017 14:59:43 +0100
Subject: OOM error caused by large array allocation in G1
In-Reply-To: <CAEyFhScVxVuPXuerEYTj04w=8S9UyMVEAB8tp_gXXgkS4fDv-Q@mail.gmail.com>
References: <CAEyFhSdj4V4rRYVRnDRbg7Pw_-4_FLw4c4mbVVwXNr3q4VXOZw@mail.gmail.com>
 <1511019424.3046.29.camel@oracle.com>
 <CAEyFhScVxVuPXuerEYTj04w=8S9UyMVEAB8tp_gXXgkS4fDv-Q@mail.gmail.com>
Message-ID: <1511272783.9073.13.camel@oracle.com>

Hi,

On Tue, 2017-11-21 at 21:48 +0800, Lijie Xu wrote:
> Hi Thomas,
> 
[...]
> > > I want to know whether my guess is right ...
> > 
> > Very likely. This is a long-standing issue (actually I have once
> > investigated about it like 10 years ago on a different regional
> > collector), and given your findings it is very likely you are
> > correct.
> > The issue also has an extra section in the tuning guide.
> 
> ==> This reference is very helpful for me. Another question is that
> "Do Parallel and CMS collectors have this defect too"? 

No. Parallel and CMS full GC always move all objects. I filed JDK-81915
65 [0] to at least avoid the OOME. Maybe it can be fixed by jdk 11.

Thanks,
  Thomas

[0] https://bugs.openjdk.java.net/browse/JDK-8191565



From daniel.daugherty at oracle.com  Tue Nov 21 14:46:10 2017
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Tue, 21 Nov 2017 09:46:10 -0500
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <D56C03FB-4A8B-48ED-9CBD-5F22C7C05F91@oracle.com>
References: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
 <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>
 <836b5c6c-5eef-9245-c16a-e71372093d9a@oracle.com>
 <82d0e473-a24f-9feb-82e9-7d04c4d7a91d@oracle.com>
 <354f7ece-66bc-c065-9ee8-f55abcc1c23e@oracle.com>
 <0ff8df8e-e93c-02d2-cb97-b13b1ac0f53f@oracle.com>
 <D56C03FB-4A8B-48ED-9CBD-5F22C7C05F91@oracle.com>
Message-ID: <56ab19ad-58d7-f65a-b29b-8503a8f6f72a@oracle.com>

Hi Robin W!

Welcome to the Thread-SMR party!!

On 11/20/17 4:48 AM, Robin Westberg wrote:
> Hi Dan,
>
> Overall I must say this looks very nice, can?t wait to use it! (Also a disclaimer: not a reviewer, and have not looked at the gc or jmvti specific changes nor the tests (yet)).

Code reviews are always welcome (OpenJDK role does not matter).


> Didn?t spot any real issues, just a few small efficiency related things:
>
> src/hotspot/share/runtime/biasedLocking.cpp
>
> 217     for (JavaThread* cur_thread = Threads::first(); cur_thread != NULL; cur_thread = cur_thread->next()) {
> 218       if (cur_thread == biased_thread) {
> 219         thread_is_alive = true;
>
> This loop could be replaced with:
>
> ThreadsListHandle tlh;
> thread_is_alive = tlh.includes(biased_thread);

Nice catch! Fixed with your proposed change.

The careful reader will notice that in revoke_bias() we are not
creating a ThreadsListHandle that is in scope for the entire
function and we are doing cached monitor info walks without an
obvious ThreadsListHandle in place.

revoke_bias() is called at a safepoint from most locations. The
one call that is not made at a safepoint is from
BiasedLocking::revoke_and_rebias() and it is made by a JavaThread
that is revoking a bias on itself which is also safe.

We should add an assertion to revoke_bias() that looks something
like this:

 ? assert(requesting_thread == Thread::current() ||
 ???????? SafepointSynchronize::is_at_safepoint(),
 ???????? "must be operating on self or at a safepoint.");

but we'll do that in a separate follow up bug since that will
require biased locking focused testing.


> src/hotspot/share/runtime/memprofiler.cpp
>
> 55-56 grabs the Threads_lock, shouldn?t be needed anymore.

Nice catch! Deleting lines 55-56.


> src/hotspot/share/runtime/thread.inline.hpp
>
> 198   TerminatedTypes l_terminated = (TerminatedTypes)
> 199       OrderAccess::load_acquire((volatile jint *) &_terminated);
> 200   return check_is_terminated(_terminated);
>
> The variable used in the return statement was intended to be l_terminated I guess?

Ouch! Looks like JavaThread::is_exiting() is right, but
JavaThread::is_terminated() has been inefficient all this
time. Fixed.


> The following are more minor suggestions / comments:
>
> src/hotspot/share/runtime/thread.cpp
>
> 3432 // operations over all threads.  It is protected by its own Mutex
> 3433 // lock, which is also used in other contexts to protect thread
>
> Should this comment perhaps be revised to mention SMR?

It definitely needs some updating... Here's a stab at it:

// The Threads class links together all active threads, and provides
// operations over all threads. It is protected by the Threads_lock,
// which is also used in other global contexts like safepointing.
// ThreadsListHandles are used to safely perform operations on one
// or more threads without the risk of the thread exiting during the
// operation.
//
// Note: The Threads_lock is currently more widely used than we
// would like. We are actively migrating Threads_lock uses to other
// mechanisms in order to reduce Threads_lock contention.

> 4745   hash_table_size--;
> 4746   hash_table_size |= hash_table_size >> 1;
> ...
>
> This calculation is repeated around line 4922 as well, perhaps put it in a function?

The hash_table_size parameter is currently unused. We were using
a different hash table before that allowed the size to be set.
Unfortunately, that hash table didn't support being freed so we
switched to ResourceHashtable.

We have a project note to come back and update the underlying
hash table to work with dynamic table sizes, but Erik hasn't
had the cycles to do it yet.


> 4828   // If someone set a handshake on us just as we entered exit path, we simple cancel it.
>
> Perhaps something like ?.. entered the exit path, we simply cancel it.?

I went with this:

 ? if (ThreadLocalHandshakes) {
 ??? // The thread is about to be deleted so cancel any handshake.
 ??? thread->cancel_handshake();
 ? }


> src/hotspot/share/runtime/thread.hpp
>
> 1179   bool on_thread_list() { return _on_thread_list; }
> 1180   void set_on_thread_list() { _on_thread_list = true; }
> 1181
> 1182   // thread has called JavaThread::exit() or is terminated
> 1183   bool is_exiting() const;
> 1184   // thread is terminated (no longer on the threads list); we compare
> 1185   // against the two non-terminated values so that a freed JavaThread
> 1186   // will also be considered terminated.
> 1187   bool check_is_terminated(TerminatedTypes l_terminated) const {
> 1188     return l_terminated != _not_terminated && l_terminated != _thread_exiting;
> 1189   }
> 1190   bool is_terminated();
>
> Use of const is inconsistent here, it?s added to 1183, should it perhaps be added to 1179 and 1190 as well then?

Fixed.


> src/hotspot/share/runtime/vm_operations.hpp
>
> 406   DeadlockCycle* result()               { return _deadlocks; };
> 407   VMOp_Type type() const                { return VMOp_FindDeadlocks; }
>
> Whitespace only change that seems spurious.

Agreed. Restored to the original.

Thanks for the review!

Dan


>
> Best regards,
> Robin
>
>> On 19 Nov 2017, at 02:49, Daniel D. Daugherty <daniel.daugherty at oracle.com> wrote:
>>
>> Greetings,
>>
>> Testing of the last round of changes revealed a hang in one of the new
>> TLH tests. Robbin has fixed the hang, updated the existing TLH test, and
>> added another TLH test for good measure.
>>
>> Here is the updated full webrev:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/
>>
>> Here is the updated delta webrev:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-delta/
>>
>> Dan ran the bits thru the usual Mach5 tier[1-5] testing and there are
>> no unexpected failures.
>>
>> We welcome comments, suggestions and feedback.
>>
>> Dan, Erik and Robbin
>>
>>
>> On 11/15/17 3:32 PM, Daniel D. Daugherty wrote:
>>> Greetings,
>>>
>>> Robbin rebased the project last night/this morning to merge with Thread
>>> Local Handshakes (TLH) and also picked up additional changesets up thru:
>>>
>>>> Changeset: fa736014cf28
>>>> Author:    cjplummer
>>>> Date:      2017-11-14 18:08 -0800
>>>> URL:http://hg.openjdk.java.net/jdk/hs/rev/fa736014cf28
>>>>
>>>> 8191049: Add alternate version of pns() that is callable from within hotspot source
>>>> Summary: added pns2() to debug.cpp
>>>> Reviewed-by: stuefe, gthornbr
>>> This is the first time we've rebased the project to bits that are this
>>> fresh (< 12 hours old at rebase time). We've done this because we think
>>> we're done with this project and are in the final review-change-retest
>>> cycle(s)... In other words, we're not planning on making any more major
>>> changes for this project... :-)
>>>
>>> *** Time for code reviewers to chime in on this thread! ***
>>>
>>> Here is the updated full webrev:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-full/
>>>
>>> Here's is the delta webrev from the 2017.11.10 rebase:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-delta/
>>>
>>> Dan has submitted the bits for the usual Mach5 tier[1-5] testing
>>> (and the new baseline also)...
>>>
>>> We're expecting this round to be a little noisier than usual because
>>> we did not rebase on a PIT snapshot so the new baseline has not been
>>> through Jesper's usual care-and-feeding of integration_blockers, etc.
>>>
>>> We welcome comments, suggestions and feedback.
>>>
>>> Dan, Erik and Robbin
>>>
>>>
>>> On 11/14/17 4:48 PM, Daniel D. Daugherty wrote:
>>>> Greetings,
>>>>
>>>> I rebased the project to the 2017.11.10 jdk/hs PIT snapshot.
>>>> (Yes, we're playing chase-the-repo...)
>>>>
>>>> Here is the updated full webrev:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-07-full/
>>>>
>>>> Unlike the previous rebase, there were no changes required to the
>>>> open code to get the rebased bits to build so there is no delta
>>>> webrev.
>>>>
>>>> These bits have been run through JPRT and I've submitted the usual
>>>> Mach5 tier[1-5] test run...
>>>>
>>>> We welcome comments, suggestions and feedback.
>>>>
>>>> Dan, Erik and Robbin
>>>>
>>>>
>>>> On 11/13/17 12:30 PM, Daniel D. Daugherty wrote:
>>>>> Greetings,
>>>>>
>>>>> I rebased the project to the 2017.10.26 jdk10/hs PIT snapshot.
>>>>>
>>>>> Here are the updated webrevs:
>>>>>
>>>>> Here's the mq comment for the change:
>>>>>
>>>>>    Rebase to 2017.10.25 PIT snapshot.
>>>>>
>>>>> Here is the full webrev:
>>>>>
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-full/
>>>>>
>>>>> And here is the delta webrev:
>>>>>
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-delta/
>>>>>
>>>>> I ran the above bits throught Mach5 tier[1-5] testing over the holiday
>>>>> weekend. Didn't see any issues in a quick look. Going to take a closer
>>>>> look today.
>>>>>
>>>>> We welcome comments, suggestions and feedback.
>>>>>
>>>>> Dan, Erik and Robbin
>>>>>
>>>>>
>>>>> On 11/8/17 1:05 PM, Daniel D. Daugherty wrote:
>>>>>> Greetings,
>>>>>>
>>>>>> Resolving one of the code review comments (from both Stefan K and Coleen)
>>>>>> on jdk10-04-full required quite a few changes so it is being done as a
>>>>>> standalone patch and corresponding webrevs:
>>>>>>
>>>>>> Here's the mq comment for the change:
>>>>>>
>>>>>>    stefank, coleenp CR - refactor most JavaThreadIterator usage to use
>>>>>>      JavaThreadIteratorWithHandle.
>>>>>>
>>>>>> Here is the full webrev:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-full/
>>>>>>
>>>>>> And here is the delta webrev:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-delta/
>>>>>>
>>>>>> We welcome comments, suggestions and feedback.
>>>>>>
>>>>>> Dan, Erik and Robbin
>>>>>>
>>>>>>
>>>>>> On 10/9/17 3:41 PM, Daniel D. Daugherty wrote:
>>>>>>> Greetings,
>>>>>>>
>>>>>>> We have a (eXtra Large) fix for the following bug:
>>>>>>>
>>>>>>> 8167108 inconsistent handling of SR_lock can lead to crashes
>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8167108
>>>>>>>
>>>>>>> This fix adds a Safe Memory Reclamation (SMR) mechanism based on
>>>>>>> Hazard Pointers to manage JavaThread lifecycle.
>>>>>>>
>>>>>>> Here's a PDF for the internal wiki that we've been using to describe
>>>>>>> and track the work on this project:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/SMR_and_JavaThread_Lifecycle-JDK10-04.pdf
>>>>>>>
>>>>>>> Dan has noticed that the indenting is wrong in some of the code quotes
>>>>>>> in the PDF that are not present in the internal wiki. We don't have a
>>>>>>> solution for that problem yet.
>>>>>>>
>>>>>>> Here's the webrev for current JDK10 version of this fix:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-04-full
>>>>>>>
>>>>>>> This fix has been run through many rounds of JPRT and Mach5 tier[2-5]
>>>>>>> testing, additional stress testing on Dan's Solaris X64 server, and
>>>>>>> additional testing on Erik and Robbin's machines.
>>>>>>>
>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>
>>>>>>> Daniel Daugherty
>>>>>>> Erik Osterlund
>>>>>>> Robbin Ehn
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>



From leo.korinth at oracle.com  Tue Nov 21 14:56:16 2017
From: leo.korinth at oracle.com (Leo Korinth)
Date: Tue, 21 Nov 2017 15:56:16 +0100
Subject: RFR: 8190408: Run G1CMRemarkTask with the appropriate amount of
 threads instead of starting up everyone
Message-ID: <f61f04b1-a61c-f7af-624e-798ec4ad4897@oracle.com>

Hi,

Removing an if statement. The if-statement did safeguard that the
worker_id was always within the number of active tasks in _cm.

However the active number of tasks in _cm is set directly before the
remark task is created (set_concurrency_and_phase()). The value is
taken from the work gang (g1h->workers()->active_workers()). Thus, as
set_concurrency*() is not called during the actual remarking, the size
of the work-gang will be in sync with the _cm->active_tasks() -- at
least during the remark phase.

The code becomes somewhat easier to read, and hopefully one does not
get confused that we have too many threads running.

Bug:
https://bugs.openjdk.java.net/browse/JDK-8190408

Webrev:
http://cr.openjdk.java.net/~lkorinth/8190408/00/

Testing:
- hs-tier1, hs-tier2

Thanks,
Leo


From thomas.schatzl at oracle.com  Tue Nov 21 15:09:02 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Tue, 21 Nov 2017 16:09:02 +0100
Subject: RFR: 8190408: Run G1CMRemarkTask with the appropriate amount of
 threads instead of starting up everyone
In-Reply-To: <f61f04b1-a61c-f7af-624e-798ec4ad4897@oracle.com>
References: <f61f04b1-a61c-f7af-624e-798ec4ad4897@oracle.com>
Message-ID: <1511276942.2258.0.camel@oracle.com>

Hi,

On Tue, 2017-11-21 at 15:56 +0100, Leo Korinth wrote:
> Hi,
> 
> Removing an if statement. The if-statement did safeguard that the
> worker_id was always within the number of active tasks in _cm.
> 
> However the active number of tasks in _cm is set directly before the
> remark task is created (set_concurrency_and_phase()). The value is
> taken from the work gang (g1h->workers()->active_workers()). Thus, as
> set_concurrency*() is not called during the actual remarking, the
> size
> of the work-gang will be in sync with the _cm->active_tasks() -- at
> least during the remark phase.
> 
> The code becomes somewhat easier to read, and hopefully one does not
> get confused that we have too many threads running.
> 
> Bug:
> https://bugs.openjdk.java.net/browse/JDK-8190408
> 
> Webrev:
> http://cr.openjdk.java.net/~lkorinth/8190408/00/
> 

Looks good.

Thanks,
  Thomas


From stefan.johansson at oracle.com  Tue Nov 21 15:34:04 2017
From: stefan.johansson at oracle.com (Stefan Johansson)
Date: Tue, 21 Nov 2017 16:34:04 +0100
Subject: RFR: 8190408: Run G1CMRemarkTask with the appropriate amount of
 threads instead of starting up everyone
In-Reply-To: <1511276942.2258.0.camel@oracle.com>
References: <f61f04b1-a61c-f7af-624e-798ec4ad4897@oracle.com>
 <1511276942.2258.0.camel@oracle.com>
Message-ID: <9bdf8347-573e-1a52-ec3f-d5cc5d73800e@oracle.com>



On 2017-11-21 16:09, Thomas Schatzl wrote:
> Hi,
>
> On Tue, 2017-11-21 at 15:56 +0100, Leo Korinth wrote:
>> Hi,
>>
>> Removing an if statement. The if-statement did safeguard that the
>> worker_id was always within the number of active tasks in _cm.
>>
>> However the active number of tasks in _cm is set directly before the
>> remark task is created (set_concurrency_and_phase()). The value is
>> taken from the work gang (g1h->workers()->active_workers()). Thus, as
>> set_concurrency*() is not called during the actual remarking, the
>> size
>> of the work-gang will be in sync with the _cm->active_tasks() -- at
>> least during the remark phase.
>>
>> The code becomes somewhat easier to read, and hopefully one does not
>> get confused that we have too many threads running.
>>
>> Bug:
>> https://bugs.openjdk.java.net/browse/JDK-8190408
>>
>> Webrev:
>> http://cr.openjdk.java.net/~lkorinth/8190408/00/
>>
> Looks good.
+1
StefanJ
>
> Thanks,
>    Thomas



From daniel.daugherty at oracle.com  Tue Nov 21 16:28:56 2017
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Tue, 21 Nov 2017 11:28:56 -0500
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <e56b44ce-28a9-822e-bbac-4b64f5f85c58@oracle.com>
References: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
 <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>
 <836b5c6c-5eef-9245-c16a-e71372093d9a@oracle.com>
 <82d0e473-a24f-9feb-82e9-7d04c4d7a91d@oracle.com>
 <354f7ece-66bc-c065-9ee8-f55abcc1c23e@oracle.com>
 <0ff8df8e-e93c-02d2-cb97-b13b1ac0f53f@oracle.com>
 <e56b44ce-28a9-822e-bbac-4b64f5f85c58@oracle.com>
Message-ID: <70cc1311-de55-5502-159b-cd0a3522cfd6@oracle.com>

Hi Coleen!

Thanks for making time to review the Thread-SMR stuff again!!

I have added back the other three OpenJDK aliases... This review is
being done on _four_ different OpenJDK aliases.

As always, replies are embedded below...


On 11/20/17 3:12 PM, coleen.phillimore at oracle.com wrote:
>
> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/src/hotspot/os/linux/os_linux.cpp.frames.html 
>
>
> I read David's comments about the threads list iterator, and I was 
> going to say that it can be cleaned up later, as the bulk of the 
> change is the SMR part but this looks truly bizarre.?? It looks like 
> it shouldn't compile because 'jt' isn't in scope.
>
> Why isn't this the syntax:
>
> JavaThreadIteratorWithHandle jtiwh;
> for (JavaThread* jt = jtiwh.first(); jt != NULL; jt = jtiwh.next()) {
> }
>
> Which would do the obvious thing without anyone having to squint at 
> the code.

See my reply to David's review for the more detailed answer.

For the above syntax, we would need braces to limit the scope of the
'jtiwh' variable. With Stefan's propsal, you get limited scope on
'jtiwh' for "free".


> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/src/hotspot/share/runtime/threadSMR.hpp.html 
>
>
> As a hater of acronmys, can this file use "Safe Memory Reclaimation"

I'm not sure what you mean by this. Do you mean rename the files?

 ? threadSMR.hpp -> threadSafeMemoryReclaimation.hpp
 ? threadSMR.cpp -> threadSafeMemoryReclaimation.cpp


> and briefly describe the concept in the beginning of the header file,
> so one knows why it's called threadSMR.hpp?

And then this part of the sentence kind of indicates that you would be
okay with the threadSMR.{c,h}pp names if a comment was added to the
header file.

Please clarify.


> It doesn't need to be long and include why Threads list needs this 
> Sometimes we tell new people that the hotspot documentation is in the 
> header files.

Yup. When I migrated stuff from thread.hpp and thread.cpp to threadSMR.hpp
and threadSMR.cpp, I should have written a header comment...

I did update a comment in thread.cpp based on Robin W's code review:

> > src/hotspot/share/runtime/thread.cpp
> >
> > 3432 // operations over all threads.? It is protected by its own Mutex
> > 3433 // lock, which is also used in other contexts to protect thread
> >
> > Should this comment perhaps be revised to mention SMR?
>
> It definitely needs some updating... Here's a stab at it:
>
> // The Threads class links together all active threads, and provides
> // operations over all threads. It is protected by the Threads_lock,
> // which is also used in other global contexts like safepointing.
> // ThreadsListHandles are used to safely perform operations on one
> // or more threads without the risk of the thread exiting during the
> // operation.
> //
> // Note: The Threads_lock is currently more widely used than we
> // would like. We are actively migrating Threads_lock uses to other
> // mechanisms in order to reduce Threads_lock contention. 

I'll take a look at adding a header comment to threadSMR.hpp.


> 186?? JavaThreadIteratorWithHandle() : _tlh(), _index(0) {}
>
> This _tlh() call should not be necessary.? The compiler should 
> generate this for you in the constructor.

Deleted.


> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/src/hotspot/share/runtime/threadSMR.cpp.html 
>
>
> ? 32 ThreadsList::ThreadsList(int entries) : _length(entries), 
> _threads(NEW_C_HEAP_ARRAY(JavaThread*, entries + 1, mtGC)), 
> _next_list(NULL) {
>
> Seems like it should be mtThread rather than mtGC.

Fixed. Definitely an artifact of Erik's original prototype when he
extracted Thread-SMR from his GC work... Thanks for catching it.


> Should
>
> ? 62?? if (EnableThreadSMRStatistics) {
>
> really be UL, ie: if (log_is_enabled(Info, thread, smr, statistics)) ?

EnableThreadSMRStatistics is used in more places than UL code.
We use it in Thread::print_*() stuff to control output of
Thread-SMR statistics info in thread dumps and hs_err_pid file
generation.

Currently thread dump and hs_err_pid file output is not generated
using UL (and probably can't be?) so we need an option to control
the Thread-SMR statistics stuff in all places.



> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/test/hotspot/jtreg/runtime/ErrorHandling/NestedThreadsListHandleInErrorHandlingTest.java.html 
>
>
> Can you use for these tests instead (there were a couple):
>
> *@requires (vm.debug == true)*

The test I cloned had this in it:

 ??? if (!Platform.isDebugBuild()) {
 ??????? // -XX:ErrorHandlerTest=N option requires debug bits.
 ??????? return;
 ??? }

and you would like me to switch to the newer mechanism?

I have updated the following tests:

 ? test/hotspot/jtreg/runtime/ErrorHandling/ErrorHandler.java
test/hotspot/jtreg/runtime/ErrorHandling/NestedThreadsListHandleInErrorHandlingTest.java
test/hotspot/jtreg/runtime/ErrorHandling/ThreadsListHandleInErrorHandlingTest.java


> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/src/hotspot/share/runtime/thread.cpp.udiff.html 
>
>
> +// thread, has been added the Threads list, the system is not at a
>
> Has "not" been added to the Threads list?? missing "not"?

Nope. If the JavaThread has been added to the Threads list
and it is not protected, then it is dangling. In other words,
a published JavaThread (on the Threads list) has to be protected
by either the system being at a safepoint or the JavaThread has
to be on some threads's ThreadsList.


>
> + return (unsigned int)(((uint32_t)(uintptr_t)s1) * 2654435761u);
>
> Can you add a comment about where this number came from?

I'll have to get that from Erik...


> I can't find the caller of threads_do_smr.

I'm guessing that's used by the GC code that depends on Thread-SMR...


> If these functions xchg_smr_thread_list, get_smr_java_thread_list, 
> inc_smr_deleted_thread_count are only used by thread.cpp, I think they 
> should go in that file and not in the .inline.hpp file to be included 
> and possibly called by other files.? I think they're private to class 
> Threads.

I have a vague memory that some of the compilers don't do inlining when
an "inline" function is in a .cpp. I believe we want these functions
to be inlined for performance reasons. Erik should probably chime in
here.


> I don't have any in-depth comments.? This looks really good from my 
> day of reading it.

Thanks for taking the time to review it again!

Dan


>
> Thanks,
> Coleen
>
> On 11/18/17 8:49 PM, Daniel D. Daugherty wrote:
>
>> Greetings,
>>
>> Testing of the last round of changes revealed a hang in one of the new
>> TLH tests. Robbin has fixed the hang, updated the existing TLH test, and
>> added another TLH test for good measure.
>>
>> Here is the updated full webrev:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/
>>
>> Here is the updated delta webrev:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-delta/
>>
>> Dan ran the bits thru the usual Mach5 tier[1-5] testing and there are
>> no unexpected failures.
>>
>> We welcome comments, suggestions and feedback.
>>
>> Dan, Erik and Robbin
>>
>>
>> On 11/15/17 3:32 PM, Daniel D. Daugherty wrote:
>>> Greetings,
>>>
>>> Robbin rebased the project last night/this morning to merge with Thread
>>> Local Handshakes (TLH) and also picked up additional changesets up 
>>> thru:
>>>
>>>> Changeset: fa736014cf28
>>>> Author:??? cjplummer
>>>> Date:????? 2017-11-14 18:08 -0800
>>>> URL:http://hg.openjdk.java.net/jdk/hs/rev/fa736014cf28
>>>>
>>>> 8191049: Add alternate version of pns() that is callable from 
>>>> within hotspot source
>>>> Summary: added pns2() to debug.cpp
>>>> Reviewed-by: stuefe, gthornbr
>>>
>>> This is the first time we've rebased the project to bits that are this
>>> fresh (< 12 hours old at rebase time). We've done this because we think
>>> we're done with this project and are in the final review-change-retest
>>> cycle(s)... In other words, we're not planning on making any more major
>>> changes for this project... :-)
>>>
>>> *** Time for code reviewers to chime in on this thread! ***
>>>
>>> Here is the updated full webrev:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-full/
>>>
>>> Here's is the delta webrev from the 2017.11.10 rebase:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-delta/
>>>
>>> Dan has submitted the bits for the usual Mach5 tier[1-5] testing
>>> (and the new baseline also)...
>>>
>>> We're expecting this round to be a little noisier than usual because
>>> we did not rebase on a PIT snapshot so the new baseline has not been
>>> through Jesper's usual care-and-feeding of integration_blockers, etc.
>>>
>>> We welcome comments, suggestions and feedback.
>>>
>>> Dan, Erik and Robbin
>>>
>>>
>>> On 11/14/17 4:48 PM, Daniel D. Daugherty wrote:
>>>> Greetings,
>>>>
>>>> I rebased the project to the 2017.11.10 jdk/hs PIT snapshot.
>>>> (Yes, we're playing chase-the-repo...)
>>>>
>>>> Here is the updated full webrev:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-07-full/
>>>>
>>>> Unlike the previous rebase, there were no changes required to the
>>>> open code to get the rebased bits to build so there is no delta
>>>> webrev.
>>>>
>>>> These bits have been run through JPRT and I've submitted the usual
>>>> Mach5 tier[1-5] test run...
>>>>
>>>> We welcome comments, suggestions and feedback.
>>>>
>>>> Dan, Erik and Robbin
>>>>
>>>>
>>>> On 11/13/17 12:30 PM, Daniel D. Daugherty wrote:
>>>>> Greetings,
>>>>>
>>>>> I rebased the project to the 2017.10.26 jdk10/hs PIT snapshot.
>>>>>
>>>>> Here are the updated webrevs:
>>>>>
>>>>> Here's the mq comment for the change:
>>>>>
>>>>> ? Rebase to 2017.10.25 PIT snapshot.
>>>>>
>>>>> Here is the full webrev:
>>>>>
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-full/
>>>>>
>>>>> And here is the delta webrev:
>>>>>
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-delta/
>>>>>
>>>>> I ran the above bits throught Mach5 tier[1-5] testing over the 
>>>>> holiday
>>>>> weekend. Didn't see any issues in a quick look. Going to take a 
>>>>> closer
>>>>> look today.
>>>>>
>>>>> We welcome comments, suggestions and feedback.
>>>>>
>>>>> Dan, Erik and Robbin
>>>>>
>>>>>
>>>>> On 11/8/17 1:05 PM, Daniel D. Daugherty wrote:
>>>>>> Greetings,
>>>>>>
>>>>>> Resolving one of the code review comments (from both Stefan K and 
>>>>>> Coleen)
>>>>>> on jdk10-04-full required quite a few changes so it is being done 
>>>>>> as a
>>>>>> standalone patch and corresponding webrevs:
>>>>>>
>>>>>> Here's the mq comment for the change:
>>>>>>
>>>>>> ? stefank, coleenp CR - refactor most JavaThreadIterator usage to 
>>>>>> use
>>>>>> ??? JavaThreadIteratorWithHandle.
>>>>>>
>>>>>> Here is the full webrev:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-full/
>>>>>>
>>>>>> And here is the delta webrev:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-delta/
>>>>>>
>>>>>> We welcome comments, suggestions and feedback.
>>>>>>
>>>>>> Dan, Erik and Robbin
>>>>>>
>>>>>>
>>>>>> On 10/9/17 3:41 PM, Daniel D. Daugherty wrote:
>>>>>>> Greetings,
>>>>>>>
>>>>>>> We have a (eXtra Large) fix for the following bug:
>>>>>>>
>>>>>>> 8167108 inconsistent handling of SR_lock can lead to crashes
>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8167108
>>>>>>>
>>>>>>> This fix adds a Safe Memory Reclamation (SMR) mechanism based on
>>>>>>> Hazard Pointers to manage JavaThread lifecycle.
>>>>>>>
>>>>>>> Here's a PDF for the internal wiki that we've been using to 
>>>>>>> describe
>>>>>>> and track the work on this project:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/SMR_and_JavaThread_Lifecycle-JDK10-04.pdf 
>>>>>>>
>>>>>>>
>>>>>>> Dan has noticed that the indenting is wrong in some of the code 
>>>>>>> quotes
>>>>>>> in the PDF that are not present in the internal wiki. We don't 
>>>>>>> have a
>>>>>>> solution for that problem yet.
>>>>>>>
>>>>>>> Here's the webrev for current JDK10 version of this fix:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-04-full
>>>>>>>
>>>>>>> This fix has been run through many rounds of JPRT and Mach5 
>>>>>>> tier[2-5]
>>>>>>> testing, additional stress testing on Dan's Solaris X64 server, and
>>>>>>> additional testing on Erik and Robbin's machines.
>>>>>>>
>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>
>>>>>>> Daniel Daugherty
>>>>>>> Erik Osterlund
>>>>>>> Robbin Ehn
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>



From coleen.phillimore at oracle.com  Tue Nov 21 17:24:11 2017
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Tue, 21 Nov 2017 12:24:11 -0500
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <70cc1311-de55-5502-159b-cd0a3522cfd6@oracle.com>
References: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
 <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>
 <836b5c6c-5eef-9245-c16a-e71372093d9a@oracle.com>
 <82d0e473-a24f-9feb-82e9-7d04c4d7a91d@oracle.com>
 <354f7ece-66bc-c065-9ee8-f55abcc1c23e@oracle.com>
 <0ff8df8e-e93c-02d2-cb97-b13b1ac0f53f@oracle.com>
 <e56b44ce-28a9-822e-bbac-4b64f5f85c58@oracle.com>
 <70cc1311-de55-5502-159b-cd0a3522cfd6@oracle.com>
Message-ID: <68b0b2bc-2d26-5442-bafb-5ce21420767b@oracle.com>



On 11/21/17 11:28 AM, Daniel D. Daugherty wrote:
> Hi Coleen!
>
> Thanks for making time to review the Thread-SMR stuff again!!
>
> I have added back the other three OpenJDK aliases... This review is
> being done on _four_ different OpenJDK aliases.
>
> As always, replies are embedded below...
>
>
> On 11/20/17 3:12 PM, coleen.phillimore at oracle.com wrote:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/src/hotspot/os/linux/os_linux.cpp.frames.html 
>>
>>
>> I read David's comments about the threads list iterator, and I was 
>> going to say that it can be cleaned up later, as the bulk of the 
>> change is the SMR part but this looks truly bizarre.?? It looks like 
>> it shouldn't compile because 'jt' isn't in scope.
>>
>> Why isn't this the syntax:
>>
>> JavaThreadIteratorWithHandle jtiwh;
>> for (JavaThread* jt = jtiwh.first(); jt != NULL; jt = jtiwh.next()) {
>> }
>>
>> Which would do the obvious thing without anyone having to squint at 
>> the code.
>
> See my reply to David's review for the more detailed answer.
>
> For the above syntax, we would need braces to limit the scope of the
> 'jtiwh' variable. With Stefan's propsal, you get limited scope on
> 'jtiwh' for "free".

Yes, thank you, I see that motivation now.
>
>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/src/hotspot/share/runtime/threadSMR.hpp.html 
>>
>>
>> As a hater of acronmys, can this file use "Safe Memory Reclaimation"
>
> I'm not sure what you mean by this. Do you mean rename the files?
>
> ? threadSMR.hpp -> threadSafeMemoryReclaimation.hpp
> ? threadSMR.cpp -> threadSafeMemoryReclaimation.cpp
>

No.

>
>> and briefly describe the concept in the beginning of the header file,
>> so one knows why it's called threadSMR.hpp?
>
> And then this part of the sentence kind of indicates that you would be
> okay with the threadSMR.{c,h}pp names if a comment was added to the
> header file.
>
> Please clarify.

Yes.? If you added a comment to the beginning of the threadSMR.hpp file 
that said what SMR stood for and what it was, that would be extremely 
helpful for future viewers of this file.

>
>
>> It doesn't need to be long and include why Threads list needs this 
>> Sometimes we tell new people that the hotspot documentation is in the 
>> header files.
>
> Yup. When I migrated stuff from thread.hpp and thread.cpp to 
> threadSMR.hpp
> and threadSMR.cpp, I should have written a header comment...
>
> I did update a comment in thread.cpp based on Robin W's code review:

Yes, I think a comment belongs in the SMR file also, or in preference.
>
>> > src/hotspot/share/runtime/thread.cpp
>> >
>> > 3432 // operations over all threads.? It is protected by its own Mutex
>> > 3433 // lock, which is also used in other contexts to protect thread
>> >
>> > Should this comment perhaps be revised to mention SMR?
>>
>> It definitely needs some updating... Here's a stab at it:
>>
>> // The Threads class links together all active threads, and provides
>> // operations over all threads. It is protected by the Threads_lock,
>> // which is also used in other global contexts like safepointing.
>> // ThreadsListHandles are used to safely perform operations on one
>> // or more threads without the risk of the thread exiting during the
>> // operation.
>> //
>> // Note: The Threads_lock is currently more widely used than we
>> // would like. We are actively migrating Threads_lock uses to other
>> // mechanisms in order to reduce Threads_lock contention. 
>
> I'll take a look at adding a header comment to threadSMR.hpp.
>
>
>> 186?? JavaThreadIteratorWithHandle() : _tlh(), _index(0) {}
>>
>> This _tlh() call should not be necessary.? The compiler should 
>> generate this for you in the constructor.
>
> Deleted.
>
>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/src/hotspot/share/runtime/threadSMR.cpp.html 
>>
>>
>> ? 32 ThreadsList::ThreadsList(int entries) : _length(entries), 
>> _threads(NEW_C_HEAP_ARRAY(JavaThread*, entries + 1, mtGC)), 
>> _next_list(NULL) {
>>
>> Seems like it should be mtThread rather than mtGC.
>
> Fixed. Definitely an artifact of Erik's original prototype when he
> extracted Thread-SMR from his GC work... Thanks for catching it.
>
>
>> Should
>>
>> ? 62?? if (EnableThreadSMRStatistics) {
>>
>> really be UL, ie: if (log_is_enabled(Info, thread, smr, statistics)) ?
>
> EnableThreadSMRStatistics is used in more places than UL code.
> We use it in Thread::print_*() stuff to control output of
> Thread-SMR statistics info in thread dumps and hs_err_pid file
> generation.

That sort of thing is also controlled by logging in general.
>
> Currently thread dump and hs_err_pid file output is not generated
> using UL (and probably can't be?) so we need an option to control
> the Thread-SMR statistics stuff in all places.
>

You can use log options in preference to this new diagnostic option in 
these cases too.?? This option looks a lot like logging to me and would 
be nice to not have to later convert it.

>
>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/test/hotspot/jtreg/runtime/ErrorHandling/NestedThreadsListHandleInErrorHandlingTest.java.html 
>>
>>
>> Can you use for these tests instead (there were a couple):
>>
>> *@requires (vm.debug == true)*
>
> The test I cloned had this in it:
>
> ??? if (!Platform.isDebugBuild()) {
> ??????? // -XX:ErrorHandlerTest=N option requires debug bits.
> ??????? return;
> ??? }
>
> and you would like me to switch to the newer mechanism?
>
> I have updated the following tests:
>
> ? test/hotspot/jtreg/runtime/ErrorHandling/ErrorHandler.java
> test/hotspot/jtreg/runtime/ErrorHandling/NestedThreadsListHandleInErrorHandlingTest.java 
>
> test/hotspot/jtreg/runtime/ErrorHandling/ThreadsListHandleInErrorHandlingTest.java 
>
>
>
Yes, the newer mechanism makes jtreg not bother to run the test, which 
makes it faster!

>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/src/hotspot/share/runtime/thread.cpp.udiff.html 
>>
>>
>> +// thread, has been added the Threads list, the system is not at a
>>
>> Has "not" been added to the Threads list?? missing "not"?
>
> Nope. If the JavaThread has been added to the Threads list
> and it is not protected, then it is dangling. In other words,
> a published JavaThread (on the Threads list) has to be protected
> by either the system being at a safepoint or the JavaThread has
> to be on some threads's ThreadsList.
>
>
>>
>> + return (unsigned int)(((uint32_t)(uintptr_t)s1) * 2654435761u);
>>
>> Can you add a comment about where this number came from?
>
> I'll have to get that from Erik...
>
>
>> I can't find the caller of threads_do_smr.
>
> I'm guessing that's used by the GC code that depends on Thread-SMR...
>

They? should add this API when the add the use of it then.? I don't see 
it in any sources.

>
>> If these functions xchg_smr_thread_list, get_smr_java_thread_list, 
>> inc_smr_deleted_thread_count are only used by thread.cpp, I think 
>> they should go in that file and not in the .inline.hpp file to be 
>> included and possibly called by other files.? I think they're private 
>> to class Threads.
>
> I have a vague memory that some of the compilers don't do inlining when
> an "inline" function is in a .cpp. I believe we want these functions
> to be inlined for performance reasons. Erik should probably chime in
> here.

I don't see why this should be.? Maybe some (older) compilers require it 
to be found before the call though, but that can still be accomplished 
in the .cpp file.
>
>
>> I don't have any in-depth comments.? This looks really good from my 
>> day of reading it.
>
> Thanks for taking the time to review it again!
>

Thanks,
Coleen

> Dan
>
>
>>
>> Thanks,
>> Coleen
>>
>> On 11/18/17 8:49 PM, Daniel D. Daugherty wrote:
>>
>>> Greetings,
>>>
>>> Testing of the last round of changes revealed a hang in one of the new
>>> TLH tests. Robbin has fixed the hang, updated the existing TLH test, 
>>> and
>>> added another TLH test for good measure.
>>>
>>> Here is the updated full webrev:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/
>>>
>>> Here is the updated delta webrev:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-delta/
>>>
>>> Dan ran the bits thru the usual Mach5 tier[1-5] testing and there are
>>> no unexpected failures.
>>>
>>> We welcome comments, suggestions and feedback.
>>>
>>> Dan, Erik and Robbin
>>>
>>>
>>> On 11/15/17 3:32 PM, Daniel D. Daugherty wrote:
>>>> Greetings,
>>>>
>>>> Robbin rebased the project last night/this morning to merge with 
>>>> Thread
>>>> Local Handshakes (TLH) and also picked up additional changesets up 
>>>> thru:
>>>>
>>>>> Changeset: fa736014cf28
>>>>> Author:??? cjplummer
>>>>> Date:????? 2017-11-14 18:08 -0800
>>>>> URL:http://hg.openjdk.java.net/jdk/hs/rev/fa736014cf28
>>>>>
>>>>> 8191049: Add alternate version of pns() that is callable from 
>>>>> within hotspot source
>>>>> Summary: added pns2() to debug.cpp
>>>>> Reviewed-by: stuefe, gthornbr
>>>>
>>>> This is the first time we've rebased the project to bits that are this
>>>> fresh (< 12 hours old at rebase time). We've done this because we 
>>>> think
>>>> we're done with this project and are in the final review-change-retest
>>>> cycle(s)... In other words, we're not planning on making any more 
>>>> major
>>>> changes for this project... :-)
>>>>
>>>> *** Time for code reviewers to chime in on this thread! ***
>>>>
>>>> Here is the updated full webrev:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-full/
>>>>
>>>> Here's is the delta webrev from the 2017.11.10 rebase:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-delta/
>>>>
>>>> Dan has submitted the bits for the usual Mach5 tier[1-5] testing
>>>> (and the new baseline also)...
>>>>
>>>> We're expecting this round to be a little noisier than usual because
>>>> we did not rebase on a PIT snapshot so the new baseline has not been
>>>> through Jesper's usual care-and-feeding of integration_blockers, etc.
>>>>
>>>> We welcome comments, suggestions and feedback.
>>>>
>>>> Dan, Erik and Robbin
>>>>
>>>>
>>>> On 11/14/17 4:48 PM, Daniel D. Daugherty wrote:
>>>>> Greetings,
>>>>>
>>>>> I rebased the project to the 2017.11.10 jdk/hs PIT snapshot.
>>>>> (Yes, we're playing chase-the-repo...)
>>>>>
>>>>> Here is the updated full webrev:
>>>>>
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-07-full/
>>>>>
>>>>> Unlike the previous rebase, there were no changes required to the
>>>>> open code to get the rebased bits to build so there is no delta
>>>>> webrev.
>>>>>
>>>>> These bits have been run through JPRT and I've submitted the usual
>>>>> Mach5 tier[1-5] test run...
>>>>>
>>>>> We welcome comments, suggestions and feedback.
>>>>>
>>>>> Dan, Erik and Robbin
>>>>>
>>>>>
>>>>> On 11/13/17 12:30 PM, Daniel D. Daugherty wrote:
>>>>>> Greetings,
>>>>>>
>>>>>> I rebased the project to the 2017.10.26 jdk10/hs PIT snapshot.
>>>>>>
>>>>>> Here are the updated webrevs:
>>>>>>
>>>>>> Here's the mq comment for the change:
>>>>>>
>>>>>> ? Rebase to 2017.10.25 PIT snapshot.
>>>>>>
>>>>>> Here is the full webrev:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-full/
>>>>>>
>>>>>> And here is the delta webrev:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-delta/
>>>>>>
>>>>>> I ran the above bits throught Mach5 tier[1-5] testing over the 
>>>>>> holiday
>>>>>> weekend. Didn't see any issues in a quick look. Going to take a 
>>>>>> closer
>>>>>> look today.
>>>>>>
>>>>>> We welcome comments, suggestions and feedback.
>>>>>>
>>>>>> Dan, Erik and Robbin
>>>>>>
>>>>>>
>>>>>> On 11/8/17 1:05 PM, Daniel D. Daugherty wrote:
>>>>>>> Greetings,
>>>>>>>
>>>>>>> Resolving one of the code review comments (from both Stefan K 
>>>>>>> and Coleen)
>>>>>>> on jdk10-04-full required quite a few changes so it is being 
>>>>>>> done as a
>>>>>>> standalone patch and corresponding webrevs:
>>>>>>>
>>>>>>> Here's the mq comment for the change:
>>>>>>>
>>>>>>> ? stefank, coleenp CR - refactor most JavaThreadIterator usage 
>>>>>>> to use
>>>>>>> ??? JavaThreadIteratorWithHandle.
>>>>>>>
>>>>>>> Here is the full webrev:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-full/
>>>>>>>
>>>>>>> And here is the delta webrev:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-delta/
>>>>>>>
>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>
>>>>>>> Dan, Erik and Robbin
>>>>>>>
>>>>>>>
>>>>>>> On 10/9/17 3:41 PM, Daniel D. Daugherty wrote:
>>>>>>>> Greetings,
>>>>>>>>
>>>>>>>> We have a (eXtra Large) fix for the following bug:
>>>>>>>>
>>>>>>>> 8167108 inconsistent handling of SR_lock can lead to crashes
>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8167108
>>>>>>>>
>>>>>>>> This fix adds a Safe Memory Reclamation (SMR) mechanism based on
>>>>>>>> Hazard Pointers to manage JavaThread lifecycle.
>>>>>>>>
>>>>>>>> Here's a PDF for the internal wiki that we've been using to 
>>>>>>>> describe
>>>>>>>> and track the work on this project:
>>>>>>>>
>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/SMR_and_JavaThread_Lifecycle-JDK10-04.pdf 
>>>>>>>>
>>>>>>>>
>>>>>>>> Dan has noticed that the indenting is wrong in some of the code 
>>>>>>>> quotes
>>>>>>>> in the PDF that are not present in the internal wiki. We don't 
>>>>>>>> have a
>>>>>>>> solution for that problem yet.
>>>>>>>>
>>>>>>>> Here's the webrev for current JDK10 version of this fix:
>>>>>>>>
>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-04-full
>>>>>>>>
>>>>>>>> This fix has been run through many rounds of JPRT and Mach5 
>>>>>>>> tier[2-5]
>>>>>>>> testing, additional stress testing on Dan's Solaris X64 server, 
>>>>>>>> and
>>>>>>>> additional testing on Erik and Robbin's machines.
>>>>>>>>
>>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>>
>>>>>>>> Daniel Daugherty
>>>>>>>> Erik Osterlund
>>>>>>>> Robbin Ehn
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>



From daniel.daugherty at oracle.com  Tue Nov 21 17:30:52 2017
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Tue, 21 Nov 2017 12:30:52 -0500
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <9cc8b08b-f373-f493-4429-662d18ff81ad@oracle.com>
References: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
 <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>
 <836b5c6c-5eef-9245-c16a-e71372093d9a@oracle.com>
 <82d0e473-a24f-9feb-82e9-7d04c4d7a91d@oracle.com>
 <354f7ece-66bc-c065-9ee8-f55abcc1c23e@oracle.com>
 <0ff8df8e-e93c-02d2-cb97-b13b1ac0f53f@oracle.com>
 <43d2f342-3f4b-5c1a-a687-c24c4efedba9@oracle.com>
 <92d1f572-a745-6f2b-1b5f-e68a1d252bab@oracle.com>
 <9cc8b08b-f373-f493-4429-662d18ff81ad@oracle.com>
Message-ID: <e043273a-9ddd-13b7-feac-2246a755c8ab@oracle.com>

Adding back the missing OpenJDK aliases...


On 11/21/17 11:14 AM, coleen.phillimore at oracle.com wrote:
>
>
> On 11/20/17 8:50 PM, Daniel D. Daugherty wrote:
>> On 11/20/17 12:51 AM, David Holmes wrote:
>>> Hi Dan,
>>>
>>> Figured I'd better cast an eye over this again before it gets pushed :)
>>
>> Thanks for reviewing again!!
>>
>>
>>> Only one thing (repeated many times) jumped out at me and apologies 
>>> if this has already been debated back and forth. I missed the 
>>> exchange where the for loop iteration was rewritten into this 
>>> unusual form:
>>>
>>> for (JavaThreadIteratorWithHandle jtiwh; JavaThread *jt = 
>>> jtiwh.next(); ) {
>>>
>>> On first reading I expect most people would mistake the control 
>>> expression for the iteration-expression due to the next(). I didn't 
>>> even know the control expression could introduce a local variable! I 
>>> find this really hard to read stylistically and far too cute/clever 
>>> for its own good. It also violates the style rules regarding 
>>> implicit boolean checks.
>>>
>>> I know Stefan proposed this as he did not like the alternate (in a 
>>> few places):
>>>
>>> +? {
>>> +??? ThreadsListHandle tlh;
>>> +??? JavaThreadIterator jti(tlh.list());
>>> +??? for (JavaThread* t = jti.first(); t != NULL; t = jti.next()) {
>>> ...
>>> +? }
>>>
>>> but it seems to me the iterator code has evolved since then and only 
>>> a couple of places need a distinct TLH separate from the actual 
>>> iterator object. So I'm more in favour of the more classic for-loop, 
>>> with the iterator declared before the loop. Or even convert to while 
>>> loops, as this iterator seems more suited to that.
>>
>> Actually both Coleen and Stefan had a problem with how much additional
>> code was added to support an iterator model. In some cases we went from
>> 1-line to 3-lines and in some cases we went from 1-line to 5-lines. For
>> Coleen, she wanted 2 of the new lines compressed into 1 new line by 
>> using
>> an iterator with an embedded ThreadsListHandle. Stefan wanted us to try
>> and get back to 1-line where possible.
>>
>> The advantages of the new style are:
>>
>> - 1-line to 1-line in all but a few cases
>> - automatic limited scope of the embedded ThreadsListHandle so we were
>> ? able to remove extra braces that were added earlier in most cases
>>
>> The disadvantages of the new style are:
>>
>> - it is an unusual for-loop style (in HotSpot)
>> - it breaks HotSpot's style rule about implied booleans
>>
>>
>> Stefan K is pretty adamant that the original iterator version where we
>> go from 1-line to 3-lines or 1-line to 5-lines is not acceptable for GC
>> code. The compiler guys haven't chimed in on this debate so I'm guessing
>> they don't have a strong opinion either way. One thing that we don't 
>> want
>> to do is have different styles for the different teams.
>>
>> So I had to make a judgement call on whether I thought Runtime could 
>> live
>> with Stefan's idea. Originally I wasn't fond of it either and 
>> breaking the
>> implied boolean style rule is a problem for me (I post that comment a 
>> lot
>> in my code reviews). However, I have to say that I'm a lot happier about
>> the compactness of the code and the fact that limited ThreadsListHandle
>> scope comes for 'free' in most cases.
>>
>> We're planning to keep the new iterator style for the following reasons:
>>
>> - smaller change footprint
>> - consistent style used for all of the teams
>> - limited ThreadsListHandle scope comes for free in most cases
>>
>> We're hoping that you can live with this decision (for now) and maybe
>> even grow to like it in the future. :-)
>
> The limited ThreadsListHandle scope, since ThreadsListHandle registers 
> and unregisters lists to SMR, seems to be worth the cost of looking at 
> this strange code pattern.?? Thank you for the explanation.
>
>>
>>
>>> Other than that just a couple of comments on variable type choices, 
>>> and a few typos spotted below.
>>
>> Replies embedded below.
>>
>>
>>>
>>> Thanks,
>>> David
>>> ---
>>>
>>> src/hotspot/share/runtime/globals.hpp
>>>
>>> 2476?? diagnostic(bool, EnableThreadSMRExtraValidityChecks, true, 
>>> ??????? \
>>> 2477?????????? "Enable Thread SMR extra validity checks") \
>>> 2478 ??????? \
>>> 2479?? diagnostic(bool, EnableThreadSMRStatistics, true, \
>>> 2480?????????? "Enable Thread SMR Statistics") ??????? \
>>>
>>> Indent of strings is off by? 3 spaces.
>>
>> Fixed.
>>
>>
>>> ---
>>>
>>> src/hotspot/share/runtime/handshake.cpp
>>>
>>> ?140?????? // There is assumption in code that Threads_lock should 
>>> be lock
>>> ?200?????? // There is assumption in code that Threads_lock should 
>>> be lock
>>>
>>> lock -> locked
>>
>> Fixed.
>>
>>
>>> 146???????? // handshake_process_by_vmthread will check the 
>>> semaphore for us again
>>>
>>> Needs period at end.
>>
>> Fixed.
>>
>>
>>> 148???????? // should be okey since the new thread will not have an 
>>> operation.
>>>
>>> okey -> okay
>>
>> Fixed.
>>
>>
>>> 151???????? // We can't warn here is since the thread do 
>>> cancel_handshake after it have been removed
>>>
>>> I think:
>>>
>>> // We can't warn here since the thread does cancel_handshake after 
>>> it has been removed
>>
>> Fixed.
>>
>>
>>> 152???????? // from ThreadsList. So we should just keep looping here 
>>> until while below return negative.
>>>
>>> from -> from the
>>>
>>> Needs period at end.
>>
>> Fixed both.
>>
>>
>>>
>>> ?204???????????? // A new thread on the ThreadsList will not have an 
>>> operation.
>>>
>>> Change period to comma, and ...
>>
>> Fixed.
>>
>>
>>> 205???????????? // Hence is skipped in handshake_process_by_vmthread.
>>>
>>> Hence is -> hence it is
>>
>> Fixed.
>>
>>
>>> ---
>>>
>>> src/hotspot/share/runtime/thread.cpp
>>>
>>> 1482???? // dcubed - Looks like the Threads_lock is for stable access
>>> 1483???? // to _jvmci_old_thread_counters and _jvmci_counters.
>>>
>>> Does this comment need revisiting?
>>
>> We've been trying to decide what to do about this comment. We'll be
>> filing a follow up bug for the Compiler team to take another look at
>> how _jvmci_old_thread_counters and _jvmci_counters are protected.
>> Threads_lock is a big hammer and there should be a less expensive
>> solution. Once that bug gets filed, this comment can go away.
>>
>>
>>> 3451 volatile jint ...
>>>
>>> Why are you using jint for field types here? (Surprised Coleen 
>>> hasn't spotted them! ;-) ).
>
> Yes, it was the jtypes I would have objected to.? And long isn't a 
> good choice on windows because it's only 32 bits there.
>>
>> volatile jint???????? Threads::_smr_delete_notify = 0;
>> volatile jint???????? Threads::_smr_deleted_thread_cnt = 0;
>> volatile jint???????? Threads::_smr_deleted_thread_time_max = 0;
>> volatile jint???????? Threads::_smr_deleted_thread_times = 0;
>> :
>> volatile jint???????? Threads::_smr_tlh_cnt = 0;
>> volatile jint???????? Threads::_smr_tlh_time_max = 0;
>> volatile jint???????? Threads::_smr_tlh_times = 0;
>>
>> _smr_delete_notify is a "flag" that indicates when an _smr_delete_lock
>> notify() operation is needed. It counts up and down and should be a 
>> fairly
>> small number.
>>
>> _smr_deleted_thread_cnt counts the number of Threads that have been
>> deleted over the life of the VM. It's an always increasing number so
>> it's size depends on how long the VM has been running.
>>
>> _smr_deleted_thread_time_max is the maximum time in millis it has
>> taken to delete a thread. This field was originally a jlong, but
>> IIRC the Atomic::cmpxchg() code was not happy on ARM... (might have
>> just been Atomic::add() that was not happy)
>>
>> _smr_deleted_thread_times accumulates the time in millis that it
>> has taken to delete threads. It's an always increasing number so
>> it's size depends on how long the VM has been running. This field
>> was originally a jlong, but IIRC the Atomic::add() code was not
>> happy on ARM... (might have just been Atomic::cmpxchg() that was
>> not happy)
>>
>> _smr_tlh_cnt counts the number of ThreadsListHandles that have been
>> deleted over the life of the VM. It's an always increasing number so
>> it's size depends on how long the VM has been running.
>>
>> _smr_tlh_time_max is the maximum time in millis it has taken to
>> delete a ThreadsListHandle. This field was originally a jlong, but
>> IIRC the Atomic::cmpxchg() code was not happy on ARM... (might have
>> just been Atomic::add() that was not happy)
>>
>> _smr_tlh_times? accumulates the time in millis that it has taken to
>> delete ThreadsListHandles. It's an always increasing number so
>> it's size depends on how long the VM has been running.? This field
>> was originally a jlong, but IIRC the Atomic::add() code was not
>> happy on ARM... (might have just been Atomic::cmpxchg() that was
>> not happy)
>>
>>
>> It just dawned on me that I'm not sure whether you think the 'jint'
>> fields should be larger or smaller or the same size but a different
>> type... :-)
>>
>> The fact that I had to write so much to explain what these fields
>> are for and how they're used indicates I need more comments. See
>> below...
>>
>>
>>> 3456 long Threads::_smr_java_thread_list_alloc_cnt = 1;
>>> 3457 long Threads::_smr_java_thread_list_free_cnt = 0;
>>>
>>> long? If you really want 64-bit counters here you won't get them on 
>>> Windows with that declaration. If you really want variable sized 
>>> counters I suggest uintptr_t; otherwise uint64_t.
>>
>> long????????????????? Threads::_smr_java_thread_list_alloc_cnt = 1;
>> long????????????????? Threads::_smr_java_thread_list_free_cnt = 0;
>>
>> _smr_java_thread_list_alloc_cnt counts the number of ThreadsLists that
>> have been allocated over the life of the VM. It's an always increasing
>> number so it's size depends on how long the VM has been running and how
>> many Threads have been created and destroyed.
>>
>> _smr_java_thread_list_free_cnt counts the number of ThreadsLists that
>> have been freed over the life of the VM. It's an always increasing
>> number so it's size depends on how long the VM has been running and how
>> many Threads have been created and destroyed.
>>
>> I can't remember why we chose 'long' and I agree it's not a good choice
>> for Win*.
>>
>>
>> Okay so it dawns on me that we haven't written down a description for
>> the various new '_cnt', '_max' and '_times" fields so I'm adding these
>> comments to thread.hpp:
>>
>
> With this comment format, it's really hard to pick out the name of the 
> field.? Nobody uses 80 columns anymore.? Can you move the comments 
> over some spaces so the field names are visible?

Yes, I'll update the patch that I have for dholmes CR resolutions.


>
>> ?private:
>> ? // Safe Memory Reclamation (SMR) support:
>> ? static Monitor*????????????? _smr_delete_lock;
>> ? // The '_cnt', '_max' and '_times" fields are enabled via
>> ? // -XX:+EnableThreadSMRStatistics:
>> ?????????????????????????????? // # of parallel threads in 
>> _smr_delete_lock->wait().
>> ? static uint????????????????? _smr_delete_lock_wait_cnt;
>> ?????????????????????????????? // Max # of parallel threads in 
>> _smr_delete_lock->wait().
>> ? static uint????????????????? _smr_delete_lock_wait_max;
>> ?????????????????????????????? // Flag to indicate when an 
>> _smr_delete_lock->notify() is needed.
>> ? static volatile uint???????? _smr_delete_notify;
>> ?????????????????????????????? // # of threads deleted over VM lifetime.
>> ? static volatile uint???????? _smr_deleted_thread_cnt;
>> ?????????????????????????????? // Max time in millis to delete a thread.
>> ? static volatile uint???????? _smr_deleted_thread_time_max;
>> ?????????????????????????????? // Cumulative time in millis to delete 
>> threads.
>> ? static volatile uint???????? _smr_deleted_thread_times;
>> ? static ThreadsList* volatile _smr_java_thread_list;
>> ? static ThreadsList*????????? get_smr_java_thread_list();
>> ? static ThreadsList* xchg_smr_java_thread_list(ThreadsList* new_list);
>> ?????????????????????????????? // # of ThreadsLists allocated over VM 
>> lifetime.
>> ? static uint64_t????????????? _smr_java_thread_list_alloc_cnt;
>> ?????????????????????????????? // # of ThreadsLists freed over VM 
>> lifetime.
>> ? static uint64_t????????????? _smr_java_thread_list_free_cnt;
>> ?????????????????????????????? // Max size ThreadsList allocated.
>> ? static uint????????????????? _smr_java_thread_list_max;
>> ?????????????????????????????? // Max # of nested ThreadsLists for a 
>> thread.
>> ? static uint????????????????? _smr_nested_thread_list_max;
>> ?????????????????????????????? // # of ThreadsListHandles deleted 
>> over VM lifetime.
>> ? static volatile uint???????? _smr_tlh_cnt;
>> ?????????????????????????????? // Max time in millis to delete a 
>> ThreadsListHandle.
>> ? static volatile jint???????? _smr_tlh_time_max;
>> ?????????????????????????????? // Cumulative time in millis to delete 
>> ThreadsListHandles.
>> ? static volatile jint???????? _smr_tlh_times;
>> ? static ThreadsList*????????? _smr_to_delete_list;
>> ?????????????????????????????? // # of parallel ThreadsLists on the 
>> to-delete list.
>> ? static uint????????????????? _smr_to_delete_list_cnt;
>> ?????????????????????????????? // Max # of parallel ThreadsLists on 
>> the to-delete list.
>> ? static uint????????????????? _smr_to_delete_list_max;
>>
>>
>> I've also gone through all the new '_cnt', '_max' and '_times" fields
>> in thread.cpp and added an impl note to explain the choice of type:
>>
>
> Since this is in the cpp file and there is so much comment text, can 
> you just make it in column 1 and leave a blank line after each field.? 
> The indentation style is really hard to read and only useful if the 
> comment is short.

Yes, I'll update the patch that I have for dholmes CR resolutions.


>
>> // Safe Memory Reclamation (SMR) support:
>> Monitor*????????????? Threads::_smr_delete_lock =
>> ????????????????????????? new Monitor(Monitor::special, 
>> "smr_delete_lock",
>> ????????????????????????????????????? false /* allow_vm_block */,
>> Monitor::_safepoint_check_never);
>> // The '_cnt', '_max' and '_times" fields are enabled via
>> // -XX:+EnableThreadSMRStatistics:
>> ????????????????????? // # of parallel threads in 
>> _smr_delete_lock->wait().
>> ????????????????????? // Impl note: Hard to imagine > 64K waiting 
>> threads so
>> ????????????????????? // this could be 16-bit, but there is no nice 
>> 16-bit
>> ????????????????????? // _FORMAT support.
>> uint????????????????? Threads::_smr_delete_lock_wait_cnt = 0;
>> ????????????????????? // Max # of parallel threads in 
>> _smr_delete_lock->wait().
>> ????????????????????? // Impl note: See _smr_delete_lock_wait_cnt note.
>> uint????????????????? Threads::_smr_delete_lock_wait_max = 0;
>> ????????????????????? // Flag to indicate when an 
>> _smr_delete_lock->notify() is needed.
>> ????????????????????? // Impl note: See _smr_delete_lock_wait_cnt note.
>> volatile uint???????? Threads::_smr_delete_notify = 0;
>> ????????????????????? // # of threads deleted over VM lifetime.
>> ????????????????????? // Impl note: Atomically incremented over VM 
>> lifetime so
>> ????????????????????? // use unsigned for more range. Unsigned 64-bit 
>> would
>> ????????????????????? // be more future proof, but 64-bit atomic inc 
>> isn't
>> ????????????????????? // available everywhere (or is it?).
>> volatile uint???????? Threads::_smr_deleted_thread_cnt = 0;
>> ????????????????????? // Max time in millis to delete a thread.
>> ????????????????????? // Impl note: 16-bit might be too small on an 
>> overloaded
>> ????????????????????? // machine. Use unsigned since this is a time 
>> value. Set
>> ????????????????????? // via Atomic::cmpxchg() in a loop for 
>> correctness.
>> volatile uint???????? Threads::_smr_deleted_thread_time_max = 0;
>> ????????????????????? // Cumulative time in millis to delete threads.
>> ????????????????????? // Impl note: Atomically added to over VM 
>> lifetime so use
>> ????????????????????? // unsigned for more range. Unsigned 64-bit 
>> would be more
>> ????????????????????? // future proof, but 64-bit atomic inc isn't 
>> available
>> ????????????????????? // everywhere (or is it?).
>> volatile uint???????? Threads::_smr_deleted_thread_times = 0;
>> ThreadsList* volatile Threads::_smr_java_thread_list = new 
>> ThreadsList(0);
>> ????????????????????? // # of ThreadsLists allocated over VM lifetime.
>> ????????????????????? // Impl note: We allocate a new ThreadsList for 
>> every
>> ????????????????????? // thread create and every thread delete so we 
>> need a
>> ????????????????????? // bigger type than the _smr_deleted_thread_cnt 
>> field.
>> uint64_t????????????? Threads::_smr_java_thread_list_alloc_cnt = 1;
>> ????????????????????? // # of ThreadsLists freed over VM lifetime.
>> ????????????????????? // Impl note: See 
>> _smr_java_thread_list_alloc_cnt note.
>> uint64_t????????????? Threads::_smr_java_thread_list_free_cnt = 0;
>> ????????????????????? // Max size ThreadsList allocated.
>> ????????????????????? // Impl note: Max # of threads alive at one 
>> time should
>> ????????????????????? // fit in unsigned 32-bit.
>> uint????????????????? Threads::_smr_java_thread_list_max = 0;
>> ????????????????????? // Max # of nested ThreadsLists for a thread.
>> ????????????????????? // Impl note: Hard to imagine > 64K nested 
>> ThreadsLists
>> ????????????????????? // so his could be 16-bit, but there is no nice 
>> 16-bit
>> ????????????????????? // _FORMAT support.
>> uint????????????????? Threads::_smr_nested_thread_list_max = 0;
>> ????????????????????? // # of ThreadsListHandles deleted over VM 
>> lifetime.
>> ????????????????????? // Impl note: Atomically incremented over VM 
>> lifetime so
>> ????????????????????? // use unsigned for more range. There will be 
>> fewer
>> ????????????????????? // ThreadsListHandles than threads so unsigned 
>> 32-bit
>> ????????????????????? // should be fine.
>> volatile uint???????? Threads::_smr_tlh_cnt = 0;
>> ????????????????????? // Max time in millis to delete a 
>> ThreadsListHandle.
>> ????????????????????? // Impl note: 16-bit might be too small on an 
>> overloaded
>> ????????????????????? // machine. Use unsigned since this is a time 
>> value. Set
>> ????????????????????? // via Atomic::cmpxchg() in a loop for 
>> correctness.
>> volatile uint???????? Threads::_smr_tlh_time_max = 0;
>> ????????????????????? // Cumulative time in millis to delete 
>> ThreadsListHandles.
>> ????????????????????? // Impl note: Atomically added to over VM 
>> lifetime so use
>> ????????????????????? // unsigned for more range. Unsigned 64-bit 
>> would be more
>> ????????????????????? // future proof, but 64-bit atomic inc isn't 
>> available
>> ????????????????????? // everywhere (or is it?).
>> volatile uint???????? Threads::_smr_tlh_times = 0;
>> ThreadsList*????????? Threads::_smr_to_delete_list = NULL;
>> ????????????????????? // # of parallel ThreadsLists on the to-delete 
>> list.
>> ????????????????????? // Impl note: Hard to imagine > 64K 
>> ThreadsLists needing
>> ????????????????????? // to be deleted so this could be 16-bit, but 
>> there is no
>> ????????????????????? // nice 16-bit _FORMAT support.
>> uint????????????????? Threads::_smr_to_delete_list_cnt = 0;
>> ????????????????????? // Max # of parallel ThreadsLists on the 
>> to-delete list.
>> ????????????????????? // Impl note: See _smr_to_delete_list_cnt note.
>> uint????????????????? Threads::_smr_to_delete_list_max = 0;
>>
>>
>> Yikes! With the additional comments, the additions to thread.hpp and
>> thread.cpp grew by a bunch... I've done a test build build on my Mac
>> with these changes and I'm about to kick off a Mach5 tier1 job...
>
> Another reason why I agreed with some earlier comment that this should 
> be in a new file because it's a new thing. I was somewhat surprised 
> that it's not in threadSMR.{hpp,cpp}.?? This refactoring could be 
> deferred but thread.cpp is too big!

More refactoring is planned for after the initial push of Thread-SMR...

Thanks for chiming in on this part of the review thread.

Dan


>
> thanks,
> Coleen
>
>>
>> Dan
>>
>>
>>
>>>
>>> ----
>>>
>>> On 19/11/2017 11:49 AM, Daniel D. Daugherty wrote:
>>>> Greetings,
>>>>
>>>> Testing of the last round of changes revealed a hang in one of the new
>>>> TLH tests. Robbin has fixed the hang, updated the existing TLH 
>>>> test, and
>>>> added another TLH test for good measure.
>>>>
>>>> Here is the updated full webrev:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/
>>>>
>>>> Here is the updated delta webrev:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-delta/
>>>>
>>>> Dan ran the bits thru the usual Mach5 tier[1-5] testing and there are
>>>> no unexpected failures.
>>>>
>>>> We welcome comments, suggestions and feedback.
>>>>
>>>> Dan, Erik and Robbin
>>>>
>>>>
>>>> On 11/15/17 3:32 PM, Daniel D. Daugherty wrote:
>>>>> Greetings,
>>>>>
>>>>> Robbin rebased the project last night/this morning to merge with 
>>>>> Thread
>>>>> Local Handshakes (TLH) and also picked up additional changesets up 
>>>>> thru:
>>>>>
>>>>>> Changeset: fa736014cf28
>>>>>> Author:??? cjplummer
>>>>>> Date:????? 2017-11-14 18:08 -0800
>>>>>> URL:http://hg.openjdk.java.net/jdk/hs/rev/fa736014cf28
>>>>>>
>>>>>> 8191049: Add alternate version of pns() that is callable from 
>>>>>> within hotspot source
>>>>>> Summary: added pns2() to debug.cpp
>>>>>> Reviewed-by: stuefe, gthornbr
>>>>>
>>>>> This is the first time we've rebased the project to bits that are 
>>>>> this
>>>>> fresh (< 12 hours old at rebase time). We've done this because we 
>>>>> think
>>>>> we're done with this project and are in the final 
>>>>> review-change-retest
>>>>> cycle(s)... In other words, we're not planning on making any more 
>>>>> major
>>>>> changes for this project... :-)
>>>>>
>>>>> *** Time for code reviewers to chime in on this thread! ***
>>>>>
>>>>> Here is the updated full webrev:
>>>>>
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-full/
>>>>>
>>>>> Here's is the delta webrev from the 2017.11.10 rebase:
>>>>>
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-delta/
>>>>>
>>>>> Dan has submitted the bits for the usual Mach5 tier[1-5] testing
>>>>> (and the new baseline also)...
>>>>>
>>>>> We're expecting this round to be a little noisier than usual because
>>>>> we did not rebase on a PIT snapshot so the new baseline has not been
>>>>> through Jesper's usual care-and-feeding of integration_blockers, etc.
>>>>>
>>>>> We welcome comments, suggestions and feedback.
>>>>>
>>>>> Dan, Erik and Robbin
>>>>>
>>>>>
>>>>> On 11/14/17 4:48 PM, Daniel D. Daugherty wrote:
>>>>>> Greetings,
>>>>>>
>>>>>> I rebased the project to the 2017.11.10 jdk/hs PIT snapshot.
>>>>>> (Yes, we're playing chase-the-repo...)
>>>>>>
>>>>>> Here is the updated full webrev:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-07-full/
>>>>>>
>>>>>> Unlike the previous rebase, there were no changes required to the
>>>>>> open code to get the rebased bits to build so there is no delta
>>>>>> webrev.
>>>>>>
>>>>>> These bits have been run through JPRT and I've submitted the usual
>>>>>> Mach5 tier[1-5] test run...
>>>>>>
>>>>>> We welcome comments, suggestions and feedback.
>>>>>>
>>>>>> Dan, Erik and Robbin
>>>>>>
>>>>>>
>>>>>> On 11/13/17 12:30 PM, Daniel D. Daugherty wrote:
>>>>>>> Greetings,
>>>>>>>
>>>>>>> I rebased the project to the 2017.10.26 jdk10/hs PIT snapshot.
>>>>>>>
>>>>>>> Here are the updated webrevs:
>>>>>>>
>>>>>>> Here's the mq comment for the change:
>>>>>>>
>>>>>>> ? Rebase to 2017.10.25 PIT snapshot.
>>>>>>>
>>>>>>> Here is the full webrev:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-full/
>>>>>>>
>>>>>>> And here is the delta webrev:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-delta/
>>>>>>>
>>>>>>> I ran the above bits throught Mach5 tier[1-5] testing over the 
>>>>>>> holiday
>>>>>>> weekend. Didn't see any issues in a quick look. Going to take a 
>>>>>>> closer
>>>>>>> look today.
>>>>>>>
>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>
>>>>>>> Dan, Erik and Robbin
>>>>>>>
>>>>>>>
>>>>>>> On 11/8/17 1:05 PM, Daniel D. Daugherty wrote:
>>>>>>>> Greetings,
>>>>>>>>
>>>>>>>> Resolving one of the code review comments (from both Stefan K 
>>>>>>>> and Coleen)
>>>>>>>> on jdk10-04-full required quite a few changes so it is being 
>>>>>>>> done as a
>>>>>>>> standalone patch and corresponding webrevs:
>>>>>>>>
>>>>>>>> Here's the mq comment for the change:
>>>>>>>>
>>>>>>>> ? stefank, coleenp CR - refactor most JavaThreadIterator usage 
>>>>>>>> to use
>>>>>>>> ??? JavaThreadIteratorWithHandle.
>>>>>>>>
>>>>>>>> Here is the full webrev:
>>>>>>>>
>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-full/
>>>>>>>>
>>>>>>>> And here is the delta webrev:
>>>>>>>>
>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-delta/
>>>>>>>>
>>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>>
>>>>>>>> Dan, Erik and Robbin
>>>>>>>>
>>>>>>>>
>>>>>>>> On 10/9/17 3:41 PM, Daniel D. Daugherty wrote:
>>>>>>>>> Greetings,
>>>>>>>>>
>>>>>>>>> We have a (eXtra Large) fix for the following bug:
>>>>>>>>>
>>>>>>>>> 8167108 inconsistent handling of SR_lock can lead to crashes
>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8167108
>>>>>>>>>
>>>>>>>>> This fix adds a Safe Memory Reclamation (SMR) mechanism based on
>>>>>>>>> Hazard Pointers to manage JavaThread lifecycle.
>>>>>>>>>
>>>>>>>>> Here's a PDF for the internal wiki that we've been using to 
>>>>>>>>> describe
>>>>>>>>> and track the work on this project:
>>>>>>>>>
>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/SMR_and_JavaThread_Lifecycle-JDK10-04.pdf 
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Dan has noticed that the indenting is wrong in some of the 
>>>>>>>>> code quotes
>>>>>>>>> in the PDF that are not present in the internal wiki. We don't 
>>>>>>>>> have a
>>>>>>>>> solution for that problem yet.
>>>>>>>>>
>>>>>>>>> Here's the webrev for current JDK10 version of this fix:
>>>>>>>>>
>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-04-full
>>>>>>>>>
>>>>>>>>> This fix has been run through many rounds of JPRT and Mach5 
>>>>>>>>> tier[2-5]
>>>>>>>>> testing, additional stress testing on Dan's Solaris X64 
>>>>>>>>> server, and
>>>>>>>>> additional testing on Erik and Robbin's machines.
>>>>>>>>>
>>>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>>>
>>>>>>>>> Daniel Daugherty
>>>>>>>>> Erik Osterlund
>>>>>>>>> Robbin Ehn
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>
>


From daniel.daugherty at oracle.com  Tue Nov 21 17:36:31 2017
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Tue, 21 Nov 2017 12:36:31 -0500
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <68b0b2bc-2d26-5442-bafb-5ce21420767b@oracle.com>
References: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
 <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>
 <836b5c6c-5eef-9245-c16a-e71372093d9a@oracle.com>
 <82d0e473-a24f-9feb-82e9-7d04c4d7a91d@oracle.com>
 <354f7ece-66bc-c065-9ee8-f55abcc1c23e@oracle.com>
 <0ff8df8e-e93c-02d2-cb97-b13b1ac0f53f@oracle.com>
 <e56b44ce-28a9-822e-bbac-4b64f5f85c58@oracle.com>
 <70cc1311-de55-5502-159b-cd0a3522cfd6@oracle.com>
 <68b0b2bc-2d26-5442-bafb-5ce21420767b@oracle.com>
Message-ID: <5e154ff2-65d4-bad4-db5e-88cf4c733c7d@oracle.com>

Thanks for keeping all the OpenJDK aliases!


On 11/21/17 12:24 PM, coleen.phillimore at oracle.com wrote:
>
>
> On 11/21/17 11:28 AM, Daniel D. Daugherty wrote:
>> Hi Coleen!
>>
>> Thanks for making time to review the Thread-SMR stuff again!!
>>
>> I have added back the other three OpenJDK aliases... This review is
>> being done on _four_ different OpenJDK aliases.
>>
>> As always, replies are embedded below...
>>
>>
>> On 11/20/17 3:12 PM, coleen.phillimore at oracle.com wrote:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/src/hotspot/os/linux/os_linux.cpp.frames.html 
>>>
>>>
>>> I read David's comments about the threads list iterator, and I was 
>>> going to say that it can be cleaned up later, as the bulk of the 
>>> change is the SMR part but this looks truly bizarre. It looks like 
>>> it shouldn't compile because 'jt' isn't in scope.
>>>
>>> Why isn't this the syntax:
>>>
>>> JavaThreadIteratorWithHandle jtiwh;
>>> for (JavaThread* jt = jtiwh.first(); jt != NULL; jt = jtiwh.next()) {
>>> }
>>>
>>> Which would do the obvious thing without anyone having to squint at 
>>> the code.
>>
>> See my reply to David's review for the more detailed answer.
>>
>> For the above syntax, we would need braces to limit the scope of the
>> 'jtiwh' variable. With Stefan's propsal, you get limited scope on
>> 'jtiwh' for "free".
>
> Yes, thank you, I see that motivation now.
>>
>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/src/hotspot/share/runtime/threadSMR.hpp.html 
>>>
>>>
>>> As a hater of acronmys, can this file use "Safe Memory Reclaimation"
>>
>> I'm not sure what you mean by this. Do you mean rename the files?
>>
>> ? threadSMR.hpp -> threadSafeMemoryReclaimation.hpp
>> ? threadSMR.cpp -> threadSafeMemoryReclaimation.cpp
>>
>
> No.
>
>>
>>> and briefly describe the concept in the beginning of the header file,
>>> so one knows why it's called threadSMR.hpp?
>>
>> And then this part of the sentence kind of indicates that you would be
>> okay with the threadSMR.{c,h}pp names if a comment was added to the
>> header file.
>>
>> Please clarify.
>
> Yes.? If you added a comment to the beginning of the threadSMR.hpp 
> file that said what SMR stood for and what it was, that would be 
> extremely helpful for future viewers of this file.

Yup. I just finished with the comment...


>
>>
>>
>>> It doesn't need to be long and include why Threads list needs this 
>>> Sometimes we tell new people that the hotspot documentation is in 
>>> the header files.
>>
>> Yup. When I migrated stuff from thread.hpp and thread.cpp to 
>> threadSMR.hpp
>> and threadSMR.cpp, I should have written a header comment...
>>
>> I did update a comment in thread.cpp based on Robin W's code review:
>
> Yes, I think a comment belongs in the SMR file also, or in preference.

Wasn't trying to say that I thought the update I did for Robin W was
sufficient...


>>
>>> > src/hotspot/share/runtime/thread.cpp
>>> >
>>> > 3432 // operations over all threads.? It is protected by its own 
>>> Mutex
>>> > 3433 // lock, which is also used in other contexts to protect thread
>>> >
>>> > Should this comment perhaps be revised to mention SMR?
>>>
>>> It definitely needs some updating... Here's a stab at it:
>>>
>>> // The Threads class links together all active threads, and provides
>>> // operations over all threads. It is protected by the Threads_lock,
>>> // which is also used in other global contexts like safepointing.
>>> // ThreadsListHandles are used to safely perform operations on one
>>> // or more threads without the risk of the thread exiting during the
>>> // operation.
>>> //
>>> // Note: The Threads_lock is currently more widely used than we
>>> // would like. We are actively migrating Threads_lock uses to other
>>> // mechanisms in order to reduce Threads_lock contention. 
>>
>> I'll take a look at adding a header comment to threadSMR.hpp.
>>
>>
>>> 186?? JavaThreadIteratorWithHandle() : _tlh(), _index(0) {}
>>>
>>> This _tlh() call should not be necessary.? The compiler should 
>>> generate this for you in the constructor.
>>
>> Deleted.
>>
>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/src/hotspot/share/runtime/threadSMR.cpp.html 
>>>
>>>
>>> ? 32 ThreadsList::ThreadsList(int entries) : _length(entries), 
>>> _threads(NEW_C_HEAP_ARRAY(JavaThread*, entries + 1, mtGC)), 
>>> _next_list(NULL) {
>>>
>>> Seems like it should be mtThread rather than mtGC.
>>
>> Fixed. Definitely an artifact of Erik's original prototype when he
>> extracted Thread-SMR from his GC work... Thanks for catching it.
>>
>>
>>> Should
>>>
>>> ? 62?? if (EnableThreadSMRStatistics) {
>>>
>>> really be UL, ie: if (log_is_enabled(Info, thread, smr, statistics)) ?
>>
>> EnableThreadSMRStatistics is used in more places than UL code.
>> We use it in Thread::print_*() stuff to control output of
>> Thread-SMR statistics info in thread dumps and hs_err_pid file
>> generation.
>
> That sort of thing is also controlled by logging in general.

Don't think I want to do that since logging may be be "up" at the
time that I need Thread::print_*() or hs_err_pid generation...

Something about chickens and eggs... :-)


>>
>> Currently thread dump and hs_err_pid file output is not generated
>> using UL (and probably can't be?) so we need an option to control
>> the Thread-SMR statistics stuff in all places.
>>
>
> You can use log options in preference to this new diagnostic option in 
> these cases too.?? This option looks a lot like logging to me and 
> would be nice to not have to later convert it.

See above reply...


>
>>
>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/test/hotspot/jtreg/runtime/ErrorHandling/NestedThreadsListHandleInErrorHandlingTest.java.html 
>>>
>>>
>>> Can you use for these tests instead (there were a couple):
>>>
>>> *@requires (vm.debug == true)*
>>
>> The test I cloned had this in it:
>>
>> ??? if (!Platform.isDebugBuild()) {
>> ??????? // -XX:ErrorHandlerTest=N option requires debug bits.
>> ??????? return;
>> ??? }
>>
>> and you would like me to switch to the newer mechanism?
>>
>> I have updated the following tests:
>>
>> ? test/hotspot/jtreg/runtime/ErrorHandling/ErrorHandler.java
>> test/hotspot/jtreg/runtime/ErrorHandling/NestedThreadsListHandleInErrorHandlingTest.java 
>>
>> test/hotspot/jtreg/runtime/ErrorHandling/ThreadsListHandleInErrorHandlingTest.java 
>>
>>
>>
> Yes, the newer mechanism makes jtreg not bother to run the test, which 
> makes it faster!

More quickly not run the test... Yup... got it...


>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/src/hotspot/share/runtime/thread.cpp.udiff.html 
>>>
>>>
>>> +// thread, has been added the Threads list, the system is not at a
>>>
>>> Has "not" been added to the Threads list?? missing "not"?
>>
>> Nope. If the JavaThread has been added to the Threads list
>> and it is not protected, then it is dangling. In other words,
>> a published JavaThread (on the Threads list) has to be protected
>> by either the system being at a safepoint or the JavaThread has
>> to be on some threads's ThreadsList.
>>
>>
>>>
>>> + return (unsigned int)(((uint32_t)(uintptr_t)s1) * 2654435761u);
>>>
>>> Can you add a comment about where this number came from?
>>
>> I'll have to get that from Erik...
>>
>>
>>> I can't find the caller of threads_do_smr.
>>
>> I'm guessing that's used by the GC code that depends on Thread-SMR...
>>
>
> They? should add this API when the add the use of it then.? I don't 
> see it in any sources.

We'll see what Erik wants to do...


>
>>
>>> If these functions xchg_smr_thread_list, get_smr_java_thread_list, 
>>> inc_smr_deleted_thread_count are only used by thread.cpp, I think 
>>> they should go in that file and not in the .inline.hpp file to be 
>>> included and possibly called by other files.? I think they're 
>>> private to class Threads.
>>
>> I have a vague memory that some of the compilers don't do inlining when
>> an "inline" function is in a .cpp. I believe we want these functions
>> to be inlined for performance reasons. Erik should probably chime in
>> here.
>
> I don't see why this should be.? Maybe some (older) compilers require 
> it to be found before the call though, but that can still be 
> accomplished in the .cpp file.

Again, we'll see what Erik wants to do...


>>
>>
>>> I don't have any in-depth comments. This looks really good from my 
>>> day of reading it.
>>
>> Thanks for taking the time to review it again!
>>
>
> Thanks,
> Coleen

And thanks again...

Dan


>
>> Dan
>>
>>
>>>
>>> Thanks,
>>> Coleen
>>>
>>> On 11/18/17 8:49 PM, Daniel D. Daugherty wrote:
>>>
>>>> Greetings,
>>>>
>>>> Testing of the last round of changes revealed a hang in one of the new
>>>> TLH tests. Robbin has fixed the hang, updated the existing TLH 
>>>> test, and
>>>> added another TLH test for good measure.
>>>>
>>>> Here is the updated full webrev:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/
>>>>
>>>> Here is the updated delta webrev:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-delta/
>>>>
>>>> Dan ran the bits thru the usual Mach5 tier[1-5] testing and there are
>>>> no unexpected failures.
>>>>
>>>> We welcome comments, suggestions and feedback.
>>>>
>>>> Dan, Erik and Robbin
>>>>
>>>>
>>>> On 11/15/17 3:32 PM, Daniel D. Daugherty wrote:
>>>>> Greetings,
>>>>>
>>>>> Robbin rebased the project last night/this morning to merge with 
>>>>> Thread
>>>>> Local Handshakes (TLH) and also picked up additional changesets up 
>>>>> thru:
>>>>>
>>>>>> Changeset: fa736014cf28
>>>>>> Author:??? cjplummer
>>>>>> Date:????? 2017-11-14 18:08 -0800
>>>>>> URL:http://hg.openjdk.java.net/jdk/hs/rev/fa736014cf28
>>>>>>
>>>>>> 8191049: Add alternate version of pns() that is callable from 
>>>>>> within hotspot source
>>>>>> Summary: added pns2() to debug.cpp
>>>>>> Reviewed-by: stuefe, gthornbr
>>>>>
>>>>> This is the first time we've rebased the project to bits that are 
>>>>> this
>>>>> fresh (< 12 hours old at rebase time). We've done this because we 
>>>>> think
>>>>> we're done with this project and are in the final 
>>>>> review-change-retest
>>>>> cycle(s)... In other words, we're not planning on making any more 
>>>>> major
>>>>> changes for this project... :-)
>>>>>
>>>>> *** Time for code reviewers to chime in on this thread! ***
>>>>>
>>>>> Here is the updated full webrev:
>>>>>
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-full/
>>>>>
>>>>> Here's is the delta webrev from the 2017.11.10 rebase:
>>>>>
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-delta/
>>>>>
>>>>> Dan has submitted the bits for the usual Mach5 tier[1-5] testing
>>>>> (and the new baseline also)...
>>>>>
>>>>> We're expecting this round to be a little noisier than usual because
>>>>> we did not rebase on a PIT snapshot so the new baseline has not been
>>>>> through Jesper's usual care-and-feeding of integration_blockers, etc.
>>>>>
>>>>> We welcome comments, suggestions and feedback.
>>>>>
>>>>> Dan, Erik and Robbin
>>>>>
>>>>>
>>>>> On 11/14/17 4:48 PM, Daniel D. Daugherty wrote:
>>>>>> Greetings,
>>>>>>
>>>>>> I rebased the project to the 2017.11.10 jdk/hs PIT snapshot.
>>>>>> (Yes, we're playing chase-the-repo...)
>>>>>>
>>>>>> Here is the updated full webrev:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-07-full/
>>>>>>
>>>>>> Unlike the previous rebase, there were no changes required to the
>>>>>> open code to get the rebased bits to build so there is no delta
>>>>>> webrev.
>>>>>>
>>>>>> These bits have been run through JPRT and I've submitted the usual
>>>>>> Mach5 tier[1-5] test run...
>>>>>>
>>>>>> We welcome comments, suggestions and feedback.
>>>>>>
>>>>>> Dan, Erik and Robbin
>>>>>>
>>>>>>
>>>>>> On 11/13/17 12:30 PM, Daniel D. Daugherty wrote:
>>>>>>> Greetings,
>>>>>>>
>>>>>>> I rebased the project to the 2017.10.26 jdk10/hs PIT snapshot.
>>>>>>>
>>>>>>> Here are the updated webrevs:
>>>>>>>
>>>>>>> Here's the mq comment for the change:
>>>>>>>
>>>>>>> ? Rebase to 2017.10.25 PIT snapshot.
>>>>>>>
>>>>>>> Here is the full webrev:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-full/
>>>>>>>
>>>>>>> And here is the delta webrev:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-delta/
>>>>>>>
>>>>>>> I ran the above bits throught Mach5 tier[1-5] testing over the 
>>>>>>> holiday
>>>>>>> weekend. Didn't see any issues in a quick look. Going to take a 
>>>>>>> closer
>>>>>>> look today.
>>>>>>>
>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>
>>>>>>> Dan, Erik and Robbin
>>>>>>>
>>>>>>>
>>>>>>> On 11/8/17 1:05 PM, Daniel D. Daugherty wrote:
>>>>>>>> Greetings,
>>>>>>>>
>>>>>>>> Resolving one of the code review comments (from both Stefan K 
>>>>>>>> and Coleen)
>>>>>>>> on jdk10-04-full required quite a few changes so it is being 
>>>>>>>> done as a
>>>>>>>> standalone patch and corresponding webrevs:
>>>>>>>>
>>>>>>>> Here's the mq comment for the change:
>>>>>>>>
>>>>>>>> ? stefank, coleenp CR - refactor most JavaThreadIterator usage 
>>>>>>>> to use
>>>>>>>> ??? JavaThreadIteratorWithHandle.
>>>>>>>>
>>>>>>>> Here is the full webrev:
>>>>>>>>
>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-full/
>>>>>>>>
>>>>>>>> And here is the delta webrev:
>>>>>>>>
>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-delta/
>>>>>>>>
>>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>>
>>>>>>>> Dan, Erik and Robbin
>>>>>>>>
>>>>>>>>
>>>>>>>> On 10/9/17 3:41 PM, Daniel D. Daugherty wrote:
>>>>>>>>> Greetings,
>>>>>>>>>
>>>>>>>>> We have a (eXtra Large) fix for the following bug:
>>>>>>>>>
>>>>>>>>> 8167108 inconsistent handling of SR_lock can lead to crashes
>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8167108
>>>>>>>>>
>>>>>>>>> This fix adds a Safe Memory Reclamation (SMR) mechanism based on
>>>>>>>>> Hazard Pointers to manage JavaThread lifecycle.
>>>>>>>>>
>>>>>>>>> Here's a PDF for the internal wiki that we've been using to 
>>>>>>>>> describe
>>>>>>>>> and track the work on this project:
>>>>>>>>>
>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/SMR_and_JavaThread_Lifecycle-JDK10-04.pdf 
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Dan has noticed that the indenting is wrong in some of the 
>>>>>>>>> code quotes
>>>>>>>>> in the PDF that are not present in the internal wiki. We don't 
>>>>>>>>> have a
>>>>>>>>> solution for that problem yet.
>>>>>>>>>
>>>>>>>>> Here's the webrev for current JDK10 version of this fix:
>>>>>>>>>
>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-04-full
>>>>>>>>>
>>>>>>>>> This fix has been run through many rounds of JPRT and Mach5 
>>>>>>>>> tier[2-5]
>>>>>>>>> testing, additional stress testing on Dan's Solaris X64 
>>>>>>>>> server, and
>>>>>>>>> additional testing on Erik and Robbin's machines.
>>>>>>>>>
>>>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>>>
>>>>>>>>> Daniel Daugherty
>>>>>>>>> Erik Osterlund
>>>>>>>>> Robbin Ehn
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>



From sangheon.kim at oracle.com  Tue Nov 21 19:56:56 2017
From: sangheon.kim at oracle.com (sangheon.kim)
Date: Tue, 21 Nov 2017 11:56:56 -0800
Subject: RFR (M): 8190426: Lazily initialize refinement threads with
 UseDynamicNumberOfGCThreads
In-Reply-To: <1511201854.2229.4.camel@oracle.com>
References: <1509726495.2630.2.camel@oracle.com>
 <1f3e5ef0-60f3-770f-f581-8c7764e44f2e@oracle.com>
 <1511177189.2337.39.camel@oracle.com> <1511201854.2229.4.camel@oracle.com>
Message-ID: <d3ebd10a-53cb-2879-46ce-c03e5e8eeab2@oracle.com>

Hi Thomas,

On 11/20/2017 10:17 AM, Thomas Schatzl wrote:
> Hi all,
>
>    Stefan Johansson found another issue: in the original code, before
> activating a thread it checked whether that thread had already been
> activated. The changes did not do that.
>
> Here's a new webrev:
>
> http://cr.openjdk.java.net/~tschatzl/8190426/webrev.1_to_2/  (diff)
> http://cr.openjdk.java.net/~tschatzl/8190426/webrev.2/  (full)
webrev.2 looks good to me.

And thank you for addressing my comments on webrev.1.
I agree we can improve lazily allocating threads with JDK-8191082: Unify 
handling of thread sets when allocating them lazily.

Thanks,
Sangheon


>
> Testing:
> hs tier1+2
>
> Thanks,
>    Thomas
>
>
> On Mon, 2017-11-20 at 12:26 +0100, Thomas Schatzl wrote:
>> Hi Sangheon,
>>
>>    thanks for your review, and sorry for the late answer - I have been
>> travelling lately...
>>
>> On Thu, 2017-11-09 at 15:00 -0800, sangheon.kim wrote:
>>> Hi Thomas,
>>>
>>> On 11/03/2017 09:28 AM, Thomas Schatzl wrote:
>>>> Hi,
>>>>
>>>>     can I have reviews for this change that makes refinement
>>>> threads
>>>> behave the same as the other thread groups we have in G1?
>>>>
>>>> I.e. with UseDynamicNumberOfGCThreads enabled (which is off by
>>>> default) they are created lazily.
>>>>
>>>> This helps a little bit further with startup/footprint benchmarks
>>>> (if
>>>> enabled).
>>>>
>>>> CR:
>>>> https://bugs.openjdk.java.net/browse/JDK-8190426
>>>> Webrev:
>>>> http://cr.openjdk.java.net/~tschatzl/8190426/webrev/
>>>> Testing:
>>>> hs-tier1+2
>>> I like this idea, thank you for improving this.
>>>
>>> I have some comments.
>> I think I addressed all your concerns. Some of them were due to me
>> being confused about which memory allocations cause VM exit, and
>> which
>> don't. I hope I got that right now.
>>
>> I also changed the behavior of the thread startup to be more similar
>> to
>> the worker threads, namely:
>> - always start one refinement thread at startup
>> - support InjectGCWorkerCreationFailure (which is tested by our tests
>> btw)
>> - make error messages and failure modes more similar to the ones for
>> the work gangs.
>>
>> Even with that we want to look at JDK-8191082 for better specifying
>> the
>> expected behavior.
>>
>> Webrevs:
>> http://cr.openjdk.java.net/~tschatzl/8190426/webrev.0_to_1/  (diff)
>> http://cr.openjdk.java.net/~tschatzl/8190426/webrev.1/  (full)
>> Testing:
>> hs-tier1+2
>>
>> Thanks,
>>    Thomas
>>



From rkennke at redhat.com  Tue Nov 21 21:37:40 2017
From: rkennke at redhat.com (Roman Kennke)
Date: Tue, 21 Nov 2017 22:37:40 +0100
Subject: RFR: 8191564: Refactor GC related servicability code into GC
 specific subclasses
In-Reply-To: <387d2b93-cb8b-51be-a6e0-3b5f9d6cbb1c@redhat.com>
References: <387d2b93-cb8b-51be-a6e0-3b5f9d6cbb1c@redhat.com>
Message-ID: <713325a4-3ed8-cd7d-c66d-a1c1767b6d3a@redhat.com>

I had some off-band discussions with Erik Helin and re-did most of the 
changeset:
- The GC interface now resides in CollectedHeap, specifically the two 
methods memory_managers() and memory_pools(), and is implemented in the 
various concrete subclasses.
- Both methods return (by value) a GrowableArray<GCMemoryManager*> and 
GrowableArray<MemoryPool> respectively. Returning a stack-allocated 
GrowableArray seemed least complicated (avoid explicit cleanup of 
short-lived array object), and most future-proof, e.g. currently there 
is an implicit expectation to get 2 GCMemoryManagers, even though some 
GCs don't necessarily have two. The API allows for easy extension of the 
situation.

http://cr.openjdk.java.net/~rkennke/8191564/webrev.01/

I think this requires reviews from both the GC and Serviceability team.

Roman


> Currently, there's lots of GC specific code sprinkled over 
> src/hotspot/share/services. This change introduces a GC interface for 
> that, and moves all GC specific code to their respective 
> src/hotspot/share/gc directory.
> 
> http://cr.openjdk.java.net/~rkennke/8191564/webrev.00/ 
> <http://cr.openjdk.java.net/%7Erkennke/8191564/webrev.00/>
> 
> Testing: hotspot_gc and hotspot_serviceability, none showed regressions
> 
> Built minimal and server without regressions
> 
> What do you think?
> 
> Roman
> 
> 



From Derek.White at cavium.com  Tue Nov 21 21:53:02 2017
From: Derek.White at cavium.com (White, Derek)
Date: Tue, 21 Nov 2017 21:53:02 +0000
Subject: RFR (XS): 8188877: Improper synchronization in offer_termination
In-Reply-To: <1510821711.5332.3.camel@oracle.com>
References: <CY1PR0701MB16328948D4FDF19FBC30F65684710@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <3229BC22-BB8A-4D3D-8440-AB799AB5394F@oracle.com>
 <CY1PR0701MB16328E4C6563BE60CF8E99D084750@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <4D9972BA-B76C-4FB7-B24D-31D507DB1A6A@oracle.com>
 <1510821711.5332.3.camel@oracle.com>
Message-ID: <CY1PR0701MB163288CD9B58A163B2F16F7C84230@CY1PR0701MB1632.namprd07.prod.outlook.com>

Hi Kim, Thomas,

So Thomas, I think I agree with my original self too ?.

Kim's concern:
> Consider the case where there is no further work to do, and all 
> threads are heading toward offer_termination.  Until the last thread 
> makes the offer, the others will be in the spin1...spinN/yield/sleep 
> cycle, as expected.  But because there appear to be no memory barriers 
> in the spin part of that cycle, a thread in the spin part might not 
> see the final offer until it completes the full spin cycle and then 
> performs the yield (assuming yield does have sufficient memory 
> barriers to acquire the published value).

My understanding is that the "acquire" semantics are entirely about memory ordering, within a CPU. In particular it prevents "following loads" from executing before the "load acquire". 

There is nothing in the "load acquire" that causes it to synchronize with the memory system more or less quickly than a naked load. Either kind of load will eventually notice that its local cached value has been invalidated and will fetch the updated value. 

In the context of a spin-loop, there could be some large number of loads (from the same variable), comparisons, and branches in flight. Without a load-acquire, it's not clear which load will see the final value, but it doesn't matter. As long as one iteration of the loop sees that value, the loop will terminate.

BTW, I have seen several explanations of what the Intel PAUSE instruction does, in its earliest forms it seemed to be about avoiding having to clean up either: mis-predicted instructions in flight, or carefully ordering the memory loads. I might be misremembering and I can't find my sources for this though.

 - Derek

> -----Original Message-----
> From: Thomas Schatzl [mailto:thomas.schatzl at oracle.com]
> Sent: Thursday, November 16, 2017 3:42 AM
> To: Kim Barrett <kim.barrett at oracle.com>; White, Derek
> <Derek.White at cavium.com>
> Cc: hotspot-gc-dev at openjdk.java.net
> Subject: Re: RFR (XS): 8188877: Improper synchronization in
> offer_termination
> 
> On Mon, 2017-10-09 at 21:44 -0400, Kim Barrett wrote:
> > > On Oct 9, 2017, at 8:20 PM, White, Derek <Derek.White at cavium.com>
> > > wrote:
> > > > src/hotspot/share/gc/shared/taskqueue.cpp
> > > >
> > > > The change to make _offered_termination volatile prevents the
> > > > compiler from hoisting it out of the loop, but I'm not sure it is
> > > > enough to prevent unnecessary delays in recognizing the
> > > > termination condition.
> > > > In particular, while offer_termination is spinning, it's not clear
> > > > there's anything to ensure an update by some other thread will
> > > > become visible during the repeated spin cycles.??yield and sleep
> > > > seem likely to involve fences that will make such updates visible,
> > > > but not so much for SpinPause.
> > >
> > > [Derek]
> > > The concurrent access around _offered_termination is (semi*)-
> > > independent of other data accesses. So I'm not sure if more ordering
> > > constraints will help much here?
> > >
> > > The atomic incr/dec operations provide a fence to publish the
> > > change. On aarch64 this is a DMB instruction. A full sync (DSB) will
> > > also publish the change, but blocks until the publish has been
> > > acked. But it doesn't publish any faster.
> > >
> > > The only extra thing we could do is read _offered_termination using
> > > OrderAccess::load_acquire(). I think the only thing this adds in
> > > this case is ordering between the loads of _offered_termination in
> > > different??loop iterations. Is this required or helpful?
> >
> > Consider the case where there is no further work to do, and all
> > threads are heading toward offer_termination.??Until the last thread
> > makes the offer, the others will be in the spin1...spinN/yield/sleep
> > cycle, as expected.??But because there appear to be no memory barriers
> > in the spin part of that cycle, a thread in the spin part might not
> > see the final offer until it completes the full spin cycle and then
> > performs the yield (assuming yield does have sufficient memory
> > barriers to acquire the published value).
> >
> > So I think the read of _offered_termination for comparison with
> > _n_threads probably ought to be an acquire.
> 
>   I agree with Derek that adding an acquire for the load of
> _offered_termination does not seem to improve termination properties, and
> is imho unnecessary (but the volatile declaration of course is).
> 
> Thanks,
>   Thomas


From daniel.daugherty at oracle.com  Wed Nov 22 00:12:49 2017
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Tue, 21 Nov 2017 19:12:49 -0500
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <0ff8df8e-e93c-02d2-cb97-b13b1ac0f53f@oracle.com>
References: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
 <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>
 <836b5c6c-5eef-9245-c16a-e71372093d9a@oracle.com>
 <82d0e473-a24f-9feb-82e9-7d04c4d7a91d@oracle.com>
 <354f7ece-66bc-c065-9ee8-f55abcc1c23e@oracle.com>
 <0ff8df8e-e93c-02d2-cb97-b13b1ac0f53f@oracle.com>
Message-ID: <5365606e-4036-b5d7-9bac-e91e42f38d01@oracle.com>

Greetings,

*** We are wrapping up code review on this project so it is time ***
*** for the various code reviewers to chime in one last time. ***

In this latest round, we had three different reviewers chime in so we're
doing delta webrevs for each of those resolutions:

David H's resolutions:

http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.09,10.10.0-delta/

 ? mq comment for dholmes_CR:
 ??? - Fix indents, trailing spaces and various typos.
 ??? - Add descriptions for the '_cnt', '_max' and '_times" fields,
 ????? add impl notes to document the type choices.

 ? src/hotspot/share/runtime/globals.hpp change is white-space only so it
 ? is only visible via the file's patch link.


Robin W's resolutions:

http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.10.0,10.10.1-delta/

 ? mq comment for robinw_CR:
 ??? - Fix some inefficient code, update some comments, fix some indents,
 ????? and add some 'const' specifiers.

 ? src/hotspot/share/runtime/vm_operations.hpp change is white-space only so
 ? it is only visible via the file's patch link.


Coleen's resolutions:

http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.10.1,10.10.2-delta/

 ? mq comment for coleenp_CR:
 ??? - Change ThreadsList::_threads from 'mtGC' -> 'mtThread',
 ??? - add header comment to threadSMR.hpp file,
 ??? - cleanup JavaThreadIteratorWithHandle ctr,
 ??? - make ErrorHandling more efficient.


Some folks prefer to see all of the delta webrevs together so here is
a delta webrev relative to jdk10-09-full:

http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.09,10.10.2-delta/

 ? src/hotspot/share/runtime/globals.hpp and
 ? src/hotspot/share/runtime/vm_operations.hpp changes are white-space only
 ? so they are only visible via the file's patch link.


And, of course, some folks prefer to see everything all at once so here
is the latest full webrev:

http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.10-full/


Dan has submitted the bits for the usual Mach5 tier[1-5] testing. The
prelim Mach5 tier1 (JPRT equivalent) on these bits passed just fine...

The project is currently baselined on Jesper's 2017-11-17 PIT snapshot
so there will be some catching up to do before integration...

We welcome comments, suggestions and feedback.

Dan, Erik and Robbin


On 11/18/17 8:49 PM, Daniel D. Daugherty wrote:
> Greetings,
>
> Testing of the last round of changes revealed a hang in one of the new
> TLH tests. Robbin has fixed the hang, updated the existing TLH test, and
> added another TLH test for good measure.
>
> Here is the updated full webrev:
>
> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/
>
> Here is the updated delta webrev:
>
> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-delta/
>
> Dan ran the bits thru the usual Mach5 tier[1-5] testing and there are
> no unexpected failures.
>
> We welcome comments, suggestions and feedback.
>
> Dan, Erik and Robbin
>
>
> On 11/15/17 3:32 PM, Daniel D. Daugherty wrote:
>> Greetings,
>>
>> Robbin rebased the project last night/this morning to merge with Thread
>> Local Handshakes (TLH) and also picked up additional changesets up thru:
>>
>>> Changeset: fa736014cf28
>>> Author:??? cjplummer
>>> Date:????? 2017-11-14 18:08 -0800
>>> URL:http://hg.openjdk.java.net/jdk/hs/rev/fa736014cf28
>>>
>>> 8191049: Add alternate version of pns() that is callable from within 
>>> hotspot source
>>> Summary: added pns2() to debug.cpp
>>> Reviewed-by: stuefe, gthornbr
>>
>> This is the first time we've rebased the project to bits that are this
>> fresh (< 12 hours old at rebase time). We've done this because we think
>> we're done with this project and are in the final review-change-retest
>> cycle(s)... In other words, we're not planning on making any more major
>> changes for this project... :-)
>>
>> *** Time for code reviewers to chime in on this thread! ***
>>
>> Here is the updated full webrev:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-full/
>>
>> Here's is the delta webrev from the 2017.11.10 rebase:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-delta/
>>
>> Dan has submitted the bits for the usual Mach5 tier[1-5] testing
>> (and the new baseline also)...
>>
>> We're expecting this round to be a little noisier than usual because
>> we did not rebase on a PIT snapshot so the new baseline has not been
>> through Jesper's usual care-and-feeding of integration_blockers, etc.
>>
>> We welcome comments, suggestions and feedback.
>>
>> Dan, Erik and Robbin
>>
>>
>> On 11/14/17 4:48 PM, Daniel D. Daugherty wrote:
>>> Greetings,
>>>
>>> I rebased the project to the 2017.11.10 jdk/hs PIT snapshot.
>>> (Yes, we're playing chase-the-repo...)
>>>
>>> Here is the updated full webrev:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-07-full/
>>>
>>> Unlike the previous rebase, there were no changes required to the
>>> open code to get the rebased bits to build so there is no delta
>>> webrev.
>>>
>>> These bits have been run through JPRT and I've submitted the usual
>>> Mach5 tier[1-5] test run...
>>>
>>> We welcome comments, suggestions and feedback.
>>>
>>> Dan, Erik and Robbin
>>>
>>>
>>> On 11/13/17 12:30 PM, Daniel D. Daugherty wrote:
>>>> Greetings,
>>>>
>>>> I rebased the project to the 2017.10.26 jdk10/hs PIT snapshot.
>>>>
>>>> Here are the updated webrevs:
>>>>
>>>> Here's the mq comment for the change:
>>>>
>>>> ? Rebase to 2017.10.25 PIT snapshot.
>>>>
>>>> Here is the full webrev:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-full/
>>>>
>>>> And here is the delta webrev:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-delta/
>>>>
>>>> I ran the above bits throught Mach5 tier[1-5] testing over the holiday
>>>> weekend. Didn't see any issues in a quick look. Going to take a closer
>>>> look today.
>>>>
>>>> We welcome comments, suggestions and feedback.
>>>>
>>>> Dan, Erik and Robbin
>>>>
>>>>
>>>> On 11/8/17 1:05 PM, Daniel D. Daugherty wrote:
>>>>> Greetings,
>>>>>
>>>>> Resolving one of the code review comments (from both Stefan K and 
>>>>> Coleen)
>>>>> on jdk10-04-full required quite a few changes so it is being done 
>>>>> as a
>>>>> standalone patch and corresponding webrevs:
>>>>>
>>>>> Here's the mq comment for the change:
>>>>>
>>>>> ? stefank, coleenp CR - refactor most JavaThreadIterator usage to use
>>>>> ??? JavaThreadIteratorWithHandle.
>>>>>
>>>>> Here is the full webrev:
>>>>>
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-full/
>>>>>
>>>>> And here is the delta webrev:
>>>>>
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-delta/
>>>>>
>>>>> We welcome comments, suggestions and feedback.
>>>>>
>>>>> Dan, Erik and Robbin
>>>>>
>>>>>
>>>>> On 10/9/17 3:41 PM, Daniel D. Daugherty wrote:
>>>>>> Greetings,
>>>>>>
>>>>>> We have a (eXtra Large) fix for the following bug:
>>>>>>
>>>>>> 8167108 inconsistent handling of SR_lock can lead to crashes
>>>>>> https://bugs.openjdk.java.net/browse/JDK-8167108
>>>>>>
>>>>>> This fix adds a Safe Memory Reclamation (SMR) mechanism based on
>>>>>> Hazard Pointers to manage JavaThread lifecycle.
>>>>>>
>>>>>> Here's a PDF for the internal wiki that we've been using to describe
>>>>>> and track the work on this project:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/SMR_and_JavaThread_Lifecycle-JDK10-04.pdf 
>>>>>>
>>>>>>
>>>>>> Dan has noticed that the indenting is wrong in some of the code 
>>>>>> quotes
>>>>>> in the PDF that are not present in the internal wiki. We don't 
>>>>>> have a
>>>>>> solution for that problem yet.
>>>>>>
>>>>>> Here's the webrev for current JDK10 version of this fix:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-04-full
>>>>>>
>>>>>> This fix has been run through many rounds of JPRT and Mach5 
>>>>>> tier[2-5]
>>>>>> testing, additional stress testing on Dan's Solaris X64 server, and
>>>>>> additional testing on Erik and Robbin's machines.
>>>>>>
>>>>>> We welcome comments, suggestions and feedback.
>>>>>>
>>>>>> Daniel Daugherty
>>>>>> Erik Osterlund
>>>>>> Robbin Ehn
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>



From kim.barrett at oracle.com  Wed Nov 22 01:39:25 2017
From: kim.barrett at oracle.com (Kim Barrett)
Date: Tue, 21 Nov 2017 20:39:25 -0500
Subject: RFR (XS): 8188877: Improper synchronization in offer_termination
In-Reply-To: <CY1PR0701MB163288CD9B58A163B2F16F7C84230@CY1PR0701MB1632.namprd07.prod.outlook.com>
References: <CY1PR0701MB16328948D4FDF19FBC30F65684710@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <3229BC22-BB8A-4D3D-8440-AB799AB5394F@oracle.com>
 <CY1PR0701MB16328E4C6563BE60CF8E99D084750@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <4D9972BA-B76C-4FB7-B24D-31D507DB1A6A@oracle.com>
 <1510821711.5332.3.camel@oracle.com>
 <CY1PR0701MB163288CD9B58A163B2F16F7C84230@CY1PR0701MB1632.namprd07.prod.outlook.com>
Message-ID: <AF3564FF-2C19-45A7-A2AB-494E30BFC8C3@oracle.com>

> On Nov 21, 2017, at 4:53 PM, White, Derek <Derek.White at cavium.com> wrote:
> 
> Hi Kim, Thomas,
> 
> So Thomas, I think I agree with my original self too ?.

Agreed.  I fell into a common trap, and should know better.

Looks good.



From david.holmes at oracle.com  Wed Nov 22 04:26:39 2017
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 22 Nov 2017 14:26:39 +1000
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <5365606e-4036-b5d7-9bac-e91e42f38d01@oracle.com>
References: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
 <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>
 <836b5c6c-5eef-9245-c16a-e71372093d9a@oracle.com>
 <82d0e473-a24f-9feb-82e9-7d04c4d7a91d@oracle.com>
 <354f7ece-66bc-c065-9ee8-f55abcc1c23e@oracle.com>
 <0ff8df8e-e93c-02d2-cb97-b13b1ac0f53f@oracle.com>
 <5365606e-4036-b5d7-9bac-e91e42f38d01@oracle.com>
Message-ID: <4d5d448e-3902-8465-197e-18a5607fbd70@oracle.com>

Hi Dan,

On 22/11/2017 10:12 AM, Daniel D. Daugherty wrote:
> Greetings,
> 
> *** We are wrapping up code review on this project so it is time ***
> *** for the various code reviewers to chime in one last time. ***
> 
> In this latest round, we had three different reviewers chime in so we're
> doing delta webrevs for each of those resolutions:
> 
> David H's resolutions:
> 
> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.09,10.10.0-delta/ 
> 
> 
>  ? mq comment for dholmes_CR:
>  ??? - Fix indents, trailing spaces and various typos.
>  ??? - Add descriptions for the '_cnt', '_max' and '_times" fields,
>  ????? add impl notes to document the type choices.

src/hotspot/share/runtime/thread.cpp

3466 // range. Unsigned 64-bit would be more future proof, but 64-bit 
atomic inc
3467 // isn't available everywhere (or is it?).

64-bit atomics should be available on all currently supported platforms 
(the last time this was an issue was for PPC32 - and the atomic 
templates should have filled in any previous holes in the allowed 
types). But if it's always 64-bit then you'd have to use Atomic::load to 
allow for 32-bit platforms.

These counts etc are all just for statistics so we aren't really 
concerned about eventual overflows in long running VMs - right?

---

                                        // # of parallel threads in 
_smr_delete_lock->wait().
   static uint                  _smr_delete_lock_wait_cnt;
                                        // Max # of parallel threads in 
_smr_delete_lock->wait().


why are the comments indented like that? I thought this is what Coleen 
previously commented on. Please just start the comments in the first 
column - no need for any indent. Thanks.

>  ? src/hotspot/share/runtime/globals.hpp change is white-space only so it
>  ? is only visible via the file's patch link.
> 
> 
> Robin W's resolutions:
> 
> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.10.0,10.10.1-delta/ 
> 
> 
>  ? mq comment for robinw_CR:
>  ??? - Fix some inefficient code, update some comments, fix some indents,
>  ????? and add some 'const' specifiers.
> 
>  ? src/hotspot/share/runtime/vm_operations.hpp change is white-space 
> only so
>  ? it is only visible via the file's patch link.

All looks fine to me. Some good catches there from Robin!

> Coleen's resolutions:
> 
> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.10.1,10.10.2-delta/ 
> 
> 
>  ? mq comment for coleenp_CR:
>  ??? - Change ThreadsList::_threads from 'mtGC' -> 'mtThread',
>  ??? - add header comment to threadSMR.hpp file,
>  ??? - cleanup JavaThreadIteratorWithHandle ctr,
>  ??? - make ErrorHandling more efficient.

src/hotspot/share/runtime/threadSMR.hpp

+ // Thread Safe Memory Reclaimation (Thread-SMR) support.

Only one 'i' in Reclamation :)

Thanks,
David
------

> 
> Some folks prefer to see all of the delta webrevs together so here is
> a delta webrev relative to jdk10-09-full:
> 
> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.09,10.10.2-delta/ 
> 
> 
>  ? src/hotspot/share/runtime/globals.hpp and
>  ? src/hotspot/share/runtime/vm_operations.hpp changes are white-space only
>  ? so they are only visible via the file's patch link.
> 
> 
> And, of course, some folks prefer to see everything all at once so here
> is the latest full webrev:
> 
> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.10-full/
> 
> 
> Dan has submitted the bits for the usual Mach5 tier[1-5] testing. The
> prelim Mach5 tier1 (JPRT equivalent) on these bits passed just fine...
> 
> The project is currently baselined on Jesper's 2017-11-17 PIT snapshot
> so there will be some catching up to do before integration...
> 
> We welcome comments, suggestions and feedback.
> 
> Dan, Erik and Robbin
> 
> 
> On 11/18/17 8:49 PM, Daniel D. Daugherty wrote:
>> Greetings,
>>
>> Testing of the last round of changes revealed a hang in one of the new
>> TLH tests. Robbin has fixed the hang, updated the existing TLH test, and
>> added another TLH test for good measure.
>>
>> Here is the updated full webrev:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/
>>
>> Here is the updated delta webrev:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-delta/
>>
>> Dan ran the bits thru the usual Mach5 tier[1-5] testing and there are
>> no unexpected failures.
>>
>> We welcome comments, suggestions and feedback.
>>
>> Dan, Erik and Robbin
>>
>>
>> On 11/15/17 3:32 PM, Daniel D. Daugherty wrote:
>>> Greetings,
>>>
>>> Robbin rebased the project last night/this morning to merge with Thread
>>> Local Handshakes (TLH) and also picked up additional changesets up thru:
>>>
>>>> Changeset: fa736014cf28
>>>> Author:??? cjplummer
>>>> Date:????? 2017-11-14 18:08 -0800
>>>> URL:http://hg.openjdk.java.net/jdk/hs/rev/fa736014cf28
>>>>
>>>> 8191049: Add alternate version of pns() that is callable from within 
>>>> hotspot source
>>>> Summary: added pns2() to debug.cpp
>>>> Reviewed-by: stuefe, gthornbr
>>>
>>> This is the first time we've rebased the project to bits that are this
>>> fresh (< 12 hours old at rebase time). We've done this because we think
>>> we're done with this project and are in the final review-change-retest
>>> cycle(s)... In other words, we're not planning on making any more major
>>> changes for this project... :-)
>>>
>>> *** Time for code reviewers to chime in on this thread! ***
>>>
>>> Here is the updated full webrev:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-full/
>>>
>>> Here's is the delta webrev from the 2017.11.10 rebase:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-delta/
>>>
>>> Dan has submitted the bits for the usual Mach5 tier[1-5] testing
>>> (and the new baseline also)...
>>>
>>> We're expecting this round to be a little noisier than usual because
>>> we did not rebase on a PIT snapshot so the new baseline has not been
>>> through Jesper's usual care-and-feeding of integration_blockers, etc.
>>>
>>> We welcome comments, suggestions and feedback.
>>>
>>> Dan, Erik and Robbin
>>>
>>>
>>> On 11/14/17 4:48 PM, Daniel D. Daugherty wrote:
>>>> Greetings,
>>>>
>>>> I rebased the project to the 2017.11.10 jdk/hs PIT snapshot.
>>>> (Yes, we're playing chase-the-repo...)
>>>>
>>>> Here is the updated full webrev:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-07-full/
>>>>
>>>> Unlike the previous rebase, there were no changes required to the
>>>> open code to get the rebased bits to build so there is no delta
>>>> webrev.
>>>>
>>>> These bits have been run through JPRT and I've submitted the usual
>>>> Mach5 tier[1-5] test run...
>>>>
>>>> We welcome comments, suggestions and feedback.
>>>>
>>>> Dan, Erik and Robbin
>>>>
>>>>
>>>> On 11/13/17 12:30 PM, Daniel D. Daugherty wrote:
>>>>> Greetings,
>>>>>
>>>>> I rebased the project to the 2017.10.26 jdk10/hs PIT snapshot.
>>>>>
>>>>> Here are the updated webrevs:
>>>>>
>>>>> Here's the mq comment for the change:
>>>>>
>>>>> ? Rebase to 2017.10.25 PIT snapshot.
>>>>>
>>>>> Here is the full webrev:
>>>>>
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-full/
>>>>>
>>>>> And here is the delta webrev:
>>>>>
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-delta/
>>>>>
>>>>> I ran the above bits throught Mach5 tier[1-5] testing over the holiday
>>>>> weekend. Didn't see any issues in a quick look. Going to take a closer
>>>>> look today.
>>>>>
>>>>> We welcome comments, suggestions and feedback.
>>>>>
>>>>> Dan, Erik and Robbin
>>>>>
>>>>>
>>>>> On 11/8/17 1:05 PM, Daniel D. Daugherty wrote:
>>>>>> Greetings,
>>>>>>
>>>>>> Resolving one of the code review comments (from both Stefan K and 
>>>>>> Coleen)
>>>>>> on jdk10-04-full required quite a few changes so it is being done 
>>>>>> as a
>>>>>> standalone patch and corresponding webrevs:
>>>>>>
>>>>>> Here's the mq comment for the change:
>>>>>>
>>>>>> ? stefank, coleenp CR - refactor most JavaThreadIterator usage to use
>>>>>> ??? JavaThreadIteratorWithHandle.
>>>>>>
>>>>>> Here is the full webrev:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-full/
>>>>>>
>>>>>> And here is the delta webrev:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-delta/
>>>>>>
>>>>>> We welcome comments, suggestions and feedback.
>>>>>>
>>>>>> Dan, Erik and Robbin
>>>>>>
>>>>>>
>>>>>> On 10/9/17 3:41 PM, Daniel D. Daugherty wrote:
>>>>>>> Greetings,
>>>>>>>
>>>>>>> We have a (eXtra Large) fix for the following bug:
>>>>>>>
>>>>>>> 8167108 inconsistent handling of SR_lock can lead to crashes
>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8167108
>>>>>>>
>>>>>>> This fix adds a Safe Memory Reclamation (SMR) mechanism based on
>>>>>>> Hazard Pointers to manage JavaThread lifecycle.
>>>>>>>
>>>>>>> Here's a PDF for the internal wiki that we've been using to describe
>>>>>>> and track the work on this project:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/SMR_and_JavaThread_Lifecycle-JDK10-04.pdf 
>>>>>>>
>>>>>>>
>>>>>>> Dan has noticed that the indenting is wrong in some of the code 
>>>>>>> quotes
>>>>>>> in the PDF that are not present in the internal wiki. We don't 
>>>>>>> have a
>>>>>>> solution for that problem yet.
>>>>>>>
>>>>>>> Here's the webrev for current JDK10 version of this fix:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-04-full
>>>>>>>
>>>>>>> This fix has been run through many rounds of JPRT and Mach5 
>>>>>>> tier[2-5]
>>>>>>> testing, additional stress testing on Dan's Solaris X64 server, and
>>>>>>> additional testing on Erik and Robbin's machines.
>>>>>>>
>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>
>>>>>>> Daniel Daugherty
>>>>>>> Erik Osterlund
>>>>>>> Robbin Ehn
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
> 


From erik.osterlund at oracle.com  Wed Nov 22 09:07:30 2017
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Wed, 22 Nov 2017 10:07:30 +0100
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <70cc1311-de55-5502-159b-cd0a3522cfd6@oracle.com>
References: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
 <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>
 <836b5c6c-5eef-9245-c16a-e71372093d9a@oracle.com>
 <82d0e473-a24f-9feb-82e9-7d04c4d7a91d@oracle.com>
 <354f7ece-66bc-c065-9ee8-f55abcc1c23e@oracle.com>
 <0ff8df8e-e93c-02d2-cb97-b13b1ac0f53f@oracle.com>
 <e56b44ce-28a9-822e-bbac-4b64f5f85c58@oracle.com>
 <70cc1311-de55-5502-159b-cd0a3522cfd6@oracle.com>
Message-ID: <5A153E52.8000704@oracle.com>

Hi,

Some replies...

On 2017-11-21 17:28, Daniel D. Daugherty wrote:
> Hi Coleen!
>
> Thanks for making time to review the Thread-SMR stuff again!!
>
> I have added back the other three OpenJDK aliases... This review is
> being done on _four_ different OpenJDK aliases.
>
> As always, replies are embedded below...
>
>
> On 11/20/17 3:12 PM, coleen.phillimore at oracle.com wrote:
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/src/hotspot/share/runtime/threadSMR.cpp.html 
>>
>>
>>   32 ThreadsList::ThreadsList(int entries) : _length(entries), 
>> _threads(NEW_C_HEAP_ARRAY(JavaThread*, entries + 1, mtGC)), 
>> _next_list(NULL) {
>>
>> Seems like it should be mtThread rather than mtGC.
>
> Fixed. Definitely an artifact of Erik's original prototype when he
> extracted Thread-SMR from his GC work... Thanks for catching it.
>

Confirmed. At the time I considered the Threads list overheads GC 
overheads, but I agree mtThread is a better fit today.

>
>>
>> + return (unsigned int)(((uint32_t)(uintptr_t)s1) * 2654435761u);
>>
>> Can you add a comment about where this number came from?
>
> I'll have to get that from Erik...

Wow, that looks like code I wrote a *very* long time ago. :) That is a 
variation of Knuth's multiplicative hash which is outlined in a comment 
in synchronizer.cpp and referred to in that comment as a phi-based 
scheme. Basically the magic number is 2^32 * Phi (the golden ratio), 
which happens to be a useful value for building a reasonably simple yet 
pretty good hash function. It is not the optimal hash function, but 
seemed to definitely be good enough for our purposes.

Thanks,
/Erik


From aph at redhat.com  Wed Nov 22 09:13:05 2017
From: aph at redhat.com (Andrew Haley)
Date: Wed, 22 Nov 2017 09:13:05 +0000
Subject: RFR (XS): 8188877: Improper synchronization in offer_termination
In-Reply-To: <CY1PR0701MB163288CD9B58A163B2F16F7C84230@CY1PR0701MB1632.namprd07.prod.outlook.com>
References: <CY1PR0701MB16328948D4FDF19FBC30F65684710@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <3229BC22-BB8A-4D3D-8440-AB799AB5394F@oracle.com>
 <CY1PR0701MB16328E4C6563BE60CF8E99D084750@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <4D9972BA-B76C-4FB7-B24D-31D507DB1A6A@oracle.com>
 <1510821711.5332.3.camel@oracle.com>
 <CY1PR0701MB163288CD9B58A163B2F16F7C84230@CY1PR0701MB1632.namprd07.prod.outlook.com>
Message-ID: <ba0c0855-188a-decf-932b-b11200dc9fc4@redhat.com>

On 21/11/17 21:53, White, Derek wrote:
> My understanding is that the "acquire" semantics are entirely about
> memory ordering, within a CPU. In particular it prevents "following
> loads" from executing before the "load acquire".
> 
>
> There is nothing in the "load acquire" that causes it to synchronize
> with the memory system more or less quickly than a naked load.

The abstract architecture only specifies things in terms of ordering
between loads, but it has to be implemented somehow, and this is MESI
or something similar.  Stores cause invalidate messages to be sent,
and these are put into the reader's invalidate queue.  When that
reader executes a load barrier it marks all the entries currently in
its invalidate queue.  The next load will wait until all marked
entries have been applied to the reader's cache.

> Either kind of read will eventually notice that its local cached
> value has been invalidated and will fetch the updated value.

Eventually, yes.

-- 
Andrew Haley
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From erik.osterlund at oracle.com  Wed Nov 22 09:48:32 2017
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Wed, 22 Nov 2017 10:48:32 +0100
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <5e154ff2-65d4-bad4-db5e-88cf4c733c7d@oracle.com>
References: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
 <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>
 <836b5c6c-5eef-9245-c16a-e71372093d9a@oracle.com>
 <82d0e473-a24f-9feb-82e9-7d04c4d7a91d@oracle.com>
 <354f7ece-66bc-c065-9ee8-f55abcc1c23e@oracle.com>
 <0ff8df8e-e93c-02d2-cb97-b13b1ac0f53f@oracle.com>
 <e56b44ce-28a9-822e-bbac-4b64f5f85c58@oracle.com>
 <70cc1311-de55-5502-159b-cd0a3522cfd6@oracle.com>
 <68b0b2bc-2d26-5442-bafb-5ce21420767b@oracle.com>
 <5e154ff2-65d4-bad4-db5e-88cf4c733c7d@oracle.com>
Message-ID: <5A1547F0.7040206@oracle.com>

Hi,

Some replies...

On 2017-11-21 18:36, Daniel D. Daugherty wrote:
> Thanks for keeping all the OpenJDK aliases!
>
>
> On 11/21/17 12:24 PM, coleen.phillimore at oracle.com wrote:
>>
>>
>> On 11/21/17 11:28 AM, Daniel D. Daugherty wrote:
>>>
>>> On 11/20/17 3:12 PM, coleen.phillimore at oracle.com wrote:
>>>
>>>> I can't find the caller of threads_do_smr.
>>>
>>> I'm guessing that's used by the GC code that depends on Thread-SMR...
>>>
>>
>> They  should add this API when the add the use of it then.  I don't 
>> see it in any sources.
>
> We'll see what Erik wants to do...

The new iterators should be more than enough, so that closure-based API 
is not going to be missed when removed.

>
>>
>>>
>>>> If these functions xchg_smr_thread_list, get_smr_java_thread_list, 
>>>> inc_smr_deleted_thread_count are only used by thread.cpp, I think 
>>>> they should go in that file and not in the .inline.hpp file to be 
>>>> included and possibly called by other files.  I think they're 
>>>> private to class Threads.
>>>
>>> I have a vague memory that some of the compilers don't do inlining when
>>> an "inline" function is in a .cpp. I believe we want these functions
>>> to be inlined for performance reasons. Erik should probably chime in
>>> here.
>>
>> I don't see why this should be.  Maybe some (older) compilers require 
>> it to be found before the call though, but that can still be 
>> accomplished in the .cpp file.
>
> Again, we'll see what Erik wants to do...

I don't mind. Either file works for me. For me it's more intuitive if 
inline member function definitions are in the .inline.hpp files. But if 
there are strong forces to move this to the cpp file, then sure.

Thanks,
/Erik



From daniel.daugherty at oracle.com  Wed Nov 22 12:51:10 2017
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Wed, 22 Nov 2017 07:51:10 -0500
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <4d5d448e-3902-8465-197e-18a5607fbd70@oracle.com>
References: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
 <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>
 <836b5c6c-5eef-9245-c16a-e71372093d9a@oracle.com>
 <82d0e473-a24f-9feb-82e9-7d04c4d7a91d@oracle.com>
 <354f7ece-66bc-c065-9ee8-f55abcc1c23e@oracle.com>
 <0ff8df8e-e93c-02d2-cb97-b13b1ac0f53f@oracle.com>
 <5365606e-4036-b5d7-9bac-e91e42f38d01@oracle.com>
 <4d5d448e-3902-8465-197e-18a5607fbd70@oracle.com>
Message-ID: <4fd06a59-f6b5-33b2-53f5-7bfc96f866a7@oracle.com>

On 11/21/17 11:26 PM, David Holmes wrote:
> Hi Dan,
>
> On 22/11/2017 10:12 AM, Daniel D. Daugherty wrote:
>> Greetings,
>>
>> *** We are wrapping up code review on this project so it is time ***
>> *** for the various code reviewers to chime in one last time. ***
>>
>> In this latest round, we had three different reviewers chime in so we're
>> doing delta webrevs for each of those resolutions:
>>
>> David H's resolutions:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.09,10.10.0-delta/ 
>>
>>
>> ?? mq comment for dholmes_CR:
>> ???? - Fix indents, trailing spaces and various typos.
>> ???? - Add descriptions for the '_cnt', '_max' and '_times" fields,
>> ?????? add impl notes to document the type choices.
>
> src/hotspot/share/runtime/thread.cpp
>
> 3466 // range. Unsigned 64-bit would be more future proof, but 64-bit 
> atomic inc
> 3467 // isn't available everywhere (or is it?).
>
> 64-bit atomics should be available on all currently supported 
> platforms (the last time this was an issue was for PPC32 - and the 
> atomic templates should have filled in any previous holes in the 
> allowed types).

Hmmm... I ran into this when I merged the project with the atomic
templates. I got a JPRT build failure on one of the ARM platforms...
Unfortunately, I didn't save the log files so I can't quote the
error message from the template stuff...


> But if it's always 64-bit then you'd have to use Atomic::load to allow 
> for 32-bit platforms.

What's the official story on 32-bit platforms? Is that what
bit me in JPRT? i.e., did we still have a 32-bit ARM build
config'ed in JPRT a month or two ago???


> These counts etc are all just for statistics so we aren't really 
> concerned about eventual overflows in long running VMs - right?

Yup these are just statistics... overflow is not a killer.


>
> ---
>
> ?????????????????????????????????????? // # of parallel threads in 
> _smr_delete_lock->wait().
> ? static uint????????????????? _smr_delete_lock_wait_cnt;
> ?????????????????????????????????????? // Max # of parallel threads in 
> _smr_delete_lock->wait().
>
>
> why are the comments indented like that? I thought this is what Coleen 
> previously commented on. Please just start the comments in the first 
> column - no need for any indent. Thanks.

Coleen asked for the new comments in thread.cpp to be moved
over to column 1. She asked for the new comments in thread.hpp
to be indented with more spaces. I started with 2 spaces and
the variables still didn't stand out. I tried 4 spaces and
they still didn't stand out probably because of the leading
_smr_... So I went with 8 spaces...


>
>> ?? src/hotspot/share/runtime/globals.hpp change is white-space only 
>> so it
>> ?? is only visible via the file's patch link.
>>
>>
>> Robin W's resolutions:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.10.0,10.10.1-delta/ 
>>
>>
>> ?? mq comment for robinw_CR:
>> ???? - Fix some inefficient code, update some comments, fix some 
>> indents,
>> ?????? and add some 'const' specifiers.
>>
>> ?? src/hotspot/share/runtime/vm_operations.hpp change is white-space 
>> only so
>> ?? it is only visible via the file's patch link.
>
> All looks fine to me. Some good catches there from Robin!

Yes, a new pair of eyes on this code is always useful.


>
>> Coleen's resolutions:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.10.1,10.10.2-delta/ 
>>
>>
>> ?? mq comment for coleenp_CR:
>> ???? - Change ThreadsList::_threads from 'mtGC' -> 'mtThread',
>> ???? - add header comment to threadSMR.hpp file,
>> ???? - cleanup JavaThreadIteratorWithHandle ctr,
>> ???? - make ErrorHandling more efficient.
>
> src/hotspot/share/runtime/threadSMR.hpp
>
> + // Thread Safe Memory Reclaimation (Thread-SMR) support.
>
> Only one 'i' in Reclamation :)

Will fix. I tried typing both and neither looked right to me.
(I might be getting blurry eyed with this stuff)...
So I went with the spelling from Coleen's comment... :-)

Thanks for taking another look!!

Dan


>
> Thanks,
> David
> ------
>
>>
>> Some folks prefer to see all of the delta webrevs together so here is
>> a delta webrev relative to jdk10-09-full:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.09,10.10.2-delta/ 
>>
>>
>> ?? src/hotspot/share/runtime/globals.hpp and
>> ?? src/hotspot/share/runtime/vm_operations.hpp changes are 
>> white-space only
>> ?? so they are only visible via the file's patch link.
>>
>>
>> And, of course, some folks prefer to see everything all at once so here
>> is the latest full webrev:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.10-full/
>>
>>
>> Dan has submitted the bits for the usual Mach5 tier[1-5] testing. The
>> prelim Mach5 tier1 (JPRT equivalent) on these bits passed just fine...
>>
>> The project is currently baselined on Jesper's 2017-11-17 PIT snapshot
>> so there will be some catching up to do before integration...
>>
>> We welcome comments, suggestions and feedback.
>>
>> Dan, Erik and Robbin
>>
>>
>> On 11/18/17 8:49 PM, Daniel D. Daugherty wrote:
>>> Greetings,
>>>
>>> Testing of the last round of changes revealed a hang in one of the new
>>> TLH tests. Robbin has fixed the hang, updated the existing TLH test, 
>>> and
>>> added another TLH test for good measure.
>>>
>>> Here is the updated full webrev:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/
>>>
>>> Here is the updated delta webrev:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-delta/
>>>
>>> Dan ran the bits thru the usual Mach5 tier[1-5] testing and there are
>>> no unexpected failures.
>>>
>>> We welcome comments, suggestions and feedback.
>>>
>>> Dan, Erik and Robbin
>>>
>>>
>>> On 11/15/17 3:32 PM, Daniel D. Daugherty wrote:
>>>> Greetings,
>>>>
>>>> Robbin rebased the project last night/this morning to merge with 
>>>> Thread
>>>> Local Handshakes (TLH) and also picked up additional changesets up 
>>>> thru:
>>>>
>>>>> Changeset: fa736014cf28
>>>>> Author:??? cjplummer
>>>>> Date:????? 2017-11-14 18:08 -0800
>>>>> URL:http://hg.openjdk.java.net/jdk/hs/rev/fa736014cf28
>>>>>
>>>>> 8191049: Add alternate version of pns() that is callable from 
>>>>> within hotspot source
>>>>> Summary: added pns2() to debug.cpp
>>>>> Reviewed-by: stuefe, gthornbr
>>>>
>>>> This is the first time we've rebased the project to bits that are this
>>>> fresh (< 12 hours old at rebase time). We've done this because we 
>>>> think
>>>> we're done with this project and are in the final review-change-retest
>>>> cycle(s)... In other words, we're not planning on making any more 
>>>> major
>>>> changes for this project... :-)
>>>>
>>>> *** Time for code reviewers to chime in on this thread! ***
>>>>
>>>> Here is the updated full webrev:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-full/
>>>>
>>>> Here's is the delta webrev from the 2017.11.10 rebase:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-delta/
>>>>
>>>> Dan has submitted the bits for the usual Mach5 tier[1-5] testing
>>>> (and the new baseline also)...
>>>>
>>>> We're expecting this round to be a little noisier than usual because
>>>> we did not rebase on a PIT snapshot so the new baseline has not been
>>>> through Jesper's usual care-and-feeding of integration_blockers, etc.
>>>>
>>>> We welcome comments, suggestions and feedback.
>>>>
>>>> Dan, Erik and Robbin
>>>>
>>>>
>>>> On 11/14/17 4:48 PM, Daniel D. Daugherty wrote:
>>>>> Greetings,
>>>>>
>>>>> I rebased the project to the 2017.11.10 jdk/hs PIT snapshot.
>>>>> (Yes, we're playing chase-the-repo...)
>>>>>
>>>>> Here is the updated full webrev:
>>>>>
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-07-full/
>>>>>
>>>>> Unlike the previous rebase, there were no changes required to the
>>>>> open code to get the rebased bits to build so there is no delta
>>>>> webrev.
>>>>>
>>>>> These bits have been run through JPRT and I've submitted the usual
>>>>> Mach5 tier[1-5] test run...
>>>>>
>>>>> We welcome comments, suggestions and feedback.
>>>>>
>>>>> Dan, Erik and Robbin
>>>>>
>>>>>
>>>>> On 11/13/17 12:30 PM, Daniel D. Daugherty wrote:
>>>>>> Greetings,
>>>>>>
>>>>>> I rebased the project to the 2017.10.26 jdk10/hs PIT snapshot.
>>>>>>
>>>>>> Here are the updated webrevs:
>>>>>>
>>>>>> Here's the mq comment for the change:
>>>>>>
>>>>>> ? Rebase to 2017.10.25 PIT snapshot.
>>>>>>
>>>>>> Here is the full webrev:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-full/
>>>>>>
>>>>>> And here is the delta webrev:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-delta/
>>>>>>
>>>>>> I ran the above bits throught Mach5 tier[1-5] testing over the 
>>>>>> holiday
>>>>>> weekend. Didn't see any issues in a quick look. Going to take a 
>>>>>> closer
>>>>>> look today.
>>>>>>
>>>>>> We welcome comments, suggestions and feedback.
>>>>>>
>>>>>> Dan, Erik and Robbin
>>>>>>
>>>>>>
>>>>>> On 11/8/17 1:05 PM, Daniel D. Daugherty wrote:
>>>>>>> Greetings,
>>>>>>>
>>>>>>> Resolving one of the code review comments (from both Stefan K 
>>>>>>> and Coleen)
>>>>>>> on jdk10-04-full required quite a few changes so it is being 
>>>>>>> done as a
>>>>>>> standalone patch and corresponding webrevs:
>>>>>>>
>>>>>>> Here's the mq comment for the change:
>>>>>>>
>>>>>>> ? stefank, coleenp CR - refactor most JavaThreadIterator usage 
>>>>>>> to use
>>>>>>> ??? JavaThreadIteratorWithHandle.
>>>>>>>
>>>>>>> Here is the full webrev:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-full/
>>>>>>>
>>>>>>> And here is the delta webrev:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-delta/
>>>>>>>
>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>
>>>>>>> Dan, Erik and Robbin
>>>>>>>
>>>>>>>
>>>>>>> On 10/9/17 3:41 PM, Daniel D. Daugherty wrote:
>>>>>>>> Greetings,
>>>>>>>>
>>>>>>>> We have a (eXtra Large) fix for the following bug:
>>>>>>>>
>>>>>>>> 8167108 inconsistent handling of SR_lock can lead to crashes
>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8167108
>>>>>>>>
>>>>>>>> This fix adds a Safe Memory Reclamation (SMR) mechanism based on
>>>>>>>> Hazard Pointers to manage JavaThread lifecycle.
>>>>>>>>
>>>>>>>> Here's a PDF for the internal wiki that we've been using to 
>>>>>>>> describe
>>>>>>>> and track the work on this project:
>>>>>>>>
>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/SMR_and_JavaThread_Lifecycle-JDK10-04.pdf 
>>>>>>>>
>>>>>>>>
>>>>>>>> Dan has noticed that the indenting is wrong in some of the code 
>>>>>>>> quotes
>>>>>>>> in the PDF that are not present in the internal wiki. We don't 
>>>>>>>> have a
>>>>>>>> solution for that problem yet.
>>>>>>>>
>>>>>>>> Here's the webrev for current JDK10 version of this fix:
>>>>>>>>
>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-04-full
>>>>>>>>
>>>>>>>> This fix has been run through many rounds of JPRT and Mach5 
>>>>>>>> tier[2-5]
>>>>>>>> testing, additional stress testing on Dan's Solaris X64 server, 
>>>>>>>> and
>>>>>>>> additional testing on Erik and Robbin's machines.
>>>>>>>>
>>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>>
>>>>>>>> Daniel Daugherty
>>>>>>>> Erik Osterlund
>>>>>>>> Robbin Ehn
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>



From daniel.daugherty at oracle.com  Wed Nov 22 12:53:16 2017
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Wed, 22 Nov 2017 07:53:16 -0500
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <5A153E52.8000704@oracle.com>
References: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
 <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>
 <836b5c6c-5eef-9245-c16a-e71372093d9a@oracle.com>
 <82d0e473-a24f-9feb-82e9-7d04c4d7a91d@oracle.com>
 <354f7ece-66bc-c065-9ee8-f55abcc1c23e@oracle.com>
 <0ff8df8e-e93c-02d2-cb97-b13b1ac0f53f@oracle.com>
 <e56b44ce-28a9-822e-bbac-4b64f5f85c58@oracle.com>
 <70cc1311-de55-5502-159b-cd0a3522cfd6@oracle.com>
 <5A153E52.8000704@oracle.com>
Message-ID: <ea91e54c-0d69-bc13-f2f6-8bfacb9719d6@oracle.com>

On 11/22/17 4:07 AM, Erik ?sterlund wrote:
> Hi,
>
> Some replies...
>
> On 2017-11-21 17:28, Daniel D. Daugherty wrote:
>> Hi Coleen!
>>
>> Thanks for making time to review the Thread-SMR stuff again!!
>>
>> I have added back the other three OpenJDK aliases... This review is
>> being done on _four_ different OpenJDK aliases.
>>
>> As always, replies are embedded below...
>>
>>
>> On 11/20/17 3:12 PM, coleen.phillimore at oracle.com wrote:
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/src/hotspot/share/runtime/threadSMR.cpp.html 
>>>
>>>
>>> ? 32 ThreadsList::ThreadsList(int entries) : _length(entries), 
>>> _threads(NEW_C_HEAP_ARRAY(JavaThread*, entries + 1, mtGC)), 
>>> _next_list(NULL) {
>>>
>>> Seems like it should be mtThread rather than mtGC.
>>
>> Fixed. Definitely an artifact of Erik's original prototype when he
>> extracted Thread-SMR from his GC work... Thanks for catching it.
>>
>
> Confirmed. At the time I considered the Threads list overheads GC 
> overheads, but I agree mtThread is a better fit today.
>
>>
>>>
>>> + return (unsigned int)(((uint32_t)(uintptr_t)s1) * 2654435761u);
>>>
>>> Can you add a comment about where this number came from?
>>
>> I'll have to get that from Erik...
>
> Wow, that looks like code I wrote a *very* long time ago. :) That is a 
> variation of Knuth's multiplicative hash which is outlined in a 
> comment in synchronizer.cpp and referred to in that comment as a 
> phi-based scheme. Basically the magic number is 2^32 * Phi (the golden 
> ratio), which happens to be a useful value for building a reasonably 
> simple yet pretty good hash function. It is not the optimal hash 
> function, but seemed to definitely be good enough for our purposes.

So a reasonable comment would be:

// The literal value is 2^32 * Phi (the golden ratio).

If so, then I'll add it in my wrap up round...

Dan

>
> Thanks,
> /Erik



From daniel.daugherty at oracle.com  Wed Nov 22 12:54:51 2017
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Wed, 22 Nov 2017 07:54:51 -0500
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <5A1547F0.7040206@oracle.com>
References: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
 <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>
 <836b5c6c-5eef-9245-c16a-e71372093d9a@oracle.com>
 <82d0e473-a24f-9feb-82e9-7d04c4d7a91d@oracle.com>
 <354f7ece-66bc-c065-9ee8-f55abcc1c23e@oracle.com>
 <0ff8df8e-e93c-02d2-cb97-b13b1ac0f53f@oracle.com>
 <e56b44ce-28a9-822e-bbac-4b64f5f85c58@oracle.com>
 <70cc1311-de55-5502-159b-cd0a3522cfd6@oracle.com>
 <68b0b2bc-2d26-5442-bafb-5ce21420767b@oracle.com>
 <5e154ff2-65d4-bad4-db5e-88cf4c733c7d@oracle.com>
 <5A1547F0.7040206@oracle.com>
Message-ID: <6906e0e0-e5cc-a415-6db2-13eff8d76ca8@oracle.com>

On 11/22/17 4:48 AM, Erik ?sterlund wrote:
> Hi,
>
> Some replies...
>
> On 2017-11-21 18:36, Daniel D. Daugherty wrote:
>> Thanks for keeping all the OpenJDK aliases!
>>
>>
>> On 11/21/17 12:24 PM, coleen.phillimore at oracle.com wrote:
>>>
>>>
>>> On 11/21/17 11:28 AM, Daniel D. Daugherty wrote:
>>>>
>>>> On 11/20/17 3:12 PM, coleen.phillimore at oracle.com wrote:
>>>>
>>>>> I can't find the caller of threads_do_smr.
>>>>
>>>> I'm guessing that's used by the GC code that depends on Thread-SMR...
>>>>
>>>
>>> They? should add this API when the add the use of it then.? I don't 
>>> see it in any sources.
>>
>> We'll see what Erik wants to do...
>
> The new iterators should be more than enough, so that closure-based 
> API is not going to be missed when removed.

I will delete it in the wrap up round.


>
>>
>>>
>>>>
>>>>> If these functions xchg_smr_thread_list, get_smr_java_thread_list, 
>>>>> inc_smr_deleted_thread_count are only used by thread.cpp, I think 
>>>>> they should go in that file and not in the .inline.hpp file to be 
>>>>> included and possibly called by other files.? I think they're 
>>>>> private to class Threads.
>>>>
>>>> I have a vague memory that some of the compilers don't do inlining 
>>>> when
>>>> an "inline" function is in a .cpp. I believe we want these functions
>>>> to be inlined for performance reasons. Erik should probably chime in
>>>> here.
>>>
>>> I don't see why this should be.? Maybe some (older) compilers 
>>> require it to be found before the call though, but that can still be 
>>> accomplished in the .cpp file.
>>
>> Again, we'll see what Erik wants to do...
>
> I don't mind. Either file works for me. For me it's more intuitive if 
> inline member function definitions are in the .inline.hpp files. But 
> if there are strong forces to move this to the cpp file, then sure.

I prefer inline member function definitions in the .inline.hpp files.
(There might even be a style guide note about this...)

Coleen, are you okay if we leave them there?

Dan


>
> Thanks,
> /Erik
>



From coleen.phillimore at oracle.com  Wed Nov 22 13:02:32 2017
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Wed, 22 Nov 2017 08:02:32 -0500
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <6906e0e0-e5cc-a415-6db2-13eff8d76ca8@oracle.com>
References: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
 <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>
 <836b5c6c-5eef-9245-c16a-e71372093d9a@oracle.com>
 <82d0e473-a24f-9feb-82e9-7d04c4d7a91d@oracle.com>
 <354f7ece-66bc-c065-9ee8-f55abcc1c23e@oracle.com>
 <0ff8df8e-e93c-02d2-cb97-b13b1ac0f53f@oracle.com>
 <e56b44ce-28a9-822e-bbac-4b64f5f85c58@oracle.com>
 <70cc1311-de55-5502-159b-cd0a3522cfd6@oracle.com>
 <68b0b2bc-2d26-5442-bafb-5ce21420767b@oracle.com>
 <5e154ff2-65d4-bad4-db5e-88cf4c733c7d@oracle.com>
 <5A1547F0.7040206@oracle.com>
 <6906e0e0-e5cc-a415-6db2-13eff8d76ca8@oracle.com>
Message-ID: <50fdb219-4aa2-bdcc-cf69-8f60c935208f@oracle.com>



On 11/22/17 7:54 AM, Daniel D. Daugherty wrote:
> On 11/22/17 4:48 AM, Erik ?sterlund wrote:
>> Hi,
>>
>> Some replies...
>>
>> On 2017-11-21 18:36, Daniel D. Daugherty wrote:
>>> Thanks for keeping all the OpenJDK aliases!
>>>
>>>
>>> On 11/21/17 12:24 PM, coleen.phillimore at oracle.com wrote:
>>>>
>>>>
>>>> On 11/21/17 11:28 AM, Daniel D. Daugherty wrote:
>>>>>
>>>>> On 11/20/17 3:12 PM, coleen.phillimore at oracle.com wrote:
>>>>>
>>>>>> I can't find the caller of threads_do_smr.
>>>>>
>>>>> I'm guessing that's used by the GC code that depends on Thread-SMR...
>>>>>
>>>>
>>>> They? should add this API when the add the use of it then. I don't 
>>>> see it in any sources.
>>>
>>> We'll see what Erik wants to do...
>>
>> The new iterators should be more than enough, so that closure-based 
>> API is not going to be missed when removed.
>
> I will delete it in the wrap up round.
>
>

Thank you.

>>
>>>
>>>>
>>>>>
>>>>>> If these functions xchg_smr_thread_list, 
>>>>>> get_smr_java_thread_list, inc_smr_deleted_thread_count are only 
>>>>>> used by thread.cpp, I think they should go in that file and not 
>>>>>> in the .inline.hpp file to be included and possibly called by 
>>>>>> other files.? I think they're private to class Threads.
>>>>>
>>>>> I have a vague memory that some of the compilers don't do inlining 
>>>>> when
>>>>> an "inline" function is in a .cpp. I believe we want these functions
>>>>> to be inlined for performance reasons. Erik should probably chime in
>>>>> here.
>>>>
>>>> I don't see why this should be.? Maybe some (older) compilers 
>>>> require it to be found before the call though, but that can still 
>>>> be accomplished in the .cpp file.
>>>
>>> Again, we'll see what Erik wants to do...
>>
>> I don't mind. Either file works for me. For me it's more intuitive if 
>> inline member function definitions are in the .inline.hpp files. But 
>> if there are strong forces to move this to the cpp file, then sure.
>
> I prefer inline member function definitions in the .inline.hpp files.
> (There might even be a style guide note about this...)
>
> Coleen, are you okay if we leave them there?

Yes, that's fine.

Coleen

>
> Dan
>
>
>>
>> Thanks,
>> /Erik
>>
>



From erik.osterlund at oracle.com  Wed Nov 22 13:06:50 2017
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Wed, 22 Nov 2017 14:06:50 +0100
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <ea91e54c-0d69-bc13-f2f6-8bfacb9719d6@oracle.com>
References: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
 <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>
 <836b5c6c-5eef-9245-c16a-e71372093d9a@oracle.com>
 <82d0e473-a24f-9feb-82e9-7d04c4d7a91d@oracle.com>
 <354f7ece-66bc-c065-9ee8-f55abcc1c23e@oracle.com>
 <0ff8df8e-e93c-02d2-cb97-b13b1ac0f53f@oracle.com>
 <e56b44ce-28a9-822e-bbac-4b64f5f85c58@oracle.com>
 <70cc1311-de55-5502-159b-cd0a3522cfd6@oracle.com>
 <5A153E52.8000704@oracle.com>
 <ea91e54c-0d69-bc13-f2f6-8bfacb9719d6@oracle.com>
Message-ID: <5A15766A.1090701@oracle.com>

Hi Dan,

On 2017-11-22 13:53, Daniel D. Daugherty wrote:
> On 11/22/17 4:07 AM, Erik ?sterlund wrote:
>> Hi,
>>
>> Some replies...
>>
>> On 2017-11-21 17:28, Daniel D. Daugherty wrote:
>>> Hi Coleen!
>>>
>>> Thanks for making time to review the Thread-SMR stuff again!!
>>>
>>> I have added back the other three OpenJDK aliases... This review is
>>> being done on _four_ different OpenJDK aliases.
>>>
>>> As always, replies are embedded below...
>>>
>>>
>>> On 11/20/17 3:12 PM, coleen.phillimore at oracle.com wrote:
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/src/hotspot/share/runtime/threadSMR.cpp.html 
>>>>
>>>>
>>>>   32 ThreadsList::ThreadsList(int entries) : _length(entries), 
>>>> _threads(NEW_C_HEAP_ARRAY(JavaThread*, entries + 1, mtGC)), 
>>>> _next_list(NULL) {
>>>>
>>>> Seems like it should be mtThread rather than mtGC.
>>>
>>> Fixed. Definitely an artifact of Erik's original prototype when he
>>> extracted Thread-SMR from his GC work... Thanks for catching it.
>>>
>>
>> Confirmed. At the time I considered the Threads list overheads GC 
>> overheads, but I agree mtThread is a better fit today.
>>
>>>
>>>>
>>>> + return (unsigned int)(((uint32_t)(uintptr_t)s1) * 2654435761u);
>>>>
>>>> Can you add a comment about where this number came from?
>>>
>>> I'll have to get that from Erik...
>>
>> Wow, that looks like code I wrote a *very* long time ago. :) That is 
>> a variation of Knuth's multiplicative hash which is outlined in a 
>> comment in synchronizer.cpp and referred to in that comment as a 
>> phi-based scheme. Basically the magic number is 2^32 * Phi (the 
>> golden ratio), which happens to be a useful value for building a 
>> reasonably simple yet pretty good hash function. It is not the 
>> optimal hash function, but seemed to definitely be good enough for 
>> our purposes.
>
> So a reasonable comment would be:
>
> // The literal value is 2^32 * Phi (the golden ratio).
>

Yes.

> If so, then I'll add it in my wrap up round...

Excellent, thanks Dan.

/Erik

> Dan
>
>>
>> Thanks,
>> /Erik
>



From daniel.daugherty at oracle.com  Wed Nov 22 13:09:16 2017
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Wed, 22 Nov 2017 08:09:16 -0500
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <e45e0c9d-d725-c346-f8b1-0a2c57e7d457@oracle.com>
References: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
 <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>
 <836b5c6c-5eef-9245-c16a-e71372093d9a@oracle.com>
 <82d0e473-a24f-9feb-82e9-7d04c4d7a91d@oracle.com>
 <354f7ece-66bc-c065-9ee8-f55abcc1c23e@oracle.com>
 <0ff8df8e-e93c-02d2-cb97-b13b1ac0f53f@oracle.com>
 <5365606e-4036-b5d7-9bac-e91e42f38d01@oracle.com>
 <4d5d448e-3902-8465-197e-18a5607fbd70@oracle.com>
 <4fd06a59-f6b5-33b2-53f5-7bfc96f866a7@oracle.com>
 <e45e0c9d-d725-c346-f8b1-0a2c57e7d457@oracle.com>
Message-ID: <1e70bbf5-6f52-315d-85ba-fd53a01b66ca@oracle.com>

Adding back the other OpenJDK aliases...

On 11/22/17 8:01 AM, coleen.phillimore at oracle.com wrote:
>
>
> On 11/22/17 7:51 AM, Daniel D. Daugherty wrote:
>> On 11/21/17 11:26 PM, David Holmes wrote:
>>> Hi Dan,
>>>
>>> On 22/11/2017 10:12 AM, Daniel D. Daugherty wrote:
>>>> Greetings,
>>>>
>>>> *** We are wrapping up code review on this project so it is time ***
>>>> *** for the various code reviewers to chime in one last time. ***
>>>>
>>>> In this latest round, we had three different reviewers chime in so 
>>>> we're
>>>> doing delta webrevs for each of those resolutions:
>>>>
>>>> David H's resolutions:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.09,10.10.0-delta/ 
>>>>
>>>>
>>>> ?? mq comment for dholmes_CR:
>>>> ???? - Fix indents, trailing spaces and various typos.
>>>> ???? - Add descriptions for the '_cnt', '_max' and '_times" fields,
>>>> ?????? add impl notes to document the type choices.
>>>
>>> src/hotspot/share/runtime/thread.cpp
>>>
>>> 3466 // range. Unsigned 64-bit would be more future proof, but 
>>> 64-bit atomic inc
>>> 3467 // isn't available everywhere (or is it?).
>>>
>>> 64-bit atomics should be available on all currently supported 
>>> platforms (the last time this was an issue was for PPC32 - and the 
>>> atomic templates should have filled in any previous holes in the 
>>> allowed types).
>>
>> Hmmm... I ran into this when I merged the project with the atomic
>> templates. I got a JPRT build failure on one of the ARM platforms...
>> Unfortunately, I didn't save the log files so I can't quote the
>> error message from the template stuff...
>>
>>
>>> But if it's always 64-bit then you'd have to use Atomic::load to 
>>> allow for 32-bit platforms.
>>
>> What's the official story on 32-bit platforms? Is that what
>> bit me in JPRT? i.e., did we still have a 32-bit ARM build
>> config'ed in JPRT a month or two ago???
>>
>>
>>> These counts etc are all just for statistics so we aren't really 
>>> concerned about eventual overflows in long running VMs - right?
>>
>> Yup these are just statistics... overflow is not a killer.
>>
>>
>>>
>>> ---
>>>
>>> ?????????????????????????????????????? // # of parallel threads in 
>>> _smr_delete_lock->wait().
>>> ? static uint????????????????? _smr_delete_lock_wait_cnt;
>>> ?????????????????????????????????????? // Max # of parallel threads 
>>> in _smr_delete_lock->wait().
>>>
>>>
>>> why are the comments indented like that? I thought this is what 
>>> Coleen previously commented on. Please just start the comments in 
>>> the first column - no need for any indent. Thanks.
>>
>> Coleen asked for the new comments in thread.cpp to be moved
>> over to column 1. She asked for the new comments in thread.hpp
>> to be indented with more spaces. I started with 2 spaces and
>> the variables still didn't stand out. I tried 4 spaces and
>> they still didn't stand out probably because of the leading
>> _smr_... So I went with 8 spaces...
>
> Since you have the same but more indepth comments in the .cpp file, 
> and at least for my sampling the comments in the .hpp file look 
> redundant, I think you should not have the same comments in the .hpp 
> file.?? They're really values that the implementation uses, not part 
> of the interface.? My vote is to remove the comments from the .hpp file.

The first line is intended to be a 1-line summary and it is identical
between thread.cpp and thread.hpp. My thinking was that someone would
look in thread.hpp first to see what the field was for and if they
wanted the gory details about the field they could look at the impl
notes in thread.cpp...

Of course, that creates a maintenance problem in keeping the 1-liners
in sync between thread.hpp and thread.cpp. I'm okay with deleting the
1-liners from thread.hpp if folks think they should go...

Dan


>
> Coleen
>>
>>
>>>
>>>> src/hotspot/share/runtime/globals.hpp change is white-space only so it
>>>> ?? is only visible via the file's patch link.
>>>>
>>>>
>>>> Robin W's resolutions:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.10.0,10.10.1-delta/ 
>>>>
>>>>
>>>> ?? mq comment for robinw_CR:
>>>> ???? - Fix some inefficient code, update some comments, fix some 
>>>> indents,
>>>> ?????? and add some 'const' specifiers.
>>>>
>>>> ?? src/hotspot/share/runtime/vm_operations.hpp change is 
>>>> white-space only so
>>>> ?? it is only visible via the file's patch link.
>>>
>>> All looks fine to me. Some good catches there from Robin!
>>
>> Yes, a new pair of eyes on this code is always useful.
>>
>>
>>>
>>>> Coleen's resolutions:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.10.1,10.10.2-delta/ 
>>>>
>>>>
>>>> ?? mq comment for coleenp_CR:
>>>> ???? - Change ThreadsList::_threads from 'mtGC' -> 'mtThread',
>>>> ???? - add header comment to threadSMR.hpp file,
>>>> ???? - cleanup JavaThreadIteratorWithHandle ctr,
>>>> ???? - make ErrorHandling more efficient.
>>>
>>> src/hotspot/share/runtime/threadSMR.hpp
>>>
>>> + // Thread Safe Memory Reclaimation (Thread-SMR) support.
>>>
>>> Only one 'i' in Reclamation :)
>>
>> Will fix. I tried typing both and neither looked right to me.
>> (I might be getting blurry eyed with this stuff)...
>> So I went with the spelling from Coleen's comment... :-)
>>
>> Thanks for taking another look!!
>>
>> Dan
>>
>>
>>>
>>> Thanks,
>>> David
>>> ------
>>>
>>>>
>>>> Some folks prefer to see all of the delta webrevs together so here is
>>>> a delta webrev relative to jdk10-09-full:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.09,10.10.2-delta/ 
>>>>
>>>>
>>>> ?? src/hotspot/share/runtime/globals.hpp and
>>>> ?? src/hotspot/share/runtime/vm_operations.hpp changes are 
>>>> white-space only
>>>> ?? so they are only visible via the file's patch link.
>>>>
>>>>
>>>> And, of course, some folks prefer to see everything all at once so 
>>>> here
>>>> is the latest full webrev:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.10-full/
>>>>
>>>>
>>>> Dan has submitted the bits for the usual Mach5 tier[1-5] testing. The
>>>> prelim Mach5 tier1 (JPRT equivalent) on these bits passed just fine...
>>>>
>>>> The project is currently baselined on Jesper's 2017-11-17 PIT snapshot
>>>> so there will be some catching up to do before integration...
>>>>
>>>> We welcome comments, suggestions and feedback.
>>>>
>>>> Dan, Erik and Robbin
>>>>
>>>>
>>>> On 11/18/17 8:49 PM, Daniel D. Daugherty wrote:
>>>>> Greetings,
>>>>>
>>>>> Testing of the last round of changes revealed a hang in one of the 
>>>>> new
>>>>> TLH tests. Robbin has fixed the hang, updated the existing TLH 
>>>>> test, and
>>>>> added another TLH test for good measure.
>>>>>
>>>>> Here is the updated full webrev:
>>>>>
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/
>>>>>
>>>>> Here is the updated delta webrev:
>>>>>
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-delta/
>>>>>
>>>>> Dan ran the bits thru the usual Mach5 tier[1-5] testing and there are
>>>>> no unexpected failures.
>>>>>
>>>>> We welcome comments, suggestions and feedback.
>>>>>
>>>>> Dan, Erik and Robbin
>>>>>
>>>>>
>>>>> On 11/15/17 3:32 PM, Daniel D. Daugherty wrote:
>>>>>> Greetings,
>>>>>>
>>>>>> Robbin rebased the project last night/this morning to merge with 
>>>>>> Thread
>>>>>> Local Handshakes (TLH) and also picked up additional changesets 
>>>>>> up thru:
>>>>>>
>>>>>>> Changeset: fa736014cf28
>>>>>>> Author:??? cjplummer
>>>>>>> Date:????? 2017-11-14 18:08 -0800
>>>>>>> URL:http://hg.openjdk.java.net/jdk/hs/rev/fa736014cf28
>>>>>>>
>>>>>>> 8191049: Add alternate version of pns() that is callable from 
>>>>>>> within hotspot source
>>>>>>> Summary: added pns2() to debug.cpp
>>>>>>> Reviewed-by: stuefe, gthornbr
>>>>>>
>>>>>> This is the first time we've rebased the project to bits that are 
>>>>>> this
>>>>>> fresh (< 12 hours old at rebase time). We've done this because we 
>>>>>> think
>>>>>> we're done with this project and are in the final 
>>>>>> review-change-retest
>>>>>> cycle(s)... In other words, we're not planning on making any more 
>>>>>> major
>>>>>> changes for this project... :-)
>>>>>>
>>>>>> *** Time for code reviewers to chime in on this thread! ***
>>>>>>
>>>>>> Here is the updated full webrev:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-full/
>>>>>>
>>>>>> Here's is the delta webrev from the 2017.11.10 rebase:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-delta/
>>>>>>
>>>>>> Dan has submitted the bits for the usual Mach5 tier[1-5] testing
>>>>>> (and the new baseline also)...
>>>>>>
>>>>>> We're expecting this round to be a little noisier than usual because
>>>>>> we did not rebase on a PIT snapshot so the new baseline has not been
>>>>>> through Jesper's usual care-and-feeding of integration_blockers, 
>>>>>> etc.
>>>>>>
>>>>>> We welcome comments, suggestions and feedback.
>>>>>>
>>>>>> Dan, Erik and Robbin
>>>>>>
>>>>>>
>>>>>> On 11/14/17 4:48 PM, Daniel D. Daugherty wrote:
>>>>>>> Greetings,
>>>>>>>
>>>>>>> I rebased the project to the 2017.11.10 jdk/hs PIT snapshot.
>>>>>>> (Yes, we're playing chase-the-repo...)
>>>>>>>
>>>>>>> Here is the updated full webrev:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-07-full/
>>>>>>>
>>>>>>> Unlike the previous rebase, there were no changes required to the
>>>>>>> open code to get the rebased bits to build so there is no delta
>>>>>>> webrev.
>>>>>>>
>>>>>>> These bits have been run through JPRT and I've submitted the usual
>>>>>>> Mach5 tier[1-5] test run...
>>>>>>>
>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>
>>>>>>> Dan, Erik and Robbin
>>>>>>>
>>>>>>>
>>>>>>> On 11/13/17 12:30 PM, Daniel D. Daugherty wrote:
>>>>>>>> Greetings,
>>>>>>>>
>>>>>>>> I rebased the project to the 2017.10.26 jdk10/hs PIT snapshot.
>>>>>>>>
>>>>>>>> Here are the updated webrevs:
>>>>>>>>
>>>>>>>> Here's the mq comment for the change:
>>>>>>>>
>>>>>>>> ? Rebase to 2017.10.25 PIT snapshot.
>>>>>>>>
>>>>>>>> Here is the full webrev:
>>>>>>>>
>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-full/
>>>>>>>>
>>>>>>>> And here is the delta webrev:
>>>>>>>>
>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-delta/
>>>>>>>>
>>>>>>>> I ran the above bits throught Mach5 tier[1-5] testing over the 
>>>>>>>> holiday
>>>>>>>> weekend. Didn't see any issues in a quick look. Going to take a 
>>>>>>>> closer
>>>>>>>> look today.
>>>>>>>>
>>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>>
>>>>>>>> Dan, Erik and Robbin
>>>>>>>>
>>>>>>>>
>>>>>>>> On 11/8/17 1:05 PM, Daniel D. Daugherty wrote:
>>>>>>>>> Greetings,
>>>>>>>>>
>>>>>>>>> Resolving one of the code review comments (from both Stefan K 
>>>>>>>>> and Coleen)
>>>>>>>>> on jdk10-04-full required quite a few changes so it is being 
>>>>>>>>> done as a
>>>>>>>>> standalone patch and corresponding webrevs:
>>>>>>>>>
>>>>>>>>> Here's the mq comment for the change:
>>>>>>>>>
>>>>>>>>> ? stefank, coleenp CR - refactor most JavaThreadIterator usage 
>>>>>>>>> to use
>>>>>>>>> ??? JavaThreadIteratorWithHandle.
>>>>>>>>>
>>>>>>>>> Here is the full webrev:
>>>>>>>>>
>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-full/
>>>>>>>>>
>>>>>>>>> And here is the delta webrev:
>>>>>>>>>
>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-delta/
>>>>>>>>>
>>>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>>>
>>>>>>>>> Dan, Erik and Robbin
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 10/9/17 3:41 PM, Daniel D. Daugherty wrote:
>>>>>>>>>> Greetings,
>>>>>>>>>>
>>>>>>>>>> We have a (eXtra Large) fix for the following bug:
>>>>>>>>>>
>>>>>>>>>> 8167108 inconsistent handling of SR_lock can lead to crashes
>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8167108
>>>>>>>>>>
>>>>>>>>>> This fix adds a Safe Memory Reclamation (SMR) mechanism based on
>>>>>>>>>> Hazard Pointers to manage JavaThread lifecycle.
>>>>>>>>>>
>>>>>>>>>> Here's a PDF for the internal wiki that we've been using to 
>>>>>>>>>> describe
>>>>>>>>>> and track the work on this project:
>>>>>>>>>>
>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/SMR_and_JavaThread_Lifecycle-JDK10-04.pdf 
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Dan has noticed that the indenting is wrong in some of the 
>>>>>>>>>> code quotes
>>>>>>>>>> in the PDF that are not present in the internal wiki. We 
>>>>>>>>>> don't have a
>>>>>>>>>> solution for that problem yet.
>>>>>>>>>>
>>>>>>>>>> Here's the webrev for current JDK10 version of this fix:
>>>>>>>>>>
>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-04-full
>>>>>>>>>>
>>>>>>>>>> This fix has been run through many rounds of JPRT and Mach5 
>>>>>>>>>> tier[2-5]
>>>>>>>>>> testing, additional stress testing on Dan's Solaris X64 
>>>>>>>>>> server, and
>>>>>>>>>> additional testing on Erik and Robbin's machines.
>>>>>>>>>>
>>>>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>>>>
>>>>>>>>>> Daniel Daugherty
>>>>>>>>>> Erik Osterlund
>>>>>>>>>> Robbin Ehn
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>
>



From daniel.daugherty at oracle.com  Wed Nov 22 13:10:34 2017
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Wed, 22 Nov 2017 08:10:34 -0500
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <50fdb219-4aa2-bdcc-cf69-8f60c935208f@oracle.com>
References: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
 <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>
 <836b5c6c-5eef-9245-c16a-e71372093d9a@oracle.com>
 <82d0e473-a24f-9feb-82e9-7d04c4d7a91d@oracle.com>
 <354f7ece-66bc-c065-9ee8-f55abcc1c23e@oracle.com>
 <0ff8df8e-e93c-02d2-cb97-b13b1ac0f53f@oracle.com>
 <e56b44ce-28a9-822e-bbac-4b64f5f85c58@oracle.com>
 <70cc1311-de55-5502-159b-cd0a3522cfd6@oracle.com>
 <68b0b2bc-2d26-5442-bafb-5ce21420767b@oracle.com>
 <5e154ff2-65d4-bad4-db5e-88cf4c733c7d@oracle.com>
 <5A1547F0.7040206@oracle.com>
 <6906e0e0-e5cc-a415-6db2-13eff8d76ca8@oracle.com>
 <50fdb219-4aa2-bdcc-cf69-8f60c935208f@oracle.com>
Message-ID: <6fa0d4d9-4acd-99eb-9769-613735561e93@oracle.com>

On 11/22/17 8:02 AM, coleen.phillimore at oracle.com wrote:
>
>
> On 11/22/17 7:54 AM, Daniel D. Daugherty wrote:
>> On 11/22/17 4:48 AM, Erik ?sterlund wrote:
>>> Hi,
>>>
>>> Some replies...
>>>
>>> On 2017-11-21 18:36, Daniel D. Daugherty wrote:
>>>> Thanks for keeping all the OpenJDK aliases!
>>>>
>>>>
>>>> On 11/21/17 12:24 PM, coleen.phillimore at oracle.com wrote:
>>>>>
>>>>>
>>>>> On 11/21/17 11:28 AM, Daniel D. Daugherty wrote:
>>>>>>
>>>>>> On 11/20/17 3:12 PM, coleen.phillimore at oracle.com wrote:
>>>>>>
>>>>>>> I can't find the caller of threads_do_smr.
>>>>>>
>>>>>> I'm guessing that's used by the GC code that depends on 
>>>>>> Thread-SMR...
>>>>>>
>>>>>
>>>>> They? should add this API when the add the use of it then. I don't 
>>>>> see it in any sources.
>>>>
>>>> We'll see what Erik wants to do...
>>>
>>> The new iterators should be more than enough, so that closure-based 
>>> API is not going to be missed when removed.
>>
>> I will delete it in the wrap up round.
>>
>>
>
> Thank you.
>
>>>
>>>>
>>>>>
>>>>>>
>>>>>>> If these functions xchg_smr_thread_list, 
>>>>>>> get_smr_java_thread_list, inc_smr_deleted_thread_count are only 
>>>>>>> used by thread.cpp, I think they should go in that file and not 
>>>>>>> in the .inline.hpp file to be included and possibly called by 
>>>>>>> other files.? I think they're private to class Threads.
>>>>>>
>>>>>> I have a vague memory that some of the compilers don't do 
>>>>>> inlining when
>>>>>> an "inline" function is in a .cpp. I believe we want these functions
>>>>>> to be inlined for performance reasons. Erik should probably chime in
>>>>>> here.
>>>>>
>>>>> I don't see why this should be.? Maybe some (older) compilers 
>>>>> require it to be found before the call though, but that can still 
>>>>> be accomplished in the .cpp file.
>>>>
>>>> Again, we'll see what Erik wants to do...
>>>
>>> I don't mind. Either file works for me. For me it's more intuitive 
>>> if inline member function definitions are in the .inline.hpp files. 
>>> But if there are strong forces to move this to the cpp file, then sure.
>>
>> I prefer inline member function definitions in the .inline.hpp files.
>> (There might even be a style guide note about this...)
>>
>> Coleen, are you okay if we leave them there?
>
> Yes, that's fine.

Thanks for confirmation!

Dan


>
> Coleen
>
>>
>> Dan
>>
>>
>>>
>>> Thanks,
>>> /Erik
>>>
>>
>



From daniel.daugherty at oracle.com  Wed Nov 22 13:10:58 2017
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Wed, 22 Nov 2017 08:10:58 -0500
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <5A15766A.1090701@oracle.com>
References: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
 <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>
 <836b5c6c-5eef-9245-c16a-e71372093d9a@oracle.com>
 <82d0e473-a24f-9feb-82e9-7d04c4d7a91d@oracle.com>
 <354f7ece-66bc-c065-9ee8-f55abcc1c23e@oracle.com>
 <0ff8df8e-e93c-02d2-cb97-b13b1ac0f53f@oracle.com>
 <e56b44ce-28a9-822e-bbac-4b64f5f85c58@oracle.com>
 <70cc1311-de55-5502-159b-cd0a3522cfd6@oracle.com>
 <5A153E52.8000704@oracle.com>
 <ea91e54c-0d69-bc13-f2f6-8bfacb9719d6@oracle.com>
 <5A15766A.1090701@oracle.com>
Message-ID: <40b27a38-d7f2-5a3b-0022-92d3678e5971@oracle.com>

On 11/22/17 8:06 AM, Erik ?sterlund wrote:
> Hi Dan,
>
> On 2017-11-22 13:53, Daniel D. Daugherty wrote:
>> On 11/22/17 4:07 AM, Erik ?sterlund wrote:
>>> Hi,
>>>
>>> Some replies...
>>>
>>> On 2017-11-21 17:28, Daniel D. Daugherty wrote:
>>>> Hi Coleen!
>>>>
>>>> Thanks for making time to review the Thread-SMR stuff again!!
>>>>
>>>> I have added back the other three OpenJDK aliases... This review is
>>>> being done on _four_ different OpenJDK aliases.
>>>>
>>>> As always, replies are embedded below...
>>>>
>>>>
>>>> On 11/20/17 3:12 PM, coleen.phillimore at oracle.com wrote:
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/src/hotspot/share/runtime/threadSMR.cpp.html 
>>>>>
>>>>>
>>>>> ? 32 ThreadsList::ThreadsList(int entries) : _length(entries), 
>>>>> _threads(NEW_C_HEAP_ARRAY(JavaThread*, entries + 1, mtGC)), 
>>>>> _next_list(NULL) {
>>>>>
>>>>> Seems like it should be mtThread rather than mtGC.
>>>>
>>>> Fixed. Definitely an artifact of Erik's original prototype when he
>>>> extracted Thread-SMR from his GC work... Thanks for catching it.
>>>>
>>>
>>> Confirmed. At the time I considered the Threads list overheads GC 
>>> overheads, but I agree mtThread is a better fit today.
>>>
>>>>
>>>>>
>>>>> + return (unsigned int)(((uint32_t)(uintptr_t)s1) * 2654435761u);
>>>>>
>>>>> Can you add a comment about where this number came from?
>>>>
>>>> I'll have to get that from Erik...
>>>
>>> Wow, that looks like code I wrote a *very* long time ago. :) That is 
>>> a variation of Knuth's multiplicative hash which is outlined in a 
>>> comment in synchronizer.cpp and referred to in that comment as a 
>>> phi-based scheme. Basically the magic number is 2^32 * Phi (the 
>>> golden ratio), which happens to be a useful value for building a 
>>> reasonably simple yet pretty good hash function. It is not the 
>>> optimal hash function, but seemed to definitely be good enough for 
>>> our purposes.
>>
>> So a reasonable comment would be:
>>
>> // The literal value is 2^32 * Phi (the golden ratio).
>>
>
> Yes.
>
>> If so, then I'll add it in my wrap up round...
>
> Excellent, thanks Dan.


Thanks for confirmation!

Dan


>
> /Erik
>
>> Dan
>>
>>>
>>> Thanks,
>>> /Erik
>>
>



From thomas.schatzl at oracle.com  Wed Nov 22 13:18:15 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 22 Nov 2017 14:18:15 +0100
Subject: RFR (XS): 8182050: assert(_whole_heap.contains(p)) failed: Attempt
 to access p out of bounds of card marking array's _whole_heap
Message-ID: <1511356695.3392.27.camel@oracle.com>

Hi all,

  can I have reviews for this tiny change that fixes tripping an assert
when executing the post barrier for arrays for G1?

The issue is that when calling arraycopy to a zero sized array of j.l.O
a call to the post barrier is emitted. If that object at that point has
 been allocated so that its non-existent value array is located at the
end of the heap, the MemRegion passed to
G1SATBCardTableLoggingModRefBS::invalidate() has a start address just
beyond the heap the assert trips over (it has a zero-sized length).

Note that the G1 invalidation code is correct, i.e. handled the
situation correctly, just the assert is wrong in this case.

The suggested fix is to ignore zero-sized MemRegions like the other
card table implementation does. This will keep triggering the assert
for really wrong MemRegions (which is desired imho).

Thanks go to Alex who investigated and fixed a very similar issue with
the other collectors in JDK-8185591.

CR:
https://bugs.openjdk.java.net/browse/JDK-8182050
Webrev:
http://cr.openjdk.java.net/~tschatzl/8182050/webrev/
Testing:
Included jtreg test case, checking that the issue does not occur with
the change anymore.

Thanks,
  Thomas


From stefan.johansson at oracle.com  Wed Nov 22 14:55:51 2017
From: stefan.johansson at oracle.com (Stefan Johansson)
Date: Wed, 22 Nov 2017 15:55:51 +0100
Subject: Need sponsor to push attached 8187819 into
 http://hg.openjdk.java.net/jdk10/hs
In-Reply-To: <7c02edb4-1a18-e750-3c6b-28e303fb2b82@oracle.com>
References: <7c02edb4-1a18-e750-3c6b-28e303fb2b82@oracle.com>
Message-ID: <28ae08ed-7776-4565-d084-a7f466be60d0@oracle.com>

Thanks for fixing this Alex,

I will sponsor and push this change.

Cheers,
Stefan

On 2017-11-16 20:46, Alexander Harlap wrote:
> I need a sponsor to push attached8187819.patch - 
> gc/TestFullGCALot.java fails on jdk10 started with 
> "-XX:-UseCompressedOops" option
>
> Patch should go into jdk10/hs
>
> Reviewed by Thomas Schatzl? and Stefan Johansson.
>
> Thank you,
>
> Alex
>



From erik.helin at oracle.com  Wed Nov 22 15:14:05 2017
From: erik.helin at oracle.com (Erik Helin)
Date: Wed, 22 Nov 2017 16:14:05 +0100
Subject: RFR (XS): 8182050: assert(_whole_heap.contains(p)) failed:
 Attempt to access p out of bounds of card marking array's _whole_heap
In-Reply-To: <1511356695.3392.27.camel@oracle.com>
References: <1511356695.3392.27.camel@oracle.com>
Message-ID: <ca389e5d-ab3b-be2e-b217-85e8afe14115@oracle.com>

On 11/22/2017 02:18 PM, Thomas Schatzl wrote:
> Hi all,
> 
>    can I have reviews for this tiny change that fixes tripping an assert
> when executing the post barrier for arrays for G1?
> 
> The issue is that when calling arraycopy to a zero sized array of j.l.O
> a call to the post barrier is emitted. If that object at that point has
>   been allocated so that its non-existent value array is located at the
> end of the heap, the MemRegion passed to
> G1SATBCardTableLoggingModRefBS::invalidate() has a start address just
> beyond the heap the assert trips over (it has a zero-sized length).
> 
> Note that the G1 invalidation code is correct, i.e. handled the
> situation correctly, just the assert is wrong in this case.
> 
> The suggested fix is to ignore zero-sized MemRegions like the other
> card table implementation does. This will keep triggering the assert
> for really wrong MemRegions (which is desired imho).
> 
> Thanks go to Alex who investigated and fixed a very similar issue with
> the other collectors in JDK-8185591.
> 
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8182050
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8182050/webrev/

Looks good, Reviewed. Thanks Thomas for taking care of this (and Alex 
for finding the cause for JDK-8185591).

Erik

> Testing:
> Included jtreg test case, checking that the issue does not occur with
> the change anymore.
> 
> Thanks,
>    Thomas
> 


From thomas.schatzl at oracle.com  Wed Nov 22 15:24:52 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 22 Nov 2017 16:24:52 +0100
Subject: RFR (XS): 8182050: assert(_whole_heap.contains(p)) failed:
 Attempt to access p out of bounds of card marking array's _whole_heap
In-Reply-To: <ca389e5d-ab3b-be2e-b217-85e8afe14115@oracle.com>
References: <1511356695.3392.27.camel@oracle.com>
 <ca389e5d-ab3b-be2e-b217-85e8afe14115@oracle.com>
Message-ID: <1511364292.2374.1.camel@oracle.com>

Hi Erik,

On Wed, 2017-11-22 at 16:14 +0100, Erik Helin wrote:
> On 11/22/2017 02:18 PM, Thomas Schatzl wrote:
> > Hi all,
> > 
> >    can I have reviews for this tiny change that fixes tripping an
> > assert when executing the post barrier for arrays for G1?
> > [...]
> > CR:
> > https://bugs.openjdk.java.net/browse/JDK-8182050
> > Webrev:
> > http://cr.openjdk.java.net/~tschatzl/8182050/webrev/
> 
> Looks good, Reviewed. Thanks Thomas for taking care of this (and
> Alex for finding the cause for JDK-8185591).

Thanks for your review,
  Thomas



From thomas.schatzl at oracle.com  Wed Nov 22 15:25:47 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 22 Nov 2017 16:25:47 +0100
Subject: RFR (M): 8190426: Lazily initialize refinement threads with
 UseDynamicNumberOfGCThreads
In-Reply-To: <d3ebd10a-53cb-2879-46ce-c03e5e8eeab2@oracle.com>
References: <1509726495.2630.2.camel@oracle.com>
 <1f3e5ef0-60f3-770f-f581-8c7764e44f2e@oracle.com>
 <1511177189.2337.39.camel@oracle.com> <1511201854.2229.4.camel@oracle.com>
 <d3ebd10a-53cb-2879-46ce-c03e5e8eeab2@oracle.com>
Message-ID: <1511364347.2374.2.camel@oracle.com>

Hi Sangheon,

On Tue, 2017-11-21 at 11:56 -0800, sangheon.kim wrote:
> Hi Thomas,
> 
> On 11/20/2017 10:17 AM, Thomas Schatzl wrote:
> > Hi all,
> > 
> >    Stefan Johansson found another issue: in the original code,
> > beforeactivating a thread it checked whether that thread had
> > already been activated. The changes did not do that.
> > 
> > Here's a new webrev:
> > 
> > http://cr.openjdk.java.net/~tschatzl/8190426/webrev.1_to_2//  (diff
> > )
> > http://cr.openjdk.java.net/~tschatzl/8190426/webrev.2//  (full)
> 
> webrev.2 looks good to me.
> 
> And thank you for addressing my comments on webrev.1.
> I agree we can improve lazily allocating threads with JDK-8191082:
> Unify handling of thread sets when allocating them lazily.
> 

Thanks for your review,
  Thomas



From stefan.johansson at oracle.com  Wed Nov 22 15:39:08 2017
From: stefan.johansson at oracle.com (Stefan Johansson)
Date: Wed, 22 Nov 2017 16:39:08 +0100
Subject: RFR: 8189733: Cleanup Full GC setup and tear down
In-Reply-To: <1510821109.5332.1.camel@oracle.com>
References: <3b9604e1-9d2e-c0d9-7b2e-e5d9232c6888@oracle.com>
 <1510821109.5332.1.camel@oracle.com>
Message-ID: <f385d617-ca62-3684-37f6-b10b92bb49aa@oracle.com>

Thanks Thomas for the review,

I got some comments from Erik and have updated the review accordingly. 
Here are the new webrevs:
Full: http://cr.openjdk.java.net/~sjohanss/8189733/01/
Inc: http://cr.openjdk.java.net/~sjohanss/8189733/00-01/

This change removes the heap() function as it's only used by member 
functions which can use _heap. Also changed the initialization list to 
use the parameters rather than the members initialized.

Thanks,
Stefan

On 2017-11-16 09:31, Thomas Schatzl wrote:
> Hi,
>
> On Wed, 2017-11-15 at 17:24 +0100, Stefan Johansson wrote:
>> Hi,
>>
>> Please review this enhancement:
>> https://bugs.openjdk.java.net/browse/JDK-8189733
>>
>> Webrev:
>> http://cr.openjdk.java.net/~sjohanss//8189733/00/index.html
>>
>> Summary:
>> After the G1FullCollector has been introduced as part of JEP 307 it
>> makes sense to move more of the setup and tear down for the Full GC
>> into
>> this class. The G1FullGCScope is also moved into the G1FullCollector
>> since it is now only used here.
>    looks good.
>
> Thomas
>



From thomas.schatzl at oracle.com  Wed Nov 22 15:41:18 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 22 Nov 2017 16:41:18 +0100
Subject: RFR: 8189733: Cleanup Full GC setup and tear down
In-Reply-To: <f385d617-ca62-3684-37f6-b10b92bb49aa@oracle.com>
References: <3b9604e1-9d2e-c0d9-7b2e-e5d9232c6888@oracle.com>
 <1510821109.5332.1.camel@oracle.com>
 <f385d617-ca62-3684-37f6-b10b92bb49aa@oracle.com>
Message-ID: <1511365278.2374.3.camel@oracle.com>

Hi,

On Wed, 2017-11-22 at 16:39 +0100, Stefan Johansson wrote:
> Thanks Thomas for the review,
> 
> I got some comments from Erik and have updated the review
> accordingly. 
> Here are the new webrevs:
> Full: http://cr.openjdk.java.net/~sjohanss/8189733/01/
> Inc: http://cr.openjdk.java.net/~sjohanss/8189733/00-01/
> 
> This change removes the heap() function as it's only used by member 
> functions which can use _heap. Also changed the initialization list
> to use the parameters rather than the members initialized.
> 

  still good.

Thomas



From stefan.johansson at oracle.com  Wed Nov 22 15:44:22 2017
From: stefan.johansson at oracle.com (Stefan Johansson)
Date: Wed, 22 Nov 2017 16:44:22 +0100
Subject: RFR (XS): 8182050: assert(_whole_heap.contains(p)) failed:
 Attempt to access p out of bounds of card marking array's _whole_heap
In-Reply-To: <ca389e5d-ab3b-be2e-b217-85e8afe14115@oracle.com>
References: <1511356695.3392.27.camel@oracle.com>
 <ca389e5d-ab3b-be2e-b217-85e8afe14115@oracle.com>
Message-ID: <1ccc5212-0736-df5d-a79a-21a609af6403@oracle.com>



On 2017-11-22 16:14, Erik Helin wrote:
> On 11/22/2017 02:18 PM, Thomas Schatzl wrote:
>> Hi all,
>>
>> ?? can I have reviews for this tiny change that fixes tripping an assert
>> when executing the post barrier for arrays for G1?
>>
>> The issue is that when calling arraycopy to a zero sized array of j.l.O
>> a call to the post barrier is emitted. If that object at that point has
>> ? been allocated so that its non-existent value array is located at the
>> end of the heap, the MemRegion passed to
>> G1SATBCardTableLoggingModRefBS::invalidate() has a start address just
>> beyond the heap the assert trips over (it has a zero-sized length).
>>
>> Note that the G1 invalidation code is correct, i.e. handled the
>> situation correctly, just the assert is wrong in this case.
>>
>> The suggested fix is to ignore zero-sized MemRegions like the other
>> card table implementation does. This will keep triggering the assert
>> for really wrong MemRegions (which is desired imho).
>>
>> Thanks go to Alex who investigated and fixed a very similar issue with
>> the other collectors in JDK-8185591.
>>
>> CR:
>> https://bugs.openjdk.java.net/browse/JDK-8182050
>> Webrev:
>> http://cr.openjdk.java.net/~tschatzl/8182050/webrev/
>
> Looks good, Reviewed. Thanks Thomas for taking care of this (and Alex 
> for finding the cause for JDK-8185591).
I agree, great work guys! Change looks good,

Stefan
>
> Erik
>
>> Testing:
>> Included jtreg test case, checking that the issue does not occur with
>> the change anymore.
>>
>> Thanks,
>> ?? Thomas
>>



From erik.helin at oracle.com  Wed Nov 22 15:49:02 2017
From: erik.helin at oracle.com (Erik Helin)
Date: Wed, 22 Nov 2017 16:49:02 +0100
Subject: RFR (XXS): 8179244: Assert failed in
 instanceMirrorKlass.inline.hpp
In-Reply-To: <1511178963.2337.46.camel@oracle.com>
References: <1511178963.2337.46.camel@oracle.com>
Message-ID: <8dcf0048-1c3a-5a3c-bc2b-5230efd80972@oracle.com>

On 11/20/2017 12:56 PM, Thomas Schatzl wrote:
> Hi all,
> 
>    can I have reviews for this small comment update that clarifies that
> the assert in that location is too strong and should be commented out.
> 
> The issue is that an assert fails during CMS precleaning if class
> loading occurred where the mirror ended up in the old generation (in
> some situations CMS allocates directly into old gen). This assert found
> that a klass that has had a NULL value suddenly changed its value to
> something else.
> 
> This is normal behavior in the situation described above.
> 
> Also checked that we do not miss iterating over a CLD for such oops -
> there is actually explicit support in class loading for this, where
> such CLDs (and their mirrors) are kept alive until remark and iterated
> over again there. (Look for "CMS support" in classLoaderData.?pp).
> 
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8179244
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8179244/webrev/

Looks good, Reviewed, I think the comment is on the right level.

Although the comment and the bug (and understanding how this all hangs 
together) is far from trivial, I think this actual patch is trivial (it 
just adds a comment). Therefore, in my opinion, this patch only requires 
one (R)eviewer.

Thanks,
Erik

> Testing:
> local compilation (just a comment added)
> 
> Thanks,
>    Thomas
> 


From thomas.schatzl at oracle.com  Wed Nov 22 16:32:54 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 22 Nov 2017 17:32:54 +0100
Subject: RFR (XXS): 8179244: Assert failed in
 instanceMirrorKlass.inline.hpp
In-Reply-To: <8dcf0048-1c3a-5a3c-bc2b-5230efd80972@oracle.com>
References: <1511178963.2337.46.camel@oracle.com>
 <8dcf0048-1c3a-5a3c-bc2b-5230efd80972@oracle.com>
Message-ID: <1511368374.2247.1.camel@oracle.com>

Hi Erik,

On Wed, 2017-11-22 at 16:49 +0100, Erik Helin wrote:
> On 11/20/2017 12:56 PM, Thomas Schatzl wrote:
> > Hi all,
> > 
> >    can I have reviews for this small comment update that clarifies
> > that the assert in that location is too strong and should be
> > commented out.
> > 
> > The issue is that an assert fails during CMS precleaning if class
> > loading occurred where the mirror ended up in the old generation
> > (in some situations CMS allocates directly into old gen). This
> > assert found that a klass that has had a NULL value suddenly
> > changed its value to something else.
> > 
> > This is normal behavior in the situation described above.
> > 
> > Also checked that we do not miss iterating over a CLD for such oops
> > - there is actually explicit support in class loading for this,
> > where such CLDs (and their mirrors) are kept alive until remark and
> > iterated over again there. (Look for "CMS support" in
> > classLoaderData.?pp).
> > 
> > CR:
> > https://bugs.openjdk.java.net/browse/JDK-8179244
> > Webrev:
> > http://cr.openjdk.java.net/~tschatzl/8179244/webrev/
> 
> Looks good, Reviewed, I think the comment is on the right level.

Thanks.

> Although the comment and the bug (and understanding how this all
> hangs together) is far from trivial, I think this actual patch is
> trivial (it just adds a comment). Therefore, in my opinion, this
> patch only requires one (R)eviewer.

Fine with me. I will push it tomorrow together with the others that are
finished by then.

Thanks,
  Thomas



From thomas.schatzl at oracle.com  Wed Nov 22 16:33:47 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 22 Nov 2017 17:33:47 +0100
Subject: RFR (XS): 8182050: assert(_whole_heap.contains(p)) failed:
 Attempt to access p out of bounds of card marking array's _whole_heap
In-Reply-To: <1ccc5212-0736-df5d-a79a-21a609af6403@oracle.com>
References: <1511356695.3392.27.camel@oracle.com>
 <ca389e5d-ab3b-be2e-b217-85e8afe14115@oracle.com>
 <1ccc5212-0736-df5d-a79a-21a609af6403@oracle.com>
Message-ID: <1511368427.2247.2.camel@oracle.com>

Hi,

On Wed, 2017-11-22 at 16:44 +0100, Stefan Johansson wrote:
> 
> On 2017-11-22 16:14, Erik Helin wrote:
> > On 11/22/2017 02:18 PM, Thomas Schatzl wrote:
> > > Hi all,
> > > 
> > >    can I have reviews for this tiny change that fixes tripping an
> > > assert when executing the post barrier for arrays for G1?
> > > 
[...]
> > > CR:
> > > https://bugs.openjdk.java.net/browse/JDK-8182050
> > > Webrev:
> > > http://cr.openjdk.java.net/~tschatzl/8182050/webrev/
> > 
> > Looks good, Reviewed. Thanks Thomas for taking care of this (and
> > Alex 
> > for finding the cause for JDK-8185591).
> 
> I agree, great work guys! Change looks good,

Thanks for your review,
  Thomas



From stefan.johansson at oracle.com  Wed Nov 22 16:34:58 2017
From: stefan.johansson at oracle.com (Stefan Johansson)
Date: Wed, 22 Nov 2017 17:34:58 +0100
Subject: RFR (M): 8190426: Lazily initialize refinement threads with
 UseDynamicNumberOfGCThreads
In-Reply-To: <d3ebd10a-53cb-2879-46ce-c03e5e8eeab2@oracle.com>
References: <1509726495.2630.2.camel@oracle.com>
 <1f3e5ef0-60f3-770f-f581-8c7764e44f2e@oracle.com>
 <1511177189.2337.39.camel@oracle.com> <1511201854.2229.4.camel@oracle.com>
 <d3ebd10a-53cb-2879-46ce-c03e5e8eeab2@oracle.com>
Message-ID: <add6cbe7-be3b-d926-a8b8-aca14668965b@oracle.com>



On 2017-11-21 20:56, sangheon.kim wrote:
> Hi Thomas,
>
> On 11/20/2017 10:17 AM, Thomas Schatzl wrote:
>> Hi all,
>>
>> ?? Stefan Johansson found another issue: in the original code, before
>> activating a thread it checked whether that thread had already been
>> activated. The changes did not do that.
>>
>> Here's a new webrev:
>>
>> http://cr.openjdk.java.net/~tschatzl/8190426/webrev.1_to_2/ (diff)
>> http://cr.openjdk.java.net/~tschatzl/8190426/webrev.2/? (full)
> webrev.2 looks good to me.
>
> And thank you for addressing my comments on webrev.1.
> I agree we can improve lazily allocating threads with JDK-8191082: 
> Unify handling of thread sets when allocating them lazily.
>
Looks good Thomas,
Stefan
> Thanks,
> Sangheon
>
>
>>
>> Testing:
>> hs tier1+2
>>
>> Thanks,
>> ?? Thomas
>>
>>
>> On Mon, 2017-11-20 at 12:26 +0100, Thomas Schatzl wrote:
>>> Hi Sangheon,
>>>
>>> ?? thanks for your review, and sorry for the late answer - I have been
>>> travelling lately...
>>>
>>> On Thu, 2017-11-09 at 15:00 -0800, sangheon.kim wrote:
>>>> Hi Thomas,
>>>>
>>>> On 11/03/2017 09:28 AM, Thomas Schatzl wrote:
>>>>> Hi,
>>>>>
>>>>> ??? can I have reviews for this change that makes refinement
>>>>> threads
>>>>> behave the same as the other thread groups we have in G1?
>>>>>
>>>>> I.e. with UseDynamicNumberOfGCThreads enabled (which is off by
>>>>> default) they are created lazily.
>>>>>
>>>>> This helps a little bit further with startup/footprint benchmarks
>>>>> (if
>>>>> enabled).
>>>>>
>>>>> CR:
>>>>> https://bugs.openjdk.java.net/browse/JDK-8190426
>>>>> Webrev:
>>>>> http://cr.openjdk.java.net/~tschatzl/8190426/webrev/
>>>>> Testing:
>>>>> hs-tier1+2
>>>> I like this idea, thank you for improving this.
>>>>
>>>> I have some comments.
>>> I think I addressed all your concerns. Some of them were due to me
>>> being confused about which memory allocations cause VM exit, and
>>> which
>>> don't. I hope I got that right now.
>>>
>>> I also changed the behavior of the thread startup to be more similar
>>> to
>>> the worker threads, namely:
>>> - always start one refinement thread at startup
>>> - support InjectGCWorkerCreationFailure (which is tested by our tests
>>> btw)
>>> - make error messages and failure modes more similar to the ones for
>>> the work gangs.
>>>
>>> Even with that we want to look at JDK-8191082 for better specifying
>>> the
>>> expected behavior.
>>>
>>> Webrevs:
>>> http://cr.openjdk.java.net/~tschatzl/8190426/webrev.0_to_1/ (diff)
>>> http://cr.openjdk.java.net/~tschatzl/8190426/webrev.1/? (full)
>>> Testing:
>>> hs-tier1+2
>>>
>>> Thanks,
>>> ?? Thomas
>>>
>



From thomas.schatzl at oracle.com  Wed Nov 22 16:43:26 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 22 Nov 2017 17:43:26 +0100
Subject: RFR (M): 8190426: Lazily initialize refinement threads with
 UseDynamicNumberOfGCThreads
In-Reply-To: <add6cbe7-be3b-d926-a8b8-aca14668965b@oracle.com>
References: <1509726495.2630.2.camel@oracle.com>
 <1f3e5ef0-60f3-770f-f581-8c7764e44f2e@oracle.com>
 <1511177189.2337.39.camel@oracle.com> <1511201854.2229.4.camel@oracle.com>
 <d3ebd10a-53cb-2879-46ce-c03e5e8eeab2@oracle.com>
 <add6cbe7-be3b-d926-a8b8-aca14668965b@oracle.com>
Message-ID: <1511369006.2247.3.camel@oracle.com>

Hi Stefan,

On Wed, 2017-11-22 at 17:34 +0100, Stefan Johansson wrote:
> 
> On 2017-11-21 20:56, sangheon.kim wrote:
> > Hi Thomas,
> > 
> > On 11/20/2017 10:17 AM, Thomas Schatzl wrote:
> > > Hi all,
> > > 
> > >    Stefan Johansson found another issue: in the original code,
> > > before activating a thread it checked whether that thread had
> > > already been activated. The changes did not do that.
> > > [...]
> > 
> > webrev.2 looks good to me.
> > 
> > And thank you for addressing my comments on webrev.1.
> > I agree we can improve lazily allocating threads with JDK-8191082: 
> > Unify handling of thread sets when allocating them lazily.
> > 
> 
> Looks good Thomas,
> Stefan

Thanks for your review,
  Thomas


From erik.helin at oracle.com  Wed Nov 22 18:18:26 2017
From: erik.helin at oracle.com (Erik Helin)
Date: Wed, 22 Nov 2017 19:18:26 +0100
Subject: RFR: 8189733: Cleanup Full GC setup and tear down
In-Reply-To: <f385d617-ca62-3684-37f6-b10b92bb49aa@oracle.com>
References: <3b9604e1-9d2e-c0d9-7b2e-e5d9232c6888@oracle.com>
 <1510821109.5332.1.camel@oracle.com>
 <f385d617-ca62-3684-37f6-b10b92bb49aa@oracle.com>
Message-ID: <a73bb9b5-5405-e2b8-35d8-34c50b0cb9af@oracle.com>

On 11/22/2017 04:39 PM, Stefan Johansson wrote:
> Thanks Thomas for the review,
> 
> I got some comments from Erik and have updated the review accordingly. 
> Here are the new webrevs:
> Full: http://cr.openjdk.java.net/~sjohanss/8189733/01/
> Inc: http://cr.openjdk.java.net/~sjohanss/8189733/00-01/

Looks good, Reviewed. Nice cleanup!

Thanks,
Erik

> This change removes the heap() function as it's only used by member 
> functions which can use _heap. Also changed the initialization list to 
> use the parameters rather than the members initialized.
> 
> Thanks,
> Stefan
> 
> On 2017-11-16 09:31, Thomas Schatzl wrote:
>> Hi,
>>
>> On Wed, 2017-11-15 at 17:24 +0100, Stefan Johansson wrote:
>>> Hi,
>>>
>>> Please review this enhancement:
>>> https://bugs.openjdk.java.net/browse/JDK-8189733
>>>
>>> Webrev:
>>> http://cr.openjdk.java.net/~sjohanss//8189733/00/index.html
>>>
>>> Summary:
>>> After the G1FullCollector has been introduced as part of JEP 307 it
>>> makes sense to move more of the setup and tear down for the Full GC
>>> into
>>> this class. The G1FullGCScope is also moved into the G1FullCollector
>>> since it is now only used here.
>> ?? looks good.
>>
>> Thomas
>>
> 


From rkennke at redhat.com  Wed Nov 22 18:20:54 2017
From: rkennke at redhat.com (Roman Kennke)
Date: Wed, 22 Nov 2017 19:20:54 +0100
Subject: RFR: 8191564: Refactor GC related servicability code into GC
 specific subclasses
In-Reply-To: <713325a4-3ed8-cd7d-c66d-a1c1767b6d3a@redhat.com>
References: <387d2b93-cb8b-51be-a6e0-3b5f9d6cbb1c@redhat.com>
 <713325a4-3ed8-cd7d-c66d-a1c1767b6d3a@redhat.com>
Message-ID: <2a7e6307-8c0d-ad4b-e50c-a7e1c2311080@redhat.com>

After a few more discussions with Erik I made some more changes.

MemoryService is now unaware of the number and meaning of GC memory 
managers (minor vs major). This should be better for GCs that don't make 
that distinction and/or use more different GCs (e.g. minor, 
intermediate, full).

This means that I needed to add some abstractions:
- GCMemoryManager now has gc_end_message() which is used by 
GCNotifier::pushNotification().
- gc_begin() and gc_end() methods in MemoryService now accept a 
GCMemoryManager* instead of bull full_gc
- Same for TraceMemoryManagerStats
- Generation now knows about the corresponding GCMemoryManager

Please review the full change:
http://cr.openjdk.java.net/~rkennke/8191564/webrev.02/

Thanks, Roman


> I had some off-band discussions with Erik Helin and re-did most of the 
> changeset:
> - The GC interface now resides in CollectedHeap, specifically the two 
> methods memory_managers() and memory_pools(), and is implemented in the 
> various concrete subclasses.
> - Both methods return (by value) a GrowableArray<GCMemoryManager*> and 
> GrowableArray<MemoryPool> respectively. Returning a stack-allocated 
> GrowableArray seemed least complicated (avoid explicit cleanup of 
> short-lived array object), and most future-proof, e.g. currently there 
> is an implicit expectation to get 2 GCMemoryManagers, even though some 
> GCs don't necessarily have two. The API allows for easy extension of the 
> situation.
> 
> http://cr.openjdk.java.net/~rkennke/8191564/webrev.01/
> 
> I think this requires reviews from both the GC and Serviceability team.
> 
> Roman
> 
> 
>> Currently, there's lots of GC specific code sprinkled over 
>> src/hotspot/share/services. This change introduces a GC interface for 
>> that, and moves all GC specific code to their respective 
>> src/hotspot/share/gc directory.
>>
>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.00/ 
>> <http://cr.openjdk.java.net/%7Erkennke/8191564/webrev.00/>
>>
>> Testing: hotspot_gc and hotspot_serviceability, none showed regressions
>>
>> Built minimal and server without regressions
>>
>> What do you think?
>>
>> Roman
>>
>>
> 



From sangheon.kim at oracle.com  Wed Nov 22 21:13:27 2017
From: sangheon.kim at oracle.com (sangheon.kim)
Date: Wed, 22 Nov 2017 13:13:27 -0800
Subject: RFR(XS): 8178497: Bug in MutableNUMASpace::ensure_parsability
In-Reply-To: <1511263671.9073.7.camel@oracle.com>
References: <8f8e7c5e-5de8-ee26-acb4-e90662dad0af@oracle.com>
 <1511263671.9073.7.camel@oracle.com>
Message-ID: <fab1ed66-cc34-6f81-350c-84f450da01e4@oracle.com>

Hi Thomas,

On 11/21/2017 03:27 AM, Thomas Schatzl wrote:
> Hi Sangheon,
>
> On Mon, 2017-11-20 at 11:05 -0800, sangheon.kim wrote:
>> Hi all,
>>
>> Can I have some reviews for this tiny pointer arithmetic change?
>> Current code doesn't use pointer arithmetic, so the the resulting
>> values are wrong(too small). i.e. adding two values first and then
>> typecast to HeapWord* which is wrong here.
>> e.g.
>> intptr_t cur_top = X;
>> size_t touched_words = XX;
>> ...
>> align_down((HeapWord*)(cur_top + touched_words), XXX);
>>
>> This should be 'align_down( (HeapWord*)cur_top + touched_words,
>> XXXX);'.
>>
>> As I don't see any good reason of using 'intptr_t', changed to use
>> 'HeapWord*' and changed related stuff.
>>
>> I didn't add regression test or some further investigation as this
>> is clear that the calculation is wrong. And hard to provoke the
>> problem outside.
>>
>> CR: https://bugs.openjdk.java.net/browse/JDK-8178497
>> Webrev: http://cr.openjdk.java.net/~sangheki/8178497/webrev.0
>> Testing: local building
>    actually after reading this code a bit in more detail I think the
> failure mode is "only" performance loss. I.e. the "invalid" part of a
> MutableNUMASpace is freed and reallocated for better NUMA locality.
Agree.

>
> With a too small invalid size what happens is that this locality
> improvement will not happen for that part of the MutableNUMASpace. I do
> not think there is a useful way of creating a reproducer.
>
> Looks good.
Thanks for the review.

Thanks,
Sangheon


>
> Thanks,
>    Thomas
>



From sangheon.kim at oracle.com  Wed Nov 22 21:23:33 2017
From: sangheon.kim at oracle.com (sangheon.kim)
Date: Wed, 22 Nov 2017 13:23:33 -0800
Subject: RFR(XS): 8178497: Bug in MutableNUMASpace::ensure_parsability
In-Reply-To: <e6739b0a-dd9c-77bc-cfa3-d5b411aa40c9@oracle.com>
References: <8f8e7c5e-5de8-ee26-acb4-e90662dad0af@oracle.com>
 <e6739b0a-dd9c-77bc-cfa3-d5b411aa40c9@oracle.com>
Message-ID: <5b43c769-ed7d-bb6f-507c-f8a5c2e3562f@oracle.com>

Hi Stefan,

Thank you for reviewing this.

On 11/21/2017 05:29 AM, Stefan Johansson wrote:
> Hi Sangheon,
>
> On 2017-11-20 20:05, sangheon.kim wrote:
>> Hi all,
>>
>> Can I have some reviews for this tiny pointer arithmetic change?
>> Current code doesn't use pointer arithmetic, so the the resulting 
>> values are wrong(too small). i.e. adding two values first and then 
>> typecast to HeapWord* which is wrong here.
>> e.g.
>> intptr_t cur_top = X;
>> size_t touched_words = XX;
>> ...
>> align_down((HeapWord*)(cur_top + touched_words), XXX);
>>
>> This should be 'align_down( (HeapWord*)cur_top + touched_words, XXXX);'.
>>
>> As I don't see any good reason of using 'intptr_t', changed to use 
>> 'HeapWord*' and changed related stuff.
>>
>> I didn't add regression test or some further investigation as this is 
>> clear that the calculation is wrong. And hard to provoke the problem 
>> outside.
>>
>> CR: https://bugs.openjdk.java.net/browse/JDK-8178497
>> Webrev: http://cr.openjdk.java.net/~sangheki/8178497/webrev.0
> Thanks for taking care of this. I think the changes is good, but I 
> would feel even more certain if you ran some testing with NUMA and 
> Parallel to make sure we haven't overlooked anything.
Okay, I added some explanation about what would happen.
And also ran hundred runs of GCOld with NUMA enabled on Solaris, of 
course no problem.

Hope this is sufficient to go. :)

Thanks,
Sangheon


>
> Thanks,
> StefanJ
>> Testing: local building
>>
>> Thanks,
>> Sangheon
>



From daniel.daugherty at oracle.com  Wed Nov 22 21:48:50 2017
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Wed, 22 Nov 2017 16:48:50 -0500
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <A20B6F1C-98A2-444F-8FED-E51E54B4CE64@oracle.com>
References: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
 <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>
 <836b5c6c-5eef-9245-c16a-e71372093d9a@oracle.com>
 <82d0e473-a24f-9feb-82e9-7d04c4d7a91d@oracle.com>
 <354f7ece-66bc-c065-9ee8-f55abcc1c23e@oracle.com>
 <0ff8df8e-e93c-02d2-cb97-b13b1ac0f53f@oracle.com>
 <5365606e-4036-b5d7-9bac-e91e42f38d01@oracle.com>
 <A20B6F1C-98A2-444F-8FED-E51E54B4CE64@oracle.com>
Message-ID: <06e3a59e-9225-ee56-eefe-2614e29bd8f9@oracle.com>

Thanks Jerry!

Dan


On 11/22/17 3:21 PM, Gerald Thornbrugh wrote:
> Hi Dan,
>
> Your changes look good.
>
> Jerry
>
>> On Nov 21, 2017, at 5:12 PM, Daniel D. Daugherty <daniel.daugherty at oracle.com> wrote:
>>
>> Greetings,
>>
>> *** We are wrapping up code review on this project so it is time ***
>> *** for the various code reviewers to chime in one last time. ***
>>
>> In this latest round, we had three different reviewers chime in so we're
>> doing delta webrevs for each of those resolutions:
>>
>> David H's resolutions:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.09,10.10.0-delta/
>>
>>    mq comment for dholmes_CR:
>>      - Fix indents, trailing spaces and various typos.
>>      - Add descriptions for the '_cnt', '_max' and '_times" fields,
>>        add impl notes to document the type choices.
>>
>>    src/hotspot/share/runtime/globals.hpp change is white-space only so it
>>    is only visible via the file's patch link.
>>
>>
>> Robin W's resolutions:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.10.0,10.10.1-delta/
>>
>>    mq comment for robinw_CR:
>>      - Fix some inefficient code, update some comments, fix some indents,
>>        and add some 'const' specifiers.
>>
>>    src/hotspot/share/runtime/vm_operations.hpp change is white-space only so
>>    it is only visible via the file's patch link.
>>
>>
>> Coleen's resolutions:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.10.1,10.10.2-delta/
>>
>>    mq comment for coleenp_CR:
>>      - Change ThreadsList::_threads from 'mtGC' -> 'mtThread',
>>      - add header comment to threadSMR.hpp file,
>>      - cleanup JavaThreadIteratorWithHandle ctr,
>>      - make ErrorHandling more efficient.
>>
>>
>> Some folks prefer to see all of the delta webrevs together so here is
>> a delta webrev relative to jdk10-09-full:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.09,10.10.2-delta/
>>
>>    src/hotspot/share/runtime/globals.hpp and
>>    src/hotspot/share/runtime/vm_operations.hpp changes are white-space only
>>    so they are only visible via the file's patch link.
>>
>>
>> And, of course, some folks prefer to see everything all at once so here
>> is the latest full webrev:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.10-full/
>>
>>
>> Dan has submitted the bits for the usual Mach5 tier[1-5] testing. The
>> prelim Mach5 tier1 (JPRT equivalent) on these bits passed just fine...
>>
>> The project is currently baselined on Jesper's 2017-11-17 PIT snapshot
>> so there will be some catching up to do before integration...
>>
>> We welcome comments, suggestions and feedback.
>>
>> Dan, Erik and Robbin
>>
>>
>> On 11/18/17 8:49 PM, Daniel D. Daugherty wrote:
>>> Greetings,
>>>
>>> Testing of the last round of changes revealed a hang in one of the new
>>> TLH tests. Robbin has fixed the hang, updated the existing TLH test, and
>>> added another TLH test for good measure.
>>>
>>> Here is the updated full webrev:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/
>>>
>>> Here is the updated delta webrev:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-delta/
>>>
>>> Dan ran the bits thru the usual Mach5 tier[1-5] testing and there are
>>> no unexpected failures.
>>>
>>> We welcome comments, suggestions and feedback.
>>>
>>> Dan, Erik and Robbin
>>>
>>>
>>> On 11/15/17 3:32 PM, Daniel D. Daugherty wrote:
>>>> Greetings,
>>>>
>>>> Robbin rebased the project last night/this morning to merge with Thread
>>>> Local Handshakes (TLH) and also picked up additional changesets up thru:
>>>>
>>>>> Changeset: fa736014cf28
>>>>> Author:    cjplummer
>>>>> Date:      2017-11-14 18:08 -0800
>>>>> URL:http://hg.openjdk.java.net/jdk/hs/rev/fa736014cf28
>>>>>
>>>>> 8191049: Add alternate version of pns() that is callable from within hotspot source
>>>>> Summary: added pns2() to debug.cpp
>>>>> Reviewed-by: stuefe, gthornbr
>>>> This is the first time we've rebased the project to bits that are this
>>>> fresh (< 12 hours old at rebase time). We've done this because we think
>>>> we're done with this project and are in the final review-change-retest
>>>> cycle(s)... In other words, we're not planning on making any more major
>>>> changes for this project... :-)
>>>>
>>>> *** Time for code reviewers to chime in on this thread! ***
>>>>
>>>> Here is the updated full webrev:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-full/
>>>>
>>>> Here's is the delta webrev from the 2017.11.10 rebase:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-delta/
>>>>
>>>> Dan has submitted the bits for the usual Mach5 tier[1-5] testing
>>>> (and the new baseline also)...
>>>>
>>>> We're expecting this round to be a little noisier than usual because
>>>> we did not rebase on a PIT snapshot so the new baseline has not been
>>>> through Jesper's usual care-and-feeding of integration_blockers, etc.
>>>>
>>>> We welcome comments, suggestions and feedback.
>>>>
>>>> Dan, Erik and Robbin
>>>>
>>>>
>>>> On 11/14/17 4:48 PM, Daniel D. Daugherty wrote:
>>>>> Greetings,
>>>>>
>>>>> I rebased the project to the 2017.11.10 jdk/hs PIT snapshot.
>>>>> (Yes, we're playing chase-the-repo...)
>>>>>
>>>>> Here is the updated full webrev:
>>>>>
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-07-full/
>>>>>
>>>>> Unlike the previous rebase, there were no changes required to the
>>>>> open code to get the rebased bits to build so there is no delta
>>>>> webrev.
>>>>>
>>>>> These bits have been run through JPRT and I've submitted the usual
>>>>> Mach5 tier[1-5] test run...
>>>>>
>>>>> We welcome comments, suggestions and feedback.
>>>>>
>>>>> Dan, Erik and Robbin
>>>>>
>>>>>
>>>>> On 11/13/17 12:30 PM, Daniel D. Daugherty wrote:
>>>>>> Greetings,
>>>>>>
>>>>>> I rebased the project to the 2017.10.26 jdk10/hs PIT snapshot.
>>>>>>
>>>>>> Here are the updated webrevs:
>>>>>>
>>>>>> Here's the mq comment for the change:
>>>>>>
>>>>>>    Rebase to 2017.10.25 PIT snapshot.
>>>>>>
>>>>>> Here is the full webrev:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-full/
>>>>>>
>>>>>> And here is the delta webrev:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-delta/
>>>>>>
>>>>>> I ran the above bits throught Mach5 tier[1-5] testing over the holiday
>>>>>> weekend. Didn't see any issues in a quick look. Going to take a closer
>>>>>> look today.
>>>>>>
>>>>>> We welcome comments, suggestions and feedback.
>>>>>>
>>>>>> Dan, Erik and Robbin
>>>>>>
>>>>>>
>>>>>> On 11/8/17 1:05 PM, Daniel D. Daugherty wrote:
>>>>>>> Greetings,
>>>>>>>
>>>>>>> Resolving one of the code review comments (from both Stefan K and Coleen)
>>>>>>> on jdk10-04-full required quite a few changes so it is being done as a
>>>>>>> standalone patch and corresponding webrevs:
>>>>>>>
>>>>>>> Here's the mq comment for the change:
>>>>>>>
>>>>>>>    stefank, coleenp CR - refactor most JavaThreadIterator usage to use
>>>>>>>      JavaThreadIteratorWithHandle.
>>>>>>>
>>>>>>> Here is the full webrev:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-full/
>>>>>>>
>>>>>>> And here is the delta webrev:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-delta/
>>>>>>>
>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>
>>>>>>> Dan, Erik and Robbin
>>>>>>>
>>>>>>>
>>>>>>> On 10/9/17 3:41 PM, Daniel D. Daugherty wrote:
>>>>>>>> Greetings,
>>>>>>>>
>>>>>>>> We have a (eXtra Large) fix for the following bug:
>>>>>>>>
>>>>>>>> 8167108 inconsistent handling of SR_lock can lead to crashes
>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8167108
>>>>>>>>
>>>>>>>> This fix adds a Safe Memory Reclamation (SMR) mechanism based on
>>>>>>>> Hazard Pointers to manage JavaThread lifecycle.
>>>>>>>>
>>>>>>>> Here's a PDF for the internal wiki that we've been using to describe
>>>>>>>> and track the work on this project:
>>>>>>>>
>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/SMR_and_JavaThread_Lifecycle-JDK10-04.pdf
>>>>>>>>
>>>>>>>> Dan has noticed that the indenting is wrong in some of the code quotes
>>>>>>>> in the PDF that are not present in the internal wiki. We don't have a
>>>>>>>> solution for that problem yet.
>>>>>>>>
>>>>>>>> Here's the webrev for current JDK10 version of this fix:
>>>>>>>>
>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-04-full
>>>>>>>>
>>>>>>>> This fix has been run through many rounds of JPRT and Mach5 tier[2-5]
>>>>>>>> testing, additional stress testing on Dan's Solaris X64 server, and
>>>>>>>> additional testing on Erik and Robbin's machines.
>>>>>>>>
>>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>>
>>>>>>>> Daniel Daugherty
>>>>>>>> Erik Osterlund
>>>>>>>> Robbin Ehn
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>



From alexander.harlap at oracle.com  Thu Nov 23 00:25:29 2017
From: alexander.harlap at oracle.com (Alexander Harlap)
Date: Wed, 22 Nov 2017 19:25:29 -0500
Subject: Need sponsor to push attached 8185591 into
 http://hg.openjdk.java.net/jdk10/hs
Message-ID: <9dbf43c5-116f-552e-fcfc-9672830902fb@oracle.com>

I need a sponsor to push attached 8185591.open.patch - 
guarantee(_byte_map[_guard_index] == last_card) failed: card table guard 
has been modified.

Patch should go into jdk10/hs.

Reviewed by Thomas Schatzl? and Kim Barrett.

Thank you,

Alex

-------------- next part --------------
# HG changeset patch
# User aharlap
# Date 1511395081 18000
# Node ID e7c78fc0a6525dc5bcd15bd766ea5c4a240efcf5
# Parent  d85284ccd1bd865ebb1391d921be7a8ebfc5f2c9
8185591: guarantee(_byte_map[_guard_index] == last_card) failed: card table guard has been modified
Summary: handle zero count in gen_write_ref_array_post_barrier()
Reviewed-by: tschatzl, kbarrett

diff -r d85284ccd1bd -r e7c78fc0a652 src/hotspot/cpu/arm/stubGenerator_arm.cpp
--- a/src/hotspot/cpu/arm/stubGenerator_arm.cpp	Fri Nov 03 17:09:25 2017 -0700
+++ b/src/hotspot/cpu/arm/stubGenerator_arm.cpp	Wed Nov 22 18:58:01 2017 -0500
@@ -2968,7 +2968,9 @@
         CardTableModRefBS* ct = barrier_set_cast<CardTableModRefBS>(bs);
         assert(sizeof(*ct->byte_map_base) == sizeof(jbyte), "adjust this code");
 
-        Label L_cardtable_loop;
+        Label L_cardtable_loop, L_done;
+
+        __ cbz_32(count, L_done); // zero count - nothing to do
 
         __ add_ptr_scaled_int32(count, addr, count, LogBytesPerHeapOop);
         __ sub(count, count, BytesPerHeapOop);                            // last addr
@@ -2987,6 +2989,7 @@
         __ strb(zero, Address(addr, 1, post_indexed));
         __ subs(count, count, 1);
         __ b(L_cardtable_loop, ge);
+        __ BIND(L_done);
       }
       break;
     case BarrierSet::ModRef:
diff -r d85284ccd1bd -r e7c78fc0a652 src/hotspot/cpu/sparc/stubGenerator_sparc.cpp
--- a/src/hotspot/cpu/sparc/stubGenerator_sparc.cpp	Fri Nov 03 17:09:25 2017 -0700
+++ b/src/hotspot/cpu/sparc/stubGenerator_sparc.cpp	Wed Nov 22 18:58:01 2017 -0500
@@ -898,7 +898,9 @@
           assert(sizeof(*ct->byte_map_base) == sizeof(jbyte), "adjust this code");
           assert_different_registers(addr, count, tmp);
 
-          Label L_loop;
+          Label L_loop, L_done;
+
+          __ cmp_and_br_short(count, 0, Assembler::equal, Assembler::pt, L_done); // zero count - nothing to do
 
           __ sll_ptr(count, LogBytesPerHeapOop, count);
           __ sub(count, BytesPerHeapOop, count);
@@ -914,6 +916,7 @@
           __ subcc(count, 1, count);
           __ brx(Assembler::greaterEqual, false, Assembler::pt, L_loop);
           __ delayed()->add(addr, 1, addr);
+        __ BIND(L_done);
         }
         break;
       case BarrierSet::ModRef:
diff -r d85284ccd1bd -r e7c78fc0a652 src/hotspot/cpu/x86/stubGenerator_x86_64.cpp
--- a/src/hotspot/cpu/x86/stubGenerator_x86_64.cpp	Fri Nov 03 17:09:25 2017 -0700
+++ b/src/hotspot/cpu/x86/stubGenerator_x86_64.cpp	Wed Nov 22 18:58:01 2017 -0500
@@ -1264,9 +1264,12 @@
           CardTableModRefBS* ct = barrier_set_cast<CardTableModRefBS>(bs);
           assert(sizeof(*ct->byte_map_base) == sizeof(jbyte), "adjust this code");
 
-          Label L_loop;
+          Label L_loop, L_done;
           const Register end = count;
 
+          __ testl(count, count);
+          __ jcc(Assembler::zero, L_done); // zero count - nothing to do
+
           __ leaq(end, Address(start, count, TIMES_OOP, 0));  // end == start+count*oop_size
           __ subptr(end, BytesPerHeapOop); // end - 1 to make inclusive
           __ shrptr(start, CardTableModRefBS::card_shift);
@@ -1280,6 +1283,7 @@
           __ movb(Address(start, count, Address::times_1), 0);
           __ decrement(count);
           __ jcc(Assembler::greaterEqual, L_loop);
+        __ BIND(L_done);
         }
         break;
       default:

From daniel.daugherty at oracle.com  Thu Nov 23 02:16:35 2017
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Wed, 22 Nov 2017 21:16:35 -0500
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <5365606e-4036-b5d7-9bac-e91e42f38d01@oracle.com>
References: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
 <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>
 <836b5c6c-5eef-9245-c16a-e71372093d9a@oracle.com>
 <82d0e473-a24f-9feb-82e9-7d04c4d7a91d@oracle.com>
 <354f7ece-66bc-c065-9ee8-f55abcc1c23e@oracle.com>
 <0ff8df8e-e93c-02d2-cb97-b13b1ac0f53f@oracle.com>
 <5365606e-4036-b5d7-9bac-e91e42f38d01@oracle.com>
Message-ID: <567339a6-e36a-5416-5c34-2a87cbfa6cdd@oracle.com>

Greetings,

I've made minor tweaks to the Thread-SMR project based on the remaining
code review comments over the last couple of days. I've also rebased the
project to jdk/hs bits as of mid-afternoon (my TZ) on 2017.11.22. I'm
running baseline Mach5 Tier[1-5] testing and prototype Mach5 Tier[1-5]
testing...

Here's a delta webrev for the latest round of tweaks:

http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-11-delta


Here's the proposed commit message:

$ cat SMR_prototype_10/open/commit.txt
8167108: inconsistent handling of SR_lock can lead to crashes
Summary: Add Thread Safe Memory Reclamation (Thread-SMR) mechanism.
Reviewed-by: coleenp, dcubed, dholmes, eosterlund, gthornbr, kbarrett, 
rehn, sspitsyn, stefank
Contributed-by: daniel.daugherty at oracle.com, erik.osterlund at oracle.com, 
robbin.ehn at oracle.com

I've gone through 880+ emails in my archive for this project and added
anyone that made a code review comment. Reviewers are not in my usual
by-comment-posting order; it was way too hard to figure that out... So
reviewers and contributors are simply in alpha-sort order...

Here's the collection of follow-up bugs that I filed based on sifting
through the email archive:

8191784 JavaThread::collect_counters() should stop using Threads_lock
https://bugs.openjdk.java.net/browse/JDK-8191784

8191785 revoke_bias() needs a new assertion
https://bugs.openjdk.java.net/browse/JDK-8191785

8191786 Thread-SMR hash table size should be dynamic
https://bugs.openjdk.java.net/browse/JDK-8191786

8191787 move private inline functions from thread.inline.hpp -> thread.cpp
https://bugs.openjdk.java.net/browse/JDK-8191787

8191789 migrate more Thread-SMR stuff from thread.[ch]pp -> threadSMR.[ch]pp
https://bugs.openjdk.java.net/browse/JDK-8191789

8191798 redo nested ThreadsListHandle to drop Threads_lock
https://bugs.openjdk.java.net/browse/JDK-8191789

I have undoubtedly missed at least one future idea that someone had
and for that my apologies...

Thanks again to everyone who contributed to the Thread-SMR project!!

Special thanks goes to Karen Kinnear, Kim Barrett and David Holmes for
their rigorous review of the design that we documented on the wiki.
Thanks for keeping us honest!

I also have to thank my co-contributor Erik ?sterlund for starting the
ball rolling on this crazy project with his presentation on Safe Memory
Reclamation, for the initial prototype and for all your patches. Erik,
hopefully we got far enough in your crazy vision... :-)

Thanks to my co-contributor Robbin Ehn for proposing that we do early
code reviews in the form of patches. Don't like something? Fix it with
a patch and a new test if appropriate!! A pretty cool way to work that
was new to me.

Finally, many thanks to Jerry Thornbrugh for all the early code reviews,
whiteboard chats and phone calls. Thanks for asking the right questions
and pointing out some places where our logic was faulty. Also thanks for
keeping me sane when I was literally on my own in Monument, CO.

Dan


On 11/21/17 7:12 PM, Daniel D. Daugherty wrote:
> Greetings,
>
> *** We are wrapping up code review on this project so it is time ***
> *** for the various code reviewers to chime in one last time. ***
>
> In this latest round, we had three different reviewers chime in so we're
> doing delta webrevs for each of those resolutions:
>
> David H's resolutions:
>
> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.09,10.10.0-delta/ 
>
>
> ? mq comment for dholmes_CR:
> ??? - Fix indents, trailing spaces and various typos.
> ??? - Add descriptions for the '_cnt', '_max' and '_times" fields,
> ????? add impl notes to document the type choices.
>
> ? src/hotspot/share/runtime/globals.hpp change is white-space only so it
> ? is only visible via the file's patch link.
>
>
> Robin W's resolutions:
>
> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.10.0,10.10.1-delta/ 
>
>
> ? mq comment for robinw_CR:
> ??? - Fix some inefficient code, update some comments, fix some indents,
> ????? and add some 'const' specifiers.
>
> ? src/hotspot/share/runtime/vm_operations.hpp change is white-space 
> only so
> ? it is only visible via the file's patch link.
>
>
> Coleen's resolutions:
>
> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.10.1,10.10.2-delta/ 
>
>
> ? mq comment for coleenp_CR:
> ??? - Change ThreadsList::_threads from 'mtGC' -> 'mtThread',
> ??? - add header comment to threadSMR.hpp file,
> ??? - cleanup JavaThreadIteratorWithHandle ctr,
> ??? - make ErrorHandling more efficient.
>
>
> Some folks prefer to see all of the delta webrevs together so here is
> a delta webrev relative to jdk10-09-full:
>
> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.09,10.10.2-delta/ 
>
>
> ? src/hotspot/share/runtime/globals.hpp and
> ? src/hotspot/share/runtime/vm_operations.hpp changes are white-space 
> only
> ? so they are only visible via the file's patch link.
>
>
> And, of course, some folks prefer to see everything all at once so here
> is the latest full webrev:
>
> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.10-full/
>
>
> Dan has submitted the bits for the usual Mach5 tier[1-5] testing. The
> prelim Mach5 tier1 (JPRT equivalent) on these bits passed just fine...
>
> The project is currently baselined on Jesper's 2017-11-17 PIT snapshot
> so there will be some catching up to do before integration...
>
> We welcome comments, suggestions and feedback.
>
> Dan, Erik and Robbin
>
>
> On 11/18/17 8:49 PM, Daniel D. Daugherty wrote:
>> Greetings,
>>
>> Testing of the last round of changes revealed a hang in one of the new
>> TLH tests. Robbin has fixed the hang, updated the existing TLH test, and
>> added another TLH test for good measure.
>>
>> Here is the updated full webrev:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/
>>
>> Here is the updated delta webrev:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-delta/
>>
>> Dan ran the bits thru the usual Mach5 tier[1-5] testing and there are
>> no unexpected failures.
>>
>> We welcome comments, suggestions and feedback.
>>
>> Dan, Erik and Robbin
>>
>>
>> On 11/15/17 3:32 PM, Daniel D. Daugherty wrote:
>>> Greetings,
>>>
>>> Robbin rebased the project last night/this morning to merge with Thread
>>> Local Handshakes (TLH) and also picked up additional changesets up 
>>> thru:
>>>
>>>> Changeset: fa736014cf28
>>>> Author:??? cjplummer
>>>> Date:????? 2017-11-14 18:08 -0800
>>>> URL:http://hg.openjdk.java.net/jdk/hs/rev/fa736014cf28
>>>>
>>>> 8191049: Add alternate version of pns() that is callable from 
>>>> within hotspot source
>>>> Summary: added pns2() to debug.cpp
>>>> Reviewed-by: stuefe, gthornbr
>>>
>>> This is the first time we've rebased the project to bits that are this
>>> fresh (< 12 hours old at rebase time). We've done this because we think
>>> we're done with this project and are in the final review-change-retest
>>> cycle(s)... In other words, we're not planning on making any more major
>>> changes for this project... :-)
>>>
>>> *** Time for code reviewers to chime in on this thread! ***
>>>
>>> Here is the updated full webrev:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-full/
>>>
>>> Here's is the delta webrev from the 2017.11.10 rebase:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-delta/
>>>
>>> Dan has submitted the bits for the usual Mach5 tier[1-5] testing
>>> (and the new baseline also)...
>>>
>>> We're expecting this round to be a little noisier than usual because
>>> we did not rebase on a PIT snapshot so the new baseline has not been
>>> through Jesper's usual care-and-feeding of integration_blockers, etc.
>>>
>>> We welcome comments, suggestions and feedback.
>>>
>>> Dan, Erik and Robbin
>>>
>>>
>>> On 11/14/17 4:48 PM, Daniel D. Daugherty wrote:
>>>> Greetings,
>>>>
>>>> I rebased the project to the 2017.11.10 jdk/hs PIT snapshot.
>>>> (Yes, we're playing chase-the-repo...)
>>>>
>>>> Here is the updated full webrev:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-07-full/
>>>>
>>>> Unlike the previous rebase, there were no changes required to the
>>>> open code to get the rebased bits to build so there is no delta
>>>> webrev.
>>>>
>>>> These bits have been run through JPRT and I've submitted the usual
>>>> Mach5 tier[1-5] test run...
>>>>
>>>> We welcome comments, suggestions and feedback.
>>>>
>>>> Dan, Erik and Robbin
>>>>
>>>>
>>>> On 11/13/17 12:30 PM, Daniel D. Daugherty wrote:
>>>>> Greetings,
>>>>>
>>>>> I rebased the project to the 2017.10.26 jdk10/hs PIT snapshot.
>>>>>
>>>>> Here are the updated webrevs:
>>>>>
>>>>> Here's the mq comment for the change:
>>>>>
>>>>> ? Rebase to 2017.10.25 PIT snapshot.
>>>>>
>>>>> Here is the full webrev:
>>>>>
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-full/
>>>>>
>>>>> And here is the delta webrev:
>>>>>
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-delta/
>>>>>
>>>>> I ran the above bits throught Mach5 tier[1-5] testing over the 
>>>>> holiday
>>>>> weekend. Didn't see any issues in a quick look. Going to take a 
>>>>> closer
>>>>> look today.
>>>>>
>>>>> We welcome comments, suggestions and feedback.
>>>>>
>>>>> Dan, Erik and Robbin
>>>>>
>>>>>
>>>>> On 11/8/17 1:05 PM, Daniel D. Daugherty wrote:
>>>>>> Greetings,
>>>>>>
>>>>>> Resolving one of the code review comments (from both Stefan K and 
>>>>>> Coleen)
>>>>>> on jdk10-04-full required quite a few changes so it is being done 
>>>>>> as a
>>>>>> standalone patch and corresponding webrevs:
>>>>>>
>>>>>> Here's the mq comment for the change:
>>>>>>
>>>>>> ? stefank, coleenp CR - refactor most JavaThreadIterator usage to 
>>>>>> use
>>>>>> ??? JavaThreadIteratorWithHandle.
>>>>>>
>>>>>> Here is the full webrev:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-full/
>>>>>>
>>>>>> And here is the delta webrev:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-delta/
>>>>>>
>>>>>> We welcome comments, suggestions and feedback.
>>>>>>
>>>>>> Dan, Erik and Robbin
>>>>>>
>>>>>>
>>>>>> On 10/9/17 3:41 PM, Daniel D. Daugherty wrote:
>>>>>>> Greetings,
>>>>>>>
>>>>>>> We have a (eXtra Large) fix for the following bug:
>>>>>>>
>>>>>>> 8167108 inconsistent handling of SR_lock can lead to crashes
>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8167108
>>>>>>>
>>>>>>> This fix adds a Safe Memory Reclamation (SMR) mechanism based on
>>>>>>> Hazard Pointers to manage JavaThread lifecycle.
>>>>>>>
>>>>>>> Here's a PDF for the internal wiki that we've been using to 
>>>>>>> describe
>>>>>>> and track the work on this project:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/SMR_and_JavaThread_Lifecycle-JDK10-04.pdf 
>>>>>>>
>>>>>>>
>>>>>>> Dan has noticed that the indenting is wrong in some of the code 
>>>>>>> quotes
>>>>>>> in the PDF that are not present in the internal wiki. We don't 
>>>>>>> have a
>>>>>>> solution for that problem yet.
>>>>>>>
>>>>>>> Here's the webrev for current JDK10 version of this fix:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-04-full
>>>>>>>
>>>>>>> This fix has been run through many rounds of JPRT and Mach5 
>>>>>>> tier[2-5]
>>>>>>> testing, additional stress testing on Dan's Solaris X64 server, and
>>>>>>> additional testing on Erik and Robbin's machines.
>>>>>>>
>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>
>>>>>>> Daniel Daugherty
>>>>>>> Erik Osterlund
>>>>>>> Robbin Ehn
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>



From robbin.ehn at oracle.com  Thu Nov 23 08:20:01 2017
From: robbin.ehn at oracle.com (Robbin Ehn)
Date: Thu, 23 Nov 2017 09:20:01 +0100
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <567339a6-e36a-5416-5c34-2a87cbfa6cdd@oracle.com>
References: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
 <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>
 <836b5c6c-5eef-9245-c16a-e71372093d9a@oracle.com>
 <82d0e473-a24f-9feb-82e9-7d04c4d7a91d@oracle.com>
 <354f7ece-66bc-c065-9ee8-f55abcc1c23e@oracle.com>
 <0ff8df8e-e93c-02d2-cb97-b13b1ac0f53f@oracle.com>
 <5365606e-4036-b5d7-9bac-e91e42f38d01@oracle.com>
 <567339a6-e36a-5416-5c34-2a87cbfa6cdd@oracle.com>
Message-ID: <5976a6bf-c9b0-c0d5-d5f9-fcdf7dc9a1a8@oracle.com>

Thanks Dan for dragging this freight train to the docks, it's time to ship it!

Created follow-up bug:
8191809: Threads::number_of_threads() is not 'MT-safe'
https://bugs.openjdk.java.net/browse/JDK-8191809

/Robbin

On 2017-11-23 03:16, Daniel D. Daugherty wrote:
> Greetings,
> 
> I've made minor tweaks to the Thread-SMR project based on the remaining
> code review comments over the last couple of days. I've also rebased the
> project to jdk/hs bits as of mid-afternoon (my TZ) on 2017.11.22. I'm
> running baseline Mach5 Tier[1-5] testing and prototype Mach5 Tier[1-5]
> testing...
> 
> Here's a delta webrev for the latest round of tweaks:
> 
> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-11-delta
> 
> 
> Here's the proposed commit message:
> 
> $ cat SMR_prototype_10/open/commit.txt
> 8167108: inconsistent handling of SR_lock can lead to crashes
> Summary: Add Thread Safe Memory Reclamation (Thread-SMR) mechanism.
> Reviewed-by: coleenp, dcubed, dholmes, eosterlund, gthornbr, kbarrett, rehn, sspitsyn, stefank
> Contributed-by: daniel.daugherty at oracle.com, erik.osterlund at oracle.com, robbin.ehn at oracle.com
> 
> I've gone through 880+ emails in my archive for this project and added
> anyone that made a code review comment. Reviewers are not in my usual
> by-comment-posting order; it was way too hard to figure that out... So
> reviewers and contributors are simply in alpha-sort order...
> 
> Here's the collection of follow-up bugs that I filed based on sifting
> through the email archive:
> 
> 8191784 JavaThread::collect_counters() should stop using Threads_lock
> https://bugs.openjdk.java.net/browse/JDK-8191784
> 
> 8191785 revoke_bias() needs a new assertion
> https://bugs.openjdk.java.net/browse/JDK-8191785
> 
> 8191786 Thread-SMR hash table size should be dynamic
> https://bugs.openjdk.java.net/browse/JDK-8191786
> 
> 8191787 move private inline functions from thread.inline.hpp -> thread.cpp
> https://bugs.openjdk.java.net/browse/JDK-8191787
> 
> 8191789 migrate more Thread-SMR stuff from thread.[ch]pp -> threadSMR.[ch]pp
> https://bugs.openjdk.java.net/browse/JDK-8191789
> 
> 8191798 redo nested ThreadsListHandle to drop Threads_lock
> https://bugs.openjdk.java.net/browse/JDK-8191789
> 
> I have undoubtedly missed at least one future idea that someone had
> and for that my apologies...
> 
> Thanks again to everyone who contributed to the Thread-SMR project!!
> 
> Special thanks goes to Karen Kinnear, Kim Barrett and David Holmes for
> their rigorous review of the design that we documented on the wiki.
> Thanks for keeping us honest!
> 
> I also have to thank my co-contributor Erik ?sterlund for starting the
> ball rolling on this crazy project with his presentation on Safe Memory
> Reclamation, for the initial prototype and for all your patches. Erik,
> hopefully we got far enough in your crazy vision... :-)
> 
> Thanks to my co-contributor Robbin Ehn for proposing that we do early
> code reviews in the form of patches. Don't like something? Fix it with
> a patch and a new test if appropriate!! A pretty cool way to work that
> was new to me.
> 
> Finally, many thanks to Jerry Thornbrugh for all the early code reviews,
> whiteboard chats and phone calls. Thanks for asking the right questions
> and pointing out some places where our logic was faulty. Also thanks for
> keeping me sane when I was literally on my own in Monument, CO.
> 
> Dan
> 
> 
> On 11/21/17 7:12 PM, Daniel D. Daugherty wrote:
>> Greetings,
>>
>> *** We are wrapping up code review on this project so it is time ***
>> *** for the various code reviewers to chime in one last time. ***
>>
>> In this latest round, we had three different reviewers chime in so we're
>> doing delta webrevs for each of those resolutions:
>>
>> David H's resolutions:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.09,10.10.0-delta/
>>
>> ? mq comment for dholmes_CR:
>> ??? - Fix indents, trailing spaces and various typos.
>> ??? - Add descriptions for the '_cnt', '_max' and '_times" fields,
>> ????? add impl notes to document the type choices.
>>
>> ? src/hotspot/share/runtime/globals.hpp change is white-space only so it
>> ? is only visible via the file's patch link.
>>
>>
>> Robin W's resolutions:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.10.0,10.10.1-delta/
>>
>> ? mq comment for robinw_CR:
>> ??? - Fix some inefficient code, update some comments, fix some indents,
>> ????? and add some 'const' specifiers.
>>
>> ? src/hotspot/share/runtime/vm_operations.hpp change is white-space only so
>> ? it is only visible via the file's patch link.
>>
>>
>> Coleen's resolutions:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.10.1,10.10.2-delta/
>>
>> ? mq comment for coleenp_CR:
>> ??? - Change ThreadsList::_threads from 'mtGC' -> 'mtThread',
>> ??? - add header comment to threadSMR.hpp file,
>> ??? - cleanup JavaThreadIteratorWithHandle ctr,
>> ??? - make ErrorHandling more efficient.
>>
>>
>> Some folks prefer to see all of the delta webrevs together so here is
>> a delta webrev relative to jdk10-09-full:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.09,10.10.2-delta/
>>
>> ? src/hotspot/share/runtime/globals.hpp and
>> ? src/hotspot/share/runtime/vm_operations.hpp changes are white-space only
>> ? so they are only visible via the file's patch link.
>>
>>
>> And, of course, some folks prefer to see everything all at once so here
>> is the latest full webrev:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.10-full/
>>
>>
>> Dan has submitted the bits for the usual Mach5 tier[1-5] testing. The
>> prelim Mach5 tier1 (JPRT equivalent) on these bits passed just fine...
>>
>> The project is currently baselined on Jesper's 2017-11-17 PIT snapshot
>> so there will be some catching up to do before integration...
>>
>> We welcome comments, suggestions and feedback.
>>
>> Dan, Erik and Robbin
>>
>>
>> On 11/18/17 8:49 PM, Daniel D. Daugherty wrote:
>>> Greetings,
>>>
>>> Testing of the last round of changes revealed a hang in one of the new
>>> TLH tests. Robbin has fixed the hang, updated the existing TLH test, and
>>> added another TLH test for good measure.
>>>
>>> Here is the updated full webrev:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/
>>>
>>> Here is the updated delta webrev:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-delta/
>>>
>>> Dan ran the bits thru the usual Mach5 tier[1-5] testing and there are
>>> no unexpected failures.
>>>
>>> We welcome comments, suggestions and feedback.
>>>
>>> Dan, Erik and Robbin
>>>
>>>
>>> On 11/15/17 3:32 PM, Daniel D. Daugherty wrote:
>>>> Greetings,
>>>>
>>>> Robbin rebased the project last night/this morning to merge with Thread
>>>> Local Handshakes (TLH) and also picked up additional changesets up thru:
>>>>
>>>>> Changeset: fa736014cf28
>>>>> Author:??? cjplummer
>>>>> Date:????? 2017-11-14 18:08 -0800
>>>>> URL:http://hg.openjdk.java.net/jdk/hs/rev/fa736014cf28
>>>>>
>>>>> 8191049: Add alternate version of pns() that is callable from within hotspot source
>>>>> Summary: added pns2() to debug.cpp
>>>>> Reviewed-by: stuefe, gthornbr
>>>>
>>>> This is the first time we've rebased the project to bits that are this
>>>> fresh (< 12 hours old at rebase time). We've done this because we think
>>>> we're done with this project and are in the final review-change-retest
>>>> cycle(s)... In other words, we're not planning on making any more major
>>>> changes for this project... :-)
>>>>
>>>> *** Time for code reviewers to chime in on this thread! ***
>>>>
>>>> Here is the updated full webrev:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-full/
>>>>
>>>> Here's is the delta webrev from the 2017.11.10 rebase:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-delta/
>>>>
>>>> Dan has submitted the bits for the usual Mach5 tier[1-5] testing
>>>> (and the new baseline also)...
>>>>
>>>> We're expecting this round to be a little noisier than usual because
>>>> we did not rebase on a PIT snapshot so the new baseline has not been
>>>> through Jesper's usual care-and-feeding of integration_blockers, etc.
>>>>
>>>> We welcome comments, suggestions and feedback.
>>>>
>>>> Dan, Erik and Robbin
>>>>
>>>>
>>>> On 11/14/17 4:48 PM, Daniel D. Daugherty wrote:
>>>>> Greetings,
>>>>>
>>>>> I rebased the project to the 2017.11.10 jdk/hs PIT snapshot.
>>>>> (Yes, we're playing chase-the-repo...)
>>>>>
>>>>> Here is the updated full webrev:
>>>>>
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-07-full/
>>>>>
>>>>> Unlike the previous rebase, there were no changes required to the
>>>>> open code to get the rebased bits to build so there is no delta
>>>>> webrev.
>>>>>
>>>>> These bits have been run through JPRT and I've submitted the usual
>>>>> Mach5 tier[1-5] test run...
>>>>>
>>>>> We welcome comments, suggestions and feedback.
>>>>>
>>>>> Dan, Erik and Robbin
>>>>>
>>>>>
>>>>> On 11/13/17 12:30 PM, Daniel D. Daugherty wrote:
>>>>>> Greetings,
>>>>>>
>>>>>> I rebased the project to the 2017.10.26 jdk10/hs PIT snapshot.
>>>>>>
>>>>>> Here are the updated webrevs:
>>>>>>
>>>>>> Here's the mq comment for the change:
>>>>>>
>>>>>> ? Rebase to 2017.10.25 PIT snapshot.
>>>>>>
>>>>>> Here is the full webrev:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-full/
>>>>>>
>>>>>> And here is the delta webrev:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-delta/
>>>>>>
>>>>>> I ran the above bits throught Mach5 tier[1-5] testing over the holiday
>>>>>> weekend. Didn't see any issues in a quick look. Going to take a closer
>>>>>> look today.
>>>>>>
>>>>>> We welcome comments, suggestions and feedback.
>>>>>>
>>>>>> Dan, Erik and Robbin
>>>>>>
>>>>>>
>>>>>> On 11/8/17 1:05 PM, Daniel D. Daugherty wrote:
>>>>>>> Greetings,
>>>>>>>
>>>>>>> Resolving one of the code review comments (from both Stefan K and Coleen)
>>>>>>> on jdk10-04-full required quite a few changes so it is being done as a
>>>>>>> standalone patch and corresponding webrevs:
>>>>>>>
>>>>>>> Here's the mq comment for the change:
>>>>>>>
>>>>>>> ? stefank, coleenp CR - refactor most JavaThreadIterator usage to use
>>>>>>> ??? JavaThreadIteratorWithHandle.
>>>>>>>
>>>>>>> Here is the full webrev:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-full/
>>>>>>>
>>>>>>> And here is the delta webrev:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-delta/
>>>>>>>
>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>
>>>>>>> Dan, Erik and Robbin
>>>>>>>
>>>>>>>
>>>>>>> On 10/9/17 3:41 PM, Daniel D. Daugherty wrote:
>>>>>>>> Greetings,
>>>>>>>>
>>>>>>>> We have a (eXtra Large) fix for the following bug:
>>>>>>>>
>>>>>>>> 8167108 inconsistent handling of SR_lock can lead to crashes
>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8167108
>>>>>>>>
>>>>>>>> This fix adds a Safe Memory Reclamation (SMR) mechanism based on
>>>>>>>> Hazard Pointers to manage JavaThread lifecycle.
>>>>>>>>
>>>>>>>> Here's a PDF for the internal wiki that we've been using to describe
>>>>>>>> and track the work on this project:
>>>>>>>>
>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/SMR_and_JavaThread_Lifecycle-JDK10-04.pdf
>>>>>>>>
>>>>>>>> Dan has noticed that the indenting is wrong in some of the code quotes
>>>>>>>> in the PDF that are not present in the internal wiki. We don't have a
>>>>>>>> solution for that problem yet.
>>>>>>>>
>>>>>>>> Here's the webrev for current JDK10 version of this fix:
>>>>>>>>
>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-04-full
>>>>>>>>
>>>>>>>> This fix has been run through many rounds of JPRT and Mach5 tier[2-5]
>>>>>>>> testing, additional stress testing on Dan's Solaris X64 server, and
>>>>>>>> additional testing on Erik and Robbin's machines.
>>>>>>>>
>>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>>
>>>>>>>> Daniel Daugherty
>>>>>>>> Erik Osterlund
>>>>>>>> Robbin Ehn
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
> 


From stefan.johansson at oracle.com  Thu Nov 23 09:17:14 2017
From: stefan.johansson at oracle.com (Stefan Johansson)
Date: Thu, 23 Nov 2017 10:17:14 +0100
Subject: RFR(XS): 8178497: Bug in MutableNUMASpace::ensure_parsability
In-Reply-To: <5b43c769-ed7d-bb6f-507c-f8a5c2e3562f@oracle.com>
References: <8f8e7c5e-5de8-ee26-acb4-e90662dad0af@oracle.com>
 <e6739b0a-dd9c-77bc-cfa3-d5b411aa40c9@oracle.com>
 <5b43c769-ed7d-bb6f-507c-f8a5c2e3562f@oracle.com>
Message-ID: <ae0e52b1-a07c-1204-a69a-9b94d9ce952c@oracle.com>



On 2017-11-22 22:23, sangheon.kim wrote:
> Hi Stefan,
>
> Thank you for reviewing this.
>
> On 11/21/2017 05:29 AM, Stefan Johansson wrote:
>> Hi Sangheon,
>>
>> On 2017-11-20 20:05, sangheon.kim wrote:
>>> Hi all,
>>>
>>> Can I have some reviews for this tiny pointer arithmetic change?
>>> Current code doesn't use pointer arithmetic, so the the resulting 
>>> values are wrong(too small). i.e. adding two values first and then 
>>> typecast to HeapWord* which is wrong here.
>>> e.g.
>>> intptr_t cur_top = X;
>>> size_t touched_words = XX;
>>> ...
>>> align_down((HeapWord*)(cur_top + touched_words), XXX);
>>>
>>> This should be 'align_down( (HeapWord*)cur_top + touched_words, 
>>> XXXX);'.
>>>
>>> As I don't see any good reason of using 'intptr_t', changed to use 
>>> 'HeapWord*' and changed related stuff.
>>>
>>> I didn't add regression test or some further investigation as this 
>>> is clear that the calculation is wrong. And hard to provoke the 
>>> problem outside.
>>>
>>> CR: https://bugs.openjdk.java.net/browse/JDK-8178497
>>> Webrev: http://cr.openjdk.java.net/~sangheki/8178497/webrev.0
>> Thanks for taking care of this. I think the changes is good, but I 
>> would feel even more certain if you ran some testing with NUMA and 
>> Parallel to make sure we haven't overlooked anything.
> Okay, I added some explanation about what would happen.
> And also ran hundred runs of GCOld with NUMA enabled on Solaris, of 
> course no problem.
>
> Hope this is sufficient to go. :)
>
Thanks Sangheon,

That's great.

Cheers,
Stefan
> Thanks,
> Sangheon
>
>
>>
>> Thanks,
>> StefanJ
>>> Testing: local building
>>>
>>> Thanks,
>>> Sangheon
>>
>



From daniel.daugherty at oracle.com  Thu Nov 23 13:31:30 2017
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Thu, 23 Nov 2017 08:31:30 -0500
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <5976a6bf-c9b0-c0d5-d5f9-fcdf7dc9a1a8@oracle.com>
References: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
 <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>
 <836b5c6c-5eef-9245-c16a-e71372093d9a@oracle.com>
 <82d0e473-a24f-9feb-82e9-7d04c4d7a91d@oracle.com>
 <354f7ece-66bc-c065-9ee8-f55abcc1c23e@oracle.com>
 <0ff8df8e-e93c-02d2-cb97-b13b1ac0f53f@oracle.com>
 <5365606e-4036-b5d7-9bac-e91e42f38d01@oracle.com>
 <567339a6-e36a-5416-5c34-2a87cbfa6cdd@oracle.com>
 <5976a6bf-c9b0-c0d5-d5f9-fcdf7dc9a1a8@oracle.com>
Message-ID: <28edcbec-4e10-6c3c-df46-5dba25a93c58@oracle.com>

On 11/23/17 3:20 AM, Robbin Ehn wrote:
> Thanks Dan for dragging this freight train to the docks, it's time to 
> ship it!

Working on that very thing today... will likely push late today (my TZ)...


> Created follow-up bug:
> 8191809: Threads::number_of_threads() is not 'MT-safe'
> https://bugs.openjdk.java.net/browse/JDK-8191809

Nice follow-up! Thanks for filing it.

Dan


>
> /Robbin
>
> On 2017-11-23 03:16, Daniel D. Daugherty wrote:
>> Greetings,
>>
>> I've made minor tweaks to the Thread-SMR project based on the remaining
>> code review comments over the last couple of days. I've also rebased the
>> project to jdk/hs bits as of mid-afternoon (my TZ) on 2017.11.22. I'm
>> running baseline Mach5 Tier[1-5] testing and prototype Mach5 Tier[1-5]
>> testing...
>>
>> Here's a delta webrev for the latest round of tweaks:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-11-delta
>>
>>
>> Here's the proposed commit message:
>>
>> $ cat SMR_prototype_10/open/commit.txt
>> 8167108: inconsistent handling of SR_lock can lead to crashes
>> Summary: Add Thread Safe Memory Reclamation (Thread-SMR) mechanism.
>> Reviewed-by: coleenp, dcubed, dholmes, eosterlund, gthornbr, 
>> kbarrett, rehn, sspitsyn, stefank
>> Contributed-by: daniel.daugherty at oracle.com, 
>> erik.osterlund at oracle.com, robbin.ehn at oracle.com
>>
>> I've gone through 880+ emails in my archive for this project and added
>> anyone that made a code review comment. Reviewers are not in my usual
>> by-comment-posting order; it was way too hard to figure that out... So
>> reviewers and contributors are simply in alpha-sort order...
>>
>> Here's the collection of follow-up bugs that I filed based on sifting
>> through the email archive:
>>
>> 8191784 JavaThread::collect_counters() should stop using Threads_lock
>> https://bugs.openjdk.java.net/browse/JDK-8191784
>>
>> 8191785 revoke_bias() needs a new assertion
>> https://bugs.openjdk.java.net/browse/JDK-8191785
>>
>> 8191786 Thread-SMR hash table size should be dynamic
>> https://bugs.openjdk.java.net/browse/JDK-8191786
>>
>> 8191787 move private inline functions from thread.inline.hpp -> 
>> thread.cpp
>> https://bugs.openjdk.java.net/browse/JDK-8191787
>>
>> 8191789 migrate more Thread-SMR stuff from thread.[ch]pp -> 
>> threadSMR.[ch]pp
>> https://bugs.openjdk.java.net/browse/JDK-8191789
>>
>> 8191798 redo nested ThreadsListHandle to drop Threads_lock
>> https://bugs.openjdk.java.net/browse/JDK-8191789
>>
>> I have undoubtedly missed at least one future idea that someone had
>> and for that my apologies...
>>
>> Thanks again to everyone who contributed to the Thread-SMR project!!
>>
>> Special thanks goes to Karen Kinnear, Kim Barrett and David Holmes for
>> their rigorous review of the design that we documented on the wiki.
>> Thanks for keeping us honest!
>>
>> I also have to thank my co-contributor Erik ?sterlund for starting the
>> ball rolling on this crazy project with his presentation on Safe Memory
>> Reclamation, for the initial prototype and for all your patches. Erik,
>> hopefully we got far enough in your crazy vision... :-)
>>
>> Thanks to my co-contributor Robbin Ehn for proposing that we do early
>> code reviews in the form of patches. Don't like something? Fix it with
>> a patch and a new test if appropriate!! A pretty cool way to work that
>> was new to me.
>>
>> Finally, many thanks to Jerry Thornbrugh for all the early code reviews,
>> whiteboard chats and phone calls. Thanks for asking the right questions
>> and pointing out some places where our logic was faulty. Also thanks for
>> keeping me sane when I was literally on my own in Monument, CO.
>>
>> Dan
>>
>>
>> On 11/21/17 7:12 PM, Daniel D. Daugherty wrote:
>>> Greetings,
>>>
>>> *** We are wrapping up code review on this project so it is time ***
>>> *** for the various code reviewers to chime in one last time. ***
>>>
>>> In this latest round, we had three different reviewers chime in so 
>>> we're
>>> doing delta webrevs for each of those resolutions:
>>>
>>> David H's resolutions:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.09,10.10.0-delta/ 
>>>
>>>
>>> ? mq comment for dholmes_CR:
>>> ??? - Fix indents, trailing spaces and various typos.
>>> ??? - Add descriptions for the '_cnt', '_max' and '_times" fields,
>>> ????? add impl notes to document the type choices.
>>>
>>> ? src/hotspot/share/runtime/globals.hpp change is white-space only 
>>> so it
>>> ? is only visible via the file's patch link.
>>>
>>>
>>> Robin W's resolutions:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.10.0,10.10.1-delta/ 
>>>
>>>
>>> ? mq comment for robinw_CR:
>>> ??? - Fix some inefficient code, update some comments, fix some 
>>> indents,
>>> ????? and add some 'const' specifiers.
>>>
>>> ? src/hotspot/share/runtime/vm_operations.hpp change is white-space 
>>> only so
>>> ? it is only visible via the file's patch link.
>>>
>>>
>>> Coleen's resolutions:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.10.1,10.10.2-delta/ 
>>>
>>>
>>> ? mq comment for coleenp_CR:
>>> ??? - Change ThreadsList::_threads from 'mtGC' -> 'mtThread',
>>> ??? - add header comment to threadSMR.hpp file,
>>> ??? - cleanup JavaThreadIteratorWithHandle ctr,
>>> ??? - make ErrorHandling more efficient.
>>>
>>>
>>> Some folks prefer to see all of the delta webrevs together so here is
>>> a delta webrev relative to jdk10-09-full:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.09,10.10.2-delta/ 
>>>
>>>
>>> ? src/hotspot/share/runtime/globals.hpp and
>>> ? src/hotspot/share/runtime/vm_operations.hpp changes are 
>>> white-space only
>>> ? so they are only visible via the file's patch link.
>>>
>>>
>>> And, of course, some folks prefer to see everything all at once so here
>>> is the latest full webrev:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.10-full/
>>>
>>>
>>> Dan has submitted the bits for the usual Mach5 tier[1-5] testing. The
>>> prelim Mach5 tier1 (JPRT equivalent) on these bits passed just fine...
>>>
>>> The project is currently baselined on Jesper's 2017-11-17 PIT snapshot
>>> so there will be some catching up to do before integration...
>>>
>>> We welcome comments, suggestions and feedback.
>>>
>>> Dan, Erik and Robbin
>>>
>>>
>>> On 11/18/17 8:49 PM, Daniel D. Daugherty wrote:
>>>> Greetings,
>>>>
>>>> Testing of the last round of changes revealed a hang in one of the new
>>>> TLH tests. Robbin has fixed the hang, updated the existing TLH 
>>>> test, and
>>>> added another TLH test for good measure.
>>>>
>>>> Here is the updated full webrev:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/
>>>>
>>>> Here is the updated delta webrev:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-delta/
>>>>
>>>> Dan ran the bits thru the usual Mach5 tier[1-5] testing and there are
>>>> no unexpected failures.
>>>>
>>>> We welcome comments, suggestions and feedback.
>>>>
>>>> Dan, Erik and Robbin
>>>>
>>>>
>>>> On 11/15/17 3:32 PM, Daniel D. Daugherty wrote:
>>>>> Greetings,
>>>>>
>>>>> Robbin rebased the project last night/this morning to merge with 
>>>>> Thread
>>>>> Local Handshakes (TLH) and also picked up additional changesets up 
>>>>> thru:
>>>>>
>>>>>> Changeset: fa736014cf28
>>>>>> Author:??? cjplummer
>>>>>> Date:????? 2017-11-14 18:08 -0800
>>>>>> URL:http://hg.openjdk.java.net/jdk/hs/rev/fa736014cf28
>>>>>>
>>>>>> 8191049: Add alternate version of pns() that is callable from 
>>>>>> within hotspot source
>>>>>> Summary: added pns2() to debug.cpp
>>>>>> Reviewed-by: stuefe, gthornbr
>>>>>
>>>>> This is the first time we've rebased the project to bits that are 
>>>>> this
>>>>> fresh (< 12 hours old at rebase time). We've done this because we 
>>>>> think
>>>>> we're done with this project and are in the final 
>>>>> review-change-retest
>>>>> cycle(s)... In other words, we're not planning on making any more 
>>>>> major
>>>>> changes for this project... :-)
>>>>>
>>>>> *** Time for code reviewers to chime in on this thread! ***
>>>>>
>>>>> Here is the updated full webrev:
>>>>>
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-full/
>>>>>
>>>>> Here's is the delta webrev from the 2017.11.10 rebase:
>>>>>
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-delta/
>>>>>
>>>>> Dan has submitted the bits for the usual Mach5 tier[1-5] testing
>>>>> (and the new baseline also)...
>>>>>
>>>>> We're expecting this round to be a little noisier than usual because
>>>>> we did not rebase on a PIT snapshot so the new baseline has not been
>>>>> through Jesper's usual care-and-feeding of integration_blockers, etc.
>>>>>
>>>>> We welcome comments, suggestions and feedback.
>>>>>
>>>>> Dan, Erik and Robbin
>>>>>
>>>>>
>>>>> On 11/14/17 4:48 PM, Daniel D. Daugherty wrote:
>>>>>> Greetings,
>>>>>>
>>>>>> I rebased the project to the 2017.11.10 jdk/hs PIT snapshot.
>>>>>> (Yes, we're playing chase-the-repo...)
>>>>>>
>>>>>> Here is the updated full webrev:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-07-full/
>>>>>>
>>>>>> Unlike the previous rebase, there were no changes required to the
>>>>>> open code to get the rebased bits to build so there is no delta
>>>>>> webrev.
>>>>>>
>>>>>> These bits have been run through JPRT and I've submitted the usual
>>>>>> Mach5 tier[1-5] test run...
>>>>>>
>>>>>> We welcome comments, suggestions and feedback.
>>>>>>
>>>>>> Dan, Erik and Robbin
>>>>>>
>>>>>>
>>>>>> On 11/13/17 12:30 PM, Daniel D. Daugherty wrote:
>>>>>>> Greetings,
>>>>>>>
>>>>>>> I rebased the project to the 2017.10.26 jdk10/hs PIT snapshot.
>>>>>>>
>>>>>>> Here are the updated webrevs:
>>>>>>>
>>>>>>> Here's the mq comment for the change:
>>>>>>>
>>>>>>> ? Rebase to 2017.10.25 PIT snapshot.
>>>>>>>
>>>>>>> Here is the full webrev:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-full/
>>>>>>>
>>>>>>> And here is the delta webrev:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-delta/
>>>>>>>
>>>>>>> I ran the above bits throught Mach5 tier[1-5] testing over the 
>>>>>>> holiday
>>>>>>> weekend. Didn't see any issues in a quick look. Going to take a 
>>>>>>> closer
>>>>>>> look today.
>>>>>>>
>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>
>>>>>>> Dan, Erik and Robbin
>>>>>>>
>>>>>>>
>>>>>>> On 11/8/17 1:05 PM, Daniel D. Daugherty wrote:
>>>>>>>> Greetings,
>>>>>>>>
>>>>>>>> Resolving one of the code review comments (from both Stefan K 
>>>>>>>> and Coleen)
>>>>>>>> on jdk10-04-full required quite a few changes so it is being 
>>>>>>>> done as a
>>>>>>>> standalone patch and corresponding webrevs:
>>>>>>>>
>>>>>>>> Here's the mq comment for the change:
>>>>>>>>
>>>>>>>> ? stefank, coleenp CR - refactor most JavaThreadIterator usage 
>>>>>>>> to use
>>>>>>>> ??? JavaThreadIteratorWithHandle.
>>>>>>>>
>>>>>>>> Here is the full webrev:
>>>>>>>>
>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-full/
>>>>>>>>
>>>>>>>> And here is the delta webrev:
>>>>>>>>
>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-delta/
>>>>>>>>
>>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>>
>>>>>>>> Dan, Erik and Robbin
>>>>>>>>
>>>>>>>>
>>>>>>>> On 10/9/17 3:41 PM, Daniel D. Daugherty wrote:
>>>>>>>>> Greetings,
>>>>>>>>>
>>>>>>>>> We have a (eXtra Large) fix for the following bug:
>>>>>>>>>
>>>>>>>>> 8167108 inconsistent handling of SR_lock can lead to crashes
>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8167108
>>>>>>>>>
>>>>>>>>> This fix adds a Safe Memory Reclamation (SMR) mechanism based on
>>>>>>>>> Hazard Pointers to manage JavaThread lifecycle.
>>>>>>>>>
>>>>>>>>> Here's a PDF for the internal wiki that we've been using to 
>>>>>>>>> describe
>>>>>>>>> and track the work on this project:
>>>>>>>>>
>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/SMR_and_JavaThread_Lifecycle-JDK10-04.pdf 
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Dan has noticed that the indenting is wrong in some of the 
>>>>>>>>> code quotes
>>>>>>>>> in the PDF that are not present in the internal wiki. We don't 
>>>>>>>>> have a
>>>>>>>>> solution for that problem yet.
>>>>>>>>>
>>>>>>>>> Here's the webrev for current JDK10 version of this fix:
>>>>>>>>>
>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-04-full
>>>>>>>>>
>>>>>>>>> This fix has been run through many rounds of JPRT and Mach5 
>>>>>>>>> tier[2-5]
>>>>>>>>> testing, additional stress testing on Dan's Solaris X64 
>>>>>>>>> server, and
>>>>>>>>> additional testing on Erik and Robbin's machines.
>>>>>>>>>
>>>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>>>
>>>>>>>>> Daniel Daugherty
>>>>>>>>> Erik Osterlund
>>>>>>>>> Robbin Ehn
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>



From thomas.schatzl at oracle.com  Thu Nov 23 13:38:46 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 23 Nov 2017 14:38:46 +0100
Subject: Need sponsor to push attached 8185591 into
 http://hg.openjdk.java.net/jdk10/hs
In-Reply-To: <9dbf43c5-116f-552e-fcfc-9672830902fb@oracle.com>
References: <9dbf43c5-116f-552e-fcfc-9672830902fb@oracle.com>
Message-ID: <1511444326.2477.0.camel@oracle.com>

Hi,

On Wed, 2017-11-22 at 19:25 -0500, Alexander Harlap wrote:
> I need a sponsor to push attached 8185591.open.patch - 
> guarantee(_byte_map[_guard_index] == last_card) failed: card table
> guard 
> has been modified.
> 
> Patch should go into jdk10/hs.
> 
> Reviewed by Thomas Schatzl  and Kim Barrett.
> 

  I will sponsor.

Thomas


From nils.eliasson at oracle.com  Thu Nov 23 21:59:48 2017
From: nils.eliasson at oracle.com (Nils Eliasson)
Date: Thu, 23 Nov 2017 22:59:48 +0100
Subject: RFR(L): 8186027: C2: loop strip mining
In-Reply-To: <dk6shd5t6x7.fsf@rwestrel.remote.csb>
References: <dk6k20cxtak.fsf@rwestrel.remote.csb>
 <35a024b5-6632-0488-a001-14f960257fc7@oracle.com>
 <dk67evjnvww.fsf@rwestrel.remote.csb>
 <8b9f8985-daae-b3b4-ba48-3d9c6185ed95@oracle.com>
 <0972a0db-2115-daa3-9990-7d58915a74a5@oracle.com>
 <dk61sktw8hn.fsf@rwestrel.remote.csb>
 <3e3e2048-5ee5-f3c2-8ec0-cdd36cff07af@oracle.com>
 <dk64lpmuzrg.fsf@rwestrel.remote.csb>
 <7ae8403a-b650-5171-5d79-0107d7d99059@oracle.com>
 <dk6shd5t6x7.fsf@rwestrel.remote.csb>
Message-ID: <f2a0b0c2-6642-785c-0e47-9d62ad8dae9f@oracle.com>

Hi,


On 2017-11-23 15:18, Roland Westrelin wrote:
> Hi Vladimir,
>
>> I am running testing again. But if this will repeat and presence of this
>> Sparse.small regression suggesting to me that may be we should keep this
>> optimization off by default - keep UseCountedLoopSafepoints false.
>>
>> We may switch it on later with additional changes which address regressions.
>>
>> What do you think?
> If the inner loop runs for a small number of iterations and the compiler
> can't statically prove it, I don't see a way to remove the overhead of
> loop strip mining entirely. So I'm not optimistic the regression can be
> fixed.
Agreed. In other words: Loop strip mining adds a guarantee that 
time-to-safepoint won't be too long, and that has a small cost

The current situation is that we have some extra performance with 
UseCountedLoopSafepoints default off, but let some users have a bad 
experience when they encounter long time-to-safepoint times or failures 
(https://bugs.openjdk.java.net/browse/JDK-5014723). I rather turn the 
table and have loop strip mining on, and let the power users experiment 
with turning it off for any uncertain performance boost.

> If loop strip mining defaults to false, would there we be any regular
> testing on your side?
We would have to add some.
>
> It seems to me that it would make sense to enable loop strip mining
> depending on what GC is used: it makes little sense for parallel gc but
> we'll want it enabled for Shenandoah for instance. Where does G1 fit? I
> can't really say and I don't have a strong opinion. But as I understand,
> G1 was made default under the assumption that users would be ok trading
> throughput for better latency. Maybe, that same reasoning applies to
> loop strip mining?

Scimark.sparse.small show a regression, but having long 
time-to-safepoint has a throughput cost in some settings like the 
companion benchmark scimark.sparse.large. Numbers using G1:

-XX:-UseCountedLoopSafepoints (default) ~86 ops/m
-XX:+UseCountedLoopSafepoints ~106 ops/m
-XX:+UseCountedLoopSafepoints -XX:LoopStripMiningIter=1000 ~111 ops/m

I would prefer having it on by default, at least in G1. Let's ask the G1 
GC-team on their opinion.

// Nils

>
> Roland.



From thomas.schatzl at oracle.com  Fri Nov 24 09:22:23 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Fri, 24 Nov 2017 10:22:23 +0100
Subject: RFR(L): 8186027: C2: loop strip mining
In-Reply-To: <f2a0b0c2-6642-785c-0e47-9d62ad8dae9f@oracle.com>
References: <dk6k20cxtak.fsf@rwestrel.remote.csb>
 <35a024b5-6632-0488-a001-14f960257fc7@oracle.com>
 <dk67evjnvww.fsf@rwestrel.remote.csb>
 <8b9f8985-daae-b3b4-ba48-3d9c6185ed95@oracle.com>
 <0972a0db-2115-daa3-9990-7d58915a74a5@oracle.com>
 <dk61sktw8hn.fsf@rwestrel.remote.csb>
 <3e3e2048-5ee5-f3c2-8ec0-cdd36cff07af@oracle.com>
 <dk64lpmuzrg.fsf@rwestrel.remote.csb>
 <7ae8403a-b650-5171-5d79-0107d7d99059@oracle.com>
 <dk6shd5t6x7.fsf@rwestrel.remote.csb>
 <f2a0b0c2-6642-785c-0e47-9d62ad8dae9f@oracle.com>
Message-ID: <1511515343.2425.11.camel@oracle.com>

Hi Nils,

On Thu, 2017-11-23 at 22:59 +0100, Nils Eliasson wrote:
> Hi,
> 
> 
> On 2017-11-23 15:18, Roland Westrelin wrote:
> > Hi Vladimir,
> > 
> > > I am running testing again. But if this will repeat and presence
> > > of this Sparse.small regression suggesting to me that may be we
> > > should keep this optimization off by default - keep
> > > UseCountedLoopSafepoints false.
> > > 
> > > We may switch it on later with additional changes which address
> > > regressions.
> > > 
> > > What do you think?
> > 
> > If the inner loop runs for a small number of iterations and the
> > compiler can't statically prove it, I don't see a way to remove the
> > overhead of loop strip mining entirely. So I'm not optimistic the
> > regression can be fixed.
> 
> Agreed. In other words: Loop strip mining adds a guarantee that 
> time-to-safepoint won't be too long, and that has a small cost
> 
> The current situation is that we have some extra performance with 
> UseCountedLoopSafepoints default off, but let some users have a bad 
> experience when they encounter long time-to-safepoint times or
> failures (https://bugs.openjdk.java.net/browse/JDK-5014723). I rather
> turn the table and have loop strip mining on, and let the power users
> experiment with turning it off for any uncertain performance boost.
> 
> > If loop strip mining defaults to false, would there we be any
> > regular testing on your side?
> 
> We would have to add some.
> > 
> > It seems to me that it would make sense to enable loop strip mining
> > depending on what GC is used: it makes little sense for parallel gc
> > but we'll want it enabled for Shenandoah for instance. Where does
> > G1 fit? I can't really say and I don't have a strong opinion. But
> > as I understand, G1 was made default under the assumption that
> > users would be ok trading throughput for better latency. Maybe,
> > that same reasoning applies to loop strip mining?
> 
> Scimark.sparse.small show a regression, but having long 
> time-to-safepoint has a throughput cost in some settings like the 
> companion benchmark scimark.sparse.large. Numbers using G1:
> 
> -XX:-UseCountedLoopSafepoints (default) ~86 ops/m
> -XX:+UseCountedLoopSafepoints ~106 ops/m
> -XX:+UseCountedLoopSafepoints -XX:LoopStripMiningIter=1000 ~111 ops/m
> 
> I would prefer having it on by default, at least in G1. Let's ask the
> G1 GC-team on their opinion.

  our perf team uses -XX:+UseCountedLoopSafepoints for _all_ collectors
 for some time now. When asked, they replied that predictability of
results is very important for them too.

We also closed out some perf regressions (e.g. JDK-8177704) due to the
problems with -XX:-UseCountedLoopSafepoints after you posted these
results in agreement with the perf team.

So I am all for making it default for G1 (and I am sure others agree),
if not for all GCs. However I recommend having a separate CR for
changing the defaults. Makes it easier reverting it in case things go
wrong.

Thanks,
  Thomas



From stefan.karlsson at oracle.com  Fri Nov 24 13:38:38 2017
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Fri, 24 Nov 2017 14:38:38 +0100
Subject: RFR: 8191861: Move and refactor hSpaceCounters
Message-ID: <7d8a576b-3661-2587-22bf-88706357ddc3@oracle.com>

Hi all,

Please review this patch to move and refactor HSpaceCounters.

http://cr.openjdk.java.net/~stefank/8191861/webrev.01
https://bugs.openjdk.java.net/browse/JDK-8191861

The patch moves hSpaceCounters.hpp/cpp out from gc/g1 into gc/shared, so 
that it can be reused by other GCs.

The patch also removes the dependency between HSpaceCounters and 
GenerationCounters. The passed in GenerationCounters object was only 
used to provide the name_space of the perf counter. This patch changes 
G1 to explicitly pass in the name_space string instead of the 
GenerationCounters.

This patch builds upon these two patches:
 
http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-November/029267.html
 
http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-November/029268.html

Thanks,
StefanK


From per.liden at oracle.com  Fri Nov 24 14:05:53 2017
From: per.liden at oracle.com (Per Liden)
Date: Fri, 24 Nov 2017 15:05:53 +0100
Subject: RFR: 8191861: Move and refactor hSpaceCounters
In-Reply-To: <7d8a576b-3661-2587-22bf-88706357ddc3@oracle.com>
References: <7d8a576b-3661-2587-22bf-88706357ddc3@oracle.com>
Message-ID: <270f5cdc-83bc-b054-9a7d-9f0636d12359@oracle.com>

Looks good!

/Per

On 2017-11-24 14:38, Stefan Karlsson wrote:
> Hi all,
>
> Please review this patch to move and refactor HSpaceCounters.
>
> http://cr.openjdk.java.net/~stefank/8191861/webrev.01
> https://bugs.openjdk.java.net/browse/JDK-8191861
>
> The patch moves hSpaceCounters.hpp/cpp out from gc/g1 into gc/shared, so
> that it can be reused by other GCs.
>
> The patch also removes the dependency between HSpaceCounters and
> GenerationCounters. The passed in GenerationCounters object was only
> used to provide the name_space of the perf counter. This patch changes
> G1 to explicitly pass in the name_space string instead of the
> GenerationCounters.
>
> This patch builds upon these two patches:
>
> http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-November/029267.html
>
>
> http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-November/029268.html
>
>
> Thanks,
> StefanK


From erik.helin at oracle.com  Fri Nov 24 14:11:12 2017
From: erik.helin at oracle.com (Erik Helin)
Date: Fri, 24 Nov 2017 15:11:12 +0100
Subject: RFR: 8191861: Move and refactor hSpaceCounters
In-Reply-To: <7d8a576b-3661-2587-22bf-88706357ddc3@oracle.com>
References: <7d8a576b-3661-2587-22bf-88706357ddc3@oracle.com>
Message-ID: <951da5b1-52bb-bb92-bef6-88d9ad033966@oracle.com>

On 11/24/2017 02:38 PM, Stefan Karlsson wrote:
> Hi all,
> 
> Please review this patch to move and refactor HSpaceCounters.
> 
> http://cr.openjdk.java.net/~stefank/8191861/webrev.01
> https://bugs.openjdk.java.net/browse/JDK-8191861
> 
> The patch moves hSpaceCounters.hpp/cpp out from gc/g1 into gc/shared, so 
> that it can be reused by other GCs.

Looks good, but could you please add a local variable for the expression

_young_collection_counters->name_space()

that is repeated three times now in g1MonitoringSupport.cpp. I don't 
need a re-review for this small additional change.

Thanks,
Erik

> The patch also removes the dependency between HSpaceCounters and 
> GenerationCounters. The passed in GenerationCounters object was only 
> used to provide the name_space of the perf counter. This patch changes 
> G1 to explicitly pass in the name_space string instead of the 
> GenerationCounters.
> 
> This patch builds upon these two patches:
> 
> http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-November/029267.html 
> 
> 
> http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-November/029268.html 
> 
> 
> Thanks,
> StefanK


From rkennke at redhat.com  Fri Nov 24 14:11:55 2017
From: rkennke at redhat.com (Roman Kennke)
Date: Fri, 24 Nov 2017 15:11:55 +0100
Subject: RFR: 8191861: Move and refactor hSpaceCounters
In-Reply-To: <7d8a576b-3661-2587-22bf-88706357ddc3@oracle.com>
References: <7d8a576b-3661-2587-22bf-88706357ddc3@oracle.com>
Message-ID: <B4576513-4160-47E4-AAFB-E843710D09C9@redhat.com>

Looks good to me too.

This is welcome because it's one oft the parts that Shenandoah uses from G1.

Thanks, Roman


Am 24. November 2017 14:38:38 MEZ schrieb Stefan Karlsson <stefan.karlsson at oracle.com>:
>Hi all,
>
>Please review this patch to move and refactor HSpaceCounters.
>
>http://cr.openjdk.java.net/~stefank/8191861/webrev.01
>https://bugs.openjdk.java.net/browse/JDK-8191861
>
>The patch moves hSpaceCounters.hpp/cpp out from gc/g1 into gc/shared,
>so 
>that it can be reused by other GCs.
>
>The patch also removes the dependency between HSpaceCounters and 
>GenerationCounters. The passed in GenerationCounters object was only 
>used to provide the name_space of the perf counter. This patch changes 
>G1 to explicitly pass in the name_space string instead of the 
>GenerationCounters.
>
>This patch builds upon these two patches:
> 
>http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-November/029267.html
> 
>http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-November/029268.html
>
>Thanks,
>StefanK

-- 
Diese Nachricht wurde von meinem Android-Ger?t mit K-9 Mail gesendet.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20171124/3ef89ac2/attachment.htm>

From per.liden at oracle.com  Fri Nov 24 14:20:06 2017
From: per.liden at oracle.com (Per Liden)
Date: Fri, 24 Nov 2017 15:20:06 +0100
Subject: RFR: 8191862: Warn about UseNUMA/UseLargePages only when using
 ParallelGC
Message-ID: <f63cc912-5ac8-f9df-299d-964e44debecf@oracle.com>

Hi,

We currently warn about using UseAdaptiveSizePolicy and 
UseAdaptiveNUMAChunkSizing in combination with UseNUMA and 
UseLargePages. This warning is only relevant for ParallelGC, but we 
print it regardless of which GC has been selected.

Webrev: http://cr.openjdk.java.net/~pliden/8191862/webrev.0/
Bug: https://bugs.openjdk.java.net/browse/JDK-8191862

/Per


From per.liden at oracle.com  Fri Nov 24 14:55:22 2017
From: per.liden at oracle.com (Per Liden)
Date: Fri, 24 Nov 2017 15:55:22 +0100
Subject: RFR(s): 8191864: Provide a public destructor for WorkGang
Message-ID: <ffd347e5-8804-25d7-474e-4a3378507bbe@oracle.com>

Hi,

WorkGang's destructor is currently declared private, meaning a WorkGang 
can't be deleted, instantiated as a value or placed on the stack. This 
patch adds a public destructor. I also added a protected destructor on 
AbstractWorkGang to guard against attempts to delete using a 
AbstractWorkGang pointer, which would require a virtual destructor.

Webrev: http://cr.openjdk.java.net/~pliden/8191864/
Bug: https://bugs.openjdk.java.net/browse/JDK-8191864

/Per


From erik.helin at oracle.com  Fri Nov 24 15:38:38 2017
From: erik.helin at oracle.com (Erik Helin)
Date: Fri, 24 Nov 2017 16:38:38 +0100
Subject: RFR: 8080345: With perm gen gone, perfdata counter
 sun.gc.policy.generations should be 2, not 3
Message-ID: <d3482f10-3409-598a-1883-6ae4c844007f@oracle.com>

Hi all,

I picked up this patch that Ramki posted quite a while ago: when I 
updated the tooling I forgot to update the performance counter 
'sun.gc.policy.generations'. This patch takes care of this problem, 
sun.gc.policy.generations now reports 2 instead of 3.

I also added a test that verifies the fix. As part of adding the test I 
had to move the test support classes PerfCounter.java and 
PerfCounters.java from test/hotspot/jtreg/gc/metaspace to 
test/hotspot/jtreg/gc/testlibrary.

Bug:
https://bugs.openjdk.java.net/browse/JDK-8080345

Patch:
http://cr.openjdk.java.net/~ehelin/8080345/00/

Testing:
- hs tier1-5
- newly added jtreg test
- jstat and jstatd tests under test/jdk/sun/tools/jstat{d}

Thanks,
Erik


From thomas.schatzl at oracle.com  Fri Nov 24 16:03:01 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Fri, 24 Nov 2017 17:03:01 +0100
Subject: RFR: 8080345: With perm gen gone, perfdata counter
 sun.gc.policy.generations should be 2, not 3
In-Reply-To: <d3482f10-3409-598a-1883-6ae4c844007f@oracle.com>
References: <d3482f10-3409-598a-1883-6ae4c844007f@oracle.com>
Message-ID: <1511539381.2425.13.camel@oracle.com>

Hi,

On Fri, 2017-11-24 at 16:38 +0100, Erik Helin wrote:
> Hi all,
> 
> I picked up this patch that Ramki posted quite a while ago: when I 
> updated the tooling I forgot to update the performance counter 
> 'sun.gc.policy.generations'. This patch takes care of this problem, 
> sun.gc.policy.generations now reports 2 instead of 3.
> 
> I also added a test that verifies the fix. As part of adding the test
> I 
> had to move the test support classes PerfCounter.java and 
> PerfCounters.java from test/hotspot/jtreg/gc/metaspace to 
> test/hotspot/jtreg/gc/testlibrary.
> 
> Bug:
> https://bugs.openjdk.java.net/browse/JDK-8080345
> 
> Patch:
> http://cr.openjdk.java.net/~ehelin/8080345/00/
> 
> Testing:
> - hs tier1-5
> - newly added jtreg test
> - jstat and jstatd tests under test/jdk/sun/tools/jstat{d}

  looks good.

Thomas



From daniel.daugherty at oracle.com  Fri Nov 24 18:13:20 2017
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Fri, 24 Nov 2017 13:13:20 -0500
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <567339a6-e36a-5416-5c34-2a87cbfa6cdd@oracle.com>
References: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
 <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>
 <836b5c6c-5eef-9245-c16a-e71372093d9a@oracle.com>
 <82d0e473-a24f-9feb-82e9-7d04c4d7a91d@oracle.com>
 <354f7ece-66bc-c065-9ee8-f55abcc1c23e@oracle.com>
 <0ff8df8e-e93c-02d2-cb97-b13b1ac0f53f@oracle.com>
 <5365606e-4036-b5d7-9bac-e91e42f38d01@oracle.com>
 <567339a6-e36a-5416-5c34-2a87cbfa6cdd@oracle.com>
Message-ID: <f5d21f65-ae84-3366-aaf9-6f5a336e97a9@oracle.com>

Greetings,

The Thread-SMR bits were pushed to jdk/hs late last night!

Mach5 Tier[1-5] on the exact bits that were pushed showed
no unexpected failures. I've looked at the jdk/hs CI pipeline
test results for my push and for the push before and after
mine and I don't see anything that worries me there either.

Obviously I'll be keeping an eye on testing for the next
several days, but so far everything looks good!!

Dan


On 11/22/17 9:16 PM, Daniel D. Daugherty wrote:
> Greetings,
>
> I've made minor tweaks to the Thread-SMR project based on the remaining
> code review comments over the last couple of days. I've also rebased the
> project to jdk/hs bits as of mid-afternoon (my TZ) on 2017.11.22. I'm
> running baseline Mach5 Tier[1-5] testing and prototype Mach5 Tier[1-5]
> testing...
>
> Here's a delta webrev for the latest round of tweaks:
>
> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-11-delta
>
>
> Here's the proposed commit message:
>
> $ cat SMR_prototype_10/open/commit.txt
> 8167108: inconsistent handling of SR_lock can lead to crashes
> Summary: Add Thread Safe Memory Reclamation (Thread-SMR) mechanism.
> Reviewed-by: coleenp, dcubed, dholmes, eosterlund, gthornbr, kbarrett, 
> rehn, sspitsyn, stefank
> Contributed-by: daniel.daugherty at oracle.com, 
> erik.osterlund at oracle.com, robbin.ehn at oracle.com
>
> I've gone through 880+ emails in my archive for this project and added
> anyone that made a code review comment. Reviewers are not in my usual
> by-comment-posting order; it was way too hard to figure that out... So
> reviewers and contributors are simply in alpha-sort order...
>
> Here's the collection of follow-up bugs that I filed based on sifting
> through the email archive:
>
> 8191784 JavaThread::collect_counters() should stop using Threads_lock
> https://bugs.openjdk.java.net/browse/JDK-8191784
>
> 8191785 revoke_bias() needs a new assertion
> https://bugs.openjdk.java.net/browse/JDK-8191785
>
> 8191786 Thread-SMR hash table size should be dynamic
> https://bugs.openjdk.java.net/browse/JDK-8191786
>
> 8191787 move private inline functions from thread.inline.hpp -> 
> thread.cpp
> https://bugs.openjdk.java.net/browse/JDK-8191787
>
> 8191789 migrate more Thread-SMR stuff from thread.[ch]pp -> 
> threadSMR.[ch]pp
> https://bugs.openjdk.java.net/browse/JDK-8191789
>
> 8191798 redo nested ThreadsListHandle to drop Threads_lock
> https://bugs.openjdk.java.net/browse/JDK-8191789
>
> I have undoubtedly missed at least one future idea that someone had
> and for that my apologies...
>
> Thanks again to everyone who contributed to the Thread-SMR project!!
>
> Special thanks goes to Karen Kinnear, Kim Barrett and David Holmes for
> their rigorous review of the design that we documented on the wiki.
> Thanks for keeping us honest!
>
> I also have to thank my co-contributor Erik ?sterlund for starting the
> ball rolling on this crazy project with his presentation on Safe Memory
> Reclamation, for the initial prototype and for all your patches. Erik,
> hopefully we got far enough in your crazy vision... :-)
>
> Thanks to my co-contributor Robbin Ehn for proposing that we do early
> code reviews in the form of patches. Don't like something? Fix it with
> a patch and a new test if appropriate!! A pretty cool way to work that
> was new to me.
>
> Finally, many thanks to Jerry Thornbrugh for all the early code reviews,
> whiteboard chats and phone calls. Thanks for asking the right questions
> and pointing out some places where our logic was faulty. Also thanks for
> keeping me sane when I was literally on my own in Monument, CO.
>
> Dan
>
>
> On 11/21/17 7:12 PM, Daniel D. Daugherty wrote:
>> Greetings,
>>
>> *** We are wrapping up code review on this project so it is time ***
>> *** for the various code reviewers to chime in one last time. ***
>>
>> In this latest round, we had three different reviewers chime in so we're
>> doing delta webrevs for each of those resolutions:
>>
>> David H's resolutions:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.09,10.10.0-delta/ 
>>
>>
>> ? mq comment for dholmes_CR:
>> ??? - Fix indents, trailing spaces and various typos.
>> ??? - Add descriptions for the '_cnt', '_max' and '_times" fields,
>> ????? add impl notes to document the type choices.
>>
>> ? src/hotspot/share/runtime/globals.hpp change is white-space only so it
>> ? is only visible via the file's patch link.
>>
>>
>> Robin W's resolutions:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.10.0,10.10.1-delta/ 
>>
>>
>> ? mq comment for robinw_CR:
>> ??? - Fix some inefficient code, update some comments, fix some indents,
>> ????? and add some 'const' specifiers.
>>
>> ? src/hotspot/share/runtime/vm_operations.hpp change is white-space 
>> only so
>> ? it is only visible via the file's patch link.
>>
>>
>> Coleen's resolutions:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.10.1,10.10.2-delta/ 
>>
>>
>> ? mq comment for coleenp_CR:
>> ??? - Change ThreadsList::_threads from 'mtGC' -> 'mtThread',
>> ??? - add header comment to threadSMR.hpp file,
>> ??? - cleanup JavaThreadIteratorWithHandle ctr,
>> ??? - make ErrorHandling more efficient.
>>
>>
>> Some folks prefer to see all of the delta webrevs together so here is
>> a delta webrev relative to jdk10-09-full:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.09,10.10.2-delta/ 
>>
>>
>> ? src/hotspot/share/runtime/globals.hpp and
>> ? src/hotspot/share/runtime/vm_operations.hpp changes are white-space 
>> only
>> ? so they are only visible via the file's patch link.
>>
>>
>> And, of course, some folks prefer to see everything all at once so here
>> is the latest full webrev:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.10-full/
>>
>>
>> Dan has submitted the bits for the usual Mach5 tier[1-5] testing. The
>> prelim Mach5 tier1 (JPRT equivalent) on these bits passed just fine...
>>
>> The project is currently baselined on Jesper's 2017-11-17 PIT snapshot
>> so there will be some catching up to do before integration...
>>
>> We welcome comments, suggestions and feedback.
>>
>> Dan, Erik and Robbin
>>
>>
>> On 11/18/17 8:49 PM, Daniel D. Daugherty wrote:
>>> Greetings,
>>>
>>> Testing of the last round of changes revealed a hang in one of the new
>>> TLH tests. Robbin has fixed the hang, updated the existing TLH test, 
>>> and
>>> added another TLH test for good measure.
>>>
>>> Here is the updated full webrev:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/
>>>
>>> Here is the updated delta webrev:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-delta/
>>>
>>> Dan ran the bits thru the usual Mach5 tier[1-5] testing and there are
>>> no unexpected failures.
>>>
>>> We welcome comments, suggestions and feedback.
>>>
>>> Dan, Erik and Robbin
>>>
>>>
>>> On 11/15/17 3:32 PM, Daniel D. Daugherty wrote:
>>>> Greetings,
>>>>
>>>> Robbin rebased the project last night/this morning to merge with 
>>>> Thread
>>>> Local Handshakes (TLH) and also picked up additional changesets up 
>>>> thru:
>>>>
>>>>> Changeset: fa736014cf28
>>>>> Author:??? cjplummer
>>>>> Date:????? 2017-11-14 18:08 -0800
>>>>> URL:http://hg.openjdk.java.net/jdk/hs/rev/fa736014cf28
>>>>>
>>>>> 8191049: Add alternate version of pns() that is callable from 
>>>>> within hotspot source
>>>>> Summary: added pns2() to debug.cpp
>>>>> Reviewed-by: stuefe, gthornbr
>>>>
>>>> This is the first time we've rebased the project to bits that are this
>>>> fresh (< 12 hours old at rebase time). We've done this because we 
>>>> think
>>>> we're done with this project and are in the final review-change-retest
>>>> cycle(s)... In other words, we're not planning on making any more 
>>>> major
>>>> changes for this project... :-)
>>>>
>>>> *** Time for code reviewers to chime in on this thread! ***
>>>>
>>>> Here is the updated full webrev:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-full/
>>>>
>>>> Here's is the delta webrev from the 2017.11.10 rebase:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-delta/
>>>>
>>>> Dan has submitted the bits for the usual Mach5 tier[1-5] testing
>>>> (and the new baseline also)...
>>>>
>>>> We're expecting this round to be a little noisier than usual because
>>>> we did not rebase on a PIT snapshot so the new baseline has not been
>>>> through Jesper's usual care-and-feeding of integration_blockers, etc.
>>>>
>>>> We welcome comments, suggestions and feedback.
>>>>
>>>> Dan, Erik and Robbin
>>>>
>>>>
>>>> On 11/14/17 4:48 PM, Daniel D. Daugherty wrote:
>>>>> Greetings,
>>>>>
>>>>> I rebased the project to the 2017.11.10 jdk/hs PIT snapshot.
>>>>> (Yes, we're playing chase-the-repo...)
>>>>>
>>>>> Here is the updated full webrev:
>>>>>
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-07-full/
>>>>>
>>>>> Unlike the previous rebase, there were no changes required to the
>>>>> open code to get the rebased bits to build so there is no delta
>>>>> webrev.
>>>>>
>>>>> These bits have been run through JPRT and I've submitted the usual
>>>>> Mach5 tier[1-5] test run...
>>>>>
>>>>> We welcome comments, suggestions and feedback.
>>>>>
>>>>> Dan, Erik and Robbin
>>>>>
>>>>>
>>>>> On 11/13/17 12:30 PM, Daniel D. Daugherty wrote:
>>>>>> Greetings,
>>>>>>
>>>>>> I rebased the project to the 2017.10.26 jdk10/hs PIT snapshot.
>>>>>>
>>>>>> Here are the updated webrevs:
>>>>>>
>>>>>> Here's the mq comment for the change:
>>>>>>
>>>>>> ? Rebase to 2017.10.25 PIT snapshot.
>>>>>>
>>>>>> Here is the full webrev:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-full/
>>>>>>
>>>>>> And here is the delta webrev:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-delta/
>>>>>>
>>>>>> I ran the above bits throught Mach5 tier[1-5] testing over the 
>>>>>> holiday
>>>>>> weekend. Didn't see any issues in a quick look. Going to take a 
>>>>>> closer
>>>>>> look today.
>>>>>>
>>>>>> We welcome comments, suggestions and feedback.
>>>>>>
>>>>>> Dan, Erik and Robbin
>>>>>>
>>>>>>
>>>>>> On 11/8/17 1:05 PM, Daniel D. Daugherty wrote:
>>>>>>> Greetings,
>>>>>>>
>>>>>>> Resolving one of the code review comments (from both Stefan K 
>>>>>>> and Coleen)
>>>>>>> on jdk10-04-full required quite a few changes so it is being 
>>>>>>> done as a
>>>>>>> standalone patch and corresponding webrevs:
>>>>>>>
>>>>>>> Here's the mq comment for the change:
>>>>>>>
>>>>>>> ? stefank, coleenp CR - refactor most JavaThreadIterator usage 
>>>>>>> to use
>>>>>>> ??? JavaThreadIteratorWithHandle.
>>>>>>>
>>>>>>> Here is the full webrev:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-full/
>>>>>>>
>>>>>>> And here is the delta webrev:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-delta/
>>>>>>>
>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>
>>>>>>> Dan, Erik and Robbin
>>>>>>>
>>>>>>>
>>>>>>> On 10/9/17 3:41 PM, Daniel D. Daugherty wrote:
>>>>>>>> Greetings,
>>>>>>>>
>>>>>>>> We have a (eXtra Large) fix for the following bug:
>>>>>>>>
>>>>>>>> 8167108 inconsistent handling of SR_lock can lead to crashes
>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8167108
>>>>>>>>
>>>>>>>> This fix adds a Safe Memory Reclamation (SMR) mechanism based on
>>>>>>>> Hazard Pointers to manage JavaThread lifecycle.
>>>>>>>>
>>>>>>>> Here's a PDF for the internal wiki that we've been using to 
>>>>>>>> describe
>>>>>>>> and track the work on this project:
>>>>>>>>
>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/SMR_and_JavaThread_Lifecycle-JDK10-04.pdf 
>>>>>>>>
>>>>>>>>
>>>>>>>> Dan has noticed that the indenting is wrong in some of the code 
>>>>>>>> quotes
>>>>>>>> in the PDF that are not present in the internal wiki. We don't 
>>>>>>>> have a
>>>>>>>> solution for that problem yet.
>>>>>>>>
>>>>>>>> Here's the webrev for current JDK10 version of this fix:
>>>>>>>>
>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-04-full
>>>>>>>>
>>>>>>>> This fix has been run through many rounds of JPRT and Mach5 
>>>>>>>> tier[2-5]
>>>>>>>> testing, additional stress testing on Dan's Solaris X64 server, 
>>>>>>>> and
>>>>>>>> additional testing on Erik and Robbin's machines.
>>>>>>>>
>>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>>
>>>>>>>> Daniel Daugherty
>>>>>>>> Erik Osterlund
>>>>>>>> Robbin Ehn
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>



From stefan.karlsson at oracle.com  Mon Nov 27 08:56:02 2017
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Mon, 27 Nov 2017 09:56:02 +0100
Subject: RFR: 8191861: Move and refactor hSpaceCounters
In-Reply-To: <270f5cdc-83bc-b054-9a7d-9f0636d12359@oracle.com>
References: <7d8a576b-3661-2587-22bf-88706357ddc3@oracle.com>
 <270f5cdc-83bc-b054-9a7d-9f0636d12359@oracle.com>
Message-ID: <aa58ad4c-37a3-f09b-0f53-e89ae9d95751@oracle.com>

Thanks, Per!

StefanK

On 2017-11-24 15:05, Per Liden wrote:
> Looks good!
> 
> /Per
> 
> On 2017-11-24 14:38, Stefan Karlsson wrote:
>> Hi all,
>>
>> Please review this patch to move and refactor HSpaceCounters.
>>
>> http://cr.openjdk.java.net/~stefank/8191861/webrev.01
>> https://bugs.openjdk.java.net/browse/JDK-8191861
>>
>> The patch moves hSpaceCounters.hpp/cpp out from gc/g1 into gc/shared, so
>> that it can be reused by other GCs.
>>
>> The patch also removes the dependency between HSpaceCounters and
>> GenerationCounters. The passed in GenerationCounters object was only
>> used to provide the name_space of the perf counter. This patch changes
>> G1 to explicitly pass in the name_space string instead of the
>> GenerationCounters.
>>
>> This patch builds upon these two patches:
>>
>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-November/029267.html 
>>
>>
>>
>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-November/029268.html 
>>
>>
>>
>> Thanks,
>> StefanK


From stefan.karlsson at oracle.com  Mon Nov 27 08:56:43 2017
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Mon, 27 Nov 2017 09:56:43 +0100
Subject: RFR: 8191861: Move and refactor hSpaceCounters
In-Reply-To: <951da5b1-52bb-bb92-bef6-88d9ad033966@oracle.com>
References: <7d8a576b-3661-2587-22bf-88706357ddc3@oracle.com>
 <951da5b1-52bb-bb92-bef6-88d9ad033966@oracle.com>
Message-ID: <12c57543-8267-2188-2d96-6bd9615fa9fd@oracle.com>

On 2017-11-24 15:11, Erik Helin wrote:
> On 11/24/2017 02:38 PM, Stefan Karlsson wrote:
>> Hi all,
>>
>> Please review this patch to move and refactor HSpaceCounters.
>>
>> http://cr.openjdk.java.net/~stefank/8191861/webrev.01
>> https://bugs.openjdk.java.net/browse/JDK-8191861
>>
>> The patch moves hSpaceCounters.hpp/cpp out from gc/g1 into gc/shared, 
>> so that it can be reused by other GCs.
> 
> Looks good, but could you please add a local variable for the expression
> 
> _young_collection_counters->name_space()
> 
> that is repeated three times now in g1MonitoringSupport.cpp. I don't 
> need a re-review for this small additional change.

Done.

Thanks for the review!

StefanK

> 
> Thanks,
> Erik
> 
>> The patch also removes the dependency between HSpaceCounters and 
>> GenerationCounters. The passed in GenerationCounters object was only 
>> used to provide the name_space of the perf counter. This patch changes 
>> G1 to explicitly pass in the name_space string instead of the 
>> GenerationCounters.
>>
>> This patch builds upon these two patches:
>>
>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-November/029267.html 
>>
>>
>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-November/029268.html 
>>
>>
>> Thanks,
>> StefanK


From stefan.karlsson at oracle.com  Mon Nov 27 08:57:24 2017
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Mon, 27 Nov 2017 09:57:24 +0100
Subject: RFR: 8191861: Move and refactor hSpaceCounters
In-Reply-To: <B4576513-4160-47E4-AAFB-E843710D09C9@redhat.com>
References: <7d8a576b-3661-2587-22bf-88706357ddc3@oracle.com>
 <B4576513-4160-47E4-AAFB-E843710D09C9@redhat.com>
Message-ID: <52fa680b-0d94-ad2b-3b20-dd3c58a1aa24@oracle.com>

Thanks, Roman!

StefanK

On 2017-11-24 15:11, Roman Kennke wrote:
> Looks good to me too.
> 
> This is welcome because it's one oft the parts that Shenandoah uses from G1.
> 
> Thanks, Roman
> 
> 
> Am 24. November 2017 14:38:38 MEZ schrieb Stefan Karlsson 
> <stefan.karlsson at oracle.com>:
> 
>     Hi all,
> 
>     Please review this patch to move and refactor HSpaceCounters.
> 
>     http://cr.openjdk.java.net/~stefank/8191861/webrev.01
>     https://bugs.openjdk.java.net/browse/JDK-8191861
> 
>     The patch moves hSpaceCounters.hpp/cpp out from gc/g1 into gc/shared, so
>     that it can be reused by other GCs.
> 
>     The patch also removes the dependency between HSpaceCounters and
>     GenerationCounters. The passed in GenerationCounters object was only
>     used to provide the name_space of the perf counter. This patch changes
>     G1 to explicitly pass in the name_space string instead of the
>     GenerationCounters.
> 
>     This patch builds upon these two patches:
>       
>     http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-November/029267.html
>       
>     http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-November/029268.html
> 
>     Thanks,
>     StefanK
> 
> 
> -- 
> Diese Nachricht wurde von meinem Android-Ger?t mit K-9 Mail gesendet.


From erik.osterlund at oracle.com  Mon Nov 27 09:06:59 2017
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Mon, 27 Nov 2017 10:06:59 +0100
Subject: RFR (S): 8191888: Refactor ClassLoaderData::remove_handle to use the
 Access API
Message-ID: <5A1BD5B3.7030808@oracle.com>

Hi,

The ClassLoaderData::remove_handle() member function currently uses 
explicit G1 SATB barriers to remove an oop from the root set, as these 
handles are not necessarily walked by the GC in a safepoint. Therefore 
G1 needs pre-write barriers.

This should now be modeled as a 
RootAccess<IN_CONCURRENT_ROOT>::oop_store instead. This maps to 
performing a pre-write SATB barrier with G1, but other GCs are free to 
do other things as necessary.

Webrev:
http://cr.openjdk.java.net/~eosterlund/8191888/webrev.00/

Bug:
https://bugs.openjdk.java.net/browse/JDK-8191888

Thanks,
/Erik


From stefan.johansson at oracle.com  Mon Nov 27 09:05:37 2017
From: stefan.johansson at oracle.com (Stefan Johansson)
Date: Mon, 27 Nov 2017 10:05:37 +0100
Subject: RFR: 8080345: With perm gen gone, perfdata counter
 sun.gc.policy.generations should be 2, not 3
In-Reply-To: <d3482f10-3409-598a-1883-6ae4c844007f@oracle.com>
References: <d3482f10-3409-598a-1883-6ae4c844007f@oracle.com>
Message-ID: <fb4516ee-b838-5f7b-8fe7-f919b9701a6e@oracle.com>

Hi Erik,

Thanks for taking care of this patch.

On 2017-11-24 16:38, Erik Helin wrote:
> Hi all,
>
> I picked up this patch that Ramki posted quite a while ago: when I 
> updated the tooling I forgot to update the performance counter 
> 'sun.gc.policy.generations'. This patch takes care of this problem, 
> sun.gc.policy.generations now reports 2 instead of 3.
>
> I also added a test that verifies the fix. As part of adding the test 
> I had to move the test support classes PerfCounter.java and 
> PerfCounters.java from test/hotspot/jtreg/gc/metaspace to 
> test/hotspot/jtreg/gc/testlibrary.
>
> Bug:
> https://bugs.openjdk.java.net/browse/JDK-8080345
>
> Patch:
> http://cr.openjdk.java.net/~ehelin/8080345/00/
>
Looks good,
Stefan

> Testing:
> - hs tier1-5
> - newly added jtreg test
> - jstat and jstatd tests under test/jdk/sun/tools/jstat{d}
>
> Thanks,
> Erik



From thomas.schatzl at oracle.com  Mon Nov 27 09:30:28 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 27 Nov 2017 10:30:28 +0100
Subject: RFR (XS): 8188877: Improper synchronization in offer_termination
In-Reply-To: <ba0c0855-188a-decf-932b-b11200dc9fc4@redhat.com>
References: <CY1PR0701MB16328948D4FDF19FBC30F65684710@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <3229BC22-BB8A-4D3D-8440-AB799AB5394F@oracle.com>
 <CY1PR0701MB16328E4C6563BE60CF8E99D084750@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <4D9972BA-B76C-4FB7-B24D-31D507DB1A6A@oracle.com>
 <1510821711.5332.3.camel@oracle.com>
 <CY1PR0701MB163288CD9B58A163B2F16F7C84230@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <ba0c0855-188a-decf-932b-b11200dc9fc4@redhat.com>
Message-ID: <1511775028.2398.3.camel@oracle.com>

Hi all,

On Wed, 2017-11-22 at 09:13 +0000, Andrew Haley wrote:
> On 21/11/17 21:53, White, Derek wrote:
> > My understanding is that the "acquire" semantics are entirely about
> > memory ordering, within a CPU. In particular it prevents "following
> > loads" from executing before the "load acquire".
> > 
> > 
> > There is nothing in the "load acquire" that causes it to
> > synchronize
> > with the memory system more or less quickly than a naked load.
> 
> The abstract architecture only specifies things in terms of ordering
> between loads, but it has to be implemented somehow, and this is MESI
> or something similar.  Stores cause invalidate messages to be sent,
> and these are put into the reader's invalidate queue.  When that
> reader executes a load barrier it marks all the entries currently in
> its invalidate queue.  The next load will wait until all marked
> entries have been applied to the reader's cache.
> 
> > Either kind of read will eventually notice that its local cached
> > value has been invalidated and will fetch the updated value.
> 
> Eventually, yes.
> 

  so, summing up this discussion, the change is good to go? I think we
can always add any implementation dependent optimizations later.

If everybody agrees, I will push it.

Thanks,
  Thomas



From rkennke at redhat.com  Mon Nov 27 09:30:56 2017
From: rkennke at redhat.com (Roman Kennke)
Date: Mon, 27 Nov 2017 10:30:56 +0100
Subject: RFR: 8191564: Refactor GC related servicability code into GC
 specific subclasses
In-Reply-To: <2a7e6307-8c0d-ad4b-e50c-a7e1c2311080@redhat.com>
References: <387d2b93-cb8b-51be-a6e0-3b5f9d6cbb1c@redhat.com>
 <713325a4-3ed8-cd7d-c66d-a1c1767b6d3a@redhat.com>
 <2a7e6307-8c0d-ad4b-e50c-a7e1c2311080@redhat.com>
Message-ID: <56567db1-caf6-ed05-ed12-336edfca0c75@redhat.com>

Erik implemented a few more refactorings and touch-ups, and here is our 
final (pending reviews) webrev:

http://cr.openjdk.java.net/~rkennke/8191564/webrev.04/

Compared to webrev.02, it improves the handling of gc-end-message, 
avoids dragging the GCMemoryManager through Generation and a few little 
related refactorings.

Ok to push now?

Roman

> After a few more discussions with Erik I made some more changes.
> 
> MemoryService is now unaware of the number and meaning of GC memory 
> managers (minor vs major). This should be better for GCs that don't make 
> that distinction and/or use more different GCs (e.g. minor, 
> intermediate, full).
> 
> This means that I needed to add some abstractions:
> - GCMemoryManager now has gc_end_message() which is used by 
> GCNotifier::pushNotification().
> - gc_begin() and gc_end() methods in MemoryService now accept a 
> GCMemoryManager* instead of bull full_gc
> - Same for TraceMemoryManagerStats
> - Generation now knows about the corresponding GCMemoryManager
> 
> Please review the full change:
> http://cr.openjdk.java.net/~rkennke/8191564/webrev.02/
> 
> Thanks, Roman
> 
> 
>> I had some off-band discussions with Erik Helin and re-did most of the 
>> changeset:
>> - The GC interface now resides in CollectedHeap, specifically the two 
>> methods memory_managers() and memory_pools(), and is implemented in 
>> the various concrete subclasses.
>> - Both methods return (by value) a GrowableArray<GCMemoryManager*> and 
>> GrowableArray<MemoryPool> respectively. Returning a stack-allocated 
>> GrowableArray seemed least complicated (avoid explicit cleanup of 
>> short-lived array object), and most future-proof, e.g. currently there 
>> is an implicit expectation to get 2 GCMemoryManagers, even though some 
>> GCs don't necessarily have two. The API allows for easy extension of 
>> the situation.
>>
>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.01/
>>
>> I think this requires reviews from both the GC and Serviceability team.
>>
>> Roman
>>
>>
>>> Currently, there's lots of GC specific code sprinkled over 
>>> src/hotspot/share/services. This change introduces a GC interface for 
>>> that, and moves all GC specific code to their respective 
>>> src/hotspot/share/gc directory.
>>>
>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.00/ 
>>> <http://cr.openjdk.java.net/%7Erkennke/8191564/webrev.00/>
>>>
>>> Testing: hotspot_gc and hotspot_serviceability, none showed regressions
>>>
>>> Built minimal and server without regressions
>>>
>>> What do you think?
>>>
>>> Roman
>>>
>>>
>>
> 



From thomas.schatzl at oracle.com  Mon Nov 27 09:32:01 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 27 Nov 2017 10:32:01 +0100
Subject: RFR (XS): 8188877: Improper synchronization in offer_termination
In-Reply-To: <1511775028.2398.3.camel@oracle.com>
References: <CY1PR0701MB16328948D4FDF19FBC30F65684710@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <3229BC22-BB8A-4D3D-8440-AB799AB5394F@oracle.com>
 <CY1PR0701MB16328E4C6563BE60CF8E99D084750@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <4D9972BA-B76C-4FB7-B24D-31D507DB1A6A@oracle.com>
 <1510821711.5332.3.camel@oracle.com>
 <CY1PR0701MB163288CD9B58A163B2F16F7C84230@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <ba0c0855-188a-decf-932b-b11200dc9fc4@redhat.com>
 <1511775028.2398.3.camel@oracle.com>
Message-ID: <1511775121.2398.4.camel@oracle.com>

Hi again,

On Mon, 2017-11-27 at 10:30 +0100, Thomas Schatzl wrote:
> Hi all,
> 
> On Wed, 2017-11-22 at 09:13 +0000, Andrew Haley wrote:
> > On 21/11/17 21:53, White, Derek wrote:
> > > My understanding is that the "acquire" semantics are entirely
> > > about
> > > memory ordering, within a CPU. In particular it prevents
> > > "following
> > > loads" from executing before the "load acquire".
> > > 
> > > 
> > > There is nothing in the "load acquire" that causes it to
> > > synchronize
> > > with the memory system more or less quickly than a naked load.
> > 
> > The abstract architecture only specifies things in terms of
> > ordering
> > between loads, but it has to be implemented somehow, and this is
> > MESI
> > or something similar.  Stores cause invalidate messages to be sent,
> > and these are put into the reader's invalidate queue.  When that
> > reader executes a load barrier it marks all the entries currently
> > in
> > its invalidate queue.  The next load will wait until all marked
> > entries have been applied to the reader's cache.
> > 
> > > Either kind of read will eventually notice that its local cached
> > > value has been invalidated and will fetch the updated value.
> > 
> > Eventually, yes.
> > 
> 
>   so, summing up this discussion, the change is good to go? I think
> we can always add any implementation dependent optimizations later.
> 
> If everybody agrees, I will push it.

  and btw, looks good :)

Thomas



From erik.osterlund at oracle.com  Mon Nov 27 11:28:48 2017
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Mon, 27 Nov 2017 12:28:48 +0100
Subject: RFR (S): 8191894: Refactor weak references in JvmtiTagHashmap to use
 the Access API
Message-ID: <5A1BF6F0.5030903@oracle.com>

Hi,

The JVMTI tag hashmap has weak oop references that are handled using raw 
oop accesses and a G1-specific SATB enqueue call when leaking out 
objects from the tag map, requiring them to be marked as live by G1.

This should now be refactored to use the Access API to annotate that 
these are ON_PHANTOM_OOP_REF, and refactor the raw oop loads to use 
ON_PHANTOM_OOP_REF | AS_NO_KEEPALIVE.

Webrev:
http://cr.openjdk.java.net/~eosterlund/8191894/webrev.00/

Bug:
https://bugs.openjdk.java.net/browse/JDK-8191894

Thanks,
/Erik


From aph at redhat.com  Mon Nov 27 11:55:32 2017
From: aph at redhat.com (Andrew Haley)
Date: Mon, 27 Nov 2017 11:55:32 +0000
Subject: RFR (XS): 8188877: Improper synchronization in offer_termination
In-Reply-To: <1511775028.2398.3.camel@oracle.com>
References: <CY1PR0701MB16328948D4FDF19FBC30F65684710@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <3229BC22-BB8A-4D3D-8440-AB799AB5394F@oracle.com>
 <CY1PR0701MB16328E4C6563BE60CF8E99D084750@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <4D9972BA-B76C-4FB7-B24D-31D507DB1A6A@oracle.com>
 <1510821711.5332.3.camel@oracle.com>
 <CY1PR0701MB163288CD9B58A163B2F16F7C84230@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <ba0c0855-188a-decf-932b-b11200dc9fc4@redhat.com>
 <1511775028.2398.3.camel@oracle.com>
Message-ID: <cfb2ea8f-6612-9cd6-32c8-871a67ebf8cf@redhat.com>

On 27/11/17 09:30, Thomas Schatzl wrote:
>   so, summing up this discussion, the change is good to go? I think we
> can always add any implementation dependent optimizations later.
> 
> If everybody agrees, I will push it.

OK from me.  I'm still a bit nervous about the possible performance
implications of interpreted code taking an extremely long time to get
to a safepoint.  However, I did some experiments and it looks like the
worst case we're likely to encounter is about a couple of hundred
microseconds.

-- 
Andrew Haley
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From stefan.karlsson at oracle.com  Mon Nov 27 12:00:05 2017
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Mon, 27 Nov 2017 13:00:05 +0100
Subject: RFR(s): 8191864: Provide a public destructor for WorkGang
In-Reply-To: <ffd347e5-8804-25d7-474e-4a3378507bbe@oracle.com>
References: <ffd347e5-8804-25d7-474e-4a3378507bbe@oracle.com>
Message-ID: <b10c30cf-812e-b43f-6a31-09b57c40a426@oracle.com>

Looks good.

StefanK

On 2017-11-24 15:55, Per Liden wrote:
> Hi,
> 
> WorkGang's destructor is currently declared private, meaning a WorkGang 
> can't be deleted, instantiated as a value or placed on the stack. This 
> patch adds a public destructor. I also added a protected destructor on 
> AbstractWorkGang to guard against attempts to delete using a 
> AbstractWorkGang pointer, which would require a virtual destructor.
> 
> Webrev: http://cr.openjdk.java.net/~pliden/8191864/
> Bug: https://bugs.openjdk.java.net/browse/JDK-8191864
> 
> /Per


From stefan.karlsson at oracle.com  Mon Nov 27 12:06:18 2017
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Mon, 27 Nov 2017 13:06:18 +0100
Subject: RFR: 8191862: Warn about UseNUMA/UseLargePages only when using
 ParallelGC
In-Reply-To: <f63cc912-5ac8-f9df-299d-964e44debecf@oracle.com>
References: <f63cc912-5ac8-f9df-299d-964e44debecf@oracle.com>
Message-ID: <3d10daf6-9374-5ff4-bdc1-101dfa4fd8d1@oracle.com>

Looks good.

StefanK

On 2017-11-24 15:20, Per Liden wrote:
> Hi,
> 
> We currently warn about using UseAdaptiveSizePolicy and 
> UseAdaptiveNUMAChunkSizing in combination with UseNUMA and 
> UseLargePages. This warning is only relevant for ParallelGC, but we 
> print it regardless of which GC has been selected.
> 
> Webrev: http://cr.openjdk.java.net/~pliden/8191862/webrev.0/
> Bug: https://bugs.openjdk.java.net/browse/JDK-8191862
> 
> /Per


From adinn at redhat.com  Mon Nov 27 12:30:37 2017
From: adinn at redhat.com (Andrew Dinn)
Date: Mon, 27 Nov 2017 12:30:37 +0000
Subject: RFR (XS): 8188877: Improper synchronization in offer_termination
In-Reply-To: <ba0c0855-188a-decf-932b-b11200dc9fc4@redhat.com>
References: <CY1PR0701MB16328948D4FDF19FBC30F65684710@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <3229BC22-BB8A-4D3D-8440-AB799AB5394F@oracle.com>
 <CY1PR0701MB16328E4C6563BE60CF8E99D084750@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <4D9972BA-B76C-4FB7-B24D-31D507DB1A6A@oracle.com>
 <1510821711.5332.3.camel@oracle.com>
 <CY1PR0701MB163288CD9B58A163B2F16F7C84230@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <ba0c0855-188a-decf-932b-b11200dc9fc4@redhat.com>
Message-ID: <ed1ce174-186d-6548-1eb9-bd61adcb6971@redhat.com>

On 22/11/17 09:13, Andrew Haley wrote:
> On 21/11/17 21:53, White, Derek wrote:
>> My understanding is that the "acquire" semantics are entirely about
>> memory ordering, within a CPU. In particular it prevents "following
>> loads" from executing before the "load acquire".
>>
>>
>> There is nothing in the "load acquire" that causes it to synchronize
>> with the memory system more or less quickly than a naked load.
> 
> The abstract architecture only specifies things in terms of ordering
> between loads, but it has to be implemented somehow, and this is MESI
> or something similar.  Stores cause invalidate messages to be sent,
> and these are put into the reader's invalidate queue.  When that
> reader executes a load barrier it marks all the entries currently in
> its invalidate queue.  The next load will wait until all marked
> entries have been applied to the reader's cache.

That's what happens when the reader executes a read barrier. The
interesting question is what happens when the reader does not execute a
read barrier.

>> Either kind of read will eventually notice that its local cached
>> value has been invalidated and will fetch the updated value.
> 
> Eventually, yes.
That's a rather ill-defined span of time ;-)

I understand that you tested this and found that it took no longer than
a few hundred microseconds. However, I really have to ask what precisely
the reader was doing during the test?

Specifically, does the time taken to 'eventually' notice a write to the
LDRed location depend upon what other instructions are executed between
successive LDRs?

regards,


Andrew Dinn
-----------
Senior Principal Software Engineer
Red Hat UK Ltd
Registered in England and Wales under Company Registration No. 03798903
Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander


From stefan.johansson at oracle.com  Mon Nov 27 12:32:19 2017
From: stefan.johansson at oracle.com (Stefan Johansson)
Date: Mon, 27 Nov 2017 13:32:19 +0100
Subject: RFR: 8191862: Warn about UseNUMA/UseLargePages only when using
 ParallelGC
In-Reply-To: <3d10daf6-9374-5ff4-bdc1-101dfa4fd8d1@oracle.com>
References: <f63cc912-5ac8-f9df-299d-964e44debecf@oracle.com>
 <3d10daf6-9374-5ff4-bdc1-101dfa4fd8d1@oracle.com>
Message-ID: <26b7d0d2-0b4b-147b-9498-e3bd8ac72840@oracle.com>



On 2017-11-27 13:06, Stefan Karlsson wrote:
> Looks good.
>
+1
StefanJ
> StefanK
>
> On 2017-11-24 15:20, Per Liden wrote:
>> Hi,
>>
>> We currently warn about using UseAdaptiveSizePolicy and 
>> UseAdaptiveNUMAChunkSizing in combination with UseNUMA and 
>> UseLargePages. This warning is only relevant for ParallelGC, but we 
>> print it regardless of which GC has been selected.
>>
>> Webrev: http://cr.openjdk.java.net/~pliden/8191862/webrev.0/
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8191862
>>
>> /Per



From stefan.johansson at oracle.com  Mon Nov 27 12:49:11 2017
From: stefan.johansson at oracle.com (Stefan Johansson)
Date: Mon, 27 Nov 2017 13:49:11 +0100
Subject: RFR(s): 8191864: Provide a public destructor for WorkGang
In-Reply-To: <b10c30cf-812e-b43f-6a31-09b57c40a426@oracle.com>
References: <ffd347e5-8804-25d7-474e-4a3378507bbe@oracle.com>
 <b10c30cf-812e-b43f-6a31-09b57c40a426@oracle.com>
Message-ID: <43187d72-391e-cd05-fd1f-756f3cf949a0@oracle.com>



On 2017-11-27 13:00, Stefan Karlsson wrote:
> Looks good.
>
+1
StefanJ
> StefanK
>
> On 2017-11-24 15:55, Per Liden wrote:
>> Hi,
>>
>> WorkGang's destructor is currently declared private, meaning a 
>> WorkGang can't be deleted, instantiated as a value or placed on the 
>> stack. This patch adds a public destructor. I also added a protected 
>> destructor on AbstractWorkGang to guard against attempts to delete 
>> using a AbstractWorkGang pointer, which would require a virtual 
>> destructor.
>>
>> Webrev: http://cr.openjdk.java.net/~pliden/8191864/
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8191864
>>
>> /Per



From per.liden at oracle.com  Mon Nov 27 12:52:08 2017
From: per.liden at oracle.com (Per Liden)
Date: Mon, 27 Nov 2017 13:52:08 +0100
Subject: RFR(s): 8191864: Provide a public destructor for WorkGang
In-Reply-To: <43187d72-391e-cd05-fd1f-756f3cf949a0@oracle.com>
References: <ffd347e5-8804-25d7-474e-4a3378507bbe@oracle.com>
 <b10c30cf-812e-b43f-6a31-09b57c40a426@oracle.com>
 <43187d72-391e-cd05-fd1f-756f3cf949a0@oracle.com>
Message-ID: <0f7fbf0c-838b-ad41-ce2d-ef3d49e286b2@oracle.com>

Thanks Stefan & Stefan!

/Per

On 2017-11-27 13:49, Stefan Johansson wrote:
>
>
> On 2017-11-27 13:00, Stefan Karlsson wrote:
>> Looks good.
>>
> +1
> StefanJ
>> StefanK
>>
>> On 2017-11-24 15:55, Per Liden wrote:
>>> Hi,
>>>
>>> WorkGang's destructor is currently declared private, meaning a
>>> WorkGang can't be deleted, instantiated as a value or placed on the
>>> stack. This patch adds a public destructor. I also added a protected
>>> destructor on AbstractWorkGang to guard against attempts to delete
>>> using a AbstractWorkGang pointer, which would require a virtual
>>> destructor.
>>>
>>> Webrev: http://cr.openjdk.java.net/~pliden/8191864/
>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8191864
>>>
>>> /Per
>


From per.liden at oracle.com  Mon Nov 27 12:52:30 2017
From: per.liden at oracle.com (Per Liden)
Date: Mon, 27 Nov 2017 13:52:30 +0100
Subject: RFR: 8191862: Warn about UseNUMA/UseLargePages only when using
 ParallelGC
In-Reply-To: <26b7d0d2-0b4b-147b-9498-e3bd8ac72840@oracle.com>
References: <f63cc912-5ac8-f9df-299d-964e44debecf@oracle.com>
 <3d10daf6-9374-5ff4-bdc1-101dfa4fd8d1@oracle.com>
 <26b7d0d2-0b4b-147b-9498-e3bd8ac72840@oracle.com>
Message-ID: <430f9a6c-c6e7-bbef-a78a-1e14e6bac77c@oracle.com>

Thanks Stefan & Stefan!

/Per

On 2017-11-27 13:32, Stefan Johansson wrote:
>
>
> On 2017-11-27 13:06, Stefan Karlsson wrote:
>> Looks good.
>>
> +1
> StefanJ
>> StefanK
>>
>> On 2017-11-24 15:20, Per Liden wrote:
>>> Hi,
>>>
>>> We currently warn about using UseAdaptiveSizePolicy and
>>> UseAdaptiveNUMAChunkSizing in combination with UseNUMA and
>>> UseLargePages. This warning is only relevant for ParallelGC, but we
>>> print it regardless of which GC has been selected.
>>>
>>> Webrev: http://cr.openjdk.java.net/~pliden/8191862/webrev.0/
>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8191862
>>>
>>> /Per
>


From thomas.schatzl at oracle.com  Mon Nov 27 12:52:34 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 27 Nov 2017 13:52:34 +0100
Subject: RFR (S): [TESTBUG] Test for JDK-8180048
Message-ID: <1511787154.2262.8.camel@oracle.com>

Hi all,

  can I have some reviews that adds a test for JDK-8180048? That test
has not been completed in time for JDK 9, but I think it is good to
have it in JDK 10.

It tries to detect the races fixed in JDK-8180048 by parsing NMT output
before/after symbol unloading. To be sure that we exercise the buggy
code path, the test takes some time (~30s) to stress symbol unloading.

To avoid this test clogging up valuable time in lower testing tiers,
the test has been put into the "stress" test directory so that it only
executes at higher test tiers. Further I limited it to execute only
with a release build.

CR:
https://bugs.openjdk.java.net/browse/JDK-8180280
Webrev:
http://cr.openjdk.java.net/~tschatzl/8180280/webrev
Testing:
test case passing

Thanks,
  Thomas


From thomas.schatzl at oracle.com  Mon Nov 27 12:53:41 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 27 Nov 2017 13:53:41 +0100
Subject: RFR (S): 8180280: [TESTBUG] Test for JDK-8180048
Message-ID: <1511787221.2262.9.camel@oracle.com>

Hi all,

  can I have some reviews that adds a test for JDK-8180048? That test
has not been completed in time for JDK 9, but I think it is good to
have it in JDK 10.

It tries to detect the races fixed in JDK-8180048 by parsing NMT output
before/after symbol unloading. To be sure that we exercise the buggy
code path, the test takes some time (~30s) to stress symbol unloading.

To avoid this test clogging up valuable time in lower testing tiers,
the test has been put into the "stress" test directory so that it only
executes at higher test tiers. Further I limited it to execute only
with a release build.

CR:
https://bugs.openjdk.java.net/browse/JDK-8180280
Webrev:
http://cr.openjdk.java.net/~tschatzl/8180280/webrev
Testing:
test case passing

Thanks,
  Thomas



From daniel.daugherty at oracle.com  Mon Nov 27 13:22:07 2017
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Mon, 27 Nov 2017 08:22:07 -0500
Subject: RFR (S): 8191894: Refactor weak references in JvmtiTagHashmap to
 use the Access API
In-Reply-To: <5A1BF6F0.5030903@oracle.com>
References: <5A1BF6F0.5030903@oracle.com>
Message-ID: <88659101-1693-d5a4-a605-8f7b5ab53299@oracle.com>

Adding serviceability-dev at ... since this is JVM/TI related...

Dan


On 11/27/17 6:28 AM, Erik ?sterlund wrote:
> Hi,
>
> The JVMTI tag hashmap has weak oop references that are handled using 
> raw oop accesses and a G1-specific SATB enqueue call when leaking out 
> objects from the tag map, requiring them to be marked as live by G1.
>
> This should now be refactored to use the Access API to annotate that 
> these are ON_PHANTOM_OOP_REF, and refactor the raw oop loads to use 
> ON_PHANTOM_OOP_REF | AS_NO_KEEPALIVE.
>
> Webrev:
> http://cr.openjdk.java.net/~eosterlund/8191894/webrev.00/
>
> Bug:
> https://bugs.openjdk.java.net/browse/JDK-8191894
>
> Thanks,
> /Erik



From alexander.harlap at oracle.com  Mon Nov 27 14:33:28 2017
From: alexander.harlap at oracle.com (Alexander Harlap)
Date: Mon, 27 Nov 2017 09:33:28 -0500
Subject: Need sponsor to push attached 8185591 into
 http://hg.openjdk.java.net/jdk10/hs
In-Reply-To: <1511444326.2477.0.camel@oracle.com>
References: <9dbf43c5-116f-552e-fcfc-9672830902fb@oracle.com>
 <1511444326.2477.0.camel@oracle.com>
Message-ID: <ed24b162-445a-2bb6-e08b-df7dba8416a9@oracle.com>

Thank you very much!

Alex


On 11/23/2017 8:38 AM, Thomas Schatzl wrote:
> Hi,
>
> On Wed, 2017-11-22 at 19:25 -0500, Alexander Harlap wrote:
>> I need a sponsor to push attached 8185591.open.patch -
>> guarantee(_byte_map[_guard_index] == last_card) failed: card table
>> guard
>> has been modified.
>>
>> Patch should go into jdk10/hs.
>>
>> Reviewed by Thomas Schatzl  and Kim Barrett.
>>
>    I will sponsor.
>
> Thomas



From aph at redhat.com  Mon Nov 27 14:53:28 2017
From: aph at redhat.com (Andrew Haley)
Date: Mon, 27 Nov 2017 14:53:28 +0000
Subject: RFR (XS): 8188877: Improper synchronization in offer_termination
In-Reply-To: <ed1ce174-186d-6548-1eb9-bd61adcb6971@redhat.com>
References: <CY1PR0701MB16328948D4FDF19FBC30F65684710@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <3229BC22-BB8A-4D3D-8440-AB799AB5394F@oracle.com>
 <CY1PR0701MB16328E4C6563BE60CF8E99D084750@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <4D9972BA-B76C-4FB7-B24D-31D507DB1A6A@oracle.com>
 <1510821711.5332.3.camel@oracle.com>
 <CY1PR0701MB163288CD9B58A163B2F16F7C84230@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <ba0c0855-188a-decf-932b-b11200dc9fc4@redhat.com>
 <ed1ce174-186d-6548-1eb9-bd61adcb6971@redhat.com>
Message-ID: <25ec7ee7-c577-6051-a57b-f2b31c3cb3bd@redhat.com>

On 27/11/17 12:30, Andrew Dinn wrote:
> On 22/11/17 09:13, Andrew Haley wrote:
>> On 21/11/17 21:53, White, Derek wrote:
>>> My understanding is that the "acquire" semantics are entirely about
>>> memory ordering, within a CPU. In particular it prevents "following
>>> loads" from executing before the "load acquire".
>>>
>>> There is nothing in the "load acquire" that causes it to synchronize
>>> with the memory system more or less quickly than a naked load.
>>
>> The abstract architecture only specifies things in terms of ordering
>> between loads, but it has to be implemented somehow, and this is MESI
>> or something similar.  Stores cause invalidate messages to be sent,
>> and these are put into the reader's invalidate queue.  When that
>> reader executes a load barrier it marks all the entries currently in
>> its invalidate queue.  The next load will wait until all marked
>> entries have been applied to the reader's cache.
> 
> That's what happens when the reader executes a read barrier. The
> interesting question is what happens when the reader does not execute a
> read barrier.

The invalidate messages still arrive at the reader, but they sit in
the invalidate queue and aren't acted upon immediately.  Eventually
they must be processed, either lazily or because the reader's
invalidate queue fills up.

>>> Either kind of read will eventually notice that its local cached
>>> value has been invalidated and will fetch the updated value.
>>
>> Eventually, yes.
> That's a rather ill-defined span of time ;-)
> 
> I understand that you tested this and found that it took no longer than
> a few hundred microseconds. However, I really have to ask what precisely
> the reader was doing during the test?

Nothing except spinning and loading, and that's a few microseconds'
delay rather than a few hundred.

> Specifically, does the time taken to 'eventually' notice a write to the
> LDRed location depend upon what other instructions are executed between
> successive LDRs?

It's really hard to be definite about that.  In practice it may well
be that back-to-back local cache accesses saturate the CPU<->cache
interconnect so much that they delay the processing of invalidate
queue entries, but that's my speculation and it's secret sauce anyway.

It is likely, though, that if you issue a load barrier the next load
must cause the contents of the invalidate queue to be applied to the
cache, in order to ensure that everything happens in order.  So I
suspect that load barriers will cause changes to be seem earlier.
Having said that, load barriers slow down all reads for a while.

And one final caveat: I'm talking about MESI, but there are more
elaborate and sophisticated ways of making this stuff work.

-- 
Andrew Haley
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From adinn at redhat.com  Mon Nov 27 15:44:57 2017
From: adinn at redhat.com (Andrew Dinn)
Date: Mon, 27 Nov 2017 15:44:57 +0000
Subject: RFR (XS): 8188877: Improper synchronization in offer_termination
In-Reply-To: <25ec7ee7-c577-6051-a57b-f2b31c3cb3bd@redhat.com>
References: <CY1PR0701MB16328948D4FDF19FBC30F65684710@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <3229BC22-BB8A-4D3D-8440-AB799AB5394F@oracle.com>
 <CY1PR0701MB16328E4C6563BE60CF8E99D084750@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <4D9972BA-B76C-4FB7-B24D-31D507DB1A6A@oracle.com>
 <1510821711.5332.3.camel@oracle.com>
 <CY1PR0701MB163288CD9B58A163B2F16F7C84230@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <ba0c0855-188a-decf-932b-b11200dc9fc4@redhat.com>
 <ed1ce174-186d-6548-1eb9-bd61adcb6971@redhat.com>
 <25ec7ee7-c577-6051-a57b-f2b31c3cb3bd@redhat.com>
Message-ID: <f213bf00-1597-f8a2-6efa-5a8dc9a68013@redhat.com>

On 27/11/17 14:53, Andrew Haley wrote:
> On 27/11/17 12:30, Andrew Dinn wrote:
<snip>
>> That's what happens when the reader executes a read barrier. The
>> interesting question is what happens when the reader does not execute a
>> read barrier.
> 
> The invalidate messages still arrive at the reader, but they sit in
> the invalidate queue and aren't acted upon immediately.  Eventually
> they must be processed, either lazily or because the reader's
> invalidate queue fills up.

Hmm, that explanation assumes there will be other invalidate messages.
But at a STW pause that's not necessarily going to be the case. In the
worst case all other threads may could be spinning on a barrier count
while this one core/thread has a single invalidate message in its queue.

>> I understand that you tested this and found that it took no longer than
>> a few hundred microseconds. However, I really have to ask what precisely
>> the reader was doing during the test?
> 
> Nothing except spinning and loading, and that's a few microseconds'
> delay rather than a few hundred.

Ok, so it's not as if the reader noticed the invalidate because it was
writing memory or performing some other 'active' interaction with the
memory system that forced the invalidate queue to be processed (assuming
that invalidate detection is not a sneaky side-effect of a nop :-).

>> Specifically, does the time taken to 'eventually' notice a write to the
>> LDRed location depend upon what other instructions are executed between
>> successive LDRs?
> 
> It's really hard to be definite about that.  In practice it may well
> be that back-to-back local cache accesses saturate the CPU<->cache
> interconnect so much that they delay the processing of invalidate
> queue entries, but that's my speculation and it's secret sauce anyway.
 . . .
> And one final caveat: I'm talking about MESI, but there are more
> elaborate and sophisticated ways of making this stuff work.

Well yes, which is fine so long as any such elaborate sophistricat^H^H^H
improvement of current behaviour doesn't assume that a core can ignore
the invalidate queue absent an ldar (or, maybe, absent ldar union some
other subset of the available memory ops).

regards,


Andrew Dinn
-----------
Senior Principal Software Engineer
Red Hat UK Ltd
Registered in England and Wales under Company Registration No. 03798903
Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander


From aph at redhat.com  Mon Nov 27 15:57:25 2017
From: aph at redhat.com (Andrew Haley)
Date: Mon, 27 Nov 2017 15:57:25 +0000
Subject: RFR (XS): 8188877: Improper synchronization in offer_termination
In-Reply-To: <f213bf00-1597-f8a2-6efa-5a8dc9a68013@redhat.com>
References: <CY1PR0701MB16328948D4FDF19FBC30F65684710@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <3229BC22-BB8A-4D3D-8440-AB799AB5394F@oracle.com>
 <CY1PR0701MB16328E4C6563BE60CF8E99D084750@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <4D9972BA-B76C-4FB7-B24D-31D507DB1A6A@oracle.com>
 <1510821711.5332.3.camel@oracle.com>
 <CY1PR0701MB163288CD9B58A163B2F16F7C84230@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <ba0c0855-188a-decf-932b-b11200dc9fc4@redhat.com>
 <ed1ce174-186d-6548-1eb9-bd61adcb6971@redhat.com>
 <25ec7ee7-c577-6051-a57b-f2b31c3cb3bd@redhat.com>
 <f213bf00-1597-f8a2-6efa-5a8dc9a68013@redhat.com>
Message-ID: <b3948fb6-62b4-f344-348a-63ffaca4504d@redhat.com>

On 27/11/17 15:44, Andrew Dinn wrote:
> On 27/11/17 14:53, Andrew Haley wrote:
>> On 27/11/17 12:30, Andrew Dinn wrote:
> <snip>
>>> That's what happens when the reader executes a read barrier. The
>>> interesting question is what happens when the reader does not execute a
>>> read barrier.
>>
>> The invalidate messages still arrive at the reader, but they sit in
>> the invalidate queue and aren't acted upon immediately.  Eventually
>> they must be processed, either lazily or because the reader's
>> invalidate queue fills up.
> 
> Hmm, that explanation assumes there will be other invalidate messages.

No, not at all.  By "lazily" I mean that while a core has nothing else
to do it might as well process its invalidate queue, and AFAIK that is
what happens.

> But at a STW pause that's not necessarily going to be the case. In the
> worst case all other threads may could be spinning on a barrier count
> while this one core/thread has a single invalidate message in its queue.

That could be, but there are other things go on.  There are other threads
active, and invalidate messages get sent to everyone.

In practice, I've never seen more than a few microseconds of delay.

Bear in mind that the interpreter changes we just made mean that
interpreted code won't necessarily see safepoint status changes for
about 100 microseconds, so the lack of an acquiring load in our
code is really not the biggest issue.

-- 
Andrew Haley
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From erik.osterlund at oracle.com  Mon Nov 27 16:23:35 2017
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Mon, 27 Nov 2017 17:23:35 +0100
Subject: RFR: 8191564: Refactor GC related servicability code into GC
 specific subclasses
In-Reply-To: <56567db1-caf6-ed05-ed12-336edfca0c75@redhat.com>
References: <387d2b93-cb8b-51be-a6e0-3b5f9d6cbb1c@redhat.com>
 <713325a4-3ed8-cd7d-c66d-a1c1767b6d3a@redhat.com>
 <2a7e6307-8c0d-ad4b-e50c-a7e1c2311080@redhat.com>
 <56567db1-caf6-ed05-ed12-336edfca0c75@redhat.com>
Message-ID: <5A1C3C07.30800@oracle.com>

Hi Roman,

A few things:

1) The code uses the "mgr" short name for "manager" in a bunch of names. 
I would be happy if we could write out the whole thing instead of the 
abbreviation.
2) It would be great if a generation would have a pointer to its 
manager, so we do not have to pass around the manager where we already 
pass around the generation (such as GenCollectedHeap::collect_generation).
The generation could be created first, then the pools, then the 
managers, then do something like generation->set_memory_manager(x).
3) In cmsHeap.cpp:54: maxSize should preferably not use camel case.

Otherwise I think it looks good.

Thanks,
/Erik

On 2017-11-27 10:30, Roman Kennke wrote:
> Erik implemented a few more refactorings and touch-ups, and here is 
> our final (pending reviews) webrev:
>
> http://cr.openjdk.java.net/~rkennke/8191564/webrev.04/
>
> Compared to webrev.02, it improves the handling of gc-end-message, 
> avoids dragging the GCMemoryManager through Generation and a few 
> little related refactorings.
>
> Ok to push now?
>
> Roman
>
>> After a few more discussions with Erik I made some more changes.
>>
>> MemoryService is now unaware of the number and meaning of GC memory 
>> managers (minor vs major). This should be better for GCs that don't 
>> make that distinction and/or use more different GCs (e.g. minor, 
>> intermediate, full).
>>
>> This means that I needed to add some abstractions:
>> - GCMemoryManager now has gc_end_message() which is used by 
>> GCNotifier::pushNotification().
>> - gc_begin() and gc_end() methods in MemoryService now accept a 
>> GCMemoryManager* instead of bull full_gc
>> - Same for TraceMemoryManagerStats
>> - Generation now knows about the corresponding GCMemoryManager
>>
>> Please review the full change:
>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.02/
>>
>> Thanks, Roman
>>
>>
>>> I had some off-band discussions with Erik Helin and re-did most of 
>>> the changeset:
>>> - The GC interface now resides in CollectedHeap, specifically the 
>>> two methods memory_managers() and memory_pools(), and is implemented 
>>> in the various concrete subclasses.
>>> - Both methods return (by value) a GrowableArray<GCMemoryManager*> 
>>> and GrowableArray<MemoryPool> respectively. Returning a 
>>> stack-allocated GrowableArray seemed least complicated (avoid 
>>> explicit cleanup of short-lived array object), and most 
>>> future-proof, e.g. currently there is an implicit expectation to get 
>>> 2 GCMemoryManagers, even though some GCs don't necessarily have two. 
>>> The API allows for easy extension of the situation.
>>>
>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.01/
>>>
>>> I think this requires reviews from both the GC and Serviceability team.
>>>
>>> Roman
>>>
>>>
>>>> Currently, there's lots of GC specific code sprinkled over 
>>>> src/hotspot/share/services. This change introduces a GC interface 
>>>> for that, and moves all GC specific code to their respective 
>>>> src/hotspot/share/gc directory.
>>>>
>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.00/ 
>>>> <http://cr.openjdk.java.net/%7Erkennke/8191564/webrev.00/>
>>>>
>>>> Testing: hotspot_gc and hotspot_serviceability, none showed 
>>>> regressions
>>>>
>>>> Built minimal and server without regressions
>>>>
>>>> What do you think?
>>>>
>>>> Roman
>>>>
>>>>
>>>
>>
>



From vladimir.kozlov at oracle.com  Mon Nov 27 18:19:55 2017
From: vladimir.kozlov at oracle.com (Vladimir Kozlov)
Date: Mon, 27 Nov 2017 10:19:55 -0800
Subject: RFR(L): 8186027: C2: loop strip mining
In-Reply-To: <1511515343.2425.11.camel@oracle.com>
References: <dk6k20cxtak.fsf@rwestrel.remote.csb>
 <35a024b5-6632-0488-a001-14f960257fc7@oracle.com>
 <dk67evjnvww.fsf@rwestrel.remote.csb>
 <8b9f8985-daae-b3b4-ba48-3d9c6185ed95@oracle.com>
 <0972a0db-2115-daa3-9990-7d58915a74a5@oracle.com>
 <dk61sktw8hn.fsf@rwestrel.remote.csb>
 <3e3e2048-5ee5-f3c2-8ec0-cdd36cff07af@oracle.com>
 <dk64lpmuzrg.fsf@rwestrel.remote.csb>
 <7ae8403a-b650-5171-5d79-0107d7d99059@oracle.com>
 <dk6shd5t6x7.fsf@rwestrel.remote.csb>
 <f2a0b0c2-6642-785c-0e47-9d62ad8dae9f@oracle.com>
 <1511515343.2425.11.camel@oracle.com>
Message-ID: <45aa8e04-9244-4089-001f-acba24e7e4ee@oracle.com>

Second pre-integration re-run finshed and there were timeouts. Looks like it depend what hosts were used to run tests.

I agree to enable it by default on subset if GCs which are sensitive to pauses: G1 and Shenandoah.

Roland, please, prepare changeset.

I am not sure we need separate CR for setting defaults. It is simple to file bug and change only defaults if we need.

Thanks,
Vladimir

On 11/24/17 1:22 AM, Thomas Schatzl wrote:
> Hi Nils,
> 
> On Thu, 2017-11-23 at 22:59 +0100, Nils Eliasson wrote:
>> Hi,
>>
>>
>> On 2017-11-23 15:18, Roland Westrelin wrote:
>>> Hi Vladimir,
>>>
>>>> I am running testing again. But if this will repeat and presence
>>>> of this Sparse.small regression suggesting to me that may be we
>>>> should keep this optimization off by default - keep
>>>> UseCountedLoopSafepoints false.
>>>>
>>>> We may switch it on later with additional changes which address
>>>> regressions.
>>>>
>>>> What do you think?
>>>
>>> If the inner loop runs for a small number of iterations and the
>>> compiler can't statically prove it, I don't see a way to remove the
>>> overhead of loop strip mining entirely. So I'm not optimistic the
>>> regression can be fixed.
>>
>> Agreed. In other words: Loop strip mining adds a guarantee that
>> time-to-safepoint won't be too long, and that has a small cost
>>
>> The current situation is that we have some extra performance with
>> UseCountedLoopSafepoints default off, but let some users have a bad
>> experience when they encounter long time-to-safepoint times or
>> failures (https://bugs.openjdk.java.net/browse/JDK-5014723). I rather
>> turn the table and have loop strip mining on, and let the power users
>> experiment with turning it off for any uncertain performance boost.
>>
>>> If loop strip mining defaults to false, would there we be any
>>> regular testing on your side?
>>
>> We would have to add some.
>>>
>>> It seems to me that it would make sense to enable loop strip mining
>>> depending on what GC is used: it makes little sense for parallel gc
>>> but we'll want it enabled for Shenandoah for instance. Where does
>>> G1 fit? I can't really say and I don't have a strong opinion. But
>>> as I understand, G1 was made default under the assumption that
>>> users would be ok trading throughput for better latency. Maybe,
>>> that same reasoning applies to loop strip mining?
>>
>> Scimark.sparse.small show a regression, but having long
>> time-to-safepoint has a throughput cost in some settings like the
>> companion benchmark scimark.sparse.large. Numbers using G1:
>>
>> -XX:-UseCountedLoopSafepoints (default) ~86 ops/m
>> -XX:+UseCountedLoopSafepoints ~106 ops/m
>> -XX:+UseCountedLoopSafepoints -XX:LoopStripMiningIter=1000 ~111 ops/m
>>
>> I would prefer having it on by default, at least in G1. Let's ask the
>> G1 GC-team on their opinion.
> 
>    our perf team uses -XX:+UseCountedLoopSafepoints for _all_ collectors
>   for some time now. When asked, they replied that predictability of
> results is very important for them too.
> 
> We also closed out some perf regressions (e.g. JDK-8177704) due to the
> problems with -XX:-UseCountedLoopSafepoints after you posted these
> results in agreement with the perf team.
> 
> So I am all for making it default for G1 (and I am sure others agree),
> if not for all GCs. However I recommend having a separate CR for
> changing the defaults. Makes it easier reverting it in case things go
> wrong.
> 
> Thanks,
>    Thomas
> 


From thomas.schatzl at oracle.com  Mon Nov 27 19:44:42 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 27 Nov 2017 20:44:42 +0100
Subject: RFR(L): 8186027: C2: loop strip mining
In-Reply-To: <45aa8e04-9244-4089-001f-acba24e7e4ee@oracle.com>
References: <dk6k20cxtak.fsf@rwestrel.remote.csb>
 <35a024b5-6632-0488-a001-14f960257fc7@oracle.com>
 <dk67evjnvww.fsf@rwestrel.remote.csb>
 <8b9f8985-daae-b3b4-ba48-3d9c6185ed95@oracle.com>
 <0972a0db-2115-daa3-9990-7d58915a74a5@oracle.com>
 <dk61sktw8hn.fsf@rwestrel.remote.csb>
 <3e3e2048-5ee5-f3c2-8ec0-cdd36cff07af@oracle.com>
 <dk64lpmuzrg.fsf@rwestrel.remote.csb>
 <7ae8403a-b650-5171-5d79-0107d7d99059@oracle.com>
 <dk6shd5t6x7.fsf@rwestrel.remote.csb>
 <f2a0b0c2-6642-785c-0e47-9d62ad8dae9f@oracle.com>
 <1511515343.2425.11.camel@oracle.com>
 <45aa8e04-9244-4089-001f-acba24e7e4ee@oracle.com>
Message-ID: <1511811882.2241.4.camel@oracle.com>

Hi,

On Mon, 2017-11-27 at 10:19 -0800, Vladimir Kozlov wrote:
> Second pre-integration re-run finshed and there were timeouts. Looks
> like it depend what hosts were used to run tests.
> 
> I agree to enable it by default on subset if GCs which are sensitive
> to pauses: G1 and Shenandoah.
> 
> Roland, please, prepare changeset.
> 
> I am not sure we need separate CR for setting defaults. It is simple
> to file bug and change only defaults if we need.

  the reason I suggested it is because this defaults change will then
be more prominent when browsing the changesets for explanations of
regressions.

Ie. I am very sure we in the GC team would do that change seperately,
but ymmv.

Thanks,
  Thomas



From serguei.spitsyn at oracle.com  Tue Nov 28 05:43:50 2017
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Mon, 27 Nov 2017 21:43:50 -0800
Subject: RFR (S): 8191894: Refactor weak references in JvmtiTagHashmap to
 use the Access API
In-Reply-To: <88659101-1693-d5a4-a605-8f7b5ab53299@oracle.com>
References: <5A1BF6F0.5030903@oracle.com>
 <88659101-1693-d5a4-a605-8f7b5ab53299@oracle.com>
Message-ID: <2b566339-d3f8-27ab-c5c9-670aa278c9da@oracle.com>

Hi Erik,

This looks good to me.

Thanks,
Serguei


On 11/27/17 05:22, Daniel D. Daugherty wrote:
> Adding serviceability-dev at ... since this is JVM/TI related...
>
> Dan
>
>
> On 11/27/17 6:28 AM, Erik ?sterlund wrote:
>> Hi,
>>
>> The JVMTI tag hashmap has weak oop references that are handled using 
>> raw oop accesses and a G1-specific SATB enqueue call when leaking out 
>> objects from the tag map, requiring them to be marked as live by G1.
>>
>> This should now be refactored to use the Access API to annotate that 
>> these are ON_PHANTOM_OOP_REF, and refactor the raw oop loads to use 
>> ON_PHANTOM_OOP_REF | AS_NO_KEEPALIVE.
>>
>> Webrev:
>> http://cr.openjdk.java.net/~eosterlund/8191894/webrev.00/
>>
>> Bug:
>> https://bugs.openjdk.java.net/browse/JDK-8191894
>>
>> Thanks,
>> /Erik
>



From david.holmes at oracle.com  Tue Nov 28 06:59:15 2017
From: david.holmes at oracle.com (David Holmes)
Date: Tue, 28 Nov 2017 16:59:15 +1000
Subject: RFR (S): 8191888: Refactor ClassLoaderData::remove_handle to use
 the Access API
In-Reply-To: <5A1BD5B3.7030808@oracle.com>
References: <5A1BD5B3.7030808@oracle.com>
Message-ID: <d11416e1-a8b4-15f0-79a4-0e74c64bf60e@oracle.com>

Hi Erik,

Is there a non-GC person's guide to what the different forms of 
RootAccess<XXX> mean and when to use them? Or do we expect a GC person 
to always jump in to show where such things are needed? ;-)

Thanks,
David

On 27/11/2017 7:06 PM, Erik ?sterlund wrote:
> Hi,
> 
> The ClassLoaderData::remove_handle() member function currently uses 
> explicit G1 SATB barriers to remove an oop from the root set, as these 
> handles are not necessarily walked by the GC in a safepoint. Therefore 
> G1 needs pre-write barriers.
> 
> This should now be modeled as a 
> RootAccess<IN_CONCURRENT_ROOT>::oop_store instead. This maps to 
> performing a pre-write SATB barrier with G1, but other GCs are free to 
> do other things as necessary.
> 
> Webrev:
> http://cr.openjdk.java.net/~eosterlund/8191888/webrev.00/
> 
> Bug:
> https://bugs.openjdk.java.net/browse/JDK-8191888
> 
> Thanks,
> /Erik


From david.holmes at oracle.com  Tue Nov 28 08:01:27 2017
From: david.holmes at oracle.com (David Holmes)
Date: Tue, 28 Nov 2017 18:01:27 +1000
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <5976a6bf-c9b0-c0d5-d5f9-fcdf7dc9a1a8@oracle.com>
References: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
 <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>
 <836b5c6c-5eef-9245-c16a-e71372093d9a@oracle.com>
 <82d0e473-a24f-9feb-82e9-7d04c4d7a91d@oracle.com>
 <354f7ece-66bc-c065-9ee8-f55abcc1c23e@oracle.com>
 <0ff8df8e-e93c-02d2-cb97-b13b1ac0f53f@oracle.com>
 <5365606e-4036-b5d7-9bac-e91e42f38d01@oracle.com>
 <567339a6-e36a-5416-5c34-2a87cbfa6cdd@oracle.com>
 <5976a6bf-c9b0-c0d5-d5f9-fcdf7dc9a1a8@oracle.com>
Message-ID: <157a90ee-35c9-1adf-490a-9910f3ee646f@oracle.com>

Just for the record ...

On 23/11/2017 6:20 PM, Robbin Ehn wrote:
> Thanks Dan for dragging this freight train to the docks, it's time to 
> ship it!

I agree. The latest delta seems fine to me.

> Created follow-up bug:
> 8191809: Threads::number_of_threads() is not 'MT-safe'
> https://bugs.openjdk.java.net/browse/JDK-8191809

Just updated that bug - I don't see any MT issues there. :)

Cheers,
David

PS. Dan: yes JPRT was still doing 32-bit ARM a "month or two back". 
64-bit atomics should not be a concern. That said I thought all the 
atomic updates were done for ARM etc.

> /Robbin
> 
> On 2017-11-23 03:16, Daniel D. Daugherty wrote:
>> Greetings,
>>
>> I've made minor tweaks to the Thread-SMR project based on the remaining
>> code review comments over the last couple of days. I've also rebased the
>> project to jdk/hs bits as of mid-afternoon (my TZ) on 2017.11.22. I'm
>> running baseline Mach5 Tier[1-5] testing and prototype Mach5 Tier[1-5]
>> testing...
>>
>> Here's a delta webrev for the latest round of tweaks:
>>
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-11-delta
>>
>>
>> Here's the proposed commit message:
>>
>> $ cat SMR_prototype_10/open/commit.txt
>> 8167108: inconsistent handling of SR_lock can lead to crashes
>> Summary: Add Thread Safe Memory Reclamation (Thread-SMR) mechanism.
>> Reviewed-by: coleenp, dcubed, dholmes, eosterlund, gthornbr, kbarrett, 
>> rehn, sspitsyn, stefank
>> Contributed-by: daniel.daugherty at oracle.com, 
>> erik.osterlund at oracle.com, robbin.ehn at oracle.com
>>
>> I've gone through 880+ emails in my archive for this project and added
>> anyone that made a code review comment. Reviewers are not in my usual
>> by-comment-posting order; it was way too hard to figure that out... So
>> reviewers and contributors are simply in alpha-sort order...
>>
>> Here's the collection of follow-up bugs that I filed based on sifting
>> through the email archive:
>>
>> 8191784 JavaThread::collect_counters() should stop using Threads_lock
>> https://bugs.openjdk.java.net/browse/JDK-8191784
>>
>> 8191785 revoke_bias() needs a new assertion
>> https://bugs.openjdk.java.net/browse/JDK-8191785
>>
>> 8191786 Thread-SMR hash table size should be dynamic
>> https://bugs.openjdk.java.net/browse/JDK-8191786
>>
>> 8191787 move private inline functions from thread.inline.hpp -> 
>> thread.cpp
>> https://bugs.openjdk.java.net/browse/JDK-8191787
>>
>> 8191789 migrate more Thread-SMR stuff from thread.[ch]pp -> 
>> threadSMR.[ch]pp
>> https://bugs.openjdk.java.net/browse/JDK-8191789
>>
>> 8191798 redo nested ThreadsListHandle to drop Threads_lock
>> https://bugs.openjdk.java.net/browse/JDK-8191789
>>
>> I have undoubtedly missed at least one future idea that someone had
>> and for that my apologies...
>>
>> Thanks again to everyone who contributed to the Thread-SMR project!!
>>
>> Special thanks goes to Karen Kinnear, Kim Barrett and David Holmes for
>> their rigorous review of the design that we documented on the wiki.
>> Thanks for keeping us honest!
>>
>> I also have to thank my co-contributor Erik ?sterlund for starting the
>> ball rolling on this crazy project with his presentation on Safe Memory
>> Reclamation, for the initial prototype and for all your patches. Erik,
>> hopefully we got far enough in your crazy vision... :-)
>>
>> Thanks to my co-contributor Robbin Ehn for proposing that we do early
>> code reviews in the form of patches. Don't like something? Fix it with
>> a patch and a new test if appropriate!! A pretty cool way to work that
>> was new to me.
>>
>> Finally, many thanks to Jerry Thornbrugh for all the early code reviews,
>> whiteboard chats and phone calls. Thanks for asking the right questions
>> and pointing out some places where our logic was faulty. Also thanks for
>> keeping me sane when I was literally on my own in Monument, CO.
>>
>> Dan
>>
>>
>> On 11/21/17 7:12 PM, Daniel D. Daugherty wrote:
>>> Greetings,
>>>
>>> *** We are wrapping up code review on this project so it is time ***
>>> *** for the various code reviewers to chime in one last time. ***
>>>
>>> In this latest round, we had three different reviewers chime in so we're
>>> doing delta webrevs for each of those resolutions:
>>>
>>> David H's resolutions:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.09,10.10.0-delta/ 
>>>
>>>
>>> ? mq comment for dholmes_CR:
>>> ??? - Fix indents, trailing spaces and various typos.
>>> ??? - Add descriptions for the '_cnt', '_max' and '_times" fields,
>>> ????? add impl notes to document the type choices.
>>>
>>> ? src/hotspot/share/runtime/globals.hpp change is white-space only so it
>>> ? is only visible via the file's patch link.
>>>
>>>
>>> Robin W's resolutions:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.10.0,10.10.1-delta/ 
>>>
>>>
>>> ? mq comment for robinw_CR:
>>> ??? - Fix some inefficient code, update some comments, fix some indents,
>>> ????? and add some 'const' specifiers.
>>>
>>> ? src/hotspot/share/runtime/vm_operations.hpp change is white-space 
>>> only so
>>> ? it is only visible via the file's patch link.
>>>
>>>
>>> Coleen's resolutions:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.10.1,10.10.2-delta/ 
>>>
>>>
>>> ? mq comment for coleenp_CR:
>>> ??? - Change ThreadsList::_threads from 'mtGC' -> 'mtThread',
>>> ??? - add header comment to threadSMR.hpp file,
>>> ??? - cleanup JavaThreadIteratorWithHandle ctr,
>>> ??? - make ErrorHandling more efficient.
>>>
>>>
>>> Some folks prefer to see all of the delta webrevs together so here is
>>> a delta webrev relative to jdk10-09-full:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.09,10.10.2-delta/ 
>>>
>>>
>>> ? src/hotspot/share/runtime/globals.hpp and
>>> ? src/hotspot/share/runtime/vm_operations.hpp changes are white-space 
>>> only
>>> ? so they are only visible via the file's patch link.
>>>
>>>
>>> And, of course, some folks prefer to see everything all at once so here
>>> is the latest full webrev:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.10-full/
>>>
>>>
>>> Dan has submitted the bits for the usual Mach5 tier[1-5] testing. The
>>> prelim Mach5 tier1 (JPRT equivalent) on these bits passed just fine...
>>>
>>> The project is currently baselined on Jesper's 2017-11-17 PIT snapshot
>>> so there will be some catching up to do before integration...
>>>
>>> We welcome comments, suggestions and feedback.
>>>
>>> Dan, Erik and Robbin
>>>
>>>
>>> On 11/18/17 8:49 PM, Daniel D. Daugherty wrote:
>>>> Greetings,
>>>>
>>>> Testing of the last round of changes revealed a hang in one of the new
>>>> TLH tests. Robbin has fixed the hang, updated the existing TLH test, 
>>>> and
>>>> added another TLH test for good measure.
>>>>
>>>> Here is the updated full webrev:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/
>>>>
>>>> Here is the updated delta webrev:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-delta/
>>>>
>>>> Dan ran the bits thru the usual Mach5 tier[1-5] testing and there are
>>>> no unexpected failures.
>>>>
>>>> We welcome comments, suggestions and feedback.
>>>>
>>>> Dan, Erik and Robbin
>>>>
>>>>
>>>> On 11/15/17 3:32 PM, Daniel D. Daugherty wrote:
>>>>> Greetings,
>>>>>
>>>>> Robbin rebased the project last night/this morning to merge with 
>>>>> Thread
>>>>> Local Handshakes (TLH) and also picked up additional changesets up 
>>>>> thru:
>>>>>
>>>>>> Changeset: fa736014cf28
>>>>>> Author:??? cjplummer
>>>>>> Date:????? 2017-11-14 18:08 -0800
>>>>>> URL:http://hg.openjdk.java.net/jdk/hs/rev/fa736014cf28
>>>>>>
>>>>>> 8191049: Add alternate version of pns() that is callable from 
>>>>>> within hotspot source
>>>>>> Summary: added pns2() to debug.cpp
>>>>>> Reviewed-by: stuefe, gthornbr
>>>>>
>>>>> This is the first time we've rebased the project to bits that are this
>>>>> fresh (< 12 hours old at rebase time). We've done this because we 
>>>>> think
>>>>> we're done with this project and are in the final review-change-retest
>>>>> cycle(s)... In other words, we're not planning on making any more 
>>>>> major
>>>>> changes for this project... :-)
>>>>>
>>>>> *** Time for code reviewers to chime in on this thread! ***
>>>>>
>>>>> Here is the updated full webrev:
>>>>>
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-full/
>>>>>
>>>>> Here's is the delta webrev from the 2017.11.10 rebase:
>>>>>
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-delta/
>>>>>
>>>>> Dan has submitted the bits for the usual Mach5 tier[1-5] testing
>>>>> (and the new baseline also)...
>>>>>
>>>>> We're expecting this round to be a little noisier than usual because
>>>>> we did not rebase on a PIT snapshot so the new baseline has not been
>>>>> through Jesper's usual care-and-feeding of integration_blockers, etc.
>>>>>
>>>>> We welcome comments, suggestions and feedback.
>>>>>
>>>>> Dan, Erik and Robbin
>>>>>
>>>>>
>>>>> On 11/14/17 4:48 PM, Daniel D. Daugherty wrote:
>>>>>> Greetings,
>>>>>>
>>>>>> I rebased the project to the 2017.11.10 jdk/hs PIT snapshot.
>>>>>> (Yes, we're playing chase-the-repo...)
>>>>>>
>>>>>> Here is the updated full webrev:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-07-full/
>>>>>>
>>>>>> Unlike the previous rebase, there were no changes required to the
>>>>>> open code to get the rebased bits to build so there is no delta
>>>>>> webrev.
>>>>>>
>>>>>> These bits have been run through JPRT and I've submitted the usual
>>>>>> Mach5 tier[1-5] test run...
>>>>>>
>>>>>> We welcome comments, suggestions and feedback.
>>>>>>
>>>>>> Dan, Erik and Robbin
>>>>>>
>>>>>>
>>>>>> On 11/13/17 12:30 PM, Daniel D. Daugherty wrote:
>>>>>>> Greetings,
>>>>>>>
>>>>>>> I rebased the project to the 2017.10.26 jdk10/hs PIT snapshot.
>>>>>>>
>>>>>>> Here are the updated webrevs:
>>>>>>>
>>>>>>> Here's the mq comment for the change:
>>>>>>>
>>>>>>> ? Rebase to 2017.10.25 PIT snapshot.
>>>>>>>
>>>>>>> Here is the full webrev:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-full/
>>>>>>>
>>>>>>> And here is the delta webrev:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-delta/
>>>>>>>
>>>>>>> I ran the above bits throught Mach5 tier[1-5] testing over the 
>>>>>>> holiday
>>>>>>> weekend. Didn't see any issues in a quick look. Going to take a 
>>>>>>> closer
>>>>>>> look today.
>>>>>>>
>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>
>>>>>>> Dan, Erik and Robbin
>>>>>>>
>>>>>>>
>>>>>>> On 11/8/17 1:05 PM, Daniel D. Daugherty wrote:
>>>>>>>> Greetings,
>>>>>>>>
>>>>>>>> Resolving one of the code review comments (from both Stefan K 
>>>>>>>> and Coleen)
>>>>>>>> on jdk10-04-full required quite a few changes so it is being 
>>>>>>>> done as a
>>>>>>>> standalone patch and corresponding webrevs:
>>>>>>>>
>>>>>>>> Here's the mq comment for the change:
>>>>>>>>
>>>>>>>> ? stefank, coleenp CR - refactor most JavaThreadIterator usage 
>>>>>>>> to use
>>>>>>>> ??? JavaThreadIteratorWithHandle.
>>>>>>>>
>>>>>>>> Here is the full webrev:
>>>>>>>>
>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-full/
>>>>>>>>
>>>>>>>> And here is the delta webrev:
>>>>>>>>
>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-delta/
>>>>>>>>
>>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>>
>>>>>>>> Dan, Erik and Robbin
>>>>>>>>
>>>>>>>>
>>>>>>>> On 10/9/17 3:41 PM, Daniel D. Daugherty wrote:
>>>>>>>>> Greetings,
>>>>>>>>>
>>>>>>>>> We have a (eXtra Large) fix for the following bug:
>>>>>>>>>
>>>>>>>>> 8167108 inconsistent handling of SR_lock can lead to crashes
>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8167108
>>>>>>>>>
>>>>>>>>> This fix adds a Safe Memory Reclamation (SMR) mechanism based on
>>>>>>>>> Hazard Pointers to manage JavaThread lifecycle.
>>>>>>>>>
>>>>>>>>> Here's a PDF for the internal wiki that we've been using to 
>>>>>>>>> describe
>>>>>>>>> and track the work on this project:
>>>>>>>>>
>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/SMR_and_JavaThread_Lifecycle-JDK10-04.pdf 
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Dan has noticed that the indenting is wrong in some of the code 
>>>>>>>>> quotes
>>>>>>>>> in the PDF that are not present in the internal wiki. We don't 
>>>>>>>>> have a
>>>>>>>>> solution for that problem yet.
>>>>>>>>>
>>>>>>>>> Here's the webrev for current JDK10 version of this fix:
>>>>>>>>>
>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-04-full
>>>>>>>>>
>>>>>>>>> This fix has been run through many rounds of JPRT and Mach5 
>>>>>>>>> tier[2-5]
>>>>>>>>> testing, additional stress testing on Dan's Solaris X64 server, 
>>>>>>>>> and
>>>>>>>>> additional testing on Erik and Robbin's machines.
>>>>>>>>>
>>>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>>>
>>>>>>>>> Daniel Daugherty
>>>>>>>>> Erik Osterlund
>>>>>>>>> Robbin Ehn
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>


From erik.osterlund at oracle.com  Tue Nov 28 11:22:12 2017
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Tue, 28 Nov 2017 12:22:12 +0100
Subject: RFR (S): 8191894: Refactor weak references in JvmtiTagHashmap to
 use the Access API
In-Reply-To: <2b566339-d3f8-27ab-c5c9-670aa278c9da@oracle.com>
References: <5A1BF6F0.5030903@oracle.com>
 <88659101-1693-d5a4-a605-8f7b5ab53299@oracle.com>
 <2b566339-d3f8-27ab-c5c9-670aa278c9da@oracle.com>
Message-ID: <5A1D46E4.7020801@oracle.com>

Hi Serguei,

Thanks for the review.

/Erik

On 2017-11-28 06:43, serguei.spitsyn at oracle.com wrote:
> Hi Erik,
>
> This looks good to me.
>
> Thanks,
> Serguei
>
>
> On 11/27/17 05:22, Daniel D. Daugherty wrote:
>> Adding serviceability-dev at ... since this is JVM/TI related...
>>
>> Dan
>>
>>
>> On 11/27/17 6:28 AM, Erik ?sterlund wrote:
>>> Hi,
>>>
>>> The JVMTI tag hashmap has weak oop references that are handled using 
>>> raw oop accesses and a G1-specific SATB enqueue call when leaking 
>>> out objects from the tag map, requiring them to be marked as live by 
>>> G1.
>>>
>>> This should now be refactored to use the Access API to annotate that 
>>> these are ON_PHANTOM_OOP_REF, and refactor the raw oop loads to use 
>>> ON_PHANTOM_OOP_REF | AS_NO_KEEPALIVE.
>>>
>>> Webrev:
>>> http://cr.openjdk.java.net/~eosterlund/8191894/webrev.00/
>>>
>>> Bug:
>>> https://bugs.openjdk.java.net/browse/JDK-8191894
>>>
>>> Thanks,
>>> /Erik
>>
>



From rkennke at redhat.com  Tue Nov 28 11:22:22 2017
From: rkennke at redhat.com (Roman Kennke)
Date: Tue, 28 Nov 2017 12:22:22 +0100
Subject: RFR: 8191564: Refactor GC related servicability code into GC
 specific subclasses
In-Reply-To: <5A1C3C07.30800@oracle.com>
References: <387d2b93-cb8b-51be-a6e0-3b5f9d6cbb1c@redhat.com>
 <713325a4-3ed8-cd7d-c66d-a1c1767b6d3a@redhat.com>
 <2a7e6307-8c0d-ad4b-e50c-a7e1c2311080@redhat.com>
 <56567db1-caf6-ed05-ed12-336edfca0c75@redhat.com> <5A1C3C07.30800@oracle.com>
Message-ID: <b32b0580-8700-ec0f-bfb1-18a232e4d364@redhat.com>

Hi Erik,

Thanks for your review!

All of the things that you mentioned should be addressed in the 
following changes:

Differential:
http://cr.openjdk.java.net/~rkennke/8191564/webrev.05.diff/
Full:
http://cr.openjdk.java.net/~rkennke/8191564/webrev.05/

Also, Erik D (H) was so kind to contribute an additional testcase, which 
is also included in the above patch.

Ok now?

Also, I need a review from serviceability-dev too!

Thanks,
Roman


> 1) The code uses the "mgr" short name for "manager" in a bunch of names. 
> I would be happy if we could write out the whole thing instead of the 
> abbreviation.
> 2) It would be great if a generation would have a pointer to its 
> manager, so we do not have to pass around the manager where we already 
> pass around the generation (such as GenCollectedHeap::collect_generation).
> The generation could be created first, then the pools, then the 
> managers, then do something like generation->set_memory_manager(x).
> 3) In cmsHeap.cpp:54: maxSize should preferably not use camel case.
> 
> Otherwise I think it looks good.
> 
> Thanks,
> /Erik
> 
> On 2017-11-27 10:30, Roman Kennke wrote:
>> Erik implemented a few more refactorings and touch-ups, and here is 
>> our final (pending reviews) webrev:
>>
>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.04/
>>
>> Compared to webrev.02, it improves the handling of gc-end-message, 
>> avoids dragging the GCMemoryManager through Generation and a few 
>> little related refactorings.
>>
>> Ok to push now?
>>
>> Roman
>>
>>> After a few more discussions with Erik I made some more changes.
>>>
>>> MemoryService is now unaware of the number and meaning of GC memory 
>>> managers (minor vs major). This should be better for GCs that don't 
>>> make that distinction and/or use more different GCs (e.g. minor, 
>>> intermediate, full).
>>>
>>> This means that I needed to add some abstractions:
>>> - GCMemoryManager now has gc_end_message() which is used by 
>>> GCNotifier::pushNotification().
>>> - gc_begin() and gc_end() methods in MemoryService now accept a 
>>> GCMemoryManager* instead of bull full_gc
>>> - Same for TraceMemoryManagerStats
>>> - Generation now knows about the corresponding GCMemoryManager
>>>
>>> Please review the full change:
>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.02/
>>>
>>> Thanks, Roman
>>>
>>>
>>>> I had some off-band discussions with Erik Helin and re-did most of 
>>>> the changeset:
>>>> - The GC interface now resides in CollectedHeap, specifically the 
>>>> two methods memory_managers() and memory_pools(), and is implemented 
>>>> in the various concrete subclasses.
>>>> - Both methods return (by value) a GrowableArray<GCMemoryManager*> 
>>>> and GrowableArray<MemoryPool> respectively. Returning a 
>>>> stack-allocated GrowableArray seemed least complicated (avoid 
>>>> explicit cleanup of short-lived array object), and most 
>>>> future-proof, e.g. currently there is an implicit expectation to get 
>>>> 2 GCMemoryManagers, even though some GCs don't necessarily have two. 
>>>> The API allows for easy extension of the situation.
>>>>
>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.01/
>>>>
>>>> I think this requires reviews from both the GC and Serviceability team.
>>>>
>>>> Roman
>>>>
>>>>
>>>>> Currently, there's lots of GC specific code sprinkled over 
>>>>> src/hotspot/share/services. This change introduces a GC interface 
>>>>> for that, and moves all GC specific code to their respective 
>>>>> src/hotspot/share/gc directory.
>>>>>
>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.00/ 
>>>>> <http://cr.openjdk.java.net/%7Erkennke/8191564/webrev.00/>
>>>>>
>>>>> Testing: hotspot_gc and hotspot_serviceability, none showed 
>>>>> regressions
>>>>>
>>>>> Built minimal and server without regressions
>>>>>
>>>>> What do you think?
>>>>>
>>>>> Roman
>>>>>
>>>>>
>>>>
>>>
>>
> 



From erik.osterlund at oracle.com  Tue Nov 28 12:19:54 2017
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Tue, 28 Nov 2017 13:19:54 +0100
Subject: RFR (S): 8191888: Refactor ClassLoaderData::remove_handle to use
 the Access API
In-Reply-To: <d11416e1-a8b4-15f0-79a4-0e74c64bf60e@oracle.com>
References: <5A1BD5B3.7030808@oracle.com>
 <d11416e1-a8b4-15f0-79a4-0e74c64bf60e@oracle.com>
Message-ID: <5A1D546A.1040601@oracle.com>

Hi David,

That is a good question.
There is a description (written by a GC person, but still) for each 
decorator in the access.hpp file where the decorators are declared. I 
try to go in to some details there when you should and should not use a 
decorator. If we find that the text might have been biased from a GC 
perspective, we can update the comments.

The previous situation has been to use naked accesses, and then suddenly 
a GC person jumps in moaning about the lack of explicit #if 
INCLUDE_ALL_GCS if (UseG1) SATB barrier goo, or manual insertion of 
post-write barriers if a CAS was successful. There was no good way of 
knowing you had to manually insert these barriers, what barriers were 
offered to shared code, or what their semantics were.

Now at least you have a set of decorators to choose from, with some 
automatic verification if you select nonsense decorators, and 
documentation in one place what they are used for and what they mean. So 
if you do not know what you need, at least you can read the descriptions 
and hopefully figure it out, which was impossible before.

Having said that, I believe things like Coleen's OopHandle and 
PhantomOopHandle will build a layer on top of Access with possibly a few 
less details exposed that are even simpler to use for the conservative 
non-GC person that just wants the thing to work as simple as possible 
and wants to stay as far away from GC land as possible, which is sane.

And if you feel uncertain, you can always loop me in any time, and I 
will probably say "there's a decorator for that".

I hope this answers your question.

Thanks,
/Erik

On 2017-11-28 07:59, David Holmes wrote:
> Hi Erik,
>
> Is there a non-GC person's guide to what the different forms of 
> RootAccess<XXX> mean and when to use them? Or do we expect a GC person 
> to always jump in to show where such things are needed? ;-)
>
> Thanks,
> David
>
> On 27/11/2017 7:06 PM, Erik ?sterlund wrote:
>> Hi,
>>
>> The ClassLoaderData::remove_handle() member function currently uses 
>> explicit G1 SATB barriers to remove an oop from the root set, as 
>> these handles are not necessarily walked by the GC in a safepoint. 
>> Therefore G1 needs pre-write barriers.
>>
>> This should now be modeled as a 
>> RootAccess<IN_CONCURRENT_ROOT>::oop_store instead. This maps to 
>> performing a pre-write SATB barrier with G1, but other GCs are free 
>> to do other things as necessary.
>>
>> Webrev:
>> http://cr.openjdk.java.net/~eosterlund/8191888/webrev.00/
>>
>> Bug:
>> https://bugs.openjdk.java.net/browse/JDK-8191888
>>
>> Thanks,
>> /Erik



From daniel.daugherty at oracle.com  Tue Nov 28 12:31:02 2017
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Tue, 28 Nov 2017 07:31:02 -0500
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <157a90ee-35c9-1adf-490a-9910f3ee646f@oracle.com>
References: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
 <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>
 <836b5c6c-5eef-9245-c16a-e71372093d9a@oracle.com>
 <82d0e473-a24f-9feb-82e9-7d04c4d7a91d@oracle.com>
 <354f7ece-66bc-c065-9ee8-f55abcc1c23e@oracle.com>
 <0ff8df8e-e93c-02d2-cb97-b13b1ac0f53f@oracle.com>
 <5365606e-4036-b5d7-9bac-e91e42f38d01@oracle.com>
 <567339a6-e36a-5416-5c34-2a87cbfa6cdd@oracle.com>
 <5976a6bf-c9b0-c0d5-d5f9-fcdf7dc9a1a8@oracle.com>
 <157a90ee-35c9-1adf-490a-9910f3ee646f@oracle.com>
Message-ID: <a7be3013-a939-b0f1-57fd-13cea26d4da3@oracle.com>

On 11/28/17 3:01 AM, David Holmes wrote:
> Just for the record ...
>
> On 23/11/2017 6:20 PM, Robbin Ehn wrote:
>> Thanks Dan for dragging this freight train to the docks, it's time to 
>> ship it!
>
> I agree. The latest delta seems fine to me.

Thanks!


>
>> Created follow-up bug:
>> 8191809: Threads::number_of_threads() is not 'MT-safe'
>> https://bugs.openjdk.java.net/browse/JDK-8191809
>
> Just updated that bug - I don't see any MT issues there. :)
>
> Cheers,
> David
>
> PS. Dan: yes JPRT was still doing 32-bit ARM a "month or two back". 
> 64-bit atomics should not be a concern. That said I thought all the 
> atomic updates were done for ARM etc.

The atomic template updates were done for ARM. It was the new
template stuff that gave me the error about a match not being
available for one of the operations on a 64-bit field. I'm
going to guess that pre-template, we would have gotten a
runtime error or something...

I really wish I had saved that JPRT build log... :-(

Dan


>
>> /Robbin
>>
>> On 2017-11-23 03:16, Daniel D. Daugherty wrote:
>>> Greetings,
>>>
>>> I've made minor tweaks to the Thread-SMR project based on the remaining
>>> code review comments over the last couple of days. I've also rebased 
>>> the
>>> project to jdk/hs bits as of mid-afternoon (my TZ) on 2017.11.22. I'm
>>> running baseline Mach5 Tier[1-5] testing and prototype Mach5 Tier[1-5]
>>> testing...
>>>
>>> Here's a delta webrev for the latest round of tweaks:
>>>
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-11-delta
>>>
>>>
>>> Here's the proposed commit message:
>>>
>>> $ cat SMR_prototype_10/open/commit.txt
>>> 8167108: inconsistent handling of SR_lock can lead to crashes
>>> Summary: Add Thread Safe Memory Reclamation (Thread-SMR) mechanism.
>>> Reviewed-by: coleenp, dcubed, dholmes, eosterlund, gthornbr, 
>>> kbarrett, rehn, sspitsyn, stefank
>>> Contributed-by: daniel.daugherty at oracle.com, 
>>> erik.osterlund at oracle.com, robbin.ehn at oracle.com
>>>
>>> I've gone through 880+ emails in my archive for this project and added
>>> anyone that made a code review comment. Reviewers are not in my usual
>>> by-comment-posting order; it was way too hard to figure that out... So
>>> reviewers and contributors are simply in alpha-sort order...
>>>
>>> Here's the collection of follow-up bugs that I filed based on sifting
>>> through the email archive:
>>>
>>> 8191784 JavaThread::collect_counters() should stop using Threads_lock
>>> https://bugs.openjdk.java.net/browse/JDK-8191784
>>>
>>> 8191785 revoke_bias() needs a new assertion
>>> https://bugs.openjdk.java.net/browse/JDK-8191785
>>>
>>> 8191786 Thread-SMR hash table size should be dynamic
>>> https://bugs.openjdk.java.net/browse/JDK-8191786
>>>
>>> 8191787 move private inline functions from thread.inline.hpp -> 
>>> thread.cpp
>>> https://bugs.openjdk.java.net/browse/JDK-8191787
>>>
>>> 8191789 migrate more Thread-SMR stuff from thread.[ch]pp -> 
>>> threadSMR.[ch]pp
>>> https://bugs.openjdk.java.net/browse/JDK-8191789
>>>
>>> 8191798 redo nested ThreadsListHandle to drop Threads_lock
>>> https://bugs.openjdk.java.net/browse/JDK-8191789
>>>
>>> I have undoubtedly missed at least one future idea that someone had
>>> and for that my apologies...
>>>
>>> Thanks again to everyone who contributed to the Thread-SMR project!!
>>>
>>> Special thanks goes to Karen Kinnear, Kim Barrett and David Holmes for
>>> their rigorous review of the design that we documented on the wiki.
>>> Thanks for keeping us honest!
>>>
>>> I also have to thank my co-contributor Erik ?sterlund for starting the
>>> ball rolling on this crazy project with his presentation on Safe Memory
>>> Reclamation, for the initial prototype and for all your patches. Erik,
>>> hopefully we got far enough in your crazy vision... :-)
>>>
>>> Thanks to my co-contributor Robbin Ehn for proposing that we do early
>>> code reviews in the form of patches. Don't like something? Fix it with
>>> a patch and a new test if appropriate!! A pretty cool way to work that
>>> was new to me.
>>>
>>> Finally, many thanks to Jerry Thornbrugh for all the early code 
>>> reviews,
>>> whiteboard chats and phone calls. Thanks for asking the right questions
>>> and pointing out some places where our logic was faulty. Also thanks 
>>> for
>>> keeping me sane when I was literally on my own in Monument, CO.
>>>
>>> Dan
>>>
>>>
>>> On 11/21/17 7:12 PM, Daniel D. Daugherty wrote:
>>>> Greetings,
>>>>
>>>> *** We are wrapping up code review on this project so it is time ***
>>>> *** for the various code reviewers to chime in one last time. ***
>>>>
>>>> In this latest round, we had three different reviewers chime in so 
>>>> we're
>>>> doing delta webrevs for each of those resolutions:
>>>>
>>>> David H's resolutions:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.09,10.10.0-delta/ 
>>>>
>>>>
>>>> ? mq comment for dholmes_CR:
>>>> ??? - Fix indents, trailing spaces and various typos.
>>>> ??? - Add descriptions for the '_cnt', '_max' and '_times" fields,
>>>> ????? add impl notes to document the type choices.
>>>>
>>>> ? src/hotspot/share/runtime/globals.hpp change is white-space only 
>>>> so it
>>>> ? is only visible via the file's patch link.
>>>>
>>>>
>>>> Robin W's resolutions:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.10.0,10.10.1-delta/ 
>>>>
>>>>
>>>> ? mq comment for robinw_CR:
>>>> ??? - Fix some inefficient code, update some comments, fix some 
>>>> indents,
>>>> ????? and add some 'const' specifiers.
>>>>
>>>> ? src/hotspot/share/runtime/vm_operations.hpp change is white-space 
>>>> only so
>>>> ? it is only visible via the file's patch link.
>>>>
>>>>
>>>> Coleen's resolutions:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.10.1,10.10.2-delta/ 
>>>>
>>>>
>>>> ? mq comment for coleenp_CR:
>>>> ??? - Change ThreadsList::_threads from 'mtGC' -> 'mtThread',
>>>> ??? - add header comment to threadSMR.hpp file,
>>>> ??? - cleanup JavaThreadIteratorWithHandle ctr,
>>>> ??? - make ErrorHandling more efficient.
>>>>
>>>>
>>>> Some folks prefer to see all of the delta webrevs together so here is
>>>> a delta webrev relative to jdk10-09-full:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.09,10.10.2-delta/ 
>>>>
>>>>
>>>> ? src/hotspot/share/runtime/globals.hpp and
>>>> ? src/hotspot/share/runtime/vm_operations.hpp changes are 
>>>> white-space only
>>>> ? so they are only visible via the file's patch link.
>>>>
>>>>
>>>> And, of course, some folks prefer to see everything all at once so 
>>>> here
>>>> is the latest full webrev:
>>>>
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.10-full/
>>>>
>>>>
>>>> Dan has submitted the bits for the usual Mach5 tier[1-5] testing. The
>>>> prelim Mach5 tier1 (JPRT equivalent) on these bits passed just fine...
>>>>
>>>> The project is currently baselined on Jesper's 2017-11-17 PIT snapshot
>>>> so there will be some catching up to do before integration...
>>>>
>>>> We welcome comments, suggestions and feedback.
>>>>
>>>> Dan, Erik and Robbin
>>>>
>>>>
>>>> On 11/18/17 8:49 PM, Daniel D. Daugherty wrote:
>>>>> Greetings,
>>>>>
>>>>> Testing of the last round of changes revealed a hang in one of the 
>>>>> new
>>>>> TLH tests. Robbin has fixed the hang, updated the existing TLH 
>>>>> test, and
>>>>> added another TLH test for good measure.
>>>>>
>>>>> Here is the updated full webrev:
>>>>>
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/
>>>>>
>>>>> Here is the updated delta webrev:
>>>>>
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-delta/
>>>>>
>>>>> Dan ran the bits thru the usual Mach5 tier[1-5] testing and there are
>>>>> no unexpected failures.
>>>>>
>>>>> We welcome comments, suggestions and feedback.
>>>>>
>>>>> Dan, Erik and Robbin
>>>>>
>>>>>
>>>>> On 11/15/17 3:32 PM, Daniel D. Daugherty wrote:
>>>>>> Greetings,
>>>>>>
>>>>>> Robbin rebased the project last night/this morning to merge with 
>>>>>> Thread
>>>>>> Local Handshakes (TLH) and also picked up additional changesets 
>>>>>> up thru:
>>>>>>
>>>>>>> Changeset: fa736014cf28
>>>>>>> Author:??? cjplummer
>>>>>>> Date:????? 2017-11-14 18:08 -0800
>>>>>>> URL:http://hg.openjdk.java.net/jdk/hs/rev/fa736014cf28
>>>>>>>
>>>>>>> 8191049: Add alternate version of pns() that is callable from 
>>>>>>> within hotspot source
>>>>>>> Summary: added pns2() to debug.cpp
>>>>>>> Reviewed-by: stuefe, gthornbr
>>>>>>
>>>>>> This is the first time we've rebased the project to bits that are 
>>>>>> this
>>>>>> fresh (< 12 hours old at rebase time). We've done this because we 
>>>>>> think
>>>>>> we're done with this project and are in the final 
>>>>>> review-change-retest
>>>>>> cycle(s)... In other words, we're not planning on making any more 
>>>>>> major
>>>>>> changes for this project... :-)
>>>>>>
>>>>>> *** Time for code reviewers to chime in on this thread! ***
>>>>>>
>>>>>> Here is the updated full webrev:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-full/
>>>>>>
>>>>>> Here's is the delta webrev from the 2017.11.10 rebase:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-delta/
>>>>>>
>>>>>> Dan has submitted the bits for the usual Mach5 tier[1-5] testing
>>>>>> (and the new baseline also)...
>>>>>>
>>>>>> We're expecting this round to be a little noisier than usual because
>>>>>> we did not rebase on a PIT snapshot so the new baseline has not been
>>>>>> through Jesper's usual care-and-feeding of integration_blockers, 
>>>>>> etc.
>>>>>>
>>>>>> We welcome comments, suggestions and feedback.
>>>>>>
>>>>>> Dan, Erik and Robbin
>>>>>>
>>>>>>
>>>>>> On 11/14/17 4:48 PM, Daniel D. Daugherty wrote:
>>>>>>> Greetings,
>>>>>>>
>>>>>>> I rebased the project to the 2017.11.10 jdk/hs PIT snapshot.
>>>>>>> (Yes, we're playing chase-the-repo...)
>>>>>>>
>>>>>>> Here is the updated full webrev:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-07-full/
>>>>>>>
>>>>>>> Unlike the previous rebase, there were no changes required to the
>>>>>>> open code to get the rebased bits to build so there is no delta
>>>>>>> webrev.
>>>>>>>
>>>>>>> These bits have been run through JPRT and I've submitted the usual
>>>>>>> Mach5 tier[1-5] test run...
>>>>>>>
>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>
>>>>>>> Dan, Erik and Robbin
>>>>>>>
>>>>>>>
>>>>>>> On 11/13/17 12:30 PM, Daniel D. Daugherty wrote:
>>>>>>>> Greetings,
>>>>>>>>
>>>>>>>> I rebased the project to the 2017.10.26 jdk10/hs PIT snapshot.
>>>>>>>>
>>>>>>>> Here are the updated webrevs:
>>>>>>>>
>>>>>>>> Here's the mq comment for the change:
>>>>>>>>
>>>>>>>> ? Rebase to 2017.10.25 PIT snapshot.
>>>>>>>>
>>>>>>>> Here is the full webrev:
>>>>>>>>
>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-full/
>>>>>>>>
>>>>>>>> And here is the delta webrev:
>>>>>>>>
>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-delta/
>>>>>>>>
>>>>>>>> I ran the above bits throught Mach5 tier[1-5] testing over the 
>>>>>>>> holiday
>>>>>>>> weekend. Didn't see any issues in a quick look. Going to take a 
>>>>>>>> closer
>>>>>>>> look today.
>>>>>>>>
>>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>>
>>>>>>>> Dan, Erik and Robbin
>>>>>>>>
>>>>>>>>
>>>>>>>> On 11/8/17 1:05 PM, Daniel D. Daugherty wrote:
>>>>>>>>> Greetings,
>>>>>>>>>
>>>>>>>>> Resolving one of the code review comments (from both Stefan K 
>>>>>>>>> and Coleen)
>>>>>>>>> on jdk10-04-full required quite a few changes so it is being 
>>>>>>>>> done as a
>>>>>>>>> standalone patch and corresponding webrevs:
>>>>>>>>>
>>>>>>>>> Here's the mq comment for the change:
>>>>>>>>>
>>>>>>>>> ? stefank, coleenp CR - refactor most JavaThreadIterator usage 
>>>>>>>>> to use
>>>>>>>>> ??? JavaThreadIteratorWithHandle.
>>>>>>>>>
>>>>>>>>> Here is the full webrev:
>>>>>>>>>
>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-full/
>>>>>>>>>
>>>>>>>>> And here is the delta webrev:
>>>>>>>>>
>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-delta/
>>>>>>>>>
>>>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>>>
>>>>>>>>> Dan, Erik and Robbin
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 10/9/17 3:41 PM, Daniel D. Daugherty wrote:
>>>>>>>>>> Greetings,
>>>>>>>>>>
>>>>>>>>>> We have a (eXtra Large) fix for the following bug:
>>>>>>>>>>
>>>>>>>>>> 8167108 inconsistent handling of SR_lock can lead to crashes
>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8167108
>>>>>>>>>>
>>>>>>>>>> This fix adds a Safe Memory Reclamation (SMR) mechanism based on
>>>>>>>>>> Hazard Pointers to manage JavaThread lifecycle.
>>>>>>>>>>
>>>>>>>>>> Here's a PDF for the internal wiki that we've been using to 
>>>>>>>>>> describe
>>>>>>>>>> and track the work on this project:
>>>>>>>>>>
>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/SMR_and_JavaThread_Lifecycle-JDK10-04.pdf 
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Dan has noticed that the indenting is wrong in some of the 
>>>>>>>>>> code quotes
>>>>>>>>>> in the PDF that are not present in the internal wiki. We 
>>>>>>>>>> don't have a
>>>>>>>>>> solution for that problem yet.
>>>>>>>>>>
>>>>>>>>>> Here's the webrev for current JDK10 version of this fix:
>>>>>>>>>>
>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-04-full
>>>>>>>>>>
>>>>>>>>>> This fix has been run through many rounds of JPRT and Mach5 
>>>>>>>>>> tier[2-5]
>>>>>>>>>> testing, additional stress testing on Dan's Solaris X64 
>>>>>>>>>> server, and
>>>>>>>>>> additional testing on Erik and Robbin's machines.
>>>>>>>>>>
>>>>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>>>>>
>>>>>>>>>> Daniel Daugherty
>>>>>>>>>> Erik Osterlund
>>>>>>>>>> Robbin Ehn
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>



From thomas.schatzl at oracle.com  Tue Nov 28 13:07:13 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Tue, 28 Nov 2017 14:07:13 +0100
Subject: RFR (S): 8191888: Refactor ClassLoaderData::remove_handle to
 use the Access API
In-Reply-To: <5A1BD5B3.7030808@oracle.com>
References: <5A1BD5B3.7030808@oracle.com>
Message-ID: <1511874433.2867.0.camel@oracle.com>

Hi,

On Mon, 2017-11-27 at 10:06 +0100, Erik ?sterlund wrote:
> Hi,
> 
> The ClassLoaderData::remove_handle() member function currently uses 
> explicit G1 SATB barriers to remove an oop from the root set, as
> these handles are not necessarily walked by the GC in a safepoint.
> Therefore G1 needs pre-write barriers.
> 
> This should now be modeled as a 
> RootAccess<IN_CONCURRENT_ROOT>::oop_store instead. This maps to 
> performing a pre-write SATB barrier with G1, but other GCs are free
> to do other things as necessary.
> 
> Webrev:
> http://cr.openjdk.java.net/~eosterlund/8191888/webrev.00/
> 
> Bug:
> https://bugs.openjdk.java.net/browse/JDK-8191888

  looks good.

Thomas


From erik.osterlund at oracle.com  Tue Nov 28 14:09:49 2017
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Tue, 28 Nov 2017 15:09:49 +0100
Subject: RFR (S): 8191888: Refactor ClassLoaderData::remove_handle to use
 the Access API
In-Reply-To: <1511874433.2867.0.camel@oracle.com>
References: <5A1BD5B3.7030808@oracle.com> <1511874433.2867.0.camel@oracle.com>
Message-ID: <5A1D6E2D.70405@oracle.com>

Hi Thomas,

Thanks for the review.

/Erik

On 2017-11-28 14:07, Thomas Schatzl wrote:
> Hi,
>
> On Mon, 2017-11-27 at 10:06 +0100, Erik ?sterlund wrote:
>> Hi,
>>
>> The ClassLoaderData::remove_handle() member function currently uses
>> explicit G1 SATB barriers to remove an oop from the root set, as
>> these handles are not necessarily walked by the GC in a safepoint.
>> Therefore G1 needs pre-write barriers.
>>
>> This should now be modeled as a
>> RootAccess<IN_CONCURRENT_ROOT>::oop_store instead. This maps to
>> performing a pre-write SATB barrier with G1, but other GCs are free
>> to do other things as necessary.
>>
>> Webrev:
>> http://cr.openjdk.java.net/~eosterlund/8191888/webrev.00/
>>
>> Bug:
>> https://bugs.openjdk.java.net/browse/JDK-8191888
>    looks good.
>
> Thomas



From per.liden at oracle.com  Tue Nov 28 15:07:43 2017
From: per.liden at oracle.com (Per Liden)
Date: Tue, 28 Nov 2017 16:07:43 +0100
Subject: RFR (S): 8191888: Refactor ClassLoaderData::remove_handle to use
 the Access API
In-Reply-To: <5A1BD5B3.7030808@oracle.com>
References: <5A1BD5B3.7030808@oracle.com>
Message-ID: <f3c4c351-c8f7-9845-f729-bfebfaf60528@oracle.com>

Looks good!

/Per

On 2017-11-27 10:06, Erik ?sterlund wrote:
> Hi,
>
> The ClassLoaderData::remove_handle() member function currently uses
> explicit G1 SATB barriers to remove an oop from the root set, as these
> handles are not necessarily walked by the GC in a safepoint. Therefore
> G1 needs pre-write barriers.
>
> This should now be modeled as a
> RootAccess<IN_CONCURRENT_ROOT>::oop_store instead. This maps to
> performing a pre-write SATB barrier with G1, but other GCs are free to
> do other things as necessary.
>
> Webrev:
> http://cr.openjdk.java.net/~eosterlund/8191888/webrev.00/
>
> Bug:
> https://bugs.openjdk.java.net/browse/JDK-8191888
>
> Thanks,
> /Erik


From erik.osterlund at oracle.com  Tue Nov 28 15:32:25 2017
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Tue, 28 Nov 2017 16:32:25 +0100
Subject: RFR (S): 8191888: Refactor ClassLoaderData::remove_handle to use
 the Access API
In-Reply-To: <f3c4c351-c8f7-9845-f729-bfebfaf60528@oracle.com>
References: <5A1BD5B3.7030808@oracle.com>
 <f3c4c351-c8f7-9845-f729-bfebfaf60528@oracle.com>
Message-ID: <5A1D8189.10603@oracle.com>

Hi Per,

Thanks for the review.

/Erik

On 2017-11-28 16:07, Per Liden wrote:
> Looks good!
>
> /Per
>
> On 2017-11-27 10:06, Erik ?sterlund wrote:
>> Hi,
>>
>> The ClassLoaderData::remove_handle() member function currently uses
>> explicit G1 SATB barriers to remove an oop from the root set, as these
>> handles are not necessarily walked by the GC in a safepoint. Therefore
>> G1 needs pre-write barriers.
>>
>> This should now be modeled as a
>> RootAccess<IN_CONCURRENT_ROOT>::oop_store instead. This maps to
>> performing a pre-write SATB barrier with G1, but other GCs are free to
>> do other things as necessary.
>>
>> Webrev:
>> http://cr.openjdk.java.net/~eosterlund/8191888/webrev.00/
>>
>> Bug:
>> https://bugs.openjdk.java.net/browse/JDK-8191888
>>
>> Thanks,
>> /Erik



From erik.osterlund at oracle.com  Tue Nov 28 16:16:39 2017
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Tue, 28 Nov 2017 17:16:39 +0100
Subject: RFR: 8191564: Refactor GC related servicability code into GC
 specific subclasses
In-Reply-To: <b32b0580-8700-ec0f-bfb1-18a232e4d364@redhat.com>
References: <387d2b93-cb8b-51be-a6e0-3b5f9d6cbb1c@redhat.com>
 <713325a4-3ed8-cd7d-c66d-a1c1767b6d3a@redhat.com>
 <2a7e6307-8c0d-ad4b-e50c-a7e1c2311080@redhat.com>
 <56567db1-caf6-ed05-ed12-336edfca0c75@redhat.com> <5A1C3C07.30800@oracle.com>
 <b32b0580-8700-ec0f-bfb1-18a232e4d364@redhat.com>
Message-ID: <5A1D8BE7.4070008@oracle.com>

Hi Roman,

This looks better now. Nice job.
I wonder though, is it possible to extract the creation of managers and 
pools to a separate function for each collected heap?
Now I see managers are created in e.g. CMS constructor, and the pools 
are created in CMSHeap::initialize(). Perhaps initialize could call 
initialize_serviceability() that sets up those things, so that they are 
nicely separated. Unless of course there is a good reason why the 
presence of managers is needed earlier than that in the bootstrapping.

Otherwise I think this looks good!

Thanks,
/Erik

On 2017-11-28 12:22, Roman Kennke wrote:
> Hi Erik,
>
> Thanks for your review!
>
> All of the things that you mentioned should be addressed in the 
> following changes:
>
> Differential:
> http://cr.openjdk.java.net/~rkennke/8191564/webrev.05.diff/
> Full:
> http://cr.openjdk.java.net/~rkennke/8191564/webrev.05/
>
> Also, Erik D (H) was so kind to contribute an additional testcase, 
> which is also included in the above patch.
>
> Ok now?
>
> Also, I need a review from serviceability-dev too!
>
> Thanks,
> Roman
>
>
>> 1) The code uses the "mgr" short name for "manager" in a bunch of 
>> names. I would be happy if we could write out the whole thing instead 
>> of the abbreviation.
>> 2) It would be great if a generation would have a pointer to its 
>> manager, so we do not have to pass around the manager where we 
>> already pass around the generation (such as 
>> GenCollectedHeap::collect_generation).
>> The generation could be created first, then the pools, then the 
>> managers, then do something like generation->set_memory_manager(x).
>> 3) In cmsHeap.cpp:54: maxSize should preferably not use camel case.
>>
>> Otherwise I think it looks good.
>>
>> Thanks,
>> /Erik
>>
>> On 2017-11-27 10:30, Roman Kennke wrote:
>>> Erik implemented a few more refactorings and touch-ups, and here is 
>>> our final (pending reviews) webrev:
>>>
>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.04/
>>>
>>> Compared to webrev.02, it improves the handling of gc-end-message, 
>>> avoids dragging the GCMemoryManager through Generation and a few 
>>> little related refactorings.
>>>
>>> Ok to push now?
>>>
>>> Roman
>>>
>>>> After a few more discussions with Erik I made some more changes.
>>>>
>>>> MemoryService is now unaware of the number and meaning of GC memory 
>>>> managers (minor vs major). This should be better for GCs that don't 
>>>> make that distinction and/or use more different GCs (e.g. minor, 
>>>> intermediate, full).
>>>>
>>>> This means that I needed to add some abstractions:
>>>> - GCMemoryManager now has gc_end_message() which is used by 
>>>> GCNotifier::pushNotification().
>>>> - gc_begin() and gc_end() methods in MemoryService now accept a 
>>>> GCMemoryManager* instead of bull full_gc
>>>> - Same for TraceMemoryManagerStats
>>>> - Generation now knows about the corresponding GCMemoryManager
>>>>
>>>> Please review the full change:
>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.02/
>>>>
>>>> Thanks, Roman
>>>>
>>>>
>>>>> I had some off-band discussions with Erik Helin and re-did most of 
>>>>> the changeset:
>>>>> - The GC interface now resides in CollectedHeap, specifically the 
>>>>> two methods memory_managers() and memory_pools(), and is 
>>>>> implemented in the various concrete subclasses.
>>>>> - Both methods return (by value) a GrowableArray<GCMemoryManager*> 
>>>>> and GrowableArray<MemoryPool> respectively. Returning a 
>>>>> stack-allocated GrowableArray seemed least complicated (avoid 
>>>>> explicit cleanup of short-lived array object), and most 
>>>>> future-proof, e.g. currently there is an implicit expectation to 
>>>>> get 2 GCMemoryManagers, even though some GCs don't necessarily 
>>>>> have two. The API allows for easy extension of the situation.
>>>>>
>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.01/
>>>>>
>>>>> I think this requires reviews from both the GC and Serviceability 
>>>>> team.
>>>>>
>>>>> Roman
>>>>>
>>>>>
>>>>>> Currently, there's lots of GC specific code sprinkled over 
>>>>>> src/hotspot/share/services. This change introduces a GC interface 
>>>>>> for that, and moves all GC specific code to their respective 
>>>>>> src/hotspot/share/gc directory.
>>>>>>
>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.00/ 
>>>>>> <http://cr.openjdk.java.net/%7Erkennke/8191564/webrev.00/>
>>>>>>
>>>>>> Testing: hotspot_gc and hotspot_serviceability, none showed 
>>>>>> regressions
>>>>>>
>>>>>> Built minimal and server without regressions
>>>>>>
>>>>>> What do you think?
>>>>>>
>>>>>> Roman
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>



From stefan.johansson at oracle.com  Tue Nov 28 16:25:34 2017
From: stefan.johansson at oracle.com (Stefan Johansson)
Date: Tue, 28 Nov 2017 17:25:34 +0100
Subject: RFR: 8191821: Finer granularity for GC verification
Message-ID: <2319511b-8979-92b9-34f2-6a4c1af46ebf@oracle.com>

Hi,

Please review this change for enhancement:
https://bugs.openjdk.java.net/browse/JDK-8191821

Webrev:
http://cr.openjdk.java.net/~sjohanss/8191821/00/

Summary:
Heap verification is a very good way to track down GC bugs, but it comes 
with a lot of overhead. Using VerifyBeforeGC, VerifyDuringGC and 
VerifyAfterGC often slows down the execution more than is needed since 
we sometimes only want to verify certain types of GCs. This change adds 
this feature for G1 by adding a new diagnostic flag VerifyGCType. The 
new flag currently only works with G1 but can easily be added for more 
GCs if needed. The type of the flag is ccstrlist which means the option 
can be used multiple times to allow more than one type to be verified. 
The types available for G1 is, young, mixed, remark, cleanup and full. 
If the flag is not specified all GCs are verified.

Note that Verify/Before/After/During/GC is still needed to decide what 
to verify, VerifyGCType only describes when.

Testing:
* Added new Gtest for G1HeapVerifier functionality.
* Added new Jtreg test for basic command line functionality.
* Executed Jtreg tests through mach5 to make sure it passes on all 
platforms.

Thanks,
Stefan


From Derek.White at cavium.com  Tue Nov 28 16:32:31 2017
From: Derek.White at cavium.com (White, Derek)
Date: Tue, 28 Nov 2017 16:32:31 +0000
Subject: RFR (XS): 8188877: Improper synchronization in offer_termination
In-Reply-To: <1511775121.2398.4.camel@oracle.com>
References: <CY1PR0701MB16328948D4FDF19FBC30F65684710@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <3229BC22-BB8A-4D3D-8440-AB799AB5394F@oracle.com>
 <CY1PR0701MB16328E4C6563BE60CF8E99D084750@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <4D9972BA-B76C-4FB7-B24D-31D507DB1A6A@oracle.com>
 <1510821711.5332.3.camel@oracle.com>
 <CY1PR0701MB163288CD9B58A163B2F16F7C84230@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <ba0c0855-188a-decf-932b-b11200dc9fc4@redhat.com>
 <1511775028.2398.3.camel@oracle.com> <1511775121.2398.4.camel@oracle.com>
Message-ID: <CY1PR0701MB1632508D3F6C48E9FD3E87AD843A0@CY1PR0701MB1632.namprd07.prod.outlook.com>

Thomas, Kim, Andrew, Andrew,

Thanks for the review comments and push!

 - Derek

> -----Original Message-----
> From: Thomas Schatzl [mailto:thomas.schatzl at oracle.com]
> Sent: Monday, November 27, 2017 4:32 AM
> To: Andrew Haley <aph at redhat.com>; White, Derek
> <Derek.White at cavium.com>; Kim Barrett <kim.barrett at oracle.com>
> Cc: hotspot-gc-dev at openjdk.java.net
> Subject: Re: RFR (XS): 8188877: Improper synchronization in
> offer_termination
> 
> Hi again,
> 
> On Mon, 2017-11-27 at 10:30 +0100, Thomas Schatzl wrote:
> > Hi all,
> >
> > On Wed, 2017-11-22 at 09:13 +0000, Andrew Haley wrote:
> > > On 21/11/17 21:53, White, Derek wrote:
> > > > My understanding is that the "acquire" semantics are entirely
> > > > about memory ordering, within a CPU. In particular it prevents
> > > > "following loads" from executing before the "load acquire".
> > > >
> > > >
> > > > There is nothing in the "load acquire" that causes it to
> > > > synchronize with the memory system more or less quickly than a
> > > > naked load.
> > >
> > > The abstract architecture only specifies things in terms of ordering
> > > between loads, but it has to be implemented somehow, and this is
> > > MESI or something similar.  Stores cause invalidate messages to be
> > > sent, and these are put into the reader's invalidate queue.  When
> > > that reader executes a load barrier it marks all the entries
> > > currently in its invalidate queue.  The next load will wait until
> > > all marked entries have been applied to the reader's cache.
> > >
> > > > Either kind of read will eventually notice that its local cached
> > > > value has been invalidated and will fetch the updated value.
> > >
> > > Eventually, yes.
> > >
> >
> >   so, summing up this discussion, the change is good to go? I think we
> > can always add any implementation dependent optimizations later.
> >
> > If everybody agrees, I will push it.
> 
>   and btw, looks good :)
> 
> Thomas


From erik.osterlund at oracle.com  Tue Nov 28 16:50:09 2017
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Tue, 28 Nov 2017 17:50:09 +0100
Subject: RFR (S): 8192003: Refactor weak references in StringTable to use the
 Access API
Message-ID: <5A1D93C1.3020605@oracle.com>

Hi,

The StringTable contains weak references to oops. Today the weak 
semantics is managed using explicit calls to G1 SATB enqueue barriers.

Now that the Access API is available, it should be used instead as it is 
more modular.

This change fixes that by making these oops ON_PHANTOM_OOP_REF with a 
corresponding AS_NO_KEEPALIVE accessor when we do not want to keep it 
alive, much like my previous changes to other weak tables.

Webrev:
http://cr.openjdk.java.net/~eosterlund/8192003/webrev.00/

Bug:
https://bugs.openjdk.java.net/browse/JDK-8192003

Thanks,
/Erik


From vladimir.kozlov at oracle.com  Tue Nov 28 17:16:20 2017
From: vladimir.kozlov at oracle.com (Vladimir Kozlov)
Date: Tue, 28 Nov 2017 09:16:20 -0800
Subject: RFR(L): 8186027: C2: loop strip mining
In-Reply-To: <dk68tequ0q5.fsf@rwestrel.remote.csb>
References: <dk6k20cxtak.fsf@rwestrel.remote.csb>
 <35a024b5-6632-0488-a001-14f960257fc7@oracle.com>
 <dk67evjnvww.fsf@rwestrel.remote.csb>
 <8b9f8985-daae-b3b4-ba48-3d9c6185ed95@oracle.com>
 <0972a0db-2115-daa3-9990-7d58915a74a5@oracle.com>
 <dk61sktw8hn.fsf@rwestrel.remote.csb>
 <3e3e2048-5ee5-f3c2-8ec0-cdd36cff07af@oracle.com>
 <dk64lpmuzrg.fsf@rwestrel.remote.csb>
 <7ae8403a-b650-5171-5d79-0107d7d99059@oracle.com>
 <dk6shd5t6x7.fsf@rwestrel.remote.csb>
 <f2a0b0c2-6642-785c-0e47-9d62ad8dae9f@oracle.com>
 <1511515343.2425.11.camel@oracle.com>
 <45aa8e04-9244-4089-001f-acba24e7e4ee@oracle.com>
 <dk68tequ0q5.fsf@rwestrel.remote.csb>
Message-ID: <4a7c6d8c-10ed-6fb0-1238-e1cf9b65baa4@oracle.com>

UseCountedLoopSafepoints should be false by default too. Otherwise code in arguments.cpp will set LoopStripMiningIter to 
1. I will remove next change in c2_globals.hpp:

-  product(bool, UseCountedLoopSafepoints, false,                            \
+  product(bool, UseCountedLoopSafepoints, true,                             \

Thanks,
Vladimir

On 11/28/17 3:00 AM, Roland Westrelin wrote:
> 
> Hi Vladimir,
> 
>> Roland, please, prepare changeset.
> 
> Here is the changeset:
> 
> http://cr.openjdk.java.net/~roland/8186027/loop-strip-mining.patch
> 
> It includes the patch below to enable it with G1.
> 
> Roland.
> 
> diff --git a/src/hotspot/share/gc/g1/g1Arguments.cpp b/src/hotspot/share/gc/g1/g1Arguments.cpp
> --- a/src/hotspot/share/gc/g1/g1Arguments.cpp
> +++ b/src/hotspot/share/gc/g1/g1Arguments.cpp
> @@ -92,6 +92,16 @@
>     }
>   
>     log_trace(gc)("MarkStackSize: %uk  MarkStackSizeMax: %uk", (unsigned int) (MarkStackSize / K), (uint) (MarkStackSizeMax / K));
> +
> +#ifdef COMPILER2
> +  // Enable loop strip mining to offer better pause time guarantees
> +  if (FLAG_IS_DEFAULT(UseCountedLoopSafepoints)) {
> +    FLAG_SET_DEFAULT(UseCountedLoopSafepoints, true);
> +  }
> +  if (UseCountedLoopSafepoints && FLAG_IS_DEFAULT(LoopStripMiningIter)) {
> +    FLAG_SET_DEFAULT(LoopStripMiningIter, 1000);
> +  }
> +#endif
>   }
>   
>   CollectedHeap* G1Arguments::create_heap() {
> diff --git a/src/hotspot/share/opto/c2_globals.hpp b/src/hotspot/share/opto/c2_globals.hpp
> --- a/src/hotspot/share/opto/c2_globals.hpp
> +++ b/src/hotspot/share/opto/c2_globals.hpp
> @@ -741,7 +741,7 @@
>     develop(bool, RenumberLiveNodes, true,                                    \
>             "Renumber live nodes")                                            \
>                                                                               \
> -  product(uintx, LoopStripMiningIter, 1000,                                 \
> +  product(uintx, LoopStripMiningIter, 0,                                    \
>             "Number of iterations in strip mined loop")                       \
>             range(0, max_juint)                                               \
>                                                                               \
> 


From Derek.White at cavium.com  Tue Nov 28 21:11:03 2017
From: Derek.White at cavium.com (White, Derek)
Date: Tue, 28 Nov 2017 21:11:03 +0000
Subject: RFR (XS): 8188877: Improper synchronization in offer_termination
In-Reply-To: <b3948fb6-62b4-f344-348a-63ffaca4504d@redhat.com>
References: <CY1PR0701MB16328948D4FDF19FBC30F65684710@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <3229BC22-BB8A-4D3D-8440-AB799AB5394F@oracle.com>
 <CY1PR0701MB16328E4C6563BE60CF8E99D084750@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <4D9972BA-B76C-4FB7-B24D-31D507DB1A6A@oracle.com>
 <1510821711.5332.3.camel@oracle.com>
 <CY1PR0701MB163288CD9B58A163B2F16F7C84230@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <ba0c0855-188a-decf-932b-b11200dc9fc4@redhat.com>
 <ed1ce174-186d-6548-1eb9-bd61adcb6971@redhat.com>
 <25ec7ee7-c577-6051-a57b-f2b31c3cb3bd@redhat.com>
 <f213bf00-1597-f8a2-6efa-5a8dc9a68013@redhat.com>
 <b3948fb6-62b4-f344-348a-63ffaca4504d@redhat.com>
Message-ID: <CY1PR0701MB16323D91B224BE7127B6383E843A0@CY1PR0701MB1632.namprd07.prod.outlook.com>

Hi Andrew and Andrew,

Thanks for the discussion on load-acquire. This has been informative.

Related to timely notifications, one thing about offer_termination is that it's not doing a classic spin-wait. Classic spin-waits (as seen in mutex.cpp, objectMonitor.cpp, safepoint.cpp, synchronizer.cpp, for examples), will test the termination condition as part of the loop.

Offer-termination just has a simple for-loop that delays some number of cycles. As high as 4k iterations * 140 cycles (per SpinPause() on x86), could be 573,000 cycles or so. For this case, especially where the termination test is a simple load, I think we should test _offered_termination in the spin-wait. This should have low overhead on the spinning thread and impose no impact on other threads.

Unless there's disagreement I'll create an enhancement request for this. I'll add a note about the cleanups that Kim mentioned also.

 - Derek

> -----Original Message-----
> From: Andrew Haley [mailto:aph at redhat.com]
> Sent: Monday, November 27, 2017 10:57 AM
> To: Andrew Dinn <adinn at redhat.com>; White, Derek
> <Derek.White at cavium.com>; Thomas Schatzl
> <thomas.schatzl at oracle.com>; Kim Barrett <kim.barrett at oracle.com>
> Cc: hotspot-gc-dev at openjdk.java.net
> Subject: Re: RFR (XS): 8188877: Improper synchronization in
> offer_termination
> 
> On 27/11/17 15:44, Andrew Dinn wrote:
> > On 27/11/17 14:53, Andrew Haley wrote:
> >> On 27/11/17 12:30, Andrew Dinn wrote:
> > <snip>
> >>> That's what happens when the reader executes a read barrier. The
> >>> interesting question is what happens when the reader does not
> >>> execute a read barrier.
> >>
> >> The invalidate messages still arrive at the reader, but they sit in
> >> the invalidate queue and aren't acted upon immediately.  Eventually
> >> they must be processed, either lazily or because the reader's
> >> invalidate queue fills up.
> >
> > Hmm, that explanation assumes there will be other invalidate messages.
> 
> No, not at all.  By "lazily" I mean that while a core has nothing else to do it
> might as well process its invalidate queue, and AFAIK that is what happens.
> 
> > But at a STW pause that's not necessarily going to be the case. In the
> > worst case all other threads may could be spinning on a barrier count
> > while this one core/thread has a single invalidate message in its queue.
> 
> That could be, but there are other things go on.  There are other threads
> active, and invalidate messages get sent to everyone.
> 
> In practice, I've never seen more than a few microseconds of delay.
> 
> Bear in mind that the interpreter changes we just made mean that
> interpreted code won't necessarily see safepoint status changes for about
> 100 microseconds, so the lack of an acquiring load in our code is really not
> the biggest issue.
> 
> --
> Andrew Haley
> Java Platform Lead Engineer
> Red Hat UK Ltd. <https://www.redhat.com>
> EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671

From rkennke at redhat.com  Tue Nov 28 21:26:29 2017
From: rkennke at redhat.com (Roman Kennke)
Date: Tue, 28 Nov 2017 22:26:29 +0100
Subject: RFR: 8191564: Refactor GC related servicability code into GC
 specific subclasses
In-Reply-To: <5A1D8BE7.4070008@oracle.com>
References: <387d2b93-cb8b-51be-a6e0-3b5f9d6cbb1c@redhat.com>
 <713325a4-3ed8-cd7d-c66d-a1c1767b6d3a@redhat.com>
 <2a7e6307-8c0d-ad4b-e50c-a7e1c2311080@redhat.com>
 <56567db1-caf6-ed05-ed12-336edfca0c75@redhat.com> <5A1C3C07.30800@oracle.com>
 <b32b0580-8700-ec0f-bfb1-18a232e4d364@redhat.com>
 <5A1D8BE7.4070008@oracle.com>
Message-ID: <a541a13e-ff82-b73a-b841-daa2ab910e88@redhat.com>

Hi Erik,

thank you for reviewing!

You are right. I think this is a leftover from when we tried to pass the 
GCMemoryManager* into the Generation constructor. The way it is done now 
(installing the GCMmemoryManager* later through set_memory_manager()) we 
can group all serviceability related set-up into 
initialize_serviceability():

Differential:
http://cr.openjdk.java.net/~rkennke/8191564/webrev.06.diff/
Full:
http://cr.openjdk.java.net/~rkennke/8191564/webrev.06/

Notice that I hooked this up into CollectedHeap::post_initialize() and 
in doing so made that method concrete, and changed all subclass 
post_initialize() methods to call the super-class.

Good now?

Thanks, Roman


> Hi Roman,
> 
> This looks better now. Nice job.
> I wonder though, is it possible to extract the creation of managers and 
> pools to a separate function for each collected heap?
> Now I see managers are created in e.g. CMS constructor, and the pools 
> are created in CMSHeap::initialize(). Perhaps initialize could call 
> initialize_serviceability() that sets up those things, so that they are 
> nicely separated. Unless of course there is a good reason why the 
> presence of managers is needed earlier than that in the bootstrapping.
> 
> Otherwise I think this looks good!
> 
> Thanks,
> /Erik
> 
> On 2017-11-28 12:22, Roman Kennke wrote:
>> Hi Erik,
>>
>> Thanks for your review!
>>
>> All of the things that you mentioned should be addressed in the 
>> following changes:
>>
>> Differential:
>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.05.diff/
>> Full:
>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.05/
>>
>> Also, Erik D (H) was so kind to contribute an additional testcase, 
>> which is also included in the above patch.
>>
>> Ok now?
>>
>> Also, I need a review from serviceability-dev too!
>>
>> Thanks,
>> Roman
>>
>>
>>> 1) The code uses the "mgr" short name for "manager" in a bunch of 
>>> names. I would be happy if we could write out the whole thing instead 
>>> of the abbreviation.
>>> 2) It would be great if a generation would have a pointer to its 
>>> manager, so we do not have to pass around the manager where we 
>>> already pass around the generation (such as 
>>> GenCollectedHeap::collect_generation).
>>> The generation could be created first, then the pools, then the 
>>> managers, then do something like generation->set_memory_manager(x).
>>> 3) In cmsHeap.cpp:54: maxSize should preferably not use camel case.
>>>
>>> Otherwise I think it looks good.
>>>
>>> Thanks,
>>> /Erik
>>>
>>> On 2017-11-27 10:30, Roman Kennke wrote:
>>>> Erik implemented a few more refactorings and touch-ups, and here is 
>>>> our final (pending reviews) webrev:
>>>>
>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.04/
>>>>
>>>> Compared to webrev.02, it improves the handling of gc-end-message, 
>>>> avoids dragging the GCMemoryManager through Generation and a few 
>>>> little related refactorings.
>>>>
>>>> Ok to push now?
>>>>
>>>> Roman
>>>>
>>>>> After a few more discussions with Erik I made some more changes.
>>>>>
>>>>> MemoryService is now unaware of the number and meaning of GC memory 
>>>>> managers (minor vs major). This should be better for GCs that don't 
>>>>> make that distinction and/or use more different GCs (e.g. minor, 
>>>>> intermediate, full).
>>>>>
>>>>> This means that I needed to add some abstractions:
>>>>> - GCMemoryManager now has gc_end_message() which is used by 
>>>>> GCNotifier::pushNotification().
>>>>> - gc_begin() and gc_end() methods in MemoryService now accept a 
>>>>> GCMemoryManager* instead of bull full_gc
>>>>> - Same for TraceMemoryManagerStats
>>>>> - Generation now knows about the corresponding GCMemoryManager
>>>>>
>>>>> Please review the full change:
>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.02/
>>>>>
>>>>> Thanks, Roman
>>>>>
>>>>>
>>>>>> I had some off-band discussions with Erik Helin and re-did most of 
>>>>>> the changeset:
>>>>>> - The GC interface now resides in CollectedHeap, specifically the 
>>>>>> two methods memory_managers() and memory_pools(), and is 
>>>>>> implemented in the various concrete subclasses.
>>>>>> - Both methods return (by value) a GrowableArray<GCMemoryManager*> 
>>>>>> and GrowableArray<MemoryPool> respectively. Returning a 
>>>>>> stack-allocated GrowableArray seemed least complicated (avoid 
>>>>>> explicit cleanup of short-lived array object), and most 
>>>>>> future-proof, e.g. currently there is an implicit expectation to 
>>>>>> get 2 GCMemoryManagers, even though some GCs don't necessarily 
>>>>>> have two. The API allows for easy extension of the situation.
>>>>>>
>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.01/
>>>>>>
>>>>>> I think this requires reviews from both the GC and Serviceability 
>>>>>> team.
>>>>>>
>>>>>> Roman
>>>>>>
>>>>>>
>>>>>>> Currently, there's lots of GC specific code sprinkled over 
>>>>>>> src/hotspot/share/services. This change introduces a GC interface 
>>>>>>> for that, and moves all GC specific code to their respective 
>>>>>>> src/hotspot/share/gc directory.
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.00/ 
>>>>>>> <http://cr.openjdk.java.net/%7Erkennke/8191564/webrev.00/>
>>>>>>>
>>>>>>> Testing: hotspot_gc and hotspot_serviceability, none showed 
>>>>>>> regressions
>>>>>>>
>>>>>>> Built minimal and server without regressions
>>>>>>>
>>>>>>> What do you think?
>>>>>>>
>>>>>>> Roman
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
> 



From david.holmes at oracle.com  Tue Nov 28 21:28:42 2017
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 29 Nov 2017 07:28:42 +1000
Subject: RFR (S): 8191888: Refactor ClassLoaderData::remove_handle to use
 the Access API
In-Reply-To: <5A1D546A.1040601@oracle.com>
References: <5A1BD5B3.7030808@oracle.com>
 <d11416e1-a8b4-15f0-79a4-0e74c64bf60e@oracle.com>
 <5A1D546A.1040601@oracle.com>
Message-ID: <1b1473cd-2235-8c98-617e-fac4c96c5b04@oracle.com>

Hi Erik,

On 28/11/2017 10:19 PM, Erik ?sterlund wrote:
> Hi David,
> 
> That is a good question.
> There is a description (written by a GC person, but still) for each 
> decorator in the access.hpp file where the decorators are declared. I 
> try to go in to some details there when you should and should not use a 
> decorator. If we find that the text might have been biased from a GC 
> perspective, we can update the comments.

The comments serve as a reference, but I think we may lack a "tutorial". 
:) As I think someone commented earlier re the Access API, how do I know 
that, for example, IN_CONCURRENT_ROOT applies when I have no idea what 
may or may not be scanned during safepoint ??

But that's OT.

Thanks,
David



> The previous situation has been to use naked accesses, and then suddenly 
> a GC person jumps in moaning about the lack of explicit #if 
> INCLUDE_ALL_GCS if (UseG1) SATB barrier goo, or manual insertion of 
> post-write barriers if a CAS was successful. There was no good way of 
> knowing you had to manually insert these barriers, what barriers were 
> offered to shared code, or what their semantics were.
> 
> Now at least you have a set of decorators to choose from, with some 
> automatic verification if you select nonsense decorators, and 
> documentation in one place what they are used for and what they mean. So 
> if you do not know what you need, at least you can read the descriptions 
> and hopefully figure it out, which was impossible before.
> 
> Having said that, I believe things like Coleen's OopHandle and 
> PhantomOopHandle will build a layer on top of Access with possibly a few 
> less details exposed that are even simpler to use for the conservative 
> non-GC person that just wants the thing to work as simple as possible 
> and wants to stay as far away from GC land as possible, which is sane.
> 
> And if you feel uncertain, you can always loop me in any time, and I 
> will probably say "there's a decorator for that".
> 
> I hope this answers your question.
> 
> Thanks,
> /Erik
> 
> On 2017-11-28 07:59, David Holmes wrote:
>> Hi Erik,
>>
>> Is there a non-GC person's guide to what the different forms of 
>> RootAccess<XXX> mean and when to use them? Or do we expect a GC person 
>> to always jump in to show where such things are needed? ;-)
>>
>> Thanks,
>> David
>>
>> On 27/11/2017 7:06 PM, Erik ?sterlund wrote:
>>> Hi,
>>>
>>> The ClassLoaderData::remove_handle() member function currently uses 
>>> explicit G1 SATB barriers to remove an oop from the root set, as 
>>> these handles are not necessarily walked by the GC in a safepoint. 
>>> Therefore G1 needs pre-write barriers.
>>>
>>> This should now be modeled as a 
>>> RootAccess<IN_CONCURRENT_ROOT>::oop_store instead. This maps to 
>>> performing a pre-write SATB barrier with G1, but other GCs are free 
>>> to do other things as necessary.
>>>
>>> Webrev:
>>> http://cr.openjdk.java.net/~eosterlund/8191888/webrev.00/
>>>
>>> Bug:
>>> https://bugs.openjdk.java.net/browse/JDK-8191888
>>>
>>> Thanks,
>>> /Erik
> 


From poonam.bajaj at oracle.com  Tue Nov 28 22:17:14 2017
From: poonam.bajaj at oracle.com (Poonam Parhar)
Date: Tue, 28 Nov 2017 14:17:14 -0800
Subject: RFR: 8191821: Finer granularity for GC verification
In-Reply-To: <2319511b-8979-92b9-34f2-6a4c1af46ebf@oracle.com>
References: <2319511b-8979-92b9-34f2-6a4c1af46ebf@oracle.com>
Message-ID: <d6c053ed-0a63-5800-0b57-a952089b35c9@oracle.com>

Hello Stefan,

The changes look good! Thanks for implementing this enhancement.

One observation:
gcArguments.cpp: if we use VerifyGCType? with the collectors that 
currently don't support it, passing it multiple comma separated strings, 
then parse_verification_type() will print this warning message 
"VerifyGCType is not supported by this collector." for all the strings. 
It would be better to break out from the while loop in 
post_heap_initialize() if the collector does not support this option.

Thanks,
Poonam

On 11/28/2017 8:25 AM, Stefan Johansson wrote:
> Hi,
>
> Please review this change for enhancement:
> https://bugs.openjdk.java.net/browse/JDK-8191821
>
> Webrev:
> http://cr.openjdk.java.net/~sjohanss/8191821/00/
>
> Summary:
> Heap verification is a very good way to track down GC bugs, but it 
> comes with a lot of overhead. Using VerifyBeforeGC, VerifyDuringGC and 
> VerifyAfterGC often slows down the execution more than is needed since 
> we sometimes only want to verify certain types of GCs. This change 
> adds this feature for G1 by adding a new diagnostic flag VerifyGCType. 
> The new flag currently only works with G1 but can easily be added for 
> more GCs if needed. The type of the flag is ccstrlist which means the 
> option can be used multiple times to allow more than one type to be 
> verified. The types available for G1 is, young, mixed, remark, cleanup 
> and full. If the flag is not specified all GCs are verified.
>
> Note that Verify/Before/After/During/GC is still needed to decide what 
> to verify, VerifyGCType only describes when.
>
> Testing:
> * Added new Gtest for G1HeapVerifier functionality.
> * Added new Jtreg test for basic command line functionality.
> * Executed Jtreg tests through mach5 to make sure it passes on all 
> platforms.
>
> Thanks,
> Stefan

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20171128/1fd47040/attachment.htm>

From sangheon.kim at oracle.com  Wed Nov 29 07:01:41 2017
From: sangheon.kim at oracle.com (sangheon.kim)
Date: Tue, 28 Nov 2017 23:01:41 -0800
Subject: RFR: 8191821: Finer granularity for GC verification
In-Reply-To: <2319511b-8979-92b9-34f2-6a4c1af46ebf@oracle.com>
References: <2319511b-8979-92b9-34f2-6a4c1af46ebf@oracle.com>
Message-ID: <ef21b0ee-88d5-b31d-990d-c2cea863b28a@oracle.com>

Hi Stefan,

On 11/28/2017 08:25 AM, Stefan Johansson wrote:
> Hi,
>
> Please review this change for enhancement:
> https://bugs.openjdk.java.net/browse/JDK-8191821
>
> Webrev:
> http://cr.openjdk.java.net/~sjohanss/8191821/00/
>
> Summary:
> Heap verification is a very good way to track down GC bugs, but it 
> comes with a lot of overhead. Using VerifyBeforeGC, VerifyDuringGC and 
> VerifyAfterGC often slows down the execution more than is needed since 
> we sometimes only want to verify certain types of GCs. This change 
> adds this feature for G1 by adding a new diagnostic flag VerifyGCType. 
> The new flag currently only works with G1 but can easily be added for 
> more GCs if needed. The type of the flag is ccstrlist which means the 
> option can be used multiple times to allow more than one type to be 
> verified. The types available for G1 is, young, mixed, remark, cleanup 
> and full. If the flag is not specified all GCs are verified.
>
> Note that Verify/Before/After/During/GC is still needed to decide what 
> to verify, VerifyGCType only describes when.
>
> Testing:
> * Added new Gtest for G1HeapVerifier functionality.
> * Added new Jtreg test for basic command line functionality.
> * Executed Jtreg tests through mach5 to make sure it passes on all 
> platforms.
Thank you for improving this area!

Looks good, I have minor nits.

----------------------------------------------------------------------
src/hotspot/share/gc/g1/g1CollectedHeap.cpp
2987 _verifier->verify_before_gc(collector_state()->yc_type() == Mixed ? 
G1HeapVerifier::G1VerifyMixed : G1HeapVerifier::G1VerifyYoung);
3147 _verifier->verify_after_gc(collector_state()->yc_type() == Mixed ? 
G1HeapVerifier::G1VerifyMixed : G1HeapVerifier::G1VerifyYoung);
- (weak suggestion) Change for better readability? e.g. split into 3 lines

----------------------------------------------------------------------
src/hotspot/share/gc/g1/g1ConcurrentMark.cpp
1018 g1h->verifier()->verify(G1HeapVerifier::G1VerifyRemark, 
VerifyOption_G1UsePrevMarking, "During GC (before)");
1039 g1h->verifier()->verify(G1HeapVerifier::G1VerifyRemark, 
VerifyOption_G1UsePrevMarking, "During GC (overflow)");
1054 g1h->verifier()->verify(G1HeapVerifier::G1VerifyRemark, 
VerifyOption_G1UseNextMarking, "During GC (after)");
1186 g1h->verifier()->verify(G1HeapVerifier::G1VerifyCleanup, 
VerifyOption_G1UsePrevMarking, "During GC (before)");
1258 g1h->verifier()->verify(G1HeapVerifier::G1VerifyCleanup, 
VerifyOption_G1UsePrevMarking, "During GC (after)");
- (weak suggestion) Change for better readability? e.g. split into 3 lines

----------------------------------------------------------------------
src/hotspot/share/gc/g1/g1FullCollector.cpp
249???? //Only do verification if VerifyDuringGC and Verify_Full is set.
- A space between '//' and 'Only'. (Not part of your change)

----------------------------------------------------------------------
src/hotspot/share/gc/g1/g1HeapVerifier.cpp
380?? if (strcmp(type, "young") == 0) {
- strncmp

391???? log_warning(gc, verify)("VerifyGCType: '%s' is unknown. 
Available are: young, mixed, remark, cleanup and full ", type);
- "Available are" -> "Available types are"?

396?? // First enable will clear _types.
- '_types' seems old name. Probably something like 'Clear 
_enabled_verification_types if verifies all types'

----------------------------------------------------------------------
test/hotspot/gtest/gc/g1/test_g1HeapVerifier.cpp
73
- Remove additional lines.

----------------------------------------------------------------------
test/hotspot/jtreg/gc/g1/TestVerifyGCType.java
42???? public static final String? VERIFY_TAG??? = "[gc,verify]";
- There are additional space between String and VERIFY_TAG. line 43, 44, 
45 as well.

48???????? // Test with all verification enabled
49???????? OutputAnalyzer output = testWithVerificationType(new String[0]);
- Can we split into sub-tests functions for better readability?

----------------------------------------------------------------------
src/hotspot/share/runtime/globals.hpp
2276????????????? "Available types are: young, mixed, concurrent, 
full")???????? \
- "concurrent," -> "remark, cleanup and"

* I don't see any length limitation logic for VerifyGCType. Not sure how 
other ccstrlist type command-line options are handled, but it would be 
better to have one.

Thanks,
Sangheon


>
> Thanks,
> Stefan



From erik.osterlund at oracle.com  Wed Nov 29 07:31:03 2017
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Wed, 29 Nov 2017 08:31:03 +0100
Subject: RFR: 8191564: Refactor GC related servicability code into GC
 specific subclasses
In-Reply-To: <a541a13e-ff82-b73a-b841-daa2ab910e88@redhat.com>
References: <387d2b93-cb8b-51be-a6e0-3b5f9d6cbb1c@redhat.com>
 <713325a4-3ed8-cd7d-c66d-a1c1767b6d3a@redhat.com>
 <2a7e6307-8c0d-ad4b-e50c-a7e1c2311080@redhat.com>
 <56567db1-caf6-ed05-ed12-336edfca0c75@redhat.com> <5A1C3C07.30800@oracle.com>
 <b32b0580-8700-ec0f-bfb1-18a232e4d364@redhat.com>
 <5A1D8BE7.4070008@oracle.com>
 <a541a13e-ff82-b73a-b841-daa2ab910e88@redhat.com>
Message-ID: <caac7013-f85b-b4e2-73b9-b81a7a57f704@oracle.com>

Hi Roman,

Looks good now. Thanks for doing this.

Thanks,
/Erik

On 2017-11-28 22:26, Roman Kennke wrote:
> Hi Erik,
>
> thank you for reviewing!
>
> You are right. I think this is a leftover from when we tried to pass 
> the GCMemoryManager* into the Generation constructor. The way it is 
> done now (installing the GCMmemoryManager* later through 
> set_memory_manager()) we can group all serviceability related set-up 
> into initialize_serviceability():
>
> Differential:
> http://cr.openjdk.java.net/~rkennke/8191564/webrev.06.diff/
> Full:
> http://cr.openjdk.java.net/~rkennke/8191564/webrev.06/
>
> Notice that I hooked this up into CollectedHeap::post_initialize() and 
> in doing so made that method concrete, and changed all subclass 
> post_initialize() methods to call the super-class.
>
> Good now?
>
> Thanks, Roman
>
>
>> Hi Roman,
>>
>> This looks better now. Nice job.
>> I wonder though, is it possible to extract the creation of managers 
>> and pools to a separate function for each collected heap?
>> Now I see managers are created in e.g. CMS constructor, and the pools 
>> are created in CMSHeap::initialize(). Perhaps initialize could call 
>> initialize_serviceability() that sets up those things, so that they 
>> are nicely separated. Unless of course there is a good reason why the 
>> presence of managers is needed earlier than that in the bootstrapping.
>>
>> Otherwise I think this looks good!
>>
>> Thanks,
>> /Erik
>>
>> On 2017-11-28 12:22, Roman Kennke wrote:
>>> Hi Erik,
>>>
>>> Thanks for your review!
>>>
>>> All of the things that you mentioned should be addressed in the 
>>> following changes:
>>>
>>> Differential:
>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.05.diff/
>>> Full:
>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.05/
>>>
>>> Also, Erik D (H) was so kind to contribute an additional testcase, 
>>> which is also included in the above patch.
>>>
>>> Ok now?
>>>
>>> Also, I need a review from serviceability-dev too!
>>>
>>> Thanks,
>>> Roman
>>>
>>>
>>>> 1) The code uses the "mgr" short name for "manager" in a bunch of 
>>>> names. I would be happy if we could write out the whole thing 
>>>> instead of the abbreviation.
>>>> 2) It would be great if a generation would have a pointer to its 
>>>> manager, so we do not have to pass around the manager where we 
>>>> already pass around the generation (such as 
>>>> GenCollectedHeap::collect_generation).
>>>> The generation could be created first, then the pools, then the 
>>>> managers, then do something like generation->set_memory_manager(x).
>>>> 3) In cmsHeap.cpp:54: maxSize should preferably not use camel case.
>>>>
>>>> Otherwise I think it looks good.
>>>>
>>>> Thanks,
>>>> /Erik
>>>>
>>>> On 2017-11-27 10:30, Roman Kennke wrote:
>>>>> Erik implemented a few more refactorings and touch-ups, and here 
>>>>> is our final (pending reviews) webrev:
>>>>>
>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.04/
>>>>>
>>>>> Compared to webrev.02, it improves the handling of gc-end-message, 
>>>>> avoids dragging the GCMemoryManager through Generation and a few 
>>>>> little related refactorings.
>>>>>
>>>>> Ok to push now?
>>>>>
>>>>> Roman
>>>>>
>>>>>> After a few more discussions with Erik I made some more changes.
>>>>>>
>>>>>> MemoryService is now unaware of the number and meaning of GC 
>>>>>> memory managers (minor vs major). This should be better for GCs 
>>>>>> that don't make that distinction and/or use more different GCs 
>>>>>> (e.g. minor, intermediate, full).
>>>>>>
>>>>>> This means that I needed to add some abstractions:
>>>>>> - GCMemoryManager now has gc_end_message() which is used by 
>>>>>> GCNotifier::pushNotification().
>>>>>> - gc_begin() and gc_end() methods in MemoryService now accept a 
>>>>>> GCMemoryManager* instead of bull full_gc
>>>>>> - Same for TraceMemoryManagerStats
>>>>>> - Generation now knows about the corresponding GCMemoryManager
>>>>>>
>>>>>> Please review the full change:
>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.02/
>>>>>>
>>>>>> Thanks, Roman
>>>>>>
>>>>>>
>>>>>>> I had some off-band discussions with Erik Helin and re-did most 
>>>>>>> of the changeset:
>>>>>>> - The GC interface now resides in CollectedHeap, specifically 
>>>>>>> the two methods memory_managers() and memory_pools(), and is 
>>>>>>> implemented in the various concrete subclasses.
>>>>>>> - Both methods return (by value) a 
>>>>>>> GrowableArray<GCMemoryManager*> and GrowableArray<MemoryPool> 
>>>>>>> respectively. Returning a stack-allocated GrowableArray seemed 
>>>>>>> least complicated (avoid explicit cleanup of short-lived array 
>>>>>>> object), and most future-proof, e.g. currently there is an 
>>>>>>> implicit expectation to get 2 GCMemoryManagers, even though some 
>>>>>>> GCs don't necessarily have two. The API allows for easy 
>>>>>>> extension of the situation.
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.01/
>>>>>>>
>>>>>>> I think this requires reviews from both the GC and 
>>>>>>> Serviceability team.
>>>>>>>
>>>>>>> Roman
>>>>>>>
>>>>>>>
>>>>>>>> Currently, there's lots of GC specific code sprinkled over 
>>>>>>>> src/hotspot/share/services. This change introduces a GC 
>>>>>>>> interface for that, and moves all GC specific code to their 
>>>>>>>> respective src/hotspot/share/gc directory.
>>>>>>>>
>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.00/ 
>>>>>>>> <http://cr.openjdk.java.net/%7Erkennke/8191564/webrev.00/>
>>>>>>>>
>>>>>>>> Testing: hotspot_gc and hotspot_serviceability, none showed 
>>>>>>>> regressions
>>>>>>>>
>>>>>>>> Built minimal and server without regressions
>>>>>>>>
>>>>>>>> What do you think?
>>>>>>>>
>>>>>>>> Roman
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>



From rkennke at redhat.com  Wed Nov 29 08:39:04 2017
From: rkennke at redhat.com (Roman Kennke)
Date: Wed, 29 Nov 2017 09:39:04 +0100
Subject: RFR: 8191564: Refactor GC related servicability code into GC
 specific subclasses
In-Reply-To: <caac7013-f85b-b4e2-73b9-b81a7a57f704@oracle.com>
References: <387d2b93-cb8b-51be-a6e0-3b5f9d6cbb1c@redhat.com>
 <713325a4-3ed8-cd7d-c66d-a1c1767b6d3a@redhat.com>
 <2a7e6307-8c0d-ad4b-e50c-a7e1c2311080@redhat.com>
 <56567db1-caf6-ed05-ed12-336edfca0c75@redhat.com> <5A1C3C07.30800@oracle.com>
 <b32b0580-8700-ec0f-bfb1-18a232e4d364@redhat.com>
 <5A1D8BE7.4070008@oracle.com>
 <a541a13e-ff82-b73a-b841-daa2ab910e88@redhat.com>
 <caac7013-f85b-b4e2-73b9-b81a7a57f704@oracle.com>
Message-ID: <d89cb145-b197-feb6-89c2-a63a4d7a74eb@redhat.com>

Hi Erik,

thanks for the review!

I think this requires another reviewer from serviceability-dev. Who can 
I ping about this?

Roman


> Hi Roman,
> 
> Looks good now. Thanks for doing this.
> 
> Thanks,
> /Erik
> 
> On 2017-11-28 22:26, Roman Kennke wrote:
>> Hi Erik,
>>
>> thank you for reviewing!
>>
>> You are right. I think this is a leftover from when we tried to pass 
>> the GCMemoryManager* into the Generation constructor. The way it is 
>> done now (installing the GCMmemoryManager* later through 
>> set_memory_manager()) we can group all serviceability related set-up 
>> into initialize_serviceability():
>>
>> Differential:
>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.06.diff/
>> Full:
>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.06/
>>
>> Notice that I hooked this up into CollectedHeap::post_initialize() and 
>> in doing so made that method concrete, and changed all subclass 
>> post_initialize() methods to call the super-class.
>>
>> Good now?
>>
>> Thanks, Roman
>>
>>
>>> Hi Roman,
>>>
>>> This looks better now. Nice job.
>>> I wonder though, is it possible to extract the creation of managers 
>>> and pools to a separate function for each collected heap?
>>> Now I see managers are created in e.g. CMS constructor, and the pools 
>>> are created in CMSHeap::initialize(). Perhaps initialize could call 
>>> initialize_serviceability() that sets up those things, so that they 
>>> are nicely separated. Unless of course there is a good reason why the 
>>> presence of managers is needed earlier than that in the bootstrapping.
>>>
>>> Otherwise I think this looks good!
>>>
>>> Thanks,
>>> /Erik
>>>
>>> On 2017-11-28 12:22, Roman Kennke wrote:
>>>> Hi Erik,
>>>>
>>>> Thanks for your review!
>>>>
>>>> All of the things that you mentioned should be addressed in the 
>>>> following changes:
>>>>
>>>> Differential:
>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.05.diff/
>>>> Full:
>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.05/
>>>>
>>>> Also, Erik D (H) was so kind to contribute an additional testcase, 
>>>> which is also included in the above patch.
>>>>
>>>> Ok now?
>>>>
>>>> Also, I need a review from serviceability-dev too!
>>>>
>>>> Thanks,
>>>> Roman
>>>>
>>>>
>>>>> 1) The code uses the "mgr" short name for "manager" in a bunch of 
>>>>> names. I would be happy if we could write out the whole thing 
>>>>> instead of the abbreviation.
>>>>> 2) It would be great if a generation would have a pointer to its 
>>>>> manager, so we do not have to pass around the manager where we 
>>>>> already pass around the generation (such as 
>>>>> GenCollectedHeap::collect_generation).
>>>>> The generation could be created first, then the pools, then the 
>>>>> managers, then do something like generation->set_memory_manager(x).
>>>>> 3) In cmsHeap.cpp:54: maxSize should preferably not use camel case.
>>>>>
>>>>> Otherwise I think it looks good.
>>>>>
>>>>> Thanks,
>>>>> /Erik
>>>>>
>>>>> On 2017-11-27 10:30, Roman Kennke wrote:
>>>>>> Erik implemented a few more refactorings and touch-ups, and here 
>>>>>> is our final (pending reviews) webrev:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.04/
>>>>>>
>>>>>> Compared to webrev.02, it improves the handling of gc-end-message, 
>>>>>> avoids dragging the GCMemoryManager through Generation and a few 
>>>>>> little related refactorings.
>>>>>>
>>>>>> Ok to push now?
>>>>>>
>>>>>> Roman
>>>>>>
>>>>>>> After a few more discussions with Erik I made some more changes.
>>>>>>>
>>>>>>> MemoryService is now unaware of the number and meaning of GC 
>>>>>>> memory managers (minor vs major). This should be better for GCs 
>>>>>>> that don't make that distinction and/or use more different GCs 
>>>>>>> (e.g. minor, intermediate, full).
>>>>>>>
>>>>>>> This means that I needed to add some abstractions:
>>>>>>> - GCMemoryManager now has gc_end_message() which is used by 
>>>>>>> GCNotifier::pushNotification().
>>>>>>> - gc_begin() and gc_end() methods in MemoryService now accept a 
>>>>>>> GCMemoryManager* instead of bull full_gc
>>>>>>> - Same for TraceMemoryManagerStats
>>>>>>> - Generation now knows about the corresponding GCMemoryManager
>>>>>>>
>>>>>>> Please review the full change:
>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.02/
>>>>>>>
>>>>>>> Thanks, Roman
>>>>>>>
>>>>>>>
>>>>>>>> I had some off-band discussions with Erik Helin and re-did most 
>>>>>>>> of the changeset:
>>>>>>>> - The GC interface now resides in CollectedHeap, specifically 
>>>>>>>> the two methods memory_managers() and memory_pools(), and is 
>>>>>>>> implemented in the various concrete subclasses.
>>>>>>>> - Both methods return (by value) a 
>>>>>>>> GrowableArray<GCMemoryManager*> and GrowableArray<MemoryPool> 
>>>>>>>> respectively. Returning a stack-allocated GrowableArray seemed 
>>>>>>>> least complicated (avoid explicit cleanup of short-lived array 
>>>>>>>> object), and most future-proof, e.g. currently there is an 
>>>>>>>> implicit expectation to get 2 GCMemoryManagers, even though some 
>>>>>>>> GCs don't necessarily have two. The API allows for easy 
>>>>>>>> extension of the situation.
>>>>>>>>
>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.01/
>>>>>>>>
>>>>>>>> I think this requires reviews from both the GC and 
>>>>>>>> Serviceability team.
>>>>>>>>
>>>>>>>> Roman
>>>>>>>>
>>>>>>>>
>>>>>>>>> Currently, there's lots of GC specific code sprinkled over 
>>>>>>>>> src/hotspot/share/services. This change introduces a GC 
>>>>>>>>> interface for that, and moves all GC specific code to their 
>>>>>>>>> respective src/hotspot/share/gc directory.
>>>>>>>>>
>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.00/ 
>>>>>>>>> <http://cr.openjdk.java.net/%7Erkennke/8191564/webrev.00/>
>>>>>>>>>
>>>>>>>>> Testing: hotspot_gc and hotspot_serviceability, none showed 
>>>>>>>>> regressions
>>>>>>>>>
>>>>>>>>> Built minimal and server without regressions
>>>>>>>>>
>>>>>>>>> What do you think?
>>>>>>>>>
>>>>>>>>> Roman
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
> 



From erik.osterlund at oracle.com  Wed Nov 29 08:41:08 2017
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Wed, 29 Nov 2017 09:41:08 +0100
Subject: RFR (S): 8191888: Refactor ClassLoaderData::remove_handle to use
 the Access API
In-Reply-To: <1b1473cd-2235-8c98-617e-fac4c96c5b04@oracle.com>
References: <5A1BD5B3.7030808@oracle.com>
 <d11416e1-a8b4-15f0-79a4-0e74c64bf60e@oracle.com>
 <5A1D546A.1040601@oracle.com>
 <1b1473cd-2235-8c98-617e-fac4c96c5b04@oracle.com>
Message-ID: <09eb9609-b87c-a10d-fdba-192de4e535e3@oracle.com>

Hi David,

On 2017-11-28 22:28, David Holmes wrote:
> Hi Erik,
>
> On 28/11/2017 10:19 PM, Erik ?sterlund wrote:
>> Hi David,
>>
>> That is a good question.
>> There is a description (written by a GC person, but still) for each 
>> decorator in the access.hpp file where the decorators are declared. I 
>> try to go in to some details there when you should and should not use 
>> a decorator. If we find that the text might have been biased from a 
>> GC perspective, we can update the comments.
>
> The comments serve as a reference, but I think we may lack a 
> "tutorial". :) As I think someone commented earlier re the Access API, 
> how do I know that, for example, IN_CONCURRENT_ROOT applies when I 
> have no idea what may or may not be scanned during safepoint ??

Would it be helpful if I wrote a little tutorial in a comment section in 
access.hpp? Hopefully, as the code gets more populated with calls to 
this thing, it starts to look more familiar as well.

As for whether to use IN_CONCURRENT_ROOT or not, the basic idea is that 
if you do not know whether your root may be processed outside of 
safepoints, then use IN_CONCURRENT_ROOT conservatively. False positives 
are fine and only come with a small performance loss for some collectors 
(e.g. cause unnecessary G1 pre-write SATB barrier). False negatives on 
the other hand can be fatal (e.g. not performing necessary G1 pre-write 
SATB barrier). It is a little bit like using MO_SEQ_CST vs MO_ACQUIRE. 
Using MO_SEQ_CST when you are not sure is strictly safer but comes with 
a performance penalty. If you for example build thread-local values in 
value types, you might not want to take that performance penalty when 
you know your values are conceptually part of your stack and are 
processed in safepoints.

As we move forward, we might change whether assuming roots may be 
processed concurrently, outside of safepoints, should be default or not. 
AFAIK, it is currently the rare exception rather than the rule. But 
perhaps things will change over time.

Thanks,
/Erik

> But that's OT.
>
> Thanks,
> David
>
>
>
>> The previous situation has been to use naked accesses, and then 
>> suddenly a GC person jumps in moaning about the lack of explicit #if 
>> INCLUDE_ALL_GCS if (UseG1) SATB barrier goo, or manual insertion of 
>> post-write barriers if a CAS was successful. There was no good way of 
>> knowing you had to manually insert these barriers, what barriers were 
>> offered to shared code, or what their semantics were.
>>
>> Now at least you have a set of decorators to choose from, with some 
>> automatic verification if you select nonsense decorators, and 
>> documentation in one place what they are used for and what they mean. 
>> So if you do not know what you need, at least you can read the 
>> descriptions and hopefully figure it out, which was impossible before.
>>
>> Having said that, I believe things like Coleen's OopHandle and 
>> PhantomOopHandle will build a layer on top of Access with possibly a 
>> few less details exposed that are even simpler to use for the 
>> conservative non-GC person that just wants the thing to work as 
>> simple as possible and wants to stay as far away from GC land as 
>> possible, which is sane.
>>
>> And if you feel uncertain, you can always loop me in any time, and I 
>> will probably say "there's a decorator for that".
>>
>> I hope this answers your question.
>>
>> Thanks,
>> /Erik
>>
>> On 2017-11-28 07:59, David Holmes wrote:
>>> Hi Erik,
>>>
>>> Is there a non-GC person's guide to what the different forms of 
>>> RootAccess<XXX> mean and when to use them? Or do we expect a GC 
>>> person to always jump in to show where such things are needed? ;-)
>>>
>>> Thanks,
>>> David
>>>
>>> On 27/11/2017 7:06 PM, Erik ?sterlund wrote:
>>>> Hi,
>>>>
>>>> The ClassLoaderData::remove_handle() member function currently uses 
>>>> explicit G1 SATB barriers to remove an oop from the root set, as 
>>>> these handles are not necessarily walked by the GC in a safepoint. 
>>>> Therefore G1 needs pre-write barriers.
>>>>
>>>> This should now be modeled as a 
>>>> RootAccess<IN_CONCURRENT_ROOT>::oop_store instead. This maps to 
>>>> performing a pre-write SATB barrier with G1, but other GCs are free 
>>>> to do other things as necessary.
>>>>
>>>> Webrev:
>>>> http://cr.openjdk.java.net/~eosterlund/8191888/webrev.00/
>>>>
>>>> Bug:
>>>> https://bugs.openjdk.java.net/browse/JDK-8191888
>>>>
>>>> Thanks,
>>>> /Erik
>>



From erik.helin at oracle.com  Wed Nov 29 09:09:28 2017
From: erik.helin at oracle.com (Erik Helin)
Date: Wed, 29 Nov 2017 10:09:28 +0100
Subject: RFR: 8191564: Refactor GC related servicability code into GC
 specific subclasses
In-Reply-To: <a541a13e-ff82-b73a-b841-daa2ab910e88@redhat.com>
References: <387d2b93-cb8b-51be-a6e0-3b5f9d6cbb1c@redhat.com>
 <713325a4-3ed8-cd7d-c66d-a1c1767b6d3a@redhat.com>
 <2a7e6307-8c0d-ad4b-e50c-a7e1c2311080@redhat.com>
 <56567db1-caf6-ed05-ed12-336edfca0c75@redhat.com> <5A1C3C07.30800@oracle.com>
 <b32b0580-8700-ec0f-bfb1-18a232e4d364@redhat.com>
 <5A1D8BE7.4070008@oracle.com>
 <a541a13e-ff82-b73a-b841-daa2ab910e88@redhat.com>
Message-ID: <97e94494-b86c-21d2-8007-0f4d4e986ab7@oracle.com>

On 11/28/2017 10:26 PM, Roman Kennke wrote:
> Hi Erik,
> 
> thank you for reviewing!
> 
> You are right. I think this is a leftover from when we tried to pass the 
> GCMemoryManager* into the Generation constructor. The way it is done now 
> (installing the GCMmemoryManager* later through set_memory_manager()) we 
> can group all serviceability related set-up into 
> initialize_serviceability():
> 
> Differential:
> http://cr.openjdk.java.net/~rkennke/8191564/webrev.06.diff/
> Full:
> http://cr.openjdk.java.net/~rkennke/8191564/webrev.06/

Looks good to me, Reviewed. I also put version 04 through some testing 
and it successfully passed hs-tier{1,2,3,4,5}.

Thanks,
Erik

> Notice that I hooked this up into CollectedHeap::post_initialize() and 
> in doing so made that method concrete, and changed all subclass 
> post_initialize() methods to call the super-class.
> 
> Good now?
> 
> Thanks, Roman
> 
> 
>> Hi Roman,
>>
>> This looks better now. Nice job.
>> I wonder though, is it possible to extract the creation of managers 
>> and pools to a separate function for each collected heap?
>> Now I see managers are created in e.g. CMS constructor, and the pools 
>> are created in CMSHeap::initialize(). Perhaps initialize could call 
>> initialize_serviceability() that sets up those things, so that they 
>> are nicely separated. Unless of course there is a good reason why the 
>> presence of managers is needed earlier than that in the bootstrapping.
>>
>> Otherwise I think this looks good!
>>
>> Thanks,
>> /Erik
>>
>> On 2017-11-28 12:22, Roman Kennke wrote:
>>> Hi Erik,
>>>
>>> Thanks for your review!
>>>
>>> All of the things that you mentioned should be addressed in the 
>>> following changes:
>>>
>>> Differential:
>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.05.diff/
>>> Full:
>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.05/
>>>
>>> Also, Erik D (H) was so kind to contribute an additional testcase, 
>>> which is also included in the above patch.
>>>
>>> Ok now?
>>>
>>> Also, I need a review from serviceability-dev too!
>>>
>>> Thanks,
>>> Roman
>>>
>>>
>>>> 1) The code uses the "mgr" short name for "manager" in a bunch of 
>>>> names. I would be happy if we could write out the whole thing 
>>>> instead of the abbreviation.
>>>> 2) It would be great if a generation would have a pointer to its 
>>>> manager, so we do not have to pass around the manager where we 
>>>> already pass around the generation (such as 
>>>> GenCollectedHeap::collect_generation).
>>>> The generation could be created first, then the pools, then the 
>>>> managers, then do something like generation->set_memory_manager(x).
>>>> 3) In cmsHeap.cpp:54: maxSize should preferably not use camel case.
>>>>
>>>> Otherwise I think it looks good.
>>>>
>>>> Thanks,
>>>> /Erik
>>>>
>>>> On 2017-11-27 10:30, Roman Kennke wrote:
>>>>> Erik implemented a few more refactorings and touch-ups, and here is 
>>>>> our final (pending reviews) webrev:
>>>>>
>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.04/
>>>>>
>>>>> Compared to webrev.02, it improves the handling of gc-end-message, 
>>>>> avoids dragging the GCMemoryManager through Generation and a few 
>>>>> little related refactorings.
>>>>>
>>>>> Ok to push now?
>>>>>
>>>>> Roman
>>>>>
>>>>>> After a few more discussions with Erik I made some more changes.
>>>>>>
>>>>>> MemoryService is now unaware of the number and meaning of GC 
>>>>>> memory managers (minor vs major). This should be better for GCs 
>>>>>> that don't make that distinction and/or use more different GCs 
>>>>>> (e.g. minor, intermediate, full).
>>>>>>
>>>>>> This means that I needed to add some abstractions:
>>>>>> - GCMemoryManager now has gc_end_message() which is used by 
>>>>>> GCNotifier::pushNotification().
>>>>>> - gc_begin() and gc_end() methods in MemoryService now accept a 
>>>>>> GCMemoryManager* instead of bull full_gc
>>>>>> - Same for TraceMemoryManagerStats
>>>>>> - Generation now knows about the corresponding GCMemoryManager
>>>>>>
>>>>>> Please review the full change:
>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.02/
>>>>>>
>>>>>> Thanks, Roman
>>>>>>
>>>>>>
>>>>>>> I had some off-band discussions with Erik Helin and re-did most 
>>>>>>> of the changeset:
>>>>>>> - The GC interface now resides in CollectedHeap, specifically the 
>>>>>>> two methods memory_managers() and memory_pools(), and is 
>>>>>>> implemented in the various concrete subclasses.
>>>>>>> - Both methods return (by value) a 
>>>>>>> GrowableArray<GCMemoryManager*> and GrowableArray<MemoryPool> 
>>>>>>> respectively. Returning a stack-allocated GrowableArray seemed 
>>>>>>> least complicated (avoid explicit cleanup of short-lived array 
>>>>>>> object), and most future-proof, e.g. currently there is an 
>>>>>>> implicit expectation to get 2 GCMemoryManagers, even though some 
>>>>>>> GCs don't necessarily have two. The API allows for easy extension 
>>>>>>> of the situation.
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.01/
>>>>>>>
>>>>>>> I think this requires reviews from both the GC and Serviceability 
>>>>>>> team.
>>>>>>>
>>>>>>> Roman
>>>>>>>
>>>>>>>
>>>>>>>> Currently, there's lots of GC specific code sprinkled over 
>>>>>>>> src/hotspot/share/services. This change introduces a GC 
>>>>>>>> interface for that, and moves all GC specific code to their 
>>>>>>>> respective src/hotspot/share/gc directory.
>>>>>>>>
>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.00/ 
>>>>>>>> <http://cr.openjdk.java.net/%7Erkennke/8191564/webrev.00/>
>>>>>>>>
>>>>>>>> Testing: hotspot_gc and hotspot_serviceability, none showed 
>>>>>>>> regressions
>>>>>>>>
>>>>>>>> Built minimal and server without regressions
>>>>>>>>
>>>>>>>> What do you think?
>>>>>>>>
>>>>>>>> Roman
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
> 


From erik.helin at oracle.com  Wed Nov 29 09:10:40 2017
From: erik.helin at oracle.com (Erik Helin)
Date: Wed, 29 Nov 2017 10:10:40 +0100
Subject: RFR: 8191564: Refactor GC related servicability code into GC
 specific subclasses
In-Reply-To: <d89cb145-b197-feb6-89c2-a63a4d7a74eb@redhat.com>
References: <387d2b93-cb8b-51be-a6e0-3b5f9d6cbb1c@redhat.com>
 <713325a4-3ed8-cd7d-c66d-a1c1767b6d3a@redhat.com>
 <2a7e6307-8c0d-ad4b-e50c-a7e1c2311080@redhat.com>
 <56567db1-caf6-ed05-ed12-336edfca0c75@redhat.com> <5A1C3C07.30800@oracle.com>
 <b32b0580-8700-ec0f-bfb1-18a232e4d364@redhat.com>
 <5A1D8BE7.4070008@oracle.com>
 <a541a13e-ff82-b73a-b841-daa2ab910e88@redhat.com>
 <caac7013-f85b-b4e2-73b9-b81a7a57f704@oracle.com>
 <d89cb145-b197-feb6-89c2-a63a4d7a74eb@redhat.com>
Message-ID: <4b899587-f7b4-82d7-4917-41d202e26ef4@oracle.com>

On 11/29/2017 09:39 AM, Roman Kennke wrote:
> Hi Erik,
> 
> thanks for the review!
> 
> I think this requires another reviewer from serviceability-dev. Who can 
> I ping about this?

You could try the hotspot-runtime-dev email list, although I suspect 
most of the runtime developers are on serviceability-dev list as well...

Thanks,
Erik

> Roman
> 
> 
>> Hi Roman,
>>
>> Looks good now. Thanks for doing this.
>>
>> Thanks,
>> /Erik
>>
>> On 2017-11-28 22:26, Roman Kennke wrote:
>>> Hi Erik,
>>>
>>> thank you for reviewing!
>>>
>>> You are right. I think this is a leftover from when we tried to pass 
>>> the GCMemoryManager* into the Generation constructor. The way it is 
>>> done now (installing the GCMmemoryManager* later through 
>>> set_memory_manager()) we can group all serviceability related set-up 
>>> into initialize_serviceability():
>>>
>>> Differential:
>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.06.diff/
>>> Full:
>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.06/
>>>
>>> Notice that I hooked this up into CollectedHeap::post_initialize() 
>>> and in doing so made that method concrete, and changed all subclass 
>>> post_initialize() methods to call the super-class.
>>>
>>> Good now?
>>>
>>> Thanks, Roman
>>>
>>>
>>>> Hi Roman,
>>>>
>>>> This looks better now. Nice job.
>>>> I wonder though, is it possible to extract the creation of managers 
>>>> and pools to a separate function for each collected heap?
>>>> Now I see managers are created in e.g. CMS constructor, and the 
>>>> pools are created in CMSHeap::initialize(). Perhaps initialize could 
>>>> call initialize_serviceability() that sets up those things, so that 
>>>> they are nicely separated. Unless of course there is a good reason 
>>>> why the presence of managers is needed earlier than that in the 
>>>> bootstrapping.
>>>>
>>>> Otherwise I think this looks good!
>>>>
>>>> Thanks,
>>>> /Erik
>>>>
>>>> On 2017-11-28 12:22, Roman Kennke wrote:
>>>>> Hi Erik,
>>>>>
>>>>> Thanks for your review!
>>>>>
>>>>> All of the things that you mentioned should be addressed in the 
>>>>> following changes:
>>>>>
>>>>> Differential:
>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.05.diff/
>>>>> Full:
>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.05/
>>>>>
>>>>> Also, Erik D (H) was so kind to contribute an additional testcase, 
>>>>> which is also included in the above patch.
>>>>>
>>>>> Ok now?
>>>>>
>>>>> Also, I need a review from serviceability-dev too!
>>>>>
>>>>> Thanks,
>>>>> Roman
>>>>>
>>>>>
>>>>>> 1) The code uses the "mgr" short name for "manager" in a bunch of 
>>>>>> names. I would be happy if we could write out the whole thing 
>>>>>> instead of the abbreviation.
>>>>>> 2) It would be great if a generation would have a pointer to its 
>>>>>> manager, so we do not have to pass around the manager where we 
>>>>>> already pass around the generation (such as 
>>>>>> GenCollectedHeap::collect_generation).
>>>>>> The generation could be created first, then the pools, then the 
>>>>>> managers, then do something like generation->set_memory_manager(x).
>>>>>> 3) In cmsHeap.cpp:54: maxSize should preferably not use camel case.
>>>>>>
>>>>>> Otherwise I think it looks good.
>>>>>>
>>>>>> Thanks,
>>>>>> /Erik
>>>>>>
>>>>>> On 2017-11-27 10:30, Roman Kennke wrote:
>>>>>>> Erik implemented a few more refactorings and touch-ups, and here 
>>>>>>> is our final (pending reviews) webrev:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.04/
>>>>>>>
>>>>>>> Compared to webrev.02, it improves the handling of 
>>>>>>> gc-end-message, avoids dragging the GCMemoryManager through 
>>>>>>> Generation and a few little related refactorings.
>>>>>>>
>>>>>>> Ok to push now?
>>>>>>>
>>>>>>> Roman
>>>>>>>
>>>>>>>> After a few more discussions with Erik I made some more changes.
>>>>>>>>
>>>>>>>> MemoryService is now unaware of the number and meaning of GC 
>>>>>>>> memory managers (minor vs major). This should be better for GCs 
>>>>>>>> that don't make that distinction and/or use more different GCs 
>>>>>>>> (e.g. minor, intermediate, full).
>>>>>>>>
>>>>>>>> This means that I needed to add some abstractions:
>>>>>>>> - GCMemoryManager now has gc_end_message() which is used by 
>>>>>>>> GCNotifier::pushNotification().
>>>>>>>> - gc_begin() and gc_end() methods in MemoryService now accept a 
>>>>>>>> GCMemoryManager* instead of bull full_gc
>>>>>>>> - Same for TraceMemoryManagerStats
>>>>>>>> - Generation now knows about the corresponding GCMemoryManager
>>>>>>>>
>>>>>>>> Please review the full change:
>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.02/
>>>>>>>>
>>>>>>>> Thanks, Roman
>>>>>>>>
>>>>>>>>
>>>>>>>>> I had some off-band discussions with Erik Helin and re-did most 
>>>>>>>>> of the changeset:
>>>>>>>>> - The GC interface now resides in CollectedHeap, specifically 
>>>>>>>>> the two methods memory_managers() and memory_pools(), and is 
>>>>>>>>> implemented in the various concrete subclasses.
>>>>>>>>> - Both methods return (by value) a 
>>>>>>>>> GrowableArray<GCMemoryManager*> and GrowableArray<MemoryPool> 
>>>>>>>>> respectively. Returning a stack-allocated GrowableArray seemed 
>>>>>>>>> least complicated (avoid explicit cleanup of short-lived array 
>>>>>>>>> object), and most future-proof, e.g. currently there is an 
>>>>>>>>> implicit expectation to get 2 GCMemoryManagers, even though 
>>>>>>>>> some GCs don't necessarily have two. The API allows for easy 
>>>>>>>>> extension of the situation.
>>>>>>>>>
>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.01/
>>>>>>>>>
>>>>>>>>> I think this requires reviews from both the GC and 
>>>>>>>>> Serviceability team.
>>>>>>>>>
>>>>>>>>> Roman
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Currently, there's lots of GC specific code sprinkled over 
>>>>>>>>>> src/hotspot/share/services. This change introduces a GC 
>>>>>>>>>> interface for that, and moves all GC specific code to their 
>>>>>>>>>> respective src/hotspot/share/gc directory.
>>>>>>>>>>
>>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.00/ 
>>>>>>>>>> <http://cr.openjdk.java.net/%7Erkennke/8191564/webrev.00/>
>>>>>>>>>>
>>>>>>>>>> Testing: hotspot_gc and hotspot_serviceability, none showed 
>>>>>>>>>> regressions
>>>>>>>>>>
>>>>>>>>>> Built minimal and server without regressions
>>>>>>>>>>
>>>>>>>>>> What do you think?
>>>>>>>>>>
>>>>>>>>>> Roman
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
> 


From thomas.schatzl at oracle.com  Wed Nov 29 09:25:47 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 29 Nov 2017 10:25:47 +0100
Subject: RFR(M): 8190828: Test plan: JEP 8171181: Support heap
 allocation on alternative memory devices
In-Reply-To: <F89640DCD01A85489FCBA68183A6A0F3BCD54D7C@ORSMSX116.amr.corp.intel.com>
References: <F89640DCD01A85489FCBA68183A6A0F3BCD54D7C@ORSMSX116.amr.corp.intel.com>
Message-ID: <1511947547.2828.0.camel@oracle.com>

Hi,

On Mon, 2017-11-13 at 23:51 +0000, Kharbas, Kishor wrote:
> Hi!
>  
> I have developed a test plan for the implementation of 8171181.
> I would appreciate a review and further guidance from the gc-dev
> members. I am hoping to get everything done well before 18.3 code
> freeze (have a vacation planned during that time).
>  
> Test plan: https://bugs.openjdk.java.net/browse/JDK-8190828

Good.

> Test webrev: http://cr.openjdk.java.net/~kkharbas/8190980/webrev.01/

Looks good.

> JEP: https://bugs.openjdk.java.net/browse/JDK-8171181
> Implementation webrev : http://cr.openjdk.java.net/~kkharbas/8190308/
> webrev.15/

Already looked at in other email thread.

Thanks,
  Thomas


From aph at redhat.com  Wed Nov 29 10:24:09 2017
From: aph at redhat.com (Andrew Haley)
Date: Wed, 29 Nov 2017 10:24:09 +0000
Subject: RFR (XS): 8188877: Improper synchronization in offer_termination
In-Reply-To: <CY1PR0701MB16323D91B224BE7127B6383E843A0@CY1PR0701MB1632.namprd07.prod.outlook.com>
References: <CY1PR0701MB16328948D4FDF19FBC30F65684710@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <3229BC22-BB8A-4D3D-8440-AB799AB5394F@oracle.com>
 <CY1PR0701MB16328E4C6563BE60CF8E99D084750@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <4D9972BA-B76C-4FB7-B24D-31D507DB1A6A@oracle.com>
 <1510821711.5332.3.camel@oracle.com>
 <CY1PR0701MB163288CD9B58A163B2F16F7C84230@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <ba0c0855-188a-decf-932b-b11200dc9fc4@redhat.com>
 <ed1ce174-186d-6548-1eb9-bd61adcb6971@redhat.com>
 <25ec7ee7-c577-6051-a57b-f2b31c3cb3bd@redhat.com>
 <f213bf00-1597-f8a2-6efa-5a8dc9a68013@redhat.com>
 <b3948fb6-62b4-f344-348a-63ffaca4504d@redhat.com>
 <CY1PR0701MB16323D91B224BE7127B6383E843A0@CY1PR0701MB1632.namprd07.prod.outlook.com>
Message-ID: <04704dc6-b801-b209-fcea-04c517798c30@redhat.com>

On 28/11/17 21:11, White, Derek wrote:
> Offer-termination just has a simple for-loop that delays some number of cycles. As high as 4k iterations * 140 cycles (per SpinPause() on x86), could be 573,000 cycles or so. For this case, especially where the termination test is a simple load, I think we should test _offered_termination in the spin-wait. This should have low overhead on the spinning thread and impose no impact on other threads.

Please point to the exact lines of code you're talking about.

-- 
Andrew Haley
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From stefan.johansson at oracle.com  Wed Nov 29 11:10:17 2017
From: stefan.johansson at oracle.com (Stefan Johansson)
Date: Wed, 29 Nov 2017 12:10:17 +0100
Subject: RFR: 8191821: Finer granularity for GC verification
In-Reply-To: <ef21b0ee-88d5-b31d-990d-c2cea863b28a@oracle.com>
References: <2319511b-8979-92b9-34f2-6a4c1af46ebf@oracle.com>
 <ef21b0ee-88d5-b31d-990d-c2cea863b28a@oracle.com>
Message-ID: <e752bdb5-4cbc-60d6-d68d-091bbb0f1d53@oracle.com>

Thanks for reviewing Sangheon,

A lot of good comments, see my answers inline. Here are new webrevs:
Inc: http://cr.openjdk.java.net/~sjohanss/8191821/00-01/
Full: http://cr.openjdk.java.net/~sjohanss/8191821/01/

On 2017-11-29 08:01, sangheon.kim wrote:
> Hi Stefan,
>
> On 11/28/2017 08:25 AM, Stefan Johansson wrote:
>> Hi,
>>
>> Please review this change for enhancement:
>> https://bugs.openjdk.java.net/browse/JDK-8191821
>>
>> Webrev:
>> http://cr.openjdk.java.net/~sjohanss/8191821/00/
>>
>> Summary:
>> Heap verification is a very good way to track down GC bugs, but it 
>> comes with a lot of overhead. Using VerifyBeforeGC, VerifyDuringGC 
>> and VerifyAfterGC often slows down the execution more than is needed 
>> since we sometimes only want to verify certain types of GCs. This 
>> change adds this feature for G1 by adding a new diagnostic flag 
>> VerifyGCType. The new flag currently only works with G1 but can 
>> easily be added for more GCs if needed. The type of the flag is 
>> ccstrlist which means the option can be used multiple times to allow 
>> more than one type to be verified. The types available for G1 is, 
>> young, mixed, remark, cleanup and full. If the flag is not specified 
>> all GCs are verified.
>>
>> Note that Verify/Before/After/During/GC is still needed to decide 
>> what to verify, VerifyGCType only describes when.
>>
>> Testing:
>> * Added new Gtest for G1HeapVerifier functionality.
>> * Added new Jtreg test for basic command line functionality.
>> * Executed Jtreg tests through mach5 to make sure it passes on all 
>> platforms.
> Thank you for improving this area!
>
> Looks good, I have minor nits.
>
> ----------------------------------------------------------------------
> src/hotspot/share/gc/g1/g1CollectedHeap.cpp
> 2987 _verifier->verify_before_gc(collector_state()->yc_type() == Mixed 
> ? G1HeapVerifier::G1VerifyMixed : G1HeapVerifier::G1VerifyYoung);
> 3147 _verifier->verify_after_gc(collector_state()->yc_type() == Mixed 
> ? G1HeapVerifier::G1VerifyMixed : G1HeapVerifier::G1VerifyYoung);
> - (weak suggestion) Change for better readability? e.g. split into 3 
> lines
>
Fixed by adding a function to decide what type it is.
> ----------------------------------------------------------------------
> src/hotspot/share/gc/g1/g1ConcurrentMark.cpp
> 1018 g1h->verifier()->verify(G1HeapVerifier::G1VerifyRemark, 
> VerifyOption_G1UsePrevMarking, "During GC (before)");
> 1039 g1h->verifier()->verify(G1HeapVerifier::G1VerifyRemark, 
> VerifyOption_G1UsePrevMarking, "During GC (overflow)");
> 1054 g1h->verifier()->verify(G1HeapVerifier::G1VerifyRemark, 
> VerifyOption_G1UseNextMarking, "During GC (after)");
> 1186 g1h->verifier()->verify(G1HeapVerifier::G1VerifyCleanup, 
> VerifyOption_G1UsePrevMarking, "During GC (before)");
> 1258 g1h->verifier()->verify(G1HeapVerifier::G1VerifyCleanup, 
> VerifyOption_G1UsePrevMarking, "During GC (after)");
> - (weak suggestion) Change for better readability? e.g. split into 3 
> lines
>
I actually think the current version reads better, leaving as is.
> ----------------------------------------------------------------------
> src/hotspot/share/gc/g1/g1FullCollector.cpp
> 249???? //Only do verification if VerifyDuringGC and Verify_Full is set.
> - A space between '//' and 'Only'. (Not part of your change)
>
Fixed, and changed to G1VerifyFull.
> ----------------------------------------------------------------------
> src/hotspot/share/gc/g1/g1HeapVerifier.cpp
> 380?? if (strcmp(type, "young") == 0) {
> - strncmp
We can't use strncmp here since we want to ensure a perfect match.
>
> 391???? log_warning(gc, verify)("VerifyGCType: '%s' is unknown. 
> Available are: young, mixed, remark, cleanup and full ", type);
> - "Available are" -> "Available types are"?
>
Fixed.
> 396 // First enable will clear _types.
> - '_types' seems old name. Probably something like 'Clear 
> _enabled_verification_types if verifies all types'
>
Correct, fixed.
> ----------------------------------------------------------------------
> test/hotspot/gtest/gc/g1/test_g1HeapVerifier.cpp
> 73
> - Remove additional lines.
>
Fixed.
> ----------------------------------------------------------------------
> test/hotspot/jtreg/gc/g1/TestVerifyGCType.java
> 42???? public static final String? VERIFY_TAG??? = "[gc,verify]";
> - There are additional space between String and VERIFY_TAG. line 43, 
> 44, 45 as well.
>
> 48???????? // Test with all verification enabled
> 49???????? OutputAnalyzer output = testWithVerificationType(new 
> String[0]);
> - Can we split into sub-tests functions for better readability?
Fixed.
>
> ----------------------------------------------------------------------
> src/hotspot/share/runtime/globals.hpp
> 2276????????????? "Available types are: young, mixed, concurrent, 
> full")???????? \
> - "concurrent," -> "remark, cleanup and"
Fixed.
>
> * I don't see any length limitation logic for VerifyGCType. Not sure 
> how other ccstrlist type command-line options are handled, but it 
> would be better to have one.
I don't see any other ccstrlist options do any specific length handling 
and I'm not sure if there is a problem. For each time the option 
VerifyGCType is encountered on the command line a new char array will be 
allocated with NEW_C_HEAP_ARRAY and the old and new value will be added 
to the new array. This can of course fail with a native OOM, but so can 
a lot of other stuff during startup.

Thanks,
Stefan

>
> Thanks,
> Sangheon
>
>
>>
>> Thanks,
>> Stefan
>



From thomas.schatzl at oracle.com  Wed Nov 29 11:16:23 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 29 Nov 2017 12:16:23 +0100
Subject: RFR: 8191821: Finer granularity for GC verification
In-Reply-To: <2319511b-8979-92b9-34f2-6a4c1af46ebf@oracle.com>
References: <2319511b-8979-92b9-34f2-6a4c1af46ebf@oracle.com>
Message-ID: <1511954183.2828.11.camel@oracle.com>

Hi,

On Tue, 2017-11-28 at 17:25 +0100, Stefan Johansson wrote:
> Hi,
> 
> Please review this change for enhancement:
> https://bugs.openjdk.java.net/browse/JDK-8191821
> 
> Webrev:
> http://cr.openjdk.java.net/~sjohanss/8191821/00/
> 
> Summary:
> Heap verification is a very good way to track down GC bugs, but it
> comes 
> with a lot of overhead. Using VerifyBeforeGC, VerifyDuringGC and 
> VerifyAfterGC often slows down the execution more than is needed
> since 
> we sometimes only want to verify certain types of GCs. This change
> adds 
> this feature for G1 by adding a new diagnostic flag VerifyGCType.
> The 
> new flag currently only works with G1 but can easily be added for
> more 
> GCs if needed. The type of the flag is ccstrlist which means the
> option 
> can be used multiple times to allow more than one type to be
> verified. 
> The types available for G1 is, young, mixed, remark, cleanup and
> full. 
> If the flag is not specified all GCs are verified.
> 
> Note that Verify/Before/After/During/GC is still needed to decide
> what 
> to verify, VerifyGCType only describes when.
> 
> Testing:
> * Added new Gtest for G1HeapVerifier functionality.
> * Added new Jtreg test for basic command line functionality.
> * Executed Jtreg tests through mach5 to make sure it passes on all 
> platforms.
> 

  - I would like to see a special verification type for initial mark
gcs, since these are part of the concurrent cycle. I mean, the change
does not allow to verify the whole concurrent cycle alone without
enabling verification for *all* young-only gcs.

  - please change G1VerifyYoung to G1VerifyYoungOnly. Mixed gcs are
also young gcs. We should be exact in our internal naming.

  - in g1HeapVerifier.hpp consider aligning the flag values below each
other.

  - gcArguments.cpp:89: the warning message should print the given
type. Not in "VerifyGCType %s", as that would be confusing, potentially
only indicating that "type" is not supported.

  - gcArguments.cpp:109: are these really the only delimiters for
arguments. Not sure if e.g. \t is also a delimiter. Isn't there some
existing argument processing method available (or at least this
constant) elsewhere?

  - gcArguments.hpp:52: maybe some documentation what this is supposed
to do, and that the accepted types are GC specific.

  - globals.hpp:2276: the list of available types is not complete
(remark, cleanup?). Also there is no indication that the supported
types are collector specific. Or just leave them out here, and document
the available strings in the collector's parsing code header file (in
the enum?)

  - TestVerifyGCType.java: instead of using a GarbageProducer you can
directly trigger specific GCs using whitebox. This would make the tests
a lot more specific imho.

Thanks,
  Thomas


From stefan.johansson at oracle.com  Wed Nov 29 11:39:44 2017
From: stefan.johansson at oracle.com (Stefan Johansson)
Date: Wed, 29 Nov 2017 12:39:44 +0100
Subject: RFR: 8191821: Finer granularity for GC verification
In-Reply-To: <d6c053ed-0a63-5800-0b57-a952089b35c9@oracle.com>
References: <2319511b-8979-92b9-34f2-6a4c1af46ebf@oracle.com>
 <d6c053ed-0a63-5800-0b57-a952089b35c9@oracle.com>
Message-ID: <ea223992-f47b-b363-0826-0340bd8a7d55@oracle.com>

Thanks for the review Poonam,

Update webrevs with your and Sangheons comments:
Inc: http://cr.openjdk.java.net/~sjohanss/8191821/00-01/
Full: http://cr.openjdk.java.net/~sjohanss/8191821/01/

Thanks,
Stefan

On 2017-11-28 23:17, Poonam Parhar wrote:
> Hello Stefan,
>
> The changes look good! Thanks for implementing this enhancement.
>
> One observation:
> gcArguments.cpp: if we use VerifyGCType? with the collectors that 
> currently don't support it, passing it multiple comma separated 
> strings, then parse_verification_type() will print this warning 
> message "VerifyGCType is not supported by this collector." for all the 
> strings. It would be better to break out from the while loop in 
> post_heap_initialize() if the collector does not support this option.
>
> Thanks,
> Poonam
>
> On 11/28/2017 8:25 AM, Stefan Johansson wrote:
>> Hi,
>>
>> Please review this change for enhancement:
>> https://bugs.openjdk.java.net/browse/JDK-8191821
>>
>> Webrev:
>> http://cr.openjdk.java.net/~sjohanss/8191821/00/
>>
>> Summary:
>> Heap verification is a very good way to track down GC bugs, but it 
>> comes with a lot of overhead. Using VerifyBeforeGC, VerifyDuringGC 
>> and VerifyAfterGC often slows down the execution more than is needed 
>> since we sometimes only want to verify certain types of GCs. This 
>> change adds this feature for G1 by adding a new diagnostic flag 
>> VerifyGCType. The new flag currently only works with G1 but can 
>> easily be added for more GCs if needed. The type of the flag is 
>> ccstrlist which means the option can be used multiple times to allow 
>> more than one type to be verified. The types available for G1 is, 
>> young, mixed, remark, cleanup and full. If the flag is not specified 
>> all GCs are verified.
>>
>> Note that Verify/Before/After/During/GC is still needed to decide 
>> what to verify, VerifyGCType only describes when.
>>
>> Testing:
>> * Added new Gtest for G1HeapVerifier functionality.
>> * Added new Jtreg test for basic command line functionality.
>> * Executed Jtreg tests through mach5 to make sure it passes on all 
>> platforms.
>>
>> Thanks,
>> Stefan
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20171129/497a950f/attachment.htm>

From Derek.White at cavium.com  Wed Nov 29 14:01:19 2017
From: Derek.White at cavium.com (White, Derek)
Date: Wed, 29 Nov 2017 14:01:19 +0000
Subject: RFR (XS): 8188877: Improper synchronization in offer_termination
In-Reply-To: <04704dc6-b801-b209-fcea-04c517798c30@redhat.com>
References: <CY1PR0701MB16328948D4FDF19FBC30F65684710@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <3229BC22-BB8A-4D3D-8440-AB799AB5394F@oracle.com>
 <CY1PR0701MB16328E4C6563BE60CF8E99D084750@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <4D9972BA-B76C-4FB7-B24D-31D507DB1A6A@oracle.com>
 <1510821711.5332.3.camel@oracle.com>
 <CY1PR0701MB163288CD9B58A163B2F16F7C84230@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <ba0c0855-188a-decf-932b-b11200dc9fc4@redhat.com>
 <ed1ce174-186d-6548-1eb9-bd61adcb6971@redhat.com>
 <25ec7ee7-c577-6051-a57b-f2b31c3cb3bd@redhat.com>
 <f213bf00-1597-f8a2-6efa-5a8dc9a68013@redhat.com>
 <b3948fb6-62b4-f344-348a-63ffaca4504d@redhat.com>
 <CY1PR0701MB16323D91B224BE7127B6383E843A0@CY1PR0701MB1632.namprd07.prod.outlook.com>
 <04704dc6-b801-b209-fcea-04c517798c30@redhat.com>
Message-ID: <BY1PR0701MB162704E01961FEA5DF478895843B0@BY1PR0701MB1627.namprd07.prod.outlook.com>

Hi Andrew,

> -----Original Message-----
> From: Andrew Haley [mailto:aph at redhat.com]
> Sent: Wednesday, November 29, 2017 5:24 AM
> To: White, Derek <Derek.White at cavium.com>; Andrew Dinn
> <adinn at redhat.com>; Thomas Schatzl <thomas.schatzl at oracle.com>; Kim
> Barrett <kim.barrett at oracle.com>
> Cc: hotspot-gc-dev at openjdk.java.net
> Subject: Re: RFR (XS): 8188877: Improper synchronization in
> offer_termination
> 
> On 28/11/17 21:11, White, Derek wrote:
> > Offer-termination just has a simple for-loop that delays some number of
> cycles. As high as 4k iterations * 140 cycles (per SpinPause() on x86), could be
> 573,000 cycles or so. For this case, especially where the termination test is a
> simple load, I think we should test _offered_termination in the spin-wait.
> This should have low overhead on the spinning thread and impose no impact
> on other threads.
> 
> Please point to the exact lines of code you're talking about.

taskqueue.cpp, in offer_termination(), about line 209:

          for (uint j = 0; j < hard_spin_limit; j++) {
            SpinPause();
          }

hard_spin_limit ranges from 4..4096.

SpinPause() on intel is "pause" (no surprise).

Pause is up to 140 cycles on SkyLake and above (up from 10 cycles).
"Intel? 64 and IA-32 Architectures Optimization Reference Manual", Sect 8.4.7.

 - Derek

From rkennke at redhat.com  Wed Nov 29 15:38:49 2017
From: rkennke at redhat.com (Roman Kennke)
Date: Wed, 29 Nov 2017 16:38:49 +0100
Subject: RFR: 8191564: Refactor GC related servicability code into GC
 specific subclasses
In-Reply-To: <4b899587-f7b4-82d7-4917-41d202e26ef4@oracle.com>
References: <387d2b93-cb8b-51be-a6e0-3b5f9d6cbb1c@redhat.com>
 <713325a4-3ed8-cd7d-c66d-a1c1767b6d3a@redhat.com>
 <2a7e6307-8c0d-ad4b-e50c-a7e1c2311080@redhat.com>
 <56567db1-caf6-ed05-ed12-336edfca0c75@redhat.com> <5A1C3C07.30800@oracle.com>
 <b32b0580-8700-ec0f-bfb1-18a232e4d364@redhat.com>
 <5A1D8BE7.4070008@oracle.com>
 <a541a13e-ff82-b73a-b841-daa2ab910e88@redhat.com>
 <caac7013-f85b-b4e2-73b9-b81a7a57f704@oracle.com>
 <d89cb145-b197-feb6-89c2-a63a4d7a74eb@redhat.com>
 <4b899587-f7b4-82d7-4917-41d202e26ef4@oracle.com>
Message-ID: <6f47a8ab-8b7e-c19c-2e49-10664ca90dd9@redhat.com>

Including hotspot-runtime-dev...

I need one more review (esp. for the src/hotspot/share/services part):

http://cr.openjdk.java.net/~rkennke/8191564/webrev.06/

Thanks, Roman


> On 11/29/2017 09:39 AM, Roman Kennke wrote:
>> Hi Erik,
>>
>> thanks for the review!
>>
>> I think this requires another reviewer from serviceability-dev. Who 
>> can I ping about this?
> 
> You could try the hotspot-runtime-dev email list, although I suspect 
> most of the runtime developers are on serviceability-dev list as well...
> 
> Thanks,
> Erik
> 
>> Roman
>>
>>
>>> Hi Roman,
>>>
>>> Looks good now. Thanks for doing this.
>>>
>>> Thanks,
>>> /Erik
>>>
>>> On 2017-11-28 22:26, Roman Kennke wrote:
>>>> Hi Erik,
>>>>
>>>> thank you for reviewing!
>>>>
>>>> You are right. I think this is a leftover from when we tried to pass 
>>>> the GCMemoryManager* into the Generation constructor. The way it is 
>>>> done now (installing the GCMmemoryManager* later through 
>>>> set_memory_manager()) we can group all serviceability related set-up 
>>>> into initialize_serviceability():
>>>>
>>>> Differential:
>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.06.diff/
>>>> Full:
>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.06/
>>>>
>>>> Notice that I hooked this up into CollectedHeap::post_initialize() 
>>>> and in doing so made that method concrete, and changed all subclass 
>>>> post_initialize() methods to call the super-class.
>>>>
>>>> Good now?
>>>>
>>>> Thanks, Roman
>>>>
>>>>
>>>>> Hi Roman,
>>>>>
>>>>> This looks better now. Nice job.
>>>>> I wonder though, is it possible to extract the creation of managers 
>>>>> and pools to a separate function for each collected heap?
>>>>> Now I see managers are created in e.g. CMS constructor, and the 
>>>>> pools are created in CMSHeap::initialize(). Perhaps initialize 
>>>>> could call initialize_serviceability() that sets up those things, 
>>>>> so that they are nicely separated. Unless of course there is a good 
>>>>> reason why the presence of managers is needed earlier than that in 
>>>>> the bootstrapping.
>>>>>
>>>>> Otherwise I think this looks good!
>>>>>
>>>>> Thanks,
>>>>> /Erik
>>>>>
>>>>> On 2017-11-28 12:22, Roman Kennke wrote:
>>>>>> Hi Erik,
>>>>>>
>>>>>> Thanks for your review!
>>>>>>
>>>>>> All of the things that you mentioned should be addressed in the 
>>>>>> following changes:
>>>>>>
>>>>>> Differential:
>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.05.diff/
>>>>>> Full:
>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.05/
>>>>>>
>>>>>> Also, Erik D (H) was so kind to contribute an additional testcase, 
>>>>>> which is also included in the above patch.
>>>>>>
>>>>>> Ok now?
>>>>>>
>>>>>> Also, I need a review from serviceability-dev too!
>>>>>>
>>>>>> Thanks,
>>>>>> Roman
>>>>>>
>>>>>>
>>>>>>> 1) The code uses the "mgr" short name for "manager" in a bunch of 
>>>>>>> names. I would be happy if we could write out the whole thing 
>>>>>>> instead of the abbreviation.
>>>>>>> 2) It would be great if a generation would have a pointer to its 
>>>>>>> manager, so we do not have to pass around the manager where we 
>>>>>>> already pass around the generation (such as 
>>>>>>> GenCollectedHeap::collect_generation).
>>>>>>> The generation could be created first, then the pools, then the 
>>>>>>> managers, then do something like generation->set_memory_manager(x).
>>>>>>> 3) In cmsHeap.cpp:54: maxSize should preferably not use camel case.
>>>>>>>
>>>>>>> Otherwise I think it looks good.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> /Erik
>>>>>>>
>>>>>>> On 2017-11-27 10:30, Roman Kennke wrote:
>>>>>>>> Erik implemented a few more refactorings and touch-ups, and here 
>>>>>>>> is our final (pending reviews) webrev:
>>>>>>>>
>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.04/
>>>>>>>>
>>>>>>>> Compared to webrev.02, it improves the handling of 
>>>>>>>> gc-end-message, avoids dragging the GCMemoryManager through 
>>>>>>>> Generation and a few little related refactorings.
>>>>>>>>
>>>>>>>> Ok to push now?
>>>>>>>>
>>>>>>>> Roman
>>>>>>>>
>>>>>>>>> After a few more discussions with Erik I made some more changes.
>>>>>>>>>
>>>>>>>>> MemoryService is now unaware of the number and meaning of GC 
>>>>>>>>> memory managers (minor vs major). This should be better for GCs 
>>>>>>>>> that don't make that distinction and/or use more different GCs 
>>>>>>>>> (e.g. minor, intermediate, full).
>>>>>>>>>
>>>>>>>>> This means that I needed to add some abstractions:
>>>>>>>>> - GCMemoryManager now has gc_end_message() which is used by 
>>>>>>>>> GCNotifier::pushNotification().
>>>>>>>>> - gc_begin() and gc_end() methods in MemoryService now accept a 
>>>>>>>>> GCMemoryManager* instead of bull full_gc
>>>>>>>>> - Same for TraceMemoryManagerStats
>>>>>>>>> - Generation now knows about the corresponding GCMemoryManager
>>>>>>>>>
>>>>>>>>> Please review the full change:
>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.02/
>>>>>>>>>
>>>>>>>>> Thanks, Roman
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> I had some off-band discussions with Erik Helin and re-did 
>>>>>>>>>> most of the changeset:
>>>>>>>>>> - The GC interface now resides in CollectedHeap, specifically 
>>>>>>>>>> the two methods memory_managers() and memory_pools(), and is 
>>>>>>>>>> implemented in the various concrete subclasses.
>>>>>>>>>> - Both methods return (by value) a 
>>>>>>>>>> GrowableArray<GCMemoryManager*> and GrowableArray<MemoryPool> 
>>>>>>>>>> respectively. Returning a stack-allocated GrowableArray seemed 
>>>>>>>>>> least complicated (avoid explicit cleanup of short-lived array 
>>>>>>>>>> object), and most future-proof, e.g. currently there is an 
>>>>>>>>>> implicit expectation to get 2 GCMemoryManagers, even though 
>>>>>>>>>> some GCs don't necessarily have two. The API allows for easy 
>>>>>>>>>> extension of the situation.
>>>>>>>>>>
>>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.01/
>>>>>>>>>>
>>>>>>>>>> I think this requires reviews from both the GC and 
>>>>>>>>>> Serviceability team.
>>>>>>>>>>
>>>>>>>>>> Roman
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Currently, there's lots of GC specific code sprinkled over 
>>>>>>>>>>> src/hotspot/share/services. This change introduces a GC 
>>>>>>>>>>> interface for that, and moves all GC specific code to their 
>>>>>>>>>>> respective src/hotspot/share/gc directory.
>>>>>>>>>>>
>>>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.00/ 
>>>>>>>>>>> <http://cr.openjdk.java.net/%7Erkennke/8191564/webrev.00/>
>>>>>>>>>>>
>>>>>>>>>>> Testing: hotspot_gc and hotspot_serviceability, none showed 
>>>>>>>>>>> regressions
>>>>>>>>>>>
>>>>>>>>>>> Built minimal and server without regressions
>>>>>>>>>>>
>>>>>>>>>>> What do you think?
>>>>>>>>>>>
>>>>>>>>>>> Roman
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>



From stefan.johansson at oracle.com  Wed Nov 29 15:43:35 2017
From: stefan.johansson at oracle.com (Stefan Johansson)
Date: Wed, 29 Nov 2017 16:43:35 +0100
Subject: RFR: 8191821: Finer granularity for GC verification
In-Reply-To: <1511954183.2828.11.camel@oracle.com>
References: <2319511b-8979-92b9-34f2-6a4c1af46ebf@oracle.com>
 <1511954183.2828.11.camel@oracle.com>
Message-ID: <8883ff1b-de5d-e44d-f554-696016535011@oracle.com>

Thanks for reviewing Thomas,

Updated webrevs:
Inc:? http://cr.openjdk.java.net/~sjohanss/8191821/01-02/
Full: http://cr.openjdk.java.net/~sjohanss/8191821/02/

Comments inline.

On 2017-11-29 12:16, Thomas Schatzl wrote:
> Hi,
>
> On Tue, 2017-11-28 at 17:25 +0100, Stefan Johansson wrote:
>> Hi,
>>
>> Please review this change for enhancement:
>> https://bugs.openjdk.java.net/browse/JDK-8191821
>>
>> Webrev:
>> http://cr.openjdk.java.net/~sjohanss/8191821/00/
>>
>> Summary:
>> Heap verification is a very good way to track down GC bugs, but it
>> comes
>> with a lot of overhead. Using VerifyBeforeGC, VerifyDuringGC and
>> VerifyAfterGC often slows down the execution more than is needed
>> since
>> we sometimes only want to verify certain types of GCs. This change
>> adds
>> this feature for G1 by adding a new diagnostic flag VerifyGCType.
>> The
>> new flag currently only works with G1 but can easily be added for
>> more
>> GCs if needed. The type of the flag is ccstrlist which means the
>> option
>> can be used multiple times to allow more than one type to be
>> verified.
>> The types available for G1 is, young, mixed, remark, cleanup and
>> full.
>> If the flag is not specified all GCs are verified.
>>
>> Note that Verify/Before/After/During/GC is still needed to decide
>> what
>> to verify, VerifyGCType only describes when.
>>
>> Testing:
>> * Added new Gtest for G1HeapVerifier functionality.
>> * Added new Jtreg test for basic command line functionality.
>> * Executed Jtreg tests through mach5 to make sure it passes on all
>> platforms.
>>
>    - I would like to see a special verification type for initial mark
> gcs, since these are part of the concurrent cycle. I mean, the change
> does not allow to verify the whole concurrent cycle alone without
> enabling verification for *all* young-only gcs.
Fixed.
>
>    - please change G1VerifyYoung to G1VerifyYoungOnly. Mixed gcs are
> also young gcs. We should be exact in our internal naming.
Fixed.
>
>    - in g1HeapVerifier.hpp consider aligning the flag values below each
> other.
Fixed.
>
>    - gcArguments.cpp:89: the warning message should print the given
> type. Not in "VerifyGCType %s", as that would be confusing, potentially
> only indicating that "type" is not supported.
Updated per Poonams request to only be printed once and with this I 
think I can be left as is.
>
>    - gcArguments.cpp:109: are these really the only delimiters for
> arguments. Not sure if e.g. \t is also a delimiter. Isn't there some
> existing argument processing method available (or at least this
> constant) elsewhere?
So I basically used the same that were available for VerifySubSet but 
also added \n since this is the delimiter added if the flag is used 
multiple times. I think the expected way to use ccstrlist is to have one 
value per usage like:
-XX:VerifyGCType=full -XX:VerifyGCType=young

But since VerifySubSet allows comma separated values I decided to do so 
as well. I people start using tabs I'm not sure we want to support that. 
I would then rather just support "\n" and have one value per flag.
>
>    - gcArguments.hpp:52: maybe some documentation what this is supposed
> to do, and that the accepted types are GC specific.
Fixed.
>
>    - globals.hpp:2276: the list of available types is not complete
> (remark, cleanup?). Also there is no indication that the supported
> types are collector specific. Or just leave them out here, and document
> the available strings in the collector's parsing code header file (in
> the enum?)
Good point, will remove this listing and update the text.
>
>    - TestVerifyGCType.java: instead of using a GarbageProducer you can
> directly trigger specific GCs using whitebox. This would make the tests
> a lot more specific imho.
Agreed and fixed. Also added test for mixed and initial-mark now.

Thanks,
Stefan
>
> Thanks,
>    Thomas



From gerald.thornbrugh at oracle.com  Wed Nov 22 20:21:43 2017
From: gerald.thornbrugh at oracle.com (Gerald Thornbrugh)
Date: Wed, 22 Nov 2017 13:21:43 -0700
Subject: RFR(XL): 8167108 - SMR and JavaThread Lifecycle
In-Reply-To: <5365606e-4036-b5d7-9bac-e91e42f38d01@oracle.com>
References: <1e50bb73-840c-fc3a-81ad-31f83037093f@oracle.com>
 <fbdb983c-441d-b8d4-99d7-5ea68846b0be@oracle.com>
 <836b5c6c-5eef-9245-c16a-e71372093d9a@oracle.com>
 <82d0e473-a24f-9feb-82e9-7d04c4d7a91d@oracle.com>
 <354f7ece-66bc-c065-9ee8-f55abcc1c23e@oracle.com>
 <0ff8df8e-e93c-02d2-cb97-b13b1ac0f53f@oracle.com>
 <5365606e-4036-b5d7-9bac-e91e42f38d01@oracle.com>
Message-ID: <A20B6F1C-98A2-444F-8FED-E51E54B4CE64@oracle.com>

Hi Dan,

Your changes look good.

Jerry

> On Nov 21, 2017, at 5:12 PM, Daniel D. Daugherty <daniel.daugherty at oracle.com> wrote:
> 
> Greetings,
> 
> *** We are wrapping up code review on this project so it is time ***
> *** for the various code reviewers to chime in one last time. ***
> 
> In this latest round, we had three different reviewers chime in so we're
> doing delta webrevs for each of those resolutions:
> 
> David H's resolutions:
> 
> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.09,10.10.0-delta/
> 
>   mq comment for dholmes_CR:
>     - Fix indents, trailing spaces and various typos.
>     - Add descriptions for the '_cnt', '_max' and '_times" fields,
>       add impl notes to document the type choices.
> 
>   src/hotspot/share/runtime/globals.hpp change is white-space only so it
>   is only visible via the file's patch link.
> 
> 
> Robin W's resolutions:
> 
> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.10.0,10.10.1-delta/
> 
>   mq comment for robinw_CR:
>     - Fix some inefficient code, update some comments, fix some indents,
>       and add some 'const' specifiers.
> 
>   src/hotspot/share/runtime/vm_operations.hpp change is white-space only so
>   it is only visible via the file's patch link.
> 
> 
> Coleen's resolutions:
> 
> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.10.1,10.10.2-delta/
> 
>   mq comment for coleenp_CR:
>     - Change ThreadsList::_threads from 'mtGC' -> 'mtThread',
>     - add header comment to threadSMR.hpp file,
>     - cleanup JavaThreadIteratorWithHandle ctr,
>     - make ErrorHandling more efficient.
> 
> 
> Some folks prefer to see all of the delta webrevs together so here is
> a delta webrev relative to jdk10-09-full:
> 
> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.09,10.10.2-delta/
> 
>   src/hotspot/share/runtime/globals.hpp and
>   src/hotspot/share/runtime/vm_operations.hpp changes are white-space only
>   so they are only visible via the file's patch link.
> 
> 
> And, of course, some folks prefer to see everything all at once so here
> is the latest full webrev:
> 
> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-10.10-full/
> 
> 
> Dan has submitted the bits for the usual Mach5 tier[1-5] testing. The
> prelim Mach5 tier1 (JPRT equivalent) on these bits passed just fine...
> 
> The project is currently baselined on Jesper's 2017-11-17 PIT snapshot
> so there will be some catching up to do before integration...
> 
> We welcome comments, suggestions and feedback.
> 
> Dan, Erik and Robbin
> 
> 
> On 11/18/17 8:49 PM, Daniel D. Daugherty wrote:
>> Greetings,
>> 
>> Testing of the last round of changes revealed a hang in one of the new
>> TLH tests. Robbin has fixed the hang, updated the existing TLH test, and
>> added another TLH test for good measure.
>> 
>> Here is the updated full webrev:
>> 
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-full/
>> 
>> Here is the updated delta webrev:
>> 
>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-09-delta/
>> 
>> Dan ran the bits thru the usual Mach5 tier[1-5] testing and there are
>> no unexpected failures.
>> 
>> We welcome comments, suggestions and feedback.
>> 
>> Dan, Erik and Robbin
>> 
>> 
>> On 11/15/17 3:32 PM, Daniel D. Daugherty wrote:
>>> Greetings,
>>> 
>>> Robbin rebased the project last night/this morning to merge with Thread
>>> Local Handshakes (TLH) and also picked up additional changesets up thru:
>>> 
>>>> Changeset: fa736014cf28
>>>> Author:    cjplummer
>>>> Date:      2017-11-14 18:08 -0800
>>>> URL:http://hg.openjdk.java.net/jdk/hs/rev/fa736014cf28
>>>> 
>>>> 8191049: Add alternate version of pns() that is callable from within hotspot source
>>>> Summary: added pns2() to debug.cpp
>>>> Reviewed-by: stuefe, gthornbr
>>> 
>>> This is the first time we've rebased the project to bits that are this
>>> fresh (< 12 hours old at rebase time). We've done this because we think
>>> we're done with this project and are in the final review-change-retest
>>> cycle(s)... In other words, we're not planning on making any more major
>>> changes for this project... :-)
>>> 
>>> *** Time for code reviewers to chime in on this thread! ***
>>> 
>>> Here is the updated full webrev:
>>> 
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-full/
>>> 
>>> Here's is the delta webrev from the 2017.11.10 rebase:
>>> 
>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-08-delta/
>>> 
>>> Dan has submitted the bits for the usual Mach5 tier[1-5] testing
>>> (and the new baseline also)...
>>> 
>>> We're expecting this round to be a little noisier than usual because
>>> we did not rebase on a PIT snapshot so the new baseline has not been
>>> through Jesper's usual care-and-feeding of integration_blockers, etc.
>>> 
>>> We welcome comments, suggestions and feedback.
>>> 
>>> Dan, Erik and Robbin
>>> 
>>> 
>>> On 11/14/17 4:48 PM, Daniel D. Daugherty wrote:
>>>> Greetings,
>>>> 
>>>> I rebased the project to the 2017.11.10 jdk/hs PIT snapshot.
>>>> (Yes, we're playing chase-the-repo...)
>>>> 
>>>> Here is the updated full webrev:
>>>> 
>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-07-full/
>>>> 
>>>> Unlike the previous rebase, there were no changes required to the
>>>> open code to get the rebased bits to build so there is no delta
>>>> webrev.
>>>> 
>>>> These bits have been run through JPRT and I've submitted the usual
>>>> Mach5 tier[1-5] test run...
>>>> 
>>>> We welcome comments, suggestions and feedback.
>>>> 
>>>> Dan, Erik and Robbin
>>>> 
>>>> 
>>>> On 11/13/17 12:30 PM, Daniel D. Daugherty wrote:
>>>>> Greetings,
>>>>> 
>>>>> I rebased the project to the 2017.10.26 jdk10/hs PIT snapshot.
>>>>> 
>>>>> Here are the updated webrevs:
>>>>> 
>>>>> Here's the mq comment for the change:
>>>>> 
>>>>>   Rebase to 2017.10.25 PIT snapshot.
>>>>> 
>>>>> Here is the full webrev:
>>>>> 
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-full/
>>>>> 
>>>>> And here is the delta webrev:
>>>>> 
>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-06-delta/
>>>>> 
>>>>> I ran the above bits throught Mach5 tier[1-5] testing over the holiday
>>>>> weekend. Didn't see any issues in a quick look. Going to take a closer
>>>>> look today.
>>>>> 
>>>>> We welcome comments, suggestions and feedback.
>>>>> 
>>>>> Dan, Erik and Robbin
>>>>> 
>>>>> 
>>>>> On 11/8/17 1:05 PM, Daniel D. Daugherty wrote:
>>>>>> Greetings,
>>>>>> 
>>>>>> Resolving one of the code review comments (from both Stefan K and Coleen)
>>>>>> on jdk10-04-full required quite a few changes so it is being done as a
>>>>>> standalone patch and corresponding webrevs:
>>>>>> 
>>>>>> Here's the mq comment for the change:
>>>>>> 
>>>>>>   stefank, coleenp CR - refactor most JavaThreadIterator usage to use
>>>>>>     JavaThreadIteratorWithHandle.
>>>>>> 
>>>>>> Here is the full webrev:
>>>>>> 
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-full/
>>>>>> 
>>>>>> And here is the delta webrev:
>>>>>> 
>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-05-delta/
>>>>>> 
>>>>>> We welcome comments, suggestions and feedback.
>>>>>> 
>>>>>> Dan, Erik and Robbin
>>>>>> 
>>>>>> 
>>>>>> On 10/9/17 3:41 PM, Daniel D. Daugherty wrote:
>>>>>>> Greetings,
>>>>>>> 
>>>>>>> We have a (eXtra Large) fix for the following bug:
>>>>>>> 
>>>>>>> 8167108 inconsistent handling of SR_lock can lead to crashes
>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8167108
>>>>>>> 
>>>>>>> This fix adds a Safe Memory Reclamation (SMR) mechanism based on
>>>>>>> Hazard Pointers to manage JavaThread lifecycle.
>>>>>>> 
>>>>>>> Here's a PDF for the internal wiki that we've been using to describe
>>>>>>> and track the work on this project:
>>>>>>> 
>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/SMR_and_JavaThread_Lifecycle-JDK10-04.pdf 
>>>>>>> 
>>>>>>> Dan has noticed that the indenting is wrong in some of the code quotes
>>>>>>> in the PDF that are not present in the internal wiki. We don't have a
>>>>>>> solution for that problem yet.
>>>>>>> 
>>>>>>> Here's the webrev for current JDK10 version of this fix:
>>>>>>> 
>>>>>>> http://cr.openjdk.java.net/~dcubed/8167108-webrev/jdk10-04-full
>>>>>>> 
>>>>>>> This fix has been run through many rounds of JPRT and Mach5 tier[2-5]
>>>>>>> testing, additional stress testing on Dan's Solaris X64 server, and
>>>>>>> additional testing on Erik and Robbin's machines.
>>>>>>> 
>>>>>>> We welcome comments, suggestions and feedback.
>>>>>>> 
>>>>>>> Daniel Daugherty
>>>>>>> Erik Osterlund
>>>>>>> Robbin Ehn
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>>> 
>>> 
>>> 
>> 
>> 
> 



From rwestrel at redhat.com  Tue Nov 28 11:00:34 2017
From: rwestrel at redhat.com (Roland Westrelin)
Date: Tue, 28 Nov 2017 12:00:34 +0100
Subject: RFR(L): 8186027: C2: loop strip mining
In-Reply-To: <45aa8e04-9244-4089-001f-acba24e7e4ee@oracle.com>
References: <dk6k20cxtak.fsf@rwestrel.remote.csb>
 <35a024b5-6632-0488-a001-14f960257fc7@oracle.com>
 <dk67evjnvww.fsf@rwestrel.remote.csb>
 <8b9f8985-daae-b3b4-ba48-3d9c6185ed95@oracle.com>
 <0972a0db-2115-daa3-9990-7d58915a74a5@oracle.com>
 <dk61sktw8hn.fsf@rwestrel.remote.csb>
 <3e3e2048-5ee5-f3c2-8ec0-cdd36cff07af@oracle.com>
 <dk64lpmuzrg.fsf@rwestrel.remote.csb>
 <7ae8403a-b650-5171-5d79-0107d7d99059@oracle.com>
 <dk6shd5t6x7.fsf@rwestrel.remote.csb>
 <f2a0b0c2-6642-785c-0e47-9d62ad8dae9f@oracle.com>
 <1511515343.2425.11.camel@oracle.com>
 <45aa8e04-9244-4089-001f-acba24e7e4ee@oracle.com>
Message-ID: <dk68tequ0q5.fsf@rwestrel.remote.csb>


Hi Vladimir,

> Roland, please, prepare changeset.

Here is the changeset:

http://cr.openjdk.java.net/~roland/8186027/loop-strip-mining.patch

It includes the patch below to enable it with G1.

Roland.

diff --git a/src/hotspot/share/gc/g1/g1Arguments.cpp b/src/hotspot/share/gc/g1/g1Arguments.cpp
--- a/src/hotspot/share/gc/g1/g1Arguments.cpp
+++ b/src/hotspot/share/gc/g1/g1Arguments.cpp
@@ -92,6 +92,16 @@
   }
 
   log_trace(gc)("MarkStackSize: %uk  MarkStackSizeMax: %uk", (unsigned int) (MarkStackSize / K), (uint) (MarkStackSizeMax / K));
+
+#ifdef COMPILER2
+  // Enable loop strip mining to offer better pause time guarantees
+  if (FLAG_IS_DEFAULT(UseCountedLoopSafepoints)) {
+    FLAG_SET_DEFAULT(UseCountedLoopSafepoints, true);
+  }
+  if (UseCountedLoopSafepoints && FLAG_IS_DEFAULT(LoopStripMiningIter)) {
+    FLAG_SET_DEFAULT(LoopStripMiningIter, 1000);
+  }
+#endif
 }
 
 CollectedHeap* G1Arguments::create_heap() {
diff --git a/src/hotspot/share/opto/c2_globals.hpp b/src/hotspot/share/opto/c2_globals.hpp
--- a/src/hotspot/share/opto/c2_globals.hpp
+++ b/src/hotspot/share/opto/c2_globals.hpp
@@ -741,7 +741,7 @@
   develop(bool, RenumberLiveNodes, true,                                    \
           "Renumber live nodes")                                            \
                                                                             \
-  product(uintx, LoopStripMiningIter, 1000,                                 \
+  product(uintx, LoopStripMiningIter, 0,                                    \
           "Number of iterations in strip mined loop")                       \
           range(0, max_juint)                                               \
                                                                             \


From rwestrel at redhat.com  Wed Nov 29 15:33:52 2017
From: rwestrel at redhat.com (Roland Westrelin)
Date: Wed, 29 Nov 2017 16:33:52 +0100
Subject: RFR(L): 8186027: C2: loop strip mining
In-Reply-To: <4a7c6d8c-10ed-6fb0-1238-e1cf9b65baa4@oracle.com>
References: <dk6k20cxtak.fsf@rwestrel.remote.csb>
 <35a024b5-6632-0488-a001-14f960257fc7@oracle.com>
 <dk67evjnvww.fsf@rwestrel.remote.csb>
 <8b9f8985-daae-b3b4-ba48-3d9c6185ed95@oracle.com>
 <0972a0db-2115-daa3-9990-7d58915a74a5@oracle.com>
 <dk61sktw8hn.fsf@rwestrel.remote.csb>
 <3e3e2048-5ee5-f3c2-8ec0-cdd36cff07af@oracle.com>
 <dk64lpmuzrg.fsf@rwestrel.remote.csb>
 <7ae8403a-b650-5171-5d79-0107d7d99059@oracle.com>
 <dk6shd5t6x7.fsf@rwestrel.remote.csb>
 <f2a0b0c2-6642-785c-0e47-9d62ad8dae9f@oracle.com>
 <1511515343.2425.11.camel@oracle.com>
 <45aa8e04-9244-4089-001f-acba24e7e4ee@oracle.com>
 <dk68tequ0q5.fsf@rwestrel.remote.csb>
 <4a7c6d8c-10ed-6fb0-1238-e1cf9b65baa4@oracle.com>
Message-ID: <dk6lgiprten.fsf@rwestrel.remote.csb>


> UseCountedLoopSafepoints should be false by default too. Otherwise code in arguments.cpp will set LoopStripMiningIter to 
> 1. I will remove next change in c2_globals.hpp:
>
> -  product(bool, UseCountedLoopSafepoints, false,                            \
> +  product(bool, UseCountedLoopSafepoints, true,                             \

Right. Thanks for pushing this.

Roland.


From sangheon.kim at oracle.com  Wed Nov 29 21:36:10 2017
From: sangheon.kim at oracle.com (sangheon.kim)
Date: Wed, 29 Nov 2017 13:36:10 -0800
Subject: RFR: 8191821: Finer granularity for GC verification
In-Reply-To: <e752bdb5-4cbc-60d6-d68d-091bbb0f1d53@oracle.com>
References: <2319511b-8979-92b9-34f2-6a4c1af46ebf@oracle.com>
 <ef21b0ee-88d5-b31d-990d-c2cea863b28a@oracle.com>
 <e752bdb5-4cbc-60d6-d68d-091bbb0f1d53@oracle.com>
Message-ID: <77dce0ae-da01-80b0-651b-93117020f929@oracle.com>

Hi Stefan,

On 11/29/2017 03:10 AM, Stefan Johansson wrote:
> Thanks for reviewing Sangheon,
>
> A lot of good comments, see my answers inline. Here are new webrevs:
> Inc: http://cr.openjdk.java.net/~sjohanss/8191821/00-01/
> Full: http://cr.openjdk.java.net/~sjohanss/8191821/01/
webrev.01 looks good to me but as you published webrev.02, I will check 
that as well.

My answers are in-lined.

>
> On 2017-11-29 08:01, sangheon.kim wrote:
>> Hi Stefan,
>>
>> On 11/28/2017 08:25 AM, Stefan Johansson wrote:
>>> Hi,
>>>
>>> Please review this change for enhancement:
>>> https://bugs.openjdk.java.net/browse/JDK-8191821
>>>
>>> Webrev:
>>> http://cr.openjdk.java.net/~sjohanss/8191821/00/
>>>
>>> Summary:
>>> Heap verification is a very good way to track down GC bugs, but it 
>>> comes with a lot of overhead. Using VerifyBeforeGC, VerifyDuringGC 
>>> and VerifyAfterGC often slows down the execution more than is needed 
>>> since we sometimes only want to verify certain types of GCs. This 
>>> change adds this feature for G1 by adding a new diagnostic flag 
>>> VerifyGCType. The new flag currently only works with G1 but can 
>>> easily be added for more GCs if needed. The type of the flag is 
>>> ccstrlist which means the option can be used multiple times to allow 
>>> more than one type to be verified. The types available for G1 is, 
>>> young, mixed, remark, cleanup and full. If the flag is not specified 
>>> all GCs are verified.
>>>
>>> Note that Verify/Before/After/During/GC is still needed to decide 
>>> what to verify, VerifyGCType only describes when.
>>>
>>> Testing:
>>> * Added new Gtest for G1HeapVerifier functionality.
>>> * Added new Jtreg test for basic command line functionality.
>>> * Executed Jtreg tests through mach5 to make sure it passes on all 
>>> platforms.
>> Thank you for improving this area!
>>
>> Looks good, I have minor nits.
>>
>> ----------------------------------------------------------------------
>> src/hotspot/share/gc/g1/g1CollectedHeap.cpp
>> 2987 _verifier->verify_before_gc(collector_state()->yc_type() == 
>> Mixed ? G1HeapVerifier::G1VerifyMixed : G1HeapVerifier::G1VerifyYoung);
>> 3147 _verifier->verify_after_gc(collector_state()->yc_type() == Mixed 
>> ? G1HeapVerifier::G1VerifyMixed : G1HeapVerifier::G1VerifyYoung);
>> - (weak suggestion) Change for better readability? e.g. split into 3 
>> lines
>>
> Fixed by adding a function to decide what type it is.
>> ----------------------------------------------------------------------
>> src/hotspot/share/gc/g1/g1ConcurrentMark.cpp
>> 1018 g1h->verifier()->verify(G1HeapVerifier::G1VerifyRemark, 
>> VerifyOption_G1UsePrevMarking, "During GC (before)");
>> 1039 g1h->verifier()->verify(G1HeapVerifier::G1VerifyRemark, 
>> VerifyOption_G1UsePrevMarking, "During GC (overflow)");
>> 1054 g1h->verifier()->verify(G1HeapVerifier::G1VerifyRemark, 
>> VerifyOption_G1UseNextMarking, "During GC (after)");
>> 1186 g1h->verifier()->verify(G1HeapVerifier::G1VerifyCleanup, 
>> VerifyOption_G1UsePrevMarking, "During GC (before)");
>> 1258 g1h->verifier()->verify(G1HeapVerifier::G1VerifyCleanup, 
>> VerifyOption_G1UsePrevMarking, "During GC (after)");
>> - (weak suggestion) Change for better readability? e.g. split into 3 
>> lines
>>
> I actually think the current version reads better, leaving as is.
>> ----------------------------------------------------------------------
>> src/hotspot/share/gc/g1/g1FullCollector.cpp
>> 249???? //Only do verification if VerifyDuringGC and Verify_Full is set.
>> - A space between '//' and 'Only'. (Not part of your change)
>>
> Fixed, and changed to G1VerifyFull.
>> ----------------------------------------------------------------------
>> src/hotspot/share/gc/g1/g1HeapVerifier.cpp
>> 380?? if (strcmp(type, "young") == 0) {
>> - strncmp
> We can't use strncmp here since we want to ensure a perfect match.
You are right.
I was confused with the ccstrlist length limitation. Sorry for the noise.

>>
>> 391???? log_warning(gc, verify)("VerifyGCType: '%s' is unknown. 
>> Available are: young, mixed, remark, cleanup and full ", type);
>> - "Available are" -> "Available types are"?
>>
> Fixed.
>> 396 // First enable will clear _types.
>> - '_types' seems old name. Probably something like 'Clear 
>> _enabled_verification_types if verifies all types'
>>
> Correct, fixed.
>> ----------------------------------------------------------------------
>> test/hotspot/gtest/gc/g1/test_g1HeapVerifier.cpp
>> 73
>> - Remove additional lines.
>>
> Fixed.
>> ----------------------------------------------------------------------
>> test/hotspot/jtreg/gc/g1/TestVerifyGCType.java
>> 42???? public static final String? VERIFY_TAG??? = "[gc,verify]";
>> - There are additional space between String and VERIFY_TAG. line 43, 
>> 44, 45 as well.
>>
>> 48???????? // Test with all verification enabled
>> 49???????? OutputAnalyzer output = testWithVerificationType(new 
>> String[0]);
>> - Can we split into sub-tests functions for better readability?
> Fixed.
>>
>> ----------------------------------------------------------------------
>> src/hotspot/share/runtime/globals.hpp
>> 2276????????????? "Available types are: young, mixed, concurrent, 
>> full")???????? \
>> - "concurrent," -> "remark, cleanup and"
> Fixed.
>>
>> * I don't see any length limitation logic for VerifyGCType. Not sure 
>> how other ccstrlist type command-line options are handled, but it 
>> would be better to have one.
> I don't see any other ccstrlist options do any specific length 
> handling and I'm not sure if there is a problem. For each time the 
> option VerifyGCType is encountered on the command line a new char 
> array will be allocated with NEW_C_HEAP_ARRAY and the old and new 
> value will be added to the new array. This can of course fail with a 
> native OOM, but so can a lot of other stuff during startup.
I'm okay with as is since all other same type command-line options don't 
check its length as well.

One thing that I thought is that if we have enum-string pair of all 
available VerifyGCType, we can use them for printing logs(e.g. when 
printing warning of all available types) and sum of those strings' 
length would be the maximum length of the new option.

Thanks,
Sangheon

>
>
> Thanks,
> Stefan
>
>>
>> Thanks,
>> Sangheon
>>
>>
>>>
>>> Thanks,
>>> Stefan
>>
>



From manc at google.com  Wed Nov 29 23:21:05 2017
From: manc at google.com (Man Cao)
Date: Wed, 29 Nov 2017 15:21:05 -0800
Subject: Make reserved_size for compressed class space and metaspace respect
 the ergo-initialized CompressedClassSpaceSize flag value
Message-ID: <CA+w6HxarhJmwyV2V6ahy_KdpYO+Db+qH6QhbHm6U6BmL7RzZbA@mail.gmail.com>

Hello,

This patch is a follow-up fix for
https://bugs.openjdk.java.net/browse/JDK-8087291

This patch moves the call to set_compressed_class_space_size() after the
flag value for CompressedClassSpaceSize is ergo-initialized, fixing the
issue that the reserved size for compressed class space and metaspace is
excessively large when MaxMetaspaceSize is set to a small value. More
discussion about it is available here:
http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2017-November/025200.html

This code patch is attached. Could anyone review and/or sponsor this patch?

Best,
Man
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20171129/0f288e44/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: jdk10hs.patch
Type: text/x-patch
Size: 970 bytes
Desc: not available
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20171129/0f288e44/jdk10hs.patch>

From david.holmes at oracle.com  Thu Nov 30 05:32:17 2017
From: david.holmes at oracle.com (David Holmes)
Date: Thu, 30 Nov 2017 15:32:17 +1000
Subject: RFR: 8191564: Refactor GC related servicability code into GC
 specific subclasses
In-Reply-To: <6f47a8ab-8b7e-c19c-2e49-10664ca90dd9@redhat.com>
References: <387d2b93-cb8b-51be-a6e0-3b5f9d6cbb1c@redhat.com>
 <713325a4-3ed8-cd7d-c66d-a1c1767b6d3a@redhat.com>
 <2a7e6307-8c0d-ad4b-e50c-a7e1c2311080@redhat.com>
 <56567db1-caf6-ed05-ed12-336edfca0c75@redhat.com> <5A1C3C07.30800@oracle.com>
 <b32b0580-8700-ec0f-bfb1-18a232e4d364@redhat.com>
 <5A1D8BE7.4070008@oracle.com>
 <a541a13e-ff82-b73a-b841-daa2ab910e88@redhat.com>
 <caac7013-f85b-b4e2-73b9-b81a7a57f704@oracle.com>
 <d89cb145-b197-feb6-89c2-a63a4d7a74eb@redhat.com>
 <4b899587-f7b4-82d7-4917-41d202e26ef4@oracle.com>
 <6f47a8ab-8b7e-c19c-2e49-10664ca90dd9@redhat.com>
Message-ID: <c2282a5d-906b-408e-5c72-38927bc4de56@oracle.com>

Hi Roman,

Just glancing at this but did you use "hg rename" to move the files? The 
webrev suggests not.

Thanks,
David

On 30/11/2017 1:38 AM, Roman Kennke wrote:
> Including hotspot-runtime-dev...
> 
> I need one more review (esp. for the src/hotspot/share/services part):
> 
> http://cr.openjdk.java.net/~rkennke/8191564/webrev.06/
> 
> Thanks, Roman
> 
> 
>> On 11/29/2017 09:39 AM, Roman Kennke wrote:
>>> Hi Erik,
>>>
>>> thanks for the review!
>>>
>>> I think this requires another reviewer from serviceability-dev. Who 
>>> can I ping about this?
>>
>> You could try the hotspot-runtime-dev email list, although I suspect 
>> most of the runtime developers are on serviceability-dev list as well...
>>
>> Thanks,
>> Erik
>>
>>> Roman
>>>
>>>
>>>> Hi Roman,
>>>>
>>>> Looks good now. Thanks for doing this.
>>>>
>>>> Thanks,
>>>> /Erik
>>>>
>>>> On 2017-11-28 22:26, Roman Kennke wrote:
>>>>> Hi Erik,
>>>>>
>>>>> thank you for reviewing!
>>>>>
>>>>> You are right. I think this is a leftover from when we tried to 
>>>>> pass the GCMemoryManager* into the Generation constructor. The way 
>>>>> it is done now (installing the GCMmemoryManager* later through 
>>>>> set_memory_manager()) we can group all serviceability related 
>>>>> set-up into initialize_serviceability():
>>>>>
>>>>> Differential:
>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.06.diff/
>>>>> Full:
>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.06/
>>>>>
>>>>> Notice that I hooked this up into CollectedHeap::post_initialize() 
>>>>> and in doing so made that method concrete, and changed all subclass 
>>>>> post_initialize() methods to call the super-class.
>>>>>
>>>>> Good now?
>>>>>
>>>>> Thanks, Roman
>>>>>
>>>>>
>>>>>> Hi Roman,
>>>>>>
>>>>>> This looks better now. Nice job.
>>>>>> I wonder though, is it possible to extract the creation of 
>>>>>> managers and pools to a separate function for each collected heap?
>>>>>> Now I see managers are created in e.g. CMS constructor, and the 
>>>>>> pools are created in CMSHeap::initialize(). Perhaps initialize 
>>>>>> could call initialize_serviceability() that sets up those things, 
>>>>>> so that they are nicely separated. Unless of course there is a 
>>>>>> good reason why the presence of managers is needed earlier than 
>>>>>> that in the bootstrapping.
>>>>>>
>>>>>> Otherwise I think this looks good!
>>>>>>
>>>>>> Thanks,
>>>>>> /Erik
>>>>>>
>>>>>> On 2017-11-28 12:22, Roman Kennke wrote:
>>>>>>> Hi Erik,
>>>>>>>
>>>>>>> Thanks for your review!
>>>>>>>
>>>>>>> All of the things that you mentioned should be addressed in the 
>>>>>>> following changes:
>>>>>>>
>>>>>>> Differential:
>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.05.diff/
>>>>>>> Full:
>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.05/
>>>>>>>
>>>>>>> Also, Erik D (H) was so kind to contribute an additional 
>>>>>>> testcase, which is also included in the above patch.
>>>>>>>
>>>>>>> Ok now?
>>>>>>>
>>>>>>> Also, I need a review from serviceability-dev too!
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Roman
>>>>>>>
>>>>>>>
>>>>>>>> 1) The code uses the "mgr" short name for "manager" in a bunch 
>>>>>>>> of names. I would be happy if we could write out the whole thing 
>>>>>>>> instead of the abbreviation.
>>>>>>>> 2) It would be great if a generation would have a pointer to its 
>>>>>>>> manager, so we do not have to pass around the manager where we 
>>>>>>>> already pass around the generation (such as 
>>>>>>>> GenCollectedHeap::collect_generation).
>>>>>>>> The generation could be created first, then the pools, then the 
>>>>>>>> managers, then do something like generation->set_memory_manager(x).
>>>>>>>> 3) In cmsHeap.cpp:54: maxSize should preferably not use camel case.
>>>>>>>>
>>>>>>>> Otherwise I think it looks good.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> /Erik
>>>>>>>>
>>>>>>>> On 2017-11-27 10:30, Roman Kennke wrote:
>>>>>>>>> Erik implemented a few more refactorings and touch-ups, and 
>>>>>>>>> here is our final (pending reviews) webrev:
>>>>>>>>>
>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.04/
>>>>>>>>>
>>>>>>>>> Compared to webrev.02, it improves the handling of 
>>>>>>>>> gc-end-message, avoids dragging the GCMemoryManager through 
>>>>>>>>> Generation and a few little related refactorings.
>>>>>>>>>
>>>>>>>>> Ok to push now?
>>>>>>>>>
>>>>>>>>> Roman
>>>>>>>>>
>>>>>>>>>> After a few more discussions with Erik I made some more changes.
>>>>>>>>>>
>>>>>>>>>> MemoryService is now unaware of the number and meaning of GC 
>>>>>>>>>> memory managers (minor vs major). This should be better for 
>>>>>>>>>> GCs that don't make that distinction and/or use more different 
>>>>>>>>>> GCs (e.g. minor, intermediate, full).
>>>>>>>>>>
>>>>>>>>>> This means that I needed to add some abstractions:
>>>>>>>>>> - GCMemoryManager now has gc_end_message() which is used by 
>>>>>>>>>> GCNotifier::pushNotification().
>>>>>>>>>> - gc_begin() and gc_end() methods in MemoryService now accept 
>>>>>>>>>> a GCMemoryManager* instead of bull full_gc
>>>>>>>>>> - Same for TraceMemoryManagerStats
>>>>>>>>>> - Generation now knows about the corresponding GCMemoryManager
>>>>>>>>>>
>>>>>>>>>> Please review the full change:
>>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.02/
>>>>>>>>>>
>>>>>>>>>> Thanks, Roman
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> I had some off-band discussions with Erik Helin and re-did 
>>>>>>>>>>> most of the changeset:
>>>>>>>>>>> - The GC interface now resides in CollectedHeap, specifically 
>>>>>>>>>>> the two methods memory_managers() and memory_pools(), and is 
>>>>>>>>>>> implemented in the various concrete subclasses.
>>>>>>>>>>> - Both methods return (by value) a 
>>>>>>>>>>> GrowableArray<GCMemoryManager*> and GrowableArray<MemoryPool> 
>>>>>>>>>>> respectively. Returning a stack-allocated GrowableArray 
>>>>>>>>>>> seemed least complicated (avoid explicit cleanup of 
>>>>>>>>>>> short-lived array object), and most future-proof, e.g. 
>>>>>>>>>>> currently there is an implicit expectation to get 2 
>>>>>>>>>>> GCMemoryManagers, even though some GCs don't necessarily have 
>>>>>>>>>>> two. The API allows for easy extension of the situation.
>>>>>>>>>>>
>>>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.01/
>>>>>>>>>>>
>>>>>>>>>>> I think this requires reviews from both the GC and 
>>>>>>>>>>> Serviceability team.
>>>>>>>>>>>
>>>>>>>>>>> Roman
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Currently, there's lots of GC specific code sprinkled over 
>>>>>>>>>>>> src/hotspot/share/services. This change introduces a GC 
>>>>>>>>>>>> interface for that, and moves all GC specific code to their 
>>>>>>>>>>>> respective src/hotspot/share/gc directory.
>>>>>>>>>>>>
>>>>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.00/ 
>>>>>>>>>>>> <http://cr.openjdk.java.net/%7Erkennke/8191564/webrev.00/>
>>>>>>>>>>>>
>>>>>>>>>>>> Testing: hotspot_gc and hotspot_serviceability, none showed 
>>>>>>>>>>>> regressions
>>>>>>>>>>>>
>>>>>>>>>>>> Built minimal and server without regressions
>>>>>>>>>>>>
>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>
>>>>>>>>>>>> Roman
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
> 


From per.liden at oracle.com  Thu Nov 30 10:32:33 2017
From: per.liden at oracle.com (Per Liden)
Date: Thu, 30 Nov 2017 11:32:33 +0100
Subject: RFR (S): 8192003: Refactor weak references in StringTable to use
 the Access API
In-Reply-To: <5A1D93C1.3020605@oracle.com>
References: <5A1D93C1.3020605@oracle.com>
Message-ID: <81e2ffcb-4912-9fc7-55d9-0553b737a695@oracle.com>

Hi Erik,

On 2017-11-28 17:50, Erik ?sterlund wrote:
> Hi,
>
> The StringTable contains weak references to oops. Today the weak
> semantics is managed using explicit calls to G1 SATB enqueue barriers.
>
> Now that the Access API is available, it should be used instead as it is
> more modular.
>
> This change fixes that by making these oops ON_PHANTOM_OOP_REF with a
> corresponding AS_NO_KEEPALIVE accessor when we do not want to keep it
> alive, much like my previous changes to other weak tables.
>
> Webrev:
> http://cr.openjdk.java.net/~eosterlund/8192003/webrev.00/

share/classfile/javaClasses.inline.hpp
--------------------------------------

   57 typeArrayOop java_lang_String::value_no_keepalive(oop java_string) {
   58   assert(initialized && (value_offset > 0), "Must be initialized");
   59   assert(is_instance(java_string), "must be java_string");
   60   oop value = 
HeapAccess<AS_NO_KEEPALIVE>::oop_load_at(java_string, value_offset);
   61   return (typeArrayOop)value;
   62 }

How about pushing this barrier down to oopDesc, with a 
oopDesc::obj_field_special_access<DecoratorSet D>(...) function?


   76   typeArrayOop value = 
java_lang_String::value_no_keepalive(java_string);
   77   assert(initialized, "Must be initialized");
   78   assert(is_instance(java_string), "must be java_string");

Looks like you accidentally moved the value_no_keepalive() call above 
the asserts?


share/classfile/stringTable.cpp
-------------------------------

  155       oop string = string_object_no_keepalive(l);
  156       if (java_lang_String::equals(string, name, len)) {
  157         return string_object(l);
  158       }

Can we please add a comment here, saying that returning "string" would 
be very bad, or maybe even fold this a bit, so that no one will be 
tempted to ever return that value, something like:

if (java_lang_String::equals(string_object_no_keepalive(l), name, len)) {
     // Comment saying why we must call string_object() here...
     return string_object(l);
}

cheers,
Per

>
> Bug:
> https://bugs.openjdk.java.net/browse/JDK-8192003
>
> Thanks,
> /Erik


From rkennke at redhat.com  Thu Nov 30 11:30:24 2017
From: rkennke at redhat.com (Roman Kennke)
Date: Thu, 30 Nov 2017 12:30:24 +0100
Subject: RFR: 8191564: Refactor GC related servicability code into GC
 specific subclasses
In-Reply-To: <c2282a5d-906b-408e-5c72-38927bc4de56@oracle.com>
References: <387d2b93-cb8b-51be-a6e0-3b5f9d6cbb1c@redhat.com>
 <713325a4-3ed8-cd7d-c66d-a1c1767b6d3a@redhat.com>
 <2a7e6307-8c0d-ad4b-e50c-a7e1c2311080@redhat.com>
 <56567db1-caf6-ed05-ed12-336edfca0c75@redhat.com> <5A1C3C07.30800@oracle.com>
 <b32b0580-8700-ec0f-bfb1-18a232e4d364@redhat.com>
 <5A1D8BE7.4070008@oracle.com>
 <a541a13e-ff82-b73a-b841-daa2ab910e88@redhat.com>
 <caac7013-f85b-b4e2-73b9-b81a7a57f704@oracle.com>
 <d89cb145-b197-feb6-89c2-a63a4d7a74eb@redhat.com>
 <4b899587-f7b4-82d7-4917-41d202e26ef4@oracle.com>
 <6f47a8ab-8b7e-c19c-2e49-10664ca90dd9@redhat.com>
 <c2282a5d-906b-408e-5c72-38927bc4de56@oracle.com>
Message-ID: <2a0f294a-b5d8-b2a8-971d-b1aa6146db2d@redhat.com>

Hi David,

yes I did, but it's probably got lost somewhere, while I was bouncing 
the patch between me and Erik D. (I noticed that the msg also got lost). 
Both reinstated:

http://cr.openjdk.java.net/~rkennke/8191564/webrev.07/

Roman

> Hi Roman,
> 
> Just glancing at this but did you use "hg rename" to move the files? The 
> webrev suggests not.
> 
> Thanks,
> David
> 
> On 30/11/2017 1:38 AM, Roman Kennke wrote:
>> Including hotspot-runtime-dev...
>>
>> I need one more review (esp. for the src/hotspot/share/services part):
>>
>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.06/
>>
>> Thanks, Roman
>>
>>
>>> On 11/29/2017 09:39 AM, Roman Kennke wrote:
>>>> Hi Erik,
>>>>
>>>> thanks for the review!
>>>>
>>>> I think this requires another reviewer from serviceability-dev. Who 
>>>> can I ping about this?
>>>
>>> You could try the hotspot-runtime-dev email list, although I suspect 
>>> most of the runtime developers are on serviceability-dev list as well...
>>>
>>> Thanks,
>>> Erik
>>>
>>>> Roman
>>>>
>>>>
>>>>> Hi Roman,
>>>>>
>>>>> Looks good now. Thanks for doing this.
>>>>>
>>>>> Thanks,
>>>>> /Erik
>>>>>
>>>>> On 2017-11-28 22:26, Roman Kennke wrote:
>>>>>> Hi Erik,
>>>>>>
>>>>>> thank you for reviewing!
>>>>>>
>>>>>> You are right. I think this is a leftover from when we tried to 
>>>>>> pass the GCMemoryManager* into the Generation constructor. The way 
>>>>>> it is done now (installing the GCMmemoryManager* later through 
>>>>>> set_memory_manager()) we can group all serviceability related 
>>>>>> set-up into initialize_serviceability():
>>>>>>
>>>>>> Differential:
>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.06.diff/
>>>>>> Full:
>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.06/
>>>>>>
>>>>>> Notice that I hooked this up into CollectedHeap::post_initialize() 
>>>>>> and in doing so made that method concrete, and changed all 
>>>>>> subclass post_initialize() methods to call the super-class.
>>>>>>
>>>>>> Good now?
>>>>>>
>>>>>> Thanks, Roman
>>>>>>
>>>>>>
>>>>>>> Hi Roman,
>>>>>>>
>>>>>>> This looks better now. Nice job.
>>>>>>> I wonder though, is it possible to extract the creation of 
>>>>>>> managers and pools to a separate function for each collected heap?
>>>>>>> Now I see managers are created in e.g. CMS constructor, and the 
>>>>>>> pools are created in CMSHeap::initialize(). Perhaps initialize 
>>>>>>> could call initialize_serviceability() that sets up those things, 
>>>>>>> so that they are nicely separated. Unless of course there is a 
>>>>>>> good reason why the presence of managers is needed earlier than 
>>>>>>> that in the bootstrapping.
>>>>>>>
>>>>>>> Otherwise I think this looks good!
>>>>>>>
>>>>>>> Thanks,
>>>>>>> /Erik
>>>>>>>
>>>>>>> On 2017-11-28 12:22, Roman Kennke wrote:
>>>>>>>> Hi Erik,
>>>>>>>>
>>>>>>>> Thanks for your review!
>>>>>>>>
>>>>>>>> All of the things that you mentioned should be addressed in the 
>>>>>>>> following changes:
>>>>>>>>
>>>>>>>> Differential:
>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.05.diff/
>>>>>>>> Full:
>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.05/
>>>>>>>>
>>>>>>>> Also, Erik D (H) was so kind to contribute an additional 
>>>>>>>> testcase, which is also included in the above patch.
>>>>>>>>
>>>>>>>> Ok now?
>>>>>>>>
>>>>>>>> Also, I need a review from serviceability-dev too!
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Roman
>>>>>>>>
>>>>>>>>
>>>>>>>>> 1) The code uses the "mgr" short name for "manager" in a bunch 
>>>>>>>>> of names. I would be happy if we could write out the whole 
>>>>>>>>> thing instead of the abbreviation.
>>>>>>>>> 2) It would be great if a generation would have a pointer to 
>>>>>>>>> its manager, so we do not have to pass around the manager where 
>>>>>>>>> we already pass around the generation (such as 
>>>>>>>>> GenCollectedHeap::collect_generation).
>>>>>>>>> The generation could be created first, then the pools, then the 
>>>>>>>>> managers, then do something like 
>>>>>>>>> generation->set_memory_manager(x).
>>>>>>>>> 3) In cmsHeap.cpp:54: maxSize should preferably not use camel 
>>>>>>>>> case.
>>>>>>>>>
>>>>>>>>> Otherwise I think it looks good.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> /Erik
>>>>>>>>>
>>>>>>>>> On 2017-11-27 10:30, Roman Kennke wrote:
>>>>>>>>>> Erik implemented a few more refactorings and touch-ups, and 
>>>>>>>>>> here is our final (pending reviews) webrev:
>>>>>>>>>>
>>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.04/
>>>>>>>>>>
>>>>>>>>>> Compared to webrev.02, it improves the handling of 
>>>>>>>>>> gc-end-message, avoids dragging the GCMemoryManager through 
>>>>>>>>>> Generation and a few little related refactorings.
>>>>>>>>>>
>>>>>>>>>> Ok to push now?
>>>>>>>>>>
>>>>>>>>>> Roman
>>>>>>>>>>
>>>>>>>>>>> After a few more discussions with Erik I made some more changes.
>>>>>>>>>>>
>>>>>>>>>>> MemoryService is now unaware of the number and meaning of GC 
>>>>>>>>>>> memory managers (minor vs major). This should be better for 
>>>>>>>>>>> GCs that don't make that distinction and/or use more 
>>>>>>>>>>> different GCs (e.g. minor, intermediate, full).
>>>>>>>>>>>
>>>>>>>>>>> This means that I needed to add some abstractions:
>>>>>>>>>>> - GCMemoryManager now has gc_end_message() which is used by 
>>>>>>>>>>> GCNotifier::pushNotification().
>>>>>>>>>>> - gc_begin() and gc_end() methods in MemoryService now accept 
>>>>>>>>>>> a GCMemoryManager* instead of bull full_gc
>>>>>>>>>>> - Same for TraceMemoryManagerStats
>>>>>>>>>>> - Generation now knows about the corresponding GCMemoryManager
>>>>>>>>>>>
>>>>>>>>>>> Please review the full change:
>>>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.02/
>>>>>>>>>>>
>>>>>>>>>>> Thanks, Roman
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> I had some off-band discussions with Erik Helin and re-did 
>>>>>>>>>>>> most of the changeset:
>>>>>>>>>>>> - The GC interface now resides in CollectedHeap, 
>>>>>>>>>>>> specifically the two methods memory_managers() and 
>>>>>>>>>>>> memory_pools(), and is implemented in the various concrete 
>>>>>>>>>>>> subclasses.
>>>>>>>>>>>> - Both methods return (by value) a 
>>>>>>>>>>>> GrowableArray<GCMemoryManager*> and 
>>>>>>>>>>>> GrowableArray<MemoryPool> respectively. Returning a 
>>>>>>>>>>>> stack-allocated GrowableArray seemed least complicated 
>>>>>>>>>>>> (avoid explicit cleanup of short-lived array object), and 
>>>>>>>>>>>> most future-proof, e.g. currently there is an implicit 
>>>>>>>>>>>> expectation to get 2 GCMemoryManagers, even though some GCs 
>>>>>>>>>>>> don't necessarily have two. The API allows for easy 
>>>>>>>>>>>> extension of the situation.
>>>>>>>>>>>>
>>>>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.01/
>>>>>>>>>>>>
>>>>>>>>>>>> I think this requires reviews from both the GC and 
>>>>>>>>>>>> Serviceability team.
>>>>>>>>>>>>
>>>>>>>>>>>> Roman
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> Currently, there's lots of GC specific code sprinkled over 
>>>>>>>>>>>>> src/hotspot/share/services. This change introduces a GC 
>>>>>>>>>>>>> interface for that, and moves all GC specific code to their 
>>>>>>>>>>>>> respective src/hotspot/share/gc directory.
>>>>>>>>>>>>>
>>>>>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.00/ 
>>>>>>>>>>>>> <http://cr.openjdk.java.net/%7Erkennke/8191564/webrev.00/>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Testing: hotspot_gc and hotspot_serviceability, none showed 
>>>>>>>>>>>>> regressions
>>>>>>>>>>>>>
>>>>>>>>>>>>> Built minimal and server without regressions
>>>>>>>>>>>>>
>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Roman
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>



From thomas.schatzl at oracle.com  Thu Nov 30 11:54:26 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 30 Nov 2017 12:54:26 +0100
Subject: RFR: 8191821: Finer granularity for GC verification
In-Reply-To: <8883ff1b-de5d-e44d-f554-696016535011@oracle.com>
References: <2319511b-8979-92b9-34f2-6a4c1af46ebf@oracle.com>
 <1511954183.2828.11.camel@oracle.com>
 <8883ff1b-de5d-e44d-f554-696016535011@oracle.com>
Message-ID: <1512042866.2422.5.camel@oracle.com>

Hi,

On Wed, 2017-11-29 at 16:43 +0100, Stefan Johansson wrote:
> Thanks for reviewing Thomas,
> 
> Updated webrevs:
> Inc:  http://cr.openjdk.java.net/~sjohanss/8191821/01-02/
> Full: http://cr.openjdk.java.net/~sjohanss/8191821/02/
> 
> [...]
> > 

Thanks for all your changes.

Two minor issues:

- in TestVerifyGCType.java, at the end there is a typo:
s/differnt/different

- please document the return type of
GCArguments::parse_verification_type. It's non-obvious imho.

Looks good otherwise. I do not need a re-review for these two nits
above.

Thanks,
  Thomas



From david.holmes at oracle.com  Thu Nov 30 12:21:25 2017
From: david.holmes at oracle.com (David Holmes)
Date: Thu, 30 Nov 2017 22:21:25 +1000
Subject: RFR: 8191564: Refactor GC related servicability code into GC
 specific subclasses
In-Reply-To: <2a0f294a-b5d8-b2a8-971d-b1aa6146db2d@redhat.com>
References: <387d2b93-cb8b-51be-a6e0-3b5f9d6cbb1c@redhat.com>
 <713325a4-3ed8-cd7d-c66d-a1c1767b6d3a@redhat.com>
 <2a7e6307-8c0d-ad4b-e50c-a7e1c2311080@redhat.com>
 <56567db1-caf6-ed05-ed12-336edfca0c75@redhat.com> <5A1C3C07.30800@oracle.com>
 <b32b0580-8700-ec0f-bfb1-18a232e4d364@redhat.com>
 <5A1D8BE7.4070008@oracle.com>
 <a541a13e-ff82-b73a-b841-daa2ab910e88@redhat.com>
 <caac7013-f85b-b4e2-73b9-b81a7a57f704@oracle.com>
 <d89cb145-b197-feb6-89c2-a63a4d7a74eb@redhat.com>
 <4b899587-f7b4-82d7-4917-41d202e26ef4@oracle.com>
 <6f47a8ab-8b7e-c19c-2e49-10664ca90dd9@redhat.com>
 <c2282a5d-906b-408e-5c72-38927bc4de56@oracle.com>
 <2a0f294a-b5d8-b2a8-971d-b1aa6146db2d@redhat.com>
Message-ID: <5c51ba9c-f64c-b8a9-7e46-1edeffb414ba@oracle.com>

On 30/11/2017 9:30 PM, Roman Kennke wrote:
> Hi David,
> 
> yes I did, but it's probably got lost somewhere, while I was bouncing 
> the patch between me and Erik D. (I noticed that the msg also got lost). 
> Both reinstated:
> 
> http://cr.openjdk.java.net/~rkennke/8191564/webrev.07/

Thanks - that's much simpler to follow.

IIRC in the test I think you need "@requires vm.gc == null" to skip the 
test if the framework is trying to run it with an explicit GC 
configuration - else it may conflict with your hardwired GC selection.

The overall refactoring seems reasonable, but I can't really vouch for 
its accuracy. I'm not clear how/if these service APIs are accesses from 
the Java-level serviceability code.

David

> Roman
> 
>> Hi Roman,
>>
>> Just glancing at this but did you use "hg rename" to move the files? 
>> The webrev suggests not.
>>
>> Thanks,
>> David
>>
>> On 30/11/2017 1:38 AM, Roman Kennke wrote:
>>> Including hotspot-runtime-dev...
>>>
>>> I need one more review (esp. for the src/hotspot/share/services part):
>>>
>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.06/
>>>
>>> Thanks, Roman
>>>
>>>
>>>> On 11/29/2017 09:39 AM, Roman Kennke wrote:
>>>>> Hi Erik,
>>>>>
>>>>> thanks for the review!
>>>>>
>>>>> I think this requires another reviewer from serviceability-dev. Who 
>>>>> can I ping about this?
>>>>
>>>> You could try the hotspot-runtime-dev email list, although I suspect 
>>>> most of the runtime developers are on serviceability-dev list as 
>>>> well...
>>>>
>>>> Thanks,
>>>> Erik
>>>>
>>>>> Roman
>>>>>
>>>>>
>>>>>> Hi Roman,
>>>>>>
>>>>>> Looks good now. Thanks for doing this.
>>>>>>
>>>>>> Thanks,
>>>>>> /Erik
>>>>>>
>>>>>> On 2017-11-28 22:26, Roman Kennke wrote:
>>>>>>> Hi Erik,
>>>>>>>
>>>>>>> thank you for reviewing!
>>>>>>>
>>>>>>> You are right. I think this is a leftover from when we tried to 
>>>>>>> pass the GCMemoryManager* into the Generation constructor. The 
>>>>>>> way it is done now (installing the GCMmemoryManager* later 
>>>>>>> through set_memory_manager()) we can group all serviceability 
>>>>>>> related set-up into initialize_serviceability():
>>>>>>>
>>>>>>> Differential:
>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.06.diff/
>>>>>>> Full:
>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.06/
>>>>>>>
>>>>>>> Notice that I hooked this up into 
>>>>>>> CollectedHeap::post_initialize() and in doing so made that method 
>>>>>>> concrete, and changed all subclass post_initialize() methods to 
>>>>>>> call the super-class.
>>>>>>>
>>>>>>> Good now?
>>>>>>>
>>>>>>> Thanks, Roman
>>>>>>>
>>>>>>>
>>>>>>>> Hi Roman,
>>>>>>>>
>>>>>>>> This looks better now. Nice job.
>>>>>>>> I wonder though, is it possible to extract the creation of 
>>>>>>>> managers and pools to a separate function for each collected heap?
>>>>>>>> Now I see managers are created in e.g. CMS constructor, and the 
>>>>>>>> pools are created in CMSHeap::initialize(). Perhaps initialize 
>>>>>>>> could call initialize_serviceability() that sets up those 
>>>>>>>> things, so that they are nicely separated. Unless of course 
>>>>>>>> there is a good reason why the presence of managers is needed 
>>>>>>>> earlier than that in the bootstrapping.
>>>>>>>>
>>>>>>>> Otherwise I think this looks good!
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> /Erik
>>>>>>>>
>>>>>>>> On 2017-11-28 12:22, Roman Kennke wrote:
>>>>>>>>> Hi Erik,
>>>>>>>>>
>>>>>>>>> Thanks for your review!
>>>>>>>>>
>>>>>>>>> All of the things that you mentioned should be addressed in the 
>>>>>>>>> following changes:
>>>>>>>>>
>>>>>>>>> Differential:
>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.05.diff/
>>>>>>>>> Full:
>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.05/
>>>>>>>>>
>>>>>>>>> Also, Erik D (H) was so kind to contribute an additional 
>>>>>>>>> testcase, which is also included in the above patch.
>>>>>>>>>
>>>>>>>>> Ok now?
>>>>>>>>>
>>>>>>>>> Also, I need a review from serviceability-dev too!
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Roman
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> 1) The code uses the "mgr" short name for "manager" in a bunch 
>>>>>>>>>> of names. I would be happy if we could write out the whole 
>>>>>>>>>> thing instead of the abbreviation.
>>>>>>>>>> 2) It would be great if a generation would have a pointer to 
>>>>>>>>>> its manager, so we do not have to pass around the manager 
>>>>>>>>>> where we already pass around the generation (such as 
>>>>>>>>>> GenCollectedHeap::collect_generation).
>>>>>>>>>> The generation could be created first, then the pools, then 
>>>>>>>>>> the managers, then do something like 
>>>>>>>>>> generation->set_memory_manager(x).
>>>>>>>>>> 3) In cmsHeap.cpp:54: maxSize should preferably not use camel 
>>>>>>>>>> case.
>>>>>>>>>>
>>>>>>>>>> Otherwise I think it looks good.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> /Erik
>>>>>>>>>>
>>>>>>>>>> On 2017-11-27 10:30, Roman Kennke wrote:
>>>>>>>>>>> Erik implemented a few more refactorings and touch-ups, and 
>>>>>>>>>>> here is our final (pending reviews) webrev:
>>>>>>>>>>>
>>>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.04/
>>>>>>>>>>>
>>>>>>>>>>> Compared to webrev.02, it improves the handling of 
>>>>>>>>>>> gc-end-message, avoids dragging the GCMemoryManager through 
>>>>>>>>>>> Generation and a few little related refactorings.
>>>>>>>>>>>
>>>>>>>>>>> Ok to push now?
>>>>>>>>>>>
>>>>>>>>>>> Roman
>>>>>>>>>>>
>>>>>>>>>>>> After a few more discussions with Erik I made some more 
>>>>>>>>>>>> changes.
>>>>>>>>>>>>
>>>>>>>>>>>> MemoryService is now unaware of the number and meaning of GC 
>>>>>>>>>>>> memory managers (minor vs major). This should be better for 
>>>>>>>>>>>> GCs that don't make that distinction and/or use more 
>>>>>>>>>>>> different GCs (e.g. minor, intermediate, full).
>>>>>>>>>>>>
>>>>>>>>>>>> This means that I needed to add some abstractions:
>>>>>>>>>>>> - GCMemoryManager now has gc_end_message() which is used by 
>>>>>>>>>>>> GCNotifier::pushNotification().
>>>>>>>>>>>> - gc_begin() and gc_end() methods in MemoryService now 
>>>>>>>>>>>> accept a GCMemoryManager* instead of bull full_gc
>>>>>>>>>>>> - Same for TraceMemoryManagerStats
>>>>>>>>>>>> - Generation now knows about the corresponding GCMemoryManager
>>>>>>>>>>>>
>>>>>>>>>>>> Please review the full change:
>>>>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.02/
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks, Roman
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> I had some off-band discussions with Erik Helin and re-did 
>>>>>>>>>>>>> most of the changeset:
>>>>>>>>>>>>> - The GC interface now resides in CollectedHeap, 
>>>>>>>>>>>>> specifically the two methods memory_managers() and 
>>>>>>>>>>>>> memory_pools(), and is implemented in the various concrete 
>>>>>>>>>>>>> subclasses.
>>>>>>>>>>>>> - Both methods return (by value) a 
>>>>>>>>>>>>> GrowableArray<GCMemoryManager*> and 
>>>>>>>>>>>>> GrowableArray<MemoryPool> respectively. Returning a 
>>>>>>>>>>>>> stack-allocated GrowableArray seemed least complicated 
>>>>>>>>>>>>> (avoid explicit cleanup of short-lived array object), and 
>>>>>>>>>>>>> most future-proof, e.g. currently there is an implicit 
>>>>>>>>>>>>> expectation to get 2 GCMemoryManagers, even though some GCs 
>>>>>>>>>>>>> don't necessarily have two. The API allows for easy 
>>>>>>>>>>>>> extension of the situation.
>>>>>>>>>>>>>
>>>>>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.01/
>>>>>>>>>>>>>
>>>>>>>>>>>>> I think this requires reviews from both the GC and 
>>>>>>>>>>>>> Serviceability team.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Roman
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Currently, there's lots of GC specific code sprinkled over 
>>>>>>>>>>>>>> src/hotspot/share/services. This change introduces a GC 
>>>>>>>>>>>>>> interface for that, and moves all GC specific code to 
>>>>>>>>>>>>>> their respective src/hotspot/share/gc directory.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.00/ 
>>>>>>>>>>>>>> <http://cr.openjdk.java.net/%7Erkennke/8191564/webrev.00/>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Testing: hotspot_gc and hotspot_serviceability, none 
>>>>>>>>>>>>>> showed regressions
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Built minimal and server without regressions
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Roman
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>
> 


From erik.osterlund at oracle.com  Thu Nov 30 13:44:32 2017
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Thu, 30 Nov 2017 14:44:32 +0100
Subject: RFR (S): 8192003: Refactor weak references in StringTable to use
 the Access API
In-Reply-To: <81e2ffcb-4912-9fc7-55d9-0553b737a695@oracle.com>
References: <5A1D93C1.3020605@oracle.com>
 <81e2ffcb-4912-9fc7-55d9-0553b737a695@oracle.com>
Message-ID: <5A200B40.9060206@oracle.com>

Hi Per,

Thank you for reviewing this.

New full webrev:
http://cr.openjdk.java.net/~eosterlund/8192003/webrev.01/

New incremental webrev:
http://cr.openjdk.java.net/~eosterlund/8192003/webrev.00_01/

On 2017-11-30 11:32, Per Liden wrote:
> Hi Erik,
>
> On 2017-11-28 17:50, Erik ?sterlund wrote:
>> Hi,
>>
>> The StringTable contains weak references to oops. Today the weak
>> semantics is managed using explicit calls to G1 SATB enqueue barriers.
>>
>> Now that the Access API is available, it should be used instead as it is
>> more modular.
>>
>> This change fixes that by making these oops ON_PHANTOM_OOP_REF with a
>> corresponding AS_NO_KEEPALIVE accessor when we do not want to keep it
>> alive, much like my previous changes to other weak tables.
>>
>> Webrev:
>> http://cr.openjdk.java.net/~eosterlund/8192003/webrev.00/
>
> share/classfile/javaClasses.inline.hpp
> --------------------------------------
>
>   57 typeArrayOop java_lang_String::value_no_keepalive(oop java_string) {
>   58   assert(initialized && (value_offset > 0), "Must be initialized");
>   59   assert(is_instance(java_string), "must be java_string");
>   60   oop value = 
> HeapAccess<AS_NO_KEEPALIVE>::oop_load_at(java_string, value_offset);
>   61   return (typeArrayOop)value;
>   62 }
>
> How about pushing this barrier down to oopDesc, with a 
> oopDesc::obj_field_special_access<DecoratorSet D>(...) function?

Sounds doable. Fixed. (Although I called the method just 
"obj_field_special", hope nobody minds...)

>
>   76   typeArrayOop value = 
> java_lang_String::value_no_keepalive(java_string);
>   77   assert(initialized, "Must be initialized");
>   78   assert(is_instance(java_string), "must be java_string");
>
> Looks like you accidentally moved the value_no_keepalive() call above 
> the asserts?

Fixed.

>
> share/classfile/stringTable.cpp
> -------------------------------
>
>  155       oop string = string_object_no_keepalive(l);
>  156       if (java_lang_String::equals(string, name, len)) {
>  157         return string_object(l);
>  158       }
>
> Can we please add a comment here, saying that returning "string" would 
> be very bad, or maybe even fold this a bit, so that no one will be 
> tempted to ever return that value, something like:
>
> if (java_lang_String::equals(string_object_no_keepalive(l), name, len)) {
>     // Comment saying why we must call string_object() here...
>     return string_object(l);
> }

Fixed.

Thanks,
/Erik

> cheers,
> Per
>
>>
>> Bug:
>> https://bugs.openjdk.java.net/browse/JDK-8192003
>>
>> Thanks,
>> /Erik



From rkennke at redhat.com  Thu Nov 30 14:22:09 2017
From: rkennke at redhat.com (Roman Kennke)
Date: Thu, 30 Nov 2017 15:22:09 +0100
Subject: RFR: 8191564: Refactor GC related servicability code into GC
 specific subclasses
In-Reply-To: <5c51ba9c-f64c-b8a9-7e46-1edeffb414ba@oracle.com>
References: <387d2b93-cb8b-51be-a6e0-3b5f9d6cbb1c@redhat.com>
 <713325a4-3ed8-cd7d-c66d-a1c1767b6d3a@redhat.com>
 <2a7e6307-8c0d-ad4b-e50c-a7e1c2311080@redhat.com>
 <56567db1-caf6-ed05-ed12-336edfca0c75@redhat.com> <5A1C3C07.30800@oracle.com>
 <b32b0580-8700-ec0f-bfb1-18a232e4d364@redhat.com>
 <5A1D8BE7.4070008@oracle.com>
 <a541a13e-ff82-b73a-b841-daa2ab910e88@redhat.com>
 <caac7013-f85b-b4e2-73b9-b81a7a57f704@oracle.com>
 <d89cb145-b197-feb6-89c2-a63a4d7a74eb@redhat.com>
 <4b899587-f7b4-82d7-4917-41d202e26ef4@oracle.com>
 <6f47a8ab-8b7e-c19c-2e49-10664ca90dd9@redhat.com>
 <c2282a5d-906b-408e-5c72-38927bc4de56@oracle.com>
 <2a0f294a-b5d8-b2a8-971d-b1aa6146db2d@redhat.com>
 <5c51ba9c-f64c-b8a9-7e46-1edeffb414ba@oracle.com>
Message-ID: <243153d8-1836-9d3e-b242-52e17b2e2134@redhat.com>

Hi David,

I added the tag as you proposed:

Differential:
http://cr.openjdk.java.net/~rkennke/8191564/webrev.08.diff/
Full:
http://cr.openjdk.java.net/~rkennke/8191564/webrev.08/

Re-run tests without regressions.

Roman

> On 30/11/2017 9:30 PM, Roman Kennke wrote:
>> Hi David,
>>
>> yes I did, but it's probably got lost somewhere, while I was bouncing 
>> the patch between me and Erik D. (I noticed that the msg also got 
>> lost). Both reinstated:
>>
>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.07/
> 
> Thanks - that's much simpler to follow.
> 
> IIRC in the test I think you need "@requires vm.gc == null" to skip the 
> test if the framework is trying to run it with an explicit GC 
> configuration - else it may conflict with your hardwired GC selection.
> 
> The overall refactoring seems reasonable, but I can't really vouch for 
> its accuracy. I'm not clear how/if these service APIs are accesses from 
> the Java-level serviceability code.
> 
> David
> 
>> Roman
>>
>>> Hi Roman,
>>>
>>> Just glancing at this but did you use "hg rename" to move the files? 
>>> The webrev suggests not.
>>>
>>> Thanks,
>>> David
>>>
>>> On 30/11/2017 1:38 AM, Roman Kennke wrote:
>>>> Including hotspot-runtime-dev...
>>>>
>>>> I need one more review (esp. for the src/hotspot/share/services part):
>>>>
>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.06/
>>>>
>>>> Thanks, Roman
>>>>
>>>>
>>>>> On 11/29/2017 09:39 AM, Roman Kennke wrote:
>>>>>> Hi Erik,
>>>>>>
>>>>>> thanks for the review!
>>>>>>
>>>>>> I think this requires another reviewer from serviceability-dev. 
>>>>>> Who can I ping about this?
>>>>>
>>>>> You could try the hotspot-runtime-dev email list, although I 
>>>>> suspect most of the runtime developers are on serviceability-dev 
>>>>> list as well...
>>>>>
>>>>> Thanks,
>>>>> Erik
>>>>>
>>>>>> Roman
>>>>>>
>>>>>>
>>>>>>> Hi Roman,
>>>>>>>
>>>>>>> Looks good now. Thanks for doing this.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> /Erik
>>>>>>>
>>>>>>> On 2017-11-28 22:26, Roman Kennke wrote:
>>>>>>>> Hi Erik,
>>>>>>>>
>>>>>>>> thank you for reviewing!
>>>>>>>>
>>>>>>>> You are right. I think this is a leftover from when we tried to 
>>>>>>>> pass the GCMemoryManager* into the Generation constructor. The 
>>>>>>>> way it is done now (installing the GCMmemoryManager* later 
>>>>>>>> through set_memory_manager()) we can group all serviceability 
>>>>>>>> related set-up into initialize_serviceability():
>>>>>>>>
>>>>>>>> Differential:
>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.06.diff/
>>>>>>>> Full:
>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.06/
>>>>>>>>
>>>>>>>> Notice that I hooked this up into 
>>>>>>>> CollectedHeap::post_initialize() and in doing so made that 
>>>>>>>> method concrete, and changed all subclass post_initialize() 
>>>>>>>> methods to call the super-class.
>>>>>>>>
>>>>>>>> Good now?
>>>>>>>>
>>>>>>>> Thanks, Roman
>>>>>>>>
>>>>>>>>
>>>>>>>>> Hi Roman,
>>>>>>>>>
>>>>>>>>> This looks better now. Nice job.
>>>>>>>>> I wonder though, is it possible to extract the creation of 
>>>>>>>>> managers and pools to a separate function for each collected heap?
>>>>>>>>> Now I see managers are created in e.g. CMS constructor, and the 
>>>>>>>>> pools are created in CMSHeap::initialize(). Perhaps initialize 
>>>>>>>>> could call initialize_serviceability() that sets up those 
>>>>>>>>> things, so that they are nicely separated. Unless of course 
>>>>>>>>> there is a good reason why the presence of managers is needed 
>>>>>>>>> earlier than that in the bootstrapping.
>>>>>>>>>
>>>>>>>>> Otherwise I think this looks good!
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> /Erik
>>>>>>>>>
>>>>>>>>> On 2017-11-28 12:22, Roman Kennke wrote:
>>>>>>>>>> Hi Erik,
>>>>>>>>>>
>>>>>>>>>> Thanks for your review!
>>>>>>>>>>
>>>>>>>>>> All of the things that you mentioned should be addressed in 
>>>>>>>>>> the following changes:
>>>>>>>>>>
>>>>>>>>>> Differential:
>>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.05.diff/
>>>>>>>>>> Full:
>>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.05/
>>>>>>>>>>
>>>>>>>>>> Also, Erik D (H) was so kind to contribute an additional 
>>>>>>>>>> testcase, which is also included in the above patch.
>>>>>>>>>>
>>>>>>>>>> Ok now?
>>>>>>>>>>
>>>>>>>>>> Also, I need a review from serviceability-dev too!
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Roman
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> 1) The code uses the "mgr" short name for "manager" in a 
>>>>>>>>>>> bunch of names. I would be happy if we could write out the 
>>>>>>>>>>> whole thing instead of the abbreviation.
>>>>>>>>>>> 2) It would be great if a generation would have a pointer to 
>>>>>>>>>>> its manager, so we do not have to pass around the manager 
>>>>>>>>>>> where we already pass around the generation (such as 
>>>>>>>>>>> GenCollectedHeap::collect_generation).
>>>>>>>>>>> The generation could be created first, then the pools, then 
>>>>>>>>>>> the managers, then do something like 
>>>>>>>>>>> generation->set_memory_manager(x).
>>>>>>>>>>> 3) In cmsHeap.cpp:54: maxSize should preferably not use camel 
>>>>>>>>>>> case.
>>>>>>>>>>>
>>>>>>>>>>> Otherwise I think it looks good.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> /Erik
>>>>>>>>>>>
>>>>>>>>>>> On 2017-11-27 10:30, Roman Kennke wrote:
>>>>>>>>>>>> Erik implemented a few more refactorings and touch-ups, and 
>>>>>>>>>>>> here is our final (pending reviews) webrev:
>>>>>>>>>>>>
>>>>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.04/
>>>>>>>>>>>>
>>>>>>>>>>>> Compared to webrev.02, it improves the handling of 
>>>>>>>>>>>> gc-end-message, avoids dragging the GCMemoryManager through 
>>>>>>>>>>>> Generation and a few little related refactorings.
>>>>>>>>>>>>
>>>>>>>>>>>> Ok to push now?
>>>>>>>>>>>>
>>>>>>>>>>>> Roman
>>>>>>>>>>>>
>>>>>>>>>>>>> After a few more discussions with Erik I made some more 
>>>>>>>>>>>>> changes.
>>>>>>>>>>>>>
>>>>>>>>>>>>> MemoryService is now unaware of the number and meaning of 
>>>>>>>>>>>>> GC memory managers (minor vs major). This should be better 
>>>>>>>>>>>>> for GCs that don't make that distinction and/or use more 
>>>>>>>>>>>>> different GCs (e.g. minor, intermediate, full).
>>>>>>>>>>>>>
>>>>>>>>>>>>> This means that I needed to add some abstractions:
>>>>>>>>>>>>> - GCMemoryManager now has gc_end_message() which is used by 
>>>>>>>>>>>>> GCNotifier::pushNotification().
>>>>>>>>>>>>> - gc_begin() and gc_end() methods in MemoryService now 
>>>>>>>>>>>>> accept a GCMemoryManager* instead of bull full_gc
>>>>>>>>>>>>> - Same for TraceMemoryManagerStats
>>>>>>>>>>>>> - Generation now knows about the corresponding GCMemoryManager
>>>>>>>>>>>>>
>>>>>>>>>>>>> Please review the full change:
>>>>>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.02/
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks, Roman
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> I had some off-band discussions with Erik Helin and re-did 
>>>>>>>>>>>>>> most of the changeset:
>>>>>>>>>>>>>> - The GC interface now resides in CollectedHeap, 
>>>>>>>>>>>>>> specifically the two methods memory_managers() and 
>>>>>>>>>>>>>> memory_pools(), and is implemented in the various concrete 
>>>>>>>>>>>>>> subclasses.
>>>>>>>>>>>>>> - Both methods return (by value) a 
>>>>>>>>>>>>>> GrowableArray<GCMemoryManager*> and 
>>>>>>>>>>>>>> GrowableArray<MemoryPool> respectively. Returning a 
>>>>>>>>>>>>>> stack-allocated GrowableArray seemed least complicated 
>>>>>>>>>>>>>> (avoid explicit cleanup of short-lived array object), and 
>>>>>>>>>>>>>> most future-proof, e.g. currently there is an implicit 
>>>>>>>>>>>>>> expectation to get 2 GCMemoryManagers, even though some 
>>>>>>>>>>>>>> GCs don't necessarily have two. The API allows for easy 
>>>>>>>>>>>>>> extension of the situation.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.01/
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I think this requires reviews from both the GC and 
>>>>>>>>>>>>>> Serviceability team.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Roman
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Currently, there's lots of GC specific code sprinkled 
>>>>>>>>>>>>>>> over src/hotspot/share/services. This change introduces a 
>>>>>>>>>>>>>>> GC interface for that, and moves all GC specific code to 
>>>>>>>>>>>>>>> their respective src/hotspot/share/gc directory.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.00/ 
>>>>>>>>>>>>>>> <http://cr.openjdk.java.net/%7Erkennke/8191564/webrev.00/>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Testing: hotspot_gc and hotspot_serviceability, none 
>>>>>>>>>>>>>>> showed regressions
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Built minimal and server without regressions
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Roman
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>



From erik.osterlund at oracle.com  Thu Nov 30 14:55:11 2017
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Thu, 30 Nov 2017 15:55:11 +0100
Subject: RFR: 8191564: Refactor GC related servicability code into GC
 specific subclasses
In-Reply-To: <243153d8-1836-9d3e-b242-52e17b2e2134@redhat.com>
References: <387d2b93-cb8b-51be-a6e0-3b5f9d6cbb1c@redhat.com>
 <713325a4-3ed8-cd7d-c66d-a1c1767b6d3a@redhat.com>
 <2a7e6307-8c0d-ad4b-e50c-a7e1c2311080@redhat.com>
 <56567db1-caf6-ed05-ed12-336edfca0c75@redhat.com> <5A1C3C07.30800@oracle.com>
 <b32b0580-8700-ec0f-bfb1-18a232e4d364@redhat.com>
 <5A1D8BE7.4070008@oracle.com>
 <a541a13e-ff82-b73a-b841-daa2ab910e88@redhat.com>
 <caac7013-f85b-b4e2-73b9-b81a7a57f704@oracle.com>
 <d89cb145-b197-feb6-89c2-a63a4d7a74eb@redhat.com>
 <4b899587-f7b4-82d7-4917-41d202e26ef4@oracle.com>
 <6f47a8ab-8b7e-c19c-2e49-10664ca90dd9@redhat.com>
 <c2282a5d-906b-408e-5c72-38927bc4de56@oracle.com>
 <2a0f294a-b5d8-b2a8-971d-b1aa6146db2d@redhat.com>
 <5c51ba9c-f64c-b8a9-7e46-1edeffb414ba@oracle.com>
 <243153d8-1836-9d3e-b242-52e17b2e2134@redhat.com>
Message-ID: <5A201BCF.8030307@oracle.com>

Hi Roman,

Looks good.
You do need a class GCMemoryManager; declaration in generation.hpp to 
build without precompiled headers though.
I do not need a new webrev for that change.

Thanks,
/Erik

On 2017-11-30 15:22, Roman Kennke wrote:
> Hi David,
>
> I added the tag as you proposed:
>
> Differential:
> http://cr.openjdk.java.net/~rkennke/8191564/webrev.08.diff/
> Full:
> http://cr.openjdk.java.net/~rkennke/8191564/webrev.08/
>
> Re-run tests without regressions.
>
> Roman
>
>> On 30/11/2017 9:30 PM, Roman Kennke wrote:
>>> Hi David,
>>>
>>> yes I did, but it's probably got lost somewhere, while I was 
>>> bouncing the patch between me and Erik D. (I noticed that the msg 
>>> also got lost). Both reinstated:
>>>
>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.07/
>>
>> Thanks - that's much simpler to follow.
>>
>> IIRC in the test I think you need "@requires vm.gc == null" to skip 
>> the test if the framework is trying to run it with an explicit GC 
>> configuration - else it may conflict with your hardwired GC selection.
>>
>> The overall refactoring seems reasonable, but I can't really vouch 
>> for its accuracy. I'm not clear how/if these service APIs are 
>> accesses from the Java-level serviceability code.
>>
>> David
>>
>>> Roman
>>>
>>>> Hi Roman,
>>>>
>>>> Just glancing at this but did you use "hg rename" to move the 
>>>> files? The webrev suggests not.
>>>>
>>>> Thanks,
>>>> David
>>>>
>>>> On 30/11/2017 1:38 AM, Roman Kennke wrote:
>>>>> Including hotspot-runtime-dev...
>>>>>
>>>>> I need one more review (esp. for the src/hotspot/share/services 
>>>>> part):
>>>>>
>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.06/
>>>>>
>>>>> Thanks, Roman
>>>>>
>>>>>
>>>>>> On 11/29/2017 09:39 AM, Roman Kennke wrote:
>>>>>>> Hi Erik,
>>>>>>>
>>>>>>> thanks for the review!
>>>>>>>
>>>>>>> I think this requires another reviewer from serviceability-dev. 
>>>>>>> Who can I ping about this?
>>>>>>
>>>>>> You could try the hotspot-runtime-dev email list, although I 
>>>>>> suspect most of the runtime developers are on serviceability-dev 
>>>>>> list as well...
>>>>>>
>>>>>> Thanks,
>>>>>> Erik
>>>>>>
>>>>>>> Roman
>>>>>>>
>>>>>>>
>>>>>>>> Hi Roman,
>>>>>>>>
>>>>>>>> Looks good now. Thanks for doing this.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> /Erik
>>>>>>>>
>>>>>>>> On 2017-11-28 22:26, Roman Kennke wrote:
>>>>>>>>> Hi Erik,
>>>>>>>>>
>>>>>>>>> thank you for reviewing!
>>>>>>>>>
>>>>>>>>> You are right. I think this is a leftover from when we tried 
>>>>>>>>> to pass the GCMemoryManager* into the Generation constructor. 
>>>>>>>>> The way it is done now (installing the GCMmemoryManager* later 
>>>>>>>>> through set_memory_manager()) we can group all serviceability 
>>>>>>>>> related set-up into initialize_serviceability():
>>>>>>>>>
>>>>>>>>> Differential:
>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.06.diff/
>>>>>>>>> Full:
>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.06/
>>>>>>>>>
>>>>>>>>> Notice that I hooked this up into 
>>>>>>>>> CollectedHeap::post_initialize() and in doing so made that 
>>>>>>>>> method concrete, and changed all subclass post_initialize() 
>>>>>>>>> methods to call the super-class.
>>>>>>>>>
>>>>>>>>> Good now?
>>>>>>>>>
>>>>>>>>> Thanks, Roman
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Hi Roman,
>>>>>>>>>>
>>>>>>>>>> This looks better now. Nice job.
>>>>>>>>>> I wonder though, is it possible to extract the creation of 
>>>>>>>>>> managers and pools to a separate function for each collected 
>>>>>>>>>> heap?
>>>>>>>>>> Now I see managers are created in e.g. CMS constructor, and 
>>>>>>>>>> the pools are created in CMSHeap::initialize(). Perhaps 
>>>>>>>>>> initialize could call initialize_serviceability() that sets 
>>>>>>>>>> up those things, so that they are nicely separated. Unless of 
>>>>>>>>>> course there is a good reason why the presence of managers is 
>>>>>>>>>> needed earlier than that in the bootstrapping.
>>>>>>>>>>
>>>>>>>>>> Otherwise I think this looks good!
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> /Erik
>>>>>>>>>>
>>>>>>>>>> On 2017-11-28 12:22, Roman Kennke wrote:
>>>>>>>>>>> Hi Erik,
>>>>>>>>>>>
>>>>>>>>>>> Thanks for your review!
>>>>>>>>>>>
>>>>>>>>>>> All of the things that you mentioned should be addressed in 
>>>>>>>>>>> the following changes:
>>>>>>>>>>>
>>>>>>>>>>> Differential:
>>>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.05.diff/
>>>>>>>>>>> Full:
>>>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.05/
>>>>>>>>>>>
>>>>>>>>>>> Also, Erik D (H) was so kind to contribute an additional 
>>>>>>>>>>> testcase, which is also included in the above patch.
>>>>>>>>>>>
>>>>>>>>>>> Ok now?
>>>>>>>>>>>
>>>>>>>>>>> Also, I need a review from serviceability-dev too!
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Roman
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> 1) The code uses the "mgr" short name for "manager" in a 
>>>>>>>>>>>> bunch of names. I would be happy if we could write out the 
>>>>>>>>>>>> whole thing instead of the abbreviation.
>>>>>>>>>>>> 2) It would be great if a generation would have a pointer 
>>>>>>>>>>>> to its manager, so we do not have to pass around the 
>>>>>>>>>>>> manager where we already pass around the generation (such 
>>>>>>>>>>>> as GenCollectedHeap::collect_generation).
>>>>>>>>>>>> The generation could be created first, then the pools, then 
>>>>>>>>>>>> the managers, then do something like 
>>>>>>>>>>>> generation->set_memory_manager(x).
>>>>>>>>>>>> 3) In cmsHeap.cpp:54: maxSize should preferably not use 
>>>>>>>>>>>> camel case.
>>>>>>>>>>>>
>>>>>>>>>>>> Otherwise I think it looks good.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> /Erik
>>>>>>>>>>>>
>>>>>>>>>>>> On 2017-11-27 10:30, Roman Kennke wrote:
>>>>>>>>>>>>> Erik implemented a few more refactorings and touch-ups, 
>>>>>>>>>>>>> and here is our final (pending reviews) webrev:
>>>>>>>>>>>>>
>>>>>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.04/
>>>>>>>>>>>>>
>>>>>>>>>>>>> Compared to webrev.02, it improves the handling of 
>>>>>>>>>>>>> gc-end-message, avoids dragging the GCMemoryManager 
>>>>>>>>>>>>> through Generation and a few little related refactorings.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Ok to push now?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Roman
>>>>>>>>>>>>>
>>>>>>>>>>>>>> After a few more discussions with Erik I made some more 
>>>>>>>>>>>>>> changes.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> MemoryService is now unaware of the number and meaning of 
>>>>>>>>>>>>>> GC memory managers (minor vs major). This should be 
>>>>>>>>>>>>>> better for GCs that don't make that distinction and/or 
>>>>>>>>>>>>>> use more different GCs (e.g. minor, intermediate, full).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This means that I needed to add some abstractions:
>>>>>>>>>>>>>> - GCMemoryManager now has gc_end_message() which is used 
>>>>>>>>>>>>>> by GCNotifier::pushNotification().
>>>>>>>>>>>>>> - gc_begin() and gc_end() methods in MemoryService now 
>>>>>>>>>>>>>> accept a GCMemoryManager* instead of bull full_gc
>>>>>>>>>>>>>> - Same for TraceMemoryManagerStats
>>>>>>>>>>>>>> - Generation now knows about the corresponding 
>>>>>>>>>>>>>> GCMemoryManager
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Please review the full change:
>>>>>>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.02/
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks, Roman
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I had some off-band discussions with Erik Helin and 
>>>>>>>>>>>>>>> re-did most of the changeset:
>>>>>>>>>>>>>>> - The GC interface now resides in CollectedHeap, 
>>>>>>>>>>>>>>> specifically the two methods memory_managers() and 
>>>>>>>>>>>>>>> memory_pools(), and is implemented in the various 
>>>>>>>>>>>>>>> concrete subclasses.
>>>>>>>>>>>>>>> - Both methods return (by value) a 
>>>>>>>>>>>>>>> GrowableArray<GCMemoryManager*> and 
>>>>>>>>>>>>>>> GrowableArray<MemoryPool> respectively. Returning a 
>>>>>>>>>>>>>>> stack-allocated GrowableArray seemed least complicated 
>>>>>>>>>>>>>>> (avoid explicit cleanup of short-lived array object), 
>>>>>>>>>>>>>>> and most future-proof, e.g. currently there is an 
>>>>>>>>>>>>>>> implicit expectation to get 2 GCMemoryManagers, even 
>>>>>>>>>>>>>>> though some GCs don't necessarily have two. The API 
>>>>>>>>>>>>>>> allows for easy extension of the situation.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.01/
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I think this requires reviews from both the GC and 
>>>>>>>>>>>>>>> Serviceability team.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Roman
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Currently, there's lots of GC specific code sprinkled 
>>>>>>>>>>>>>>>> over src/hotspot/share/services. This change introduces 
>>>>>>>>>>>>>>>> a GC interface for that, and moves all GC specific code 
>>>>>>>>>>>>>>>> to their respective src/hotspot/share/gc directory.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8191564/webrev.00/ 
>>>>>>>>>>>>>>>> <http://cr.openjdk.java.net/%7Erkennke/8191564/webrev.00/>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Testing: hotspot_gc and hotspot_serviceability, none 
>>>>>>>>>>>>>>>> showed regressions
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Built minimal and server without regressions
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Roman
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>
>



From per.liden at oracle.com  Thu Nov 30 15:43:20 2017
From: per.liden at oracle.com (=?utf-8?Q?Per_Lid=C3=A9n?=)
Date: Thu, 30 Nov 2017 16:43:20 +0100
Subject: RFR (S): 8192003: Refactor weak references in StringTable to use
 the Access API
In-Reply-To: <5A200B40.9060206@oracle.com>
References: <5A1D93C1.3020605@oracle.com>
 <81e2ffcb-4912-9fc7-55d9-0553b737a695@oracle.com>
 <5A200B40.9060206@oracle.com>
Message-ID: <C14AEF77-BC36-4076-8C5E-BC7A76B3A647@oracle.com>

Looks good!

/Per

> On 30 Nov 2017, at 14:44, Erik ?sterlund <erik.osterlund at oracle.com> wrote:
> 
> Hi Per,
> 
> Thank you for reviewing this.
> 
> New full webrev:
> http://cr.openjdk.java.net/~eosterlund/8192003/webrev.01/
> 
> New incremental webrev:
> http://cr.openjdk.java.net/~eosterlund/8192003/webrev.00_01/
> 
>> On 2017-11-30 11:32, Per Liden wrote:
>> Hi Erik,
>> 
>>> On 2017-11-28 17:50, Erik ?sterlund wrote:
>>> Hi,
>>> 
>>> The StringTable contains weak references to oops. Today the weak
>>> semantics is managed using explicit calls to G1 SATB enqueue barriers.
>>> 
>>> Now that the Access API is available, it should be used instead as it is
>>> more modular.
>>> 
>>> This change fixes that by making these oops ON_PHANTOM_OOP_REF with a
>>> corresponding AS_NO_KEEPALIVE accessor when we do not want to keep it
>>> alive, much like my previous changes to other weak tables.
>>> 
>>> Webrev:
>>> http://cr.openjdk.java.net/~eosterlund/8192003/webrev.00/
>> 
>> share/classfile/javaClasses.inline.hpp
>> --------------------------------------
>> 
>>  57 typeArrayOop java_lang_String::value_no_keepalive(oop java_string) {
>>  58   assert(initialized && (value_offset > 0), "Must be initialized");
>>  59   assert(is_instance(java_string), "must be java_string");
>>  60   oop value = HeapAccess<AS_NO_KEEPALIVE>::oop_load_at(java_string, value_offset);
>>  61   return (typeArrayOop)value;
>>  62 }
>> 
>> How about pushing this barrier down to oopDesc, with a oopDesc::obj_field_special_access<DecoratorSet D>(...) function?
> 
> Sounds doable. Fixed. (Although I called the method just "obj_field_special", hope nobody minds...)
> 
>> 
>>  76   typeArrayOop value = java_lang_String::value_no_keepalive(java_string);
>>  77   assert(initialized, "Must be initialized");
>>  78   assert(is_instance(java_string), "must be java_string");
>> 
>> Looks like you accidentally moved the value_no_keepalive() call above the asserts?
> 
> Fixed.
> 
>> 
>> share/classfile/stringTable.cpp
>> -------------------------------
>> 
>> 155       oop string = string_object_no_keepalive(l);
>> 156       if (java_lang_String::equals(string, name, len)) {
>> 157         return string_object(l);
>> 158       }
>> 
>> Can we please add a comment here, saying that returning "string" would be very bad, or maybe even fold this a bit, so that no one will be tempted to ever return that value, something like:
>> 
>> if (java_lang_String::equals(string_object_no_keepalive(l), name, len)) {
>>    // Comment saying why we must call string_object() here...
>>    return string_object(l);
>> }
> 
> Fixed.
> 
> Thanks,
> /Erik
> 
>> cheers,
>> Per
>> 
>>> 
>>> Bug:
>>> https://bugs.openjdk.java.net/browse/JDK-8192003
>>> 
>>> Thanks,
>>> /Erik
> 



From erik.osterlund at oracle.com  Thu Nov 30 16:17:44 2017
From: erik.osterlund at oracle.com (Erik Osterlund)
Date: Thu, 30 Nov 2017 17:17:44 +0100
Subject: RFR (S): 8192003: Refactor weak references in StringTable to use
 the Access API
In-Reply-To: <C14AEF77-BC36-4076-8C5E-BC7A76B3A647@oracle.com>
References: <5A1D93C1.3020605@oracle.com>
 <81e2ffcb-4912-9fc7-55d9-0553b737a695@oracle.com>
 <5A200B40.9060206@oracle.com>
 <C14AEF77-BC36-4076-8C5E-BC7A76B3A647@oracle.com>
Message-ID: <39525A98-9025-417F-8EFE-B5FF6EE3180A@oracle.com>

Hi Per,

Thanks for the review.

/Erik

> On 30 Nov 2017, at 16:43, Per Lid?n <per.liden at oracle.com> wrote:
> 
> Looks good!
> 
> /Per
> 
>> On 30 Nov 2017, at 14:44, Erik ?sterlund <erik.osterlund at oracle.com> wrote:
>> 
>> Hi Per,
>> 
>> Thank you for reviewing this.
>> 
>> New full webrev:
>> http://cr.openjdk.java.net/~eosterlund/8192003/webrev.01/
>> 
>> New incremental webrev:
>> http://cr.openjdk.java.net/~eosterlund/8192003/webrev.00_01/
>> 
>>> On 2017-11-30 11:32, Per Liden wrote:
>>> Hi Erik,
>>> 
>>>> On 2017-11-28 17:50, Erik ?sterlund wrote:
>>>> Hi,
>>>> 
>>>> The StringTable contains weak references to oops. Today the weak
>>>> semantics is managed using explicit calls to G1 SATB enqueue barriers.
>>>> 
>>>> Now that the Access API is available, it should be used instead as it is
>>>> more modular.
>>>> 
>>>> This change fixes that by making these oops ON_PHANTOM_OOP_REF with a
>>>> corresponding AS_NO_KEEPALIVE accessor when we do not want to keep it
>>>> alive, much like my previous changes to other weak tables.
>>>> 
>>>> Webrev:
>>>> http://cr.openjdk.java.net/~eosterlund/8192003/webrev.00/
>>> 
>>> share/classfile/javaClasses.inline.hpp
>>> --------------------------------------
>>> 
>>> 57 typeArrayOop java_lang_String::value_no_keepalive(oop java_string) {
>>> 58   assert(initialized && (value_offset > 0), "Must be initialized");
>>> 59   assert(is_instance(java_string), "must be java_string");
>>> 60   oop value = HeapAccess<AS_NO_KEEPALIVE>::oop_load_at(java_string, value_offset);
>>> 61   return (typeArrayOop)value;
>>> 62 }
>>> 
>>> How about pushing this barrier down to oopDesc, with a oopDesc::obj_field_special_access<DecoratorSet D>(...) function?
>> 
>> Sounds doable. Fixed. (Although I called the method just "obj_field_special", hope nobody minds...)
>> 
>>> 
>>> 76   typeArrayOop value = java_lang_String::value_no_keepalive(java_string);
>>> 77   assert(initialized, "Must be initialized");
>>> 78   assert(is_instance(java_string), "must be java_string");
>>> 
>>> Looks like you accidentally moved the value_no_keepalive() call above the asserts?
>> 
>> Fixed.
>> 
>>> 
>>> share/classfile/stringTable.cpp
>>> -------------------------------
>>> 
>>> 155       oop string = string_object_no_keepalive(l);
>>> 156       if (java_lang_String::equals(string, name, len)) {
>>> 157         return string_object(l);
>>> 158       }
>>> 
>>> Can we please add a comment here, saying that returning "string" would be very bad, or maybe even fold this a bit, so that no one will be tempted to ever return that value, something like:
>>> 
>>> if (java_lang_String::equals(string_object_no_keepalive(l), name, len)) {
>>>   // Comment saying why we must call string_object() here...
>>>   return string_object(l);
>>> }
>> 
>> Fixed.
>> 
>> Thanks,
>> /Erik
>> 
>>> cheers,
>>> Per
>>> 
>>>> 
>>>> Bug:
>>>> https://bugs.openjdk.java.net/browse/JDK-8192003
>>>> 
>>>> Thanks,
>>>> /Erik
>> 
> 



From sangheon.kim at oracle.com  Thu Nov 30 17:52:19 2017
From: sangheon.kim at oracle.com (sangheon.kim)
Date: Thu, 30 Nov 2017 09:52:19 -0800
Subject: RFR: 8191821: Finer granularity for GC verification
In-Reply-To: <8883ff1b-de5d-e44d-f554-696016535011@oracle.com>
References: <2319511b-8979-92b9-34f2-6a4c1af46ebf@oracle.com>
 <1511954183.2828.11.camel@oracle.com>
 <8883ff1b-de5d-e44d-f554-696016535011@oracle.com>
Message-ID: <850071cd-ae61-5165-f273-1342b06f1bb5@oracle.com>

Hi Stefan,

On 11/29/2017 07:43 AM, Stefan Johansson wrote:
> Thanks for reviewing Thomas,
>
> Updated webrevs:
> Inc:? http://cr.openjdk.java.net/~sjohanss/8191821/01-02/
> Full: http://cr.openjdk.java.net/~sjohanss/8191821/02/
02 version looks good.

Thanks,
Sangheon


>
> Comments inline.
>
> On 2017-11-29 12:16, Thomas Schatzl wrote:
>> Hi,
>>
>> On Tue, 2017-11-28 at 17:25 +0100, Stefan Johansson wrote:
>>> Hi,
>>>
>>> Please review this change for enhancement:
>>> https://bugs.openjdk.java.net/browse/JDK-8191821
>>>
>>> Webrev:
>>> http://cr.openjdk.java.net/~sjohanss/8191821/00/
>>>
>>> Summary:
>>> Heap verification is a very good way to track down GC bugs, but it
>>> comes
>>> with a lot of overhead. Using VerifyBeforeGC, VerifyDuringGC and
>>> VerifyAfterGC often slows down the execution more than is needed
>>> since
>>> we sometimes only want to verify certain types of GCs. This change
>>> adds
>>> this feature for G1 by adding a new diagnostic flag VerifyGCType.
>>> The
>>> new flag currently only works with G1 but can easily be added for
>>> more
>>> GCs if needed. The type of the flag is ccstrlist which means the
>>> option
>>> can be used multiple times to allow more than one type to be
>>> verified.
>>> The types available for G1 is, young, mixed, remark, cleanup and
>>> full.
>>> If the flag is not specified all GCs are verified.
>>>
>>> Note that Verify/Before/After/During/GC is still needed to decide
>>> what
>>> to verify, VerifyGCType only describes when.
>>>
>>> Testing:
>>> * Added new Gtest for G1HeapVerifier functionality.
>>> * Added new Jtreg test for basic command line functionality.
>>> * Executed Jtreg tests through mach5 to make sure it passes on all
>>> platforms.
>>>
>> ?? - I would like to see a special verification type for initial mark
>> gcs, since these are part of the concurrent cycle. I mean, the change
>> does not allow to verify the whole concurrent cycle alone without
>> enabling verification for *all* young-only gcs.
> Fixed.
>>
>> ?? - please change G1VerifyYoung to G1VerifyYoungOnly. Mixed gcs are
>> also young gcs. We should be exact in our internal naming.
> Fixed.
>>
>> ?? - in g1HeapVerifier.hpp consider aligning the flag values below each
>> other.
> Fixed.
>>
>> ?? - gcArguments.cpp:89: the warning message should print the given
>> type. Not in "VerifyGCType %s", as that would be confusing, potentially
>> only indicating that "type" is not supported.
> Updated per Poonams request to only be printed once and with this I 
> think I can be left as is.
>>
>> ?? - gcArguments.cpp:109: are these really the only delimiters for
>> arguments. Not sure if e.g. \t is also a delimiter. Isn't there some
>> existing argument processing method available (or at least this
>> constant) elsewhere?
> So I basically used the same that were available for VerifySubSet but 
> also added \n since this is the delimiter added if the flag is used 
> multiple times. I think the expected way to use ccstrlist is to have 
> one value per usage like:
> -XX:VerifyGCType=full -XX:VerifyGCType=young
>
> But since VerifySubSet allows comma separated values I decided to do 
> so as well. I people start using tabs I'm not sure we want to support 
> that. I would then rather just support "\n" and have one value per flag.
>>
>> ?? - gcArguments.hpp:52: maybe some documentation what this is supposed
>> to do, and that the accepted types are GC specific.
> Fixed.
>>
>> ?? - globals.hpp:2276: the list of available types is not complete
>> (remark, cleanup?). Also there is no indication that the supported
>> types are collector specific. Or just leave them out here, and document
>> the available strings in the collector's parsing code header file (in
>> the enum?)
> Good point, will remove this listing and update the text.
>>
>> ?? - TestVerifyGCType.java: instead of using a GarbageProducer you can
>> directly trigger specific GCs using whitebox. This would make the tests
>> a lot more specific imho.
> Agreed and fixed. Also added test for mixed and initial-mark now.
>
> Thanks,
> Stefan
>>
>> Thanks,
>> ?? Thomas
>



From martin.doerr at sap.com  Thu Nov 30 18:08:57 2017
From: martin.doerr at sap.com (Doerr, Martin)
Date: Thu, 30 Nov 2017 18:08:57 +0000
Subject: RFR(M): 8190308: Supporting heap allocation on alternative memory
 devices and CSR review
In-Reply-To: <F89640DCD01A85489FCBA68183A6A0F3BCD54C80@ORSMSX116.amr.corp.intel.com>
References: <F89640DCD01A85489FCBA68183A6A0F3BCD4BDC5@ORSMSX116.amr.corp.intel.com>
 <a31f2457-149b-f5fb-71c3-b0595ddfa87a@oracle.com>
 <F89640DCD01A85489FCBA68183A6A0F3BCD4FCFF@ORSMSX116.amr.corp.intel.com>
 <1509719477.3207.10.camel@oracle.com>
 <F89640DCD01A85489FCBA68183A6A0F3BCD4FF76@ORSMSX116.amr.corp.intel.com>
 <9b259cdd-111a-0634-80a1-0c6ba1a8d260@oracle.com>
 <F89640DCD01A85489FCBA68183A6A0F3BCD5008C@ORSMSX116.amr.corp.intel.com>
 <4171dc3d-3a43-a455-0efb-ee5ff5640b93@oracle.com>
 <F89640DCD01A85489FCBA68183A6A0F3BCD54AC2@ORSMSX116.amr.corp.intel.com>
 <1510605610.7132.3.camel@oracle.com>
 <3e6131cd-bac3-c8db-971f-297bb13b087d@oracle.com>
 <F89640DCD01A85489FCBA68183A6A0F3BCD54C80@ORSMSX116.amr.corp.intel.com>
Message-ID: <b4390cfb2ce845dd8f939c1aeeca1899@sap.com>

Hi,

I just noticed that "MAP_NORESERVE" is not defined on AIX (os_posix.cpp reserve_mmapped_memory).
I guess it's not POSIX, so this looks like a bug.
Would it make sense to replace it by "NOT_AIX( | MAP_NORESERVE )"? Or is there a better idea?

Best regards,
Martin


-----Original Message-----
From: hotspot-gc-dev [mailto:hotspot-gc-dev-bounces at openjdk.java.net] On Behalf Of Kharbas, Kishor
Sent: Montag, 13. November 2017 22:33
To: sangheon.kim <sangheon.kim at oracle.com>; Thomas Schatzl <thomas.schatzl at oracle.com>; 'hotspot-gc-dev at openjdk.java.net' <hotspot-gc-dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net
Cc: Kharbas, Kishor <kishor.kharbas at intel.com>
Subject: RE: RFR(M): 8190308: Supporting heap allocation on alternative memory devices and CSR review

Thanks Sangheon and Thomas!

-----Original Message-----
From: sangheon.kim [mailto:sangheon.kim at oracle.com] 
Sent: Monday, November 13, 2017 12:41 PM
To: Thomas Schatzl <thomas.schatzl at oracle.com>; Kharbas, Kishor <kishor.kharbas at intel.com>; 'hotspot-gc-dev at openjdk.java.net' <hotspot-gc-dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net
Subject: Re: RFR(M): 8190308: Supporting heap allocation on alternative memory devices and CSR review

Hi Kishor,

On 11/13/2017 12:40 PM, Thomas Schatzl wrote:
> Hi Kishor,
>
> On Mon, 2017-11-13 at 19:40 +0000, Kharbas, Kishor wrote:
>> Greetings,
>>   
>> I have an updated webrev to remove compilation warning on Windows 32- 
>> bit.
>> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.15/
>> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.15_to_14/
>>   
>> Sorry missed this earlier. I request for a review on this update.
>    looks good. The other changes from webrev.13 on also look good.
+1

Thanks,
Sangheon


>
> Thanks,
>    Thomas
>
>>   
>> Thanks
>> Kishor
>>   
>> From: sangheon.kim [mailto:sangheon.kim at oracle.com]
>> Sent: Friday, November 3, 2017 4:07 PM
>> To: Kharbas, Kishor <kishor.kharbas at intel.com>; Thomas Schatzl <thoma 
>> s.schatzl at oracle.com>; 'hotspot-gc-dev at openjdk.java.net' <hotspot-gc- 
>> dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net
>> Subject: Re: RFR(M): 8190308: Supporting heap allocation on 
>> alternative memory devices and CSR review
>>   
>> Hi Kishor,
>>
>>
>> On 11/03/2017 02:59 PM, Kharbas, Kishor wrote:
>> Hi Sangheon,
>>   
>> Here is link to the updated webrev-
>> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.14/
>> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.14_to_13/
>> Looks good to me.
>>
>> Thanks,
>> Sangheon
>>
>>
>>
>>   
>> Thanks
>> Kishor
>>   
>> From: sangheon.kim [mailto:sangheon.kim at oracle.com]
>> Sent: Friday, November 3, 2017 2:38 PM
>> To: Kharbas, Kishor <kishor.kharbas at intel.com>; Thomas Schatzl <thoma 
>> s.schatzl at oracle.com>; 'hotspot-gc-dev at openjdk.java.net' <hotspot-gc- 
>> dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net
>> Subject: Re: RFR(M): 8190308: Supporting heap allocation on 
>> alternative memory devices and CSR review
>>   
>> Hi Kishor,
>>
>> On 11/03/2017 11:39 AM, Kharbas, Kishor wrote:
>> Thanks a lot!
>>   
>> Link to updated webrevs -
>> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.13/
>> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.13_to_12/
>> Thank you for fixing all.
>> Looks good to me except below.
>>
>> Could you update the copyright format in TestAllocateHeapAt.java?
>> 2? * Copyright (c) 2017 Oracle and/or its affiliates. All rights 
>> reserved.
>> - Missing comma: * Copyright (c) 2017, Oracle and/or its affiliates.
>> All rights reserved.
>>
>> Thanks,
>> Sangheon
>>
>>
>>
>>
>>   
>>   
>> @Sangheon: Please let me know if you see any corrections needed.
>>   
>> -Kishor
>>   
>> -----Original Message-----
>> From: Thomas Schatzl [mailto:thomas.schatzl at oracle.com]
>> Sent: Friday, November 3, 2017 7:31 AM
>> To: Kharbas, Kishor <kishor.kharbas at intel.com>; sangheon.kim 
>> <sangheon.kim at oracle.com>; 'hotspot-gc-dev at openjdk.java.net'
>> <hotspot-gc-dev at openjdk.java.net>; hotspot-runtime- 
>> dev at openjdk.java.net
>> Subject: Re: RFR(M): 8190308: Supporting heap allocation on 
>> alternative memory devices and CSR review
>>   
>> Hi,
>>   
>> On Fri, 2017-11-03 at 08:55 +0000, Kharbas, Kishor wrote:
>> Hi Sangheon,
>>   
>> Thanks for the review and comments. Here is an updated webrev-
>> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.12
>> http://cr.openjdk.java.net/~kkharbas/8190308/webrev.12_to_11
>>   
>> In addition to your suggested corrections, I added code to set Linux 
>> core dump filter ensuring Heap is dumped correctly when this feature 
>> is used. This is follow-up to Jini George?s comment
>> (http://openjdk.5641.n7.nabble.com/RFR-M-8171181-Supporting-heap-
>> allocation-on-alternative-memory-devices-td300109.html#a300450).
>>   
>> Some minor nits:
>>   
>>  ?- os_posix.cpp:300: please move the else next to the brace
>>  ?- arguments.cpp:4624: please add a space between "if" and the 
>> bracket
>>   
>> I do not need to see a new webrev for these changes. Looks good.
>>   
>> Thanks,
>>  ? Thomas
>>   
>>   
>>   


From mandy.chung at oracle.com  Thu Nov 30 18:12:01 2017
From: mandy.chung at oracle.com (mandy chung)
Date: Thu, 30 Nov 2017 10:12:01 -0800
Subject: RFR: 8191564: Refactor GC related servicability code into GC
 specific subclasses
In-Reply-To: <243153d8-1836-9d3e-b242-52e17b2e2134@redhat.com>
References: <387d2b93-cb8b-51be-a6e0-3b5f9d6cbb1c@redhat.com>
 <713325a4-3ed8-cd7d-c66d-a1c1767b6d3a@redhat.com>
 <2a7e6307-8c0d-ad4b-e50c-a7e1c2311080@redhat.com>
 <56567db1-caf6-ed05-ed12-336edfca0c75@redhat.com> <5A1C3C07.30800@oracle.com>
 <b32b0580-8700-ec0f-bfb1-18a232e4d364@redhat.com>
 <5A1D8BE7.4070008@oracle.com>
 <a541a13e-ff82-b73a-b841-daa2ab910e88@redhat.com>
 <caac7013-f85b-b4e2-73b9-b81a7a57f704@oracle.com>
 <d89cb145-b197-feb6-89c2-a63a4d7a74eb@redhat.com>
 <4b899587-f7b4-82d7-4917-41d202e26ef4@oracle.com>
 <6f47a8ab-8b7e-c19c-2e49-10664ca90dd9@redhat.com>
 <c2282a5d-906b-408e-5c72-38927bc4de56@oracle.com>
 <2a0f294a-b5d8-b2a8-971d-b1aa6146db2d@redhat.com>
 <5c51ba9c-f64c-b8a9-7e46-1edeffb414ba@oracle.com>
 <243153d8-1836-9d3e-b242-52e17b2e2134@redhat.com>
Message-ID: <e639fe63-ff6e-809f-7007-42019126ece0@oracle.com>



On 11/30/17 6:22 AM, Roman Kennke wrote:
> Hi David,
>
> I added the tag as you proposed:
>
> Differential:
> http://cr.openjdk.java.net/~rkennke/8191564/webrev.08.diff/
> Full:
> http://cr.openjdk.java.net/~rkennke/8191564/webrev.08/
>

This looks really good.? This version looks much better than an early 
version I looked at (thanks for regenerating webrev with hg rename).

A minor comment that GCMemoryManager could take an enum to indicate the 
type of this? GC action (major vs minor).?? This can be a future cleanup.

thanks
Mandy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20171130/22958f9c/attachment.htm>

From rkennke at redhat.com  Thu Nov 30 18:26:57 2017
From: rkennke at redhat.com (Roman Kennke)
Date: Thu, 30 Nov 2017 19:26:57 +0100
Subject: RFR: 8191564: Refactor GC related servicability code into GC
 specific subclasses
In-Reply-To: <e639fe63-ff6e-809f-7007-42019126ece0@oracle.com>
References: <387d2b93-cb8b-51be-a6e0-3b5f9d6cbb1c@redhat.com>
 <713325a4-3ed8-cd7d-c66d-a1c1767b6d3a@redhat.com>
 <2a7e6307-8c0d-ad4b-e50c-a7e1c2311080@redhat.com>
 <56567db1-caf6-ed05-ed12-336edfca0c75@redhat.com> <5A1C3C07.30800@oracle.com>
 <b32b0580-8700-ec0f-bfb1-18a232e4d364@redhat.com>
 <5A1D8BE7.4070008@oracle.com>
 <a541a13e-ff82-b73a-b841-daa2ab910e88@redhat.com>
 <caac7013-f85b-b4e2-73b9-b81a7a57f704@oracle.com>
 <d89cb145-b197-feb6-89c2-a63a4d7a74eb@redhat.com>
 <4b899587-f7b4-82d7-4917-41d202e26ef4@oracle.com>
 <6f47a8ab-8b7e-c19c-2e49-10664ca90dd9@redhat.com>
 <c2282a5d-906b-408e-5c72-38927bc4de56@oracle.com>
 <2a0f294a-b5d8-b2a8-971d-b1aa6146db2d@redhat.com>
 <5c51ba9c-f64c-b8a9-7e46-1edeffb414ba@oracle.com>
 <243153d8-1836-9d3e-b242-52e17b2e2134@redhat.com>
 <e639fe63-ff6e-809f-7007-42019126ece0@oracle.com>
Message-ID: <2da42a09-73d7-6f3e-b027-337ff6ff29a2@redhat.com>

Hi Mandy,

thanks for reviewing! I think Erik ?. already pushed it for me.

> A minor comment that GCMemoryManager could take an enum to indicate the 
> type of this? GC action (major vs minor).?? This can be a future cleanup.

This may be overly restrictive: some GCs may not necessarily provide a 
major/minor distinction, but have only one GC manager (e.g. epsilon), or 
may provide 3 or more GC managers (e.g. 'minor' partial (or young) GC, 
'intermediate' concurrent GC, 'intermediate or major' concurrent full 
GC, 'major' STW last-ditch full GC...).


Roman


From mandy.chung at oracle.com  Thu Nov 30 18:41:58 2017
From: mandy.chung at oracle.com (mandy chung)
Date: Thu, 30 Nov 2017 10:41:58 -0800
Subject: RFR: 8191564: Refactor GC related servicability code into GC
 specific subclasses
In-Reply-To: <2da42a09-73d7-6f3e-b027-337ff6ff29a2@redhat.com>
References: <387d2b93-cb8b-51be-a6e0-3b5f9d6cbb1c@redhat.com>
 <713325a4-3ed8-cd7d-c66d-a1c1767b6d3a@redhat.com>
 <2a7e6307-8c0d-ad4b-e50c-a7e1c2311080@redhat.com>
 <56567db1-caf6-ed05-ed12-336edfca0c75@redhat.com> <5A1C3C07.30800@oracle.com>
 <b32b0580-8700-ec0f-bfb1-18a232e4d364@redhat.com>
 <5A1D8BE7.4070008@oracle.com>
 <a541a13e-ff82-b73a-b841-daa2ab910e88@redhat.com>
 <caac7013-f85b-b4e2-73b9-b81a7a57f704@oracle.com>
 <d89cb145-b197-feb6-89c2-a63a4d7a74eb@redhat.com>
 <4b899587-f7b4-82d7-4917-41d202e26ef4@oracle.com>
 <6f47a8ab-8b7e-c19c-2e49-10664ca90dd9@redhat.com>
 <c2282a5d-906b-408e-5c72-38927bc4de56@oracle.com>
 <2a0f294a-b5d8-b2a8-971d-b1aa6146db2d@redhat.com>
 <5c51ba9c-f64c-b8a9-7e46-1edeffb414ba@oracle.com>
 <243153d8-1836-9d3e-b242-52e17b2e2134@redhat.com>
 <e639fe63-ff6e-809f-7007-42019126ece0@oracle.com>
 <2da42a09-73d7-6f3e-b027-337ff6ff29a2@redhat.com>
Message-ID: <67b97a9b-ccd0-3232-ae4c-2a24dc9d605f@oracle.com>



On 11/30/17 10:26 AM, Roman Kennke wrote:
> Hi Mandy,
>
> thanks for reviewing! I think Erik ?. already pushed it for me.
>
>> A minor comment that GCMemoryManager could take an enum to indicate 
>> the type of this? GC action (major vs minor).?? This can be a future 
>> cleanup.
>
> This may be overly restrictive: some GCs may not necessarily provide a 
> major/minor distinction, but have only one GC manager (e.g. epsilon), 
> or may provide 3 or more GC managers (e.g. 'minor' partial (or young) 
> GC, 'intermediate' concurrent GC, 'intermediate or major' concurrent 
> full GC, 'major' STW last-ditch full GC...).

Right - there will be more than 2 enums and major and major are just 
examples that could let GC notififer to determine the message based on 
the type.? This is minor and just a thought that can be considered in 
the future.

Thanks.
Mandy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20171130/94748011/attachment.htm>

From manc at google.com  Thu Nov 30 22:03:59 2017
From: manc at google.com (Man Cao)
Date: Thu, 30 Nov 2017 14:03:59 -0800
Subject: Make reserved_size for compressed class space and metaspace
 respect the ergo-initialized CompressedClassSpaceSize flag value
In-Reply-To: <CA+w6HxarhJmwyV2V6ahy_KdpYO+Db+qH6QhbHm6U6BmL7RzZbA@mail.gmail.com>
References: <CA+w6HxarhJmwyV2V6ahy_KdpYO+Db+qH6QhbHm6U6BmL7RzZbA@mail.gmail.com>
Message-ID: <CA+w6Hxa6RAcp1D4waqeCzz9VVhT=e5qfOe2qhg-huhh-zdj7bA@mail.gmail.com>

I realized that the email attachment is probably dropped by the mailing
list, so below is the inlined patch.

--- old/src/hotspot/share/memory/metaspace.cpp 2017-11-29
14:56:59.017118444 -0800
+++ new/src/hotspot/share/memory/metaspace.cpp 2017-11-29
14:56:58.657121375 -0800
@@ -3321,9 +3321,6 @@
   MinMetaspaceExpansion = align_down_bounded(MinMetaspaceExpansion,
_commit_alignment);
   MaxMetaspaceExpansion = align_down_bounded(MaxMetaspaceExpansion,
_commit_alignment);

-  CompressedClassSpaceSize = align_down_bounded(CompressedClassSpaceSize,
_reserve_alignment);
-  set_compressed_class_space_size(CompressedClassSpaceSize);
-
   // Initial virtual space size will be calculated at global_initialize()
   size_t min_metaspace_sz =
       VIRTUALSPACEMULTIPLIER * InitialBootClassLoaderMetaspaceSize;
@@ -3341,6 +3338,8 @@
                   min_metaspace_sz);
   }

+  CompressedClassSpaceSize = align_down_bounded(CompressedClassSpaceSize,
_reserve_alignment);
+  set_compressed_class_space_size(CompressedClassSpaceSize);
 }

 void Metaspace::global_initialize() {

Best,
Man

On Wed, Nov 29, 2017 at 3:21 PM, Man Cao <manc at google.com> wrote:

> Hello,
>
> This patch is a follow-up fix for https://bugs.openjdk.java.
> net/browse/JDK-8087291
>
> This patch moves the call to set_compressed_class_space_size() after the
> flag value for CompressedClassSpaceSize is ergo-initialized, fixing the
> issue that the reserved size for compressed class space and metaspace is
> excessively large when MaxMetaspaceSize is set to a small value. More
> discussion about it is available here:
> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/
> 2017-November/025200.html
>
> This code patch is attached. Could anyone review and/or sponsor this patch?
>
> Best,
> Man
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20171130/383a450d/attachment.htm>