From serguei.spitsyn at oracle.com  Thu Jun  1 09:08:31 2017
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Thu, 1 Jun 2017 02:08:31 -0700
Subject: Possible race in jdwp invoke handling?
In-Reply-To: <CAA-vtUxoZN+7cH6csKiP8neOJun7PfgE2bJpO78d+HScWV3bVw@mail.gmail.com>
References: <CAA-vtUxoZN+7cH6csKiP8neOJun7PfgE2bJpO78d+HScWV3bVw@mail.gmail.com>
Message-ID: <c43483a0-323a-dd67-e654-b2019251c691@oracle.com>

Hi Thomas,

I'll look at it a couple of days later as I'm busy with other stuff now.

Thanks,
Serguei


On 5/31/17 07:42, Thomas St?fe wrote:
> Hi all,
>
> I am looking at a possible race in JDWP invoke request handling and 
> would like your opinion.
>
> This is how I understand the handling of invoke events (please do 
> correct me if I am wrong):
>
> 1) JDWP InvokeXXX request comes in for thread X. Handled by the "JDWP 
> Transport Listener" thread. We call "invoker_requestInvoke()". Here, 
> under lock protection, we take the thread-local InvokeRequest 
> structure and fill it with the invoke data. We only do this if 
> request->available is true. We set request->available to false, 
> thereby preventing further attempts to add invoke requests for this 
> thread. Any subsequent JDWP invoke commands will now return with 
> errors, right?
>
> 2) In the context of a JVMTI callback for thread X the actual invoke 
> will be done. Request structure will be filled with the result 
> (exception object handle and result). request->available stays false.
>
> 3) In a third thread, the "JDWP Event Helper Thread", the return 
> packet will be sent to the debugger. In 
> invoker_completeInvokeRequest(), we have two guarded sections. In the 
> first section, we reset request->available to true (A):
>
>     eventHandler_lock(); /* for proper lock order */
>     debugMonitorEnter(invokerLock);
>
>     request = threadControl_getInvokeRequest(thread);
>     ....<skip>...
>
>     request->pending = JNI_FALSE;
>     request->started = JNI_FALSE;
> A) request->available = JNI_TRUE; /* For next time around */
>
>     ...<skip>...
>
>     debugMonitorExit(invokerLock);
>     eventHandler_unlock();
>
> Then we leave the guarded section and send the jdwp answer packet back.
>
> Then we enter a second guarded section and clean up the handles for 
> return value and exception:
>
> ...
>     eventHandler_lock(); // for proper lock order
>     debugMonitorEnter(invokerLock);
> B)    deletePotentiallySavedGlobalRefs(env, request);
>     debugMonitorExit(invokerLock);
>     eventHandler_unlock();
>
>
> ---
>
> My question is this: would it be possible for a new invoke request to 
> be incoming in the time between the first and the second guarded 
> section? So, could the following sequence happen:
>
>
> [JDWP Transport Listener]         invoker_requestInvoke (request 1)
>
> [Thread X]  invoker_doInvoke  (request 1)
>
> [JDWP Event Helper]  invoker_completeInvokeRequest  (request 1)
> debugMonitorEnter(invokerLock);
> <reset request->available and request->pending>
> debugMonitorExit(invokerLock);
> ....
>
> [JDWP Transport Listener]             invoker_requestInvoke (request 2)
>
> [Thread X]  invoker_doInvoke  (request 2)  -> overwrites 
> request->exception and request->returnValue to its new values.
>
>
>
> [JDWP Event Helper]                     ....
>  debugMonitorEnter(invokerLock);
>  deletePotentiallySavedGlobalRefs(env, request);  -> releases 
> request->exception and request->returnValue, which is now interfering 
> with request 2.
>  debugMonitorExit(invokerLock);
>
>
> ?
>
> This is all theoretical. Wanted to hear your opinions first before 
> proceeding.
>
> Kind Regards, Thomas
>
>
>
>
>
>


From christoph.langer at sap.com  Thu Jun  1 09:39:46 2017
From: christoph.langer at sap.com (Langer, Christoph)
Date: Thu, 1 Jun 2017 09:39:46 +0000
Subject: JDK 10 RFR 8181417: Code cleanups in com.sun.jdi
Message-ID: <a7e6b44d1b4d49c580a65549fe0f98e0@sap.com>

Hi,

during integration work of the jdi library for one of our projects I spotted some places where a few cleanups could be done. I'd like to contribute this. It is mostly about import statements and formatting/empty lines. I think at this stage of JDK10 it's a good time for such kind of work.

Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8181417.0/
Bug: https://bugs.openjdk.java.net/browse/JDK-8181417

Thanks and best regards
Christoph

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170601/4ec24e69/attachment.html>

From Alan.Bateman at oracle.com  Thu Jun  1 09:58:59 2017
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Thu, 1 Jun 2017 10:58:59 +0100
Subject: JDK 10 RFR 8181417: Code cleanups in com.sun.jdi
In-Reply-To: <a7e6b44d1b4d49c580a65549fe0f98e0@sap.com>
References: <a7e6b44d1b4d49c580a65549fe0f98e0@sap.com>
Message-ID: <780e8ebf-539d-a8fc-0e2a-6f04367017c0@oracle.com>

On 01/06/2017 10:39, Langer, Christoph wrote:
>
> Hi,
>
> during integration work of the jdi library for one of our projects I 
> spotted some places where a few cleanups could be done. I?d like to 
> contribute this. It is mostly about import statements and 
> formatting/empty lines. I think at this stage of JDK10 it?s a good 
> time for such kind of work.
>
> Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8181417.0/ 
> <http://cr.openjdk.java.net/%7Eclanger/webrevs/8181417.0/>
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8181417 
> <https://bugs.openjdk.java.net/browse/JDK-8181417>
>
>
serviceability-dev is the right place so dropping core-libs-dev.

In Transport, maybe Connection too, then the class description looks 
very messy now so I think that needs to reformatted so that lines are 
roughly of equal length.

The reformatting and adding/removing blank lines looks odd in places but 
not worth discussing.

-Alan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170601/838c9762/attachment-0001.html>

From ujwal.vangapally at oracle.com  Thu Jun  1 11:25:53 2017
From: ujwal.vangapally at oracle.com (Ujwal Vangapally)
Date: Thu, 1 Jun 2017 16:55:53 +0530
Subject: RFR: JDK-8178508 Co-locate remaining MM tests
In-Reply-To: <6d1f972d-dbec-7a79-4c1d-a960d8696a18@oracle.com>
References: <6d1f972d-dbec-7a79-4c1d-a960d8696a18@oracle.com>
Message-ID: <a60f4f4a-40b6-fa23-894e-02600d0ce9b7@oracle.com>

Gentle reminder.

Thanks,

Ujwal.


On 5/31/2017 10:32 AM, Ujwal Vangapally wrote:
>
> Kindly review the changes made for below bug
>
> converted tonga test to JTREG test
>
> added an additional assert statement for verifying setusageThreshold() 
> operation is successful
>
> https://bugs.openjdk.java.net/browse/JDK-8178508
>
> tonga test case is currently at this path :
>
> http://sqe-hgi.us.oracle.com/hg/index.cgi/testbase/javase/functional/9/vm/file/tip/src/nsk/regression/b6653214/  
>
>
> webrev :
> http://cr.openjdk.java.net/~uvangapally/webrev/2017/8178508/webrev.00/
>
> Thanks,
>
> Ujwal.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170601/3191eb21/attachment.html>

From thomas.stuefe at gmail.com  Thu Jun  1 12:18:16 2017
From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=)
Date: Thu, 1 Jun 2017 14:18:16 +0200
Subject: RFR(xs): 8181419: Race in jdwp invoker handling may lead to crashes
 or invalid results
Message-ID: <CAA-vtUwt_h6o5LjajKEGJbzD12kdYQ7cJN-41tKNFkNhs9SCug@mail.gmail.com>

Hi all,

please take a look at this proposed fix for a theoretical race in the jdwp
library.

Issue: https://bugs.openjdk.java.net/browse/JDK-8181419
webrev:
http://cr.openjdk.java.net/~stuefe/webrevs/8181419-Race-in-jdwp-invoker-handling-may-lead-to-crashes-or-invalid-results/webrev.00/webrev/

In short, this is an addition to Severin's fix to the jdwp invoke handling (
https://bugs.openjdk.java.net/browse/JDK-8153711).

We have a potential race condition where the delayed cleanup of the saved
returnvalue object reference and the exception reference (released
in deletePotentiallySavedGlobalRefs() ) may be overtaken by a new request
which populates the thread request structure anew. If this happens,
deletePotentiallySavedGlobalRefs() may actually release the return value /
exception references of the follow up request, if that one was already
processed.

The solution I choose is safe and conservative. We still release both
references, but use the locally saved JNI references. We just avoid
accessing the thread local request structure after it has been cleared for
reuse. This keeps timing and locking behaviour unchanged.

I am currently running jtreg tests for com/sun/jdi on AIX and Linux.

Kind Regards, Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170601/15053483/attachment.html>

From christoph.langer at sap.com  Thu Jun  1 12:19:34 2017
From: christoph.langer at sap.com (Langer, Christoph)
Date: Thu, 1 Jun 2017 12:19:34 +0000
Subject: JDK 10 RFR 8181417: Code cleanups in com.sun.jdi
In-Reply-To: <780e8ebf-539d-a8fc-0e2a-6f04367017c0@oracle.com>
References: <a7e6b44d1b4d49c580a65549fe0f98e0@sap.com>
 <780e8ebf-539d-a8fc-0e2a-6f04367017c0@oracle.com>
Message-ID: <14dc677d666148148f0dc5aa19715dcc@sap.com>

Hi Alan,

thanks for your quick turnaround on this.

I updated Transport.java and Connection.java to get the class descriptions more compact: http://cr.openjdk.java.net/~clanger/webrevs/8181417.1/

Let me know where you find the blank lines reformatting odd. I tried to follow the usual layout that after the opening brace of a class a blank line would follow, as well as in between methods and fields one empty line should be placed.

Other than that, can I consider this reviewed?

Thanks
Christoph

From: Alan Bateman [mailto:Alan.Bateman at oracle.com]
Sent: Donnerstag, 1. Juni 2017 11:59
To: Langer, Christoph <christoph.langer at sap.com>; serviceability-dev at openjdk.java.net
Subject: Re: JDK 10 RFR 8181417: Code cleanups in com.sun.jdi

On 01/06/2017 10:39, Langer, Christoph wrote:

Hi,

during integration work of the jdi library for one of our projects I spotted some places where a few cleanups could be done. I'd like to contribute this. It is mostly about import statements and formatting/empty lines. I think at this stage of JDK10 it's a good time for such kind of work.

Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8181417.0/<http://cr.openjdk.java.net/%7Eclanger/webrevs/8181417.0/>
Bug: https://bugs.openjdk.java.net/browse/JDK-8181417

serviceability-dev is the right place so dropping core-libs-dev.

In Transport, maybe Connection too, then the class description looks very messy now so I think that needs to reformatted so that lines are roughly of equal length.

The reformatting and adding/removing blank lines looks odd in places but not worth discussing.

-Alan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170601/4ebf5374/attachment.html>

From thomas.stuefe at gmail.com  Thu Jun  1 12:37:33 2017
From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=)
Date: Thu, 1 Jun 2017 14:37:33 +0200
Subject: JDK 10 RFR 8181417: Code cleanups in com.sun.jdi
In-Reply-To: <a7e6b44d1b4d49c580a65549fe0f98e0@sap.com>
References: <a7e6b44d1b4d49c580a65549fe0f98e0@sap.com>
Message-ID: <CAA-vtUzQr6c8No4Ukbf7_P9Vcn1bbLkN0HDjCCH_Cuc=YWZMHg@mail.gmail.com>

Hi Christoph, l

Copyright years need adjusting in a large number of files. Otherwise looks
ok.

..Thomas

On Thu, Jun 1, 2017 at 11:39 AM, Langer, Christoph <christoph.langer at sap.com
> wrote:

> Hi,
>
> during integration work of the jdi library for one of our projects I
> spotted some places where a few cleanups could be done. I'd like to
> contribute this. It is mostly about import statements and formatting/empty
> lines. I think at this stage of JDK10 it's a good time for such kind of
> work.
>
> Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8181417.0/
> Bug: https://bugs.openjdk.java.net/browse/JDK-8181417
>
> Thanks and best regards
> Christoph
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170601/aea0243c/attachment.html>

From sharath.ballal at oracle.com  Thu Jun  1 15:21:00 2017
From: sharath.ballal at oracle.com (Sharath Ballal)
Date: Thu, 1 Jun 2017 08:21:00 -0700 (PDT)
Subject: RFR: JDK-6760477 - Update SA to include stack traces in the heap
 dump
In-Reply-To: <2078e8c7-1394-e695-674e-ec072791b48b@oracle.com>
References: <2091a3f7-af89-4c8e-803a-22b4848c8788@default>
 <2078e8c7-1394-e695-674e-ec072791b48b@oracle.com>
Message-ID: <cfb22450-24d9-45c7-a999-bc5d18df82c5@default>

Thanks Jini.  I have addressed both the comments.

> * Looks like the stackTraces array list is unused. 
Done.  The reason of adding this was for use at other places which may require the stack trace.
> * HeapHprofBinWriter.java: Line 764: It might be better to replace 2 * 4 and similar instances with 2 * INT_SIZE.
Done



 

Thanks & Regards,

Sharath Ballal

 

 

From: Jini George 
Sent: Tuesday, May 30, 2017 2:56 PM
To: Sharath Ballal; serviceability-dev at openjdk.java.net
Subject: Re: RFR: JDK-6760477 - Update SA to include stack traces in the heap dump

 

Hi Sharath,

Looks good. A few nits.

* Looks like the stackTraces array list is unused. 

* HeapHprofBinWriter.java: Line 764: It might be better to replace 2 * 4 and similar instances with 2 * INT_SIZE.

Thanks,
Jini.

On 5/29/2017 4:14 PM, Sharath Ballal wrote:

Hi,

 

Pls review the fix for including stack traces in the heap dump in SA.

 

Bug:    https://bugs.openjdk.java.net/browse/JDK-6760477 

Webrev:  HYPERLINK "http://cr.openjdk.java.net/%7Esballal/6760477/webrev.00/"http://cr.openjdk.java.net/~sballal/6760477/webrev.00/ 

 

 

Thanks & Regards,

Sharath Ballal

 

 

 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170601/4b1964c0/attachment-0001.html>

From christoph.langer at sap.com  Thu Jun  1 15:38:33 2017
From: christoph.langer at sap.com (Langer, Christoph)
Date: Thu, 1 Jun 2017 15:38:33 +0000
Subject: JDK 10 RFR 8181417: Code cleanups in com.sun.jdi
In-Reply-To: <a7e6b44d1b4d49c580a65549fe0f98e0@sap.com>
References: <a7e6b44d1b4d49c580a65549fe0f98e0@sap.com>
Message-ID: <d2e245e424664ba78ad2b22842e61fa4@sap.com>

Hi again,

I updated my webrev to include packages com.sun.jdi.event and com.sun.jdi.request as well. With that I've added everything that I found in com.sun.jdi and its subpackages.

Please look here: http://cr.openjdk.java.net/~clanger/webrevs/8181417.1/

Best regards
Christoph

From: Langer, Christoph
Sent: Donnerstag, 1. Juni 2017 11:40
To: serviceability-dev at openjdk.java.net; core-libs-dev at openjdk.java.net
Subject: JDK 10 RFR 8181417: Code cleanups in com.sun.jdi

Hi,

during integration work of the jdi library for one of our projects I spotted some places where a few cleanups could be done. I'd like to contribute this. It is mostly about import statements and formatting/empty lines. I think at this stage of JDK10 it's a good time for such kind of work.

Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8181417.0/
Bug: https://bugs.openjdk.java.net/browse/JDK-8181417

Thanks and best regards
Christoph

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170601/d5f3baaa/attachment.html>

From david.holmes at oracle.com  Fri Jun  2 01:44:43 2017
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 2 Jun 2017 11:44:43 +1000
Subject: RFR: JDK-8180322 Move JNI spec to specs directory
In-Reply-To: <f42d4796-1ca5-cd3e-8c68-7df4c7ea30c5@oracle.com>
References: <f42d4796-1ca5-cd3e-8c68-7df4c7ea30c5@oracle.com>
Message-ID: <97f865ed-dd01-4851-87d4-0047621da5ea@oracle.com>

Hi Magnus,

Serviceability owns the jvmti.xml file, so cc'd.

The changes to externalink seem okay (I assume the GetEnv -> getenv 
change is not significant?).

The patch file seems to contain a bazillion whitespace changes so you 
might want to check what is going on there.

Thanks,
David

On 1/06/2017 10:46 PM, Magnus Ihse Bursie wrote:
> The JNI spec will move to a new place in the "specs" directory in the 
> docs image, and links to it needs to be updated.
> 
> Bug: https://bugs.openjdk.java.net/browse/JDK-8180322
> WebRev: 
> http://cr.openjdk.java.net/~ihse/JDK-8180322-move-jni-spec/webrev.01
> 
> This is a noreg-docs fix and will be pushed to jdk9.
> 
> /Magnus

From mandy.chung at oracle.com  Fri Jun  2 01:50:34 2017
From: mandy.chung at oracle.com (Mandy Chung)
Date: Thu, 1 Jun 2017 18:50:34 -0700
Subject: RFR: JDK-8180322 Move JNI spec to specs directory
In-Reply-To: <f42d4796-1ca5-cd3e-8c68-7df4c7ea30c5@oracle.com>
References: <f42d4796-1ca5-cd3e-8c68-7df4c7ea30c5@oracle.com>
Message-ID: <D887C3BB-9F2A-4484-B027-D7110C9D2457@oracle.com>


> On Jun 1, 2017, at 5:46 AM, Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com> wrote:
> 
> The JNI spec will move to a new place in the "specs" directory in the docs image, and links to it needs to be updated.
> 
> Bug: https://bugs.openjdk.java.net/browse/JDK-8180322
> WebRev: http://cr.openjdk.java.net/~ihse/JDK-8180322-move-jni-spec/webrev.01
> 
> This is a noreg-docs fix and will be pushed to jdk9.

Looks okay to me.

Mandy

From magnus.ihse.bursie at oracle.com  Fri Jun  2 09:50:18 2017
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Fri, 2 Jun 2017 11:50:18 +0200
Subject: RFR: JDK-8180322 Move JNI spec to specs directory
In-Reply-To: <97f865ed-dd01-4851-87d4-0047621da5ea@oracle.com>
References: <f42d4796-1ca5-cd3e-8c68-7df4c7ea30c5@oracle.com>
 <97f865ed-dd01-4851-87d4-0047621da5ea@oracle.com>
Message-ID: <779DF8A7-C5AA-4237-836A-4737E578BDFA@oracle.com>


> 2 juni 2017 kl. 03:44 skrev David Holmes <david.holmes at oracle.com>:
> 
> Hi Magnus,
> 
> Serviceability owns the jvmti.xml file, so cc'd.
> 
> The changes to externalink seem okay (I assume the GetEnv -> getenv change is not significant?).

It matches the changes in the source document. 

> 
> The patch file seems to contain a bazillion whitespace changes so you might want to check what is going on there.

Those were trailing whitespaces. Unfortunately, jcheck only enforces the "no trailing whitespaces" rule for certain file types (or technically extensions), and .xml is not one of them (and neither is .gmk :-(). I have instructed my editor to remove trailing whitespaces always, and sometimes this means a patch gets lots of whitespace cleanup. 

/Magnus

> 
> Thanks,
> David
> 
>> On 1/06/2017 10:46 PM, Magnus Ihse Bursie wrote:
>> The JNI spec will move to a new place in the "specs" directory in the docs image, and links to it needs to be updated.
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8180322
>> WebRev: http://cr.openjdk.java.net/~ihse/JDK-8180322-move-jni-spec/webrev.01
>> This is a noreg-docs fix and will be pushed to jdk9.
>> /Magnus


From christoph.langer at sap.com  Fri Jun  2 10:11:09 2017
From: christoph.langer at sap.com (Langer, Christoph)
Date: Fri, 2 Jun 2017 10:11:09 +0000
Subject: [XS] JDK-8180413 : avoid accessing NULL in jdk.jdwp.agent
In-Reply-To: <cc1fe434-af00-1ae0-9695-f92b29430a6b@oracle.com>
References: <f930c2163f534b6690ac6cfecb772cb1@sap.com>
 <37ac7b12-1edd-ce65-a20f-404bd3eba75e@oracle.com>
 <444e21a8d38c42338fa5c38093a32e9c@sap.com>
 <520c56a8-4379-e4b0-42ac-acde1939cc7b@oracle.com>
 <c650c04dfe524c3881e3fb5e548fbd88@sap.com>
 <cc1fe434-af00-1ae0-9695-f92b29430a6b@oracle.com>
Message-ID: <15b68799f2fa445094a72cb9c9534029@sap.com>

Hi,

I pushed the fix to jdk10 now: http://hg.openjdk.java.net/jdk10/jdk10/jdk/rev/66e9436dbaf8


Best regards
Christoph

> -----Original Message-----
> From: serguei.spitsyn at oracle.com [mailto:serguei.spitsyn at oracle.com]
> Sent: Montag, 22. Mai 2017 11:40
> To: Baesken, Matthias <matthias.baesken at sap.com>; Alan Bateman
> <Alan.Bateman at oracle.com>; core-libs-dev at openjdk.java.net;
> serviceability-dev at openjdk.java.net
> Cc: Simonis, Volker <volker.simonis at sap.com>; Langer, Christoph
> <christoph.langer at sap.com>
> Subject: Re: [XS] JDK-8180413 : avoid accessing NULL in jdk.jdwp.agent
> 
> Hello Matthias,
> 
> It is great, you are able to push this fix!
> 
> Thanks,
> Serguei
> 
> 
> 
> On 5/22/17 01:49, Baesken, Matthias wrote:
> > Hello Serguei,  the  com/sun/jdi  tests were executed ,  same number of
> passed and errors in a patched and unpatched JDK .
> > We do not need a sponsor because  the change is in JDK not HS .
> >
> > Best regards, Matthias
> >
> >
> > -----Original Message-----
> > From: serguei.spitsyn at oracle.com [mailto:serguei.spitsyn at oracle.com]
> > Sent: Freitag, 19. Mai 2017 02:18
> > To: Baesken, Matthias <matthias.baesken at sap.com>; Alan Bateman
> <Alan.Bateman at oracle.com>; core-libs-dev at openjdk.java.net;
> serviceability-dev at openjdk.java.net
> > Cc: Simonis, Volker <volker.simonis at sap.com>
> > Subject: Re: [XS] JDK-8180413 : avoid accessing NULL in jdk.jdwp.agent
> >
> > Hi Matthias,
> >
> > The fix looks good to me.
> > It must be tested at least with the JTreg com/sun/jdi tests.
> > Do you need a sponsor?
> >
> > Thanks,
> > Serguei
> >
> >
> >
> > On 5/16/17 03:21, Baesken, Matthias wrote:
> >>    Sure, I forward it to  serviceability-dev .
> >>
> >> -----Original Message-----
> >> From: Alan Bateman [mailto:Alan.Bateman at oracle.com]
> >> Sent: Dienstag, 16. Mai 2017 11:51
> >> To: Baesken, Matthias <matthias.baesken at sap.com>; core-libs-
> dev at openjdk.java.net
> >> Cc: Simonis, Volker <volker.simonis at sap.com>
> >> Subject: Re: [XS] JDK-8180413 : avoid accessing NULL in jdk.jdwp.agent
> >>
> >>
> >>
> >> On 16/05/2017 10:04, Baesken, Matthias wrote:
> >>> Hello, could you please review this small change :
> >>>
> >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8180413/
> >>>
> >>> it fixes a number of places in   jdk.jdwp.agent   where in case of an error
> it is attempted to write to NULL .
> >>>
> >>> Bug :  JDK-8180413 : avoid accessing NULL in jdk.jdwp.agent
> >>>
> >>>
> >>> https://bugs.openjdk.java.net/browse/JDK-8180413
> >>>
> >>>
> >> Can you bring this to serviceability-dev as that is the mailing list
> >> where the JDWP agent is maintained?
> >>
> >> -Alan


From Alan.Bateman at oracle.com  Fri Jun  2 12:06:27 2017
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Fri, 2 Jun 2017 13:06:27 +0100
Subject: JDK 10 RFR 8181417: Code cleanups in com.sun.jdi
In-Reply-To: <d2e245e424664ba78ad2b22842e61fa4@sap.com>
References: <a7e6b44d1b4d49c580a65549fe0f98e0@sap.com>
 <d2e245e424664ba78ad2b22842e61fa4@sap.com>
Message-ID: <f524c5ab-b67b-e269-3fe1-99f0b2a03209@oracle.com>

On 01/06/2017 16:38, Langer, Christoph wrote:
>
> Hi again,
>
> I updated my webrev to include packages com.sun.jdi.event and 
> com.sun.jdi.request as well. With that I?ve added everything that I 
> found in com.sun.jdi and its subpackages.
>
> Please look here: 
> http://cr.openjdk.java.net/~clanger/webrevs/8181417.1/ 
> <http://cr.openjdk.java.net/%7Eclanger/webrevs/8181417.1/>
>
>
Lots of noise with the blank lines but otherwise looks okay to me.

-Alan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170602/44c7b8cd/attachment-0001.html>

From christoph.langer at sap.com  Fri Jun  2 13:04:43 2017
From: christoph.langer at sap.com (Langer, Christoph)
Date: Fri, 2 Jun 2017 13:04:43 +0000
Subject: JDK 10 RFR 8181417: Code cleanups in com.sun.jdi
In-Reply-To: <f524c5ab-b67b-e269-3fe1-99f0b2a03209@oracle.com>
References: <a7e6b44d1b4d49c580a65549fe0f98e0@sap.com>
 <d2e245e424664ba78ad2b22842e61fa4@sap.com>
 <f524c5ab-b67b-e269-3fe1-99f0b2a03209@oracle.com>
Message-ID: <f71bf93d965d4abe91173b775353fee0@sap.com>

Thanks Alan,

I'll probably submit this after my vacation in 2 weeks and will run another jtreg before.

Best regards
Christoph

From: Alan Bateman [mailto:Alan.Bateman at oracle.com]
Sent: Freitag, 2. Juni 2017 14:06
To: Langer, Christoph <christoph.langer at sap.com>; serviceability-dev at openjdk.java.net
Cc: Stuefe, Thomas <thomas.stuefe at sap.com>
Subject: Re: JDK 10 RFR 8181417: Code cleanups in com.sun.jdi

On 01/06/2017 16:38, Langer, Christoph wrote:

Hi again,

I updated my webrev to include packages com.sun.jdi.event and com.sun.jdi.request as well. With that I've added everything that I found in com.sun.jdi and its subpackages.

Please look here: http://cr.openjdk.java.net/~clanger/webrevs/8181417.1/<http://cr.openjdk.java.net/%7Eclanger/webrevs/8181417.1/>

Lots of noise with the blank lines but otherwise looks okay to me.

-Alan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170602/ec1d1209/attachment.html>

From jcbeyler at google.com  Fri Jun  2 15:57:27 2017
From: jcbeyler at google.com (JC Beyler)
Date: Fri, 2 Jun 2017 08:57:27 -0700
Subject: Low-Overhead Heap Profiling
In-Reply-To: <102c59b8-25b6-8c21-8eef-1de7d0bbf629@oracle.com>
References: <CAF9BGBx8xFYudhJ=v1_hU5=nwEcC1X0NKRmC=E-RNN6SMs1ozg@mail.gmail.com>
 <CAF9BGBxj7+C6uy4dXHO-3pNorbpO7n22-9JmevM0F11xOo+MnA@mail.gmail.com>
 <CAF9BGBxrdLVmXRTDHj5hQHJhcyrkBM6+ex1TFWi=A5z=V2FSew@mail.gmail.com>
 <CAF9BGBz81YYVUsMLw9sOMR9fH7JQmyb8d_OZ9aJ47EOAqwWhEw@mail.gmail.com>
 <2af975e6-3827-bd57-0c3d-fadd54867a67@oracle.com>
 <365499b6-3f4d-a4df-9e7e-e72a739fb26b@oracle.com>
 <CAF9BGBy0T7A-+1dejYbr6f0d4s0XR-zv-WwHYqXrbe0EtCdsxQ@mail.gmail.com>
 <fb1155c6-ccf8-843c-34fc-fc4a45fc54cc@oracle.com>
 <CAF9BGBxqmXF9OFD4F9wVb-DQOj_UfNNcBhe+ARVxJ5E4cgCC1Q@mail.gmail.com>
 <102c59b8-25b6-8c21-8eef-1de7d0bbf629@oracle.com>
Message-ID: <CAF9BGBxDaCOxobJi8Uu9kwKPRpnj-KqgWLfJO2z7UX71UnNF4g@mail.gmail.com>

Dear all,

Thanks Robbin for the comments. I have left the MuxLocker for now and will
look at removing it or as a future enhancement as you say. I did make the
class extends and added a TODO for myself to test this in slowdebug.

I have also put a new webrev up that is still a work in progress but should
allow us to talk about TLAB vs C1/C2 modifications.

TLAB implementation: http://cr.openjdk.java.net/~rasbold/8171119/webrev.04/
C1/C2 implementation: http://cr.openjdk.java.net/~rasbold/8171119/webrev.03/


What this webrev has is:
   - I have put in a TLAB implementation, it is a WIP, and I have not yet
done the qualitative/quantitative study on it vs the implementation using
compilation changes but the big parts are in place
       - Caveats:
           - There is a TODO: http://cr.openjdk.java.
net/~rasbold/8171119/webrev.04/src/cpu/x86/vm/macroAssembler_x86.cpp.patch
               - I have not fixed the calculation in the case of a
FastTLABRefill case
           - This is always on right now, there is no way to turn it off,
that's also a TODO to be directed by the JVMTI API

    - I also have circumvented the AsyncGetCallTrace using the snippet of
code you showed Robbin, it works for here/now
          - But we might have to revisit this one day because it then does
not try to get some of the native stacks and jumps directly to the Java
stacks (I see cases where this could be an issue)
          - However, this has cleaned up quite a bit of the code and I have
removed all mention of ASGCT and its structures now and use directly the
JVMTI structures

   - GC is handled now, I have not yet done the qualitative/quantitative
study on it but the big parts are in place

   - Due to the way the TLAB is called, the stack walker is now correct and
produces the right stacks it seems (this is a bold sentence from my ONE
JTREG test :))

Final notes on this webrev:
   - Have to fix that TLAB case for the FastTLABRefill
   - Implement the turn on/off system for the TLAB implementation
   - Have to start looking at the data to see that it is consistent and
does gather the right samples, right frequency, etc.
   - Have to check the GC elements and what that produces
   - Run a slowdebug run and ensure I fixed all those issues you saw Robbin

As always, your comments and feedback are greatly appreciated! Happy Friday!
Jc


On Tue, May 30, 2017 at 5:33 AM, Robbin Ehn <robbin.ehn at oracle.com> wrote:

> Hi Jc,
>
> On 05/22/2017 08:47 PM, JC Beyler wrote:
>
>> Dear all,
>>
>> I have a new webrev up:
>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.03/
>>
>
> I liked this!
>
> Two small things:
>
> heapMonitoring.hpp
> class HeapMonitoring should extend AllStatic
>
> heapMonitoring.cpp
> class MuxLocker should extend StackObj
> But I think you should skip MuxLocker or push it separate generic
> enhancement.
>
> Great with the jtreg test, thanks alot!
>
>
>> This webrev has, I hope, fixed a lot of the comments from Robbin:
>>    - The casts normally are all C++ style
>>    - Moved this to jdk10-hs
>>       - I have not tested slowdebug yet, hopefully it does not break there
>>    - Added the garbage collection system:
>>       - Now live sampled allocations are tracked throughout their lifetime
>>          - When GC happens, it moves the sampled allocation information
>> to two lists: recent and frequent GC lists
>>              - Those lists use the array system that the live objects
>> were using before but have different re-use strategies
>>       - Added the JVMTI API for them via a GetFrequentGarbageTraces and
>> GetGarbageTraces
>> - Both use the same JVMTI structures
>>       - Added the calls to them for the test, though I've kept that test
>> simple for now:
>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.03/raw_fi
>> les/new/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c
>>          - As I write this, I notice my webrev is missing a final change
>> I made to the test that calls the same ReleaseTraces to each
>> live/garbage/frequent structure. This is updated in my local repo and will
>> get in the next webrev.
>>
>> Next steps for this work are:
>>     - Putting the TLAB implementation (yes not forgotten ;-))
>>     - Adding more testing and separate the current test system to check
>> things a bit more thoroughly
>>     - Have not tried to circumvent AsyncGetCallTrace yet
>>     - Still have to double check the stack walker a bit more
>>
>
> Looking forward to this.
>
> Could someone from compiler take a look please?
>
> Thanks!
>
> /Robbin
>
>
>> Happy webrev perusal!
>> Jc
>>
>>
>> On Tue, May 16, 2017 at 5:20 AM, Robbin Ehn <robbin.ehn at oracle.com
>> <mailto:robbin.ehn at oracle.com>> wrote:
>>
>>     Just a few answers,
>>
>>     On 05/15/2017 06:48 PM, JC Beyler wrote:
>>
>>         Dear all,
>>
>>         I've updated the webrev to:
>>         http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/ <
>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/>
>>         <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/ <
>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/>>
>>
>>
>>     I'll look at this later, thanks!
>>
>>
>>         Robbin,
>>         I believe I have addressed most of your items with webrev 02:
>>             - I added a JTreg test to show how it works:
>>         http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/raw_fi
>> les/new/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c
>>         <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/raw_f
>> iles/new/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c>
>>         <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/raw_f
>> iles/new/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c
>>         <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/raw_f
>> iles/new/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c>>
>>            - I've modified the code to use its own data structures both
>> internally and externally, this will make it easier to move out of
>> AsyncGetCallTrace as we move
>>         forward, that is still on my TODOs
>>            - I cleaned up the JVMTI API by passing a structure that
>> handles the num_traces and put in a ReleaseTraces as well
>>            - I cleaned up other issues as well.
>>
>>         However, I have three questions, which are probably because I'm
>> new in this community:
>>            1) My previous webrevs were based off of JDK9 by mistake. When
>> I took JDK10 via : hg clone http://hg.openjdk.java.net/jdk10/jdk10
>>         <http://hg.openjdk.java.net/jdk10/jdk10> <
>> http://hg.openjdk.java.net/jdk10/jdk10 <http://hg.openjdk.java.net/jd
>> k10/jdk10>> jdk10
>>                - I don't see code compatible with what you were showing
>> (ie your patches don't make sense for that code base; ex: klass is still
>> accessed via klass() for
>>         example in collectedHeap.inline.hpp)
>>                - Would you know what is the right hg clone command so we
>> are working on the same code base?
>>
>>
>>     We use jdk10-hs, e.g.
>>     hg tclone http://hg.openjdk.java.net/jdk10/hs <
>> http://hg.openjdk.java.net/jdk10/hs> 10-hs
>>
>>     There is sporadic big merges going from jdk9->jdk10->jdk10-hs and
>> jdk10-hs->jdk10, so 10 is moving...
>>
>>
>>            2) You mentioned I was using os::malloc, new,
>> NEW_C_HEAP_ARRAY; I cleaned out the os::malloc but which of the new vs
>> NEW_C_HEAP_ARRAY should I use. It might be
>>         that I don't understand when one uses one or the other but I see
>> both used around the code base?
>>              - Is it that new is to be used for anything internal and
>> NEW_C_HEAP_ARRAY anything provided to the JVMTI users outside of the JVM?
>>
>>
>>     We overload new operator when you extend correct base class, e.g.
>> CHeapObj<mtInternal> so use 'new'
>>     But for arrays you will need the macro NEW_C_HEAP_ARRAY.
>>
>>
>>            3) Casts: same kind question: which should I use. The code was
>> using a bit of everything, I'll refactor it entirely but I was not clear if
>> I should go to C casts
>>         or C++ casts as I see both in the codebase. What is the
>> convention I should use?
>>
>>
>>     Just be consist, use what suites you, C++ casts might be preferable,
>> if we are moving towards C++11.
>>     And use 'right' cast, e.g. going from Thread* to JavaThread* you
>> should use C cast or static_cast, not reinterpret_cast I would say.
>>
>>
>>         Final notes on this webrev:
>>             - I am still missing:
>>               - Putting a TLAB implementation so that we can compare both
>> webrevs
>>               - Have not tried to circumvent AsyncGetCallTrace
>>               - Putting in the handling of GC'd objects
>>               - Fix a stack walker issue I have seen, I think I know the
>> problem and will test that theory out for the next webrev
>>
>>         I will work on integrating those items for the next webrev!
>>
>>
>>     Thanks!
>>
>>
>>         Thanks for your help,
>>         Jc
>>
>>         Ps:  I tested this on a new repo:
>>
>>         hg clone http://hg.openjdk.java.net/jdk10/jdk10 <
>> http://hg.openjdk.java.net/jdk10/jdk10> <http://hg.openjdk.java.net/jd
>> k10/jdk10
>>         <http://hg.openjdk.java.net/jdk10/jdk10>> jdk10
>>         ... building it
>>         cd test
>>         jtreg -nativepath:<path-to-jdk10>/bu
>> ild/linux-x86_64-normal-server-release/support/test/hotspot/jtreg/native/lib/
>> -jdk
>>         <path-to-jdk10>/linux-x86_64-normal-server-release/images/jdk
>> ../hotspot/test/serviceability/jvmti/HeapMonitor/
>>
>>
>>     I'll test it out!
>>
>>     /Robbin
>>
>>
>>
>>         On Thu, May 4, 2017 at 11:21 PM, serguei.spitsyn at oracle.com
>> <mailto:serguei.spitsyn at oracle.com> <mailto:serguei.spitsyn at oracle.com
>>         <mailto:serguei.spitsyn at oracle.com>> <serguei.spitsyn at oracle.com
>> <mailto:serguei.spitsyn at oracle.com> <mailto:serguei.spitsyn at oracle.com
>>
>>         <mailto:serguei.spitsyn at oracle.com>>> wrote:
>>
>>              Robbin,
>>
>>              Thank you for forwarding!
>>              I will review it.
>>
>>              Thanks,
>>              Serguei
>>
>>
>>
>>              On 5/4/17 02:13, Robbin Ehn wrote:
>>
>>                  Hi,
>>
>>                  To me the compiler changes looks what is expected.
>>                  It would be good if someone from compiler could take a
>> look at that.
>>                  Added compiler to mail thread.
>>
>>                  Also adding Serguei, It would be good with his view also.
>>
>>                  My initial take on it, read through most of the code and
>> took it for a ride.
>>
>>                  ##############################
>>                  - Regarding the compiler changes: I think we need the
>> 'TLAB end' trickery (mentioned by Tony P)
>>                  instead of a separate check for sampling in fast path
>> for the final version.
>>
>>                  ##############################
>>                  - This patch I had to apply to get it compile on JDK 10:
>>
>>                  diff -r ac3ded340b35 src/share/vm/gc/shared/collect
>> edHeap.inline.hpp
>>                  --- a/src/share/vm/gc/shared/collectedHeap.inline.hpp
>>   Fri Apr 28 14:31:38 2017 +0200
>>                  +++ b/src/share/vm/gc/shared/collectedHeap.inline.hpp
>>   Thu May 04 10:22:56 2017 +0200
>>                  @@ -87,3 +87,3 @@
>>                        // support for object alloc event (no-op most of
>> the time)
>>                  -    if (klass() != NULL && klass()->name() != NULL) {
>>                  +    if (klass != NULL && klass->name() != NULL) {
>>                          Thread *base_thread = Thread::current();
>>                  diff -r ac3ded340b35 src/share/vm/runtime/heapMonit
>> oring.cpp
>>                  --- a/src/share/vm/runtime/heapMonitoring.cpp    Fri
>> Apr 28 14:31:38 2017 +0200
>>                  +++ b/src/share/vm/runtime/heapMonitoring.cpp    Thu
>> May 04 10:22:56 2017 +0200
>>                  @@ -316,3 +316,3 @@
>>                      JavaThread *thread = reinterpret_cast<JavaThread
>> *>(Thread::current());
>>                  -  assert(o->size() << LogHeapWordSize == byte_size,
>>                  +  assert(o->size() << LogHeapWordSize ==
>> (long)byte_size,
>>                             "Object size is incorrect.");
>>
>>                  ##############################
>>                  - This patch I had to apply to get it not asserting
>> during slowdebug:
>>
>>                  --- a/src/share/vm/runtime/heapMonitoring.cpp    Fri
>> Apr 28 15:15:16 2017 +0200
>>                  +++ b/src/share/vm/runtime/heapMonitoring.cpp    Thu
>> May 04 10:24:25 2017 +0200
>>                  @@ -32,3 +32,3 @@
>>                    // TODO(jcbeyler): should we make this into a JVMTI
>> structure?
>>                  -struct StackTraceData {
>>                  +struct StackTraceData : CHeapObj<mtInternal> {
>>                      ASGCT_CallTrace *trace;
>>                  @@ -143,3 +143,2 @@
>>                    StackTraceStorage::StackTraceStorage() :
>>                  -    _allocated_traces(new StackTraceData*[MaxHeapTraces]
>> ),
>>                        _allocated_traces_size(MaxHeapTraces),
>>                  @@ -147,2 +146,3 @@
>>                        _allocated_count(0) {
>>                  +  _allocated_traces = NEW_C_HEAP_ARRAY(StackTraceData*,
>> MaxHeapTraces, mtInternal);
>>                      memset(_allocated_traces, 0,
>> sizeof(*_allocated_traces) * MaxHeapTraces);
>>                  @@ -152,3 +152,3 @@
>>                    StackTraceStorage::~StackTraceStorage() {
>>                  -  delete[] _allocated_traces;
>>                  +  FREE_C_HEAP_ARRAY(StackTraceData*,
>> _allocated_traces);
>>                    }
>>
>>                  - Classes should extend correct base class for which
>> type of memory is used for it e.g.: CHeapObj<mt????> or StackObj or
>> AllStatic
>>                  - The style in heapMonitoring.cpp is a bit different
>> from normal vm-style, e.g. using C++ casts instead of C. You mix
>> NEW_C_HEAP_ARRAY, os::malloc and new.
>>                  - In jvmtiHeapTransition.hpp you use C cast instead.
>>
>>                  ##############################
>>                  - This patch I had apply to get traces without setting
>> an ?unrelated? capability
>>                  - Should this not be a new capability?
>>
>>                  diff -r c02a5d8785bf src/share/vm/prims/forte.cpp
>>                  --- a/src/share/vm/prims/forte.cpp    Fri Apr 28
>> 15:15:16 2017 +0200
>>                  +++ b/src/share/vm/prims/forte.cpp    Thu May 04
>> 10:24:25 2017 +0200
>>                  @@ -530,6 +530,6 @@
>>
>>                  -  if (!JvmtiExport::should_post_class_load()) {
>>                  +/*  if (!JvmtiExport::should_post_class_load()) {
>>                        trace->num_frames = ticks_no_class_load; // -1
>>                        return;
>>                  -  }
>>                  +  }*/
>>
>>                  ##############################
>>                  - forte.cpp: (I know this is not part of your changes
>> but)
>>                  find_jmethod_id_or_null give me NULL for my test.
>>                  It looks like we actually want the regular jmethod_id() ?
>>
>>                  Since we are the thread we are talking about (and in
>> same ucontext) and thread is in vm and have a last java frame,
>>                  I think most of the checks done in AsyncGetCallTrace is
>> irrelevant, so you should be-able to call forte_fill_call_trace_given_top
>> directly.
>>                  But since we might need jmethod_id() if possible to
>> avoid getting method id NULL,
>>                  we need some fixes in forte code, or just do the
>> vframStream loop inside heapMonitoring.cpp and not use forte.cpp.
>>
>>                  Something like:
>>
>>                     if (jthread->has_last_Java_frame()) { // just to be
>> safe
>>                       vframeStream vfst(jthread);
>>                       while (!vfst.at_end()) {
>>                         Method* m = vfst.method();
>>                         m->jmethod_id();
>>                         m->line_number_from_bci(vfst.bci());
>>                         vfst.next();
>>                       }
>>
>>                  - This is a bit confusing in forte.cpp,
>> trace->frames[count].lineno = bci.
>>                  Line number should be m->line_number_from_bci(bci);
>>                  Do the heapMonitoring suppose to trace with bci or line
>> number?
>>                  I would say bci, meaning we should either rename
>> ASGCT_CallFrame?lineno or use another data structure which says bci.
>>
>>                  ##############################
>>                  - // TODO(jcbeyler): remove this extra code handling the
>> extra trace for
>>                  Please fix all these TODO's :)
>>
>>                  ##############################
>>                  - heapMonitoring.hpp:
>>                  // TODO(jcbeyler): is this algorithm acceptable in open
>> source?
>>
>>                  Why is this comment here? What is the implication?
>>                  Have you tested any simpler algorithm?
>>
>>                  ##############################
>>                  - Create a sanity jtreg test.
>> (./hotspot/make/test/JtregNative.gmk for building the agent)
>>
>>                  ##############################
>>                  - monitoring_period vs HeapMonitorRate, pick rate or
>> period.
>>
>>                  ##############################
>>                  - globals.hpp
>>                  Why is MaxHeapTraces not settable/overridable from jvmti
>> interface? That would be handy.
>>
>>                  ##############################
>>                  - jvmtiStackTraceData + ASGCT_CallFrame memory
>>                  Are the agent suppose to loop through and free all
>> ASGCT_CallFrame?
>>                  Wouldn't it be better with some kinda protocol, like:
>>                  (*jvmti)->GetLiveTraces(jvmti, &stack_traces,
>> &num_traces);
>>                  (*jvmti)->ReleaseTraces(jvmti, stack_traces,
>> num_traces);
>>
>>                  Also using another data structure that have num_traces
>> inside it simplifies things.
>>                  So I'm not convinced using the async structure is the
>> best way forward.
>>
>>
>>                  I have more questions, but I think it's better if you
>> respond and update the code first.
>>
>>                  Thanks!
>>
>>                  /Robbin
>>
>>
>>                  On 04/21/2017 11:34 PM, JC Beyler wrote:
>>
>>                      Hi all,
>>
>>                      I've added size information to the allocation
>> sampling system. This allows the callback to remember the size of each
>> sampled allocation.
>>         http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/ <
>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/>
>>         <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/ <
>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/>>
>>
>>                      The new webrev.01 also adds the actual heap
>> monitoring sampling system in files:
>>         http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/sh
>> are/vm/runtime/heapMonitoring.cpp.patch
>>         <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/s
>> hare/vm/runtime/heapMonitoring.cpp.patch>
>>                      <http://cr.openjdk.java.net/~
>> rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.cpp.patch
>>         <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/s
>> hare/vm/runtime/heapMonitoring.cpp.patch>>
>>                      and
>>         http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/sh
>> are/vm/runtime/heapMonitoring.hpp.patch
>>         <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/s
>> hare/vm/runtime/heapMonitoring.hpp.patch>
>>                      <http://cr.openjdk.java.net/~
>> rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.hpp.patch
>>         <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/s
>> hare/vm/runtime/heapMonitoring.hpp.patch>>
>>
>>                      My next step is to add the GC part to the webrev,
>> which will allow users to determine what objects are live and what are
>> garbage.
>>
>>                      Thanks for your attention and let me know if there
>> are any questions!
>>
>>                      Have a wonderful Friday!
>>                      Jc
>>
>>                      On Mon, Apr 17, 2017 at 12:37 PM, JC Beyler <
>> jcbeyler at google.com <mailto:jcbeyler at google.com> <mailto:
>> jcbeyler at google.com <mailto:jcbeyler at google.com>>
>>         <mailto:jcbeyler at google.com <mailto:jcbeyler at google.com> <mailto:
>> jcbeyler at google.com <mailto:jcbeyler at google.com>>>> wrote:
>>
>>                           Hi all,
>>
>>                           I worked on getting a few numbers for overhead
>> and accuracy for my feature. I'm unsure if here is the right place to
>> provide the full data, so I
>>         am just
>>                      summarizing
>>                           here for now.
>>
>>                           - Overhead of the feature
>>
>>                           Using the Dacapo benchmark (
>> http://dacapobench.org/). My initial results are that sampling provides
>> 2.4% with a 512k sampling, 512k being our
>>         default setting.
>>
>>                           - Note: this was without the tradesoap,
>> tradebeans and tomcat benchmarks since they did not work with my JDK9
>> (issue between Dacapo and JDK9 it seems)
>>                           - I want to rerun next week to ensure number
>> stability
>>
>>                           - Accuracy of the feature
>>
>>                           I wrote a small microbenchmark that allocates
>> from two different stacktraces at a given ratio. For example, 10% of
>> stacktrace S1 and 90% from
>>         stacktrace
>>                      S2. The
>>                           microbenchmark was run 20 times, I averaged the
>> results and looked for accuracy. It seems that statistically it is sound
>> since if I allocated10%
>>         S1 and 90%
>>                      S2, with a
>>                           sampling rate of 512k, I obtained 9.61% S1 and
>> 90.49% S2.
>>
>>                           Let me know if there are any questions on the
>> numbers and if you'd like to see some more data.
>>
>>                           Note: this was done using our internal JDK8
>> implementation since the webrev provided by
>>         http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <
>> http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>
>>                      <http://cr.openjdk.java.net/~
>> rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~r
>> asbold/heapz/webrev.00/index.html>>
>>                      <http://cr.openjdk.java.net/~
>> rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~r
>> asbold/heapz/webrev.00/index.html>
>>         <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <
>> http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>>> does
>> not yet contain the whole
>>                      implementation and therefore would have been
>> misleading.
>>
>>                           Thanks,
>>                           Jc
>>
>>
>>                           On Tue, Apr 4, 2017 at 3:55 PM, JC Beyler <
>> jcbeyler at google.com <mailto:jcbeyler at google.com> <mailto:
>> jcbeyler at google.com
>>         <mailto:jcbeyler at google.com>> <mailto:jcbeyler at google.com
>> <mailto:jcbeyler at google.com> <mailto:jcbeyler at google.com <mailto:
>> jcbeyler at google.com>>>> wrote:
>>
>>                               Hi all,
>>
>>                               To move the discussion forward, with Chuck
>> Rasbold's help to make a webrev, we pushed this:
>>         http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <
>> http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>
>>         <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <
>> http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>>
>>                      <http://cr.openjdk.java.net/~
>> rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~r
>> asbold/heapz/webrev.00/index.html>
>>         <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <
>> http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>>>
>>                               415 lines changed: 399 ins; 13 del; 3 mod;
>> 51122 unchg
>>
>>                               This is not a final change that does the
>> whole proposition from the JBS entry: https://bugs.openjdk.java.net/
>> browse/JDK-8177374
>>         <https://bugs.openjdk.java.net/browse/JDK-8177374>
>>                      <https://bugs.openjdk.java.net/browse/JDK-8177374 <
>> https://bugs.openjdk.java.net/browse/JDK-8177374>>
>>                      <https://bugs.openjdk.java.net/browse/JDK-8177374 <
>> https://bugs.openjdk.java.net/browse/JDK-8177374> <
>> https://bugs.openjdk.java.net/browse/JDK-8177374
>>         <https://bugs.openjdk.java.net/browse/JDK-8177374>>>; what it
>> does show is parts of the implementation that is
>>                      proposed and hopefully can start the conversation
>> going
>>                               as I work through the details.
>>
>>                               For example, the changes to C2 are done
>> here for the allocations:
>>         http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/shar
>> e/vm/opto/macro.cpp.patch
>>         <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/sha
>> re/vm/opto/macro.cpp.patch>
>>                      <http://cr.openjdk.java.net/~
>> rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch
>>         <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/sha
>> re/vm/opto/macro.cpp.patch>>
>>                      <http://cr.openjdk.java.net/~
>> rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch
>>         <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/sha
>> re/vm/opto/macro.cpp.patch>
>>                      <http://cr.openjdk.java.net/~
>> rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch
>>         <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/sha
>> re/vm/opto/macro.cpp.patch>>>
>>
>>                               Hopefully this all makes sense and thank
>> you for all your future comments!
>>                               Jc
>>
>>
>>                               On Tue, Dec 13, 2016 at 1:11 PM, JC Beyler <
>> jcbeyler at google.com <mailto:jcbeyler at google.com> <mailto:
>> jcbeyler at google.com
>>         <mailto:jcbeyler at google.com>> <mailto:jcbeyler at google.com
>> <mailto:jcbeyler at google.com> <mailto:jcbeyler at google.com <mailto:
>> jcbeyler at google.com>>>>
>>                      wrote:
>>
>>                                   Hello all,
>>
>>                                   This is a follow-up from Jeremy's
>> initial email from last year:
>>         http://mail.openjdk.java.net/pipermail/serviceability-dev/20
>> 15-June/017543.html <http://mail.openjdk.java.net/
>> pipermail/serviceability-dev/2015-June/017543.html>
>>                      <http://mail.openjdk.java.
>> net/pipermail/serviceability-dev/2015-June/017543.html
>>         <http://mail.openjdk.java.net/pipermail/serviceability-dev/2
>> 015-June/017543.html>>
>>                      <http://mail.openjdk.java.
>> net/pipermail/serviceability-dev/2015-June/017543.html
>>         <http://mail.openjdk.java.net/pipermail/serviceability-dev/2
>> 015-June/017543.html> <http://mail.openjdk.java.net/
>> pipermail/serviceability-dev/2015-June/017543.html
>>         <http://mail.openjdk.java.net/pipermail/serviceability-dev/2
>> 015-June/017543.html>>>
>>
>>                                   I've gone ahead and started working on
>> preparing this and Jeremy and I went down the route of actually writing it
>> up in JEP form:
>>         https://bugs.openjdk.java.net/browse/JDK-8171119 <
>> https://bugs.openjdk.java.net/browse/JDK-8171119> <
>> https://bugs.openjdk.java.net/browse/JDK-8171119
>>         <https://bugs.openjdk.java.net/browse/JDK-8171119>>
>>
>>                                   I think original conversation that
>> happened last year in that thread still holds true:
>>
>>                                     - We have a patch at Google that we
>> think others might be interested in
>>                                        - It provides a means to
>> understand where the allocation hotspots are at a very low overhead
>>                                        - Since it is at a low overhead,
>> we can leave it on by default
>>
>>                                   So I come to the mailing list with
>> Jeremy's initial question:
>>                                   "I thought I would ask if there is any
>> interest / if I should write a JEP / if I should just forget it."
>>
>>                                   A year ago, it seemed some thought it
>> was a good idea, is this still true?
>>
>>                                   Thanks,
>>                                   Jc
>>
>>
>>
>>
>>
>>
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170602/2ca91384/attachment-0001.html>

From serguei.spitsyn at oracle.com  Fri Jun  2 18:47:16 2017
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Fri, 2 Jun 2017 11:47:16 -0700
Subject: [XS] JDK-8180413 : avoid accessing NULL in jdk.jdwp.agent
In-Reply-To: <15b68799f2fa445094a72cb9c9534029@sap.com>
References: <f930c2163f534b6690ac6cfecb772cb1@sap.com>
 <37ac7b12-1edd-ce65-a20f-404bd3eba75e@oracle.com>
 <444e21a8d38c42338fa5c38093a32e9c@sap.com>
 <520c56a8-4379-e4b0-42ac-acde1939cc7b@oracle.com>
 <c650c04dfe524c3881e3fb5e548fbd88@sap.com>
 <cc1fe434-af00-1ae0-9695-f92b29430a6b@oracle.com>
 <15b68799f2fa445094a72cb9c9534029@sap.com>
Message-ID: <088d54e4-9b58-784a-4c69-13db8bdc9e32@oracle.com>

Hi Christoph,

Great!

Thanks,
Serguei


On 6/2/17 03:11, Langer, Christoph wrote:
> Hi,
>
> I pushed the fix to jdk10 now: http://hg.openjdk.java.net/jdk10/jdk10/jdk/rev/66e9436dbaf8
>
>
> Best regards
> Christoph
>
>> -----Original Message-----
>> From: serguei.spitsyn at oracle.com [mailto:serguei.spitsyn at oracle.com]
>> Sent: Montag, 22. Mai 2017 11:40
>> To: Baesken, Matthias <matthias.baesken at sap.com>; Alan Bateman
>> <Alan.Bateman at oracle.com>; core-libs-dev at openjdk.java.net;
>> serviceability-dev at openjdk.java.net
>> Cc: Simonis, Volker <volker.simonis at sap.com>; Langer, Christoph
>> <christoph.langer at sap.com>
>> Subject: Re: [XS] JDK-8180413 : avoid accessing NULL in jdk.jdwp.agent
>>
>> Hello Matthias,
>>
>> It is great, you are able to push this fix!
>>
>> Thanks,
>> Serguei
>>
>>
>>
>> On 5/22/17 01:49, Baesken, Matthias wrote:
>>> Hello Serguei,  the  com/sun/jdi  tests were executed ,  same number of
>> passed and errors in a patched and unpatched JDK .
>>> We do not need a sponsor because  the change is in JDK not HS .
>>>
>>> Best regards, Matthias
>>>
>>>
>>> -----Original Message-----
>>> From: serguei.spitsyn at oracle.com [mailto:serguei.spitsyn at oracle.com]
>>> Sent: Freitag, 19. Mai 2017 02:18
>>> To: Baesken, Matthias <matthias.baesken at sap.com>; Alan Bateman
>> <Alan.Bateman at oracle.com>; core-libs-dev at openjdk.java.net;
>> serviceability-dev at openjdk.java.net
>>> Cc: Simonis, Volker <volker.simonis at sap.com>
>>> Subject: Re: [XS] JDK-8180413 : avoid accessing NULL in jdk.jdwp.agent
>>>
>>> Hi Matthias,
>>>
>>> The fix looks good to me.
>>> It must be tested at least with the JTreg com/sun/jdi tests.
>>> Do you need a sponsor?
>>>
>>> Thanks,
>>> Serguei
>>>
>>>
>>>
>>> On 5/16/17 03:21, Baesken, Matthias wrote:
>>>>     Sure, I forward it to  serviceability-dev .
>>>>
>>>> -----Original Message-----
>>>> From: Alan Bateman [mailto:Alan.Bateman at oracle.com]
>>>> Sent: Dienstag, 16. Mai 2017 11:51
>>>> To: Baesken, Matthias <matthias.baesken at sap.com>; core-libs-
>> dev at openjdk.java.net
>>>> Cc: Simonis, Volker <volker.simonis at sap.com>
>>>> Subject: Re: [XS] JDK-8180413 : avoid accessing NULL in jdk.jdwp.agent
>>>>
>>>>
>>>>
>>>> On 16/05/2017 10:04, Baesken, Matthias wrote:
>>>>> Hello, could you please review this small change :
>>>>>
>>>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8180413/
>>>>>
>>>>> it fixes a number of places in   jdk.jdwp.agent   where in case of an error
>> it is attempted to write to NULL .
>>>>> Bug :  JDK-8180413 : avoid accessing NULL in jdk.jdwp.agent
>>>>>
>>>>>
>>>>> https://bugs.openjdk.java.net/browse/JDK-8180413
>>>>>
>>>>>
>>>> Can you bring this to serviceability-dev as that is the mailing list
>>>> where the JDWP agent is maintained?
>>>>
>>>> -Alan


From serguei.spitsyn at oracle.com  Fri Jun  2 20:42:44 2017
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Fri, 2 Jun 2017 13:42:44 -0700
Subject: RFR: JDK-8180300 Move JDWP specs to specs directory
In-Reply-To: <8c99cf2c-f00b-98ef-9389-c45dfb982411@oracle.com>
References: <8c99cf2c-f00b-98ef-9389-c45dfb982411@oracle.com>
Message-ID: <74890c96-8aae-14d2-e01f-9451ed0fc592@oracle.com>

Magnus,

I've included the serviceability mailing list.
Thank you for catching these links!

But these two files are not the only fixes for this sub-task, right?
Should this webrev include your new markdown files as well?

Thanks,
Serguei



On 6/2/17 06:03, Magnus Ihse Bursie wrote:
> The JDWP spec will move to a new place in the "specs" directory in the 
> docs image, and links to it needs to be updated.
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8180300
> WebRev: 
> http://cr.openjdk.java.net/~ihse/JDK-8180300-move-jdwp-spec/webrev.01
>
> This is a noreg-docs fix and will be pushed to jdk9.
>
> /Magnus


From serguei.spitsyn at oracle.com  Fri Jun  2 21:16:24 2017
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Fri, 2 Jun 2017 14:16:24 -0700
Subject: JDK 10 RFR 8181417: Code cleanups in com.sun.jdi
In-Reply-To: <d2e245e424664ba78ad2b22842e61fa4@sap.com>
References: <a7e6b44d1b4d49c580a65549fe0f98e0@sap.com>
 <d2e245e424664ba78ad2b22842e61fa4@sap.com>
Message-ID: <7f7b7359-0a82-34e4-17a3-597d13b95492@oracle.com>

An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170602/0e0ae110/attachment.html>

From serguei.spitsyn at oracle.com  Fri Jun  2 21:26:17 2017
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Fri, 2 Jun 2017 14:26:17 -0700
Subject: RFR: JDK-8180322 Move JNI spec to specs directory
In-Reply-To: <D887C3BB-9F2A-4484-B027-D7110C9D2457@oracle.com>
References: <f42d4796-1ca5-cd3e-8c68-7df4c7ea30c5@oracle.com>
 <D887C3BB-9F2A-4484-B027-D7110C9D2457@oracle.com>
Message-ID: <7831d6bb-7e25-0886-f34d-052d2e2258b3@oracle.com>

On 6/1/17 18:50, Mandy Chung wrote:
>> On Jun 1, 2017, at 5:46 AM, Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com> wrote:
>>
>> The JNI spec will move to a new place in the "specs" directory in the docs image, and links to it needs to be updated.
>>
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8180322
>> WebRev: http://cr.openjdk.java.net/~ihse/JDK-8180322-move-jni-spec/webrev.01
>>
>> This is a noreg-docs fix and will be pushed to jdk9.
> Looks okay to me.

+1

Thanks,
Serguei

>
> Mandy


From harsha.wardhana.b at oracle.com  Mon Jun  5 05:18:14 2017
From: harsha.wardhana.b at oracle.com (Harsha Wardhana B)
Date: Mon, 5 Jun 2017 10:48:14 +0530
Subject: RFR: JDK-8178508 Co-locate remaining MM tests
In-Reply-To: <a60f4f4a-40b6-fa23-894e-02600d0ce9b7@oracle.com>
References: <6d1f972d-dbec-7a79-4c1d-a960d8696a18@oracle.com>
 <a60f4f4a-40b6-fa23-894e-02600d0ce9b7@oracle.com>
Message-ID: <11a3f964-070a-22f0-50fc-a2b0052a54df@oracle.com>

Hi Ujwal,

Looks fine.

-Harsha


On Thursday 01 June 2017 04:55 PM, Ujwal Vangapally wrote:
>
> Gentle reminder.
>
> Thanks,
>
> Ujwal.
>
>
> On 5/31/2017 10:32 AM, Ujwal Vangapally wrote:
>>
>> Kindly review the changes made for below bug
>>
>> converted tonga test to JTREG test
>>
>> added an additional assert statement for verifying 
>> setusageThreshold() operation is successful
>>
>> https://bugs.openjdk.java.net/browse/JDK-8178508
>>
>> tonga test case is currently at this path :
>>
>> http://sqe-hgi.us.oracle.com/hg/index.cgi/testbase/javase/functional/9/vm/file/tip/src/nsk/regression/b6653214/  
>>
>>
>> webrev :
>> http://cr.openjdk.java.net/~uvangapally/webrev/2017/8178508/webrev.00/
>>
>> Thanks,
>>
>> Ujwal.
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170605/0d608dd9/attachment.html>

From mandy.chung at oracle.com  Mon Jun  5 16:48:19 2017
From: mandy.chung at oracle.com (Mandy Chung)
Date: Mon, 5 Jun 2017 09:48:19 -0700
Subject: RFR: JDK-8178508 Co-locate remaining MM tests
In-Reply-To: <11a3f964-070a-22f0-50fc-a2b0052a54df@oracle.com>
References: <6d1f972d-dbec-7a79-4c1d-a960d8696a18@oracle.com>
 <a60f4f4a-40b6-fa23-894e-02600d0ce9b7@oracle.com>
 <11a3f964-070a-22f0-50fc-a2b0052a54df@oracle.com>
Message-ID: <1ED1CB49-8F63-494E-AC82-5506C9BB5FC3@oracle.com>


>> 
>> On 5/31/2017 10:32 AM, Ujwal Vangapally wrote:
>>> webrev : 
>>> http://cr.openjdk.java.net/~uvangapally/webrev/2017/8178508/webrev.00/ <http://cr.openjdk.java.net/~uvangapally/webrev/2017/8178508/webrev.00/>

The test should be in  test/java/lang/management/MemoryPoolMXBean since it?s a test for that API.  I also suggest to rename the test to LargeHeapThresholdTest (or something like that).

Mandy

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170605/1db65a2b/attachment-0001.html>

From mandy.chung at oracle.com  Mon Jun  5 16:59:13 2017
From: mandy.chung at oracle.com (Mandy Chung)
Date: Mon, 5 Jun 2017 09:59:13 -0700
Subject: RFR: JDK-8178508 Co-locate remaining MM tests
In-Reply-To: <1ED1CB49-8F63-494E-AC82-5506C9BB5FC3@oracle.com>
References: <6d1f972d-dbec-7a79-4c1d-a960d8696a18@oracle.com>
 <a60f4f4a-40b6-fa23-894e-02600d0ce9b7@oracle.com>
 <11a3f964-070a-22f0-50fc-a2b0052a54df@oracle.com>
 <1ED1CB49-8F63-494E-AC82-5506C9BB5FC3@oracle.com>
Message-ID: <FC020732-2C5C-40EA-B7E8-020A159D48F3@oracle.com>


> On Jun 5, 2017, at 9:48 AM, Mandy Chung <mandy.chung at oracle.com> wrote:
> 
> 
>>> 
>>> On 5/31/2017 10:32 AM, Ujwal Vangapally wrote:
>>>> webrev : 
>>>> http://cr.openjdk.java.net/~uvangapally/webrev/2017/8178508/webrev.00/ <http://cr.openjdk.java.net/~uvangapally/webrev/2017/8178508/webrev.00/>
> 
> The test should be in  test/java/lang/management/MemoryPoolMXBean since it?s a test for that API.  I also suggest to rename the test to LargeHeapThresholdTest (or something like that).

  41  * @run main/othervm -Xmx3000M -ea MX2G
  59                 assert i.getUsageThreshold() == TWO_G : "Usage threshold for"
The test should elimiate its dependency on -ea; otherwise the test may not fail when it runs standalone without -ea.  You can eplace the assert with an if-statement to check the condition and throw a RuntimeException instead.  

Mandy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170605/866a67b4/attachment.html>

From igor.ignatyev at oracle.com  Mon Jun  5 22:30:23 2017
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Mon, 5 Jun 2017 15:30:23 -0700
Subject: RFR: JDK-8178508 Co-locate remaining MM tests
In-Reply-To: <a60f4f4a-40b6-fa23-894e-02600d0ce9b7@oracle.com>
References: <6d1f972d-dbec-7a79-4c1d-a960d8696a18@oracle.com>
 <a60f4f4a-40b6-fa23-894e-02600d0ce9b7@oracle.com>
Message-ID: <D102EA42-BB7A-4368-9E47-E72E8CF0BB94@oracle.com>

Hello Ujwal,

we usually don't use 'assert' in our tests, we are either using static methods from jdk.test.lib.Assert class or make comparison directly in a test and throw an unchecked exception(I'd recommend AssertionError).
otherwise looks good to me.

-- Igor  
 
> On Jun 1, 2017, at 4:25 AM, Ujwal Vangapally <ujwal.vangapally at oracle.com> wrote:
> 
> Gentle reminder.
> Thanks,
> 
> Ujwal.
> 
> On 5/31/2017 10:32 AM, Ujwal Vangapally wrote:
>> Kindly review the changes made for below bug
>> 
>> converted tonga test to JTREG test
>> 
>> added an additional assert statement for verifying setusageThreshold() operation is successful 
>> https://bugs.openjdk.java.net/browse/JDK-8178508 <https://bugs.openjdk.java.net/browse/JDK-8178508>
>> 
>> tonga test case is currently at this path :
>> 
>> http://sqe-hgi.us.oracle.com/hg/index.cgi/testbase/javase/functional/9/vm/file/tip/src/nsk/regression/b6653214/ <http://sqe-hgi.us.oracle.com/hg/index.cgi/testbase/javase/functional/9/vm/file/tip/src/nsk/regression/b6653214/> 
>> 
>> 
>> webrev : 
>> http://cr.openjdk.java.net/~uvangapally/webrev/2017/8178508/webrev.00/ <http://cr.openjdk.java.net/%7Euvangapally/webrev/2017/8178508/webrev.00/>
>> 
>> Thanks,
>> 
>> Ujwal.
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170605/8316ded1/attachment.html>

From chihiro.ito at oracle.com  Wed Jun  7 09:46:07 2017
From: chihiro.ito at oracle.com (chihiro ito)
Date: Wed, 7 Jun 2017 18:46:07 +0900
Subject: [10] RFR 8181647: jhsdb jstack could not output thread name
Message-ID: <5937CB5F.4060209@oracle.com>

Hi all,

I changed to output Java thread name in jhsdb jstack as following.

jhsdb jstack --pid 25879
"main" nid=25884: (state = BLOCKED)

jhsdb jstack --mixed --pid 25879
----------------- "main" nid=25884 -----------------

Could you possibly review for this following small change? If review is 
ok, please commit this as cito.

Source:
diff --git 
a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java 
b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
--- a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
+++ b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
@@ -86,6 +86,13 @@
              try {
                 CFrame f = cdbg.topFrameForThread(th);
                 out.print("----------------- ");
+               JavaThread jthread = (JavaThread) proxyToThread.get(th);
+               if (jthread != null) {
+                   out.print("\"");
+                   out.print(jthread.getThreadName());
+                   out.print("\" ");
+               }
+               out.print("nid=");
                 out.print(th);
                 out.println(" -----------------");
                 while (f != null) {
diff --git 
a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java 
b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
--- 
a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
+++ 
b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
@@ -75,7 +75,9 @@
              for (JavaThread cur = threads.first(); cur != null; cur = 
cur.next(), i++) {
                  if (cur.isJavaThread()) {
                      Address sp = cur.getLastJavaSP();
-                    tty.print("Thread ");
+                    tty.print("\"");
+                    tty.print(cur.getThreadName());
+                    tty.print("\" nid=");
                      cur.printThreadIDOn(tty);
                      tty.print(": (state = " + cur.getThreadState());
                      if (verbose) {

Regards,
Chihiro

-- 

Chihiro Ito | Principal Consultant | +81.90.6148.8815
Oracle <http://www.oracle.com> Consultant
ORACLE Japan | Akasaka Center Bldg. | Motoakasaka 1-3-13 | 1070051 
Minato-ku, Tokyo, JAPAN

Oracle is committed to developing practices and products that help 
protect the environment <http://www.oracle.com/commitment>

From poonam.bajaj at oracle.com  Wed Jun  7 15:53:05 2017
From: poonam.bajaj at oracle.com (Poonam Parhar)
Date: Wed, 7 Jun 2017 08:53:05 -0700 (PDT)
Subject: RFR(10): JDK-8178536: OOM ERRORS + SERVICE-THREAD TAKES A PROCESSOR
 TO 100%
Message-ID: <cc9ee5be-e334-49e5-9d91-c280d721beab@default>

Hello,

Could I have reviews for the changes for the Bug  HYPERLINK "https://bugs.openjdk.java.net/browse/JDK-8178536" JDK-8178536: OOM ERRORS + SERVICE-THREAD TAKES A PROCESSOR TO 100%.

Problem: If there are listeners installed for MemoryMXBean then in the event of an OOME, JVM internal thread 'Service Thread' responsible for delivering notifications to the Java threads itself may encounter an OOM exception and get into a loop of processing its pending requests. This happens if and when the Service Thread executing the native code calls its corresponding java methods and faces an OOM exception, this pending exception makes the thread exit early from SensorInfo?trigger() function before it can update its pending requests counter (_pending_trigger_count). This pending exception is never cleared and that makes the thread loop in LowMemoryDetector::process_sensor_changes().

Hotspot changes:
http://cr.openjdk.java.net/~poonam/8178536/webrev.hotspot/
These changes check for the pending exception and clear it, and make sure that the pending requests counters are kept in sync on both the Java and the VM side.

JDK changes:
http://cr.openjdk.java.net/~poonam/8178536/webrev.jdk/
These changes make the triggerAction() a no-op since we need to call this method if MemoryService::create_MemoryUsage_obj() encounters an OOM exception and we want to avoid further potential OOM exceptions in triggerAction(MemoryUsage).

Testcase: 
I have written a testcase though I won't be integrating it along with the fix because the failure is unreliable; the Service Thread does not always fall into the situation that causes it to loop i.e. it has _pending_trigger_count>0 and hits an OOM in its Java calls.
http://cr.openjdk.java.net/~poonam/8178536/webrev.hotspot.testcase/

Thanks,
Poonam
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170607/a3c8a0d1/attachment-0001.html>

From jini.george at oracle.com  Thu Jun  8 07:42:39 2017
From: jini.george at oracle.com (Jini George)
Date: Thu, 8 Jun 2017 13:12:39 +0530
Subject: [10] RFR 8181647: jhsdb jstack could not output thread name
In-Reply-To: <5937CB5F.4060209@oracle.com>
References: <5937CB5F.4060209@oracle.com>
Message-ID: <a9a45cd1-8717-ed6b-8169-9543e904c20f@oracle.com>

Hi Chihiro,

Thank you for making this useful change. Your changes look good.

It would be great though if you could add a test case for this. Could 
you also modify the copyright year to 2017 ? One additional suggestion: 
The addition of the thread name makes the separator lines unaligned in 
the pstack/jstack --mixed cases. Like:

----------------- "Service Thread" nid=16051 -----------------
and
----------------- nid=16052 -----------------

So I am wondering if it would make sense to have the name printed out on 
a separate line to keep the separator lines aligned. But this is a nit, 
and I would leave it to you to decide on this.

Thanks,
Jini (Not a (R)eviewer)

On 6/7/2017 3:16 PM, chihiro ito wrote:
> Hi all,
>
> I changed to output Java thread name in jhsdb jstack as following.
>
> jhsdb jstack --pid 25879
> "main" nid=25884: (state = BLOCKED)
>
> jhsdb jstack --mixed --pid 25879
> ----------------- "main" nid=25884 -----------------
>
> Could you possibly review for this following small change? If review 
> is ok, please commit this as cito.
>
> Source:
> diff --git 
> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java 
> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
> --- 
> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
> +++ 
> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
> @@ -86,6 +86,13 @@
>              try {
>                 CFrame f = cdbg.topFrameForThread(th);
>                 out.print("----------------- ");
> +               JavaThread jthread = (JavaThread) proxyToThread.get(th);
> +               if (jthread != null) {
> +                   out.print("\"");
> +                   out.print(jthread.getThreadName());
> +                   out.print("\" ");
> +               }
> +               out.print("nid=");
>                 out.print(th);
>                 out.println(" -----------------");
>                 while (f != null) {
> diff --git 
> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java 
> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java 
>
> --- 
> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
> +++ 
> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
> @@ -75,7 +75,9 @@
>              for (JavaThread cur = threads.first(); cur != null; cur = 
> cur.next(), i++) {
>                  if (cur.isJavaThread()) {
>                      Address sp = cur.getLastJavaSP();
> -                    tty.print("Thread ");
> +                    tty.print("\"");
> +                    tty.print(cur.getThreadName());
> +                    tty.print("\" nid=");
>                      cur.printThreadIDOn(tty);
>                      tty.print(": (state = " + cur.getThreadState());
>                      if (verbose) {
>
> Regards,
> Chihiro
>


From chihiro.ito at oracle.com  Thu Jun  8 09:04:58 2017
From: chihiro.ito at oracle.com (chihiro ito)
Date: Thu, 8 Jun 2017 18:04:58 +0900
Subject: [10] RFR 8181647: jhsdb jstack could not output thread name
In-Reply-To: <a9a45cd1-8717-ed6b-8169-9543e904c20f@oracle.com>
References: <5937CB5F.4060209@oracle.com>
 <a9a45cd1-8717-ed6b-8169-9543e904c20f@oracle.com>
Message-ID: <5939133A.9040709@oracle.com>

Hi Jini,

Thank you for your advices. I try to add the test case and modify the 
copyright year to 2017.
Basically, I agree with your idea, but I think that the separator line 
should finally be the same as the output of the jstack command. I worry 
which is better way.

Thanks,
Chihiro

On 2017/06/08 16:42, Jini George wrote:
> Hi Chihiro,
>
> Thank you for making this useful change. Your changes look good.
>
> It would be great though if you could add a test case for this. Could 
> you also modify the copyright year to 2017 ? One additional 
> suggestion: The addition of the thread name makes the separator lines 
> unaligned in the pstack/jstack --mixed cases. Like:
>
> ----------------- "Service Thread" nid=16051 -----------------
> and
> ----------------- nid=16052 -----------------
>
> So I am wondering if it would make sense to have the name printed out 
> on a separate line to keep the separator lines aligned. But this is a 
> nit, and I would leave it to you to decide on this.
>
> Thanks,
> Jini (Not a (R)eviewer)
>
> On 6/7/2017 3:16 PM, chihiro ito wrote:
>> Hi all,
>>
>> I changed to output Java thread name in jhsdb jstack as following.
>>
>> jhsdb jstack --pid 25879
>> "main" nid=25884: (state = BLOCKED)
>>
>> jhsdb jstack --mixed --pid 25879
>> ----------------- "main" nid=25884 -----------------
>>
>> Could you possibly review for this following small change? If review 
>> is ok, please commit this as cito.
>>
>> Source:
>> diff --git 
>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java 
>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
>> --- 
>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
>> +++ 
>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
>> @@ -86,6 +86,13 @@
>>              try {
>>                 CFrame f = cdbg.topFrameForThread(th);
>>                 out.print("----------------- ");
>> +               JavaThread jthread = (JavaThread) proxyToThread.get(th);
>> +               if (jthread != null) {
>> +                   out.print("\"");
>> +                   out.print(jthread.getThreadName());
>> +                   out.print("\" ");
>> +               }
>> +               out.print("nid=");
>>                 out.print(th);
>>                 out.println(" -----------------");
>>                 while (f != null) {
>> diff --git 
>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java 
>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java 
>>
>> --- 
>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
>> +++ 
>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
>> @@ -75,7 +75,9 @@
>>              for (JavaThread cur = threads.first(); cur != null; cur 
>> = cur.next(), i++) {
>>                  if (cur.isJavaThread()) {
>>                      Address sp = cur.getLastJavaSP();
>> -                    tty.print("Thread ");
>> +                    tty.print("\"");
>> +                    tty.print(cur.getThreadName());
>> +                    tty.print("\" nid=");
>>                      cur.printThreadIDOn(tty);
>>                      tty.print(": (state = " + cur.getThreadState());
>>                      if (verbose) {
>>
>> Regards,
>> Chihiro
>>
>

-- 

Chihiro Ito | Principal Consultant | +81.90.6148.8815
Oracle <http://www.oracle.com> Consultant
ORACLE Japan | Akasaka Center Bldg. | Motoakasaka 1-3-13 | 1070051 
Minato-ku, Tokyo, JAPAN

Oracle is committed to developing practices and products that help 
protect the environment <http://www.oracle.com/commitment>

From ujwal.vangapally at oracle.com  Thu Jun  8 11:13:29 2017
From: ujwal.vangapally at oracle.com (Ujwal Vangapally)
Date: Thu, 8 Jun 2017 16:43:29 +0530
Subject: RFR: JDK-8178508 Co-locate remaining MM tests
In-Reply-To: <FC020732-2C5C-40EA-B7E8-020A159D48F3@oracle.com>
References: <6d1f972d-dbec-7a79-4c1d-a960d8696a18@oracle.com>
 <a60f4f4a-40b6-fa23-894e-02600d0ce9b7@oracle.com>
 <11a3f964-070a-22f0-50fc-a2b0052a54df@oracle.com>
 <1ED1CB49-8F63-494E-AC82-5506C9BB5FC3@oracle.com>
 <FC020732-2C5C-40EA-B7E8-020A159D48F3@oracle.com>
Message-ID: <325350fb-fd35-0d54-328e-e78ee3129c46@oracle.com>

Thanks for the Review Mandy, Igor, Harsha.

below is the link for new webrev incorporating review comments.

webrev: 
http://cr.openjdk.java.net/~uvangapally/webrev/2017/8178508/webrev.01/ 
<http://cr.openjdk.java.net/%7Euvangapally/webrev/2017/8178508/webrev.00/>


changes in new webrev other than those mentioned in review comments:

using -Xmx3000M option on unsupported machines(like window 32bit machine 
containing just 2GB user space) creates unnecessary failures in previous 
implementation, so used ProcessTools.executeTestJava() to do a sample 
run of "java -Xmx3000M -version" and took decision depending on it's 
exit value whether to use -Xmx3000M or not in actual execution .

removed  '@requires os.maxmemory >3G'  as we really don't require more 
than 3GB physical memory to verify this test.

Suggestions:

will it makes more sense to add
@requires (os.family != "windows") | (os.simpleArch != "i586")
as windows 32 bit just provides 2GB for user space.

no problem with current test as it just prints
"Ability to use big heap thresholds has NOT been verified"
on win 32bit.

Thanks,
Ujwal.


On 6/5/2017 10:29 PM, Mandy Chung wrote:
>
>> On Jun 5, 2017, at 9:48 AM, Mandy Chung <mandy.chung at oracle.com 
>> <mailto:mandy.chung at oracle.com>> wrote:
>>
>>
>>>>
>>>> On 5/31/2017 10:32 AM, Ujwal Vangapally wrote:
>>>>> webrev :
>>>>> http://cr.openjdk.java.net/~uvangapally/webrev/2017/8178508/webrev.00/ 
>>>>> <http://cr.openjdk.java.net/%7Euvangapally/webrev/2017/8178508/webrev.00/>
>>
>> The test should be in  test/java/lang/management/MemoryPoolMXBean 
>> since it?s a test for that API.  I also suggest to rename the test to 
>> LargeHeapThresholdTest (or something like that).
>
>    41  * @run main/othervm -Xmx3000M -ea MX2G
>    59                 assert i.getUsageThreshold() == TWO_G : "Usage threshold for"
> The test should elimiate its dependency on -ea; otherwise the test may 
> not fail when it runs standalone without -ea.  You can eplace the 
> assert with an if-statement to check the condition and throw a 
> RuntimeException instead.
>
> Mandy

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170608/f7aa4831/attachment.html>

From ujwal.vangapally at oracle.com  Thu Jun  8 13:16:33 2017
From: ujwal.vangapally at oracle.com (Ujwal Vangapally)
Date: Thu, 8 Jun 2017 18:46:33 +0530
Subject: RFR: JDK-8178508 Co-locate remaining MM tests
In-Reply-To: <325350fb-fd35-0d54-328e-e78ee3129c46@oracle.com>
References: <6d1f972d-dbec-7a79-4c1d-a960d8696a18@oracle.com>
 <a60f4f4a-40b6-fa23-894e-02600d0ce9b7@oracle.com>
 <11a3f964-070a-22f0-50fc-a2b0052a54df@oracle.com>
 <1ED1CB49-8F63-494E-AC82-5506C9BB5FC3@oracle.com>
 <FC020732-2C5C-40EA-B7E8-020A159D48F3@oracle.com>
 <325350fb-fd35-0d54-328e-e78ee3129c46@oracle.com>
Message-ID: <a32aa8d1-7e7b-1e47-e83a-7ffdfab1f5bf@oracle.com>

Please use the below link for new webrev, link in my previous reply is 
redirecting to old webrev

webrev : 
http://cr.openjdk.java.net/~uvangapally/webrev/2017/8178508/webrev.01/


Thanks,
Ujwal.

On 6/8/2017 4:43 PM, Ujwal Vangapally wrote:
>
> Thanks for the Review Mandy, Igor, Harsha.
>
> below is the link for new webrev incorporating review comments.
>
> webrev: 
> http://cr.openjdk.java.net/~uvangapally/webrev/2017/8178508/webrev.01/ 
> <http://cr.openjdk.java.net/%7Euvangapally/webrev/2017/8178508/webrev.00/>
>
>
> changes in new webrev other than those mentioned in review comments:
>
> using -Xmx3000M option on unsupported machines(like window 32bit 
> machine containing just 2GB user space) creates unnecessary failures 
> in previous implementation, so used ProcessTools.executeTestJava() to 
> do a sample run of "java -Xmx3000M -version" and took decision 
> depending on it's exit value whether to use -Xmx3000M or not in actual 
> execution .
>
> removed  '@requires os.maxmemory >3G'  as we really don't require more 
> than 3GB physical memory to verify this test.
>
> Suggestions:
>
> will it makes more sense to add
> @requires (os.family != "windows") | (os.simpleArch != "i586")
> as windows 32 bit just provides 2GB for user space.
>
> no problem with current test as it just prints
> "Ability to use big heap thresholds has NOT been verified"
> on win 32bit.
>
> Thanks,
> Ujwal.
>
>
> On 6/5/2017 10:29 PM, Mandy Chung wrote:
>>
>>> On Jun 5, 2017, at 9:48 AM, Mandy Chung <mandy.chung at oracle.com 
>>> <mailto:mandy.chung at oracle.com>> wrote:
>>>
>>>
>>>>>
>>>>> On 5/31/2017 10:32 AM, Ujwal Vangapally wrote:
>>>>>> webrev :
>>>>>> http://cr.openjdk.java.net/~uvangapally/webrev/2017/8178508/webrev.00/ 
>>>>>> <http://cr.openjdk.java.net/%7Euvangapally/webrev/2017/8178508/webrev.00/>
>>>
>>> The test should be in  test/java/lang/management/MemoryPoolMXBean 
>>> since it?s a test for that API.  I also suggest to rename the test 
>>> to LargeHeapThresholdTest (or something like that).
>>
>>    41  * @run main/othervm -Xmx3000M -ea MX2G
>>    59                 assert i.getUsageThreshold() == TWO_G : "Usage threshold for"
>> The test should elimiate its dependency on -ea; otherwise the test 
>> may not fail when it runs standalone without -ea.  You can eplace the 
>> assert with an if-statement to check the condition and throw a 
>> RuntimeException instead.
>>
>> Mandy
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170608/0156ebd9/attachment-0001.html>

From serguei.spitsyn at oracle.com  Fri Jun  9 00:10:28 2017
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Thu, 8 Jun 2017 17:10:28 -0700
Subject: RFR(M): 8178054: [TESTBUG] Need test for JVM TI IsModifiableModule
Message-ID: <1843de6c-2d60-3659-0b43-940dfdaed387@oracle.com>

Please, review a JDK 9 fix for the test enhancement:
https://bugs.openjdk.java.net/browse/JDK-8178054

Webrev:
http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8178054-jvmti-ismodif.2/


Summary:

   New JVMTI function IsModifiableModule() was introduced a couple of 
month ago.
   This patch adds a test coverage for it.

Testing:
   Tested the updated JTREG test serviceability/jvmti/GetNamedModule.


Thanks,
Serguei

From Alan.Bateman at oracle.com  Fri Jun  9 07:44:36 2017
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Fri, 9 Jun 2017 08:44:36 +0100
Subject: RFR(M): 8178054: [TESTBUG] Need test for JVM TI IsModifiableModule
In-Reply-To: <1843de6c-2d60-3659-0b43-940dfdaed387@oracle.com>
References: <1843de6c-2d60-3659-0b43-940dfdaed387@oracle.com>
Message-ID: <26d4a0d5-4fa9-6e3d-fd08-1ec524f3a194@oracle.com>

On 09/06/2017 01:10, serguei.spitsyn at oracle.com wrote:
> Please, review a JDK 9 fix for the test enhancement:
> https://bugs.openjdk.java.net/browse/JDK-8178054
>
> Webrev:
> http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8178054-jvmti-ismodif.2/ 
>
>
>
> Summary:
>
>   New JVMTI function IsModifiableModule() was introduced a couple of 
> month ago.
>   This patch adds a test coverage for it.
This update means that GetNamedModuleTest now tests both GetNamedModule 
and IsModifiableModule. Would be better to separate them so that there 
are two focused tests rather than one test trying to test two functions? 
If it has to be one test then I assume GetNamedModuleTest and the 
description in @summary will need to be updated.

Otherwise the test coverage looks good as it exercises both named and 
unnamed modules, plus it tests the error cases.

In passing, it looks like "@modules jdk.jdi" is missing (pre-dates your 
changes).

-Alan

From serguei.spitsyn at oracle.com  Fri Jun  9 09:07:47 2017
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Fri, 9 Jun 2017 02:07:47 -0700
Subject: RFR(M): 8178054: [TESTBUG] Need test for JVM TI IsModifiableModule
In-Reply-To: <26d4a0d5-4fa9-6e3d-fd08-1ec524f3a194@oracle.com>
References: <1843de6c-2d60-3659-0b43-940dfdaed387@oracle.com>
 <26d4a0d5-4fa9-6e3d-fd08-1ec524f3a194@oracle.com>
Message-ID: <75da7763-3c9d-a467-fa78-2b811b601ef4@oracle.com>

Hi Alan,

Thank you for the comments!

I've made a stand-alone IsModifiableModule test and also added "@modules 
jdk.jdi" to the GetNamedModuleTest.

New webrev:
http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8178054-jvmti-ismodif.3/

Thanks,
Serguei


On 6/9/17 00:44, Alan Bateman wrote:
> On 09/06/2017 01:10, serguei.spitsyn at oracle.com wrote:
>> Please, review a JDK 9 fix for the test enhancement:
>> https://bugs.openjdk.java.net/browse/JDK-8178054
>>
>> Webrev:
>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8178054-jvmti-ismodif.2/ 
>>
>>
>>
>> Summary:
>>
>>   New JVMTI function IsModifiableModule() was introduced a couple of 
>> month ago.
>>   This patch adds a test coverage for it.
> This update means that GetNamedModuleTest now tests both 
> GetNamedModule and IsModifiableModule. Would be better to separate 
> them so that there are two focused tests rather than one test trying 
> to test two functions? If it has to be one test then I assume 
> GetNamedModuleTest and the description in @summary will need to be 
> updated.
>
> Otherwise the test coverage looks good as it exercises both named and 
> unnamed modules, plus it tests the error cases.
>
> In passing, it looks like "@modules jdk.jdi" is missing (pre-dates 
> your changes).
>
> -Alan


From Alan.Bateman at oracle.com  Fri Jun  9 09:38:43 2017
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Fri, 9 Jun 2017 10:38:43 +0100
Subject: RFR(M): 8178054: [TESTBUG] Need test for JVM TI IsModifiableModule
In-Reply-To: <75da7763-3c9d-a467-fa78-2b811b601ef4@oracle.com>
References: <1843de6c-2d60-3659-0b43-940dfdaed387@oracle.com>
 <26d4a0d5-4fa9-6e3d-fd08-1ec524f3a194@oracle.com>
 <75da7763-3c9d-a467-fa78-2b811b601ef4@oracle.com>
Message-ID: <ee221a7f-daed-e9a4-c037-24787cef9f2d@oracle.com>



On 09/06/2017 10:07, serguei.spitsyn at oracle.com wrote:
> Hi Alan,
>
> Thank you for the comments!
>
> I've made a stand-alone IsModifiableModule test and also added 
> "@modules jdk.jdi" to the GetNamedModuleTest.
>
> New webrev:
> http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8178054-jvmti-ismodif.3/ 
>
This mostly looks good.

One suggestion is to drop the "not_a_module" parameter from 
test_is_modifable_module as it is not needed. Instead the test just 
needs to provide any non-Module object to check_is_modifable (it doesn't 
have to a class loader).

-Alan.

From serguei.spitsyn at oracle.com  Fri Jun  9 18:32:41 2017
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Fri, 9 Jun 2017 11:32:41 -0700
Subject: RFR(M): 8178054: [TESTBUG] Need test for JVM TI IsModifiableModule
In-Reply-To: <ee221a7f-daed-e9a4-c037-24787cef9f2d@oracle.com>
References: <1843de6c-2d60-3659-0b43-940dfdaed387@oracle.com>
 <26d4a0d5-4fa9-6e3d-fd08-1ec524f3a194@oracle.com>
 <75da7763-3c9d-a467-fa78-2b811b601ef4@oracle.com>
 <ee221a7f-daed-e9a4-c037-24787cef9f2d@oracle.com>
Message-ID: <202f1d93-2a73-c616-461a-6ef74b321128@oracle.com>

On 6/9/17 02:38, Alan Bateman wrote:
>
>
> On 09/06/2017 10:07, serguei.spitsyn at oracle.com wrote:
>> Hi Alan,
>>
>> Thank you for the comments!
>>
>> I've made a stand-alone IsModifiableModule test and also added 
>> "@modules jdk.jdi" to the GetNamedModuleTest.
>>
>> New webrev:
>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8178054-jvmti-ismodif.3/ 
>>
> This mostly looks good.
>
> One suggestion is to drop the "not_a_module" parameter from 
> test_is_modifable_module as it is not needed. Instead the test just 
> needs to provide any non-Module object to check_is_modifable (it 
> doesn't have to a class loader).

Ok, I removed the get_class_loader function, replaced the loader with a 
class and left alone the "not_a_module" parameter.
This is the simplest way to provide a non-module jobject.
Otherwise, the JNIEnv parameter has to be passed to the 
check_is_modifiable_error_codes .
Also, I've done a small refactoring by inlining the 
test_is_modifiable_module body into the *_check function.

New webrev:
http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8178054-jvmti-ismodif.4/


Thanks,
Serguei


>
> -Alan.


From Alan.Bateman at oracle.com  Sat Jun 10 07:41:30 2017
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Sat, 10 Jun 2017 08:41:30 +0100
Subject: RFR(M): 8178054: [TESTBUG] Need test for JVM TI IsModifiableModule
In-Reply-To: <202f1d93-2a73-c616-461a-6ef74b321128@oracle.com>
References: <1843de6c-2d60-3659-0b43-940dfdaed387@oracle.com>
 <26d4a0d5-4fa9-6e3d-fd08-1ec524f3a194@oracle.com>
 <75da7763-3c9d-a467-fa78-2b811b601ef4@oracle.com>
 <ee221a7f-daed-e9a4-c037-24787cef9f2d@oracle.com>
 <202f1d93-2a73-c616-461a-6ef74b321128@oracle.com>
Message-ID: <f371fe89-5df6-2621-b3cd-b9621a4f6088@oracle.com>



On 09/06/2017 19:32, serguei.spitsyn at oracle.com wrote:
>
> Ok, I removed the get_class_loader function, replaced the loader with 
> a class and left alone the "not_a_module" parameter.
> This is the simplest way to provide a non-module jobject.
> Otherwise, the JNIEnv parameter has to be passed to the 
> check_is_modifiable_error_codes .
> Also, I've done a small refactoring by inlining the 
> test_is_modifiable_module body into the *_check function.
>
> New webrev:
> http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8178054-jvmti-ismodif.4/ 
>
I think it looks good now.

-Alan.

From serguei.spitsyn at oracle.com  Sat Jun 10 11:08:19 2017
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Sat, 10 Jun 2017 04:08:19 -0700
Subject: RFR(M): 8178054: [TESTBUG] Need test for JVM TI IsModifiableModule
In-Reply-To: <f371fe89-5df6-2621-b3cd-b9621a4f6088@oracle.com>
References: <1843de6c-2d60-3659-0b43-940dfdaed387@oracle.com>
 <26d4a0d5-4fa9-6e3d-fd08-1ec524f3a194@oracle.com>
 <75da7763-3c9d-a467-fa78-2b811b601ef4@oracle.com>
 <ee221a7f-daed-e9a4-c037-24787cef9f2d@oracle.com>
 <202f1d93-2a73-c616-461a-6ef74b321128@oracle.com>
 <f371fe89-5df6-2621-b3cd-b9621a4f6088@oracle.com>
Message-ID: <44b3c266-06e4-f8d4-5b31-9115448cae73@oracle.com>



On 6/10/17 00:41, Alan Bateman wrote:
>
>
> On 09/06/2017 19:32, serguei.spitsyn at oracle.com wrote:
>>
>> Ok, I removed the get_class_loader function, replaced the loader with 
>> a class and left alone the "not_a_module" parameter.
>> This is the simplest way to provide a non-module jobject.
>> Otherwise, the JNIEnv parameter has to be passed to the 
>> check_is_modifiable_error_codes .
>> Also, I've done a small refactoring by inlining the 
>> test_is_modifiable_module body into the *_check function.
>>
>> New webrev:
>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8178054-jvmti-ismodif.4/ 
>>
> I think it looks good now.

Thank you a lot for review and good suggestions!
Serguei

>
> -Alan.


From jcbeyler at google.com  Mon Jun 12 18:11:45 2017
From: jcbeyler at google.com (JC Beyler)
Date: Mon, 12 Jun 2017 11:11:45 -0700
Subject: Low-Overhead Heap Profiling
In-Reply-To: <CAF9BGBxDaCOxobJi8Uu9kwKPRpnj-KqgWLfJO2z7UX71UnNF4g@mail.gmail.com>
References: <CAF9BGBx8xFYudhJ=v1_hU5=nwEcC1X0NKRmC=E-RNN6SMs1ozg@mail.gmail.com>
 <CAF9BGBxj7+C6uy4dXHO-3pNorbpO7n22-9JmevM0F11xOo+MnA@mail.gmail.com>
 <CAF9BGBxrdLVmXRTDHj5hQHJhcyrkBM6+ex1TFWi=A5z=V2FSew@mail.gmail.com>
 <CAF9BGBz81YYVUsMLw9sOMR9fH7JQmyb8d_OZ9aJ47EOAqwWhEw@mail.gmail.com>
 <2af975e6-3827-bd57-0c3d-fadd54867a67@oracle.com>
 <365499b6-3f4d-a4df-9e7e-e72a739fb26b@oracle.com>
 <CAF9BGBy0T7A-+1dejYbr6f0d4s0XR-zv-WwHYqXrbe0EtCdsxQ@mail.gmail.com>
 <fb1155c6-ccf8-843c-34fc-fc4a45fc54cc@oracle.com>
 <CAF9BGBxqmXF9OFD4F9wVb-DQOj_UfNNcBhe+ARVxJ5E4cgCC1Q@mail.gmail.com>
 <102c59b8-25b6-8c21-8eef-1de7d0bbf629@oracle.com>
 <CAF9BGBxDaCOxobJi8Uu9kwKPRpnj-KqgWLfJO2z7UX71UnNF4g@mail.gmail.com>
Message-ID: <CAF9BGByisZchXTsw1=mLETLnOggJShRJBDxFeOUgt+criTTGuQ@mail.gmail.com>

Dear all,

I've continued working on this and have done the following webrev:
http://cr.openjdk.java.net/~rasbold/8171119/webrev.05/

What it does:
   - Implement the turn on/off system for the TLAB implementation
      - Basically you can turn the system on/off using the JVMTI API
(StartHeapProfiling and StopHeapProfiling here
http://cr.openjdk.java.net/~rasbold/8171119/webrev.05/src/share/vm/prims/jvmti.xml.patch
)
         - I've currently implemented that the system resets the profiling
data at the StartHeapProfiling, this allows you to start profiling, stop
profiling, and then the user can query what happened during profiling as a
post-processing step :)

   - I've currently, for sake of simplicity and Robbin hinted it would be
better for now, removed the mutex code for the data but think that will
have to come back in a next patch or in this one at some point

   - I've also really worked on the testing code to make it more expandable
in this case

http://cr.openjdk.java.net/~rasbold/8171119/webrev.05/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c.patch
is
now a bit more generic and allows to turn on/off the sampling; it allows to
query the existence or not of frames passed from Java world to JNI, which
allows the test to define what should be seen and have the code in one
place.
          - That is done using the helper class
http://cr.openjdk.java.net/~rasbold/8171119/webrev.05/test/serviceability/jvmti/HeapMonitor/MyPackage/Frame.java.patch
             - Basically each test can now provide an array of Frames that
the native library can check the internal data for a match. This allows to
have various tests have their own signatures, etc.

          - This has allowed me to add a nice test here to test the wipe
out of the data:

http://cr.openjdk.java.net/~rasbold/8171119/webrev.05/test/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorOnOffTest.java.patch

  - Finally, I've done initial overhead testing and, though my data was a
bit noisy, I saw no real overhead using the Tlab approach while off. I will
try to get better data and more importantly, more stable data when turning
it on.

Things I still need to do:
   - Have to fix that TLAB case for the FastTLABRefill
   - Have to start looking at the data to see that it is consistent and
does gather the right samples, right frequency, etc.
   - Have to check the GC elements and what that produces
   - Run a slowdebug run and ensure I fixed all those issues you saw Robbin

Thanks for looking at the webrev and have a great week!
Jc

On Fri, Jun 2, 2017 at 8:57 AM, JC Beyler <jcbeyler at google.com> wrote:

> Dear all,
>
> Thanks Robbin for the comments. I have left the MuxLocker for now and will
> look at removing it or as a future enhancement as you say. I did make the
> class extends and added a TODO for myself to test this in slowdebug.
>
> I have also put a new webrev up that is still a work in progress but
> should allow us to talk about TLAB vs C1/C2 modifications.
>
> TLAB implementation: http://cr.openjdk.java.net/~
> rasbold/8171119/webrev.04/
> C1/C2 implementation: http://cr.openjdk.java.net/~
> rasbold/8171119/webrev.03/
>
> What this webrev has is:
>    - I have put in a TLAB implementation, it is a WIP, and I have not yet
> done the qualitative/quantitative study on it vs the implementation using
> compilation changes but the big parts are in place
>        - Caveats:
>            - There is a TODO: http://cr.openjdk.java.n
> et/~rasbold/8171119/webrev.04/src/cpu/x86/vm/macroAssembler_x86.cpp.patch
>                - I have not fixed the calculation in the case of a
> FastTLABRefill case
>            - This is always on right now, there is no way to turn it off,
> that's also a TODO to be directed by the JVMTI API
>
>     - I also have circumvented the AsyncGetCallTrace using the snippet of
> code you showed Robbin, it works for here/now
>           - But we might have to revisit this one day because it then does
> not try to get some of the native stacks and jumps directly to the Java
> stacks (I see cases where this could be an issue)
>           - However, this has cleaned up quite a bit of the code and I
> have removed all mention of ASGCT and its structures now and use directly
> the JVMTI structures
>
>    - GC is handled now, I have not yet done the qualitative/quantitative
> study on it but the big parts are in place
>
>    - Due to the way the TLAB is called, the stack walker is now correct
> and produces the right stacks it seems (this is a bold sentence from my ONE
> JTREG test :))
>
> Final notes on this webrev:
>    - Have to fix that TLAB case for the FastTLABRefill
>    - Implement the turn on/off system for the TLAB implementation
>    - Have to start looking at the data to see that it is consistent and
> does gather the right samples, right frequency, etc.
>    - Have to check the GC elements and what that produces
>    - Run a slowdebug run and ensure I fixed all those issues you saw Robbin
>
> As always, your comments and feedback are greatly appreciated! Happy
> Friday!
> Jc
>
>
> On Tue, May 30, 2017 at 5:33 AM, Robbin Ehn <robbin.ehn at oracle.com> wrote:
>
>> Hi Jc,
>>
>> On 05/22/2017 08:47 PM, JC Beyler wrote:
>>
>>> Dear all,
>>>
>>> I have a new webrev up:
>>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.03/
>>>
>>
>> I liked this!
>>
>> Two small things:
>>
>> heapMonitoring.hpp
>> class HeapMonitoring should extend AllStatic
>>
>> heapMonitoring.cpp
>> class MuxLocker should extend StackObj
>> But I think you should skip MuxLocker or push it separate generic
>> enhancement.
>>
>> Great with the jtreg test, thanks alot!
>>
>>
>>> This webrev has, I hope, fixed a lot of the comments from Robbin:
>>>    - The casts normally are all C++ style
>>>    - Moved this to jdk10-hs
>>>       - I have not tested slowdebug yet, hopefully it does not break
>>> there
>>>    - Added the garbage collection system:
>>>       - Now live sampled allocations are tracked throughout their
>>> lifetime
>>>          - When GC happens, it moves the sampled allocation information
>>> to two lists: recent and frequent GC lists
>>>              - Those lists use the array system that the live objects
>>> were using before but have different re-use strategies
>>>       - Added the JVMTI API for them via a GetFrequentGarbageTraces and
>>> GetGarbageTraces
>>> - Both use the same JVMTI structures
>>>       - Added the calls to them for the test, though I've kept that test
>>> simple for now:
>>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.03/raw_fi
>>> les/new/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c
>>>          - As I write this, I notice my webrev is missing a final change
>>> I made to the test that calls the same ReleaseTraces to each
>>> live/garbage/frequent structure. This is updated in my local repo and will
>>> get in the next webrev.
>>>
>>> Next steps for this work are:
>>>     - Putting the TLAB implementation (yes not forgotten ;-))
>>>     - Adding more testing and separate the current test system to check
>>> things a bit more thoroughly
>>>     - Have not tried to circumvent AsyncGetCallTrace yet
>>>     - Still have to double check the stack walker a bit more
>>>
>>
>> Looking forward to this.
>>
>> Could someone from compiler take a look please?
>>
>> Thanks!
>>
>> /Robbin
>>
>>
>>> Happy webrev perusal!
>>> Jc
>>>
>>>
>>> On Tue, May 16, 2017 at 5:20 AM, Robbin Ehn <robbin.ehn at oracle.com
>>> <mailto:robbin.ehn at oracle.com>> wrote:
>>>
>>>     Just a few answers,
>>>
>>>     On 05/15/2017 06:48 PM, JC Beyler wrote:
>>>
>>>         Dear all,
>>>
>>>         I've updated the webrev to:
>>>         http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/ <
>>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/>
>>>         <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/ <
>>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/>>
>>>
>>>
>>>     I'll look at this later, thanks!
>>>
>>>
>>>         Robbin,
>>>         I believe I have addressed most of your items with webrev 02:
>>>             - I added a JTreg test to show how it works:
>>>         http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/raw_fi
>>> les/new/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c
>>>         <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/raw_f
>>> iles/new/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c>
>>>         <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/raw_f
>>> iles/new/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c
>>>         <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/raw_f
>>> iles/new/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c>>
>>>            - I've modified the code to use its own data structures both
>>> internally and externally, this will make it easier to move out of
>>> AsyncGetCallTrace as we move
>>>         forward, that is still on my TODOs
>>>            - I cleaned up the JVMTI API by passing a structure that
>>> handles the num_traces and put in a ReleaseTraces as well
>>>            - I cleaned up other issues as well.
>>>
>>>         However, I have three questions, which are probably because I'm
>>> new in this community:
>>>            1) My previous webrevs were based off of JDK9 by mistake.
>>> When I took JDK10 via : hg clone http://hg.openjdk.java.net/jdk10/jdk10
>>>         <http://hg.openjdk.java.net/jdk10/jdk10> <
>>> http://hg.openjdk.java.net/jdk10/jdk10 <http://hg.openjdk.java.net/jd
>>> k10/jdk10>> jdk10
>>>                - I don't see code compatible with what you were showing
>>> (ie your patches don't make sense for that code base; ex: klass is still
>>> accessed via klass() for
>>>         example in collectedHeap.inline.hpp)
>>>                - Would you know what is the right hg clone command so we
>>> are working on the same code base?
>>>
>>>
>>>     We use jdk10-hs, e.g.
>>>     hg tclone http://hg.openjdk.java.net/jdk10/hs <
>>> http://hg.openjdk.java.net/jdk10/hs> 10-hs
>>>
>>>     There is sporadic big merges going from jdk9->jdk10->jdk10-hs and
>>> jdk10-hs->jdk10, so 10 is moving...
>>>
>>>
>>>            2) You mentioned I was using os::malloc, new,
>>> NEW_C_HEAP_ARRAY; I cleaned out the os::malloc but which of the new vs
>>> NEW_C_HEAP_ARRAY should I use. It might be
>>>         that I don't understand when one uses one or the other but I see
>>> both used around the code base?
>>>              - Is it that new is to be used for anything internal and
>>> NEW_C_HEAP_ARRAY anything provided to the JVMTI users outside of the JVM?
>>>
>>>
>>>     We overload new operator when you extend correct base class, e.g.
>>> CHeapObj<mtInternal> so use 'new'
>>>     But for arrays you will need the macro NEW_C_HEAP_ARRAY.
>>>
>>>
>>>            3) Casts: same kind question: which should I use. The code
>>> was using a bit of everything, I'll refactor it entirely but I was not
>>> clear if I should go to C casts
>>>         or C++ casts as I see both in the codebase. What is the
>>> convention I should use?
>>>
>>>
>>>     Just be consist, use what suites you, C++ casts might be preferable,
>>> if we are moving towards C++11.
>>>     And use 'right' cast, e.g. going from Thread* to JavaThread* you
>>> should use C cast or static_cast, not reinterpret_cast I would say.
>>>
>>>
>>>         Final notes on this webrev:
>>>             - I am still missing:
>>>               - Putting a TLAB implementation so that we can compare
>>> both webrevs
>>>               - Have not tried to circumvent AsyncGetCallTrace
>>>               - Putting in the handling of GC'd objects
>>>               - Fix a stack walker issue I have seen, I think I know the
>>> problem and will test that theory out for the next webrev
>>>
>>>         I will work on integrating those items for the next webrev!
>>>
>>>
>>>     Thanks!
>>>
>>>
>>>         Thanks for your help,
>>>         Jc
>>>
>>>         Ps:  I tested this on a new repo:
>>>
>>>         hg clone http://hg.openjdk.java.net/jdk10/jdk10 <
>>> http://hg.openjdk.java.net/jdk10/jdk10> <http://hg.openjdk.java.net/jd
>>> k10/jdk10
>>>         <http://hg.openjdk.java.net/jdk10/jdk10>> jdk10
>>>         ... building it
>>>         cd test
>>>         jtreg -nativepath:<path-to-jdk10>/bu
>>> ild/linux-x86_64-normal-server-release/support/test/hotspot/jtreg/native/lib/
>>> -jdk
>>>         <path-to-jdk10>/linux-x86_64-normal-server-release/images/jdk
>>> ../hotspot/test/serviceability/jvmti/HeapMonitor/
>>>
>>>
>>>     I'll test it out!
>>>
>>>     /Robbin
>>>
>>>
>>>
>>>         On Thu, May 4, 2017 at 11:21 PM, serguei.spitsyn at oracle.com
>>> <mailto:serguei.spitsyn at oracle.com> <mailto:serguei.spitsyn at oracle.com
>>>         <mailto:serguei.spitsyn at oracle.com>> <serguei.spitsyn at oracle.com
>>> <mailto:serguei.spitsyn at oracle.com> <mailto:serguei.spitsyn at oracle.com
>>>
>>>         <mailto:serguei.spitsyn at oracle.com>>> wrote:
>>>
>>>              Robbin,
>>>
>>>              Thank you for forwarding!
>>>              I will review it.
>>>
>>>              Thanks,
>>>              Serguei
>>>
>>>
>>>
>>>              On 5/4/17 02:13, Robbin Ehn wrote:
>>>
>>>                  Hi,
>>>
>>>                  To me the compiler changes looks what is expected.
>>>                  It would be good if someone from compiler could take a
>>> look at that.
>>>                  Added compiler to mail thread.
>>>
>>>                  Also adding Serguei, It would be good with his view
>>> also.
>>>
>>>                  My initial take on it, read through most of the code
>>> and took it for a ride.
>>>
>>>                  ##############################
>>>                  - Regarding the compiler changes: I think we need the
>>> 'TLAB end' trickery (mentioned by Tony P)
>>>                  instead of a separate check for sampling in fast path
>>> for the final version.
>>>
>>>                  ##############################
>>>                  - This patch I had to apply to get it compile on JDK 10:
>>>
>>>                  diff -r ac3ded340b35 src/share/vm/gc/shared/collect
>>> edHeap.inline.hpp
>>>                  --- a/src/share/vm/gc/shared/collectedHeap.inline.hpp
>>>   Fri Apr 28 14:31:38 2017 +0200
>>>                  +++ b/src/share/vm/gc/shared/collectedHeap.inline.hpp
>>>   Thu May 04 10:22:56 2017 +0200
>>>                  @@ -87,3 +87,3 @@
>>>                        // support for object alloc event (no-op most of
>>> the time)
>>>                  -    if (klass() != NULL && klass()->name() != NULL) {
>>>                  +    if (klass != NULL && klass->name() != NULL) {
>>>                          Thread *base_thread = Thread::current();
>>>                  diff -r ac3ded340b35 src/share/vm/runtime/heapMonit
>>> oring.cpp
>>>                  --- a/src/share/vm/runtime/heapMonitoring.cpp    Fri
>>> Apr 28 14:31:38 2017 +0200
>>>                  +++ b/src/share/vm/runtime/heapMonitoring.cpp    Thu
>>> May 04 10:22:56 2017 +0200
>>>                  @@ -316,3 +316,3 @@
>>>                      JavaThread *thread = reinterpret_cast<JavaThread
>>> *>(Thread::current());
>>>                  -  assert(o->size() << LogHeapWordSize == byte_size,
>>>                  +  assert(o->size() << LogHeapWordSize ==
>>> (long)byte_size,
>>>                             "Object size is incorrect.");
>>>
>>>                  ##############################
>>>                  - This patch I had to apply to get it not asserting
>>> during slowdebug:
>>>
>>>                  --- a/src/share/vm/runtime/heapMonitoring.cpp    Fri
>>> Apr 28 15:15:16 2017 +0200
>>>                  +++ b/src/share/vm/runtime/heapMonitoring.cpp    Thu
>>> May 04 10:24:25 2017 +0200
>>>                  @@ -32,3 +32,3 @@
>>>                    // TODO(jcbeyler): should we make this into a JVMTI
>>> structure?
>>>                  -struct StackTraceData {
>>>                  +struct StackTraceData : CHeapObj<mtInternal> {
>>>                      ASGCT_CallTrace *trace;
>>>                  @@ -143,3 +143,2 @@
>>>                    StackTraceStorage::StackTraceStorage() :
>>>                  -    _allocated_traces(new
>>> StackTraceData*[MaxHeapTraces]),
>>>                        _allocated_traces_size(MaxHeapTraces),
>>>                  @@ -147,2 +146,3 @@
>>>                        _allocated_count(0) {
>>>                  +  _allocated_traces = NEW_C_HEAP_ARRAY(StackTraceData*,
>>> MaxHeapTraces, mtInternal);
>>>                      memset(_allocated_traces, 0,
>>> sizeof(*_allocated_traces) * MaxHeapTraces);
>>>                  @@ -152,3 +152,3 @@
>>>                    StackTraceStorage::~StackTraceStorage() {
>>>                  -  delete[] _allocated_traces;
>>>                  +  FREE_C_HEAP_ARRAY(StackTraceData*,
>>> _allocated_traces);
>>>                    }
>>>
>>>                  - Classes should extend correct base class for which
>>> type of memory is used for it e.g.: CHeapObj<mt????> or StackObj or
>>> AllStatic
>>>                  - The style in heapMonitoring.cpp is a bit different
>>> from normal vm-style, e.g. using C++ casts instead of C. You mix
>>> NEW_C_HEAP_ARRAY, os::malloc and new.
>>>                  - In jvmtiHeapTransition.hpp you use C cast instead.
>>>
>>>                  ##############################
>>>                  - This patch I had apply to get traces without setting
>>> an ?unrelated? capability
>>>                  - Should this not be a new capability?
>>>
>>>                  diff -r c02a5d8785bf src/share/vm/prims/forte.cpp
>>>                  --- a/src/share/vm/prims/forte.cpp    Fri Apr 28
>>> 15:15:16 2017 +0200
>>>                  +++ b/src/share/vm/prims/forte.cpp    Thu May 04
>>> 10:24:25 2017 +0200
>>>                  @@ -530,6 +530,6 @@
>>>
>>>                  -  if (!JvmtiExport::should_post_class_load()) {
>>>                  +/*  if (!JvmtiExport::should_post_class_load()) {
>>>                        trace->num_frames = ticks_no_class_load; // -1
>>>                        return;
>>>                  -  }
>>>                  +  }*/
>>>
>>>                  ##############################
>>>                  - forte.cpp: (I know this is not part of your changes
>>> but)
>>>                  find_jmethod_id_or_null give me NULL for my test.
>>>                  It looks like we actually want the regular jmethod_id()
>>> ?
>>>
>>>                  Since we are the thread we are talking about (and in
>>> same ucontext) and thread is in vm and have a last java frame,
>>>                  I think most of the checks done in AsyncGetCallTrace is
>>> irrelevant, so you should be-able to call forte_fill_call_trace_given_top
>>> directly.
>>>                  But since we might need jmethod_id() if possible to
>>> avoid getting method id NULL,
>>>                  we need some fixes in forte code, or just do the
>>> vframStream loop inside heapMonitoring.cpp and not use forte.cpp.
>>>
>>>                  Something like:
>>>
>>>                     if (jthread->has_last_Java_frame()) { // just to be
>>> safe
>>>                       vframeStream vfst(jthread);
>>>                       while (!vfst.at_end()) {
>>>                         Method* m = vfst.method();
>>>                         m->jmethod_id();
>>>                         m->line_number_from_bci(vfst.bci());
>>>                         vfst.next();
>>>                       }
>>>
>>>                  - This is a bit confusing in forte.cpp,
>>> trace->frames[count].lineno = bci.
>>>                  Line number should be m->line_number_from_bci(bci);
>>>                  Do the heapMonitoring suppose to trace with bci or line
>>> number?
>>>                  I would say bci, meaning we should either rename
>>> ASGCT_CallFrame?lineno or use another data structure which says bci.
>>>
>>>                  ##############################
>>>                  - // TODO(jcbeyler): remove this extra code handling
>>> the extra trace for
>>>                  Please fix all these TODO's :)
>>>
>>>                  ##############################
>>>                  - heapMonitoring.hpp:
>>>                  // TODO(jcbeyler): is this algorithm acceptable in open
>>> source?
>>>
>>>                  Why is this comment here? What is the implication?
>>>                  Have you tested any simpler algorithm?
>>>
>>>                  ##############################
>>>                  - Create a sanity jtreg test.
>>> (./hotspot/make/test/JtregNative.gmk for building the agent)
>>>
>>>                  ##############################
>>>                  - monitoring_period vs HeapMonitorRate, pick rate or
>>> period.
>>>
>>>                  ##############################
>>>                  - globals.hpp
>>>                  Why is MaxHeapTraces not settable/overridable from
>>> jvmti interface? That would be handy.
>>>
>>>                  ##############################
>>>                  - jvmtiStackTraceData + ASGCT_CallFrame memory
>>>                  Are the agent suppose to loop through and free all
>>> ASGCT_CallFrame?
>>>                  Wouldn't it be better with some kinda protocol, like:
>>>                  (*jvmti)->GetLiveTraces(jvmti, &stack_traces,
>>> &num_traces);
>>>                  (*jvmti)->ReleaseTraces(jvmti, stack_traces,
>>> num_traces);
>>>
>>>                  Also using another data structure that have num_traces
>>> inside it simplifies things.
>>>                  So I'm not convinced using the async structure is the
>>> best way forward.
>>>
>>>
>>>                  I have more questions, but I think it's better if you
>>> respond and update the code first.
>>>
>>>                  Thanks!
>>>
>>>                  /Robbin
>>>
>>>
>>>                  On 04/21/2017 11:34 PM, JC Beyler wrote:
>>>
>>>                      Hi all,
>>>
>>>                      I've added size information to the allocation
>>> sampling system. This allows the callback to remember the size of each
>>> sampled allocation.
>>>         http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/ <
>>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/>
>>>         <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/ <
>>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/>>
>>>
>>>                      The new webrev.01 also adds the actual heap
>>> monitoring sampling system in files:
>>>         http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/sh
>>> are/vm/runtime/heapMonitoring.cpp.patch
>>>         <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/s
>>> hare/vm/runtime/heapMonitoring.cpp.patch>
>>>                      <http://cr.openjdk.java.net/~
>>> rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.cpp.patch
>>>         <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/s
>>> hare/vm/runtime/heapMonitoring.cpp.patch>>
>>>                      and
>>>         http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/sh
>>> are/vm/runtime/heapMonitoring.hpp.patch
>>>         <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/s
>>> hare/vm/runtime/heapMonitoring.hpp.patch>
>>>                      <http://cr.openjdk.java.net/~
>>> rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.hpp.patch
>>>         <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/s
>>> hare/vm/runtime/heapMonitoring.hpp.patch>>
>>>
>>>                      My next step is to add the GC part to the webrev,
>>> which will allow users to determine what objects are live and what are
>>> garbage.
>>>
>>>                      Thanks for your attention and let me know if there
>>> are any questions!
>>>
>>>                      Have a wonderful Friday!
>>>                      Jc
>>>
>>>                      On Mon, Apr 17, 2017 at 12:37 PM, JC Beyler <
>>> jcbeyler at google.com <mailto:jcbeyler at google.com> <mailto:
>>> jcbeyler at google.com <mailto:jcbeyler at google.com>>
>>>         <mailto:jcbeyler at google.com <mailto:jcbeyler at google.com>
>>> <mailto:jcbeyler at google.com <mailto:jcbeyler at google.com>>>> wrote:
>>>
>>>                           Hi all,
>>>
>>>                           I worked on getting a few numbers for overhead
>>> and accuracy for my feature. I'm unsure if here is the right place to
>>> provide the full data, so I
>>>         am just
>>>                      summarizing
>>>                           here for now.
>>>
>>>                           - Overhead of the feature
>>>
>>>                           Using the Dacapo benchmark (
>>> http://dacapobench.org/). My initial results are that sampling provides
>>> 2.4% with a 512k sampling, 512k being our
>>>         default setting.
>>>
>>>                           - Note: this was without the tradesoap,
>>> tradebeans and tomcat benchmarks since they did not work with my JDK9
>>> (issue between Dacapo and JDK9 it seems)
>>>                           - I want to rerun next week to ensure number
>>> stability
>>>
>>>                           - Accuracy of the feature
>>>
>>>                           I wrote a small microbenchmark that allocates
>>> from two different stacktraces at a given ratio. For example, 10% of
>>> stacktrace S1 and 90% from
>>>         stacktrace
>>>                      S2. The
>>>                           microbenchmark was run 20 times, I averaged
>>> the results and looked for accuracy. It seems that statistically it is
>>> sound since if I allocated10%
>>>         S1 and 90%
>>>                      S2, with a
>>>                           sampling rate of 512k, I obtained 9.61% S1 and
>>> 90.49% S2.
>>>
>>>                           Let me know if there are any questions on the
>>> numbers and if you'd like to see some more data.
>>>
>>>                           Note: this was done using our internal JDK8
>>> implementation since the webrev provided by
>>>         http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <
>>> http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>
>>>                      <http://cr.openjdk.java.net/~
>>> rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~r
>>> asbold/heapz/webrev.00/index.html>>
>>>                      <http://cr.openjdk.java.net/~
>>> rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~r
>>> asbold/heapz/webrev.00/index.html>
>>>         <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html
>>> <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>>> does
>>> not yet contain the whole
>>>                      implementation and therefore would have been
>>> misleading.
>>>
>>>                           Thanks,
>>>                           Jc
>>>
>>>
>>>                           On Tue, Apr 4, 2017 at 3:55 PM, JC Beyler <
>>> jcbeyler at google.com <mailto:jcbeyler at google.com> <mailto:
>>> jcbeyler at google.com
>>>         <mailto:jcbeyler at google.com>> <mailto:jcbeyler at google.com
>>> <mailto:jcbeyler at google.com> <mailto:jcbeyler at google.com <mailto:
>>> jcbeyler at google.com>>>> wrote:
>>>
>>>                               Hi all,
>>>
>>>                               To move the discussion forward, with Chuck
>>> Rasbold's help to make a webrev, we pushed this:
>>>         http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <
>>> http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>
>>>         <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html
>>> <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>>
>>>                      <http://cr.openjdk.java.net/~
>>> rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~r
>>> asbold/heapz/webrev.00/index.html>
>>>         <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html
>>> <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>>>
>>>                               415 lines changed: 399 ins; 13 del; 3 mod;
>>> 51122 unchg
>>>
>>>                               This is not a final change that does the
>>> whole proposition from the JBS entry: https://bugs.openjdk.java.net/
>>> browse/JDK-8177374
>>>         <https://bugs.openjdk.java.net/browse/JDK-8177374>
>>>                      <https://bugs.openjdk.java.net/browse/JDK-8177374 <
>>> https://bugs.openjdk.java.net/browse/JDK-8177374>>
>>>                      <https://bugs.openjdk.java.net/browse/JDK-8177374 <
>>> https://bugs.openjdk.java.net/browse/JDK-8177374> <
>>> https://bugs.openjdk.java.net/browse/JDK-8177374
>>>         <https://bugs.openjdk.java.net/browse/JDK-8177374>>>; what it
>>> does show is parts of the implementation that is
>>>                      proposed and hopefully can start the conversation
>>> going
>>>                               as I work through the details.
>>>
>>>                               For example, the changes to C2 are done
>>> here for the allocations:
>>>         http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/shar
>>> e/vm/opto/macro.cpp.patch
>>>         <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/sha
>>> re/vm/opto/macro.cpp.patch>
>>>                      <http://cr.openjdk.java.net/~
>>> rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch
>>>         <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/sha
>>> re/vm/opto/macro.cpp.patch>>
>>>                      <http://cr.openjdk.java.net/~
>>> rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch
>>>         <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/sha
>>> re/vm/opto/macro.cpp.patch>
>>>                      <http://cr.openjdk.java.net/~
>>> rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch
>>>         <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/sha
>>> re/vm/opto/macro.cpp.patch>>>
>>>
>>>                               Hopefully this all makes sense and thank
>>> you for all your future comments!
>>>                               Jc
>>>
>>>
>>>                               On Tue, Dec 13, 2016 at 1:11 PM, JC Beyler
>>> <jcbeyler at google.com <mailto:jcbeyler at google.com> <mailto:
>>> jcbeyler at google.com
>>>         <mailto:jcbeyler at google.com>> <mailto:jcbeyler at google.com
>>> <mailto:jcbeyler at google.com> <mailto:jcbeyler at google.com <mailto:
>>> jcbeyler at google.com>>>>
>>>                      wrote:
>>>
>>>                                   Hello all,
>>>
>>>                                   This is a follow-up from Jeremy's
>>> initial email from last year:
>>>         http://mail.openjdk.java.net/pipermail/serviceability-dev/20
>>> 15-June/017543.html <http://mail.openjdk.java.net/
>>> pipermail/serviceability-dev/2015-June/017543.html>
>>>                      <http://mail.openjdk.java.net
>>> /pipermail/serviceability-dev/2015-June/017543.html
>>>         <http://mail.openjdk.java.net/pipermail/serviceability-dev/2
>>> 015-June/017543.html>>
>>>                      <http://mail.openjdk.java.net
>>> /pipermail/serviceability-dev/2015-June/017543.html
>>>         <http://mail.openjdk.java.net/pipermail/serviceability-dev/2
>>> 015-June/017543.html> <http://mail.openjdk.java.net/
>>> pipermail/serviceability-dev/2015-June/017543.html
>>>         <http://mail.openjdk.java.net/pipermail/serviceability-dev/2
>>> 015-June/017543.html>>>
>>>
>>>                                   I've gone ahead and started working on
>>> preparing this and Jeremy and I went down the route of actually writing it
>>> up in JEP form:
>>>         https://bugs.openjdk.java.net/browse/JDK-8171119 <
>>> https://bugs.openjdk.java.net/browse/JDK-8171119> <
>>> https://bugs.openjdk.java.net/browse/JDK-8171119
>>>         <https://bugs.openjdk.java.net/browse/JDK-8171119>>
>>>
>>>                                   I think original conversation that
>>> happened last year in that thread still holds true:
>>>
>>>                                     - We have a patch at Google that we
>>> think others might be interested in
>>>                                        - It provides a means to
>>> understand where the allocation hotspots are at a very low overhead
>>>                                        - Since it is at a low overhead,
>>> we can leave it on by default
>>>
>>>                                   So I come to the mailing list with
>>> Jeremy's initial question:
>>>                                   "I thought I would ask if there is any
>>> interest / if I should write a JEP / if I should just forget it."
>>>
>>>                                   A year ago, it seemed some thought it
>>> was a good idea, is this still true?
>>>
>>>                                   Thanks,
>>>                                   Jc
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170612/d93a01c3/attachment-0001.html>

From mandy.chung at oracle.com  Tue Jun 13 01:58:28 2017
From: mandy.chung at oracle.com (Mandy Chung)
Date: Mon, 12 Jun 2017 18:58:28 -0700
Subject: RFR: JDK-8178508 Co-locate remaining MM tests
In-Reply-To: <325350fb-fd35-0d54-328e-e78ee3129c46@oracle.com>
References: <6d1f972d-dbec-7a79-4c1d-a960d8696a18@oracle.com>
 <a60f4f4a-40b6-fa23-894e-02600d0ce9b7@oracle.com>
 <11a3f964-070a-22f0-50fc-a2b0052a54df@oracle.com>
 <1ED1CB49-8F63-494E-AC82-5506C9BB5FC3@oracle.com>
 <FC020732-2C5C-40EA-B7E8-020A159D48F3@oracle.com>
 <325350fb-fd35-0d54-328e-e78ee3129c46@oracle.com>
Message-ID: <7B0DD73D-176E-4EFC-A01A-81D54CE0B918@oracle.com>


> On Jun 8, 2017, at 4:13 AM, Ujwal Vangapally <ujwal.vangapally at oracle.com> wrote:
> 
> Thanks for the Review Mandy, Igor, Harsha.
> 
> below is the link for new webrev incorporating review comments.
> 
> webrev: http://cr.openjdk.java.net/~uvangapally/webrev/2017/8178508/webrev.01/ <http://cr.openjdk.java.net/%7Euvangapally/webrev/2017/8178508/webrev.00/>

As you originally had, using @run main/othervm -Xmx3000M LargeHeapThresholdTest is much simpler.  I think it?s better to say @requires to run only only 64-bit platforms.

Can you move the multi-line @summary to the last tag.

Mandy

> changes in new webrev other than those mentioned in review comments:
> 
> using -Xmx3000M option on unsupported machines(like window 32bit machine containing just 2GB user space) creates unnecessary failures in previous implementation, so used ProcessTools.executeTestJava() to do a sample run of "java -Xmx3000M -version" and took decision depending on it's exit value whether to use -Xmx3000M or not in actual execution .
> 
> removed  '@requires os.maxmemory >3G'  as we really don't require more than 3GB physical memory to verify this test.
>  
> Suggestions:
> 
> will it makes more sense to add 
> @requires (os.family != "windows") | (os.simpleArch != "i586") 
> as windows 32 bit just provides 2GB for user space.
> 
> no problem with current test as it just prints 
> "Ability to use big heap thresholds has NOT been verified" 
> on win 32bit.
> 
> Thanks,
> Ujwal.
> 
> 
> On 6/5/2017 10:29 PM, Mandy Chung wrote:
>> 
>>> On Jun 5, 2017, at 9:48 AM, Mandy Chung <mandy.chung at oracle.com <mailto:mandy.chung at oracle.com>> wrote:
>>> 
>>> 
>>>>> 
>>>>> On 5/31/2017 10:32 AM, Ujwal Vangapally wrote:
>>>>>> webrev : 
>>>>>> http://cr.openjdk.java.net/~uvangapally/webrev/2017/8178508/webrev.00/ <http://cr.openjdk.java.net/%7Euvangapally/webrev/2017/8178508/webrev.00/>
>>> 
>>> The test should be in  test/java/lang/management/MemoryPoolMXBean since it?s a test for that API.  I also suggest to rename the test to LargeHeapThresholdTest (or something like that).
>> 
>>   41  * @run main/othervm -Xmx3000M -ea MX2G
>>   59                 assert i.getUsageThreshold() == TWO_G : "Usage threshold for"
>> The test should elimiate its dependency on -ea; otherwise the test may not fail when it runs standalone           without -ea.  You can eplace the assert with an if-statement to check the condition and throw a RuntimeException instead.  
>> 
>> Mandy
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170612/a7a69dc4/attachment.html>

From yasuenag at gmail.com  Tue Jun 13 05:10:14 2017
From: yasuenag at gmail.com (Yasumasa Suenaga)
Date: Tue, 13 Jun 2017 14:10:14 +0900
Subject: JDK-8151815: Could not parse core image with JSnap.
Message-ID: <CAGFVN2DU=5i6k3XU_C=q+swAvX-NxZJntpfC1WmsC_tA66j3qw@mail.gmail.com>

Hi all,

I want to discuss about JDK-8151815: Could not parse core image with JSnap.


In last year, I found JSnap cannot parse coredump and I've sent review
request for it as JDK-8151815. However it has not been reviewed yet
[1].

We've discussed about safety implementation, but we could not get consensus.
IMHO all SA tools should be handled java processes and core images,
and PerfCounter value is useful. So I fix this issue.

I uploaded new webrev for this issue. I think this patch is safety
because new flag PerfMemory::_destroyed guards double free, and all
members in PerfMemory is accessible (they are not munmap'ed)

  http://cr.openjdk.java.net/~ysuenaga/JDK-8151815/webrev.03/


Can you cooperate?


Thanks,

Yasumasa


[1] http://mail.openjdk.java.net/pipermail/serviceability-dev/2016-April/019480.html

From ujwal.vangapally at oracle.com  Tue Jun 13 08:39:29 2017
From: ujwal.vangapally at oracle.com (Ujwal Vangapally)
Date: Tue, 13 Jun 2017 14:09:29 +0530
Subject: RFR: JDK-8178508 Co-locate remaining MM tests
In-Reply-To: <7B0DD73D-176E-4EFC-A01A-81D54CE0B918@oracle.com>
References: <6d1f972d-dbec-7a79-4c1d-a960d8696a18@oracle.com>
 <a60f4f4a-40b6-fa23-894e-02600d0ce9b7@oracle.com>
 <11a3f964-070a-22f0-50fc-a2b0052a54df@oracle.com>
 <1ED1CB49-8F63-494E-AC82-5506C9BB5FC3@oracle.com>
 <FC020732-2C5C-40EA-B7E8-020A159D48F3@oracle.com>
 <325350fb-fd35-0d54-328e-e78ee3129c46@oracle.com>
 <7B0DD73D-176E-4EFC-A01A-81D54CE0B918@oracle.com>
Message-ID: <6d71f5a4-4762-a300-862e-7bd4e95676b8@oracle.com>

Hi Mandy,

made multi-line @summary as last tag.

yes it was simpler before and we can use '@requires only 64 bit', but we 
won't be able to verify it on Linux 32 bit in this case.

As it is possible to run -Xmx3000M on Linux 32 bit machines most of the 
time but can't guarantee it all the time.

Hence used ProcessTools.executeTestJava() to make decision about using 
-Xmx3000M.

I think it would be better if we can make Test run on Linux 32 bit 
machine also.

webrev : 
http://cr.openjdk.java.net/~uvangapally/webrev/2017/8178508/webrev.02/

Thanks,

Ujwal.


On 6/13/2017 7:28 AM, Mandy Chung wrote:
>
>> On Jun 8, 2017, at 4:13 AM, Ujwal Vangapally 
>> <ujwal.vangapally at oracle.com <mailto:ujwal.vangapally at oracle.com>> wrote:
>>
>> Thanks for the Review Mandy, Igor, Harsha.
>>
>> below is the link for new webrev incorporating review comments.
>>
>> webrev: 
>> http://cr.openjdk.java.net/~uvangapally/webrev/2017/8178508/webrev.01/ 
>> <http://cr.openjdk.java.net/%7Euvangapally/webrev/2017/8178508/webrev.00/>
>>
>>
>
> As you originally had, using @run main/othervm -Xmx3000M 
> LargeHeapThresholdTest is much simpler.  I think it?s better to say 
> @requires to run only only 64-bit platforms.
>
> Can you move the multi-line @summary to the last tag.
>
> Mandy
>
>> changes in new webrev other than those mentioned in review comments:
>>
>> using -Xmx3000M option on unsupported machines(like window 32bit 
>> machine containing just 2GB user space) creates unnecessary failures 
>> in previous implementation, so used ProcessTools.executeTestJava() to 
>> do a sample run of "java -Xmx3000M -version" and took decision 
>> depending on it's exit value whether to use -Xmx3000M or not in 
>> actual execution .
>>
>> removed  '@requires os.maxmemory >3G'  as we really don't require 
>> more than 3GB physical memory to verify this test.
>>
>> Suggestions:
>>
>> will it makes more sense to add
>> @requires (os.family != "windows") | (os.simpleArch != "i586")
>> as windows 32 bit just provides 2GB for user space.
>>
>> no problem with current test as it just prints
>> "Ability to use big heap thresholds has NOT been verified"
>> on win 32bit.
>>
>> Thanks,
>> Ujwal.
>>
>>
>> On 6/5/2017 10:29 PM, Mandy Chung wrote:
>>>
>>>> On Jun 5, 2017, at 9:48 AM, Mandy Chung <mandy.chung at oracle.com 
>>>> <mailto:mandy.chung at oracle.com>> wrote:
>>>>
>>>>
>>>>>>
>>>>>> On 5/31/2017 10:32 AM, Ujwal Vangapally wrote:
>>>>>>> webrev :
>>>>>>> http://cr.openjdk.java.net/~uvangapally/webrev/2017/8178508/webrev.00/ 
>>>>>>> <http://cr.openjdk.java.net/%7Euvangapally/webrev/2017/8178508/webrev.00/>
>>>>
>>>> The test should be in  test/java/lang/management/MemoryPoolMXBean 
>>>> since it?s a test for that API.  I also suggest to rename the test 
>>>> to LargeHeapThresholdTest (or something like that).
>>>
>>>    41  * @run main/othervm -Xmx3000M -ea MX2G
>>>    59                 assert i.getUsageThreshold() == TWO_G : "Usage threshold for"
>>> The test should elimiate its dependency on -ea; otherwise the test 
>>> may not fail when it runs standalone without -ea.  You can eplace 
>>> the assert with an if-statement to check the condition and throw a 
>>> RuntimeException instead.
>>>
>>> Mandy
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170613/e022af73/attachment-0001.html>

From robbin.ehn at oracle.com  Tue Jun 13 09:19:15 2017
From: robbin.ehn at oracle.com (Robbin Ehn)
Date: Tue, 13 Jun 2017 11:19:15 +0200
Subject: Low-Overhead Heap Profiling
In-Reply-To: <CAF9BGByisZchXTsw1=mLETLnOggJShRJBDxFeOUgt+criTTGuQ@mail.gmail.com>
References: <CAF9BGBx8xFYudhJ=v1_hU5=nwEcC1X0NKRmC=E-RNN6SMs1ozg@mail.gmail.com>
 <CAF9BGBxj7+C6uy4dXHO-3pNorbpO7n22-9JmevM0F11xOo+MnA@mail.gmail.com>
 <CAF9BGBxrdLVmXRTDHj5hQHJhcyrkBM6+ex1TFWi=A5z=V2FSew@mail.gmail.com>
 <CAF9BGBz81YYVUsMLw9sOMR9fH7JQmyb8d_OZ9aJ47EOAqwWhEw@mail.gmail.com>
 <2af975e6-3827-bd57-0c3d-fadd54867a67@oracle.com>
 <365499b6-3f4d-a4df-9e7e-e72a739fb26b@oracle.com>
 <CAF9BGBy0T7A-+1dejYbr6f0d4s0XR-zv-WwHYqXrbe0EtCdsxQ@mail.gmail.com>
 <fb1155c6-ccf8-843c-34fc-fc4a45fc54cc@oracle.com>
 <CAF9BGBxqmXF9OFD4F9wVb-DQOj_UfNNcBhe+ARVxJ5E4cgCC1Q@mail.gmail.com>
 <102c59b8-25b6-8c21-8eef-1de7d0bbf629@oracle.com>
 <CAF9BGBxDaCOxobJi8Uu9kwKPRpnj-KqgWLfJO2z7UX71UnNF4g@mail.gmail.com>
 <CAF9BGByisZchXTsw1=mLETLnOggJShRJBDxFeOUgt+criTTGuQ@mail.gmail.com>
Message-ID: <26a795b8-bdfd-506c-a772-ff9ce88e5f9a@oracle.com>

Hi JC,

This version really makes my happy, awesome work!

Since we got no attention from compiler and you manage to get ride of most of the compiler changes, I dropped compiler but added runtime instead.
Serguei, when you have time, please chime in.

On 06/12/2017 08:11 PM, JC Beyler wrote:
> Dear all,
> 
> I've continued working on this and have done the following webrev:
> http://cr.openjdk.java.net/~rasbold/8171119/webrev.05/
> 
> What it does:
>     - Implement the turn on/off system for the TLAB implementation
>        - Basically you can turn the system on/off using the JVMTI API (StartHeapProfiling and StopHeapProfiling here 
> http://cr.openjdk.java.net/~rasbold/8171119/webrev.05/src/share/vm/prims/jvmti.xml.patch)
>           - I've currently implemented that the system resets the profiling data at the StartHeapProfiling, this allows you to start profiling, stop profiling, and then the 
> user can query what happened during profiling as a post-processing step :)

Nice addition.

> 
>     - I've currently, for sake of simplicity and Robbin hinted it would be better for now, removed the mutex code for the data but think that will have to come back in a 
> next patch or in this one at some point

29 // Keep muxlock for now
....
237   // Protects the traces currently sampled (below).
238   volatile intptr_t _allocated_traces_lock[1];

This seems unused now, and I do not see any locks at all?

> 
>     - I've also really worked on the testing code to make it more expandable in this case
> http://cr.openjdk.java.net/~rasbold/8171119/webrev.05/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c.patch is now a bit more generic and allows to turn on/off the 
> sampling; it allows to query the existence or not of frames passed from Java world to JNI, which allows the test to define what should be seen and have the code in one place.
>            - That is done using the helper class http://cr.openjdk.java.net/~rasbold/8171119/webrev.05/test/serviceability/jvmti/HeapMonitor/MyPackage/Frame.java.patch
>               - Basically each test can now provide an array of Frames that the native library can check the internal data for a match. This allows to have various tests 
> have their own signatures, etc.

To get a more potential users it's nice with a Java API and as you say simplifies tests, good.

> 
>            - This has allowed me to add a nice test here to test the wipe out of the data:
> http://cr.openjdk.java.net/~rasbold/8171119/webrev.05/test/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorOnOffTest.java.patch
> 
>    - Finally, I've done initial overhead testing and, though my data was a bit noisy, I saw no real overhead using the Tlab approach while off. I will try to get better 
> data and more importantly, more stable data when turning it on.

I think it's key to have numbers showing ~0% performance impact here.

> 
> Things I still need to do:
>     - Have to fix that TLAB case for the FastTLABRefill
>     - Have to start looking at the data to see that it is consistent and does gather the right samples, right frequency, etc.
>     - Have to check the GC elements and what that produces
>     - Run a slowdebug run and ensure I fixed all those issues you saw Robbin

Also we will need support for some more platform, as the patch stands now, it's only the src/cpu/x86/vm/macroAssembler_x86.cpp part that needs to be done for other platforms?

Thanks!

/Robbin

> 
> Thanks for looking at the webrev and have a great week!
> Jc
> 
> On Fri, Jun 2, 2017 at 8:57 AM, JC Beyler <jcbeyler at google.com <mailto:jcbeyler at google.com>> wrote:
> 
>     Dear all,
> 
>     Thanks Robbin for the comments. I have left the MuxLocker for now and will look at removing it or as a future enhancement as you say. I did make the class extends and
>     added a TODO for myself to test this in slowdebug.
> 
>     I have also put a new webrev up that is still a work in progress but should allow us to talk about TLAB vs C1/C2 modifications.
> 
>     TLAB implementation: http://cr.openjdk.java.net/~rasbold/8171119/webrev.04/ <http://cr.openjdk.java.net/~rasbold/8171119/webrev.04/>
>     C1/C2 implementation: http://cr.openjdk.java.net/~rasbold/8171119/webrev.03/ <http://cr.openjdk.java.net/~rasbold/8171119/webrev.03/>
> 
>     What this webrev has is:
>         - I have put in a TLAB implementation, it is a WIP, and I have not yet done the qualitative/quantitative study on it vs the implementation using compilation changes
>     but the big parts are in place
>             - Caveats:
>                 - There is a TODO: http://cr.openjdk.java.net/~rasbold/8171119/webrev.04/src/cpu/x86/vm/macroAssembler_x86.cpp.patch
>     <http://cr.openjdk.java.net/~rasbold/8171119/webrev.04/src/cpu/x86/vm/macroAssembler_x86.cpp.patch>
>                     - I have not fixed the calculation in the case of a FastTLABRefill case
>                 - This is always on right now, there is no way to turn it off, that's also a TODO to be directed by the JVMTI API
> 
>          - I also have circumvented the AsyncGetCallTrace using the snippet of code you showed Robbin, it works for here/now
>                - But we might have to revisit this one day because it then does not try to get some of the native stacks and jumps directly to the Java stacks (I see cases
>     where this could be an issue)
>                - However, this has cleaned up quite a bit of the code and I have removed all mention of ASGCT and its structures now and use directly the JVMTI structures
> 
>         - GC is handled now, I have not yet done the qualitative/quantitative study on it but the big parts are in place
> 
>         - Due to the way the TLAB is called, the stack walker is now correct and produces the right stacks it seems (this is a bold sentence from my ONE JTREG test :))
> 
>     Final notes on this webrev:
>         - Have to fix that TLAB case for the FastTLABRefill
>         - Implement the turn on/off system for the TLAB implementation
>         - Have to start looking at the data to see that it is consistent and does gather the right samples, right frequency, etc.
>         - Have to check the GC elements and what that produces
>         - Run a slowdebug run and ensure I fixed all those issues you saw Robbin
> 
>     As always, your comments and feedback are greatly appreciated! Happy Friday!
>     Jc
> 
> 
>     On Tue, May 30, 2017 at 5:33 AM, Robbin Ehn <robbin.ehn at oracle.com <mailto:robbin.ehn at oracle.com>> wrote:
> 
>         Hi Jc,
> 
>         On 05/22/2017 08:47 PM, JC Beyler wrote:
> 
>             Dear all,
> 
>             I have a new webrev up:
>             http://cr.openjdk.java.net/~rasbold/8171119/webrev.03/ <http://cr.openjdk.java.net/~rasbold/8171119/webrev.03/>
> 
> 
>         I liked this!
> 
>         Two small things:
> 
>         heapMonitoring.hpp
>         class HeapMonitoring should extend AllStatic
> 
>         heapMonitoring.cpp
>         class MuxLocker should extend StackObj
>         But I think you should skip MuxLocker or push it separate generic enhancement.
> 
>         Great with the jtreg test, thanks alot!
> 
> 
>             This webrev has, I hope, fixed a lot of the comments from Robbin:
>                 - The casts normally are all C++ style
>                 - Moved this to jdk10-hs
>                    - I have not tested slowdebug yet, hopefully it does not break there
>                 - Added the garbage collection system:
>                    - Now live sampled allocations are tracked throughout their lifetime
>                       - When GC happens, it moves the sampled allocation information to two lists: recent and frequent GC lists
>                           - Those lists use the array system that the live objects were using before but have different re-use strategies
>                    - Added the JVMTI API for them via a GetFrequentGarbageTraces and GetGarbageTraces
>             - Both use the same JVMTI structures
>                    - Added the calls to them for the test, though I've kept that test simple for now:
>             http://cr.openjdk.java.net/~rasbold/8171119/webrev.03/raw_files/new/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c
>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.03/raw_files/new/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c>
>                       - As I write this, I notice my webrev is missing a final change I made to the test that calls the same ReleaseTraces to each live/garbage/frequent
>             structure. This is updated in my local repo and will get in the next webrev.
> 
>             Next steps for this work are:
>                  - Putting the TLAB implementation (yes not forgotten ;-))
>                  - Adding more testing and separate the current test system to check things a bit more thoroughly
>                  - Have not tried to circumvent AsyncGetCallTrace yet
>                  - Still have to double check the stack walker a bit more
> 
> 
>         Looking forward to this.
> 
>         Could someone from compiler take a look please?
> 
>         Thanks!
> 
>         /Robbin
> 
> 
>             Happy webrev perusal!
>             Jc
> 
> 
>             On Tue, May 16, 2017 at 5:20 AM, Robbin Ehn <robbin.ehn at oracle.com <mailto:robbin.ehn at oracle.com> <mailto:robbin.ehn at oracle.com <mailto:robbin.ehn at oracle.com>>>
>             wrote:
> 
>                  Just a few answers,
> 
>                  On 05/15/2017 06:48 PM, JC Beyler wrote:
> 
>                      Dear all,
> 
>                      I've updated the webrev to:
>             http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/ <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/>
>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/ <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/>>
>                      <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/ <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/>
>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/ <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/>>>
> 
> 
>                  I'll look at this later, thanks!
> 
> 
>                      Robbin,
>                      I believe I have addressed most of your items with webrev 02:
>                          - I added a JTreg test to show how it works:
>             http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/raw_files/new/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c
>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/raw_files/new/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c>
>                      <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/raw_files/new/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c
>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/raw_files/new/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c>>
>                      <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/raw_files/new/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c
>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/raw_files/new/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c>
>                      <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/raw_files/new/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c
>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/raw_files/new/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c>>>
>                         - I've modified the code to use its own data structures both internally and externally, this will make it easier to move out of AsyncGetCallTrace as
>             we move
>                      forward, that is still on my TODOs
>                         - I cleaned up the JVMTI API by passing a structure that handles the num_traces and put in a ReleaseTraces as well
>                         - I cleaned up other issues as well.
> 
>                      However, I have three questions, which are probably because I'm new in this community:
>                         1) My previous webrevs were based off of JDK9 by mistake. When I took JDK10 via : hg clone http://hg.openjdk.java.net/jdk10/jdk10
>             <http://hg.openjdk.java.net/jdk10/jdk10>
>                      <http://hg.openjdk.java.net/jdk10/jdk10 <http://hg.openjdk.java.net/jdk10/jdk10>> <http://hg.openjdk.java.net/jdk10/jdk10
>             <http://hg.openjdk.java.net/jdk10/jdk10> <http://hg.openjdk.java.net/jdk10/jdk10 <http://hg.openjdk.java.net/jdk10/jdk10>>> jdk10
>                             - I don't see code compatible with what you were showing (ie your patches don't make sense for that code base; ex: klass is still accessed via
>             klass() for
>                      example in collectedHeap.inline.hpp)
>                             - Would you know what is the right hg clone command so we are working on the same code base?
> 
> 
>                  We use jdk10-hs, e.g.
>                  hg tclone http://hg.openjdk.java.net/jdk10/hs <http://hg.openjdk.java.net/jdk10/hs> <http://hg.openjdk.java.net/jdk10/hs
>             <http://hg.openjdk.java.net/jdk10/hs>> 10-hs
> 
>                  There is sporadic big merges going from jdk9->jdk10->jdk10-hs and jdk10-hs->jdk10, so 10 is moving...
> 
> 
>                         2) You mentioned I was using os::malloc, new, NEW_C_HEAP_ARRAY; I cleaned out the os::malloc but which of the new vs NEW_C_HEAP_ARRAY should I use.
>             It might be
>                      that I don't understand when one uses one or the other but I see both used around the code base?
>                           - Is it that new is to be used for anything internal and NEW_C_HEAP_ARRAY anything provided to the JVMTI users outside of the JVM?
> 
> 
>                  We overload new operator when you extend correct base class, e.g. CHeapObj<mtInternal> so use 'new'
>                  But for arrays you will need the macro NEW_C_HEAP_ARRAY.
> 
> 
>                         3) Casts: same kind question: which should I use. The code was using a bit of everything, I'll refactor it entirely but I was not clear if I should
>             go to C casts
>                      or C++ casts as I see both in the codebase. What is the convention I should use?
> 
> 
>                  Just be consist, use what suites you, C++ casts might be preferable, if we are moving towards C++11.
>                  And use 'right' cast, e.g. going from Thread* to JavaThread* you should use C cast or static_cast, not reinterpret_cast I would say.
> 
> 
>                      Final notes on this webrev:
>                          - I am still missing:
>                            - Putting a TLAB implementation so that we can compare both webrevs
>                            - Have not tried to circumvent AsyncGetCallTrace
>                            - Putting in the handling of GC'd objects
>                            - Fix a stack walker issue I have seen, I think I know the problem and will test that theory out for the next webrev
> 
>                      I will work on integrating those items for the next webrev!
> 
> 
>                  Thanks!
> 
> 
>                      Thanks for your help,
>                      Jc
> 
>                      Ps:  I tested this on a new repo:
> 
>                      hg clone http://hg.openjdk.java.net/jdk10/jdk10 <http://hg.openjdk.java.net/jdk10/jdk10> <http://hg.openjdk.java.net/jdk10/jdk10
>             <http://hg.openjdk.java.net/jdk10/jdk10>> <http://hg.openjdk.java.net/jdk10/jdk10 <http://hg.openjdk.java.net/jdk10/jdk10>
>                      <http://hg.openjdk.java.net/jdk10/jdk10 <http://hg.openjdk.java.net/jdk10/jdk10>>> jdk10
>                      ... building it
>                      cd test
>                      jtreg -nativepath:<path-to-jdk10>/build/linux-x86_64-normal-server-release/support/test/hotspot/jtreg/native/lib/ -jdk
>                      <path-to-jdk10>/linux-x86_64-normal-server-release/images/jdk ../hotspot/test/serviceability/jvmti/HeapMonitor/
> 
> 
>                  I'll test it out!
> 
>                  /Robbin
> 
> 
> 
>                      On Thu, May 4, 2017 at 11:21 PM, serguei.spitsyn at oracle.com <mailto:serguei.spitsyn at oracle.com> <mailto:serguei.spitsyn at oracle.com
>             <mailto:serguei.spitsyn at oracle.com>> <mailto:serguei.spitsyn at oracle.com <mailto:serguei.spitsyn at oracle.com>
>                      <mailto:serguei.spitsyn at oracle.com <mailto:serguei.spitsyn at oracle.com>>> <serguei.spitsyn at oracle.com <mailto:serguei.spitsyn at oracle.com>
>             <mailto:serguei.spitsyn at oracle.com <mailto:serguei.spitsyn at oracle.com>> <mailto:serguei.spitsyn at oracle.com <mailto:serguei.spitsyn at oracle.com>
> 
>                      <mailto:serguei.spitsyn at oracle.com <mailto:serguei.spitsyn at oracle.com>>>> wrote:
> 
>                           Robbin,
> 
>                           Thank you for forwarding!
>                           I will review it.
> 
>                           Thanks,
>                           Serguei
> 
> 
> 
>                           On 5/4/17 02:13, Robbin Ehn wrote:
> 
>                               Hi,
> 
>                               To me the compiler changes looks what is expected.
>                               It would be good if someone from compiler could take a look at that.
>                               Added compiler to mail thread.
> 
>                               Also adding Serguei, It would be good with his view also.
> 
>                               My initial take on it, read through most of the code and took it for a ride.
> 
>                               ##############################
>                               - Regarding the compiler changes: I think we need the 'TLAB end' trickery (mentioned by Tony P)
>                               instead of a separate check for sampling in fast path for the final version.
> 
>                               ##############################
>                               - This patch I had to apply to get it compile on JDK 10:
> 
>                               diff -r ac3ded340b35 src/share/vm/gc/shared/collectedHeap.inline.hpp
>                               --- a/src/share/vm/gc/shared/collectedHeap.inline.hpp    Fri Apr 28 14:31:38 2017 +0200
>                               +++ b/src/share/vm/gc/shared/collectedHeap.inline.hpp    Thu May 04 10:22:56 2017 +0200
>                               @@ -87,3 +87,3 @@
>                                     // support for object alloc event (no-op most of the time)
>                               -    if (klass() != NULL && klass()->name() != NULL) {
>                               +    if (klass != NULL && klass->name() != NULL) {
>                                       Thread *base_thread = Thread::current();
>                               diff -r ac3ded340b35 src/share/vm/runtime/heapMonitoring.cpp
>                               --- a/src/share/vm/runtime/heapMonitoring.cpp    Fri Apr 28 14:31:38 2017 +0200
>                               +++ b/src/share/vm/runtime/heapMonitoring.cpp    Thu May 04 10:22:56 2017 +0200
>                               @@ -316,3 +316,3 @@
>                                   JavaThread *thread = reinterpret_cast<JavaThread *>(Thread::current());
>                               -  assert(o->size() << LogHeapWordSize == byte_size,
>                               +  assert(o->size() << LogHeapWordSize == (long)byte_size,
>                                          "Object size is incorrect.");
> 
>                               ##############################
>                               - This patch I had to apply to get it not asserting during slowdebug:
> 
>                               --- a/src/share/vm/runtime/heapMonitoring.cpp    Fri Apr 28 15:15:16 2017 +0200
>                               +++ b/src/share/vm/runtime/heapMonitoring.cpp    Thu May 04 10:24:25 2017 +0200
>                               @@ -32,3 +32,3 @@
>                                 // TODO(jcbeyler): should we make this into a JVMTI structure?
>                               -struct StackTraceData {
>                               +struct StackTraceData : CHeapObj<mtInternal> {
>                                   ASGCT_CallTrace *trace;
>                               @@ -143,3 +143,2 @@
>                                 StackTraceStorage::StackTraceStorage() :
>                               -    _allocated_traces(new StackTraceData*[MaxHeapTraces]),
>                                     _allocated_traces_size(MaxHeapTraces),
>                               @@ -147,2 +146,3 @@
>                                     _allocated_count(0) {
>                               +  _allocated_traces = NEW_C_HEAP_ARRAY(StackTraceData*, MaxHeapTraces, mtInternal);
>                                   memset(_allocated_traces, 0, sizeof(*_allocated_traces) * MaxHeapTraces);
>                               @@ -152,3 +152,3 @@
>                                 StackTraceStorage::~StackTraceStorage() {
>                               -  delete[] _allocated_traces;
>                               +  FREE_C_HEAP_ARRAY(StackTraceData*, _allocated_traces);
>                                 }
> 
>                               - Classes should extend correct base class for which type of memory is used for it e.g.: CHeapObj<mt????> or StackObj or AllStatic
>                               - The style in heapMonitoring.cpp is a bit different from normal vm-style, e.g. using C++ casts instead of C. You mix NEW_C_HEAP_ARRAY,
>             os::malloc and new.
>                               - In jvmtiHeapTransition.hpp you use C cast instead.
> 
>                               ##############################
>                               - This patch I had apply to get traces without setting an ?unrelated? capability
>                               - Should this not be a new capability?
> 
>                               diff -r c02a5d8785bf src/share/vm/prims/forte.cpp
>                               --- a/src/share/vm/prims/forte.cpp    Fri Apr 28 15:15:16 2017 +0200
>                               +++ b/src/share/vm/prims/forte.cpp    Thu May 04 10:24:25 2017 +0200
>                               @@ -530,6 +530,6 @@
> 
>                               -  if (!JvmtiExport::should_post_class_load()) {
>                               +/*  if (!JvmtiExport::should_post_class_load()) {
>                                     trace->num_frames = ticks_no_class_load; // -1
>                                     return;
>                               -  }
>                               +  }*/
> 
>                               ##############################
>                               - forte.cpp: (I know this is not part of your changes but)
>                               find_jmethod_id_or_null give me NULL for my test.
>                               It looks like we actually want the regular jmethod_id() ?
> 
>                               Since we are the thread we are talking about (and in same ucontext) and thread is in vm and have a last java frame,
>                               I think most of the checks done in AsyncGetCallTrace is irrelevant, so you should be-able to call forte_fill_call_trace_given_top directly.
>                               But since we might need jmethod_id() if possible to avoid getting method id NULL,
>                               we need some fixes in forte code, or just do the vframStream loop inside heapMonitoring.cpp and not use forte.cpp.
> 
>                               Something like:
> 
>                                  if (jthread->has_last_Java_frame()) { // just to be safe
>                                    vframeStream vfst(jthread);
>                                    while (!vfst.at_end()) {
>                                      Method* m = vfst.method();
>                                      m->jmethod_id();
>                                      m->line_number_from_bci(vfst.bci());
>                                      vfst.next();
>                                    }
> 
>                               - This is a bit confusing in forte.cpp, trace->frames[count].lineno = bci.
>                               Line number should be m->line_number_from_bci(bci);
>                               Do the heapMonitoring suppose to trace with bci or line number?
>                               I would say bci, meaning we should either rename ASGCT_CallFrame?lineno or use another data structure which says bci.
> 
>                               ##############################
>                               - // TODO(jcbeyler): remove this extra code handling the extra trace for
>                               Please fix all these TODO's :)
> 
>                               ##############################
>                               - heapMonitoring.hpp:
>                               // TODO(jcbeyler): is this algorithm acceptable in open source?
> 
>                               Why is this comment here? What is the implication?
>                               Have you tested any simpler algorithm?
> 
>                               ##############################
>                               - Create a sanity jtreg test. (./hotspot/make/test/JtregNative.gmk for building the agent)
> 
>                               ##############################
>                               - monitoring_period vs HeapMonitorRate, pick rate or period.
> 
>                               ##############################
>                               - globals.hpp
>                               Why is MaxHeapTraces not settable/overridable from jvmti interface? That would be handy.
> 
>                               ##############################
>                               - jvmtiStackTraceData + ASGCT_CallFrame memory
>                               Are the agent suppose to loop through and free all ASGCT_CallFrame?
>                               Wouldn't it be better with some kinda protocol, like:
>                               (*jvmti)->GetLiveTraces(jvmti, &stack_traces, &num_traces);
>                               (*jvmti)->ReleaseTraces(jvmti, stack_traces, num_traces);
> 
>                               Also using another data structure that have num_traces inside it simplifies things.
>                               So I'm not convinced using the async structure is the best way forward.
> 
> 
>                               I have more questions, but I think it's better if you respond and update the code first.
> 
>                               Thanks!
> 
>                               /Robbin
> 
> 
>                               On 04/21/2017 11:34 PM, JC Beyler wrote:
> 
>                                   Hi all,
> 
>                                   I've added size information to the allocation sampling system. This allows the callback to remember the size of each sampled allocation.
>             http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/ <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/>
>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/ <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/>>
>                      <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/ <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/>
>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/ <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/>>>
> 
>                                   The new webrev.01 also adds the actual heap monitoring sampling system in files:
>             http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.cpp.patch
>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.cpp.patch>
>                      <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.cpp.patch
>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.cpp.patch>>
>                                   <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.cpp.patch
>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.cpp.patch>
>                      <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.cpp.patch
>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.cpp.patch>>>
>                                   and
>             http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.hpp.patch
>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.hpp.patch>
>                      <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.hpp.patch
>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.hpp.patch>>
>                                   <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.hpp.patch
>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.hpp.patch>
>                      <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.hpp.patch
>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.hpp.patch>>>
> 
>                                   My next step is to add the GC part to the webrev, which will allow users to determine what objects are live and what are garbage.
> 
>                                   Thanks for your attention and let me know if there are any questions!
> 
>                                   Have a wonderful Friday!
>                                   Jc
> 
>                                   On Mon, Apr 17, 2017 at 12:37 PM, JC Beyler <jcbeyler at google.com <mailto:jcbeyler at google.com> <mailto:jcbeyler at google.com
>             <mailto:jcbeyler at google.com>> <mailto:jcbeyler at google.com <mailto:jcbeyler at google.com> <mailto:jcbeyler at google.com <mailto:jcbeyler at google.com>>>
>                      <mailto:jcbeyler at google.com <mailto:jcbeyler at google.com> <mailto:jcbeyler at google.com <mailto:jcbeyler at google.com>> <mailto:jcbeyler at google.com
>             <mailto:jcbeyler at google.com> <mailto:jcbeyler at google.com <mailto:jcbeyler at google.com>>>>> wrote:
> 
>                                        Hi all,
> 
>                                        I worked on getting a few numbers for overhead and accuracy for my feature. I'm unsure if here is the right place to provide the full
>             data, so I
>                      am just
>                                   summarizing
>                                        here for now.
> 
>                                        - Overhead of the feature
> 
>                                        Using the Dacapo benchmark (http://dacapobench.org/). My initial results are that sampling provides 2.4% with a 512k sampling, 512k
>             being our
>                      default setting.
> 
>                                        - Note: this was without the tradesoap, tradebeans and tomcat benchmarks since they did not work with my JDK9 (issue between Dacapo
>             and JDK9 it seems)
>                                        - I want to rerun next week to ensure number stability
> 
>                                        - Accuracy of the feature
> 
>                                        I wrote a small microbenchmark that allocates from two different stacktraces at a given ratio. For example, 10% of stacktrace S1 and
>             90% from
>                      stacktrace
>                                   S2. The
>                                        microbenchmark was run 20 times, I averaged the results and looked for accuracy. It seems that statistically it is sound since if I
>             allocated10%
>                      S1 and 90%
>                                   S2, with a
>                                        sampling rate of 512k, I obtained 9.61% S1 and 90.49% S2.
> 
>                                        Let me know if there are any questions on the numbers and if you'd like to see some more data.
> 
>                                        Note: this was done using our internal JDK8 implementation since the webrev provided by
>             http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>
>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>>
>                                   <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>
>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>>>
>                                   <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>
>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>>
>                      <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>
>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>>>> does not yet contain the whole
>                                   implementation and therefore would have been misleading.
> 
>                                        Thanks,
>                                        Jc
> 
> 
>                                        On Tue, Apr 4, 2017 at 3:55 PM, JC Beyler <jcbeyler at google.com <mailto:jcbeyler at google.com> <mailto:jcbeyler at google.com
>             <mailto:jcbeyler at google.com>> <mailto:jcbeyler at google.com <mailto:jcbeyler at google.com>
>                      <mailto:jcbeyler at google.com <mailto:jcbeyler at google.com>>> <mailto:jcbeyler at google.com <mailto:jcbeyler at google.com> <mailto:jcbeyler at google.com
>             <mailto:jcbeyler at google.com>> <mailto:jcbeyler at google.com <mailto:jcbeyler at google.com> <mailto:jcbeyler at google.com <mailto:jcbeyler at google.com>>>>> wrote:
> 
>                                            Hi all,
> 
>                                            To move the discussion forward, with Chuck Rasbold's help to make a webrev, we pushed this:
>             http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>
>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>>
>                      <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>
>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>>>
>                                   <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>
>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>>
>                      <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>
>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>>>>
>                                            415 lines changed: 399 ins; 13 del; 3 mod; 51122 unchg
> 
>                                            This is not a final change that does the whole proposition from the JBS entry: https://bugs.openjdk.java.net/browse/JDK-8177374
>             <https://bugs.openjdk.java.net/browse/JDK-8177374>
>                      <https://bugs.openjdk.java.net/browse/JDK-8177374 <https://bugs.openjdk.java.net/browse/JDK-8177374>>
>                                   <https://bugs.openjdk.java.net/browse/JDK-8177374 <https://bugs.openjdk.java.net/browse/JDK-8177374>
>             <https://bugs.openjdk.java.net/browse/JDK-8177374 <https://bugs.openjdk.java.net/browse/JDK-8177374>>>
>                                   <https://bugs.openjdk.java.net/browse/JDK-8177374 <https://bugs.openjdk.java.net/browse/JDK-8177374>
>             <https://bugs.openjdk.java.net/browse/JDK-8177374 <https://bugs.openjdk.java.net/browse/JDK-8177374>> <https://bugs.openjdk.java.net/browse/JDK-8177374
>             <https://bugs.openjdk.java.net/browse/JDK-8177374>
>                      <https://bugs.openjdk.java.net/browse/JDK-8177374 <https://bugs.openjdk.java.net/browse/JDK-8177374>>>>; what it does show is parts of the
>             implementation that is
>                                   proposed and hopefully can start the conversation going
>                                            as I work through the details.
> 
>                                            For example, the changes to C2 are done here for the allocations:
>             http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch
>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch>
>                      <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch
>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch>>
>                                   <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch
>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch>
>                      <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch
>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch>>>
>                                   <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch
>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch>
>                      <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch
>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch>>
>                                   <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch
>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch>
>                      <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch
>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch>>>>
> 
>                                            Hopefully this all makes sense and thank you for all your future comments!
>                                            Jc
> 
> 
>                                            On Tue, Dec 13, 2016 at 1:11 PM, JC Beyler <jcbeyler at google.com <mailto:jcbeyler at google.com> <mailto:jcbeyler at google.com
>             <mailto:jcbeyler at google.com>> <mailto:jcbeyler at google.com <mailto:jcbeyler at google.com>
>                      <mailto:jcbeyler at google.com <mailto:jcbeyler at google.com>>> <mailto:jcbeyler at google.com <mailto:jcbeyler at google.com> <mailto:jcbeyler at google.com
>             <mailto:jcbeyler at google.com>> <mailto:jcbeyler at google.com <mailto:jcbeyler at google.com> <mailto:jcbeyler at google.com <mailto:jcbeyler at google.com>>>>>
>                                   wrote:
> 
>                                                Hello all,
> 
>                                                This is a follow-up from Jeremy's initial email from last year:
>             http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-June/017543.html
>             <http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-June/017543.html>
>             <http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-June/017543.html <http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-June/017543.html>>
>                                   <http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-June/017543.html
>             <http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-June/017543.html>
>                      <http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-June/017543.html
>             <http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-June/017543.html>>>
>                                   <http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-June/017543.html
>             <http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-June/017543.html>
>                      <http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-June/017543.html
>             <http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-June/017543.html>>
>             <http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-June/017543.html <http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-June/017543.html>
>                      <http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-June/017543.html
>             <http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-June/017543.html>>>>
> 
>                                                I've gone ahead and started working on preparing this and Jeremy and I went down the route of actually writing it up in JEP form:
>             https://bugs.openjdk.java.net/browse/JDK-8171119 <https://bugs.openjdk.java.net/browse/JDK-8171119> <https://bugs.openjdk.java.net/browse/JDK-8171119
>             <https://bugs.openjdk.java.net/browse/JDK-8171119>> <https://bugs.openjdk.java.net/browse/JDK-8171119 <https://bugs.openjdk.java.net/browse/JDK-8171119>
>                      <https://bugs.openjdk.java.net/browse/JDK-8171119 <https://bugs.openjdk.java.net/browse/JDK-8171119>>>
> 
>                                                I think original conversation that happened last year in that thread still holds true:
> 
>                                                  - We have a patch at Google that we think others might be interested in
>                                                     - It provides a means to understand where the allocation hotspots are at a very low overhead
>                                                     - Since it is at a low overhead, we can leave it on by default
> 
>                                                So I come to the mailing list with Jeremy's initial question:
>                                                "I thought I would ask if there is any interest / if I should write a JEP / if I should just forget it."
> 
>                                                A year ago, it seemed some thought it was a good idea, is this still true?
> 
>                                                Thanks,
>                                                Jc
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 

From robbin.ehn at oracle.com  Tue Jun 13 09:34:00 2017
From: robbin.ehn at oracle.com (Robbin Ehn)
Date: Tue, 13 Jun 2017 11:34:00 +0200
Subject: Low-Overhead Heap Profiling
In-Reply-To: <26a795b8-bdfd-506c-a772-ff9ce88e5f9a@oracle.com>
References: <CAF9BGBx8xFYudhJ=v1_hU5=nwEcC1X0NKRmC=E-RNN6SMs1ozg@mail.gmail.com>
 <CAF9BGBxj7+C6uy4dXHO-3pNorbpO7n22-9JmevM0F11xOo+MnA@mail.gmail.com>
 <CAF9BGBxrdLVmXRTDHj5hQHJhcyrkBM6+ex1TFWi=A5z=V2FSew@mail.gmail.com>
 <CAF9BGBz81YYVUsMLw9sOMR9fH7JQmyb8d_OZ9aJ47EOAqwWhEw@mail.gmail.com>
 <2af975e6-3827-bd57-0c3d-fadd54867a67@oracle.com>
 <365499b6-3f4d-a4df-9e7e-e72a739fb26b@oracle.com>
 <CAF9BGBy0T7A-+1dejYbr6f0d4s0XR-zv-WwHYqXrbe0EtCdsxQ@mail.gmail.com>
 <fb1155c6-ccf8-843c-34fc-fc4a45fc54cc@oracle.com>
 <CAF9BGBxqmXF9OFD4F9wVb-DQOj_UfNNcBhe+ARVxJ5E4cgCC1Q@mail.gmail.com>
 <102c59b8-25b6-8c21-8eef-1de7d0bbf629@oracle.com>
 <CAF9BGBxDaCOxobJi8Uu9kwKPRpnj-KqgWLfJO2z7UX71UnNF4g@mail.gmail.com>
 <CAF9BGByisZchXTsw1=mLETLnOggJShRJBDxFeOUgt+criTTGuQ@mail.gmail.com>
 <26a795b8-bdfd-506c-a772-ff9ce88e5f9a@oracle.com>
Message-ID: <42b128d2-eb1b-f14a-5ff5-6aec9e91fcfa@oracle.com>

Hi again,

Forgot to say added Dan, he might be interested in JVMTI changes at least.

On 06/13/2017 11:19 AM, Robbin Ehn wrote:
> 
> To get a more potential users it's nice with a Java API and as you say simplifies tests, good.

Obviously this is not a public API, but the code for doing it from java will at least but out there.

Thanks!

/Robbin

> 
>>
>>            - This has allowed me to add a nice test here to test the wipe out of the data:
>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.05/test/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorOnOffTest.java.patch
>>
>>    - Finally, I've done initial overhead testing and, though my data was a bit noisy, I saw no real overhead using the Tlab approach while off. I will try to get better 
>> data and more importantly, more stable data when turning it on.
> 
> I think it's key to have numbers showing ~0% performance impact here.
> 
>>
>> Things I still need to do:
>>     - Have to fix that TLAB case for the FastTLABRefill
>>     - Have to start looking at the data to see that it is consistent and does gather the right samples, right frequency, etc.
>>     - Have to check the GC elements and what that produces
>>     - Run a slowdebug run and ensure I fixed all those issues you saw Robbin
> 
> Also we will need support for some more platform, as the patch stands now, it's only the src/cpu/x86/vm/macroAssembler_x86.cpp part that needs to be done for other platforms?
> 
> Thanks!
> 
> /Robbin
> 
>>
>> Thanks for looking at the webrev and have a great week!
>> Jc
>>
>> On Fri, Jun 2, 2017 at 8:57 AM, JC Beyler <jcbeyler at google.com <mailto:jcbeyler at google.com>> wrote:
>>
>>     Dear all,
>>
>>     Thanks Robbin for the comments. I have left the MuxLocker for now and will look at removing it or as a future enhancement as you say. I did make the class extends and
>>     added a TODO for myself to test this in slowdebug.
>>
>>     I have also put a new webrev up that is still a work in progress but should allow us to talk about TLAB vs C1/C2 modifications.
>>
>>     TLAB implementation: http://cr.openjdk.java.net/~rasbold/8171119/webrev.04/ <http://cr.openjdk.java.net/~rasbold/8171119/webrev.04/>
>>     C1/C2 implementation: http://cr.openjdk.java.net/~rasbold/8171119/webrev.03/ <http://cr.openjdk.java.net/~rasbold/8171119/webrev.03/>
>>
>>     What this webrev has is:
>>         - I have put in a TLAB implementation, it is a WIP, and I have not yet done the qualitative/quantitative study on it vs the implementation using compilation changes
>>     but the big parts are in place
>>             - Caveats:
>>                 - There is a TODO: http://cr.openjdk.java.net/~rasbold/8171119/webrev.04/src/cpu/x86/vm/macroAssembler_x86.cpp.patch
>>     <http://cr.openjdk.java.net/~rasbold/8171119/webrev.04/src/cpu/x86/vm/macroAssembler_x86.cpp.patch>
>>                     - I have not fixed the calculation in the case of a FastTLABRefill case
>>                 - This is always on right now, there is no way to turn it off, that's also a TODO to be directed by the JVMTI API
>>
>>          - I also have circumvented the AsyncGetCallTrace using the snippet of code you showed Robbin, it works for here/now
>>                - But we might have to revisit this one day because it then does not try to get some of the native stacks and jumps directly to the Java stacks (I see cases
>>     where this could be an issue)
>>                - However, this has cleaned up quite a bit of the code and I have removed all mention of ASGCT and its structures now and use directly the JVMTI structures
>>
>>         - GC is handled now, I have not yet done the qualitative/quantitative study on it but the big parts are in place
>>
>>         - Due to the way the TLAB is called, the stack walker is now correct and produces the right stacks it seems (this is a bold sentence from my ONE JTREG test :))
>>
>>     Final notes on this webrev:
>>         - Have to fix that TLAB case for the FastTLABRefill
>>         - Implement the turn on/off system for the TLAB implementation
>>         - Have to start looking at the data to see that it is consistent and does gather the right samples, right frequency, etc.
>>         - Have to check the GC elements and what that produces
>>         - Run a slowdebug run and ensure I fixed all those issues you saw Robbin
>>
>>     As always, your comments and feedback are greatly appreciated! Happy Friday!
>>     Jc
>>
>>
>>     On Tue, May 30, 2017 at 5:33 AM, Robbin Ehn <robbin.ehn at oracle.com <mailto:robbin.ehn at oracle.com>> wrote:
>>
>>         Hi Jc,
>>
>>         On 05/22/2017 08:47 PM, JC Beyler wrote:
>>
>>             Dear all,
>>
>>             I have a new webrev up:
>>             http://cr.openjdk.java.net/~rasbold/8171119/webrev.03/ <http://cr.openjdk.java.net/~rasbold/8171119/webrev.03/>
>>
>>
>>         I liked this!
>>
>>         Two small things:
>>
>>         heapMonitoring.hpp
>>         class HeapMonitoring should extend AllStatic
>>
>>         heapMonitoring.cpp
>>         class MuxLocker should extend StackObj
>>         But I think you should skip MuxLocker or push it separate generic enhancement.
>>
>>         Great with the jtreg test, thanks alot!
>>
>>
>>             This webrev has, I hope, fixed a lot of the comments from Robbin:
>>                 - The casts normally are all C++ style
>>                 - Moved this to jdk10-hs
>>                    - I have not tested slowdebug yet, hopefully it does not break there
>>                 - Added the garbage collection system:
>>                    - Now live sampled allocations are tracked throughout their lifetime
>>                       - When GC happens, it moves the sampled allocation information to two lists: recent and frequent GC lists
>>                           - Those lists use the array system that the live objects were using before but have different re-use strategies
>>                    - Added the JVMTI API for them via a GetFrequentGarbageTraces and GetGarbageTraces
>>             - Both use the same JVMTI structures
>>                    - Added the calls to them for the test, though I've kept that test simple for now:
>>             http://cr.openjdk.java.net/~rasbold/8171119/webrev.03/raw_files/new/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c
>>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.03/raw_files/new/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c>
>>                       - As I write this, I notice my webrev is missing a final change I made to the test that calls the same ReleaseTraces to each live/garbage/frequent
>>             structure. This is updated in my local repo and will get in the next webrev.
>>
>>             Next steps for this work are:
>>                  - Putting the TLAB implementation (yes not forgotten ;-))
>>                  - Adding more testing and separate the current test system to check things a bit more thoroughly
>>                  - Have not tried to circumvent AsyncGetCallTrace yet
>>                  - Still have to double check the stack walker a bit more
>>
>>
>>         Looking forward to this.
>>
>>         Could someone from compiler take a look please?
>>
>>         Thanks!
>>
>>         /Robbin
>>
>>
>>             Happy webrev perusal!
>>             Jc
>>
>>
>>             On Tue, May 16, 2017 at 5:20 AM, Robbin Ehn <robbin.ehn at oracle.com <mailto:robbin.ehn at oracle.com> <mailto:robbin.ehn at oracle.com <mailto:robbin.ehn at oracle.com>>>
>>             wrote:
>>
>>                  Just a few answers,
>>
>>                  On 05/15/2017 06:48 PM, JC Beyler wrote:
>>
>>                      Dear all,
>>
>>                      I've updated the webrev to:
>>             http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/ <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/>
>>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/ <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/>>
>>                      <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/ <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/>
>>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/ <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/>>>
>>
>>
>>                  I'll look at this later, thanks!
>>
>>
>>                      Robbin,
>>                      I believe I have addressed most of your items with webrev 02:
>>                          - I added a JTreg test to show how it works:
>>             http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/raw_files/new/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c
>>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/raw_files/new/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c>
>>                      <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/raw_files/new/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c
>>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/raw_files/new/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c>>
>>                      <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/raw_files/new/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c
>>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/raw_files/new/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c>
>>                      <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/raw_files/new/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c
>>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/raw_files/new/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c>>>
>>                         - I've modified the code to use its own data structures both internally and externally, this will make it easier to move out of AsyncGetCallTrace as
>>             we move
>>                      forward, that is still on my TODOs
>>                         - I cleaned up the JVMTI API by passing a structure that handles the num_traces and put in a ReleaseTraces as well
>>                         - I cleaned up other issues as well.
>>
>>                      However, I have three questions, which are probably because I'm new in this community:
>>                         1) My previous webrevs were based off of JDK9 by mistake. When I took JDK10 via : hg clone http://hg.openjdk.java.net/jdk10/jdk10
>>             <http://hg.openjdk.java.net/jdk10/jdk10>
>>                      <http://hg.openjdk.java.net/jdk10/jdk10 <http://hg.openjdk.java.net/jdk10/jdk10>> <http://hg.openjdk.java.net/jdk10/jdk10
>>             <http://hg.openjdk.java.net/jdk10/jdk10> <http://hg.openjdk.java.net/jdk10/jdk10 <http://hg.openjdk.java.net/jdk10/jdk10>>> jdk10
>>                             - I don't see code compatible with what you were showing (ie your patches don't make sense for that code base; ex: klass is still accessed via
>>             klass() for
>>                      example in collectedHeap.inline.hpp)
>>                             - Would you know what is the right hg clone command so we are working on the same code base?
>>
>>
>>                  We use jdk10-hs, e.g.
>>                  hg tclone http://hg.openjdk.java.net/jdk10/hs <http://hg.openjdk.java.net/jdk10/hs> <http://hg.openjdk.java.net/jdk10/hs
>>             <http://hg.openjdk.java.net/jdk10/hs>> 10-hs
>>
>>                  There is sporadic big merges going from jdk9->jdk10->jdk10-hs and jdk10-hs->jdk10, so 10 is moving...
>>
>>
>>                         2) You mentioned I was using os::malloc, new, NEW_C_HEAP_ARRAY; I cleaned out the os::malloc but which of the new vs NEW_C_HEAP_ARRAY should I use.
>>             It might be
>>                      that I don't understand when one uses one or the other but I see both used around the code base?
>>                           - Is it that new is to be used for anything internal and NEW_C_HEAP_ARRAY anything provided to the JVMTI users outside of the JVM?
>>
>>
>>                  We overload new operator when you extend correct base class, e.g. CHeapObj<mtInternal> so use 'new'
>>                  But for arrays you will need the macro NEW_C_HEAP_ARRAY.
>>
>>
>>                         3) Casts: same kind question: which should I use. The code was using a bit of everything, I'll refactor it entirely but I was not clear if I should
>>             go to C casts
>>                      or C++ casts as I see both in the codebase. What is the convention I should use?
>>
>>
>>                  Just be consist, use what suites you, C++ casts might be preferable, if we are moving towards C++11.
>>                  And use 'right' cast, e.g. going from Thread* to JavaThread* you should use C cast or static_cast, not reinterpret_cast I would say.
>>
>>
>>                      Final notes on this webrev:
>>                          - I am still missing:
>>                            - Putting a TLAB implementation so that we can compare both webrevs
>>                            - Have not tried to circumvent AsyncGetCallTrace
>>                            - Putting in the handling of GC'd objects
>>                            - Fix a stack walker issue I have seen, I think I know the problem and will test that theory out for the next webrev
>>
>>                      I will work on integrating those items for the next webrev!
>>
>>
>>                  Thanks!
>>
>>
>>                      Thanks for your help,
>>                      Jc
>>
>>                      Ps:  I tested this on a new repo:
>>
>>                      hg clone http://hg.openjdk.java.net/jdk10/jdk10 <http://hg.openjdk.java.net/jdk10/jdk10> <http://hg.openjdk.java.net/jdk10/jdk10
>>             <http://hg.openjdk.java.net/jdk10/jdk10>> <http://hg.openjdk.java.net/jdk10/jdk10 <http://hg.openjdk.java.net/jdk10/jdk10>
>>                      <http://hg.openjdk.java.net/jdk10/jdk10 <http://hg.openjdk.java.net/jdk10/jdk10>>> jdk10
>>                      ... building it
>>                      cd test
>>                      jtreg -nativepath:<path-to-jdk10>/build/linux-x86_64-normal-server-release/support/test/hotspot/jtreg/native/lib/ -jdk
>>                      <path-to-jdk10>/linux-x86_64-normal-server-release/images/jdk ../hotspot/test/serviceability/jvmti/HeapMonitor/
>>
>>
>>                  I'll test it out!
>>
>>                  /Robbin
>>
>>
>>
>>                      On Thu, May 4, 2017 at 11:21 PM, serguei.spitsyn at oracle.com <mailto:serguei.spitsyn at oracle.com> <mailto:serguei.spitsyn at oracle.com
>>             <mailto:serguei.spitsyn at oracle.com>> <mailto:serguei.spitsyn at oracle.com <mailto:serguei.spitsyn at oracle.com>
>>                      <mailto:serguei.spitsyn at oracle.com <mailto:serguei.spitsyn at oracle.com>>> <serguei.spitsyn at oracle.com <mailto:serguei.spitsyn at oracle.com>
>>             <mailto:serguei.spitsyn at oracle.com <mailto:serguei.spitsyn at oracle.com>> <mailto:serguei.spitsyn at oracle.com <mailto:serguei.spitsyn at oracle.com>
>>
>>                      <mailto:serguei.spitsyn at oracle.com <mailto:serguei.spitsyn at oracle.com>>>> wrote:
>>
>>                           Robbin,
>>
>>                           Thank you for forwarding!
>>                           I will review it.
>>
>>                           Thanks,
>>                           Serguei
>>
>>
>>
>>                           On 5/4/17 02:13, Robbin Ehn wrote:
>>
>>                               Hi,
>>
>>                               To me the compiler changes looks what is expected.
>>                               It would be good if someone from compiler could take a look at that.
>>                               Added compiler to mail thread.
>>
>>                               Also adding Serguei, It would be good with his view also.
>>
>>                               My initial take on it, read through most of the code and took it for a ride.
>>
>>                               ##############################
>>                               - Regarding the compiler changes: I think we need the 'TLAB end' trickery (mentioned by Tony P)
>>                               instead of a separate check for sampling in fast path for the final version.
>>
>>                               ##############################
>>                               - This patch I had to apply to get it compile on JDK 10:
>>
>>                               diff -r ac3ded340b35 src/share/vm/gc/shared/collectedHeap.inline.hpp
>>                               --- a/src/share/vm/gc/shared/collectedHeap.inline.hpp    Fri Apr 28 14:31:38 2017 +0200
>>                               +++ b/src/share/vm/gc/shared/collectedHeap.inline.hpp    Thu May 04 10:22:56 2017 +0200
>>                               @@ -87,3 +87,3 @@
>>                                     // support for object alloc event (no-op most of the time)
>>                               -    if (klass() != NULL && klass()->name() != NULL) {
>>                               +    if (klass != NULL && klass->name() != NULL) {
>>                                       Thread *base_thread = Thread::current();
>>                               diff -r ac3ded340b35 src/share/vm/runtime/heapMonitoring.cpp
>>                               --- a/src/share/vm/runtime/heapMonitoring.cpp    Fri Apr 28 14:31:38 2017 +0200
>>                               +++ b/src/share/vm/runtime/heapMonitoring.cpp    Thu May 04 10:22:56 2017 +0200
>>                               @@ -316,3 +316,3 @@
>>                                   JavaThread *thread = reinterpret_cast<JavaThread *>(Thread::current());
>>                               -  assert(o->size() << LogHeapWordSize == byte_size,
>>                               +  assert(o->size() << LogHeapWordSize == (long)byte_size,
>>                                          "Object size is incorrect.");
>>
>>                               ##############################
>>                               - This patch I had to apply to get it not asserting during slowdebug:
>>
>>                               --- a/src/share/vm/runtime/heapMonitoring.cpp    Fri Apr 28 15:15:16 2017 +0200
>>                               +++ b/src/share/vm/runtime/heapMonitoring.cpp    Thu May 04 10:24:25 2017 +0200
>>                               @@ -32,3 +32,3 @@
>>                                 // TODO(jcbeyler): should we make this into a JVMTI structure?
>>                               -struct StackTraceData {
>>                               +struct StackTraceData : CHeapObj<mtInternal> {
>>                                   ASGCT_CallTrace *trace;
>>                               @@ -143,3 +143,2 @@
>>                                 StackTraceStorage::StackTraceStorage() :
>>                               -    _allocated_traces(new StackTraceData*[MaxHeapTraces]),
>>                                     _allocated_traces_size(MaxHeapTraces),
>>                               @@ -147,2 +146,3 @@
>>                                     _allocated_count(0) {
>>                               +  _allocated_traces = NEW_C_HEAP_ARRAY(StackTraceData*, MaxHeapTraces, mtInternal);
>>                                   memset(_allocated_traces, 0, sizeof(*_allocated_traces) * MaxHeapTraces);
>>                               @@ -152,3 +152,3 @@
>>                                 StackTraceStorage::~StackTraceStorage() {
>>                               -  delete[] _allocated_traces;
>>                               +  FREE_C_HEAP_ARRAY(StackTraceData*, _allocated_traces);
>>                                 }
>>
>>                               - Classes should extend correct base class for which type of memory is used for it e.g.: CHeapObj<mt????> or StackObj or AllStatic
>>                               - The style in heapMonitoring.cpp is a bit different from normal vm-style, e.g. using C++ casts instead of C. You mix NEW_C_HEAP_ARRAY,
>>             os::malloc and new.
>>                               - In jvmtiHeapTransition.hpp you use C cast instead.
>>
>>                               ##############################
>>                               - This patch I had apply to get traces without setting an ?unrelated? capability
>>                               - Should this not be a new capability?
>>
>>                               diff -r c02a5d8785bf src/share/vm/prims/forte.cpp
>>                               --- a/src/share/vm/prims/forte.cpp    Fri Apr 28 15:15:16 2017 +0200
>>                               +++ b/src/share/vm/prims/forte.cpp    Thu May 04 10:24:25 2017 +0200
>>                               @@ -530,6 +530,6 @@
>>
>>                               -  if (!JvmtiExport::should_post_class_load()) {
>>                               +/*  if (!JvmtiExport::should_post_class_load()) {
>>                                     trace->num_frames = ticks_no_class_load; // -1
>>                                     return;
>>                               -  }
>>                               +  }*/
>>
>>                               ##############################
>>                               - forte.cpp: (I know this is not part of your changes but)
>>                               find_jmethod_id_or_null give me NULL for my test.
>>                               It looks like we actually want the regular jmethod_id() ?
>>
>>                               Since we are the thread we are talking about (and in same ucontext) and thread is in vm and have a last java frame,
>>                               I think most of the checks done in AsyncGetCallTrace is irrelevant, so you should be-able to call forte_fill_call_trace_given_top directly.
>>                               But since we might need jmethod_id() if possible to avoid getting method id NULL,
>>                               we need some fixes in forte code, or just do the vframStream loop inside heapMonitoring.cpp and not use forte.cpp.
>>
>>                               Something like:
>>
>>                                  if (jthread->has_last_Java_frame()) { // just to be safe
>>                                    vframeStream vfst(jthread);
>>                                    while (!vfst.at_end()) {
>>                                      Method* m = vfst.method();
>>                                      m->jmethod_id();
>>                                      m->line_number_from_bci(vfst.bci());
>>                                      vfst.next();
>>                                    }
>>
>>                               - This is a bit confusing in forte.cpp, trace->frames[count].lineno = bci.
>>                               Line number should be m->line_number_from_bci(bci);
>>                               Do the heapMonitoring suppose to trace with bci or line number?
>>                               I would say bci, meaning we should either rename ASGCT_CallFrame?lineno or use another data structure which says bci.
>>
>>                               ##############################
>>                               - // TODO(jcbeyler): remove this extra code handling the extra trace for
>>                               Please fix all these TODO's :)
>>
>>                               ##############################
>>                               - heapMonitoring.hpp:
>>                               // TODO(jcbeyler): is this algorithm acceptable in open source?
>>
>>                               Why is this comment here? What is the implication?
>>                               Have you tested any simpler algorithm?
>>
>>                               ##############################
>>                               - Create a sanity jtreg test. (./hotspot/make/test/JtregNative.gmk for building the agent)
>>
>>                               ##############################
>>                               - monitoring_period vs HeapMonitorRate, pick rate or period.
>>
>>                               ##############################
>>                               - globals.hpp
>>                               Why is MaxHeapTraces not settable/overridable from jvmti interface? That would be handy.
>>
>>                               ##############################
>>                               - jvmtiStackTraceData + ASGCT_CallFrame memory
>>                               Are the agent suppose to loop through and free all ASGCT_CallFrame?
>>                               Wouldn't it be better with some kinda protocol, like:
>>                               (*jvmti)->GetLiveTraces(jvmti, &stack_traces, &num_traces);
>>                               (*jvmti)->ReleaseTraces(jvmti, stack_traces, num_traces);
>>
>>                               Also using another data structure that have num_traces inside it simplifies things.
>>                               So I'm not convinced using the async structure is the best way forward.
>>
>>
>>                               I have more questions, but I think it's better if you respond and update the code first.
>>
>>                               Thanks!
>>
>>                               /Robbin
>>
>>
>>                               On 04/21/2017 11:34 PM, JC Beyler wrote:
>>
>>                                   Hi all,
>>
>>                                   I've added size information to the allocation sampling system. This allows the callback to remember the size of each sampled allocation.
>>             http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/ <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/>
>>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/ <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/>>
>>                      <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/ <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/>
>>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/ <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/>>>
>>
>>                                   The new webrev.01 also adds the actual heap monitoring sampling system in files:
>>             http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.cpp.patch
>>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.cpp.patch>
>>                      <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.cpp.patch
>>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.cpp.patch>>
>>                                   <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.cpp.patch
>>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.cpp.patch>
>>                      <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.cpp.patch
>>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.cpp.patch>>>
>>                                   and
>>             http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.hpp.patch
>>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.hpp.patch>
>>                      <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.hpp.patch
>>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.hpp.patch>>
>>                                   <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.hpp.patch
>>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.hpp.patch>
>>                      <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.hpp.patch
>>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.hpp.patch>>>
>>
>>                                   My next step is to add the GC part to the webrev, which will allow users to determine what objects are live and what are garbage.
>>
>>                                   Thanks for your attention and let me know if there are any questions!
>>
>>                                   Have a wonderful Friday!
>>                                   Jc
>>
>>                                   On Mon, Apr 17, 2017 at 12:37 PM, JC Beyler <jcbeyler at google.com <mailto:jcbeyler at google.com> <mailto:jcbeyler at google.com
>>             <mailto:jcbeyler at google.com>> <mailto:jcbeyler at google.com <mailto:jcbeyler at google.com> <mailto:jcbeyler at google.com <mailto:jcbeyler at google.com>>>
>>                      <mailto:jcbeyler at google.com <mailto:jcbeyler at google.com> <mailto:jcbeyler at google.com <mailto:jcbeyler at google.com>> <mailto:jcbeyler at google.com
>>             <mailto:jcbeyler at google.com> <mailto:jcbeyler at google.com <mailto:jcbeyler at google.com>>>>> wrote:
>>
>>                                        Hi all,
>>
>>                                        I worked on getting a few numbers for overhead and accuracy for my feature. I'm unsure if here is the right place to provide the full
>>             data, so I
>>                      am just
>>                                   summarizing
>>                                        here for now.
>>
>>                                        - Overhead of the feature
>>
>>                                        Using the Dacapo benchmark (http://dacapobench.org/). My initial results are that sampling provides 2.4% with a 512k sampling, 512k
>>             being our
>>                      default setting.
>>
>>                                        - Note: this was without the tradesoap, tradebeans and tomcat benchmarks since they did not work with my JDK9 (issue between Dacapo
>>             and JDK9 it seems)
>>                                        - I want to rerun next week to ensure number stability
>>
>>                                        - Accuracy of the feature
>>
>>                                        I wrote a small microbenchmark that allocates from two different stacktraces at a given ratio. For example, 10% of stacktrace S1 and
>>             90% from
>>                      stacktrace
>>                                   S2. The
>>                                        microbenchmark was run 20 times, I averaged the results and looked for accuracy. It seems that statistically it is sound since if I
>>             allocated10%
>>                      S1 and 90%
>>                                   S2, with a
>>                                        sampling rate of 512k, I obtained 9.61% S1 and 90.49% S2.
>>
>>                                        Let me know if there are any questions on the numbers and if you'd like to see some more data.
>>
>>                                        Note: this was done using our internal JDK8 implementation since the webrev provided by
>>             http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>
>>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>>
>>                                   <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>
>>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>>>
>>                                   <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>
>>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>>
>>                      <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>
>>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>>>> does not yet contain the 
>> whole
>>                                   implementation and therefore would have been misleading.
>>
>>                                        Thanks,
>>                                        Jc
>>
>>
>>                                        On Tue, Apr 4, 2017 at 3:55 PM, JC Beyler <jcbeyler at google.com <mailto:jcbeyler at google.com> <mailto:jcbeyler at google.com
>>             <mailto:jcbeyler at google.com>> <mailto:jcbeyler at google.com <mailto:jcbeyler at google.com>
>>                      <mailto:jcbeyler at google.com <mailto:jcbeyler at google.com>>> <mailto:jcbeyler at google.com <mailto:jcbeyler at google.com> <mailto:jcbeyler at google.com
>>             <mailto:jcbeyler at google.com>> <mailto:jcbeyler at google.com <mailto:jcbeyler at google.com> <mailto:jcbeyler at google.com <mailto:jcbeyler at google.com>>>>> wrote:
>>
>>                                            Hi all,
>>
>>                                            To move the discussion forward, with Chuck Rasbold's help to make a webrev, we pushed this:
>>             http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>
>>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>>
>>                      <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>
>>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>>>
>>                                   <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>
>>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>>
>>                      <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>
>>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>>>>
>>                                            415 lines changed: 399 ins; 13 del; 3 mod; 51122 unchg
>>
>>                                            This is not a final change that does the whole proposition from the JBS entry: https://bugs.openjdk.java.net/browse/JDK-8177374
>>             <https://bugs.openjdk.java.net/browse/JDK-8177374>
>>                      <https://bugs.openjdk.java.net/browse/JDK-8177374 <https://bugs.openjdk.java.net/browse/JDK-8177374>>
>>                                   <https://bugs.openjdk.java.net/browse/JDK-8177374 <https://bugs.openjdk.java.net/browse/JDK-8177374>
>>             <https://bugs.openjdk.java.net/browse/JDK-8177374 <https://bugs.openjdk.java.net/browse/JDK-8177374>>>
>>                                   <https://bugs.openjdk.java.net/browse/JDK-8177374 <https://bugs.openjdk.java.net/browse/JDK-8177374>
>>             <https://bugs.openjdk.java.net/browse/JDK-8177374 <https://bugs.openjdk.java.net/browse/JDK-8177374>> <https://bugs.openjdk.java.net/browse/JDK-8177374
>>             <https://bugs.openjdk.java.net/browse/JDK-8177374>
>>                      <https://bugs.openjdk.java.net/browse/JDK-8177374 <https://bugs.openjdk.java.net/browse/JDK-8177374>>>>; what it does show is parts of the
>>             implementation that is
>>                                   proposed and hopefully can start the conversation going
>>                                            as I work through the details.
>>
>>                                            For example, the changes to C2 are done here for the allocations:
>>             http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch
>>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch>
>>                      <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch
>>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch>>
>>                                   <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch
>>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch>
>>                      <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch
>>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch>>>
>>                                   <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch
>>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch>
>>                      <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch
>>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch>>
>>                                   <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch
>>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch>
>>                      <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch
>>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch>>>>
>>
>>                                            Hopefully this all makes sense and thank you for all your future comments!
>>                                            Jc
>>
>>
>>                                            On Tue, Dec 13, 2016 at 1:11 PM, JC Beyler <jcbeyler at google.com <mailto:jcbeyler at google.com> <mailto:jcbeyler at google.com
>>             <mailto:jcbeyler at google.com>> <mailto:jcbeyler at google.com <mailto:jcbeyler at google.com>
>>                      <mailto:jcbeyler at google.com <mailto:jcbeyler at google.com>>> <mailto:jcbeyler at google.com <mailto:jcbeyler at google.com> <mailto:jcbeyler at google.com
>>             <mailto:jcbeyler at google.com>> <mailto:jcbeyler at google.com <mailto:jcbeyler at google.com> <mailto:jcbeyler at google.com <mailto:jcbeyler at google.com>>>>>
>>                                   wrote:
>>
>>                                                Hello all,
>>
>>                                                This is a follow-up from Jeremy's initial email from last year:
>>             http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-June/017543.html
>>             <http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-June/017543.html>
>>             <http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-June/017543.html 
>> <http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-June/017543.html>>
>>                                   <http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-June/017543.html
>>             <http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-June/017543.html>
>>                      <http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-June/017543.html
>>             <http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-June/017543.html>>>
>>                                   <http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-June/017543.html
>>             <http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-June/017543.html>
>>                      <http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-June/017543.html
>>             <http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-June/017543.html>>
>>             <http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-June/017543.html 
>> <http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-June/017543.html>
>>                      <http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-June/017543.html
>>             <http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-June/017543.html>>>>
>>
>>                                                I've gone ahead and started working on preparing this and Jeremy and I went down the route of actually writing it up in JEP 
>> form:
>>             https://bugs.openjdk.java.net/browse/JDK-8171119 <https://bugs.openjdk.java.net/browse/JDK-8171119> <https://bugs.openjdk.java.net/browse/JDK-8171119
>>             <https://bugs.openjdk.java.net/browse/JDK-8171119>> <https://bugs.openjdk.java.net/browse/JDK-8171119 <https://bugs.openjdk.java.net/browse/JDK-8171119>
>>                      <https://bugs.openjdk.java.net/browse/JDK-8171119 <https://bugs.openjdk.java.net/browse/JDK-8171119>>>
>>
>>                                                I think original conversation that happened last year in that thread still holds true:
>>
>>                                                  - We have a patch at Google that we think others might be interested in
>>                                                     - It provides a means to understand where the allocation hotspots are at a very low overhead
>>                                                     - Since it is at a low overhead, we can leave it on by default
>>
>>                                                So I come to the mailing list with Jeremy's initial question:
>>                                                "I thought I would ask if there is any interest / if I should write a JEP / if I should just forget it."
>>
>>                                                A year ago, it seemed some thought it was a good idea, is this still true?
>>
>>                                                Thanks,
>>                                                Jc
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>

From thomas.stuefe at gmail.com  Tue Jun 13 13:55:37 2017
From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=)
Date: Tue, 13 Jun 2017 15:55:37 +0200
Subject: RFR(xs): 8181419: Race in jdwp invoker handling may lead to
 crashes or invalid results
In-Reply-To: <CAA-vtUwt_h6o5LjajKEGJbzD12kdYQ7cJN-41tKNFkNhs9SCug@mail.gmail.com>
References: <CAA-vtUwt_h6o5LjajKEGJbzD12kdYQ7cJN-41tKNFkNhs9SCug@mail.gmail.com>
Message-ID: <CAA-vtUw+RGyJXOpnsuaRfZW-D3Eh5RKeGOa1w7=e=CYhu7SF0g@mail.gmail.com>

Ping... Anyone?

On Thu, Jun 1, 2017 at 2:18 PM, Thomas St?fe <thomas.stuefe at gmail.com>
wrote:

> Hi all,
>
> please take a look at this proposed fix for a theoretical race in the jdwp
> library.
>
> Issue: https://bugs.openjdk.java.net/browse/JDK-8181419
> webrev: http://cr.openjdk.java.net/~stuefe/webrevs/
> 8181419-Race-in-jdwp-invoker-handling-may-lead-to-crashes-
> or-invalid-results/webrev.00/webrev/
>
> In short, this is an addition to Severin's fix to the jdwp invoke handling
> (https://bugs.openjdk.java.net/browse/JDK-8153711).
>
> We have a potential race condition where the delayed cleanup of the saved
> returnvalue object reference and the exception reference (released in
> deletePotentiallySavedGlobalRefs() ) may be overtaken by a new request
> which populates the thread request structure anew. If this happens,
> deletePotentiallySavedGlobalRefs() may actually release the return value
> / exception references of the follow up request, if that one was already
> processed.
>
> The solution I choose is safe and conservative. We still release both
> references, but use the locally saved JNI references. We just avoid
> accessing the thread local request structure after it has been cleared for
> reuse. This keeps timing and locking behaviour unchanged.
>
> I am currently running jtreg tests for com/sun/jdi on AIX and Linux.
>
> Kind Regards, Thomas
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170613/b3ee1968/attachment.html>

From thomas.schatzl at oracle.com  Tue Jun 13 15:03:46 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Tue, 13 Jun 2017 17:03:46 +0200
Subject: Low-Overhead Heap Profiling
In-Reply-To: <CAF9BGByisZchXTsw1=mLETLnOggJShRJBDxFeOUgt+criTTGuQ@mail.gmail.com>
References: <CAF9BGBx8xFYudhJ=v1_hU5=nwEcC1X0NKRmC=E-RNN6SMs1ozg@mail.gmail.com>
 <CAF9BGBxj7+C6uy4dXHO-3pNorbpO7n22-9JmevM0F11xOo+MnA@mail.gmail.com>
 <CAF9BGBxrdLVmXRTDHj5hQHJhcyrkBM6+ex1TFWi=A5z=V2FSew@mail.gmail.com>
 <CAF9BGBz81YYVUsMLw9sOMR9fH7JQmyb8d_OZ9aJ47EOAqwWhEw@mail.gmail.com>
 <2af975e6-3827-bd57-0c3d-fadd54867a67@oracle.com>
 <365499b6-3f4d-a4df-9e7e-e72a739fb26b@oracle.com>
 <CAF9BGBy0T7A-+1dejYbr6f0d4s0XR-zv-WwHYqXrbe0EtCdsxQ@mail.gmail.com>
 <fb1155c6-ccf8-843c-34fc-fc4a45fc54cc@oracle.com>
 <CAF9BGBxqmXF9OFD4F9wVb-DQOj_UfNNcBhe+ARVxJ5E4cgCC1Q@mail.gmail.com>
 <102c59b8-25b6-8c21-8eef-1de7d0bbf629@oracle.com>
 <CAF9BGBxDaCOxobJi8Uu9kwKPRpnj-KqgWLfJO2z7UX71UnNF4g@mail.gmail.com>
 <CAF9BGByisZchXTsw1=mLETLnOggJShRJBDxFeOUgt+criTTGuQ@mail.gmail.com>
Message-ID: <1497366226.2829.109.camel@oracle.com>

Hi all,

On Mon, 2017-06-12 at 11:11 -0700, JC Beyler wrote:
> Dear all,
> 
> I've continued working on this and have done the following webrev:
> http://cr.openjdk.java.net/~rasbold/8171119/webrev.05/

[...]
> Things I still need to do:
> ? ?- Have to fix that TLAB case for the FastTLABRefill
> ? ?- Have to start looking at the data to see that it is consistent
> and does gather the right samples, right frequency, etc.
> ? ?- Have to check the GC elements and what that produces
> ? ?- Run a slowdebug run and ensure I fixed all those issues you saw
> Robbin
> 
> Thanks for looking at the webrev and have a great week!

? scratching a bit on the surface of this change, so apologies for
rather shallow comments:

- macroAssembler_x86.cpp:5604: while this is compiler code, and I am
not sure this is final, please avoid littering the code with TODO
remarks :) They tend to be candidates for later wtf moments only.

Just file a CR for that.

- calling?HeapMonitoring::do_weak_oops() (which should probably be
called weak_oops_do() like other similar methods) only if string
deduplication is enabled (in g1CollectedHeap.cpp:4511) seems wrong.

The call should be at least around 6 lines up outside the if.

Preferentially in a method like process_weak_jni_handles(), including
additional logging. (No new (G1) gc phase without minimal logging :)).

Seeing the changes in referenceProcess.cpp, you need to add the call to
HeapMonitoring::do_weak_oops() exactly similar to
process_weak_jni_handles() in case there is no reference processing
done.

- psParallelCompact.cpp:2172 similar to other places where the change
adds the call to HeapMonitoring::do_weak_oops(), remove the empty line.

- the change doubles the size of
CollectedHeap::allocate_from_tlab_slow() above the "small and nice"
threshold. Maybe it could be refactored a bit.

- referenceProcessor.cpp:40: please make sure that the includes are
sorted alphabetically (I did not check the other files yet).

- referenceProcessor.cpp:261: the change should add logging about the
number of references encountered, maybe after the corresponding "JNI
weak reference count" log message.

-?threadLocalAllocBuffer.cpp:331: one more "TODO"

- threadLocalAllocBuffer.hpp: ThreadLocalAllocBuffer class
documentation should be updated about the sampling additions. I would
have no clue what the difference between "actual_end" and "end" would
be from the given information.

- threadLocalAllocBuffer.hpp:130: extra whitespace ;)

- some copyrights :)

- in heapMonitoring.hpp: there are some random comments about some code
that has been grabbed from "util/math/fastmath.[h|cc]". I can't tell
whether this is code that can be used but I assume that Noam Shazeer is
okay with that (i.e. that's all Google code).

- heapMonitoring.hpp/cpp static constant naming does not correspond to
Hotspot's. Additionally, in Hotspot static methods are cased like other
methods.

- in heapMonitoring.cpp there are a few cryptic comments at the top
that seem to refer to internal stuff that should probably be removed.

I did not think through the impact of the TLAB changes on collector
behavior yet (if there are). Also I did not check for problems with
concurrent mark and SATB/G1 (if there are).

Thanks,
? Thomas


From serguei.spitsyn at oracle.com  Tue Jun 13 20:56:20 2017
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Tue, 13 Jun 2017 13:56:20 -0700
Subject: RFR(M): 8178054: [TESTBUG] Need test for JVM TI IsModifiableModule
In-Reply-To: <44b3c266-06e4-f8d4-5b31-9115448cae73@oracle.com>
References: <1843de6c-2d60-3659-0b43-940dfdaed387@oracle.com>
 <26d4a0d5-4fa9-6e3d-fd08-1ec524f3a194@oracle.com>
 <75da7763-3c9d-a467-fa78-2b811b601ef4@oracle.com>
 <ee221a7f-daed-e9a4-c037-24787cef9f2d@oracle.com>
 <202f1d93-2a73-c616-461a-6ef74b321128@oracle.com>
 <f371fe89-5df6-2621-b3cd-b9621a4f6088@oracle.com>
 <44b3c266-06e4-f8d4-5b31-9115448cae73@oracle.com>
Message-ID: <7d394e2b-b3d4-92ed-033e-6cc99ec9350e@oracle.com>

An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170613/722230cd/attachment.html>

From serguei.spitsyn at oracle.com  Tue Jun 13 21:16:56 2017
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Tue, 13 Jun 2017 14:16:56 -0700
Subject: RFR(xs): 8181419: Race in jdwp invoker handling may lead to
 crashes or invalid results
In-Reply-To: <CAA-vtUw+RGyJXOpnsuaRfZW-D3Eh5RKeGOa1w7=e=CYhu7SF0g@mail.gmail.com>
References: <CAA-vtUwt_h6o5LjajKEGJbzD12kdYQ7cJN-41tKNFkNhs9SCug@mail.gmail.com>
 <CAA-vtUw+RGyJXOpnsuaRfZW-D3Eh5RKeGOa1w7=e=CYhu7SF0g@mail.gmail.com>
Message-ID: <9e1963f0-9d27-241a-f1de-9cfea607b91d@oracle.com>

An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170613/0ca2756e/attachment.html>

From igor.ignatyev at oracle.com  Tue Jun 13 22:07:36 2017
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Tue, 13 Jun 2017 15:07:36 -0700
Subject: RFR(M): 8178054: [TESTBUG] Need test for JVM TI IsModifiableModule
In-Reply-To: <7d394e2b-b3d4-92ed-033e-6cc99ec9350e@oracle.com>
References: <1843de6c-2d60-3659-0b43-940dfdaed387@oracle.com>
 <26d4a0d5-4fa9-6e3d-fd08-1ec524f3a194@oracle.com>
 <75da7763-3c9d-a467-fa78-2b811b601ef4@oracle.com>
 <ee221a7f-daed-e9a4-c037-24787cef9f2d@oracle.com>
 <202f1d93-2a73-c616-461a-6ef74b321128@oracle.com>
 <f371fe89-5df6-2621-b3cd-b9621a4f6088@oracle.com>
 <44b3c266-06e4-f8d4-5b31-9115448cae73@oracle.com>
 <7d394e2b-b3d4-92ed-033e-6cc99ec9350e@oracle.com>
Message-ID: <90D23DA6-0F2F-4AE6-9D3E-6819AF650C3C@oracle.com>

HI Serguei,

generally it looks good, but you still need at least one test to check that IsModifiableModule returns false than it must.

minor nit: I'd prefer to have java/lang/AssertionError instead of Exception as EXC_CNAME in libIsModifiableModuleTest.c as it clearly shows that some of test checks failed.

Thanks,
-- Igor

> On Jun 13, 2017, at 1:56 PM, serguei.spitsyn at oracle.com wrote:
> 
> PING..
> 
> This is for jdk 9.
> I have one review from Alan and need at least one more.
> Added hotspot-test and hotspot_dev_confidential mailing lists to the "To:" list.
> 
> Thanks,
> Serguei
> 
> 
> On 6/10/17 04:08, serguei.spitsyn at oracle.com <mailto:serguei.spitsyn at oracle.com> wrote:
>> 
>> 
>> On 6/10/17 00:41, Alan Bateman wrote: 
>>> 
>>> 
>>> On 09/06/2017 19:32, serguei.spitsyn at oracle.com <mailto:serguei.spitsyn at oracle.com> wrote: 
>>>> 
>>>> Ok, I removed the get_class_loader function, replaced the loader with a class and left alone the "not_a_module" parameter. 
>>>> This is the simplest way to provide a non-module jobject. 
>>>> Otherwise, the JNIEnv parameter has to be passed to the check_is_modifiable_error_codes . 
>>>> Also, I've done a small refactoring by inlining the test_is_modifiable_module body into the *_check function. 
>>>> 
>>>> New webrev: 
>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8178054-jvmti-ismodif.4/ <http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8178054-jvmti-ismodif.4/> 
>>> I think it looks good now. 
>> 
>> Thank you a lot for review and good suggestions! 
>> Serguei 
>> 
>>> 
>>> -Alan. 
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170613/e1b109aa/attachment-0001.html>

From serguei.spitsyn at oracle.com  Tue Jun 13 22:12:57 2017
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Tue, 13 Jun 2017 15:12:57 -0700
Subject: RFR(M): 8178054: [TESTBUG] Need test for JVM TI IsModifiableModule
In-Reply-To: <90D23DA6-0F2F-4AE6-9D3E-6819AF650C3C@oracle.com>
References: <1843de6c-2d60-3659-0b43-940dfdaed387@oracle.com>
 <26d4a0d5-4fa9-6e3d-fd08-1ec524f3a194@oracle.com>
 <75da7763-3c9d-a467-fa78-2b811b601ef4@oracle.com>
 <ee221a7f-daed-e9a4-c037-24787cef9f2d@oracle.com>
 <202f1d93-2a73-c616-461a-6ef74b321128@oracle.com>
 <f371fe89-5df6-2621-b3cd-b9621a4f6088@oracle.com>
 <44b3c266-06e4-f8d4-5b31-9115448cae73@oracle.com>
 <7d394e2b-b3d4-92ed-033e-6cc99ec9350e@oracle.com>
 <90D23DA6-0F2F-4AE6-9D3E-6819AF650C3C@oracle.com>
Message-ID: <1bd102a6-a179-5120-20f1-acf4c3b3d6d7@oracle.com>

An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170613/a3e4d590/attachment.html>

From igor.ignatyev at oracle.com  Tue Jun 13 22:20:13 2017
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Tue, 13 Jun 2017 15:20:13 -0700
Subject: RFR(M): 8178054: [TESTBUG] Need test for JVM TI IsModifiableModule
In-Reply-To: <1bd102a6-a179-5120-20f1-acf4c3b3d6d7@oracle.com>
References: <1843de6c-2d60-3659-0b43-940dfdaed387@oracle.com>
 <26d4a0d5-4fa9-6e3d-fd08-1ec524f3a194@oracle.com>
 <75da7763-3c9d-a467-fa78-2b811b601ef4@oracle.com>
 <ee221a7f-daed-e9a4-c037-24787cef9f2d@oracle.com>
 <202f1d93-2a73-c616-461a-6ef74b321128@oracle.com>
 <f371fe89-5df6-2621-b3cd-b9621a4f6088@oracle.com>
 <44b3c266-06e4-f8d4-5b31-9115448cae73@oracle.com>
 <7d394e2b-b3d4-92ed-033e-6cc99ec9350e@oracle.com>
 <90D23DA6-0F2F-4AE6-9D3E-6819AF650C3C@oracle.com>
 <1bd102a6-a179-5120-20f1-acf4c3b3d6d7@oracle.com>
Message-ID: <CB17A298-A9CE-42D4-8CE4-A095D7943E5D@oracle.com>

I see. I asumme that such test will be added when we introduce such modules and update IsModifiableModule method, however I'd recommend to file an RFE so we won't forget about it later.

Cheers,
-- Igor
> On Jun 13, 2017, at 3:12 PM, serguei.spitsyn at oracle.com wrote:
> 
> Hi Igor,
> 
> Thank you a lot for review!
> There are no non-modifiable modules in jdk 9.
> So that it is impossible to test that IsModifiableModule returns false.
> Thank you for the suggestion, I'll replace the Exception with java/lang/AssertionError.
> 
> Thanks,
> Serguei
> 
> 
> On 6/13/17 15:07, Igor Ignatyev wrote:
>> HI Serguei,
>> 
>> generally it looks good, but you still need at least one test to check that IsModifiableModule returns false than it must.
>> 
>> minor nit: I'd prefer to have java/lang/AssertionError instead of Exception as EXC_CNAME in libIsModifiableModuleTest.c as it clearly shows that some of test checks failed.
>> 
>> Thanks,
>> -- Igor
>> 
>>> On Jun 13, 2017, at 1:56 PM, serguei.spitsyn at oracle.com <mailto:serguei.spitsyn at oracle.com> wrote:
>>> 
>>> PING..
>>> 
>>> This is for jdk 9.
>>> I have one review from Alan and need at least one more.
>>> Added hotspot-test and hotspot_dev_confidential mailing lists to the "To:" list.
>>> 
>>> Thanks,
>>> Serguei
>>> 
>>> 
>>> On 6/10/17 04:08, serguei.spitsyn at oracle.com <mailto:serguei.spitsyn at oracle.com> wrote:
>>>> 
>>>> 
>>>> On 6/10/17 00:41, Alan Bateman wrote: 
>>>>> 
>>>>> 
>>>>> On 09/06/2017 19:32, serguei.spitsyn at oracle.com <mailto:serguei.spitsyn at oracle.com> wrote: 
>>>>>> 
>>>>>> Ok, I removed the get_class_loader function, replaced the loader with a class and left alone the "not_a_module" parameter. 
>>>>>> This is the simplest way to provide a non-module jobject. 
>>>>>> Otherwise, the JNIEnv parameter has to be passed to the check_is_modifiable_error_codes . 
>>>>>> Also, I've done a small refactoring by inlining the test_is_modifiable_module body into the *_check function. 
>>>>>> 
>>>>>> New webrev: 
>>>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8178054-jvmti-ismodif.4/ <http://cr.openjdk.java.net/%7Esspitsyn/webrevs/2017/hotspot/8178054-jvmti-ismodif.4/> 
>>>>> I think it looks good now. 
>>>> 
>>>> Thank you a lot for review and good suggestions! 
>>>> Serguei 
>>>> 
>>>>> 
>>>>> -Alan. 
>>>> 
>>> 
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170613/07bb5115/attachment.html>

From serguei.spitsyn at oracle.com  Tue Jun 13 22:37:03 2017
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Tue, 13 Jun 2017 15:37:03 -0700
Subject: RFR(M): 8178054: [TESTBUG] Need test for JVM TI IsModifiableModule
In-Reply-To: <CB17A298-A9CE-42D4-8CE4-A095D7943E5D@oracle.com>
References: <1843de6c-2d60-3659-0b43-940dfdaed387@oracle.com>
 <26d4a0d5-4fa9-6e3d-fd08-1ec524f3a194@oracle.com>
 <75da7763-3c9d-a467-fa78-2b811b601ef4@oracle.com>
 <ee221a7f-daed-e9a4-c037-24787cef9f2d@oracle.com>
 <202f1d93-2a73-c616-461a-6ef74b321128@oracle.com>
 <f371fe89-5df6-2621-b3cd-b9621a4f6088@oracle.com>
 <44b3c266-06e4-f8d4-5b31-9115448cae73@oracle.com>
 <7d394e2b-b3d4-92ed-033e-6cc99ec9350e@oracle.com>
 <90D23DA6-0F2F-4AE6-9D3E-6819AF650C3C@oracle.com>
 <1bd102a6-a179-5120-20f1-acf4c3b3d6d7@oracle.com>
 <CB17A298-A9CE-42D4-8CE4-A095D7943E5D@oracle.com>
Message-ID: <39c648ff-b0a5-a156-de34-ece4ea8fed84@oracle.com>

An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170613/722e9b03/attachment-0001.html>

From serguei.spitsyn at oracle.com  Tue Jun 13 22:42:18 2017
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Tue, 13 Jun 2017 15:42:18 -0700
Subject: RFR(M): 8178054: [TESTBUG] Need test for JVM TI IsModifiableModule
In-Reply-To: <7d394e2b-b3d4-92ed-033e-6cc99ec9350e@oracle.com>
References: <1843de6c-2d60-3659-0b43-940dfdaed387@oracle.com>
 <26d4a0d5-4fa9-6e3d-fd08-1ec524f3a194@oracle.com>
 <75da7763-3c9d-a467-fa78-2b811b601ef4@oracle.com>
 <ee221a7f-daed-e9a4-c037-24787cef9f2d@oracle.com>
 <202f1d93-2a73-c616-461a-6ef74b321128@oracle.com>
 <f371fe89-5df6-2621-b3cd-b9621a4f6088@oracle.com>
 <44b3c266-06e4-f8d4-5b31-9115448cae73@oracle.com>
 <7d394e2b-b3d4-92ed-033e-6cc99ec9350e@oracle.com>
Message-ID: <fcd01233-29bd-bf92-18e6-1193f06a78f4@oracle.com>

An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170613/78b18ce8/attachment.html>

From serguei.spitsyn at oracle.com  Tue Jun 13 23:36:43 2017
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Tue, 13 Jun 2017 16:36:43 -0700
Subject: RFR(xs): 8181419: Race in jdwp invoker handling may lead to
 crashes or invalid results
In-Reply-To: <9e1963f0-9d27-241a-f1de-9cfea607b91d@oracle.com>
References: <CAA-vtUwt_h6o5LjajKEGJbzD12kdYQ7cJN-41tKNFkNhs9SCug@mail.gmail.com>
 <CAA-vtUw+RGyJXOpnsuaRfZW-D3Eh5RKeGOa1w7=e=CYhu7SF0g@mail.gmail.com>
 <9e1963f0-9d27-241a-f1de-9cfea607b91d@oracle.com>
Message-ID: <8a82b2e6-d29a-1e1e-c16d-ff8560ee0203@oracle.com>

An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170613/c3e99a66/attachment.html>

From mandy.chung at oracle.com  Wed Jun 14 01:48:57 2017
From: mandy.chung at oracle.com (Mandy Chung)
Date: Tue, 13 Jun 2017 18:48:57 -0700
Subject: RFR: JDK-8178508 Co-locate remaining MM tests
In-Reply-To: <6d71f5a4-4762-a300-862e-7bd4e95676b8@oracle.com>
References: <6d1f972d-dbec-7a79-4c1d-a960d8696a18@oracle.com>
 <a60f4f4a-40b6-fa23-894e-02600d0ce9b7@oracle.com>
 <11a3f964-070a-22f0-50fc-a2b0052a54df@oracle.com>
 <1ED1CB49-8F63-494E-AC82-5506C9BB5FC3@oracle.com>
 <FC020732-2C5C-40EA-B7E8-020A159D48F3@oracle.com>
 <325350fb-fd35-0d54-328e-e78ee3129c46@oracle.com>
 <7B0DD73D-176E-4EFC-A01A-81D54CE0B918@oracle.com>
 <6d71f5a4-4762-a300-862e-7bd4e95676b8@oracle.com>
Message-ID: <13C2E57A-F0C9-425A-81B8-79CA10F807BD@oracle.com>


> On Jun 13, 2017, at 1:39 AM, Ujwal Vangapally <ujwal.vangapally at oracle.com> wrote:
> 
> Hi Mandy,
> 
> made multi-line @summary as last tag.
> 
> yes it was simpler before and we can use '@requires only 64 bit', but we won't be able to verify it on Linux 32 bit in this case.
> As it is possible to run -Xmx3000M on Linux 32 bit machines most of the time but can't guarantee it all the time.
> 
> Hence used ProcessTools.executeTestJava() to make decision about using -Xmx3000M.
> 
> I think it would be better if we can make Test run on Linux 32 bit machine also.
> webrev : http://cr.openjdk.java.net/~uvangapally/webrev/2017/8178508/webrev.02/ <http://cr.openjdk.java.net/~uvangapally/webrev/2017/8178508/webrev.02/>
> 

I still prefer the simplified version to use jtreg @requires to specify the platforms it should run on, like you suggest to use @requires (os.family != "windows") | (os.simpleArch != "i586") 

Mandy

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170613/02ad4a7b/attachment.html>

From ujwal.vangapally at oracle.com  Wed Jun 14 03:13:59 2017
From: ujwal.vangapally at oracle.com (Ujwal Vangapally)
Date: Wed, 14 Jun 2017 08:43:59 +0530
Subject: RFR: JDK-8178508 Co-locate remaining MM tests
In-Reply-To: <13C2E57A-F0C9-425A-81B8-79CA10F807BD@oracle.com>
References: <6d1f972d-dbec-7a79-4c1d-a960d8696a18@oracle.com>
 <a60f4f4a-40b6-fa23-894e-02600d0ce9b7@oracle.com>
 <11a3f964-070a-22f0-50fc-a2b0052a54df@oracle.com>
 <1ED1CB49-8F63-494E-AC82-5506C9BB5FC3@oracle.com>
 <FC020732-2C5C-40EA-B7E8-020A159D48F3@oracle.com>
 <325350fb-fd35-0d54-328e-e78ee3129c46@oracle.com>
 <7B0DD73D-176E-4EFC-A01A-81D54CE0B918@oracle.com>
 <6d71f5a4-4762-a300-862e-7bd4e95676b8@oracle.com>
 <13C2E57A-F0C9-425A-81B8-79CA10F807BD@oracle.com>
Message-ID: <a28d2ece-a08f-cc83-12c8-ab43ebfb237b@oracle.com>

Thanks for the review Mandy.

updated webrev with simplified version using @requires os.simpleArch=="x64".

can't use @requires (os.family != "windows") | (os.simpleArch != 
"i586")alone  as sometimes it might fail on Linux 32bit as well

webrev : 
http://cr.openjdk.java.net/~uvangapally/webrev/2017/8178508/webrev.03/

-Ujwal



On 6/14/2017 7:18 AM, Mandy Chung wrote:
>
>> On Jun 13, 2017, at 1:39 AM, Ujwal Vangapally 
>> <ujwal.vangapally at oracle.com <mailto:ujwal.vangapally at oracle.com>> wrote:
>>
>> Hi Mandy,
>>
>> made multi-line @summary as last tag.
>>
>> yes it was simpler before and we can use '@requires only 64 bit', but 
>> we won't be able to verify it on Linux 32 bit in this case.
>>
>> As it is possible to run -Xmx3000M on Linux 32 bit machines most of 
>> the time but can't guarantee it all the time.
>>
>> Hence used ProcessTools.executeTestJava() to make decision about 
>> using -Xmx3000M.
>>
>> I think it would be better if we can make Test run on Linux 32 bit 
>> machine also.
>>
>> webrev : 
>> http://cr.openjdk.java.net/~uvangapally/webrev/2017/8178508/webrev.02/
>>
>>
>
> I still prefer the simplified version to use jtreg @requires to 
> specify the platforms it should run on, like you suggest to use 
> @requires (os.family != "windows") | (os.simpleArch != "i586")
>
> Mandy
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170614/fe21aa07/attachment.html>

From mandy.chung at oracle.com  Wed Jun 14 04:10:15 2017
From: mandy.chung at oracle.com (Mandy Chung)
Date: Tue, 13 Jun 2017 21:10:15 -0700
Subject: RFR: JDK-8178508 Co-locate remaining MM tests
In-Reply-To: <a28d2ece-a08f-cc83-12c8-ab43ebfb237b@oracle.com>
References: <6d1f972d-dbec-7a79-4c1d-a960d8696a18@oracle.com>
 <a60f4f4a-40b6-fa23-894e-02600d0ce9b7@oracle.com>
 <11a3f964-070a-22f0-50fc-a2b0052a54df@oracle.com>
 <1ED1CB49-8F63-494E-AC82-5506C9BB5FC3@oracle.com>
 <FC020732-2C5C-40EA-B7E8-020A159D48F3@oracle.com>
 <325350fb-fd35-0d54-328e-e78ee3129c46@oracle.com>
 <7B0DD73D-176E-4EFC-A01A-81D54CE0B918@oracle.com>
 <6d71f5a4-4762-a300-862e-7bd4e95676b8@oracle.com>
 <13C2E57A-F0C9-425A-81B8-79CA10F807BD@oracle.com>
 <a28d2ece-a08f-cc83-12c8-ab43ebfb237b@oracle.com>
Message-ID: <6E24B6D9-22CA-4BED-B845-6302C5562619@oracle.com>


> On Jun 13, 2017, at 8:13 PM, Ujwal Vangapally <ujwal.vangapally at oracle.com> wrote:
> 
> Thanks for the review Mandy.
> 
> updated webrev with simplified version using @requires os.simpleArch=="x64".
> 
> can't use @requires (os.family != "windows") | (os.simpleArch != "i586") alone  as sometimes it might fail on Linux 32bit as well 
> 
> webrev : http://cr.openjdk.java.net/~uvangapally/webrev/2017/8178508/webrev.03 <http://cr.openjdk.java.net/~uvangapally/webrev/2017/8178508/webrev.03>
> 

This version looks good.

Mandy

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170613/5c8cec2f/attachment-0001.html>

From thomas.stuefe at gmail.com  Wed Jun 14 04:17:11 2017
From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=)
Date: Wed, 14 Jun 2017 06:17:11 +0200
Subject: RFR(xs): 8181419: Race in jdwp invoker handling may lead to
 crashes or invalid results
In-Reply-To: <8a82b2e6-d29a-1e1e-c16d-ff8560ee0203@oracle.com>
References: <CAA-vtUwt_h6o5LjajKEGJbzD12kdYQ7cJN-41tKNFkNhs9SCug@mail.gmail.com>
 <CAA-vtUw+RGyJXOpnsuaRfZW-D3Eh5RKeGOa1w7=e=CYhu7SF0g@mail.gmail.com>
 <9e1963f0-9d27-241a-f1de-9cfea607b91d@oracle.com>
 <8a82b2e6-d29a-1e1e-c16d-ff8560ee0203@oracle.com>
Message-ID: <CAA-vtUyzkw-Zb-s+3yJNHe42hQ5LxmGFt8MiOVP5t6nW1pEqfQ@mail.gmail.com>

Hi Sergey,

thank you for the review!

I ran jdi jtreg tests on both AIX and Linux x64, I had some issues but they
seemed preexisting (same issues without my patch). Could you please also
run tests you think are necessary on your platforms? Thanks!

best Regards, Thomas

On Wed, Jun 14, 2017 at 1:36 AM, serguei.spitsyn at oracle.com <
serguei.spitsyn at oracle.com> wrote:

> On 6/13/17 14:16, serguei.spitsyn at oracle.com wrote:
>
> Hi Thomas,
>
> The fix looks great to me.
>
>
> Forgot to say that the copyright comment need an update.
>
> Thanks,
> Serguei
>
>
> Thank you for identifying and fixing the problem!
>
> Thanks,
> Serguei
>
>
> On 6/13/17 06:55, Thomas St?fe wrote:
>
> Ping... Anyone?
>
> On Thu, Jun 1, 2017 at 2:18 PM, Thomas St?fe <thomas.stuefe at gmail.com>
> wrote:
>
>> Hi all,
>>
>> please take a look at this proposed fix for a theoretical race in the
>> jdwp library.
>>
>> Issue: https://bugs.openjdk.java.net/browse/JDK-8181419
>> webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8181419-
>> Race-in-jdwp-invoker-handling-may-lead-to-crashes-or-
>> invalid-results/webrev.00/webrev/
>>
>> In short, this is an addition to Severin's fix to the jdwp invoke
>> handling (https://bugs.openjdk.java.net/browse/JDK-8153711).
>>
>> We have a potential race condition where the delayed cleanup of the saved
>> returnvalue object reference and the exception reference (released
>> in deletePotentiallySavedGlobalRefs() ) may be overtaken by a new
>> request which populates the thread request structure anew. If this happens,
>> deletePotentiallySavedGlobalRefs() may actually release the return value
>> / exception references of the follow up request, if that one was already
>> processed.
>>
>> The solution I choose is safe and conservative. We still release both
>> references, but use the locally saved JNI references. We just avoid
>> accessing the thread local request structure after it has been cleared for
>> reuse. This keeps timing and locking behaviour unchanged.
>>
>> I am currently running jtreg tests for com/sun/jdi on AIX and Linux.
>>
>> Kind Regards, Thomas
>>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170614/7c40d783/attachment.html>

From yasuenag at gmail.com  Wed Jun 14 04:22:56 2017
From: yasuenag at gmail.com (Yasumasa Suenaga)
Date: Wed, 14 Jun 2017 13:22:56 +0900
Subject: JDK-8153333: [REDO] STW phases at Concurrent GC should count in
 PerfCounter
Message-ID: <CAGFVN2CoZiXRsr3McK69PSNz1bYwDvnx+LkqEaJc25c4uuDovw@mail.gmail.com>

Hi all,

I changed PerfCounter to show CGC STW phase in jstat in JDK-8151674.
However, it occurred several jtreg test failure, so it was back-outed.

I want to resume to work for this issue.

http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/hotspot/
http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/jdk/

These changes are work fine on jtreg test as below:

  hotspot/test/serviceability/tmtools/jstat
  jdk/test/sun/tools


Since JDK 9, default GC algorithm is set to G1.
So I think this change is useful to watch GC behavior through jstat.

I cannot access JPRT. Could you help?


Thanks,

Yasumasa

From Alan.Bateman at oracle.com  Wed Jun 14 07:07:40 2017
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Wed, 14 Jun 2017 08:07:40 +0100
Subject: RFR(M): 8178054: [TESTBUG] Need test for JVM TI IsModifiableModule
In-Reply-To: <39c648ff-b0a5-a156-de34-ece4ea8fed84@oracle.com>
References: <1843de6c-2d60-3659-0b43-940dfdaed387@oracle.com>
 <26d4a0d5-4fa9-6e3d-fd08-1ec524f3a194@oracle.com>
 <75da7763-3c9d-a467-fa78-2b811b601ef4@oracle.com>
 <ee221a7f-daed-e9a4-c037-24787cef9f2d@oracle.com>
 <202f1d93-2a73-c616-461a-6ef74b321128@oracle.com>
 <f371fe89-5df6-2621-b3cd-b9621a4f6088@oracle.com>
 <44b3c266-06e4-f8d4-5b31-9115448cae73@oracle.com>
 <7d394e2b-b3d4-92ed-033e-6cc99ec9350e@oracle.com>
 <90D23DA6-0F2F-4AE6-9D3E-6819AF650C3C@oracle.com>
 <1bd102a6-a179-5120-20f1-acf4c3b3d6d7@oracle.com>
 <CB17A298-A9CE-42D4-8CE4-A095D7943E5D@oracle.com>
 <39c648ff-b0a5-a156-de34-ece4ea8fed84@oracle.com>
Message-ID: <a2ca2c8c-01e0-35ca-f066-6bbd7ab9caa1@oracle.com>



On 13/06/2017 23:37, serguei.spitsyn at oracle.com wrote:
> I've filed a test enhancement and targeted it to tbd_minor:
> https://bugs.openjdk.java.net/browse/JDK-8182130
>     update the IsModifiableModule test if any module becomes 
> non-modifiable
>
I suggest closing this, it's pointless. If someday that some of the core 
modules are made non-modifable then there will be tests developed as 
part of that work.

-Alan

From chihiro.ito at oracle.com  Wed Jun 14 07:17:42 2017
From: chihiro.ito at oracle.com (chihiro ito)
Date: Wed, 14 Jun 2017 16:17:42 +0900
Subject: [10] RFR 8181647: jhsdb jstack could not output thread name
In-Reply-To: <5939133A.9040709@oracle.com>
References: <5937CB5F.4060209@oracle.com>
 <a9a45cd1-8717-ed6b-8169-9543e904c20f@oracle.com>
 <5939133A.9040709@oracle.com>
Message-ID: <5940E316.7060104@oracle.com>

Hi all,

I added a test case and modified previous patch including fix the 
copyright year to 2017.
I changed to output Java thread name next the separator lines in "jhsdb 
jstack --mixed" case as following.

----------------- 32117 -----------------
"main"
0x00007f6c8feafa82    __pthread_cond_timedwait + 0x132

Could you possibly review for this following small change? If review is 
ok, please commit this as cito.


Source:
diff --git 
a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java 
b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
--- a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
+++ b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
@@ -1,5 +1,5 @@
  /*
- * Copyright (c) 2003, 2013, Oracle and/or its affiliates. All rights 
reserved.
+ * Copyright (c) 2003, 2017, Oracle and/or its affiliates. All rights 
reserved.
   * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
   *
   * This code is free software; you can redistribute it and/or modify it
@@ -88,6 +88,12 @@
                 out.print("----------------- ");
                 out.print(th);
                 out.println(" -----------------");
+               JavaThread jthread = (JavaThread) proxyToThread.get(th);
+               if (jthread != null) {
+                   out.print("\"");
+                   out.print(jthread.getThreadName());
+                   out.println("\"");
+               }
                 while (f != null) {
                    ClosestSymbol sym = f.closestSymbolToPC();
                    Address pc = f.pc();
diff --git 
a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java 
b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
--- 
a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
+++ 
b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
@@ -1,5 +1,5 @@
  /*
- * Copyright (c) 2002, 2013, Oracle and/or its affiliates. All rights 
reserved.
+ * Copyright (c) 2002, 2017, Oracle and/or its affiliates. All rights 
reserved.
   * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
   *
   * This code is free software; you can redistribute it and/or modify it
@@ -75,7 +75,9 @@
              for (JavaThread cur = threads.first(); cur != null; cur = 
cur.next(), i++) {
                  if (cur.isJavaThread()) {
                      Address sp = cur.getLastJavaSP();
-                    tty.print("Thread ");
+                    tty.print("\"");
+                    tty.print(cur.getThreadName());
+                    tty.print("\" nid=");
                      cur.printThreadIDOn(tty);
                      tty.print(": (state = " + cur.getThreadState());
                      if (verbose) {
diff --git a/test/serviceability/sa/JhsdbThreadNameTest.java 
b/test/serviceability/sa/JhsdbThreadNameTest.java
new file mode 100644
--- /dev/null
+++ b/test/serviceability/sa/JhsdbThreadNameTest.java
@@ -0,0 +1,107 @@
+/*
+ * Copyright (c) 2017, Oracle and/or its affiliates. All rights reserved.
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+ *
+ * This code is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 only, as
+ * published by the Free Software Foundation.
+ *
+ * This code is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+ * version 2 for more details (a copy is included in the LICENSE file that
+ * accompanied this code).
+ *
+ * You should have received a copy of the GNU General Public License 
version
+ * 2 along with this work; if not, write to the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
+ *
+ * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
+ * or visit www.oracle.com if you need additional information or have any
+ * questions.
+ */
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+import java.util.function.Consumer;
+
+import jdk.test.lib.apps.LingeredApp;
+import jdk.test.lib.JDKToolLauncher;
+import jdk.test.lib.Platform;
+import jdk.test.lib.process.OutputAnalyzer;
+import jdk.test.lib.Utils;
+
+/*
+ * @test
+ * @library /test/lib
+ * @run main/othervm JhsdbThreadNameTest
+ */
+public class JhsdbThreadNameTest {
+
+    private static String notMixedModeThreadNames[] = 
{"Common-Cleaner", "Signal Dispatcher", "Finalizer", "Reference 
Handler", "main"};
+    private static String mixedModeThreadNames[] = {"C2 
CompilerThread0", "C1 CompilerThread1", "Sweeper thread", "Service Thread"};
+
+    private static void startHsdbJstack(boolean mixed) throws Exception {
+
+        LingeredApp app = null;
+
+        try {
+            List<String> vmArgs = new ArrayList<String>();
+            vmArgs.add("-Xmx10m");
+            vmArgs.addAll(Utils.getVmOptions());
+
+            app = LingeredApp.startApp(vmArgs);
+            System.out.println("Started LingeredApp with pid " + 
app.getPid());
+
+            JDKToolLauncher jhsdbLauncher = 
JDKToolLauncher.createUsingTestJDK("jhsdb");
+
+            jhsdbLauncher.addToolArg("jstack");
+            jhsdbLauncher.addToolArg("--pid");
+            jhsdbLauncher.addToolArg(Long.toString(app.getPid()));
+
+            if (mixed) {
+                jhsdbLauncher.addToolArg("--mixed");
+            }
+            ProcessBuilder pb = new ProcessBuilder();
+            pb.command(jhsdbLauncher.getCommand());
+            Process jhsdb = pb.start();
+
+            jhsdb.waitFor();
+
+            OutputAnalyzer out = new OutputAnalyzer(jhsdb);
+
+ 
Arrays.stream(notMixedModeThreadNames).map(JhsdbThreadNameTest::addQuotationMarks).forEach(out::shouldContain);
+            Consumer<String> testMethod = null;
+            if (mixed) {
+                testMethod = out::shouldContain;
+            } else {
+                testMethod = out::shouldNotContain;
+            }
+ 
Arrays.stream(mixedModeThreadNames).map(JhsdbThreadNameTest::addQuotationMarks).forEach(testMethod);
+
+        } catch (InterruptedException ie) {
+            throw new Error("Problem awaiting the child process: " + 
ie, ie);
+        } catch (Exception attachE) {
+            throw new Error("Couldn't start jhsdb, attach to 
LingeredApp or match ThreadName: " + attachE);
+
+        } finally {
+            LingeredApp.stopApp(app);
+        }
+    }
+
+    private static String addQuotationMarks(String str) {
+        return "\"" + str + "\"";
+    }
+
+    public static void main(String[] args) throws Exception {
+
+        if (!Platform.shouldSAAttach()) {
+            System.out.println("SA attach not expected to work - test 
skipped.");
+            return;
+        }
+
+        startHsdbJstack(true);
+        startHsdbJstack(false);
+    }
+}


Regards,
Chihiro


On 2017/06/08 18:04, chihiro ito wrote:
> Hi Jini,
>
> Thank you for your advices. I try to add the test case and modify the 
> copyright year to 2017.
> Basically, I agree with your idea, but I think that the separator line 
> should finally be the same as the output of the jstack command. I 
> worry which is better way.
>
> Thanks,
> Chihiro
>
> On 2017/06/08 16:42, Jini George wrote:
>> Hi Chihiro,
>>
>> Thank you for making this useful change. Your changes look good.
>>
>> It would be great though if you could add a test case for this. Could 
>> you also modify the copyright year to 2017 ? One additional 
>> suggestion: The addition of the thread name makes the separator lines 
>> unaligned in the pstack/jstack --mixed cases. Like:
>>
>> ----------------- "Service Thread" nid=16051 -----------------
>> and
>> ----------------- nid=16052 -----------------
>>
>> So I am wondering if it would make sense to have the name printed out 
>> on a separate line to keep the separator lines aligned. But this is a 
>> nit, and I would leave it to you to decide on this.
>>
>> Thanks,
>> Jini (Not a (R)eviewer)
>>
>> On 6/7/2017 3:16 PM, chihiro ito wrote:
>>> Hi all,
>>>
>>> I changed to output Java thread name in jhsdb jstack as following.
>>>
>>> jhsdb jstack --pid 25879
>>> "main" nid=25884: (state = BLOCKED)
>>>
>>> jhsdb jstack --mixed --pid 25879
>>> ----------------- "main" nid=25884 -----------------
>>>
>>> Could you possibly review for this following small change? If review 
>>> is ok, please commit this as cito.
>>>
>>> Source:
>>> diff --git 
>>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java 
>>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
>>> --- 
>>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
>>> +++ 
>>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
>>> @@ -86,6 +86,13 @@
>>>              try {
>>>                 CFrame f = cdbg.topFrameForThread(th);
>>>                 out.print("----------------- ");
>>> +               JavaThread jthread = (JavaThread) 
>>> proxyToThread.get(th);
>>> +               if (jthread != null) {
>>> +                   out.print("\"");
>>> +                   out.print(jthread.getThreadName());
>>> +                   out.print("\" ");
>>> +               }
>>> +               out.print("nid=");
>>>                 out.print(th);
>>>                 out.println(" -----------------");
>>>                 while (f != null) {
>>> diff --git 
>>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java 
>>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java 
>>>
>>> --- 
>>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
>>> +++ 
>>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
>>> @@ -75,7 +75,9 @@
>>>              for (JavaThread cur = threads.first(); cur != null; cur 
>>> = cur.next(), i++) {
>>>                  if (cur.isJavaThread()) {
>>>                      Address sp = cur.getLastJavaSP();
>>> -                    tty.print("Thread ");
>>> +                    tty.print("\"");
>>> +                    tty.print(cur.getThreadName());
>>> +                    tty.print("\" nid=");
>>>                      cur.printThreadIDOn(tty);
>>>                      tty.print(": (state = " + cur.getThreadState());
>>>                      if (verbose) {
>>>
>>> Regards,
>>> Chihiro
>>>
>>
>

-- 

Chihiro Ito | Principal Consultant | +81.90.6148.8815
Oracle <http://www.oracle.com> Consultant
ORACLE Japan | Akasaka Center Bldg. | Motoakasaka 1-3-13 | 1070051 
Minato-ku, Tokyo, JAPAN

Oracle is committed to developing practices and products that help 
protect the environment <http://www.oracle.com/commitment>

From serguei.spitsyn at oracle.com  Wed Jun 14 07:22:05 2017
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Wed, 14 Jun 2017 00:22:05 -0700
Subject: RFR(M): 8178054: [TESTBUG] Need test for JVM TI IsModifiableModule
In-Reply-To: <a2ca2c8c-01e0-35ca-f066-6bbd7ab9caa1@oracle.com>
References: <1843de6c-2d60-3659-0b43-940dfdaed387@oracle.com>
 <26d4a0d5-4fa9-6e3d-fd08-1ec524f3a194@oracle.com>
 <75da7763-3c9d-a467-fa78-2b811b601ef4@oracle.com>
 <ee221a7f-daed-e9a4-c037-24787cef9f2d@oracle.com>
 <202f1d93-2a73-c616-461a-6ef74b321128@oracle.com>
 <f371fe89-5df6-2621-b3cd-b9621a4f6088@oracle.com>
 <44b3c266-06e4-f8d4-5b31-9115448cae73@oracle.com>
 <7d394e2b-b3d4-92ed-033e-6cc99ec9350e@oracle.com>
 <90D23DA6-0F2F-4AE6-9D3E-6819AF650C3C@oracle.com>
 <1bd102a6-a179-5120-20f1-acf4c3b3d6d7@oracle.com>
 <CB17A298-A9CE-42D4-8CE4-A095D7943E5D@oracle.com>
 <39c648ff-b0a5-a156-de34-ece4ea8fed84@oracle.com>
 <a2ca2c8c-01e0-35ca-f066-6bbd7ab9caa1@oracle.com>
Message-ID: <1b812362-d2e7-41b1-ee39-d039cbea323f@oracle.com>

Done.

Thanks,
Serguei


On 6/14/17 00:07, Alan Bateman wrote:
>
>
> On 13/06/2017 23:37, serguei.spitsyn at oracle.com wrote:
>> I've filed a test enhancement and targeted it to tbd_minor:
>> https://bugs.openjdk.java.net/browse/JDK-8182130
>>     update the IsModifiableModule test if any module becomes 
>> non-modifiable
>>
> I suggest closing this, it's pointless. If someday that some of the 
> core modules are made non-modifable then there will be tests developed 
> as part of that work.
>
> -Alan


From ecki at zusammenkunft.net  Wed Jun 14 07:51:10 2017
From: ecki at zusammenkunft.net (Bernd Eckenfels)
Date: Wed, 14 Jun 2017 07:51:10 +0000
Subject: [10] RFR 8181647: jhsdb jstack could not output thread name
In-Reply-To: <5940E316.7060104@oracle.com>
References: <5937CB5F.4060209@oracle.com>
 <a9a45cd1-8717-ed6b-8169-9543e904c20f@oracle.com>
 <5939133A.9040709@oracle.com>,<5940E316.7060104@oracle.com>
Message-ID: <HE1PR08MB2795EF131855EC452EAF9D3FFFC30@HE1PR08MB2795.eurprd08.prod.outlook.com>

I don't understand why this format is totally different from the normal stack traces? At least the header with the stack names could be similar?

Gruss
Bernd
--
http://bernd.eckenfels.net
________________________________
From: serviceability-dev <serviceability-dev-bounces at openjdk.java.net> on behalf of chihiro ito <chihiro.ito at oracle.com>
Sent: Wednesday, June 14, 2017 9:17:42 AM
To: Jini George; serviceability-dev at openjdk.java.net
Subject: Re: [10] RFR 8181647: jhsdb jstack could not output thread name

Hi all,

I added a test case and modified previous patch including fix the
copyright year to 2017.
I changed to output Java thread name next the separator lines in "jhsdb
jstack --mixed" case as following.

----------------- 32117 -----------------
"main"
0x00007f6c8feafa82    __pthread_cond_timedwait + 0x132

Could you possibly review for this following small change? If review is
ok, please commit this as cito.


Source:
diff --git
a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
--- a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
+++ b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
@@ -1,5 +1,5 @@
  /*
- * Copyright (c) 2003, 2013, Oracle and/or its affiliates. All rights
reserved.
+ * Copyright (c) 2003, 2017, Oracle and/or its affiliates. All rights
reserved.
   * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
   *
   * This code is free software; you can redistribute it and/or modify it
@@ -88,6 +88,12 @@
                 out.print("----------------- ");
                 out.print(th);
                 out.println(" -----------------");
+               JavaThread jthread = (JavaThread) proxyToThread.get(th);
+               if (jthread != null) {
+                   out.print("\"");
+                   out.print(jthread.getThreadName());
+                   out.println("\"");
+               }
                 while (f != null) {
                    ClosestSymbol sym = f.closestSymbolToPC();
                    Address pc = f.pc();
diff --git
a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
---
a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
+++
b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
@@ -1,5 +1,5 @@
  /*
- * Copyright (c) 2002, 2013, Oracle and/or its affiliates. All rights
reserved.
+ * Copyright (c) 2002, 2017, Oracle and/or its affiliates. All rights
reserved.
   * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
   *
   * This code is free software; you can redistribute it and/or modify it
@@ -75,7 +75,9 @@
              for (JavaThread cur = threads.first(); cur != null; cur =
cur.next(), i++) {
                  if (cur.isJavaThread()) {
                      Address sp = cur.getLastJavaSP();
-                    tty.print("Thread ");
+                    tty.print("\"");
+                    tty.print(cur.getThreadName());
+                    tty.print("\" nid=");
                      cur.printThreadIDOn(tty);
                      tty.print(": (state = " + cur.getThreadState());
                      if (verbose) {
diff --git a/test/serviceability/sa/JhsdbThreadNameTest.java
b/test/serviceability/sa/JhsdbThreadNameTest.java
new file mode 100644
--- /dev/null
+++ b/test/serviceability/sa/JhsdbThreadNameTest.java
@@ -0,0 +1,107 @@
+/*
+ * Copyright (c) 2017, Oracle and/or its affiliates. All rights reserved.
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+ *
+ * This code is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 only, as
+ * published by the Free Software Foundation.
+ *
+ * This code is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+ * version 2 for more details (a copy is included in the LICENSE file that
+ * accompanied this code).
+ *
+ * You should have received a copy of the GNU General Public License
version
+ * 2 along with this work; if not, write to the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
+ *
+ * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
+ * or visit www.oracle.com<http://www.oracle.com> if you need additional information or have any
+ * questions.
+ */
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+import java.util.function.Consumer;
+
+import jdk.test.lib.apps.LingeredApp;
+import jdk.test.lib.JDKToolLauncher;
+import jdk.test.lib.Platform;
+import jdk.test.lib.process.OutputAnalyzer;
+import jdk.test.lib.Utils;
+
+/*
+ * @test
+ * @library /test/lib
+ * @run main/othervm JhsdbThreadNameTest
+ */
+public class JhsdbThreadNameTest {
+
+    private static String notMixedModeThreadNames[] =
{"Common-Cleaner", "Signal Dispatcher", "Finalizer", "Reference
Handler", "main"};
+    private static String mixedModeThreadNames[] = {"C2
CompilerThread0", "C1 CompilerThread1", "Sweeper thread", "Service Thread"};
+
+    private static void startHsdbJstack(boolean mixed) throws Exception {
+
+        LingeredApp app = null;
+
+        try {
+            List<String> vmArgs = new ArrayList<String>();
+            vmArgs.add("-Xmx10m");
+            vmArgs.addAll(Utils.getVmOptions());
+
+            app = LingeredApp.startApp(vmArgs);
+            System.out.println("Started LingeredApp with pid " +
app.getPid());
+
+            JDKToolLauncher jhsdbLauncher =
JDKToolLauncher.createUsingTestJDK("jhsdb");
+
+            jhsdbLauncher.addToolArg("jstack");
+            jhsdbLauncher.addToolArg("--pid");
+            jhsdbLauncher.addToolArg(Long.toString(app.getPid()));
+
+            if (mixed) {
+                jhsdbLauncher.addToolArg("--mixed");
+            }
+            ProcessBuilder pb = new ProcessBuilder();
+            pb.command(jhsdbLauncher.getCommand());
+            Process jhsdb = pb.start();
+
+            jhsdb.waitFor();
+
+            OutputAnalyzer out = new OutputAnalyzer(jhsdb);
+
+
Arrays.stream(notMixedModeThreadNames).map(JhsdbThreadNameTest::addQuotationMarks).forEach(out::shouldContain);
+            Consumer<String> testMethod = null;
+            if (mixed) {
+                testMethod = out::shouldContain;
+            } else {
+                testMethod = out::shouldNotContain;
+            }
+
Arrays.stream(mixedModeThreadNames).map(JhsdbThreadNameTest::addQuotationMarks).forEach(testMethod);
+
+        } catch (InterruptedException ie) {
+            throw new Error("Problem awaiting the child process: " +
ie, ie);
+        } catch (Exception attachE) {
+            throw new Error("Couldn't start jhsdb, attach to
LingeredApp or match ThreadName: " + attachE);
+
+        } finally {
+            LingeredApp.stopApp(app);
+        }
+    }
+
+    private static String addQuotationMarks(String str) {
+        return "\"" + str + "\"";
+    }
+
+    public static void main(String[] args) throws Exception {
+
+        if (!Platform.shouldSAAttach()) {
+            System.out.println("SA attach not expected to work - test
skipped.");
+            return;
+        }
+
+        startHsdbJstack(true);
+        startHsdbJstack(false);
+    }
+}


Regards,
Chihiro


On 2017/06/08 18:04, chihiro ito wrote:
> Hi Jini,
>
> Thank you for your advices. I try to add the test case and modify the
> copyright year to 2017.
> Basically, I agree with your idea, but I think that the separator line
> should finally be the same as the output of the jstack command. I
> worry which is better way.
>
> Thanks,
> Chihiro
>
> On 2017/06/08 16:42, Jini George wrote:
>> Hi Chihiro,
>>
>> Thank you for making this useful change. Your changes look good.
>>
>> It would be great though if you could add a test case for this. Could
>> you also modify the copyright year to 2017 ? One additional
>> suggestion: The addition of the thread name makes the separator lines
>> unaligned in the pstack/jstack --mixed cases. Like:
>>
>> ----------------- "Service Thread" nid=16051 -----------------
>> and
>> ----------------- nid=16052 -----------------
>>
>> So I am wondering if it would make sense to have the name printed out
>> on a separate line to keep the separator lines aligned. But this is a
>> nit, and I would leave it to you to decide on this.
>>
>> Thanks,
>> Jini (Not a (R)eviewer)
>>
>> On 6/7/2017 3:16 PM, chihiro ito wrote:
>>> Hi all,
>>>
>>> I changed to output Java thread name in jhsdb jstack as following.
>>>
>>> jhsdb jstack --pid 25879
>>> "main" nid=25884: (state = BLOCKED)
>>>
>>> jhsdb jstack --mixed --pid 25879
>>> ----------------- "main" nid=25884 -----------------
>>>
>>> Could you possibly review for this following small change? If review
>>> is ok, please commit this as cito.
>>>
>>> Source:
>>> diff --git
>>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
>>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
>>> ---
>>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
>>> +++
>>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
>>> @@ -86,6 +86,13 @@
>>>              try {
>>>                 CFrame f = cdbg.topFrameForThread(th);
>>>                 out.print("----------------- ");
>>> +               JavaThread jthread = (JavaThread)
>>> proxyToThread.get(th);
>>> +               if (jthread != null) {
>>> +                   out.print("\"");
>>> +                   out.print(jthread.getThreadName());
>>> +                   out.print("\" ");
>>> +               }
>>> +               out.print("nid=");
>>>                 out.print(th);
>>>                 out.println(" -----------------");
>>>                 while (f != null) {
>>> diff --git
>>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
>>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
>>>
>>> ---
>>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
>>> +++
>>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
>>> @@ -75,7 +75,9 @@
>>>              for (JavaThread cur = threads.first(); cur != null; cur
>>> = cur.next(), i++) {
>>>                  if (cur.isJavaThread()) {
>>>                      Address sp = cur.getLastJavaSP();
>>> -                    tty.print("Thread ");
>>> +                    tty.print("\"");
>>> +                    tty.print(cur.getThreadName());
>>> +                    tty.print("\" nid=");
>>>                      cur.printThreadIDOn(tty);
>>>                      tty.print(": (state = " + cur.getThreadState());
>>>                      if (verbose) {
>>>
>>> Regards,
>>> Chihiro
>>>
>>
>

--

Chihiro Ito | Principal Consultant | +81.90.6148.8815
Oracle <http://www.oracle.com> Consultant
ORACLE Japan | Akasaka Center Bldg. | Motoakasaka 1-3-13 | 1070051
Minato-ku, Tokyo, JAPAN

Oracle is committed to developing practices and products that help
protect the environment <http://www.oracle.com/commitment>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170614/92db301a/attachment-0001.html>

From ecki at zusammenkunft.net  Wed Jun 14 15:40:05 2017
From: ecki at zusammenkunft.net (Bernd)
Date: Wed, 14 Jun 2017 17:40:05 +0200
Subject: Missing Class Informations in heapdumps?
Message-ID: <CABOR3+yugCj42RU5Q1H-DKUqf79oxr0u4iB2OHdANG_DN-_emw@mail.gmail.com>

Hello,

while researching on the following StackOverflow Question (dealing with
minimum footprint on startup) I noticed that the jmap generated heapdump
does not contain all class instances and the ones which are contained do
not list most of the fields of a class.

https://stackoverflow.com/a/44397441/13189

The original question was, why a hello world java class loads so many
classes and especially has so many (115) java.lang.ref.Field instances in
the (jmap -histo) histogram.

To explain this I was expecting to simply query the instances in MAT,
however the Field instances do not show up as reachable and can therefore
not be queries (I think those are cached Field accessors for some system
classes, but I cannot see that in the dump file).

Both Eclipse MAT and jvisualvm Profiler do not show them (and do only show
a small number of class instances and now fields for them).

Is this a known limitation that not all class instances and their fields
are present in the dump or is this more a problem of the tools displaying
the dump? Is this restricted to classes from the system classloader (as I
do not recall having seen that problem in application classes).

Bernd
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170614/dc0ce6e2/attachment.html>

From ujwal.vangapally at oracle.com  Wed Jun 14 17:06:32 2017
From: ujwal.vangapally at oracle.com (Ujwal Vangapally)
Date: Wed, 14 Jun 2017 22:36:32 +0530
Subject: RFR: JDK-8178508 Co-locate remaining MM tests
In-Reply-To: <6E24B6D9-22CA-4BED-B845-6302C5562619@oracle.com>
References: <6d1f972d-dbec-7a79-4c1d-a960d8696a18@oracle.com>
 <a60f4f4a-40b6-fa23-894e-02600d0ce9b7@oracle.com>
 <11a3f964-070a-22f0-50fc-a2b0052a54df@oracle.com>
 <1ED1CB49-8F63-494E-AC82-5506C9BB5FC3@oracle.com>
 <FC020732-2C5C-40EA-B7E8-020A159D48F3@oracle.com>
 <325350fb-fd35-0d54-328e-e78ee3129c46@oracle.com>
 <7B0DD73D-176E-4EFC-A01A-81D54CE0B918@oracle.com>
 <6d71f5a4-4762-a300-862e-7bd4e95676b8@oracle.com>
 <13C2E57A-F0C9-425A-81B8-79CA10F807BD@oracle.com>
 <a28d2ece-a08f-cc83-12c8-ab43ebfb237b@oracle.com>
 <6E24B6D9-22CA-4BED-B845-6302C5562619@oracle.com>
Message-ID: <99a09ffe-9f6c-b37b-fd14-a99b9a6b180f@oracle.com>

Hi Igor can you kindly review this.

Thanks,

Ujwal


On 6/14/2017 9:40 AM, Mandy Chung wrote:
>
>> On Jun 13, 2017, at 8:13 PM, Ujwal Vangapally 
>> <ujwal.vangapally at oracle.com <mailto:ujwal.vangapally at oracle.com>> wrote:
>>
>> Thanks for the review Mandy.
>>
>> updated webrev with simplified version using @requires 
>> os.simpleArch=="x64".
>>
>> can't use @requires (os.family != "windows") | (os.simpleArch != 
>> "i586")alone  as sometimes it might fail on Linux 32bit as well
>>
>> webrev : 
>> http://cr.openjdk.java.net/~uvangapally/webrev/2017/8178508/webrev.03
>>
>>
>
> This version looks good.
>
> Mandy
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170614/243e88ff/attachment.html>

From igor.ignatyev at oracle.com  Wed Jun 14 17:08:50 2017
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Wed, 14 Jun 2017 10:08:50 -0700
Subject: RFR: JDK-8178508 Co-locate remaining MM tests
In-Reply-To: <99a09ffe-9f6c-b37b-fd14-a99b9a6b180f@oracle.com>
References: <6d1f972d-dbec-7a79-4c1d-a960d8696a18@oracle.com>
 <a60f4f4a-40b6-fa23-894e-02600d0ce9b7@oracle.com>
 <11a3f964-070a-22f0-50fc-a2b0052a54df@oracle.com>
 <1ED1CB49-8F63-494E-AC82-5506C9BB5FC3@oracle.com>
 <FC020732-2C5C-40EA-B7E8-020A159D48F3@oracle.com>
 <325350fb-fd35-0d54-328e-e78ee3129c46@oracle.com>
 <7B0DD73D-176E-4EFC-A01A-81D54CE0B918@oracle.com>
 <6d71f5a4-4762-a300-862e-7bd4e95676b8@oracle.com>
 <13C2E57A-F0C9-425A-81B8-79CA10F807BD@oracle.com>
 <a28d2ece-a08f-cc83-12c8-ab43ebfb237b@oracle.com>
 <6E24B6D9-22CA-4BED-B845-6302C5562619@oracle.com>
 <99a09ffe-9f6c-b37b-fd14-a99b9a6b180f@oracle.com>
Message-ID: <03FACA7B-6E6D-474D-B515-26781D6E386C@oracle.com>

looks good to me as well.

-- Igor
> On Jun 14, 2017, at 10:06 AM, Ujwal Vangapally <ujwal.vangapally at oracle.com> wrote:
> 
> Hi Igor can you kindly review this.
> 
> Thanks,
> 
> Ujwal
> 
> On 6/14/2017 9:40 AM, Mandy Chung wrote:
>> 
>>> On Jun 13, 2017, at 8:13 PM, Ujwal Vangapally <ujwal.vangapally at oracle.com <mailto:ujwal.vangapally at oracle.com>> wrote:
>>> 
>>> Thanks for the review Mandy.
>>> 
>>> updated webrev with simplified version using @requires os.simpleArch=="x64".
>>> 
>>> can't use @requires (os.family != "windows") | (os.simpleArch != "i586") alone  as sometimes it might fail on Linux 32bit as well 
>>> 
>>> webrev : http://cr.openjdk.java.net/~uvangapally/webrev/2017/8178508/webrev.03 <http://cr.openjdk.java.net/%7Euvangapally/webrev/2017/8178508/webrev.03>
>>> 
>> 
>> This version looks good.
>> 
>> Mandy
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170614/cc423664/attachment.html>

From ujwal.vangapally at oracle.com  Wed Jun 14 17:12:39 2017
From: ujwal.vangapally at oracle.com (Ujwal Vangapally)
Date: Wed, 14 Jun 2017 22:42:39 +0530
Subject: RFR: JDK-8178508 Co-locate remaining MM tests
In-Reply-To: <03FACA7B-6E6D-474D-B515-26781D6E386C@oracle.com>
References: <6d1f972d-dbec-7a79-4c1d-a960d8696a18@oracle.com>
 <a60f4f4a-40b6-fa23-894e-02600d0ce9b7@oracle.com>
 <11a3f964-070a-22f0-50fc-a2b0052a54df@oracle.com>
 <1ED1CB49-8F63-494E-AC82-5506C9BB5FC3@oracle.com>
 <FC020732-2C5C-40EA-B7E8-020A159D48F3@oracle.com>
 <325350fb-fd35-0d54-328e-e78ee3129c46@oracle.com>
 <7B0DD73D-176E-4EFC-A01A-81D54CE0B918@oracle.com>
 <6d71f5a4-4762-a300-862e-7bd4e95676b8@oracle.com>
 <13C2E57A-F0C9-425A-81B8-79CA10F807BD@oracle.com>
 <a28d2ece-a08f-cc83-12c8-ab43ebfb237b@oracle.com>
 <6E24B6D9-22CA-4BED-B845-6302C5562619@oracle.com>
 <99a09ffe-9f6c-b37b-fd14-a99b9a6b180f@oracle.com>
 <03FACA7B-6E6D-474D-B515-26781D6E386C@oracle.com>
Message-ID: <07c2d36b-d1d3-7daa-d6ba-539539a59e8e@oracle.com>

Thanks a lot for the review Mandy, Igor, Harsha .

-Ujwal.


On 6/14/2017 10:38 PM, Igor Ignatyev wrote:
> looks good to me as well.
>
> -- Igor
>> On Jun 14, 2017, at 10:06 AM, Ujwal Vangapally 
>> <ujwal.vangapally at oracle.com <mailto:ujwal.vangapally at oracle.com>> wrote:
>>
>> Hi Igor can you kindly review this.
>>
>> Thanks,
>>
>> Ujwal
>>
>>
>> On 6/14/2017 9:40 AM, Mandy Chung wrote:
>>>
>>>> On Jun 13, 2017, at 8:13 PM, Ujwal Vangapally 
>>>> <ujwal.vangapally at oracle.com <mailto:ujwal.vangapally at oracle.com>> 
>>>> wrote:
>>>>
>>>> Thanks for the review Mandy.
>>>>
>>>> updated webrev with simplified version using @requires 
>>>> os.simpleArch=="x64".
>>>>
>>>> can't use @requires (os.family != "windows") | (os.simpleArch != 
>>>> "i586")alone  as sometimes it might fail on Linux 32bit as well
>>>>
>>>> webrev : 
>>>> http://cr.openjdk.java.net/~uvangapally/webrev/2017/8178508/webrev.03
>>>>
>>>>
>>>
>>> This version looks good.
>>>
>>> Mandy
>>>
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170614/73893f60/attachment-0001.html>

From mandy.chung at oracle.com  Wed Jun 14 17:18:04 2017
From: mandy.chung at oracle.com (Mandy Chung)
Date: Wed, 14 Jun 2017 10:18:04 -0700
Subject: RFR(10): JDK-8178536: OOM ERRORS + SERVICE-THREAD TAKES A
 PROCESSOR TO 100%
In-Reply-To: <cc9ee5be-e334-49e5-9d91-c280d721beab@default>
References: <cc9ee5be-e334-49e5-9d91-c280d721beab@default>
Message-ID: <0FB47AE2-E83E-479E-8AFE-F6D765AFECD2@oracle.com>


> On Jun 7, 2017, at 8:53 AM, Poonam Parhar <poonam.bajaj at oracle.com> wrote:
> 
> Hello,
> 
> Could I have reviews for the changes for the Bug JDK-8178536 <https://bugs.openjdk.java.net/browse/JDK-8178536>: OOM ERRORS + SERVICE-THREAD TAKES A PROCESSOR TO 100%.
> 
> Problem: If there are listeners installed for MemoryMXBean then in the event of an OOME, JVM internal thread 'Service Thread' responsible for delivering notifications to the Java threads itself may encounter an OOM exception and get into a loop of processing its pending requests. This happens if and when the Service Thread executing the native code calls its corresponding java methods and faces an OOM exception, this pending exception makes the thread exit early from SensorInfo?trigger() function before it can update its pending requests counter (_pending_trigger_count). This pending exception is never cleared and that makes the thread loop in LowMemoryDetector::process_sensor_changes().
> 
> Hotspot changes:
> http://cr.openjdk.java.net/~poonam/8178536/webrev.hotspot/ <http://cr.openjdk.java.net/~poonam/8178536/webrev.hotspot/>
> These changes check for the pending exception and clear it, and make sure that the pending requests counters are kept in sync on both the Java and the VM side.
> 
This fix would drop any notification when it runs out of memory.  This approach is reasonable.  Even we preallocate the notification object, the listener code may fail due to OOM. 

There are two places doing the object allocation - VM allocates MemoryUsage and the Sensor::trigger implementation.  Another idea is to change Sensor::trigger to take the values of init, used, committed, max instead MemoryUsage so that the VM code can be simplified and the Java code will catch all exception and return a boolean to indicate success/fail.   This will also prepare moving the invocation of the user listener code in a Java thread instead of ServiceThread to address the issue about ServiceThread is a  hidden thread that can?t be debugged. 

> JDK changes:
> 
> http://cr.openjdk.java.net/~poonam/8178536/webrev.jdk/ <http://cr.openjdk.java.net/~poonam/8178536/webrev.jdk/>
> These changes make the triggerAction() a no-op since we need to call this method if MemoryService::create_MemoryUsage_obj() encounters an OOM exception and we want to avoid further potential OOM exceptions in triggerAction(MemoryUsage).
> 
> 
> Testcase:
> 
> I have written a testcase though I won't be integrating it along with the fix because the failure is unreliable; the Service Thread does not always fall into the situation that causes it to loop i.e. it has _pending_trigger_count>0 and hits an OOM in its Java calls.
> 
> http://cr.openjdk.java.net/~poonam/8178536/webrev.hotspot.testcase/ <http://cr.openjdk.java.net/~poonam/8178536/webrev.hotspot.testcase/>
OK.

Mandy

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170614/6ccc263b/attachment.html>

From poonam.bajaj at oracle.com  Wed Jun 14 19:58:33 2017
From: poonam.bajaj at oracle.com (Poonam Parhar)
Date: Wed, 14 Jun 2017 12:58:33 -0700 (PDT)
Subject: RFR(10): JDK-8178536: OOM ERRORS + SERVICE-THREAD TAKES A
 PROCESSOR TO 100%
In-Reply-To: <0FB47AE2-E83E-479E-8AFE-F6D765AFECD2@oracle.com>
References: <cc9ee5be-e334-49e5-9d91-c280d721beab@default>
 <0FB47AE2-E83E-479E-8AFE-F6D765AFECD2@oracle.com>
Message-ID: <51fca6ba-d036-4ea2-9507-0b10633fe71c@default>

Hello Mandy,

?

Thanks for taking a look at the changes. My comments inlined below:

?

From: Mandy Chung 
Sent: Wednesday, June 14, 2017 10:18 AM
To: Poonam Parhar
Cc: hotspot-runtime-dev at openjdk.java.net; serviceability-dev
Subject: Re: RFR(10): JDK-8178536: OOM ERRORS + SERVICE-THREAD TAKES A PROCESSOR TO 100%

?

?

On Jun 7, 2017, at 8:53 AM, Poonam Parhar <HYPERLINK "mailto:poonam.bajaj at oracle.com"poonam.bajaj at oracle.com> wrote:

?

Hello,

Could I have reviews for the changes for the Bug HYPERLINK "https://bugs.openjdk.java.net/browse/JDK-8178536"JDK-8178536: OOM ERRORS + SERVICE-THREAD TAKES A PROCESSOR TO 100%.

Problem: If there are listeners installed for MemoryMXBean then in the event of an OOME, JVM internal thread 'Service Thread' responsible for delivering notifications to the Java threads itself may encounter an OOM exception and get into a loop of processing its pending requests. This happens if and when the Service Thread executing the native code calls its corresponding java methods and faces an OOM exception, this pending exception makes the thread exit early from SensorInfo?trigger() function before it can update its pending requests counter (_pending_trigger_count). This pending exception is never cleared and that makes the thread loop in LowMemoryDetector::process_sensor_changes().

Hotspot changes:

http://cr.openjdk.java.net/~poonam/8178536/webrev.hotspot/

These changes check for the pending exception and clear it, and make sure that the pending requests counters are kept in sync on both the Java and the VM side.

[Poonam] First of all apologies for this weird bold font. Not sure why my email client sent two emails one with this bold font and one without.

This fix would drop any notification when it runs out of memory. ?This approach is reasonable. ?Even we preallocate the notification object, the listener code may fail due to OOM.?

?

There are two places doing the object allocation - VM allocates MemoryUsage and the Sensor::trigger implementation. ?Another idea is to change Sensor::trigger to take the values of init, used, committed, max instead MemoryUsage so that the VM code can be simplified and the Java code will catch all exception and return a boolean to indicate success/fail. ? This will also prepare moving the invocation of the user listener code in a Java thread instead of ServiceThread to address the issue about ServiceThread is a ?hidden thread that can?t be debugged.?


[Poonam] yes, that could also be another way. But for this bug, I would like to keep the changes in sync with how the exceptions in the Java code called from the VM side of the Service Thread are handled. For example, ServiceThread::service_thread_entry() also calls GCNotifier::sendNotification() and here?s how the pending exception is handled.

?

?

??? if(has_gc_notification_event) {

??????? GCNotifier::sendNotification(CHECK);

??? }

void GCNotifier::sendNotification(TRAPS) {

? GCNotifier::sendNotificationInternal(THREAD);

? // Clearing pending exception to avoid premature termination of

? // the service thread

? if (HAS_PENDING_EXCEPTION) {

??? CLEAR_PENDING_EXCEPTION;

? }

}

?

When we get to the issue of enabling the debugging of the user code currently being executed on the ServiceThread, we will have to be looking at these other notifications as well and I would prefer that we make those changes together under the same bug and not with this change.

?

Please let me know what you think.

?

Thanks,

Poonam

?

JDK changes:

http://cr.openjdk.java.net/~poonam/8178536/webrev.jdk/

These changes make the triggerAction() a no-op since we need to call this method if MemoryService::create_MemoryUsage_obj() encounters an OOM exception and we want to avoid further potential OOM exceptions in triggerAction(MemoryUsage).

?

Testcase: 

I have written a testcase though I won't be integrating it along with the fix because the failure is unreliable; the Service Thread does not always fall into the situation that causes it to loop i.e. it has _pending_trigger_count>0 and hits an OOM in its Java calls.

http://cr.openjdk.java.net/~poonam/8178536/webrev.hotspot.testcase/

?

OK.

?

Mandy

?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170614/f4bdec00/attachment-0001.html>

From mandy.chung at oracle.com  Wed Jun 14 22:14:09 2017
From: mandy.chung at oracle.com (Mandy Chung)
Date: Wed, 14 Jun 2017 15:14:09 -0700
Subject: RFR(10): JDK-8178536: OOM ERRORS + SERVICE-THREAD TAKES A
 PROCESSOR TO 100%
In-Reply-To: <51fca6ba-d036-4ea2-9507-0b10633fe71c@default>
References: <cc9ee5be-e334-49e5-9d91-c280d721beab@default>
 <0FB47AE2-E83E-479E-8AFE-F6D765AFECD2@oracle.com>
 <51fca6ba-d036-4ea2-9507-0b10633fe71c@default>
Message-ID: <32C6D3CC-3B24-4D4C-ABE0-1C76FA086E48@oracle.com>


> On Jun 14, 2017, at 12:58 PM, Poonam Parhar <poonam.bajaj at oracle.com> wrote:
> 
> 
> [Poonam] yes, that could also be another way. But for this bug, I would like to keep the changes in sync with how the exceptions in the Java code called from the VM side of the Service Thread are handled. For example, ServiceThread::service_thread_entry() also calls GCNotifier::sendNotification() and here?s how the pending exception is handled.
>  
>  
> 


You can still keep the clear pending exception.   The main thing I suggest is to change the trigger method to take the raw long value parameters instead of MemoryUsage and the object allocation is all done in Java.

>  :
>  
> When we get to the issue of enabling the debugging of the user code currently being executed on the ServiceThread, we will have to be looking at these other notifications as well and I would prefer that we make those changes together under the same bug and not with this change.
>  
> Please let me know what you think.
> 

I?m okay if you want to resolve this issue as is.  And follow up my feedback together with the other issue with ServiceThread.

Mandy

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170614/5d8ea2f2/attachment.html>

From poonam.bajaj at oracle.com  Thu Jun 15 00:39:18 2017
From: poonam.bajaj at oracle.com (Poonam Parhar)
Date: Wed, 14 Jun 2017 17:39:18 -0700
Subject: RFR(10): JDK-8178536: OOM ERRORS + SERVICE-THREAD TAKES A
 PROCESSOR TO 100%
In-Reply-To: <32C6D3CC-3B24-4D4C-ABE0-1C76FA086E48@oracle.com>
References: <cc9ee5be-e334-49e5-9d91-c280d721beab@default>
 <0FB47AE2-E83E-479E-8AFE-F6D765AFECD2@oracle.com>
 <51fca6ba-d036-4ea2-9507-0b10633fe71c@default>
 <32C6D3CC-3B24-4D4C-ABE0-1C76FA086E48@oracle.com>
Message-ID: <3fa8172c-3e23-bddf-93a0-47e15c82e69f@oracle.com>


On 6/14/2017 3:14 PM, Mandy Chung wrote:
>
>> On Jun 14, 2017, at 12:58 PM, Poonam Parhar <poonam.bajaj at oracle.com 
>> <mailto:poonam.bajaj at oracle.com>> wrote:
>>
>>
>>
>> [Poonam] yes, that could also be another way. But for this bug, I 
>> would like to keep the changes in sync with how the exceptions in the 
>> Java code called from the VM side of the Service Thread are handled. 
>> For example, ServiceThread::service_thread_entry() also calls 
>> GCNotifier::sendNotification() and here?s how the pending exception 
>> is handled.//
>> *//*
>>
>
> You can still keep the clear pending exception.   The main thing I 
> suggest is to change the trigger method to take the raw long value 
> parameters instead of MemoryUsage and the object allocation is all 
> done in Java.
>
>>  :
>>
>> When we get to the issue of enabling the debugging of the user code 
>> currently being executed on the ServiceThread, we will have to be 
>> looking at these other notifications as well and I would prefer that 
>> we make those changes together under the same bug and not with this 
>> change.
>>
>> Please let me know what you think.
>>
>>
>
> I?m okay if you want to resolve this issue as is.  And follow up my 
> feedback together with the other issue with ServiceThread.

Thanks Mandy! Yes, I would like to integrate this fix as is, and change 
the trigger() parameters for the other issue.

Regards,
Poonam

>
> Mandy
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170614/3bf83acd/attachment.html>

From thomas.stuefe at gmail.com  Thu Jun 15 06:31:16 2017
From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=)
Date: Thu, 15 Jun 2017 08:31:16 +0200
Subject: RFR(xs): 8181419: Race in jdwp invoker handling may lead to
 crashes or invalid results
In-Reply-To: <9e1963f0-9d27-241a-f1de-9cfea607b91d@oracle.com>
References: <CAA-vtUwt_h6o5LjajKEGJbzD12kdYQ7cJN-41tKNFkNhs9SCug@mail.gmail.com>
 <CAA-vtUw+RGyJXOpnsuaRfZW-D3Eh5RKeGOa1w7=e=CYhu7SF0g@mail.gmail.com>
 <9e1963f0-9d27-241a-f1de-9cfea607b91d@oracle.com>
Message-ID: <CAA-vtUz3yDZ_j9nfv14f7qcn62_ZNnfG6yC-gK3sDpQKren-eQ@mail.gmail.com>

Hi Folks,

may I please have a second reviewer?

Thank you!

Kind Regards, Thomas

On Tue, Jun 13, 2017 at 11:16 PM, serguei.spitsyn at oracle.com <
serguei.spitsyn at oracle.com> wrote:

> Hi Thomas,
>
> The fix looks great to me.
> Thank you for identifying and fixing the problem!
>
> Thanks,
> Serguei
>
>
>
> On 6/13/17 06:55, Thomas St?fe wrote:
>
> Ping... Anyone?
>
> On Thu, Jun 1, 2017 at 2:18 PM, Thomas St?fe <thomas.stuefe at gmail.com>
> wrote:
>
>> Hi all,
>>
>> please take a look at this proposed fix for a theoretical race in the
>> jdwp library.
>>
>> Issue: https://bugs.openjdk.java.net/browse/JDK-8181419
>> webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8181419-
>> Race-in-jdwp-invoker-handling-may-lead-to-crashes-or-
>> invalid-results/webrev.00/webrev/
>>
>> In short, this is an addition to Severin's fix to the jdwp invoke
>> handling (https://bugs.openjdk.java.net/browse/JDK-8153711).
>>
>> We have a potential race condition where the delayed cleanup of the saved
>> returnvalue object reference and the exception reference (released
>> in deletePotentiallySavedGlobalRefs() ) may be overtaken by a new
>> request which populates the thread request structure anew. If this happens,
>> deletePotentiallySavedGlobalRefs() may actually release the return value
>> / exception references of the follow up request, if that one was already
>> processed.
>>
>> The solution I choose is safe and conservative. We still release both
>> references, but use the locally saved JNI references. We just avoid
>> accessing the thread local request structure after it has been cleared for
>> reuse. This keeps timing and locking behaviour unchanged.
>>
>> I am currently running jtreg tests for com/sun/jdi on AIX and Linux.
>>
>> Kind Regards, Thomas
>>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170615/734aee0d/attachment-0001.html>

From serguei.spitsyn at oracle.com  Thu Jun 15 18:00:48 2017
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Thu, 15 Jun 2017 11:00:48 -0700
Subject: RFR(xs): 8181419: Race in jdwp invoker handling may lead to
 crashes or invalid results
In-Reply-To: <CAA-vtUyzkw-Zb-s+3yJNHe42hQ5LxmGFt8MiOVP5t6nW1pEqfQ@mail.gmail.com>
References: <CAA-vtUwt_h6o5LjajKEGJbzD12kdYQ7cJN-41tKNFkNhs9SCug@mail.gmail.com>
 <CAA-vtUw+RGyJXOpnsuaRfZW-D3Eh5RKeGOa1w7=e=CYhu7SF0g@mail.gmail.com>
 <9e1963f0-9d27-241a-f1de-9cfea607b91d@oracle.com>
 <8a82b2e6-d29a-1e1e-c16d-ff8560ee0203@oracle.com>
 <CAA-vtUyzkw-Zb-s+3yJNHe42hQ5LxmGFt8MiOVP5t6nW1pEqfQ@mail.gmail.com>
Message-ID: <23f2b805-3a39-93ca-42f4-a589bf98cdd6@oracle.com>

An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170615/6220ee55/attachment.html>

From serguei.spitsyn at oracle.com  Thu Jun 15 18:59:07 2017
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Thu, 15 Jun 2017 11:59:07 -0700
Subject: RFR(xs): 8181419: Race in jdwp invoker handling may lead to
 crashes or invalid results
In-Reply-To: <23f2b805-3a39-93ca-42f4-a589bf98cdd6@oracle.com>
References: <CAA-vtUwt_h6o5LjajKEGJbzD12kdYQ7cJN-41tKNFkNhs9SCug@mail.gmail.com>
 <CAA-vtUw+RGyJXOpnsuaRfZW-D3Eh5RKeGOa1w7=e=CYhu7SF0g@mail.gmail.com>
 <9e1963f0-9d27-241a-f1de-9cfea607b91d@oracle.com>
 <8a82b2e6-d29a-1e1e-c16d-ff8560ee0203@oracle.com>
 <CAA-vtUyzkw-Zb-s+3yJNHe42hQ5LxmGFt8MiOVP5t6nW1pEqfQ@mail.gmail.com>
 <23f2b805-3a39-93ca-42f4-a589bf98cdd6@oracle.com>
Message-ID: <2fdcb080-d177-42c5-4943-d275129c3b3e@oracle.com>

An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170615/ae62e334/attachment.html>

From amit.sapre at oracle.com  Fri Jun 16 11:17:44 2017
From: amit.sapre at oracle.com (Amit Sapre)
Date: Fri, 16 Jun 2017 04:17:44 -0700 (PDT)
Subject: RFR (JDK 9) : JDK-8181895 - javax management docs contain links to
 technotes
Message-ID: <83e4c980-0cd8-4758-8e6b-663c70de3b05@default>

Hi,

 

Please review the doc  changes for removing technotes links

 

Bug ID : https://bugs.openjdk.java.net/browse/JDK-8181895 

Webrev : http://cr.openjdk.java.net/~asapre/webrev/2017/JDK-8181895/webrev.00/

 

Thanks,

Amit
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170616/214f0e2a/attachment.html>

From christoph.langer at sap.com  Fri Jun 16 12:18:21 2017
From: christoph.langer at sap.com (Langer, Christoph)
Date: Fri, 16 Jun 2017 12:18:21 +0000
Subject: JDK 10 RFR 8181417: Code cleanups in com.sun.jdi
In-Reply-To: <7f7b7359-0a82-34e4-17a3-597d13b95492@oracle.com>
References: <a7e6b44d1b4d49c580a65549fe0f98e0@sap.com>
 <d2e245e424664ba78ad2b22842e61fa4@sap.com>
 <7f7b7359-0a82-34e4-17a3-597d13b95492@oracle.com>
Message-ID: <3f52b4f69694406991768424cb76031a@sap.com>

Hi,

I just submitted http://hg.openjdk.java.net/jdk10/jdk10/jdk/rev/c0f1d48e3ffa after thoroughly reviewing all copyright headers, cleaning a few additional spots in the same sense, running a build and the jdi jtreg test suite.

Sorry for leaving this open for such a long time and the scarce initial bug description.

Best regards
Christoph

From: serguei.spitsyn at oracle.com [mailto:serguei.spitsyn at oracle.com]
Sent: Freitag, 2. Juni 2017 23:16
To: Langer, Christoph <christoph.langer at sap.com>; serviceability-dev at openjdk.java.net
Cc: Stuefe, Thomas <thomas.stuefe at sap.com>
Subject: Re: JDK 10 RFR 8181417: Code cleanups in com.sun.jdi

Hi Langer,

Thank you for the contribution!
It looks good (agreed with Alan on a lot of noise with the adding/removing empty lines).
Many files (including new in this version of webrev) miss the copyright comment update.

Thanks,
Serguei



On 6/1/17 08:38, Langer, Christoph wrote:
Hi again,

I updated my webrev to include packages com.sun.jdi.event and com.sun.jdi.request as well. With that I?ve added everything that I found in com.sun.jdi and its subpackages.

Please look here: http://cr.openjdk.java.net/~clanger/webrevs/8181417.1/<http://cr.openjdk.java.net/%7Eclanger/webrevs/8181417.1/>

Best regards
Christoph

From: Langer, Christoph
Sent: Donnerstag, 1. Juni 2017 11:40
To: serviceability-dev at openjdk.java.net<mailto:serviceability-dev at openjdk.java.net>; core-libs-dev at openjdk.java.net<mailto:core-libs-dev at openjdk.java.net>
Subject: JDK 10 RFR 8181417: Code cleanups in com.sun.jdi

Hi,

during integration work of the jdi library for one of our projects I spotted some places where a few cleanups could be done. I?d like to contribute this. It is mostly about import statements and formatting/empty lines. I think at this stage of JDK10 it?s a good time for such kind of work.

Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8181417.0/<http://cr.openjdk.java.net/%7Eclanger/webrevs/8181417.0/>
Bug: https://bugs.openjdk.java.net/browse/JDK-8181417

Thanks and best regards
Christoph


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170616/a0e5357f/attachment.html>

From daniel.daugherty at oracle.com  Fri Jun 16 17:12:46 2017
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Fri, 16 Jun 2017 11:12:46 -0600
Subject: RFR(10): JDK-8178536: OOM ERRORS + SERVICE-THREAD TAKES A
 PROCESSOR TO 100%
In-Reply-To: <51fca6ba-d036-4ea2-9507-0b10633fe71c@default>
References: <cc9ee5be-e334-49e5-9d91-c280d721beab@default>
 <0FB47AE2-E83E-479E-8AFE-F6D765AFECD2@oracle.com>
 <51fca6ba-d036-4ea2-9507-0b10633fe71c@default>
Message-ID: <0c339de1-a093-950c-52ff-d7d9a3ce129b@oracle.com>

 > Hotspot changes:
 >
 > http://cr.openjdk.java.net/~poonam/8178536/webrev.hotspot/

src/share/vm/services/lowMemoryDetector.cpp
     nit - please update the copyright year before pushing.

     L307:     // call Sensor::trigger(int, MemoryUsage) to send 
notification to listeners.
         nit - 'call' -> 'Call'

     L308:     // when OOME occurs and fails to allocate MemoryUsage 
object, call
         nit - 'when' -> 'When'

     L328:        // we just clear the OOM pending exception that we 
might have encountered
         nit 'we' -> 'We'

     L328, L330-32 - jcheck is going to complain about the trailing 
blanks on
         these lines.

Thumbs up on the HotSpot changes, but these low memory situations
are fraught with unexpected failure modes and code paths.

Dan


On 6/14/17 1:58 PM, Poonam Parhar wrote:
> Hello Mandy,
>
>   
>
> Thanks for taking a look at the changes. My comments inlined below:
>
>   
>
> From: Mandy Chung
> Sent: Wednesday, June 14, 2017 10:18 AM
> To: Poonam Parhar
> Cc: hotspot-runtime-dev at openjdk.java.net; serviceability-dev
> Subject: Re: RFR(10): JDK-8178536: OOM ERRORS + SERVICE-THREAD TAKES A PROCESSOR TO 100%
>
>   
>
>   
>
> On Jun 7, 2017, at 8:53 AM, Poonam Parhar <HYPERLINK "mailto:poonam.bajaj at oracle.com"poonam.bajaj at oracle.com> wrote:
>
>   
>
> Hello,
>
> Could I have reviews for the changes for the Bug HYPERLINK "https://bugs.openjdk.java.net/browse/JDK-8178536"JDK-8178536: OOM ERRORS + SERVICE-THREAD TAKES A PROCESSOR TO 100%.
>
> Problem: If there are listeners installed for MemoryMXBean then in the event of an OOME, JVM internal thread 'Service Thread' responsible for delivering notifications to the Java threads itself may encounter an OOM exception and get into a loop of processing its pending requests. This happens if and when the Service Thread executing the native code calls its corresponding java methods and faces an OOM exception, this pending exception makes the thread exit early from SensorInfo?trigger() function before it can update its pending requests counter (_pending_trigger_count). This pending exception is never cleared and that makes the thread loop in LowMemoryDetector::process_sensor_changes().
>
> Hotspot changes:
>
> http://cr.openjdk.java.net/~poonam/8178536/webrev.hotspot/
>
> These changes check for the pending exception and clear it, and make sure that the pending requests counters are kept in sync on both the Java and the VM side.
>
> [Poonam] First of all apologies for this weird bold font. Not sure why my email client sent two emails one with this bold font and one without.
>
> This fix would drop any notification when it runs out of memory.  This approach is reasonable.  Even we preallocate the notification object, the listener code may fail due to OOM.
>
>   
>
> There are two places doing the object allocation - VM allocates MemoryUsage and the Sensor::trigger implementation.  Another idea is to change Sensor::trigger to take the values of init, used, committed, max instead MemoryUsage so that the VM code can be simplified and the Java code will catch all exception and return a boolean to indicate success/fail.   This will also prepare moving the invocation of the user listener code in a Java thread instead of ServiceThread to address the issue about ServiceThread is a  hidden thread that can?t be debugged.
>
>
> [Poonam] yes, that could also be another way. But for this bug, I would like to keep the changes in sync with how the exceptions in the Java code called from the VM side of the Service Thread are handled. For example, ServiceThread::service_thread_entry() also calls GCNotifier::sendNotification() and here?s how the pending exception is handled.
>
>   
>
>   
>
>      if(has_gc_notification_event) {
>
>          GCNotifier::sendNotification(CHECK);
>
>      }
>
> void GCNotifier::sendNotification(TRAPS) {
>
>    GCNotifier::sendNotificationInternal(THREAD);
>
>    // Clearing pending exception to avoid premature termination of
>
>    // the service thread
>
>    if (HAS_PENDING_EXCEPTION) {
>
>      CLEAR_PENDING_EXCEPTION;
>
>    }
>
> }
>
>   
>
> When we get to the issue of enabling the debugging of the user code currently being executed on the ServiceThread, we will have to be looking at these other notifications as well and I would prefer that we make those changes together under the same bug and not with this change.
>
>   
>
> Please let me know what you think.
>
>   
>
> Thanks,
>
> Poonam
>
>   
>
> JDK changes:
>
> http://cr.openjdk.java.net/~poonam/8178536/webrev.jdk/
>
> These changes make the triggerAction() a no-op since we need to call this method if MemoryService::create_MemoryUsage_obj() encounters an OOM exception and we want to avoid further potential OOM exceptions in triggerAction(MemoryUsage).
>
>   
>
> Testcase:
>
> I have written a testcase though I won't be integrating it along with the fix because the failure is unreliable; the Service Thread does not always fall into the situation that causes it to loop i.e. it has _pending_trigger_count>0 and hits an OOM in its Java calls.
>
> http://cr.openjdk.java.net/~poonam/8178536/webrev.hotspot.testcase/
>
>   
>
> OK.
>
>   
>
> Mandy
>
>   
>


From poonam.bajaj at oracle.com  Fri Jun 16 17:58:54 2017
From: poonam.bajaj at oracle.com (Poonam Parhar)
Date: Fri, 16 Jun 2017 10:58:54 -0700 (PDT)
Subject: RFR(10): JDK-8178536: OOM ERRORS + SERVICE-THREAD TAKES A
 PROCESSOR TO 100%
In-Reply-To: <0c339de1-a093-950c-52ff-d7d9a3ce129b@oracle.com>
References: <cc9ee5be-e334-49e5-9d91-c280d721beab@default>
 <0FB47AE2-E83E-479E-8AFE-F6D765AFECD2@oracle.com>
 <51fca6ba-d036-4ea2-9507-0b10633fe71c@default>
 <0c339de1-a093-950c-52ff-d7d9a3ce129b@oracle.com>
Message-ID: <eeb95855-b11c-47bd-8dc3-213d39b570ad@default>

Thanks Dan! I will incorporate your suggestions.

Regards,
Poonam

> -----Original Message-----
> From: Daniel D. Daugherty
> Sent: Friday, June 16, 2017 10:13 AM
> To: Poonam Parhar; Mandy Chung
> Cc: serviceability-dev; hotspot-runtime-dev at openjdk.java.net
> Subject: Re: RFR(10): JDK-8178536: OOM ERRORS + SERVICE-THREAD TAKES A
> PROCESSOR TO 100%
> 
>  > Hotspot changes:
>  >
>  > http://cr.openjdk.java.net/~poonam/8178536/webrev.hotspot/
> 
> src/share/vm/services/lowMemoryDetector.cpp
>      nit - please update the copyright year before pushing.
> 
>      L307:     // call Sensor::trigger(int, MemoryUsage) to send
> notification to listeners.
>          nit - 'call' -> 'Call'
> 
>      L308:     // when OOME occurs and fails to allocate MemoryUsage
> object, call
>          nit - 'when' -> 'When'
> 
>      L328:        // we just clear the OOM pending exception that we
> might have encountered
>          nit 'we' -> 'We'
> 
>      L328, L330-32 - jcheck is going to complain about the trailing
> blanks on
>          these lines.
> 
> Thumbs up on the HotSpot changes, but these low memory situations are
> fraught with unexpected failure modes and code paths.
> 
> Dan
> 
> 
> On 6/14/17 1:58 PM, Poonam Parhar wrote:
> > Hello Mandy,
> >
> >
> >
> > Thanks for taking a look at the changes. My comments inlined below:
> >
> >
> >
> > From: Mandy Chung
> > Sent: Wednesday, June 14, 2017 10:18 AM
> > To: Poonam Parhar
> > Cc: hotspot-runtime-dev at openjdk.java.net; serviceability-dev
> > Subject: Re: RFR(10): JDK-8178536: OOM ERRORS + SERVICE-THREAD TAKES
> A
> > PROCESSOR TO 100%
> >
> >
> >
> >
> >
> > On Jun 7, 2017, at 8:53 AM, Poonam Parhar <HYPERLINK
> "mailto:poonam.bajaj at oracle.com"poonam.bajaj at oracle.com> wrote:
> >
> >
> >
> > Hello,
> >
> > Could I have reviews for the changes for the Bug HYPERLINK
> "https://bugs.openjdk.java.net/browse/JDK-8178536"JDK-8178536: OOM
> ERRORS + SERVICE-THREAD TAKES A PROCESSOR TO 100%.
> >
> > Problem: If there are listeners installed for MemoryMXBean then in
> the event of an OOME, JVM internal thread 'Service Thread' responsible
> for delivering notifications to the Java threads itself may encounter
> an OOM exception and get into a loop of processing its pending
> requests. This happens if and when the Service Thread executing the
> native code calls its corresponding java methods and faces an OOM
> exception, this pending exception makes the thread exit early from
> SensorInfo?trigger() function before it can update its pending requests
> counter (_pending_trigger_count). This pending exception is never
> cleared and that makes the thread loop in
> LowMemoryDetector::process_sensor_changes().
> >
> > Hotspot changes:
> >
> > http://cr.openjdk.java.net/~poonam/8178536/webrev.hotspot/
> >
> > These changes check for the pending exception and clear it, and make
> sure that the pending requests counters are kept in sync on both the
> Java and the VM side.
> >
> > [Poonam] First of all apologies for this weird bold font. Not sure
> why my email client sent two emails one with this bold font and one
> without.
> >
> > This fix would drop any notification when it runs out of memory.
> This approach is reasonable.  Even we preallocate the notification
> object, the listener code may fail due to OOM.
> >
> >
> >
> > There are two places doing the object allocation - VM allocates
> MemoryUsage and the Sensor::trigger implementation.  Another idea is to
> change Sensor::trigger to take the values of init, used, committed, max
> instead MemoryUsage so that the VM code can be simplified and the Java
> code will catch all exception and return a boolean to indicate
> success/fail.   This will also prepare moving the invocation of the
> user listener code in a Java thread instead of ServiceThread to address
> the issue about ServiceThread is a  hidden thread that can?t be
> debugged.
> >
> >
> > [Poonam] yes, that could also be another way. But for this bug, I
> would like to keep the changes in sync with how the exceptions in the
> Java code called from the VM side of the Service Thread are handled.
> For example, ServiceThread::service_thread_entry() also calls
> GCNotifier::sendNotification() and here?s how the pending exception is
> handled.
> >
> >
> >
> >
> >
> >      if(has_gc_notification_event) {
> >
> >          GCNotifier::sendNotification(CHECK);
> >
> >      }
> >
> > void GCNotifier::sendNotification(TRAPS) {
> >
> >    GCNotifier::sendNotificationInternal(THREAD);
> >
> >    // Clearing pending exception to avoid premature termination of
> >
> >    // the service thread
> >
> >    if (HAS_PENDING_EXCEPTION) {
> >
> >      CLEAR_PENDING_EXCEPTION;
> >
> >    }
> >
> > }
> >
> >
> >
> > When we get to the issue of enabling the debugging of the user code
> currently being executed on the ServiceThread, we will have to be
> looking at these other notifications as well and I would prefer that we
> make those changes together under the same bug and not with this
> change.
> >
> >
> >
> > Please let me know what you think.
> >
> >
> >
> > Thanks,
> >
> > Poonam
> >
> >
> >
> > JDK changes:
> >
> > http://cr.openjdk.java.net/~poonam/8178536/webrev.jdk/
> >
> > These changes make the triggerAction() a no-op since we need to call
> this method if MemoryService::create_MemoryUsage_obj() encounters an
> OOM exception and we want to avoid further potential OOM exceptions in
> triggerAction(MemoryUsage).
> >
> >
> >
> > Testcase:
> >
> > I have written a testcase though I won't be integrating it along with
> the fix because the failure is unreliable; the Service Thread does not
> always fall into the situation that causes it to loop i.e. it has
> _pending_trigger_count>0 and hits an OOM in its Java calls.
> >
> > http://cr.openjdk.java.net/~poonam/8178536/webrev.hotspot.testcase/
> >
> >
> >
> > OK.
> >
> >
> >
> > Mandy
> >
> >
> >
> 

From vladimir.kozlov at oracle.com  Sat Jun 17 17:04:19 2017
From: vladimir.kozlov at oracle.com (Vladimir Kozlov)
Date: Sat, 17 Jun 2017 10:04:19 -0700
Subject: RFR(10): JDK-8178536: OOM ERRORS + SERVICE-THREAD TAKES A
 PROCESSOR TO 100%
In-Reply-To: <cc9ee5be-e334-49e5-9d91-c280d721beab@default>
References: <cc9ee5be-e334-49e5-9d91-c280d721beab@default>
Message-ID: <daa4127d-a84a-f10b-bf34-808e224f7786@oracle.com>

Hi Poonam,

You need to follow PRD2 process - add label and "Fix Request" comment.

http://openjdk.java.net/projects/jdk9/fix-request-process

I will approve after that.

Regards,
Vladimir

On 6/7/17 8:53 AM, Poonam Parhar wrote:
> Hello,
> 
> Could I have reviews for the changes for the Bug  HYPERLINK "https://bugs.openjdk.java.net/browse/JDK-8178536" JDK-8178536: OOM ERRORS + SERVICE-THREAD TAKES A PROCESSOR TO 100%.
> 
> Problem: If there are listeners installed for MemoryMXBean then in the event of an OOME, JVM internal thread 'Service Thread' responsible for delivering notifications to the Java threads itself may encounter an OOM exception and get into a loop of processing its pending requests. This happens if and when the Service Thread executing the native code calls its corresponding java methods and faces an OOM exception, this pending exception makes the thread exit early from SensorInfo?trigger() function before it can update its pending requests counter (_pending_trigger_count). This pending exception is never cleared and that makes the thread loop in LowMemoryDetector::process_sensor_changes().
> 
> Hotspot changes:
> http://cr.openjdk.java.net/~poonam/8178536/webrev.hotspot/
> These changes check for the pending exception and clear it, and make sure that the pending requests counters are kept in sync on both the Java and the VM side.
> 
> JDK changes:
> http://cr.openjdk.java.net/~poonam/8178536/webrev.jdk/
> These changes make the triggerAction() a no-op since we need to call this method if MemoryService::create_MemoryUsage_obj() encounters an OOM exception and we want to avoid further potential OOM exceptions in triggerAction(MemoryUsage).
> 
> Testcase:
> I have written a testcase though I won't be integrating it along with the fix because the failure is unreliable; the Service Thread does not always fall into the situation that causes it to loop i.e. it has _pending_trigger_count>0 and hits an OOM in its Java calls.
> http://cr.openjdk.java.net/~poonam/8178536/webrev.hotspot.testcase/
> 
> Thanks,
> Poonam
> 

From serguei.spitsyn at oracle.com  Sat Jun 17 22:42:59 2017
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Sat, 17 Jun 2017 15:42:59 -0700
Subject: RFR(xs): 8181419: Race in jdwp invoker handling may lead to
 crashes or invalid results
In-Reply-To: <23f2b805-3a39-93ca-42f4-a589bf98cdd6@oracle.com>
References: <CAA-vtUwt_h6o5LjajKEGJbzD12kdYQ7cJN-41tKNFkNhs9SCug@mail.gmail.com>
 <CAA-vtUw+RGyJXOpnsuaRfZW-D3Eh5RKeGOa1w7=e=CYhu7SF0g@mail.gmail.com>
 <9e1963f0-9d27-241a-f1de-9cfea607b91d@oracle.com>
 <8a82b2e6-d29a-1e1e-c16d-ff8560ee0203@oracle.com>
 <CAA-vtUyzkw-Zb-s+3yJNHe42hQ5LxmGFt8MiOVP5t6nW1pEqfQ@mail.gmail.com>
 <23f2b805-3a39-93ca-42f4-a589bf98cdd6@oracle.com>
Message-ID: <f61fa99a-0da4-eb82-657d-5dc45fd2f7e8@oracle.com>

An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170617/fe789713/attachment.html>

From chihiro.ito at oracle.com  Sun Jun 18 04:02:22 2017
From: chihiro.ito at oracle.com (chihiro ito)
Date: Sun, 18 Jun 2017 13:02:22 +0900
Subject: [10] RFR 8181647: jhsdb jstack could not output thread name
In-Reply-To: <HE1PR08MB2795EF131855EC452EAF9D3FFFC30@HE1PR08MB2795.eurprd08.prod.outlook.com>
References: <5937CB5F.4060209@oracle.com>
 <a9a45cd1-8717-ed6b-8169-9543e904c20f@oracle.com>
 <5939133A.9040709@oracle.com> <5940E316.7060104@oracle.com>
 <HE1PR08MB2795EF131855EC452EAF9D3FFFC30@HE1PR08MB2795.eurprd08.prod.outlook.com>
Message-ID: <5945FB4E.9070502@oracle.com>

At first I thought to print just each thread name, but I tried to make 
it as similar as possible to JStack as following.

Mixed mode:
----------------- 26476 -----------------
"main" #1 prio=5 tid=0x00007f6894019000 nid=0x676c waiting on condition 
[0x00007f689b7ae000]
    java.lang.Thread.State: TIMED_WAITING (sleeping)
    JavaThread state: _thread_blocked
0x00007f689b185a82    __pthread_cond_timedwait + 0x132

No mixed mode:
"main" #1 prio=5 tid=0x00007f6894019000 nid=0x676c waiting on condition 
[0x00007f689b7ae000]
    java.lang.Thread.State: TIMED_WAITING (sleeping)
    JavaThread state: _thread_blocked
  - java.lang.Thread.sleep(long) @bci=0 (Interpreted frame)


This change passed a test by jtreg.

SOURCE_HOME=/home/user/repo/jdk10-hs
jtreg -dir:${SOURCE_HOME}/hotspot/test 
-testjdk:${SOURCE_HOME}/build/linux-x86_64-normal-server-slowdebug/jdk/ 
serviceability/sa/JhsdbThreadInfoTest.java
Test results: passed: 1


Source:
diff --git 
a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/oops/OopUtilities.java 
b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/oops/OopUtilities.java
--- 
a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/oops/OopUtilities.java
+++ 
b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/oops/OopUtilities.java
@@ -1,5 +1,5 @@
  /*
- * Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights 
reserved.
+ * Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights 
reserved.
   * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
   *
   * This code is free software; you can redistribute it and/or modify it
@@ -59,20 +59,20 @@
    // parkBlocker field is new since 1.6
    private static OopField threadParkBlockerField;

+  private static IntField threadPriorityField;
+  private static BooleanField threadDaemonField;
+
    // possible values of java_lang_Thread::ThreadStatus
    private static int THREAD_STATUS_NEW;
-  /*
-    Other enum constants are not needed as of now. Uncomment these as 
and when needed.

-    private static int THREAD_STATUS_RUNNABLE;
-    private static int THREAD_STATUS_SLEEPING;
-    private static int THREAD_STATUS_IN_OBJECT_WAIT;
-    private static int THREAD_STATUS_IN_OBJECT_WAIT_TIMED;
-    private static int THREAD_STATUS_PARKED;
-    private static int THREAD_STATUS_PARKED_TIMED;
-    private static int THREAD_STATUS_BLOCKED_ON_MONITOR_ENTER;
-    private static int THREAD_STATUS_TERMINATED;
-  */
+  private static int THREAD_STATUS_RUNNABLE;
+  private static int THREAD_STATUS_SLEEPING;
+  private static int THREAD_STATUS_IN_OBJECT_WAIT;
+  private static int THREAD_STATUS_IN_OBJECT_WAIT_TIMED;
+  private static int THREAD_STATUS_PARKED;
+  private static int THREAD_STATUS_PARKED_TIMED;
+  private static int THREAD_STATUS_BLOCKED_ON_MONITOR_ENTER;
+  private static int THREAD_STATUS_TERMINATED;

    // java.util.concurrent.locks.AbstractOwnableSynchronizer fields
    private static OopField absOwnSyncOwnerThreadField;
@@ -229,20 +229,19 @@
        threadStatusField = (IntField) k.findField("threadStatus", "I");
        threadParkBlockerField = (OopField) k.findField("parkBlocker",
                                       "Ljava/lang/Object;");
+      threadPriorityField = (IntField) k.findField("priority", "I");
+      threadDaemonField = (BooleanField) k.findField("daemon", "Z");
        TypeDataBase db = VM.getVM().getTypeDataBase();
        THREAD_STATUS_NEW = 
db.lookupIntConstant("java_lang_Thread::NEW").intValue();
-      /*
-        Other enum constants are not needed as of now. Uncomment these 
as and when needed.

-        THREAD_STATUS_RUNNABLE = 
db.lookupIntConstant("java_lang_Thread::RUNNABLE").intValue();
-        THREAD_STATUS_SLEEPING = 
db.lookupIntConstant("java_lang_Thread::SLEEPING").intValue();
-        THREAD_STATUS_IN_OBJECT_WAIT = 
db.lookupIntConstant("java_lang_Thread::IN_OBJECT_WAIT").intValue();
-        THREAD_STATUS_IN_OBJECT_WAIT_TIMED = 
db.lookupIntConstant("java_lang_Thread::IN_OBJECT_WAIT_TIMED").intValue();
-        THREAD_STATUS_PARKED = 
db.lookupIntConstant("java_lang_Thread::PARKED").intValue();
-        THREAD_STATUS_PARKED_TIMED = 
db.lookupIntConstant("java_lang_Thread::PARKED_TIMED").intValue();
-        THREAD_STATUS_BLOCKED_ON_MONITOR_ENTER = 
db.lookupIntConstant("java_lang_Thread::BLOCKED_ON_MONITOR_ENTER").intValue();
-        THREAD_STATUS_TERMINATED = 
db.lookupIntConstant("java_lang_Thread::TERMINATED").intValue();
-      */
+      THREAD_STATUS_RUNNABLE = 
db.lookupIntConstant("java_lang_Thread::RUNNABLE").intValue();
+      THREAD_STATUS_SLEEPING = 
db.lookupIntConstant("java_lang_Thread::SLEEPING").intValue();
+      THREAD_STATUS_IN_OBJECT_WAIT = 
db.lookupIntConstant("java_lang_Thread::IN_OBJECT_WAIT").intValue();
+      THREAD_STATUS_IN_OBJECT_WAIT_TIMED = 
db.lookupIntConstant("java_lang_Thread::IN_OBJECT_WAIT_TIMED").intValue();
+      THREAD_STATUS_PARKED = 
db.lookupIntConstant("java_lang_Thread::PARKED").intValue();
+      THREAD_STATUS_PARKED_TIMED = 
db.lookupIntConstant("java_lang_Thread::PARKED_TIMED").intValue();
+      THREAD_STATUS_BLOCKED_ON_MONITOR_ENTER = 
db.lookupIntConstant("java_lang_Thread::BLOCKED_ON_MONITOR_ENTER").intValue();
+      THREAD_STATUS_TERMINATED = 
db.lookupIntConstant("java_lang_Thread::TERMINATED").intValue();

        if (Assert.ASSERTS_ENABLED) {
          // it is okay to miss threadStatusField, because this was
@@ -331,4 +330,46 @@
        return absOwnSyncOwnerThreadField.getValue(oop);
      }
    }
+
+  public static int threadOopGetPriority(Oop threadOop) {
+    initThreadFields();
+    if (threadPriorityField != null) {
+      return threadPriorityField.getValue(threadOop);
+    } else {
+      return 0;
+    }
+  }
+
+  public static boolean threadOopGetDaemon(Oop threadOop) {
+    initThreadFields();
+    if (threadDaemonField != null) {
+      return threadDaemonField.getValue(threadOop);
+    } else {
+      return false;
+    }
+  }
+
+  public static String threadOopGetThreadStatusName(Oop threadOop) {
+    int status = OopUtilities.threadOopGetThreadStatus(threadOop);
+    if( status == THREAD_STATUS_NEW ){
+      return "NEW";
+    }else if(status == THREAD_STATUS_RUNNABLE ){
+      return "RUNNABLE";
+    }else if(status == THREAD_STATUS_SLEEPING ){
+      return "TIMED_WAITING (sleeping)";
+    }else if(status == THREAD_STATUS_IN_OBJECT_WAIT ){
+      return "WAITING (on object monitor)";
+    }else if(status == THREAD_STATUS_IN_OBJECT_WAIT_TIMED ){
+      return "TIMED_WAITING (on object monitor)";
+    }else if(status == THREAD_STATUS_PARKED ){
+      return "WAITING (parking)";
+    }else if(status == THREAD_STATUS_PARKED_TIMED ){
+      return "TIMED_WAITING (parking)";
+    }else if(status == THREAD_STATUS_BLOCKED_ON_MONITOR_ENTER ){
+      return "BLOCKED (on object monitor)";
+    }else if(status == THREAD_STATUS_TERMINATED ){
+      return "TERMINATED";
+    }
+    return "UNKNOWN";
+  }
  }
diff --git 
a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/JavaThread.java 
b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/JavaThread.java
--- 
a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/JavaThread.java
+++ 
b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/JavaThread.java
@@ -475,4 +475,36 @@
      return access.getLastSP(addr);
    }

+
+  public void printThreadInfoOn(PrintStream out){
+
+    Oop threadOop = this.getThreadObj();
+
+    out.print("\"");
+    out.print(this.getThreadName());
+    out.print("\" #");
+    out.print(OopUtilities.threadOopGetTID(threadOop));
+    if( OopUtilities.threadOopGetDaemon(threadOop) ){
+      out.print(" daemon");
+    }
+    out.print(" prio=");
+    out.print(OopUtilities.threadOopGetPriority(threadOop));
+    out.print(" tid=");
+ out.print(String.format("0x%016x",this.getAddress().asLongValue()));
+    out.print(" nid=");
+    out.print(String.format("0x%x ",this.getOSThread().threadId()));
+    out.print(getOSThread().getThreadState().getPrintVal());
+    out.print(" [");
+    if( this.getLastJavaSP() == null){
+      out.print(String.format("0x%016x",0L));
+    } else {
+      Address maskAddress = this.getLastJavaSP().andWithMask(0xFFF);
+ out.print(this.getLastJavaSP().xorWithMask(maskAddress.asLongValue()));
+    }
+    out.println("]");
+    out.print("   java.lang.Thread.State: ");
+ out.println(OopUtilities.threadOopGetThreadStatusName(threadOop));
+    out.print("   JavaThread state: _thread_");
+    out.println(this.getThreadState().toString().toLowerCase());
+  }
  }
diff --git 
a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/OSThread.java 
b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/OSThread.java
--- 
a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/OSThread.java
+++ 
b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/OSThread.java
@@ -1,5 +1,5 @@
  /*
- * Copyright (c) 2004, 2013, Oracle and/or its affiliates. All rights 
reserved.
+ * Copyright (c) 2004, 2017, Oracle and/or its affiliates. All rights 
reserved.
   * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
   *
   * This code is free software; you can redistribute it and/or modify it
@@ -33,6 +33,19 @@
  public class OSThread extends VMObject {
      private static JIntField interruptedField;
      private static Field threadIdField;
+    private static CIntegerField threadStateField;
+
+    // ThreadStates read from underlying process
+    private static int ALLOCATED;
+    private static int INITIALIZED;
+    private static int RUNNABLE;
+    private static int MONITOR_WAIT;
+    private static int CONDVAR_WAIT;
+    private static int OBJECT_WAIT;
+    private static int BREAKPOINTED;
+    private static int SLEEPING;
+    private static int ZOMBIE;
+
      static {
          VM.registerVMInitializedObserver(new Observer() {
              public void update(Observable o, Object data) {
@@ -45,6 +58,17 @@
          Type type = db.lookupType("OSThread");
          interruptedField = type.getJIntField("_interrupted");
          threadIdField = type.getField("_thread_id");
+        threadStateField = type.getCIntegerField("_state");
+
+        ALLOCATED = db.lookupIntConstant("ALLOCATED").intValue();
+        INITIALIZED = db.lookupIntConstant("INITIALIZED").intValue();
+        RUNNABLE = db.lookupIntConstant("RUNNABLE").intValue();
+        MONITOR_WAIT = db.lookupIntConstant("MONITOR_WAIT").intValue();
+        CONDVAR_WAIT = db.lookupIntConstant("CONDVAR_WAIT").intValue();
+        OBJECT_WAIT = db.lookupIntConstant("OBJECT_WAIT").intValue();
+        BREAKPOINTED = db.lookupIntConstant("BREAKPOINTED").intValue();
+        SLEEPING = db.lookupIntConstant("SLEEPING").intValue();
+        ZOMBIE = db.lookupIntConstant("ZOMBIE").intValue();
      }

      public OSThread(Address addr) {
@@ -59,4 +83,30 @@
          return threadIdField.getJInt(addr);
      }

+    public ThreadState getThreadState() {
+        int val = (int)threadStateField.getValue(addr);
+        if (val ==  ALLOCATED) {
+            return ThreadState.ALLOCATED;
+        } else if (val ==  INITIALIZED) {
+            return ThreadState.INITIALIZED;
+        } else if (val ==  RUNNABLE) {
+            return ThreadState.RUNNABLE;
+        } else if (val ==  MONITOR_WAIT) {
+            return ThreadState.MONITOR_WAIT;
+        } else if (val ==  CONDVAR_WAIT) {
+            return ThreadState.CONDVAR_WAIT;
+        } else if (val ==  OBJECT_WAIT) {
+            return ThreadState.OBJECT_WAIT;
+        } else if (val ==  BREAKPOINTED) {
+            return ThreadState.BREAKPOINTED;
+        } else if (val ==  BREAKPOINTED) {
+            return ThreadState.BREAKPOINTED;
+        } else if (val ==  SLEEPING) {
+            return ThreadState.SLEEPING;
+        } else if (val ==  ZOMBIE) {
+            return ThreadState.ZOMBIE;
+        } else {
+            throw new RuntimeException("Illegal thread state " + val);
+        }
+    }
  }
diff --git 
a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/ThreadState.java 
b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/ThreadState.java
new file mode 100644
--- /dev/null
+++ 
b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/ThreadState.java
@@ -0,0 +1,60 @@
+/*
+ * Copyright (c) 2017, Oracle and/or its affiliates. All rights reserved.
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+ *
+ * This code is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 only, as
+ * published by the Free Software Foundation.
+ *
+ * This code is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+ * version 2 for more details (a copy is included in the LICENSE file that
+ * accompanied this code).
+ *
+ * You should have received a copy of the GNU General Public License 
version
+ * 2 along with this work; if not, write to the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
+ *
+ * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
+ * or visit www.oracle.com if you need additional information or have any
+ * questions.
+ */
+
+package sun.jvm.hotspot.runtime;
+
+/** This is a type-safe enum mirroring the ThreadState enum in
+ osThread.hpp. The conversion between the underlying ints
+ and these values is done in OSThread. */
+
+public class ThreadState {
+
+    private String printVal;
+
+    /** Memory has been allocated but not initialized */
+    public static final ThreadState ALLOCATED = new 
ThreadState("allocated");
+    /** The thread has been initialized but yet started */
+    public static final ThreadState INITIALIZED = new 
ThreadState("initialized");
+    /** Has been started and is runnable, but not necessarily running */
+    public static final ThreadState RUNNABLE = new ThreadState("runnable");
+    /** Waiting on a contended monitor lock */
+    public static final ThreadState MONITOR_WAIT = new 
ThreadState("waiting for monitor entry");
+    /** Waiting on a condition variable */
+    public static final ThreadState CONDVAR_WAIT = new 
ThreadState("waiting on condition");
+    /** Waiting on an Object.wait() call */
+    public static final ThreadState OBJECT_WAIT = new ThreadState("in 
Object.wait()");
+    /** Suspended at breakpoint */
+    public static final ThreadState BREAKPOINTED = new ThreadState("at 
breakpoint");
+    /** Thread.sleep() */
+    public static final ThreadState SLEEPING = new ThreadState("sleeping");
+    /** All done, but not reclaimed yet */
+    public static final ThreadState ZOMBIE = new ThreadState("zombie");
+
+    private ThreadState(String printVal){
+        this.printVal = printVal;
+    }
+
+    public String getPrintVal() {
+        return printVal;
+    }
+}
diff --git 
a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java 
b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
--- a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
+++ b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
@@ -1,5 +1,5 @@
  /*
- * Copyright (c) 2003, 2013, Oracle and/or its affiliates. All rights 
reserved.
+ * Copyright (c) 2003, 2017, Oracle and/or its affiliates. All rights 
reserved.
   * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
   *
   * This code is free software; you can redistribute it and/or modify it
@@ -88,6 +88,10 @@
                 out.print("----------------- ");
                 out.print(th);
                 out.println(" -----------------");
+               JavaThread jthread = (JavaThread) proxyToThread.get(th);
+               if (jthread != null) {
+                  jthread.printThreadInfoOn(out);
+               }
                 while (f != null) {
                    ClosestSymbol sym = f.closestSymbolToPC();
                    Address pc = f.pc();
diff --git 
a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java 
b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
--- 
a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
+++ 
b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
@@ -1,5 +1,5 @@
  /*
- * Copyright (c) 2002, 2013, Oracle and/or its affiliates. All rights 
reserved.
+ * Copyright (c) 2002, 2017, Oracle and/or its affiliates. All rights 
reserved.
   * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
   *
   * This code is free software; you can redistribute it and/or modify it
@@ -74,14 +74,7 @@
              int i = 1;
              for (JavaThread cur = threads.first(); cur != null; cur = 
cur.next(), i++) {
                  if (cur.isJavaThread()) {
-                    Address sp = cur.getLastJavaSP();
-                    tty.print("Thread ");
-                    cur.printThreadIDOn(tty);
-                    tty.print(": (state = " + cur.getThreadState());
-                    if (verbose) {
-                        tty.println(", current Java SP = " + sp);
-                    }
-                    tty.println(')');
+                    cur.printThreadInfoOn(tty);
                      try {
                          for (JavaVFrame vf = 
cur.getLastJavaVFrameDbg(); vf != null; vf = vf.javaSender()) {
                              Method method = vf.getMethod();
diff --git a/src/share/vm/runtime/vmStructs.cpp 
b/src/share/vm/runtime/vmStructs.cpp
--- a/src/share/vm/runtime/vmStructs.cpp
+++ b/src/share/vm/runtime/vmStructs.cpp
@@ -981,6 +981,7 @@
/************/ \
\
    volatile_nonstatic_field(OSThread, _interrupted, 
jint)                                  \
+  volatile_nonstatic_field(OSThread, _state, 
ThreadState)                                 \
\
/************************/ \
    /* OopMap and OopMapSet */ \
@@ -2186,6 +2187,7 @@
declare_integer_type(Generation::Name) \
declare_integer_type(InstanceKlass::ClassState) \
declare_integer_type(JavaThreadState) \
+ declare_integer_type(ThreadState) \
declare_integer_type(Location::Type) \
declare_integer_type(Location::Where) \
declare_integer_type(Flag::Flags) \
@@ -2443,6 +2445,20 @@
declare_constant(JavaThread::_not_terminated) \
declare_constant(JavaThread::_thread_exiting) \
\
+ /*******************/ \
+  /* JavaThreadState */                                                   \
+ /*******************/ \
+ \
+ declare_constant(ALLOCATED) \
+ declare_constant(INITIALIZED) \
+ declare_constant(RUNNABLE) \
+ declare_constant(MONITOR_WAIT) \
+ declare_constant(CONDVAR_WAIT) \
+ declare_constant(OBJECT_WAIT) \
+ declare_constant(BREAKPOINTED) \
+ declare_constant(SLEEPING) \
+ declare_constant(ZOMBIE) \
+ \
/******************************/ \
    /* Klass misc. enum constants 
*/                                        \
/******************************/ \
diff --git a/test/serviceability/sa/JhsdbThreadInfoTest.java 
b/test/serviceability/sa/JhsdbThreadInfoTest.java
new file mode 100644
--- /dev/null
+++ b/test/serviceability/sa/JhsdbThreadInfoTest.java
@@ -0,0 +1,87 @@
+/*
+ * Copyright (c) 2017, Oracle and/or its affiliates. All rights reserved.
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+ *
+ * This code is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 only, as
+ * published by the Free Software Foundation.
+ *
+ * This code is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+ * version 2 for more details (a copy is included in the LICENSE file that
+ * accompanied this code).
+ *
+ * You should have received a copy of the GNU General Public License 
version
+ * 2 along with this work; if not, write to the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
+ *
+ * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
+ * or visit www.oracle.com if you need additional information or have any
+ * questions.
+ */
+
+import jdk.test.lib.apps.LingeredApp;
+import jdk.test.lib.JDKToolLauncher;
+import jdk.test.lib.Platform;
+import jdk.test.lib.process.OutputAnalyzer;
+import jdk.test.lib.Utils;
+
+/*
+ * @test
+ * @library /test/lib
+ * @run main JhsdbThreadInfoTest
+ */
+public class JhsdbThreadInfoTest {
+
+    public static void main(String[] args) throws Exception {
+
+        if (!Platform.shouldSAAttach()) {
+            System.out.println("SA attach not expected to work - test 
skipped.");
+            return;
+        }
+
+        LingeredApp app = null;
+
+        try {
+            app = LingeredApp.startApp(Utils.getVmOptions());
+            System.out.println("Started LingeredApp with pid " + 
app.getPid());
+
+            JDKToolLauncher jhsdbLauncher = 
JDKToolLauncher.createUsingTestJDK("jhsdb");
+
+            jhsdbLauncher.addToolArg("jstack");
+            jhsdbLauncher.addToolArg("--pid");
+            jhsdbLauncher.addToolArg(Long.toString(app.getPid()));
+
+            ProcessBuilder pb = new ProcessBuilder();
+            pb.command(jhsdbLauncher.getCommand());
+            Process jhsdb = pb.start();
+
+            jhsdb.waitFor();
+
+            OutputAnalyzer out = new OutputAnalyzer(jhsdb);
+
+            System.out.println(out.getStdout());
+            System.err.println(out.getStderr());
+
+            out.shouldMatch("\".+\" #\\d+ daemon prio=\\d+ 
tid=0x[0-9a-f]+ nid=0x[0-9a-f]+ .+ \\[0x[0-9a-f]+]");
+            out.shouldMatch("\"main\" #\\d+ prio=\\d+ tid=0x[0-9a-f]+ 
nid=0x[0-9a-f]+ .+ \\[0x[0-9a-f]+]");
+            out.shouldMatch("   java.lang.Thread.State: .+");
+            out.shouldMatch("   JavaThread state: _thread_.+");
+
+            out.shouldNotContain("   java.lang.Thread.State: UNKNOWN");
+            out.stderrShouldBeEmpty();
+
+            System.out.println("Test Completed");
+
+
+        } catch (InterruptedException ie) {
+            throw new Error("Problem awaiting the child process: " + 
ie, ie);
+        } catch (Exception attachE) {
+            throw new Error("Couldn't start jhsdb, attach to 
LingeredApp or match ThreadName: " + attachE);
+
+        } finally {
+            LingeredApp.stopApp(app);
+        }
+    }
+}


Regards,
Chihiro


On 2017/06/14 16:51, Bernd Eckenfels wrote:
> I don't understand why this format is totally different from the 
> normal stack traces? At least the header with the stack names could be 
> similar?
>
> Gruss
> Bernd
> -- 
> http://bernd.eckenfels.net
> ------------------------------------------------------------------------
> *From:* serviceability-dev 
> <serviceability-dev-bounces at openjdk.java.net> on behalf of chihiro ito 
> <chihiro.ito at oracle.com>
> *Sent:* Wednesday, June 14, 2017 9:17:42 AM
> *To:* Jini George; serviceability-dev at openjdk.java.net
> *Subject:* Re: [10] RFR 8181647: jhsdb jstack could not output thread 
> name
> Hi all,
>
> I added a test case and modified previous patch including fix the
> copyright year to 2017.
> I changed to output Java thread name next the separator lines in "jhsdb
> jstack --mixed" case as following.
>
> ----------------- 32117 -----------------
> "main"
> 0x00007f6c8feafa82    __pthread_cond_timedwait + 0x132
>
> Could you possibly review for this following small change? If review is
> ok, please commit this as cito.
>
>
> Source:
> diff --git
> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
> --- 
> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
> +++ 
> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
> @@ -1,5 +1,5 @@
>   /*
> - * Copyright (c) 2003, 2013, Oracle and/or its affiliates. All rights
> reserved.
> + * Copyright (c) 2003, 2017, Oracle and/or its affiliates. All rights
> reserved.
>    * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
>    *
>    * This code is free software; you can redistribute it and/or modify it
> @@ -88,6 +88,12 @@
>                  out.print("----------------- ");
>                  out.print(th);
>                  out.println(" -----------------");
> +               JavaThread jthread = (JavaThread) proxyToThread.get(th);
> +               if (jthread != null) {
> +                   out.print("\"");
> +                   out.print(jthread.getThreadName());
> +                   out.println("\"");
> +               }
>                  while (f != null) {
>                     ClosestSymbol sym = f.closestSymbolToPC();
>                     Address pc = f.pc();
> diff --git
> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java 
>
> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
> ---
> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
> +++
> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
> @@ -1,5 +1,5 @@
>   /*
> - * Copyright (c) 2002, 2013, Oracle and/or its affiliates. All rights
> reserved.
> + * Copyright (c) 2002, 2017, Oracle and/or its affiliates. All rights
> reserved.
>    * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
>    *
>    * This code is free software; you can redistribute it and/or modify it
> @@ -75,7 +75,9 @@
>               for (JavaThread cur = threads.first(); cur != null; cur =
> cur.next(), i++) {
>                   if (cur.isJavaThread()) {
>                       Address sp = cur.getLastJavaSP();
> -                    tty.print("Thread ");
> +                    tty.print("\"");
> +                    tty.print(cur.getThreadName());
> +                    tty.print("\" nid=");
>                       cur.printThreadIDOn(tty);
>                       tty.print(": (state = " + cur.getThreadState());
>                       if (verbose) {
> diff --git a/test/serviceability/sa/JhsdbThreadNameTest.java
> b/test/serviceability/sa/JhsdbThreadNameTest.java
> new file mode 100644
> --- /dev/null
> +++ b/test/serviceability/sa/JhsdbThreadNameTest.java
> @@ -0,0 +1,107 @@
> +/*
> + * Copyright (c) 2017, Oracle and/or its affiliates. All rights reserved.
> + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> + *
> + * This code is free software; you can redistribute it and/or modify it
> + * under the terms of the GNU General Public License version 2 only, as
> + * published by the Free Software Foundation.
> + *
> + * This code is distributed in the hope that it will be useful, but 
> WITHOUT
> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
> + * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
> + * version 2 for more details (a copy is included in the LICENSE file 
> that
> + * accompanied this code).
> + *
> + * You should have received a copy of the GNU General Public License
> version
> + * 2 along with this work; if not, write to the Free Software Foundation,
> + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
> + *
> + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 
> 94065 USA
> + * or visit www.oracle.com <http://www.oracle.com> if you need 
> additional information or have any
> + * questions.
> + */
> +
> +import java.util.ArrayList;
> +import java.util.Arrays;
> +import java.util.List;
> +import java.util.function.Consumer;
> +
> +import jdk.test.lib.apps.LingeredApp;
> +import jdk.test.lib.JDKToolLauncher;
> +import jdk.test.lib.Platform;
> +import jdk.test.lib.process.OutputAnalyzer;
> +import jdk.test.lib.Utils;
> +
> +/*
> + * @test
> + * @library /test/lib
> + * @run main/othervm JhsdbThreadNameTest
> + */
> +public class JhsdbThreadNameTest {
> +
> +    private static String notMixedModeThreadNames[] =
> {"Common-Cleaner", "Signal Dispatcher", "Finalizer", "Reference
> Handler", "main"};
> +    private static String mixedModeThreadNames[] = {"C2
> CompilerThread0", "C1 CompilerThread1", "Sweeper thread", "Service 
> Thread"};
> +
> +    private static void startHsdbJstack(boolean mixed) throws Exception {
> +
> +        LingeredApp app = null;
> +
> +        try {
> +            List<String> vmArgs = new ArrayList<String>();
> +            vmArgs.add("-Xmx10m");
> +            vmArgs.addAll(Utils.getVmOptions());
> +
> +            app = LingeredApp.startApp(vmArgs);
> +            System.out.println("Started LingeredApp with pid " +
> app.getPid());
> +
> +            JDKToolLauncher jhsdbLauncher =
> JDKToolLauncher.createUsingTestJDK("jhsdb");
> +
> +            jhsdbLauncher.addToolArg("jstack");
> +            jhsdbLauncher.addToolArg("--pid");
> + jhsdbLauncher.addToolArg(Long.toString(app.getPid()));
> +
> +            if (mixed) {
> +                jhsdbLauncher.addToolArg("--mixed");
> +            }
> +            ProcessBuilder pb = new ProcessBuilder();
> +            pb.command(jhsdbLauncher.getCommand());
> +            Process jhsdb = pb.start();
> +
> +            jhsdb.waitFor();
> +
> +            OutputAnalyzer out = new OutputAnalyzer(jhsdb);
> +
> +
> Arrays.stream(notMixedModeThreadNames).map(JhsdbThreadNameTest::addQuotationMarks).forEach(out::shouldContain);
> +            Consumer<String> testMethod = null;
> +            if (mixed) {
> +                testMethod = out::shouldContain;
> +            } else {
> +                testMethod = out::shouldNotContain;
> +            }
> +
> Arrays.stream(mixedModeThreadNames).map(JhsdbThreadNameTest::addQuotationMarks).forEach(testMethod);
> +
> +        } catch (InterruptedException ie) {
> +            throw new Error("Problem awaiting the child process: " +
> ie, ie);
> +        } catch (Exception attachE) {
> +            throw new Error("Couldn't start jhsdb, attach to
> LingeredApp or match ThreadName: " + attachE);
> +
> +        } finally {
> +            LingeredApp.stopApp(app);
> +        }
> +    }
> +
> +    private static String addQuotationMarks(String str) {
> +        return "\"" + str + "\"";
> +    }
> +
> +    public static void main(String[] args) throws Exception {
> +
> +        if (!Platform.shouldSAAttach()) {
> +            System.out.println("SA attach not expected to work - test
> skipped.");
> +            return;
> +        }
> +
> +        startHsdbJstack(true);
> +        startHsdbJstack(false);
> +    }
> +}
>
>
> Regards,
> Chihiro
>
>
> On 2017/06/08 18:04, chihiro ito wrote:
> > Hi Jini,
> >
> > Thank you for your advices. I try to add the test case and modify the
> > copyright year to 2017.
> > Basically, I agree with your idea, but I think that the separator line
> > should finally be the same as the output of the jstack command. I
> > worry which is better way.
> >
> > Thanks,
> > Chihiro
> >
> > On 2017/06/08 16:42, Jini George wrote:
> >> Hi Chihiro,
> >>
> >> Thank you for making this useful change. Your changes look good.
> >>
> >> It would be great though if you could add a test case for this. Could
> >> you also modify the copyright year to 2017 ? One additional
> >> suggestion: The addition of the thread name makes the separator lines
> >> unaligned in the pstack/jstack --mixed cases. Like:
> >>
> >> ----------------- "Service Thread" nid=16051 -----------------
> >> and
> >> ----------------- nid=16052 -----------------
> >>
> >> So I am wondering if it would make sense to have the name printed out
> >> on a separate line to keep the separator lines aligned. But this is a
> >> nit, and I would leave it to you to decide on this.
> >>
> >> Thanks,
> >> Jini (Not a (R)eviewer)
> >>
> >> On 6/7/2017 3:16 PM, chihiro ito wrote:
> >>> Hi all,
> >>>
> >>> I changed to output Java thread name in jhsdb jstack as following.
> >>>
> >>> jhsdb jstack --pid 25879
> >>> "main" nid=25884: (state = BLOCKED)
> >>>
> >>> jhsdb jstack --mixed --pid 25879
> >>> ----------------- "main" nid=25884 -----------------
> >>>
> >>> Could you possibly review for this following small change? If review
> >>> is ok, please commit this as cito.
> >>>
> >>> Source:
> >>> diff --git
> >>> 
> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
> >>> 
> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
> >>> ---
> >>> 
> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
> >>> +++
> >>> 
> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
> >>> @@ -86,6 +86,13 @@
> >>>              try {
> >>>                 CFrame f = cdbg.topFrameForThread(th);
> >>>                 out.print("----------------- ");
> >>> +               JavaThread jthread = (JavaThread)
> >>> proxyToThread.get(th);
> >>> +               if (jthread != null) {
> >>> +                   out.print("\"");
> >>> + out.print(jthread.getThreadName());
> >>> +                   out.print("\" ");
> >>> +               }
> >>> +               out.print("nid=");
> >>>                 out.print(th);
> >>>                 out.println(" -----------------");
> >>>                 while (f != null) {
> >>> diff --git
> >>> 
> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java 
>
> >>> 
> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java 
>
> >>>
> >>> ---
> >>> 
> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
> >>> +++
> >>> 
> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
> >>> @@ -75,7 +75,9 @@
> >>>              for (JavaThread cur = threads.first(); cur != null; cur
> >>> = cur.next(), i++) {
> >>>                  if (cur.isJavaThread()) {
> >>>                      Address sp = cur.getLastJavaSP();
> >>> -                    tty.print("Thread ");
> >>> +                    tty.print("\"");
> >>> + tty.print(cur.getThreadName());
> >>> +                    tty.print("\" nid=");
> >>>                      cur.printThreadIDOn(tty);
> >>>                      tty.print(": (state = " + cur.getThreadState());
> >>>                      if (verbose) {
> >>>
> >>> Regards,
> >>> Chihiro
> >>>
> >>
> >
>
> -- 
>
> Chihiro Ito | Principal Consultant | +81.90.6148.8815
> Oracle <http://www.oracle.com> Consultant
> ORACLE Japan | Akasaka Center Bldg. | Motoakasaka 1-3-13 | 1070051
> Minato-ku, Tokyo, JAPAN
>
> Oracle is committed to developing practices and products that help
> protect the environment <http://www.oracle.com/commitment>

-- 

Chihiro Ito | Principal Consultant | +81.90.6148.8815
Oracle <http://www.oracle.com> Consultant
ORACLE Japan | Akasaka Center Bldg. | Motoakasaka 1-3-13 | 1070051 
Minato-ku, Tokyo, JAPAN

Oracle is committed to developing practices and products that help 
protect the environment <http://www.oracle.com/commitment>

From thomas.stuefe at gmail.com  Sun Jun 18 04:50:33 2017
From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=)
Date: Sun, 18 Jun 2017 04:50:33 +0000
Subject: RFR(xs): 8181419: Race in jdwp invoker handling may lead to
 crashes or invalid results
In-Reply-To: <f61fa99a-0da4-eb82-657d-5dc45fd2f7e8@oracle.com>
References: <CAA-vtUwt_h6o5LjajKEGJbzD12kdYQ7cJN-41tKNFkNhs9SCug@mail.gmail.com>
 <CAA-vtUw+RGyJXOpnsuaRfZW-D3Eh5RKeGOa1w7=e=CYhu7SF0g@mail.gmail.com>
 <9e1963f0-9d27-241a-f1de-9cfea607b91d@oracle.com>
 <8a82b2e6-d29a-1e1e-c16d-ff8560ee0203@oracle.com>
 <CAA-vtUyzkw-Zb-s+3yJNHe42hQ5LxmGFt8MiOVP5t6nW1pEqfQ@mail.gmail.com>
 <23f2b805-3a39-93ca-42f4-a589bf98cdd6@oracle.com>
 <f61fa99a-0da4-eb82-657d-5dc45fd2f7e8@oracle.com>
Message-ID: <CAA-vtUywVeKup9-q4H1pb9W-dMHB2D7d9nt1bxzz4f0ZETZs3w@mail.gmail.com>

Thank you Serguei!

..Thomas

On Sun 18. Jun 2017 at 00:42, serguei.spitsyn at oracle.com <
serguei.spitsyn at oracle.com> wrote:

> Hi Thomas,
>
> An update on this.
>
> I ran the nsk.jdi and jtreg jdk.jdi tests on Linux-x64.
> The results are pretty good.
> One jtreg test failed:
>  com/sun/jdi/LineNumberInfo.java: Test javac regressions in the generation
> of line number info
>
> This failure seems to be unrelated to your fix as it is about incorrect
> line number information.
>
> Thanks,
> Serguei
>
>
>
>
> On 6/15/17 11:00, serguei.spitsyn at oracle.com wrote:
>
> Hi Thomas,
>
> Sure.
>
>
> I'll also run some internal jdi tests.
>
> Thanks,
> Serguei
>
>
> On 6/13/17 21:17, Thomas St?fe wrote:
>
> Hi Sergey,
>
> thank you for the review!
>
> I ran jdi jtreg tests on both AIX and Linux x64, I had some issues but
> they seemed preexisting (same issues without my patch). Could you please
> also run tests you think are necessary on your platforms? Thanks!
>
> best Regards, Thomas
>
> On Wed, Jun 14, 2017 at 1:36 AM, serguei.spitsyn at oracle.com <
> serguei.spitsyn at oracle.com> wrote:
>
>> On 6/13/17 14:16, serguei.spitsyn at oracle.com wrote:
>>
>> Hi Thomas,
>>
>> The fix looks great to me.
>>
>>
>> Forgot to say that the copyright comment need an update.
>>
>> Thanks,
>> Serguei
>>
>>
>> Thank you for identifying and fixing the problem!
>>
>> Thanks,
>> Serguei
>>
>>
>> On 6/13/17 06:55, Thomas St?fe wrote:
>>
>> Ping... Anyone?
>>
>> On Thu, Jun 1, 2017 at 2:18 PM, Thomas St?fe <thomas.stuefe at gmail.com>
>> wrote:
>>
>>> Hi all,
>>>
>>> please take a look at this proposed fix for a theoretical race in the
>>> jdwp library.
>>>
>>> Issue: https://bugs.openjdk.java.net/browse/JDK-8181419
>>> webrev:
>>> http://cr.openjdk.java.net/~stuefe/webrevs/8181419-Race-in-jdwp-invoker-handling-may-lead-to-crashes-or-invalid-results/webrev.00/webrev/
>>>
>>> In short, this is an addition to Severin's fix to the jdwp invoke
>>> handling (https://bugs.openjdk.java.net/browse/JDK-8153711).
>>>
>>> We have a potential race condition where the delayed cleanup of the
>>> saved returnvalue object reference and the exception reference (released
>>> in deletePotentiallySavedGlobalRefs() ) may be overtaken by a new request
>>> which populates the thread request structure anew. If this happens,
>>> deletePotentiallySavedGlobalRefs() may actually release the return value /
>>> exception references of the follow up request, if that one was already
>>> processed.
>>>
>>> The solution I choose is safe and conservative. We still release both
>>> references, but use the locally saved JNI references. We just avoid
>>> accessing the thread local request structure after it has been cleared for
>>> reuse. This keeps timing and locking behaviour unchanged.
>>>
>>> I am currently running jtreg tests for com/sun/jdi on AIX and Linux.
>>>
>>> Kind Regards, Thomas
>>>
>>
>>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170618/a3bb6332/attachment.html>

From sgehwolf at redhat.com  Mon Jun 19 09:10:53 2017
From: sgehwolf at redhat.com (Severin Gehwolf)
Date: Mon, 19 Jun 2017 11:10:53 +0200
Subject: RFR(xs): 8181419: Race in jdwp invoker handling may lead to
 crashes or invalid results
In-Reply-To: <CAA-vtUw+RGyJXOpnsuaRfZW-D3Eh5RKeGOa1w7=e=CYhu7SF0g@mail.gmail.com>
References: <CAA-vtUwt_h6o5LjajKEGJbzD12kdYQ7cJN-41tKNFkNhs9SCug@mail.gmail.com>
 <CAA-vtUw+RGyJXOpnsuaRfZW-D3Eh5RKeGOa1w7=e=CYhu7SF0g@mail.gmail.com>
Message-ID: <1497863453.3566.3.camel@redhat.com>

Hi Thomas,

On Tue, 2017-06-13 at 15:55 +0200, Thomas St?fe wrote:
> Ping... Anyone?
> 
> On Thu, Jun 1, 2017 at 2:18 PM, Thomas St?fe <thomas.stuefe at gmail.com
> > wrote:
> > Hi all,
> > 
> > please take a look at this proposed fix for a theoretical race in
> > the jdwp library.
> > 
> > Issue:?https://bugs.openjdk.java.net/browse/JDK-8181419
> > webrev:?http://cr.openjdk.java.net/~stuefe/webrevs/8181419-Race-in-jdwp-invoker-handling-may-lead-to-crashes-or-invalid-results/webrev.00/webrev/
> > 
> > In short, this is an addition to Severin's fix to the jdwp invoke
> > handling (https://bugs.openjdk.java.net/browse/JDK-8153711).
> > 
> > We have a potential race condition where the delayed cleanup of the
> > saved returnvalue object reference and the exception reference
> > (released in?deletePotentiallySavedGlobalRefs() ) may be overtaken
> > by a new request which populates the thread request structure anew.
> > If this happens, deletePotentiallySavedGlobalRefs() may actually
> > release the return value / exception references of the follow up
> > request, if that one was already processed.
> > 
> > The solution I choose is safe and conservative. We still release
> > both references, but use the locally saved JNI references. We just
> > avoid accessing the thread local request structure after it has
> > been cleared for reuse. This keeps timing and locking behaviour
> > unchanged.
> > 
> > I am currently running jtreg tests for com/sun/jdi on AIX and
> > Linux.

The fix makes sense to me. Looks good! I'm not an OpenJDK Reviewer.

Cheers,
Severin

From thomas.stuefe at gmail.com  Mon Jun 19 09:23:26 2017
From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=)
Date: Mon, 19 Jun 2017 11:23:26 +0200
Subject: RFR(xs): 8181419: Race in jdwp invoker handling may lead to
 crashes or invalid results
In-Reply-To: <1497863453.3566.3.camel@redhat.com>
References: <CAA-vtUwt_h6o5LjajKEGJbzD12kdYQ7cJN-41tKNFkNhs9SCug@mail.gmail.com>
 <CAA-vtUw+RGyJXOpnsuaRfZW-D3Eh5RKeGOa1w7=e=CYhu7SF0g@mail.gmail.com>
 <1497863453.3566.3.camel@redhat.com>
Message-ID: <CAA-vtUy5MZ7c3qBONdUM-uAqH4pGu7xCvYJnrKCqMpJECdavUQ@mail.gmail.com>

Thank you Severin!

On Mon, Jun 19, 2017 at 11:10 AM, Severin Gehwolf <sgehwolf at redhat.com>
wrote:

> Hi Thomas,
>
> On Tue, 2017-06-13 at 15:55 +0200, Thomas St?fe wrote:
> > Ping... Anyone?
> >
> > On Thu, Jun 1, 2017 at 2:18 PM, Thomas St?fe <thomas.stuefe at gmail.com
> > > wrote:
> > > Hi all,
> > >
> > > please take a look at this proposed fix for a theoretical race in
> > > the jdwp library.
> > >
> > > Issue: https://bugs.openjdk.java.net/browse/JDK-8181419
> > > webrev: http://cr.openjdk.java.net/~stuefe/webrevs/
> 8181419-Race-in-jdwp-invoker-handling-may-lead-to-crashes-
> or-invalid-results/webrev.00/webrev/
> > >
> > > In short, this is an addition to Severin's fix to the jdwp invoke
> > > handling (https://bugs.openjdk.java.net/browse/JDK-8153711).
> > >
> > > We have a potential race condition where the delayed cleanup of the
> > > saved returnvalue object reference and the exception reference
> > > (released in deletePotentiallySavedGlobalRefs() ) may be overtaken
> > > by a new request which populates the thread request structure anew.
> > > If this happens, deletePotentiallySavedGlobalRefs() may actually
> > > release the return value / exception references of the follow up
> > > request, if that one was already processed.
> > >
> > > The solution I choose is safe and conservative. We still release
> > > both references, but use the locally saved JNI references. We just
> > > avoid accessing the thread local request structure after it has
> > > been cleared for reuse. This keeps timing and locking behaviour
> > > unchanged.
> > >
> > > I am currently running jtreg tests for com/sun/jdi on AIX and
> > > Linux.
>
> The fix makes sense to me. Looks good! I'm not an OpenJDK Reviewer.
>
> Cheers,
> Severin
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170619/07de6a2d/attachment.html>

From christoph.langer at sap.com  Mon Jun 19 09:31:58 2017
From: christoph.langer at sap.com (Langer, Christoph)
Date: Mon, 19 Jun 2017 09:31:58 +0000
Subject: RFR(xs): 8181419: Race in jdwp invoker handling may lead to
 crashes or invalid results
In-Reply-To: <CAA-vtUz3yDZ_j9nfv14f7qcn62_ZNnfG6yC-gK3sDpQKren-eQ@mail.gmail.com>
References: <CAA-vtUwt_h6o5LjajKEGJbzD12kdYQ7cJN-41tKNFkNhs9SCug@mail.gmail.com>
 <CAA-vtUw+RGyJXOpnsuaRfZW-D3Eh5RKeGOa1w7=e=CYhu7SF0g@mail.gmail.com>
 <9e1963f0-9d27-241a-f1de-9cfea607b91d@oracle.com>
 <CAA-vtUz3yDZ_j9nfv14f7qcn62_ZNnfG6yC-gK3sDpQKren-eQ@mail.gmail.com>
Message-ID: <4f6445a8bb9a49b2a236b65d879da7f9@sap.com>

Hi Thomas,

this looks good to me. You still need to update the current year in the copyright header.

Best regards
Christoph

From: serviceability-dev [mailto:serviceability-dev-bounces at openjdk.java.net] On Behalf Of Thomas St?fe
Sent: Donnerstag, 15. Juni 2017 08:31
To: serviceability-dev at openjdk.java.net
Subject: Re: RFR(xs): 8181419: Race in jdwp invoker handling may lead to crashes or invalid results

Hi Folks,

may I please have a second reviewer?

Thank you!

Kind Regards, Thomas

On Tue, Jun 13, 2017 at 11:16 PM, serguei.spitsyn at oracle.com<mailto:serguei.spitsyn at oracle.com> <serguei.spitsyn at oracle.com<mailto:serguei.spitsyn at oracle.com>> wrote:
Hi Thomas,

The fix looks great to me.
Thank you for identifying and fixing the problem!

Thanks,
Serguei



On 6/13/17 06:55, Thomas St?fe wrote:
Ping... Anyone?

On Thu, Jun 1, 2017 at 2:18 PM, Thomas St?fe <thomas.stuefe at gmail.com<mailto:thomas.stuefe at gmail.com>> wrote:
Hi all,

please take a look at this proposed fix for a theoretical race in the jdwp library.

Issue: https://bugs.openjdk.java.net/browse/JDK-8181419
webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8181419-Race-in-jdwp-invoker-handling-may-lead-to-crashes-or-invalid-results/webrev.00/webrev/<http://cr.openjdk.java.net/%7Estuefe/webrevs/8181419-Race-in-jdwp-invoker-handling-may-lead-to-crashes-or-invalid-results/webrev.00/webrev/>

In short, this is an addition to Severin's fix to the jdwp invoke handling (https://bugs.openjdk.java.net/browse/JDK-8153711).

We have a potential race condition where the delayed cleanup of the saved returnvalue object reference and the exception reference (released in deletePotentiallySavedGlobalRefs() ) may be overtaken by a new request which populates the thread request structure anew. If this happens, deletePotentiallySavedGlobalRefs() may actually release the return value / exception references of the follow up request, if that one was already processed.

The solution I choose is safe and conservative. We still release both references, but use the locally saved JNI references. We just avoid accessing the thread local request structure after it has been cleared for reuse. This keeps timing and locking behaviour unchanged.

I am currently running jtreg tests for com/sun/jdi on AIX and Linux.

Kind Regards, Thomas



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170619/d0f6a0c3/attachment-0001.html>

From thomas.stuefe at gmail.com  Mon Jun 19 09:34:36 2017
From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=)
Date: Mon, 19 Jun 2017 11:34:36 +0200
Subject: RFR(xs): 8181419: Race in jdwp invoker handling may lead to
 crashes or invalid results
In-Reply-To: <4f6445a8bb9a49b2a236b65d879da7f9@sap.com>
References: <CAA-vtUwt_h6o5LjajKEGJbzD12kdYQ7cJN-41tKNFkNhs9SCug@mail.gmail.com>
 <CAA-vtUw+RGyJXOpnsuaRfZW-D3Eh5RKeGOa1w7=e=CYhu7SF0g@mail.gmail.com>
 <9e1963f0-9d27-241a-f1de-9cfea607b91d@oracle.com>
 <CAA-vtUz3yDZ_j9nfv14f7qcn62_ZNnfG6yC-gK3sDpQKren-eQ@mail.gmail.com>
 <4f6445a8bb9a49b2a236b65d879da7f9@sap.com>
Message-ID: <CAA-vtUzb8QuzEB8wJp6FsSJ0Jg+7aHZXCzQ7KzW_9J1b4pdWSA@mail.gmail.com>

Thank you Christoph!

On Mon, Jun 19, 2017 at 11:31 AM, Langer, Christoph <
christoph.langer at sap.com> wrote:

> Hi Thomas,
>
>
>
> this looks good to me. You still need to update the current year in the
> copyright header.
>
>
>
> Best regards
>
> Christoph
>
>
>
> *From:* serviceability-dev [mailto:serviceability-dev-
> bounces at openjdk.java.net] *On Behalf Of *Thomas St?fe
> *Sent:* Donnerstag, 15. Juni 2017 08:31
> *To:* serviceability-dev at openjdk.java.net
> *Subject:* Re: RFR(xs): 8181419: Race in jdwp invoker handling may lead
> to crashes or invalid results
>
>
>
> Hi Folks,
>
>
>
> may I please have a second reviewer?
>
>
>
> Thank you!
>
>
>
> Kind Regards, Thomas
>
>
>
> On Tue, Jun 13, 2017 at 11:16 PM, serguei.spitsyn at oracle.com <
> serguei.spitsyn at oracle.com> wrote:
>
> Hi Thomas,
>
> The fix looks great to me.
> Thank you for identifying and fixing the problem!
>
> Thanks,
> Serguei
>
>
>
>
> On 6/13/17 06:55, Thomas St?fe wrote:
>
> Ping... Anyone?
>
>
>
> On Thu, Jun 1, 2017 at 2:18 PM, Thomas St?fe <thomas.stuefe at gmail.com>
> wrote:
>
> Hi all,
>
>
>
> please take a look at this proposed fix for a theoretical race in the jdwp
> library.
>
>
>
> Issue: https://bugs.openjdk.java.net/browse/JDK-8181419
>
> webrev: http://cr.openjdk.java.net/~stuefe/webrevs/
> 8181419-Race-in-jdwp-invoker-handling-may-lead-to-crashes-
> or-invalid-results/webrev.00/webrev/
>
>
>
> In short, this is an addition to Severin's fix to the jdwp invoke handling
> (https://bugs.openjdk.java.net/browse/JDK-8153711).
>
>
>
> We have a potential race condition where the delayed cleanup of the saved
> returnvalue object reference and the exception reference (released in
> deletePotentiallySavedGlobalRefs() ) may be overtaken by a new request
> which populates the thread request structure anew. If this happens,
> deletePotentiallySavedGlobalRefs() may actually release the return value
> / exception references of the follow up request, if that one was already
> processed.
>
>
>
> The solution I choose is safe and conservative. We still release both
> references, but use the locally saved JNI references. We just avoid
> accessing the thread local request structure after it has been cleared for
> reuse. This keeps timing and locking behaviour unchanged.
>
>
>
> I am currently running jtreg tests for com/sun/jdi on AIX and Linux.
>
>
>
> Kind Regards, Thomas
>
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170619/c593eede/attachment.html>

From ujwal.vangapally at oracle.com  Mon Jun 19 10:52:51 2017
From: ujwal.vangapally at oracle.com (Ujwal Vangapally)
Date: Mon, 19 Jun 2017 16:22:51 +0530
Subject: RFR: JDK-8173180 VirtualMachine.startLocalManagementAgent() returns
 URI with unreliable IP address
Message-ID: <247efda9-804c-7ed6-e42c-6588eec78716@oracle.com>

Hi,

Kindly review the fix for bug below

https://bugs.openjdk.java.net/browse/JDK-8173180

webrev: 
http://cr.openjdk.java.net/~uvangapally/webrev/2017/8173180/webrev.00/

Thanks,

Ujwal

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170619/aa5b6f74/attachment.html>

From daniel.fuchs at oracle.com  Mon Jun 19 16:53:04 2017
From: daniel.fuchs at oracle.com (Daniel Fuchs)
Date: Mon, 19 Jun 2017 17:53:04 +0100
Subject: RFR: JDK-8173180 VirtualMachine.startLocalManagementAgent()
 returns URI with unreliable IP address
In-Reply-To: <247efda9-804c-7ed6-e42c-6588eec78716@oracle.com>
References: <247efda9-804c-7ed6-e42c-6588eec78716@oracle.com>
Message-ID: <a8aa2492-cd21-580d-5e4c-b530628ff94c@oracle.com>

Hi,

If I'm not mistaken then this will make it impossible
for earlier release to interoperate with newer releases
as the LocalRMIClientSocketFactory class will not be
present the client tries to deserialize the stub.

best regards,

-- daniel

On 19/06/2017 11:52, Ujwal Vangapally wrote:
> Hi,
> 
> Kindly review the fix for bug below
> 
> https://bugs.openjdk.java.net/browse/JDK-8173180
> 
> webrev: 
> http://cr.openjdk.java.net/~uvangapally/webrev/2017/8173180/webrev.00/
> 
> Thanks,
> 
> Ujwal
> 


From mandy.chung at oracle.com  Mon Jun 19 19:47:39 2017
From: mandy.chung at oracle.com (Mandy Chung)
Date: Mon, 19 Jun 2017 12:47:39 -0700
Subject: Review Request JDK-8181834: Broken link in jdk.jdi module
 documentation
Message-ID: <2823421C-9D7D-400A-8DB9-5F89B7BC0365@oracle.com>

Serguei,

Can you review this patch fixing the link to JPDA?

Mandy



diff --git a/src/jdk.jdi/share/classes/module-info.java b/src/jdk.jdi/share/classes/module-info.java
--- a/src/jdk.jdi/share/classes/module-info.java
+++ b/src/jdk.jdi/share/classes/module-info.java
@@ -39,12 +39,13 @@
  * creation, etc. The ability to inspect a suspended thread's state, local
  * variables, stack backtrace, etc.
  * <p>
- * JDI is the highest-layer of the Java Platform Debugger Architecture (JPDA).
- * For more information on the Java Platform Debugger Architecture, see the <a
- * href="{@docRoot}/../../../../technotes/guides/jpda/index.html"> Java
- * Platform Debugger Architecture documentation</a> for this release and the <a
- * href="http://java.sun.com/products/jpda">Java Platform Debugger Architecture
- * website</a>.
+ * JDI is the highest-layer of the 
+ * <a href="${docRoot}/../specs/jpda/jpda.html">
+ * Java Platform Debugger Architecture (JPDA)</a>.
+ * <p>
+ * This module includes a simple command-line debugger,
+ * <em>{@index jdb jdb tool}</em>.
+ *
  * <h3>Global Exceptions</h3>
  * <p>
  * This section documents exceptions which apply to the entire API and are thus
@@ -102,10 +103,6 @@
  *   unloaded.
  * </blockquote>
  *
- * <h3>jdb</h3>
- *
- * <em>{@index jdb jdb tool}</em> is a simple command-line debugger provided
- * in this module.
  *
  * <dl style="font-family:'DejaVu Sans', Arial, Helvetica, sans serif">
  * <dt class="simpleTagLabel">Tool Guides:
@@ -119,6 +116,8 @@
  *
  * @moduleGraph
  * @since 9
+ * @see <a href="${docRoot}/../specs/jpda/jpda.html">
+ * Java Platform Debugger Architecture (JPDA)</a>
  */
 module jdk.jdi {
     requires jdk.attach;


From serguei.spitsyn at oracle.com  Mon Jun 19 20:17:01 2017
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Mon, 19 Jun 2017 13:17:01 -0700
Subject: Review Request JDK-8181834: Broken link in jdk.jdi module
 documentation
In-Reply-To: <2823421C-9D7D-400A-8DB9-5F89B7BC0365@oracle.com>
References: <2823421C-9D7D-400A-8DB9-5F89B7BC0365@oracle.com>
Message-ID: <6b8f9243-72d9-53c7-409f-f670598aab59@oracle.com>

Hi Mandy,

I've built the docs with your patch and the link is still not resolved.
It points to the docs/api/specs/jpda/jpda.html

But it is resolved if the patch is corrected with:
   + * <a href="${docRoot}/../../specs/jpda/jpda.html">


Is it 9 or 10?

Thanks,
Serguei


On 6/19/17 12:47, Mandy Chung wrote:
> Serguei,
>
> Can you review this patch fixing the link to JPDA?
>
> Mandy
>
>
>
> diff --git a/src/jdk.jdi/share/classes/module-info.java b/src/jdk.jdi/share/classes/module-info.java
> --- a/src/jdk.jdi/share/classes/module-info.java
> +++ b/src/jdk.jdi/share/classes/module-info.java
> @@ -39,12 +39,13 @@
>    * creation, etc. The ability to inspect a suspended thread's state, local
>    * variables, stack backtrace, etc.
>    * <p>
> - * JDI is the highest-layer of the Java Platform Debugger Architecture (JPDA).
> - * For more information on the Java Platform Debugger Architecture, see the <a
> - * href="{@docRoot}/../../../../technotes/guides/jpda/index.html"> Java
> - * Platform Debugger Architecture documentation</a> for this release and the <a
> - * href="http://java.sun.com/products/jpda">Java Platform Debugger Architecture
> - * website</a>.
> + * JDI is the highest-layer of the
> + * <a href="${docRoot}/../specs/jpda/jpda.html">
> + * Java Platform Debugger Architecture (JPDA)</a>.
> + * <p>
> + * This module includes a simple command-line debugger,
> + * <em>{@index jdb jdb tool}</em>.
> + *
>    * <h3>Global Exceptions</h3>
>    * <p>
>    * This section documents exceptions which apply to the entire API and are thus
> @@ -102,10 +103,6 @@
>    *   unloaded.
>    * </blockquote>
>    *
> - * <h3>jdb</h3>
> - *
> - * <em>{@index jdb jdb tool}</em> is a simple command-line debugger provided
> - * in this module.
>    *
>    * <dl style="font-family:'DejaVu Sans', Arial, Helvetica, sans serif">
>    * <dt class="simpleTagLabel">Tool Guides:
> @@ -119,6 +116,8 @@
>    *
>    * @moduleGraph
>    * @since 9
> + * @see <a href="${docRoot}/../specs/jpda/jpda.html">
> + * Java Platform Debugger Architecture (JPDA)</a>
>    */
>   module jdk.jdi {
>       requires jdk.attach;
>


From mandy.chung at oracle.com  Mon Jun 19 20:23:06 2017
From: mandy.chung at oracle.com (Mandy Chung)
Date: Mon, 19 Jun 2017 13:23:06 -0700
Subject: Review Request JDK-8181834: Broken link in jdk.jdi module
 documentation
In-Reply-To: <6b8f9243-72d9-53c7-409f-f670598aab59@oracle.com>
References: <2823421C-9D7D-400A-8DB9-5F89B7BC0365@oracle.com>
 <6b8f9243-72d9-53c7-409f-f670598aab59@oracle.com>
Message-ID: <7AF44A54-29A9-4774-93AA-0ADCC575282A@oracle.com>


> On Jun 19, 2017, at 1:17 PM, serguei.spitsyn at oracle.com wrote:
> 
> Hi Mandy,
> 
> I've built the docs with your patch and the link is still not resolved.
> It points to the docs/api/specs/jpda/jpda.html
> 

It should point to docs/specs/jpda/jpda.html. I verified in my build.
Are you sure your build is correct?

> But it is resolved if the patch is corrected with:
>  + * <a href="${docRoot}/../../specs/jpda/jpda.html?>

> 
> Is it 9 or 10?
> 

I want to fix this simple docs bug in JDK 9.

Mandy

> Thanks,
> Serguei
> 
> 
> On 6/19/17 12:47, Mandy Chung wrote:
>> Serguei,
>> 
>> Can you review this patch fixing the link to JPDA?
>> 
>> Mandy
>> 
>> 
>> 
>> diff --git a/src/jdk.jdi/share/classes/module-info.java b/src/jdk.jdi/share/classes/module-info.java
>> --- a/src/jdk.jdi/share/classes/module-info.java
>> +++ b/src/jdk.jdi/share/classes/module-info.java
>> @@ -39,12 +39,13 @@
>>   * creation, etc. The ability to inspect a suspended thread's state, local
>>   * variables, stack backtrace, etc.
>>   * <p>
>> - * JDI is the highest-layer of the Java Platform Debugger Architecture (JPDA).
>> - * For more information on the Java Platform Debugger Architecture, see the <a
>> - * href="{@docRoot}/../../../../technotes/guides/jpda/index.html"> Java
>> - * Platform Debugger Architecture documentation</a> for this release and the <a
>> - * href="http://java.sun.com/products/jpda">Java Platform Debugger Architecture
>> - * website</a>.
>> + * JDI is the highest-layer of the
>> + * <a href="${docRoot}/../specs/jpda/jpda.html">
>> + * Java Platform Debugger Architecture (JPDA)</a>.
>> + * <p>
>> + * This module includes a simple command-line debugger,
>> + * <em>{@index jdb jdb tool}</em>.
>> + *
>>   * <h3>Global Exceptions</h3>
>>   * <p>
>>   * This section documents exceptions which apply to the entire API and are thus
>> @@ -102,10 +103,6 @@
>>   *   unloaded.
>>   * </blockquote>
>>   *
>> - * <h3>jdb</h3>
>> - *
>> - * <em>{@index jdb jdb tool}</em> is a simple command-line debugger provided
>> - * in this module.
>>   *
>>   * <dl style="font-family:'DejaVu Sans', Arial, Helvetica, sans serif">
>>   * <dt class="simpleTagLabel">Tool Guides:
>> @@ -119,6 +116,8 @@
>>   *
>>   * @moduleGraph
>>   * @since 9
>> + * @see <a href="${docRoot}/../specs/jpda/jpda.html">
>> + * Java Platform Debugger Architecture (JPDA)</a>
>>   */
>>  module jdk.jdi {
>>      requires jdk.attach;
>> 
> 


From serguei.spitsyn at oracle.com  Mon Jun 19 20:31:06 2017
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Mon, 19 Jun 2017 13:31:06 -0700
Subject: Review Request JDK-8181834: Broken link in jdk.jdi module
 documentation
In-Reply-To: <7AF44A54-29A9-4774-93AA-0ADCC575282A@oracle.com>
References: <2823421C-9D7D-400A-8DB9-5F89B7BC0365@oracle.com>
 <6b8f9243-72d9-53c7-409f-f670598aab59@oracle.com>
 <7AF44A54-29A9-4774-93AA-0ADCC575282A@oracle.com>
Message-ID: <ca6c9dc9-7b28-4581-c980-43a42df9f772@oracle.com>




On 6/19/17 13:23, Mandy Chung wrote:
>> On Jun 19, 2017, at 1:17 PM, serguei.spitsyn at oracle.com wrote:
>>
>> Hi Mandy,
>>
>> I've built the docs with your patch and the link is still not resolved.
>> It points to the docs/api/specs/jpda/jpda.html
>>
> It should point to docs/specs/jpda/jpda.html. I verified in my build.

Good. I'm Ok with the fix then.

> Are you sure your build is correct?

No, I'm not sure.


>> But it is resolved if the patch is corrected with:
>>   + * <a href="${docRoot}/../../specs/jpda/jpda.html?>
>> Is it 9 or 10?
>>
> I want to fix this simple docs bug in JDK 9.

Ok, thanks.

-Serguei

>
> Mandy
>
>> Thanks,
>> Serguei
>>
>>
>> On 6/19/17 12:47, Mandy Chung wrote:
>>> Serguei,
>>>
>>> Can you review this patch fixing the link to JPDA?
>>>
>>> Mandy
>>>
>>>
>>>
>>> diff --git a/src/jdk.jdi/share/classes/module-info.java b/src/jdk.jdi/share/classes/module-info.java
>>> --- a/src/jdk.jdi/share/classes/module-info.java
>>> +++ b/src/jdk.jdi/share/classes/module-info.java
>>> @@ -39,12 +39,13 @@
>>>    * creation, etc. The ability to inspect a suspended thread's state, local
>>>    * variables, stack backtrace, etc.
>>>    * <p>
>>> - * JDI is the highest-layer of the Java Platform Debugger Architecture (JPDA).
>>> - * For more information on the Java Platform Debugger Architecture, see the <a
>>> - * href="{@docRoot}/../../../../technotes/guides/jpda/index.html"> Java
>>> - * Platform Debugger Architecture documentation</a> for this release and the <a
>>> - * href="http://java.sun.com/products/jpda">Java Platform Debugger Architecture
>>> - * website</a>.
>>> + * JDI is the highest-layer of the
>>> + * <a href="${docRoot}/../specs/jpda/jpda.html">
>>> + * Java Platform Debugger Architecture (JPDA)</a>.
>>> + * <p>
>>> + * This module includes a simple command-line debugger,
>>> + * <em>{@index jdb jdb tool}</em>.
>>> + *
>>>    * <h3>Global Exceptions</h3>
>>>    * <p>
>>>    * This section documents exceptions which apply to the entire API and are thus
>>> @@ -102,10 +103,6 @@
>>>    *   unloaded.
>>>    * </blockquote>
>>>    *
>>> - * <h3>jdb</h3>
>>> - *
>>> - * <em>{@index jdb jdb tool}</em> is a simple command-line debugger provided
>>> - * in this module.
>>>    *
>>>    * <dl style="font-family:'DejaVu Sans', Arial, Helvetica, sans serif">
>>>    * <dt class="simpleTagLabel">Tool Guides:
>>> @@ -119,6 +116,8 @@
>>>    *
>>>    * @moduleGraph
>>>    * @since 9
>>> + * @see <a href="${docRoot}/../specs/jpda/jpda.html">
>>> + * Java Platform Debugger Architecture (JPDA)</a>
>>>    */
>>>   module jdk.jdi {
>>>       requires jdk.attach;
>>>


From mandy.chung at oracle.com  Mon Jun 19 20:34:32 2017
From: mandy.chung at oracle.com (Mandy Chung)
Date: Mon, 19 Jun 2017 13:34:32 -0700
Subject: Review Request JDK-8181834: Broken link in jdk.jdi module
 documentation
In-Reply-To: <ca6c9dc9-7b28-4581-c980-43a42df9f772@oracle.com>
References: <2823421C-9D7D-400A-8DB9-5F89B7BC0365@oracle.com>
 <6b8f9243-72d9-53c7-409f-f670598aab59@oracle.com>
 <7AF44A54-29A9-4774-93AA-0ADCC575282A@oracle.com>
 <ca6c9dc9-7b28-4581-c980-43a42df9f772@oracle.com>
Message-ID: <E5D52A15-B873-4D12-BB3E-ADE670DE2D96@oracle.com>

+ * <a href="{@docRoot}/../specs/jpda/jpda.html?>

I sent a wrong patch.  The above is the corrected link.

Mandy

> On Jun 19, 2017, at 1:31 PM, serguei.spitsyn at oracle.com wrote:
> 
> 
> 
> 
> On 6/19/17 13:23, Mandy Chung wrote:
>>> On Jun 19, 2017, at 1:17 PM, serguei.spitsyn at oracle.com wrote:
>>> 
>>> Hi Mandy,
>>> 
>>> I've built the docs with your patch and the link is still not resolved.
>>> It points to the docs/api/specs/jpda/jpda.html
>>> 
>> It should point to docs/specs/jpda/jpda.html. I verified in my build.
> 
> Good. I'm Ok with the fix then.
> 
>> Are you sure your build is correct?
> 
> No, I'm not sure.
> 
> 
>>> But it is resolved if the patch is corrected with:
>>>  + * <a href="${docRoot}/../../specs/jpda/jpda.html?>
>>> Is it 9 or 10?
>>> 
>> I want to fix this simple docs bug in JDK 9.
> 
> Ok, thanks.
> 
> -Serguei
> 
>> 
>> Mandy
>> 
>>> Thanks,
>>> Serguei
>>> 
>>> 
>>> On 6/19/17 12:47, Mandy Chung wrote:
>>>> Serguei,
>>>> 
>>>> Can you review this patch fixing the link to JPDA?
>>>> 
>>>> Mandy
>>>> 
>>>> 
>>>> 
>>>> diff --git a/src/jdk.jdi/share/classes/module-info.java b/src/jdk.jdi/share/classes/module-info.java
>>>> --- a/src/jdk.jdi/share/classes/module-info.java
>>>> +++ b/src/jdk.jdi/share/classes/module-info.java
>>>> @@ -39,12 +39,13 @@
>>>>   * creation, etc. The ability to inspect a suspended thread's state, local
>>>>   * variables, stack backtrace, etc.
>>>>   * <p>
>>>> - * JDI is the highest-layer of the Java Platform Debugger Architecture (JPDA).
>>>> - * For more information on the Java Platform Debugger Architecture, see the <a
>>>> - * href="{@docRoot}/../../../../technotes/guides/jpda/index.html"> Java
>>>> - * Platform Debugger Architecture documentation</a> for this release and the <a
>>>> - * href="http://java.sun.com/products/jpda">Java Platform Debugger Architecture
>>>> - * website</a>.
>>>> + * JDI is the highest-layer of the
>>>> + * <a href="${docRoot}/../specs/jpda/jpda.html">
>>>> + * Java Platform Debugger Architecture (JPDA)</a>.
>>>> + * <p>
>>>> + * This module includes a simple command-line debugger,
>>>> + * <em>{@index jdb jdb tool}</em>.
>>>> + *
>>>>   * <h3>Global Exceptions</h3>
>>>>   * <p>
>>>>   * This section documents exceptions which apply to the entire API and are thus
>>>> @@ -102,10 +103,6 @@
>>>>   *   unloaded.
>>>>   * </blockquote>
>>>>   *
>>>> - * <h3>jdb</h3>
>>>> - *
>>>> - * <em>{@index jdb jdb tool}</em> is a simple command-line debugger provided
>>>> - * in this module.
>>>>   *
>>>>   * <dl style="font-family:'DejaVu Sans', Arial, Helvetica, sans serif">
>>>>   * <dt class="simpleTagLabel">Tool Guides:
>>>> @@ -119,6 +116,8 @@
>>>>   *
>>>>   * @moduleGraph
>>>>   * @since 9
>>>> + * @see <a href="${docRoot}/../specs/jpda/jpda.html">
>>>> + * Java Platform Debugger Architecture (JPDA)</a>
>>>>   */
>>>>  module jdk.jdi {
>>>>      requires jdk.attach;
>>>> 
> 


From serguei.spitsyn at oracle.com  Mon Jun 19 20:38:46 2017
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Mon, 19 Jun 2017 13:38:46 -0700
Subject: Review Request JDK-8181834: Broken link in jdk.jdi module
 documentation
In-Reply-To: <E5D52A15-B873-4D12-BB3E-ADE670DE2D96@oracle.com>
References: <2823421C-9D7D-400A-8DB9-5F89B7BC0365@oracle.com>
 <6b8f9243-72d9-53c7-409f-f670598aab59@oracle.com>
 <7AF44A54-29A9-4774-93AA-0ADCC575282A@oracle.com>
 <ca6c9dc9-7b28-4581-c980-43a42df9f772@oracle.com>
 <E5D52A15-B873-4D12-BB3E-ADE670DE2D96@oracle.com>
Message-ID: <e5e00a42-684f-f109-4d3c-922d84fcace6@oracle.com>

Thumbs up.

Thanks,
Serguei

On 6/19/17 13:34, Mandy Chung wrote:
> + * <a href="{@docRoot}/../specs/jpda/jpda.html?>
>
> I sent a wrong patch.  The above is the corrected link.
>
> Mandy
>
>> On Jun 19, 2017, at 1:31 PM, serguei.spitsyn at oracle.com wrote:
>>
>>
>>
>>
>> On 6/19/17 13:23, Mandy Chung wrote:
>>>> On Jun 19, 2017, at 1:17 PM, serguei.spitsyn at oracle.com wrote:
>>>>
>>>> Hi Mandy,
>>>>
>>>> I've built the docs with your patch and the link is still not resolved.
>>>> It points to the docs/api/specs/jpda/jpda.html
>>>>
>>> It should point to docs/specs/jpda/jpda.html. I verified in my build.
>> Good. I'm Ok with the fix then.
>>
>>> Are you sure your build is correct?
>> No, I'm not sure.
>>
>>
>>>> But it is resolved if the patch is corrected with:
>>>>   + * <a href="${docRoot}/../../specs/jpda/jpda.html?>
>>>> Is it 9 or 10?
>>>>
>>> I want to fix this simple docs bug in JDK 9.
>> Ok, thanks.
>>
>> -Serguei
>>
>>> Mandy
>>>
>>>> Thanks,
>>>> Serguei
>>>>
>>>>
>>>> On 6/19/17 12:47, Mandy Chung wrote:
>>>>> Serguei,
>>>>>
>>>>> Can you review this patch fixing the link to JPDA?
>>>>>
>>>>> Mandy
>>>>>
>>>>>
>>>>>
>>>>> diff --git a/src/jdk.jdi/share/classes/module-info.java b/src/jdk.jdi/share/classes/module-info.java
>>>>> --- a/src/jdk.jdi/share/classes/module-info.java
>>>>> +++ b/src/jdk.jdi/share/classes/module-info.java
>>>>> @@ -39,12 +39,13 @@
>>>>>    * creation, etc. The ability to inspect a suspended thread's state, local
>>>>>    * variables, stack backtrace, etc.
>>>>>    * <p>
>>>>> - * JDI is the highest-layer of the Java Platform Debugger Architecture (JPDA).
>>>>> - * For more information on the Java Platform Debugger Architecture, see the <a
>>>>> - * href="{@docRoot}/../../../../technotes/guides/jpda/index.html"> Java
>>>>> - * Platform Debugger Architecture documentation</a> for this release and the <a
>>>>> - * href="http://java.sun.com/products/jpda">Java Platform Debugger Architecture
>>>>> - * website</a>.
>>>>> + * JDI is the highest-layer of the
>>>>> + * <a href="${docRoot}/../specs/jpda/jpda.html">
>>>>> + * Java Platform Debugger Architecture (JPDA)</a>.
>>>>> + * <p>
>>>>> + * This module includes a simple command-line debugger,
>>>>> + * <em>{@index jdb jdb tool}</em>.
>>>>> + *
>>>>>    * <h3>Global Exceptions</h3>
>>>>>    * <p>
>>>>>    * This section documents exceptions which apply to the entire API and are thus
>>>>> @@ -102,10 +103,6 @@
>>>>>    *   unloaded.
>>>>>    * </blockquote>
>>>>>    *
>>>>> - * <h3>jdb</h3>
>>>>> - *
>>>>> - * <em>{@index jdb jdb tool}</em> is a simple command-line debugger provided
>>>>> - * in this module.
>>>>>    *
>>>>>    * <dl style="font-family:'DejaVu Sans', Arial, Helvetica, sans serif">
>>>>>    * <dt class="simpleTagLabel">Tool Guides:
>>>>> @@ -119,6 +116,8 @@
>>>>>    *
>>>>>    * @moduleGraph
>>>>>    * @since 9
>>>>> + * @see <a href="${docRoot}/../specs/jpda/jpda.html">
>>>>> + * Java Platform Debugger Architecture (JPDA)</a>
>>>>>    */
>>>>>   module jdk.jdi {
>>>>>       requires jdk.attach;
>>>>>


From harsha.wardhana.b at oracle.com  Tue Jun 20 06:10:13 2017
From: harsha.wardhana.b at oracle.com (Harsha Wardhana B)
Date: Tue, 20 Jun 2017 11:40:13 +0530
Subject: RFR : 8182485 - JMX connections should have configurable
 ObjectInputFilter
Message-ID: <b1ddf0ac-34cf-9137-b8b5-800643289af1@oracle.com>

Hi,

Please review the below RFE,

JDK-8182485 : JMX connections should have configurable ObjectInputFilter

having webrev at

http://cr.openjdk.java.net/~hb/8182485/webrev.00/

The enhancement adds ObjectInputFilter to JMX connections to configure 
filters during deserialization.

-Harsha





From harsha.wardhana.b at oracle.com  Tue Jun 20 08:11:54 2017
From: harsha.wardhana.b at oracle.com (Harsha Wardhana B)
Date: Tue, 20 Jun 2017 13:41:54 +0530
Subject: RFR: JDK-8173180 VirtualMachine.startLocalManagementAgent()
 returns URI with unreliable IP address
In-Reply-To: <a8aa2492-cd21-580d-5e4c-b530628ff94c@oracle.com>
References: <247efda9-804c-7ed6-e42c-6588eec78716@oracle.com>
 <a8aa2492-cd21-580d-5e4c-b530628ff94c@oracle.com>
Message-ID: <2b08267b-f4aa-9847-71eb-6b160ac9be2c@oracle.com>

Hi Daniel,

The fix is applicable only to local JMX agent and the most common use 
case would be client and server running from same JVM.
It is highly unlikely that local JMX agent will be started to cater for 
out-of-jvm clients.

I don't see how introducing this fix can cause new interoperability 
problems. Can you please elaborate?

-Harsha

On Monday 19 June 2017 10:23 PM, Daniel Fuchs wrote:
> Hi,
>
> If I'm not mistaken then this will make it impossible
> for earlier release to interoperate with newer releases
> as the LocalRMIClientSocketFactory class will not be
> present the client tries to deserialize the stub.
>
> best regards,
>
> -- daniel
>
> On 19/06/2017 11:52, Ujwal Vangapally wrote:
>> Hi,
>>
>> Kindly review the fix for bug below
>>
>> https://bugs.openjdk.java.net/browse/JDK-8173180
>>
>> webrev: 
>> http://cr.openjdk.java.net/~uvangapally/webrev/2017/8173180/webrev.00/
>>
>> Thanks,
>>
>> Ujwal
>>
>


From daniel.fuchs at oracle.com  Tue Jun 20 09:10:27 2017
From: daniel.fuchs at oracle.com (Daniel Fuchs)
Date: Tue, 20 Jun 2017 10:10:27 +0100
Subject: RFR: JDK-8173180 VirtualMachine.startLocalManagementAgent()
 returns URI with unreliable IP address
In-Reply-To: <2b08267b-f4aa-9847-71eb-6b160ac9be2c@oracle.com>
References: <247efda9-804c-7ed6-e42c-6588eec78716@oracle.com>
 <a8aa2492-cd21-580d-5e4c-b530628ff94c@oracle.com>
 <2b08267b-f4aa-9847-71eb-6b160ac9be2c@oracle.com>
Message-ID: <b0b78e4b-75a8-2187-f019-39456128751d@oracle.com>

Hi Harsha,

Maybe I'm missing something.
How is the local agent started?

If it's started when you connect jconsole to a process
by specifying the process ID - then I suspect this will
prevent e.g. jconsole or jvisualvm running on JDK 8 to
connect to your process.

Can you verify that it's not the case?

best regards,

-- daniel

On 20/06/2017 09:11, Harsha Wardhana B wrote:
> Hi Daniel,
> 
> The fix is applicable only to local JMX agent and the most common use 
> case would be client and server running from same JVM.
> It is highly unlikely that local JMX agent will be started to cater for 
> out-of-jvm clients.
> 
> I don't see how introducing this fix can cause new interoperability 
> problems. Can you please elaborate?
> 
> -Harsha
> 
> On Monday 19 June 2017 10:23 PM, Daniel Fuchs wrote:
>> Hi,
>>
>> If I'm not mistaken then this will make it impossible
>> for earlier release to interoperate with newer releases
>> as the LocalRMIClientSocketFactory class will not be
>> present the client tries to deserialize the stub.
>>
>> best regards,
>>
>> -- daniel
>>
>> On 19/06/2017 11:52, Ujwal Vangapally wrote:
>>> Hi,
>>>
>>> Kindly review the fix for bug below
>>>
>>> https://bugs.openjdk.java.net/browse/JDK-8173180
>>>
>>> webrev: 
>>> http://cr.openjdk.java.net/~uvangapally/webrev/2017/8173180/webrev.00/
>>>
>>> Thanks,
>>>
>>> Ujwal
>>>
>>
> 


From ujwal.vangapally at oracle.com  Tue Jun 20 11:42:51 2017
From: ujwal.vangapally at oracle.com (Ujwal Vangapally)
Date: Tue, 20 Jun 2017 17:12:51 +0530
Subject: RFR: JDK-8173180 VirtualMachine.startLocalManagementAgent()
 returns URI with unreliable IP address
In-Reply-To: <b0b78e4b-75a8-2187-f019-39456128751d@oracle.com>
References: <247efda9-804c-7ed6-e42c-6588eec78716@oracle.com>
 <a8aa2492-cd21-580d-5e4c-b530628ff94c@oracle.com>
 <2b08267b-f4aa-9847-71eb-6b160ac9be2c@oracle.com>
 <b0b78e4b-75a8-2187-f019-39456128751d@oracle.com>
Message-ID: <9d83842b-47f8-e2ce-34f2-68675858919b@oracle.com>

Thanks for the Review Daniel, Harsha.

Yes with this fix JConsole running on JDK 8 won't be able to connect to 
a process running on higher version of Java containing this change.

I think even Harsha is agreeing this, his point is that the use case 
where a client running on JDK 8 trying to connect to a process running 
on latest releases is rare. As mostly for Local Management both client 
and server will be running on same machine using same JDK for starting 
both client and server .

please correct me if this is not the case.

can we have this fix with interoperability issue between versions ?

Thanks,

Ujwal.



On 6/20/2017 2:40 PM, Daniel Fuchs wrote:
> Hi Harsha,
>
> Maybe I'm missing something.
> How is the local agent started?
>
> If it's started when you connect jconsole to a process
> by specifying the process ID - then I suspect this will
> prevent e.g. jconsole or jvisualvm running on JDK 8 to
> connect to your process.
>
> Can you verify that it's not the case?
>
> best regards,
>
> -- daniel
>
> On 20/06/2017 09:11, Harsha Wardhana B wrote:
>> Hi Daniel,
>>
>> The fix is applicable only to local JMX agent and the most common use 
>> case would be client and server running from same JVM.
>> It is highly unlikely that local JMX agent will be started to cater 
>> for out-of-jvm clients.
>>
>> I don't see how introducing this fix can cause new interoperability 
>> problems. Can you please elaborate?
>>
>> -Harsha
>>
>> On Monday 19 June 2017 10:23 PM, Daniel Fuchs wrote:
>>> Hi,
>>>
>>> If I'm not mistaken then this will make it impossible
>>> for earlier release to interoperate with newer releases
>>> as the LocalRMIClientSocketFactory class will not be
>>> present the client tries to deserialize the stub.
>>>
>>> best regards,
>>>
>>> -- daniel
>>>
>>> On 19/06/2017 11:52, Ujwal Vangapally wrote:
>>>> Hi,
>>>>
>>>> Kindly review the fix for bug below
>>>>
>>>> https://bugs.openjdk.java.net/browse/JDK-8173180
>>>>
>>>> webrev: 
>>>> http://cr.openjdk.java.net/~uvangapally/webrev/2017/8173180/webrev.00/
>>>>
>>>> Thanks,
>>>>
>>>> Ujwal
>>>>
>>>
>>
>


From jini.george at oracle.com  Tue Jun 20 12:16:45 2017
From: jini.george at oracle.com (Jini George)
Date: Tue, 20 Jun 2017 17:46:45 +0530
Subject: [10] RFR 8181647: jhsdb jstack could not output thread name
In-Reply-To: <5940E316.7060104@oracle.com>
References: <5937CB5F.4060209@oracle.com>
 <a9a45cd1-8717-ed6b-8169-9543e904c20f@oracle.com>
 <5939133A.9040709@oracle.com> <5940E316.7060104@oracle.com>
Message-ID: <e8c222d4-90d0-2097-7dc4-db3f7ff776c0@oracle.com>

Your changes look good overall, Chihiro. A few points on the test case, 
though.

* The test case checks for the existence of the compiler threads and the 
sweeper thread which would not exist when the test case is run with 
options like -Xint. You might want to remove those.
* You might have to skip the test on OSX due to jstack issues on OSX.
* Please correct the alignment on lines 74 and 81 for the test case.

Thank you,
Jini. (Not a (R)eviewer)


On 6/14/2017 12:47 PM, chihiro ito wrote:
> Hi all,
>
> I added a test case and modified previous patch including fix the 
> copyright year to 2017.
> I changed to output Java thread name next the separator lines in 
> "jhsdb jstack --mixed" case as following.
>
> ----------------- 32117 -----------------
> "main"
> 0x00007f6c8feafa82    __pthread_cond_timedwait + 0x132
>
> Could you possibly review for this following small change? If review 
> is ok, please commit this as cito.
>
>
> Source:
> diff --git 
> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java 
> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
> --- 
> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
> +++ 
> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
> @@ -1,5 +1,5 @@
>  /*
> - * Copyright (c) 2003, 2013, Oracle and/or its affiliates. All rights 
> reserved.
> + * Copyright (c) 2003, 2017, Oracle and/or its affiliates. All rights 
> reserved.
>   * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
>   *
>   * This code is free software; you can redistribute it and/or modify it
> @@ -88,6 +88,12 @@
>                 out.print("----------------- ");
>                 out.print(th);
>                 out.println(" -----------------");
> +               JavaThread jthread = (JavaThread) proxyToThread.get(th);
> +               if (jthread != null) {
> +                   out.print("\"");
> +                   out.print(jthread.getThreadName());
> +                   out.println("\"");
> +               }
>                 while (f != null) {
>                    ClosestSymbol sym = f.closestSymbolToPC();
>                    Address pc = f.pc();
> diff --git 
> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java 
> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java 
>
> --- 
> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
> +++ 
> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
> @@ -1,5 +1,5 @@
>  /*
> - * Copyright (c) 2002, 2013, Oracle and/or its affiliates. All rights 
> reserved.
> + * Copyright (c) 2002, 2017, Oracle and/or its affiliates. All rights 
> reserved.
>   * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
>   *
>   * This code is free software; you can redistribute it and/or modify it
> @@ -75,7 +75,9 @@
>              for (JavaThread cur = threads.first(); cur != null; cur = 
> cur.next(), i++) {
>                  if (cur.isJavaThread()) {
>                      Address sp = cur.getLastJavaSP();
> -                    tty.print("Thread ");
> +                    tty.print("\"");
> +                    tty.print(cur.getThreadName());
> +                    tty.print("\" nid=");
>                      cur.printThreadIDOn(tty);
>                      tty.print(": (state = " + cur.getThreadState());
>                      if (verbose) {
> diff --git a/test/serviceability/sa/JhsdbThreadNameTest.java 
> b/test/serviceability/sa/JhsdbThreadNameTest.java
> new file mode 100644
> --- /dev/null
> +++ b/test/serviceability/sa/JhsdbThreadNameTest.java
> @@ -0,0 +1,107 @@
> +/*
> + * Copyright (c) 2017, Oracle and/or its affiliates. All rights 
> reserved.
> + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> + *
> + * This code is free software; you can redistribute it and/or modify it
> + * under the terms of the GNU General Public License version 2 only, as
> + * published by the Free Software Foundation.
> + *
> + * This code is distributed in the hope that it will be useful, but 
> WITHOUT
> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
> + * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
> + * version 2 for more details (a copy is included in the LICENSE file 
> that
> + * accompanied this code).
> + *
> + * You should have received a copy of the GNU General Public License 
> version
> + * 2 along with this work; if not, write to the Free Software 
> Foundation,
> + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
> + *
> + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 
> 94065 USA
> + * or visit www.oracle.com if you need additional information or have 
> any
> + * questions.
> + */
> +
> +import java.util.ArrayList;
> +import java.util.Arrays;
> +import java.util.List;
> +import java.util.function.Consumer;
> +
> +import jdk.test.lib.apps.LingeredApp;
> +import jdk.test.lib.JDKToolLauncher;
> +import jdk.test.lib.Platform;
> +import jdk.test.lib.process.OutputAnalyzer;
> +import jdk.test.lib.Utils;
> +
> +/*
> + * @test
> + * @library /test/lib
> + * @run main/othervm JhsdbThreadNameTest
> + */
> +public class JhsdbThreadNameTest {
> +
> +    private static String notMixedModeThreadNames[] = 
> {"Common-Cleaner", "Signal Dispatcher", "Finalizer", "Reference 
> Handler", "main"};
> +    private static String mixedModeThreadNames[] = {"C2 
> CompilerThread0", "C1 CompilerThread1", "Sweeper thread", "Service 
> Thread"};
> +
> +    private static void startHsdbJstack(boolean mixed) throws 
> Exception {
> +
> +        LingeredApp app = null;
> +
> +        try {
> +            List<String> vmArgs = new ArrayList<String>();
> +            vmArgs.add("-Xmx10m");
> +            vmArgs.addAll(Utils.getVmOptions());
> +
> +            app = LingeredApp.startApp(vmArgs);
> +            System.out.println("Started LingeredApp with pid " + 
> app.getPid());
> +
> +            JDKToolLauncher jhsdbLauncher = 
> JDKToolLauncher.createUsingTestJDK("jhsdb");
> +
> +            jhsdbLauncher.addToolArg("jstack");
> +            jhsdbLauncher.addToolArg("--pid");
> + jhsdbLauncher.addToolArg(Long.toString(app.getPid()));
> +
> +            if (mixed) {
> +                jhsdbLauncher.addToolArg("--mixed");
> +            }
> +            ProcessBuilder pb = new ProcessBuilder();
> +            pb.command(jhsdbLauncher.getCommand());
> +            Process jhsdb = pb.start();
> +
> +            jhsdb.waitFor();
> +
> +            OutputAnalyzer out = new OutputAnalyzer(jhsdb);
> +
> + 
> Arrays.stream(notMixedModeThreadNames).map(JhsdbThreadNameTest::addQuotationMarks).forEach(out::shouldContain);
> +            Consumer<String> testMethod = null;
> +            if (mixed) {
> +                testMethod = out::shouldContain;
> +            } else {
> +                testMethod = out::shouldNotContain;
> +            }
> + 
> Arrays.stream(mixedModeThreadNames).map(JhsdbThreadNameTest::addQuotationMarks).forEach(testMethod);
> +
> +        } catch (InterruptedException ie) {
> +            throw new Error("Problem awaiting the child process: " + 
> ie, ie);
> +        } catch (Exception attachE) {
> +            throw new Error("Couldn't start jhsdb, attach to 
> LingeredApp or match ThreadName: " + attachE);
> +
> +        } finally {
> +            LingeredApp.stopApp(app);
> +        }
> +    }
> +
> +    private static String addQuotationMarks(String str) {
> +        return "\"" + str + "\"";
> +    }
> +
> +    public static void main(String[] args) throws Exception {
> +
> +        if (!Platform.shouldSAAttach()) {
> +            System.out.println("SA attach not expected to work - test 
> skipped.");
> +            return;
> +        }
> +
> +        startHsdbJstack(true);
> +        startHsdbJstack(false);
> +    }
> +}
>
>
> Regards,
> Chihiro
>
>
> On 2017/06/08 18:04, chihiro ito wrote:
>> Hi Jini,
>>
>> Thank you for your advices. I try to add the test case and modify the 
>> copyright year to 2017.
>> Basically, I agree with your idea, but I think that the separator 
>> line should finally be the same as the output of the jstack command. 
>> I worry which is better way.
>>
>> Thanks,
>> Chihiro
>>
>> On 2017/06/08 16:42, Jini George wrote:
>>> Hi Chihiro,
>>>
>>> Thank you for making this useful change. Your changes look good.
>>>
>>> It would be great though if you could add a test case for this. 
>>> Could you also modify the copyright year to 2017 ? One additional 
>>> suggestion: The addition of the thread name makes the separator 
>>> lines unaligned in the pstack/jstack --mixed cases. Like:
>>>
>>> ----------------- "Service Thread" nid=16051 -----------------
>>> and
>>> ----------------- nid=16052 -----------------
>>>
>>> So I am wondering if it would make sense to have the name printed 
>>> out on a separate line to keep the separator lines aligned. But this 
>>> is a nit, and I would leave it to you to decide on this.
>>>
>>> Thanks,
>>> Jini (Not a (R)eviewer)
>>>
>>> On 6/7/2017 3:16 PM, chihiro ito wrote:
>>>> Hi all,
>>>>
>>>> I changed to output Java thread name in jhsdb jstack as following.
>>>>
>>>> jhsdb jstack --pid 25879
>>>> "main" nid=25884: (state = BLOCKED)
>>>>
>>>> jhsdb jstack --mixed --pid 25879
>>>> ----------------- "main" nid=25884 -----------------
>>>>
>>>> Could you possibly review for this following small change? If 
>>>> review is ok, please commit this as cito.
>>>>
>>>> Source:
>>>> diff --git 
>>>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java 
>>>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java 
>>>>
>>>> --- 
>>>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java 
>>>>
>>>> +++ 
>>>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java 
>>>>
>>>> @@ -86,6 +86,13 @@
>>>>              try {
>>>>                 CFrame f = cdbg.topFrameForThread(th);
>>>>                 out.print("----------------- ");
>>>> +               JavaThread jthread = (JavaThread) 
>>>> proxyToThread.get(th);
>>>> +               if (jthread != null) {
>>>> +                   out.print("\"");
>>>> +                   out.print(jthread.getThreadName());
>>>> +                   out.print("\" ");
>>>> +               }
>>>> +               out.print("nid=");
>>>>                 out.print(th);
>>>>                 out.println(" -----------------");
>>>>                 while (f != null) {
>>>> diff --git 
>>>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java 
>>>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java 
>>>>
>>>> --- 
>>>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
>>>> +++ 
>>>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
>>>> @@ -75,7 +75,9 @@
>>>>              for (JavaThread cur = threads.first(); cur != null; 
>>>> cur = cur.next(), i++) {
>>>>                  if (cur.isJavaThread()) {
>>>>                      Address sp = cur.getLastJavaSP();
>>>> -                    tty.print("Thread ");
>>>> +                    tty.print("\"");
>>>> +                    tty.print(cur.getThreadName());
>>>> +                    tty.print("\" nid=");
>>>>                      cur.printThreadIDOn(tty);
>>>>                      tty.print(": (state = " + cur.getThreadState());
>>>>                      if (verbose) {
>>>>
>>>> Regards,
>>>> Chihiro
>>>>
>>>
>>
>


From roger.riggs at oracle.com  Tue Jun 20 13:06:09 2017
From: roger.riggs at oracle.com (Roger Riggs)
Date: Tue, 20 Jun 2017 09:06:09 -0400
Subject: RFR: JDK-8173180 VirtualMachine.startLocalManagementAgent()
 returns URI with unreliable IP address
In-Reply-To: <9d83842b-47f8-e2ce-34f2-68675858919b@oracle.com>
References: <247efda9-804c-7ed6-e42c-6588eec78716@oracle.com>
 <a8aa2492-cd21-580d-5e4c-b530628ff94c@oracle.com>
 <2b08267b-f4aa-9847-71eb-6b160ac9be2c@oracle.com>
 <b0b78e4b-75a8-2187-f019-39456128751d@oracle.com>
 <9d83842b-47f8-e2ce-34f2-68675858919b@oracle.com>
Message-ID: <f9bc0f98-bb9f-7f6a-25af-2503efcc39ad@oracle.com>

Hi,

I would expect to see a mix of versions in many operational cases. For 
example, in a large
deployment there will be a mix of versions active and the folks 
monitoring it should not
have to change their management tools unnecessarily.

The usual rule for interoperability between versions is that it should 
work between the
previous and next versions.  (+/- 1).

If there is any issue with compatibility between versions, you'll need 
to file a CCC
to make sure it gets adequate review.

Roger


On 6/20/17 7:42 AM, Ujwal Vangapally wrote:
> Thanks for the Review Daniel, Harsha.
>
> Yes with this fix JConsole running on JDK 8 won't be able to connect 
> to a process running on higher version of Java containing this change.
>
> I think even Harsha is agreeing this, his point is that the use case 
> where a client running on JDK 8 trying to connect to a process running 
> on latest releases is rare. As mostly for Local Management both client 
> and server will be running on same machine using same JDK for starting 
> both client and server .
>
> please correct me if this is not the case.
>
> can we have this fix with interoperability issue between versions ?
>
> Thanks,
>
> Ujwal.
>
>
>
> On 6/20/2017 2:40 PM, Daniel Fuchs wrote:
>> Hi Harsha,
>>
>> Maybe I'm missing something.
>> How is the local agent started?
>>
>> If it's started when you connect jconsole to a process
>> by specifying the process ID - then I suspect this will
>> prevent e.g. jconsole or jvisualvm running on JDK 8 to
>> connect to your process.
>>
>> Can you verify that it's not the case?
>>
>> best regards,
>>
>> -- daniel
>>
>> On 20/06/2017 09:11, Harsha Wardhana B wrote:
>>> Hi Daniel,
>>>
>>> The fix is applicable only to local JMX agent and the most common 
>>> use case would be client and server running from same JVM.
>>> It is highly unlikely that local JMX agent will be started to cater 
>>> for out-of-jvm clients.
>>>
>>> I don't see how introducing this fix can cause new interoperability 
>>> problems. Can you please elaborate?
>>>
>>> -Harsha
>>>
>>> On Monday 19 June 2017 10:23 PM, Daniel Fuchs wrote:
>>>> Hi,
>>>>
>>>> If I'm not mistaken then this will make it impossible
>>>> for earlier release to interoperate with newer releases
>>>> as the LocalRMIClientSocketFactory class will not be
>>>> present the client tries to deserialize the stub.
>>>>
>>>> best regards,
>>>>
>>>> -- daniel
>>>>
>>>> On 19/06/2017 11:52, Ujwal Vangapally wrote:
>>>>> Hi,
>>>>>
>>>>> Kindly review the fix for bug below
>>>>>
>>>>> https://bugs.openjdk.java.net/browse/JDK-8173180
>>>>>
>>>>> webrev: 
>>>>> http://cr.openjdk.java.net/~uvangapally/webrev/2017/8173180/webrev.00/ 
>>>>>
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Ujwal
>>>>>
>>>>
>>>
>>
>


From daniel.fuchs at oracle.com  Tue Jun 20 13:09:34 2017
From: daniel.fuchs at oracle.com (Daniel Fuchs)
Date: Tue, 20 Jun 2017 14:09:34 +0100
Subject: RFR: JDK-8173180 VirtualMachine.startLocalManagementAgent()
 returns URI with unreliable IP address
In-Reply-To: <9d83842b-47f8-e2ce-34f2-68675858919b@oracle.com>
References: <247efda9-804c-7ed6-e42c-6588eec78716@oracle.com>
 <a8aa2492-cd21-580d-5e4c-b530628ff94c@oracle.com>
 <2b08267b-f4aa-9847-71eb-6b160ac9be2c@oracle.com>
 <b0b78e4b-75a8-2187-f019-39456128751d@oracle.com>
 <9d83842b-47f8-e2ce-34f2-68675858919b@oracle.com>
Message-ID: <40fd294d-be6f-b54a-3bb8-87a2b971521e@oracle.com>

On 20/06/2017 12:42, Ujwal Vangapally wrote:
> Thanks for the Review Daniel, Harsha.
> 
> Yes with this fix JConsole running on JDK 8 won't be able to connect to 
> a process running on higher version of Java containing this change.
> 
> I think even Harsha is agreeing this, his point is that the use case 
> where a client running on JDK 8 trying to connect to a process running 
> on latest releases is rare. As mostly for Local Management both client 
> and server will be running on same machine using same JDK for starting 
> both client and server .
> 
> please correct me if this is not the case.

Well - I believe it could be quite common to use VisualVM running
on JDK 8 to connect to JDK 9 processes. JMX is all about
interoperability so I'd be very reluctant to break this 'by
default'.

> can we have this fix with interoperability issue between versions ?

Maybe you could make the fix conditional to a system property being
defined? Also are you sure what you are proposing is the correct fix
for this issue?

Note that - if I'm not mistaken - you can work around the original issue
by specifying -Djava.rmi.server.hostname=<address> on the command line.
http://docs.oracle.com/javase/8/docs/technotes/guides/rmi/javarmiproperties.html
This will instruct RMI to put the specified <address> inside its stubs.
The inconvenience is that this is global (for all RMI stubs, just not
only those used by JMX) - but would that be an issue for the use case
mentioned in the bug?

best regards,

-- daniel

> 
> Thanks,
> 
> Ujwal.
> 
> 
> 
> On 6/20/2017 2:40 PM, Daniel Fuchs wrote:
>> Hi Harsha,
>>
>> Maybe I'm missing something.
>> How is the local agent started?
>>
>> If it's started when you connect jconsole to a process
>> by specifying the process ID - then I suspect this will
>> prevent e.g. jconsole or jvisualvm running on JDK 8 to
>> connect to your process.
>>
>> Can you verify that it's not the case?
>>
>> best regards,
>>
>> -- daniel
>>
>> On 20/06/2017 09:11, Harsha Wardhana B wrote:
>>> Hi Daniel,
>>>
>>> The fix is applicable only to local JMX agent and the most common use 
>>> case would be client and server running from same JVM.
>>> It is highly unlikely that local JMX agent will be started to cater 
>>> for out-of-jvm clients.
>>>
>>> I don't see how introducing this fix can cause new interoperability 
>>> problems. Can you please elaborate?
>>>
>>> -Harsha
>>>
>>> On Monday 19 June 2017 10:23 PM, Daniel Fuchs wrote:
>>>> Hi,
>>>>
>>>> If I'm not mistaken then this will make it impossible
>>>> for earlier release to interoperate with newer releases
>>>> as the LocalRMIClientSocketFactory class will not be
>>>> present the client tries to deserialize the stub.
>>>>
>>>> best regards,
>>>>
>>>> -- daniel
>>>>
>>>> On 19/06/2017 11:52, Ujwal Vangapally wrote:
>>>>> Hi,
>>>>>
>>>>> Kindly review the fix for bug below
>>>>>
>>>>> https://bugs.openjdk.java.net/browse/JDK-8173180
>>>>>
>>>>> webrev: 
>>>>> http://cr.openjdk.java.net/~uvangapally/webrev/2017/8173180/webrev.00/
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Ujwal
>>>>>
>>>>
>>>
>>
> 


From chihiro.ito at oracle.com  Tue Jun 20 13:39:50 2017
From: chihiro.ito at oracle.com (chihiro ito)
Date: Tue, 20 Jun 2017 22:39:50 +0900
Subject: [10] RFR 8181647: jhsdb jstack could not output thread name
In-Reply-To: <e8c222d4-90d0-2097-7dc4-db3f7ff776c0@oracle.com>
References: <5937CB5F.4060209@oracle.com>
 <a9a45cd1-8717-ed6b-8169-9543e904c20f@oracle.com>
 <5939133A.9040709@oracle.com> <5940E316.7060104@oracle.com>
 <e8c222d4-90d0-2097-7dc4-db3f7ff776c0@oracle.com>
Message-ID: <594925A6.4070007@oracle.com>

Thank you for checking.
However, a few days ago, I have changed the output to be more similar to 
jstack.
Could you possibly check for latest one?

Best regards,
Chihiro

On 2017/06/20 21:16, Jini George wrote:
> Your changes look good overall, Chihiro. A few points on the test 
> case, though.
>
> * The test case checks for the existence of the compiler threads and 
> the sweeper thread which would not exist when the test case is run 
> with options like -Xint. You might want to remove those.
> * You might have to skip the test on OSX due to jstack issues on OSX.
> * Please correct the alignment on lines 74 and 81 for the test case.
>
> Thank you,
> Jini. (Not a (R)eviewer)
>
>
> On 6/14/2017 12:47 PM, chihiro ito wrote:
>> Hi all,
>>
>> I added a test case and modified previous patch including fix the 
>> copyright year to 2017.
>> I changed to output Java thread name next the separator lines in 
>> "jhsdb jstack --mixed" case as following.
>>
>> ----------------- 32117 -----------------
>> "main"
>> 0x00007f6c8feafa82    __pthread_cond_timedwait + 0x132
>>
>> Could you possibly review for this following small change? If review 
>> is ok, please commit this as cito.
>>
>>
>> Source:
>> diff --git 
>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java 
>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
>> --- 
>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
>> +++ 
>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
>> @@ -1,5 +1,5 @@
>>  /*
>> - * Copyright (c) 2003, 2013, Oracle and/or its affiliates. All 
>> rights reserved.
>> + * Copyright (c) 2003, 2017, Oracle and/or its affiliates. All 
>> rights reserved.
>>   * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
>>   *
>>   * This code is free software; you can redistribute it and/or modify it
>> @@ -88,6 +88,12 @@
>>                 out.print("----------------- ");
>>                 out.print(th);
>>                 out.println(" -----------------");
>> +               JavaThread jthread = (JavaThread) proxyToThread.get(th);
>> +               if (jthread != null) {
>> +                   out.print("\"");
>> +                   out.print(jthread.getThreadName());
>> +                   out.println("\"");
>> +               }
>>                 while (f != null) {
>>                    ClosestSymbol sym = f.closestSymbolToPC();
>>                    Address pc = f.pc();
>> diff --git 
>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java 
>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java 
>>
>> --- 
>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
>> +++ 
>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
>> @@ -1,5 +1,5 @@
>>  /*
>> - * Copyright (c) 2002, 2013, Oracle and/or its affiliates. All 
>> rights reserved.
>> + * Copyright (c) 2002, 2017, Oracle and/or its affiliates. All 
>> rights reserved.
>>   * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
>>   *
>>   * This code is free software; you can redistribute it and/or modify it
>> @@ -75,7 +75,9 @@
>>              for (JavaThread cur = threads.first(); cur != null; cur 
>> = cur.next(), i++) {
>>                  if (cur.isJavaThread()) {
>>                      Address sp = cur.getLastJavaSP();
>> -                    tty.print("Thread ");
>> +                    tty.print("\"");
>> +                    tty.print(cur.getThreadName());
>> +                    tty.print("\" nid=");
>>                      cur.printThreadIDOn(tty);
>>                      tty.print(": (state = " + cur.getThreadState());
>>                      if (verbose) {
>> diff --git a/test/serviceability/sa/JhsdbThreadNameTest.java 
>> b/test/serviceability/sa/JhsdbThreadNameTest.java
>> new file mode 100644
>> --- /dev/null
>> +++ b/test/serviceability/sa/JhsdbThreadNameTest.java
>> @@ -0,0 +1,107 @@
>> +/*
>> + * Copyright (c) 2017, Oracle and/or its affiliates. All rights 
>> reserved.
>> + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
>> + *
>> + * This code is free software; you can redistribute it and/or modify it
>> + * under the terms of the GNU General Public License version 2 only, as
>> + * published by the Free Software Foundation.
>> + *
>> + * This code is distributed in the hope that it will be useful, but 
>> WITHOUT
>> + * ANY WARRANTY; without even the implied warranty of 
>> MERCHANTABILITY or
>> + * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public 
>> License
>> + * version 2 for more details (a copy is included in the LICENSE 
>> file that
>> + * accompanied this code).
>> + *
>> + * You should have received a copy of the GNU General Public License 
>> version
>> + * 2 along with this work; if not, write to the Free Software 
>> Foundation,
>> + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
>> + *
>> + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 
>> 94065 USA
>> + * or visit www.oracle.com if you need additional information or 
>> have any
>> + * questions.
>> + */
>> +
>> +import java.util.ArrayList;
>> +import java.util.Arrays;
>> +import java.util.List;
>> +import java.util.function.Consumer;
>> +
>> +import jdk.test.lib.apps.LingeredApp;
>> +import jdk.test.lib.JDKToolLauncher;
>> +import jdk.test.lib.Platform;
>> +import jdk.test.lib.process.OutputAnalyzer;
>> +import jdk.test.lib.Utils;
>> +
>> +/*
>> + * @test
>> + * @library /test/lib
>> + * @run main/othervm JhsdbThreadNameTest
>> + */
>> +public class JhsdbThreadNameTest {
>> +
>> +    private static String notMixedModeThreadNames[] = 
>> {"Common-Cleaner", "Signal Dispatcher", "Finalizer", "Reference 
>> Handler", "main"};
>> +    private static String mixedModeThreadNames[] = {"C2 
>> CompilerThread0", "C1 CompilerThread1", "Sweeper thread", "Service 
>> Thread"};
>> +
>> +    private static void startHsdbJstack(boolean mixed) throws 
>> Exception {
>> +
>> +        LingeredApp app = null;
>> +
>> +        try {
>> +            List<String> vmArgs = new ArrayList<String>();
>> +            vmArgs.add("-Xmx10m");
>> +            vmArgs.addAll(Utils.getVmOptions());
>> +
>> +            app = LingeredApp.startApp(vmArgs);
>> +            System.out.println("Started LingeredApp with pid " + 
>> app.getPid());
>> +
>> +            JDKToolLauncher jhsdbLauncher = 
>> JDKToolLauncher.createUsingTestJDK("jhsdb");
>> +
>> +            jhsdbLauncher.addToolArg("jstack");
>> +            jhsdbLauncher.addToolArg("--pid");
>> + jhsdbLauncher.addToolArg(Long.toString(app.getPid()));
>> +
>> +            if (mixed) {
>> +                jhsdbLauncher.addToolArg("--mixed");
>> +            }
>> +            ProcessBuilder pb = new ProcessBuilder();
>> +            pb.command(jhsdbLauncher.getCommand());
>> +            Process jhsdb = pb.start();
>> +
>> +            jhsdb.waitFor();
>> +
>> +            OutputAnalyzer out = new OutputAnalyzer(jhsdb);
>> +
>> + 
>> Arrays.stream(notMixedModeThreadNames).map(JhsdbThreadNameTest::addQuotationMarks).forEach(out::shouldContain);
>> +            Consumer<String> testMethod = null;
>> +            if (mixed) {
>> +                testMethod = out::shouldContain;
>> +            } else {
>> +                testMethod = out::shouldNotContain;
>> +            }
>> + 
>> Arrays.stream(mixedModeThreadNames).map(JhsdbThreadNameTest::addQuotationMarks).forEach(testMethod);
>> +
>> +        } catch (InterruptedException ie) {
>> +            throw new Error("Problem awaiting the child process: " + 
>> ie, ie);
>> +        } catch (Exception attachE) {
>> +            throw new Error("Couldn't start jhsdb, attach to 
>> LingeredApp or match ThreadName: " + attachE);
>> +
>> +        } finally {
>> +            LingeredApp.stopApp(app);
>> +        }
>> +    }
>> +
>> +    private static String addQuotationMarks(String str) {
>> +        return "\"" + str + "\"";
>> +    }
>> +
>> +    public static void main(String[] args) throws Exception {
>> +
>> +        if (!Platform.shouldSAAttach()) {
>> +            System.out.println("SA attach not expected to work - 
>> test skipped.");
>> +            return;
>> +        }
>> +
>> +        startHsdbJstack(true);
>> +        startHsdbJstack(false);
>> +    }
>> +}
>>
>>
>> Regards,
>> Chihiro
>>
>>
>> On 2017/06/08 18:04, chihiro ito wrote:
>>> Hi Jini,
>>>
>>> Thank you for your advices. I try to add the test case and modify 
>>> the copyright year to 2017.
>>> Basically, I agree with your idea, but I think that the separator 
>>> line should finally be the same as the output of the jstack command. 
>>> I worry which is better way.
>>>
>>> Thanks,
>>> Chihiro
>>>
>>> On 2017/06/08 16:42, Jini George wrote:
>>>> Hi Chihiro,
>>>>
>>>> Thank you for making this useful change. Your changes look good.
>>>>
>>>> It would be great though if you could add a test case for this. 
>>>> Could you also modify the copyright year to 2017 ? One additional 
>>>> suggestion: The addition of the thread name makes the separator 
>>>> lines unaligned in the pstack/jstack --mixed cases. Like:
>>>>
>>>> ----------------- "Service Thread" nid=16051 -----------------
>>>> and
>>>> ----------------- nid=16052 -----------------
>>>>
>>>> So I am wondering if it would make sense to have the name printed 
>>>> out on a separate line to keep the separator lines aligned. But 
>>>> this is a nit, and I would leave it to you to decide on this.
>>>>
>>>> Thanks,
>>>> Jini (Not a (R)eviewer)
>>>>
>>>> On 6/7/2017 3:16 PM, chihiro ito wrote:
>>>>> Hi all,
>>>>>
>>>>> I changed to output Java thread name in jhsdb jstack as following.
>>>>>
>>>>> jhsdb jstack --pid 25879
>>>>> "main" nid=25884: (state = BLOCKED)
>>>>>
>>>>> jhsdb jstack --mixed --pid 25879
>>>>> ----------------- "main" nid=25884 -----------------
>>>>>
>>>>> Could you possibly review for this following small change? If 
>>>>> review is ok, please commit this as cito.
>>>>>
>>>>> Source:
>>>>> diff --git 
>>>>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java 
>>>>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java 
>>>>>
>>>>> --- 
>>>>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java 
>>>>>
>>>>> +++ 
>>>>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java 
>>>>>
>>>>> @@ -86,6 +86,13 @@
>>>>>              try {
>>>>>                 CFrame f = cdbg.topFrameForThread(th);
>>>>>                 out.print("----------------- ");
>>>>> +               JavaThread jthread = (JavaThread) 
>>>>> proxyToThread.get(th);
>>>>> +               if (jthread != null) {
>>>>> +                   out.print("\"");
>>>>> +                   out.print(jthread.getThreadName());
>>>>> +                   out.print("\" ");
>>>>> +               }
>>>>> +               out.print("nid=");
>>>>>                 out.print(th);
>>>>>                 out.println(" -----------------");
>>>>>                 while (f != null) {
>>>>> diff --git 
>>>>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java 
>>>>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java 
>>>>>
>>>>> --- 
>>>>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
>>>>> +++ 
>>>>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
>>>>> @@ -75,7 +75,9 @@
>>>>>              for (JavaThread cur = threads.first(); cur != null; 
>>>>> cur = cur.next(), i++) {
>>>>>                  if (cur.isJavaThread()) {
>>>>>                      Address sp = cur.getLastJavaSP();
>>>>> -                    tty.print("Thread ");
>>>>> +                    tty.print("\"");
>>>>> +                    tty.print(cur.getThreadName());
>>>>> +                    tty.print("\" nid=");
>>>>>                      cur.printThreadIDOn(tty);
>>>>>                      tty.print(": (state = " + cur.getThreadState());
>>>>>                      if (verbose) {
>>>>>
>>>>> Regards,
>>>>> Chihiro
>>>>>
>>>>
>>>
>>
>

-- 

Chihiro Ito | Principal Consultant | +81.90.6148.8815
Oracle <http://www.oracle.com> Consultant
ORACLE Japan | Akasaka Center Bldg. | Motoakasaka 1-3-13 | 1070051 
Minato-ku, Tokyo, JAPAN

Oracle is committed to developing practices and products that help 
protect the environment <http://www.oracle.com/commitment>

From harsha.wardhana.b at oracle.com  Tue Jun 20 13:45:59 2017
From: harsha.wardhana.b at oracle.com (Harsha Wardhana B)
Date: Tue, 20 Jun 2017 19:15:59 +0530
Subject: RFR: JDK-8173180 VirtualMachine.startLocalManagementAgent()
 returns URI with unreliable IP address
In-Reply-To: <40fd294d-be6f-b54a-3bb8-87a2b971521e@oracle.com>
References: <247efda9-804c-7ed6-e42c-6588eec78716@oracle.com>
 <a8aa2492-cd21-580d-5e4c-b530628ff94c@oracle.com>
 <2b08267b-f4aa-9847-71eb-6b160ac9be2c@oracle.com>
 <b0b78e4b-75a8-2187-f019-39456128751d@oracle.com>
 <9d83842b-47f8-e2ce-34f2-68675858919b@oracle.com>
 <40fd294d-be6f-b54a-3bb8-87a2b971521e@oracle.com>
Message-ID: <07010699-c9fd-697f-e506-4a98e7f0bbd1@oracle.com>

Hi Daniel,


On Tuesday 20 June 2017 06:39 PM, Daniel Fuchs wrote:
> On 20/06/2017 12:42, Ujwal Vangapally wrote:
>> Thanks for the Review Daniel, Harsha.
>>
>> Yes with this fix JConsole running on JDK 8 won't be able to connect 
>> to a process running on higher version of Java containing this change.
>>
>> I think even Harsha is agreeing this, his point is that the use case 
>> where a client running on JDK 8 trying to connect to a process 
>> running on latest releases is rare. As mostly for Local Management 
>> both client and server will be running on same machine using same JDK 
>> for starting both client and server .
>>
>> please correct me if this is not the case.
>
> Well - I believe it could be quite common to use VisualVM running
> on JDK 8 to connect to JDK 9 processes. JMX is all about
> interoperability so I'd be very reluctant to break this 'by
> default'.
Agreed.
>
>> can we have this fix with interoperability issue between versions ?
>
> Maybe you could make the fix conditional to a system property being
> defined? Also are you sure what you are proposing is the correct fix
> for this issue?
It's more of a workaround than a fix as we don't see any other way to 
fix this issue.
>
> Note that - if I'm not mistaken - you can work around the original issue
> by specifying -Djava.rmi.server.hostname=<address> on the command line.
> http://docs.oracle.com/javase/8/docs/technotes/guides/rmi/javarmiproperties.html 
>
> This will instruct RMI to put the specified <address> inside its stubs.
> The inconvenience is that this is global (for all RMI stubs, just not
> only those used by JMX) - but would that be an issue for the use case
> mentioned in the bug?
We had the same consideration that setting the RMI property will change 
the Interface address for all the stubs which may not be an acceptable 
fix in this case.
One possible fix (enhancement) would be to define a JMX system property 
that will control the Interface address that goes into RMI stubs. If 
that seems like too much work, we can close this issue as 'won't fix' as 
we don't have any acceptable fix.
>
> best regards,
>
> -- daniel

-Harsha
>
>>
>> Thanks,
>>
>> Ujwal.
>>
>>
>>
>> On 6/20/2017 2:40 PM, Daniel Fuchs wrote:
>>> Hi Harsha,
>>>
>>> Maybe I'm missing something.
>>> How is the local agent started?
>>>
>>> If it's started when you connect jconsole to a process
>>> by specifying the process ID - then I suspect this will
>>> prevent e.g. jconsole or jvisualvm running on JDK 8 to
>>> connect to your process.
>>>
>>> Can you verify that it's not the case?
>>>
>>> best regards,
>>>
>>> -- daniel
>>>
>>> On 20/06/2017 09:11, Harsha Wardhana B wrote:
>>>> Hi Daniel,
>>>>
>>>> The fix is applicable only to local JMX agent and the most common 
>>>> use case would be client and server running from same JVM.
>>>> It is highly unlikely that local JMX agent will be started to cater 
>>>> for out-of-jvm clients.
>>>>
>>>> I don't see how introducing this fix can cause new interoperability 
>>>> problems. Can you please elaborate?
>>>>
>>>> -Harsha
>>>>
>>>> On Monday 19 June 2017 10:23 PM, Daniel Fuchs wrote:
>>>>> Hi,
>>>>>
>>>>> If I'm not mistaken then this will make it impossible
>>>>> for earlier release to interoperate with newer releases
>>>>> as the LocalRMIClientSocketFactory class will not be
>>>>> present the client tries to deserialize the stub.
>>>>>
>>>>> best regards,
>>>>>
>>>>> -- daniel
>>>>>
>>>>> On 19/06/2017 11:52, Ujwal Vangapally wrote:
>>>>>> Hi,
>>>>>>
>>>>>> Kindly review the fix for bug below
>>>>>>
>>>>>> https://bugs.openjdk.java.net/browse/JDK-8173180
>>>>>>
>>>>>> webrev: 
>>>>>> http://cr.openjdk.java.net/~uvangapally/webrev/2017/8173180/webrev.00/ 
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Ujwal
>>>>>>
>>>>>
>>>>
>>>
>>
>

From mandy.chung at oracle.com  Tue Jun 20 15:37:43 2017
From: mandy.chung at oracle.com (Mandy Chung)
Date: Tue, 20 Jun 2017 08:37:43 -0700
Subject: Review Request JDK-8182596: Fix broken links in
 com.sun.tools.attach.VirtualMachine
Message-ID: <17D5BAAC-38DC-4949-ACC7-58DE92F5550C@oracle.com>

diff --git a/src/jdk.attach/share/classes/com/sun/tools/attach/VirtualMachine.java b/src/jdk.attach/share/classes/com/sun/tools/attach/VirtualMachine.java
--- a/src/jdk.attach/share/classes/com/sun/tools/attach/VirtualMachine.java
+++ b/src/jdk.attach/share/classes/com/sun/tools/attach/VirtualMachine.java
@@ -59,9 +59,9 @@
  * {@link java.lang.instrument} for a detailed description on how these agents
  * are loaded and started). The {@link #loadAgentLibrary loadAgentLibrary} and
  * {@link #loadAgentPath loadAgentPath} methods are used to load agents that
- * are deployed either in a dynamic library or statically linked into the VM and make use of the <a
- * href="../../../../../../../../technotes/guides/jvmti/index.html">JVM Tools
- * Interface</a>. </p>
+ * are deployed either in a dynamic library or statically linked into the VM and
+ * make use of the <a href="{@docRoot}/../specs/jvmti.html">JVM Tools Interface</a>.
+ * </p>
  *
  * <p> In addition to loading agents a VirtualMachine provides read access to the
  * {@link java.lang.System#getProperties() system properties} in the target VM.
@@ -289,8 +289,8 @@
     /**
      * Loads an agent library.
      *
-     * <p> A <a href="../../../../../../../../technotes/guides/jvmti/index.html">JVM
-     * TI</a> client is called an <i>agent</i>. It is developed in a native language.
+     * <p> A <a href="{@docRoot}/../specs/jvmti.html">JVM TI</a>
+     * client is called an <i>agent</i>. It is developed in a native language.
      * A JVM TI agent is deployed in a platform specific manner but it is typically the
      * platform equivalent of a dynamic library. Alternatively, it may be statically linked into the VM.
      * This method causes the given agent library to be loaded into the target
@@ -298,8 +298,8 @@
      * It then causes the target VM to invoke the {@code Agent_OnAttach} function
      * or, for a statically linked agent named 'L', the {@code Agent_OnAttach_L} function
      * as specified in the
-     * <a href="../../../../../../../../technotes/guides/jvmti/index.html"> JVM Tools
-     * Interface</a> specification. Note that the {@code Agent_OnAttach[_L]}
+     * <a href="{@docRoot}/../specs/jvmti.html">JVM Tools Interface</a> specification.
+     * Note that the {@code Agent_OnAttach[_L]}
      * function is invoked even if the agent library was loaded prior to invoking
      * this method.
      *
@@ -380,8 +380,8 @@
     /**
      * Load a native agent library by full pathname.
      *
-     * <p> A <a href="../../../../../../../../technotes/guides/jvmti/index.html">JVM
-     * TI</a> client is called an <i>agent</i>. It is developed in a native language.
+     * <p> A <a href="{@docRoot}/../specs/jvmti.html">JVM TI</a>
+     * client is called an <i>agent</i>. It is developed in a native language.
      * A JVM TI agent is deployed in a platform specific manner but it is typically the
      * platform equivalent of a dynamic library. Alternatively, the native
      * library specified by the agentPath parameter may be statically
@@ -397,8 +397,7 @@
      * It then causes the target VM to invoke the {@code Agent_OnAttach}
      * function or, for a statically linked agent named 'L', the
      * {@code Agent_OnAttach_L} function as specified in the
-     * <a href="../../../../../../../../technotes/guides/jvmti/index.html"> JVM Tools
-     * Interface</a> specification.
+     * <a href="{@docRoot}/../specs/jvmti.html">JVM Tools Interface</a> specification.
      * Note that the {@code Agent_OnAttach[_L]}
      * function is invoked even if the agent library was loaded prior to invoking
      * this method.
@@ -611,9 +610,9 @@
      * way as on the command line, you need to specify at least the
      * {@code com.sun.management.jmxremote.port} property.
      *
-     * <p> See the online documentation for <a
-     * href="../../../../../../../../technotes/guides/management/agent.html">
-     * Monitoring and Management Using JMX Technology</a> for further details.
+     * <p> See the online documentation for 
+     * {@extLink monitoring_and_management_using_jmx_technology
+     * Monitoring and Management Using JMX Technology} for further details.
      *
      * @param   agentProperties
      *          A Properties object containing the configuration properties
@@ -642,9 +641,9 @@
     /**
      * Starts the local JMX management agent in the target virtual machine.
      *
-     * <p> See the online documentation for <a
-     * href="../../../../../../../../technotes/guides/management/agent.html">
-     * Monitoring and Management Using JMX Technology</a> for further details.
+     * <p> See the online documentation for 
+     * {@extLink monitoring_and_management_using_jmx_technology
+     * Monitoring and Management Using JMX Technology} for further details.
      *
      * @return  The String representation of the local connector's service address.
      *          The value can be parsed by the

From Alan.Bateman at oracle.com  Tue Jun 20 15:41:05 2017
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Tue, 20 Jun 2017 16:41:05 +0100
Subject: Review Request JDK-8182596: Fix broken links in
 com.sun.tools.attach.VirtualMachine
In-Reply-To: <17D5BAAC-38DC-4949-ACC7-58DE92F5550C@oracle.com>
References: <17D5BAAC-38DC-4949-ACC7-58DE92F5550C@oracle.com>
Message-ID: <bf2ef888-2a8c-8b8d-5552-711fcbf5f34a@oracle.com>

Looks good.

On 20/06/2017 16:37, Mandy Chung wrote:
> diff --git a/src/jdk.attach/share/classes/com/sun/tools/attach/VirtualMachine.java b/src/jdk.attach/share/classes/com/sun/tools/attach/VirtualMachine.java
> --- a/src/jdk.attach/share/classes/com/sun/tools/attach/VirtualMachine.java
> +++ b/src/jdk.attach/share/classes/com/sun/tools/attach/VirtualMachine.java
> @@ -59,9 +59,9 @@
>    * {@link java.lang.instrument} for a detailed description on how these agents
>    * are loaded and started). The {@link #loadAgentLibrary loadAgentLibrary} and
>    * {@link #loadAgentPath loadAgentPath} methods are used to load agents that
> - * are deployed either in a dynamic library or statically linked into the VM and make use of the <a
> - * href="../../../../../../../../technotes/guides/jvmti/index.html">JVM Tools
> - * Interface</a>. </p>
> + * are deployed either in a dynamic library or statically linked into the VM and
> + * make use of the <a href="{@docRoot}/../specs/jvmti.html">JVM Tools Interface</a>.
> + * </p>
>    *
>    * <p> In addition to loading agents a VirtualMachine provides read access to the
>    * {@link java.lang.System#getProperties() system properties} in the target VM.
> @@ -289,8 +289,8 @@
>       /**
>        * Loads an agent library.
>        *
> -     * <p> A <a href="../../../../../../../../technotes/guides/jvmti/index.html">JVM
> -     * TI</a> client is called an <i>agent</i>. It is developed in a native language.
> +     * <p> A <a href="{@docRoot}/../specs/jvmti.html">JVM TI</a>
> +     * client is called an <i>agent</i>. It is developed in a native language.
>        * A JVM TI agent is deployed in a platform specific manner but it is typically the
>        * platform equivalent of a dynamic library. Alternatively, it may be statically linked into the VM.
>        * This method causes the given agent library to be loaded into the target
> @@ -298,8 +298,8 @@
>        * It then causes the target VM to invoke the {@code Agent_OnAttach} function
>        * or, for a statically linked agent named 'L', the {@code Agent_OnAttach_L} function
>        * as specified in the
> -     * <a href="../../../../../../../../technotes/guides/jvmti/index.html"> JVM Tools
> -     * Interface</a> specification. Note that the {@code Agent_OnAttach[_L]}
> +     * <a href="{@docRoot}/../specs/jvmti.html">JVM Tools Interface</a> specification.
> +     * Note that the {@code Agent_OnAttach[_L]}
>        * function is invoked even if the agent library was loaded prior to invoking
>        * this method.
>        *
> @@ -380,8 +380,8 @@
>       /**
>        * Load a native agent library by full pathname.
>        *
> -     * <p> A <a href="../../../../../../../../technotes/guides/jvmti/index.html">JVM
> -     * TI</a> client is called an <i>agent</i>. It is developed in a native language.
> +     * <p> A <a href="{@docRoot}/../specs/jvmti.html">JVM TI</a>
> +     * client is called an <i>agent</i>. It is developed in a native language.
>        * A JVM TI agent is deployed in a platform specific manner but it is typically the
>        * platform equivalent of a dynamic library. Alternatively, the native
>        * library specified by the agentPath parameter may be statically
> @@ -397,8 +397,7 @@
>        * It then causes the target VM to invoke the {@code Agent_OnAttach}
>        * function or, for a statically linked agent named 'L', the
>        * {@code Agent_OnAttach_L} function as specified in the
> -     * <a href="../../../../../../../../technotes/guides/jvmti/index.html"> JVM Tools
> -     * Interface</a> specification.
> +     * <a href="{@docRoot}/../specs/jvmti.html">JVM Tools Interface</a> specification.
>        * Note that the {@code Agent_OnAttach[_L]}
>        * function is invoked even if the agent library was loaded prior to invoking
>        * this method.
> @@ -611,9 +610,9 @@
>        * way as on the command line, you need to specify at least the
>        * {@code com.sun.management.jmxremote.port} property.
>        *
> -     * <p> See the online documentation for <a
> -     * href="../../../../../../../../technotes/guides/management/agent.html">
> -     * Monitoring and Management Using JMX Technology</a> for further details.
> +     * <p> See the online documentation for
> +     * {@extLink monitoring_and_management_using_jmx_technology
> +     * Monitoring and Management Using JMX Technology} for further details.
>        *
>        * @param   agentProperties
>        *          A Properties object containing the configuration properties
> @@ -642,9 +641,9 @@
>       /**
>        * Starts the local JMX management agent in the target virtual machine.
>        *
> -     * <p> See the online documentation for <a
> -     * href="../../../../../../../../technotes/guides/management/agent.html">
> -     * Monitoring and Management Using JMX Technology</a> for further details.
> +     * <p> See the online documentation for
> +     * {@extLink monitoring_and_management_using_jmx_technology
> +     * Monitoring and Management Using JMX Technology} for further details.
>        *
>        * @return  The String representation of the local connector's service address.
>        *          The value can be parsed by the


From serguei.spitsyn at oracle.com  Tue Jun 20 16:39:12 2017
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Tue, 20 Jun 2017 09:39:12 -0700
Subject: Review Request JDK-8182596: Fix broken links in
 com.sun.tools.attach.VirtualMachine
In-Reply-To: <17D5BAAC-38DC-4949-ACC7-58DE92F5550C@oracle.com>
References: <17D5BAAC-38DC-4949-ACC7-58DE92F5550C@oracle.com>
Message-ID: <11e01f4d-01ae-536d-7705-51dc428dff39@oracle.com>

Looks good.

Thanks,
Serguei

On 6/20/17 08:37, Mandy Chung wrote:
> diff --git a/src/jdk.attach/share/classes/com/sun/tools/attach/VirtualMachine.java b/src/jdk.attach/share/classes/com/sun/tools/attach/VirtualMachine.java
> --- a/src/jdk.attach/share/classes/com/sun/tools/attach/VirtualMachine.java
> +++ b/src/jdk.attach/share/classes/com/sun/tools/attach/VirtualMachine.java
> @@ -59,9 +59,9 @@
>    * {@link java.lang.instrument} for a detailed description on how these agents
>    * are loaded and started). The {@link #loadAgentLibrary loadAgentLibrary} and
>    * {@link #loadAgentPath loadAgentPath} methods are used to load agents that
> - * are deployed either in a dynamic library or statically linked into the VM and make use of the <a
> - * href="../../../../../../../../technotes/guides/jvmti/index.html">JVM Tools
> - * Interface</a>. </p>
> + * are deployed either in a dynamic library or statically linked into the VM and
> + * make use of the <a href="{@docRoot}/../specs/jvmti.html">JVM Tools Interface</a>.
> + * </p>
>    *
>    * <p> In addition to loading agents a VirtualMachine provides read access to the
>    * {@link java.lang.System#getProperties() system properties} in the target VM.
> @@ -289,8 +289,8 @@
>       /**
>        * Loads an agent library.
>        *
> -     * <p> A <a href="../../../../../../../../technotes/guides/jvmti/index.html">JVM
> -     * TI</a> client is called an <i>agent</i>. It is developed in a native language.
> +     * <p> A <a href="{@docRoot}/../specs/jvmti.html">JVM TI</a>
> +     * client is called an <i>agent</i>. It is developed in a native language.
>        * A JVM TI agent is deployed in a platform specific manner but it is typically the
>        * platform equivalent of a dynamic library. Alternatively, it may be statically linked into the VM.
>        * This method causes the given agent library to be loaded into the target
> @@ -298,8 +298,8 @@
>        * It then causes the target VM to invoke the {@code Agent_OnAttach} function
>        * or, for a statically linked agent named 'L', the {@code Agent_OnAttach_L} function
>        * as specified in the
> -     * <a href="../../../../../../../../technotes/guides/jvmti/index.html"> JVM Tools
> -     * Interface</a> specification. Note that the {@code Agent_OnAttach[_L]}
> +     * <a href="{@docRoot}/../specs/jvmti.html">JVM Tools Interface</a> specification.
> +     * Note that the {@code Agent_OnAttach[_L]}
>        * function is invoked even if the agent library was loaded prior to invoking
>        * this method.
>        *
> @@ -380,8 +380,8 @@
>       /**
>        * Load a native agent library by full pathname.
>        *
> -     * <p> A <a href="../../../../../../../../technotes/guides/jvmti/index.html">JVM
> -     * TI</a> client is called an <i>agent</i>. It is developed in a native language.
> +     * <p> A <a href="{@docRoot}/../specs/jvmti.html">JVM TI</a>
> +     * client is called an <i>agent</i>. It is developed in a native language.
>        * A JVM TI agent is deployed in a platform specific manner but it is typically the
>        * platform equivalent of a dynamic library. Alternatively, the native
>        * library specified by the agentPath parameter may be statically
> @@ -397,8 +397,7 @@
>        * It then causes the target VM to invoke the {@code Agent_OnAttach}
>        * function or, for a statically linked agent named 'L', the
>        * {@code Agent_OnAttach_L} function as specified in the
> -     * <a href="../../../../../../../../technotes/guides/jvmti/index.html"> JVM Tools
> -     * Interface</a> specification.
> +     * <a href="{@docRoot}/../specs/jvmti.html">JVM Tools Interface</a> specification.
>        * Note that the {@code Agent_OnAttach[_L]}
>        * function is invoked even if the agent library was loaded prior to invoking
>        * this method.
> @@ -611,9 +610,9 @@
>        * way as on the command line, you need to specify at least the
>        * {@code com.sun.management.jmxremote.port} property.
>        *
> -     * <p> See the online documentation for <a
> -     * href="../../../../../../../../technotes/guides/management/agent.html">
> -     * Monitoring and Management Using JMX Technology</a> for further details.
> +     * <p> See the online documentation for
> +     * {@extLink monitoring_and_management_using_jmx_technology
> +     * Monitoring and Management Using JMX Technology} for further details.
>        *
>        * @param   agentProperties
>        *          A Properties object containing the configuration properties
> @@ -642,9 +641,9 @@
>       /**
>        * Starts the local JMX management agent in the target virtual machine.
>        *
> -     * <p> See the online documentation for <a
> -     * href="../../../../../../../../technotes/guides/management/agent.html">
> -     * Monitoring and Management Using JMX Technology</a> for further details.
> +     * <p> See the online documentation for
> +     * {@extLink monitoring_and_management_using_jmx_technology
> +     * Monitoring and Management Using JMX Technology} for further details.
>        *
>        * @return  The String representation of the local connector's service address.
>        *          The value can be parsed by the


From daniel.fuchs at oracle.com  Tue Jun 20 17:24:06 2017
From: daniel.fuchs at oracle.com (Daniel Fuchs)
Date: Tue, 20 Jun 2017 18:24:06 +0100
Subject: RFR : 8182485 - JMX connections should have configurable
 ObjectInputFilter
In-Reply-To: <b1ddf0ac-34cf-9137-b8b5-800643289af1@oracle.com>
References: <b1ddf0ac-34cf-9137-b8b5-800643289af1@oracle.com>
Message-ID: <e202cf03-a135-aece-2f16-45736ebae065@oracle.com>

The fix looks good to me Harsha.

best regards,

-- daniel

On 20/06/2017 07:10, Harsha Wardhana B wrote:
> Hi,
> 
> Please review the below RFE,
> 
> JDK-8182485 : JMX connections should have configurable ObjectInputFilter
> 
> having webrev at
> 
> http://cr.openjdk.java.net/~hb/8182485/webrev.00/
> 
> The enhancement adds ObjectInputFilter to JMX connections to configure 
> filters during deserialization.
> 
> -Harsha
> 
> 
> 
> 


From Roger.Riggs at Oracle.com  Tue Jun 20 18:00:24 2017
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Tue, 20 Jun 2017 14:00:24 -0400
Subject: RFR : 8182485 - JMX connections should have configurable
 ObjectInputFilter
In-Reply-To: <e202cf03-a135-aece-2f16-45736ebae065@oracle.com>
References: <b1ddf0ac-34cf-9137-b8b5-800643289af1@oracle.com>
 <e202cf03-a135-aece-2f16-45736ebae065@oracle.com>
Message-ID: <7b911c07-065b-ce52-4364-4b78c390bdd4@Oracle.com>

+1

Roger


On 6/20/2017 1:24 PM, Daniel Fuchs wrote:
> The fix looks good to me Harsha.
>
> best regards,
>
> -- daniel
>
> On 20/06/2017 07:10, Harsha Wardhana B wrote:
>> Hi,
>>
>> Please review the below RFE,
>>
>> JDK-8182485 : JMX connections should have configurable ObjectInputFilter
>>
>> having webrev at
>>
>> http://cr.openjdk.java.net/~hb/8182485/webrev.00/
>>
>> The enhancement adds ObjectInputFilter to JMX connections to 
>> configure filters during deserialization.
>>
>> -Harsha
>>
>>
>>
>>
>


From Ronald_Servant at ca.ibm.com  Tue Jun 20 19:45:13 2017
From: Ronald_Servant at ca.ibm.com (Ronald Servant)
Date: Tue, 20 Jun 2017 19:45:13 +0000
Subject: Remove METHOD_ENTRY and METHOD_EXIT events from needed_capabilities
 in JDWP Agent
Message-ID: <OF17185C46.8474EA24-ON00258145.006B861B-00258145.006C82BF@notes.na.collabserv.com>

An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170620/0fb70479/attachment.html>

From jcbeyler at google.com  Tue Jun 20 21:22:06 2017
From: jcbeyler at google.com (JC Beyler)
Date: Tue, 20 Jun 2017 14:22:06 -0700
Subject: Low-Overhead Heap Profiling
In-Reply-To: <26a795b8-bdfd-506c-a772-ff9ce88e5f9a@oracle.com>
References: <CAF9BGBx8xFYudhJ=v1_hU5=nwEcC1X0NKRmC=E-RNN6SMs1ozg@mail.gmail.com>
 <CAF9BGBxj7+C6uy4dXHO-3pNorbpO7n22-9JmevM0F11xOo+MnA@mail.gmail.com>
 <CAF9BGBxrdLVmXRTDHj5hQHJhcyrkBM6+ex1TFWi=A5z=V2FSew@mail.gmail.com>
 <CAF9BGBz81YYVUsMLw9sOMR9fH7JQmyb8d_OZ9aJ47EOAqwWhEw@mail.gmail.com>
 <2af975e6-3827-bd57-0c3d-fadd54867a67@oracle.com>
 <365499b6-3f4d-a4df-9e7e-e72a739fb26b@oracle.com>
 <CAF9BGBy0T7A-+1dejYbr6f0d4s0XR-zv-WwHYqXrbe0EtCdsxQ@mail.gmail.com>
 <fb1155c6-ccf8-843c-34fc-fc4a45fc54cc@oracle.com>
 <CAF9BGBxqmXF9OFD4F9wVb-DQOj_UfNNcBhe+ARVxJ5E4cgCC1Q@mail.gmail.com>
 <102c59b8-25b6-8c21-8eef-1de7d0bbf629@oracle.com>
 <CAF9BGBxDaCOxobJi8Uu9kwKPRpnj-KqgWLfJO2z7UX71UnNF4g@mail.gmail.com>
 <CAF9BGByisZchXTsw1=mLETLnOggJShRJBDxFeOUgt+criTTGuQ@mail.gmail.com>
 <26a795b8-bdfd-506c-a772-ff9ce88e5f9a@oracle.com>
Message-ID: <CAF9BGBwePW3HE=bvZAe-9UsT20SU_9BOX9L02Y9qZdXTyjw8Aw@mail.gmail.com>

Hi all,

As I work through the remarks it seems easier to handle this in maybe two
shorter emails than one handling it all. So let me start with Robbin's
questions/remarks:

- I removed the lock relicas, so that is handled.

- For performance numbers, is there a set of benchmarks used by
serviceability-dev to show overhead? Currently, I've been using DaCapo for
example since it should show any issues quickly,

- For the helper API that I'm doing for the tests, would you want it more
polished straight away? My idea was to have something simple for testing
and then work on something a bit more open to users afterwards, either
directly in the test folders or somewhere else. Let me know what you think.

Thanks again and I'll be answering Thomas' comments in a bit,
Jc


On Tue, Jun 13, 2017 at 2:19 AM, Robbin Ehn <robbin.ehn at oracle.com> wrote:

> Hi JC,
>
> This version really makes my happy, awesome work!
>
> Since we got no attention from compiler and you manage to get ride of most
> of the compiler changes, I dropped compiler but added runtime instead.
> Serguei, when you have time, please chime in.
>
> On 06/12/2017 08:11 PM, JC Beyler wrote:
>
>> Dear all,
>>
>> I've continued working on this and have done the following webrev:
>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.05/
>>
>> What it does:
>>     - Implement the turn on/off system for the TLAB implementation
>>        - Basically you can turn the system on/off using the JVMTI API
>> (StartHeapProfiling and StopHeapProfiling here
>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.05/src/sh
>> are/vm/prims/jvmti.xml.patch)
>>           - I've currently implemented that the system resets the
>> profiling data at the StartHeapProfiling, this allows you to start
>> profiling, stop profiling, and then the user can query what happened during
>> profiling as a post-processing step :)
>>
>
> Nice addition.
>
>
>>     - I've currently, for sake of simplicity and Robbin hinted it would
>> be better for now, removed the mutex code for the data but think that will
>> have to come back in a next patch or in this one at some point
>>
>
> 29 // Keep muxlock for now
> ....
> 237   // Protects the traces currently sampled (below).
> 238   volatile intptr_t _allocated_traces_lock[1];
>
> This seems unused now, and I do not see any locks at all?
>
>
>>     - I've also really worked on the testing code to make it more
>> expandable in this case
>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.05/test/
>> serviceability/jvmti/HeapMonitor/libHeapMonitor.c.patch is now a bit
>> more generic and allows to turn on/off the sampling; it allows to query the
>> existence or not of frames passed from Java world to JNI, which allows the
>> test to define what should be seen and have the code in one place.
>>            - That is done using the helper class
>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.05/test/
>> serviceability/jvmti/HeapMonitor/MyPackage/Frame.java.patch
>>               - Basically each test can now provide an array of Frames
>> that the native library can check the internal data for a match. This
>> allows to have various tests have their own signatures, etc.
>>
>
> To get a more potential users it's nice with a Java API and as you say
> simplifies tests, good.
>
>
>>            - This has allowed me to add a nice test here to test the wipe
>> out of the data:
>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.05/test/
>> serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorOnOffT
>> est.java.patch
>>
>>    - Finally, I've done initial overhead testing and, though my data was
>> a bit noisy, I saw no real overhead using the Tlab approach while off. I
>> will try to get better data and more importantly, more stable data when
>> turning it on.
>>
>
> I think it's key to have numbers showing ~0% performance impact here.
>
>
>> Things I still need to do:
>>     - Have to fix that TLAB case for the FastTLABRefill
>>     - Have to start looking at the data to see that it is consistent and
>> does gather the right samples, right frequency, etc.
>>     - Have to check the GC elements and what that produces
>>     - Run a slowdebug run and ensure I fixed all those issues you saw
>> Robbin
>>
>
> Also we will need support for some more platform, as the patch stands now,
> it's only the src/cpu/x86/vm/macroAssembler_x86.cpp part that needs to be
> done for other platforms?
>
> Thanks!
>
> /Robbin
>
>
>> Thanks for looking at the webrev and have a great week!
>> Jc
>>
>> On Fri, Jun 2, 2017 at 8:57 AM, JC Beyler <jcbeyler at google.com <mailto:
>> jcbeyler at google.com>> wrote:
>>
>>     Dear all,
>>
>>     Thanks Robbin for the comments. I have left the MuxLocker for now and
>> will look at removing it or as a future enhancement as you say. I did make
>> the class extends and
>>     added a TODO for myself to test this in slowdebug.
>>
>>     I have also put a new webrev up that is still a work in progress but
>> should allow us to talk about TLAB vs C1/C2 modifications.
>>
>>     TLAB implementation: http://cr.openjdk.java.net/~ra
>> sbold/8171119/webrev.04/ <http://cr.openjdk.java.net/~r
>> asbold/8171119/webrev.04/>
>>     C1/C2 implementation: http://cr.openjdk.java.net/~ra
>> sbold/8171119/webrev.03/ <http://cr.openjdk.java.net/~r
>> asbold/8171119/webrev.03/>
>>
>>     What this webrev has is:
>>         - I have put in a TLAB implementation, it is a WIP, and I have
>> not yet done the qualitative/quantitative study on it vs the implementation
>> using compilation changes
>>     but the big parts are in place
>>             - Caveats:
>>                 - There is a TODO: http://cr.openjdk.java.net/~ra
>> sbold/8171119/webrev.04/src/cpu/x86/vm/macroAssembler_x86.cpp.patch
>>     <http://cr.openjdk.java.net/~rasbold/8171119/webrev.04/src/c
>> pu/x86/vm/macroAssembler_x86.cpp.patch>
>>                     - I have not fixed the calculation in the case of a
>> FastTLABRefill case
>>                 - This is always on right now, there is no way to turn it
>> off, that's also a TODO to be directed by the JVMTI API
>>
>>          - I also have circumvented the AsyncGetCallTrace using the
>> snippet of code you showed Robbin, it works for here/now
>>                - But we might have to revisit this one day because it
>> then does not try to get some of the native stacks and jumps directly to
>> the Java stacks (I see cases
>>     where this could be an issue)
>>                - However, this has cleaned up quite a bit of the code and
>> I have removed all mention of ASGCT and its structures now and use directly
>> the JVMTI structures
>>
>>         - GC is handled now, I have not yet done the
>> qualitative/quantitative study on it but the big parts are in place
>>
>>         - Due to the way the TLAB is called, the stack walker is now
>> correct and produces the right stacks it seems (this is a bold sentence
>> from my ONE JTREG test :))
>>
>>     Final notes on this webrev:
>>         - Have to fix that TLAB case for the FastTLABRefill
>>         - Implement the turn on/off system for the TLAB implementation
>>         - Have to start looking at the data to see that it is consistent
>> and does gather the right samples, right frequency, etc.
>>         - Have to check the GC elements and what that produces
>>         - Run a slowdebug run and ensure I fixed all those issues you saw
>> Robbin
>>
>>     As always, your comments and feedback are greatly appreciated! Happy
>> Friday!
>>     Jc
>>
>>
>>     On Tue, May 30, 2017 at 5:33 AM, Robbin Ehn <robbin.ehn at oracle.com
>> <mailto:robbin.ehn at oracle.com>> wrote:
>>
>>         Hi Jc,
>>
>>         On 05/22/2017 08:47 PM, JC Beyler wrote:
>>
>>             Dear all,
>>
>>             I have a new webrev up:
>>             http://cr.openjdk.java.net/~rasbold/8171119/webrev.03/ <
>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.03/>
>>
>>
>>         I liked this!
>>
>>         Two small things:
>>
>>         heapMonitoring.hpp
>>         class HeapMonitoring should extend AllStatic
>>
>>         heapMonitoring.cpp
>>         class MuxLocker should extend StackObj
>>         But I think you should skip MuxLocker or push it separate generic
>> enhancement.
>>
>>         Great with the jtreg test, thanks alot!
>>
>>
>>             This webrev has, I hope, fixed a lot of the comments from
>> Robbin:
>>                 - The casts normally are all C++ style
>>                 - Moved this to jdk10-hs
>>                    - I have not tested slowdebug yet, hopefully it does
>> not break there
>>                 - Added the garbage collection system:
>>                    - Now live sampled allocations are tracked throughout
>> their lifetime
>>                       - When GC happens, it moves the sampled allocation
>> information to two lists: recent and frequent GC lists
>>                           - Those lists use the array system that the
>> live objects were using before but have different re-use strategies
>>                    - Added the JVMTI API for them via a
>> GetFrequentGarbageTraces and GetGarbageTraces
>>             - Both use the same JVMTI structures
>>                    - Added the calls to them for the test, though I've
>> kept that test simple for now:
>>             http://cr.openjdk.java.net/~rasbold/8171119/webrev.03/raw_fi
>> les/new/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c
>>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.03/raw_f
>> iles/new/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c>
>>                       - As I write this, I notice my webrev is missing a
>> final change I made to the test that calls the same ReleaseTraces to each
>> live/garbage/frequent
>>             structure. This is updated in my local repo and will get in
>> the next webrev.
>>
>>             Next steps for this work are:
>>                  - Putting the TLAB implementation (yes not forgotten ;-))
>>                  - Adding more testing and separate the current test
>> system to check things a bit more thoroughly
>>                  - Have not tried to circumvent AsyncGetCallTrace yet
>>                  - Still have to double check the stack walker a bit more
>>
>>
>>         Looking forward to this.
>>
>>         Could someone from compiler take a look please?
>>
>>         Thanks!
>>
>>         /Robbin
>>
>>
>>             Happy webrev perusal!
>>             Jc
>>
>>
>>             On Tue, May 16, 2017 at 5:20 AM, Robbin Ehn <
>> robbin.ehn at oracle.com <mailto:robbin.ehn at oracle.com> <mailto:
>> robbin.ehn at oracle.com <mailto:robbin.ehn at oracle.com>>>
>>             wrote:
>>
>>                  Just a few answers,
>>
>>                  On 05/15/2017 06:48 PM, JC Beyler wrote:
>>
>>                      Dear all,
>>
>>                      I've updated the webrev to:
>>             http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/ <
>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/>
>>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/ <
>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/>>
>>                      <http://cr.openjdk.java.net/~
>> rasbold/8171119/webrev.02/ <http://cr.openjdk.java.net/~r
>> asbold/8171119/webrev.02/>
>>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/ <
>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/>>>
>>
>>
>>                  I'll look at this later, thanks!
>>
>>
>>                      Robbin,
>>                      I believe I have addressed most of your items with
>> webrev 02:
>>                          - I added a JTreg test to show how it works:
>>             http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/raw_fi
>> les/new/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c
>>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/raw_f
>> iles/new/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c>
>>                      <http://cr.openjdk.java.net/~
>> rasbold/8171119/webrev.02/raw_files/new/test/serviceability/
>> jvmti/HeapMonitor/libHeapMonitor.c
>>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/raw_f
>> iles/new/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c>>
>>                      <http://cr.openjdk.java.net/~
>> rasbold/8171119/webrev.02/raw_files/new/test/serviceability/
>> jvmti/HeapMonitor/libHeapMonitor.c
>>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/raw_f
>> iles/new/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c>
>>                      <http://cr.openjdk.java.net/~
>> rasbold/8171119/webrev.02/raw_files/new/test/serviceability/
>> jvmti/HeapMonitor/libHeapMonitor.c
>>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.02/raw_f
>> iles/new/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c>>>
>>                         - I've modified the code to use its own data
>> structures both internally and externally, this will make it easier to move
>> out of AsyncGetCallTrace as
>>             we move
>>                      forward, that is still on my TODOs
>>                         - I cleaned up the JVMTI API by passing a
>> structure that handles the num_traces and put in a ReleaseTraces as well
>>                         - I cleaned up other issues as well.
>>
>>                      However, I have three questions, which are probably
>> because I'm new in this community:
>>                         1) My previous webrevs were based off of JDK9 by
>> mistake. When I took JDK10 via : hg clone http://hg.openjdk.java.net/jdk
>> 10/jdk10
>>             <http://hg.openjdk.java.net/jdk10/jdk10>
>>                      <http://hg.openjdk.java.net/jdk10/jdk10 <
>> http://hg.openjdk.java.net/jdk10/jdk10>> <http://hg.openjdk.java.net/jd
>> k10/jdk10
>>             <http://hg.openjdk.java.net/jdk10/jdk10> <
>> http://hg.openjdk.java.net/jdk10/jdk10 <http://hg.openjdk.java.net/jd
>> k10/jdk10>>> jdk10
>>                             - I don't see code compatible with what you
>> were showing (ie your patches don't make sense for that code base; ex:
>> klass is still accessed via
>>             klass() for
>>                      example in collectedHeap.inline.hpp)
>>                             - Would you know what is the right hg clone
>> command so we are working on the same code base?
>>
>>
>>                  We use jdk10-hs, e.g.
>>                  hg tclone http://hg.openjdk.java.net/jdk10/hs <
>> http://hg.openjdk.java.net/jdk10/hs> <http://hg.openjdk.java.net/jdk10/hs
>>             <http://hg.openjdk.java.net/jdk10/hs>> 10-hs
>>
>>                  There is sporadic big merges going from
>> jdk9->jdk10->jdk10-hs and jdk10-hs->jdk10, so 10 is moving...
>>
>>
>>                         2) You mentioned I was using os::malloc, new,
>> NEW_C_HEAP_ARRAY; I cleaned out the os::malloc but which of the new vs
>> NEW_C_HEAP_ARRAY should I use.
>>             It might be
>>                      that I don't understand when one uses one or the
>> other but I see both used around the code base?
>>                           - Is it that new is to be used for anything
>> internal and NEW_C_HEAP_ARRAY anything provided to the JVMTI users outside
>> of the JVM?
>>
>>
>>                  We overload new operator when you extend correct base
>> class, e.g. CHeapObj<mtInternal> so use 'new'
>>                  But for arrays you will need the macro NEW_C_HEAP_ARRAY.
>>
>>
>>                         3) Casts: same kind question: which should I use.
>> The code was using a bit of everything, I'll refactor it entirely but I was
>> not clear if I should
>>             go to C casts
>>                      or C++ casts as I see both in the codebase. What is
>> the convention I should use?
>>
>>
>>                  Just be consist, use what suites you, C++ casts might be
>> preferable, if we are moving towards C++11.
>>                  And use 'right' cast, e.g. going from Thread* to
>> JavaThread* you should use C cast or static_cast, not reinterpret_cast I
>> would say.
>>
>>
>>                      Final notes on this webrev:
>>                          - I am still missing:
>>                            - Putting a TLAB implementation so that we can
>> compare both webrevs
>>                            - Have not tried to circumvent
>> AsyncGetCallTrace
>>                            - Putting in the handling of GC'd objects
>>                            - Fix a stack walker issue I have seen, I
>> think I know the problem and will test that theory out for the next webrev
>>
>>                      I will work on integrating those items for the next
>> webrev!
>>
>>
>>                  Thanks!
>>
>>
>>                      Thanks for your help,
>>                      Jc
>>
>>                      Ps:  I tested this on a new repo:
>>
>>                      hg clone http://hg.openjdk.java.net/jdk10/jdk10 <
>> http://hg.openjdk.java.net/jdk10/jdk10> <http://hg.openjdk.java.net/jd
>> k10/jdk10
>>             <http://hg.openjdk.java.net/jdk10/jdk10>> <
>> http://hg.openjdk.java.net/jdk10/jdk10 <http://hg.openjdk.java.net/jd
>> k10/jdk10>
>>                      <http://hg.openjdk.java.net/jdk10/jdk10 <
>> http://hg.openjdk.java.net/jdk10/jdk10>>> jdk10
>>                      ... building it
>>                      cd test
>>                      jtreg -nativepath:<path-to-jdk10>/bu
>> ild/linux-x86_64-normal-server-release/support/test/hotspot/jtreg/native/lib/
>> -jdk
>>                      <path-to-jdk10>/linux-x86_64-
>> normal-server-release/images/jdk ../hotspot/test/serviceability
>> /jvmti/HeapMonitor/
>>
>>
>>                  I'll test it out!
>>
>>                  /Robbin
>>
>>
>>
>>                      On Thu, May 4, 2017 at 11:21 PM,
>> serguei.spitsyn at oracle.com <mailto:serguei.spitsyn at oracle.com> <mailto:
>> serguei.spitsyn at oracle.com
>>             <mailto:serguei.spitsyn at oracle.com>> <mailto:
>> serguei.spitsyn at oracle.com <mailto:serguei.spitsyn at oracle.com>
>>                      <mailto:serguei.spitsyn at oracle.com <mailto:
>> serguei.spitsyn at oracle.com>>> <serguei.spitsyn at oracle.com <mailto:
>> serguei.spitsyn at oracle.com>
>>             <mailto:serguei.spitsyn at oracle.com <mailto:
>> serguei.spitsyn at oracle.com>> <mailto:serguei.spitsyn at oracle.com <mailto:
>> serguei.spitsyn at oracle.com>
>>
>>                      <mailto:serguei.spitsyn at oracle.com <mailto:
>> serguei.spitsyn at oracle.com>>>> wrote:
>>
>>                           Robbin,
>>
>>                           Thank you for forwarding!
>>                           I will review it.
>>
>>                           Thanks,
>>                           Serguei
>>
>>
>>
>>                           On 5/4/17 02:13, Robbin Ehn wrote:
>>
>>                               Hi,
>>
>>                               To me the compiler changes looks what is
>> expected.
>>                               It would be good if someone from compiler
>> could take a look at that.
>>                               Added compiler to mail thread.
>>
>>                               Also adding Serguei, It would be good with
>> his view also.
>>
>>                               My initial take on it, read through most of
>> the code and took it for a ride.
>>
>>                               ##############################
>>                               - Regarding the compiler changes: I think
>> we need the 'TLAB end' trickery (mentioned by Tony P)
>>                               instead of a separate check for sampling in
>> fast path for the final version.
>>
>>                               ##############################
>>                               - This patch I had to apply to get it
>> compile on JDK 10:
>>
>>                               diff -r ac3ded340b35
>> src/share/vm/gc/shared/collectedHeap.inline.hpp
>>                               --- a/src/share/vm/gc/shared/collectedHeap.inline.hpp
>>   Fri Apr 28 14:31:38 2017 +0200
>>                               +++ b/src/share/vm/gc/shared/collectedHeap.inline.hpp
>>   Thu May 04 10:22:56 2017 +0200
>>                               @@ -87,3 +87,3 @@
>>                                     // support for object alloc event
>> (no-op most of the time)
>>                               -    if (klass() != NULL && klass()->name()
>> != NULL) {
>>                               +    if (klass != NULL && klass->name() !=
>> NULL) {
>>                                       Thread *base_thread =
>> Thread::current();
>>                               diff -r ac3ded340b35
>> src/share/vm/runtime/heapMonitoring.cpp
>>                               --- a/src/share/vm/runtime/heapMonitoring.cpp
>>   Fri Apr 28 14:31:38 2017 +0200
>>                               +++ b/src/share/vm/runtime/heapMonitoring.cpp
>>   Thu May 04 10:22:56 2017 +0200
>>                               @@ -316,3 +316,3 @@
>>                                   JavaThread *thread =
>> reinterpret_cast<JavaThread *>(Thread::current());
>>                               -  assert(o->size() << LogHeapWordSize ==
>> byte_size,
>>                               +  assert(o->size() << LogHeapWordSize ==
>> (long)byte_size,
>>                                          "Object size is incorrect.");
>>
>>                               ##############################
>>                               - This patch I had to apply to get it not
>> asserting during slowdebug:
>>
>>                               --- a/src/share/vm/runtime/heapMonitoring.cpp
>>   Fri Apr 28 15:15:16 2017 +0200
>>                               +++ b/src/share/vm/runtime/heapMonitoring.cpp
>>   Thu May 04 10:24:25 2017 +0200
>>                               @@ -32,3 +32,3 @@
>>                                 // TODO(jcbeyler): should we make this
>> into a JVMTI structure?
>>                               -struct StackTraceData {
>>                               +struct StackTraceData :
>> CHeapObj<mtInternal> {
>>                                   ASGCT_CallTrace *trace;
>>                               @@ -143,3 +143,2 @@
>>                                 StackTraceStorage::StackTraceStorage() :
>>                               -    _allocated_traces(new
>> StackTraceData*[MaxHeapTraces]),
>>                                     _allocated_traces_size(MaxHeap
>> Traces),
>>                               @@ -147,2 +146,3 @@
>>                                     _allocated_count(0) {
>>                               +  _allocated_traces =
>> NEW_C_HEAP_ARRAY(StackTraceData*, MaxHeapTraces, mtInternal);
>>                                   memset(_allocated_traces, 0,
>> sizeof(*_allocated_traces) * MaxHeapTraces);
>>                               @@ -152,3 +152,3 @@
>>                                 StackTraceStorage::~StackTraceStorage() {
>>                               -  delete[] _allocated_traces;
>>                               +  FREE_C_HEAP_ARRAY(StackTraceData*,
>> _allocated_traces);
>>                                 }
>>
>>                               - Classes should extend correct base class
>> for which type of memory is used for it e.g.: CHeapObj<mt????> or StackObj
>> or AllStatic
>>                               - The style in heapMonitoring.cpp is a bit
>> different from normal vm-style, e.g. using C++ casts instead of C. You mix
>> NEW_C_HEAP_ARRAY,
>>             os::malloc and new.
>>                               - In jvmtiHeapTransition.hpp you use C cast
>> instead.
>>
>>                               ##############################
>>                               - This patch I had apply to get traces
>> without setting an ?unrelated? capability
>>                               - Should this not be a new capability?
>>
>>                               diff -r c02a5d8785bf
>> src/share/vm/prims/forte.cpp
>>                               --- a/src/share/vm/prims/forte.cpp    Fri
>> Apr 28 15:15:16 2017 +0200
>>                               +++ b/src/share/vm/prims/forte.cpp    Thu
>> May 04 10:24:25 2017 +0200
>>                               @@ -530,6 +530,6 @@
>>
>>                               -  if (!JvmtiExport::should_post_class_load())
>> {
>>                               +/*  if (!JvmtiExport::should_post_class_load())
>> {
>>                                     trace->num_frames =
>> ticks_no_class_load; // -1
>>                                     return;
>>                               -  }
>>                               +  }*/
>>
>>                               ##############################
>>                               - forte.cpp: (I know this is not part of
>> your changes but)
>>                               find_jmethod_id_or_null give me NULL for my
>> test.
>>                               It looks like we actually want the regular
>> jmethod_id() ?
>>
>>                               Since we are the thread we are talking
>> about (and in same ucontext) and thread is in vm and have a last java frame,
>>                               I think most of the checks done in
>> AsyncGetCallTrace is irrelevant, so you should be-able to call
>> forte_fill_call_trace_given_top directly.
>>                               But since we might need jmethod_id() if
>> possible to avoid getting method id NULL,
>>                               we need some fixes in forte code, or just
>> do the vframStream loop inside heapMonitoring.cpp and not use forte.cpp.
>>
>>                               Something like:
>>
>>                                  if (jthread->has_last_Java_frame()) {
>> // just to be safe
>>                                    vframeStream vfst(jthread);
>>                                    while (!vfst.at_end()) {
>>                                      Method* m = vfst.method();
>>                                      m->jmethod_id();
>>                                      m->line_number_from_bci(vfst.bci());
>>                                      vfst.next();
>>                                    }
>>
>>                               - This is a bit confusing in forte.cpp,
>> trace->frames[count].lineno = bci.
>>                               Line number should be
>> m->line_number_from_bci(bci);
>>                               Do the heapMonitoring suppose to trace with
>> bci or line number?
>>                               I would say bci, meaning we should either
>> rename ASGCT_CallFrame?lineno or use another data structure which says bci.
>>
>>                               ##############################
>>                               - // TODO(jcbeyler): remove this extra code
>> handling the extra trace for
>>                               Please fix all these TODO's :)
>>
>>                               ##############################
>>                               - heapMonitoring.hpp:
>>                               // TODO(jcbeyler): is this algorithm
>> acceptable in open source?
>>
>>                               Why is this comment here? What is the
>> implication?
>>                               Have you tested any simpler algorithm?
>>
>>                               ##############################
>>                               - Create a sanity jtreg test.
>> (./hotspot/make/test/JtregNative.gmk for building the agent)
>>
>>                               ##############################
>>                               - monitoring_period vs HeapMonitorRate,
>> pick rate or period.
>>
>>                               ##############################
>>                               - globals.hpp
>>                               Why is MaxHeapTraces not
>> settable/overridable from jvmti interface? That would be handy.
>>
>>                               ##############################
>>                               - jvmtiStackTraceData + ASGCT_CallFrame
>> memory
>>                               Are the agent suppose to loop through and
>> free all ASGCT_CallFrame?
>>                               Wouldn't it be better with some kinda
>> protocol, like:
>>                               (*jvmti)->GetLiveTraces(jvmti,
>> &stack_traces, &num_traces);
>>                               (*jvmti)->ReleaseTraces(jvmti,
>> stack_traces, num_traces);
>>
>>                               Also using another data structure that have
>> num_traces inside it simplifies things.
>>                               So I'm not convinced using the async
>> structure is the best way forward.
>>
>>
>>                               I have more questions, but I think it's
>> better if you respond and update the code first.
>>
>>                               Thanks!
>>
>>                               /Robbin
>>
>>
>>                               On 04/21/2017 11:34 PM, JC Beyler wrote:
>>
>>                                   Hi all,
>>
>>                                   I've added size information to the
>> allocation sampling system. This allows the callback to remember the size
>> of each sampled allocation.
>>             http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/ <
>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/>
>>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/ <
>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/>>
>>                      <http://cr.openjdk.java.net/~
>> rasbold/8171119/webrev.01/ <http://cr.openjdk.java.net/~r
>> asbold/8171119/webrev.01/>
>>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/ <
>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/>>>
>>
>>                                   The new webrev.01 also adds the actual
>> heap monitoring sampling system in files:
>>             http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/sh
>> are/vm/runtime/heapMonitoring.cpp.patch
>>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/s
>> hare/vm/runtime/heapMonitoring.cpp.patch>
>>                      <http://cr.openjdk.java.net/~
>> rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.cpp.patch
>>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/s
>> hare/vm/runtime/heapMonitoring.cpp.patch>>
>>                                   <http://cr.openjdk.java.net/~r
>> asbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.cpp.patch
>>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/s
>> hare/vm/runtime/heapMonitoring.cpp.patch>
>>                      <http://cr.openjdk.java.net/~
>> rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.cpp.patch
>>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/s
>> hare/vm/runtime/heapMonitoring.cpp.patch>>>
>>                                   and
>>             http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/sh
>> are/vm/runtime/heapMonitoring.hpp.patch
>>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/s
>> hare/vm/runtime/heapMonitoring.hpp.patch>
>>                      <http://cr.openjdk.java.net/~
>> rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.hpp.patch
>>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/s
>> hare/vm/runtime/heapMonitoring.hpp.patch>>
>>                                   <http://cr.openjdk.java.net/~r
>> asbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.hpp.patch
>>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/s
>> hare/vm/runtime/heapMonitoring.hpp.patch>
>>                      <http://cr.openjdk.java.net/~
>> rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.hpp.patch
>>             <http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/s
>> hare/vm/runtime/heapMonitoring.hpp.patch>>>
>>
>>                                   My next step is to add the GC part to
>> the webrev, which will allow users to determine what objects are live and
>> what are garbage.
>>
>>                                   Thanks for your attention and let me
>> know if there are any questions!
>>
>>                                   Have a wonderful Friday!
>>                                   Jc
>>
>>                                   On Mon, Apr 17, 2017 at 12:37 PM, JC
>> Beyler <jcbeyler at google.com <mailto:jcbeyler at google.com> <mailto:
>> jcbeyler at google.com
>>             <mailto:jcbeyler at google.com>> <mailto:jcbeyler at google.com
>> <mailto:jcbeyler at google.com> <mailto:jcbeyler at google.com <mailto:
>> jcbeyler at google.com>>>
>>                      <mailto:jcbeyler at google.com <mailto:
>> jcbeyler at google.com> <mailto:jcbeyler at google.com <mailto:
>> jcbeyler at google.com>> <mailto:jcbeyler at google.com
>>             <mailto:jcbeyler at google.com> <mailto:jcbeyler at google.com
>> <mailto:jcbeyler at google.com>>>>> wrote:
>>
>>                                        Hi all,
>>
>>                                        I worked on getting a few numbers
>> for overhead and accuracy for my feature. I'm unsure if here is the right
>> place to provide the full
>>             data, so I
>>                      am just
>>                                   summarizing
>>                                        here for now.
>>
>>                                        - Overhead of the feature
>>
>>                                        Using the Dacapo benchmark (
>> http://dacapobench.org/). My initial results are that sampling provides
>> 2.4% with a 512k sampling, 512k
>>             being our
>>                      default setting.
>>
>>                                        - Note: this was without the
>> tradesoap, tradebeans and tomcat benchmarks since they did not work with my
>> JDK9 (issue between Dacapo
>>             and JDK9 it seems)
>>                                        - I want to rerun next week to
>> ensure number stability
>>
>>                                        - Accuracy of the feature
>>
>>                                        I wrote a small microbenchmark
>> that allocates from two different stacktraces at a given ratio. For
>> example, 10% of stacktrace S1 and
>>             90% from
>>                      stacktrace
>>                                   S2. The
>>                                        microbenchmark was run 20 times, I
>> averaged the results and looked for accuracy. It seems that statistically
>> it is sound since if I
>>             allocated10%
>>                      S1 and 90%
>>                                   S2, with a
>>                                        sampling rate of 512k, I obtained
>> 9.61% S1 and 90.49% S2.
>>
>>                                        Let me know if there are any
>> questions on the numbers and if you'd like to see some more data.
>>
>>                                        Note: this was done using our
>> internal JDK8 implementation since the webrev provided by
>>             http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.ht
>> ml <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>
>>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.h
>> tml <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>>
>>                                   <http://cr.openjdk.java.net/~r
>> asbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~r
>> asbold/heapz/webrev.00/index.html>
>>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.h
>> tml <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>>>
>>                                   <http://cr.openjdk.java.net/~r
>> asbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~r
>> asbold/heapz/webrev.00/index.html>
>>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.h
>> tml <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>>
>>                      <http://cr.openjdk.java.net/~
>> rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~r
>> asbold/heapz/webrev.00/index.html>
>>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.h
>> tml <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>>>>
>> does not yet contain the whole
>>                                   implementation and therefore would have
>> been misleading.
>>
>>                                        Thanks,
>>                                        Jc
>>
>>
>>                                        On Tue, Apr 4, 2017 at 3:55 PM, JC
>> Beyler <jcbeyler at google.com <mailto:jcbeyler at google.com> <mailto:
>> jcbeyler at google.com
>>             <mailto:jcbeyler at google.com>> <mailto:jcbeyler at google.com
>> <mailto:jcbeyler at google.com>
>>                      <mailto:jcbeyler at google.com <mailto:
>> jcbeyler at google.com>>> <mailto:jcbeyler at google.com <mailto:
>> jcbeyler at google.com> <mailto:jcbeyler at google.com
>>             <mailto:jcbeyler at google.com>> <mailto:jcbeyler at google.com
>> <mailto:jcbeyler at google.com> <mailto:jcbeyler at google.com <mailto:
>> jcbeyler at google.com>>>>> wrote:
>>
>>                                            Hi all,
>>
>>                                            To move the discussion
>> forward, with Chuck Rasbold's help to make a webrev, we pushed this:
>>             http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.ht
>> ml <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>
>>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.h
>> tml <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>>
>>                      <http://cr.openjdk.java.net/~
>> rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~r
>> asbold/heapz/webrev.00/index.html>
>>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.h
>> tml <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>>>
>>                                   <http://cr.openjdk.java.net/~r
>> asbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~r
>> asbold/heapz/webrev.00/index.html>
>>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.h
>> tml <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>>
>>                      <http://cr.openjdk.java.net/~
>> rasbold/heapz/webrev.00/index.html <http://cr.openjdk.java.net/~r
>> asbold/heapz/webrev.00/index.html>
>>             <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.h
>> tml <http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html>>>>
>>                                            415 lines changed: 399 ins; 13
>> del; 3 mod; 51122 unchg
>>
>>                                            This is not a final change
>> that does the whole proposition from the JBS entry:
>> https://bugs.openjdk.java.net/browse/JDK-8177374
>>             <https://bugs.openjdk.java.net/browse/JDK-8177374>
>>                      <https://bugs.openjdk.java.net/browse/JDK-8177374 <
>> https://bugs.openjdk.java.net/browse/JDK-8177374>>
>>                                   <https://bugs.openjdk.java.net
>> /browse/JDK-8177374 <https://bugs.openjdk.java.net/browse/JDK-8177374>
>>             <https://bugs.openjdk.java.net/browse/JDK-8177374 <
>> https://bugs.openjdk.java.net/browse/JDK-8177374>>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170620/0891563d/attachment-0001.html>

From harsha.wardhana.b at oracle.com  Wed Jun 21 03:08:38 2017
From: harsha.wardhana.b at oracle.com (Harsha Wardhana B)
Date: Wed, 21 Jun 2017 08:38:38 +0530
Subject: RFR : 8182485 - JMX connections should have configurable
 ObjectInputFilter
In-Reply-To: <7b911c07-065b-ce52-4364-4b78c390bdd4@Oracle.com>
References: <b1ddf0ac-34cf-9137-b8b5-800643289af1@oracle.com>
 <e202cf03-a135-aece-2f16-45736ebae065@oracle.com>
 <7b911c07-065b-ce52-4364-4b78c390bdd4@Oracle.com>
Message-ID: <6ff76a62-896d-83f0-9e8c-be6473e26c02@oracle.com>

Thanks for the review Daniel,Roger.

-Harsha


On Tuesday 20 June 2017 11:30 PM, Roger Riggs wrote:
> +1
>
> Roger
>
>
> On 6/20/2017 1:24 PM, Daniel Fuchs wrote:
>> The fix looks good to me Harsha.
>>
>> best regards,
>>
>> -- daniel
>>
>> On 20/06/2017 07:10, Harsha Wardhana B wrote:
>>> Hi,
>>>
>>> Please review the below RFE,
>>>
>>> JDK-8182485 : JMX connections should have configurable 
>>> ObjectInputFilter
>>>
>>> having webrev at
>>>
>>> http://cr.openjdk.java.net/~hb/8182485/webrev.00/
>>>
>>> The enhancement adds ObjectInputFilter to JMX connections to 
>>> configure filters during deserialization.
>>>
>>> -Harsha
>>>
>>>
>>>
>>>
>>
>


From Alan.Bateman at oracle.com  Wed Jun 21 08:10:07 2017
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Wed, 21 Jun 2017 09:10:07 +0100
Subject: Remove METHOD_ENTRY and METHOD_EXIT events from
 needed_capabilities in JDWP Agent
In-Reply-To: <OF17185C46.8474EA24-ON00258145.006B861B-00258145.006C82BF@notes.na.collabserv.com>
References: <OF17185C46.8474EA24-ON00258145.006B861B-00258145.006C82BF@notes.na.collabserv.com>
Message-ID: <df117e76-589b-98b6-7845-aff816d7a9f1@oracle.com>

On 20/06/2017 20:45, Ronald Servant wrote:
> Looking at the Agent_OnLoad() code in debuginit.c of libjdwp, I see 
> that both the METHOD_ENTRY and METHOD_EXIT are listed as required, but 
> I don't see where they are ever enabled in the agent.
> In the J9 JVM making it possible to enable these events has severe 
> performance penalties. Since the events are not enabled in the JDWP 
> agent, simply reserved, we were hoping you'd be open to removing the 
> reservation of the capabilities.
>
If the JDWP agent never enables these events then it shouldn't need to 
obtain the capabilities. I have a vague recollection that they are 
enabled when single stepping, are you sure this isn't the case? Have you 
run the tests with without enabling the capabilities?

-Alan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170621/7b43924d/attachment.html>

From Ronald_Servant at ca.ibm.com  Wed Jun 21 14:35:15 2017
From: Ronald_Servant at ca.ibm.com (Ronald Servant)
Date: Wed, 21 Jun 2017 14:35:15 +0000
Subject: Remove METHOD_ENTRY and METHOD_EXIT events from
 needed_capabilities in JDWP Agent
In-Reply-To: <df117e76-589b-98b6-7845-aff816d7a9f1@oracle.com>
References: <df117e76-589b-98b6-7845-aff816d7a9f1@oracle.com>,
 <OF17185C46.8474EA24-ON00258145.006B861B-00258145.006C82BF@notes.na.collabserv.com>
Message-ID: <OFAF5D7ACD.CA57A818-ON00258146.004FDD28-00258146.005021DC@notes.na.collabserv.com>

An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170621/c0515a24/attachment.html>

From jcbeyler at google.com  Wed Jun 21 20:45:27 2017
From: jcbeyler at google.com (JC Beyler)
Date: Wed, 21 Jun 2017 13:45:27 -0700
Subject: Low-Overhead Heap Profiling
In-Reply-To: <1497366226.2829.109.camel@oracle.com>
References: <CAF9BGBx8xFYudhJ=v1_hU5=nwEcC1X0NKRmC=E-RNN6SMs1ozg@mail.gmail.com>
 <CAF9BGBxj7+C6uy4dXHO-3pNorbpO7n22-9JmevM0F11xOo+MnA@mail.gmail.com>
 <CAF9BGBxrdLVmXRTDHj5hQHJhcyrkBM6+ex1TFWi=A5z=V2FSew@mail.gmail.com>
 <CAF9BGBz81YYVUsMLw9sOMR9fH7JQmyb8d_OZ9aJ47EOAqwWhEw@mail.gmail.com>
 <2af975e6-3827-bd57-0c3d-fadd54867a67@oracle.com>
 <365499b6-3f4d-a4df-9e7e-e72a739fb26b@oracle.com>
 <CAF9BGBy0T7A-+1dejYbr6f0d4s0XR-zv-WwHYqXrbe0EtCdsxQ@mail.gmail.com>
 <fb1155c6-ccf8-843c-34fc-fc4a45fc54cc@oracle.com>
 <CAF9BGBxqmXF9OFD4F9wVb-DQOj_UfNNcBhe+ARVxJ5E4cgCC1Q@mail.gmail.com>
 <102c59b8-25b6-8c21-8eef-1de7d0bbf629@oracle.com>
 <CAF9BGBxDaCOxobJi8Uu9kwKPRpnj-KqgWLfJO2z7UX71UnNF4g@mail.gmail.com>
 <CAF9BGByisZchXTsw1=mLETLnOggJShRJBDxFeOUgt+criTTGuQ@mail.gmail.com>
 <1497366226.2829.109.camel@oracle.com>
Message-ID: <CAF9BGBx8376y_yeMFnMC8PHSqKDaFc5NcbFerQ2up0YVKtSKGg@mail.gmail.com>

Hi all,

First off: Thanks again to Robbin and Thomas for their reviews :)

Next, I've uploaded a new webrev:
http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/

Here is an update:

- @Robbin, I forgot to say that yes I need to look at implementing this for
the other architectures and testing it before it is all ready to go. Is it
common to have it working on all possible combinations or is there a subset
that I should be doing first and we can do the others later?
- I've tested slowdebug, built and ran the JTreg tests I wrote with
slowdebug and fixed a few more issues
- I've refactored a bit of the code following Thomas' comments
   - I think I've handled all the comments from Thomas (I put comments
inline below for the specifics)

The biggest addition to this webrev is that there is more testing, I've
added two tests for looking specifically at the two garbage sampling data
Recent vs Frequent:

http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/test/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorRecentTest.java.patch

   and

http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/test/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorFrequentTest.java.patch

I've also refactored the JNI library a bit:
http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c.patch
.
   - Side question: I was looking at trying to make this a multi-file
library so you would not have all the code in one spot. It seems this is
not really done?
      - Is there a solution to this? What I really wanted was:
        - The core library that will help testing be easier
        - The JNI code for each Java class separate and calling into the
core library

- Following Thomas' comments on statistics, I want to add some quality
assurance tests and find that the easiest way would be to have a few
counters of what is happening in the sampler and expose that to the user.
   - I'll be adding that in the next version if no one sees any objections
to that.
   - This will allow me to add a sanity test in JTreg about number of
samples and average of sampling rate

@Thomas: I had a few questions that I inlined below but I will summarize
the "bigger ones" here:
   - You mentioned constants are not using the right conventions, I looked
around and didn't see any convention except normal naming then for static
constants. Is that right?
   - You mentioned putting the weak_oops_do in a separate method and
logging, I inlined my answer below and would appreciate your feedback there
too.

Thanks again for your reviews and patience!
Jc

PS: I've also inlined my answers to Thomas below:

On Tue, Jun 13, 2017 at 8:03 AM, Thomas Schatzl <thomas.schatzl at oracle.com>
wrote:

> Hi all,
>
> On Mon, 2017-06-12 at 11:11 -0700, JC Beyler wrote:
> > Dear all,
> >
> > I've continued working on this and have done the following webrev:
> > http://cr.openjdk.java.net/~rasbold/8171119/webrev.05/
>
> [...]
> > Things I still need to do:
> >    - Have to fix that TLAB case for the FastTLABRefill
> >    - Have to start looking at the data to see that it is consistent
> > and does gather the right samples, right frequency, etc.
> >    - Have to check the GC elements and what that produces
> >    - Run a slowdebug run and ensure I fixed all those issues you saw
> > Robbin
> >
> > Thanks for looking at the webrev and have a great week!
>
>   scratching a bit on the surface of this change, so apologies for
> rather shallow comments:
>
> - macroAssembler_x86.cpp:5604: while this is compiler code, and I am
> not sure this is final, please avoid littering the code with TODO
> remarks :) They tend to be candidates for later wtf moments only.
>
> Just file a CR for that.
>
>
Newcomer question: what is a CR and not sure I have the rights to do that
yet ? :)


> - calling HeapMonitoring::do_weak_oops() (which should probably be
> called weak_oops_do() like other similar methods) only if string
> deduplication is enabled (in g1CollectedHeap.cpp:4511) seems wrong.


> The call should be at least around 6 lines up outside the if.
>
> Preferentially in a method like process_weak_jni_handles(), including
> additional logging. (No new (G1) gc phase without minimal logging :)).
>

Done but really not sure because:

I put for logging:
  log_develop_trace(gc, freelist)("G1ConcRegionFreeing [other] : heap
monitoring");

Since weak_jni_handles didn't have logging for me to be inspired from, I
did that but unconvinced this is what should be done.


>
> Seeing the changes in referenceProcess.cpp, you need to add the call to
> HeapMonitoring::do_weak_oops() exactly similar to
> process_weak_jni_handles() in case there is no reference processing
> done.
>
> - psParallelCompact.cpp:2172 similar to other places where the change
> adds the call to HeapMonitoring::do_weak_oops(), remove the empty line.
>

Done from what I can tell in the whole webrev. (The only empty lines I
still see are when I maintain an empty line, exception being
http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/src/share/vm/gc/shared/collectedHeap.cpp.patch
where I think it was "nicer")


>
> - the change doubles the size of
> CollectedHeap::allocate_from_tlab_slow() above the "small and nice"
> threshold. Maybe it could be refactored a bit.
>

Done I think, it looks better to me :).


>
> - referenceProcessor.cpp:40: please make sure that the includes are
> sorted alphabetically (I did not check the other files yet).
>

Done and checked all other files normally.


>
> - referenceProcessor.cpp:261: the change should add logging about the
> number of references encountered, maybe after the corresponding "JNI
> weak reference count" log message.
>

Just to double check, are you saying that you'd like to have the heap
sampler to keep in store how many sampled objects were encountered in the
HeapMonitoring::weak_oops_do?
   - Would a return of the method with the number of handled references and
logging that work?
   - Additionally, would you prefer it in a separate block with its
GCTraceTime?


> - threadLocalAllocBuffer.cpp:331: one more "TODO"
>

Removed it and added it to my personal todos to look at.


>
> - threadLocalAllocBuffer.hpp: ThreadLocalAllocBuffer class
> documentation should be updated about the sampling additions. I would
> have no clue what the difference between "actual_end" and "end" would
> be from the given information.
>

If you are talking about the comments in this file, I made them more clear
I hope in the new webrev. If it was somewhere else, let me know where to
change.


>
> - threadLocalAllocBuffer.hpp:130: extra whitespace ;)
>

Fixed :)


>
> - some copyrights :)
>
>
I think I fixed all the issues, if you see specific issues, let me know.


> - in heapMonitoring.hpp: there are some random comments about some code
> that has been grabbed from "util/math/fastmath.[h|cc]". I can't tell
> whether this is code that can be used but I assume that Noam Shazeer is
> okay with that (i.e. that's all Google code).
>

Jeremy and I double checked and we can release that as I thought. I removed
the comment from that piece of code entirely.


>
> - heapMonitoring.hpp/cpp static constant naming does not correspond to
> Hotspot's. Additionally, in Hotspot static methods are cased like other
> methods.
>

I think I fixed the methods to be cased the same way as all other methods.
For static constants, I was not sure. I fixed a few other variables but I
could not seem to really see a consistent trend for constants. I made them
as variables but I'm not sure now.



>
> - in heapMonitoring.cpp there are a few cryptic comments at the top
> that seem to refer to internal stuff that should probably be removed.
>

Sorry about that! My personal todos not cleared out.


>
> I did not think through the impact of the TLAB changes on collector
> behavior yet (if there are). Also I did not check for problems with
> concurrent mark and SATB/G1 (if there are).
>

I would love to know your thoughts on this, I think this is fine. I see
issues with multiple threads right now hitting the stack storage instance.
Previous webrevs had a mutex lock here but we took it out for simplificity
(and only for now).


>
> Thanks,
>   Thomas
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170621/3f8ce0d8/attachment-0001.html>

From serguei.spitsyn at oracle.com  Fri Jun 23 10:01:12 2017
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Fri, 23 Jun 2017 03:01:12 -0700
Subject: Low-Overhead Heap Profiling
In-Reply-To: <CAF9BGBx8376y_yeMFnMC8PHSqKDaFc5NcbFerQ2up0YVKtSKGg@mail.gmail.com>
References: <CAF9BGBx8xFYudhJ=v1_hU5=nwEcC1X0NKRmC=E-RNN6SMs1ozg@mail.gmail.com>
 <CAF9BGBxj7+C6uy4dXHO-3pNorbpO7n22-9JmevM0F11xOo+MnA@mail.gmail.com>
 <CAF9BGBxrdLVmXRTDHj5hQHJhcyrkBM6+ex1TFWi=A5z=V2FSew@mail.gmail.com>
 <CAF9BGBz81YYVUsMLw9sOMR9fH7JQmyb8d_OZ9aJ47EOAqwWhEw@mail.gmail.com>
 <2af975e6-3827-bd57-0c3d-fadd54867a67@oracle.com>
 <365499b6-3f4d-a4df-9e7e-e72a739fb26b@oracle.com>
 <CAF9BGBy0T7A-+1dejYbr6f0d4s0XR-zv-WwHYqXrbe0EtCdsxQ@mail.gmail.com>
 <fb1155c6-ccf8-843c-34fc-fc4a45fc54cc@oracle.com>
 <CAF9BGBxqmXF9OFD4F9wVb-DQOj_UfNNcBhe+ARVxJ5E4cgCC1Q@mail.gmail.com>
 <102c59b8-25b6-8c21-8eef-1de7d0bbf629@oracle.com>
 <CAF9BGBxDaCOxobJi8Uu9kwKPRpnj-KqgWLfJO2z7UX71UnNF4g@mail.gmail.com>
 <CAF9BGByisZchXTsw1=mLETLnOggJShRJBDxFeOUgt+criTTGuQ@mail.gmail.com>
 <1497366226.2829.109.camel@oracle.com>
 <CAF9BGBx8376y_yeMFnMC8PHSqKDaFc5NcbFerQ2up0YVKtSKGg@mail.gmail.com>
Message-ID: <82d08353-4cdb-c771-d0bc-117341f53dcf@oracle.com>

An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170623/d8b5a67d/attachment.html>

From thomas.schatzl at oracle.com  Fri Jun 23 10:52:27 2017
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Fri, 23 Jun 2017 12:52:27 +0200
Subject: Low-Overhead Heap Profiling
In-Reply-To: <CAF9BGBx8376y_yeMFnMC8PHSqKDaFc5NcbFerQ2up0YVKtSKGg@mail.gmail.com>
References: <CAF9BGBx8xFYudhJ=v1_hU5=nwEcC1X0NKRmC=E-RNN6SMs1ozg@mail.gmail.com>
 <CAF9BGBxj7+C6uy4dXHO-3pNorbpO7n22-9JmevM0F11xOo+MnA@mail.gmail.com>
 <CAF9BGBxrdLVmXRTDHj5hQHJhcyrkBM6+ex1TFWi=A5z=V2FSew@mail.gmail.com>
 <CAF9BGBz81YYVUsMLw9sOMR9fH7JQmyb8d_OZ9aJ47EOAqwWhEw@mail.gmail.com>
 <2af975e6-3827-bd57-0c3d-fadd54867a67@oracle.com>
 <365499b6-3f4d-a4df-9e7e-e72a739fb26b@oracle.com>
 <CAF9BGBy0T7A-+1dejYbr6f0d4s0XR-zv-WwHYqXrbe0EtCdsxQ@mail.gmail.com>
 <fb1155c6-ccf8-843c-34fc-fc4a45fc54cc@oracle.com>
 <CAF9BGBxqmXF9OFD4F9wVb-DQOj_UfNNcBhe+ARVxJ5E4cgCC1Q@mail.gmail.com>
 <102c59b8-25b6-8c21-8eef-1de7d0bbf629@oracle.com>
 <CAF9BGBxDaCOxobJi8Uu9kwKPRpnj-KqgWLfJO2z7UX71UnNF4g@mail.gmail.com>
 <CAF9BGByisZchXTsw1=mLETLnOggJShRJBDxFeOUgt+criTTGuQ@mail.gmail.com>
 <1497366226.2829.109.camel@oracle.com>
 <CAF9BGBx8376y_yeMFnMC8PHSqKDaFc5NcbFerQ2up0YVKtSKGg@mail.gmail.com>
Message-ID: <1498215147.2741.34.camel@oracle.com>

Hi,

On Wed, 2017-06-21 at 13:45 -0700, JC Beyler wrote:
> Hi all,
> 
> First off: Thanks again to Robbin and Thomas for their reviews :)
> 
> Next, I've uploaded a new webrev:
> http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/
> 
> Here is an update:
> 
> - @Robbin, I forgot to say that yes I need to look at implementing
> this for the other architectures and testing it before it is all
> ready to go. Is it common to have it working on all possible
> combinations or is there a subset that I should be doing first and we
> can do the others later?
> - I've tested slowdebug, built and ran the JTreg tests I wrote with
> slowdebug and fixed a few more issues
> - I've refactored a bit of the code following Thomas' comments
> ? ?- I think I've handled all the comments from Thomas (I put
> comments inline below for the specifics)

Thanks for handling all those.

> - Following Thomas' comments on statistics, I want to add some
> quality assurance tests and find that the easiest way would be to
> have a few counters of what is happening in the sampler and expose
> that to the user.
> ? ?- I'll be adding that in the next version if no one sees any
> objections to that.
> ? ?- This will allow me to add a sanity test in JTreg about number of
> samples and average of sampling rate
> 
> @Thomas: I had a few questions that I inlined below but I will
> summarize the "bigger ones" here:
> ? ?- You mentioned constants are not using the right conventions, I
> looked around and didn't see any convention except normal naming then
> for static constants. Is that right?

I looked through?https://wiki.openjdk.java.net/display/HotSpot/StyleGui
de?and the rule is to "follow an existing pattern and must have a
distinct appearance from other names". Which does not help a lot I
guess :/ The GC team started using upper camel case, e.g.
SomeOtherConstant, but very likely this is probably not applied
consistently throughout. So I am fine with not adding another style
(like kMaxStackDepth with the "k" in front with some unknown meaning)
is fine.

(Chances are you will find that style somewhere used anyway too,
apologies if so :/)

> PS: I've also inlined my answers to Thomas below:
> 
> On Tue, Jun 13, 2017 at 8:03 AM, Thomas Schatzl <thomas.schatzl at oracl
> e.com> wrote:
> > Hi all,
> > 
> > On Mon, 2017-06-12 at 11:11 -0700, JC Beyler wrote:
> > > Dear all,
> > >
> > > I've continued working on this and have done the following
> > webrev:
> > > http://cr.openjdk.java.net/~rasbold/8171119/webrev.05/
> > 
> > [...]
> > > Things I still need to do:
> > > ? ?- Have to fix that TLAB case for the FastTLABRefill
> > > ? ?- Have to start looking at the data to see that it is
> > consistent and does gather the right samples, right frequency, etc.
> > > ? ?- Have to check the GC elements and what that produces
> > > ? ?- Run a slowdebug run and ensure I fixed all those issues you
> > saw > Robbin
> > >
> > > Thanks for looking at the webrev and have a great week!
> > 
> > ? scratching a bit on the surface of this change, so apologies for
> > rather shallow comments:
> > 
> > - macroAssembler_x86.cpp:5604: while this is compiler code, and I
> > am not sure this is final, please avoid littering the code with
> > TODO remarks :) They tend to be candidates for later wtf moments
> > only.
> > 
> > Just file a CR for that.
> > 
> Newcomer question: what is a CR and not sure I have the rights to do
> that yet ? :)

Apologies. CR is a change request, this suggests to file a bug in the
bug tracker. And you are right, you can't just create a new account in
the OpenJDK JIRA yourselves. :(

I was mostly referring to the "... but it is a TODO" part of that
comment in macroassembler_x86.cpp. Comments about the why of the code
are appreciated.

[Note that I now understand that this is to some degree still work in
progress. As long as the final changeset does no contain TODO's I am
fine (and it's not a hard objection, rather their use in "final" code
is typically limited in my experience)]

5603???// Currently, if this happens, just set back the actual end to?
where it was.
5604???// We miss?a chance to sample here.

Would be okay, if explaining "this" and the "why" of missing a chance
to sample here would be best.

Like maybe:

// If we needed to refill TLABs, just set the actual end point to?
// the end of the TLAB again. We do not sample here although we could.

I am not sure whether "miss a chance to sample" meant "we could, but
consciously don't because it's not that useful" or "it would be
necessary but don't because it's too complicated to do.".

Looking at the original comment once more, I am also not sure if that
comment shouldn't referring to the "end" variable (not actual_end)
because that's the variable that is responsible for taking the sampling
path? (Going from the member description of ThreadLocalAllocBuffer).

But maybe I am only confused and it's best to just leave the comment
away. :)

Thinking about it some more, doesn't this not-sampling in this case
mean that sampling does not work in any collector that does inline TLAB
allocation at the moment? (Or is inline TLAB alloc automatically
disabled with sampling somehow?)

That would indeed be a bigger TODO then :)

> > - calling?HeapMonitoring::do_weak_oops() (which should probably be
> > called weak_oops_do() like other similar methods) only if string
> > deduplication is enabled (in g1CollectedHeap.cpp:4511) seems wrong.
> 
> The call should be at least around 6 lines up outside the if.
> 
> Preferentially in a method like process_weak_jni_handles(), including
> additional logging. (No new (G1) gc phase without minimal logging
> :)).
> Done but really not sure because:
> 
> I put for logging:
> ? log_develop_trace(gc, freelist)("G1ConcRegionFreeing [other] : heap
> monitoring");

I would think that "gc, ref" would be more appropriate log tags for
this similar to jni handles.
(I am als not sure what weak reference handling has to do with
G1ConcRegionFreeing, so I am a bit puzzled)

> Since weak_jni_handles didn't have logging for me to be inspired
> from, I did that but unconvinced this is what should be done.

The JNI handle processing does have logging, but only in
ReferenceProcessor::process_discovered_references(). In
process_weak_jni_handles() only overall time is measured (in a G1
specific way, since only G1 supports disabling reference procesing) :/

The code in ReferenceProcessor prints both time taken
referenceProcessor.cpp:254, as well as the count, but strangely only in
debug VMs.

I have no idea why this logging is that unimportant to only print that
in a debug VM. However there are reviews out for changing this area a
bit, so it might be useful to wait for that (JDK-8173335).

> > - the change doubles the size of
> > CollectedHeap::allocate_from_tlab_slow() above the "small and nice"
> > threshold. Maybe it could be refactored a bit.
> Done I think, it looks better to me :).

In ThreadLocalAllocBuffer::handle_sample() I think the
set_back_actual_end()/pick_next_sample() calls could be hoisted out of
the "if" :)

> > - referenceProcessor.cpp:261: the change should add logging about
> > the number of references encountered, maybe after the corresponding
> > "JNI weak reference count" log message.
> Just to double check, are you saying that you'd like to have the heap
> sampler to keep in store how many sampled objects were encountered in
> the HeapMonitoring::weak_oops_do?
> ? ?- Would a return of the method with the number of handled
> references and logging that work?

Yes, it's fine if HeapMonitoring::weak_oops_do() only returned the
number of processed weak oops.

> ? ?- Additionally, would you prefer it in a separate block with its
> GCTraceTime?

Yes. Both kinds of information is interesting: while the time taken is
typically more important, the next question would be why, and the
number of references typically goes a long way there.

See above though, it is probably best to wait a bit.

> > -?threadLocalAllocBuffer.cpp:331: one more "TODO"
> Removed it and added it to my personal todos to look at.
> ?
> > 
> > - threadLocalAllocBuffer.hpp: ThreadLocalAllocBuffer class
> > documentation should be updated about the sampling additions. I
> > would have no clue what the difference between "actual_end" and
> > "end" would be from the given information.
> If you are talking about the comments in this file, I made them more
> clear I hope in the new webrev. If it was somewhere else, let me know
> where to change.

Thanks, that's much better. Maybe a note in the comment of the class
that ThreadLocalBuffer provides some sampling facility by modifying the
end() of the TLAB to cause "frequent" calls into the runtime call where
actual sampling takes place.

> > - in heapMonitoring.hpp: there are some random comments about some
> > code that has been grabbed from "util/math/fastmath.[h|cc]". I
> > can't tell whether this is code that can be used but I assume that
> > Noam Shazeer is okay with that (i.e. that's all Google code).
> Jeremy and I double checked and we can release that as I thought. I
> removed the comment from that piece of code entirely.

Thanks.

> > - heapMonitoring.hpp/cpp static constant naming does not correspond
> > to Hotspot's. Additionally, in Hotspot static methods are cased
> > like other methods.
> I think I fixed the methods to be cased the same way as all other
> methods. For static constants, I was not sure. I fixed a few other
> variables but I could not seem to really see a consistent trend for
> constants. I made them as variables but I'm not sure now.

Sorry again, style is a kind of mess. The goal of my suggestions here
is only to prevent yet another style creeping in.
?
> > - in heapMonitoring.cpp there are a few cryptic comments at the top
> > that seem to refer to internal stuff that should probably be
> > removed.
> Sorry about that! My personal todos not cleared out.

I am happy about comments, but I simply did not understand any of that
and I do not know about other readers as well.

If you think you will remember removing/updating them until the review
proper (I misunderstood the review situation a little it seems).

> > I did not think through the impact of the TLAB changes on collector
> > behavior yet (if there are). Also I did not check for problems with
> > concurrent mark and SATB/G1 (if there are).
> I would love to know your thoughts on this, I think this is fine. I 

I think so too now. No objects are made live out of thin air :)

> see issues with multiple threads right now hitting the stack storage
> instance. Previous webrevs had a mutex lock here but we took it out
> for simplificity (and only for now).

:) When looking at this after some thinking I now assume for this
review that this code is not MT safe at all. There seems to be more
synchronization missing than just the one for the StackTraceStorage. So
no comments about this here.

Also, this would require some kind of specification of what is allowed
to be called when and where.

One potentially relevant observation about locking here: depending on
sampling frequency, StackTraceStore::add_trace() may be rather
frequently called. I assume that you are going to do measurements :)

I am not sure what the expected usage of the API is, but
StackTraceStore::add_trace() seems to be able to grow without bounds.
Only a GC truncates them to the live ones. That in itself seems to be
problematic (GCs can be *wide* apart), and of course some of the API
methods add to that because they duplicate that unbounded array. Do you
have any concerns/measurements about this?

Also, these stack traces might hold on to huge arrays. Any
consideration of that? Particularly it might be the cause for OOMEs in
tight memory situations.

- please consider adding a safepoint check in
HeapMonitoring::weak_oops_do to prevent accidental misuse.

- in struct StackTraceStorage, the public fields may also need
underscores. At least some files in the runtime directory have structs
with underscored public members (and some don't). The runtime team
should probably comment on that.

- In?StackTraceStorage::weak_oops_do(), when examining the
StackTraceData, maybe it is useful to consider having a non-NULL
reference outside of the heap's reserved space an error. There should
be no oop outside of the heap's reserved space ever.

Unless you allow storing random values in?StackTraceData::obj, which I
would not encourage.

-?HeapMonitoring::weak_oops_do() does not seem to use the
passed?AbstractRefProcTaskExecutor.

- I do not understand allowing to call this method with a NULL
complete_gc closure. This would mean that objects referenced from the
object that is referenced by the StackTraceData are not pulled, meaning
they would get stale.

- same with is_alive parameter value of NULL

- heapMonitoring.cpp:590: I do not completely understand the purpose of
this code: in the end this results in a fixed value directly dependent
on the Thread address anyway? In the end this results in a fixed value
directly dependent on the Thread address anyway?
IOW, what is special about exactly 20 rounds?

- also I would consider stripping a few bits of the threads' address as
initialization value for your rng. The last three bits (and probably
more, check whether the Thread object is allocated on special
boundaries) are always zero for them.
Not sure if the given "random" value is random enough before/after,
this method, so just skip that comment if you think this is not
required.

Some more random nits I did not find a place to put anywhere:

- ThreadLocalAllocBuffer::_extra_space does not seem to be used
anywhere?

- Maybe indent the declaration of ThreadLocalAllocBuffer::_bytes_until_sample to align below the other members of that group.

Thanks,
? Thomas


From jcbeyler at google.com  Fri Jun 23 16:58:48 2017
From: jcbeyler at google.com (JC Beyler)
Date: Fri, 23 Jun 2017 09:58:48 -0700
Subject: Low-Overhead Heap Profiling
In-Reply-To: <82d08353-4cdb-c771-d0bc-117341f53dcf@oracle.com>
References: <CAF9BGBx8xFYudhJ=v1_hU5=nwEcC1X0NKRmC=E-RNN6SMs1ozg@mail.gmail.com>
 <CAF9BGBxj7+C6uy4dXHO-3pNorbpO7n22-9JmevM0F11xOo+MnA@mail.gmail.com>
 <CAF9BGBxrdLVmXRTDHj5hQHJhcyrkBM6+ex1TFWi=A5z=V2FSew@mail.gmail.com>
 <CAF9BGBz81YYVUsMLw9sOMR9fH7JQmyb8d_OZ9aJ47EOAqwWhEw@mail.gmail.com>
 <2af975e6-3827-bd57-0c3d-fadd54867a67@oracle.com>
 <365499b6-3f4d-a4df-9e7e-e72a739fb26b@oracle.com>
 <CAF9BGBy0T7A-+1dejYbr6f0d4s0XR-zv-WwHYqXrbe0EtCdsxQ@mail.gmail.com>
 <fb1155c6-ccf8-843c-34fc-fc4a45fc54cc@oracle.com>
 <CAF9BGBxqmXF9OFD4F9wVb-DQOj_UfNNcBhe+ARVxJ5E4cgCC1Q@mail.gmail.com>
 <102c59b8-25b6-8c21-8eef-1de7d0bbf629@oracle.com>
 <CAF9BGBxDaCOxobJi8Uu9kwKPRpnj-KqgWLfJO2z7UX71UnNF4g@mail.gmail.com>
 <CAF9BGByisZchXTsw1=mLETLnOggJShRJBDxFeOUgt+criTTGuQ@mail.gmail.com>
 <1497366226.2829.109.camel@oracle.com>
 <CAF9BGBx8376y_yeMFnMC8PHSqKDaFc5NcbFerQ2up0YVKtSKGg@mail.gmail.com>
 <82d08353-4cdb-c771-d0bc-117341f53dcf@oracle.com>
Message-ID: <CAF9BGBzZCjZedDKfFwX9DmqsQ04_hxsHwZ6UpB1i_r1F-=rT-A@mail.gmail.com>

Hi Serguei and all,

Thanks Serguei for your review. I believe I have not answered many
questions but raised more but I hope this helps understand what I am trying
to do.

I have inlined my answers to make it easier to answer everything (and to be
honest, ask questions).

@Thomas: Thank you for your thorough answer as well, I'll be working on it
and will answer when I get the fixes and comments in a handled state, most
likely beginning of next week :)

On Fri, Jun 23, 2017 at 3:01 AM, serguei.spitsyn at oracle.com <
serguei.spitsyn at oracle.com> wrote:

> Hi JC,
>
> Some initial JVMTI-specific questions/comments.
>
> I was not able to apply your patch as the are many merge conflicts.
> Could you, please, re-base it on the latest JDK 10 sources?
>

This is weird, I did a pull right now and it worked. I hesitated I was
using hg correctly so I just did:
hg clone http://hg.openjdk.java.net/jdk10/hs jdk10hs
bash ./get_source.sh
wget http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/hotspot.patch
patch -p1 < hotspot.patch

(Maybe the patch is not the right way of doing things, there might be a hg
equivalent?).

And it seemed to work:
bash ./configure --with-boot-jdk=/path/to/jdk1.8.0-latest
--with-debug-level=slowdebug
make all images JOBS=48

Any idea why it is working for me and not you? I am assuming I am doing
something wrong but I don't know what!


>
> I think, it would make sense to introduce new optional capability,
> something like:
>    can_sample_heap_allocations
>

I will work on adding that for the next webrev, Robbin had mentioned I
should and I dropped the ball on that one.


>
> Do you consider this API to be used by a single agent or it is a
> multi-agent feature?
> What if one agent calls the StartHeapSampling() but another calls one of
> the others.
> The API specs needs to clarify this.
>
> I'm not sure you had a chance to carefully think on what JVMTI phases are
> allowed for new functions.
>

- I am pretty sure I have NOT really thought about this as carefully as you
have because this is the first time I am doing this and these questions
already make me go "hmmm" :)

I have a tendency to work through issues and try to keep the design simple
first. Your question seems to imply I can make this work for a single agent
world and have a means to explicitly disable it for multi-agent for now.

I looked quickly in the prims folder for anything mentioning single agent
vs multi agent but did not find anything. Does this get handled s this more
a documentation thing and tough luck for the user.

This page:
https://docs.oracle.com/javase/7/docs/platform/jvmti/jvmti.html#environments

seems to say the latter. That the documentation should say that this is not
a multiple agent support (yet?) and there is nothing we can do. Is that
right?



>
>
> http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/src/
> share/vm/runtime/heapMonitoring.cpp.html
>
> I do not see the variable HeapMonitoring::_enabled is checked in the
> functions:
>     HeapMonitoring::get_live_traces(),
>     HeapMonitoring::get_garbage_traces(),
>     HeapMonitoring::get_frequent_garbage_traces(),
>     HeapMonitoring::release_traces()
>
>
So it is not checked there because the API allows you to:
  - Start sampling (which flips the enabled variable)
  - Stop sampling (which flips back the enabled variable)
  - Get the sampled traces at that point

I think the API documentation is not clear for this right now, so I'll
change it for the next webrev. Let me also know if you think this is a bad
idea. I thought it would be useful because it would allow you to start/stop
the sampling and then later on go back and see what was sampled at a later
time.


> A question about a multi-agent feature from above applies here as well.
>

I'm open to either supporting multi-agent if you think it is important or
doing whatever can be done to, as a first step, make it a single agent
support. Let me know how to do that.


>
>
> Thanks,
> Serguei
>

Thank you!
Jc


>
>
>
>
> On 6/21/17 13:45, JC Beyler wrote:
>
> Hi all,
>
> First off: Thanks again to Robbin and Thomas for their reviews :)
>
> Next, I've uploaded a new webrev:
> http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/
>
> Here is an update:
>
> - @Robbin, I forgot to say that yes I need to look at implementing this
> for the other architectures and testing it before it is all ready to go. Is
> it common to have it working on all possible combinations or is there a
> subset that I should be doing first and we can do the others later?
> - I've tested slowdebug, built and ran the JTreg tests I wrote with
> slowdebug and fixed a few more issues
> - I've refactored a bit of the code following Thomas' comments
>    - I think I've handled all the comments from Thomas (I put comments
> inline below for the specifics)
>
> The biggest addition to this webrev is that there is more testing, I've
> added two tests for looking specifically at the two garbage sampling data
> Recent vs Frequent:
>    http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/
> test/serviceability/jvmti/HeapMonitor/MyPackage/
> HeapMonitorRecentTest.java.patch
>    and
>    http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/
> test/serviceability/jvmti/HeapMonitor/MyPackage/
> HeapMonitorFrequentTest.java.patch
>
> I've also refactored the JNI library a bit: http://cr.openjdk.java.net/~
> rasbold/8171119/webrev.06/test/serviceability/jvmti/
> HeapMonitor/libHeapMonitor.c.patch.
>    - Side question: I was looking at trying to make this a multi-file
> library so you would not have all the code in one spot. It seems this is
> not really done?
>       - Is there a solution to this? What I really wanted was:
>         - The core library that will help testing be easier
>         - The JNI code for each Java class separate and calling into the
> core library
>
> - Following Thomas' comments on statistics, I want to add some quality
> assurance tests and find that the easiest way would be to have a few
> counters of what is happening in the sampler and expose that to the user.
>    - I'll be adding that in the next version if no one sees any objections
> to that.
>    - This will allow me to add a sanity test in JTreg about number of
> samples and average of sampling rate
>
> @Thomas: I had a few questions that I inlined below but I will summarize
> the "bigger ones" here:
>    - You mentioned constants are not using the right conventions, I looked
> around and didn't see any convention except normal naming then for static
> constants. Is that right?
>    - You mentioned putting the weak_oops_do in a separate method and
> logging, I inlined my answer below and would appreciate your feedback there
> too.
>
> Thanks again for your reviews and patience!
> Jc
>
> PS: I've also inlined my answers to Thomas below:
>
> On Tue, Jun 13, 2017 at 8:03 AM, Thomas Schatzl <thomas.schatzl at oracle.com
> > wrote:
>
>> Hi all,
>>
>> On Mon, 2017-06-12 at 11:11 -0700, JC Beyler wrote:
>> > Dear all,
>> >
>> > I've continued working on this and have done the following webrev:
>> > http://cr.openjdk.java.net/~rasbold/8171119/webrev.05/
>>
>> [...]
>> > Things I still need to do:
>> >    - Have to fix that TLAB case for the FastTLABRefill
>> >    - Have to start looking at the data to see that it is consistent
>> > and does gather the right samples, right frequency, etc.
>> >    - Have to check the GC elements and what that produces
>> >    - Run a slowdebug run and ensure I fixed all those issues you saw
>> > Robbin
>> >
>> > Thanks for looking at the webrev and have a great week!
>>
>>   scratching a bit on the surface of this change, so apologies for
>> rather shallow comments:
>>
>> - macroAssembler_x86.cpp:5604: while this is compiler code, and I am
>> not sure this is final, please avoid littering the code with TODO
>> remarks :) They tend to be candidates for later wtf moments only.
>>
>> Just file a CR for that.
>>
>>
> Newcomer question: what is a CR and not sure I have the rights to do that
> yet ? :)
>
>
>> - calling HeapMonitoring::do_weak_oops() (which should probably be
>> called weak_oops_do() like other similar methods) only if string
>> deduplication is enabled (in g1CollectedHeap.cpp:4511) seems wrong.
>
>
>> The call should be at least around 6 lines up outside the if.
>>
>> Preferentially in a method like process_weak_jni_handles(), including
>> additional logging. (No new (G1) gc phase without minimal logging :)).
>>
>
> Done but really not sure because:
>
> I put for logging:
>   log_develop_trace(gc, freelist)("G1ConcRegionFreeing [other] : heap
> monitoring");
>
> Since weak_jni_handles didn't have logging for me to be inspired from, I
> did that but unconvinced this is what should be done.
>
>
>>
>> Seeing the changes in referenceProcess.cpp, you need to add the call to
>> HeapMonitoring::do_weak_oops() exactly similar to
>> process_weak_jni_handles() in case there is no reference processing
>> done.
>>
>> - psParallelCompact.cpp:2172 similar to other places where the change
>> adds the call to HeapMonitoring::do_weak_oops(), remove the empty line.
>>
>
> Done from what I can tell in the whole webrev. (The only empty lines I
> still see are when I maintain an empty line, exception being
> http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/src/
> share/vm/gc/shared/collectedHeap.cpp.patch where I think it was "nicer")
>
>
>>
>> - the change doubles the size of
>> CollectedHeap::allocate_from_tlab_slow() above the "small and nice"
>> threshold. Maybe it could be refactored a bit.
>>
>
> Done I think, it looks better to me :).
>
>
>>
>> - referenceProcessor.cpp:40: please make sure that the includes are
>> sorted alphabetically (I did not check the other files yet).
>>
>
> Done and checked all other files normally.
>
>
>>
>> - referenceProcessor.cpp:261: the change should add logging about the
>> number of references encountered, maybe after the corresponding "JNI
>> weak reference count" log message.
>>
>
> Just to double check, are you saying that you'd like to have the heap
> sampler to keep in store how many sampled objects were encountered in the
> HeapMonitoring::weak_oops_do?
>    - Would a return of the method with the number of handled references
> and logging that work?
>    - Additionally, would you prefer it in a separate block with its
> GCTraceTime?
>
>
>> - threadLocalAllocBuffer.cpp:331: one more "TODO"
>>
>
> Removed it and added it to my personal todos to look at.
>
>
>>
>> - threadLocalAllocBuffer.hpp: ThreadLocalAllocBuffer class
>> documentation should be updated about the sampling additions. I would
>> have no clue what the difference between "actual_end" and "end" would
>> be from the given information.
>>
>
> If you are talking about the comments in this file, I made them more clear
> I hope in the new webrev. If it was somewhere else, let me know where to
> change.
>
>
>>
>> - threadLocalAllocBuffer.hpp:130: extra whitespace ;)
>>
>
> Fixed :)
>
>
>>
>> - some copyrights :)
>>
>>
> I think I fixed all the issues, if you see specific issues, let me know.
>
>
>> - in heapMonitoring.hpp: there are some random comments about some code
>> that has been grabbed from "util/math/fastmath.[h|cc]". I can't tell
>> whether this is code that can be used but I assume that Noam Shazeer is
>> okay with that (i.e. that's all Google code).
>>
>
> Jeremy and I double checked and we can release that as I thought. I
> removed the comment from that piece of code entirely.
>
>
>>
>> - heapMonitoring.hpp/cpp static constant naming does not correspond to
>> Hotspot's. Additionally, in Hotspot static methods are cased like other
>> methods.
>>
>
> I think I fixed the methods to be cased the same way as all other methods.
> For static constants, I was not sure. I fixed a few other variables but I
> could not seem to really see a consistent trend for constants. I made them
> as variables but I'm not sure now.
>
>
>
>>
>> - in heapMonitoring.cpp there are a few cryptic comments at the top
>> that seem to refer to internal stuff that should probably be removed.
>>
>
> Sorry about that! My personal todos not cleared out.
>
>
>>
>> I did not think through the impact of the TLAB changes on collector
>> behavior yet (if there are). Also I did not check for problems with
>> concurrent mark and SATB/G1 (if there are).
>>
>
> I would love to know your thoughts on this, I think this is fine. I see
> issues with multiple threads right now hitting the stack storage instance.
> Previous webrevs had a mutex lock here but we took it out for simplificity
> (and only for now).
>
>
>>
>> Thanks,
>>   Thomas
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170623/0279b1c7/attachment-0001.html>

From jeremymanson at google.com  Fri Jun 23 23:08:44 2017
From: jeremymanson at google.com (Jeremy Manson)
Date: Fri, 23 Jun 2017 16:08:44 -0700
Subject: Low-Overhead Heap Profiling
In-Reply-To: <1498215147.2741.34.camel@oracle.com>
References: <CAF9BGBx8xFYudhJ=v1_hU5=nwEcC1X0NKRmC=E-RNN6SMs1ozg@mail.gmail.com>
 <CAF9BGBxj7+C6uy4dXHO-3pNorbpO7n22-9JmevM0F11xOo+MnA@mail.gmail.com>
 <CAF9BGBxrdLVmXRTDHj5hQHJhcyrkBM6+ex1TFWi=A5z=V2FSew@mail.gmail.com>
 <CAF9BGBz81YYVUsMLw9sOMR9fH7JQmyb8d_OZ9aJ47EOAqwWhEw@mail.gmail.com>
 <2af975e6-3827-bd57-0c3d-fadd54867a67@oracle.com>
 <365499b6-3f4d-a4df-9e7e-e72a739fb26b@oracle.com>
 <CAF9BGBy0T7A-+1dejYbr6f0d4s0XR-zv-WwHYqXrbe0EtCdsxQ@mail.gmail.com>
 <fb1155c6-ccf8-843c-34fc-fc4a45fc54cc@oracle.com>
 <CAF9BGBxqmXF9OFD4F9wVb-DQOj_UfNNcBhe+ARVxJ5E4cgCC1Q@mail.gmail.com>
 <102c59b8-25b6-8c21-8eef-1de7d0bbf629@oracle.com>
 <CAF9BGBxDaCOxobJi8Uu9kwKPRpnj-KqgWLfJO2z7UX71UnNF4g@mail.gmail.com>
 <CAF9BGByisZchXTsw1=mLETLnOggJShRJBDxFeOUgt+criTTGuQ@mail.gmail.com>
 <1497366226.2829.109.camel@oracle.com>
 <CAF9BGBx8376y_yeMFnMC8PHSqKDaFc5NcbFerQ2up0YVKtSKGg@mail.gmail.com>
 <1498215147.2741.34.camel@oracle.com>
Message-ID: <CAPYFHW3-MqOpo7nr6GRMuaGVWy8ULytb2=nVnao8rvyWeU_nwA@mail.gmail.com>

On Fri, Jun 23, 2017 at 3:52 AM, Thomas Schatzl <thomas.schatzl at oracle.com>
wrote:

>
> I looked through https://wiki.openjdk.java.net/display/HotSpot/StyleGui
> de and the rule is to "follow an existing pattern and must have a
> distinct appearance from other names". Which does not help a lot I
> guess :/ The GC team started using upper camel case, e.g.
> SomeOtherConstant, but very likely this is probably not applied
> consistently throughout. So I am fine with not adding another style
> (like kMaxStackDepth with the "k" in front with some unknown meaning)
> is fine.
>


In case you're curious: the k- prefix for constants is found in Hungarian
notation.  It's used as part of Google C++ style to indicate global (for
some definition) variables declared constexpr or const.  See
https://google.github.io/styleguide/cppguide.html#Constant_Names.

Generally, in our patches, we've ping-ponged on use of Hotspot style versus
Google style.  We tend to use Hotspot if it is obvious to us what that is,
but Google otherwise.

Obviously, it should match Hotspot style if it is being submitted, assuming
we can find out what Hotspot style is. :)

Jeremy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170623/8e95a33c/attachment.html>

From serguei.spitsyn at oracle.com  Sat Jun 24 04:06:21 2017
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Fri, 23 Jun 2017 21:06:21 -0700
Subject: [Urgent JDK 9] RFR: 8182844: Fix broken links in the generated
 jvmti.html
Message-ID: <bdee3e7d-75f4-9aa9-723b-fe9a17e5a902@oracle.com>

An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170623/4302992b/attachment.html>

From christoph.langer at sap.com  Sat Jun 24 05:36:51 2017
From: christoph.langer at sap.com (Langer, Christoph)
Date: Sat, 24 Jun 2017 05:36:51 +0000
Subject: [Urgent JDK 9] RFR: 8182844: Fix broken links in the generated
 jvmti.html
In-Reply-To: <bdee3e7d-75f4-9aa9-723b-fe9a17e5a902@oracle.com>
References: <bdee3e7d-75f4-9aa9-723b-fe9a17e5a902@oracle.com>
Message-ID: <aa3731554fde4e9b973675c14e9749a0@sap.com>

Hi Serguei,

the change looks fine and fixes the URLs.

Best regards
Christoph

From: serviceability-dev [mailto:serviceability-dev-bounces at openjdk.java.net] On Behalf Of serguei.spitsyn at oracle.com
Sent: Samstag, 24. Juni 2017 06:06
To: serviceability-dev at openjdk.java.net; Mandy Chung <Mandy.Chung at oracle.com>; Alan Bateman <Alan.Bateman at oracle.com>
Subject: [Urgent JDK 9] RFR: 8182844: Fix broken links in the generated jvmti.html

Please, review this urgent JDK 9 review request for the bug:
  https://bugs.openjdk.java.net/browse/JDK-8182844

Webrev:
  http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8182844-jvmti-spec-links/


Summary:

  The broken links are:
        <a href="http://docs.oracle.com/javase/9/docs/technotes/guides/jpda/architecture.html">Java
        <a href="http://docs.oracle.com/javase/9/jni/invocation.html#getenv">
        <a href="http://docs.oracle.com/javase/9/jni/types.html#modified-utf-8-strings">
        <a href="http://docs.oracle.com/javase/9/jni/design.html">Java
        <a href="http://docs.oracle.com/javase/9/jni/functions.html#local-references">
          <a href="http://docs.oracle.com/javase/9/jni/design.html#java-exceptions">Java Exceptions</a>
      <a href="http://docs.oracle.com/javase/9/jni/invocation.html#attaching-to-the-vm">Attaching to the VM</a>.
            <a href="http://docs.oracle.com/javase/9/jni/types.html#type-signatures">JNI
            <a href="http://docs.oracle.com/javase/9/jni/index.html">JNI Specification</a>
            <a href="http://docs.oracle.com/javase/9/jni/index.html">JNI Specification</a>
          See <a href="http://docs.oracle.com/javase/9/jni/functions.html">JNI
      path to a <a href="http://docs.oracle.com/javase/9/docs/technotes/guides/jar/jar.html">
     <a href="http://docs.oracle.com/javase/9/docs/technotes/guides/jar/jar.html">JAR file</a> to be
     <a href="http://docs.oracle.com/javase/9/jni/functions.html#interface-function-table">

  There are two issues:
    - 3 links have the incorrect part "docs/technotes/guides" that now has to be "docs/specs".
    - the jvmti.xsl injects the obsolete prefix to every link:
        "http://docs.oracle.com/javase/9/"<http://docs.oracle.com/javase/9/>

  The fix is to replace the "docs/technotes/guides" with "docs/specs"
  and skip injecting the incorrect prefix "http://docs.oracle.com/javase/9/"<http://docs.oracle.com/javase/9/>.
  So that all links become relative to the location of the jvmti.html.


  Also, it can be too late to push this at this time.
  Sorry that I've discovered this so late.
  One question is if the priority of this bug has to be raised to P1,
  or maybe this bug needs to be deferred to 9u.


Thanks,
Serguei
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170624/2e2b7950/attachment-0001.html>

From serguei.spitsyn at oracle.com  Sat Jun 24 07:33:27 2017
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Sat, 24 Jun 2017 00:33:27 -0700
Subject: [Urgent JDK 9] RFR: 8182844: Fix broken links in the generated
 jvmti.html
In-Reply-To: <aa3731554fde4e9b973675c14e9749a0@sap.com>
References: <bdee3e7d-75f4-9aa9-723b-fe9a17e5a902@oracle.com>
 <aa3731554fde4e9b973675c14e9749a0@sap.com>
Message-ID: <22b99d9a-f606-97e2-7db1-2d82e4dfcf2b@oracle.com>

An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170624/5104229d/attachment.html>

From daniel.daugherty at oracle.com  Sat Jun 24 13:40:04 2017
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Sat, 24 Jun 2017 07:40:04 -0600
Subject: [Urgent JDK 9] RFR: 8182844: Fix broken links in the generated
 jvmti.html
In-Reply-To: <bdee3e7d-75f4-9aa9-723b-fe9a17e5a902@oracle.com>
References: <bdee3e7d-75f4-9aa9-723b-fe9a17e5a902@oracle.com>
Message-ID: <fd5891a1-db0b-8629-6dbb-2bbc542ff331@oracle.com>

On 6/23/17 10:06 PM, serguei.spitsyn at oracle.com wrote:
> Please, review this urgent JDK 9 review request for the bug:
> https://bugs.openjdk.java.net/browse/JDK-8182844
>
> Webrev:
> http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8182844-jvmti-spec-links/

src/share/vm/prims/jvmti.xml
     No comments.

src/share/vm/prims/jvmti.xsl
     No comments.

Thumbs up!


If you want this fixed in JDK9, the priority has to be raised to P1.
I don't think JDK9 should go out the door with broken doc links.
That gives a bad impression so I would raise this to a P1 and
start the process of requesting approval for the push.

Dan



>
>
> Summary:
>
>   The broken links are:
>         <a 
> href="http://docs.oracle.com/javase/9/docs/technotes/guides/jpda/architecture.html">Java 
>
>         <a 
> href="http://docs.oracle.com/javase/9/jni/invocation.html#getenv">
>         <a 
> href="http://docs.oracle.com/javase/9/jni/types.html#modified-utf-8-strings"> 
>
>         <a href="http://docs.oracle.com/javase/9/jni/design.html">Java
>         <a 
> href="http://docs.oracle.com/javase/9/jni/functions.html#local-references"> 
>
>           <a 
> href="http://docs.oracle.com/javase/9/jni/design.html#java-exceptions">Java 
> Exceptions</a>
>       <a 
> href="http://docs.oracle.com/javase/9/jni/invocation.html#attaching-to-the-vm">Attaching 
> to the VM</a>.
>             <a 
> href="http://docs.oracle.com/javase/9/jni/types.html#type-signatures">JNI
>             <a 
> href="http://docs.oracle.com/javase/9/jni/index.html">JNI 
> Specification</a>
>             <a 
> href="http://docs.oracle.com/javase/9/jni/index.html">JNI 
> Specification</a>
>           See <a 
> href="http://docs.oracle.com/javase/9/jni/functions.html">JNI
>       path to a <a 
> href="http://docs.oracle.com/javase/9/docs/technotes/guides/jar/jar.html"> 
>
>      <a 
> href="http://docs.oracle.com/javase/9/docs/technotes/guides/jar/jar.html">JAR 
> file</a> to be
>      <a 
> href="http://docs.oracle.com/javase/9/jni/functions.html#interface-function-table"> 
>
>
>   There are two issues:
>     - 3 links have the incorrect part "docs/technotes/guides" that now 
> has to be "docs/specs".
>     - the jvmti.xsl injects the obsolete prefix to every link:
> "http://docs.oracle.com/javase/9/"
>
>   The fix is to replace the "docs/technotes/guides" with "docs/specs"
>   and skip injecting the incorrect prefix 
> "http://docs.oracle.com/javase/9/".
>   So that all links become relative to the location of the jvmti.html.
>
>
>   Also, it can be too late to push this at this time.
>   Sorry that I've discovered this so late.
>   One question is if the priority of this bug has to be raised to P1,
>   or maybe this bug needs to be deferred to 9u.
>
>
> Thanks,
> Serguei

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170624/411f1918/attachment.html>

From serguei.spitsyn at oracle.com  Sat Jun 24 17:26:34 2017
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Sat, 24 Jun 2017 10:26:34 -0700
Subject: [Urgent JDK 9] RFR: 8182844: Fix broken links in the generated
 jvmti.html
In-Reply-To: <fd5891a1-db0b-8629-6dbb-2bbc542ff331@oracle.com>
References: <bdee3e7d-75f4-9aa9-723b-fe9a17e5a902@oracle.com>
 <fd5891a1-db0b-8629-6dbb-2bbc542ff331@oracle.com>
Message-ID: <638ab41e-37d4-b00d-6114-46924e7294a9@oracle.com>

An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170624/0b79b978/attachment.html>

From serguei.spitsyn at oracle.com  Sat Jun 24 17:27:55 2017
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Sat, 24 Jun 2017 10:27:55 -0700
Subject: [Urgent JDK 9] RFR: 8182844: Fix broken links in the generated
 jvmti.html
In-Reply-To: <638ab41e-37d4-b00d-6114-46924e7294a9@oracle.com>
References: <bdee3e7d-75f4-9aa9-723b-fe9a17e5a902@oracle.com>
 <fd5891a1-db0b-8629-6dbb-2bbc542ff331@oracle.com>
 <638ab41e-37d4-b00d-6114-46924e7294a9@oracle.com>
Message-ID: <da1f4477-425b-8444-74e2-dc189f830f72@oracle.com>

An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170624/d087ce20/attachment-0001.html>

From daniel.daugherty at oracle.com  Sat Jun 24 19:44:16 2017
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Sat, 24 Jun 2017 13:44:16 -0600
Subject: [Urgent JDK 9] RFR: 8182844: Fix broken links in the generated
 jvmti.html
In-Reply-To: <da1f4477-425b-8444-74e2-dc189f830f72@oracle.com>
References: <bdee3e7d-75f4-9aa9-723b-fe9a17e5a902@oracle.com>
 <fd5891a1-db0b-8629-6dbb-2bbc542ff331@oracle.com>
 <638ab41e-37d4-b00d-6114-46924e7294a9@oracle.com>
 <da1f4477-425b-8444-74e2-dc189f830f72@oracle.com>
Message-ID: <297a81a3-71c4-fe72-60a5-f228872cad00@oracle.com>

On 6/24/17 11:27 AM, serguei.spitsyn at oracle.com wrote:
> On 6/24/17 10:26, serguei.spitsyn at oracle.com wrote:
>> On 6/24/17 06:40, Daniel D. Daugherty wrote:
>>> On 6/23/17 10:06 PM, serguei.spitsyn at oracle.com wrote:
>>>> Please, review this urgent JDK 9 review request for the bug:
>>>> https://bugs.openjdk.java.net/browse/JDK-8182844
>>>>
>>>> Webrev:
>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8182844-jvmti-spec-links/
>>>
>>> src/share/vm/prims/jvmti.xml
>>>     No comments.
>>>
>>> src/share/vm/prims/jvmti.xsl
>>>     No comments.
>>>
>>> Thumbs up!
>> Thanks a lot, Dam!
>
> Sorry for a typo.
> I wanted to say "Dan".

No worries! Trust me, I'm used to people saying "That damn Dan..."

Here's a snippet from Mark's internal e-mail:

 > We should only fix P1 bugs from this point onward, and all bugs must go
 > through the fix-request process -- even if they are "noreg-doc".

Don't know why the OpenJDK9 e-mail wasn't as clear, but...

Dan



>
> Thanks,
> Serguei
>
>>
>>>
>>>
>>> If you want this fixed in JDK9, the priority has to be raised to P1.
>>> I don't think JDK9 should go out the door with broken doc links.
>>> That gives a bad impression so I would raise this to a P1 and
>>> start the process of requesting approval for the push.
>>
>> I've made priority P1.
>> The following P1 doc bug that is in the jdk 9 queue does not have 
>> jdk9-fix-request.
>> Do we need it for doc bugs?
>>
>> Thanks,
>> Serguei
>>
>>
>>>
>>> Dan
>>>
>>>
>>>
>>>>
>>>>
>>>> Summary:
>>>>
>>>>   The broken links are:
>>>>         <a 
>>>> href="http://docs.oracle.com/javase/9/docs/technotes/guides/jpda/architecture.html">Java 
>>>>
>>>>         <a 
>>>> href="http://docs.oracle.com/javase/9/jni/invocation.html#getenv">
>>>>         <a 
>>>> href="http://docs.oracle.com/javase/9/jni/types.html#modified-utf-8-strings"> 
>>>>
>>>>         <a href="http://docs.oracle.com/javase/9/jni/design.html">Java
>>>>         <a 
>>>> href="http://docs.oracle.com/javase/9/jni/functions.html#local-references"> 
>>>>
>>>>           <a 
>>>> href="http://docs.oracle.com/javase/9/jni/design.html#java-exceptions">Java 
>>>> Exceptions</a>
>>>>       <a 
>>>> href="http://docs.oracle.com/javase/9/jni/invocation.html#attaching-to-the-vm">Attaching 
>>>> to the VM</a>.
>>>>             <a 
>>>> href="http://docs.oracle.com/javase/9/jni/types.html#type-signatures">JNI 
>>>>
>>>>             <a 
>>>> href="http://docs.oracle.com/javase/9/jni/index.html">JNI 
>>>> Specification</a>
>>>>             <a 
>>>> href="http://docs.oracle.com/javase/9/jni/index.html">JNI 
>>>> Specification</a>
>>>>           See <a 
>>>> href="http://docs.oracle.com/javase/9/jni/functions.html">JNI
>>>>       path to a <a 
>>>> href="http://docs.oracle.com/javase/9/docs/technotes/guides/jar/jar.html"> 
>>>>
>>>>      <a 
>>>> href="http://docs.oracle.com/javase/9/docs/technotes/guides/jar/jar.html">JAR 
>>>> file</a> to be
>>>>      <a 
>>>> href="http://docs.oracle.com/javase/9/jni/functions.html#interface-function-table"> 
>>>>
>>>>
>>>>   There are two issues:
>>>>     - 3 links have the incorrect part "docs/technotes/guides" that 
>>>> now has to be "docs/specs".
>>>>     - the jvmti.xsl injects the obsolete prefix to every link:
>>>> "http://docs.oracle.com/javase/9/"
>>>>
>>>>   The fix is to replace the "docs/technotes/guides" with "docs/specs"
>>>>   and skip injecting the incorrect prefix 
>>>> "http://docs.oracle.com/javase/9/".
>>>>   So that all links become relative to the location of the jvmti.html.
>>>>
>>>>
>>>>   Also, it can be too late to push this at this time.
>>>>   Sorry that I've discovered this so late.
>>>>   One question is if the priority of this bug has to be raised to P1,
>>>>   or maybe this bug needs to be deferred to 9u.
>>>>
>>>>
>>>> Thanks,
>>>> Serguei
>>>
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170624/b418d125/attachment.html>

From serguei.spitsyn at oracle.com  Sat Jun 24 20:09:58 2017
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Sat, 24 Jun 2017 13:09:58 -0700
Subject: [Urgent JDK 9] RFR: 8182844: Fix broken links in the generated
 jvmti.html
In-Reply-To: <297a81a3-71c4-fe72-60a5-f228872cad00@oracle.com>
References: <bdee3e7d-75f4-9aa9-723b-fe9a17e5a902@oracle.com>
 <fd5891a1-db0b-8629-6dbb-2bbc542ff331@oracle.com>
 <638ab41e-37d4-b00d-6114-46924e7294a9@oracle.com>
 <da1f4477-425b-8444-74e2-dc189f830f72@oracle.com>
 <297a81a3-71c4-fe72-60a5-f228872cad00@oracle.com>
Message-ID: <db3a79f7-5c96-e2f5-0f72-ee1797f44e30@oracle.com>

An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170624/26d53b4c/attachment-0001.html>

From mandy.chung at oracle.com  Sat Jun 24 22:14:51 2017
From: mandy.chung at oracle.com (Mandy Chung)
Date: Sun, 25 Jun 2017 06:14:51 +0800
Subject: [Urgent JDK 9] RFR: 8182844: Fix broken links in the generated
 jvmti.html
In-Reply-To: <638ab41e-37d4-b00d-6114-46924e7294a9@oracle.com>
References: <bdee3e7d-75f4-9aa9-723b-fe9a17e5a902@oracle.com>
 <fd5891a1-db0b-8629-6dbb-2bbc542ff331@oracle.com>
 <638ab41e-37d4-b00d-6114-46924e7294a9@oracle.com>
Message-ID: <CC9EB84C-F5EC-4187-9B05-47F0154964EF@oracle.com>


> On Jun 25, 2017, at 1:26 AM, serguei.spitsyn at oracle.com wrote:
> 
>> 
>> 
>> If you want this fixed in JDK9, the priority has to be raised to P1.
>> I don't think JDK9 should go out the door with broken doc links.
>> That gives a bad impression so I would raise this to a P1 and
>> start the process of requesting approval for the push.
> 
> I've made priority P1.
> The following P1 doc bug that is in the jdk 9 queue does not have jdk9-fix-request.
> Do we need it for doc bugs?

This is not a showstopper for JDK 9 and it doesn?t justify P1.  Since we are in RC milestone, we should fix the broken links in JDK 10.

Mandy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170625/a2888014/attachment.html>

From daniel.daugherty at oracle.com  Sun Jun 25 02:05:18 2017
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Sat, 24 Jun 2017 20:05:18 -0600
Subject: [Urgent JDK 9] RFR: 8182844: Fix broken links in the generated
 jvmti.html
In-Reply-To: <CC9EB84C-F5EC-4187-9B05-47F0154964EF@oracle.com>
References: <bdee3e7d-75f4-9aa9-723b-fe9a17e5a902@oracle.com>
 <fd5891a1-db0b-8629-6dbb-2bbc542ff331@oracle.com>
 <638ab41e-37d4-b00d-6114-46924e7294a9@oracle.com>
 <CC9EB84C-F5EC-4187-9B05-47F0154964EF@oracle.com>
Message-ID: <cce7c92f-e354-ec83-8ffe-15a7cf65bb3b@oracle.com>

On 6/24/17 4:14 PM, Mandy Chung wrote:
>
>> On Jun 25, 2017, at 1:26 AM, serguei.spitsyn at oracle.com 
>> <mailto:serguei.spitsyn at oracle.com> wrote:
>>
>>>
>>>
>>> If you want this fixed in JDK9, the priority has to be raised to P1.
>>> I don't think JDK9 should go out the door with broken doc links.
>>> That gives a bad impression so I would raise this to a P1 and
>>> start the process of requesting approval for the push.
>>
>> I've made priority P1.
>> The following P1 doc bug that is in the jdk 9 queue does not have 
>> jdk9-fix-request.
>> Do we need it for doc bugs?
>
> This is not a showstopper for JDK 9 and it doesn?t justify P1.  Since 
> we are in RC milestone, we should fix the broken links in JDK 10.

I'll politely disagree. We just made the changes that broke the
doc links very recently so it fits the criteria of a new bug.
I think going out with these links broken would not give a good
impression.

Dan


>
> Mandy

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170624/7320b580/attachment.html>

From serguei.spitsyn at oracle.com  Sun Jun 25 05:46:35 2017
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Sat, 24 Jun 2017 22:46:35 -0700
Subject: [Urgent JDK 9] RFR: 8182844: Fix broken links in the generated
 jvmti.html
In-Reply-To: <cce7c92f-e354-ec83-8ffe-15a7cf65bb3b@oracle.com>
References: <bdee3e7d-75f4-9aa9-723b-fe9a17e5a902@oracle.com>
 <fd5891a1-db0b-8629-6dbb-2bbc542ff331@oracle.com>
 <638ab41e-37d4-b00d-6114-46924e7294a9@oracle.com>
 <CC9EB84C-F5EC-4187-9B05-47F0154964EF@oracle.com>
 <cce7c92f-e354-ec83-8ffe-15a7cf65bb3b@oracle.com>
Message-ID: <efd1b090-4bd8-998d-0f49-2a254ff53702@oracle.com>

An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170624/1dd9d1e5/attachment.html>

From Alan.Bateman at oracle.com  Mon Jun 26 06:20:11 2017
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Mon, 26 Jun 2017 07:20:11 +0100
Subject: [Urgent JDK 9] RFR: 8182844: Fix broken links in the generated
 jvmti.html
In-Reply-To: <cce7c92f-e354-ec83-8ffe-15a7cf65bb3b@oracle.com>
References: <bdee3e7d-75f4-9aa9-723b-fe9a17e5a902@oracle.com>
 <fd5891a1-db0b-8629-6dbb-2bbc542ff331@oracle.com>
 <638ab41e-37d4-b00d-6114-46924e7294a9@oracle.com>
 <CC9EB84C-F5EC-4187-9B05-47F0154964EF@oracle.com>
 <cce7c92f-e354-ec83-8ffe-15a7cf65bb3b@oracle.com>
Message-ID: <c7d2a0ba-6188-29d0-c26e-50ae965ae239@oracle.com>



On 25/06/2017 03:05, Daniel D. Daugherty wrote:
> On 6/24/17 4:14 PM, Mandy Chung wrote:
>> :
>>
>> This is not a showstopper for JDK 9 and it doesn?t justify P1.  Since 
>> we are in RC milestone, we should fix the broken links in JDK 10.
>
> I'll politely disagree. We just made the changes that broke the
> doc links very recently so it fits the criteria of a new bug.
> I think going out with these links broken would not give a good
> impression.
Mandy is correct. Only P1 showstoppers are allowed now and I don't think 
broken links in the JVM TI spec to the JAR or JPDA docs are either a P1 
or a showstopper issue.

-Alan

From andrew_m_leonard at uk.ibm.com  Thu Jun 22 16:31:06 2017
From: andrew_m_leonard at uk.ibm.com (Andrew Leonard)
Date: Thu, 22 Jun 2017 17:31:06 +0100
Subject: Fw: Proposal:JdpController.getProcessId() VM compatibility improvement
Message-ID: <OF2003829A.B802760D-ON80258147.005A7365-80258147.005ABEB1@notes.na.collabserv.com>

Hello,
I would like to propose the change below to the 
JdpController.getProcessId() method which currently uses a reflection hack 
and the VMManagement interface. Instead for JDK9+ we can use the 
ProcessHandle.current().getPid() method, which will also allow better 
compatibility with other pluggable VMs.

jdk/src/jdk.management.agent/share/classes/sun/management/jdp/JdpController.java
36d35
< import sun.management.VMManagement;
136,149c135,136
<         try {
<             // Get the current process id using a reflection hack
<             RuntimeMXBean runtime = 
ManagementFactory.getRuntimeMXBean();
<             Field jvm = runtime.getClass().getDeclaredField("jvm");
<             jvm.setAccessible(true);
< 
<             VMManagement mgmt = (sun.management.VMManagement) 
jvm.get(runtime);
<             Method pid_method = 
mgmt.getClass().getDeclaredMethod("getProcessId");
<             pid_method.setAccessible(true);
<             Integer pid = (Integer) pid_method.invoke(mgmt);
<             return pid;
<         } catch(Exception ex) {
<             return null;
<         }
---
>         // Get the current process id
>         return (int)ProcessHandle.current().getPid();

I'd appreciate any feedback please, and help obtaining a suitable sponsor 
and contributor?

Thanks
Andrew

Andrew Leonard
Java Runtimes Development
IBM Hursley
IBM United Kingdom Ltd
Phone internal: 245913, external: 01962 815913
internet email: andrew_m_leonard at uk.ibm.com 

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170622/6b775f42/attachment.html>

From Roger.Riggs at Oracle.com  Mon Jun 26 18:58:45 2017
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Mon, 26 Jun 2017 14:58:45 -0400
Subject: Proposal:JdpController.getProcessId() VM compatibility improvement
In-Reply-To: <OFE4AC364D.7A71FF1A-ON8025814B.004FDD9C-8025814B.0050639A@notes.na.collabserv.com>
References: <OF505F1325.D4423114-ON80258147.00474E6E-80258147.004A90D7@notes.na.collabserv.com>
 <15439b10-c6c2-aa2f-6509-66845629b9e5@oracle.com>
 <OF7DF04E8A.40ED4187-ON80258147.005A1AAE-80258147.005A67E5@notes.na.collabserv.com>
 <a50affa3-b577-55c5-6cef-8126230f5872@Oracle.com>
 <OFDD8A45C0.4897652F-ON80258148.003BD38C-80258148.003C3BD7@notes.na.collabserv.com>
 <ea6b12dd-63ed-40b1-d1b9-c96cc8cb6877@oracle.com>
 <OFE4AC364D.7A71FF1A-ON8025814B.004FDD9C-8025814B.0050639A@notes.na.collabserv.com>
Message-ID: <7ccbad84-416c-5884-be7a-7f54214007c9@Oracle.com>

Hi Andrew,

Redirecting back to serviceability-dev at openjdk.java.net, it  is the 
better list for this topic.

I don't know these tests but it seems odd to stick that assert into every
callback to packetFromThisVMReceived.  Perhaps someone more familiar can 
review and sponsor.

Thanks, Roger




On 6/26/2017 10:37 AM, Andrew Leonard wrote:
> Hi Roger,
> I have now updated to the latest openjdk9 and examined the existing 
> jdp tests. I have added to those tests to assert the JDP packet 
> processId, which was in fact "null", so it was failing to set it 
> before... Hence, with this testcase update they fail without my patch, 
> and succeed with the new patch.
> Being new to this contribution process, where do I go from here please?
> Thanks
> Andrew
>
> diff -r 2425838cfb5e 
> src/jdk.management.agent/share/classes/sun/management/jdp/JdpController.java
> --- 
> a/src/jdk.management.agent/share/classes/sun/management/jdp/JdpController.java 
>      Fri Jun 23 14:32:59 2017 -0400
> +++ 
> b/src/jdk.management.agent/share/classes/sun/management/jdp/JdpController.java 
>      Mon Jun 26 15:31:15 2017 +0100
> @@ -133,20 +133,8 @@
>
>      // Get the process id of the current running Java process
>      private static Integer getProcessId() {
> -        try {
> -  // Get the current process id using a reflection hack
> -  RuntimeMXBean runtime = ManagementFactory.getRuntimeMXBean();
> -  Field jvm = runtime.getClass().getDeclaredField("jvm");
> -  jvm.setAccessible(true);
> -
> -  VMManagement mgmt = (sun.management.VMManagement) jvm.get(runtime);
> -  Method pid_method = mgmt.getClass().getDeclaredMethod("getProcessId");
> -  pid_method.setAccessible(true);
> -  Integer pid = (Integer) pid_method.invoke(mgmt);
> -  return pid;
> -        } catch(Exception ex) {
> -  return null;
> -        }
> +        // Get the current process id
> +        return (int)ProcessHandle.current().pid();
>      }
>
> diff -r 2425838cfb5e test/sun/management/jdp/JdpOnTestCase.java
> --- a/test/sun/management/jdp/JdpOnTestCase.java        Fri Jun 23 
> 14:32:59 2017 -0400
> +++ b/test/sun/management/jdp/JdpOnTestCase.java        Mon Jun 26 
> 15:30:51 2017 +0100
> @@ -31,6 +31,7 @@
>
>  import java.net.SocketTimeoutException;
>  import java.util.Map;
> +import static jdk.testlibrary.Asserts.assertNotEquals;
>
>  public class JdpOnTestCase extends JdpTestCase {
>
> @@ -58,6 +59,7 @@
>          final String jdpName = payload.get("INSTANCE_NAME");
>          log.fine("Received correct JDP packet #" + 
> String.valueOf(receivedJDPpackets) +
>      ", jdp.name=" + jdpName);
> +        assertNotEquals(null, payload.get("PROCESS_ID"), "Expected 
> payload.get(\"PROCESS_ID\") to be not null.");
>      }
>
>
>
>
> Andrew Leonard
> Java Runtimes Development
> IBM Hursley
> IBM United Kingdom Ltd
> Phone internal: 245913, external: 01962 815913
> internet email: andrew_m_leonard at uk.ibm.com
>
>
>
>
> From: Roger Riggs <roger.riggs at oracle.com>
> To: Andrew Leonard <andrew_m_leonard at uk.ibm.com>
> Cc: core-libs-dev at openjdk.java.net
> Date: 23/06/2017 13:57
> Subject: Re: Proposal:JdpController.getProcessId() VM compatibility 
> improvement
> ------------------------------------------------------------------------
>
>
>
> Hi Leonard,
>
> No need to stray from your intended focus.
> The other is just another cleanup.
>
> Thanks, Roger
>
>
> On 6/23/17 6:57 AM, Andrew Leonard wrote:
> Thanks Roger,
> So yes that issue does look similar, although I was only looking in 
> the management interfaces. I'll have a peek at those references to see 
> if they are similar, it would make sense to clean those up too, to be 
> consistent. Your thoughts?
> I'll also see if there's a junit for this method, if not i'll see if I 
> can add one.
> Cheers
> Andrew
>
> Andrew Leonard
> Java Runtimes Development
> IBM Hursley
> IBM United Kingdom Ltd
> Phone internal: 245913, external: 01962 815913
> internet email: _andrew_m_leonard at uk.ibm.com_ 
> <mailto:andrew_m_leonard at uk.ibm.com>
>
>
>
>
> From: Roger Riggs _<Roger.Riggs at Oracle.com>_ 
> <mailto:Roger.Riggs at Oracle.com>
> To: _core-libs-dev at openjdk.java.net_ 
> <mailto:core-libs-dev at openjdk.java.net>
> Date: 22/06/2017 18:03
> Subject: Re: Proposal:JdpController.getProcessId() VM compatibility 
> improvement
> Sent by: "core-libs-dev" _<core-libs-dev-bounces at openjdk.java.net>_ 
> <mailto:core-libs-dev-bounces at openjdk.java.net>
> ------------------------------------------------------------------------
>
>
>
> HI,
>
> This looks like:
>
> JDK-8074569 <_https://bugs.openjdk.java.net/browse/JDK-8074569_> Update
> implementation to use ProcessHandle.current() to get the PID
>
> I'm happy to sponsor.
>
> Roger
>
> On 6/22/2017 12:27 PM, Andrew Leonard wrote:
> > Thanks Alan,
> > Yes, you're right the exception swallowing can be removed too.
> > I'll re-post to serviceability-dev...
> > Cheers
> > Andrew
> >
> > Andrew Leonard
> > Java Runtimes Development
> > IBM Hursley
> > IBM United Kingdom Ltd
> > Phone internal: 245913, external: 01962 815913
> > internet email: _andrew_m_leonard at uk.ibm.com_ 
> <mailto:andrew_m_leonard at uk.ibm.com>
> >
> >
> >
> >
> > From:   Alan Bateman _<Alan.Bateman at oracle.com>_ 
> <mailto:Alan.Bateman at oracle.com>
> > To:     Andrew Leonard _<andrew_m_leonard at uk.ibm.com>_ 
> <mailto:andrew_m_leonard at uk.ibm.com>,
> > _core-libs-dev at openjdk.java.net_ <mailto:core-libs-dev at openjdk.java.net>
> > Date:   22/06/2017 17:21
> > Subject:        Re: Proposal:JdpController.getProcessId() VM 
> compatibility
> > improvement
> >
> >
> >
> >
> > This seems a good clean-up, esp. as java.management doesn't open
> > sun.management to jdk.management.agent so I'll bet this wasn't working
> > any. I assume the exception swallowing can be removed too.
> >
> > Can you bring this to serviceability-dev as this is where this code is
> > maintained?
> >
> > On 22/06/2017 14:34, Andrew Leonard wrote:
> >> Hello,
> >> I would like to propose the change below to the
> >> JdpController.getProcessId() method which currently uses a reflection
> > hack
> >> and the VMManagement interface. Instead for JDK9+ we can use the
> >> ProcessHandle.current().getPid() method, which will also allow better
> >> compatibility with other pluggable VMs.
> >>
> >>
> > 
> jdk/src/jdk.management.agent/share/classes/sun/management/jdp/JdpController.java
> >> 36d35
> >> < import sun.management.VMManagement;
> >> 137,146c136,137
> >> <             // Get the current process id using a reflection hack
> >> <             RuntimeMXBean runtime =
> >> ManagementFactory.getRuntimeMXBean();
> >> <             Field jvm = runtime.getClass().getDeclaredField("jvm");
> >> <             jvm.setAccessible(true);
> >> <
> >> <             VMManagement mgmt = (sun.management.VMManagement)
> >> jvm.get(runtime);
> >> <             Method pid_method =
> >> mgmt.getClass().getDeclaredMethod("getProcessId");
> >> <             pid_method.setAccessible(true);
> >> <             Integer pid = (Integer) pid_method.invoke(mgmt);
> >> <             return pid;
> >> ---
> >>>               // Get the current process id
> >>>               return (int)ProcessHandle.current().getPid();
> >> I'd appreciate any feedback please, and how I would go about 
> obtaining a
> >> sponsor and contributor?
> >>
> >> Thanks
> >> Andrew
> >>
> >> Andrew Leonard
> >> Java Runtimes Development
> >> IBM Hursley
> >> IBM United Kingdom Ltd
> >> Phone internal: 245913, external: 01962 815913
> >> internet email: _andrew_m_leonard at uk.ibm.com_ 
> <mailto:andrew_m_leonard at uk.ibm.com>
> >>
> >> Unless stated otherwise above:
> >> IBM United Kingdom Limited - Registered in England and Wales with 
> number
> >> 741598.
> >> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
> > 3AU
> >
> >
> >
> >
> > Unless stated otherwise above:
> > IBM United Kingdom Limited - Registered in England and Wales with number
> > 741598.
> > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire 
> PO6 3AU
>
>
>
>
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with 
> number 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 
> 3AU
>
>
>
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with 
> number 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170626/5da06483/attachment-0001.html>

From serguei.spitsyn at oracle.com  Tue Jun 27 05:53:52 2017
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Mon, 26 Jun 2017 22:53:52 -0700
Subject: [Urgent JDK 9] RFR: 8182844: Fix broken links in the generated
 jvmti.html
In-Reply-To: <c7d2a0ba-6188-29d0-c26e-50ae965ae239@oracle.com>
References: <bdee3e7d-75f4-9aa9-723b-fe9a17e5a902@oracle.com>
 <fd5891a1-db0b-8629-6dbb-2bbc542ff331@oracle.com>
 <638ab41e-37d4-b00d-6114-46924e7294a9@oracle.com>
 <CC9EB84C-F5EC-4187-9B05-47F0154964EF@oracle.com>
 <cce7c92f-e354-ec83-8ffe-15a7cf65bb3b@oracle.com>
 <c7d2a0ba-6188-29d0-c26e-50ae965ae239@oracle.com>
Message-ID: <af5755f1-9fc3-41b6-dd79-6d2de6ba88fc@oracle.com>

Sorry, I forgot to send this message yesterday.
-----

Dan, Mandy and Alan,

Ok, I've updated the bug accordingly.

Thanks!
Serguei


On 6/25/17 23:20, Alan Bateman wrote:
>
>
> On 25/06/2017 03:05, Daniel D. Daugherty wrote:
>> On 6/24/17 4:14 PM, Mandy Chung wrote:
>>> :
>>>
>>> This is not a showstopper for JDK 9 and it doesn?t justify P1.  
>>> Since we are in RC milestone, we should fix the broken links in JDK 10.
>>
>> I'll politely disagree. We just made the changes that broke the
>> doc links very recently so it fits the criteria of a new bug.
>> I think going out with these links broken would not give a good
>> impression.
> Mandy is correct. Only P1 showstoppers are allowed now and I don't 
> think broken links in the JVM TI spec to the JAR or JPDA docs are 
> either a P1 or a showstopper issue.
>
> -Alan


From andrew_m_leonard at uk.ibm.com  Tue Jun 27 08:09:32 2017
From: andrew_m_leonard at uk.ibm.com (Andrew Leonard)
Date: Tue, 27 Jun 2017 09:09:32 +0100
Subject: Proposal:JdpController.getProcessId() VM compatibility improvement
In-Reply-To: <7ccbad84-416c-5884-be7a-7f54214007c9@Oracle.com>
References: <OF505F1325.D4423114-ON80258147.00474E6E-80258147.004A90D7@notes.na.collabserv.com>
 <15439b10-c6c2-aa2f-6509-66845629b9e5@oracle.com>
 <OF7DF04E8A.40ED4187-ON80258147.005A1AAE-80258147.005A67E5@notes.na.collabserv.com>
 <a50affa3-b577-55c5-6cef-8126230f5872@Oracle.com>
 <OFDD8A45C0.4897652F-ON80258148.003BD38C-80258148.003C3BD7@notes.na.collabserv.com>
 <ea6b12dd-63ed-40b1-d1b9-c96cc8cb6877@oracle.com>
 <OFE4AC364D.7A71FF1A-ON8025814B.004FDD9C-8025814B.0050639A@notes.na.collabserv.com>
 <7ccbad84-416c-5884-be7a-7f54214007c9@Oracle.com>
Message-ID: <OFAAC24269.7108A021-ON8025814C.002CC188-8025814C.002CD347@notes.na.collabserv.com>

Thanks Roger for taking a look, i'll see what the serviceability community 
think,
cheers
Andrew

Andrew Leonard
Java Runtimes Development
IBM Hursley
IBM United Kingdom Ltd
Phone internal: 245913, external: 01962 815913
internet email: andrew_m_leonard at uk.ibm.com 




From:   Roger Riggs <Roger.Riggs at Oracle.com>
To:     Andrew Leonard <andrew_m_leonard at uk.ibm.com>
Cc:     serviceability-dev at openjdk.java.net, Core-Libs-Dev 
<core-libs-dev at openjdk.java.net>
Date:   26/06/2017 19:59
Subject:        Re: Proposal:JdpController.getProcessId() VM compatibility 
improvement



Hi Andrew,

Redirecting back to serviceability-dev at openjdk.java.net, it  is the better 
list for this topic.

I don't know these tests but it seems odd to stick that assert into every  

callback to packetFromThisVMReceived.  Perhaps someone more familiar can 
review and sponsor.

Thanks, Roger




On 6/26/2017 10:37 AM, Andrew Leonard wrote:
Hi Roger, 
I have now updated to the latest openjdk9 and examined the existing jdp 
tests. I have added to those tests to assert the JDP packet processId, 
which was in fact "null", so it was failing to set it before... Hence, 
with this testcase update they fail without my patch, and succeed with the 
new patch. 
Being new to this contribution process, where do I go from here please? 
Thanks 
Andrew 

diff -r 2425838cfb5e 
src/jdk.management.agent/share/classes/sun/management/jdp/JdpController.java
--- 
a/src/jdk.management.agent/share/classes/sun/management/jdp/JdpController.java  
     Fri Jun 23 14:32:59 2017 -0400 
+++ 
b/src/jdk.management.agent/share/classes/sun/management/jdp/JdpController.java  
     Mon Jun 26 15:31:15 2017 +0100 
@@ -133,20 +133,8 @@ 
  
     // Get the process id of the current running Java process 
     private static Integer getProcessId() { 
-        try { 
-            // Get the current process id using a reflection hack 
-            RuntimeMXBean runtime = ManagementFactory.getRuntimeMXBean(); 

-            Field jvm = runtime.getClass().getDeclaredField("jvm"); 
-            jvm.setAccessible(true); 
- 
-            VMManagement mgmt = (sun.management.VMManagement) 
jvm.get(runtime); 
-            Method pid_method = 
mgmt.getClass().getDeclaredMethod("getProcessId"); 
-            pid_method.setAccessible(true); 
-            Integer pid = (Integer) pid_method.invoke(mgmt); 
-            return pid; 
-        } catch(Exception ex) { 
-            return null; 
-        } 
+        // Get the current process id 
+        return (int)ProcessHandle.current().pid(); 
     } 

diff -r 2425838cfb5e test/sun/management/jdp/JdpOnTestCase.java 
--- a/test/sun/management/jdp/JdpOnTestCase.java        Fri Jun 23 
14:32:59 2017 -0400 
+++ b/test/sun/management/jdp/JdpOnTestCase.java        Mon Jun 26 
15:30:51 2017 +0100 
@@ -31,6 +31,7 @@ 
  
 import java.net.SocketTimeoutException; 
 import java.util.Map; 
+import static jdk.testlibrary.Asserts.assertNotEquals; 
  
 public class JdpOnTestCase extends JdpTestCase { 
  
@@ -58,6 +59,7 @@ 
         final String jdpName = payload.get("INSTANCE_NAME"); 
         log.fine("Received correct JDP packet #" + 
String.valueOf(receivedJDPpackets) + 
                 ", jdp.name=" + jdpName); 
+        assertNotEquals(null, payload.get("PROCESS_ID"), "Expected 
payload.get(\"PROCESS_ID\") to be not null."); 
     } 




Andrew Leonard
Java Runtimes Development
IBM Hursley
IBM United Kingdom Ltd
Phone internal: 245913, external: 01962 815913
internet email: andrew_m_leonard at uk.ibm.com 




From:        Roger Riggs <roger.riggs at oracle.com> 
To:        Andrew Leonard <andrew_m_leonard at uk.ibm.com> 
Cc:        core-libs-dev at openjdk.java.net 
Date:        23/06/2017 13:57 
Subject:        Re: Proposal:JdpController.getProcessId() VM compatibility 
improvement 



Hi Leonard,

No need to stray from your intended focus.  
The other is just another cleanup.

Thanks, Roger


On 6/23/17 6:57 AM, Andrew Leonard wrote: 
Thanks Roger, 
So yes that issue does look similar, although I was only looking in the 
management interfaces. I'll have a peek at those references to see if they 
are similar, it would make sense to clean those up too, to be consistent. 
Your thoughts? 
I'll also see if there's a junit for this method, if not i'll see if I can 
add one. 
Cheers 
Andrew 

Andrew Leonard
Java Runtimes Development
IBM Hursley
IBM United Kingdom Ltd
Phone internal: 245913, external: 01962 815913
internet email: andrew_m_leonard at uk.ibm.com 




From:        Roger Riggs <Roger.Riggs at Oracle.com> 
To:        core-libs-dev at openjdk.java.net 
Date:        22/06/2017 18:03 
Subject:        Re: Proposal:JdpController.getProcessId() VM compatibility 
improvement 
Sent by:        "core-libs-dev" <core-libs-dev-bounces at openjdk.java.net> 



HI,

This looks like:

JDK-8074569 <https://bugs.openjdk.java.net/browse/JDK-8074569> Update 
implementation to use ProcessHandle.current() to get the PID

I'm happy to sponsor.

Roger

On 6/22/2017 12:27 PM, Andrew Leonard wrote:
> Thanks Alan,
> Yes, you're right the exception swallowing can be removed too.
> I'll re-post to serviceability-dev...
> Cheers
> Andrew
>
> Andrew Leonard
> Java Runtimes Development
> IBM Hursley
> IBM United Kingdom Ltd
> Phone internal: 245913, external: 01962 815913
> internet email: andrew_m_leonard at uk.ibm.com
>
>
>
>
> From:   Alan Bateman <Alan.Bateman at oracle.com>
> To:     Andrew Leonard <andrew_m_leonard at uk.ibm.com>,
> core-libs-dev at openjdk.java.net
> Date:   22/06/2017 17:21
> Subject:        Re: Proposal:JdpController.getProcessId() VM 
compatibility
> improvement
>
>
>
>
> This seems a good clean-up, esp. as java.management doesn't open
> sun.management to jdk.management.agent so I'll bet this wasn't working
> any. I assume the exception swallowing can be removed too.
>
> Can you bring this to serviceability-dev as this is where this code is
> maintained?
>
> On 22/06/2017 14:34, Andrew Leonard wrote:
>> Hello,
>> I would like to propose the change below to the
>> JdpController.getProcessId() method which currently uses a reflection
> hack
>> and the VMManagement interface. Instead for JDK9+ we can use the
>> ProcessHandle.current().getPid() method, which will also allow better
>> compatibility with other pluggable VMs.
>>
>>
> 
jdk/src/jdk.management.agent/share/classes/sun/management/jdp/JdpController.java
>> 36d35
>> < import sun.management.VMManagement;
>> 137,146c136,137
>> <             // Get the current process id using a reflection hack
>> <             RuntimeMXBean runtime =
>> ManagementFactory.getRuntimeMXBean();
>> <             Field jvm = runtime.getClass().getDeclaredField("jvm");
>> <             jvm.setAccessible(true);
>> <
>> <             VMManagement mgmt = (sun.management.VMManagement)
>> jvm.get(runtime);
>> <             Method pid_method =
>> mgmt.getClass().getDeclaredMethod("getProcessId");
>> <             pid_method.setAccessible(true);
>> <             Integer pid = (Integer) pid_method.invoke(mgmt);
>> <             return pid;
>> ---
>>>               // Get the current process id
>>>               return (int)ProcessHandle.current().getPid();
>> I'd appreciate any feedback please, and how I would go about obtaining 
a
>> sponsor and contributor?
>>
>> Thanks
>> Andrew
>>
>> Andrew Leonard
>> Java Runtimes Development
>> IBM Hursley
>> IBM United Kingdom Ltd
>> Phone internal: 245913, external: 01962 815913
>> internet email: andrew_m_leonard at uk.ibm.com
>>
>> Unless stated otherwise above:
>> IBM United Kingdom Limited - Registered in England and Wales with 
number
>> 741598.
>> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
> 3AU
>
>
>
>
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 
3AU




Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU 




Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU



Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170627/bce79fed/attachment-0001.html>

From christoph.langer at sap.com  Tue Jun 27 18:41:40 2017
From: christoph.langer at sap.com (Langer, Christoph)
Date: Tue, 27 Jun 2017 18:41:40 +0000
Subject: Proposal:JdpController.getProcessId() VM compatibility improvement
In-Reply-To: <OF2003829A.B802760D-ON80258147.005A7365-80258147.005ABEB1@notes.na.collabserv.com>
References: <OF2003829A.B802760D-ON80258147.005A7365-80258147.005ABEB1@notes.na.collabserv.com>
Message-ID: <a5c8cddda4e64ddb9a9a003a7edd81f9@sap.com>

Hi Andrew,

I can help you with this contribution.

First question: Do you want to create a fix completely resolving Roger's bug 8074569 [1] or shall I create a new bug for you? In the latter case, please send me a title and a long text (you can send this in a private Email).

Then, when we have the bug, you'll need to create a WebRev and post it for review. Are you familiar with that process? Do you have some close colleague who can assist you with that and upload a WebRev to cr.openjdk.java.net? If not I can do that for you when you send me the data. We can discuss that in a private mail thread as well.

Best regards
Christoph

[1] https://bugs.openjdk.java.net/browse/JDK-8074569

From: serviceability-dev [mailto:serviceability-dev-bounces at openjdk.java.net] On Behalf Of Andrew Leonard
Sent: Donnerstag, 22. Juni 2017 18:31
To: serviceability-dev at openjdk.java.net
Subject: Fw: Proposal:JdpController.getProcessId() VM compatibility improvement

Hello,
I would like to propose the change below to the JdpController.getProcessId() method which currently uses a reflection hack and the VMManagement interface. Instead for JDK9+ we can use the ProcessHandle.current().getPid() method, which will also allow better compatibility with other pluggable VMs.

jdk/src/jdk.management.agent/share/classes/sun/management/jdp/JdpController.java
36d35
< import sun.management.VMManagement;
136,149c135,136
<         try {
<             // Get the current process id using a reflection hack
<             RuntimeMXBean runtime = ManagementFactory.getRuntimeMXBean();
<             Field jvm = runtime.getClass().getDeclaredField("jvm");
<             jvm.setAccessible(true);
<
<             VMManagement mgmt = (sun.management.VMManagement) jvm.get(runtime);
<             Method pid_method = mgmt.getClass().getDeclaredMethod("getProcessId");
<             pid_method.setAccessible(true);
<             Integer pid = (Integer) pid_method.invoke(mgmt);
<             return pid;
<         } catch(Exception ex) {
<             return null;
<         }
---
>         // Get the current process id
>         return (int)ProcessHandle.current().getPid();

I'd appreciate any feedback please, and help obtaining a suitable sponsor and contributor?

Thanks
Andrew

Andrew Leonard
Java Runtimes Development
IBM Hursley
IBM United Kingdom Ltd
Phone internal: 245913, external: 01962 815913
internet email: andrew_m_leonard at uk.ibm.com<mailto:andrew_m_leonard at uk.ibm.com>

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170627/660fb1bc/attachment.html>

From christoph.langer at sap.com  Tue Jun 27 19:09:13 2017
From: christoph.langer at sap.com (Langer, Christoph)
Date: Tue, 27 Jun 2017 19:09:13 +0000
Subject: RFR 10 (L): 8183012: Code cleanup in com.sun.tools.jdi
Message-ID: <ebb2e14c731c4f37b9d45d6050e4df8d@sap.com>

Hi,

I've got another contribution for cleaning up the jdk.jdi classes. This time it's about package com.sun.tools.jdi.

Bug: https://bugs.openjdk.java.net/browse/JDK-8183012
Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8183012.0/

What I've done:
I've largely cleaned up the import statements (removing obsolete imports, expanding "*" wildcards, sorting). Furthermore I did quite a few whitespace cleanups to make the code look nicer. And I removed the types in constructors of typed classes.

The more interesting stuff which should probably closely be reviewed is the following:
a) Remove unnecessary casts in BooleanValueImpl.java, ByteValueImpl.java, CharValueImpl.java, FloatValueImpl.java, LongValueImpl.java, PacketStream.java, ShortValueImpl.java
b) Remove redundant super interfaces in the class declarations of ClassLoaderReferenceImpl.java, ConnectorImpl.java, RawCommandLineLauncher.java, SunCommandLineLauncher.java, ThreadGroupReferenceImpl.java, ThreadReferenceImpl.java
c) ObjectReferenceImpl.java: Remove some code in void validateClassMethodInvocation(Method method, int options). Some very old comment suggests that the code was still there because nobody divined what it was useful for. But I couldn't find anything that seems to be really useful here, except if the caller maybe wants to get some exception if such occurred in the course of resolving the class or something like that.
d) SocketTransportService.java: pull out SocketConnection to an own file SocketConnection.java, pull class SocketTransportServiceCapabilities into anonymous class within SocketTransportService.capabilities()
e) RawCommandLineLauncher.java, SunCommandLineLauncher.java: Modifications to constructing the TransportService object in its constructors

Module jdk.jdi still builds without warnings after my change and I'm currently running the jdi jtreg suite.

Thanks in advance and best regards
Christoph

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170627/04fbffa0/attachment.html>

From jcbeyler at google.com  Tue Jun 27 22:04:26 2017
From: jcbeyler at google.com (JC Beyler)
Date: Tue, 27 Jun 2017 15:04:26 -0700
Subject: Low-Overhead Heap Profiling
In-Reply-To: <1498215147.2741.34.camel@oracle.com>
References: <CAF9BGBx8xFYudhJ=v1_hU5=nwEcC1X0NKRmC=E-RNN6SMs1ozg@mail.gmail.com>
 <CAF9BGBxj7+C6uy4dXHO-3pNorbpO7n22-9JmevM0F11xOo+MnA@mail.gmail.com>
 <CAF9BGBxrdLVmXRTDHj5hQHJhcyrkBM6+ex1TFWi=A5z=V2FSew@mail.gmail.com>
 <CAF9BGBz81YYVUsMLw9sOMR9fH7JQmyb8d_OZ9aJ47EOAqwWhEw@mail.gmail.com>
 <2af975e6-3827-bd57-0c3d-fadd54867a67@oracle.com>
 <365499b6-3f4d-a4df-9e7e-e72a739fb26b@oracle.com>
 <CAF9BGBy0T7A-+1dejYbr6f0d4s0XR-zv-WwHYqXrbe0EtCdsxQ@mail.gmail.com>
 <fb1155c6-ccf8-843c-34fc-fc4a45fc54cc@oracle.com>
 <CAF9BGBxqmXF9OFD4F9wVb-DQOj_UfNNcBhe+ARVxJ5E4cgCC1Q@mail.gmail.com>
 <102c59b8-25b6-8c21-8eef-1de7d0bbf629@oracle.com>
 <CAF9BGBxDaCOxobJi8Uu9kwKPRpnj-KqgWLfJO2z7UX71UnNF4g@mail.gmail.com>
 <CAF9BGByisZchXTsw1=mLETLnOggJShRJBDxFeOUgt+criTTGuQ@mail.gmail.com>
 <1497366226.2829.109.camel@oracle.com>
 <CAF9BGBx8376y_yeMFnMC8PHSqKDaFc5NcbFerQ2up0YVKtSKGg@mail.gmail.com>
 <1498215147.2741.34.camel@oracle.com>
Message-ID: <CAF9BGBwaRu+12iRMm6U+n8j_+3izaEcv1OExdFKqZczEE7SJ0A@mail.gmail.com>

Dear Thomas et al,

Here is the newest webrev:
http://cr.openjdk.java.net/~rasbold/8171119/webrev.07/

Thomas, I "think" I have answered all your remarks. The summary is:

- The statistic system is up and provides insight on what the heap sampler
is doing
   - I've noticed that, though the sampling rate is at the right mean, we
are missing some samples, I have not yet tracked out why (details below)

- I've run a tiny benchmark that is the worse case: it is a very tight loop
and allocated a small array
   - In this case, I see no overhead when the system is off so that is a
good start :)
   - I see right now a high overhead in this case when sampling is on. This
is not a really too surprising but I'm going to see if this is consistent
with our internal implementation. The benchmark is really allocation
stressful so I'm not too surprised but I want to do the due diligence.

 - The statistic system up is up and I have a new test
http://cr.openjdk.java.net/~rasbold/8171119/webrev.07/test/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatTest.java.patch
    - I did a bit of a study about the random generator here, more details
are below but basically it seems to work well

 - I added a capability but since this is the first time doing this, I was
not sure I did it right
   - I did add a test though for it and the test seems to do what I expect
(all methods are failing with the JVMTI_ERROR_MUST_POSSESS_CAPABILITY
error).
       -
http://cr.openjdk.java.net/~rasbold/8171119/webrev.07/test/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorNoCapabilityTest.java.patch

 - I still need to figure out what to do about the multi-agent vs
single-agent issue

 - As far as measurements, it seems I still need to look at:
   - Why we do the 20 random calls first, are they necessary?
   - Look at the mean of the sampling rate that the random generator does
and also what is actually sampled
   - What is the overhead in terms of memory/performance when on?

I have inlined my answers, I think I got them all in the new webrev, let me
know your thoughts.

Thanks again!
Jc


On Fri, Jun 23, 2017 at 3:52 AM, Thomas Schatzl <thomas.schatzl at oracle.com>
wrote:

> Hi,
>
> On Wed, 2017-06-21 at 13:45 -0700, JC Beyler wrote:
> > Hi all,
> >
> > First off: Thanks again to Robbin and Thomas for their reviews :)
> >
> > Next, I've uploaded a new webrev:
> > http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/
> >
> > Here is an update:
> >
> > - @Robbin, I forgot to say that yes I need to look at implementing
> > this for the other architectures and testing it before it is all
> > ready to go. Is it common to have it working on all possible
> > combinations or is there a subset that I should be doing first and we
> > can do the others later?
> > - I've tested slowdebug, built and ran the JTreg tests I wrote with
> > slowdebug and fixed a few more issues
> > - I've refactored a bit of the code following Thomas' comments
> >    - I think I've handled all the comments from Thomas (I put
> > comments inline below for the specifics)
>
> Thanks for handling all those.
>
> > - Following Thomas' comments on statistics, I want to add some
> > quality assurance tests and find that the easiest way would be to
> > have a few counters of what is happening in the sampler and expose
> > that to the user.
> >    - I'll be adding that in the next version if no one sees any
> > objections to that.
> >    - This will allow me to add a sanity test in JTreg about number of
> > samples and average of sampling rate
> >
> > @Thomas: I had a few questions that I inlined below but I will
> > summarize the "bigger ones" here:
> >    - You mentioned constants are not using the right conventions, I
> > looked around and didn't see any convention except normal naming then
> > for static constants. Is that right?
>
> I looked through https://wiki.openjdk.java.net/display/HotSpot/StyleGui
> de and the rule is to "follow an existing pattern and must have a
> distinct appearance from other names". Which does not help a lot I
> guess :/ The GC team started using upper camel case, e.g.
> SomeOtherConstant, but very likely this is probably not applied
> consistently throughout. So I am fine with not adding another style
> (like kMaxStackDepth with the "k" in front with some unknown meaning)
> is fine.
>
> (Chances are you will find that style somewhere used anyway too,
> apologies if so :/)
>

Thanks for that link, now I know where to look. I used the upper camel case
in my code as well then :) I should have gotten them all.


>
> > PS: I've also inlined my answers to Thomas below:
> >
> > On Tue, Jun 13, 2017 at 8:03 AM, Thomas Schatzl <thomas.schatzl at oracl
> > e.com> wrote:
> > > Hi all,
> > >
> > > On Mon, 2017-06-12 at 11:11 -0700, JC Beyler wrote:
> > > > Dear all,
> > > >
> > > > I've continued working on this and have done the following
> > > webrev:
> > > > http://cr.openjdk.java.net/~rasbold/8171119/webrev.05/
> > >
> > > [...]
> > > > Things I still need to do:
> > > >    - Have to fix that TLAB case for the FastTLABRefill
> > > >    - Have to start looking at the data to see that it is
> > > consistent and does gather the right samples, right frequency, etc.
> > > >    - Have to check the GC elements and what that produces
> > > >    - Run a slowdebug run and ensure I fixed all those issues you
> > > saw > Robbin
> > > >
> > > > Thanks for looking at the webrev and have a great week!
> > >
> > >   scratching a bit on the surface of this change, so apologies for
> > > rather shallow comments:
> > >
> > > - macroAssembler_x86.cpp:5604: while this is compiler code, and I
> > > am not sure this is final, please avoid littering the code with
> > > TODO remarks :) They tend to be candidates for later wtf moments
> > > only.
> > >
> > > Just file a CR for that.
> > >
> > Newcomer question: what is a CR and not sure I have the rights to do
> > that yet ? :)
>
> Apologies. CR is a change request, this suggests to file a bug in the
> bug tracker. And you are right, you can't just create a new account in
> the OpenJDK JIRA yourselves. :(
>

Ok good to know, I'll continue with my own todo list but I'll work hard on
not letting it slip in the webrevs anymore :)


>
> I was mostly referring to the "... but it is a TODO" part of that
> comment in macroassembler_x86.cpp. Comments about the why of the code
> are appreciated.


> [Note that I now understand that this is to some degree still work in
> progress. As long as the final changeset does no contain TODO's I am
> fine (and it's not a hard objection, rather their use in "final" code
> is typically limited in my experience)]
>
> 5603   // Currently, if this happens, just set back the actual end to
> where it was.
> 5604   // We miss a chance to sample here.
>
> Would be okay, if explaining "this" and the "why" of missing a chance
> to sample here would be best.
>
> Like maybe:
>
> // If we needed to refill TLABs, just set the actual end point to
> // the end of the TLAB again. We do not sample here although we could.
>
>
Done with your comment, it works well in my mind.


> I am not sure whether "miss a chance to sample" meant "we could, but
> consciously don't because it's not that useful" or "it would be
> necessary but don't because it's too complicated to do.".
>
> Looking at the original comment once more, I am also not sure if that
> comment shouldn't referring to the "end" variable (not actual_end)
> because that's the variable that is responsible for taking the sampling
> path? (Going from the member description of ThreadLocalAllocBuffer).
>

I've moved this code and it no longer shows up here but the rationale and
answer was:

So.. Yes, end is the variable provoking the sampling. Actual end is the
actual end of the TLAB.

What was happening here is that the code is resetting _end to point towards
the end of the new TLAB. Because, we now have the end for sampling and
_actual_end for the actual end, we need to update the actual_end as well.

Normally, were we to do the real work here, we would calculate the (end -
start) offset, then do:

- Set the new end to : start + (old_end - old_start)
- Set the actual end like we do here now where it because it is the actual
end.

Why is this not done here now anymore?
  - I was still debating which path to take:
     - Do it in the fast refill code, it has its perks:
         - In a world where fast refills are happening all the time or a
lot, we can augment there the code to do the sampling
     - Remember what we had as an end before leaving the slowpath and check
on return
         - This is what I'm doing now, it removes the need to go fix up all
fast refill paths but if you remain in fast refill paths, you won't get
sampling. I have to think of the consequences of that, maybe a future
change later on?
            - I have the statistics now so I'm going to study that
               -> By the way, though my statistics are showing I'm missing
some samples, if I turn off FastTlabRefill, it is the same loss so for now,
it seems this does not occur in my simple test.



>
> But maybe I am only confused and it's best to just leave the comment
> away. :)
>
> Thinking about it some more, doesn't this not-sampling in this case
> mean that sampling does not work in any collector that does inline TLAB
> allocation at the moment? (Or is inline TLAB alloc automatically
> disabled with sampling somehow?)


> That would indeed be a bigger TODO then :)
>

Agreed, this remark made me think that perhaps as a first step the new way
of doing it is better but I did have to:
 - Remove the const of the ThreadLocalBuffer remaining and hard_end methods
 - Move hard_end out of the header file to have a bit more logic there

Please let me know what you think of that and if you prefer it this way or
changing the fast refills. (I prefer this way now because it is more
incremental).


>
> > > - calling HeapMonitoring::do_weak_oops() (which should probably be
> > > called weak_oops_do() like other similar methods) only if string
> > > deduplication is enabled (in g1CollectedHeap.cpp:4511) seems wrong.
> >
> > The call should be at least around 6 lines up outside the if.
> >
> > Preferentially in a method like process_weak_jni_handles(), including
> > additional logging. (No new (G1) gc phase without minimal logging
> > :)).
> > Done but really not sure because:
> >
> > I put for logging:
> >   log_develop_trace(gc, freelist)("G1ConcRegionFreeing [other] : heap
> > monitoring");
>
> I would think that "gc, ref" would be more appropriate log tags for
> this similar to jni handles.
> (I am als not sure what weak reference handling has to do with
> G1ConcRegionFreeing, so I am a bit puzzled)
>

I was not sure what to put for the tags or really as the message. I cleaned
it up a bit now to:
   log_develop_trace(gc, ref)("HeapSampling [other] : heap monitoring
processing");



> > Since weak_jni_handles didn't have logging for me to be inspired
> > from, I did that but unconvinced this is what should be done.
>
> The JNI handle processing does have logging, but only in
> ReferenceProcessor::process_discovered_references(). In
> process_weak_jni_handles() only overall time is measured (in a G1
> specific way, since only G1 supports disabling reference procesing) :/
>
> The code in ReferenceProcessor prints both time taken
> referenceProcessor.cpp:254, as well as the count, but strangely only in
> debug VMs.
>
> I have no idea why this logging is that unimportant to only print that
> in a debug VM. However there are reviews out for changing this area a
> bit, so it might be useful to wait for that (JDK-8173335).
>

I cleaned it up a bit anyway and now it returns the count of objects that
are in the system.


>
> > > - the change doubles the size of
> > > CollectedHeap::allocate_from_tlab_slow() above the "small and nice"
> > > threshold. Maybe it could be refactored a bit.
> > Done I think, it looks better to me :).
>
> In ThreadLocalAllocBuffer::handle_sample() I think the
> set_back_actual_end()/pick_next_sample() calls could be hoisted out of
> the "if" :)
>

Done!


>
> > > - referenceProcessor.cpp:261: the change should add logging about
> > > the number of references encountered, maybe after the corresponding
> > > "JNI weak reference count" log message.
> > Just to double check, are you saying that you'd like to have the heap
> > sampler to keep in store how many sampled objects were encountered in
> > the HeapMonitoring::weak_oops_do?
> >    - Would a return of the method with the number of handled
> > references and logging that work?
>
> Yes, it's fine if HeapMonitoring::weak_oops_do() only returned the
> number of processed weak oops.
>

Done also (but I admit I have not tested the output yet) :)


>
> >    - Additionally, would you prefer it in a separate block with its
> > GCTraceTime?
>
> Yes. Both kinds of information is interesting: while the time taken is
> typically more important, the next question would be why, and the
> number of references typically goes a long way there.
>
> See above though, it is probably best to wait a bit.
>

Agreed that I "could" wait but, if it's ok, I'll just refactor/remove this
when we get closer to something final. Either, JDK-8173335
has gone in and I will notice it now or it will soon and I can change it
then.



>
> > > - threadLocalAllocBuffer.cpp:331: one more "TODO"
> > Removed it and added it to my personal todos to look at.
> >
> > >
> > > - threadLocalAllocBuffer.hpp: ThreadLocalAllocBuffer class
> > > documentation should be updated about the sampling additions. I
> > > would have no clue what the difference between "actual_end" and
> > > "end" would be from the given information.
> > If you are talking about the comments in this file, I made them more
> > clear I hope in the new webrev. If it was somewhere else, let me know
> > where to change.
>
> Thanks, that's much better. Maybe a note in the comment of the class
> that ThreadLocalBuffer provides some sampling facility by modifying the
> end() of the TLAB to cause "frequent" calls into the runtime call where
> actual sampling takes place.
>

Done, I think it's better now. Added something about the slow_path_end as
well.


>
> > > - in heapMonitoring.hpp: there are some random comments about some
> > > code that has been grabbed from "util/math/fastmath.[h|cc]". I
> > > can't tell whether this is code that can be used but I assume that
> > > Noam Shazeer is okay with that (i.e. that's all Google code).
> > Jeremy and I double checked and we can release that as I thought. I
> > removed the comment from that piece of code entirely.
>
> Thanks.
>
> > > - heapMonitoring.hpp/cpp static constant naming does not correspond
> > > to Hotspot's. Additionally, in Hotspot static methods are cased
> > > like other methods.
> > I think I fixed the methods to be cased the same way as all other
> > methods. For static constants, I was not sure. I fixed a few other
> > variables but I could not seem to really see a consistent trend for
> > constants. I made them as variables but I'm not sure now.
>
> Sorry again, style is a kind of mess. The goal of my suggestions here
> is only to prevent yet another style creeping in.
>
> > > - in heapMonitoring.cpp there are a few cryptic comments at the top
> > > that seem to refer to internal stuff that should probably be
> > > removed.
> > Sorry about that! My personal todos not cleared out.
>
> I am happy about comments, but I simply did not understand any of that
> and I do not know about other readers as well.
>
> If you think you will remember removing/updating them until the review
> proper (I misunderstood the review situation a little it seems).
>
> > > I did not think through the impact of the TLAB changes on collector
> > > behavior yet (if there are). Also I did not check for problems with
> > > concurrent mark and SATB/G1 (if there are).
> > I would love to know your thoughts on this, I think this is fine. I
>
> I think so too now. No objects are made live out of thin air :)
>
> > see issues with multiple threads right now hitting the stack storage
> > instance. Previous webrevs had a mutex lock here but we took it out
> > for simplificity (and only for now).
>
> :) When looking at this after some thinking I now assume for this
> review that this code is not MT safe at all. There seems to be more
> synchronization missing than just the one for the StackTraceStorage. So
> no comments about this here.
>

I doubled checked a bit (quickly I admit) but it seems that synchronization
in StackTraceStorage is really all you need (all methods lead to a
StackTraceStorage one and can be multithreaded outside of that).
There is a question about the initialization where the
method HeapMonitoring::initialize_profiling is not thread safe.
It would work (famous last words) and not crash if there was a race but we
could add a synchronization point there as well (and therefore on the stop
as well).

But anyway I will really check and do this once we add back synchronization.


>
> Also, this would require some kind of specification of what is allowed
> to be called when and where.
>

Would we specify this with the methods in the jvmti.xml file? We could
start by specifying in each that they are not thread safe but I saw no
mention of that for other methods.


>
> One potentially relevant observation about locking here: depending on
> sampling frequency, StackTraceStore::add_trace() may be rather
> frequently called. I assume that you are going to do measurements :)
>

Though we don't have the TLAB implementation in our code, the compiler
generated sampler uses 2% of overhead with a 512k sampling rate. I can do
real measurements when the code settles and we can see how costly this is
as a TLAB implementation.
However, my theory is that if the rate is 512k, the memory/performance
overhead should be minimal since it is what we saw with our code/workloads
(though not called the same way, we call it essentially at the same rate).
If you have a benchmark you'd like me to test, let me know!

Right now, with my really small test, this does use a bit of overhead even
for a 512k sample size. I don't know yet why, I'm going to see what is
going on.

Finally, I think it is not reasonable to suppose the overhead to be
negligible if the sampling rate used is too low. The user should know that
the lower the rate, the higher the overhead (documentation TODO?).


>
> I am not sure what the expected usage of the API is, but
> StackTraceStore::add_trace() seems to be able to grow without bounds.
> Only a GC truncates them to the live ones. That in itself seems to be
> problematic (GCs can be *wide* apart), and of course some of the API
> methods add to that because they duplicate that unbounded array. Do you
> have any concerns/measurements about this?
>

So, the theory is that yes add_trace can be able to grow without bounds but
it grows at a sample per 512k of allocated space. The stacks it gathers are
currently maxed at 64 (I'd like to expand that to an option to the user
though at some point). So I have no concerns because:

- If really this is taking a lot of space, that means the job is keeping a
lot of objects in memory as well, therefore the entire heap is getting huge
- If this is the case, you will be triggering a GC at some point anyway.

(I'm putting under the rug the issue of "What if we set the rate to 1 for
example" because as you lower the sampling rate, we cannot guarantee low
overhead; the idea behind this feature is to have a means of having
meaningful allocated samples at a low overhead)

I have no measurements really right now but since I now have some
statistics I can poll, I will look a bit more at this question.

I have the same last sentence than above: the user should expect this to
happen if the sampling rate is too small. That probably can be reflected in
the StartHeapSampling as a note : careful this might impact your
performance.


>
> Also, these stack traces might hold on to huge arrays. Any
> consideration of that? Particularly it might be the cause for OOMEs in
> tight memory situations.
>

There is a stack size maximum that is set to 64 so it should not hold huge
arrays. I don't think this is an issue but I can double check with a test
or two.


>
> - please consider adding a safepoint check in
> HeapMonitoring::weak_oops_do to prevent accidental misuse.
>
> - in struct StackTraceStorage, the public fields may also need
> underscores. At least some files in the runtime directory have structs
> with underscored public members (and some don't). The runtime team
> should probably comment on that.
>

Agreed I did not know. I looked around and a lot of structs did not have
them it seemed so I left it as is. I will happily change it if someone
prefers (I was not sure if you really preferred or not, your sentence
seemed to be more a note of "this might need to change but I don't know if
the runtime team enforces that", let me know if I read that wrongly).


>
> - In StackTraceStorage::weak_oops_do(), when examining the
> StackTraceData, maybe it is useful to consider having a non-NULL
> reference outside of the heap's reserved space an error. There should
> be no oop outside of the heap's reserved space ever.
>
> Unless you allow storing random values in StackTraceData::obj, which I
> would not encourage.
>

I suppose you are talking about this part:
if ((value != NULL && Universe::heap()->is_in_reserved(value)) &&
          (is_alive == NULL || is_alive->do_object_b(value))) {

What you are saying is that I could have something like:
if (value != my_non_null_reference &&
          (is_alive == NULL || is_alive->do_object_b(value))) {

Is that what you meant? Is there really a reason to do so? When I look at
the code, is_in_reserved seems like a O(1) method call. I'm not even sure
we can have a NULL value to be honest. I might have to study that to see if
this was not a paranoid test to begin with.

The is_alive code has now morphed due to the comment below.



>
> - HeapMonitoring::weak_oops_do() does not seem to use the
> passed AbstractRefProcTaskExecutor.
>

It did use it:
 size_t HeapMonitoring::weak_oops_do(
    AbstractRefProcTaskExecutor *task_executor,
    BoolObjectClosure* is_alive,
    OopClosure *f,
    VoidClosure *complete_gc) {
  assert(SafepointSynchronize::is_at_safepoint(), "must be at safepoint");

  if (task_executor != NULL) {
    task_executor->set_single_threaded_mode();
  }
  return StackTraceStorage::storage()->weak_oops_do(is_alive, f,
complete_gc);
}

But due to the comment below, I refactored this, so this is no longer here.
Now I have an always true closure that is passed.


> - I do not understand allowing to call this method with a NULL
> complete_gc closure. This would mean that objects referenced from the
> object that is referenced by the StackTraceData are not pulled, meaning
> they would get stale.
>
> - same with is_alive parameter value of NULL
>

So these questions made me look a bit closer at this code. This code I
think was written this way to have a very small impact on the file but you
are right, there is no reason for this here. I've simplified the code by
making in referenceProcessor.cpp a process_HeapSampling method that handles
everything there.

The code allowed NULLs because it depended on where you were coming from
and how the code was being called.

- I added a static always_true variable and pass that now to be more
consistent with the rest of the code.
- I moved the complete_gc into process_phaseHeapSampling now (new method)
and handle the task_executor and the complete_gc there
   - Newbie question: in our code we did a set_single_threaded_mode but I
see that process_phaseJNI does it right before its call, do I need to do it
for the process_phaseHeapSample?

That API is much cleaner (in my mind) and is consistent with what is done
around it (again in my mind).


> - heapMonitoring.cpp:590: I do not completely understand the purpose of
> this code: in the end this results in a fixed value directly dependent
> on the Thread address anyway? In the end this results in a fixed value
> directly dependent on the Thread address anyway?
> IOW, what is special about exactly 20 rounds?
>

So we really want a fast random number generator that has a specific mean
(512k is the default we use). The code uses the thread address as the start
number of the sequence (why not, it is random enough is rationale). Then
instead of just starting there, we prime the sequence and really only start
at the 21st number, it is arbitrary and I have not done a study to see if
we could do more or less of that.

As I have the statistics of the system up and running, I'll run some
experiments to see if this is needed, is 20 good, or not.


>
> - also I would consider stripping a few bits of the threads' address as
> initialization value for your rng. The last three bits (and probably
> more, check whether the Thread object is allocated on special
> boundaries) are always zero for them.
> Not sure if the given "random" value is random enough before/after,
> this method, so just skip that comment if you think this is not
> required.
>

I don't know is the honest answer. I think what is important is that we
tend towards a mean and it is random "enough" to not fall in pitfalls of
only sampling a subset of objects due to their allocation order. I added
that as test to do to see if it changes the mean in any way for the 512k
default value and/or if the first 1000 elements look better.


>
> Some more random nits I did not find a place to put anywhere:
>
> - ThreadLocalAllocBuffer::_extra_space does not seem to be used
> anywhere?
>

Good catch :).


>
> - Maybe indent the declaration of ThreadLocalAllocBuffer::_bytes_until_sample
> to align below the other members of that group.
>
>
Done moved it up a bit to have non static members together and static
separate.


> Thanks,
>   Thomas
>
>
Thanks for your review!
Jc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170627/b067eade/attachment-0001.html>

From serguei.spitsyn at oracle.com  Wed Jun 28 04:26:41 2017
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Tue, 27 Jun 2017 21:26:41 -0700
Subject: Low-Overhead Heap Profiling
In-Reply-To: <CAF9BGBzZCjZedDKfFwX9DmqsQ04_hxsHwZ6UpB1i_r1F-=rT-A@mail.gmail.com>
References: <CAF9BGBx8xFYudhJ=v1_hU5=nwEcC1X0NKRmC=E-RNN6SMs1ozg@mail.gmail.com>
 <CAF9BGBxj7+C6uy4dXHO-3pNorbpO7n22-9JmevM0F11xOo+MnA@mail.gmail.com>
 <CAF9BGBxrdLVmXRTDHj5hQHJhcyrkBM6+ex1TFWi=A5z=V2FSew@mail.gmail.com>
 <CAF9BGBz81YYVUsMLw9sOMR9fH7JQmyb8d_OZ9aJ47EOAqwWhEw@mail.gmail.com>
 <2af975e6-3827-bd57-0c3d-fadd54867a67@oracle.com>
 <365499b6-3f4d-a4df-9e7e-e72a739fb26b@oracle.com>
 <CAF9BGBy0T7A-+1dejYbr6f0d4s0XR-zv-WwHYqXrbe0EtCdsxQ@mail.gmail.com>
 <fb1155c6-ccf8-843c-34fc-fc4a45fc54cc@oracle.com>
 <CAF9BGBxqmXF9OFD4F9wVb-DQOj_UfNNcBhe+ARVxJ5E4cgCC1Q@mail.gmail.com>
 <102c59b8-25b6-8c21-8eef-1de7d0bbf629@oracle.com>
 <CAF9BGBxDaCOxobJi8Uu9kwKPRpnj-KqgWLfJO2z7UX71UnNF4g@mail.gmail.com>
 <CAF9BGByisZchXTsw1=mLETLnOggJShRJBDxFeOUgt+criTTGuQ@mail.gmail.com>
 <1497366226.2829.109.camel@oracle.com>
 <CAF9BGBx8376y_yeMFnMC8PHSqKDaFc5NcbFerQ2up0YVKtSKGg@mail.gmail.com>
 <82d08353-4cdb-c771-d0bc-117341f53dcf@oracle.com>
 <CAF9BGBzZCjZedDKfFwX9DmqsQ04_hxsHwZ6UpB1i_r1F-=rT-A@mail.gmail.com>
Message-ID: <41d75264-bc7c-8901-fa6c-59ca9f0625b1@oracle.com>

An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170627/56e6abf1/attachment-0001.html>

From serguei.spitsyn at oracle.com  Wed Jun 28 06:58:26 2017
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Tue, 27 Jun 2017 23:58:26 -0700
Subject: Low-Overhead Heap Profiling
In-Reply-To: <41d75264-bc7c-8901-fa6c-59ca9f0625b1@oracle.com>
References: <CAF9BGBx8xFYudhJ=v1_hU5=nwEcC1X0NKRmC=E-RNN6SMs1ozg@mail.gmail.com>
 <CAF9BGBxrdLVmXRTDHj5hQHJhcyrkBM6+ex1TFWi=A5z=V2FSew@mail.gmail.com>
 <CAF9BGBz81YYVUsMLw9sOMR9fH7JQmyb8d_OZ9aJ47EOAqwWhEw@mail.gmail.com>
 <2af975e6-3827-bd57-0c3d-fadd54867a67@oracle.com>
 <365499b6-3f4d-a4df-9e7e-e72a739fb26b@oracle.com>
 <CAF9BGBy0T7A-+1dejYbr6f0d4s0XR-zv-WwHYqXrbe0EtCdsxQ@mail.gmail.com>
 <fb1155c6-ccf8-843c-34fc-fc4a45fc54cc@oracle.com>
 <CAF9BGBxqmXF9OFD4F9wVb-DQOj_UfNNcBhe+ARVxJ5E4cgCC1Q@mail.gmail.com>
 <102c59b8-25b6-8c21-8eef-1de7d0bbf629@oracle.com>
 <CAF9BGBxDaCOxobJi8Uu9kwKPRpnj-KqgWLfJO2z7UX71UnNF4g@mail.gmail.com>
 <CAF9BGByisZchXTsw1=mLETLnOggJShRJBDxFeOUgt+criTTGuQ@mail.gmail.com>
 <1497366226.2829.109.camel@oracle.com>
 <CAF9BGBx8376y_yeMFnMC8PHSqKDaFc5NcbFerQ2up0YVKtSKGg@mail.gmail.com>
 <82d08353-4cdb-c771-d0bc-117341f53dcf@oracle.com>
 <CAF9BGBzZCjZedDKfFwX9DmqsQ04_hxsHwZ6UpB1i_r1F-=rT-A@mail.gmail.com>
 <41d75264-bc7c-8901-fa6c-59ca9f0625b1@oracle.com>
Message-ID: <da854917-e450-23c8-a641-b1136845b7ee@oracle.com>

An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170627/67666015/attachment-0001.html>

From serguei.spitsyn at oracle.com  Wed Jun 28 07:26:48 2017
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Wed, 28 Jun 2017 00:26:48 -0700
Subject: Low-Overhead Heap Profiling
In-Reply-To: <da854917-e450-23c8-a641-b1136845b7ee@oracle.com>
References: <CAF9BGBx8xFYudhJ=v1_hU5=nwEcC1X0NKRmC=E-RNN6SMs1ozg@mail.gmail.com>
 <CAF9BGBz81YYVUsMLw9sOMR9fH7JQmyb8d_OZ9aJ47EOAqwWhEw@mail.gmail.com>
 <2af975e6-3827-bd57-0c3d-fadd54867a67@oracle.com>
 <365499b6-3f4d-a4df-9e7e-e72a739fb26b@oracle.com>
 <CAF9BGBy0T7A-+1dejYbr6f0d4s0XR-zv-WwHYqXrbe0EtCdsxQ@mail.gmail.com>
 <fb1155c6-ccf8-843c-34fc-fc4a45fc54cc@oracle.com>
 <CAF9BGBxqmXF9OFD4F9wVb-DQOj_UfNNcBhe+ARVxJ5E4cgCC1Q@mail.gmail.com>
 <102c59b8-25b6-8c21-8eef-1de7d0bbf629@oracle.com>
 <CAF9BGBxDaCOxobJi8Uu9kwKPRpnj-KqgWLfJO2z7UX71UnNF4g@mail.gmail.com>
 <CAF9BGByisZchXTsw1=mLETLnOggJShRJBDxFeOUgt+criTTGuQ@mail.gmail.com>
 <1497366226.2829.109.camel@oracle.com>
 <CAF9BGBx8376y_yeMFnMC8PHSqKDaFc5NcbFerQ2up0YVKtSKGg@mail.gmail.com>
 <82d08353-4cdb-c771-d0bc-117341f53dcf@oracle.com>
 <CAF9BGBzZCjZedDKfFwX9DmqsQ04_hxsHwZ6UpB1i_r1F-=rT-A@mail.gmail.com>
 <41d75264-bc7c-8901-fa6c-59ca9f0625b1@oracle.com>
 <da854917-e450-23c8-a641-b1136845b7ee@oracle.com>
Message-ID: <362d8353-87ef-f5d2-48f7-b2398c97cb32@oracle.com>

An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170628/2bc11bb7/attachment-0001.html>

From robbin.ehn at oracle.com  Wed Jun 28 07:51:35 2017
From: robbin.ehn at oracle.com (Robbin Ehn)
Date: Wed, 28 Jun 2017 09:51:35 +0200
Subject: Low-Overhead Heap Profiling
In-Reply-To: <CAF9BGBx8376y_yeMFnMC8PHSqKDaFc5NcbFerQ2up0YVKtSKGg@mail.gmail.com>
References: <CAF9BGBx8xFYudhJ=v1_hU5=nwEcC1X0NKRmC=E-RNN6SMs1ozg@mail.gmail.com>
 <CAF9BGBxj7+C6uy4dXHO-3pNorbpO7n22-9JmevM0F11xOo+MnA@mail.gmail.com>
 <CAF9BGBxrdLVmXRTDHj5hQHJhcyrkBM6+ex1TFWi=A5z=V2FSew@mail.gmail.com>
 <CAF9BGBz81YYVUsMLw9sOMR9fH7JQmyb8d_OZ9aJ47EOAqwWhEw@mail.gmail.com>
 <2af975e6-3827-bd57-0c3d-fadd54867a67@oracle.com>
 <365499b6-3f4d-a4df-9e7e-e72a739fb26b@oracle.com>
 <CAF9BGBy0T7A-+1dejYbr6f0d4s0XR-zv-WwHYqXrbe0EtCdsxQ@mail.gmail.com>
 <fb1155c6-ccf8-843c-34fc-fc4a45fc54cc@oracle.com>
 <CAF9BGBxqmXF9OFD4F9wVb-DQOj_UfNNcBhe+ARVxJ5E4cgCC1Q@mail.gmail.com>
 <102c59b8-25b6-8c21-8eef-1de7d0bbf629@oracle.com>
 <CAF9BGBxDaCOxobJi8Uu9kwKPRpnj-KqgWLfJO2z7UX71UnNF4g@mail.gmail.com>
 <CAF9BGByisZchXTsw1=mLETLnOggJShRJBDxFeOUgt+criTTGuQ@mail.gmail.com>
 <1497366226.2829.109.camel@oracle.com>
 <CAF9BGBx8376y_yeMFnMC8PHSqKDaFc5NcbFerQ2up0YVKtSKGg@mail.gmail.com>
Message-ID: <4702c8d2-d740-c7c8-902e-8f73f2f4847e@oracle.com>

Hi,

On 06/21/2017 10:45 PM, JC Beyler wrote:
> Hi all,
> 
> First off: Thanks again to Robbin and Thomas for their reviews :)
> 
> Next, I've uploaded a new webrev:
> http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/
> 
> Here is an update:
> 
> - @Robbin, I forgot to say that yes I need to look at implementing this for the other architectures and testing it before it is all ready to go. Is it common to have it 
> working on all possible combinations or is there a subset that I should be doing first and we can do the others later?

I will change my mind here and proposed a different approach.
Right now the only compiler platform change you have is for the FastTLABRefill.
FastTLABRefill works for all gc except G1 (CMS is deprecated) but only in C1 compiled code.
The performance angle of this is not really relevant, since then you will have C2 compiled code.

So I suggested that we should remove FastTLABRefill completely, I will try to makes this happen but most likely not before my long vacation.

That means you can avoid all platform specific code (as patch is now), correct?

If so just keep the x86 support in there and we remove that if I succeed, plus you can support all platforms when FastTLABRefill=false.

Sounds good?

If you had any other questions I missed please let me know.
I'll look at your latest webrev, and response to that also.

Thanks!

/Robbin


> - I've tested slowdebug, built and ran the JTreg tests I wrote with slowdebug and fixed a few more issues
> - I've refactored a bit of the code following Thomas' comments
>     - I think I've handled all the comments from Thomas (I put comments inline below for the specifics)
> 
> The biggest addition to this webrev is that there is more testing, I've added two tests for looking specifically at the two garbage sampling data Recent vs Frequent:
> http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/test/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorRecentTest.java.patch
>     and
> http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/test/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorFrequentTest.java.patch
> 
> I've also refactored the JNI library a bit: http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c.patch.
>     - Side question: I was looking at trying to make this a multi-file library so you would not have all the code in one spot. It seems this is not really done?
>        - Is there a solution to this? What I really wanted was:
>          - The core library that will help testing be easier
>          - The JNI code for each Java class separate and calling into the core library
> 
> - Following Thomas' comments on statistics, I want to add some quality assurance tests and find that the easiest way would be to have a few counters of what is happening in 
> the sampler and expose that to the user.
>     - I'll be adding that in the next version if no one sees any objections to that.
>     - This will allow me to add a sanity test in JTreg about number of samples and average of sampling rate
> 
> @Thomas: I had a few questions that I inlined below but I will summarize the "bigger ones" here:
>     - You mentioned constants are not using the right conventions, I looked around and didn't see any convention except normal naming then for static constants. Is that right?
>     - You mentioned putting the weak_oops_do in a separate method and logging, I inlined my answer below and would appreciate your feedback there too.
> 
> Thanks again for your reviews and patience!
> Jc
> 
> PS: I've also inlined my answers to Thomas below:
> 
> On Tue, Jun 13, 2017 at 8:03 AM, Thomas Schatzl <thomas.schatzl at oracle.com <mailto:thomas.schatzl at oracle.com>> wrote:
> 
>     Hi all,
> 
>     On Mon, 2017-06-12 at 11:11 -0700, JC Beyler wrote:
>     > Dear all,
>     >
>     > I've continued working on this and have done the following webrev:
>     > http://cr.openjdk.java.net/~rasbold/8171119/webrev.05/ <http://cr.openjdk.java.net/~rasbold/8171119/webrev.05/>
> 
>     [...]
>     > Things I still need to do:
>     >    - Have to fix that TLAB case for the FastTLABRefill
>     >    - Have to start looking at the data to see that it is consistent
>     > and does gather the right samples, right frequency, etc.
>     >    - Have to check the GC elements and what that produces
>     >    - Run a slowdebug run and ensure I fixed all those issues you saw
>     > Robbin
>     >
>     > Thanks for looking at the webrev and have a great week!
> 
>        scratching a bit on the surface of this change, so apologies for
>     rather shallow comments:
> 
>     - macroAssembler_x86.cpp:5604: while this is compiler code, and I am
>     not sure this is final, please avoid littering the code with TODO
>     remarks :) They tend to be candidates for later wtf moments only.
> 
>     Just file a CR for that.
> 
> 
> Newcomer question: what is a CR and not sure I have the rights to do that yet ? :)
> 
>     - calling HeapMonitoring::do_weak_oops() (which should probably be
>     called weak_oops_do() like other similar methods) only if string
>     deduplication is enabled (in g1CollectedHeap.cpp:4511) seems wrong.
> 
> 
>     The call should be at least around 6 lines up outside the if.
> 
>     Preferentially in a method like process_weak_jni_handles(), including
>     additional logging. (No new (G1) gc phase without minimal logging :)).
> 
> 
> Done but really not sure because:
> 
> I put for logging:
>    log_develop_trace(gc, freelist)("G1ConcRegionFreeing [other] : heap monitoring");
> 
> Since weak_jni_handles didn't have logging for me to be inspired from, I did that but unconvinced this is what should be done.
> 
> 
>     Seeing the changes in referenceProcess.cpp, you need to add the call to
>     HeapMonitoring::do_weak_oops() exactly similar to
>     process_weak_jni_handles() in case there is no reference processing
>     done.
> 
>     - psParallelCompact.cpp:2172 similar to other places where the change
>     adds the call to HeapMonitoring::do_weak_oops(), remove the empty line.
> 
> 
> Done from what I can tell in the whole webrev. (The only empty lines I still see are when I maintain an empty line, exception being 
> http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/src/share/vm/gc/shared/collectedHeap.cpp.patch where I think it was "nicer")
> 
> 
>     - the change doubles the size of
>     CollectedHeap::allocate_from_tlab_slow() above the "small and nice"
>     threshold. Maybe it could be refactored a bit.
> 
> 
> Done I think, it looks better to me :).
> 
> 
>     - referenceProcessor.cpp:40: please make sure that the includes are
>     sorted alphabetically (I did not check the other files yet).
> 
> 
> Done and checked all other files normally.
> 
> 
>     - referenceProcessor.cpp:261: the change should add logging about the
>     number of references encountered, maybe after the corresponding "JNI
>     weak reference count" log message.
> 
> 
> Just to double check, are you saying that you'd like to have the heap sampler to keep in store how many sampled objects were encountered in the HeapMonitoring::weak_oops_do?
>     - Would a return of the method with the number of handled references and logging that work?
>     - Additionally, would you prefer it in a separate block with its GCTraceTime?
> 
> 
>     - threadLocalAllocBuffer.cpp:331: one more "TODO"
> 
> 
> Removed it and added it to my personal todos to look at.
> 
> 
>     - threadLocalAllocBuffer.hpp: ThreadLocalAllocBuffer class
>     documentation should be updated about the sampling additions. I would
>     have no clue what the difference between "actual_end" and "end" would
>     be from the given information.
> 
> 
> If you are talking about the comments in this file, I made them more clear I hope in the new webrev. If it was somewhere else, let me know where to change.
> 
> 
>     - threadLocalAllocBuffer.hpp:130: extra whitespace ;)
> 
> 
> Fixed :)
> 
> 
>     - some copyrights :)
> 
> 
> I think I fixed all the issues, if you see specific issues, let me know.
> 
>     - in heapMonitoring.hpp: there are some random comments about some code
>     that has been grabbed from "util/math/fastmath.[h|cc]". I can't tell
>     whether this is code that can be used but I assume that Noam Shazeer is
>     okay with that (i.e. that's all Google code).
> 
> 
> Jeremy and I double checked and we can release that as I thought. I removed the comment from that piece of code entirely.
> 
> 
>     - heapMonitoring.hpp/cpp static constant naming does not correspond to
>     Hotspot's. Additionally, in Hotspot static methods are cased like other
>     methods.
> 
> 
> I think I fixed the methods to be cased the same way as all other methods. For static constants, I was not sure. I fixed a few other variables but I could not seem to 
> really see a consistent trend for constants. I made them as variables but I'm not sure now.
> 
> 
>     - in heapMonitoring.cpp there are a few cryptic comments at the top
>     that seem to refer to internal stuff that should probably be removed.
> 
> 
> Sorry about that! My personal todos not cleared out.
> 
> 
>     I did not think through the impact of the TLAB changes on collector
>     behavior yet (if there are). Also I did not check for problems with
>     concurrent mark and SATB/G1 (if there are).
> 
> 
> I would love to know your thoughts on this, I think this is fine. I see issues with multiple threads right now hitting the stack storage instance. Previous webrevs had a 
> mutex lock here but we took it out for simplificity (and only for now).
> 
> 
>     Thanks,
>        Thomas
> 
> 

From serguei.spitsyn at oracle.com  Wed Jun 28 08:03:57 2017
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Wed, 28 Jun 2017 01:03:57 -0700
Subject: RFR 10 (L): 8183012: Code cleanup in com.sun.tools.jdi
In-Reply-To: <ebb2e14c731c4f37b9d45d6050e4df8d@sap.com>
References: <ebb2e14c731c4f37b9d45d6050e4df8d@sap.com>
Message-ID: <dd1ca4a9-74cf-46f9-e210-cfdb2d2669a9@oracle.com>

An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170628/21776a75/attachment.html>

From robbin.ehn at oracle.com  Wed Jun 28 08:12:06 2017
From: robbin.ehn at oracle.com (Robbin Ehn)
Date: Wed, 28 Jun 2017 10:12:06 +0200
Subject: Low-Overhead Heap Profiling
In-Reply-To: <CAF9BGBwaRu+12iRMm6U+n8j_+3izaEcv1OExdFKqZczEE7SJ0A@mail.gmail.com>
References: <CAF9BGBx8xFYudhJ=v1_hU5=nwEcC1X0NKRmC=E-RNN6SMs1ozg@mail.gmail.com>
 <CAF9BGBxj7+C6uy4dXHO-3pNorbpO7n22-9JmevM0F11xOo+MnA@mail.gmail.com>
 <CAF9BGBxrdLVmXRTDHj5hQHJhcyrkBM6+ex1TFWi=A5z=V2FSew@mail.gmail.com>
 <CAF9BGBz81YYVUsMLw9sOMR9fH7JQmyb8d_OZ9aJ47EOAqwWhEw@mail.gmail.com>
 <2af975e6-3827-bd57-0c3d-fadd54867a67@oracle.com>
 <365499b6-3f4d-a4df-9e7e-e72a739fb26b@oracle.com>
 <CAF9BGBy0T7A-+1dejYbr6f0d4s0XR-zv-WwHYqXrbe0EtCdsxQ@mail.gmail.com>
 <fb1155c6-ccf8-843c-34fc-fc4a45fc54cc@oracle.com>
 <CAF9BGBxqmXF9OFD4F9wVb-DQOj_UfNNcBhe+ARVxJ5E4cgCC1Q@mail.gmail.com>
 <102c59b8-25b6-8c21-8eef-1de7d0bbf629@oracle.com>
 <CAF9BGBxDaCOxobJi8Uu9kwKPRpnj-KqgWLfJO2z7UX71UnNF4g@mail.gmail.com>
 <CAF9BGByisZchXTsw1=mLETLnOggJShRJBDxFeOUgt+criTTGuQ@mail.gmail.com>
 <1497366226.2829.109.camel@oracle.com>
 <CAF9BGBx8376y_yeMFnMC8PHSqKDaFc5NcbFerQ2up0YVKtSKGg@mail.gmail.com>
 <1498215147.2741.34.camel@oracle.com>
 <CAF9BGBwaRu+12iRMm6U+n8j_+3izaEcv1OExdFKqZczEE7SJ0A@mail.gmail.com>
Message-ID: <044f8c75-72f3-79fd-af47-7ee875c071fd@oracle.com>

Hi,

On 06/28/2017 12:04 AM, JC Beyler wrote:
> Dear Thomas et al,
> 
> Here is the newest webrev:
> http://cr.openjdk.java.net/~rasbold/8171119/webrev.07/

You have some more bits to in there but generally this looks good and really nice with more tests.
I'll do and deep dive and re-test this when I get back from my long vacation with whatever patch version you have then.

Also I think it's time you provide incremental (v06->07 changes) as well as complete change-sets.

Thanks, Robbin



> 
> Thomas, I "think" I have answered all your remarks. The summary is:
> 
> - The statistic system is up and provides insight on what the heap sampler is doing
>     - I've noticed that, though the sampling rate is at the right mean, we are missing some samples, I have not yet tracked out why (details below)
> 
> - I've run a tiny benchmark that is the worse case: it is a very tight loop and allocated a small array
>     - In this case, I see no overhead when the system is off so that is a good start :)
>     - I see right now a high overhead in this case when sampling is on. This is not a really too surprising but I'm going to see if this is consistent with our internal 
> implementation. The benchmark is really allocation stressful so I'm not too surprised but I want to do the due diligence.
> 
>   - The statistic system up is up and I have a new test 
> http://cr.openjdk.java.net/~rasbold/8171119/webrev.07/test/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatTest.java.patch
>      - I did a bit of a study about the random generator here, more details are below but basically it seems to work well
> 
>   - I added a capability but since this is the first time doing this, I was not sure I did it right
>     - I did add a test though for it and the test seems to do what I expect (all methods are failing with the JVMTI_ERROR_MUST_POSSESS_CAPABILITY error).
>         - http://cr.openjdk.java.net/~rasbold/8171119/webrev.07/test/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorNoCapabilityTest.java.patch
> 
>   - I still need to figure out what to do about the multi-agent vs single-agent issue
> 
>   - As far as measurements, it seems I still need to look at:
>     - Why we do the 20 random calls first, are they necessary?
>     - Look at the mean of the sampling rate that the random generator does and also what is actually sampled
>     - What is the overhead in terms of memory/performance when on?
> 
> I have inlined my answers, I think I got them all in the new webrev, let me know your thoughts.
> 
> Thanks again!
> Jc
> 
> 
> On Fri, Jun 23, 2017 at 3:52 AM, Thomas Schatzl <thomas.schatzl at oracle.com <mailto:thomas.schatzl at oracle.com>> wrote:
> 
>     Hi,
> 
>     On Wed, 2017-06-21 at 13:45 -0700, JC Beyler wrote:
>     > Hi all,
>     >
>     > First off: Thanks again to Robbin and Thomas for their reviews :)
>     >
>     > Next, I've uploaded a new webrev:
>     > http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/ <http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/>
>     >
>     > Here is an update:
>     >
>     > - @Robbin, I forgot to say that yes I need to look at implementing
>     > this for the other architectures and testing it before it is all
>     > ready to go. Is it common to have it working on all possible
>     > combinations or is there a subset that I should be doing first and we
>     > can do the others later?
>     > - I've tested slowdebug, built and ran the JTreg tests I wrote with
>     > slowdebug and fixed a few more issues
>     > - I've refactored a bit of the code following Thomas' comments
>     >    - I think I've handled all the comments from Thomas (I put
>     > comments inline below for the specifics)
> 
>     Thanks for handling all those.
> 
>     > - Following Thomas' comments on statistics, I want to add some
>     > quality assurance tests and find that the easiest way would be to
>     > have a few counters of what is happening in the sampler and expose
>     > that to the user.
>     >    - I'll be adding that in the next version if no one sees any
>     > objections to that.
>     >    - This will allow me to add a sanity test in JTreg about number of
>     > samples and average of sampling rate
>     >
>     > @Thomas: I had a few questions that I inlined below but I will
>     > summarize the "bigger ones" here:
>     >    - You mentioned constants are not using the right conventions, I
>     > looked around and didn't see any convention except normal naming then
>     > for static constants. Is that right?
> 
>     I looked through https://wiki.openjdk.java.net/display/HotSpot/StyleGui <https://wiki.openjdk.java.net/display/HotSpot/StyleGui>
>     de and the rule is to "follow an existing pattern and must have a
>     distinct appearance from other names". Which does not help a lot I
>     guess :/ The GC team started using upper camel case, e.g.
>     SomeOtherConstant, but very likely this is probably not applied
>     consistently throughout. So I am fine with not adding another style
>     (like kMaxStackDepth with the "k" in front with some unknown meaning)
>     is fine.
> 
>     (Chances are you will find that style somewhere used anyway too,
>     apologies if so :/)
> 
> 
> Thanks for that link, now I know where to look. I used the upper camel case in my code as well then :) I should have gotten them all.
> 
> 
>      > PS: I've also inlined my answers to Thomas below:
>      >
>      > On Tue, Jun 13, 2017 at 8:03 AM, Thomas Schatzl <thomas.schatzl at oracl
>      > e.com <http://e.com>> wrote:
>      > > Hi all,
>      > >
>      > > On Mon, 2017-06-12 at 11:11 -0700, JC Beyler wrote:
>      > > > Dear all,
>      > > >
>      > > > I've continued working on this and have done the following
>      > > webrev:
>      > > > http://cr.openjdk.java.net/~rasbold/8171119/webrev.05/ <http://cr.openjdk.java.net/~rasbold/8171119/webrev.05/>
>      > >
>      > > [...]
>      > > > Things I still need to do:
>      > > >    - Have to fix that TLAB case for the FastTLABRefill
>      > > >    - Have to start looking at the data to see that it is
>      > > consistent and does gather the right samples, right frequency, etc.
>      > > >    - Have to check the GC elements and what that produces
>      > > >    - Run a slowdebug run and ensure I fixed all those issues you
>      > > saw > Robbin
>      > > >
>      > > > Thanks for looking at the webrev and have a great week!
>      > >
>      > >   scratching a bit on the surface of this change, so apologies for
>      > > rather shallow comments:
>      > >
>      > > - macroAssembler_x86.cpp:5604: while this is compiler code, and I
>      > > am not sure this is final, please avoid littering the code with
>      > > TODO remarks :) They tend to be candidates for later wtf moments
>      > > only.
>      > >
>      > > Just file a CR for that.
>      > >
>      > Newcomer question: what is a CR and not sure I have the rights to do
>      > that yet ? :)
> 
>     Apologies. CR is a change request, this suggests to file a bug in the
>     bug tracker. And you are right, you can't just create a new account in
>     the OpenJDK JIRA yourselves. :(
> 
> 
> Ok good to know, I'll continue with my own todo list but I'll work hard on not letting it slip in the webrevs anymore :)
> 
> 
>     I was mostly referring to the "... but it is a TODO" part of that
>     comment in macroassembler_x86.cpp. Comments about the why of the code
>     are appreciated. 
> 
> 
>     [Note that I now understand that this is to some degree still work in
>     progress. As long as the final changeset does no contain TODO's I am
>     fine (and it's not a hard objection, rather their use in "final" code
>     is typically limited in my experience)]
> 
>     5603   // Currently, if this happens, just set back the actual end to
>     where it was.
>     5604   // We miss a chance to sample here.
> 
>     Would be okay, if explaining "this" and the "why" of missing a chance
>     to sample here would be best.
> 
>     Like maybe:
> 
>     // If we needed to refill TLABs, just set the actual end point to
>     // the end of the TLAB again. We do not sample here although we could.
> 
> Done with your comment, it works well in my mind.
> 
>     I am not sure whether "miss a chance to sample" meant "we could, but
>     consciously don't because it's not that useful" or "it would be
>     necessary but don't because it's too complicated to do.".
> 
>     Looking at the original comment once more, I am also not sure if that
>     comment shouldn't referring to the "end" variable (not actual_end)
>     because that's the variable that is responsible for taking the sampling
>     path? (Going from the member description of ThreadLocalAllocBuffer).
> 
> 
> I've moved this code and it no longer shows up here but the rationale and answer was:
> 
> So.. Yes, end is the variable provoking the sampling. Actual end is the actual end of the TLAB.
> 
> What was happening here is that the code is resetting _end to point towards the end of the new TLAB. Because, we now have the end for sampling and _actual_end for the 
> actual end, we need to update the actual_end as well.
> 
> Normally, were we to do the real work here, we would calculate the (end - start) offset, then do:
> 
> - Set the new end to : start + (old_end - old_start)
> - Set the actual end like we do here now where it because it is the actual end.
> 
> Why is this not done here now anymore?
>    - I was still debating which path to take:
>       - Do it in the fast refill code, it has its perks:
>           - In a world where fast refills are happening all the time or a lot, we can augment there the code to do the sampling
>       - Remember what we had as an end before leaving the slowpath and check on return
>           - This is what I'm doing now, it removes the need to go fix up all fast refill paths but if you remain in fast refill paths, you won't get sampling. I have to 
> think of the consequences of that, maybe a future change later on?
>              - I have the statistics now so I'm going to study that
>                 -> By the way, though my statistics are showing I'm missing some samples, if I turn off FastTlabRefill, it is the same loss so for now, it seems this does 
> not occur in my simple test.
> 
> 
> 
>     But maybe I am only confused and it's best to just leave the comment
>     away. :)
> 
>     Thinking about it some more, doesn't this not-sampling in this case
>     mean that sampling does not work in any collector that does inline TLAB
>     allocation at the moment? (Or is inline TLAB alloc automatically
>     disabled with sampling somehow?) 
> 
> 
>     That would indeed be a bigger TODO then :)
> 
> 
> Agreed, this remark made me think that perhaps as a first step the new way of doing it is better but I did have to:
>   - Remove the const of the ThreadLocalBuffer remaining and hard_end methods
>   - Move hard_end out of the header file to have a bit more logic there
> 
> Please let me know what you think of that and if you prefer it this way or changing the fast refills. (I prefer this way now because it is more incremental).
> 
> 
>     > > - calling HeapMonitoring::do_weak_oops() (which should probably be
>     > > called weak_oops_do() like other similar methods) only if string
>     > > deduplication is enabled (in g1CollectedHeap.cpp:4511) seems wrong.
>     >
>     > The call should be at least around 6 lines up outside the if.
>     >
>     > Preferentially in a method like process_weak_jni_handles(), including
>     > additional logging. (No new (G1) gc phase without minimal logging
>     > :)).
>     > Done but really not sure because:
>     >
>     > I put for logging:
>     >   log_develop_trace(gc, freelist)("G1ConcRegionFreeing [other] : heap
>     > monitoring");
> 
>     I would think that "gc, ref" would be more appropriate log tags for
>     this similar to jni handles.
>     (I am als not sure what weak reference handling has to do with
>     G1ConcRegionFreeing, so I am a bit puzzled)
> 
> 
> I was not sure what to put for the tags or really as the message. I cleaned it up a bit now to:
>     log_develop_trace(gc, ref)("HeapSampling [other] : heap monitoring processing");
> 
> 
> 
>     > Since weak_jni_handles didn't have logging for me to be inspired
>     > from, I did that but unconvinced this is what should be done.
> 
>     The JNI handle processing does have logging, but only in
>     ReferenceProcessor::process_discovered_references(). In
>     process_weak_jni_handles() only overall time is measured (in a G1
>     specific way, since only G1 supports disabling reference procesing) :/
> 
>     The code in ReferenceProcessor prints both time taken
>     referenceProcessor.cpp:254, as well as the count, but strangely only in
>     debug VMs.
> 
>     I have no idea why this logging is that unimportant to only print that
>     in a debug VM. However there are reviews out for changing this area a
>     bit, so it might be useful to wait for that (JDK-8173335).
> 
> 
> I cleaned it up a bit anyway and now it returns the count of objects that are in the system.
> 
> 
>     > > - the change doubles the size of
>     > > CollectedHeap::allocate_from_tlab_slow() above the "small and nice"
>     > > threshold. Maybe it could be refactored a bit.
>     > Done I think, it looks better to me :).
> 
>     In ThreadLocalAllocBuffer::handle_sample() I think the
>     set_back_actual_end()/pick_next_sample() calls could be hoisted out of
>     the "if" :)
> 
> 
> Done!
> 
> 
>     > > - referenceProcessor.cpp:261: the change should add logging about
>     > > the number of references encountered, maybe after the corresponding
>     > > "JNI weak reference count" log message.
>     > Just to double check, are you saying that you'd like to have the heap
>     > sampler to keep in store how many sampled objects were encountered in
>     > the HeapMonitoring::weak_oops_do?
>     >    - Would a return of the method with the number of handled
>     > references and logging that work?
> 
>     Yes, it's fine if HeapMonitoring::weak_oops_do() only returned the
>     number of processed weak oops.
> 
> 
> Done also (but I admit I have not tested the output yet) :)
> 
> 
>     >    - Additionally, would you prefer it in a separate block with its
>     > GCTraceTime?
> 
>     Yes. Both kinds of information is interesting: while the time taken is
>     typically more important, the next question would be why, and the
>     number of references typically goes a long way there.
> 
>     See above though, it is probably best to wait a bit.
> 
> 
> Agreed that I "could" wait but, if it's ok, I'll just refactor/remove this when we get closer to something final. Either, JDK-8173335
> has gone in and I will notice it now or it will soon and I can change it then.
> 
> 
>     > > - threadLocalAllocBuffer.cpp:331: one more "TODO"
>     > Removed it and added it to my personal todos to look at.
>     >  
>     > >
>     > > - threadLocalAllocBuffer.hpp: ThreadLocalAllocBuffer class
>     > > documentation should be updated about the sampling additions. I
>     > > would have no clue what the difference between "actual_end" and
>     > > "end" would be from the given information.
>     > If you are talking about the comments in this file, I made them more
>     > clear I hope in the new webrev. If it was somewhere else, let me know
>     > where to change.
> 
>     Thanks, that's much better. Maybe a note in the comment of the class
>     that ThreadLocalBuffer provides some sampling facility by modifying the
>     end() of the TLAB to cause "frequent" calls into the runtime call where
>     actual sampling takes place.
> 
> 
> Done, I think it's better now. Added something about the slow_path_end as well.
> 
> 
>     > > - in heapMonitoring.hpp: there are some random comments about some
>     > > code that has been grabbed from "util/math/fastmath.[h|cc]". I
>     > > can't tell whether this is code that can be used but I assume that
>     > > Noam Shazeer is okay with that (i.e. that's all Google code).
>     > Jeremy and I double checked and we can release that as I thought. I
>     > removed the comment from that piece of code entirely.
> 
>     Thanks.
> 
>     > > - heapMonitoring.hpp/cpp static constant naming does not correspond
>     > > to Hotspot's. Additionally, in Hotspot static methods are cased
>     > > like other methods.
>     > I think I fixed the methods to be cased the same way as all other
>     > methods. For static constants, I was not sure. I fixed a few other
>     > variables but I could not seem to really see a consistent trend for
>     > constants. I made them as variables but I'm not sure now.
> 
>     Sorry again, style is a kind of mess. The goal of my suggestions here
>     is only to prevent yet another style creeping in.
> 
>     > > - in heapMonitoring.cpp there are a few cryptic comments at the top
>     > > that seem to refer to internal stuff that should probably be
>     > > removed.
>     > Sorry about that! My personal todos not cleared out.
> 
>     I am happy about comments, but I simply did not understand any of that
>     and I do not know about other readers as well.
> 
>     If you think you will remember removing/updating them until the review
>     proper (I misunderstood the review situation a little it seems).
> 
>     > > I did not think through the impact of the TLAB changes on collector
>     > > behavior yet (if there are). Also I did not check for problems with
>     > > concurrent mark and SATB/G1 (if there are).
>     > I would love to know your thoughts on this, I think this is fine. I
> 
>     I think so too now. No objects are made live out of thin air :)
> 
>     > see issues with multiple threads right now hitting the stack storage
>     > instance. Previous webrevs had a mutex lock here but we took it out
>     > for simplificity (and only for now).
> 
>     :) When looking at this after some thinking I now assume for this
>     review that this code is not MT safe at all. There seems to be more
>     synchronization missing than just the one for the StackTraceStorage. So
>     no comments about this here.
> 
> 
> I doubled checked a bit (quickly I admit) but it seems that synchronization in StackTraceStorage is really all you need (all methods lead to a StackTraceStorage one and can 
> be multithreaded outside of that).
> There is a question about the initialization where the method HeapMonitoring::initialize_profiling is not thread safe.
> It would work (famous last words) and not crash if there was a race but we could add a synchronization point there as well (and therefore on the stop as well).
> 
> But anyway I will really check and do this once we add back synchronization.
> 
> 
>     Also, this would require some kind of specification of what is allowed
>     to be called when and where.
> 
> 
> Would we specify this with the methods in the jvmti.xml file? We could start by specifying in each that they are not thread safe but I saw no mention of that for other methods.
> 
> 
>     One potentially relevant observation about locking here: depending on
>     sampling frequency, StackTraceStore::add_trace() may be rather
>     frequently called. I assume that you are going to do measurements :)
> 
> 
> Though we don't have the TLAB implementation in our code, the compiler generated sampler uses 2% of overhead with a 512k sampling rate. I can do real measurements when the 
> code settles and we can see how costly this is as a TLAB implementation.
> However, my theory is that if the rate is 512k, the memory/performance overhead should be minimal since it is what we saw with our code/workloads (though not called the 
> same way, we call it essentially at the same rate).
> If you have a benchmark you'd like me to test, let me know!
> 
> Right now, with my really small test, this does use a bit of overhead even for a 512k sample size. I don't know yet why, I'm going to see what is going on.
> 
> Finally, I think it is not reasonable to suppose the overhead to be negligible if the sampling rate used is too low. The user should know that the lower the rate, the 
> higher the overhead (documentation TODO?).
> 
> 
>     I am not sure what the expected usage of the API is, but
>     StackTraceStore::add_trace() seems to be able to grow without bounds.
>     Only a GC truncates them to the live ones. That in itself seems to be
>     problematic (GCs can be *wide* apart), and of course some of the API
>     methods add to that because they duplicate that unbounded array. Do you
>     have any concerns/measurements about this?
> 
> 
> So, the theory is that yes add_trace can be able to grow without bounds but it grows at a sample per 512k of allocated space. The stacks it gathers are currently maxed at 
> 64 (I'd like to expand that to an option to the user though at some point). So I have no concerns because:
> 
> - If really this is taking a lot of space, that means the job is keeping a lot of objects in memory as well, therefore the entire heap is getting huge
> - If this is the case, you will be triggering a GC at some point anyway.
> 
> (I'm putting under the rug the issue of "What if we set the rate to 1 for example" because as you lower the sampling rate, we cannot guarantee low overhead; the idea behind 
> this feature is to have a means of having meaningful allocated samples at a low overhead)
> 
> I have no measurements really right now but since I now have some statistics I can poll, I will look a bit more at this question.
> 
> I have the same last sentence than above: the user should expect this to happen if the sampling rate is too small. That probably can be reflected in the StartHeapSampling 
> as a note : careful this might impact your performance.
> 
> 
>     Also, these stack traces might hold on to huge arrays. Any
>     consideration of that? Particularly it might be the cause for OOMEs in
>     tight memory situations.
> 
> 
> There is a stack size maximum that is set to 64 so it should not hold huge arrays. I don't think this is an issue but I can double check with a test or two.
> 
> 
>     - please consider adding a safepoint check in
>     HeapMonitoring::weak_oops_do to prevent accidental misuse.
> 
>     - in struct StackTraceStorage, the public fields may also need
>     underscores. At least some files in the runtime directory have structs
>     with underscored public members (and some don't). The runtime team
>     should probably comment on that.
> 
> 
> Agreed I did not know. I looked around and a lot of structs did not have them it seemed so I left it as is. I will happily change it if someone prefers (I was not sure if 
> you really preferred or not, your sentence seemed to be more a note of "this might need to change but I don't know if the runtime team enforces that", let me know if I read 
> that wrongly).
> 
> 
>     - In StackTraceStorage::weak_oops_do(), when examining the
>     StackTraceData, maybe it is useful to consider having a non-NULL
>     reference outside of the heap's reserved space an error. There should
>     be no oop outside of the heap's reserved space ever.
> 
>     Unless you allow storing random values in StackTraceData::obj, which I
>     would not encourage.
> 
> 
> I suppose you are talking about this part:
> if ((value != NULL && Universe::heap()->is_in_reserved(value)) &&
>            (is_alive == NULL || is_alive->do_object_b(value))) {
> 
> What you are saying is that I could have something like:
> if (value != my_non_null_reference &&
>            (is_alive == NULL || is_alive->do_object_b(value))) {
> 
> Is that what you meant? Is there really a reason to do so? When I look at the code, is_in_reserved seems like a O(1) method call. I'm not even sure we can have a NULL value 
> to be honest. I might have to study that to see if this was not a paranoid test to begin with.
> 
> The is_alive code has now morphed due to the comment below.
> 
> 
> 
>     - HeapMonitoring::weak_oops_do() does not seem to use the
>     passed AbstractRefProcTaskExecutor.
> 
> 
> It did use it:
>   size_t HeapMonitoring::weak_oops_do(
>      AbstractRefProcTaskExecutor *task_executor,
>      BoolObjectClosure* is_alive,
>      OopClosure *f,
>      VoidClosure *complete_gc) {
>    assert(SafepointSynchronize::is_at_safepoint(), "must be at safepoint");
> 
>    if (task_executor != NULL) {
>      task_executor->set_single_threaded_mode();
>    }
>    return StackTraceStorage::storage()->weak_oops_do(is_alive, f, complete_gc);
> }
> 
> But due to the comment below, I refactored this, so this is no longer here. Now I have an always true closure that is passed.
> 
> 
>     - I do not understand allowing to call this method with a NULL
>     complete_gc closure. This would mean that objects referenced from the
>     object that is referenced by the StackTraceData are not pulled, meaning
>     they would get stale.
> 
>     - same with is_alive parameter value of NULL
> 
> 
> So these questions made me look a bit closer at this code. This code I think was written this way to have a very small impact on the file but you are right, there is no 
> reason for this here. I've simplified the code by making in referenceProcessor.cpp a process_HeapSampling method that handles everything there.
> 
> The code allowed NULLs because it depended on where you were coming from and how the code was being called.
> 
> - I added a static always_true variable and pass that now to be more consistent with the rest of the code.
> - I moved the complete_gc into process_phaseHeapSampling now (new method) and handle the task_executor and the complete_gc there
>     - Newbie question: in our code we did a set_single_threaded_mode but I see that process_phaseJNI does it right before its call, do I need to do it for the 
> process_phaseHeapSample?
> That API is much cleaner (in my mind) and is consistent with what is done around it (again in my mind).
> 
> 
>     - heapMonitoring.cpp:590: I do not completely understand the purpose of
>     this code: in the end this results in a fixed value directly dependent
>     on the Thread address anyway? In the end this results in a fixed value
>     directly dependent on the Thread address anyway?
>     IOW, what is special about exactly 20 rounds?
> 
> 
> So we really want a fast random number generator that has a specific mean (512k is the default we use). The code uses the thread address as the start number of the sequence 
> (why not, it is random enough is rationale). Then instead of just starting there, we prime the sequence and really only start at the 21st number, it is arbitrary and I have 
> not done a study to see if we could do more or less of that.
> 
> As I have the statistics of the system up and running, I'll run some experiments to see if this is needed, is 20 good, or not.
> 
> 
>     - also I would consider stripping a few bits of the threads' address as
>     initialization value for your rng. The last three bits (and probably
>     more, check whether the Thread object is allocated on special
>     boundaries) are always zero for them.
>     Not sure if the given "random" value is random enough before/after,
>     this method, so just skip that comment if you think this is not
>     required.
> 
> 
> I don't know is the honest answer. I think what is important is that we tend towards a mean and it is random "enough" to not fall in pitfalls of only sampling a subset of 
> objects due to their allocation order. I added that as test to do to see if it changes the mean in any way for the 512k default value and/or if the first 1000 elements look 
> better.
> 
> 
>     Some more random nits I did not find a place to put anywhere:
> 
>     - ThreadLocalAllocBuffer::_extra_space does not seem to be used
>     anywhere?
> 
> 
> Good catch :).
> 
> 
>     - Maybe indent the declaration of ThreadLocalAllocBuffer::_bytes_until_sample to align below the other members of that group.
> 
> 
> Done moved it up a bit to have non static members together and static separate.
> 
>     Thanks,
>        Thomas
> 
> 
> Thanks for your review!
> Jc
> 

From christoph.langer at sap.com  Wed Jun 28 08:43:36 2017
From: christoph.langer at sap.com (Langer, Christoph)
Date: Wed, 28 Jun 2017 08:43:36 +0000
Subject: RFR 10 (L): 8183012: Code cleanup in com.sun.tools.jdi
In-Reply-To: <dd1ca4a9-74cf-46f9-e210-cfdb2d2669a9@oracle.com>
References: <ebb2e14c731c4f37b9d45d6050e4df8d@sap.com>
 <dd1ca4a9-74cf-46f9-e210-cfdb2d2669a9@oracle.com>
Message-ID: <519e7b050c1b4806956dcc479035ebc2@sap.com>

Hi Serguei,

thanks for the review. The reordering of the imports was done by my Eclipse tooling. And as it builds (and also jtreg did not show a regression (on Windows)), I would suspect it is ok.

Best regards
Christoph

From: serguei.spitsyn at oracle.com [mailto:serguei.spitsyn at oracle.com]
Sent: Mittwoch, 28. Juni 2017 10:04
To: Langer, Christoph <christoph.langer at sap.com>; serviceability-dev at openjdk.java.net
Subject: Re: RFR 10 (L): 8183012: Code cleanup in com.sun.tools.jdi

Hi Christoph,


It looks good to me.
I do not have time to check the imports and their sorting.


Thanks a lot for the cleanup!
Serguei


On 6/27/17 12:09, Langer, Christoph wrote:
Hi,

I?ve got another contribution for cleaning up the jdk.jdi classes. This time it?s about package com.sun.tools.jdi.

Bug: https://bugs.openjdk.java.net/browse/JDK-8183012
Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8183012.0/<http://cr.openjdk.java.net/%7Eclanger/webrevs/8183012.0/>

What I?ve done:
I?ve largely cleaned up the import statements (removing obsolete imports, expanding ?*? wildcards, sorting). Furthermore I did quite a few whitespace cleanups to make the code look nicer. And I removed the types in constructors of typed classes.

The more interesting stuff which should probably closely be reviewed is the following:
a) Remove unnecessary casts in BooleanValueImpl.java, ByteValueImpl.java, CharValueImpl.java, FloatValueImpl.java, LongValueImpl.java, PacketStream.java, ShortValueImpl.java
b) Remove redundant super interfaces in the class declarations of ClassLoaderReferenceImpl.java, ConnectorImpl.java, RawCommandLineLauncher.java, SunCommandLineLauncher.java, ThreadGroupReferenceImpl.java, ThreadReferenceImpl.java
c) ObjectReferenceImpl.java: Remove some code in void validateClassMethodInvocation(Method method, int options). Some very old comment suggests that the code was still there because nobody divined what it was useful for. But I couldn?t find anything that seems to be really useful here, except if the caller maybe wants to get some exception if such occurred in the course of resolving the class or something like that.
d) SocketTransportService.java: pull out SocketConnection to an own file SocketConnection.java, pull class SocketTransportServiceCapabilities into anonymous class within SocketTransportService.capabilities()
e) RawCommandLineLauncher.java, SunCommandLineLauncher.java: Modifications to constructing the TransportService object in its constructors

Module jdk.jdi still builds without warnings after my change and I?m currently running the jdi jtreg suite.

Thanks in advance and best regards
Christoph


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170628/99cf48a9/attachment.html>

From serguei.spitsyn at oracle.com  Wed Jun 28 08:45:44 2017
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Wed, 28 Jun 2017 01:45:44 -0700
Subject: RFR 10 (L): 8183012: Code cleanup in com.sun.tools.jdi
In-Reply-To: <519e7b050c1b4806956dcc479035ebc2@sap.com>
References: <ebb2e14c731c4f37b9d45d6050e4df8d@sap.com>
 <dd1ca4a9-74cf-46f9-e210-cfdb2d2669a9@oracle.com>
 <519e7b050c1b4806956dcc479035ebc2@sap.com>
Message-ID: <c3d04aae-a230-d7bc-c9fa-b421868b002b@oracle.com>

An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170628/dd3762b2/attachment.html>

From thomas.stuefe at gmail.com  Wed Jun 28 13:39:45 2017
From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=)
Date: Wed, 28 Jun 2017 15:39:45 +0200
Subject: RFR 10 (L): 8183012: Code cleanup in com.sun.tools.jdi
In-Reply-To: <ebb2e14c731c4f37b9d45d6050e4df8d@sap.com>
References: <ebb2e14c731c4f37b9d45d6050e4df8d@sap.com>
Message-ID: <CAA-vtUzV8vt7Gp=-Hu7QOA=Eb2zbKUfTDpJmVO_BZ5X3SvnXfg@mail.gmail.com>

Hi Christoph,

Thanks for the cleanup! Here some minor remarks (like Serguei, I did not
check the expanded imports).

---

com/sun/tools/jdi/InvokableTypeImpl.java:

import com.sun.jdi.VMCannotBeModifiedException; ?Why do we need to import
this type, we do not seem to use it?

---

ObjectReferenceImpl:

the removed code: Looks like part of the checks are missing since a long
time. The remains could be understood as a check that the class for the
method to be invoked exists in the debuggee.  But then, there are no
guarantees anyway that inbetween firing the invoke command to the debuggee
and the debuggee processing the invoke command that class may not be
unloaded, so we may just rely on the jdwp invoke command failing for that
case. So, ok to remove it IMHO.

---

SDE.java:

@SuppressWarnings("unused") ? which member is unused?

---

"SocketTransportService.java: pull out SocketConnection to an own file
SocketConnection.java" - why?

---

com/sun/tools/jdi/VMModifiers.java:

Not touched by your change, but weird: In VMModifiers, some of the
constants share numerical values:

28 public interface VMModifiers {
...
  35     int VOLATILE = 0x00000040;      /* can cache in registers */
  36     int BRIDGE = 0x00000040;        /* Bridge method generated by
compiler */

  37     int TRANSIENT = 0x00000080;     /* not persistant */
  38     int VARARGS = 0x00000080;       /* Method accepts var. args*/
...

So, VOLATILE == BRIDGE and TRANSIENT == VARARGS? This seems wrong, no?



Kind Regards, Thomas




On Tue, Jun 27, 2017 at 9:09 PM, Langer, Christoph <christoph.langer at sap.com
> wrote:

> Hi,
>
>
>
> I?ve got another contribution for cleaning up the jdk.jdi classes. This
> time it?s about package com.sun.tools.jdi.
>
>
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8183012
>
> Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8183012.0/
>
>
>
> What I?ve done:
>
> I?ve largely cleaned up the import statements (removing obsolete imports,
> expanding ?*? wildcards, sorting). Furthermore I did quite a few whitespace
> cleanups to make the code look nicer. And I removed the types in
> constructors of typed classes.
>
>
>
> The more interesting stuff which should probably closely be reviewed is
> the following:
>
> a) Remove unnecessary casts in BooleanValueImpl.java, ByteValueImpl.java,
> CharValueImpl.java, FloatValueImpl.java, LongValueImpl.java,
> PacketStream.java, ShortValueImpl.java
>
> b) Remove redundant super interfaces in the class declarations of
> ClassLoaderReferenceImpl.java, ConnectorImpl.java,
> RawCommandLineLauncher.java, SunCommandLineLauncher.java,
> ThreadGroupReferenceImpl.java, ThreadReferenceImpl.java
>
> c) ObjectReferenceImpl.java: Remove some code in void
> validateClassMethodInvocation(Method method, int options). Some very old
> comment suggests that the code was still there because nobody divined what
> it was useful for. But I couldn?t find anything that seems to be really
> useful here, except if the caller maybe wants to get some exception if such
> occurred in the course of resolving the class or something like that.
>
> d) SocketTransportService.java: pull out SocketConnection to an own file
> SocketConnection.java, pull class SocketTransportServiceCapabilities into
> anonymous class within SocketTransportService.capabilities()
>
> e) RawCommandLineLauncher.java, SunCommandLineLauncher.java: Modifications
> to constructing the TransportService object in its constructors
>
>
>
> Module jdk.jdi still builds without warnings after my change and I?m
> currently running the jdi jtreg suite.
>
>
>
> Thanks in advance and best regards
>
> Christoph
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170628/45d12334/attachment.html>

From christoph.langer at sap.com  Wed Jun 28 14:47:44 2017
From: christoph.langer at sap.com (Langer, Christoph)
Date: Wed, 28 Jun 2017 14:47:44 +0000
Subject: RFR 10 (L): 8183012: Code cleanup in com.sun.tools.jdi
In-Reply-To: <CAA-vtUzV8vt7Gp=-Hu7QOA=Eb2zbKUfTDpJmVO_BZ5X3SvnXfg@mail.gmail.com>
References: <ebb2e14c731c4f37b9d45d6050e4df8d@sap.com>
 <CAA-vtUzV8vt7Gp=-Hu7QOA=Eb2zbKUfTDpJmVO_BZ5X3SvnXfg@mail.gmail.com>
Message-ID: <5b4867b4d420439297e4239f9313b97a@sap.com>

Hi Thomas,

thanks for your review!

> com/sun/tools/jdi/InvokableTypeImpl.java:
> 
> import com.sun.jdi.VMCannotBeModifiedException; ?Why do we need to
> import this type, we do not seem to use it?

This is needed for the documentation, see line 97, the throws documentation.

> ObjectReferenceImpl:
> 
> the removed code: Looks like part of the checks are missing since a long time.
> The remains could be understood as a check that the class for the method to
> be invoked exists in the debuggee.? But then, there are no guarantees
> anyway that inbetween firing the invoke command to the debuggee and the
> debuggee processing the invoke command that class may not be unloaded,
> so we may just rely on the jdwp invoke command failing for that case. So, ok
> to remove it IMHO.

I thought so, too.

> SDE.java:
> 
> @SuppressWarnings("unused") ? which member is unused?

It's about member "njplsEnd". Still it looks cleaner not to remove it but to suppress the warning.

> "SocketTransportService.java: pull out SocketConnection to an own file
> SocketConnection.java" - why?

In the codebase that I was merging the package with, the class SocketConnection needed to be declared public for some reason.
This is only allowed if the code lives in a file which is named like the class. And since I think it's generally not wrong to have classes
in their own file and I find other package private classes of com.sun.tools.jdi their own class files, too, I thought it's a win-win to pull it out. :)
It will ease future merging.

> com/sun/tools/jdi/VMModifiers.java:
> 
> Not touched by your change, but weird: In VMModifiers, some of the
> constants share numerical values:
> 
> 28 public interface VMModifiers {
> ...
> ? 35 ? ? int VOLATILE = 0x00000040; ? ? ?/* can cache in registers */
> ? 36 ? ? int BRIDGE = 0x00000040; ? ? ? ?/* Bridge method generated by compiler
> */
> 
> ? 37 ? ? int TRANSIENT = 0x00000080; ? ? /* not persistant */
> ? 38 ? ? int VARARGS = 0x00000080; ? ? ? /* Method accepts var. args*/
> ...
> 
> So, VOLATILE == BRIDGE and TRANSIENT == VARARGS? This seems wrong, no?

This really looks weird, I agree. But it's out of scope for me to dig further... :)

As I addressed all the points you mentioned, I will consider the change reviewed by you.

Best regards
Christoph


From thomas.stuefe at gmail.com  Wed Jun 28 14:56:01 2017
From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=)
Date: Wed, 28 Jun 2017 16:56:01 +0200
Subject: RFR 10 (L): 8183012: Code cleanup in com.sun.tools.jdi
In-Reply-To: <5b4867b4d420439297e4239f9313b97a@sap.com>
References: <ebb2e14c731c4f37b9d45d6050e4df8d@sap.com>
 <CAA-vtUzV8vt7Gp=-Hu7QOA=Eb2zbKUfTDpJmVO_BZ5X3SvnXfg@mail.gmail.com>
 <5b4867b4d420439297e4239f9313b97a@sap.com>
Message-ID: <CAA-vtUz1ruSkarb55W0nX9G-D43J8+E_uSR39N3DC3LebFngFA@mail.gmail.com>

Hi Christoph,



On Wed, Jun 28, 2017 at 4:47 PM, Langer, Christoph <christoph.langer at sap.com
> wrote:

> Hi Thomas,
>
> thanks for your review!
>
> > com/sun/tools/jdi/InvokableTypeImpl.java:
> >
> > import com.sun.jdi.VMCannotBeModifiedException; ?Why do we need to
> > import this type, we do not seem to use it?
>
> This is needed for the documentation, see line 97, the throws
> documentation.
>
> > ObjectReferenceImpl:
> >
> > the removed code: Looks like part of the checks are missing since a long
> time.
> > The remains could be understood as a check that the class for the method
> to
> > be invoked exists in the debuggee.  But then, there are no guarantees
> > anyway that inbetween firing the invoke command to the debuggee and the
> > debuggee processing the invoke command that class may not be unloaded,
> > so we may just rely on the jdwp invoke command failing for that case.
> So, ok
> > to remove it IMHO.
>
> I thought so, too.
>
> > SDE.java:
> >
> > @SuppressWarnings("unused") ? which member is unused?
>
> It's about member "njplsEnd". Still it looks cleaner not to remove it but
> to suppress the warning.
>
> > "SocketTransportService.java: pull out SocketConnection to an own file
> > SocketConnection.java" - why?
>
> In the codebase that I was merging the package with, the class
> SocketConnection needed to be declared public for some reason.
> This is only allowed if the code lives in a file which is named like the
> class. And since I think it's generally not wrong to have classes
> in their own file and I find other package private classes of
> com.sun.tools.jdi their own class files, too, I thought it's a win-win to
> pull it out. :)
> It will ease future merging.
>
> > com/sun/tools/jdi/VMModifiers.java:
> >
> > Not touched by your change, but weird: In VMModifiers, some of the
> > constants share numerical values:
> >
> > 28 public interface VMModifiers {
> > ...
> >   35     int VOLATILE = 0x00000040;      /* can cache in registers */
> >   36     int BRIDGE = 0x00000040;        /* Bridge method generated by
> compiler
> > */
> >
> >   37     int TRANSIENT = 0x00000080;     /* not persistant */
> >   38     int VARARGS = 0x00000080;       /* Method accepts var. args*/
> > ...
> >
> > So, VOLATILE == BRIDGE and TRANSIENT == VARARGS? This seems wrong, no?
>
> This really looks weird, I agree. But it's out of scope for me to dig
> further... :)
>
> As I addressed all the points you mentioned, I will consider the change
> reviewed by you.
>
> Best regards
> Christoph
>
>
Thanks for the answers, and I am fine with the change, no need for a new
webrev.

Kind Regards, Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170628/85596863/attachment.html>

From chihiro.ito at oracle.com  Wed Jun 28 15:02:11 2017
From: chihiro.ito at oracle.com (chihiro ito)
Date: Thu, 29 Jun 2017 00:02:11 +0900
Subject: [10] RFR 8181647: jhsdb jstack could not output thread name
In-Reply-To: <5945FB4E.9070502@oracle.com>
References: <5937CB5F.4060209@oracle.com>
 <a9a45cd1-8717-ed6b-8169-9543e904c20f@oracle.com>
 <5939133A.9040709@oracle.com> <5940E316.7060104@oracle.com>
 <HE1PR08MB2795EF131855EC452EAF9D3FFFC30@HE1PR08MB2795.eurprd08.prod.outlook.com>
 <5945FB4E.9070502@oracle.com>
Message-ID: <5953C4F3.1000802@oracle.com>

Hi all,

In last week, I've posted review request [1].
Could you possibly review for this following small change? If review is 
ok, please commit this as cito.

[1] 
http://mail.openjdk.java.net/pipermail/serviceability-dev/2017-June/021430.html

Thanks,
Chihiro (Contributer)

On 2017/06/18 13:02, chihiro ito wrote:
> At first I thought to print just each thread name, but I tried to make 
> it as similar as possible to JStack as following.
>
> Mixed mode:
> ----------------- 26476 -----------------
> "main" #1 prio=5 tid=0x00007f6894019000 nid=0x676c waiting on 
> condition [0x00007f689b7ae000]
>    java.lang.Thread.State: TIMED_WAITING (sleeping)
>    JavaThread state: _thread_blocked
> 0x00007f689b185a82    __pthread_cond_timedwait + 0x132
>
> No mixed mode:
> "main" #1 prio=5 tid=0x00007f6894019000 nid=0x676c waiting on 
> condition [0x00007f689b7ae000]
>    java.lang.Thread.State: TIMED_WAITING (sleeping)
>    JavaThread state: _thread_blocked
>  - java.lang.Thread.sleep(long) @bci=0 (Interpreted frame)
>
>
> This change passed a test by jtreg.
>
> SOURCE_HOME=/home/user/repo/jdk10-hs
> jtreg -dir:${SOURCE_HOME}/hotspot/test 
> -testjdk:${SOURCE_HOME}/build/linux-x86_64-normal-server-slowdebug/jdk/ serviceability/sa/JhsdbThreadInfoTest.java 
>
> Test results: passed: 1
>
>
> Source:
> diff --git 
> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/oops/OopUtilities.java 
> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/oops/OopUtilities.java 
>
> --- 
> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/oops/OopUtilities.java
> +++ 
> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/oops/OopUtilities.java
> @@ -1,5 +1,5 @@
>  /*
> - * Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights 
> reserved.
> + * Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights 
> reserved.
>   * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
>   *
>   * This code is free software; you can redistribute it and/or modify it
> @@ -59,20 +59,20 @@
>    // parkBlocker field is new since 1.6
>    private static OopField threadParkBlockerField;
>
> +  private static IntField threadPriorityField;
> +  private static BooleanField threadDaemonField;
> +
>    // possible values of java_lang_Thread::ThreadStatus
>    private static int THREAD_STATUS_NEW;
> -  /*
> -    Other enum constants are not needed as of now. Uncomment these as 
> and when needed.
>
> -    private static int THREAD_STATUS_RUNNABLE;
> -    private static int THREAD_STATUS_SLEEPING;
> -    private static int THREAD_STATUS_IN_OBJECT_WAIT;
> -    private static int THREAD_STATUS_IN_OBJECT_WAIT_TIMED;
> -    private static int THREAD_STATUS_PARKED;
> -    private static int THREAD_STATUS_PARKED_TIMED;
> -    private static int THREAD_STATUS_BLOCKED_ON_MONITOR_ENTER;
> -    private static int THREAD_STATUS_TERMINATED;
> -  */
> +  private static int THREAD_STATUS_RUNNABLE;
> +  private static int THREAD_STATUS_SLEEPING;
> +  private static int THREAD_STATUS_IN_OBJECT_WAIT;
> +  private static int THREAD_STATUS_IN_OBJECT_WAIT_TIMED;
> +  private static int THREAD_STATUS_PARKED;
> +  private static int THREAD_STATUS_PARKED_TIMED;
> +  private static int THREAD_STATUS_BLOCKED_ON_MONITOR_ENTER;
> +  private static int THREAD_STATUS_TERMINATED;
>
>    // java.util.concurrent.locks.AbstractOwnableSynchronizer fields
>    private static OopField absOwnSyncOwnerThreadField;
> @@ -229,20 +229,19 @@
>        threadStatusField = (IntField) k.findField("threadStatus", "I");
>        threadParkBlockerField = (OopField) k.findField("parkBlocker",
>                                       "Ljava/lang/Object;");
> +      threadPriorityField = (IntField) k.findField("priority", "I");
> +      threadDaemonField = (BooleanField) k.findField("daemon", "Z");
>        TypeDataBase db = VM.getVM().getTypeDataBase();
>        THREAD_STATUS_NEW = 
> db.lookupIntConstant("java_lang_Thread::NEW").intValue();
> -      /*
> -        Other enum constants are not needed as of now. Uncomment 
> these as and when needed.
>
> -        THREAD_STATUS_RUNNABLE = 
> db.lookupIntConstant("java_lang_Thread::RUNNABLE").intValue();
> -        THREAD_STATUS_SLEEPING = 
> db.lookupIntConstant("java_lang_Thread::SLEEPING").intValue();
> -        THREAD_STATUS_IN_OBJECT_WAIT = 
> db.lookupIntConstant("java_lang_Thread::IN_OBJECT_WAIT").intValue();
> -        THREAD_STATUS_IN_OBJECT_WAIT_TIMED = 
> db.lookupIntConstant("java_lang_Thread::IN_OBJECT_WAIT_TIMED").intValue();
> -        THREAD_STATUS_PARKED = 
> db.lookupIntConstant("java_lang_Thread::PARKED").intValue();
> -        THREAD_STATUS_PARKED_TIMED = 
> db.lookupIntConstant("java_lang_Thread::PARKED_TIMED").intValue();
> -        THREAD_STATUS_BLOCKED_ON_MONITOR_ENTER = 
> db.lookupIntConstant("java_lang_Thread::BLOCKED_ON_MONITOR_ENTER").intValue();
> -        THREAD_STATUS_TERMINATED = 
> db.lookupIntConstant("java_lang_Thread::TERMINATED").intValue();
> -      */
> +      THREAD_STATUS_RUNNABLE = 
> db.lookupIntConstant("java_lang_Thread::RUNNABLE").intValue();
> +      THREAD_STATUS_SLEEPING = 
> db.lookupIntConstant("java_lang_Thread::SLEEPING").intValue();
> +      THREAD_STATUS_IN_OBJECT_WAIT = 
> db.lookupIntConstant("java_lang_Thread::IN_OBJECT_WAIT").intValue();
> +      THREAD_STATUS_IN_OBJECT_WAIT_TIMED = 
> db.lookupIntConstant("java_lang_Thread::IN_OBJECT_WAIT_TIMED").intValue();
> +      THREAD_STATUS_PARKED = 
> db.lookupIntConstant("java_lang_Thread::PARKED").intValue();
> +      THREAD_STATUS_PARKED_TIMED = 
> db.lookupIntConstant("java_lang_Thread::PARKED_TIMED").intValue();
> +      THREAD_STATUS_BLOCKED_ON_MONITOR_ENTER = 
> db.lookupIntConstant("java_lang_Thread::BLOCKED_ON_MONITOR_ENTER").intValue();
> +      THREAD_STATUS_TERMINATED = 
> db.lookupIntConstant("java_lang_Thread::TERMINATED").intValue();
>
>        if (Assert.ASSERTS_ENABLED) {
>          // it is okay to miss threadStatusField, because this was
> @@ -331,4 +330,46 @@
>        return absOwnSyncOwnerThreadField.getValue(oop);
>      }
>    }
> +
> +  public static int threadOopGetPriority(Oop threadOop) {
> +    initThreadFields();
> +    if (threadPriorityField != null) {
> +      return threadPriorityField.getValue(threadOop);
> +    } else {
> +      return 0;
> +    }
> +  }
> +
> +  public static boolean threadOopGetDaemon(Oop threadOop) {
> +    initThreadFields();
> +    if (threadDaemonField != null) {
> +      return threadDaemonField.getValue(threadOop);
> +    } else {
> +      return false;
> +    }
> +  }
> +
> +  public static String threadOopGetThreadStatusName(Oop threadOop) {
> +    int status = OopUtilities.threadOopGetThreadStatus(threadOop);
> +    if( status == THREAD_STATUS_NEW ){
> +      return "NEW";
> +    }else if(status == THREAD_STATUS_RUNNABLE ){
> +      return "RUNNABLE";
> +    }else if(status == THREAD_STATUS_SLEEPING ){
> +      return "TIMED_WAITING (sleeping)";
> +    }else if(status == THREAD_STATUS_IN_OBJECT_WAIT ){
> +      return "WAITING (on object monitor)";
> +    }else if(status == THREAD_STATUS_IN_OBJECT_WAIT_TIMED ){
> +      return "TIMED_WAITING (on object monitor)";
> +    }else if(status == THREAD_STATUS_PARKED ){
> +      return "WAITING (parking)";
> +    }else if(status == THREAD_STATUS_PARKED_TIMED ){
> +      return "TIMED_WAITING (parking)";
> +    }else if(status == THREAD_STATUS_BLOCKED_ON_MONITOR_ENTER ){
> +      return "BLOCKED (on object monitor)";
> +    }else if(status == THREAD_STATUS_TERMINATED ){
> +      return "TERMINATED";
> +    }
> +    return "UNKNOWN";
> +  }
>  }
> diff --git 
> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/JavaThread.java 
> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/JavaThread.java 
>
> --- 
> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/JavaThread.java
> +++ 
> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/JavaThread.java
> @@ -475,4 +475,36 @@
>      return access.getLastSP(addr);
>    }
>
> +
> +  public void printThreadInfoOn(PrintStream out){
> +
> +    Oop threadOop = this.getThreadObj();
> +
> +    out.print("\"");
> +    out.print(this.getThreadName());
> +    out.print("\" #");
> +    out.print(OopUtilities.threadOopGetTID(threadOop));
> +    if( OopUtilities.threadOopGetDaemon(threadOop) ){
> +      out.print(" daemon");
> +    }
> +    out.print(" prio=");
> +    out.print(OopUtilities.threadOopGetPriority(threadOop));
> +    out.print(" tid=");
> + out.print(String.format("0x%016x",this.getAddress().asLongValue()));
> +    out.print(" nid=");
> +    out.print(String.format("0x%x ",this.getOSThread().threadId()));
> +    out.print(getOSThread().getThreadState().getPrintVal());
> +    out.print(" [");
> +    if( this.getLastJavaSP() == null){
> +      out.print(String.format("0x%016x",0L));
> +    } else {
> +      Address maskAddress = this.getLastJavaSP().andWithMask(0xFFF);
> + out.print(this.getLastJavaSP().xorWithMask(maskAddress.asLongValue()));
> +    }
> +    out.println("]");
> +    out.print("   java.lang.Thread.State: ");
> + out.println(OopUtilities.threadOopGetThreadStatusName(threadOop));
> +    out.print("   JavaThread state: _thread_");
> +    out.println(this.getThreadState().toString().toLowerCase());
> +  }
>  }
> diff --git 
> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/OSThread.java 
> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/OSThread.java 
>
> --- 
> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/OSThread.java
> +++ 
> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/OSThread.java
> @@ -1,5 +1,5 @@
>  /*
> - * Copyright (c) 2004, 2013, Oracle and/or its affiliates. All rights 
> reserved.
> + * Copyright (c) 2004, 2017, Oracle and/or its affiliates. All rights 
> reserved.
>   * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
>   *
>   * This code is free software; you can redistribute it and/or modify it
> @@ -33,6 +33,19 @@
>  public class OSThread extends VMObject {
>      private static JIntField interruptedField;
>      private static Field threadIdField;
> +    private static CIntegerField threadStateField;
> +
> +    // ThreadStates read from underlying process
> +    private static int ALLOCATED;
> +    private static int INITIALIZED;
> +    private static int RUNNABLE;
> +    private static int MONITOR_WAIT;
> +    private static int CONDVAR_WAIT;
> +    private static int OBJECT_WAIT;
> +    private static int BREAKPOINTED;
> +    private static int SLEEPING;
> +    private static int ZOMBIE;
> +
>      static {
>          VM.registerVMInitializedObserver(new Observer() {
>              public void update(Observable o, Object data) {
> @@ -45,6 +58,17 @@
>          Type type = db.lookupType("OSThread");
>          interruptedField = type.getJIntField("_interrupted");
>          threadIdField = type.getField("_thread_id");
> +        threadStateField = type.getCIntegerField("_state");
> +
> +        ALLOCATED = db.lookupIntConstant("ALLOCATED").intValue();
> +        INITIALIZED = db.lookupIntConstant("INITIALIZED").intValue();
> +        RUNNABLE = db.lookupIntConstant("RUNNABLE").intValue();
> +        MONITOR_WAIT = db.lookupIntConstant("MONITOR_WAIT").intValue();
> +        CONDVAR_WAIT = db.lookupIntConstant("CONDVAR_WAIT").intValue();
> +        OBJECT_WAIT = db.lookupIntConstant("OBJECT_WAIT").intValue();
> +        BREAKPOINTED = db.lookupIntConstant("BREAKPOINTED").intValue();
> +        SLEEPING = db.lookupIntConstant("SLEEPING").intValue();
> +        ZOMBIE = db.lookupIntConstant("ZOMBIE").intValue();
>      }
>
>      public OSThread(Address addr) {
> @@ -59,4 +83,30 @@
>          return threadIdField.getJInt(addr);
>      }
>
> +    public ThreadState getThreadState() {
> +        int val = (int)threadStateField.getValue(addr);
> +        if (val ==  ALLOCATED) {
> +            return ThreadState.ALLOCATED;
> +        } else if (val ==  INITIALIZED) {
> +            return ThreadState.INITIALIZED;
> +        } else if (val ==  RUNNABLE) {
> +            return ThreadState.RUNNABLE;
> +        } else if (val ==  MONITOR_WAIT) {
> +            return ThreadState.MONITOR_WAIT;
> +        } else if (val ==  CONDVAR_WAIT) {
> +            return ThreadState.CONDVAR_WAIT;
> +        } else if (val ==  OBJECT_WAIT) {
> +            return ThreadState.OBJECT_WAIT;
> +        } else if (val ==  BREAKPOINTED) {
> +            return ThreadState.BREAKPOINTED;
> +        } else if (val ==  BREAKPOINTED) {
> +            return ThreadState.BREAKPOINTED;
> +        } else if (val ==  SLEEPING) {
> +            return ThreadState.SLEEPING;
> +        } else if (val ==  ZOMBIE) {
> +            return ThreadState.ZOMBIE;
> +        } else {
> +            throw new RuntimeException("Illegal thread state " + val);
> +        }
> +    }
>  }
> diff --git 
> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/ThreadState.java 
> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/ThreadState.java 
>
> new file mode 100644
> --- /dev/null
> +++ 
> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/ThreadState.java
> @@ -0,0 +1,60 @@
> +/*
> + * Copyright (c) 2017, Oracle and/or its affiliates. All rights 
> reserved.
> + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> + *
> + * This code is free software; you can redistribute it and/or modify it
> + * under the terms of the GNU General Public License version 2 only, as
> + * published by the Free Software Foundation.
> + *
> + * This code is distributed in the hope that it will be useful, but 
> WITHOUT
> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
> + * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
> + * version 2 for more details (a copy is included in the LICENSE file 
> that
> + * accompanied this code).
> + *
> + * You should have received a copy of the GNU General Public License 
> version
> + * 2 along with this work; if not, write to the Free Software 
> Foundation,
> + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
> + *
> + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 
> 94065 USA
> + * or visit www.oracle.com if you need additional information or have 
> any
> + * questions.
> + */
> +
> +package sun.jvm.hotspot.runtime;
> +
> +/** This is a type-safe enum mirroring the ThreadState enum in
> + osThread.hpp. The conversion between the underlying ints
> + and these values is done in OSThread. */
> +
> +public class ThreadState {
> +
> +    private String printVal;
> +
> +    /** Memory has been allocated but not initialized */
> +    public static final ThreadState ALLOCATED = new 
> ThreadState("allocated");
> +    /** The thread has been initialized but yet started */
> +    public static final ThreadState INITIALIZED = new 
> ThreadState("initialized");
> +    /** Has been started and is runnable, but not necessarily running */
> +    public static final ThreadState RUNNABLE = new 
> ThreadState("runnable");
> +    /** Waiting on a contended monitor lock */
> +    public static final ThreadState MONITOR_WAIT = new 
> ThreadState("waiting for monitor entry");
> +    /** Waiting on a condition variable */
> +    public static final ThreadState CONDVAR_WAIT = new 
> ThreadState("waiting on condition");
> +    /** Waiting on an Object.wait() call */
> +    public static final ThreadState OBJECT_WAIT = new ThreadState("in 
> Object.wait()");
> +    /** Suspended at breakpoint */
> +    public static final ThreadState BREAKPOINTED = new 
> ThreadState("at breakpoint");
> +    /** Thread.sleep() */
> +    public static final ThreadState SLEEPING = new 
> ThreadState("sleeping");
> +    /** All done, but not reclaimed yet */
> +    public static final ThreadState ZOMBIE = new ThreadState("zombie");
> +
> +    private ThreadState(String printVal){
> +        this.printVal = printVal;
> +    }
> +
> +    public String getPrintVal() {
> +        return printVal;
> +    }
> +}
> diff --git 
> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java 
>
> --- 
> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
> +++ 
> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
> @@ -1,5 +1,5 @@
>  /*
> - * Copyright (c) 2003, 2013, Oracle and/or its affiliates. All rights 
> reserved.
> + * Copyright (c) 2003, 2017, Oracle and/or its affiliates. All rights 
> reserved.
>   * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
>   *
>   * This code is free software; you can redistribute it and/or modify it
> @@ -88,6 +88,10 @@
>                 out.print("----------------- ");
>                 out.print(th);
>                 out.println(" -----------------");
> +               JavaThread jthread = (JavaThread) proxyToThread.get(th);
> +               if (jthread != null) {
> +                  jthread.printThreadInfoOn(out);
> +               }
>                 while (f != null) {
>                    ClosestSymbol sym = f.closestSymbolToPC();
>                    Address pc = f.pc();
> diff --git 
> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java 
> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java 
>
> --- 
> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
> +++ 
> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
> @@ -1,5 +1,5 @@
>  /*
> - * Copyright (c) 2002, 2013, Oracle and/or its affiliates. All rights 
> reserved.
> + * Copyright (c) 2002, 2017, Oracle and/or its affiliates. All rights 
> reserved.
>   * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
>   *
>   * This code is free software; you can redistribute it and/or modify it
> @@ -74,14 +74,7 @@
>              int i = 1;
>              for (JavaThread cur = threads.first(); cur != null; cur = 
> cur.next(), i++) {
>                  if (cur.isJavaThread()) {
> -                    Address sp = cur.getLastJavaSP();
> -                    tty.print("Thread ");
> -                    cur.printThreadIDOn(tty);
> -                    tty.print(": (state = " + cur.getThreadState());
> -                    if (verbose) {
> -                        tty.println(", current Java SP = " + sp);
> -                    }
> -                    tty.println(')');
> +                    cur.printThreadInfoOn(tty);
>                      try {
>                          for (JavaVFrame vf = 
> cur.getLastJavaVFrameDbg(); vf != null; vf = vf.javaSender()) {
>                              Method method = vf.getMethod();
> diff --git a/src/share/vm/runtime/vmStructs.cpp 
> b/src/share/vm/runtime/vmStructs.cpp
> --- a/src/share/vm/runtime/vmStructs.cpp
> +++ b/src/share/vm/runtime/vmStructs.cpp
> @@ -981,6 +981,7 @@
> /************/ \
> \
>    volatile_nonstatic_field(OSThread, _interrupted, 
> jint)                                  \
> +  volatile_nonstatic_field(OSThread, _state, 
> ThreadState)                                 \
> \
> /************************/ \
>    /* OopMap and OopMapSet */ \
> @@ -2186,6 +2187,7 @@
> declare_integer_type(Generation::Name) \
> declare_integer_type(InstanceKlass::ClassState) \
> declare_integer_type(JavaThreadState) \
> + declare_integer_type(ThreadState) \
> declare_integer_type(Location::Type) \
> declare_integer_type(Location::Where) \
> declare_integer_type(Flag::Flags) \
> @@ -2443,6 +2445,20 @@
> declare_constant(JavaThread::_not_terminated) \
> declare_constant(JavaThread::_thread_exiting) \
> \
> + /*******************/ \
> +  /* JavaThreadState 
> */                                                   \
> + /*******************/ \
> + \
> + declare_constant(ALLOCATED) \
> + declare_constant(INITIALIZED) \
> + declare_constant(RUNNABLE) \
> + declare_constant(MONITOR_WAIT) \
> + declare_constant(CONDVAR_WAIT) \
> + declare_constant(OBJECT_WAIT) \
> + declare_constant(BREAKPOINTED) \
> + declare_constant(SLEEPING) \
> + declare_constant(ZOMBIE) \
> + \
> /******************************/ \
>    /* Klass misc. enum constants 
> */                                        \
> /******************************/ \
> diff --git a/test/serviceability/sa/JhsdbThreadInfoTest.java 
> b/test/serviceability/sa/JhsdbThreadInfoTest.java
> new file mode 100644
> --- /dev/null
> +++ b/test/serviceability/sa/JhsdbThreadInfoTest.java
> @@ -0,0 +1,87 @@
> +/*
> + * Copyright (c) 2017, Oracle and/or its affiliates. All rights 
> reserved.
> + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> + *
> + * This code is free software; you can redistribute it and/or modify it
> + * under the terms of the GNU General Public License version 2 only, as
> + * published by the Free Software Foundation.
> + *
> + * This code is distributed in the hope that it will be useful, but 
> WITHOUT
> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
> + * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
> + * version 2 for more details (a copy is included in the LICENSE file 
> that
> + * accompanied this code).
> + *
> + * You should have received a copy of the GNU General Public License 
> version
> + * 2 along with this work; if not, write to the Free Software 
> Foundation,
> + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
> + *
> + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 
> 94065 USA
> + * or visit www.oracle.com if you need additional information or have 
> any
> + * questions.
> + */
> +
> +import jdk.test.lib.apps.LingeredApp;
> +import jdk.test.lib.JDKToolLauncher;
> +import jdk.test.lib.Platform;
> +import jdk.test.lib.process.OutputAnalyzer;
> +import jdk.test.lib.Utils;
> +
> +/*
> + * @test
> + * @library /test/lib
> + * @run main JhsdbThreadInfoTest
> + */
> +public class JhsdbThreadInfoTest {
> +
> +    public static void main(String[] args) throws Exception {
> +
> +        if (!Platform.shouldSAAttach()) {
> +            System.out.println("SA attach not expected to work - test 
> skipped.");
> +            return;
> +        }
> +
> +        LingeredApp app = null;
> +
> +        try {
> +            app = LingeredApp.startApp(Utils.getVmOptions());
> +            System.out.println("Started LingeredApp with pid " + 
> app.getPid());
> +
> +            JDKToolLauncher jhsdbLauncher = 
> JDKToolLauncher.createUsingTestJDK("jhsdb");
> +
> +            jhsdbLauncher.addToolArg("jstack");
> +            jhsdbLauncher.addToolArg("--pid");
> + jhsdbLauncher.addToolArg(Long.toString(app.getPid()));
> +
> +            ProcessBuilder pb = new ProcessBuilder();
> +            pb.command(jhsdbLauncher.getCommand());
> +            Process jhsdb = pb.start();
> +
> +            jhsdb.waitFor();
> +
> +            OutputAnalyzer out = new OutputAnalyzer(jhsdb);
> +
> +            System.out.println(out.getStdout());
> +            System.err.println(out.getStderr());
> +
> +            out.shouldMatch("\".+\" #\\d+ daemon prio=\\d+ 
> tid=0x[0-9a-f]+ nid=0x[0-9a-f]+ .+ \\[0x[0-9a-f]+]");
> +            out.shouldMatch("\"main\" #\\d+ prio=\\d+ tid=0x[0-9a-f]+ 
> nid=0x[0-9a-f]+ .+ \\[0x[0-9a-f]+]");
> +            out.shouldMatch("   java.lang.Thread.State: .+");
> +            out.shouldMatch("   JavaThread state: _thread_.+");
> +
> +            out.shouldNotContain("   java.lang.Thread.State: UNKNOWN");
> +            out.stderrShouldBeEmpty();
> +
> +            System.out.println("Test Completed");
> +
> +
> +        } catch (InterruptedException ie) {
> +            throw new Error("Problem awaiting the child process: " + 
> ie, ie);
> +        } catch (Exception attachE) {
> +            throw new Error("Couldn't start jhsdb, attach to 
> LingeredApp or match ThreadName: " + attachE);
> +
> +        } finally {
> +            LingeredApp.stopApp(app);
> +        }
> +    }
> +}
>
>
> Regards,
> Chihiro
>
>
> On 2017/06/14 16:51, Bernd Eckenfels wrote:
>> I don't understand why this format is totally different from the 
>> normal stack traces? At least the header with the stack names could 
>> be similar?
>>
>> Gruss
>> Bernd
>> -- 
>> http://bernd.eckenfels.net
>> ------------------------------------------------------------------------
>> *From:* serviceability-dev 
>> <serviceability-dev-bounces at openjdk.java.net> on behalf of chihiro 
>> ito <chihiro.ito at oracle.com>
>> *Sent:* Wednesday, June 14, 2017 9:17:42 AM
>> *To:* Jini George; serviceability-dev at openjdk.java.net
>> *Subject:* Re: [10] RFR 8181647: jhsdb jstack could not output thread 
>> name
>> Hi all,
>>
>> I added a test case and modified previous patch including fix the
>> copyright year to 2017.
>> I changed to output Java thread name next the separator lines in "jhsdb
>> jstack --mixed" case as following.
>>
>> ----------------- 32117 -----------------
>> "main"
>> 0x00007f6c8feafa82    __pthread_cond_timedwait + 0x132
>>
>> Could you possibly review for this following small change? If review is
>> ok, please commit this as cito.
>>
>>
>> Source:
>> diff --git
>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
>> --- 
>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
>> +++ 
>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
>> @@ -1,5 +1,5 @@
>>   /*
>> - * Copyright (c) 2003, 2013, Oracle and/or its affiliates. All rights
>> reserved.
>> + * Copyright (c) 2003, 2017, Oracle and/or its affiliates. All rights
>> reserved.
>>    * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
>>    *
>>    * This code is free software; you can redistribute it and/or 
>> modify it
>> @@ -88,6 +88,12 @@
>>                  out.print("----------------- ");
>>                  out.print(th);
>>                  out.println(" -----------------");
>> +               JavaThread jthread = (JavaThread) proxyToThread.get(th);
>> +               if (jthread != null) {
>> +                   out.print("\"");
>> +                   out.print(jthread.getThreadName());
>> +                   out.println("\"");
>> +               }
>>                  while (f != null) {
>>                     ClosestSymbol sym = f.closestSymbolToPC();
>>                     Address pc = f.pc();
>> diff --git
>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java 
>>
>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java 
>>
>> ---
>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java 
>>
>> +++
>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java 
>>
>> @@ -1,5 +1,5 @@
>>   /*
>> - * Copyright (c) 2002, 2013, Oracle and/or its affiliates. All rights
>> reserved.
>> + * Copyright (c) 2002, 2017, Oracle and/or its affiliates. All rights
>> reserved.
>>    * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
>>    *
>>    * This code is free software; you can redistribute it and/or 
>> modify it
>> @@ -75,7 +75,9 @@
>>               for (JavaThread cur = threads.first(); cur != null; cur =
>> cur.next(), i++) {
>>                   if (cur.isJavaThread()) {
>>                       Address sp = cur.getLastJavaSP();
>> -                    tty.print("Thread ");
>> +                    tty.print("\"");
>> +                    tty.print(cur.getThreadName());
>> +                    tty.print("\" nid=");
>>                       cur.printThreadIDOn(tty);
>>                       tty.print(": (state = " + cur.getThreadState());
>>                       if (verbose) {
>> diff --git a/test/serviceability/sa/JhsdbThreadNameTest.java
>> b/test/serviceability/sa/JhsdbThreadNameTest.java
>> new file mode 100644
>> --- /dev/null
>> +++ b/test/serviceability/sa/JhsdbThreadNameTest.java
>> @@ -0,0 +1,107 @@
>> +/*
>> + * Copyright (c) 2017, Oracle and/or its affiliates. All rights 
>> reserved.
>> + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
>> + *
>> + * This code is free software; you can redistribute it and/or modify it
>> + * under the terms of the GNU General Public License version 2 only, as
>> + * published by the Free Software Foundation.
>> + *
>> + * This code is distributed in the hope that it will be useful, but 
>> WITHOUT
>> + * ANY WARRANTY; without even the implied warranty of 
>> MERCHANTABILITY or
>> + * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public 
>> License
>> + * version 2 for more details (a copy is included in the LICENSE 
>> file that
>> + * accompanied this code).
>> + *
>> + * You should have received a copy of the GNU General Public License
>> version
>> + * 2 along with this work; if not, write to the Free Software 
>> Foundation,
>> + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
>> + *
>> + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 
>> 94065 USA
>> + * or visit www.oracle.com <http://www.oracle.com> if you need 
>> additional information or have any
>> + * questions.
>> + */
>> +
>> +import java.util.ArrayList;
>> +import java.util.Arrays;
>> +import java.util.List;
>> +import java.util.function.Consumer;
>> +
>> +import jdk.test.lib.apps.LingeredApp;
>> +import jdk.test.lib.JDKToolLauncher;
>> +import jdk.test.lib.Platform;
>> +import jdk.test.lib.process.OutputAnalyzer;
>> +import jdk.test.lib.Utils;
>> +
>> +/*
>> + * @test
>> + * @library /test/lib
>> + * @run main/othervm JhsdbThreadNameTest
>> + */
>> +public class JhsdbThreadNameTest {
>> +
>> +    private static String notMixedModeThreadNames[] =
>> {"Common-Cleaner", "Signal Dispatcher", "Finalizer", "Reference
>> Handler", "main"};
>> +    private static String mixedModeThreadNames[] = {"C2
>> CompilerThread0", "C1 CompilerThread1", "Sweeper thread", "Service 
>> Thread"};
>> +
>> +    private static void startHsdbJstack(boolean mixed) throws 
>> Exception {
>> +
>> +        LingeredApp app = null;
>> +
>> +        try {
>> +            List<String> vmArgs = new ArrayList<String>();
>> +            vmArgs.add("-Xmx10m");
>> +            vmArgs.addAll(Utils.getVmOptions());
>> +
>> +            app = LingeredApp.startApp(vmArgs);
>> +            System.out.println("Started LingeredApp with pid " +
>> app.getPid());
>> +
>> +            JDKToolLauncher jhsdbLauncher =
>> JDKToolLauncher.createUsingTestJDK("jhsdb");
>> +
>> +            jhsdbLauncher.addToolArg("jstack");
>> +            jhsdbLauncher.addToolArg("--pid");
>> + jhsdbLauncher.addToolArg(Long.toString(app.getPid()));
>> +
>> +            if (mixed) {
>> +                jhsdbLauncher.addToolArg("--mixed");
>> +            }
>> +            ProcessBuilder pb = new ProcessBuilder();
>> +            pb.command(jhsdbLauncher.getCommand());
>> +            Process jhsdb = pb.start();
>> +
>> +            jhsdb.waitFor();
>> +
>> +            OutputAnalyzer out = new OutputAnalyzer(jhsdb);
>> +
>> +
>> Arrays.stream(notMixedModeThreadNames).map(JhsdbThreadNameTest::addQuotationMarks).forEach(out::shouldContain); 
>>
>> +            Consumer<String> testMethod = null;
>> +            if (mixed) {
>> +                testMethod = out::shouldContain;
>> +            } else {
>> +                testMethod = out::shouldNotContain;
>> +            }
>> +
>> Arrays.stream(mixedModeThreadNames).map(JhsdbThreadNameTest::addQuotationMarks).forEach(testMethod); 
>>
>> +
>> +        } catch (InterruptedException ie) {
>> +            throw new Error("Problem awaiting the child process: " +
>> ie, ie);
>> +        } catch (Exception attachE) {
>> +            throw new Error("Couldn't start jhsdb, attach to
>> LingeredApp or match ThreadName: " + attachE);
>> +
>> +        } finally {
>> +            LingeredApp.stopApp(app);
>> +        }
>> +    }
>> +
>> +    private static String addQuotationMarks(String str) {
>> +        return "\"" + str + "\"";
>> +    }
>> +
>> +    public static void main(String[] args) throws Exception {
>> +
>> +        if (!Platform.shouldSAAttach()) {
>> +            System.out.println("SA attach not expected to work - test
>> skipped.");
>> +            return;
>> +        }
>> +
>> +        startHsdbJstack(true);
>> +        startHsdbJstack(false);
>> +    }
>> +}
>>
>>
>> Regards,
>> Chihiro
>>
>>
>> On 2017/06/08 18:04, chihiro ito wrote:
>> > Hi Jini,
>> >
>> > Thank you for your advices. I try to add the test case and modify the
>> > copyright year to 2017.
>> > Basically, I agree with your idea, but I think that the separator line
>> > should finally be the same as the output of the jstack command. I
>> > worry which is better way.
>> >
>> > Thanks,
>> > Chihiro
>> >
>> > On 2017/06/08 16:42, Jini George wrote:
>> >> Hi Chihiro,
>> >>
>> >> Thank you for making this useful change. Your changes look good.
>> >>
>> >> It would be great though if you could add a test case for this. Could
>> >> you also modify the copyright year to 2017 ? One additional
>> >> suggestion: The addition of the thread name makes the separator lines
>> >> unaligned in the pstack/jstack --mixed cases. Like:
>> >>
>> >> ----------------- "Service Thread" nid=16051 -----------------
>> >> and
>> >> ----------------- nid=16052 -----------------
>> >>
>> >> So I am wondering if it would make sense to have the name printed out
>> >> on a separate line to keep the separator lines aligned. But this is a
>> >> nit, and I would leave it to you to decide on this.
>> >>
>> >> Thanks,
>> >> Jini (Not a (R)eviewer)
>> >>
>> >> On 6/7/2017 3:16 PM, chihiro ito wrote:
>> >>> Hi all,
>> >>>
>> >>> I changed to output Java thread name in jhsdb jstack as following.
>> >>>
>> >>> jhsdb jstack --pid 25879
>> >>> "main" nid=25884: (state = BLOCKED)
>> >>>
>> >>> jhsdb jstack --mixed --pid 25879
>> >>> ----------------- "main" nid=25884 -----------------
>> >>>
>> >>> Could you possibly review for this following small change? If review
>> >>> is ok, please commit this as cito.
>> >>>
>> >>> Source:
>> >>> diff --git
>> >>> 
>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
>> >>> 
>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
>> >>> ---
>> >>> 
>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
>> >>> +++
>> >>> 
>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
>> >>> @@ -86,6 +86,13 @@
>> >>>              try {
>> >>>                 CFrame f = cdbg.topFrameForThread(th);
>> >>>                 out.print("----------------- ");
>> >>> +               JavaThread jthread = (JavaThread)
>> >>> proxyToThread.get(th);
>> >>> +               if (jthread != null) {
>> >>> +                   out.print("\"");
>> >>> + out.print(jthread.getThreadName());
>> >>> +                   out.print("\" ");
>> >>> +               }
>> >>> +               out.print("nid=");
>> >>>                 out.print(th);
>> >>>                 out.println(" -----------------");
>> >>>                 while (f != null) {
>> >>> diff --git
>> >>> 
>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java 
>>
>> >>> 
>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java 
>>
>> >>>
>> >>> ---
>> >>> 
>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
>> >>> +++
>> >>> 
>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
>> >>> @@ -75,7 +75,9 @@
>> >>>              for (JavaThread cur = threads.first(); cur != null; cur
>> >>> = cur.next(), i++) {
>> >>>                  if (cur.isJavaThread()) {
>> >>>                      Address sp = cur.getLastJavaSP();
>> >>> -                    tty.print("Thread ");
>> >>> +                    tty.print("\"");
>> >>> + tty.print(cur.getThreadName());
>> >>> +                    tty.print("\" nid=");
>> >>>                      cur.printThreadIDOn(tty);
>> >>>                      tty.print(": (state = " + 
>> cur.getThreadState());
>> >>>                      if (verbose) {
>> >>>
>> >>> Regards,
>> >>> Chihiro
>> >>>
>> >>
>> >
>>
>> -- 
>>
>> Chihiro Ito | Principal Consultant | +81.90.6148.8815
>> Oracle <http://www.oracle.com> Consultant
>> ORACLE Japan | Akasaka Center Bldg. | Motoakasaka 1-3-13 | 1070051
>> Minato-ku, Tokyo, JAPAN
>>
>> Oracle is committed to developing practices and products that help
>> protect the environment <http://www.oracle.com/commitment>
>

-- 

Chihiro Ito | Principal Consultant | +81.90.6148.8815
Oracle <http://www.oracle.com> Consultant
ORACLE Japan | Akasaka Center Bldg. | Motoakasaka 1-3-13 | 1070051 
Minato-ku, Tokyo, JAPAN

Oracle is committed to developing practices and products that help 
protect the environment <http://www.oracle.com/commitment>

From jcbeyler at google.com  Wed Jun 28 21:10:12 2017
From: jcbeyler at google.com (JC Beyler)
Date: Wed, 28 Jun 2017 14:10:12 -0700
Subject: Low-Overhead Heap Profiling
In-Reply-To: <362d8353-87ef-f5d2-48f7-b2398c97cb32@oracle.com>
References: <CAF9BGBx8xFYudhJ=v1_hU5=nwEcC1X0NKRmC=E-RNN6SMs1ozg@mail.gmail.com>
 <CAF9BGBz81YYVUsMLw9sOMR9fH7JQmyb8d_OZ9aJ47EOAqwWhEw@mail.gmail.com>
 <2af975e6-3827-bd57-0c3d-fadd54867a67@oracle.com>
 <365499b6-3f4d-a4df-9e7e-e72a739fb26b@oracle.com>
 <CAF9BGBy0T7A-+1dejYbr6f0d4s0XR-zv-WwHYqXrbe0EtCdsxQ@mail.gmail.com>
 <fb1155c6-ccf8-843c-34fc-fc4a45fc54cc@oracle.com>
 <CAF9BGBxqmXF9OFD4F9wVb-DQOj_UfNNcBhe+ARVxJ5E4cgCC1Q@mail.gmail.com>
 <102c59b8-25b6-8c21-8eef-1de7d0bbf629@oracle.com>
 <CAF9BGBxDaCOxobJi8Uu9kwKPRpnj-KqgWLfJO2z7UX71UnNF4g@mail.gmail.com>
 <CAF9BGByisZchXTsw1=mLETLnOggJShRJBDxFeOUgt+criTTGuQ@mail.gmail.com>
 <1497366226.2829.109.camel@oracle.com>
 <CAF9BGBx8376y_yeMFnMC8PHSqKDaFc5NcbFerQ2up0YVKtSKGg@mail.gmail.com>
 <82d08353-4cdb-c771-d0bc-117341f53dcf@oracle.com>
 <CAF9BGBzZCjZedDKfFwX9DmqsQ04_hxsHwZ6UpB1i_r1F-=rT-A@mail.gmail.com>
 <41d75264-bc7c-8901-fa6c-59ca9f0625b1@oracle.com>
 <da854917-e450-23c8-a641-b1136845b7ee@oracle.com>
 <362d8353-87ef-f5d2-48f7-b2398c97cb32@oracle.com>
Message-ID: <CAF9BGBwwVdiDA1x+otJ_S9qAdoQST+Od=0_rrPi3RSoaOb+QXA@mail.gmail.com>

Hi Serguei,

I'm answering your emails in one spot to make it easier for everyone to
follow. I think I've gathered all your remarks in one spot:

Before I start: No worries for going on vacation, I am also soon going for
a few weeks :)

1) hg import:

Strange, I tried this to be sure I was not doing something wrong:
hg clone http://hg.openjdk.java.net/jdk10/hs jdk10hs-testpatch
bash ./get_source.sh
wget http://cr.openjdk.java.net/~rasbold/8171119/webrev.07/hotspot.patch
hg import ../hotspot.patch
hg import ../hotspot.patch
applying ../hotspot.patch
abort: no username supplied (see "hg help config")
hg status

This worked for me.

Is it by any chance that I recently moved to the hs branch?


2) JVMTI System

Yes this system seems slightly different than the turn on/set your
callbacks and event notifications...

  - We talked about having a callback system but it is for a different use
case:
      - Have a callback for each sampled allocation
         - Theoretically I suppose we could do this here and move the code
from heapMonitoring into a library outside of the JVM to offer the same
effect. I'm not sure on the overhead though but I can do a comparison. I've
had conversations about this and Jeremy was worried that a callback system
would add more overhead instead of a full sampler inside the JVM that you
can poll like we do here

  - Start/stop and initialization/finalization can definitely be separated,
I've just left it as the code was written originally but could move it, it
might simplify a bit the code too
      - Because of our way of doing it, it seems that we keep the
capability on always, though you could imagine that you would only need the
capability for the initialization and all other methods fail/do nothing if
not initialized. That would allow us to disable the capability later.
         - What do you think?

  - I think I will go for single agent, I don't think we care about
multi-agent right now, so I'll stay on this path and leave it on the
onload_solo


3) Capabilities:
  - I had already added the can_sample_heap capability (not sure if you saw
it) and had seen that it does work as I thought it would. I added a JTreg
test to check each method here:
       Search for
Java_MyPackage_HeapMonitorNoCapabilityTest_allSamplingMethodsFail
in
http://cr.openjdk.java.net/~rasbold/8171119/webrev.07/test/serviceability/jvmti/HeapMonitor/libHeapMonitor.c.patch

- I had not put it as onload_solo and had not put the print out for
debugging, so I did do that by following the can_generate_breakpoint_events
as you recommended
- Currently I don't have an associated event because I'm not sure it
applies to here but if you want me to consider it and try it out to see the
overhead of such a change, I'm happy to try

4) Changing structures:
- I've moved jvmtiCallFrame to jvmtiFrameInfo
- For jvmtiStackTrace, it seems that for historical reasons we were using
env_id but I don't in the current code so I removed it. So the only change
between jvmtiStackTrace and jvmtiStackInfo is the size of the object
allocated. Before doing that work, any issues with me adding it in your
opinion?
       - If not but there is more work to make it consistent across the
board, perhaps that should be a first change and then do this on top.

Thanks for all your comments!
Jc




On Wed, Jun 28, 2017 at 12:26 AM, serguei.spitsyn at oracle.com <
serguei.spitsyn at oracle.com> wrote:

> One more comment on the JVMTI spec update (jvmti.xml).
>
> In the Heap Monitoring section new typedefs are defined:
> Call Frame
>
> typedef struct {
>     jint bci;
>     jmethodID method_id;
> } jvmtiCallFrame;
>
> Stack Trace
>
> typedef struct {
>     JNIEnv* env_id;
>     jvmtiCallFrame* frames;
>     jint frame_count;
>     jint size;
>     jlong thread_id;
> } jvmtiStackTrace;
>
>
> I think, it'd make sense to use similar typedefs defined in the Stack
> Frame section:
>   jvmtiFrameInfo and jvmtiStackInfo
>
>
> Thanks,
> Serguei
>
>
>
> On 6/27/17 23:58, serguei.spitsyn at oracle.com wrote:
>
> Hi JC,
>
> Please, find more comments below.
>
>
> On 6/27/17 21:26, serguei.spitsyn at oracle.com wrote:
>
> Hi JC,
>
> Sorry for a latency in reply.
> I'm going to a 4-week vacation.
> But I'll try to help you occasionally when there will be access to the
> Internet.
>
>
>
> On 6/23/17 09:58, JC Beyler wrote:
>
> Hi Serguei and all,
>
> Thanks Serguei for your review. I believe I have not answered many
> questions but raised more but I hope this helps understand what I am trying
> to do.
>
> I have inlined my answers to make it easier to answer everything (and to
> be honest, ask questions).
>
> @Thomas: Thank you for your thorough answer as well, I'll be working on it
> and will answer when I get the fixes and comments in a handled state, most
> likely beginning of next week :)
>
> On Fri, Jun 23, 2017 at 3:01 AM, serguei.spitsyn at oracle.com <
> serguei.spitsyn at oracle.com> wrote:
>
>> Hi JC,
>>
>> > Some initial JVMTI-specific questions/comments.
>>
> . . .
>
> I think, it would make sense to introduce new optional capability,
> something like:
>    can_sample_heap_allocations
>
> I will work on adding that for the next webrev, Robbin had mentioned I
> should and I dropped the ball on that one.
>
>
> I see a couple of more emails from you.
> Will try to help when you have some problems or questions.
>
> > Do you consider this API to be used by a single agent or it is a
> multi-agent feature?
> > What if one agent calls the StartHeapSampling() but another calls one
> of the others.
> > The API specs needs to clarify this.
>
> > I'm not sure you had a chance to carefully think on what JVMTI phases
> are allowed for new functions.
>
> - I am pretty sure I have NOT really thought about this as carefully as
> you have because this is the first time I am doing this and these questions
> already make me go "hmmm" :)
>
>
> Expectable. :)
>
>
> I have a tendency to work through issues and try to keep the design simple
> first. Your question seems to imply I can make this work for a single agent
> world and have a means to explicitly disable it for multi-agent for now.
>
>
> Right.
> I agree, this will allow to make a progress and get more experience first.
>
>
>
> I looked quickly in the prims folder for anything mentioning single agent
> vs multi agent but did not find anything. Does this get handled s this more
> a documentation thing and tough luck for the user.
>
> This page: https://docs.oracle.com/javase/7/docs/platform/jvmti/
> jvmti.html#environments
>
> seems to say the latter. That the documentation should say that this is
> not a multiple agent support (yet?) and there is nothing we can do. Is that
> right?
>
>
>
> You can look at these two functions in the jvmtiManageCapabilities.cpp:
>
>  144 jvmtiCapabilities JvmtiManageCapabilities::init_always_solo_capabilities()
> {
>  145   jvmtiCapabilities jc;
>  146
>  147   memset(&jc, 0, sizeof(jc));
>  148   jc.can_suspend = 1;
>  149   return jc;
>  150 }
>  151
>  152
>  153 jvmtiCapabilities JvmtiManageCapabilities::init_onload_solo_capabilities()
> {
>  154   jvmtiCapabilities jc;
>  155
>  156   memset(&jc, 0, sizeof(jc));
>  157   jc.can_generate_field_modification_events = 1;
>  158   jc.can_generate_field_access_events = 1;
>  159   jc.can_generate_breakpoint_events = 1;
>  160   return jc;
>  161 }
>
>
> The can_suspend is always solo but other 3 capabilities are onload solo.
> Probably, onload solo would work well for your purpose.
> So that it is possible to follow one of these pattern, for example,
> can_generate_breakpoint_events.
> You can check all uses of it in the JVMTI spec and the hotspot sources.
>
> Also, look at the generated file:
>   build/linux-x86_64-normal-server-release/hotspot/variant-server/gensrc/
> jvmtifiles/jvmtiEnter.cpp
>
>   (You may have different kind of build so, just replace the suffix
> "linux-x86_64-normal-server-release" with yours.
>
> You will find a couple of conditions like this:
>
> 2873   if (jvmti_env->get_capabilities()->can_generate_breakpoint_events
> == 0) {
> 2874     return JVMTI_ERROR_MUST_POSSESS_CAPABILITY;
> 2875   }
>
> In your case such checks will be also auto-generated from your version of
> the jvmti.xml.
> The newly generated jvmti.html can be found in the same build folder too.
>
> You may need to look at this JVMTI spec section:
>   http://docs.oracle.com/javase/8/docs/platform/jvmti/jvmti.
> html#capability
>
> on how to add the needed capability.
>
>
> Please, ask questions if you have.
> I hope, somebody will be able to answer, or I'll do it when get a chance.
>
>
> Thanks,
> Serguei
>
>
>
>> > http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/src/sh
>> are/vm/runtime/heapMonitoring.cpp.html
>>
>> > I do not see the variable HeapMonitoring::_enabled is checked in the
>> functions:
>> >     HeapMonitoring::get_live_traces(),
>> >     HeapMonitoring::get_garbage_traces(),
>> >     HeapMonitoring::get_frequent_garbage_traces(),
>> >     HeapMonitoring::release_traces()
>>
>> So it is not checked there because the API allows you to:
>   - Start sampling (which flips the enabled variable)
>   - Stop sampling (which flips back the enabled variable)
>   - Get the sampled traces at that point
>
> I think the API documentation is not clear for this right now, so I'll
> change it for the next webrev. Let me also know if you think this is a bad
> idea.
>
>
> I agree, it is better at least to document this behavior.
>
>
> I thought it would be useful because it would allow you to start/stop the
> sampling and then later on go back and see what was sampled at a later time.
>
>
> I need to think more on this.
> The whole style is different than that used in the JVMTI.
> Normally, each agent does some steps like the following:
>
>   - add the capabilities required by the agent (normally in the ONLOAD
> phase)
>   - set the agent event callbacks
>   - enable/disable event notification mode
>   - relinquish the capabilities that are not needed anymore
>
> In your approach there is no tight association with any agent yet.
> Also, it seems, the heap monitoring initialization/finalization
> are not separated from sampling starting/stopping.
> The heap allocation events are processed by the VM itself (in GC or JVMTI).
>
> It is not clear yet if it could be somehow adjusted to JVMTI style
> and if it is really necessary.
>
>
> A question about a multi-agent feature from above applies here as well.
>>
>
> I'm open to either supporting multi-agent if you think it is important or
> doing whatever can be done to, as a first step, make it a single agent
> support. Let me know how to do that.
>
>
> I do not think it is really important to support multi-agents here.
> However, it is important to make a choice sooner rather than later as it
> impacts the design.
>
>
> Thanks,
> Serguei
>
>   > Thanks,
>   > Serguei
>
> Thank you!
> Jc
>
>
>>
>>
>>
>>
>> On 6/21/17 13:45, JC Beyler wrote:
>>
>> Hi all,
>>
>> First off: Thanks again to Robbin and Thomas for their reviews :)
>>
>> Next, I've uploaded a new webrev:
>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/
>>
>> Here is an update:
>>
>> - @Robbin, I forgot to say that yes I need to look at implementing this
>> for the other architectures and testing it before it is all ready to go. Is
>> it common to have it working on all possible combinations or is there a
>> subset that I should be doing first and we can do the others later?
>> - I've tested slowdebug, built and ran the JTreg tests I wrote with
>> slowdebug and fixed a few more issues
>> - I've refactored a bit of the code following Thomas' comments
>>    - I think I've handled all the comments from Thomas (I put comments
>> inline below for the specifics)
>>
>> The biggest addition to this webrev is that there is more testing, I've
>> added two tests for looking specifically at the two garbage sampling data
>> Recent vs Frequent:
>>    http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/test/
>> serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorRecent
>> Test.java.patch
>>    and
>>    http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/test/
>> serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorFreque
>> ntTest.java.patch
>>
>> I've also refactored the JNI library a bit:
>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/test/
>> serviceability/jvmti/HeapMonitor/libHeapMonitor.c.patch.
>>    - Side question: I was looking at trying to make this a multi-file
>> library so you would not have all the code in one spot. It seems this is
>> not really done?
>>       - Is there a solution to this? What I really wanted was:
>>         - The core library that will help testing be easier
>>         - The JNI code for each Java class separate and calling into the
>> core library
>>
>> - Following Thomas' comments on statistics, I want to add some quality
>> assurance tests and find that the easiest way would be to have a few
>> counters of what is happening in the sampler and expose that to the user.
>>    - I'll be adding that in the next version if no one sees any
>> objections to that.
>>    - This will allow me to add a sanity test in JTreg about number of
>> samples and average of sampling rate
>>
>> @Thomas: I had a few questions that I inlined below but I will summarize
>> the "bigger ones" here:
>>    - You mentioned constants are not using the right conventions, I
>> looked around and didn't see any convention except normal naming then for
>> static constants. Is that right?
>>    - You mentioned putting the weak_oops_do in a separate method and
>> logging, I inlined my answer below and would appreciate your feedback there
>> too.
>>
>> Thanks again for your reviews and patience!
>> Jc
>>
>> PS: I've also inlined my answers to Thomas below:
>>
>> On Tue, Jun 13, 2017 at 8:03 AM, Thomas Schatzl <
>> thomas.schatzl at oracle.com> wrote:
>>
>>> Hi all,
>>>
>>> On Mon, 2017-06-12 at 11:11 -0700, JC Beyler wrote:
>>> > Dear all,
>>> >
>>> > I've continued working on this and have done the following webrev:
>>> > http://cr.openjdk.java.net/~rasbold/8171119/webrev.05/
>>>
>>> [...]
>>> > Things I still need to do:
>>> >    - Have to fix that TLAB case for the FastTLABRefill
>>> >    - Have to start looking at the data to see that it is consistent
>>> > and does gather the right samples, right frequency, etc.
>>> >    - Have to check the GC elements and what that produces
>>> >    - Run a slowdebug run and ensure I fixed all those issues you saw
>>> > Robbin
>>> >
>>> > Thanks for looking at the webrev and have a great week!
>>>
>>>   scratching a bit on the surface of this change, so apologies for
>>> rather shallow comments:
>>>
>>> - macroAssembler_x86.cpp:5604: while this is compiler code, and I am
>>> not sure this is final, please avoid littering the code with TODO
>>> remarks :) They tend to be candidates for later wtf moments only.
>>>
>>> Just file a CR for that.
>>>
>>>
>> Newcomer question: what is a CR and not sure I have the rights to do that
>> yet ? :)
>>
>>
>>> - calling HeapMonitoring::do_weak_oops() (which should probably be
>>> called weak_oops_do() like other similar methods) only if string
>>> deduplication is enabled (in g1CollectedHeap.cpp:4511) seems wrong.
>>
>>
>>> The call should be at least around 6 lines up outside the if.
>>>
>>> Preferentially in a method like process_weak_jni_handles(), including
>>> additional logging. (No new (G1) gc phase without minimal logging :)).
>>>
>>
>> Done but really not sure because:
>>
>> I put for logging:
>>   log_develop_trace(gc, freelist)("G1ConcRegionFreeing [other] : heap
>> monitoring");
>>
>> Since weak_jni_handles didn't have logging for me to be inspired from, I
>> did that but unconvinced this is what should be done.
>>
>>
>>>
>>> Seeing the changes in referenceProcess.cpp, you need to add the call to
>>> HeapMonitoring::do_weak_oops() exactly similar to
>>> process_weak_jni_handles() in case there is no reference processing
>>> done.
>>>
>>> - psParallelCompact.cpp:2172 similar to other places where the change
>>> adds the call to HeapMonitoring::do_weak_oops(), remove the empty line.
>>>
>>
>> Done from what I can tell in the whole webrev. (The only empty lines I
>> still see are when I maintain an empty line, exception being
>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/src/sh
>> are/vm/gc/shared/collectedHeap.cpp.patch where I think it was "nicer")
>>
>>
>>>
>>> - the change doubles the size of
>>> CollectedHeap::allocate_from_tlab_slow() above the "small and nice"
>>> threshold. Maybe it could be refactored a bit.
>>>
>>
>> Done I think, it looks better to me :).
>>
>>
>>>
>>> - referenceProcessor.cpp:40: please make sure that the includes are
>>> sorted alphabetically (I did not check the other files yet).
>>>
>>
>> Done and checked all other files normally.
>>
>>
>>>
>>> - referenceProcessor.cpp:261: the change should add logging about the
>>> number of references encountered, maybe after the corresponding "JNI
>>> weak reference count" log message.
>>>
>>
>> Just to double check, are you saying that you'd like to have the heap
>> sampler to keep in store how many sampled objects were encountered in the
>> HeapMonitoring::weak_oops_do?
>>    - Would a return of the method with the number of handled references
>> and logging that work?
>>    - Additionally, would you prefer it in a separate block with its
>> GCTraceTime?
>>
>>
>>> - threadLocalAllocBuffer.cpp:331: one more "TODO"
>>>
>>
>> Removed it and added it to my personal todos to look at.
>>
>>
>>>
>>> - threadLocalAllocBuffer.hpp: ThreadLocalAllocBuffer class
>>> documentation should be updated about the sampling additions. I would
>>> have no clue what the difference between "actual_end" and "end" would
>>> be from the given information.
>>>
>>
>> If you are talking about the comments in this file, I made them more
>> clear I hope in the new webrev. If it was somewhere else, let me know where
>> to change.
>>
>>
>>>
>>> - threadLocalAllocBuffer.hpp:130: extra whitespace ;)
>>>
>>
>> Fixed :)
>>
>>
>>>
>>> - some copyrights :)
>>>
>>>
>> I think I fixed all the issues, if you see specific issues, let me know.
>>
>>
>>> - in heapMonitoring.hpp: there are some random comments about some code
>>> that has been grabbed from "util/math/fastmath.[h|cc]". I can't tell
>>> whether this is code that can be used but I assume that Noam Shazeer is
>>> okay with that (i.e. that's all Google code).
>>>
>>
>> Jeremy and I double checked and we can release that as I thought. I
>> removed the comment from that piece of code entirely.
>>
>>
>>>
>>> - heapMonitoring.hpp/cpp static constant naming does not correspond to
>>> Hotspot's. Additionally, in Hotspot static methods are cased like other
>>> methods.
>>>
>>
>> I think I fixed the methods to be cased the same way as all other
>> methods. For static constants, I was not sure. I fixed a few other
>> variables but I could not seem to really see a consistent trend for
>> constants. I made them as variables but I'm not sure now.
>>
>>
>>
>>>
>>> - in heapMonitoring.cpp there are a few cryptic comments at the top
>>> that seem to refer to internal stuff that should probably be removed.
>>>
>>
>> Sorry about that! My personal todos not cleared out.
>>
>>
>>>
>>> I did not think through the impact of the TLAB changes on collector
>>> behavior yet (if there are). Also I did not check for problems with
>>> concurrent mark and SATB/G1 (if there are).
>>>
>>
>> I would love to know your thoughts on this, I think this is fine. I see
>> issues with multiple threads right now hitting the stack storage instance.
>> Previous webrevs had a mutex lock here but we took it out for simplificity
>> (and only for now).
>>
>>
>>>
>>> Thanks,
>>>   Thomas
>>>
>>>
>>
>>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170628/f5b6978d/attachment-0001.html>

From jcbeyler at google.com  Wed Jun 28 22:15:33 2017
From: jcbeyler at google.com (JC Beyler)
Date: Wed, 28 Jun 2017 15:15:33 -0700
Subject: Low-Overhead Heap Profiling
In-Reply-To: <044f8c75-72f3-79fd-af47-7ee875c071fd@oracle.com>
References: <CAF9BGBx8xFYudhJ=v1_hU5=nwEcC1X0NKRmC=E-RNN6SMs1ozg@mail.gmail.com>
 <CAF9BGBxj7+C6uy4dXHO-3pNorbpO7n22-9JmevM0F11xOo+MnA@mail.gmail.com>
 <CAF9BGBxrdLVmXRTDHj5hQHJhcyrkBM6+ex1TFWi=A5z=V2FSew@mail.gmail.com>
 <CAF9BGBz81YYVUsMLw9sOMR9fH7JQmyb8d_OZ9aJ47EOAqwWhEw@mail.gmail.com>
 <2af975e6-3827-bd57-0c3d-fadd54867a67@oracle.com>
 <365499b6-3f4d-a4df-9e7e-e72a739fb26b@oracle.com>
 <CAF9BGBy0T7A-+1dejYbr6f0d4s0XR-zv-WwHYqXrbe0EtCdsxQ@mail.gmail.com>
 <fb1155c6-ccf8-843c-34fc-fc4a45fc54cc@oracle.com>
 <CAF9BGBxqmXF9OFD4F9wVb-DQOj_UfNNcBhe+ARVxJ5E4cgCC1Q@mail.gmail.com>
 <102c59b8-25b6-8c21-8eef-1de7d0bbf629@oracle.com>
 <CAF9BGBxDaCOxobJi8Uu9kwKPRpnj-KqgWLfJO2z7UX71UnNF4g@mail.gmail.com>
 <CAF9BGByisZchXTsw1=mLETLnOggJShRJBDxFeOUgt+criTTGuQ@mail.gmail.com>
 <1497366226.2829.109.camel@oracle.com>
 <CAF9BGBx8376y_yeMFnMC8PHSqKDaFc5NcbFerQ2up0YVKtSKGg@mail.gmail.com>
 <1498215147.2741.34.camel@oracle.com>
 <CAF9BGBwaRu+12iRMm6U+n8j_+3izaEcv1OExdFKqZczEE7SJ0A@mail.gmail.com>
 <044f8c75-72f3-79fd-af47-7ee875c071fd@oracle.com>
Message-ID: <CAF9BGBysr6UAu5H71XGBrvPQPaHVx6tceQpQb5POTie1m4htvg@mail.gmail.com>

Hi Robbin,

A) Fast Refill

Sounds good for the fast refill, it will be simpler without that code so
I'll be happy if it gets removed :).

Right now, I fixed it in a way where this no longer is an issue for fast
refill turned on. If it does get removed, then my code can be simplified,
which is always good. When you return we can see if we rather I simplify my
code first and then we work on removing FastTLABRefill or the other way
around.

B) Incremental changes

Again another newbiew question here...

For showing the incremental changes, is there a link that explains how to
do that? I apologize for my newbie questions all the time :)

Right now, I do:

 ksh ../webrev.ksh -m -N

That generates a webrev.zip and send it to Chuck Rasbold. He then uploads
it to a new webrev.

I tried commiting my change and adding a small change. Then if I just do
ksh ../webrev.ksh without any options, it seems to produce a similar page
but now with only the changes I had (so the 06-07 comparison you were
talking about) and a changeset that has it all. I imagine that is what you
meant.

Which means that my workflow would become:

1) Make changes
2) Make a webrev without any options to show just the differences with the
tip
3) Amend my changes to my local commit so that I have it done with
4) Go to 1

Does that seem correct to you?

Note that when I do this, I only see the full change of a file in the full
change set (Side note here: now the page says change set and not patch,
which is maybe why Serguei was having issues?).

Thanks!
Jc



On Wed, Jun 28, 2017 at 1:12 AM, Robbin Ehn <robbin.ehn at oracle.com> wrote:

> Hi,
>
> On 06/28/2017 12:04 AM, JC Beyler wrote:
>
>> Dear Thomas et al,
>>
>> Here is the newest webrev:
>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.07/
>>
>
> You have some more bits to in there but generally this looks good and
> really nice with more tests.
> I'll do and deep dive and re-test this when I get back from my long
> vacation with whatever patch version you have then.
>
> Also I think it's time you provide incremental (v06->07 changes) as well
> as complete change-sets.
>
> Thanks, Robbin
>
>
>
>
>> Thomas, I "think" I have answered all your remarks. The summary is:
>>
>> - The statistic system is up and provides insight on what the heap
>> sampler is doing
>>     - I've noticed that, though the sampling rate is at the right mean,
>> we are missing some samples, I have not yet tracked out why (details below)
>>
>> - I've run a tiny benchmark that is the worse case: it is a very tight
>> loop and allocated a small array
>>     - In this case, I see no overhead when the system is off so that is a
>> good start :)
>>     - I see right now a high overhead in this case when sampling is on.
>> This is not a really too surprising but I'm going to see if this is
>> consistent with our internal implementation. The benchmark is really
>> allocation stressful so I'm not too surprised but I want to do the due
>> diligence.
>>
>>   - The statistic system up is up and I have a new test
>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.07/test/
>> serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatTest.java.patch
>>      - I did a bit of a study about the random generator here, more
>> details are below but basically it seems to work well
>>
>>   - I added a capability but since this is the first time doing this, I
>> was not sure I did it right
>>     - I did add a test though for it and the test seems to do what I
>> expect (all methods are failing with the JVMTI_ERROR_MUST_POSSESS_CAPABILITY
>> error).
>>         - http://cr.openjdk.java.net/~rasbold/8171119/webrev.07/test/
>> serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorNoCapa
>> bilityTest.java.patch
>>
>>   - I still need to figure out what to do about the multi-agent vs
>> single-agent issue
>>
>>   - As far as measurements, it seems I still need to look at:
>>     - Why we do the 20 random calls first, are they necessary?
>>     - Look at the mean of the sampling rate that the random generator
>> does and also what is actually sampled
>>     - What is the overhead in terms of memory/performance when on?
>>
>> I have inlined my answers, I think I got them all in the new webrev, let
>> me know your thoughts.
>>
>> Thanks again!
>> Jc
>>
>>
>> On Fri, Jun 23, 2017 at 3:52 AM, Thomas Schatzl <
>> thomas.schatzl at oracle.com <mailto:thomas.schatzl at oracle.com>> wrote:
>>
>>     Hi,
>>
>>     On Wed, 2017-06-21 at 13:45 -0700, JC Beyler wrote:
>>     > Hi all,
>>     >
>>     > First off: Thanks again to Robbin and Thomas for their reviews :)
>>     >
>>     > Next, I've uploaded a new webrev:
>>     > http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/ <
>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/>
>>
>>     >
>>     > Here is an update:
>>     >
>>     > - @Robbin, I forgot to say that yes I need to look at implementing
>>     > this for the other architectures and testing it before it is all
>>     > ready to go. Is it common to have it working on all possible
>>     > combinations or is there a subset that I should be doing first and
>> we
>>     > can do the others later?
>>     > - I've tested slowdebug, built and ran the JTreg tests I wrote with
>>     > slowdebug and fixed a few more issues
>>     > - I've refactored a bit of the code following Thomas' comments
>>     >    - I think I've handled all the comments from Thomas (I put
>>     > comments inline below for the specifics)
>>
>>     Thanks for handling all those.
>>
>>     > - Following Thomas' comments on statistics, I want to add some
>>     > quality assurance tests and find that the easiest way would be to
>>     > have a few counters of what is happening in the sampler and expose
>>     > that to the user.
>>     >    - I'll be adding that in the next version if no one sees any
>>     > objections to that.
>>     >    - This will allow me to add a sanity test in JTreg about number
>> of
>>     > samples and average of sampling rate
>>     >
>>     > @Thomas: I had a few questions that I inlined below but I will
>>     > summarize the "bigger ones" here:
>>     >    - You mentioned constants are not using the right conventions, I
>>     > looked around and didn't see any convention except normal naming
>> then
>>     > for static constants. Is that right?
>>
>>     I looked through https://wiki.openjdk.java.net/
>> display/HotSpot/StyleGui <https://wiki.openjdk.java.net
>> /display/HotSpot/StyleGui>
>>     de and the rule is to "follow an existing pattern and must have a
>>     distinct appearance from other names". Which does not help a lot I
>>     guess :/ The GC team started using upper camel case, e.g.
>>     SomeOtherConstant, but very likely this is probably not applied
>>     consistently throughout. So I am fine with not adding another style
>>     (like kMaxStackDepth with the "k" in front with some unknown meaning)
>>     is fine.
>>
>>     (Chances are you will find that style somewhere used anyway too,
>>     apologies if so :/)
>>
>>
>> Thanks for that link, now I know where to look. I used the upper camel
>> case in my code as well then :) I should have gotten them all.
>>
>>
>>      > PS: I've also inlined my answers to Thomas below:
>>      >
>>      > On Tue, Jun 13, 2017 at 8:03 AM, Thomas Schatzl
>> <thomas.schatzl at oracl
>>      > e.com <http://e.com>> wrote:
>>      > > Hi all,
>>      > >
>>      > > On Mon, 2017-06-12 at 11:11 -0700, JC Beyler wrote:
>>      > > > Dear all,
>>      > > >
>>      > > > I've continued working on this and have done the following
>>      > > webrev:
>>      > > > http://cr.openjdk.java.net/~rasbold/8171119/webrev.05/ <
>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.05/>
>>
>>      > >
>>      > > [...]
>>      > > > Things I still need to do:
>>      > > >    - Have to fix that TLAB case for the FastTLABRefill
>>      > > >    - Have to start looking at the data to see that it is
>>      > > consistent and does gather the right samples, right frequency,
>> etc.
>>      > > >    - Have to check the GC elements and what that produces
>>      > > >    - Run a slowdebug run and ensure I fixed all those issues
>> you
>>      > > saw > Robbin
>>      > > >
>>      > > > Thanks for looking at the webrev and have a great week!
>>      > >
>>      > >   scratching a bit on the surface of this change, so apologies
>> for
>>      > > rather shallow comments:
>>      > >
>>      > > - macroAssembler_x86.cpp:5604: while this is compiler code, and I
>>      > > am not sure this is final, please avoid littering the code with
>>      > > TODO remarks :) They tend to be candidates for later wtf moments
>>      > > only.
>>      > >
>>      > > Just file a CR for that.
>>      > >
>>      > Newcomer question: what is a CR and not sure I have the rights to
>> do
>>      > that yet ? :)
>>
>>     Apologies. CR is a change request, this suggests to file a bug in the
>>     bug tracker. And you are right, you can't just create a new account in
>>     the OpenJDK JIRA yourselves. :(
>>
>>
>> Ok good to know, I'll continue with my own todo list but I'll work hard
>> on not letting it slip in the webrevs anymore :)
>>
>>
>>     I was mostly referring to the "... but it is a TODO" part of that
>>     comment in macroassembler_x86.cpp. Comments about the why of the code
>>     are appreciated.
>>
>>     [Note that I now understand that this is to some degree still work in
>>     progress. As long as the final changeset does no contain TODO's I am
>>     fine (and it's not a hard objection, rather their use in "final" code
>>     is typically limited in my experience)]
>>
>>     5603   // Currently, if this happens, just set back the actual end to
>>     where it was.
>>     5604   // We miss a chance to sample here.
>>
>>     Would be okay, if explaining "this" and the "why" of missing a chance
>>     to sample here would be best.
>>
>>     Like maybe:
>>
>>     // If we needed to refill TLABs, just set the actual end point to
>>     // the end of the TLAB again. We do not sample here although we could.
>>
>> Done with your comment, it works well in my mind.
>>
>>     I am not sure whether "miss a chance to sample" meant "we could, but
>>     consciously don't because it's not that useful" or "it would be
>>     necessary but don't because it's too complicated to do.".
>>
>>     Looking at the original comment once more, I am also not sure if that
>>     comment shouldn't referring to the "end" variable (not actual_end)
>>     because that's the variable that is responsible for taking the
>> sampling
>>     path? (Going from the member description of ThreadLocalAllocBuffer).
>>
>>
>> I've moved this code and it no longer shows up here but the rationale and
>> answer was:
>>
>> So.. Yes, end is the variable provoking the sampling. Actual end is the
>> actual end of the TLAB.
>>
>> What was happening here is that the code is resetting _end to point
>> towards the end of the new TLAB. Because, we now have the end for sampling
>> and _actual_end for the actual end, we need to update the actual_end as
>> well.
>>
>> Normally, were we to do the real work here, we would calculate the (end -
>> start) offset, then do:
>>
>> - Set the new end to : start + (old_end - old_start)
>> - Set the actual end like we do here now where it because it is the
>> actual end.
>>
>> Why is this not done here now anymore?
>>    - I was still debating which path to take:
>>       - Do it in the fast refill code, it has its perks:
>>           - In a world where fast refills are happening all the time or a
>> lot, we can augment there the code to do the sampling
>>       - Remember what we had as an end before leaving the slowpath and
>> check on return
>>           - This is what I'm doing now, it removes the need to go fix up
>> all fast refill paths but if you remain in fast refill paths, you won't get
>> sampling. I have to think of the consequences of that, maybe a future
>> change later on?
>>              - I have the statistics now so I'm going to study that
>>                 -> By the way, though my statistics are showing I'm
>> missing some samples, if I turn off FastTlabRefill, it is the same loss so
>> for now, it seems this does not occur in my simple test.
>>
>>
>>
>>     But maybe I am only confused and it's best to just leave the comment
>>     away. :)
>>
>>     Thinking about it some more, doesn't this not-sampling in this case
>>     mean that sampling does not work in any collector that does inline
>> TLAB
>>     allocation at the moment? (Or is inline TLAB alloc automatically
>>     disabled with sampling somehow?)
>>
>>     That would indeed be a bigger TODO then :)
>>
>>
>> Agreed, this remark made me think that perhaps as a first step the new
>> way of doing it is better but I did have to:
>>   - Remove the const of the ThreadLocalBuffer remaining and hard_end
>> methods
>>   - Move hard_end out of the header file to have a bit more logic there
>>
>> Please let me know what you think of that and if you prefer it this way
>> or changing the fast refills. (I prefer this way now because it is more
>> incremental).
>>
>>
>>     > > - calling HeapMonitoring::do_weak_oops() (which should probably be
>>     > > called weak_oops_do() like other similar methods) only if string
>>     > > deduplication is enabled (in g1CollectedHeap.cpp:4511) seems
>> wrong.
>>     >
>>     > The call should be at least around 6 lines up outside the if.
>>     >
>>     > Preferentially in a method like process_weak_jni_handles(),
>> including
>>     > additional logging. (No new (G1) gc phase without minimal logging
>>     > :)).
>>     > Done but really not sure because:
>>     >
>>     > I put for logging:
>>     >   log_develop_trace(gc, freelist)("G1ConcRegionFreeing [other] :
>> heap
>>     > monitoring");
>>
>>     I would think that "gc, ref" would be more appropriate log tags for
>>     this similar to jni handles.
>>     (I am als not sure what weak reference handling has to do with
>>     G1ConcRegionFreeing, so I am a bit puzzled)
>>
>>
>> I was not sure what to put for the tags or really as the message. I
>> cleaned it up a bit now to:
>>     log_develop_trace(gc, ref)("HeapSampling [other] : heap monitoring
>> processing");
>>
>>
>>
>>     > Since weak_jni_handles didn't have logging for me to be inspired
>>     > from, I did that but unconvinced this is what should be done.
>>
>>     The JNI handle processing does have logging, but only in
>>     ReferenceProcessor::process_discovered_references(). In
>>     process_weak_jni_handles() only overall time is measured (in a G1
>>     specific way, since only G1 supports disabling reference procesing) :/
>>
>>     The code in ReferenceProcessor prints both time taken
>>     referenceProcessor.cpp:254, as well as the count, but strangely only
>> in
>>     debug VMs.
>>
>>     I have no idea why this logging is that unimportant to only print that
>>     in a debug VM. However there are reviews out for changing this area a
>>     bit, so it might be useful to wait for that (JDK-8173335).
>>
>>
>> I cleaned it up a bit anyway and now it returns the count of objects that
>> are in the system.
>>
>>
>>     > > - the change doubles the size of
>>     > > CollectedHeap::allocate_from_tlab_slow() above the "small and
>> nice"
>>     > > threshold. Maybe it could be refactored a bit.
>>     > Done I think, it looks better to me :).
>>
>>     In ThreadLocalAllocBuffer::handle_sample() I think the
>>     set_back_actual_end()/pick_next_sample() calls could be hoisted out
>> of
>>     the "if" :)
>>
>>
>> Done!
>>
>>
>>     > > - referenceProcessor.cpp:261: the change should add logging about
>>     > > the number of references encountered, maybe after the
>> corresponding
>>     > > "JNI weak reference count" log message.
>>     > Just to double check, are you saying that you'd like to have the
>> heap
>>     > sampler to keep in store how many sampled objects were encountered
>> in
>>     > the HeapMonitoring::weak_oops_do?
>>     >    - Would a return of the method with the number of handled
>>     > references and logging that work?
>>
>>     Yes, it's fine if HeapMonitoring::weak_oops_do() only returned the
>>     number of processed weak oops.
>>
>>
>> Done also (but I admit I have not tested the output yet) :)
>>
>>
>>     >    - Additionally, would you prefer it in a separate block with its
>>     > GCTraceTime?
>>
>>     Yes. Both kinds of information is interesting: while the time taken is
>>     typically more important, the next question would be why, and the
>>     number of references typically goes a long way there.
>>
>>     See above though, it is probably best to wait a bit.
>>
>>
>> Agreed that I "could" wait but, if it's ok, I'll just refactor/remove
>> this when we get closer to something final. Either, JDK-8173335
>> has gone in and I will notice it now or it will soon and I can change it
>> then.
>>
>>
>>     > > - threadLocalAllocBuffer.cpp:331: one more "TODO"
>>     > Removed it and added it to my personal todos to look at.
>>     >      > >
>>     > > - threadLocalAllocBuffer.hpp: ThreadLocalAllocBuffer class
>>     > > documentation should be updated about the sampling additions. I
>>     > > would have no clue what the difference between "actual_end" and
>>     > > "end" would be from the given information.
>>     > If you are talking about the comments in this file, I made them more
>>     > clear I hope in the new webrev. If it was somewhere else, let me
>> know
>>     > where to change.
>>
>>     Thanks, that's much better. Maybe a note in the comment of the class
>>     that ThreadLocalBuffer provides some sampling facility by modifying
>> the
>>     end() of the TLAB to cause "frequent" calls into the runtime call
>> where
>>     actual sampling takes place.
>>
>>
>> Done, I think it's better now. Added something about the slow_path_end as
>> well.
>>
>>
>>     > > - in heapMonitoring.hpp: there are some random comments about some
>>     > > code that has been grabbed from "util/math/fastmath.[h|cc]". I
>>     > > can't tell whether this is code that can be used but I assume that
>>     > > Noam Shazeer is okay with that (i.e. that's all Google code).
>>     > Jeremy and I double checked and we can release that as I thought. I
>>     > removed the comment from that piece of code entirely.
>>
>>     Thanks.
>>
>>     > > - heapMonitoring.hpp/cpp static constant naming does not
>> correspond
>>     > > to Hotspot's. Additionally, in Hotspot static methods are cased
>>     > > like other methods.
>>     > I think I fixed the methods to be cased the same way as all other
>>     > methods. For static constants, I was not sure. I fixed a few other
>>     > variables but I could not seem to really see a consistent trend for
>>     > constants. I made them as variables but I'm not sure now.
>>
>>     Sorry again, style is a kind of mess. The goal of my suggestions here
>>     is only to prevent yet another style creeping in.
>>
>>     > > - in heapMonitoring.cpp there are a few cryptic comments at the
>> top
>>     > > that seem to refer to internal stuff that should probably be
>>     > > removed.
>>     > Sorry about that! My personal todos not cleared out.
>>
>>     I am happy about comments, but I simply did not understand any of that
>>     and I do not know about other readers as well.
>>
>>     If you think you will remember removing/updating them until the review
>>     proper (I misunderstood the review situation a little it seems).
>>
>>     > > I did not think through the impact of the TLAB changes on
>> collector
>>     > > behavior yet (if there are). Also I did not check for problems
>> with
>>     > > concurrent mark and SATB/G1 (if there are).
>>     > I would love to know your thoughts on this, I think this is fine. I
>>
>>     I think so too now. No objects are made live out of thin air :)
>>
>>     > see issues with multiple threads right now hitting the stack storage
>>     > instance. Previous webrevs had a mutex lock here but we took it out
>>     > for simplificity (and only for now).
>>
>>     :) When looking at this after some thinking I now assume for this
>>     review that this code is not MT safe at all. There seems to be more
>>     synchronization missing than just the one for the StackTraceStorage.
>> So
>>     no comments about this here.
>>
>>
>> I doubled checked a bit (quickly I admit) but it seems that
>> synchronization in StackTraceStorage is really all you need (all methods
>> lead to a StackTraceStorage one and can be multithreaded outside of that).
>> There is a question about the initialization where the method
>> HeapMonitoring::initialize_profiling is not thread safe.
>> It would work (famous last words) and not crash if there was a race but
>> we could add a synchronization point there as well (and therefore on the
>> stop as well).
>>
>> But anyway I will really check and do this once we add back
>> synchronization.
>>
>>
>>     Also, this would require some kind of specification of what is allowed
>>     to be called when and where.
>>
>>
>> Would we specify this with the methods in the jvmti.xml file? We could
>> start by specifying in each that they are not thread safe but I saw no
>> mention of that for other methods.
>>
>>
>>     One potentially relevant observation about locking here: depending on
>>     sampling frequency, StackTraceStore::add_trace() may be rather
>>     frequently called. I assume that you are going to do measurements :)
>>
>>
>> Though we don't have the TLAB implementation in our code, the compiler
>> generated sampler uses 2% of overhead with a 512k sampling rate. I can do
>> real measurements when the code settles and we can see how costly this is
>> as a TLAB implementation.
>> However, my theory is that if the rate is 512k, the memory/performance
>> overhead should be minimal since it is what we saw with our code/workloads
>> (though not called the same way, we call it essentially at the same rate).
>> If you have a benchmark you'd like me to test, let me know!
>>
>> Right now, with my really small test, this does use a bit of overhead
>> even for a 512k sample size. I don't know yet why, I'm going to see what is
>> going on.
>>
>> Finally, I think it is not reasonable to suppose the overhead to be
>> negligible if the sampling rate used is too low. The user should know that
>> the lower the rate, the higher the overhead (documentation TODO?).
>>
>>
>>     I am not sure what the expected usage of the API is, but
>>     StackTraceStore::add_trace() seems to be able to grow without bounds.
>>     Only a GC truncates them to the live ones. That in itself seems to be
>>     problematic (GCs can be *wide* apart), and of course some of the API
>>     methods add to that because they duplicate that unbounded array. Do
>> you
>>     have any concerns/measurements about this?
>>
>>
>> So, the theory is that yes add_trace can be able to grow without bounds
>> but it grows at a sample per 512k of allocated space. The stacks it gathers
>> are currently maxed at 64 (I'd like to expand that to an option to the user
>> though at some point). So I have no concerns because:
>>
>> - If really this is taking a lot of space, that means the job is keeping
>> a lot of objects in memory as well, therefore the entire heap is getting
>> huge
>> - If this is the case, you will be triggering a GC at some point anyway.
>>
>> (I'm putting under the rug the issue of "What if we set the rate to 1 for
>> example" because as you lower the sampling rate, we cannot guarantee low
>> overhead; the idea behind this feature is to have a means of having
>> meaningful allocated samples at a low overhead)
>>
>> I have no measurements really right now but since I now have some
>> statistics I can poll, I will look a bit more at this question.
>>
>> I have the same last sentence than above: the user should expect this to
>> happen if the sampling rate is too small. That probably can be reflected in
>> the StartHeapSampling as a note : careful this might impact your
>> performance.
>>
>>
>>     Also, these stack traces might hold on to huge arrays. Any
>>     consideration of that? Particularly it might be the cause for OOMEs in
>>     tight memory situations.
>>
>>
>> There is a stack size maximum that is set to 64 so it should not hold
>> huge arrays. I don't think this is an issue but I can double check with a
>> test or two.
>>
>>
>>     - please consider adding a safepoint check in
>>     HeapMonitoring::weak_oops_do to prevent accidental misuse.
>>
>>     - in struct StackTraceStorage, the public fields may also need
>>     underscores. At least some files in the runtime directory have structs
>>     with underscored public members (and some don't). The runtime team
>>     should probably comment on that.
>>
>>
>> Agreed I did not know. I looked around and a lot of structs did not have
>> them it seemed so I left it as is. I will happily change it if someone
>> prefers (I was not sure if you really preferred or not, your sentence
>> seemed to be more a note of "this might need to change but I don't know if
>> the runtime team enforces that", let me know if I read that wrongly).
>>
>>
>>     - In StackTraceStorage::weak_oops_do(), when examining the
>>     StackTraceData, maybe it is useful to consider having a non-NULL
>>     reference outside of the heap's reserved space an error. There should
>>     be no oop outside of the heap's reserved space ever.
>>
>>     Unless you allow storing random values in StackTraceData::obj, which I
>>     would not encourage.
>>
>>
>> I suppose you are talking about this part:
>> if ((value != NULL && Universe::heap()->is_in_reserved(value)) &&
>>            (is_alive == NULL || is_alive->do_object_b(value))) {
>>
>> What you are saying is that I could have something like:
>> if (value != my_non_null_reference &&
>>            (is_alive == NULL || is_alive->do_object_b(value))) {
>>
>> Is that what you meant? Is there really a reason to do so? When I look at
>> the code, is_in_reserved seems like a O(1) method call. I'm not even sure
>> we can have a NULL value to be honest. I might have to study that to see if
>> this was not a paranoid test to begin with.
>>
>> The is_alive code has now morphed due to the comment below.
>>
>>
>>
>>     - HeapMonitoring::weak_oops_do() does not seem to use the
>>     passed AbstractRefProcTaskExecutor.
>>
>>
>> It did use it:
>>   size_t HeapMonitoring::weak_oops_do(
>>      AbstractRefProcTaskExecutor *task_executor,
>>      BoolObjectClosure* is_alive,
>>      OopClosure *f,
>>      VoidClosure *complete_gc) {
>>    assert(SafepointSynchronize::is_at_safepoint(), "must be at
>> safepoint");
>>
>>    if (task_executor != NULL) {
>>      task_executor->set_single_threaded_mode();
>>    }
>>    return StackTraceStorage::storage()->weak_oops_do(is_alive, f,
>> complete_gc);
>> }
>>
>> But due to the comment below, I refactored this, so this is no longer
>> here. Now I have an always true closure that is passed.
>>
>>
>>     - I do not understand allowing to call this method with a NULL
>>     complete_gc closure. This would mean that objects referenced from the
>>     object that is referenced by the StackTraceData are not pulled,
>> meaning
>>     they would get stale.
>>
>>     - same with is_alive parameter value of NULL
>>
>>
>> So these questions made me look a bit closer at this code. This code I
>> think was written this way to have a very small impact on the file but you
>> are right, there is no reason for this here. I've simplified the code by
>> making in referenceProcessor.cpp a process_HeapSampling method that handles
>> everything there.
>>
>> The code allowed NULLs because it depended on where you were coming from
>> and how the code was being called.
>>
>> - I added a static always_true variable and pass that now to be more
>> consistent with the rest of the code.
>> - I moved the complete_gc into process_phaseHeapSampling now (new method)
>> and handle the task_executor and the complete_gc there
>>     - Newbie question: in our code we did a set_single_threaded_mode but
>> I see that process_phaseJNI does it right before its call, do I need to do
>> it for the process_phaseHeapSample?
>> That API is much cleaner (in my mind) and is consistent with what is done
>> around it (again in my mind).
>>
>>
>>     - heapMonitoring.cpp:590: I do not completely understand the purpose
>> of
>>     this code: in the end this results in a fixed value directly dependent
>>     on the Thread address anyway? In the end this results in a fixed value
>>     directly dependent on the Thread address anyway?
>>     IOW, what is special about exactly 20 rounds?
>>
>>
>> So we really want a fast random number generator that has a specific mean
>> (512k is the default we use). The code uses the thread address as the start
>> number of the sequence (why not, it is random enough is rationale). Then
>> instead of just starting there, we prime the sequence and really only start
>> at the 21st number, it is arbitrary and I have not done a study to see if
>> we could do more or less of that.
>>
>> As I have the statistics of the system up and running, I'll run some
>> experiments to see if this is needed, is 20 good, or not.
>>
>>
>>     - also I would consider stripping a few bits of the threads' address
>> as
>>     initialization value for your rng. The last three bits (and probably
>>     more, check whether the Thread object is allocated on special
>>     boundaries) are always zero for them.
>>     Not sure if the given "random" value is random enough before/after,
>>     this method, so just skip that comment if you think this is not
>>     required.
>>
>>
>> I don't know is the honest answer. I think what is important is that we
>> tend towards a mean and it is random "enough" to not fall in pitfalls of
>> only sampling a subset of objects due to their allocation order. I added
>> that as test to do to see if it changes the mean in any way for the 512k
>> default value and/or if the first 1000 elements look better.
>>
>>
>>     Some more random nits I did not find a place to put anywhere:
>>
>>     - ThreadLocalAllocBuffer::_extra_space does not seem to be used
>>     anywhere?
>>
>>
>> Good catch :).
>>
>>
>>     - Maybe indent the declaration of ThreadLocalAllocBuffer::_bytes_until_sample
>> to align below the other members of that group.
>>
>>
>> Done moved it up a bit to have non static members together and static
>> separate.
>>
>>     Thanks,
>>        Thomas
>>
>>
>> Thanks for your review!
>> Jc
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170628/770d7942/attachment-0001.html>

From jcbeyler at google.com  Wed Jun 28 22:15:34 2017
From: jcbeyler at google.com (JC Beyler)
Date: Wed, 28 Jun 2017 15:15:34 -0700
Subject: Low-Overhead Heap Profiling
In-Reply-To: <044f8c75-72f3-79fd-af47-7ee875c071fd@oracle.com>
References: <CAF9BGBx8xFYudhJ=v1_hU5=nwEcC1X0NKRmC=E-RNN6SMs1ozg@mail.gmail.com>
 <CAF9BGBxj7+C6uy4dXHO-3pNorbpO7n22-9JmevM0F11xOo+MnA@mail.gmail.com>
 <CAF9BGBxrdLVmXRTDHj5hQHJhcyrkBM6+ex1TFWi=A5z=V2FSew@mail.gmail.com>
 <CAF9BGBz81YYVUsMLw9sOMR9fH7JQmyb8d_OZ9aJ47EOAqwWhEw@mail.gmail.com>
 <2af975e6-3827-bd57-0c3d-fadd54867a67@oracle.com>
 <365499b6-3f4d-a4df-9e7e-e72a739fb26b@oracle.com>
 <CAF9BGBy0T7A-+1dejYbr6f0d4s0XR-zv-WwHYqXrbe0EtCdsxQ@mail.gmail.com>
 <fb1155c6-ccf8-843c-34fc-fc4a45fc54cc@oracle.com>
 <CAF9BGBxqmXF9OFD4F9wVb-DQOj_UfNNcBhe+ARVxJ5E4cgCC1Q@mail.gmail.com>
 <102c59b8-25b6-8c21-8eef-1de7d0bbf629@oracle.com>
 <CAF9BGBxDaCOxobJi8Uu9kwKPRpnj-KqgWLfJO2z7UX71UnNF4g@mail.gmail.com>
 <CAF9BGByisZchXTsw1=mLETLnOggJShRJBDxFeOUgt+criTTGuQ@mail.gmail.com>
 <1497366226.2829.109.camel@oracle.com>
 <CAF9BGBx8376y_yeMFnMC8PHSqKDaFc5NcbFerQ2up0YVKtSKGg@mail.gmail.com>
 <1498215147.2741.34.camel@oracle.com>
 <CAF9BGBwaRu+12iRMm6U+n8j_+3izaEcv1OExdFKqZczEE7SJ0A@mail.gmail.com>
 <044f8c75-72f3-79fd-af47-7ee875c071fd@oracle.com>
Message-ID: <CAF9BGBx7KR9vHgBtYVS5cWKRwdX-ggLn2X2P8cmbgu4+AnV=vg@mail.gmail.com>

Hi Robbin,

A) Fast Refill

Sounds good for the fast refill, it will be simpler without that code so
I'll be happy if it gets removed :).

Right now, I fixed it in a way where this no longer is an issue for fast
refill turned on. If it does get removed, then my code can be simplified,
which is always good. When you return we can see if we rather I simplify my
code first and then we work on removing FastTLABRefill or the other way
around.

B) Incremental changes

Again another newbiew question here...

For showing the incremental changes, is there a link that explains how to
do that? I apologize for my newbie questions all the time :)

Right now, I do:

 ksh ../webrev.ksh -m -N

That generates a webrev.zip and send it to Chuck Rasbold. He then uploads
it to a new webrev.

I tried commiting my change and adding a small change. Then if I just do
ksh ../webrev.ksh without any options, it seems to produce a similar page
but now with only the changes I had (so the 06-07 comparison you were
talking about) and a changeset that has it all. I imagine that is what you
meant.

Which means that my workflow would become:

1) Make changes
2) Make a webrev without any options to show just the differences with the
tip
3) Amend my changes to my local commit so that I have it done with
4) Go to 1

Does that seem correct to you?

Note that when I do this, I only see the full change of a file in the full
change set (Side note here: now the page says change set and not patch,
which is maybe why Serguei was having issues?).

Thanks!
Jc



On Wed, Jun 28, 2017 at 1:12 AM, Robbin Ehn <robbin.ehn at oracle.com> wrote:

> Hi,
>
> On 06/28/2017 12:04 AM, JC Beyler wrote:
>
>> Dear Thomas et al,
>>
>> Here is the newest webrev:
>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.07/
>>
>
> You have some more bits to in there but generally this looks good and
> really nice with more tests.
> I'll do and deep dive and re-test this when I get back from my long
> vacation with whatever patch version you have then.
>
> Also I think it's time you provide incremental (v06->07 changes) as well
> as complete change-sets.
>
> Thanks, Robbin
>
>
>
>
>> Thomas, I "think" I have answered all your remarks. The summary is:
>>
>> - The statistic system is up and provides insight on what the heap
>> sampler is doing
>>     - I've noticed that, though the sampling rate is at the right mean,
>> we are missing some samples, I have not yet tracked out why (details below)
>>
>> - I've run a tiny benchmark that is the worse case: it is a very tight
>> loop and allocated a small array
>>     - In this case, I see no overhead when the system is off so that is a
>> good start :)
>>     - I see right now a high overhead in this case when sampling is on.
>> This is not a really too surprising but I'm going to see if this is
>> consistent with our internal implementation. The benchmark is really
>> allocation stressful so I'm not too surprised but I want to do the due
>> diligence.
>>
>>   - The statistic system up is up and I have a new test
>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.07/test/
>> serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatTest.java.patch
>>      - I did a bit of a study about the random generator here, more
>> details are below but basically it seems to work well
>>
>>   - I added a capability but since this is the first time doing this, I
>> was not sure I did it right
>>     - I did add a test though for it and the test seems to do what I
>> expect (all methods are failing with the JVMTI_ERROR_MUST_POSSESS_CAPABILITY
>> error).
>>         - http://cr.openjdk.java.net/~rasbold/8171119/webrev.07/test/
>> serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorNoCapa
>> bilityTest.java.patch
>>
>>   - I still need to figure out what to do about the multi-agent vs
>> single-agent issue
>>
>>   - As far as measurements, it seems I still need to look at:
>>     - Why we do the 20 random calls first, are they necessary?
>>     - Look at the mean of the sampling rate that the random generator
>> does and also what is actually sampled
>>     - What is the overhead in terms of memory/performance when on?
>>
>> I have inlined my answers, I think I got them all in the new webrev, let
>> me know your thoughts.
>>
>> Thanks again!
>> Jc
>>
>>
>> On Fri, Jun 23, 2017 at 3:52 AM, Thomas Schatzl <
>> thomas.schatzl at oracle.com <mailto:thomas.schatzl at oracle.com>> wrote:
>>
>>     Hi,
>>
>>     On Wed, 2017-06-21 at 13:45 -0700, JC Beyler wrote:
>>     > Hi all,
>>     >
>>     > First off: Thanks again to Robbin and Thomas for their reviews :)
>>     >
>>     > Next, I've uploaded a new webrev:
>>     > http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/ <
>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/>
>>
>>     >
>>     > Here is an update:
>>     >
>>     > - @Robbin, I forgot to say that yes I need to look at implementing
>>     > this for the other architectures and testing it before it is all
>>     > ready to go. Is it common to have it working on all possible
>>     > combinations or is there a subset that I should be doing first and
>> we
>>     > can do the others later?
>>     > - I've tested slowdebug, built and ran the JTreg tests I wrote with
>>     > slowdebug and fixed a few more issues
>>     > - I've refactored a bit of the code following Thomas' comments
>>     >    - I think I've handled all the comments from Thomas (I put
>>     > comments inline below for the specifics)
>>
>>     Thanks for handling all those.
>>
>>     > - Following Thomas' comments on statistics, I want to add some
>>     > quality assurance tests and find that the easiest way would be to
>>     > have a few counters of what is happening in the sampler and expose
>>     > that to the user.
>>     >    - I'll be adding that in the next version if no one sees any
>>     > objections to that.
>>     >    - This will allow me to add a sanity test in JTreg about number
>> of
>>     > samples and average of sampling rate
>>     >
>>     > @Thomas: I had a few questions that I inlined below but I will
>>     > summarize the "bigger ones" here:
>>     >    - You mentioned constants are not using the right conventions, I
>>     > looked around and didn't see any convention except normal naming
>> then
>>     > for static constants. Is that right?
>>
>>     I looked through https://wiki.openjdk.java.net/
>> display/HotSpot/StyleGui <https://wiki.openjdk.java.net
>> /display/HotSpot/StyleGui>
>>     de and the rule is to "follow an existing pattern and must have a
>>     distinct appearance from other names". Which does not help a lot I
>>     guess :/ The GC team started using upper camel case, e.g.
>>     SomeOtherConstant, but very likely this is probably not applied
>>     consistently throughout. So I am fine with not adding another style
>>     (like kMaxStackDepth with the "k" in front with some unknown meaning)
>>     is fine.
>>
>>     (Chances are you will find that style somewhere used anyway too,
>>     apologies if so :/)
>>
>>
>> Thanks for that link, now I know where to look. I used the upper camel
>> case in my code as well then :) I should have gotten them all.
>>
>>
>>      > PS: I've also inlined my answers to Thomas below:
>>      >
>>      > On Tue, Jun 13, 2017 at 8:03 AM, Thomas Schatzl
>> <thomas.schatzl at oracl
>>      > e.com <http://e.com>> wrote:
>>      > > Hi all,
>>      > >
>>      > > On Mon, 2017-06-12 at 11:11 -0700, JC Beyler wrote:
>>      > > > Dear all,
>>      > > >
>>      > > > I've continued working on this and have done the following
>>      > > webrev:
>>      > > > http://cr.openjdk.java.net/~rasbold/8171119/webrev.05/ <
>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.05/>
>>
>>      > >
>>      > > [...]
>>      > > > Things I still need to do:
>>      > > >    - Have to fix that TLAB case for the FastTLABRefill
>>      > > >    - Have to start looking at the data to see that it is
>>      > > consistent and does gather the right samples, right frequency,
>> etc.
>>      > > >    - Have to check the GC elements and what that produces
>>      > > >    - Run a slowdebug run and ensure I fixed all those issues
>> you
>>      > > saw > Robbin
>>      > > >
>>      > > > Thanks for looking at the webrev and have a great week!
>>      > >
>>      > >   scratching a bit on the surface of this change, so apologies
>> for
>>      > > rather shallow comments:
>>      > >
>>      > > - macroAssembler_x86.cpp:5604: while this is compiler code, and I
>>      > > am not sure this is final, please avoid littering the code with
>>      > > TODO remarks :) They tend to be candidates for later wtf moments
>>      > > only.
>>      > >
>>      > > Just file a CR for that.
>>      > >
>>      > Newcomer question: what is a CR and not sure I have the rights to
>> do
>>      > that yet ? :)
>>
>>     Apologies. CR is a change request, this suggests to file a bug in the
>>     bug tracker. And you are right, you can't just create a new account in
>>     the OpenJDK JIRA yourselves. :(
>>
>>
>> Ok good to know, I'll continue with my own todo list but I'll work hard
>> on not letting it slip in the webrevs anymore :)
>>
>>
>>     I was mostly referring to the "... but it is a TODO" part of that
>>     comment in macroassembler_x86.cpp. Comments about the why of the code
>>     are appreciated.
>>
>>     [Note that I now understand that this is to some degree still work in
>>     progress. As long as the final changeset does no contain TODO's I am
>>     fine (and it's not a hard objection, rather their use in "final" code
>>     is typically limited in my experience)]
>>
>>     5603   // Currently, if this happens, just set back the actual end to
>>     where it was.
>>     5604   // We miss a chance to sample here.
>>
>>     Would be okay, if explaining "this" and the "why" of missing a chance
>>     to sample here would be best.
>>
>>     Like maybe:
>>
>>     // If we needed to refill TLABs, just set the actual end point to
>>     // the end of the TLAB again. We do not sample here although we could.
>>
>> Done with your comment, it works well in my mind.
>>
>>     I am not sure whether "miss a chance to sample" meant "we could, but
>>     consciously don't because it's not that useful" or "it would be
>>     necessary but don't because it's too complicated to do.".
>>
>>     Looking at the original comment once more, I am also not sure if that
>>     comment shouldn't referring to the "end" variable (not actual_end)
>>     because that's the variable that is responsible for taking the
>> sampling
>>     path? (Going from the member description of ThreadLocalAllocBuffer).
>>
>>
>> I've moved this code and it no longer shows up here but the rationale and
>> answer was:
>>
>> So.. Yes, end is the variable provoking the sampling. Actual end is the
>> actual end of the TLAB.
>>
>> What was happening here is that the code is resetting _end to point
>> towards the end of the new TLAB. Because, we now have the end for sampling
>> and _actual_end for the actual end, we need to update the actual_end as
>> well.
>>
>> Normally, were we to do the real work here, we would calculate the (end -
>> start) offset, then do:
>>
>> - Set the new end to : start + (old_end - old_start)
>> - Set the actual end like we do here now where it because it is the
>> actual end.
>>
>> Why is this not done here now anymore?
>>    - I was still debating which path to take:
>>       - Do it in the fast refill code, it has its perks:
>>           - In a world where fast refills are happening all the time or a
>> lot, we can augment there the code to do the sampling
>>       - Remember what we had as an end before leaving the slowpath and
>> check on return
>>           - This is what I'm doing now, it removes the need to go fix up
>> all fast refill paths but if you remain in fast refill paths, you won't get
>> sampling. I have to think of the consequences of that, maybe a future
>> change later on?
>>              - I have the statistics now so I'm going to study that
>>                 -> By the way, though my statistics are showing I'm
>> missing some samples, if I turn off FastTlabRefill, it is the same loss so
>> for now, it seems this does not occur in my simple test.
>>
>>
>>
>>     But maybe I am only confused and it's best to just leave the comment
>>     away. :)
>>
>>     Thinking about it some more, doesn't this not-sampling in this case
>>     mean that sampling does not work in any collector that does inline
>> TLAB
>>     allocation at the moment? (Or is inline TLAB alloc automatically
>>     disabled with sampling somehow?)
>>
>>     That would indeed be a bigger TODO then :)
>>
>>
>> Agreed, this remark made me think that perhaps as a first step the new
>> way of doing it is better but I did have to:
>>   - Remove the const of the ThreadLocalBuffer remaining and hard_end
>> methods
>>   - Move hard_end out of the header file to have a bit more logic there
>>
>> Please let me know what you think of that and if you prefer it this way
>> or changing the fast refills. (I prefer this way now because it is more
>> incremental).
>>
>>
>>     > > - calling HeapMonitoring::do_weak_oops() (which should probably be
>>     > > called weak_oops_do() like other similar methods) only if string
>>     > > deduplication is enabled (in g1CollectedHeap.cpp:4511) seems
>> wrong.
>>     >
>>     > The call should be at least around 6 lines up outside the if.
>>     >
>>     > Preferentially in a method like process_weak_jni_handles(),
>> including
>>     > additional logging. (No new (G1) gc phase without minimal logging
>>     > :)).
>>     > Done but really not sure because:
>>     >
>>     > I put for logging:
>>     >   log_develop_trace(gc, freelist)("G1ConcRegionFreeing [other] :
>> heap
>>     > monitoring");
>>
>>     I would think that "gc, ref" would be more appropriate log tags for
>>     this similar to jni handles.
>>     (I am als not sure what weak reference handling has to do with
>>     G1ConcRegionFreeing, so I am a bit puzzled)
>>
>>
>> I was not sure what to put for the tags or really as the message. I
>> cleaned it up a bit now to:
>>     log_develop_trace(gc, ref)("HeapSampling [other] : heap monitoring
>> processing");
>>
>>
>>
>>     > Since weak_jni_handles didn't have logging for me to be inspired
>>     > from, I did that but unconvinced this is what should be done.
>>
>>     The JNI handle processing does have logging, but only in
>>     ReferenceProcessor::process_discovered_references(). In
>>     process_weak_jni_handles() only overall time is measured (in a G1
>>     specific way, since only G1 supports disabling reference procesing) :/
>>
>>     The code in ReferenceProcessor prints both time taken
>>     referenceProcessor.cpp:254, as well as the count, but strangely only
>> in
>>     debug VMs.
>>
>>     I have no idea why this logging is that unimportant to only print that
>>     in a debug VM. However there are reviews out for changing this area a
>>     bit, so it might be useful to wait for that (JDK-8173335).
>>
>>
>> I cleaned it up a bit anyway and now it returns the count of objects that
>> are in the system.
>>
>>
>>     > > - the change doubles the size of
>>     > > CollectedHeap::allocate_from_tlab_slow() above the "small and
>> nice"
>>     > > threshold. Maybe it could be refactored a bit.
>>     > Done I think, it looks better to me :).
>>
>>     In ThreadLocalAllocBuffer::handle_sample() I think the
>>     set_back_actual_end()/pick_next_sample() calls could be hoisted out
>> of
>>     the "if" :)
>>
>>
>> Done!
>>
>>
>>     > > - referenceProcessor.cpp:261: the change should add logging about
>>     > > the number of references encountered, maybe after the
>> corresponding
>>     > > "JNI weak reference count" log message.
>>     > Just to double check, are you saying that you'd like to have the
>> heap
>>     > sampler to keep in store how many sampled objects were encountered
>> in
>>     > the HeapMonitoring::weak_oops_do?
>>     >    - Would a return of the method with the number of handled
>>     > references and logging that work?
>>
>>     Yes, it's fine if HeapMonitoring::weak_oops_do() only returned the
>>     number of processed weak oops.
>>
>>
>> Done also (but I admit I have not tested the output yet) :)
>>
>>
>>     >    - Additionally, would you prefer it in a separate block with its
>>     > GCTraceTime?
>>
>>     Yes. Both kinds of information is interesting: while the time taken is
>>     typically more important, the next question would be why, and the
>>     number of references typically goes a long way there.
>>
>>     See above though, it is probably best to wait a bit.
>>
>>
>> Agreed that I "could" wait but, if it's ok, I'll just refactor/remove
>> this when we get closer to something final. Either, JDK-8173335
>> has gone in and I will notice it now or it will soon and I can change it
>> then.
>>
>>
>>     > > - threadLocalAllocBuffer.cpp:331: one more "TODO"
>>     > Removed it and added it to my personal todos to look at.
>>     >      > >
>>     > > - threadLocalAllocBuffer.hpp: ThreadLocalAllocBuffer class
>>     > > documentation should be updated about the sampling additions. I
>>     > > would have no clue what the difference between "actual_end" and
>>     > > "end" would be from the given information.
>>     > If you are talking about the comments in this file, I made them more
>>     > clear I hope in the new webrev. If it was somewhere else, let me
>> know
>>     > where to change.
>>
>>     Thanks, that's much better. Maybe a note in the comment of the class
>>     that ThreadLocalBuffer provides some sampling facility by modifying
>> the
>>     end() of the TLAB to cause "frequent" calls into the runtime call
>> where
>>     actual sampling takes place.
>>
>>
>> Done, I think it's better now. Added something about the slow_path_end as
>> well.
>>
>>
>>     > > - in heapMonitoring.hpp: there are some random comments about some
>>     > > code that has been grabbed from "util/math/fastmath.[h|cc]". I
>>     > > can't tell whether this is code that can be used but I assume that
>>     > > Noam Shazeer is okay with that (i.e. that's all Google code).
>>     > Jeremy and I double checked and we can release that as I thought. I
>>     > removed the comment from that piece of code entirely.
>>
>>     Thanks.
>>
>>     > > - heapMonitoring.hpp/cpp static constant naming does not
>> correspond
>>     > > to Hotspot's. Additionally, in Hotspot static methods are cased
>>     > > like other methods.
>>     > I think I fixed the methods to be cased the same way as all other
>>     > methods. For static constants, I was not sure. I fixed a few other
>>     > variables but I could not seem to really see a consistent trend for
>>     > constants. I made them as variables but I'm not sure now.
>>
>>     Sorry again, style is a kind of mess. The goal of my suggestions here
>>     is only to prevent yet another style creeping in.
>>
>>     > > - in heapMonitoring.cpp there are a few cryptic comments at the
>> top
>>     > > that seem to refer to internal stuff that should probably be
>>     > > removed.
>>     > Sorry about that! My personal todos not cleared out.
>>
>>     I am happy about comments, but I simply did not understand any of that
>>     and I do not know about other readers as well.
>>
>>     If you think you will remember removing/updating them until the review
>>     proper (I misunderstood the review situation a little it seems).
>>
>>     > > I did not think through the impact of the TLAB changes on
>> collector
>>     > > behavior yet (if there are). Also I did not check for problems
>> with
>>     > > concurrent mark and SATB/G1 (if there are).
>>     > I would love to know your thoughts on this, I think this is fine. I
>>
>>     I think so too now. No objects are made live out of thin air :)
>>
>>     > see issues with multiple threads right now hitting the stack storage
>>     > instance. Previous webrevs had a mutex lock here but we took it out
>>     > for simplificity (and only for now).
>>
>>     :) When looking at this after some thinking I now assume for this
>>     review that this code is not MT safe at all. There seems to be more
>>     synchronization missing than just the one for the StackTraceStorage.
>> So
>>     no comments about this here.
>>
>>
>> I doubled checked a bit (quickly I admit) but it seems that
>> synchronization in StackTraceStorage is really all you need (all methods
>> lead to a StackTraceStorage one and can be multithreaded outside of that).
>> There is a question about the initialization where the method
>> HeapMonitoring::initialize_profiling is not thread safe.
>> It would work (famous last words) and not crash if there was a race but
>> we could add a synchronization point there as well (and therefore on the
>> stop as well).
>>
>> But anyway I will really check and do this once we add back
>> synchronization.
>>
>>
>>     Also, this would require some kind of specification of what is allowed
>>     to be called when and where.
>>
>>
>> Would we specify this with the methods in the jvmti.xml file? We could
>> start by specifying in each that they are not thread safe but I saw no
>> mention of that for other methods.
>>
>>
>>     One potentially relevant observation about locking here: depending on
>>     sampling frequency, StackTraceStore::add_trace() may be rather
>>     frequently called. I assume that you are going to do measurements :)
>>
>>
>> Though we don't have the TLAB implementation in our code, the compiler
>> generated sampler uses 2% of overhead with a 512k sampling rate. I can do
>> real measurements when the code settles and we can see how costly this is
>> as a TLAB implementation.
>> However, my theory is that if the rate is 512k, the memory/performance
>> overhead should be minimal since it is what we saw with our code/workloads
>> (though not called the same way, we call it essentially at the same rate).
>> If you have a benchmark you'd like me to test, let me know!
>>
>> Right now, with my really small test, this does use a bit of overhead
>> even for a 512k sample size. I don't know yet why, I'm going to see what is
>> going on.
>>
>> Finally, I think it is not reasonable to suppose the overhead to be
>> negligible if the sampling rate used is too low. The user should know that
>> the lower the rate, the higher the overhead (documentation TODO?).
>>
>>
>>     I am not sure what the expected usage of the API is, but
>>     StackTraceStore::add_trace() seems to be able to grow without bounds.
>>     Only a GC truncates them to the live ones. That in itself seems to be
>>     problematic (GCs can be *wide* apart), and of course some of the API
>>     methods add to that because they duplicate that unbounded array. Do
>> you
>>     have any concerns/measurements about this?
>>
>>
>> So, the theory is that yes add_trace can be able to grow without bounds
>> but it grows at a sample per 512k of allocated space. The stacks it gathers
>> are currently maxed at 64 (I'd like to expand that to an option to the user
>> though at some point). So I have no concerns because:
>>
>> - If really this is taking a lot of space, that means the job is keeping
>> a lot of objects in memory as well, therefore the entire heap is getting
>> huge
>> - If this is the case, you will be triggering a GC at some point anyway.
>>
>> (I'm putting under the rug the issue of "What if we set the rate to 1 for
>> example" because as you lower the sampling rate, we cannot guarantee low
>> overhead; the idea behind this feature is to have a means of having
>> meaningful allocated samples at a low overhead)
>>
>> I have no measurements really right now but since I now have some
>> statistics I can poll, I will look a bit more at this question.
>>
>> I have the same last sentence than above: the user should expect this to
>> happen if the sampling rate is too small. That probably can be reflected in
>> the StartHeapSampling as a note : careful this might impact your
>> performance.
>>
>>
>>     Also, these stack traces might hold on to huge arrays. Any
>>     consideration of that? Particularly it might be the cause for OOMEs in
>>     tight memory situations.
>>
>>
>> There is a stack size maximum that is set to 64 so it should not hold
>> huge arrays. I don't think this is an issue but I can double check with a
>> test or two.
>>
>>
>>     - please consider adding a safepoint check in
>>     HeapMonitoring::weak_oops_do to prevent accidental misuse.
>>
>>     - in struct StackTraceStorage, the public fields may also need
>>     underscores. At least some files in the runtime directory have structs
>>     with underscored public members (and some don't). The runtime team
>>     should probably comment on that.
>>
>>
>> Agreed I did not know. I looked around and a lot of structs did not have
>> them it seemed so I left it as is. I will happily change it if someone
>> prefers (I was not sure if you really preferred or not, your sentence
>> seemed to be more a note of "this might need to change but I don't know if
>> the runtime team enforces that", let me know if I read that wrongly).
>>
>>
>>     - In StackTraceStorage::weak_oops_do(), when examining the
>>     StackTraceData, maybe it is useful to consider having a non-NULL
>>     reference outside of the heap's reserved space an error. There should
>>     be no oop outside of the heap's reserved space ever.
>>
>>     Unless you allow storing random values in StackTraceData::obj, which I
>>     would not encourage.
>>
>>
>> I suppose you are talking about this part:
>> if ((value != NULL && Universe::heap()->is_in_reserved(value)) &&
>>            (is_alive == NULL || is_alive->do_object_b(value))) {
>>
>> What you are saying is that I could have something like:
>> if (value != my_non_null_reference &&
>>            (is_alive == NULL || is_alive->do_object_b(value))) {
>>
>> Is that what you meant? Is there really a reason to do so? When I look at
>> the code, is_in_reserved seems like a O(1) method call. I'm not even sure
>> we can have a NULL value to be honest. I might have to study that to see if
>> this was not a paranoid test to begin with.
>>
>> The is_alive code has now morphed due to the comment below.
>>
>>
>>
>>     - HeapMonitoring::weak_oops_do() does not seem to use the
>>     passed AbstractRefProcTaskExecutor.
>>
>>
>> It did use it:
>>   size_t HeapMonitoring::weak_oops_do(
>>      AbstractRefProcTaskExecutor *task_executor,
>>      BoolObjectClosure* is_alive,
>>      OopClosure *f,
>>      VoidClosure *complete_gc) {
>>    assert(SafepointSynchronize::is_at_safepoint(), "must be at
>> safepoint");
>>
>>    if (task_executor != NULL) {
>>      task_executor->set_single_threaded_mode();
>>    }
>>    return StackTraceStorage::storage()->weak_oops_do(is_alive, f,
>> complete_gc);
>> }
>>
>> But due to the comment below, I refactored this, so this is no longer
>> here. Now I have an always true closure that is passed.
>>
>>
>>     - I do not understand allowing to call this method with a NULL
>>     complete_gc closure. This would mean that objects referenced from the
>>     object that is referenced by the StackTraceData are not pulled,
>> meaning
>>     they would get stale.
>>
>>     - same with is_alive parameter value of NULL
>>
>>
>> So these questions made me look a bit closer at this code. This code I
>> think was written this way to have a very small impact on the file but you
>> are right, there is no reason for this here. I've simplified the code by
>> making in referenceProcessor.cpp a process_HeapSampling method that handles
>> everything there.
>>
>> The code allowed NULLs because it depended on where you were coming from
>> and how the code was being called.
>>
>> - I added a static always_true variable and pass that now to be more
>> consistent with the rest of the code.
>> - I moved the complete_gc into process_phaseHeapSampling now (new method)
>> and handle the task_executor and the complete_gc there
>>     - Newbie question: in our code we did a set_single_threaded_mode but
>> I see that process_phaseJNI does it right before its call, do I need to do
>> it for the process_phaseHeapSample?
>> That API is much cleaner (in my mind) and is consistent with what is done
>> around it (again in my mind).
>>
>>
>>     - heapMonitoring.cpp:590: I do not completely understand the purpose
>> of
>>     this code: in the end this results in a fixed value directly dependent
>>     on the Thread address anyway? In the end this results in a fixed value
>>     directly dependent on the Thread address anyway?
>>     IOW, what is special about exactly 20 rounds?
>>
>>
>> So we really want a fast random number generator that has a specific mean
>> (512k is the default we use). The code uses the thread address as the start
>> number of the sequence (why not, it is random enough is rationale). Then
>> instead of just starting there, we prime the sequence and really only start
>> at the 21st number, it is arbitrary and I have not done a study to see if
>> we could do more or less of that.
>>
>> As I have the statistics of the system up and running, I'll run some
>> experiments to see if this is needed, is 20 good, or not.
>>
>>
>>     - also I would consider stripping a few bits of the threads' address
>> as
>>     initialization value for your rng. The last three bits (and probably
>>     more, check whether the Thread object is allocated on special
>>     boundaries) are always zero for them.
>>     Not sure if the given "random" value is random enough before/after,
>>     this method, so just skip that comment if you think this is not
>>     required.
>>
>>
>> I don't know is the honest answer. I think what is important is that we
>> tend towards a mean and it is random "enough" to not fall in pitfalls of
>> only sampling a subset of objects due to their allocation order. I added
>> that as test to do to see if it changes the mean in any way for the 512k
>> default value and/or if the first 1000 elements look better.
>>
>>
>>     Some more random nits I did not find a place to put anywhere:
>>
>>     - ThreadLocalAllocBuffer::_extra_space does not seem to be used
>>     anywhere?
>>
>>
>> Good catch :).
>>
>>
>>     - Maybe indent the declaration of ThreadLocalAllocBuffer::_bytes_until_sample
>> to align below the other members of that group.
>>
>>
>> Done moved it up a bit to have non static members together and static
>> separate.
>>
>>     Thanks,
>>        Thomas
>>
>>
>> Thanks for your review!
>> Jc
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170628/67608415/attachment-0001.html>

From yasuenag at gmail.com  Wed Jun 28 22:40:47 2017
From: yasuenag at gmail.com (Yasumasa Suenaga)
Date: Thu, 29 Jun 2017 07:40:47 +0900
Subject: [10] RFR 8181647: jhsdb jstack could not output thread name
In-Reply-To: <5953C4F3.1000802@oracle.com>
References: <5937CB5F.4060209@oracle.com>
 <a9a45cd1-8717-ed6b-8169-9543e904c20f@oracle.com>
 <5939133A.9040709@oracle.com> <5940E316.7060104@oracle.com>
 <HE1PR08MB2795EF131855EC452EAF9D3FFFC30@HE1PR08MB2795.eurprd08.prod.outlook.com>
 <5945FB4E.9070502@oracle.com> <5953C4F3.1000802@oracle.com>
Message-ID: <c5086795-8704-6fe1-02a8-2df22de4861a@gmail.com>

Hi chihiro,

getThreadState() in OSThread.java:

>> +        } else if (val ==  BREAKPOINTED) {
>> +            return ThreadState.BREAKPOINTED;
>> +        } else if (val ==  BREAKPOINTED) {
>> +            return ThreadState.BREAKPOINTED;

These conditions are duplicated.


Please upload webrev if you can :-)


Yasumasa


On 2017/06/29 0:02, chihiro ito wrote:
> Hi all,
>
> In last week, I've posted review request [1].
> Could you possibly review for this following small change? If review is ok, please commit this as cito.
>
> [1] http://mail.openjdk.java.net/pipermail/serviceability-dev/2017-June/021430.html
>
> Thanks,
> Chihiro (Contributer)
>
> On 2017/06/18 13:02, chihiro ito wrote:
>> At first I thought to print just each thread name, but I tried to make it as similar as possible to JStack as following.
>>
>> Mixed mode:
>> ----------------- 26476 -----------------
>> "main" #1 prio=5 tid=0x00007f6894019000 nid=0x676c waiting on condition [0x00007f689b7ae000]
>>    java.lang.Thread.State: TIMED_WAITING (sleeping)
>>    JavaThread state: _thread_blocked
>> 0x00007f689b185a82    __pthread_cond_timedwait + 0x132
>>
>> No mixed mode:
>> "main" #1 prio=5 tid=0x00007f6894019000 nid=0x676c waiting on condition [0x00007f689b7ae000]
>>    java.lang.Thread.State: TIMED_WAITING (sleeping)
>>    JavaThread state: _thread_blocked
>>  - java.lang.Thread.sleep(long) @bci=0 (Interpreted frame)
>>
>>
>> This change passed a test by jtreg.
>>
>> SOURCE_HOME=/home/user/repo/jdk10-hs
>> jtreg -dir:${SOURCE_HOME}/hotspot/test -testjdk:${SOURCE_HOME}/build/linux-x86_64-normal-server-slowdebug/jdk/ serviceability/sa/JhsdbThreadInfoTest.java
>> Test results: passed: 1
>>
>>
>> Source:
>> diff --git a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/oops/OopUtilities.java b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/oops/OopUtilities.java
>> --- a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/oops/OopUtilities.java
>> +++ b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/oops/OopUtilities.java
>> @@ -1,5 +1,5 @@
>>  /*
>> - * Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.
>> + * Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved.
>>   * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
>>   *
>>   * This code is free software; you can redistribute it and/or modify it
>> @@ -59,20 +59,20 @@
>>    // parkBlocker field is new since 1.6
>>    private static OopField threadParkBlockerField;
>>
>> +  private static IntField threadPriorityField;
>> +  private static BooleanField threadDaemonField;
>> +
>>    // possible values of java_lang_Thread::ThreadStatus
>>    private static int THREAD_STATUS_NEW;
>> -  /*
>> -    Other enum constants are not needed as of now. Uncomment these as and when needed.
>>
>> -    private static int THREAD_STATUS_RUNNABLE;
>> -    private static int THREAD_STATUS_SLEEPING;
>> -    private static int THREAD_STATUS_IN_OBJECT_WAIT;
>> -    private static int THREAD_STATUS_IN_OBJECT_WAIT_TIMED;
>> -    private static int THREAD_STATUS_PARKED;
>> -    private static int THREAD_STATUS_PARKED_TIMED;
>> -    private static int THREAD_STATUS_BLOCKED_ON_MONITOR_ENTER;
>> -    private static int THREAD_STATUS_TERMINATED;
>> -  */
>> +  private static int THREAD_STATUS_RUNNABLE;
>> +  private static int THREAD_STATUS_SLEEPING;
>> +  private static int THREAD_STATUS_IN_OBJECT_WAIT;
>> +  private static int THREAD_STATUS_IN_OBJECT_WAIT_TIMED;
>> +  private static int THREAD_STATUS_PARKED;
>> +  private static int THREAD_STATUS_PARKED_TIMED;
>> +  private static int THREAD_STATUS_BLOCKED_ON_MONITOR_ENTER;
>> +  private static int THREAD_STATUS_TERMINATED;
>>
>>    // java.util.concurrent.locks.AbstractOwnableSynchronizer fields
>>    private static OopField absOwnSyncOwnerThreadField;
>> @@ -229,20 +229,19 @@
>>        threadStatusField = (IntField) k.findField("threadStatus", "I");
>>        threadParkBlockerField = (OopField) k.findField("parkBlocker",
>>                                       "Ljava/lang/Object;");
>> +      threadPriorityField = (IntField) k.findField("priority", "I");
>> +      threadDaemonField = (BooleanField) k.findField("daemon", "Z");
>>        TypeDataBase db = VM.getVM().getTypeDataBase();
>>        THREAD_STATUS_NEW = db.lookupIntConstant("java_lang_Thread::NEW").intValue();
>> -      /*
>> -        Other enum constants are not needed as of now. Uncomment these as and when needed.
>>
>> -        THREAD_STATUS_RUNNABLE = db.lookupIntConstant("java_lang_Thread::RUNNABLE").intValue();
>> -        THREAD_STATUS_SLEEPING = db.lookupIntConstant("java_lang_Thread::SLEEPING").intValue();
>> -        THREAD_STATUS_IN_OBJECT_WAIT = db.lookupIntConstant("java_lang_Thread::IN_OBJECT_WAIT").intValue();
>> -        THREAD_STATUS_IN_OBJECT_WAIT_TIMED = db.lookupIntConstant("java_lang_Thread::IN_OBJECT_WAIT_TIMED").intValue();
>> -        THREAD_STATUS_PARKED = db.lookupIntConstant("java_lang_Thread::PARKED").intValue();
>> -        THREAD_STATUS_PARKED_TIMED = db.lookupIntConstant("java_lang_Thread::PARKED_TIMED").intValue();
>> -        THREAD_STATUS_BLOCKED_ON_MONITOR_ENTER = db.lookupIntConstant("java_lang_Thread::BLOCKED_ON_MONITOR_ENTER").intValue();
>> -        THREAD_STATUS_TERMINATED = db.lookupIntConstant("java_lang_Thread::TERMINATED").intValue();
>> -      */
>> +      THREAD_STATUS_RUNNABLE = db.lookupIntConstant("java_lang_Thread::RUNNABLE").intValue();
>> +      THREAD_STATUS_SLEEPING = db.lookupIntConstant("java_lang_Thread::SLEEPING").intValue();
>> +      THREAD_STATUS_IN_OBJECT_WAIT = db.lookupIntConstant("java_lang_Thread::IN_OBJECT_WAIT").intValue();
>> +      THREAD_STATUS_IN_OBJECT_WAIT_TIMED = db.lookupIntConstant("java_lang_Thread::IN_OBJECT_WAIT_TIMED").intValue();
>> +      THREAD_STATUS_PARKED = db.lookupIntConstant("java_lang_Thread::PARKED").intValue();
>> +      THREAD_STATUS_PARKED_TIMED = db.lookupIntConstant("java_lang_Thread::PARKED_TIMED").intValue();
>> +      THREAD_STATUS_BLOCKED_ON_MONITOR_ENTER = db.lookupIntConstant("java_lang_Thread::BLOCKED_ON_MONITOR_ENTER").intValue();
>> +      THREAD_STATUS_TERMINATED = db.lookupIntConstant("java_lang_Thread::TERMINATED").intValue();
>>
>>        if (Assert.ASSERTS_ENABLED) {
>>          // it is okay to miss threadStatusField, because this was
>> @@ -331,4 +330,46 @@
>>        return absOwnSyncOwnerThreadField.getValue(oop);
>>      }
>>    }
>> +
>> +  public static int threadOopGetPriority(Oop threadOop) {
>> +    initThreadFields();
>> +    if (threadPriorityField != null) {
>> +      return threadPriorityField.getValue(threadOop);
>> +    } else {
>> +      return 0;
>> +    }
>> +  }
>> +
>> +  public static boolean threadOopGetDaemon(Oop threadOop) {
>> +    initThreadFields();
>> +    if (threadDaemonField != null) {
>> +      return threadDaemonField.getValue(threadOop);
>> +    } else {
>> +      return false;
>> +    }
>> +  }
>> +
>> +  public static String threadOopGetThreadStatusName(Oop threadOop) {
>> +    int status = OopUtilities.threadOopGetThreadStatus(threadOop);
>> +    if( status == THREAD_STATUS_NEW ){
>> +      return "NEW";
>> +    }else if(status == THREAD_STATUS_RUNNABLE ){
>> +      return "RUNNABLE";
>> +    }else if(status == THREAD_STATUS_SLEEPING ){
>> +      return "TIMED_WAITING (sleeping)";
>> +    }else if(status == THREAD_STATUS_IN_OBJECT_WAIT ){
>> +      return "WAITING (on object monitor)";
>> +    }else if(status == THREAD_STATUS_IN_OBJECT_WAIT_TIMED ){
>> +      return "TIMED_WAITING (on object monitor)";
>> +    }else if(status == THREAD_STATUS_PARKED ){
>> +      return "WAITING (parking)";
>> +    }else if(status == THREAD_STATUS_PARKED_TIMED ){
>> +      return "TIMED_WAITING (parking)";
>> +    }else if(status == THREAD_STATUS_BLOCKED_ON_MONITOR_ENTER ){
>> +      return "BLOCKED (on object monitor)";
>> +    }else if(status == THREAD_STATUS_TERMINATED ){
>> +      return "TERMINATED";
>> +    }
>> +    return "UNKNOWN";
>> +  }
>>  }
>> diff --git a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/JavaThread.java b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/JavaThread.java
>> --- a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/JavaThread.java
>> +++ b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/JavaThread.java
>> @@ -475,4 +475,36 @@
>>      return access.getLastSP(addr);
>>    }
>>
>> +
>> +  public void printThreadInfoOn(PrintStream out){
>> +
>> +    Oop threadOop = this.getThreadObj();
>> +
>> +    out.print("\"");
>> +    out.print(this.getThreadName());
>> +    out.print("\" #");
>> +    out.print(OopUtilities.threadOopGetTID(threadOop));
>> +    if( OopUtilities.threadOopGetDaemon(threadOop) ){
>> +      out.print(" daemon");
>> +    }
>> +    out.print(" prio=");
>> +    out.print(OopUtilities.threadOopGetPriority(threadOop));
>> +    out.print(" tid=");
>> + out.print(String.format("0x%016x",this.getAddress().asLongValue()));
>> +    out.print(" nid=");
>> +    out.print(String.format("0x%x ",this.getOSThread().threadId()));
>> +    out.print(getOSThread().getThreadState().getPrintVal());
>> +    out.print(" [");
>> +    if( this.getLastJavaSP() == null){
>> +      out.print(String.format("0x%016x",0L));
>> +    } else {
>> +      Address maskAddress = this.getLastJavaSP().andWithMask(0xFFF);
>> + out.print(this.getLastJavaSP().xorWithMask(maskAddress.asLongValue()));
>> +    }
>> +    out.println("]");
>> +    out.print("   java.lang.Thread.State: ");
>> + out.println(OopUtilities.threadOopGetThreadStatusName(threadOop));
>> +    out.print("   JavaThread state: _thread_");
>> +    out.println(this.getThreadState().toString().toLowerCase());
>> +  }
>>  }
>> diff --git a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/OSThread.java b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/OSThread.java
>> --- a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/OSThread.java
>> +++ b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/OSThread.java
>> @@ -1,5 +1,5 @@
>>  /*
>> - * Copyright (c) 2004, 2013, Oracle and/or its affiliates. All rights reserved.
>> + * Copyright (c) 2004, 2017, Oracle and/or its affiliates. All rights reserved.
>>   * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
>>   *
>>   * This code is free software; you can redistribute it and/or modify it
>> @@ -33,6 +33,19 @@
>>  public class OSThread extends VMObject {
>>      private static JIntField interruptedField;
>>      private static Field threadIdField;
>> +    private static CIntegerField threadStateField;
>> +
>> +    // ThreadStates read from underlying process
>> +    private static int ALLOCATED;
>> +    private static int INITIALIZED;
>> +    private static int RUNNABLE;
>> +    private static int MONITOR_WAIT;
>> +    private static int CONDVAR_WAIT;
>> +    private static int OBJECT_WAIT;
>> +    private static int BREAKPOINTED;
>> +    private static int SLEEPING;
>> +    private static int ZOMBIE;
>> +
>>      static {
>>          VM.registerVMInitializedObserver(new Observer() {
>>              public void update(Observable o, Object data) {
>> @@ -45,6 +58,17 @@
>>          Type type = db.lookupType("OSThread");
>>          interruptedField = type.getJIntField("_interrupted");
>>          threadIdField = type.getField("_thread_id");
>> +        threadStateField = type.getCIntegerField("_state");
>> +
>> +        ALLOCATED = db.lookupIntConstant("ALLOCATED").intValue();
>> +        INITIALIZED = db.lookupIntConstant("INITIALIZED").intValue();
>> +        RUNNABLE = db.lookupIntConstant("RUNNABLE").intValue();
>> +        MONITOR_WAIT = db.lookupIntConstant("MONITOR_WAIT").intValue();
>> +        CONDVAR_WAIT = db.lookupIntConstant("CONDVAR_WAIT").intValue();
>> +        OBJECT_WAIT = db.lookupIntConstant("OBJECT_WAIT").intValue();
>> +        BREAKPOINTED = db.lookupIntConstant("BREAKPOINTED").intValue();
>> +        SLEEPING = db.lookupIntConstant("SLEEPING").intValue();
>> +        ZOMBIE = db.lookupIntConstant("ZOMBIE").intValue();
>>      }
>>
>>      public OSThread(Address addr) {
>> @@ -59,4 +83,30 @@
>>          return threadIdField.getJInt(addr);
>>      }
>>
>> +    public ThreadState getThreadState() {
>> +        int val = (int)threadStateField.getValue(addr);
>> +        if (val ==  ALLOCATED) {
>> +            return ThreadState.ALLOCATED;
>> +        } else if (val ==  INITIALIZED) {
>> +            return ThreadState.INITIALIZED;
>> +        } else if (val ==  RUNNABLE) {
>> +            return ThreadState.RUNNABLE;
>> +        } else if (val ==  MONITOR_WAIT) {
>> +            return ThreadState.MONITOR_WAIT;
>> +        } else if (val ==  CONDVAR_WAIT) {
>> +            return ThreadState.CONDVAR_WAIT;
>> +        } else if (val ==  OBJECT_WAIT) {
>> +            return ThreadState.OBJECT_WAIT;
>> +        } else if (val ==  BREAKPOINTED) {
>> +            return ThreadState.BREAKPOINTED;
>> +        } else if (val ==  BREAKPOINTED) {
>> +            return ThreadState.BREAKPOINTED;
>> +        } else if (val ==  SLEEPING) {
>> +            return ThreadState.SLEEPING;
>> +        } else if (val ==  ZOMBIE) {
>> +            return ThreadState.ZOMBIE;
>> +        } else {
>> +            throw new RuntimeException("Illegal thread state " + val);
>> +        }
>> +    }
>>  }
>> diff --git a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/ThreadState.java b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/ThreadState.java
>> new file mode 100644
>> --- /dev/null
>> +++ b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/ThreadState.java
>> @@ -0,0 +1,60 @@
>> +/*
>> + * Copyright (c) 2017, Oracle and/or its affiliates. All rights reserved.
>> + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
>> + *
>> + * This code is free software; you can redistribute it and/or modify it
>> + * under the terms of the GNU General Public License version 2 only, as
>> + * published by the Free Software Foundation.
>> + *
>> + * This code is distributed in the hope that it will be useful, but WITHOUT
>> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
>> + * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
>> + * version 2 for more details (a copy is included in the LICENSE file that
>> + * accompanied this code).
>> + *
>> + * You should have received a copy of the GNU General Public License version
>> + * 2 along with this work; if not, write to the Free Software Foundation,
>> + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
>> + *
>> + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
>> + * or visit www.oracle.com if you need additional information or have any
>> + * questions.
>> + */
>> +
>> +package sun.jvm.hotspot.runtime;
>> +
>> +/** This is a type-safe enum mirroring the ThreadState enum in
>> + osThread.hpp. The conversion between the underlying ints
>> + and these values is done in OSThread. */
>> +
>> +public class ThreadState {
>> +
>> +    private String printVal;
>> +
>> +    /** Memory has been allocated but not initialized */
>> +    public static final ThreadState ALLOCATED = new ThreadState("allocated");
>> +    /** The thread has been initialized but yet started */
>> +    public static final ThreadState INITIALIZED = new ThreadState("initialized");
>> +    /** Has been started and is runnable, but not necessarily running */
>> +    public static final ThreadState RUNNABLE = new ThreadState("runnable");
>> +    /** Waiting on a contended monitor lock */
>> +    public static final ThreadState MONITOR_WAIT = new ThreadState("waiting for monitor entry");
>> +    /** Waiting on a condition variable */
>> +    public static final ThreadState CONDVAR_WAIT = new ThreadState("waiting on condition");
>> +    /** Waiting on an Object.wait() call */
>> +    public static final ThreadState OBJECT_WAIT = new ThreadState("in Object.wait()");
>> +    /** Suspended at breakpoint */
>> +    public static final ThreadState BREAKPOINTED = new ThreadState("at breakpoint");
>> +    /** Thread.sleep() */
>> +    public static final ThreadState SLEEPING = new ThreadState("sleeping");
>> +    /** All done, but not reclaimed yet */
>> +    public static final ThreadState ZOMBIE = new ThreadState("zombie");
>> +
>> +    private ThreadState(String printVal){
>> +        this.printVal = printVal;
>> +    }
>> +
>> +    public String getPrintVal() {
>> +        return printVal;
>> +    }
>> +}
>> diff --git a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
>> --- a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
>> +++ b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
>> @@ -1,5 +1,5 @@
>>  /*
>> - * Copyright (c) 2003, 2013, Oracle and/or its affiliates. All rights reserved.
>> + * Copyright (c) 2003, 2017, Oracle and/or its affiliates. All rights reserved.
>>   * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
>>   *
>>   * This code is free software; you can redistribute it and/or modify it
>> @@ -88,6 +88,10 @@
>>                 out.print("----------------- ");
>>                 out.print(th);
>>                 out.println(" -----------------");
>> +               JavaThread jthread = (JavaThread) proxyToThread.get(th);
>> +               if (jthread != null) {
>> +                  jthread.printThreadInfoOn(out);
>> +               }
>>                 while (f != null) {
>>                    ClosestSymbol sym = f.closestSymbolToPC();
>>                    Address pc = f.pc();
>> diff --git a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
>> --- a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
>> +++ b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
>> @@ -1,5 +1,5 @@
>>  /*
>> - * Copyright (c) 2002, 2013, Oracle and/or its affiliates. All rights reserved.
>> + * Copyright (c) 2002, 2017, Oracle and/or its affiliates. All rights reserved.
>>   * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
>>   *
>>   * This code is free software; you can redistribute it and/or modify it
>> @@ -74,14 +74,7 @@
>>              int i = 1;
>>              for (JavaThread cur = threads.first(); cur != null; cur = cur.next(), i++) {
>>                  if (cur.isJavaThread()) {
>> -                    Address sp = cur.getLastJavaSP();
>> -                    tty.print("Thread ");
>> -                    cur.printThreadIDOn(tty);
>> -                    tty.print(": (state = " + cur.getThreadState());
>> -                    if (verbose) {
>> -                        tty.println(", current Java SP = " + sp);
>> -                    }
>> -                    tty.println(')');
>> +                    cur.printThreadInfoOn(tty);
>>                      try {
>>                          for (JavaVFrame vf = cur.getLastJavaVFrameDbg(); vf != null; vf = vf.javaSender()) {
>>                              Method method = vf.getMethod();
>> diff --git a/src/share/vm/runtime/vmStructs.cpp b/src/share/vm/runtime/vmStructs.cpp
>> --- a/src/share/vm/runtime/vmStructs.cpp
>> +++ b/src/share/vm/runtime/vmStructs.cpp
>> @@ -981,6 +981,7 @@
>> /************/ \
>> \
>>    volatile_nonstatic_field(OSThread, _interrupted, jint)                                  \
>> +  volatile_nonstatic_field(OSThread, _state, ThreadState)                                 \
>> \
>> /************************/ \
>>    /* OopMap and OopMapSet */ \
>> @@ -2186,6 +2187,7 @@
>> declare_integer_type(Generation::Name) \
>> declare_integer_type(InstanceKlass::ClassState) \
>> declare_integer_type(JavaThreadState) \
>> + declare_integer_type(ThreadState) \
>> declare_integer_type(Location::Type) \
>> declare_integer_type(Location::Where) \
>> declare_integer_type(Flag::Flags) \
>> @@ -2443,6 +2445,20 @@
>> declare_constant(JavaThread::_not_terminated) \
>> declare_constant(JavaThread::_thread_exiting) \
>> \
>> + /*******************/ \
>> +  /* JavaThreadState */                                                   \
>> + /*******************/ \
>> + \
>> + declare_constant(ALLOCATED) \
>> + declare_constant(INITIALIZED) \
>> + declare_constant(RUNNABLE) \
>> + declare_constant(MONITOR_WAIT) \
>> + declare_constant(CONDVAR_WAIT) \
>> + declare_constant(OBJECT_WAIT) \
>> + declare_constant(BREAKPOINTED) \
>> + declare_constant(SLEEPING) \
>> + declare_constant(ZOMBIE) \
>> + \
>> /******************************/ \
>>    /* Klass misc. enum constants */                                        \
>> /******************************/ \
>> diff --git a/test/serviceability/sa/JhsdbThreadInfoTest.java b/test/serviceability/sa/JhsdbThreadInfoTest.java
>> new file mode 100644
>> --- /dev/null
>> +++ b/test/serviceability/sa/JhsdbThreadInfoTest.java
>> @@ -0,0 +1,87 @@
>> +/*
>> + * Copyright (c) 2017, Oracle and/or its affiliates. All rights reserved.
>> + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
>> + *
>> + * This code is free software; you can redistribute it and/or modify it
>> + * under the terms of the GNU General Public License version 2 only, as
>> + * published by the Free Software Foundation.
>> + *
>> + * This code is distributed in the hope that it will be useful, but WITHOUT
>> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
>> + * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
>> + * version 2 for more details (a copy is included in the LICENSE file that
>> + * accompanied this code).
>> + *
>> + * You should have received a copy of the GNU General Public License version
>> + * 2 along with this work; if not, write to the Free Software Foundation,
>> + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
>> + *
>> + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
>> + * or visit www.oracle.com if you need additional information or have any
>> + * questions.
>> + */
>> +
>> +import jdk.test.lib.apps.LingeredApp;
>> +import jdk.test.lib.JDKToolLauncher;
>> +import jdk.test.lib.Platform;
>> +import jdk.test.lib.process.OutputAnalyzer;
>> +import jdk.test.lib.Utils;
>> +
>> +/*
>> + * @test
>> + * @library /test/lib
>> + * @run main JhsdbThreadInfoTest
>> + */
>> +public class JhsdbThreadInfoTest {
>> +
>> +    public static void main(String[] args) throws Exception {
>> +
>> +        if (!Platform.shouldSAAttach()) {
>> +            System.out.println("SA attach not expected to work - test skipped.");
>> +            return;
>> +        }
>> +
>> +        LingeredApp app = null;
>> +
>> +        try {
>> +            app = LingeredApp.startApp(Utils.getVmOptions());
>> +            System.out.println("Started LingeredApp with pid " + app.getPid());
>> +
>> +            JDKToolLauncher jhsdbLauncher = JDKToolLauncher.createUsingTestJDK("jhsdb");
>> +
>> +            jhsdbLauncher.addToolArg("jstack");
>> +            jhsdbLauncher.addToolArg("--pid");
>> + jhsdbLauncher.addToolArg(Long.toString(app.getPid()));
>> +
>> +            ProcessBuilder pb = new ProcessBuilder();
>> +            pb.command(jhsdbLauncher.getCommand());
>> +            Process jhsdb = pb.start();
>> +
>> +            jhsdb.waitFor();
>> +
>> +            OutputAnalyzer out = new OutputAnalyzer(jhsdb);
>> +
>> +            System.out.println(out.getStdout());
>> +            System.err.println(out.getStderr());
>> +
>> +            out.shouldMatch("\".+\" #\\d+ daemon prio=\\d+ tid=0x[0-9a-f]+ nid=0x[0-9a-f]+ .+ \\[0x[0-9a-f]+]");
>> +            out.shouldMatch("\"main\" #\\d+ prio=\\d+ tid=0x[0-9a-f]+ nid=0x[0-9a-f]+ .+ \\[0x[0-9a-f]+]");
>> +            out.shouldMatch("   java.lang.Thread.State: .+");
>> +            out.shouldMatch("   JavaThread state: _thread_.+");
>> +
>> +            out.shouldNotContain("   java.lang.Thread.State: UNKNOWN");
>> +            out.stderrShouldBeEmpty();
>> +
>> +            System.out.println("Test Completed");
>> +
>> +
>> +        } catch (InterruptedException ie) {
>> +            throw new Error("Problem awaiting the child process: " + ie, ie);
>> +        } catch (Exception attachE) {
>> +            throw new Error("Couldn't start jhsdb, attach to LingeredApp or match ThreadName: " + attachE);
>> +
>> +        } finally {
>> +            LingeredApp.stopApp(app);
>> +        }
>> +    }
>> +}
>>
>>
>> Regards,
>> Chihiro
>>
>>
>> On 2017/06/14 16:51, Bernd Eckenfels wrote:
>>> I don't understand why this format is totally different from the normal stack traces? At least the header with the stack names could be similar?
>>>
>>> Gruss
>>> Bernd
>>> --
>>> http://bernd.eckenfels.net
>>> ------------------------------------------------------------------------
>>> *From:* serviceability-dev <serviceability-dev-bounces at openjdk.java.net> on behalf of chihiro ito <chihiro.ito at oracle.com>
>>> *Sent:* Wednesday, June 14, 2017 9:17:42 AM
>>> *To:* Jini George; serviceability-dev at openjdk.java.net
>>> *Subject:* Re: [10] RFR 8181647: jhsdb jstack could not output thread name
>>> Hi all,
>>>
>>> I added a test case and modified previous patch including fix the
>>> copyright year to 2017.
>>> I changed to output Java thread name next the separator lines in "jhsdb
>>> jstack --mixed" case as following.
>>>
>>> ----------------- 32117 -----------------
>>> "main"
>>> 0x00007f6c8feafa82    __pthread_cond_timedwait + 0x132
>>>
>>> Could you possibly review for this following small change? If review is
>>> ok, please commit this as cito.
>>>
>>>
>>> Source:
>>> diff --git
>>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
>>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
>>> --- a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
>>> +++ b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
>>> @@ -1,5 +1,5 @@
>>>   /*
>>> - * Copyright (c) 2003, 2013, Oracle and/or its affiliates. All rights
>>> reserved.
>>> + * Copyright (c) 2003, 2017, Oracle and/or its affiliates. All rights
>>> reserved.
>>>    * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
>>>    *
>>>    * This code is free software; you can redistribute it and/or modify it
>>> @@ -88,6 +88,12 @@
>>>                  out.print("----------------- ");
>>>                  out.print(th);
>>>                  out.println(" -----------------");
>>> +               JavaThread jthread = (JavaThread) proxyToThread.get(th);
>>> +               if (jthread != null) {
>>> +                   out.print("\"");
>>> +                   out.print(jthread.getThreadName());
>>> +                   out.println("\"");
>>> +               }
>>>                  while (f != null) {
>>>                     ClosestSymbol sym = f.closestSymbolToPC();
>>>                     Address pc = f.pc();
>>> diff --git
>>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
>>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
>>> ---
>>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
>>> +++
>>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
>>> @@ -1,5 +1,5 @@
>>>   /*
>>> - * Copyright (c) 2002, 2013, Oracle and/or its affiliates. All rights
>>> reserved.
>>> + * Copyright (c) 2002, 2017, Oracle and/or its affiliates. All rights
>>> reserved.
>>>    * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
>>>    *
>>>    * This code is free software; you can redistribute it and/or modify it
>>> @@ -75,7 +75,9 @@
>>>               for (JavaThread cur = threads.first(); cur != null; cur =
>>> cur.next(), i++) {
>>>                   if (cur.isJavaThread()) {
>>>                       Address sp = cur.getLastJavaSP();
>>> -                    tty.print("Thread ");
>>> +                    tty.print("\"");
>>> +                    tty.print(cur.getThreadName());
>>> +                    tty.print("\" nid=");
>>>                       cur.printThreadIDOn(tty);
>>>                       tty.print(": (state = " + cur.getThreadState());
>>>                       if (verbose) {
>>> diff --git a/test/serviceability/sa/JhsdbThreadNameTest.java
>>> b/test/serviceability/sa/JhsdbThreadNameTest.java
>>> new file mode 100644
>>> --- /dev/null
>>> +++ b/test/serviceability/sa/JhsdbThreadNameTest.java
>>> @@ -0,0 +1,107 @@
>>> +/*
>>> + * Copyright (c) 2017, Oracle and/or its affiliates. All rights reserved.
>>> + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
>>> + *
>>> + * This code is free software; you can redistribute it and/or modify it
>>> + * under the terms of the GNU General Public License version 2 only, as
>>> + * published by the Free Software Foundation.
>>> + *
>>> + * This code is distributed in the hope that it will be useful, but WITHOUT
>>> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
>>> + * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
>>> + * version 2 for more details (a copy is included in the LICENSE file that
>>> + * accompanied this code).
>>> + *
>>> + * You should have received a copy of the GNU General Public License
>>> version
>>> + * 2 along with this work; if not, write to the Free Software Foundation,
>>> + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
>>> + *
>>> + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
>>> + * or visit www.oracle.com <http://www.oracle.com> if you need additional information or have any
>>> + * questions.
>>> + */
>>> +
>>> +import java.util.ArrayList;
>>> +import java.util.Arrays;
>>> +import java.util.List;
>>> +import java.util.function.Consumer;
>>> +
>>> +import jdk.test.lib.apps.LingeredApp;
>>> +import jdk.test.lib.JDKToolLauncher;
>>> +import jdk.test.lib.Platform;
>>> +import jdk.test.lib.process.OutputAnalyzer;
>>> +import jdk.test.lib.Utils;
>>> +
>>> +/*
>>> + * @test
>>> + * @library /test/lib
>>> + * @run main/othervm JhsdbThreadNameTest
>>> + */
>>> +public class JhsdbThreadNameTest {
>>> +
>>> +    private static String notMixedModeThreadNames[] =
>>> {"Common-Cleaner", "Signal Dispatcher", "Finalizer", "Reference
>>> Handler", "main"};
>>> +    private static String mixedModeThreadNames[] = {"C2
>>> CompilerThread0", "C1 CompilerThread1", "Sweeper thread", "Service Thread"};
>>> +
>>> +    private static void startHsdbJstack(boolean mixed) throws Exception {
>>> +
>>> +        LingeredApp app = null;
>>> +
>>> +        try {
>>> +            List<String> vmArgs = new ArrayList<String>();
>>> +            vmArgs.add("-Xmx10m");
>>> +            vmArgs.addAll(Utils.getVmOptions());
>>> +
>>> +            app = LingeredApp.startApp(vmArgs);
>>> +            System.out.println("Started LingeredApp with pid " +
>>> app.getPid());
>>> +
>>> +            JDKToolLauncher jhsdbLauncher =
>>> JDKToolLauncher.createUsingTestJDK("jhsdb");
>>> +
>>> +            jhsdbLauncher.addToolArg("jstack");
>>> +            jhsdbLauncher.addToolArg("--pid");
>>> + jhsdbLauncher.addToolArg(Long.toString(app.getPid()));
>>> +
>>> +            if (mixed) {
>>> +                jhsdbLauncher.addToolArg("--mixed");
>>> +            }
>>> +            ProcessBuilder pb = new ProcessBuilder();
>>> +            pb.command(jhsdbLauncher.getCommand());
>>> +            Process jhsdb = pb.start();
>>> +
>>> +            jhsdb.waitFor();
>>> +
>>> +            OutputAnalyzer out = new OutputAnalyzer(jhsdb);
>>> +
>>> +
>>> Arrays.stream(notMixedModeThreadNames).map(JhsdbThreadNameTest::addQuotationMarks).forEach(out::shouldContain);
>>> +            Consumer<String> testMethod = null;
>>> +            if (mixed) {
>>> +                testMethod = out::shouldContain;
>>> +            } else {
>>> +                testMethod = out::shouldNotContain;
>>> +            }
>>> +
>>> Arrays.stream(mixedModeThreadNames).map(JhsdbThreadNameTest::addQuotationMarks).forEach(testMethod);
>>> +
>>> +        } catch (InterruptedException ie) {
>>> +            throw new Error("Problem awaiting the child process: " +
>>> ie, ie);
>>> +        } catch (Exception attachE) {
>>> +            throw new Error("Couldn't start jhsdb, attach to
>>> LingeredApp or match ThreadName: " + attachE);
>>> +
>>> +        } finally {
>>> +            LingeredApp.stopApp(app);
>>> +        }
>>> +    }
>>> +
>>> +    private static String addQuotationMarks(String str) {
>>> +        return "\"" + str + "\"";
>>> +    }
>>> +
>>> +    public static void main(String[] args) throws Exception {
>>> +
>>> +        if (!Platform.shouldSAAttach()) {
>>> +            System.out.println("SA attach not expected to work - test
>>> skipped.");
>>> +            return;
>>> +        }
>>> +
>>> +        startHsdbJstack(true);
>>> +        startHsdbJstack(false);
>>> +    }
>>> +}
>>>
>>>
>>> Regards,
>>> Chihiro
>>>
>>>
>>> On 2017/06/08 18:04, chihiro ito wrote:
>>> > Hi Jini,
>>> >
>>> > Thank you for your advices. I try to add the test case and modify the
>>> > copyright year to 2017.
>>> > Basically, I agree with your idea, but I think that the separator line
>>> > should finally be the same as the output of the jstack command. I
>>> > worry which is better way.
>>> >
>>> > Thanks,
>>> > Chihiro
>>> >
>>> > On 2017/06/08 16:42, Jini George wrote:
>>> >> Hi Chihiro,
>>> >>
>>> >> Thank you for making this useful change. Your changes look good.
>>> >>
>>> >> It would be great though if you could add a test case for this. Could
>>> >> you also modify the copyright year to 2017 ? One additional
>>> >> suggestion: The addition of the thread name makes the separator lines
>>> >> unaligned in the pstack/jstack --mixed cases. Like:
>>> >>
>>> >> ----------------- "Service Thread" nid=16051 -----------------
>>> >> and
>>> >> ----------------- nid=16052 -----------------
>>> >>
>>> >> So I am wondering if it would make sense to have the name printed out
>>> >> on a separate line to keep the separator lines aligned. But this is a
>>> >> nit, and I would leave it to you to decide on this.
>>> >>
>>> >> Thanks,
>>> >> Jini (Not a (R)eviewer)
>>> >>
>>> >> On 6/7/2017 3:16 PM, chihiro ito wrote:
>>> >>> Hi all,
>>> >>>
>>> >>> I changed to output Java thread name in jhsdb jstack as following.
>>> >>>
>>> >>> jhsdb jstack --pid 25879
>>> >>> "main" nid=25884: (state = BLOCKED)
>>> >>>
>>> >>> jhsdb jstack --mixed --pid 25879
>>> >>> ----------------- "main" nid=25884 -----------------
>>> >>>
>>> >>> Could you possibly review for this following small change? If review
>>> >>> is ok, please commit this as cito.
>>> >>>
>>> >>> Source:
>>> >>> diff --git
>>> >>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
>>> >>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
>>> >>> ---
>>> >>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
>>> >>> +++
>>> >>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java
>>> >>> @@ -86,6 +86,13 @@
>>> >>>              try {
>>> >>>                 CFrame f = cdbg.topFrameForThread(th);
>>> >>>                 out.print("----------------- ");
>>> >>> +               JavaThread jthread = (JavaThread)
>>> >>> proxyToThread.get(th);
>>> >>> +               if (jthread != null) {
>>> >>> +                   out.print("\"");
>>> >>> + out.print(jthread.getThreadName());
>>> >>> +                   out.print("\" ");
>>> >>> +               }
>>> >>> +               out.print("nid=");
>>> >>>                 out.print(th);
>>> >>>                 out.println(" -----------------");
>>> >>>                 while (f != null) {
>>> >>> diff --git
>>> >>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
>>> >>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
>>> >>>
>>> >>> ---
>>> >>> a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
>>> >>> +++
>>> >>> b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
>>> >>> @@ -75,7 +75,9 @@
>>> >>>              for (JavaThread cur = threads.first(); cur != null; cur
>>> >>> = cur.next(), i++) {
>>> >>>                  if (cur.isJavaThread()) {
>>> >>>                      Address sp = cur.getLastJavaSP();
>>> >>> -                    tty.print("Thread ");
>>> >>> +                    tty.print("\"");
>>> >>> + tty.print(cur.getThreadName());
>>> >>> +                    tty.print("\" nid=");
>>> >>>                      cur.printThreadIDOn(tty);
>>> >>>                      tty.print(": (state = " + cur.getThreadState());
>>> >>>                      if (verbose) {
>>> >>>
>>> >>> Regards,
>>> >>> Chihiro
>>> >>>
>>> >>
>>> >
>>>
>>> --
>>>
>>> Chihiro Ito | Principal Consultant | +81.90.6148.8815
>>> Oracle <http://www.oracle.com> Consultant
>>> ORACLE Japan | Akasaka Center Bldg. | Motoakasaka 1-3-13 | 1070051
>>> Minato-ku, Tokyo, JAPAN
>>>
>>> Oracle is committed to developing practices and products that help
>>> protect the environment <http://www.oracle.com/commitment>
>>
>

From robbin.ehn at oracle.com  Thu Jun 29 06:46:42 2017
From: robbin.ehn at oracle.com (Robbin Ehn)
Date: Thu, 29 Jun 2017 08:46:42 +0200
Subject: Low-Overhead Heap Profiling
In-Reply-To: <CAF9BGBx7KR9vHgBtYVS5cWKRwdX-ggLn2X2P8cmbgu4+AnV=vg@mail.gmail.com>
References: <CAF9BGBx8xFYudhJ=v1_hU5=nwEcC1X0NKRmC=E-RNN6SMs1ozg@mail.gmail.com>
 <CAF9BGBz81YYVUsMLw9sOMR9fH7JQmyb8d_OZ9aJ47EOAqwWhEw@mail.gmail.com>
 <2af975e6-3827-bd57-0c3d-fadd54867a67@oracle.com>
 <365499b6-3f4d-a4df-9e7e-e72a739fb26b@oracle.com>
 <CAF9BGBy0T7A-+1dejYbr6f0d4s0XR-zv-WwHYqXrbe0EtCdsxQ@mail.gmail.com>
 <fb1155c6-ccf8-843c-34fc-fc4a45fc54cc@oracle.com>
 <CAF9BGBxqmXF9OFD4F9wVb-DQOj_UfNNcBhe+ARVxJ5E4cgCC1Q@mail.gmail.com>
 <102c59b8-25b6-8c21-8eef-1de7d0bbf629@oracle.com>
 <CAF9BGBxDaCOxobJi8Uu9kwKPRpnj-KqgWLfJO2z7UX71UnNF4g@mail.gmail.com>
 <CAF9BGByisZchXTsw1=mLETLnOggJShRJBDxFeOUgt+criTTGuQ@mail.gmail.com>
 <1497366226.2829.109.camel@oracle.com>
 <CAF9BGBx8376y_yeMFnMC8PHSqKDaFc5NcbFerQ2up0YVKtSKGg@mail.gmail.com>
 <1498215147.2741.34.camel@oracle.com>
 <CAF9BGBwaRu+12iRMm6U+n8j_+3izaEcv1OExdFKqZczEE7SJ0A@mail.gmail.com>
 <044f8c75-72f3-79fd-af47-7ee875c071fd@oracle.com>
 <CAF9BGBx7KR9vHgBtYVS5cWKRwdX-ggLn2X2P8cmbgu4+AnV=vg@mail.gmail.com>
Message-ID: <23f4e6f5-c94e-01f7-ef1d-5e328d4823c8@oracle.com>

Hi JC,

On 06/29/2017 12:15 AM, JC Beyler wrote:
> B) Incremental changes

I guess the most common work flow here is using mq :
hg qnew fix_v1
edit files
hg qrefresh
hg qnew fix_v2
edit files
hg qrefresh

if you do hg log you will see 2 commits

webrev.ksh -r -2 -o my_inc_v1_v2
webrev.ksh -o my_full_v2


In  your .hgrc you might need:
[extensions]
mq =

/Robbin

> 
> Again another newbiew question here...
> 
> For showing the incremental changes, is there a link that explains how to do that? I apologize for my newbie questions all the time :)
> 
> Right now, I do:
> 
>   ksh ../webrev.ksh -m -N
> 
> That generates a webrev.zip and send it to Chuck Rasbold. He then uploads it to a new webrev.
> 
> I tried commiting my change and adding a small change. Then if I just do ksh ../webrev.ksh without any options, it seems to produce a similar page but now with only the 
> changes I had (so the 06-07 comparison you were talking about) and a changeset that has it all. I imagine that is what you meant.
> 
> Which means that my workflow would become:
> 
> 1) Make changes
> 2) Make a webrev without any options to show just the differences with the tip
> 3) Amend my changes to my local commit so that I have it done with
> 4) Go to 1
> 
> Does that seem correct to you?
> 
> Note that when I do this, I only see the full change of a file in the full change set (Side note here: now the page says change set and not patch, which is maybe why 
> Serguei was having issues?).
> 
> Thanks!
> Jc
> 
> 
> 
> On Wed, Jun 28, 2017 at 1:12 AM, Robbin Ehn <robbin.ehn at oracle.com <mailto:robbin.ehn at oracle.com>> wrote:
> 
>     Hi,
> 
>     On 06/28/2017 12:04 AM, JC Beyler wrote:
> 
>         Dear Thomas et al,
> 
>         Here is the newest webrev:
>         http://cr.openjdk.java.net/~rasbold/8171119/webrev.07/ <http://cr.openjdk.java.net/~rasbold/8171119/webrev.07/>
> 
> 
>     You have some more bits to in there but generally this looks good and really nice with more tests.
>     I'll do and deep dive and re-test this when I get back from my long vacation with whatever patch version you have then.
> 
>     Also I think it's time you provide incremental (v06->07 changes) as well as complete change-sets.
> 
>     Thanks, Robbin
> 
> 
> 
> 
>         Thomas, I "think" I have answered all your remarks. The summary is:
> 
>         - The statistic system is up and provides insight on what the heap sampler is doing
>              - I've noticed that, though the sampling rate is at the right mean, we are missing some samples, I have not yet tracked out why (details below)
> 
>         - I've run a tiny benchmark that is the worse case: it is a very tight loop and allocated a small array
>              - In this case, I see no overhead when the system is off so that is a good start :)
>              - I see right now a high overhead in this case when sampling is on. This is not a really too surprising but I'm going to see if this is consistent with our
>         internal implementation. The benchmark is really allocation stressful so I'm not too surprised but I want to do the due diligence.
> 
>            - The statistic system up is up and I have a new test
>         http://cr.openjdk.java.net/~rasbold/8171119/webrev.07/test/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatTest.java.patch
>         <http://cr.openjdk.java.net/~rasbold/8171119/webrev.07/test/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatTest.java.patch>
>               - I did a bit of a study about the random generator here, more details are below but basically it seems to work well
> 
>            - I added a capability but since this is the first time doing this, I was not sure I did it right
>              - I did add a test though for it and the test seems to do what I expect (all methods are failing with the JVMTI_ERROR_MUST_POSSESS_CAPABILITY error).
>                  - http://cr.openjdk.java.net/~rasbold/8171119/webrev.07/test/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorNoCapabilityTest.java.patch
>         <http://cr.openjdk.java.net/~rasbold/8171119/webrev.07/test/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorNoCapabilityTest.java.patch>
> 
>            - I still need to figure out what to do about the multi-agent vs single-agent issue
> 
>            - As far as measurements, it seems I still need to look at:
>              - Why we do the 20 random calls first, are they necessary?
>              - Look at the mean of the sampling rate that the random generator does and also what is actually sampled
>              - What is the overhead in terms of memory/performance when on?
> 
>         I have inlined my answers, I think I got them all in the new webrev, let me know your thoughts.
> 
>         Thanks again!
>         Jc
> 
> 
>         On Fri, Jun 23, 2017 at 3:52 AM, Thomas Schatzl <thomas.schatzl at oracle.com <mailto:thomas.schatzl at oracle.com> <mailto:thomas.schatzl at oracle.com
>         <mailto:thomas.schatzl at oracle.com>>> wrote:
> 
>              Hi,
> 
>              On Wed, 2017-06-21 at 13:45 -0700, JC Beyler wrote:
>              > Hi all,
>              >
>              > First off: Thanks again to Robbin and Thomas for their reviews :)
>              >
>              > Next, I've uploaded a new webrev:
>              > http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/ <http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/>
>         <http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/ <http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/>>
> 
>              >
>              > Here is an update:
>              >
>              > - @Robbin, I forgot to say that yes I need to look at implementing
>              > this for the other architectures and testing it before it is all
>              > ready to go. Is it common to have it working on all possible
>              > combinations or is there a subset that I should be doing first and we
>              > can do the others later?
>              > - I've tested slowdebug, built and ran the JTreg tests I wrote with
>              > slowdebug and fixed a few more issues
>              > - I've refactored a bit of the code following Thomas' comments
>              >    - I think I've handled all the comments from Thomas (I put
>              > comments inline below for the specifics)
> 
>              Thanks for handling all those.
> 
>              > - Following Thomas' comments on statistics, I want to add some
>              > quality assurance tests and find that the easiest way would be to
>              > have a few counters of what is happening in the sampler and expose
>              > that to the user.
>              >    - I'll be adding that in the next version if no one sees any
>              > objections to that.
>              >    - This will allow me to add a sanity test in JTreg about number of
>              > samples and average of sampling rate
>              >
>              > @Thomas: I had a few questions that I inlined below but I will
>              > summarize the "bigger ones" here:
>              >    - You mentioned constants are not using the right conventions, I
>              > looked around and didn't see any convention except normal naming then
>              > for static constants. Is that right?
> 
>              I looked through https://wiki.openjdk.java.net/display/HotSpot/StyleGui <https://wiki.openjdk.java.net/display/HotSpot/StyleGui>
>         <https://wiki.openjdk.java.net/display/HotSpot/StyleGui <https://wiki.openjdk.java.net/display/HotSpot/StyleGui>>
>              de and the rule is to "follow an existing pattern and must have a
>              distinct appearance from other names". Which does not help a lot I
>              guess :/ The GC team started using upper camel case, e.g.
>              SomeOtherConstant, but very likely this is probably not applied
>              consistently throughout. So I am fine with not adding another style
>              (like kMaxStackDepth with the "k" in front with some unknown meaning)
>              is fine.
> 
>              (Chances are you will find that style somewhere used anyway too,
>              apologies if so :/)
> 
> 
>         Thanks for that link, now I know where to look. I used the upper camel case in my code as well then :) I should have gotten them all.
> 
> 
>               > PS: I've also inlined my answers to Thomas below:
>               >
>               > On Tue, Jun 13, 2017 at 8:03 AM, Thomas Schatzl <thomas.schatzl at oracl
>               > e.com <http://e.com> <http://e.com>> wrote:
>               > > Hi all,
>               > >
>               > > On Mon, 2017-06-12 at 11:11 -0700, JC Beyler wrote:
>               > > > Dear all,
>               > > >
>               > > > I've continued working on this and have done the following
>               > > webrev:
>               > > > http://cr.openjdk.java.net/~rasbold/8171119/webrev.05/ <http://cr.openjdk.java.net/~rasbold/8171119/webrev.05/>
>         <http://cr.openjdk.java.net/~rasbold/8171119/webrev.05/ <http://cr.openjdk.java.net/~rasbold/8171119/webrev.05/>>
> 
>               > >
>               > > [...]
>               > > > Things I still need to do:
>               > > >    - Have to fix that TLAB case for the FastTLABRefill
>               > > >    - Have to start looking at the data to see that it is
>               > > consistent and does gather the right samples, right frequency, etc.
>               > > >    - Have to check the GC elements and what that produces
>               > > >    - Run a slowdebug run and ensure I fixed all those issues you
>               > > saw > Robbin
>               > > >
>               > > > Thanks for looking at the webrev and have a great week!
>               > >
>               > >   scratching a bit on the surface of this change, so apologies for
>               > > rather shallow comments:
>               > >
>               > > - macroAssembler_x86.cpp:5604: while this is compiler code, and I
>               > > am not sure this is final, please avoid littering the code with
>               > > TODO remarks :) They tend to be candidates for later wtf moments
>               > > only.
>               > >
>               > > Just file a CR for that.
>               > >
>               > Newcomer question: what is a CR and not sure I have the rights to do
>               > that yet ? :)
> 
>              Apologies. CR is a change request, this suggests to file a bug in the
>              bug tracker. And you are right, you can't just create a new account in
>              the OpenJDK JIRA yourselves. :(
> 
> 
>         Ok good to know, I'll continue with my own todo list but I'll work hard on not letting it slip in the webrevs anymore :)
> 
> 
>              I was mostly referring to the "... but it is a TODO" part of that
>              comment in macroassembler_x86.cpp. Comments about the why of the code
>              are appreciated.
> 
>              [Note that I now understand that this is to some degree still work in
>              progress. As long as the final changeset does no contain TODO's I am
>              fine (and it's not a hard objection, rather their use in "final" code
>              is typically limited in my experience)]
> 
>              5603   // Currently, if this happens, just set back the actual end to
>              where it was.
>              5604   // We miss a chance to sample here.
> 
>              Would be okay, if explaining "this" and the "why" of missing a chance
>              to sample here would be best.
> 
>              Like maybe:
> 
>              // If we needed to refill TLABs, just set the actual end point to
>              // the end of the TLAB again. We do not sample here although we could.
> 
>         Done with your comment, it works well in my mind.
> 
>              I am not sure whether "miss a chance to sample" meant "we could, but
>              consciously don't because it's not that useful" or "it would be
>              necessary but don't because it's too complicated to do.".
> 
>              Looking at the original comment once more, I am also not sure if that
>              comment shouldn't referring to the "end" variable (not actual_end)
>              because that's the variable that is responsible for taking the sampling
>              path? (Going from the member description of ThreadLocalAllocBuffer).
> 
> 
>         I've moved this code and it no longer shows up here but the rationale and answer was:
> 
>         So.. Yes, end is the variable provoking the sampling. Actual end is the actual end of the TLAB.
> 
>         What was happening here is that the code is resetting _end to point towards the end of the new TLAB. Because, we now have the end for sampling and _actual_end for
>         the actual end, we need to update the actual_end as well.
> 
>         Normally, were we to do the real work here, we would calculate the (end - start) offset, then do:
> 
>         - Set the new end to : start + (old_end - old_start)
>         - Set the actual end like we do here now where it because it is the actual end.
> 
>         Why is this not done here now anymore?
>             - I was still debating which path to take:
>                - Do it in the fast refill code, it has its perks:
>                    - In a world where fast refills are happening all the time or a lot, we can augment there the code to do the sampling
>                - Remember what we had as an end before leaving the slowpath and check on return
>                    - This is what I'm doing now, it removes the need to go fix up all fast refill paths but if you remain in fast refill paths, you won't get sampling. I
>         have to think of the consequences of that, maybe a future change later on?
>                       - I have the statistics now so I'm going to study that
>                          -> By the way, though my statistics are showing I'm missing some samples, if I turn off FastTlabRefill, it is the same loss so for now, it seems
>         this does not occur in my simple test.
> 
> 
> 
>              But maybe I am only confused and it's best to just leave the comment
>              away. :)
> 
>              Thinking about it some more, doesn't this not-sampling in this case
>              mean that sampling does not work in any collector that does inline TLAB
>              allocation at the moment? (Or is inline TLAB alloc automatically
>              disabled with sampling somehow?)
> 
>              That would indeed be a bigger TODO then :)
> 
> 
>         Agreed, this remark made me think that perhaps as a first step the new way of doing it is better but I did have to:
>            - Remove the const of the ThreadLocalBuffer remaining and hard_end methods
>            - Move hard_end out of the header file to have a bit more logic there
> 
>         Please let me know what you think of that and if you prefer it this way or changing the fast refills. (I prefer this way now because it is more incremental).
> 
> 
>              > > - calling HeapMonitoring::do_weak_oops() (which should probably be
>              > > called weak_oops_do() like other similar methods) only if string
>              > > deduplication is enabled (in g1CollectedHeap.cpp:4511) seems wrong.
>              >
>              > The call should be at least around 6 lines up outside the if.
>              >
>              > Preferentially in a method like process_weak_jni_handles(), including
>              > additional logging. (No new (G1) gc phase without minimal logging
>              > :)).
>              > Done but really not sure because:
>              >
>              > I put for logging:
>              >   log_develop_trace(gc, freelist)("G1ConcRegionFreeing [other] : heap
>              > monitoring");
> 
>              I would think that "gc, ref" would be more appropriate log tags for
>              this similar to jni handles.
>              (I am als not sure what weak reference handling has to do with
>              G1ConcRegionFreeing, so I am a bit puzzled)
> 
> 
>         I was not sure what to put for the tags or really as the message. I cleaned it up a bit now to:
>              log_develop_trace(gc, ref)("HeapSampling [other] : heap monitoring processing");
> 
> 
> 
>              > Since weak_jni_handles didn't have logging for me to be inspired
>              > from, I did that but unconvinced this is what should be done.
> 
>              The JNI handle processing does have logging, but only in
>              ReferenceProcessor::process_discovered_references(). In
>              process_weak_jni_handles() only overall time is measured (in a G1
>              specific way, since only G1 supports disabling reference procesing) :/
> 
>              The code in ReferenceProcessor prints both time taken
>              referenceProcessor.cpp:254, as well as the count, but strangely only in
>              debug VMs.
> 
>              I have no idea why this logging is that unimportant to only print that
>              in a debug VM. However there are reviews out for changing this area a
>              bit, so it might be useful to wait for that (JDK-8173335).
> 
> 
>         I cleaned it up a bit anyway and now it returns the count of objects that are in the system.
> 
> 
>              > > - the change doubles the size of
>              > > CollectedHeap::allocate_from_tlab_slow() above the "small and nice"
>              > > threshold. Maybe it could be refactored a bit.
>              > Done I think, it looks better to me :).
> 
>              In ThreadLocalAllocBuffer::handle_sample() I think the
>              set_back_actual_end()/pick_next_sample() calls could be hoisted out of
>              the "if" :)
> 
> 
>         Done!
> 
> 
>              > > - referenceProcessor.cpp:261: the change should add logging about
>              > > the number of references encountered, maybe after the corresponding
>              > > "JNI weak reference count" log message.
>              > Just to double check, are you saying that you'd like to have the heap
>              > sampler to keep in store how many sampled objects were encountered in
>              > the HeapMonitoring::weak_oops_do?
>              >    - Would a return of the method with the number of handled
>              > references and logging that work?
> 
>              Yes, it's fine if HeapMonitoring::weak_oops_do() only returned the
>              number of processed weak oops.
> 
> 
>         Done also (but I admit I have not tested the output yet) :)
> 
> 
>              >    - Additionally, would you prefer it in a separate block with its
>              > GCTraceTime?
> 
>              Yes. Both kinds of information is interesting: while the time taken is
>              typically more important, the next question would be why, and the
>              number of references typically goes a long way there.
> 
>              See above though, it is probably best to wait a bit.
> 
> 
>         Agreed that I "could" wait but, if it's ok, I'll just refactor/remove this when we get closer to something final. Either, JDK-8173335
>         has gone in and I will notice it now or it will soon and I can change it then.
> 
> 
>              > > - threadLocalAllocBuffer.cpp:331: one more "TODO"
>              > Removed it and added it to my personal todos to look at.
>              >      > >
>              > > - threadLocalAllocBuffer.hpp: ThreadLocalAllocBuffer class
>              > > documentation should be updated about the sampling additions. I
>              > > would have no clue what the difference between "actual_end" and
>              > > "end" would be from the given information.
>              > If you are talking about the comments in this file, I made them more
>              > clear I hope in the new webrev. If it was somewhere else, let me know
>              > where to change.
> 
>              Thanks, that's much better. Maybe a note in the comment of the class
>              that ThreadLocalBuffer provides some sampling facility by modifying the
>              end() of the TLAB to cause "frequent" calls into the runtime call where
>              actual sampling takes place.
> 
> 
>         Done, I think it's better now. Added something about the slow_path_end as well.
> 
> 
>              > > - in heapMonitoring.hpp: there are some random comments about some
>              > > code that has been grabbed from "util/math/fastmath.[h|cc]". I
>              > > can't tell whether this is code that can be used but I assume that
>              > > Noam Shazeer is okay with that (i.e. that's all Google code).
>              > Jeremy and I double checked and we can release that as I thought. I
>              > removed the comment from that piece of code entirely.
> 
>              Thanks.
> 
>              > > - heapMonitoring.hpp/cpp static constant naming does not correspond
>              > > to Hotspot's. Additionally, in Hotspot static methods are cased
>              > > like other methods.
>              > I think I fixed the methods to be cased the same way as all other
>              > methods. For static constants, I was not sure. I fixed a few other
>              > variables but I could not seem to really see a consistent trend for
>              > constants. I made them as variables but I'm not sure now.
> 
>              Sorry again, style is a kind of mess. The goal of my suggestions here
>              is only to prevent yet another style creeping in.
> 
>              > > - in heapMonitoring.cpp there are a few cryptic comments at the top
>              > > that seem to refer to internal stuff that should probably be
>              > > removed.
>              > Sorry about that! My personal todos not cleared out.
> 
>              I am happy about comments, but I simply did not understand any of that
>              and I do not know about other readers as well.
> 
>              If you think you will remember removing/updating them until the review
>              proper (I misunderstood the review situation a little it seems).
> 
>              > > I did not think through the impact of the TLAB changes on collector
>              > > behavior yet (if there are). Also I did not check for problems with
>              > > concurrent mark and SATB/G1 (if there are).
>              > I would love to know your thoughts on this, I think this is fine. I
> 
>              I think so too now. No objects are made live out of thin air :)
> 
>              > see issues with multiple threads right now hitting the stack storage
>              > instance. Previous webrevs had a mutex lock here but we took it out
>              > for simplificity (and only for now).
> 
>              :) When looking at this after some thinking I now assume for this
>              review that this code is not MT safe at all. There seems to be more
>              synchronization missing than just the one for the StackTraceStorage. So
>              no comments about this here.
> 
> 
>         I doubled checked a bit (quickly I admit) but it seems that synchronization in StackTraceStorage is really all you need (all methods lead to a StackTraceStorage one
>         and can be multithreaded outside of that).
>         There is a question about the initialization where the method HeapMonitoring::initialize_profiling is not thread safe.
>         It would work (famous last words) and not crash if there was a race but we could add a synchronization point there as well (and therefore on the stop as well).
> 
>         But anyway I will really check and do this once we add back synchronization.
> 
> 
>              Also, this would require some kind of specification of what is allowed
>              to be called when and where.
> 
> 
>         Would we specify this with the methods in the jvmti.xml file? We could start by specifying in each that they are not thread safe but I saw no mention of that for
>         other methods.
> 
> 
>              One potentially relevant observation about locking here: depending on
>              sampling frequency, StackTraceStore::add_trace() may be rather
>              frequently called. I assume that you are going to do measurements :)
> 
> 
>         Though we don't have the TLAB implementation in our code, the compiler generated sampler uses 2% of overhead with a 512k sampling rate. I can do real measurements
>         when the code settles and we can see how costly this is as a TLAB implementation.
>         However, my theory is that if the rate is 512k, the memory/performance overhead should be minimal since it is what we saw with our code/workloads (though not called
>         the same way, we call it essentially at the same rate).
>         If you have a benchmark you'd like me to test, let me know!
> 
>         Right now, with my really small test, this does use a bit of overhead even for a 512k sample size. I don't know yet why, I'm going to see what is going on.
> 
>         Finally, I think it is not reasonable to suppose the overhead to be negligible if the sampling rate used is too low. The user should know that the lower the rate,
>         the higher the overhead (documentation TODO?).
> 
> 
>              I am not sure what the expected usage of the API is, but
>              StackTraceStore::add_trace() seems to be able to grow without bounds.
>              Only a GC truncates them to the live ones. That in itself seems to be
>              problematic (GCs can be *wide* apart), and of course some of the API
>              methods add to that because they duplicate that unbounded array. Do you
>              have any concerns/measurements about this?
> 
> 
>         So, the theory is that yes add_trace can be able to grow without bounds but it grows at a sample per 512k of allocated space. The stacks it gathers are currently
>         maxed at 64 (I'd like to expand that to an option to the user though at some point). So I have no concerns because:
> 
>         - If really this is taking a lot of space, that means the job is keeping a lot of objects in memory as well, therefore the entire heap is getting huge
>         - If this is the case, you will be triggering a GC at some point anyway.
> 
>         (I'm putting under the rug the issue of "What if we set the rate to 1 for example" because as you lower the sampling rate, we cannot guarantee low overhead; the
>         idea behind this feature is to have a means of having meaningful allocated samples at a low overhead)
> 
>         I have no measurements really right now but since I now have some statistics I can poll, I will look a bit more at this question.
> 
>         I have the same last sentence than above: the user should expect this to happen if the sampling rate is too small. That probably can be reflected in the
>         StartHeapSampling as a note : careful this might impact your performance.
> 
> 
>              Also, these stack traces might hold on to huge arrays. Any
>              consideration of that? Particularly it might be the cause for OOMEs in
>              tight memory situations.
> 
> 
>         There is a stack size maximum that is set to 64 so it should not hold huge arrays. I don't think this is an issue but I can double check with a test or two.
> 
> 
>              - please consider adding a safepoint check in
>              HeapMonitoring::weak_oops_do to prevent accidental misuse.
> 
>              - in struct StackTraceStorage, the public fields may also need
>              underscores. At least some files in the runtime directory have structs
>              with underscored public members (and some don't). The runtime team
>              should probably comment on that.
> 
> 
>         Agreed I did not know. I looked around and a lot of structs did not have them it seemed so I left it as is. I will happily change it if someone prefers (I was not
>         sure if you really preferred or not, your sentence seemed to be more a note of "this might need to change but I don't know if the runtime team enforces that", let
>         me know if I read that wrongly).
> 
> 
>              - In StackTraceStorage::weak_oops_do(), when examining the
>              StackTraceData, maybe it is useful to consider having a non-NULL
>              reference outside of the heap's reserved space an error. There should
>              be no oop outside of the heap's reserved space ever.
> 
>              Unless you allow storing random values in StackTraceData::obj, which I
>              would not encourage.
> 
> 
>         I suppose you are talking about this part:
>         if ((value != NULL && Universe::heap()->is_in_reserved(value)) &&
>                     (is_alive == NULL || is_alive->do_object_b(value))) {
> 
>         What you are saying is that I could have something like:
>         if (value != my_non_null_reference &&
>                     (is_alive == NULL || is_alive->do_object_b(value))) {
> 
>         Is that what you meant? Is there really a reason to do so? When I look at the code, is_in_reserved seems like a O(1) method call. I'm not even sure we can have a
>         NULL value to be honest. I might have to study that to see if this was not a paranoid test to begin with.
> 
>         The is_alive code has now morphed due to the comment below.
> 
> 
> 
>              - HeapMonitoring::weak_oops_do() does not seem to use the
>              passed AbstractRefProcTaskExecutor.
> 
> 
>         It did use it:
>            size_t HeapMonitoring::weak_oops_do(
>               AbstractRefProcTaskExecutor *task_executor,
>               BoolObjectClosure* is_alive,
>               OopClosure *f,
>               VoidClosure *complete_gc) {
>             assert(SafepointSynchronize::is_at_safepoint(), "must be at safepoint");
> 
>             if (task_executor != NULL) {
>               task_executor->set_single_threaded_mode();
>             }
>             return StackTraceStorage::storage()->weak_oops_do(is_alive, f, complete_gc);
>         }
> 
>         But due to the comment below, I refactored this, so this is no longer here. Now I have an always true closure that is passed.
> 
> 
>              - I do not understand allowing to call this method with a NULL
>              complete_gc closure. This would mean that objects referenced from the
>              object that is referenced by the StackTraceData are not pulled, meaning
>              they would get stale.
> 
>              - same with is_alive parameter value of NULL
> 
> 
>         So these questions made me look a bit closer at this code. This code I think was written this way to have a very small impact on the file but you are right, there
>         is no reason for this here. I've simplified the code by making in referenceProcessor.cpp a process_HeapSampling method that handles everything there.
> 
>         The code allowed NULLs because it depended on where you were coming from and how the code was being called.
> 
>         - I added a static always_true variable and pass that now to be more consistent with the rest of the code.
>         - I moved the complete_gc into process_phaseHeapSampling now (new method) and handle the task_executor and the complete_gc there
>              - Newbie question: in our code we did a set_single_threaded_mode but I see that process_phaseJNI does it right before its call, do I need to do it for the
>         process_phaseHeapSample?
>         That API is much cleaner (in my mind) and is consistent with what is done around it (again in my mind).
> 
> 
>              - heapMonitoring.cpp:590: I do not completely understand the purpose of
>              this code: in the end this results in a fixed value directly dependent
>              on the Thread address anyway? In the end this results in a fixed value
>              directly dependent on the Thread address anyway?
>              IOW, what is special about exactly 20 rounds?
> 
> 
>         So we really want a fast random number generator that has a specific mean (512k is the default we use). The code uses the thread address as the start number of the
>         sequence (why not, it is random enough is rationale). Then instead of just starting there, we prime the sequence and really only start at the 21st number, it is
>         arbitrary and I have not done a study to see if we could do more or less of that.
> 
>         As I have the statistics of the system up and running, I'll run some experiments to see if this is needed, is 20 good, or not.
> 
> 
>              - also I would consider stripping a few bits of the threads' address as
>              initialization value for your rng. The last three bits (and probably
>              more, check whether the Thread object is allocated on special
>              boundaries) are always zero for them.
>              Not sure if the given "random" value is random enough before/after,
>              this method, so just skip that comment if you think this is not
>              required.
> 
> 
>         I don't know is the honest answer. I think what is important is that we tend towards a mean and it is random "enough" to not fall in pitfalls of only sampling a
>         subset of objects due to their allocation order. I added that as test to do to see if it changes the mean in any way for the 512k default value and/or if the first
>         1000 elements look better.
> 
> 
>              Some more random nits I did not find a place to put anywhere:
> 
>              - ThreadLocalAllocBuffer::_extra_space does not seem to be used
>              anywhere?
> 
> 
>         Good catch :).
> 
> 
>              - Maybe indent the declaration of ThreadLocalAllocBuffer::_bytes_until_sample to align below the other members of that group.
> 
> 
>         Done moved it up a bit to have non static members together and static separate.
> 
>              Thanks,
>                 Thomas
> 
> 
>         Thanks for your review!
>         Jc
> 
> 

From andrew_m_leonard at uk.ibm.com  Thu Jun 29 13:57:28 2017
From: andrew_m_leonard at uk.ibm.com (Andrew Leonard)
Date: Thu, 29 Jun 2017 14:57:28 +0100
Subject: RFR 8183123 : JDP packets have no processId context set
Message-ID: <OF1D472515.ACDCB152-ON8025814D.00513AE8-8025814E.004CAD09@notes.na.collabserv.com>

Hi All,
Please can I get some review feedback for my changes for this issue: 
https://bugs.openjdk.java.net/browse/JDK-8183123
The webrev patch has been uploaded here: 
http://cr.openjdk.java.net/~clanger/webrevs/8183123.0/

Essentially the fix entails:
- Replacing invalid process id query logic with call to 
ProcessHandle.current().getPid().
- Update testcase to cover the failing scenario. Thus it fails without my 
patch, and succeeds with it.

Thanks
Andrew

Andrew Leonard
Java Runtimes Development
IBM Hursley
IBM United Kingdom Ltd
Phone internal: 245913, external: 01962 815913
internet email: andrew_m_leonard at uk.ibm.com 

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170629/5baaa5d3/attachment.html>

From christoph.langer at sap.com  Thu Jun 29 14:38:01 2017
From: christoph.langer at sap.com (Langer, Christoph)
Date: Thu, 29 Jun 2017 14:38:01 +0000
Subject: RFR 8183123 : JDP packets have no processId context set
In-Reply-To: <OF1D472515.ACDCB152-ON8025814D.00513AE8-8025814E.004CAD09@notes.na.collabserv.com>
References: <OF1D472515.ACDCB152-ON8025814D.00513AE8-8025814E.004CAD09@notes.na.collabserv.com>
Message-ID: <70b2804ebcf14b3694a35f6038d99555@sap.com>

Hi Andrew,

the fix looks ok to me. I will sponsor it after we get another review.

The copyright years need to be updated but I can do that before pushing. In the test I think it looks nicer to put the new "import static ..." line with one line of spacing to the others. But that's maybe my personal taste. I could rearrange that before pushing if you like.

Did you run the jtreg tests, e.g. group 'jdk_management'?

Best regards
Christoph

From: serviceability-dev [mailto:serviceability-dev-bounces at openjdk.java.net] On Behalf Of Andrew Leonard
Sent: Donnerstag, 29. Juni 2017 15:57
To: serviceability-dev at openjdk.java.net
Subject: RFR 8183123 : JDP packets have no processId context set

Hi All,
Please can I get some review feedback for my changes for this issue: https://bugs.openjdk.java.net/browse/JDK-8183123
The webrev patch has been uploaded here: http://cr.openjdk.java.net/~clanger/webrevs/8183123.0/

Essentially the fix entails:
- Replacing invalid process id query logic with call to ProcessHandle.current().getPid().
- Update testcase to cover the failing scenario. Thus it fails without my patch, and succeeds with it.

Thanks
Andrew

Andrew Leonard
Java Runtimes Development
IBM Hursley
IBM United Kingdom Ltd
Phone internal: 245913, external: 01962 815913
internet email: andrew_m_leonard at uk.ibm.com<mailto:andrew_m_leonard at uk.ibm.com>

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170629/9723f916/attachment.html>

From andrew_m_leonard at uk.ibm.com  Thu Jun 29 16:15:52 2017
From: andrew_m_leonard at uk.ibm.com (Andrew Leonard)
Date: Thu, 29 Jun 2017 17:15:52 +0100
Subject: RFR 8183123 : JDP packets have no processId context set
In-Reply-To: <70b2804ebcf14b3694a35f6038d99555@sap.com>
References: <OF1D472515.ACDCB152-ON8025814D.00513AE8-8025814E.004CAD09@notes.na.collabserv.com>
 <70b2804ebcf14b3694a35f6038d99555@sap.com>
Message-ID: <OF5E4315B2.D13BFBF3-ON8025814E.005943AC-8025814E.00595916@notes.na.collabserv.com>

Thanks Christoph,
Yes, jdk_management group all pass :-)

==============================
Test summary
==============================
   TEST                                                           TOTAL 
PASS  FAIL ERROR 
   jtreg:jdk/test:jdk_management                       129   129     0 0 
==============================
TEST SUCCESS

Cheers
Andrew

Andrew Leonard
Java Runtimes Development
IBM Hursley
IBM United Kingdom Ltd
Phone internal: 245913, external: 01962 815913
internet email: andrew_m_leonard at uk.ibm.com 




From:   "Langer, Christoph" <christoph.langer at sap.com>
To:     Andrew Leonard <andrew_m_leonard at uk.ibm.com>
Cc:     "serviceability-dev at openjdk.java.net" 
<serviceability-dev at openjdk.java.net>
Date:   29/06/2017 15:38
Subject:        RE: RFR 8183123 : JDP packets have no processId context 
set



Hi Andrew,
 
the fix looks ok to me. I will sponsor it after we get another review.
 
The copyright years need to be updated but I can do that before pushing. 
In the test I think it looks nicer to put the new ?import static ?? line 
with one line of spacing to the others. But that?s maybe my personal 
taste. I could rearrange that before pushing if you like.
 
Did you run the jtreg tests, e.g. group ?jdk_management??
 
Best regards
Christoph
 
From: serviceability-dev [
mailto:serviceability-dev-bounces at openjdk.java.net] On Behalf Of Andrew 
Leonard
Sent: Donnerstag, 29. Juni 2017 15:57
To: serviceability-dev at openjdk.java.net
Subject: RFR 8183123 : JDP packets have no processId context set
 
Hi All, 
Please can I get some review feedback for my changes for this issue: 
https://bugs.openjdk.java.net/browse/JDK-8183123 
The webrev patch has been uploaded here: 
http://cr.openjdk.java.net/~clanger/webrevs/8183123.0/ 

Essentially the fix entails: 
- Replacing invalid process id query logic with call to 
ProcessHandle.current().getPid(). 
- Update testcase to cover the failing scenario. Thus it fails without my 
patch, and succeeds with it. 

Thanks 
Andrew 

Andrew Leonard
Java Runtimes Development
IBM Hursley
IBM United Kingdom Ltd
Phone internal: 245913, external: 01962 815913
internet email: andrew_m_leonard at uk.ibm.com 

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170629/736b5b0d/attachment-0001.html>

From jcbeyler at google.com  Fri Jun 30 05:01:30 2017
From: jcbeyler at google.com (JC Beyler)
Date: Thu, 29 Jun 2017 22:01:30 -0700
Subject: Low-Overhead Heap Profiling
In-Reply-To: <23f4e6f5-c94e-01f7-ef1d-5e328d4823c8@oracle.com>
References: <CAF9BGBx8xFYudhJ=v1_hU5=nwEcC1X0NKRmC=E-RNN6SMs1ozg@mail.gmail.com>
 <CAF9BGBz81YYVUsMLw9sOMR9fH7JQmyb8d_OZ9aJ47EOAqwWhEw@mail.gmail.com>
 <2af975e6-3827-bd57-0c3d-fadd54867a67@oracle.com>
 <365499b6-3f4d-a4df-9e7e-e72a739fb26b@oracle.com>
 <CAF9BGBy0T7A-+1dejYbr6f0d4s0XR-zv-WwHYqXrbe0EtCdsxQ@mail.gmail.com>
 <fb1155c6-ccf8-843c-34fc-fc4a45fc54cc@oracle.com>
 <CAF9BGBxqmXF9OFD4F9wVb-DQOj_UfNNcBhe+ARVxJ5E4cgCC1Q@mail.gmail.com>
 <102c59b8-25b6-8c21-8eef-1de7d0bbf629@oracle.com>
 <CAF9BGBxDaCOxobJi8Uu9kwKPRpnj-KqgWLfJO2z7UX71UnNF4g@mail.gmail.com>
 <CAF9BGByisZchXTsw1=mLETLnOggJShRJBDxFeOUgt+criTTGuQ@mail.gmail.com>
 <1497366226.2829.109.camel@oracle.com>
 <CAF9BGBx8376y_yeMFnMC8PHSqKDaFc5NcbFerQ2up0YVKtSKGg@mail.gmail.com>
 <1498215147.2741.34.camel@oracle.com>
 <CAF9BGBwaRu+12iRMm6U+n8j_+3izaEcv1OExdFKqZczEE7SJ0A@mail.gmail.com>
 <044f8c75-72f3-79fd-af47-7ee875c071fd@oracle.com>
 <CAF9BGBx7KR9vHgBtYVS5cWKRwdX-ggLn2X2P8cmbgu4+AnV=vg@mail.gmail.com>
 <23f4e6f5-c94e-01f7-ef1d-5e328d4823c8@oracle.com>
Message-ID: <CAF9BGBzbf6e809jNQKh1C-zco9VR-Z003Qu6dKcr7yM4a_Zjbg@mail.gmail.com>

Thanks Robbin,

This seems to have worked. When I have the next webrev ready, we will find
out but I'm fairly confident it will work!

Thanks agian!
Jc

On Wed, Jun 28, 2017 at 11:46 PM, Robbin Ehn <robbin.ehn at oracle.com> wrote:

> Hi JC,
>
> On 06/29/2017 12:15 AM, JC Beyler wrote:
>
>> B) Incremental changes
>>
>
> I guess the most common work flow here is using mq :
> hg qnew fix_v1
> edit files
> hg qrefresh
> hg qnew fix_v2
> edit files
> hg qrefresh
>
> if you do hg log you will see 2 commits
>
> webrev.ksh -r -2 -o my_inc_v1_v2
> webrev.ksh -o my_full_v2
>
>
> In  your .hgrc you might need:
> [extensions]
> mq =
>
> /Robbin
>
>
>> Again another newbiew question here...
>>
>> For showing the incremental changes, is there a link that explains how to
>> do that? I apologize for my newbie questions all the time :)
>>
>> Right now, I do:
>>
>>   ksh ../webrev.ksh -m -N
>>
>> That generates a webrev.zip and send it to Chuck Rasbold. He then uploads
>> it to a new webrev.
>>
>> I tried commiting my change and adding a small change. Then if I just do
>> ksh ../webrev.ksh without any options, it seems to produce a similar page
>> but now with only the changes I had (so the 06-07 comparison you were
>> talking about) and a changeset that has it all. I imagine that is what you
>> meant.
>>
>> Which means that my workflow would become:
>>
>> 1) Make changes
>> 2) Make a webrev without any options to show just the differences with
>> the tip
>> 3) Amend my changes to my local commit so that I have it done with
>> 4) Go to 1
>>
>> Does that seem correct to you?
>>
>> Note that when I do this, I only see the full change of a file in the
>> full change set (Side note here: now the page says change set and not
>> patch, which is maybe why Serguei was having issues?).
>>
>> Thanks!
>> Jc
>>
>>
>>
>> On Wed, Jun 28, 2017 at 1:12 AM, Robbin Ehn <robbin.ehn at oracle.com
>> <mailto:robbin.ehn at oracle.com>> wrote:
>>
>>     Hi,
>>
>>     On 06/28/2017 12:04 AM, JC Beyler wrote:
>>
>>         Dear Thomas et al,
>>
>>         Here is the newest webrev:
>>         http://cr.openjdk.java.net/~rasbold/8171119/webrev.07/ <
>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.07/>
>>
>>
>>
>>     You have some more bits to in there but generally this looks good and
>> really nice with more tests.
>>     I'll do and deep dive and re-test this when I get back from my long
>> vacation with whatever patch version you have then.
>>
>>     Also I think it's time you provide incremental (v06->07 changes) as
>> well as complete change-sets.
>>
>>     Thanks, Robbin
>>
>>
>>
>>
>>         Thomas, I "think" I have answered all your remarks. The summary
>> is:
>>
>>         - The statistic system is up and provides insight on what the
>> heap sampler is doing
>>              - I've noticed that, though the sampling rate is at the
>> right mean, we are missing some samples, I have not yet tracked out why
>> (details below)
>>
>>         - I've run a tiny benchmark that is the worse case: it is a very
>> tight loop and allocated a small array
>>              - In this case, I see no overhead when the system is off so
>> that is a good start :)
>>              - I see right now a high overhead in this case when sampling
>> is on. This is not a really too surprising but I'm going to see if this is
>> consistent with our
>>         internal implementation. The benchmark is really allocation
>> stressful so I'm not too surprised but I want to do the due diligence.
>>
>>            - The statistic system up is up and I have a new test
>>         http://cr.openjdk.java.net/~rasbold/8171119/webrev.07/test/
>> serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatTest.java.patch
>>         <http://cr.openjdk.java.net/~rasbold/8171119/webrev.07/test/
>> serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatTest.java.patch
>> >
>>               - I did a bit of a study about the random generator here,
>> more details are below but basically it seems to work well
>>
>>            - I added a capability but since this is the first time doing
>> this, I was not sure I did it right
>>              - I did add a test though for it and the test seems to do
>> what I expect (all methods are failing with the
>> JVMTI_ERROR_MUST_POSSESS_CAPABILITY error).
>>                  - http://cr.openjdk.java.net/~ra
>> sbold/8171119/webrev.07/test/serviceability/jvmti/HeapMonit
>> or/MyPackage/HeapMonitorNoCapabilityTest.java.patch
>>         <http://cr.openjdk.java.net/~rasbold/8171119/webrev.07/test/
>> serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorNoCapa
>> bilityTest.java.patch>
>>
>>            - I still need to figure out what to do about the multi-agent
>> vs single-agent issue
>>
>>            - As far as measurements, it seems I still need to look at:
>>              - Why we do the 20 random calls first, are they necessary?
>>              - Look at the mean of the sampling rate that the random
>> generator does and also what is actually sampled
>>              - What is the overhead in terms of memory/performance when
>> on?
>>
>>         I have inlined my answers, I think I got them all in the new
>> webrev, let me know your thoughts.
>>
>>         Thanks again!
>>         Jc
>>
>>
>>         On Fri, Jun 23, 2017 at 3:52 AM, Thomas Schatzl <
>> thomas.schatzl at oracle.com <mailto:thomas.schatzl at oracle.com> <mailto:
>> thomas.schatzl at oracle.com
>>
>>         <mailto:thomas.schatzl at oracle.com>>> wrote:
>>
>>              Hi,
>>
>>              On Wed, 2017-06-21 at 13:45 -0700, JC Beyler wrote:
>>              > Hi all,
>>              >
>>              > First off: Thanks again to Robbin and Thomas for their
>> reviews :)
>>              >
>>              > Next, I've uploaded a new webrev:
>>              > http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/ <
>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/>
>>         <http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/ <
>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.06/>>
>>
>>              >
>>              > Here is an update:
>>              >
>>              > - @Robbin, I forgot to say that yes I need to look at
>> implementing
>>              > this for the other architectures and testing it before it
>> is all
>>              > ready to go. Is it common to have it working on all
>> possible
>>              > combinations or is there a subset that I should be doing
>> first and we
>>              > can do the others later?
>>              > - I've tested slowdebug, built and ran the JTreg tests I
>> wrote with
>>              > slowdebug and fixed a few more issues
>>              > - I've refactored a bit of the code following Thomas'
>> comments
>>              >    - I think I've handled all the comments from Thomas (I
>> put
>>              > comments inline below for the specifics)
>>
>>              Thanks for handling all those.
>>
>>              > - Following Thomas' comments on statistics, I want to add
>> some
>>              > quality assurance tests and find that the easiest way
>> would be to
>>              > have a few counters of what is happening in the sampler
>> and expose
>>              > that to the user.
>>              >    - I'll be adding that in the next version if no one
>> sees any
>>              > objections to that.
>>              >    - This will allow me to add a sanity test in JTreg
>> about number of
>>              > samples and average of sampling rate
>>              >
>>              > @Thomas: I had a few questions that I inlined below but I
>> will
>>              > summarize the "bigger ones" here:
>>              >    - You mentioned constants are not using the right
>> conventions, I
>>              > looked around and didn't see any convention except normal
>> naming then
>>              > for static constants. Is that right?
>>
>>              I looked through https://wiki.openjdk.java.net/
>> display/HotSpot/StyleGui <https://wiki.openjdk.java.net
>> /display/HotSpot/StyleGui>
>>         <https://wiki.openjdk.java.net/display/HotSpot/StyleGui <
>> https://wiki.openjdk.java.net/display/HotSpot/StyleGui>>
>>              de and the rule is to "follow an existing pattern and must
>> have a
>>              distinct appearance from other names". Which does not help a
>> lot I
>>              guess :/ The GC team started using upper camel case, e.g.
>>              SomeOtherConstant, but very likely this is probably not
>> applied
>>              consistently throughout. So I am fine with not adding
>> another style
>>              (like kMaxStackDepth with the "k" in front with some unknown
>> meaning)
>>              is fine.
>>
>>              (Chances are you will find that style somewhere used anyway
>> too,
>>              apologies if so :/)
>>
>>
>>         Thanks for that link, now I know where to look. I used the upper
>> camel case in my code as well then :) I should have gotten them all.
>>
>>
>>               > PS: I've also inlined my answers to Thomas below:
>>               >
>>               > On Tue, Jun 13, 2017 at 8:03 AM, Thomas Schatzl
>> <thomas.schatzl at oracl
>>               > e.com <http://e.com> <http://e.com>> wrote:
>>               > > Hi all,
>>               > >
>>               > > On Mon, 2017-06-12 at 11:11 -0700, JC Beyler wrote:
>>               > > > Dear all,
>>               > > >
>>               > > > I've continued working on this and have done the
>> following
>>               > > webrev:
>>               > > > http://cr.openjdk.java.net/~ra
>> sbold/8171119/webrev.05/ <http://cr.openjdk.java.net/~r
>> asbold/8171119/webrev.05/>
>>         <http://cr.openjdk.java.net/~rasbold/8171119/webrev.05/ <
>> http://cr.openjdk.java.net/~rasbold/8171119/webrev.05/>>
>>
>>               > >
>>               > > [...]
>>               > > > Things I still need to do:
>>               > > >    - Have to fix that TLAB case for the FastTLABRefill
>>               > > >    - Have to start looking at the data to see that it
>> is
>>               > > consistent and does gather the right samples, right
>> frequency, etc.
>>               > > >    - Have to check the GC elements and what that
>> produces
>>               > > >    - Run a slowdebug run and ensure I fixed all those
>> issues you
>>               > > saw > Robbin
>>               > > >
>>               > > > Thanks for looking at the webrev and have a great
>> week!
>>               > >
>>               > >   scratching a bit on the surface of this change, so
>> apologies for
>>               > > rather shallow comments:
>>               > >
>>               > > - macroAssembler_x86.cpp:5604: while this is compiler
>> code, and I
>>               > > am not sure this is final, please avoid littering the
>> code with
>>               > > TODO remarks :) They tend to be candidates for later
>> wtf moments
>>               > > only.
>>               > >
>>               > > Just file a CR for that.
>>               > >
>>               > Newcomer question: what is a CR and not sure I have the
>> rights to do
>>               > that yet ? :)
>>
>>              Apologies. CR is a change request, this suggests to file a
>> bug in the
>>              bug tracker. And you are right, you can't just create a new
>> account in
>>              the OpenJDK JIRA yourselves. :(
>>
>>
>>         Ok good to know, I'll continue with my own todo list but I'll
>> work hard on not letting it slip in the webrevs anymore :)
>>
>>
>>              I was mostly referring to the "... but it is a TODO" part of
>> that
>>              comment in macroassembler_x86.cpp. Comments about the why of
>> the code
>>              are appreciated.
>>
>>              [Note that I now understand that this is to some degree
>> still work in
>>              progress. As long as the final changeset does no contain
>> TODO's I am
>>              fine (and it's not a hard objection, rather their use in
>> "final" code
>>              is typically limited in my experience)]
>>
>>              5603   // Currently, if this happens, just set back the
>> actual end to
>>              where it was.
>>              5604   // We miss a chance to sample here.
>>
>>              Would be okay, if explaining "this" and the "why" of missing
>> a chance
>>              to sample here would be best.
>>
>>              Like maybe:
>>
>>              // If we needed to refill TLABs, just set the actual end
>> point to
>>              // the end of the TLAB again. We do not sample here although
>> we could.
>>
>>         Done with your comment, it works well in my mind.
>>
>>              I am not sure whether "miss a chance to sample" meant "we
>> could, but
>>              consciously don't because it's not that useful" or "it would
>> be
>>              necessary but don't because it's too complicated to do.".
>>
>>              Looking at the original comment once more, I am also not
>> sure if that
>>              comment shouldn't referring to the "end" variable (not
>> actual_end)
>>              because that's the variable that is responsible for taking
>> the sampling
>>              path? (Going from the member description of
>> ThreadLocalAllocBuffer).
>>
>>
>>         I've moved this code and it no longer shows up here but the
>> rationale and answer was:
>>
>>         So.. Yes, end is the variable provoking the sampling. Actual end
>> is the actual end of the TLAB.
>>
>>         What was happening here is that the code is resetting _end to
>> point towards the end of the new TLAB. Because, we now have the end for
>> sampling and _actual_end for
>>         the actual end, we need to update the actual_end as well.
>>
>>         Normally, were we to do the real work here, we would calculate
>> the (end - start) offset, then do:
>>
>>         - Set the new end to : start + (old_end - old_start)
>>         - Set the actual end like we do here now where it because it is
>> the actual end.
>>
>>         Why is this not done here now anymore?
>>             - I was still debating which path to take:
>>                - Do it in the fast refill code, it has its perks:
>>                    - In a world where fast refills are happening all the
>> time or a lot, we can augment there the code to do the sampling
>>                - Remember what we had as an end before leaving the
>> slowpath and check on return
>>                    - This is what I'm doing now, it removes the need to
>> go fix up all fast refill paths but if you remain in fast refill paths, you
>> won't get sampling. I
>>         have to think of the consequences of that, maybe a future change
>> later on?
>>                       - I have the statistics now so I'm going to study
>> that
>>                          -> By the way, though my statistics are showing
>> I'm missing some samples, if I turn off FastTlabRefill, it is the same loss
>> so for now, it seems
>>         this does not occur in my simple test.
>>
>>
>>
>>              But maybe I am only confused and it's best to just leave the
>> comment
>>              away. :)
>>
>>              Thinking about it some more, doesn't this not-sampling in
>> this case
>>              mean that sampling does not work in any collector that does
>> inline TLAB
>>              allocation at the moment? (Or is inline TLAB alloc
>> automatically
>>              disabled with sampling somehow?)
>>
>>              That would indeed be a bigger TODO then :)
>>
>>
>>         Agreed, this remark made me think that perhaps as a first step
>> the new way of doing it is better but I did have to:
>>            - Remove the const of the ThreadLocalBuffer remaining and
>> hard_end methods
>>            - Move hard_end out of the header file to have a bit more
>> logic there
>>
>>         Please let me know what you think of that and if you prefer it
>> this way or changing the fast refills. (I prefer this way now because it is
>> more incremental).
>>
>>
>>              > > - calling HeapMonitoring::do_weak_oops() (which should
>> probably be
>>              > > called weak_oops_do() like other similar methods) only
>> if string
>>              > > deduplication is enabled (in g1CollectedHeap.cpp:4511)
>> seems wrong.
>>              >
>>              > The call should be at least around 6 lines up outside the
>> if.
>>              >
>>              > Preferentially in a method like
>> process_weak_jni_handles(), including
>>              > additional logging. (No new (G1) gc phase without minimal
>> logging
>>              > :)).
>>              > Done but really not sure because:
>>              >
>>              > I put for logging:
>>              >   log_develop_trace(gc, freelist)("G1ConcRegionFreeing
>> [other] : heap
>>              > monitoring");
>>
>>              I would think that "gc, ref" would be more appropriate log
>> tags for
>>              this similar to jni handles.
>>              (I am als not sure what weak reference handling has to do
>> with
>>              G1ConcRegionFreeing, so I am a bit puzzled)
>>
>>
>>         I was not sure what to put for the tags or really as the message.
>> I cleaned it up a bit now to:
>>              log_develop_trace(gc, ref)("HeapSampling [other] : heap
>> monitoring processing");
>>
>>
>>
>>              > Since weak_jni_handles didn't have logging for me to be
>> inspired
>>              > from, I did that but unconvinced this is what should be
>> done.
>>
>>              The JNI handle processing does have logging, but only in
>>              ReferenceProcessor::process_discovered_references(). In
>>              process_weak_jni_handles() only overall time is measured (in
>> a G1
>>              specific way, since only G1 supports disabling reference
>> procesing) :/
>>
>>              The code in ReferenceProcessor prints both time taken
>>              referenceProcessor.cpp:254, as well as the count, but
>> strangely only in
>>              debug VMs.
>>
>>              I have no idea why this logging is that unimportant to only
>> print that
>>              in a debug VM. However there are reviews out for changing
>> this area a
>>              bit, so it might be useful to wait for that (JDK-8173335).
>>
>>
>>         I cleaned it up a bit anyway and now it returns the count of
>> objects that are in the system.
>>
>>
>>              > > - the change doubles the size of
>>              > > CollectedHeap::allocate_from_tlab_slow() above the
>> "small and nice"
>>              > > threshold. Maybe it could be refactored a bit.
>>              > Done I think, it looks better to me :).
>>
>>              In ThreadLocalAllocBuffer::handle_sample() I think the
>>              set_back_actual_end()/pick_next_sample() calls could be
>> hoisted out of
>>              the "if" :)
>>
>>
>>         Done!
>>
>>
>>              > > - referenceProcessor.cpp:261: the change should add
>> logging about
>>              > > the number of references encountered, maybe after the
>> corresponding
>>              > > "JNI weak reference count" log message.
>>              > Just to double check, are you saying that you'd like to
>> have the heap
>>              > sampler to keep in store how many sampled objects were
>> encountered in
>>              > the HeapMonitoring::weak_oops_do?
>>              >    - Would a return of the method with the number of
>> handled
>>              > references and logging that work?
>>
>>              Yes, it's fine if HeapMonitoring::weak_oops_do() only
>> returned the
>>              number of processed weak oops.
>>
>>
>>         Done also (but I admit I have not tested the output yet) :)
>>
>>
>>              >    - Additionally, would you prefer it in a separate block
>> with its
>>              > GCTraceTime?
>>
>>              Yes. Both kinds of information is interesting: while the
>> time taken is
>>              typically more important, the next question would be why,
>> and the
>>              number of references typically goes a long way there.
>>
>>              See above though, it is probably best to wait a bit.
>>
>>
>>         Agreed that I "could" wait but, if it's ok, I'll just
>> refactor/remove this when we get closer to something final. Either,
>> JDK-8173335
>>         has gone in and I will notice it now or it will soon and I can
>> change it then.
>>
>>
>>              > > - threadLocalAllocBuffer.cpp:331: one more "TODO"
>>              > Removed it and added it to my personal todos to look at.
>>              >      > >
>>              > > - threadLocalAllocBuffer.hpp: ThreadLocalAllocBuffer
>> class
>>              > > documentation should be updated about the sampling
>> additions. I
>>              > > would have no clue what the difference between
>> "actual_end" and
>>              > > "end" would be from the given information.
>>              > If you are talking about the comments in this file, I made
>> them more
>>              > clear I hope in the new webrev. If it was somewhere else,
>> let me know
>>              > where to change.
>>
>>              Thanks, that's much better. Maybe a note in the comment of
>> the class
>>              that ThreadLocalBuffer provides some sampling facility by
>> modifying the
>>              end() of the TLAB to cause "frequent" calls into the runtime
>> call where
>>              actual sampling takes place.
>>
>>
>>         Done, I think it's better now. Added something about the
>> slow_path_end as well.
>>
>>
>>              > > - in heapMonitoring.hpp: there are some random comments
>> about some
>>              > > code that has been grabbed from
>> "util/math/fastmath.[h|cc]". I
>>              > > can't tell whether this is code that can be used but I
>> assume that
>>              > > Noam Shazeer is okay with that (i.e. that's all Google
>> code).
>>              > Jeremy and I double checked and we can release that as I
>> thought. I
>>              > removed the comment from that piece of code entirely.
>>
>>              Thanks.
>>
>>              > > - heapMonitoring.hpp/cpp static constant naming does not
>> correspond
>>              > > to Hotspot's. Additionally, in Hotspot static methods
>> are cased
>>              > > like other methods.
>>              > I think I fixed the methods to be cased the same way as
>> all other
>>              > methods. For static constants, I was not sure. I fixed a
>> few other
>>              > variables but I could not seem to really see a consistent
>> trend for
>>              > constants. I made them as variables but I'm not sure now.
>>
>>              Sorry again, style is a kind of mess. The goal of my
>> suggestions here
>>              is only to prevent yet another style creeping in.
>>
>>              > > - in heapMonitoring.cpp there are a few cryptic comments
>> at the top
>>              > > that seem to refer to internal stuff that should
>> probably be
>>              > > removed.
>>              > Sorry about that! My personal todos not cleared out.
>>
>>              I am happy about comments, but I simply did not understand
>> any of that
>>              and I do not know about other readers as well.
>>
>>              If you think you will remember removing/updating them until
>> the review
>>              proper (I misunderstood the review situation a little it
>> seems).
>>
>>              > > I did not think through the impact of the TLAB changes
>> on collector
>>              > > behavior yet (if there are). Also I did not check for
>> problems with
>>              > > concurrent mark and SATB/G1 (if there are).
>>              > I would love to know your thoughts on this, I think this
>> is fine. I
>>
>>              I think so too now. No objects are made live out of thin air
>> :)
>>
>>              > see issues with multiple threads right now hitting the
>> stack storage
>>              > instance. Previous webrevs had a mutex lock here but we
>> took it out
>>              > for simplificity (and only for now).
>>
>>              :) When looking at this after some thinking I now assume for
>> this
>>              review that this code is not MT safe at all. There seems to
>> be more
>>              synchronization missing than just the one for the
>> StackTraceStorage. So
>>              no comments about this here.
>>
>>
>>         I doubled checked a bit (quickly I admit) but it seems that
>> synchronization in StackTraceStorage is really all you need (all methods
>> lead to a StackTraceStorage one
>>         and can be multithreaded outside of that).
>>         There is a question about the initialization where the method
>> HeapMonitoring::initialize_profiling is not thread safe.
>>         It would work (famous last words) and not crash if there was a
>> race but we could add a synchronization point there as well (and therefore
>> on the stop as well).
>>
>>         But anyway I will really check and do this once we add back
>> synchronization.
>>
>>
>>              Also, this would require some kind of specification of what
>> is allowed
>>              to be called when and where.
>>
>>
>>         Would we specify this with the methods in the jvmti.xml file? We
>> could start by specifying in each that they are not thread safe but I saw
>> no mention of that for
>>         other methods.
>>
>>
>>              One potentially relevant observation about locking here:
>> depending on
>>              sampling frequency, StackTraceStore::add_trace() may be
>> rather
>>              frequently called. I assume that you are going to do
>> measurements :)
>>
>>
>>         Though we don't have the TLAB implementation in our code, the
>> compiler generated sampler uses 2% of overhead with a 512k sampling rate. I
>> can do real measurements
>>         when the code settles and we can see how costly this is as a TLAB
>> implementation.
>>         However, my theory is that if the rate is 512k, the
>> memory/performance overhead should be minimal since it is what we saw with
>> our code/workloads (though not called
>>         the same way, we call it essentially at the same rate).
>>         If you have a benchmark you'd like me to test, let me know!
>>
>>         Right now, with my really small test, this does use a bit of
>> overhead even for a 512k sample size. I don't know yet why, I'm going to
>> see what is going on.
>>
>>         Finally, I think it is not reasonable to suppose the overhead to
>> be negligible if the sampling rate used is too low. The user should know
>> that the lower the rate,
>>         the higher the overhead (documentation TODO?).
>>
>>
>>              I am not sure what the expected usage of the API is, but
>>              StackTraceStore::add_trace() seems to be able to grow
>> without bounds.
>>              Only a GC truncates them to the live ones. That in itself
>> seems to be
>>              problematic (GCs can be *wide* apart), and of course some of
>> the API
>>              methods add to that because they duplicate that unbounded
>> array. Do you
>>              have any concerns/measurements about this?
>>
>>
>>         So, the theory is that yes add_trace can be able to grow without
>> bounds but it grows at a sample per 512k of allocated space. The stacks it
>> gathers are currently
>>         maxed at 64 (I'd like to expand that to an option to the user
>> though at some point). So I have no concerns because:
>>
>>         - If really this is taking a lot of space, that means the job is
>> keeping a lot of objects in memory as well, therefore the entire heap is
>> getting huge
>>         - If this is the case, you will be triggering a GC at some point
>> anyway.
>>
>>         (I'm putting under the rug the issue of "What if we set the rate
>> to 1 for example" because as you lower the sampling rate, we cannot
>> guarantee low overhead; the
>>         idea behind this feature is to have a means of having meaningful
>> allocated samples at a low overhead)
>>
>>         I have no measurements really right now but since I now have some
>> statistics I can poll, I will look a bit more at this question.
>>
>>         I have the same last sentence than above: the user should expect
>> this to happen if the sampling rate is too small. That probably can be
>> reflected in the
>>         StartHeapSampling as a note : careful this might impact your
>> performance.
>>
>>
>>              Also, these stack traces might hold on to huge arrays. Any
>>              consideration of that? Particularly it might be the cause
>> for OOMEs in
>>              tight memory situations.
>>
>>
>>         There is a stack size maximum that is set to 64 so it should not
>> hold huge arrays. I don't think this is an issue but I can double check
>> with a test or two.
>>
>>
>>              - please consider adding a safepoint check in
>>              HeapMonitoring::weak_oops_do to prevent accidental misuse.
>>
>>              - in struct StackTraceStorage, the public fields may also
>> need
>>              underscores. At least some files in the runtime directory
>> have structs
>>              with underscored public members (and some don't). The
>> runtime team
>>              should probably comment on that.
>>
>>
>>         Agreed I did not know. I looked around and a lot of structs did
>> not have them it seemed so I left it as is. I will happily change it if
>> someone prefers (I was not
>>         sure if you really preferred or not, your sentence seemed to be
>> more a note of "this might need to change but I don't know if the runtime
>> team enforces that", let
>>         me know if I read that wrongly).
>>
>>
>>              - In StackTraceStorage::weak_oops_do(), when examining the
>>              StackTraceData, maybe it is useful to consider having a
>> non-NULL
>>              reference outside of the heap's reserved space an error.
>> There should
>>              be no oop outside of the heap's reserved space ever.
>>
>>              Unless you allow storing random values in
>> StackTraceData::obj, which I
>>              would not encourage.
>>
>>
>>         I suppose you are talking about this part:
>>         if ((value != NULL && Universe::heap()->is_in_reserved(value)) &&
>>                     (is_alive == NULL || is_alive->do_object_b(value))) {
>>
>>         What you are saying is that I could have something like:
>>         if (value != my_non_null_reference &&
>>                     (is_alive == NULL || is_alive->do_object_b(value))) {
>>
>>         Is that what you meant? Is there really a reason to do so? When I
>> look at the code, is_in_reserved seems like a O(1) method call. I'm not
>> even sure we can have a
>>         NULL value to be honest. I might have to study that to see if
>> this was not a paranoid test to begin with.
>>
>>         The is_alive code has now morphed due to the comment below.
>>
>>
>>
>>              - HeapMonitoring::weak_oops_do() does not seem to use the
>>              passed AbstractRefProcTaskExecutor.
>>
>>
>>         It did use it:
>>            size_t HeapMonitoring::weak_oops_do(
>>               AbstractRefProcTaskExecutor *task_executor,
>>               BoolObjectClosure* is_alive,
>>               OopClosure *f,
>>               VoidClosure *complete_gc) {
>>             assert(SafepointSynchronize::is_at_safepoint(), "must be at
>> safepoint");
>>
>>             if (task_executor != NULL) {
>>               task_executor->set_single_threaded_mode();
>>             }
>>             return StackTraceStorage::storage()->weak_oops_do(is_alive,
>> f, complete_gc);
>>         }
>>
>>         But due to the comment below, I refactored this, so this is no
>> longer here. Now I have an always true closure that is passed.
>>
>>
>>              - I do not understand allowing to call this method with a
>> NULL
>>              complete_gc closure. This would mean that objects referenced
>> from the
>>              object that is referenced by the StackTraceData are not
>> pulled, meaning
>>              they would get stale.
>>
>>              - same with is_alive parameter value of NULL
>>
>>
>>         So these questions made me look a bit closer at this code. This
>> code I think was written this way to have a very small impact on the file
>> but you are right, there
>>         is no reason for this here. I've simplified the code by making in
>> referenceProcessor.cpp a process_HeapSampling method that handles
>> everything there.
>>
>>         The code allowed NULLs because it depended on where you were
>> coming from and how the code was being called.
>>
>>         - I added a static always_true variable and pass that now to be
>> more consistent with the rest of the code.
>>         - I moved the complete_gc into process_phaseHeapSampling now (new
>> method) and handle the task_executor and the complete_gc there
>>              - Newbie question: in our code we did a
>> set_single_threaded_mode but I see that process_phaseJNI does it right
>> before its call, do I need to do it for the
>>         process_phaseHeapSample?
>>         That API is much cleaner (in my mind) and is consistent with what
>> is done around it (again in my mind).
>>
>>
>>              - heapMonitoring.cpp:590: I do not completely understand the
>> purpose of
>>              this code: in the end this results in a fixed value directly
>> dependent
>>              on the Thread address anyway? In the end this results in a
>> fixed value
>>              directly dependent on the Thread address anyway?
>>              IOW, what is special about exactly 20 rounds?
>>
>>
>>         So we really want a fast random number generator that has a
>> specific mean (512k is the default we use). The code uses the thread
>> address as the start number of the
>>         sequence (why not, it is random enough is rationale). Then
>> instead of just starting there, we prime the sequence and really only start
>> at the 21st number, it is
>>         arbitrary and I have not done a study to see if we could do more
>> or less of that.
>>
>>         As I have the statistics of the system up and running, I'll run
>> some experiments to see if this is needed, is 20 good, or not.
>>
>>
>>              - also I would consider stripping a few bits of the threads'
>> address as
>>              initialization value for your rng. The last three bits (and
>> probably
>>              more, check whether the Thread object is allocated on special
>>              boundaries) are always zero for them.
>>              Not sure if the given "random" value is random enough
>> before/after,
>>              this method, so just skip that comment if you think this is
>> not
>>              required.
>>
>>
>>         I don't know is the honest answer. I think what is important is
>> that we tend towards a mean and it is random "enough" to not fall in
>> pitfalls of only sampling a
>>         subset of objects due to their allocation order. I added that as
>> test to do to see if it changes the mean in any way for the 512k default
>> value and/or if the first
>>         1000 elements look better.
>>
>>
>>              Some more random nits I did not find a place to put anywhere:
>>
>>              - ThreadLocalAllocBuffer::_extra_space does not seem to be
>> used
>>              anywhere?
>>
>>
>>         Good catch :).
>>
>>
>>              - Maybe indent the declaration of
>> ThreadLocalAllocBuffer::_bytes_until_sample to align below the other
>> members of that group.
>>
>>
>>         Done moved it up a bit to have non static members together and
>> static separate.
>>
>>              Thanks,
>>                 Thomas
>>
>>
>>         Thanks for your review!
>>         Jc
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20170629/6b7bc0f6/attachment-0001.html>