RFR(S): 8200720: Print accumulated number of allocated bytes in thread dump

[Haug, Gunter]

Hi Jc,

Thank you for your reply and sorry for being late with mine!
You’re completely right with everything you write. It would be best to either piggy back our counters in your ThreadHeapSampler side structure (probably rename it and incorporate the improvements you suggest) or even better have the sampling system provide this information as you describe. 

OTOH as mentioned, we output more data (cpu time, number of defined classes, number of files opened, number of socks opened, number of bytes read/written via file and socket io and pthread-id), so it would also be an option to implement a separate side structure to hold data required.

Thank you very much,
Gunter




From: JC Beyler <jcbeyler at google.com>
Date: Thursday, 5. April 2018 at 18:02
To: "Haug, Gunter" <gunter.haug at sap.com>
Cc: hotspot-dev Source Developers <hotspot-dev at openjdk.java.net>
Subject: Re: RFR(S): 8200720: Print accumulated number of allocated bytes in thread dump

Hi Gunter, 

I really am new to the Java scene so I really am not speaking about the implementation and its potential integration into the code. Or even would the feature be accepted or not, however, from what I've seen in the webrev you provided:
  - In theory, adding to Thread is not really accepted and a side structure is better, perhaps we could piggy back on the same structure, since I'm trying to add one called ThreadHeapSampler, we could rename it to ThreadHeapInformation and both use that (see http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.10/src/hotspot/share/runtime/threadHeapSampler.cpp.html) in order to not have different structures provide similar information topics.

  -  bytes_in_tlab should be words_in_tlab if I'm not mistaken

  - seems like you are missing allocations that do not fit in the TLAB due to too big, no?

On another note, depending on what your users want, this might not be relevant:

We actually implemented a similar approach but it was on top of the sampling technique I'm pushing currently. It provides the same information you do but also the bytes allocated still live for a given thread due to how our internal system is implemented (because our users preferred to know how much a thread was holding instead of how much a thread allocated). With the work I'm doing here (via JDK-8171119), it might be reasonable and do-able to have the sampling system provide this information on the native agent side (and thus you can do your implementation and provide the data to your users without this change). Would this perhaps be of interest to you? We could build an example test and you can test it on your side and I can test it on mine and we can see how far off either technique is in regards to the information about per thread allocation?

I have not done the overhead calculation of this in a long time, we could wrap that together and see if that's acceptable. I'm not sure you get a 0% as you do but it would be small.

Thanks,
Jc

On Thu, Apr 5, 2018 at 3:28 AM Haug, Gunter <mailto:gunter.haug at sap.com> wrote:
Hi,

can I please have a review and a sponsor for the following enhancement:

http://cr.openjdk.java.net/~ghaug/webrevs/8200720
https://bugs.openjdk.java.net/browse/JDK-8200720

We at SAP have extended the thread dumps (obtained by jstack or jcmd) by several fields providing thread specific information. These extensions are quite popular with our support team. With some knowledge of the architecture of the application, they often allow for quick and simple diagnosis of a running system. Therefore we would like to contribute these enhancements.

I took the quite simple accumulated number of bytes allocated by a thread as an example. This only works with TLAB active. Provided it is known how the application should behave, a misbehaving thread can identified easily.

There is no measurable overhead for this enhancement. However, it may be a problem that the format of the output is changed. Tools parsing the output may have to be changed.

Here is an example of the output generated:

---------------------------------------------------------
"main" #1 prio=5 os_prio=31 allocated=243048448B tid=0x00007fca98000800 nid=0x1d03 waiting on condition  [0x0000000109baa000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
        at java.lang.Thread.sleep(java.base/Native Method)
               ...
---------------------------------------------------------

As mentioned above, we have a whole bunch of other enhancements to the thread dump similar to this one and would be willing to contribute them if there is any interest.

Thanks and best regards,
Gunter