RFR(L) 8237354: Add option to jcmd to write a gzipped heap dump

[David Holmes]

As this is a decades old FAQ ...

On 12/02/2020 8:50 am, David Holmes wrote:
> On 11/02/2020 10:55 pm, Yasumasa Suenaga wrote:
>> Hi Ralf,
>>
>> On 2020/02/11 0:33, Schmelter, Ralf wrote:
>>> Hi Yasumasa,
>>>
>>>>    You can use `DCmdArgument<jlong>` for -gz option.
>>>
>>> That is what I originally tried. But then you always have to supply a 
>>> compression level (just specifying -gz doesn't work). Since I would 
>>> expect most users never caring about the compression level, I 
>>> switched to a string option, which can handle this pattern.
>>
>> I think you can modify DCmdArgument<jlong>::parse_value() to allow the 
>> operation without argument.
>> Or you can add new impl function for integer types which can handle 
>> default value.
>>
>>
>>>> _nr_of_threads, _id_to_write, _current in CompressionBackend should 
>>>> be added `volatile` at least.
>>>
>>> I don't think that is needed. Apart from the initialization, they are 
>>> only changed under lock protection.
>>
>> I concerned with compiler optimization.
>> They are class members and they are used in `while(true)` loop.
>> Of course the problem would not appear in all C++ compiler, but I 
>> guess it is more safely if `volatile` is added.
> 
> As long as the variable is only accessed under the lock then volatile is 
> not needed. If a compiler hoisted accesses outside of locked regions 
> then all MT code could be broken.

https://danluu.com/threads-faq/#Q56

And the same applies for non-POSIX platform compilers.

Cheers,
David
-----

> David
> -----
> 
> 
>>
>>>> BTW how much processing time is different between single threaded 
>>>> and multi threaded?
>>>
>>> I've benchmarked an example, which creates a ~31 GB uncompressed 
>>> hprof file, with a VM which doesn't use any background threads. Here 
>>> are the size of the create files, the compression level and the time 
>>> spend:
>>>
>>> Uncompressed, 31.6 G, 71 sec
>>> gzipped level 1, 7.57 G, 463 sec (x6.5)
>>> gzipped level 3, 7.10 G, 609 sec (x8.6)
>>> gzipped level 6, 6.49 G, 1415 sec (x19.9)
>>>
>>> So even the fastest gzip compression makes writing the dump at least 
>>> 5 times as slow.
>>>
>>>> Also I want to know what number is set to ParallelGCThreads.
>>>> ParallelGCThreads seems to affect to thread num for GZip compression.
>>>
>>> Originally, I've tried to use the WorkGang (CollectedHeap:: 
>>> get_safepoint_workers()) of the GC to do the work. But this wouldn't 
>>> work because Shenandoah could not iterate the heap from a worker 
>>> thread. So I've opted to start the needed threads itself for the time 
>>> of the heap dump. I've used ParallelGCThreads as the maximum number 
>>> of threads, since this is what would be used for a GC too. So it 
>>> should not clog up the machine more than a GC. Maybe it would be even 
>>> better to additionally limit the threads by the compression level.
>>
>> Thanks!
>>
>> Yasumasa (ysuenaga)
>>
>>
>>> Best regards,
>>> Ralf Schmelter
>>>
>>> -----Original Message-----
>>> From: Yasumasa Suenaga <suenaga at oss.nttdata.com>
>>> Sent: Samstag, 8. Februar 2020 14:46
>>> To: Schmelter, Ralf <ralf.schmelter at sap.com>; OpenJDK Serviceability 
>>> <serviceability-dev at openjdk.java.net>
>>> Cc: yasuenag at gmail.com
>>> Subject: Re: RFR(L) 8237354: Add option to jcmd to write a gzipped 
>>> heap dump
>>>
>>> Hi Ralf,
>>>
>>>
>>> - diagnosticCommand.cpp
>>>    You can use `DCmdArgument<jlong>` for -gz option.
>>>    If you want to use lesser type (e.g. int, unsigned char), I guess 
>>> you need to modify GenDCmdArgument class.
>>>
>>> - heapDumper.cpp
>>>    _nr_of_threads, _id_to_write, _current in CompressionBackend 
>>> should be added `volatile` at least.
>>>    (Other values need to be checked)
>>>
>>>
>>> BTW how much processing time is different between single threaded and 
>>> multi threaded?
>>> Also I want to know what number is set to ParallelGCThreads.
>>> ParallelGCThreads seems to affect to thread num for GZip compression.
>>>
>>>
>>> Thanks,
>>>
>>> Yasumasa
>>>