RFR(L) 8237354: Add option to jcmd to write a gzipped heap dump

[David Holmes]

On 11/02/2020 10:55 pm, Yasumasa Suenaga wrote:
> Hi Ralf,
> 
> On 2020/02/11 0:33, Schmelter, Ralf wrote:
>> Hi Yasumasa,
>>
>>>    You can use `DCmdArgument<jlong>` for -gz option.
>>
>> That is what I originally tried. But then you always have to supply a 
>> compression level (just specifying -gz doesn't work). Since I would 
>> expect most users never caring about the compression level, I switched 
>> to a string option, which can handle this pattern.
> 
> I think you can modify DCmdArgument<jlong>::parse_value() to allow the 
> operation without argument.
> Or you can add new impl function for integer types which can handle 
> default value.
> 
> 
>>> _nr_of_threads, _id_to_write, _current in CompressionBackend should 
>>> be added `volatile` at least.
>>
>> I don't think that is needed. Apart from the initialization, they are 
>> only changed under lock protection.
> 
> I concerned with compiler optimization.
> They are class members and they are used in `while(true)` loop.
> Of course the problem would not appear in all C++ compiler, but I guess 
> it is more safely if `volatile` is added.

As long as the variable is only accessed under the lock then volatile is 
not needed. If a compiler hoisted accesses outside of locked regions 
then all MT code could be broken.

David
-----


> 
>>> BTW how much processing time is different between single threaded and 
>>> multi threaded?
>>
>> I've benchmarked an example, which creates a ~31 GB uncompressed hprof 
>> file, with a VM which doesn't use any background threads. Here are the 
>> size of the create files, the compression level and the time spend:
>>
>> Uncompressed, 31.6 G, 71 sec
>> gzipped level 1, 7.57 G, 463 sec (x6.5)
>> gzipped level 3, 7.10 G, 609 sec (x8.6)
>> gzipped level 6, 6.49 G, 1415 sec (x19.9)
>>
>> So even the fastest gzip compression makes writing the dump at least 5 
>> times as slow.
>>
>>> Also I want to know what number is set to ParallelGCThreads.
>>> ParallelGCThreads seems to affect to thread num for GZip compression.
>>
>> Originally, I've tried to use the WorkGang (CollectedHeap:: 
>> get_safepoint_workers()) of the GC to do the work. But this wouldn't 
>> work because Shenandoah could not iterate the heap from a worker 
>> thread. So I've opted to start the needed threads itself for the time 
>> of the heap dump. I've used ParallelGCThreads as the maximum number of 
>> threads, since this is what would be used for a GC too. So it should 
>> not clog up the machine more than a GC. Maybe it would be even better 
>> to additionally limit the threads by the compression level.
> 
> Thanks!
> 
> Yasumasa (ysuenaga)
> 
> 
>> Best regards,
>> Ralf Schmelter
>>
>> -----Original Message-----
>> From: Yasumasa Suenaga <suenaga at oss.nttdata.com>
>> Sent: Samstag, 8. Februar 2020 14:46
>> To: Schmelter, Ralf <ralf.schmelter at sap.com>; OpenJDK Serviceability 
>> <serviceability-dev at openjdk.java.net>
>> Cc: yasuenag at gmail.com
>> Subject: Re: RFR(L) 8237354: Add option to jcmd to write a gzipped 
>> heap dump
>>
>> Hi Ralf,
>>
>>
>> - diagnosticCommand.cpp
>>    You can use `DCmdArgument<jlong>` for -gz option.
>>    If you want to use lesser type (e.g. int, unsigned char), I guess 
>> you need to modify GenDCmdArgument class.
>>
>> - heapDumper.cpp
>>    _nr_of_threads, _id_to_write, _current in CompressionBackend should 
>> be added `volatile` at least.
>>    (Other values need to be checked)
>>
>>
>> BTW how much processing time is different between single threaded and 
>> multi threaded?
>> Also I want to know what number is set to ParallelGCThreads.
>> ParallelGCThreads seems to affect to thread num for GZip compression.
>>
>>
>> Thanks,
>>
>> Yasumasa
>>