RFR: 8252842: Extend jmap to support parallel heap dump [v10]

Wed Feb 24 00:10:41 UTC 2021

On Wed, 24 Feb 2021 00:02:33 GMT, Lin Zang <lzang at openjdk.org> wrote:

>>> I have reconsidered the solution of “dumpheapext”, IMHO maybe the name is too specific, for example, if in future there is more arguments for `-histo`, we have to made another command called "inspectheapext".
>>> 
>>> How about create a new command named (e.g.) "jmapext", and make the "dumpheapext" work as the subcommand and be passed to JVM as the 1st argument of "jmapext". Then in future we don't bothering adding new command, just subcommand (or argument) like "inspectheapext". And hence there may be more easier to extend other commands?
>> 
>> Well, if you want to go that route, maybe you just add a new command called "attachcmd", and it can support any command, new and old. But then that kind of points out the real issue with existing commands in that they are rigid with how many arguments they support, and I think they all suffer from that.
>> 
>> There have been other suggestions on how to deal with this, including having the 1st argument include all the arguments. I think the issue there is that it fails with older JDKs. However, the solution for older JDKs might just be the same as is suggested with "dumpheapext", which is just retry using the old argument passing method if the first one fails (and not including any of the new arguments that the old JDK won't understand anyway). So in general, I think any solution we come up with will fail with older JDKs, and need to fallback to retrying with a command we know will work.
>> 
>> How is `jmap dump:gz=...` working with older JDKs? I tried it against JDK8 and didn't get any errors, but it wasn't gzipped either.
>
> Hi @plummercj ,
> 
>> maybe you just add a new command called "attachcmd", and it can support any command, new and old
> 
> I think make it work only for newly added arguments for different command could reduce the implementation complexity. The old command doesn't need to handle the exception. 
> And I also found in this way the "attachcmd" will be quite similar with `jcmd` command, it accept different command and pass all argument with one string. Then there maybe another solution -- for "parallel" argument of `jmap -dump`, just change it to the `jcmd` command and passing all arguments as a string.  Then we don't bother with adding the `dumpheapext` command.
>  
> 
>> So in general, I think any solution we come up with will fail with older JDKs, and need to fallback to retrying with a command we know will work
> 
> Yes, no matter what name it is, the new command must have the ablility to eat the unsupported command exception and then print error info. 
> 
> 
> 
>> How is `jmap dump:gz=...` working with older JDKs? I tried it against JDK8 and didn't get any errors, but it wasn't gzipped either.
> 
> The older version for `jmap dump` only passing arguments that is recognized (`file` and `live or all`), and siliently drop the unrecognized arguments without any output. 
> I added the logic of print error message in JDK16, with JBS https://bugs.openjdk.java.net/browse/JDK-8251347

> Hi @linzang
> 
> > This data are really interest to me, it seems using gzipped dump is faster than unzipped dump, is the because of disk writing or something else? I would investigate more about it~
> 
> I would guess that is the case. In the compressed case we write about 800 MB per second, that is as fast as my SSD goes.
> 
> Here are some actual results I've gotten (uncompressed):
> 
> ```
> jmap.exe -dump:file=ser.hprof,all 30600 
> Dumping heap to ser.hprof ...
> Heap dump file created [32009882362 bytes in 59.303 secs]
> 
> jmap.exe -dump:file=par1.hprof,parallel=1,all 30600
> Dumping heap to par1.hprof ...
> Heap dump file created [32009885809 bytes in 72.719 secs]
> 
> jmap.exe -dump:file=par2.hprof,parallel=2,all 30600
> Dumping heap to par2.hprof ...
> Heap dump file created [32009881876 bytes in 57.546 secs]
> 
> jmap.exe -dump:file=par4.hprof,parallel=4,all 30600
> Dumping heap to par4.hprof ...
> Heap dump file created [32009882956 bytes in 44.301 secs]
> 
> jmap.exe -dump:file=par5.hprof,parallel=5,all 30600
> Dumping heap to par5.hprof ...
> Heap dump file created [32009882164 bytes in 40.282 secs]
> 
> jmap.exe -dump:file=par6.hprof,parallel=6,all 30600
> Dumping heap to par6.hprof ...
> Heap dump file created [32009881156 bytes in 45.988 secs]
> ```
> 
> And here for the compressed case:
> 
> ```
> jmap.exe -dump:file=par1.hprof.gz,parallel=1,all,gz=1 52372
> Dumping heap to par1.hprof.gz ...
> Heap dump file created [8076994216 bytes in 54.057 secs]
> 
> jmap.exe -dump:file=par2.hprof.gz,parallel=2,all,gz=1 52372
> Dumping heap to par2.hprof.gz ...
> Heap dump file created [8075859421 bytes in 43.442 secs]
> 
> jmap.exe -dump:file=par4.hprof.gz,parallel=4,all,gz=1 52372
> Dumping heap to par4.hprof.gz ...
> Heap dump file created [8075886152 bytes in 28.710 secs]
> 
> jmap.exe -dump:file=par6.hprof.gz,parallel=6,all,gz=1 52372
> Dumping heap to par6.hprof.gz ...
> Heap dump file created [8075758374 bytes in 25.730 secs]
> 
> jmap.exe -dump:file=par8.hprof.gz,parallel=8,all,gz=1 52372
> Dumping heap to par8.hprof.gz ...
> Heap dump file created [8075652558 bytes in 26.039 secs]
> 
> jmap.exe -dump:file=par16.hprof.gz,parallel=16,all,gz=1 52372
> Dumping heap to par16.hprof.gz ...
> Heap dump file created [8075644423 bytes in 31.977 secs]
> 
> jmap.exe -dump:file=par24.hprof.gz,parallel=24,all,gz=1 52372
> Dumping heap to par24.hprof.gz ...
> Heap dump file created [8075579546 bytes in 41.094 secs]
> ```
> 
> Best regards,
> Ralf

Hi Ralf, 

The data are really helpful. Thanks for the thorough listing and sharing!

BRs,
Lin

-------------

PR: https://git.openjdk.java.net/jdk/pull/2261