[8u] RFR backport of JDK-8144732: VM_HeapDumper hits assert with bad dump_len

[Andrew John Hughes]

On 25/09/2019 07:25, Denghui Dong wrote:
> Hi all, 
>   I'd like to request a backport of JDK-8144732.
> 
>   In our production environment, many application use large heap, and there are some
> big arrays in the heap. When developers use jmap to dump heap, and use Eclipse MAT(mostly) 
> or jhat to analyze the file, often got error. For example:
> 
> public class BigArray {
>   public static void main(String[] args) throws Exception {
>     long[] b = new long[1024 * 1024 * 1024 / 2];
>     Object o = new Object();
>     synchronized(o) {
>       o.wait(60000);
>     }
>   }
> }
> 
>   If you run the above code, and use jmap to generate a dump file, then use jhat to parse it,
> you will got a warning message:
> 
> "WARNING: Error reading heap dump or heap dump segment:  Byte count is -4294967296 instead of 0"
> 
>   Eclipse MAT also can't parse the dump file correctly.
> 
>   The root cause is the length of the segment exceeds the limit.
> 
>   I found that JDK-8144732 can resolve this problem, because it can truncate the array whose
> size is too large and ensure a segment length within limit.
> 
> The patch (from JDK9) doesn't apply cleanly.
> 
> Original bug: https://bugs.openjdk.java.net/browse/JDK-8150432
> 
> Original patch: http://hg.openjdk.java.net/jdk9/dev/jdk/rev/91a26000bfb5
> 
> My webrev: http://cr.openjdk.java.net/~luchsh/8144732_8u/
> 
> Testing:
>   jdk/test/demo/jvmti/hprof/HeapDumpTest.java passed.
>   jdk/test/sun/tools/jhat/HatHeapDump1Test.java passed.
> 
>   What's your comments ?
> 
> Thanks
> Denghui Dong
> 

I'm concerned that this alters the HPROF format used for dumps under 2GB
[0] Is there no other way of fixing the bug without altering this, which
may have an impact on tools expecting to parse HPROF 1.0.1 format data.

I guess HPROF 1.0.2 support was already required for larger dumps, so
I'm not sure how much of an issue this is, but it's definitely a
compatibility change.

[0] https://bugs.openjdk.java.net/browse/JDK-8174881
-- 
Andrew :)

Senior Free Java Software Engineer
Red Hat, Inc. (http://www.redhat.com)

PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net)
Fingerprint = 5132 579D D154 0ED2 3E04  C5A0 CFDA 0F9B 3596 4222
https://keybase.io/gnu_andrew