JDK-8005604, instance size/metaspace data in HPROF

[Vladimir Sitnikov]

Seems like HPROF does not include information on non-heap memory.
It would be nice to have accurate class size (e.g. with @Contended), space
wastage, permgen/metaspace (e.g. constant pool size or whatever is in
C-heap), code cache blobs in the heap dump.

Alexey Shipilёv raised similar question regarding instance size CR 8005604:
HPROF should report the actual instance size [1], [2], however it seems no
progress is made.

Creating brand-new format does not look like an easy solution, and it is
not clear how new features will be added later. It is not clear why new
format would be better in terms of efforts to update tooling.

What do you think if we keep HPROF format completely intact and dump
additional information via synthetic classes and objects?

For instance, to dump precise instance size of java class, we include
"vm.native.synthetic.ClassLayout { int instanceSize; int spaceLoss; }"
class in the generated HPROF file. For each java class we dump "instance"
of this ClassInfo.
This will reveal true instance size, space loss (e.g. alignment,
@contended, etc) to the heap analyzer.
The resulting HPROF will be fully compatible with existing analyzers, so no
significant harm is made.

To associate java.lang.Class and its vm.native metadata, a new synthetic
static reference from java.lang.Class to vm.native.synthetic.ClassInfo is
added.
Another approach is to reference from vm.native to java.lang.Class, however
it will be harder to analyze (more clicks, more complex queries)

Pros:
1) We can dump real instance size or any new information while keeping
HPROF format intact
2) Current tools will parse and display the dump just fine. With moderate
effort tools can be improved to use this "metadata" for calculations (and
proper accounting of "used heap size")
3) Query language of memory analyzers (e.g. map-reduce-javascript in
VirtualVM, OQL/SQL in Eclipse MAT, etc) can leverage this new data. The
approach of synthetic classes does not require to update analyzer for that.
4) More details (e.g. code cache blobs or whatever) can be added in the
similar way to the dump while still keeping forward and backward
compatibility in terms of file format

Cons:
1) I am not sure if iteration over permgen/metaspace/codecache is safe in
terms of crashes (or whatever). It is not good a good idea to crash when
writing HPROF dump (especially during OutOfMemoryError), however I believe
at least basic information (e.g. instance size) should be available in a
safe way.
2) Existing analyzers will charge "heap size" for synthetic
classes/objects. I do not think it is a big deal, however, if we dump
permgen/metaspace/codecache, then the amount of synthetic objects might be
significant enough to confuse non-expecting engineers


1: https://bugs.openjdk.java.net/browse/JDK-8005604
2:
http://mail.openjdk.java.net/pipermail/serviceability-dev/2012-December/007852.html

-- 
Regards,
Vladimir Sitnikov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20140218/9085839c/attachment-0001.html