RFC: regarding metaspace(metadata?) dump

[Yi Yang]

Hi,
Internally, we often receive feedback from users and ask for help on metaspace-related issues, for example
1. Users are eager to know which GroovyClassLoader loads which classes, why they are not unloaded,
and why they are leading to Metaspace OOME.
2. They want to know the class structure of dynamically generated classes in some scenarios such as 
deserialization
3. Finding memory leaking about duplicated classes
...
Internally we implemented a metaspace dump that generates human-readable text, it looks something like this:
[Basic Information]
Dump Reason : JCMD
MaxMetaspaceSize : 18446744073709547520 B
CompressedClassSpaceSize : 1073741824 B
Class Space Used : 309992 B
Class Space Capacity : 395264 B
...
[Class Loader Data]
ClassLoaderData : loader = 0x000000008024f928, loader_klass = 0x0000000800010098, loader_klass_name = 
sun/misc/Launcher$AppClassLoader, label = N/A
 Class Used Chunks :
 * Chunk : [0x0000000800060000, 0x0000000800060230, 0x0000000800060800)
 NonClass Used Chunks :
 * Chunk : [0x00007fd8379c1000, 0x00007fd8379c1350, 0x00007fd8379c2000)
 Klasses :
 Klass : 0x0000000800060028, name = Test, size = 520 B
 ConstantPool : 0x00007fd8379c1050, size = 296 B
...
It has been working effectively for several years and has helped many users solve metaspace-related problems.
But a more user-friendly way is that JDK can inherently support this capability. We hope that format of the metaspace
dump file can take both flexibility and compatibility into account, and the content of dump file should be detailed
enough to meet the needs of both application developers and lower-level developers.
Based on above considerations, I think using JSON as its file format is an appropriate solution(But XML or binary 
format are still not excluded as candidates). Specifically, in earlier thoughts, I thought the format of the metaspace
file could be as follows(pretty printed)
https://gist.github.com/y1yang0/ab3034b6381b8a9d215602c89af4e9c3 <https://gist.github.com/y1yang0/ab3034b6381b8a9d215602c89af4e9c3 >
Using the JSON format, we can flexibly add new fields without breaking compatibility. It is debatable as to which data
to write. We can reach a consensus that third-party parsers(Metaspace Analyzer Tool) can at least reconstruct Java
source code from the dump file. Based on this, we can write more useful information for low-level troubleshooting
or debugging. (e.g. the init_state of InstanceKlass).
 In addition, we can even output the native code and associated information with regard to Method, third-party parser
 can reconstruct the human-readable assembly representation of the compiled method based on dump file. To some extent,
we have implemented code cache dump by the way. For this reason, I'm not sure if the title of the RFC proposal should
be called metaspace dump, maybe metadata dump? It looks more like a metadata-dump framework.
Do you have any thoughts about metaspace/metadata dump? Looking forward to hearing your feedback, any comments are invaluable!
Best regards,
Yi Yang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-runtime-dev/attachments/20230111/10b5246b/attachment.htm>