<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
CC-ing serviceability.<br>
<br>
Hi Yi,<br>
<br>
In general, I think it's good to have tools for understanding the
internal layout of the class metadata layouts.<br>
<br>
I think there are overlaps between your proposal and existing tools.
For example, there are jcmd options such as VM.class_hierarchy and
VM.classes, etc.<br>
<br>
The Serviceability Agent can also be used to analyze the contents of
the class metadata.<br>
<br>
Dd you look at the existing tools and see how they match up with
your requirements?<br>
<br>
I'd be interested in seeing your implementation and compare it with
the existing tools.<br>
<br>
<br>
<div class="moz-cite-prefix">On 1/11/2023 4:56 AM, Yi Yang wrote:<br>
</div>
<blockquote type="cite" cite="mid:cfb6adf7-3eef-42a2-a82b-5fdf638428c4.qingfeng.yy@alibaba-inc.com">
<div class="__aliyun_email_body_block">
<div style="line-height:1.7;font-family:tahoma;font-size:14.0px;color:#000000;">
<div style="clear:both;">Hi,<br>
</div>
<div style="clear:both;"><br>
</div>
<div style="clear:both;">
<div style="clear:both;">Internally, we often receive
feedback from users and ask for help on metaspace-related
issues, for example</div>
<div style="clear:both;">1. Users are eager to know which
GroovyClassLoader loads which classes, why they are not
unloaded,</div>
<div style="clear:both;">and why they are leading to
Metaspace OOME.</div>
<div style="clear:both;">2. They want to know the class
structure of dynamically generated classes in some
scenarios such as </div>
<div style="clear:both;">deserialization</div>
<div style="clear:both;">3. Finding memory leaking about
duplicated classes</div>
<div style="clear:both;">...</div>
<div style="clear:both;">Internally we implemented a
metaspace dump that generates human-readable text, it
looks something like this:</div>
<div style="clear:both;"><br>
</div>
<div style="clear:both;">[Basic Information]</div>
<div style="clear:both;">Dump Reason : JCMD</div>
<div style="clear:both;">MaxMetaspaceSize :
18446744073709547520 B</div>
<div style="clear:both;">CompressedClassSpaceSize :
1073741824 B</div>
<div style="clear:both;">Class Space Used : 309992 B</div>
<div style="clear:both;">Class Space Capacity : 395264 B</div>
<div style="clear:both;">...</div>
<div style="clear:both;">[Class Loader Data]</div>
<div style="clear:both;">ClassLoaderData : loader =
0x000000008024f928, loader_klass = 0x0000000800010098,
loader_klass_name = </div>
<div style="clear:both;">sun/misc/Launcher$AppClassLoader,
label = N/A</div>
<div style="clear:both;"> Class Used Chunks :</div>
<div style="clear:both;"> * Chunk : [0x0000000800060000,
0x0000000800060230, 0x0000000800060800)</div>
<div style="clear:both;"> NonClass Used Chunks :</div>
<div style="clear:both;"> * Chunk : [0x00007fd8379c1000,
0x00007fd8379c1350, 0x00007fd8379c2000)</div>
<div style="clear:both;"> Klasses :</div>
<div style="clear:both;"> Klass : 0x0000000800060028,
name = Test, size = 520 B</div>
<div style="clear:both;"> ConstantPool :
0x00007fd8379c1050, size = 296 B</div>
<div style="clear:both;">...</div>
<div style="clear:both;"><br>
</div>
<div style="clear:both;">It has been working effectively for
several years and has helped many users solve
metaspace-related problems.</div>
<div style="clear:both;">But a more user-friendly way is
that JDK can inherently support this capability. We hope
that format of the metaspace</div>
<div style="clear:both;">dump file can take both flexibility
and compatibility into account, and the content of dump
file should be detailed</div>
<div style="clear:both;">enough to meet the needs of both
application developers and lower-level developers.</div>
<div style="clear:both;"><br>
</div>
<div style="clear:both;">Based on above considerations, I
think using JSON as its file format is an appropriate
solution(But XML or binary </div>
<div style="clear:both;">format are still not excluded as
candidates). Specifically, in earlier thoughts, I thought
the format of the metaspace</div>
<div style="clear:both;">file could be as follows(pretty
printed)</div>
<div style="clear:both;"><br>
</div>
<div style="clear:both;"><a href="https://gist.github.com/y1yang0/ab3034b6381b8a9d215602c89af4e9c3" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">https://gist.github.com/y1yang0/ab3034b6381b8a9d215602c89af4e9c3</a></div>
<div style="clear:both;"><br>
</div>
<div style="clear:both;">Using the JSON format, we can
flexibly add new fields without breaking compatibility. It
is debatable as to which data</div>
<div style="clear:both;">to write. We can reach a consensus
that third-party parsers(Metaspace Analyzer Tool) can at
least reconstruct Java</div>
<div style="clear:both;">source code from the dump file. </div>
</div>
</div>
</div>
</blockquote>
<br>
This may be quite difficult, because the metadata contains rewritten
Java bytecodes. The rewriting format may be dependent on the JDK
version. Also, the class linkage (the resolution of constant pool
information) will be vastly from one JDK version to another. So
using writing a third party tool that can work with multiple JDK
versions will be quite hard. Also, defining a "portable" format for
the dump will be difficult, since we don't know how the internal
data structure will evolve in the future.<br>
<br>
Thanks<br>
- Ioi<br>
<br>
<br>
<blockquote type="cite" cite="mid:cfb6adf7-3eef-42a2-a82b-5fdf638428c4.qingfeng.yy@alibaba-inc.com">
<div class="__aliyun_email_body_block">
<div style="line-height:1.7;font-family:tahoma;font-size:14.0px;color:#000000;">
<div style="clear:both;">
<div style="clear:both;">Based on this, we can write more
useful information for low-level troubleshooting</div>
<div style="clear:both;">or debugging. (e.g. the init_state
of InstanceKlass).</div>
<div style="clear:both;"> In addition, we can even output
the native code and associated information with regard to
Method, third-party parser</div>
<div style="clear:both;"> can reconstruct the human-readable
assembly representation of the compiled method based on
dump file. To some extent,</div>
<div style="clear:both;">we have implemented code cache dump
by the way. For this reason, I'm not sure if the title of
the RFC proposal should</div>
<div style="clear:both;">be called metaspace dump, maybe
metadata dump? It looks more like a metadata-dump
framework.</div>
<div style="clear:both;"><br>
</div>
<div style="clear:both;">Do you have any thoughts about
metaspace/metadata dump? Looking forward to hearing your
feedback, any comments are invaluable!</div>
<div style="clear:both;"><br>
</div>
<div style="clear:both;">Best regards,</div>
<span>Yi Yang</span></div>
</div>
</div>
</blockquote>
<br>
</body>
</html>