RFR[9](M): 8037476: Tag MetaSpace objects with size and type to make them traceable at runtime

Fri Mar 14 21:33:08 UTC 2014

Hi,

I'm looking for some comments and feedback on the following change
which enhances MetaSpace objects with tags containng their size and
type to make them traceable at runtime:

http://cr.openjdk.java.net/~simonis/webrevs/8037476/
https://bugs.openjdk.java.net/browse/JDK-8037476

The HotSpot VM contains some debugging helper functions which can be
called during a live debugging session. One of these functions
("find(intptr_t x)" in debug.cpp) calls os::print_location() which
tries very hard to print some useful information for a given pointer
(e.g. if the pointer points into the Java heap, the function will call
"print_on()" for the corresponding oop which provides a lot of useful
meta-information for that object). This functionality is not only
restricted to live debugging. It is for example also used in the
hs_err reporting in the "Register to memory mapping" section.

In the past (i.e. before the PermGen removal in HS25)
os::print_location() worked equally well for true Java objects and
meta objects (like Symbols, Klass mirrors, etc.) because these meta
objects were stores as plain oops in the permanent generation of the
Java heap. However, after the permgen removal, his metadata is now
stored in the so called "Metaspace" which currently does not allow for
a reverse mapping of a native pointer back to the owning meta object.
The consequence is that os::print_location() currently only prints "is
pointing into metadata" for all native pointers pointing into the
metaspace.

This change transparently adds a tag in front of every MetaspaceObj
which contains the object's size and type. Together with a dispatching
method which is necessary because some of the classes in the
MetaspaceObj type hierarchy don't have a virtual function table it
becomes now possible to walk and print all the MetaspaceObj objects in
a typesafe way.

Of course, tagging each MetaspaceObj adds some space overhead but in
my opinion, this overhead is not very big. I've done some measuring
(with -XX:+PrintGCDetails -XX:+Verbose) and found that on 64-bit
platforms the tagging consumes about 4% additional 'used' metaspace,
but only about 3% additional committed metaspace memory (that's
probably because we waste some space anyway because of chunking).

On 32-bit, the overhead is a little bigger - currently about 6%.
However that's because of the fact that although a Metaword is only 32
bit on a 32-bit platform, the memory allocated for MetaspaceObjs is
kept 64-bit aligned. In order to keep the prototype simple, the
current implementation just uses two Metawords for the tag (although
one would be enough to hold the tag information). Changing this is
possible and could theoretically lead to an even smaller overhead on
32-bit platforms compared to 64-bit ones (because we could use the
otherwise wasted Metaword in about 50% of the allocations).
Unfortunately, the current Metaspace implementation does the padding
to 64-bit before the actual allocation which makes it a little tricky
to implement the tagging. The solution would be to not pad the
MetaspaceObj's size to 64-bit but instead align their starting address
during allocation.

The current implementation is protected with "NOT_PRODUCT" and
switchable at startup with -XX:+UseMetaspaceTags. If metaspace tagging
is switched on, we can now call "find()" in the debugger for a
metaspace address and get again an output as attached at the end of
this mail.

Also the output in the "Register to memory mapping" section of the
hs_err file can now contain more useful information (i.e. see the line
for "R13" which was simple "is pointing into metadata" before):

Register to memory mapping:

RAX=0x0000000854bf6698 is an unknown value
...
R11=0x00007fbb412312c0 is at entry_point+32 in (nmethod*)0x00007fbb41231150
R12=0x0000000000000000 is an unknown value
R13=0x00007fbb552ed9e7 is pointing into metadata
(start=0x00007fbb552ed9b0, size=2, tag=9, type=ConstMethod)
{constMethod}
 - method:       0x00007fbb552eda00 {method} {0x00007fbb552eda00}
'<clinit>' '()V' in 'sun/font/FontManagerNativeLibrary'
R14=0x00007fbb19345d08 is pointing into the stack for thread: 0x00007fbae400b800

As a future enhancement and if we decide to make this permanent
feature (i.e. not switachable at startup) this could also be
implemented more elegantly by simply storing the actual size and type
of every object directly in the 'MetaspaceObj'. Another possibility
would be to store just the object size in 'MetaspaceObj' and add
virtual print() function, although that would probably require even
more overhead.

I've tested this implementation so far on Linux/x86/x86_64/ppc64 and
did some basic testing on Solaris/Sparcv9. The only thing I haven't
looked at until now is support for class data sharing.

Thank you and best regards,
Volker

Below you can see the output of the debugger helper function "find()"
with the new metaspace tagging implementation when called during a
live debugging session:

(gdb) call find(0x00007fffec41d4c0)

"Executing find"
0x00007fffec41d4c0 is pointing into metadata
(start=0x00007fffec41d4c0, size=5, tag=8, type=Method)
{method}
 - this oop:          0x00007fffec41d4c0
 - method holder:     synchronized 'IntLoopWithGC'
 - constants:         0x00007fffec41d098 constant pool [78]
{0x00007fffec41d098} for synchronized 'IntLoopWithGC'
cache=0x00007fffec41d698
 - access:            0xc0000008  static
 - name:              'loop'
 - signature:         '(I)V'
...

(gdb) call find(0xafdda210)

"Executing find"
0xafdda210 is pointing into metadata (start=0xafdda210, size=2, tag=5,
type=TypeArrayU4)
0: 0xafdda300
1: 0xafdda260
2: 0xafdda3f8

(gdb) call find(0xafdda2a0)

"Executing find"
0xafdda2a0 is pointing into metadata (start=0xafdda2a0, size=20,
tag=9, type=ConstMethod)
{constMethod}
 - method:       0xafdda300 {method} {0xafdda300} 'loop' '(I)V' in
synchronized 'IntLoopWithGC'
 - stackmap data:       Array<T>(0xafdda340)

(gdb) call findpc(0x00007fffec41d300)

"Executing findpc"
0x00007fffec41d300 is pointing into metadata
(start=0x00007fffec41d098, size=82, tag=11, type=ConstantPool)
{constant pool}
 - holder: 0x0000000100060038
 - cache: 0x00007fffec41d698
 - resolved_references: 0x00000000d6bd8000
 - reference_map: 0x00007fffec41d870
 -   1 : Method : klass_index=23 name_and_type_index=39
 -   2 : Integer : 1000000
 -   3 : Field : klass_index=22 name_and_type_index=40
...