PermGen Elimination project is promoting

[Coleen Phillimore]

On 9/27/2012 3:45 AM, Kirk Pepperdine wrote:
> Hi Jon,
>
> Thanks for the reply. As you know the problems with classloading and meta-data are not going to go away with the removal of perm-space, they are just being pushed else where. So I should have been a bit more explicit with my concerns. It's great that we're (once again) going to muck up the GC logs... ;-) but I fear that we're moving data into C heap and if my experience with slab allocators we've can be used as an indicator, dealing with problems that have been moved into C heap can be extremely difficult to deal with especially when you only see them in prod. The tools to deal with these problems are no where nearly as robust as they are in Java and they are an order of magnitude more difficult to use. It seems I'd like an API or a tool that would allow me to dump all of this meta-data out of C heap.

Hi,

  I am guessing that the problems that you are referring to are memory 
leak problems and not being able to tell which bits of metadata are 
being leaked.    That's a valid concern.  Did you use jmap and jhat to 
analyze your java heap and found metadata memory leaks?   And is the 
sort of functionality that you would like to see with what we are 
calling metaspace?

Metadata isn't self describing now, except via the vtable pointer for 
some bits of metadata.  But we do report on how much memory is used with 
native memory tracking.  This feature is new and we welcome enhancement 
request to this.

If you give us some use cases and requirements, we can file a detailed 
RFE so that we can support the sort of debugging that would be generally 
useful.   The serviceability agent will report close to the same 
information for metadata as it did when the metadata was in the Java 
heap.   I think.  I haven't really tried to use the SA.

Other potential problems in production, like crashes, can be debugged 
with full debugging symbols (FDS) feature.   Since the metadata is now 
in C++ and not opaque "oop" types, gdb and dbx debuggers are much 
happier to decode their fields.

Thank you for all of your feedback.
Coleen

>
> -- Kirk
>
> On 2012-09-27, at 7:23 AM, Jon Masamitsu<jon.masamitsu at oracle.com>  wrote:
>
>> Kirk,
>>
>> There is some information labeled with "Metaspace" in the GC output
>> that says how much memory is being used for class metadata.   It's
>> not really GC info but can be affected by GC (i.e., class unloading
>> during GC will reduce the used class metadata).  There are jstat
>> counters that currently carry the same name as the old perm
>> gen names.   Those names WILL change.  I'm checking on what
>> else there will be.
>>
>> Jon
>>
>> On 9/26/2012 9:24 PM, Kirk Pepperdine wrote:
>>> +1 on the summary comment.
>>>
>>> I've always wondered if the removal of perm gen was going to buy us (Java community) anything. Now I see with the move of meta-data to C heap that was it has bought us is the move of data from a space where we had tools to analyze problems that cropped up to a space where we don't. So, my question is; what facilities will be put into place that would allow us to view the class meta-data in a production VM?
>>>
>>> Regards,
>>> Kirk
>>>
>>> On 2012-09-27, at 5:06 AM, Srinivas Ramakrishna<ysr1729 at gmail.com>   wrote:
>>>
>>>> Nice summary, thanks!
>>>>
>>>> ysr1729
>>>>
>>>> On Sep 26, 2012, at 13:31, Jon Masamitsu<jon.masamitsu at oracle.com>   wrote:
>>>>
>>>>> We're expecting to promote hotspot with the perm gen elimination changes
>>>>> into JDK8 this week.  The last webrev for initial integration into hotspot
>>>>> is at
>>>>>
>>>>> http://cr.openjdk.java.net/~coleenp/metadata8/
>>>>>
>>>>> Basically this is the removal of the permanent generation and the use
>>>>> of native memory for hotspot's representation of class metadata.
>>>>>
>>>>> Generally speaking this change is invisible to the
>>>>> Java application.  However, depending on the allocation
>>>>> behavior, more Java heap may be used.  Also the
>>>>> amount of native memory used for the class metadata may be
>>>>> less than or more than the memory used previously by the
>>>>> permanent generation.  Basically the message is that
>>>>> you should not see any differences but then again you
>>>>> may.
>>>>>
>>>>> This change is in hsx 25 b2 and is expected to promote
>>>>> into jdk8 b58.
>>>>>
>>>>> These are some of the details.
>>>>>
>>>>>    - Most allocations for the class metadata previously
>>>>> done out of the permanent generation are now allocated
>>>>> out of native memory. Some miscellaneous  data has
>>>>> been moved to the Java heap.
>>>>>
>>>>>    - The permanent generation has been removed. The
>>>>> PermSize and  MaxPermSize are ignored and a warning is
>>>>> issued if they are present on the command line.
>>>>>
>>>>>    - The klasses that were used to described class
>>>>> metadata have been  removed (klassKlass and it's derivatives).
>>>>>
>>>>>    - By default class metadata allocation is only limited
>>>>> by the  amount of available native memory. Use the new flag
>>>>> MaxMetaspaceSize to limit the amount of  native memory used
>>>>> for class metadata. It is analogous to MaxPermSize.
>>>>>
>>>>>    - In 64bit VM when compressed oops are used a special
>>>>> fixed size  space is used for classes to set a compressed
>>>>> class pointer in object's header. The size  of that
>>>>> class's space is controlled by ClassMetaspaceSize flag
>>>>> with default value 2Mbytes.
>>>>>
>>>>>    - A garbage collection may be induced to collect dead
>>>>> classloaders  and classes. The first garbage collection
>>>>> will be induced when the class metadata  usage reaches
>>>>> MetaspaceSize (12Mbytes on the 32bit client VM and 16Mbytes
>>>>> on the 32bit  server VM with larger sizes on the 64bit VM's).
>>>>> Set MetaspaceSize to a higher value to delay the induced garbage
>>>>> collections. After an induced garbage collection the
>>>>> class metadata usage needed to induce the next garbage
>>>>> collection may be  increased.
>>>>>
>>>>>    - Whereas class metadata was previously garbage
>>>>> collected as part of the permanent generation, without
>>>>> the permanent generation the storage for class metadata
>>>>> is explicitly managed. C-heap allocation is not used.
>>>>>
>>>>>    - jstat counter names for metadata allocations
>>>>> have not been  updated. The old permanent generation names
>>>>> are still used but will be updated in the  near future.