RFR (L) 8072061: Automatically determine optimal sizes for the CDS regions

[Ioi Lam]

Hi Colleen,

Thanks for the review and all of your help to fix my previous 
implementations, which were a bit hacky to say the least :-)


On 6/30/17 3:16 PM, coleen.phillimore at oracle.com wrote:
>
> Ioi, I have one comment and one question.
>
> http://cr.openjdk.java.net/~iklam/jdk10/8072061-auto-size-cds-region.v10/hotspot/src/share/vm/memory/metaspace.cpp.udiff.html 
>
>
> This cleanup is worth all of the change.   It's nice to not conflate 
> metaspace allocation with the creating shared region anymore.   It 
> started out as a little bit of extra code but has expanded to all of 
> what you removed.   So I guess you've convinced me that copying 
> metadata is worth doing.
>
> http://cr.openjdk.java.net/~iklam/jdk10/8072061-auto-size-cds-region.v10/hotspot/src/share/vm/utilities/hashtable.cpp.udiff.html 
>
>
> Why do you have to include metaspaceShared.hpp here?
>
Oops, that was added by accident. I removed it.

Thanks
- Ioi
> I've prereviewed this and have no further comments. This looks good!  
> Thank you for making the changes I suggested in the pre-review.  
> Starting with copy_and_compact() it makes sense what this is doing and 
> looks like it'll be easier to understand overall.
>
> Thanks!
> Coleen
>
>
> On 6/29/17 6:55 PM, Ioi Lam wrote:
>> https://bugs.openjdk.java.net/browse/JDK-8072061
>> http://cr.openjdk.java.net/~iklam/jdk10/8072061-auto-size-cds-region.v10/ 
>>
>>
>> ------------------------------------------------------------------------------ 
>>
>> Motivation:
>>
>>   The CDS archive is divided in several regions such as RO, RW, etc. In
>>   JDK 9, when you are dumping with a custom classlist, you must manually
>>   specify the region sizes using flags such as -XX:SharedReadOnlySize.
>>   This is error prone and cumbersome.
>>
>>   This RFE makes it possible to automatically determine the optimal 
>> sizes
>>   of the regions.
>>
>> Overview:
>>
>>   With this RFE, during CDS dump time, all class metadata are first
>>   allocated outside of the shared regions. Then, we scan all of the
>>   reachable metadata objects, determine whether they are RO or RW, and
>>   then copy them into the destination regions.
>>
>>   After all objects are copied, we scan and relocate all the pointers to
>>   point to the new copies.
>>
>> Notes to reviewers:
>>
>>   To implement the copy and relocation operations, we have added the new
>>   MetaspaceClosure API for walking all the reachablemetadata objects, 
>> and
>>   scanning all the pointers embedded in these objects.
>>
>>   Please start in metaspaceShared.cppand follow the code path:
>>
>>     ArchiveCompactor::copy_and_compact()
>>     -> iterate_roots()
>>       -> SystemDictionary::classes_do(it)
>>         -> Dictionary::classes_do(MetaspaceClosure* it)
>> -> it->push(probe->klass_addr())
>>
>>   At this point you will get into metaspaceClosure.hpp. Please read the
>>   comments around MetaspaceClosure::Ref about the (lack of) C++ vtables,
>>   and why the template classes ObjectRef, PrimitiveArrayRef and
>>   PointerArrayRef are needed.
>>
>>   Once you understand how ObjectRefand PointerArrayRef work, you can
>>   see that the above it->push(probe->klass_addr()) call will be invoking
>>   this template method:
>>
>>      template <class T> void push(T** mpp, Writability w = _default) {
>>        ObjectRef<T> ref(mpp);
>>        push_impl(&ref, w);
>>      }
>>
>>   which will eventually call into Klass::metaspace_pointers_do(), which
>>   will eventually recursively iterate over all the Klasses, Methods,
>>   ConstantPools and other types of MetaspaceObj.
>>
>> Testingstatus:
>>
>>   All CDS tests have passed locally on Linux. All new tests have been
>> implemented.
>>
>> I am waiting to run RBT tests on other platforms.
>>
>> Thanks
>> - Ioi
>