RFR (S) 8008217: CDS: Class data sharing limits the malloc heap on Solaris

[Coleen Phillimore]

Summary: In 64bit VM move CDS archive address to 32G on all platforms 
using new flag SharedBaseAddress. In 32bit VM set CDS archive address to 
2Gb on Linux and let other OSs pick the address.

open webrev at http://cr.openjdk.java.net/~coleenp/8008217_2/
bug link at http://bugs.sun.com/view_bug.do?bug_id=8008217_2

Tjhis is a rereview based on Vladimir's comments.  On linux 32 the 
SharedBaseAddress is set to 2G.  The default address for dumping isn't 
available for restoring (for no good reason I can tell, linux just mmaps 
a different address).   2G seems to work for a default though.   If the 
heap is large (not default) and overlaps 2G, the CDS is turned off.   
CDS is mapped after the Java heap is initialized.

On Windows and Solaris 32 bit the default mmap address for -Xshare:dump 
is used.   This is usually successful to mmap for -Xshare:on with 
default heap size.

On 64 bit 32G is used as a base address, which is generally above the 
Java heap.   This was tested on all platforms (again) but mostly on 
linux 32 bit.

Thanks,
Coleen


On 3/18/2013 7:23 PM, Coleen Phillimore wrote:
> On 3/18/2013 7:00 PM, Vladimir Kozlov wrote:
>> On 3/18/13 3:30 PM, Coleen Phillimore wrote:
>>> On 3/18/2013 1:32 PM, Vladimir Kozlov wrote:
>>>> Coleen,
>>>>
>>>> First, your description is not accurate. Should be something like 
>>>> this:
>>>>
>>>> "In 64bit VM move CDS archive address to 32G on all platforms using
>>>> new flag SharedBaseAddress.
>>>> In 32bit VM set CDS archive address to 2Gb on Linux and let other OSs
>>>> pick the address."
>>>
>>> Yes, that sounds better than what I wrote.
>>>
>>>>
>>>> I think you need to always set SharedBaseAddress to 32G with the flag
>>>> description that it is default for 64-bit VM. Special flag setting for
>>>> Linux in 32bit VM could be done in VirtualSpaceNode() or in
>>>> arguments.cpp (where CDS values are set) under #ifdef if the value is
>>>> default.
>>>
>>> I don't understand this comment.   Does it mean make it 32G and 2G in
>>> globals.hpp rather than in the globals_os_cpu.hpp files?  I don't have
>>> good experiences with special case code in arguments.cpp and the
>>> complexity we keep adding to it, and want to avoid that.
>>
>> If you had different values on different Linux flavors then setting 
>> in globals_os_cpu.hpp is good. But you have the same value. If you 
>> need to change 2G value later you will need to change again all these 
>> files.
>
> This is true.  You convinced me.   I'd rather code in the declaration 
> than these #ifdefs in arguments.cpp.  There are too many #ifdefs in 
> arguments.cpp:
>
>   product(uintx, SharedBaseAddress, NOT_LP64(SOLARIS_ONLY(0) 
> NOT_SOLARIS(3*G))\
> LP64_ONLY(32*G), \
>           "Address to allocate shared memory if OS default doesn't 
> work")   \
> \
> On my Linux 32 bit machine 3G doesn't work (java -version doesn't get 
> back the address).  If I let Linux 32 bit default, it picks an address 
> close to 3G, and Java -version does work.
>
> Maybe it's better that Linux and Windows pick an address, which is the 
> way they used to work (or not) before I happened to test them with 
> NPG.   I'd have to retest Windows with this because I don't remember 
> what the results there were.   What do you think?
>
> thanks,
> Coleen
>
>
>>
>> There is Arguments::set_shared_spaces_flags() method where you can do 
>> it (at the end of the method):
>>
>> #ifndef _LP64
>>   if (UseSharedSpaces && FLAG_IS_DEFAULT(SharedBaseAddress)) {
>> #ifdef LINUX
>>     // set CDS archive address to 2Gb on Linux in 32bit VM
>>     FLAG_SET_DEFAULT(SharedBaseAddress, 2*G);
>> #else
>>     // let other OSs pick the address
>>     FLAG_SET_DEFAULT(SharedBaseAddress, 0);
>> #endif
>>   }
>> #endif
>>
>>>
>>>>
>>>> Also setting in 32bit VM the CDS address on Linux to 2G may limit
>>>> ability to specify large java heap size since 2G is in the middle of
>>>> virtual space. Can it be somewhere at the top of virtual space (4G -
>>>> cds_size)? Or at the bottom? Did you look on heap mapping in that
>>>> case? Note, HeapBaseMinAddress was never used in 32bit VM before NPG.
>>>>
>>>
>>> I was wondering what a better address than 2G would be for Linux. Which
>>> is why I put the settings in the different os_cpu files.  If I let 
>>> Linux
>>> default, it picks a high address for -Xshare:dump but then cannot get
>>> this higher address when ute tests are run.   2G is sort of in the
>>> middle which always seems to work, and with 32 bit, the Java heap
>>> shouldn't be that large anyway because otherwise people would be using
>>> 64 bits.   Maybe 3G might be better, or 3.5G.
>>> I didn't dare try to use lower addresses because some other application
>>> specific data would be more likely to want these addresses. So, that's
>>> another reason I made it an option and split their settings out.
>>
>> I think you should go as high as possible to leave more space for 
>> java heap. It would be nice to determine the address dynamically if 
>> possible. I think the problem with UTE is testing is done in Virtual 
>> Machines and not on hardware.
>>
>> I am concern about escalations from customers later that they can't 
>> specify big heap any more with 32bit VM. I would suggest to ask our 
>> embedded group to look on this.
>>
>> Thanks,
>> Vladimir
>>
>>>
>>> Thanks,
>>> Coleen
>>>
>>>> Thanks,
>>>> Vladimir
>>>>
>>>> On 3/18/13 7:51 AM, Coleen Phillimore wrote:
>>>>>
>>>>> Adding in some more potential reviewers...  please review!
>>>>> Thanks,
>>>>> Coleen
>>>>>
>>>>> On 03/15/2013 02:25 PM, Coleen Phillimore wrote:
>>>>>> Summary: Don't use HeapBaseMinAddress on solaris, let the OS pick 
>>>>>> the
>>>>>> CDS archive address on solaris.  Move all CDS archive addresses 
>>>>>> up to
>>>>>> 32G on 64 bit.
>>>>>>
>>>>>> On Linux, this still uses 2*G address because running ute, we can't
>>>>>> seem get the same address that -Xshare:dump choses. Solaris 
>>>>>> chooses a
>>>>>> higher address which seems to work fine.
>>>>>>
>>>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8008217/
>>>>>> bug link at http://bugs.sun.com/view_bug.do?bug_id=8008217
>>>>>>
>>>>>> Tested manually with gc/gctests/ManyObjects on all platforms. To add
>>>>>> a jtreg test, we'd have to introduce a new flag to change the 
>>>>>> location
>>>>>> of the shared archive because we don't have write access to 
>>>>>> JAVA_HOME
>>>>>> while testing.   Do people think this is worth doing?
>>>>>>
>>>>>> Thanks,
>>>>>> Coleen
>>>>>
>>>
>