RFR: 10: 8160638: solaris JVM unable to allocate more than 2GB of direct byte buffers when max heap is <= 2GB

[David Holmes]

On 18/04/2017 8:51 PM, Vladimir Kempik wrote:
> Hello
>
> Sorry if it sounds a bit complicated, it's actually not that much.

:)

> 18.04.17 7:03, David Holmes пишет:
>> Hi Vladimir,
>>
>> On 10/04/2017 11:30 PM, Vladimir Kempik wrote:
>>> Hello
>>>
>>> Please review this fix for bug JDK-8160638
>>> <https://bugs.openjdk.java.net/browse/JDK-8160638>
>>>
>>> The issue is with solaris only.
>>>
>>> When java mmaps heap (with compressed Oops enabled), mmaped heap become
>>> upper limit for any native mallocs.
>>>
>>> So when heap is starting at 2 gb, the maximum we can malloc is 2gb.
>>>
>>> Native malloc is used by Direct Memory Buffers, so even with
>>> -XX:MaxDirectMemorySize=100g we are still limited with less than 2 gb of
>>> memory for Direct Memory Buffers.
>>>
>>> The fix moves HeapBaseMinAddress to upper space when it's needed for
>>> MaxDirectMemorySize to operate properly, leaving about 1 gb of native
>>> memory for java's needs.
>>
>> Why is this not handled directly in Arguments::set_heap_size()?
>
> I don't change heap size here, only location of heap.

True, but set_heap_size already does some checking of, and potential 
adjustment of HeapBaseMinAddress. And it isn't immediately obvious when 
your checks happen in relation to that code.

>>
>> I admit I'm unclear about the "magic numbers" involved here. From the
>> bug report we have to have a heap >2G for things to work okay. So not
>> sure why we adjust the HeapBaseMinAddress the way you do, or why 1GB
>> is significant in the calculations ??
> heap size doesn't have much to do with the issue, the issue happens when
> compressedOops get enabled, and it usualy happens with heap <=2G.
>
> for example, if we have 2gb of java heap (with compressed Oops),
> starting at 2gb location (so java heap is located from 2gb to 4gb in
> memory map) then we are limited with malloc allocations on solaris,
> malloc will only allocate until we hit the wall of java heap starting at
> 2gb. malloc won't be able to allocate past 4gb limit in this case.
> By default the heap is starting at 2gb on amd64, and 6gb on sparcv9,
> however 0-4gb space is already used on sparcv9, so sparcv9 has same 2G
> of free space before heap base.
> since java internals is using this 2gb of c++ heap as well I thought it
> would be good to not occupy more than half of that space with Direct
> Memory Buffers.

So if I understand this right, the direct byte buffers will always be 
allocated from this before-heap-memory ie below HeapBaseMinAddress, so 
we're shifting that up to make room for the space requested by 
MaxDirectMemorySize if it is >2GB.

I don't get the 1GB adjustment though - doesn't that reduce what is 
available for direct byte buffers if "java internals" use more than 1GB 
itself?

I think a picture would paint a thousand words here :)

Thanks,
David

>
> Thanks, Vladimir
>>
>> May be best for a GC person to review this.
>>
>> Thanks,
>> David
>>
>>> Testing: jprt, included testcase.
>>>
>>> Webrev - http://cr.openjdk.java.net/~vkempik/8160638/webrev.00/
>>>
>>> Bug - https://bugs.openjdk.java.net/browse/JDK-8160638
>>>
>>> Thanks, Vladimir
>>>
>