RFR: 10: 8160638: solaris JVM unable to allocate more than 2GB of direct byte buffers when max heap is <= 2GB

[Zhengyu Gu]

On 04/18/2017 08:22 AM, David Holmes wrote:
> On 18/04/2017 8:51 PM, Vladimir Kempik wrote:
>> Hello
>>
>> Sorry if it sounds a bit complicated, it's actually not that much.
>
> :)
>
>> 18.04.17 7:03, David Holmes пишет:
>>> Hi Vladimir,
>>>
>>> On 10/04/2017 11:30 PM, Vladimir Kempik wrote:
>>>> Hello
>>>>
>>>> Please review this fix for bug JDK-8160638
>>>> <https://bugs.openjdk.java.net/browse/JDK-8160638>
>>>>
>>>> The issue is with solaris only.
>>>>
>>>> When java mmaps heap (with compressed Oops enabled), mmaped heap become
>>>> upper limit for any native mallocs.
>>>>
>>>> So when heap is starting at 2 gb, the maximum we can malloc is 2gb.
>>>>
>>>> Native malloc is used by Direct Memory Buffers, so even with
>>>> -XX:MaxDirectMemorySize=100g we are still limited with less than 2
>>>> gb of
>>>> memory for Direct Memory Buffers.
>>>>
>>>> The fix moves HeapBaseMinAddress to upper space when it's needed for
>>>> MaxDirectMemorySize to operate properly, leaving about 1 gb of native
>>>> memory for java's needs.
>>>
>>> Why is this not handled directly in Arguments::set_heap_size()?
>>
>> I don't change heap size here, only location of heap.
>
> True, but set_heap_size already does some checking of, and potential
> adjustment of HeapBaseMinAddress. And it isn't immediately obvious when
> your checks happen in relation to that code.
>
>>>
>>> I admit I'm unclear about the "magic numbers" involved here. From the
>>> bug report we have to have a heap >2G for things to work okay. So not
>>> sure why we adjust the HeapBaseMinAddress the way you do, or why 1GB
>>> is significant in the calculations ??
>> heap size doesn't have much to do with the issue, the issue happens when
>> compressedOops get enabled, and it usualy happens with heap <=2G.
>>
>> for example, if we have 2gb of java heap (with compressed Oops),
>> starting at 2gb location (so java heap is located from 2gb to 4gb in
>> memory map) then we are limited with malloc allocations on solaris,
>> malloc will only allocate until we hit the wall of java heap starting at
>> 2gb. malloc won't be able to allocate past 4gb limit in this case.
>> By default the heap is starting at 2gb on amd64, and 6gb on sparcv9,
>> however 0-4gb space is already used on sparcv9, so sparcv9 has same 2G
>> of free space before heap base.
>> since java internals is using this 2gb of c++ heap as well I thought it
>> would be good to not occupy more than half of that space with Direct
>> Memory Buffers.
>
> So if I understand this right, the direct byte buffers will always be
> allocated from this before-heap-memory ie below HeapBaseMinAddress, so
> we're shifting that up to make room for the space requested by
> MaxDirectMemorySize if it is >2GB.

This is really the malloc implementation problem in Solaris, which has 
low malloc base address by default and uses sbrk to expand malloc heap.

It will still run into trouble if "UseCompressedClassPointers" flag is on.

Thanks,

-Zhengyu

>
> I don't get the 1GB adjustment though - doesn't that reduce what is
> available for direct byte buffers if "java internals" use more than 1GB
> itself?
>
> I think a picture would paint a thousand words here :)
>
> Thanks,
> David
>
>>
>> Thanks, Vladimir
>>>
>>> May be best for a GC person to review this.
>>>
>>> Thanks,
>>> David
>>>
>>>> Testing: jprt, included testcase.
>>>>
>>>> Webrev - http://cr.openjdk.java.net/~vkempik/8160638/webrev.00/
>>>>
>>>> Bug - https://bugs.openjdk.java.net/browse/JDK-8160638
>>>>
>>>> Thanks, Vladimir
>>>>
>>