RFR: 10: 8160638: solaris JVM unable to allocate more than 2GB of direct byte buffers when max heap is <= 2GB

[Zhengyu Gu]

Hi Vladimir,

Sorry, my bad. Somehow, I had impression that compressed class space is 
below the heap, and has address < 4G.

-Zhengyu




On 04/18/2017 02:17 PM, Vladimir Kempik wrote:
> Hello Zhengyu Gu
>
> How will UseCompressedClassPointers run into trouble?
>
> UseCompressedClassPointers requires UseCompressedOops to be enabled, and
> in this case the code will adjust heap into higher space.
>
> and java  -Xmx2g -XX:MaxDirectMemorySize=9g -XX:+UseCompressedOops
> -XX:+UseCompressedCPointers -XX:HeapBaseMinAddress=10g MaxDirectMemory
>
> causes no troubles for me in test on jdk with no patch.
>
> Thanks, Vladimir
>
>
> 18.04.17 15:50, Zhengyu Gu пишет:
>>
>>
>> On 04/18/2017 08:22 AM, David Holmes wrote:
>>> On 18/04/2017 8:51 PM, Vladimir Kempik wrote:
>>>> Hello
>>>>
>>>> Sorry if it sounds a bit complicated, it's actually not that much.
>>>
>>> :)
>>>
>>>> 18.04.17 7:03, David Holmes пишет:
>>>>> Hi Vladimir,
>>>>>
>>>>> On 10/04/2017 11:30 PM, Vladimir Kempik wrote:
>>>>>> Hello
>>>>>>
>>>>>> Please review this fix for bug JDK-8160638
>>>>>> <https://bugs.openjdk.java.net/browse/JDK-8160638>
>>>>>>
>>>>>> The issue is with solaris only.
>>>>>>
>>>>>> When java mmaps heap (with compressed Oops enabled), mmaped heap
>>>>>> become
>>>>>> upper limit for any native mallocs.
>>>>>>
>>>>>> So when heap is starting at 2 gb, the maximum we can malloc is 2gb.
>>>>>>
>>>>>> Native malloc is used by Direct Memory Buffers, so even with
>>>>>> -XX:MaxDirectMemorySize=100g we are still limited with less than 2
>>>>>> gb of
>>>>>> memory for Direct Memory Buffers.
>>>>>>
>>>>>> The fix moves HeapBaseMinAddress to upper space when it's needed for
>>>>>> MaxDirectMemorySize to operate properly, leaving about 1 gb of native
>>>>>> memory for java's needs.
>>>>>
>>>>> Why is this not handled directly in Arguments::set_heap_size()?
>>>>
>>>> I don't change heap size here, only location of heap.
>>>
>>> True, but set_heap_size already does some checking of, and potential
>>> adjustment of HeapBaseMinAddress. And it isn't immediately obvious when
>>> your checks happen in relation to that code.
>>>
>>>>>
>>>>> I admit I'm unclear about the "magic numbers" involved here. From the
>>>>> bug report we have to have a heap >2G for things to work okay. So not
>>>>> sure why we adjust the HeapBaseMinAddress the way you do, or why 1GB
>>>>> is significant in the calculations ??
>>>> heap size doesn't have much to do with the issue, the issue happens
>>>> when
>>>> compressedOops get enabled, and it usualy happens with heap <=2G.
>>>>
>>>> for example, if we have 2gb of java heap (with compressed Oops),
>>>> starting at 2gb location (so java heap is located from 2gb to 4gb in
>>>> memory map) then we are limited with malloc allocations on solaris,
>>>> malloc will only allocate until we hit the wall of java heap
>>>> starting at
>>>> 2gb. malloc won't be able to allocate past 4gb limit in this case.
>>>> By default the heap is starting at 2gb on amd64, and 6gb on sparcv9,
>>>> however 0-4gb space is already used on sparcv9, so sparcv9 has same 2G
>>>> of free space before heap base.
>>>> since java internals is using this 2gb of c++ heap as well I thought it
>>>> would be good to not occupy more than half of that space with Direct
>>>> Memory Buffers.
>>>
>>> So if I understand this right, the direct byte buffers will always be
>>> allocated from this before-heap-memory ie below HeapBaseMinAddress, so
>>> we're shifting that up to make room for the space requested by
>>> MaxDirectMemorySize if it is >2GB.
>>
>> This is really the malloc implementation problem in Solaris, which has
>> low malloc base address by default and uses sbrk to expand malloc heap.
>>
>> It will still run into trouble if "UseCompressedClassPointers" flag is
>> on.
>
>>
>> Thanks,
>>
>> -Zhengyu
>>
>>>
>>> I don't get the 1GB adjustment though - doesn't that reduce what is
>>> available for direct byte buffers if "java internals" use more than 1GB
>>> itself?
>>>
>>> I think a picture would paint a thousand words here :)
>>>
>>> Thanks,
>>> David
>>>
>>>>
>>>> Thanks, Vladimir
>>>>>
>>>>> May be best for a GC person to review this.
>>>>>
>>>>> Thanks,
>>>>> David
>>>>>
>>>>>> Testing: jprt, included testcase.
>>>>>>
>>>>>> Webrev - http://cr.openjdk.java.net/~vkempik/8160638/webrev.00/
>>>>>>
>>>>>> Bug - https://bugs.openjdk.java.net/browse/JDK-8160638
>>>>>>
>>>>>> Thanks, Vladimir
>>>>>>
>>>>
>