[ZGC] [aarch64] Unable to allocate heap for certain Linux kernel configurations

[Stefan Karlsson]

On 2020-08-28 12:37, Christoph Göttschkes wrote:
> Hi Stefan,
> 
> On 2020-08-28 11:38, Christoph Göttschkes wrote:
>> Maybe there could be a mechanism which tries to allocate memory beyond 
>> certain addresses to try and detect the number of bits available? On 
>> my machine, for instance, the ZGC implementation tries to allocate 
>> memory with different starting addresses, but always gets an address 
>> back which is way smaller (because of the kernel limitations). Maybe, 
>> the ZGC implementation could store this information (the number of 
>> bits in addresses returned by mmap) and use this information to try 
>> and make another loop, which tries to allocate the heap with a reduced 
>> number of bits used for the addresses. This could also be a HotSpot 
>> option, to speed things up during startup if one knows that the 
>> machine uses a "weird" configuration.
> 
> I made a small prototype implementation of the idea I had, which 
> introduces a new HotSpot option, which re-adjusts itself if allocation 
> fails, due to not enough address bits available:
> 
> https://cr.openjdk.java.net/~cgo/8252500/prototype-webrev.00/
> 
> This isn't finished by any means. It is only implemented for aarch64 and 
> even compilation might fail for other platforms.
> 
> On the two test devices I have, it works as expected. The one which has 
> 4 pagetable levels, the allocation succeeds right in the beginning. On 
> the one with only 3 pagetable levels, the implementation first tries to 
> allocate memory with too high addresses and then re-adjusts the 
> available address bits and tries again. Configuring ZAddressBits low 
> enough makes the allocation succeed right in the beginning.
> 
> Maybe the patch is generic enough to fix your problems as well?

I'm not aware of any other supported platform where this would fail, 
other than AArch64. So, personally I'd like to keep a fix local to the 
AArch64 port, if possible.

A variant to your patch could be to add some code that you run once 
inside ZPlatformAddressOffsetBits, where you use mmap to probe out the 
valid address bits:

  size_t ZPlatformAddressOffsetBits() {
    // Runs once
    static size_t ZAddressBits = probe_valid_max_address_bit();
	
    const size_t max_address_offset_bits = ZAddressBits - 3;
    const size_t min_address_offset_bits = max_address_offset_bits - 2;
    const size_t address_offset = round_up_power_of_2(MaxHeapSize * 
ZVirtualToPhysicalRatio);
    const size_t address_offset_bits = log2_intptr(address_offset);
    return clamp(address_offset_bits, min_address_offset_bits, 
max_address_offset_bits);
  }

I think such a localized patch would be easier to get integrated into 
the code base.

StefanK

> 
> -- Christoph
>