[ZGC] [aarch64] Unable to allocate heap for certain Linux kernel configurations

[Stuart Monteith]

On 28/08/2020 11:42, Stefan Karlsson wrote:
> On 2020-08-28 11:38, Christoph Göttschkes wrote:
>> Hi Stefan, thanks for your feedback. Looks like this case isn't as exotic as I first thought. May I ask which kind of 
>> machine this is? Also a small embedded board?
> 
> It was one of our compiler devs that ran into this. I don't think it was a small machine, but rather that was configured 
> differently then other AArch64 machines that we've run on.
> 
>>
>> On 2020-08-28 11:04, Stefan Karlsson wrote:
>>> I think we hit a very similar problem during some internal testing on one machine. I have a patch to workaround that 
>>> problem:
>>>
>>> https://cr.openjdk.java.net/~stefank/prototype/zaarch-va/webrev.01/
>>
>> Your patch works with some modifications. In my case, only 39 bits are available in the virtual address space. I put 
>> that value as "va_bits" and it works.
> 
> OK. Good to know.
> 
>>
>>> Unfortunately, this patch only solves the problem on a very specific setup, and I don't think it covers your 
>>> use-case. Hopefully, someone with enough AArch64 machine config knowledge would be able to extend this patch to also 
>>> cover all possible combinations.
>>
>> I don't think the possible combinations are the problem. [1] shows them (sorry, didn't put that link in the last mail) 
>> for Linux. I think the real problem is detecting this and making the addressing scheme adjust itself.
>>
>> Maybe there could be a mechanism which tries to allocate memory beyond certain addresses to try and detect the number 
>> of bits available? On my machine, for instance, the ZGC implementation tries to allocate memory with different 
>> starting addresses, but always gets an address back which is way smaller (because of the kernel limitations). Maybe, 
>> the ZGC implementation could store this information (the number of bits in addresses returned by mmap) and use this 
>> information to try and make another loop, which tries to allocate the heap with a reduced number of bits used for the 
>> addresses. This could also be a HotSpot option, to speed things up during startup if one knows that the machine uses a 
>> "weird" configuration.
>>
>>> I think creating a bug report would be a good start. Do you have an openjdk user name? If not I can create a bug report.
>>
>> Yes, I can create bug reports. I used my first mail and created one [2].
> 
> Thanks! We have had some brief discussions with Stuart (CC:ed), who created the AArch64 port, about this problem. Maybe 
> he has had some time to think about it, and have some additional insights or ideas.
> 
> Thanks,
> StefanK
> 
>>
>> -- Christoph
>>
>> [1] 
>> https://urldefense.com/v3/__https://www.kernel.org/doc/Documentation/arm64/memory.txt__;!!GqivPVa7Brio!LfyuMvR1e95FLlQs-8R-nh4-o89GIOgBFv1QU5_Y8OwPMh7ckvRjmrgp0vjyD0UaGUhW$ 
>> [2] https://bugs.openjdk.java.net/browse/JDK-8252500
>>

Hi,
	That's right - I have been exploring options on this, and I had a similar solution at one point to finding the address 
space size. From speaking with people familiar with the arm64 linux kernel, there is no good way to query the available 
address space except for probining it and testing what is there. Thinking we could do with a general-purpose routine, I 
experimented with a routine that forks the process and probes the address space non-destructively. MAP_FIXED implicitly 
destroys any existing mappings. Of course, ZGC mmaps memory at fixed addresses anyhow, so the concern about embedded the 
JVM in your program and destroying existing mappings turned out to be moot, as we'd be doing that anyway.

CCing Monica as the Windows platform might have similar issues.


Thanks,
	Stuart


BR,
	Stuart