[ZGC] [aarch64] Unable to allocate heap for certain Linux kernel configurations

[Christoph Göttschkes]

Hi Stuart.

On 2020-08-28 16:56, Stuart Monteith wrote:
> Hi,
>      That's right - I have been exploring options on this, and I had a 
> similar solution at one point to finding the address space size. From 
> speaking with people familiar with the arm64 linux kernel, there is no 
> good way to query the available address space except for probining it 
> and testing what is there. Thinking we could do with a general-purpose 
> routine, I experimented with a routine that forks the process and probes 
> the address space non-destructively. MAP_FIXED implicitly destroys any 
> existing mappings. Of course, ZGC mmaps memory at fixed addresses 
> anyhow, so the concern about embedded the JVM in your program and 
> destroying existing mappings turned out to be moot, as we'd be doing 
> that anyway.
After reading your comment, the only other viable solution I came up 
with is using a combination of msync and mmap. I didn't fully look into 
this yet, but made a small prototype to show you the idea and to get 
early feedback. Maybe someone already looked into this and knows that 
some edge case doesn't work.

General idea:
First use msync to check if there is already a mapping for the page. If 
msync returns ENOMEM, this either means: there is no mapping yet, or the 
address is invalid. After getting ENOMEM, we can use mmap to try and map 
the page. If we get back the same address, we know the address is valid. 
If the address is different, we know it is invalid. I also don't want to 
use MAP_FIXED, but maybe MAP_FIXED_NOREPLACE (as mentioned by Florian) 
would be a solution, but it was only introduced in Linux 4.17, so I 
guess we would need a fallback solution.

I tested this on different Linux aarch64 devices and it seems to work. 
You can find the prototype here:

https://cr.openjdk.java.net/~cgo/8252500/prototype-webrev.01/

-- Christoph