Review request (hs24): 8007074: SIGSEGV at ParMarkBitMap::verify_clear()

[Daniel D. Daugherty]

On 7/9/13 8:21 AM, Stefan Karlsson wrote:
> On 07/09/2013 01:06 PM, Florian Weimer wrote:
>> On 07/02/2013 06:57 PM, Stefan Karlsson wrote:
>>> When the JVM starts it reserves a memory area for the entire Java heap.
>>> We use mmap(...MAP_NORESERVE...) to reserve a contiguous chunk of 
>>> memory
>>> that no other
>>> subsystem of the JVM, or Java program, will be allowed to mmap into.
>>>
>>> The reservation of the memory only reflects the maximum possible heap
>>> size, but often a smaller heap size is used if the memory pressure is
>>> low. The part of
>>> the heap that is actually used is committed with mmap(...MAP_FIXED...).
>>
>> Since you mention this, here's a really old pet peeve of mine.
>>
>> This does not work as intended.  mmap(...MAP_NORESERVE...) actually 
>> commits the memory.  The correct way to reserve address space on 
>> Linux is to drop MAP_NORESERVE and map the memory with PROT_NONE. 
>> Subsequently, you can commit the memory using mprotect() and suitable 
>> protection flags.  This way, the default heap size will no longer 
>> cause failures with vm.overcommit_memory=2.
>
> I wrote a small test program that tries to mmap 2TB on my 8GB machine. 
> This the outcome from the experiment:
>
> overcommit_memory = 0
>            | MAP_NORESERVE | no MAP_NORESERVE |
>            +---------------+------------------+
> PROT_WRITE |  succeeded    | failed           |
> PROT_NONE  |  succeeded    | succeeded        |
>            +---------------+------------------+
>
> overcommit_memory = 1
>            | MAP_NORESERVE | no MAP_NORESERVE |
>            +---------------+------------------+
> PROT_WRITE |  succeeded    | succeeded        |
> PROT_NONE  |  succeeded    | succeeded        |
>            +---------------+------------------+
>
> overcommit_memory = 2
>            | MAP_NORESERVE | no MAP_NORESERVE |
>            +---------------+------------------+
> PROT_WRITE |  failed       | failed           |
> PROT_NONE  |  succeeded    | succeeded        |
>            +---------------+------------------+
>
> With PROT_NONE we don't commit memory, independent of the usage of 
> MAP_NORESERVE.

It definitely looks like PROT_NONE has clearer/more consistent
semantics than does MAP_NORESERVE when it comes to making a
reservation.

So what does mprotect() do when you change a PROT_NONE mapping
to something else and there is no swap space available? Do you lose
your original PROT_NONE mapping entry?

Dan







>
> Note that we recently changed from using PROT_READ|PROT_WRITE to 
> PROT_NONE when reserving memory in JDK8:
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=8012015
>
>
> Me and Per Lidén discussed your suggestion to use mprotect to commit 
> memory instead of re-mmapping. We think that doing that might actually 
> solve the problem in:
> http://hg.openjdk.java.net/hsx/hotspot-rt/hotspot/rev/a837fa3d3f86
>
> where we today shutdown when mmap(...MAP_FIXED...) removes the 
> previous mapping when mmap fails for small pages.
>
> We could probably also change this fix for the large pages bug 
> (8007074) to use the same mprotect strategy. However, we would still 
> need most of the alignment changes from the 8007074 fix.
>
>>
>> Would you be interested in a patch that improves matters in this area?
>
> I think so, but I'll defer the question to the hotspot-runtime-dev team.
>
> If we want to use the mprotect strategy for small we should probably 
> try to use it for large pages as well.
>
> thanks,
> StefanK
>
>>
>> (Note that this only applies to Linux, not necessarily other systems.)
>>
>
>