Idea: aliased heap for checking to-space invariant

[Roman Kennke]

Am 30.01.2018 um 16:07 schrieb Aleksey Shipilev:
> So I have been walking and muttering to myself how we cannot mprotect(PROT_READ) the collection set,
> because we have to accept the fwdptr update in the same page. We used to mprotect cset for
> verification, but that code basically mprotect(PROT_WRITE)-ed the page when fwdptr write had
> faulted, restarted the fwdptr update, accepting everything else after that too. Thus it was became
> too racy to be useful. This was the reason for us to ditch that verification part, and instead rely
> on explicit ShenandoahStoreCheck machinery.
> 
> Then it hit me: the memory protection is enforced on virtual pages, not on physical pages, which
> means we can use the aliased heap to accept the fwdptr stores, while normal heap cset is protected
> from writes! I.e. have the normal heap WRITE|READ as usual, have the alias heap WRITE|READ as usual,
> then when cset is selected WRITE-protect the cset, and watch out for failures. The fwdptr updates
> from WB code should instead go via the aliased heap that is WRITE-enabled.
> 
> This gives us several advantages:
>    *) We capture all bad writes mechanically, instead of hoping we covered all ShStoreCheck cases
>    *) The upstream exposure in .ad and platform-specific macro-assemblers goes away
>    *) Roman's work on aliased heaps is not in vain :)
>    *) We don't arrive to the mess with "differently-shaped" pointers to both normal and aliased heap,
> because we never leak aliased heap pointers anywhere: we just use that as the location for the
> fwdptr CAS.
> 
> We can (and probably should) only enable this for verification, so we don't have any ill effects for
> non-verificated modes (which would just do the same thing they do today).
> 

This sounds like a great idea! I think it would work.

If we are going to introduce the machinery for multi-mapping (and we 
might eventually get it through ZGC anyway), we might want to think 
about finishing off the safe-oom-during-evac issue. Your inspiration 
inspired me: what if we create the 2nd mapping only on demand? I.e. when 
we hit oom-during-evac, we create the 2nd mapping, use it to safely flip 
the blocking bit in the forwarding pointer, and when the whole 
oom-during-evac sequence is over, we unmap the 2nd mapping. This way 
we'd keep memory counters in the kernel normal until we hit the very 
unlikely error case, and then only for a limited time. WDYT?

Roman