Fixing the OOM-during-evac

[Roman Kennke]

Here's the current state. It should be complete, except for possible
assembly parts in the WB fast-path.
It's now done such that we can use a scope object
(ShenandoahOOMDuringEvacScope) around the parts that require it. This
will enter and leave the OOM-during-evac protected region. If entered
successfully, it will set a thread-local flag that communicates to the
enclosed evacuate_object() that it may evacuate, or else (safely)
return with RB(obj). evacuate_object() also checks (via corresponding
flag), that it is always enclosed by a ShenandoahOOMDuringEvacScope. I
checked all code paths that may lead to evacuate_object() and
installed such a scope object in all the right places.

Please give it a bit of pre-review and let me know what you think:
http://cr.openjdk.java.net/~rkennke/safe-oom-during-evac-counter.patch

I'd also be interested in relevant performance comparison / evaluation.

Meanwhile I'll think about how to do the assembly parts. :-)

Cheers, Roman

On Wed, Feb 28, 2018 at 4:00 PM, Aleksey Shipilev <shade at redhat.com> wrote:
> On 02/28/2018 02:53 PM, Roman Kennke wrote:
>> Here's my current prototype which seems to pass initial tests with
>> -XX:+ShenandoahOOMDuringEvacALot
>>
>> http://cr.openjdk.java.net/~rkennke/safe-oom-during-evac-counter.patch
>
> From the first glance, it looks right. But, we need to prove all corner cases are covered...
>
> -Aleksey
>
>