RFR: Refactor allocation failure and explicit GC handling

[Roman Kennke]

Am 17.01.2018 um 09:32 schrieb Aleksey Shipilev:
> On 01/17/2018 03:45 AM, Zhengyu Gu wrote:
>> ShenandoahConcurrentThread::handle_alloc_failure() now takes monitor lock, and this method is in
>> ShenandoahHeap::allocate_memory() path, in turn, can be called inside write barrier ... seems to be
>> the scenario we talked before, that we *can not* do.
> 
> Note that allocate_memory on the *shared/TLAB* allocation path was taking a lock in the old code
> too: see the path in ShHeap::allocate_memory -> ShHeap::collect(_allocation_failure) ->
> ShConcThread::do_full_gc. The trick here is not to lock when shared_gc/GCLAB allocation fails, and
> this is why we have separate ::handle_alloc_failure_evac().
> 

Yes. We must not take locks under the write-barrier, because that is a 
leaf-call and must not possibly take a safepoint. It's ok to take locks 
in the allocation(-failure) path, because that is a no-leaf call, and 
may take safepoints.

  Infact, this is what happens with other GCs: allocation failure goes 
straight to VMThread::execute() ... I wonder if we could also do this 
and avoid the locking? But then, how to communicate with the ShConcThread ?

Roman