RFR 8247670: Shenandoah: deadlock during class unloading OOME

[Zhengyu Gu]

Thanks and pushed.

-Zhengyu

On 7/10/20 2:14 PM, Roman Kennke wrote:
> 
> Ok, let's do that.
> 
> Thanks,
> Roman
> 
> 
> On Fri, 2020-07-10 at 14:04 -0400, Zhengyu Gu wrote:
>> The deadlock is caused by JDK-8245288. After further investigation,
>> JDK-8245288 change does not seem to provide improvement that reported
>> in
>> original bug. Let's just backout JDK-8245288.
>>
>> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8247670/webrev.01/
>>
>> Test:
>>     hotspot_gc_shenandoah
>>
>> Thanks,
>>
>> -Zhengyu
>>
>>
>> On 6/16/20 3:48 PM, Zhengyu Gu wrote:
>>> The deadlock is caused by one thread holding per-nmethod lock,
>>> then
>>> encountering evac-oom. At the same time, another thread entering
>>> evac-oom scope, then acquiring the same per-nmethod lock.
>>>
>>> The first thread expects the second thread to see evac-oom and exit
>>> the
>>> scope, but the second thread is blocked on acquiring per-nmethod
>>> lock.
>>>
>>> The solution is to introduce an abortable locker on per-nmethod
>>> lock. If
>>> the second thread can not acquire the lock, but see evac-oom, it
>>> simply
>>> aborts, so it can exit evac-oom scope.
>>>
>>> The solution does come with penalties:
>>>
>>> If the second thread is a Java thread (via nmethod entry barrier),
>>> the
>>> nmethod will be deopt.
>>>
>>> If the second thread is worker, it causes current code root
>>> processing
>>> to abort, then restart.
>>>
>>>
>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8247670
>>> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8247670/webrev.00/
>>>
>>> Test:
>>>     hotspot_gc_shenandoah (x86_64 and aarch64)
>>>
>>> Thanks,
>>>
>>> -Zhengyu
>