RFR 8247670: Shenandoah: deadlock during class unloading OOME

[Roman Kennke]

Ok, let's do that.

Thanks,
Roman


On Fri, 2020-07-10 at 14:04 -0400, Zhengyu Gu wrote:
> The deadlock is caused by JDK-8245288. After further investigation, 
> JDK-8245288 change does not seem to provide improvement that reported
> in 
> original bug. Let's just backout JDK-8245288.
> 
> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8247670/webrev.01/
> 
> Test:
>    hotspot_gc_shenandoah
> 
> Thanks,
> 
> -Zhengyu
> 
> 
> On 6/16/20 3:48 PM, Zhengyu Gu wrote:
> > The deadlock is caused by one thread holding per-nmethod lock,
> > then 
> > encountering evac-oom. At the same time, another thread entering 
> > evac-oom scope, then acquiring the same per-nmethod lock.
> > 
> > The first thread expects the second thread to see evac-oom and exit
> > the 
> > scope, but the second thread is blocked on acquiring per-nmethod
> > lock.
> > 
> > The solution is to introduce an abortable locker on per-nmethod
> > lock. If 
> > the second thread can not acquire the lock, but see evac-oom, it
> > simply 
> > aborts, so it can exit evac-oom scope.
> > 
> > The solution does come with penalties:
> > 
> > If the second thread is a Java thread (via nmethod entry barrier),
> > the 
> > nmethod will be deopt.
> > 
> > If the second thread is worker, it causes current code root
> > processing 
> > to abort, then restart.
> > 
> > 
> > Bug: https://bugs.openjdk.java.net/browse/JDK-8247670
> > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8247670/webrev.00/
> > 
> > Test:
> >    hotspot_gc_shenandoah (x86_64 and aarch64)
> > 
> > Thanks,
> > 
> > -Zhengyu