RFR (S/M) JDK-8010196 NPG: Internal Error: Metaspace allocation lock -- possible deadlock

[Christian Tornqvist]

Hi Mikael,

Test looks good apart from two minor issues:

Spelling error:
43         throw new RuntimeException("Unalbe to load class file");

Don't use System.exit() in jtreg
(http://openjdk.java.net/jtreg/faq.html#question2.6):
120       System.exit(1);

Thanks,
Christian

-----Original Message-----
From: hotspot-runtime-dev-bounces at openjdk.java.net
[mailto:hotspot-runtime-dev-bounces at openjdk.java.net] On Behalf Of Mikael
Gerdin
Sent: den 9 april 2013 11:44
To: Leonid Mesnik
Cc: hotspot-gc-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net
Subject: Re: RFR (S/M) JDK-8010196 NPG: Internal Error: Metaspace allocation
lock -- possible deadlock

Good news!
I managed to write a regression test by adding a bit more dependencies every
iteration:

http://cr.openjdk.java.net/~mgerdin/8010196/webrev.0-regtest/webrev/

/Mikael


On 04/08/2013 06:34 PM, Mikael Gerdin wrote:
> Leonid,
>
> On 04/08/2013 04:19 PM, Leonid Mesnik wrote:
>> Mikael
>>
>> Is it possible to include regression test which reproduce problem?
>> (based on BigArityTest.java)
>
> I can't say I understand what BigArityTest does that makes it 
> introduce cross-CLD dependencies.
> I did write a test which does cause dependencies but I failed to make 
> it reproduce the problem, even though some of the dependencies were 
> added during a G1 concurrent mark cycle.
>
> /Mikael
>
>>
>> Leonid
>>
>> On 04/08/2013 05:49 PM, Mikael Gerdin wrote:
>>> Hi
>>>
>>> The problem is that when running the G1 garbage collector a call to 
>>> objArrayOopDesc::obj_at_put can in rare cases cause the G1 dirty 
>>> card queue buffer to fill up and this will cause G1 to take
>>> DirtyCardQ_CBL_mon/16 to return the full buffer and get a new one.
>>>
>>> Adding dependencies to a ClassLoaderData is currently protected by 
>>> the per-metaspace "Metaspace allocation lock"/5 (which protects more 
>>> than just allocations).
>>>
>>> Because I want to avoid messing around with the lock ordering I 
>>> suggest that we use an ObjectLocker on the _dependencies oop in 
>>> ClassLoaderData.
>>> _dependencies is a 2-element objArrayOop, in effect it's an ad-hoc 
>>> linked list of ClassLoaders/Classes which must be kept alive by this 
>>> CLD.
>>>
>>> Using an ObjectLocker on the head element of the linked list should 
>>> be at least as good as using the metaspace_lock(). There should not 
>>> be any new deadlock issues with VM mutexes since any application 
>>> thread could use a similar synchronized linked list construct.
>>>
>>> As a bonus I suggest that we factor out the dependency handling from 
>>> ClassLoaderData into a inner class: ClassLoaderData::Dependencies 
>>> and let Dependencies manage the head of the linked list by itself. 
>>> This should make it more clear that the dependency list is not 
>>> guarded by the metaspace lock anymore.
>>>
>>> Buglinks:
>>> https://jbs.oracle.com/bugs/browse/JDK-8010196
>>> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=8010196
>>>
>>> Testing:
>>> default jprt run
>>> jdk/test/java/lang/invoke/BigArityTest.java (which could reproduce 
>>> the issue with -XX:+UseG1GC -XX:G1UpdateBufferSize=1)
>>>
>>> Webrevs:
>>> Incremental:
>>> http://cr.openjdk.java.net/~mgerdin/8010196/webrev.0-fix/webrev
>>> http://cr.openjdk.java.net/~mgerdin/8010196/webrev.0-factor-out/webr
>>> ev 
>>> http://cr.openjdk.java.net/~mgerdin/8010196/webrev.0-rename/webrev
>>> Everything:
>>> http://cr.openjdk.java.net/~mgerdin/8010196/webrev.0/webrev
>>>
>>> Thanks
>>> /Mikael
>>
>>
>