RFR (s): JDK-8013129 Possible deadlock with Metaspace locks due to mixed usage of safepoint aware and non-safepoint aware locking

[Coleen Phillimore]

On 4/26/2013 8:08 AM, Mikael Gerdin wrote:
>> Now an interesting thing with these changes is that one only applies to
>> the VMThread - so what does presence or absence of a safepoint check
>> mean for the VMThread itself? There is only one difference in that case:
>> locking with a safepoint check allows "lock sneaking". But I don't think
>> I need to expand on that here - see mutex.cpp.
>
> The separate VMThread and non-vmthread code paths in this change are 
> unnecessary and I thought Coleen had a change which was going to fix 
> that but I may have been mistaken.

Yes, unifying those code path is in the fix for 8011610: NPG: Class 
metaspace limits number of classes for UseCompressedKlassPointers which 
I'm still working on.
>
>>
>> But this does raise a follow on concern: when the thread is not the
>> VMThread, can it hit a safepoint or otherwise block in the region whilst
>> holding the Metaspace lock? If the answer is yes then we still have a
>> potential safepoint deadlock because the VMThread will not be able to
>> acquire the lock during the safepoint VM_operation and will block.
>
> Currently we don't block in the allocation path and if we need to 
> trigger a GC I believe we release the locks before starting a 
> VM_operation.

The reason we can't check for safepoint in these locks is because GC 
tries to allocate while in the safepoint so takes out these locks 
again.   Which is a different can of worms.

>
>>
>> Aside: this also makes me wonder about the safety of other
>> lock-without-safepoint-check usages.
>
> Indeed, if there is a deadlock possiblility when mixing 
> lock-without-safepoint-check and "normal" lock then we should consider 
> enforcing that a lock is used only for one of the two paths, perhaps 
> using the C++ compiler / type system.

Yes, that would be ideal.

Coleen

>
> /Mikael
>
>>