RFR: 8234096: Mutex rank ordering problem JVMTI LoadedClassesClosure and G1 SATB queue

[Roman Kennke]

A user reported a mutex rank ordering problem between JVMTI 
LoadedClassesClosure and G1's SATB queues. As far as I can tell, this is 
caused by JDK-8187577:

ClassLoaderData::loaded_classes_do() acquires the "Metaspace allocation 
lock/5" and LoadedClassesClosure tries to enqueue it into G1's SATB 
queue, thus attempts to acquire "SATB_Q_CBL_mon/16"

See bug report for more details and discussion.
https://bugs.openjdk.java.net/browse/JDK-8234096

The bug is 8u-specific. I forward-ported the testcase to 11 and 14 (will 
post that under different bug-ID asap), and could not reproduce it 
there. I also analyzed the code path, and confirmed that the bug is 
really not present there.

The fix that I want to propose is simple: instead of enqueueing the oops 
during loaded_classes_do() (and thus under lock), we can enqueue it 
during extraction of same class-mirrors into the target array. This is 
ok because there is no safepoint between the two phases.

Webrev:
http://cr.openjdk.java.net/~rkennke/JDK-8234096/webrev.02/

Testing: The new test fails without the fix, and passes with it. 
hotspot_all doesn't show regressions

Can I please get a review for this change?

Thanks, Roman