RFR(S) 8166364: fatal error: acquiring lock DirtyCardQ_CBL_mon/16 out of order with lock Module_lock/6 -- possible deadlock

[harold seigel]

Hi,

Please review this fix for JDK-8166364.

This fix moves the setting of the module fields in the class mirrors of 
the fixup_module_list outside of the Module_lock.  The determination of 
whether a mirror should be added to the fixup_module_list is still done 
under Module_lock as is the defining of module java.base.  This prevents 
any synchronization issues with a mirror being erroneously added to the 
fixup_module_list after module java.base is defined.  The other piece is 
that the VM, in Modules::define_javabase_module(), guarantees under 
Module_lock that only one thread will ever successfully define module 
java.base.

Open webrev: http://cr.openjdk.java.net/~hseigel/bug_8166364/

JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8166364

The fix was tested with the JCK Lang and vm tests, and JTreg hotspot, 
java/io, java/lang, and java/util tests using both fastdebug and 
slowdebug builds.  The nsk cololocated and the non-colocated quick tests 
were also run against a slowdebug build.

Thanks, Harold