RFR: 8231584: Deadlock with ClassLoader.findLibrary and System.loadLibrary call

[Anton Kozlov]

On 30.09.2019 21:18, Mandy Chung wrote:
> Skimming through the implementation, it seems to me that the Runtime::loadLibrary0 does not need to be synchronized. 
> ClassLoader::loadLibrary0 should ensure that a native library of a given name is loaded and registered only once.  

Interesting. I get confused at first by a comment nearby [1]. At first sight, it assumes that Runtime.loadLibrary0 is synchronized. But it's guarded by a monitor of loadedLibraryNames.

May be this will be simpler approach.

[1]: http://hg.openjdk.java.net/jdk/jdk/file/a7c95e2f8814/src/java.base/share/classes/java/lang/ClassLoader.java#l2471

> Can you explain why you think ClassLoader::findLibrary needs to be called with a lock?

I wanted to save findLibrary behavior. The code seems to be there for a long time. I think there are chances some user classLoader could unintentionally depend on findLibrary being called synchronously. For example, a cache for a file system would need a synchronization in custom findLibrary if the outer lock would be removed.

But the lock introduces same deadlock, when class being initialized is loaded from the loader with findLibrary overridden. See the test in updated webrev. Removing synchronization solves the issue. Syncing on loader object fixes the test but only because a classLoader lock is taken when resolve is started. I was unable to find an evidence syncing on the loader would be generally applicable.

New intermediate webrev: (while investigating)

http://cr.openjdk.java.net/~akozlov/8231584/webrev.01/

I removed the sync around findLibrary to resolve the another deadlock; test is improved.

Thanks,
Anton