JDK8: Unloading native JNI library

[David Holmes]

Hi Vemund,

What OS do you see this on? Are you able to test different OS?

Thanks,
David

On 24/08/2017 10:37 PM, Vemund Ostgaard wrote:
> Hello,
> 
> I am struggling with a problem related to unloading (and reloading) a 
> native JNI library that has been loaded with System.load(). I have been 
> referred to this mailing list as the right place to ask, so seeking your 
> advice on the issue.
> 
> My goal was to load a JNI library from a JVM process and be able to 
> reload an updated copy of the same library later without restarting the 
> JVM. The library might be loaded from the exact same path as before or 
> from a different path.
> 
> The testing I have done (more details below) indicates to me that the 
> first JNI library I load with System.load() does not get unloaded 
> completely when its loading classloader is garbage collected, subsequent 
> attempts to load a library from the same path indicates that the native 
> library kept the state it had from the first loaded copy.
> 
> Surprisingly though, any other copy of the native library that is loaded 
> from other file system locations than the first path are apparently 
> unloaded when their loading classloader is garbage collected, and 
> subsequent attempts to load from those paths indicate that those 
> libraries do not keep their state.
> 
> So, I am curious whether there is something special about the first time 
> System.load() is used from application code in a new JVM, and whether 
> there is some way I can avoid that behavior.
> 
> 
> Details about my testing:
> 
> I have added these methods to my native JNI library:
> 
> JNIEXPORT jint JNICALL JNI_OnLoad(JavaVM *jvm, void *reserved)
> {
>      counter++;
>      printf("JNI_OnLoad called: %d\n", counter);
>      (void)jvm;
>      (void)reserved;
>      return JNI_VERSION_1_4;
> }
> 
> JNIEXPORT void JNICALL JNI_OnUnload(JavaVM *jvm, void *reserved)
> {
>      printf( "JNI_OnUnload called: %d\n", counter);
>      (void)jvm;
>      (void)reserved;
>      return;
> }
> 
> The counter is a global variable in the same file, and it is initialized 
> to 0. The plan was to see if the counter was initialized to 0 again 
> after the library is unloaded and loaded again, and also to see that 
> these methods were called at all. I also recompiled the native library 
> in two other versions with the counter initialized to 100 and 200 
> respectively, to be able to tell the difference between different copies 
> of the library being loaded/unloaded.
> 
> I wrote a small java class that calls System.load() to load my native 
> JNI library, and a small custom classloader that extends URL 
> classloader. To both of these classes I added overrides for finalize() 
> just to see it being called and printing out a unique ID to know what 
> copy of each class was being finalized.
> 
> The third Java class is just a main() method that calls a test() method 
> with the test logic using the other two classes to test loading and 
> unloading of the native library. When the test() method returns to the 
> main() method, the classloader object goes out of scope and I call 
> System.gc() a few times to get everything garbage collected. The test() 
> method also returns a WeakReference() to the classloader object so I can 
> call WeakReference.get() before and after the System.gc() calls to 
> verify that this becomes null. I interpret the WeakReference becoming 
> null as a sign that my classloader is getting garbage collected.
> 
> My testing shows that in all cases:
> - JNI_OnLoad() and JNI_OnUnload() gets called as expected
> - The finalizer() methods for my two classes are called as expected
> - The WeakReference.get() for my custom classloader returns null after a 
> few System.gc() calls and sleeping a couple seconds total.
> 
> As already mentioned, the first JNI library loaded keeps its state, the 
> counter printed from JNI_OnLoad() and JNI_OnUnload() keeps increasing 
> for every repeated load+unload from that file location, while for all 
> other libraries loaded from other file locations, the state is 
> apparently reset. This holds true also if using multiple custom 
> classloaders or identical copies of the JNI library in different 
> locations. The key factor seems to only be loading from the same file 
> location as used the first time.
> 
> I can't see that my code behaves any different for the multiple times I 
> try to load and unload the JNI libraries, so I am wondering if there is 
> some JNI bug or known behavior that explains what I see?
> 
> 
> Thank you,
> Vemund