Review Request JDK-8164512: Replace ClassLoader use of finalizer with phantom reference to unload native library

[David Holmes]

On 27/09/2017 1:36 PM, mandy chung wrote:
> On 9/26/17 7:35 PM, David Holmes wrote:
>> On 27/09/2017 12:11 PM, mandy chung wrote:
>>> On 9/26/17 7:06 PM, David Holmes wrote:
>>>>
>>>>> It is not tied with the Cleaner change. Instead, the FindClass bug 
>>>>> blocks the finalizer to Cleaner change.
>>>>>
>>>>> FindClass bug is uncovered when I implemented the change from 
>>>>> finalizer to Cleaner (or phantom reference).   There is a test 
>>>>> calling FindClass to look for a class defined by the class loader 
>>>>> being unloaded, say L. L is not Gc'ed and so FindClass successfully 
>>>>> finds the class (which resurrect the class loader which was marked 
>>>>> finalizable).
>>>>>
>>>>> Is that clearer?
>>>>
>>>> So the issue is only that this test breaks?? 
>>>
>>> No.  The test reveals a bug in JNI_FindClass that uses a class loader 
>>> being finalized as the context when NativeLibrary is the caller.
>>>> And you want to change the FindClass spec to make it clear the test 
>>>> is what needs to be changed?
>>> No.   It is a bug in the hotspot implementation.   The JNI spec says 
>>> that the context of JNI_OnUnload being called is unknown. The hotspot 
>>> implementation of FindClass uses the class loader associated with 
>>> that native library as the context when invoked from JNI_OnUnload 
>>> which is wrong.
>>
>> I'm not sure I agree it is wrong. As I've said elsewhere there's a 
>> good chance that if you are trying to load classes via FindClass as 
>> part of a unload hook (which implies you are using custom 
>> classloaders), then it may be only the current loader or a parent 
>> (still custom) can load that class. But we're on the fringe of 
>> realistic expectations here as the context is specified as being 
>> "unknown".
>>
> For a native unload hook to access some class defined by this class 
> loader, definitely it should not write to any fields since the class and 
> class loader are not strongly reachable.   Reading the current state 
> stored in the class can be done by writing to the native fields.

Yes that is a good point - but as the spec says due to the unknown 
context the hook has to be very careful about what it tries to do. I 
agree it is doubtful that anyone can, or should, be relying on the 
direct use of the classloader that has become unreachable, but ...

> I'd like to know what other use cases that FindClass must ressurrect a 
> class defined by this class loader or find a class defined by its 
> ancestor if you have any in mind that the existing code can't be 
> replaced due to the proposed change.

... I can easily imagine a subsystem that runs under a custom loader and 
which then instantiates further execution contexts (per connection for 
example) each with their own classloader and which can be reclaimed 
after the request is complete. I can then easily imagine that they use 
an unload hook to record statistics about native library use, and that 
the statistics classes are in the top-level custom loader, and not 
locatable from the system loader.

While the spec makes no guarantees this will work it only says 
programmers "should be conservative in their use of VM services" which 
strongly suggests to me a "try it and see if it works" approach. In the 
current code while loading from the loader being reclaimed is highly 
dubious, delegating through that loader seems fairly reasonable to me.

>> That said given the spec says "unknown" the behaviour of the VM could 
>> change and still be in spec.
>>
>> I presume that when using a cleaner the current classloader that would 
>> be used by FindClass is the system loader? Hence the observed 
>> behaviour of FindClass "changes" if you switch to the cleaner from the 
>> finalizer - and can't be reverted to the old behaviour by using a 
>> command-line flag. Hence if we want to be able to revert we have to do 
>> that in a FindClass-only change first. Then drop-in the cleaner update 
>> and remove the flag.
>>
>>> I will file a separate JBS issue to separate this JNI bug.
>>
>> Okay. I see this as a RFE not a bug per-se: change from "unknown 
>> context" to a specific well known context.
> This case is arguable whether it's considered as a RFE or a bug because 
> the current spec of JNI_OnUnload and JNI_FindClass are not aligned.  I 
> lean toward a bug.    The bottom line:  do you agree with this proposed 
> JNI spec change?

I don't think the spec _has_ to change because I disagree that there is 
a misalignment between JNI_OnUnload and JNI_FindClass. FindClass clearly 
states it uses the current loader or else the system loader if there is 
no notion of a current loader. OnUnload says it runs in an unknown 
context, so you don't know what the current loader may be, or even if 
there is one. But regardless a call to FindClass from OnUnload should 
use the current loader if it exists, or else the system loader. The fact 
it may be dubious to use the current loader when it is itself in the 
process of being unloaded does not impinge on the voracity of the spec 
in my opinion.

So you can change to using a Cleaner instead of a finalizer and while it 
will behave differently, that change in behaviour does not violate the 
spec in any way - again in my opinion.

Now if you want to pave the way for a future switch to Cleaner by 
changing the spec for JNI_OnUnload such that it must be executed in a 
context where (equivalently) there either is no current loader or the 
current loader is the system loader, then I do not oppose that. But the 
only purpose that serves is to allow a migration path to the new 
behaviour - and then forever locks us in.

Note however I would not want to see the implementation of FindClass 
having to special case this - I would hope it just happens naturally if 
the Cleaner thread reports the current class loader as the system 
loader. Does it?

Thanks,
David

> 
> Mandy