JNI WeakGlobalRefs

[Hans Boehm]

On Sat, Jul 24, 2021 at 5:23 AM Kim Barrett <kim.barrett at oracle.com> wrote:

> > On Jul 21, 2021, at 9:25 PM, Hans Boehm <hboehm at google.com> wrote:
> >
> > I'm concerned that the current semantics of JNI WeakGlobalRefs are still
> > dangerous in a very subtle way that is hidden in the spec. The current
> > (14+) spec says:
>
> I'm responding to various parts of your different emails here, and not
> being
> linear in the discussion. Hopefully I'm not being too confusing.
>
> > (I found some discussion from JDK-8220617, including a message I posted.
> > Unfortunately, it seems to me that all of us overlooked this issue?)
>
> I don't think the problems arising from WeakGlobalRef's being able to
> access
> post-WeakReference clearing or post-finalization were overlooked. What you
> are saying is "well known", and there are multiple previous discussions.
> The
> most recent I can find is here:
>
> https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-November/043919.html
>
> and in particular, here:
>
> https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-November/043924.html
>
> > Based on the current spec, you can get a similar effect to a finalizer by
> > enqueuing a WeakReference. In reality, it might be the case that in the
> > absence of finalizers, GlobalWeakRefs are cleared when WeakReferences are
> > enqueued, But I don't think the spec says that. The implementation is
> still
> > allowed to enqueue WeakReferences, leave the object around for a while,
> and
> > clear PhantomReferences and GlobalWeakRefs later, leaving a window during
> > which WeakReferences have been enqueued and processed, but GlobalWeakRefs
> > will continue to generate strong references to the object.
>
> It's true that the spec permits the GC to use different "certain point in
> time"s for deciding whether an object has reached the various strengths.
> But
> to my knowledge (and Gil Tene's as well:
>
> https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2016-January/016298.html
> and I would trust him more than me on this question), no implementation
> actually does that, other than as required to support finalization. There's
> no obvious benefit, and doing so likely has undesirable costs
> (implementation
> complexity, runtime speed/space, pause durations, &etc.).
>
Thanks for the useful pointers!

To me, the problem is not really resurrection, but unintentional
resurrection
of no-longer-valid objects. But in a Java context, I agree that's a
meaningless
distinction.


> Assuming that, your initial problem description isn't possible:
>
> > Consider what happens if I have a WeakGlobalRef W that refers to a Java
> > object A which, possibly indirectly, relies on an object F, where F is
> > finalizable, i.e.
> >
> > W - - -> A -----> ... -----> F
> >
> > Assume that F becomes invalid once it is finalized, e.g. because the
> > finalizer deallocates a native object that F relies on. This seems to be
> a
> > very common case. We are then exposed to the following scenario:
> >
> > 0) At some point, there are no longer any other references to A or F.
> > 1) F is enqueued for finalization.
> > 2) W is dereferenced by Thread 1, yielding a strong reference to A and
> > transitively to F.
> > 3) F is finalized.
> > 4) Thread 1 uses A and F, accessing F, which is no longer valid.
> > 5) Crash, or possibly memory corruption followed by a later crash
> elsewhere.
>
> F is only going to become finalizable at the same time that W is cleared.
> So
> Step 2 can't happen.
>
Interesting point. This perhaps shouldn't actually happen unless A and F
are strongly connected, i.e. A is also reachable from F. (Even if it's not,
the
implementation needs to preclude that by not clearing W until it has traced
objects
reachable from F. But I think that's at most a performance issue.)

But I'm not sure how one would specify that, or how much it actually helps.
At a minimum, it still has to break if A is a client supplied object
which is itself finalizable, e.g. because the object's type is a finalizable
subclass of what the implementer of W expected.


>
> There are other ways the strength of WeakGlobalRefs relative to
> WeakReference and finalization can get one in trouble though.
>
> I think the description of the reference strengths and the associated
> behaviors ought to be improved. But again, finalization makes that
> complicated. My impression is that there's a reluctance to make even
> clarifications in this area that have to deal with the complexity induced
> by
> finalization, when removing finalization would simplify so much. (And yeah,
> that's despite the slow rate of progress on removing finalization.)
>
> The deficiencies of resurrection-based cleanup, and the comparative
> superiority of proxy-based cleanup, have been known for a long time. (I
> have
> archived discussions on the topic from the late 1980s and early 1990s, some
> including you as a participant.) Finalization is long past its expiration
> date. It seems to me that rather than spending time discussing how to
> continue to work around its problems, it would be more productive to work
> toward removal. Some progress has been made. There's a punch-list for
> removing finalizers in the jdk here:
> https://bugs.openjdk.java.net/browse/JDK-8253568
> There is also work being done on tooling to help with the process.  Things
> like checking for usage of finalization, enabling or disabling at various
> granularities, measuring and reporting its costs, and so on.
>
>From my perspective, this looks like a great direction, but we still have
a long way to go here. To really solve the problem, and converge on a simple
model, it looks to me like we need to:

1) Remove all uses of finalization from the standard library. (In progress.)
2) Remove all uses of finalization from other Java libraries and
application code.
3) Revise the spec to require simultaneous clearing of all kinds of
Soft/Weak/Phantom References.

IMO, (2) will take a really long time, even with tooling. (AFAICT, one
technical
obstacle is that the current Cleaner spec provides new easy way to share
Cleaners
across separately developed libraries. risking thread proliferation, and
thus not
making Cleaners a consistent win over finalizers today. But the main reason
is that
finalizers are fairly widely used, and it will take time to touch all that
code. I don't
see a way to mechanize that transformation.)

If I had to speculate on a date for (3), I would guess somewhere in the
2030-2050
range. Is there an argument that I should be more optimistic?

In the meantime, it still seems to me that WeakGlobalRefs are more likely
to be
used incorrectly than correctly, though you've convinced me that actual
unpredictable failures are a bit less likely than I had thought.

Hans