[jmm-dev] Non-core-model issues status

[Hans Boehm]

I can see arguments for putting the annotation on

1) A field that is logically invalidated by finalization, implying that
reachability of the enclosing object needs to be preserved as long as it is
logically reachable.

2) The class that contains such a field.

I think (1) has the advantage that we can restrict the more careful
treatment to methods accessing that field rather than all methods.  (2) may
be slightly simpler to use.

As Paul suggests, we could also (3) annotate the methods accessing such a
field.  That seems less convenient and more error-prone to me.  In
particular, for something like Android BigInteger, we'd end up annotating
most of the methods, rather than either a single field or a single class.
It does seem better to me than explicit reachabilityFence(), which would
require dozens of much more subtle (and largely untestable) annotations.

Hans

On Thu, Oct 9, 2014 at 8:02 AM, Paul Benedict <pbenedict at apache.org> wrote:

> I think the annotation belongs on a method and it forces the JVM (or
> compiler) to do the reachability fence boilerplate code automatically -- or
> whatever strategy becomes agreed upon. I don't think it's a "good" solution
> to throw the weight of the responsibility to the developer to write more
> boilerplate code.
>
>
> Cheers,
> Paul
>
> On Tue, Oct 7, 2014 at 5:23 PM, Hans Boehm <boehm at acm.org> wrote:
>
>> On Sun, Oct 5, 2014 at 7:24 AM, Doug Lea <dl at cs.oswego.edu> wrote:
>> >
>> > On 10/03/2014 08:20 PM, Hans Boehm wrote:
>> >>
>> >> Let me try restating the proposed finalization rules and moving the
>> annotation
>> >> from a field to the containing class.  As far as I can tell, the
>> annotation
>> >> we've been discussing really only impacts the treatment of the
>> containing class,
>> >> so that seems to be a more logical place for it.
>> >
>> >
>> > Thanks. Seeing this fleshed out leads me again to prefer a simpler
>> > (for us) option: Defining reachabilityFence() and introducing but not
>> > requiring support for your annotation @ReachabilitySensitive (modulo
>> > names for these).  Backing up to explain why...
>> >
>> > Finalization reveals a mismatch between JVM-level lifetimes and
>> > source-code-level block scopes. Most programmers implicity expect that
>> > reachability continues to the end of enclosing block. But the only
>> > Java syntactic constructs guaranteed to preserve this entail locking.
>> > Probably the best default advice is to always use synchronized
>> > blocks/methods (or other j.u.c locks in try-finally constructions)
>> > when accessing finalizable resources (as well as in the finalize
>> > methods themselves). But when people can't or don't want to use
>> > locking (not even the hacky and possibly slow but effective trailing
>> > "synchronized(x){}") they should be able to use try {...}  finally
>> > {reachabilityFence(x);} (which acts as an RCU-ish read-lock with
>> > respect to GC). Or, in some cases (e.g., ThreadPoolExecutor), do
>> > nothing, because nothing bad can happen anyway.
>> Note that the locking that's naturally required in finalizers protects the
>> underlying resource pool to which the resources being cleaned up will be
>> returned.  I don't know of many cases in which that naturally suffices to
>> handle the premature finalization issues.  The locking we're talking about
>> is almost always just there to prevent premature finalization.
>>
>> I'm not convinced that the version of ThreadPoolExecutor.java I found is
>> officially correct as written, though I can believe that it doesn't fail
>> in
>> practice.  Since it doesn't lock the ThreadPoolExecutor itself, and
>> instead
>> uses a mainLock field, it seems to me that the semantics allows a
>> ThreadPoolExecutor to be shut down by the finalizer while e.g. an
>> execute()
>> call is in progress but the call hasn't done anything yet except cache the
>> fields of the ThreadPoolExecutor object, so that it doesn't need the
>> object
>> itself anymore.  I really do think that nearly all non-debug uses of
>> finalizers are affected.
>>
>> >
>> > Automated placement of any of these forms of lifetime control must be
>> > done at source level, because scope information is not guaranteed to
>> > be preserved in bytecode.  (Although live-range debugging information
>> > in class files might suffice in many cases.)  Different languages
>> > running on JVMs might have different scoping constructs and rules.  To
>> > nail them down, languages would need to add C++-destructor-like rules
>> > as Hans sketched out.  But even if they do, in Java and probably all
>> > JVM-hosted languages, a front-end compiler's ability to automate
>> > placement of reachability fences is limited by static type
>> > information.  For example, a reference of nominal type Object might be
>> > a FileHandle, and even a call to an Object method like hashCode()
>> > might require reachabililty protection.  Insisting on the elimination
>> > of all possible lifetime mismatches requires compilers to always
>> > insert reachability fences unless provably not necessary, either
>> > because the object is known to be of a type without a finalizer, or
>> > one of the above manual techniques already occurs in the source code.
>> > Even though it is rarely necessary to insert reachability fences, most
>> > proofs require analyses that front-end compilers do not perform.
>>
>> I think you're actually making things look a bit worse than they are.
>>  (Which is nontrivial in this area!)  If an object p has a field f that's
>> cleaned up by a finalizer, then any function that cares about premature
>> finalization must at some point have a reference to p with static type S
>> (e.g. FileHandle) that includes the field f.  Otherwise it couldn't access
>> f, and wouldn't care if it were cleaned up.  If it accesses f via a call
>> to
>> another method m2, then m2 must have the reference to p with the right
>> static type.  Depending on how we deal with expression temporaries, this
>> argument may or may not be 100% solid, but I think it covers all common
>> use
>> cases.  I do think that looking at static types can buy as something here.
>> The problem is that just looking at the presence of a finalize() method
>> doesn't suffice; with Java's variety of finalization-like mechanisms, we
>> need an annotation.
>>
>> This is very similar to proposed solution 2 in
>> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2261.html .
>>
>> >
>> > So, guaranteeing lack of scope-lifetime surprise doesn't look like a
>> > promising option. Instead we can require something like Hans sketched
>> > out that is good but imperfect, or fall back to introducing annotation
>> > @ReachabilitySensitive (implicitly applied to classes with non-default
>> > finalize()), but not requiring any compile-time or run-time
>> > properties. And then encouraging tool/IDE developers to support it to
>> > help programmers avoid bugs.  These might include points-to analyses
>> > to minimize impact, warnings about upcasts from
>> > @ReachabilitySensitive, suggested alternatives to using finalize(),
>> > and so on. If people using finalization came to rely on such tools,
>> > I'd expect the tools would become better at this than we could ever
>> > mandate as part of language/VM spec.
>>
>> A down side of not requiring it in the language is that the code produced
>> by the tools would look rather ugly.  For example, the Android
>> Biginteger.multiply() method current looks unsurprisingly like
>>
>>     public BigInteger multiply(BigInteger value) {
>>         return new BigInteger(BigInt.product(getBigInt(),
>> value.getBigInt()));
>>     }
>>
>> I think it becomes (using "reachabilityFence" as the name, although
>> Ihttp://
>> www.tutorialspoint.com/java/util/ personally don't like it):
>>
>>     public BigInteger multiply(BigInteger value) {
>>         try {
>>             BigInteger result = new BigInteger(BigInt.product(getBigInt(),
>> value.getBigInt()));
>>         finally {
>>             reachabilityFence(this);
>>             reachabilityFence(value);
>>         }
>>         return result;
>>     }
>>
>> and similarly for many of the other functions in the file.
>>
>> And I'm not sure this is actually a complete answer.  Do we allow
>> executions in which the result is finalized before the product() call
>> completes, if the result isn't used?  I thought we didn't, but I don't see
>> the words to that effect.  Do I also need a reachabilityFence(result)?
>>
>> >
>> > As always, I realize that finding a middle ground between deprecating
>> > and "fixing" finalize() can be a tough sell to both sides.  Other
>> > ideas welcome.
>>
>> Deprecating it seems like a non-starter to me.  I'm OK with deprecating
>> finalize(), but that only makes this problem worse.  And I haven't heard
>> arguments for deprecating java.lang.ref, which would really make things
>> easier for us (though probably harder for users).
>>
>> Hans
>>
>> >
>> > -Doug
>> >
>> >>
>> >> 1. A class is reachability-sensitive if it has a non-default finalizer
>> or is
>> >> suitably annotated (e.g. because its instances are cleaned up by a
>> finalizer
>> >> guardian or through a reference queue).
>> >>
>> >> 2. A reference variable is reachability-sensitive if its static type
>> is a
>> >> reachability-sensitive class (Q1: or array of such?).
>> >>
>> >> 3. The end of the scope of a reachability-sensitive variable
>> synchronizes with
>> >> the invocation of the finalize() method on the object to which it last
>> >> referred.  For this purpose, the "this" reference is treated as an
>> implicit
>> >> parameter to member functions.  (Q2: The treatment of expression
>> temporary
>> >> reachability-sensitive references is unclear.  Do we treat them as
>> though they
>> >> had a scope that lasts through the end of the full expression, as for
>> C++
>> >> destructors?)
>> >>
>> >> I think this is implementable at modest performance cost, though
>> non-trivial
>> >> compiler engineering cost.  To enforce (3) the compiler mist either
>> treat the
>> >> end of such a scope as a release operation (though without the actual
>> fence) or
>> >> refrain from moving operations across the next GC safe-point.  The GC
>> should
>> >> then guarantee sufficient synchronization.  Hard real-time GCs may have
>> other
>> >> issues, but I would be surprised if this became expensive to enforce.
>> >>
>> >> We don't enforce a corresponding property for references whose static
>> type is
>> >> not reachability-sensitive while their dynamic type is.  I think that's
>> >> generally OK, since the fields being cleaned up by a finalizer can't be
>> accessed
>> >> in such a context without entering another one in which the static
>> >> reachability-sensitive type is visible.  There's usually a similar
>> argument for
>> >> indirect references to reachability-sensitive objects.
>> >>
>> >> It seems to me that this would implicitly make most code that either
>> naively
>> >> used finalizers (instead of finalizer guardians or java.lang.ref) or
>> correctly
>> >> annotated finalizable classes correct.
>> >>
>> >> Q3: Where does this leave the current conditions in
>> >>
>> http://docs.oracle.com/javase/specs/jls/se8/html/jls-12.html#jls-12.6.2
>> ?  Can
>> >> we just drop it?
>> >>
>> >> Hans
>>
>
>