Feedback and comments on ARM proposal - resend

[Joseph D. Darcy]

There was a problem with this message getting through to the list 
earlier today; resending.

-Joe

-=-=-=-

Hello.

Josh, thanks for sending in a proposal which has garnered so much 
discussion!

To begin, a handful of typos: under syntax, "Disposable<?>" should now 
be "Disposable", in semantics and compilation, the desugaring of an ARM 
block with a Finally or Catches portion should list them not as 
"Finally_opt Catches_opt" but as "Catches_opt Finally_opt"; for 
migration, the SQL classes are in package javax.sql rather than java.sql 
and the relevant type names are bit different than the listed ones.  
Onto more substantive matters.

I have some reservations about the ARM proposal.

A few background thoughts, the design of Java assumes garbage collection 
is available as the primary resource reclamation mechanism.  When your 
resource reclamation can map into garbage collection, things work fairly 
well and naturally benefit from the many ongoing engineering 
improvements in that area.  However, that leaves resources with a block 
structured life cyle pattern (open files, etc.) without an elegant 
solution in Java today.  Finalizers have serious limitations and while 
nested try-catch-finally blocks are possible, they are verbose and 
tricky to get right, as detailed in the proposal.

There are a few possible solutions to managing the block structured 
resources:

Library-based solutions:

* Weak references:  Reference queues to process less-than-strong 
references can be used to call a method after an object is no longer 
reachable.  However, this approach shares some of the problems of 
finalizers, including no guarantees of prompt execution and exceptions 
occurring in separate thread.

* Closures: If something like BGGA closures were in the platform, the 
tricky cleanup code could be implemented as a library wrapping around 
the application logic.  Better still, variants of the library could be 
implemented tailored to the needs of different kinds of resources.  
Closures will certainly *not* be in JDK 7, so a closure-based solution 
is not an option for this release.

Language-based solutions:

* Destructors as in C++: Destructors in C++ rely on objects having 
different "storage classes," that is living on the stack versus the 
heap.  Java has a uniform allocation model and adding storage classes 
would be impractical and undesirable.

* New statement form: As discussed in this proposal, given a good 
desugaring, the tricky cleanup code could be generated by the compiler.  
That change doesn't fundamentally depend on any features not currently 
in the platform, but there are limitations in how well the desugaring 
fits all the use-cases of interest.

While managing block structure resources surely is a problem, a language 
solution is at or near the limit of the size of a coin.

All else being equal and given the effort that would be involved, I'd 
prefer a change that generally supported block-structured resources 
rather than "just" the IO related "Closeable" ones.  I think doing this 
would include handling "close" methods that are not declared or expected 
to throw exceptions.

( If any distinct handling is done for close methods that are not 
expected to throw exceptions, the test would probably need to be done on 
the dynamic type of the Throwable so that the Liskov substitution 
principle was followed.  Assuming a degenerate exception could indicate 
it did and did not have a close method that threw exceptions, the 
runtime treatment of such an exception should not depend on the static 
type of the variable referring to the exception.

For example, perhaps reasonable semantics are for a Disposable resource 
to have a secondary exception during close added to the original 
exception's suppressed exception list while if "DisposableQuiet" 
resource gets an exception during close, the *new* exception should be 
propagated out with the old exception being suppressed?)


Other comments interspersed.

>
>   Automatic Resource Management
>
> *AUTHOR: *Joshua Bloch
>
> *OVERVIEW*
>
> FEATURE SUMMARY: A /resource/ is as an object that must be closed 
> manually, such as 
> a java.io.InputStream, OutputStream, Reader, Writer, Formatter; java.nio.Channel; java.net.socket; java.sql.Connection, Statement, ResultSet, 
> or java.awt.Graphics.
>
>

[snip]

> Like the for-each statement (introduced in Java 1.5), the automatic 
> resource management statement is a small piece of syntactic sugar with 
> a very high power-to-weight ratio.
>

The for-each statement in JDK 5 was a clear win.  While there were 
details of the design to be debated (what interface should be accepted, 
etc.) the basic semantics were uncontested: starting with the first 
element, one by one visit the next element of the structure until 
reaching the end.  As has been discussed on the list, the desired 
semantics of the ARM blocks are less clear, such as how any secondary 
exception should be handled.  I would expect the same behavior would not 
be appropriate in all cases. Also as noted on the list and in the 
proposal itself, there is a big difference in getting an exception on 
closing an input stream versus an output stream.

> MAJOR DISADVANTAGE: Like all syntactic sugar, this construct removes a 
> bit of Java's "what you see is what you get" character ("transparency").
>

Another disadvantage is further coupling between the language 
specification and libraries.


[snip]

>  
>
> *DETAILS*
>
> * *
>
> SPECIFICATION: What follows is not intended to be a formal JLS-quality 
> specification. It emphasizes brevity and clarity over thoroughness.
>

Adding in some analysis of the specification areas of concerns listed in 
http://blogs.sun.com/darcy/entry/so_you_want_to_change

> * How does the grammar need to be revised?
Understood.

> * How is the type system affected?

The handling of resource-ness could be considered a minor type system 
change.

> * Are any new conversions defined?

No.

> * Are naming conventions or name visibility modified?
Shouldn't be.

> * Is the existing structure of packages, classes, or interfaces changed?

No.

> * How can the new feature be annotated?

Yes; the resource declarations can be annotated.

> * Is method resolution impacted?
No; although if multiple clean-up methods are supported, "close", 
"flush", there will need to be rules about which ones are called if more 
than one is present.

At present, the desugaring appears to call MyResource.close rather than 
Disposable.close, which seems to be the right thing to do.

> * How does the change impact source compatibility?

Removing resource-ness will affect code that uses that type as a 
resource, but removing the implementation of an interface is already 
source-incompatible.

> * How does the change impact binary compatibility?
Depending on how the desugaring is structured, removing the 
resource-ness of a class, but leaving the close method, may or may not 
impact pre-existing class files which use the class as a resource.  
(Removing the implementation of an interface is already binary 
incompatible.)

> * Does the feature affect the reachability of code or the definite 
> assignment of variables?

Yes; while the desguraring should cover this, future drafts should 
verify there are no complications.


[snip]

>  
>
> An automatic resource management statement with multiple local 
> variable declarations and no /Finally/ or /Catches/ would behave as if 
> (recursively) replaced by the following source code:
>
>  
>
> {
>
>     final /LocalVariableDeclaration/ ;              // First variable 
> declaration
>
>     try( /ResourceDeclarations/ ) /Block/ finally {  // Remaining 
> resource declarations
>
>         /localVar/.close();  // /localVar/ is the variable declared 
> in /LocalVariableDeclaration/
>
>     }
>
> }
>

Hmm, if the resource declarations wanted to misbehave, they could 
prematurely call close on earlier declarations; something like:

try(Resource1 resource1 = new Resource1();
     Resource2 = closeAndAllocate(resource1) ) {...}

static Resouce2 closeAndAllocate(Closable closeable) {
    try {
       closeable.close();
    catch(Exception exception) {
        ; // bye-bye
    }
    return new Resource2();
}

That should still mostly be okay since the second call to close is a 
no-op by the Closeable specification, but suppressing the exception from 
close could be thwarted.

What other naughty things could be done to or with close methods?

[snip]
 
>
> These simple semantics solve most of the problems described above, but 
> they leave one problem unsolved: if the /Block/ throws one exception, 
> and the automatically generated close invocation throws another, the 
> latter exception supplants the former.  This could be corrected by 
> using a slightly more complex de-sugaring for the 
> single-local-variable-declaration form of the construct:
>
>  
>
> {
>
>     final LocalVariableDeclaration ;
>
>     boolean #suppressSecondaryException = false;
>
>     try Block catch (final Throwable #t) {
>
>         #suppressSecondaryException = true;
>
>         throw #t;
>
>     } finally {
>
>         if (#suppressSecondaryException)
>
>             try { localVar.close(); } catch(Exception #ignore) { }
>
>         else
>
>             localVar.close();
>
>     }
>
> }
>

In this more involved desugaring that suppresses a secondary exception, 
a slight modification may be needed to call any new methods to add the 
suppressed exceptions to the primary one.

Perhaps the state variable could be eliminated by a lower-level 
desugaring including gotos, but that is a minor implementation detail.

>  
>
> The variables #t, #suppressSecondaryException, and #ignore are 
> compiler-generated identifiers that are distinct from one and other, 
> and from any other identifiers (compiler-generated or otherwise) that 
> are in scope (JLS §6.3) at the point where the automatic resource 
> management statement occurs.
>
>
> This de-sugaring takes advantage of the ability to rethrow a final 
> caught exception, which has been proposed for Java 7.  The present 
> proposal does /not/ depend on this ability.  In its absence, one could 
> use a method such as sneakyThrow (/Java Puzzlers/, Puzzle 43).
>

Care would need to be taken to generate an appropriate line number table 
for debugging purposes.

[snip]

>  
>
> *Retaining suppressed exceptions* - As described above, the construct 
> simply discards exceptions that are suppressed. It would probably be 
> better to attach them to the exception in whose favor they are being 
> suppressed. This would entail adding two new methods 
> to Throwable, void addSuppressedException(Throwable) and Throwable[] getSuppressedExceptions().
>

Having more mutable state in a Throwable is a bit disconcerting, but I 
suppose necessary.  An external reference to the original exception 
could be stashed away somewhere so creating and propagating an augmented 
copy of the exception with the additional suppressing information 
wouldn't be correct in general.
>
>  
>
> *Ignoring certain **close failures* - One shortcoming of the construct 
> as described is that it does not provide a way for the programmer to 
> indicate that exceptions thrown when closing a resource should be 
> ignored. In the case of the copy method, ideally the program would 
> ignore exceptions thrown when closing the InputStream, but not 
> the OutputStream.  There are several ways this could be achieved.
>

What did you have in mind here?  Different disposable interfaces?  
Annotations on the resource type?

[snip]

Thinking a bit speculatively, if multi-catch and final rethrow are 
added, would that have any weird interactions if used with ARM blocks?

In related matters, I'm working on a writeup on the "finally" variation, 
an alternative I don't favor, and will send it out in the next day or two.

-Joe