Proposal: Automatic Resource Management

[Neal Gafter]

The proposed changes to existing APIs introduce an incompatibility.
This, for example, is legal today:

interface A extends java.sql.Connection, java.io.Closeable {}

However, since you propose that Connection would extend
Disposable<SqlException> and Closable would extend
Disposable<IOException>, interface A is broken after the proposed
change.  Also, the changes you propose to existing APIs introduce
changes to overload resolution that can break existing code.

You also missed a few disadvantages.

o All of the difficult decisions that you defer to the expert group,
such as the handling of exceptions, suggest that this solution is
likely to be a poor match for many uses, resulting in the sort of
shoehorning of code into the construct that results in errors.

o This approach of adding point solutions one-by-one to a language for
particular use cases results in a general degradation of the quality
of the language over time with little lifting of the level of
abstraction, so any change of this sort should be viewed with great
skepticism.

o Given the short time frame that project coin is to complete its work
and the nature of this solution -- attempting to maximize the utility
in the most common use cases while minimizing the likelihood of user
error in using the construct -- it seems unlikely that we will not
have the time to experiment with all the variables in realistic
contexts.  Therefore, one should add to the disadvantages the very
real chance that the language construct is likely to get some of these
aspects wrong if added in the JDK7 timeframe.  The fact that these
issues haven't been resolved in the 32+ months since you started
working on this proposal, or in the 13+ months since a prototype has
been available, suggests that 3-6 months is not a reasonable timeframe
to resolve them, and even hints that there may be no generally
acceptable solution.

You also forgot to mention the obvious alternatives.  Most
prominently, control abstraction via closures (BGGA or JCA) allows API
designers to tune the behavior for the various use cases (APIs)
without the need for point-solution API-specific language changes.

Generally, this proposal appears to leave too many open issues to be
suitable for inclusion in the coin project/JSR.


On Fri, Feb 27, 2009 at 10:29 PM, Joshua Bloch <jjb at google.com> wrote:
> Automatic Resource Management
>
> *AUTHOR: *Joshua Bloch
>
> *OVERVIEW*
>
> FEATURE SUMMARY: A *resource* is as an object that must be closed manually,
> such as a java.io.InputStream, OutputStream, Reader, Writer, Formatter;
> java.nio.Channel;java.net.socket; java.sql.Connection, Statement, ResultSet,
> or java.awt.Graphics.
>
> The *automatic resource management statement* is a form of the try statement
> that declares one or more resources. The scope of these resource
> declarations is limited to the statement. When the statement completes,
> whether normally or abruptly, all of its resources are closed automatically.
>
> MAJOR ADVANTAGE: The automatic resource management statement obviates the
> need for manual resource termination, which has proven ugly and error prone.
> Even good programmers get it wrong most of the time. For example, Sun’s
> guide to Persistent Connections (http://tinyurl.com/6b5jc7) gets it wrong in
> code that claims to be exemplary. Likewise, the solution on page 88 of Bloch
> and Gafter’s *Java Puzzlers* (Addison-Wesley, 2006) is badly broken, and no
> one ever noticed. In fact, 2/3 of the uses of the close method in the JDK
> itself are wrong! The price for failing to terminate resources properly is
> resource leaks or even outright failures, which may be silent (as in *Java
> Puzzlers*).
>
> Even the “correct” idioms for manual resource management are deficient: if
> an exception is thrown in the try block, and another when closing the
> resource in the finally block, the second exception supplants the first,
> making it difficult to determine the real cause of the trouble. While it is
> possible to write code to suppress the second exception in favor of the
> first, virtually no one does, as it is just too verbose. This is not a
> theoretical problem; it has greatly complicated the debugging of large
> systems.
>
> A secondary advantage of the automatic resource management construct is that
> it could emit the code to suppress the uninteresting (second) exception in
> favor of the interesting (first) one with no effort on the part of the
> programmer, and no loss to the clarity of the program.
>
> Like the for-each statement (introduced in Java 1.5), the automatic resource
> management statement is a small piece of syntactic sugar with a very high
> power-to-weight ratio.
>
> MAJOR DISADVANTAGE: Like all syntactic sugar, this construct removes a bit
> of Java’s “what you see is what you get” character (“transparency”).
>
> ALTERNATIVES: The benefits of this proposal cannot be had without a language
> change. Absent a language change, you must close resources manually. That is
> why Java’s competitors have automatic resource management constructs (C#
> has using blocks and C++ has destructors).
>
> *EXAMPLES*
>
> SIMPLE EXAMPLE: Here is a static method to read the first line of a file,
> demonstrating the minimal (nearly) correct code to release a single resource
> today.
>
>    static String readFirstLineFromFile(String path) throws IOException {
>
>        BufferedReader br = new BufferedReader(new FileReader(path));
>
>        try {
>
>            return br.readLine();
>
>        } finally {
>
>            br.close();
>
>        }
>
>    }
>
> Unfortunately, if the readLine and close invocations both throw exceptions,
> the latter exception supplants the former.  The only practical way around
> this today would to be to ignore any exception thrown by the close invocation.
> While this might be reasonable in the case of a Reader or InputStream, it
> would be disastrous for a Writer or OutputStream.
>
> Here’s how it would look with an automatic resource management statement:
>
>
>
>    static String readFirstLineFromFile2(String path) throws IOException {
>
>        try (BufferedReader br = new BufferedReader(new FileReader(path)) {
>
>           return br.readLine();
>
>        }
>
>    }
>
> ADVANCED EXAMPLE*:* Here is a static method to make a copy of a file,
> demonstrating the minimal correct code to release two resources today:
>
>    static void copy(String src, String dest) throws IOException {
>
>        InputStream in = new FileInputStream(src);
>
>        try {
>
>            OutputStream out = new FileOutputStream(dest);
>
>            try {
>
>                byte[] buf = new byte[8 * 1024];
>
>                int n;
>
>                while ((n = in.read(buf)) >= 0)
>
>                    out.write(buf, 0, n);
>
>            } finally {
>
>                out.close();
>
>            }
>
>        } finally {
>
>            in.close();
>
>        }
>
>    }
>
> Here’s how it would look with an automatic resource management statement:
>
>    static void copy(String src, String dest) throws IOException {
>
>        try (InputStream in = new FileInputStream(src);
>
>             OutputStream out = new FileOutputStream(dest)) {
>
>            byte[] buf = new byte[8192];
>
>            int n;
>
>            while ((n = in.read(buf)) >= 0)
>
>                out.write(buf, 0, n);
>
>        }
>
>    }
>
>
>
> *DETAILS*
>
> * *
>
> SPECIFICATION: What follows is not intended to be a formal JLS-quality
> specification. It emphasizes brevity and clarity over thoroughness.
>
>
>
> SYNTAX: The production for *TryStatement* in JLS §14.20 would be extended
> with this alternative:
>
>
>
> *TryStatement*:
>
>    try ( *ResourceDeclarations* ) *Block Catchesopt Finallyopt*
>
> *ResourceDeclarations*:
>
>    *LocalVariableDeclaration*
>
>    *LocalVariableDeclaration* ; *ResourceDeclarations*
>
> * *
>
> The type of each *LocalVariableDeclaration* in a *ResourceDeclarations *must
> be a subtype of Disposable<?>.  Such types are known as *resource types*.
>
>
>
> SEMANTICS and COMPILATION: An automatic resource management statement with a
> single local variable declaration and no *Finally* or *Catches* would behave
> as if replaced by the following source code:
>
>
>
> {
>
>    final *LocalVariableDeclaration* ;
>
>    try *Block* finally {
>
>        *localVar*.close();  // *localVar* is the variable declared in *
> LocalVariableDeclaration*
>
>    }
>
> }
>
>
>
> An automatic resource management statement with multiple local variable
> declarations and no *Finally* or *Catches* would behave as if (recursively)
> replaced by the following source code:
>
>
>
> {
>
>    final *LocalVariableDeclaration* ;              // First variable
> declaration
>
>    try( *ResourceDeclarations* ) *Block* finally {  // Remaining resource
> declarations
>
>        *localVar*.close();  // *localVar* is the variable declared in *
> LocalVariableDeclaration*
>
>    }
>
> }
>
>
>
> When you initially de-sugar an automatic resource management statement with
> n resource declarations for n > 1, you get an automatic resource management
> statement with n-1 resource declarations. After n such replacements, you
> have n nested try-finally statements, and the de-sugaring is complete. Note
> that resource declarations are implicitly final. For consistency with
> existing constructs with implicit modifiers, it is legal (though
> discouraged) for the programmer to provide an explicit final modifier.
>
>
>
> Note that the close method is only called on resources whose declarations
> execute without throwing an exception, and that the first such exception
> causes the statement to complete abruptly.
>
>
>
> An automatic resource management statement with a *Finally* or *Catches* would
> behave as if replaced by the following code (which contains an automatic
> resource management statement with no *Finally* or *Catches that must be
> expanded as per the desugaring above*):
>
>
>
> try {
>
>    try ( *ResourceDeclarations* ) *Block*
>
> } *Finallyopt Catchesopt*
>
>
>
> These simple semantics solve most of the problems described above, but they
> leave one problem unsolved: if the *Block* throws one exception, and the
> automatically generated closeinvocation throws another, the latter exception
> supplants the former.  This could be corrected by using a slightly more
> complex de-sugaring for the single-local-variable-declaration form of the
> construct:
>
>
>
> {
>
>    final LocalVariableDeclaration ;
>
>    boolean #suppressSecondaryException = false;
>
>    try Block catch (final Throwable #t) {
>
>        #suppressSecondaryException = true;
>
>        throw #t;
>
>    } finally {
>
>        if (#suppressSecondaryException)
>
>            try { localVar.close(); } catch(Exception #ignore) { }
>
>        else
>
>            localVar.close();
>
>    }
>
> }
>
>
>
> The variables #t, #suppressSecondaryException, and #ignore are
> compiler-generated identifiers that are distinct from one and other, and
> from any other identifiers (compiler-generated or otherwise) that are in
> scope (JLS §6.3) at the point where the automatic resource management
> statement occurs.
>
> This de-sugaring takes advantage of the ability to rethrow a final caught
> exception, which has been proposed for Java 7.  The present proposal does *
> not* depend on this ability.  In its absence, one could use a method such
> as sneakyThrow (*Java Puzzlers*, Puzzle 43).
>
>
>
> TYPE SYSTEM: The proposal has no effect on the type system.
>
>
>
> TESTING: The proposed construct can be tested by writing automatic resource
> management statements with a number of resources varying from 1 to some
> upper limit (say 10). Each resource can throw an exception (or not) during
> initialization, use, or termination. JUnit assertions are added to check
> that all resources opened are automatically closed, and that the correct
> exception (if any) is thrown.
>
>
>
> LIBRARY SUPPORT: A class must implement a designated interface to make it
> eligible for automatic resource management. An obvious choice would be
> Closeable, but unfortunately its close method is specified to throw
> IOException, which precludes its use in a general purpose resource
> management facility. It is, however, possible to retrofit Closeable with a
> parameterized superinterface:
>
>
>
> *package** java.lang;*
>
> */***
>
> * * A resource that must be closed when it is no longer needed.*
>
> * **
>
> * * @param X the type of exception thrown by the close method (or*
>
> * *     {@link RuntimeException} if the close method is not permitted*
>
> * *     to throw any checked exceptions).*
>
> * */*
>
> *public interface Disposable<X extends Throwable> {*
>
> *    void close() throws X;*
>
> *}*
>
>
>
> package java.io;
>
> public interface Closeable extends Disposable<IOException> {
>
>    void close() throws IOException;
>
> }
>
>
>
> Other existing interfaces can be similarly retrofitted, for example:
>
>
>
> package java.sql;
>
> interface Connection extends Disposable<SQLException> {
>
>    void close() throws SQLException;
>
>    ...    // (and all the other members of the Connection interface)
>
> }
>
>
>
> REFLECTIVE APIS: This proposal has no effect on core reflective APIs. The
> tree API inside javac (
> http://java.sun.com/javase/6/docs/jdk/api/javac/tree/index.html) would
> require a minor extension.
>
>
>
> OTHER CHANGES: No other parts of the platform need be to be updated.
>
>
>
> MIGRATION: Any resource that must be closed manually should be retrofitted
> to implement the Disposable interface.  In the JDK, this includes
> java.io.Closeable;java.sql.Connection, Statement, and ResultSet. New code
> using classes that implement Disposable should use automatic resource
> management statements (for clarity and correctness).  Manual resource
> management in existing code can be replaced by automatic resource management
> for increased clarity and improved behavior. Given the number of resource
> management errors observed in existing code, it may be worth the time to do
> this systematically.  It is very easy to do this with any modern IDE, which
> can search for uses of a method (in this case, Disposable.close()).
>
>
>
> *COMPATIBILITY*
>
> * *
>
> BREAKING CHANGES: All previously valid programs remain valid, and their
> semantics is unaffected.
>
> * *
>
> EXISTING PROGRAMS: Source and class files of earlier versions are unaffected
> by the feature.  No new overloadings or overridings can occur.
>
> *REFERENCES*
>
> * *
>
> EXISTING BUGS: 4888664, 4364906, 4331290, 4215007, 4120342.
>
>
>
> *ADDITIONAL FEATURES*
>
>
>
> Here are several features that might be added to the construct if the expert
> group deemed it advisable:
>
>
>
> *Retaining suppressed exceptions* - As described above, the construct simply
> discards exceptions that are suppressed. It would probably be better to
> attach them to the exception in whose favor they are being suppressed. This
> would entail adding two new methods to Throwable, void
> addSuppressedException(Throwable) and Throwable[]getSuppressedExceptions().
>
>
>
> *Ignoring certain **close failures* - One shortcoming of the construct as
> described is that it does not provide a way for the programmer to indicate
> that exceptions thrown when closing a resource should be ignored. In the
> case of the copy method, ideally the program would ignore exceptions thrown
> when closing the InputStream, but not theOutputStream.  There are several
> ways this could be achieved.
>
>
>
> *Expressions in place of declarations* - It was suggested that the automatic
> resource management statement could be defined to allow an expression in
> place of a variable declaration, permitting the use of preexisting
> variables.  This feature was consciously omitted to make it more difficult
> to access a closed resource accidentally.
>
>
>
> *DESIGN ALTERNATIVES*
>
>
>
> *Modifier in place of block* - An alterative to a block construct is a new
> modifier that could be added to any local variable declaration for a
> variable that extends Disposable.  This is more flexible and less verbose,
> but more dissimilar to existing Java language constructs.
>
>
>
> *Annotation to indicate termination method *- An alternative to the proposed
> use of the Disposable interface is an annotation on the resource termination
> method. This allows the use of a different method names (such as destroy
>  and terminate) and eases the use of the new construct with existing
> resource types.  But it is more “magical” and does not mesh as well with
> Java’s type system.
>
>