Automatic Resource Management, V.2
Joshua Bloch
jjb at google.com
Sun Apr 19 23:45:15 PDT 2009
Fellow Coiners,
Thanks for all your feedback on the Automatic Resource Management proposal.
I have revised the proposal in light of these comments. The new version
(V.2) can be seen at this URL:
http://docs.google.com/Doc?id=ddv8ts74_3fs7483dp . It is included below for
archival purposes.
Josh
Automatic Resource Management (V.2)
*AUTHOR: *Joshua Bloch
*OVERVIEW*
FEATURE SUMMARY: A *resource* is as an object that must be closed manually,
such as a java.io.InputStream, OutputStream, Reader, Writer, Formatter;
java.nio.Channel; java.net.socket; java.sql.Connection, Statement, ResultSet,
or java.awt.Graphics. The *automatic resource management statement* is a
form of the try statement that declares one or more resources. The scope of
these resource declarations is limited to the statement. When the statement
completes, whether normally or abruptly, all of its resources are closed
automatically.
MAJOR ADVANTAGE: The automatic resource management statement obviates the
need for manual resource termination, which has proven ugly and error prone.
Even good programmers get it wrong most of the time. For example, Sun’s
guide to Persistent Connections (http://tinyurl.com/6b5jc7) gets it wrong in
code that claims to be exemplary. Likewise, the solution on page 88 of Bloch
and Gafter’s *Java Puzzlers* (Addison-Wesley, 2006) is badly broken, and no
one ever noticed. In fact, 2/3 of the uses of the close method in the JDK
itself are wrong! The price for failing to terminate resources properly is
resource leaks or even outright failures, which may be silent (as in *Java
Puzzlers*).
Even the “correct” idioms for manual resource management are deficient: if
an exception is thrown in the try block, and another when closing the
resource in the finally block, the second exception supplants the first,
making it difficult to determine the real cause of the trouble. In other
words, by responsibly closing your resource even during some catastrophic
event, you can actually erase all record of the event, resulting in many
wasted hours of debugging. While it is possible to write code to suppress
the second exception in favor of the first, virtually no one does, as it is
just too verbose. This is not a theoretical problem; it has greatly
complicated the debugging of large systems. A secondary advantage of the
automatic resource management construct is that it can suppress the
uninteresting (second) exception in favor of the interesting (first) one
with no effort on the part of the programmer, and no loss in the clarity of
the program.
Like the for-each statement (introduced in Java 1.5), the automatic resource
management statement is a small piece of syntactic sugar with a very high
power-to-weight ratio.
MAJOR DISADVANTAGE: Like all syntactic sugar, this construct removes a bit
of Java’s “what you see is what you get” character (“transparency”). Another
disadvantage is increased coupling between the language specification and
libraries.
ALTERNATIVES: The benefits of this proposal cannot be had without a language
change. Absent a language change, you must close resources manually. That is
why Java’s competitors have automatic resource management constructs (C#
has using blocks and C++ has destructors).
*NONTRIVIAL CHANGES IN THIS VERSION*
1. Renamed interface Disposable to AutoCloseable and moved it from
package java.lang to java.lang.auto.
2. Enhanced statement to allow expressions in place of declarations to
designate resources.
3. Added support for retention of suppressed exceptions.
4. Added list of types to be retrofitted.
5. Added design FAQ with additional examples.
*EXAMPLES*
SIMPLE EXAMPLE: Here is a static method to read the first line of a file,
demonstrating the minimal (nearly) correct code to release a single resource
today.
static String readFirstLineFromFile(String path) throws IOException {
BufferedReader br = new BufferedReader(new FileReader(path));
try {
return br.readLine();
} finally {
br.close();
}
}
Unfortunately, if the readLine and close invocations both throw exceptions,
the latter exception supplants the former. The only practical way around
this today would to be to ignore any exception thrown by the close invocation.
While this might be reasonable in the case of a Reader or InputStream, it
would be disastrous for a Writer or OutputStream.
Here’s how it would look with an automatic resource management statement:
static String readFirstLineFromFile2(String path) throws IOException {
try (BufferedReader br = new BufferedReader(new FileReader(path)) {
return br.readLine();
}
}
ADVANCED EXAMPLES*:* Here is a static method to make a copy of a file,
demonstrating the minimal correct code to release two resources today:
static void copy(String src, String dest) throws IOException {
InputStream in = new FileInputStream(src);
try {
OutputStream out = new FileOutputStream(dest);
try {
byte[] buf = new byte[8 * 1024];
int n;
while ((n = in.read(buf)) >= 0)
out.write(buf, 0, n);
} finally {
out.close();
}
} finally {
in.close();
}
}
Here’s how it would look with an automatic resource management statement:
static void copy(String src, String dest) throws IOException {
try (InputStream in = new FileInputStream(src);
OutputStream out = new FileOutputStream(dest)) {
byte[] buf = new byte[8192];
int n;
while ((n = in.read(buf)) >= 0)
out.write(buf, 0, n);
}
}
*DETAILS*
* *
SPECIFICATION: What follows is not intended to be a formal JLS-quality
specification. It emphasizes brevity and clarity over thoroughness.
SYNTAX: The production for *TryStatement* in JLS §14.20 would be extended
with this alternative:
*TryStatement*:
try ( *ResourceSpecifications* ) *Block Catchesopt Finallyopt*
*Resource**Specifications*:
*Resource**Specification*
*Resource**Specification *; *Resource**Specifications*
*Resource**Specification**:*
* **LocalVariableDeclaration*
* Expression*
* *
The type of each *LocalVariableDeclaration* or *Expression *in a *
ResourceDeclarations *must be a subtype of AutoCloseable. Such types are
known as *resource types*.
SEMANTICS and COMPILATION: An automatic resource management statement with a
single local variable declaration and no *Finally* or *Catches* would behave
as if replaced by the following source code:
{
final *LocalVariableDeclaration* ;
try *Block* finally {
*localVar*.close(); // *localVar* is the variable declared in *
LocalVariableDeclaration*
}
}
An automatic resource management statement with a single resource expression
and no *Finally* or *Catches* would behave as if replaced by the following
source code:
{
final *T**ype* #localVar = *Expression*;
try *Block* finally {
#*localVar*.close();
}
}
The variable #localVar is a compiler-generated identifier distinct from any
other identifiers (compiler-generated or otherwise) that are in scope (JLS
§6.3) at the point where the automatic resource management statement occurs.
The *Type *in the above desugaring is replaced by the static type of *
Expression*.
An automatic resource management statement with multiple resource
specifications (declarations or expressions) and no *Finally* or
*Catches* would
behave as if (recursively) replaced by the following source code:
{
final *LocalVariableDeclaration* ; // First variable
declaration
try( *ResourceDeclarations* ) *Block* finally { // Remaining resource
declarations
*localVar*.close(); // *localVar* is the variable declared in *
LocalVariableDeclaration*
}
}
When you initially desugar an automatic resource management statement with n
resource specifications for n > 1, you get an automatic resource management
statement with n-1 resource declarations. After n such replacements, you
have n nested try-finally statements, and the desugaring is complete. A
single declaration that declares multiple resources is treated as if it were
multiple declarations, each declaring a single resource. Note that resource
declarations are implicitly final. For consistency with existing constructs
with implicit modifiers, it is legal (though discouraged) for the programmer
to provide an explicit final modifier.
Note that the close method is only called on resources whose declarations
(or expressions) execute without throwing an exception, and that the first
such exception causes the statement to complete abruptly.
An automatic resource management statement with a *Finally* or *Catches* would
behave as if replaced by the following code (which contains an automatic
resource management statement with no *Finally* or *Catches that must be
expanded as per the desugaring above*). Note that resources are closed
before any explicit finally or catch clauses have executed, and that
resource variable declarations are not in scope in explicit finally and
catch clauses:
try {
try ( *ResourceDeclarations* ) *Block*
}* Catchesopt Finallyopt*
The above desugarings are simplified for pedagogical purposes. They solve
most of the problems described above but one problem remains: if the
*Block* throws
one exception, and the automatically generated close invocation throws
another, the latter exception supplants the former. This shortcoming is
corrected by using a slightly more complex desugaring for the
single-local-variable-declaration forms of the construct. In this
desugaring, an exception thrown during the lifetime of a resource (the *
Block*) receives priority over an exception thrown by an automatically
generated close invocation. Such a *suppressed exception* is not discarded,
but is associated with the *primary exception *(the one thrown from the *
Block*), and can be extracted using Throwable's getSuppressedExceptions
method.
The following desugaring should be regarded as definitive (though the
compiler is free to emit any object code that obeys its semantics):
{
final *LocalVariableDeclaration* ; // Or final *T**ype* #localVar = *
Expression *;
Throwable #primaryException = null;
try *Block* catch (final Throwable #t) {
#primaryException = t;
throw #t;
} finally {
if (#primaryException != null) {
try {
localVar.close();
} catch(Exception #suppressedException) {
#primaryException.addSuppressedException(#suppressedException);
}
} else {
localVar.close();
}
}
}
The variables #primaryException, #t, and #suppressedException are
compiler-generated identifiers that are distinct from one and other, and
from any other identifiers (compiler-generated or otherwise) that are in
scope (JLS §6.3) at the point where the automatic resource management
statement occurs.
This de-sugaring takes advantage of the ability to rethrow a final caught
exception, which has been proposed for Java 7. The present proposal does *
not* depend on this ability. In its absence, one could use a method such as
sneakyThrow (*Java Puzzlers*, Puzzle 43).
It is suggested that the line number table emitted by the compiler make the
automatically generated resource-closing code appear to come from the last
line of the *Block* (which will contain a close bracket), immediately prior
to any *Catches**opt* or *Finally**opt*.
TYPE SYSTEM: The proposal has no effect on the type system (other than
classifying certain types as resources).
REACHABILITY AND DEFINITE ASSIGNMENT: The reachability and definite
assignment rules for this construct are specified by the desugaring above.
It has been noted that under certain pathological conditions, the compiler
must generate an error message indicating that the construct can throw an
exception that cannot occur. These pathological conditions are unlikely to
occur in any real code. If we ever wish to address this issue, we could do
so by modifying the DA/DU rules for the language so that the following
program becomes legal:
public static void main(String[] args) {
if (false)
throw new Exception(); // Compiler error!
}
TESTING: The proposed construct can be tested by writing automatic resource
management statements with a number of resources varying from 1 to some
upper limit (say 10). Each resource can throw an exception (or not) during
initialization, use, or termination. JUnit assertions are added to check
that all resources opened are automatically closed, and that the correct
exception (if any) is thrown.
LIBRARY SUPPORT: A class must implement a designated interface to make it
eligible for automatic resource management. An obvious choice would be
Closeable, but unfortunately its close method is specified to throw
IOException, which precludes its use in a general purpose resource
management facility. It is, however, possible to retrofit Closeable with a
superinterface:
*package** java.lang.auto;*
*/***
* * A resource that must be closed when it is no longer needed.*
* */*
*public interface AutoCloseable {*
* void close() throws Exception;*
*}*
package java.io;
public interface Closeable *extends AutoCloseable* {
void close() throws IOException;
}
Retrofitting java.io.Closeable to implement AutoCloseable will also provide
automatic resource management for its 11 subinterfacs and 81 implementing
classes in the JDK. Other existing classes and interfaces can be similarly
retrofitted, for example:
package java.sql;
interface Connection *extends AutoCloseable* {
void close() throws SQLException;
... // (and all the other members of the Connection interface)
}
Other types that should be retrofitted include java.sql.Statement,
PreparedStatement, ResultSet; and java.awt.Graphics. Other possibilities
include java.nio.channels.FileLock; javax.imageio.ImageReader, ImageWriter;
javax.naming.Context; javax.xml.stream.XMLEventReader, XMLEventWriter,
XMLStreamReader, XMLStreamWriter; java.beans.XMLEncoder, and XMLDecoder. It
is best to err on the side of caution, and to only retrofit classes that we
know are appropriate for automatic resource management. More classes can be
retrofitted over time, and new types can be designed with automatic resource
management in mind. See this document for additional
information<http://docs.google.com/Doc?id=ddv8ts74_1d7tz65fd>
.
To support automatic retention of suppressed exceptions (for high-quality
diagnostics), we add these methods to java.lang.Throwable:
/**
* Adds the specified exception to the list of exceptions that were
suppressed,
* typically by the automatic resource management statement, in order to
* deliver this exception.
*/
public void addSuppressedException(Throwable suppressedException);
/**
* Returns an array containing all of the exceptions that were suppressed,
* typically by the automatic resource management statement, in order to
* deliver this exception.
*/
public Throwable[] getSuppressedExceptions().
The printStackTrace method should be modified to append the stack traces for
any suppressed exceptions.
REFLECTIVE APIS: This proposal has no effect on core reflective APIs. The
tree API inside javac (
http://java.sun.com/javase/6/docs/jdk/api/javac/tree/index.html) would
require a minor extension.
OTHER CHANGES: No other parts of the platform need be to be updated.
MIGRATION: As discussed above in *Library Support*, any lexically scoped
resource type that must be closed manually should be retrofitted to
implement the AutoCloseable interface. New code using classes that
implement AutoCloseable should use automatic resource management statements
(for clarity and correctness). Manual resource management in existing code
can be replaced by automatic resource management for increased clarity and
improved behavior. Given the number of resource management errors observed
in existing code, it may be worth the time to do this systematically. It is
very easy to do this with any modern IDE, which can search for all uses of a
method (such as java.io.Closeable.close()).
*COMPATIBILITY*
* *
BREAKING CHANGES: All previously valid programs remain valid, and their
semantics is unaffected.
* *
EXISTING PROGRAMS: Source and class files of earlier versions are unaffected
by this feature. No new overloadings or overridings can occur.
*REFERENCES*
* *
EXISTING BUGS: 4888664, 4364906, 4331290, 4215007, 4120342.
*DESIGN FAQ*
*1. Does the construct support locks?*
The automatic resource management statement was not designed for use with
locks. The synchronized block already handles Java's built-in monitor locks.
The java.util.concurrent.locks.Lock interface and its implementations were
added to handle some special cases that built-in synchronization doesn't
support, such as non-block-structured locking, and most programmers have no
need for them. (Many uses of Lock would be better replaced by a
higher-level java.util.concurrent abstraction such as Semaphore,
CountDownLatch, or CyclicBarrier.)
Also, it is quite straightforward to acquire and release these locks
manually, so even legitimate users of the Lock interface have little
incentive to use a more automated solution:
myLock.lock();
try {
// Access the resource protected by myLock
} finally {
myLock.unlock();
}
That said, it *is* possible to use the proposed automatic resource
management statement for lock management with the aid of an adapter like
this one:
interface AutoLock extends AutoCloseable {
AutoLock lock(); // Locks lock and returns this
void close(); // Unlocks lock
}
static AutoLock autoLock(final Lock lock) {
return new AutoLock() {
public AutoLock lock() { lock.lock(); return this; }
public void close() { lock.unlock(); }
};
}
With this adapter, you can use the following code to perform locked access
with automatic unlocking:
AutoLock al = autoLock(myLock);
...
try(al.lock()) {
// Access the resource protected by lck
}
*2. Does the construct allow access to resources after an exception is
thrown from within the block, but before the resource is closed?*
Not directly. The need for this functionality is rare. The workaround is to
nest a try-catch block inside the automatic resource management statement:
try (LineNumberReader reader = getInput()) {
try {
parseInput(reader);
} catch (CharacterCodingException ex) {
report("Char encoding error at line " + reader.getLineNumber());
throw ex; // Rethrow the exception
}
}
* 3. Does the construct work properly with “decorated” resources?*
There is a low probability of a resource leak when the construct is used in
this fashion:
try (BufferedReader br = new BufferedReader(new FileReader(path)) {
... // Use buffered reader
}
The problem is that there is a “window of vulnerability” in between the time
the FileReader is constructed and the time the BufferedReader is
constructed. If the BufferedReader construction fails, say, because there is
insufficient memory for the buffer, the construct will *not *close the
FileReader. This is unfortunate but generally acceptable. The window of
vulnerability is small, and it seems likely that the application is moribund
if decorator construction fails.
In the unlikely event that this level of risk is unacceptable for your
application, you can eliminate the window of vulnerability by treating the
object and its decorator as separate resources:
try (FileReader fr = new FileReader(path) {
try (BufferedReader br = new BufferedReader(fr) {
... // Use buffered reader
}
}
* 4. Does the construct support resource types whose termination method has
a name other than **close**?*
Not directly. There are two obvious ways in which the construct could have
supported this functionality. One is to designate the termination method
with an annotation. But this violates the rule that annotations cannot
change the semantics of a program (JLS §9.7). The other is to use an actual
modifier to denote the termination method (the finally keyword has been
proposed for this purpose). This would, however, require a class file format
change, and a mandate changes to tools such as JavaDoc. The resulting
construct would be more “magical,” and wouldn't mesh as well with Java’s
type system.
The proposed construct is an “80-20” solution. It is aimed squarely at
solving the serious problem of Closeables. Most resources can be retrofitted
to implement AutoCloseable, but a few can't (e.g., interface types that do
not already contain a parameterless close method and class types that
contain a parameterless close method with the wrong return type or
semantics). For such resources, we recommend the use of an adapter, such as
AutoLock above. The exact form of the adapter will differ depending on the
details of the resource.
*5. Some failures of the **close** method can be safely ignored (e.g.,
closing a file that was open for read). Does the construct provide for this?
*
No. While this functionality seems attractive, it is not clear that it's
worth the added complexity. As a practical matter these “harmless
exceptions” rarely if ever occur, so a program will be no more robust if
these exceptions are ignored. If you feel you must ignore them, there is a
workaround, but it isn't pretty:
static void copy(String src, String dest) throws IOException {
boolean done = false;
try (InputStream in = new FileInputStream(src)) {
try(OutputStream out = new FileOutputStream(dest)) {
byte[] buf = new byte[8192];
int n;
while ((n = in.read(buf)) >= 0)
out.write(buf, 0, n);
}
done = true;
} catch(IOException e) {
if (!done)
throw e;
}
}
*ACKNOWLEDGMENTS*
I thank the members of the coin-dev list and my colleagues at Google for all
of the useful input on earlier versions of this proposal.
More information about the coin-dev
mailing list