Some Classes with a public void close() don't implement AutoCloseable

[John Rose]

On Apr 17, 2020, at 9:35 AM, Florian Weimer <fw at deneb.enyo.de> wrote:
> 
> * forax:
> 
>> - until lambdas are retrofitted as inline types, i'm worry people
>> will use forceTryable with a ReentrantLock, try(var __ =
>> forceTryable(lock::unlock)) { ... }  because most of the time, the
>> lambda will not be allocated but not always, sometimes it may still
>> be found escaping by the JITs.
> 
> The JDK itself routinely does something very similar when registering
> cleaners.  Is this lack of robustness in the presence of OOMEs a
> quality-of-implementation issue?  Or less or more important?

Turning this around in my mind, it seems like the root cause of
this non-robustness is a minor mis-feature of TWR.  TWR says
to itself, “I only have one job, and that is to call close”.  That’s
fine, but that means that means that the object to be closed
must be created *and* opened before TWR takes control.
If the closing operation is mediated by an adapter, the adapter
will almost certainly be created *after* the original (unadapted)
object is created and opened.  This is the point where an OOME
could bollix things up.  To defend against such failures, TWR
would have to define a distinct point where the object is opened.
This point would be a precise state transition immediately before
entry into the block covered by catches and the `finally` block.

Because TWR has the reasonable simplification that it only cares
about closing—the opening is up to the coder—it cannot help
precisely define the opening point; the opening point happens
“some time before” the TWR gets to work.  That “some time before”
happens either textually before the TWR, or inside the TWR’s
head expression.

If the actual logical open succeeds, but then there is an adapter
step in the TWR’s head expression which causes an exception
to be thrown, then we have a problem we could call a “lost
close”.  The TWR seems to promise that its head action will
*always* be undone by an auto-close, but that promise has
fine print; there are small gaps between the logical open and
the actual finalization guarantees of the TWR.

(Apologies if this has already been discussed to death.  I was
only a tangential part of the design discussions where the
problem of lost closes would have been covered.  I’m sure
the problem of lost closes was covered, but I’m not sure
how deep was the argument that dismissed the problem.
It seems to me that the lore about lost closes has grown
years after TWR was designed.  For example, we have had
to build JVM features to assist—partially—with the problem.)

There are possible workarounds for lost closes, which ask the
user of the TWR block to place an “open” expression or statement
explicitly somewhere in the TWR:

try (var locker = new Locker()) {
   locker.open();  // explicit open statement ensures all setup happens first
}

or:

try (var __ = new Locker()
      .open()  // explicit open expression ensures all setup happens first
   ) {
}

The problem with these workarounds is they require a lot of cooperation
from the user.  Which means they are error-prone.  Which means designers
of abstractions for the head expressions of TWR avoid making such
requirements on end users, and so place the logical open operation
deep inside the TWR head expression (abstracted away from the end
user’s control).

I am going out on a limb saying TWR has a “mis-feature” because the
focus of TWR on close is obviously a reasonable Java-like simplification of
TWR design.  Culturally, we go simple when we can; there seemed at the
time to be little or no downside to omitting consideration of `open` and
focusing only on `close`.

It’s arguably only a mis-feature because programmers do not think
about “close” operations as something completely separate from
“open” operations; they think in terms of balanced pairs, which
bracket transactional code.  TWR purports to help programmers
“balance the brackets” but it doesn’t help them precisely manage
the opening brackets, just the closing ones.  To the extent that
this leads programmers into bracketing mismatches problems,
that’s a mis-feature, despite TWR’s claim to do its one job
flawlessly; the problem is that TWR in practice ought to help
the programmer with both brackets, not just one.

How might this mis-feature be corrected?  Maybe it can’t.
Maybe there’s no way to deal with lost closes other than
educating programmers to avoid allocations or other
computations in the head of the TWR, at least allocations
after the logical open.  (Those are the ones which might cause
an OOME or SOE that would lead to a lost close.)

But (and this is why I’m going on at length here) maybe there
is a way to give the user more help avoiding lost closes.
It seems that if TWR were willing to define open operations
separately, some progress could be made.  Suppose the object
at the head of a TWR implemented an optional new interface
`java.lang.AutoOpenable`.  In such cases, the TWR would
call a nullary `open` operation on that object, immediately
before entering the block covered by the `finally` clause
that calls `close`. Would this help?  Yes, it would, because
the designer of the abstraction at the head of the TWR could
take care to do all argument validation and storage allocation
*without* doing the actual logical open or allocate the resource
that needs to be closed (or seize the lock).  The evaluation
of the head expression(s) of the TWR would (as before) not
be under any finally clause or catches.  The new feature here
would be that the designer of the abstraction (at the head
of the TWR) would know that the expression creating
the abstraction would do all necessary storage allocation,
adapter creation, argument validation, and (if necessary)
stack banging, *before* the final critical opening step.
It would not be up to the end user; it would be more under
the control of the abstraction, to sequence the preparatory
steps *before* the logical open.

Anyway, I think this discussion may have shaken loose a
new idea (at least new to me) for dealing with a truly vexed
problem.

I filed https://bugs.openjdk.java.net/browse/JDK-8243098 to track.

— John