MayHoldCloseableResource vs AutoCloseable

[Zhong Yu]

On Mon, Jul 8, 2013 at 10:06 AM, Paul Benedict <pbenedict at apache.org> wrote:
> Thanks Brian. So I go back to my question. Why not just use the annotation
> and dump the interface? If any AutoCloseable sub-interface is annotated
> with @MayHoldResource, resource leak warnings and catching can be disabled.

That'll require we annotate majority of methods in JDK that return
Streams (which require no close()). In Brian's solution only a few IO
methods need to be annotated.

However the solution may not work in some applications where most
Streams hold resources and require close(); it'll be a nuance to
annotate most of their Stream yielding methods.

One of EG's motivations to add close() directly on Stream seems to be
that if a stream is associated with some resources, a derived stream
should be associated with these resources too, and the responsibility
to free these resources can be transferred from the original stream to
the derived stream. However then the programmer has to mentally keep
track of the transfer of responsibility. It's arguably more
complicated than pinning the responsibility on the original stream.

What I mean is, in the previous design the IO methods returned
CloseableStream<T>

    interface CloseableStream<T> extends AutoCloseable, Stream<T>

and EG didn't like it because, say, CloseableStream.map() does not
return a CloseableStream, so the derived streams are not closeable.
Adding close() to all Streams solves that problem.

One can object that why do we need to make derived streams closeable
too? All we need to ensure is to close the original stream; we don't
need to close derived streams. If we need to leak derived stream to
another scope, e.g. return it from a method body, we can then wrap it
in CloseableStream, thus transferring close() from the original stream
to this derived stream.

What we really need here is an intersection type Stream&Closeable;
Java doesn't have denotable intersection type, but there are several
ways to approximate that effect. We've seen two solutions with
subtyping. There are also 3 other solutions with composition:

1.
        Pair<Stream<Path>, Closeable> listFiles(dir);

2.
        CloseableAnd<Stream<Path>> listFiles(dir);

        interface CloseableAnd<X> extends Closeable
            X getX();

3.
        StreamAnd<Closeable, Path> listFiles(dir);

        interface StreamAnd<X, T> extends Stream<T>
            X getX();


My personal favorite is (2). It's slightly more verbose than
CloseableStream at use site, but it's more generic and it works on any
objects associated with GC-resistant resources.


Zhong Yu




> The current sub-interface, imo, does not achieve anything.
>
>
> On Mon, Jul 8, 2013 at 9:54 AM, Brian Goetz <brian.goetz at oracle.com> wrote:
>
>> This was all extensively hashed out on the EG list, you should read the
>> discussions.
>>
>> The difference is in the presumption.  An AC object is *assumed to*
>> require closing, unless you can demonstrate a reason why it does not
>> require closing (e.g., ByteArrayInputStream holds no GC-resistent resources
>> and its close() method does nothing.)
>>
>> A MHCR object should be presumed *not to* require closing, unless you know
>> that it does (e.g., an IO-backed stream.)
>>
>> Static analysis tools are one audience for this distinction, but there's
>> another important one: the brains of people who read code.
>>
>> Not closing something that holds a file handle is a big problem, but at
>> the same time, we don't want to ask people to write:
>>
>> int sum =
>>   list.stream()
>>       .filter(...)
>>       .map(...)
>>       .sum();
>>
>> as
>>
>> int sum;
>> try (IntStream s = list.stream()
>>                        .filter(...)
>>                        .map(...)) {
>>     sum = s.sum();
>> }
>>
>> just because list *might* hold a file handle.
>>
>> On the other hand, we want streams to be closeable, because if they do
>> hold resources, we want for derived streams (like concat(a, b)) to be able
>> to ass the close message to their constituent components.
>>
>> So the problem here is "must implement a close mechanism, but 99% of the
>> time, users should pretend they don't and just go on with normal coding."
>>  And if they're going to implement a close mechanism, NOT implementing AC
>> is dumb.
>>
>> The reality is there are lots of places where we rely on users to know
>> stuff that can't be determined from the static type system.  For example:
>>
>>  - This object is thread-confined (or thread-safe), so I can access its
>> state without additional synchronization
>>  - This object is immutable, so I can freely share it with other code
>> without copying it
>>
>> We routinely rely on the user to know when certain things are safe, and
>> very often the user does know these things.  The same is true with "does
>> this stream hold a GC-resistent resource like a file handle."  Because
>> streams are designed to be created and traversed in the same expression,
>> most of the time, the user already knows with certainty.  In those cases
>> where a framework has to manipulate streams (like Stream.concat) and
>> therefore has to operate in generality, it can take the conservative route
>> and assume that closing is needed.
>>
>> You can consider this as a loose end from Coin, where AC only considered
>> the case where it was reasonable to presume that failure to close was an
>> error.  The case where closing is supported but not expected was not
>> handled by AC.  MHCR attempts to address that.  Is it beautiful? Certainly
>> not.  But, as Doug said, it "provides something better than any other
>> related schemes I know."  If the name weren't so clunky, I daresay you
>> might not have even noticed.
>>
>>
>>
>> On 7/8/2013 10:20 AM, Paul Benedict wrote:
>>
>>> What are the semantic differences between these two interfaces? I come
>>> away
>>> with no programming difference (i.e., you will always need to close the
>>> stream because it *could* hold a resource); but it's more of a hint to
>>> IDEs
>>> not to display a "resource leak" warning if try-with-resources is not
>>> used.
>>>
>>> Am I correct? The only reason I ask is because it seems this interface can
>>> be done without. I think it would be much more palatable to use just the
>>> annotation only rather than the sub-interface.
>>>
>>> Paul
>>>
>>>
>