Cancelable streams

[David Holmes]

On 4/11/2012 11:01 PM, Brian Goetz wrote:
>> I'm familiar with infinite streams (I was a tech. reviewer for the original Wizard Book) but I'm not familiar with your approach to cancellation.  Is there any precedent for it?
>
> This is a use case that shows up in real-time analytics.  There's a stream of data, and you want to chew on it for as long as you have cycles, but want to shut down gracefully when asked to.

This is a can of worms. In real-time we handle that with asynchronous 
cancellation techniques. Otherwise every step of the way you have to be 
aware of, and allow for, cancellation. :( Of course this is nothing new 
- cooperative cancellation requires cooperation.

>> To handle cancellation, I would signal the generator to stop returning elements, or to generate a (null) marker to signal end of stream.
>
>>
>> In other words, cancellation should be handled at the generator, not by Streams in general.
>
> And this discussion is basically over the use of the word "should" in the above sentence.  There are lots of places it *could* be handled, including just telling the user to cut the the power to the box when they're done.

In essence I think you want to insert a "gate" operation into the 
pipeline - while the gate is open elements can be read from the stream, 
but when it is closed they can not. The question then becomes how you 
control the gate, either from within the pipeline (predicate based) or 
externally (time-trigerred callback).

Adds a whole new dimension to things.

David