Infinite stream in parallel mode

[Brian Goetz]

We're almost there!  There's a few pieces missing.

1.  A Spliterator that can deal with arbitrary iterators.  The obvious 
splitting strategy there is pretty weak; split into (first, rest).  This 
can probably be improved by splitting into (first n, rest), where n 
varies with the number of splits.  So the first split is (first, rest), 
the second split is (first two, rest), the next is (first four, rest) up 
until you hit some target size (derived from core count.)  This lets 
early splits create some work to get some forking going faster, while 
later splits have lower task management overhead once the pool is hot.

2.  Some way of canceling a source.  Right now, the only way a 
computation on an infinite stream completes is that an intermediate 
operation (like limit) truncates the stream or the terminal operation 
(like findFirst) cancels the computation.  But it is also useful to do 
something like:

   infiniteEventStream.parallel().forEach(...)

and let it burn until you're "tired" (e.g., timer expires, shutdown is 
requested, an adequately good solution is found.)  What we need is a way 
of wiring that termination criteria back into the source, so it says "no 
more elemnets" and the computation can quiesce naturally. (This latter 
one applies equally to serial and parallel modes.)



On 11/16/2012 9:49 AM, Georgiy Rakov wrote:
> Hello,
>
> I wonder if it is possible to process infinite stream in parallel mode now.
> Maybe it is going to be implemented later somehow?
>
> Thanks, Gerogiy.
>