Regression (b86): OOM on parallel stream

[Brian Goetz]

Yes, in parallel, a limit operation (currently) needs to buffer its 
entire results.  There are optimizations we can apply (not yet done) on 
SIZED streams and UNORDERED streams that remove this restriction, but in 
the general case, there's going to be a lot of buffering when run in 
parallel.  This is because limit() is constrained to delivering the 
elements in encounter order.

Though in this case, buffering 200000 longs should not run out of 
memory; your heap size is probably tiny -- and the 10m wait was GC 
thrashing.

On 4/19/2013 8:08 AM, Mallwitz, Christian wrote:
> Hi,
>
> The following throws (after a 10+ minute wait) an OOM - removing the parallel() bit produces the expected result of 200000.
>
> Thanks
> Christian
>
> public class OOM {
>      public static void main(String[] args) {
>          System.out.println(
>                  java.util.stream.Streams.iterate(1L, n -> n + 1L)
>                          .parallel()
>                          .filter(l -> l % 100 == 0).limit(200_000).count());
>      }
> }
>
> Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
> 	at java.util.stream.SpinedBuffer.ensureCapacity(SpinedBuffer.java:129)
> 	at java.util.stream.Nodes$SpinedNodeBuilder.begin(Nodes.java:1278)
> 	at java.util.stream.Sink$ChainedReference.begin(Sink.java:252)
> 	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:452)
> 	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:443)
> 	at java.util.stream.SliceOps$SliceTask.doLeaf(SliceOps.java:328)
> 	at java.util.stream.SliceOps$SliceTask.doLeaf(SliceOps.java:273)
> 	at java.util.stream.AbstractTask.compute(AbstractTask.java:284)
> 	at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:710)
> 	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:260)
> 	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1012)
> 	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1631)
> 	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
>
>