Regression (b86): OOM on parallel stream

[Paul Sandoz]

Hi Christian,

Thanks. It's a known issue that limit (and substream) can cause OOMEs on infinite or very large inputs.

When evaluated in parallel limit buffers content (it acts like a barrier). We have a number of ideas on the table to investigate that will either avoid buffering and/or reduce the potential for OOMEs.

Paul.

On Apr 19, 2013, at 2:08 PM, "Mallwitz, Christian" <christian.mallwitz at commerzbank.com> wrote:

> Hi,
> 
> The following throws (after a 10+ minute wait) an OOM - removing the parallel() bit produces the expected result of 200000.
> 
> Thanks
> Christian
> 
> public class OOM {
>    public static void main(String[] args) {
>        System.out.println(
>                java.util.stream.Streams.iterate(1L, n -> n + 1L)
>                        .parallel()
>                        .filter(l -> l % 100 == 0).limit(200_000).count());
>    }
> }
> 
> Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
> 	at java.util.stream.SpinedBuffer.ensureCapacity(SpinedBuffer.java:129)
> 	at java.util.stream.Nodes$SpinedNodeBuilder.begin(Nodes.java:1278)
> 	at java.util.stream.Sink$ChainedReference.begin(Sink.java:252)
> 	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:452)
> 	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:443)
> 	at java.util.stream.SliceOps$SliceTask.doLeaf(SliceOps.java:328)
> 	at java.util.stream.SliceOps$SliceTask.doLeaf(SliceOps.java:273)
> 	at java.util.stream.AbstractTask.compute(AbstractTask.java:284)
> 	at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:710)
> 	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:260)
> 	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1012)
> 	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1631)
> 	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> 
>