StreamOpFlag.* and OutOfMemoryError when going parall

[Brian Goetz]

There's a good reason you are struggling with it -- this API is not 
intended for "casual" builders of streams.  The intention is that 99.9% 
of users will never write code to call the Streams.stream() method 
directly; they will use arrays, collections, generators, or other 
packaged providers of streams.

A safe default is 0.  You can OR together the IS_ version of the flags 
if you know something about the nature of your stream.  You probably 
want IS_ORDERED if you want to represent a stream for which the order of 
elements means something (e.g., a range generator, a List, an array, etc.)

The reason it is failing with OOME is that the parallel implementation 
of limit() computes the entire results rather than operating fully 
lazily.  While it would be preferable to operate more lazily, a lazy 
parallel implementation of operations like limit is not trivial.  We 
hope to improve this in the future.

On 12/5/2012 1:18 PM, Christian Mallwitz wrote:
> Hi,
>
> Using lambda-8-b67-linux-i586-03_dec_2012 I'm trying to compute n (not
> necessarily the first n) prime numbers:
>
> import java.util.*;
> import java.util.function.*;
> import java.util.stream.*;
>
> public class LambdaExample3 {
>
>      public static boolean isPrime(long n) {
>          if (n <= 1) { return false; }
>          if (n == 2) { return true; }
>          for (int i = 2; i <= Math.sqrt(n) + 1; i++) { if (n % i == 0)
> return false; }
>          return true;
>      }
>
>      public static void main(String[] args) {
>
>          Stream<Long> stream =
> Streams.parallel(Streams.spliteratorUnknownSize(new Iterator<Long>() {
>                  private long n = 0;
>                  @Override public boolean hasNext() { return true; }
>                  @Override public Long next() { return ++n; }
>                  }),
>              // fails with OutOfMemoryError
>              // StreamOpFlag.toStreamFlags(StreamOpFlag.NOT_SIZED,
> StreamOpFlag.INITIAL_OPS_VALUE)
>              // fails with OutOfMemoryError
>              // StreamOpFlag.NOT_SIZED
>              // works, but no speed-up to non-parallel version
>              StreamOpFlag.INITIAL_OPS_VALUE
>              );
>
>          stream
>              .filter(LambdaExample3::isPrime)
>              .limit(300000)
>              .forEach(l -> { /*System.out.println(l);*/ });
>      }
> }
>
> I'm struggling with the StreamOpFlag parameter. What should I pick?
> INITIAL_OPS_VALUE seems to work but isn't running anything in parallel
> (at least it is not faster than the serial version). NOT_SIZED isn't
> working but failing miserably with an  OutOfMemoryError. IS_PARALLEL
> is not needed because I already use parallel() - should specifying
> IS_PARALLEL and using Streams.stream() supposed to go parallel as
> well?
>
> Is the OutOfMemoryError caused by a bug? The OutOfMemoryError is
> reported from a fork/join pool thread so at least it is going
> parallel? Am I missing something?
>
> Thanks
> Christian
>