Regression (b92): OOM on filter/limit operation on unbound/parallel stream

[Mallwitz, Christian]

Hi,

Re-reporting problem for build 92: the filter/limit on unbound, parallel stream operation behaves very weird. How weird? Well:

- the serial version uses about 16M of heap, the parallel gobbles up all of the 1GB defined as max
- it throws an OOM after running for hours while it is __NOT__ garbage collecting much but
- the filter predicate is called on millions of elements more than what can be observed using a serial stream
- I can't observe any parallelism (based on number of cores and system load - 4 cores and only 25% load)

I'm using 1.8.0-ea-lambda-nightly-h4544-20130526-b92-b00 on a Windows XP 32 bit machine - VM args: -mx1g -XX:+PrintCommandLineFlags -verbose:gc -XX:+PrintGCTimeStamps

My test program tries to find the first 283,146 prime numbers (which are all below 4,000,000). It executes the filter/limit operation on a serial stream first and prints out numbers tested for primality occasionally.

Thanks
Christian





package com.snuffbumble.lambda;

import java.util.stream.LongStream;
import java.util.stream.Stream;

public class OOM {

    public static void main(String... ignored) {

        // first 283_146 primes are all < 4_000_000

        firstNPrimes(283_146, getIteratorBasedStream(false)); // only minor GC - max heap 16M
        firstNPrimes(283_146, getIteratorBasedStream(true));  // full GC - up to max heap 1GB and subsequent OOM
    }

    private static Stream<Long> getIteratorBasedStream(boolean produceParallelStream) {
        Stream<Long> s = LongStream.iterate(1L, n -> n + 1L).boxed();
        return produceParallelStream ? s.parallel() : s;
    }

    private static void firstNPrimes(int n, Stream<Long> stream) {
        System.out.println(String.format("firstNPrimes (%8d, %5b): %8d", n, stream.isParallel(),
                stream
                        .filter(OOM::isPrime)
                        .limit(n) // limit after primality test
                        .count()));
    }

    private static long start_millis = System.currentTimeMillis();
    private static long max_n_seen = 3_999_970L;

    private static boolean isPrime(long n) {

        if (n >= max_n_seen) {
            System.out.println(String.format("%.3f: %d", (System.currentTimeMillis() - start_millis)/1000.0, n));
            max_n_seen = n + (n/20L); // new max_n_seen 5% larger than current n
        }

        if (n <= 1) { return false; }
        if (n == 2) { return true; }
        if (n % 2 == 0) { return false; }
        for (int i = 3; i <= (int) Math.sqrt(n); i += 2) { if (n % i == 0) return false; }
        return true;
    }
}