RFR 8190974 Parallel stream execution within a custom ForkJoinPool should obey the parallelism

[Paul Sandoz]

> On 8 Nov 2017, at 15:48, Martin Buchholz <martinrb at google.com> wrote:
> 
> You probably want a different summary.
> + * @summary Tests counting of streams containing Integer.MAX_VALUE + 1 elements
> 
Doh, indeed.


> Will this fail if common pool parallelism is odd?
> +            int splitsForPHalfC = countSplits(new ForkJoinPool(ForkJoinPool.getCommonPoolParallelism() / 2));
> 

Yes, the number of splits will be parallelism * 4 rounded up to the nearest power of 2.

I modified the common pool testing and removed the doubling of the parallelism since this might result in OOME when creating the threads.


> I would drop the word "factor" below:
> +     * Default target factor of leaf tasks for parallel decomposition.
> 
Ok.


> Do you ever test with
> 
> -Djava.util.concurrent.ForkJoinPool.common.parallelism=0
> 
>  * @run junit/othervm/timeout=1000
>  *      --add-opens java.base/java.util.concurrent=ALL-UNNAMED
>  *      --add-opens java.base/java.lang=ALL-UNNAMED
>  *      -Djsr166.testImplementationDetails=true
>  *      -Djava.util.concurrent.ForkJoinPool.common.parallelism=0
>  *      JSR166TestCase
> 

Added. I had to move the test outside of the stream test area, which is set up to run in a more restricted manner.

WebRev updated in place 

Thanks,
Paul.

> 
> 
> 
> On Wed, Nov 8, 2017 at 1:01 PM, Paul Sandoz <paul.sandoz at oracle.com <mailto:paul.sandoz at oracle.com>> wrote:
> Hi,
> 
> Please review this patch to ensure that a parallel stream obeys the parallelism of a custom fork join pool when it is executed within that pool:
> 
>   http://cr.openjdk.java.net/~psandoz/jdk10/JDK-8190974-par-stream-custom-pool/webrev/ <http://cr.openjdk.java.net/~psandoz/jdk10/JDK-8190974-par-stream-custom-pool/webrev/> <http://cr.openjdk.java.net/~psandoz/jdk10/JDK-8190974-par-stream-custom-pool/webrev/ <http://cr.openjdk.java.net/~psandoz/jdk10/JDK-8190974-par-stream-custom-pool/webrev/>>
> 
> Streams currently do not support capabilities to control the level of parallelism and therefore resources utilised (tricky API design problem to get right).
> 
> At the moment the trick is to run stream executions within a custom pool, however the number of fork join tasks created will be in proportion to the parallelism of the common pool thus the execution will be over-provisioned. This can be especially noticeable on large machines with many cores.
> 
> Paul.
>