RFR 8190974 Parallel stream execution within a custom ForkJoinPool should obey the parallelism

[Martin Buchholz]

Looks good, but have comments as always.

Fix type: exection

Unrelated typo: the this slice spliterator

I would test core libraries under taskset with an odd number of cpus



On Wed, Nov 8, 2017 at 4:43 PM, Paul Sandoz <paul.sandoz at oracle.com> wrote:

>
> On 8 Nov 2017, at 15:48, Martin Buchholz <martinrb at google.com> wrote:
>
> You probably want a different summary.
>
> + * @summary Tests counting of streams containing Integer.MAX_VALUE + 1 elements
>
>
> Doh, indeed.
>
>
> Will this fail if common pool parallelism is odd?
>
> +            int splitsForPHalfC = countSplits(new ForkJoinPool(ForkJoinPool.getCommonPoolParallelism() / 2));
>
>
>
> Yes, the number of splits will be parallelism * 4 rounded up to the
> nearest power of 2.
>
> I modified the common pool testing and removed the doubling of the
> parallelism since this might result in OOME when creating the threads.
>
>
> I would drop the word "factor" below:
>
> +     * Default target factor of leaf tasks for parallel decomposition.
>
>
> Ok.
>
>
> Do you ever test with
>
> -Djava.util.concurrent.ForkJoinPool.common.parallelism=0
>
>  * @run junit/othervm/timeout=1000
>  *      --add-opens java.base/java.util.concurrent=ALL-UNNAMED
>  *      --add-opens java.base/java.lang=ALL-UNNAMED
>  *      -Djsr166.testImplementationDetails=true
>  *      -Djava.util.concurrent.ForkJoinPool.common.parallelism=0
>  *      JSR166TestCase
>
>
> Added. I had to move the test outside of the stream test area, which is
> set up to run in a more restricted manner.
>
> WebRev updated in place
>
> Thanks,
> Paul.
>
>
>
>
> On Wed, Nov 8, 2017 at 1:01 PM, Paul Sandoz <paul.sandoz at oracle.com>
> wrote:
>
>> Hi,
>>
>> Please review this patch to ensure that a parallel stream obeys the
>> parallelism of a custom fork join pool when it is executed within that pool:
>>
>>   http://cr.openjdk.java.net/~psandoz/jdk10/JDK-8190974-par-st
>> ream-custom-pool/webrev/ <http://cr.openjdk.java.net/~p
>> sandoz/jdk10/JDK-8190974-par-stream-custom-pool/webrev/>
>>
>> Streams currently do not support capabilities to control the level of
>> parallelism and therefore resources utilised (tricky API design problem to
>> get right).
>>
>> At the moment the trick is to run stream executions within a custom pool,
>> however the number of fork join tasks created will be in proportion to the
>> parallelism of the common pool thus the execution will be over-provisioned.
>> This can be especially noticeable on large machines with many cores.
>>
>> Paul.
>
>
>
>