Avoiding Side Effects of ForkJoinPool.managedBlock

[Alan Bateman]

On 18/01/2024 09:30, Johannes Spangenberg wrote:
> Hello,
>
> I have a question about dealing with side effects caused by ForkJoinPool. I am not certain if this mailing list is the appropriate platform, but I hope it is. The issue I am facing is that running code within a ForkJoinPool may change the behavior of the code. These changes in behavior have resulted in non-deterministic test failures in one of the repositories I work on. JUnit uses a ForkJoinPool when you enable parallel test execution. I would like to disable or avoid the side effects of various methods if called from a ForkJoinPool. So far, I haven't found a solution, except for completely replacing the use of the ForkJoinPool in JUnit with some custom scheduler which is not a ForkJoinPool.
>
> Is there a solution to force an "unmanaged" block (as opposed to ForkJoinPool.managedBlock)? Is there alternatively a good approach to transfer CPU bound subtasks to another thread pool while blocking the ForkJoinWorkerThread without compensation? I have implemented a workaround which I explain below, but I am not sure if this will remain effective in future JDK versions. I am currently using JDK 17 but were also able to reproduce the issues with JDK 21.
>
> I have observed the following side-effects caused by managed blocks or similar mechanisms:
>
> 1. Parallel stream methods execute another task (i.e. JUnit test) from the pool recursively, which is particularly problematic if your code utilizes any ThreadLocal.
>
> 2. Blocking methods spawn around 256 threads in total to "compensate" for the blocking operation. Consequently, you end up with up to 256 tests running concurrently, each of them might or might not be CPU bound (unknown to the ForkJoinPool).
>
> 3. Blocking methods may throw a RejectedExecutionException when the thread limit is reached. This is effectively a race condition which may lead to exceptions.
>
> I have not been able to determine under which circumstances each behavior occurs. I am unaware of any thorough documentation that clearly outlines the expected behavior in different scenarios with different blocking methods. While (1.) and (3.) have caused test failures, (2.) simply causes JUnit to run 256 tests in parallel instead of the intended 12. I attached a JUnit test to reproduce (1.) and (3.), but it might not fail on every run.
>
> Many of the blocking methods of the standard library include a check if the current thread is an instance of ForkJoinWorkerThread. My current workaround involves wrapping the code that makes blocking calls into a FutureTask which is executed on another thread and then joining this task afterwards. As of now, FutureTask.get() seems not to implement any of the side-effects. As the missing instanceof-check in FutureTask makes it inconsistent with other Futures like CompletableFuture, I fear it might be considered a "bug". I would like to know a safe solution which is specified to continue to work in future JDKs.
>
I think it would be useful to understand how JUnit creates the 
ForkJoinPool. The reason is that it controls the parallelism and "max 
pool size". If there is interference due to managedBlocker then JUnit 
can set maxPoolSize to the same value as parallelism.

On the REE, this is also controlled by JUnit when it creates the FJP. 
The saturate parameter is the predicate that is determines if REE is 
thrown or the pool continues without an additional thread.

-Alan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/core-libs-dev/attachments/20240118/cce86b3f/attachment-0001.htm>