Avoiding Side Effects of ForkJoinPool.managedBlock

[Johannes Spangenberg]

Hello,

I have a question about dealing with side effects caused by ForkJoinPool. I am not certain if this mailing list is the appropriate platform, but I hope it is. The issue I am facing is that running code within a ForkJoinPool may change the behavior of the code. These changes in behavior have resulted in non-deterministic test failures in one of the repositories I work on. JUnit uses a ForkJoinPool when you enable parallel test execution. I would like to disable or avoid the side effects of various methods if called from a ForkJoinPool. So far, I haven't found a solution, except for completely replacing the use of the ForkJoinPool in JUnit with some custom scheduler which is not a ForkJoinPool.

Is there a solution to force an "unmanaged" block (as opposed to ForkJoinPool.managedBlock)? Is there alternatively a good approach to transfer CPU bound subtasks to another thread pool while blocking the ForkJoinWorkerThread without compensation? I have implemented a workaround which I explain below, but I am not sure if this will remain effective in future JDK versions. I am currently using JDK 17 but were also able to reproduce the issues with JDK 21.

I have observed the following side-effects caused by managed blocks or similar mechanisms:

1. Parallel stream methods execute another task (i.e. JUnit test) from the pool recursively, which is particularly problematic if your code utilizes any ThreadLocal.

2. Blocking methods spawn around 256 threads in total to "compensate" for the blocking operation. Consequently, you end up with up to 256 tests running concurrently, each of them might or might not be CPU bound (unknown to the ForkJoinPool).

3. Blocking methods may throw a RejectedExecutionException when the thread limit is reached. This is effectively a race condition which may lead to exceptions.

I have not been able to determine under which circumstances each behavior occurs. I am unaware of any thorough documentation that clearly outlines the expected behavior in different scenarios with different blocking methods. While (1.) and (3.) have caused test failures, (2.) simply causes JUnit to run 256 tests in parallel instead of the intended 12. I attached a JUnit test to reproduce (1.) and (3.), but it might not fail on every run.

Many of the blocking methods of the standard library include a check if the current thread is an instance of ForkJoinWorkerThread. My current workaround involves wrapping the code that makes blocking calls into a FutureTask which is executed on another thread and then joining this task afterwards. As of now, FutureTask.get() seems not to implement any of the side-effects. As the missing instanceof-check in FutureTask makes it inconsistent with other Futures like CompletableFuture, I fear it might be considered a "bug". I would like to know a safe solution which is specified to continue to work in future JDKs.

PS: There is also a ticket on the JUnit project about this topic, but it only talks about side-effect (2.), but not the other side-effects we observed.
https://github.com/junit-team/junit5/issues/3108

Thanks,
Johannes

-------------- next part --------------
A non-text attachment was scrubbed...
Name: ForkJoinPoolTest.java
Type: application/octet-stream
Size: 1994 bytes
Desc: ForkJoinPoolTest.java
URL: <https://mail.openjdk.org/pipermail/core-libs-dev/attachments/20240118/de997cb5/ForkJoinPoolTest-0001.java>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 7798 bytes
Desc: not available
URL: <https://mail.openjdk.org/pipermail/core-libs-dev/attachments/20240118/de997cb5/smime-0001.p7s>