Resource Constrained Thread Per Task Executor

Attila Kelemen attila.kelemen85 at
Mon May 13 20:36:15 UTC 2024

You don't actually need fancy classes from JDK to implement this
(definitely no internal ones). Though it wouldn't be bad to have a utility
for this out of the box, the implementation of such an `ExecutorService`
(if you only need an `Executor` it is even simpler) is relatively
straightforward. Assuming you already have an `ExecutorService` with a
thread per task implementation, you just have to `acquire` the semaphore
before submitting a task to the wrapped executor, and `release` it once
the submitted task completes. If you only need an `Executor` then it is
simpler, because then you just need to do this with the `execute` method,
and also in that case you might trivially replace the wrapped `Executor`
with a `ThreadFactory` and enforce a thread per task policy.

Colin Redmond <colin.redmond at> ezt írta (időpont: 2024. máj.
13., H, 21:25):

> Hello,
> I have been experimenting with Virtual Threads in a Spring Integration
> project that is essentially a producer consumer project. it has a small
> number of threads that reads from an external queue (pubsub) and passes the
> received message off to an ThreadPerTaskExecutor executor service. It then
> executes several IO heavy steps.
> Overall Virtual threads have been amazing, before I was running with
> 500-600 platform threads. Switching to virtual threads has enabled me to
> scale 20-30% more for the same resources (CPU, Memory).
> However I am running into one problem. Since this is a spring integration
> project I don't have a lot of control over how the work is propagated
> between threads. The thread that reads from the external queue runs much
> faster than the internal threads processing the work. When running with
> high load the service fails OOM because it can start 100k+ virtual threads,
> and more importantly those virtual threads have 100k+ messages to be
> processed which take up a lot of memory.
> I resolved the issue by using a ThreadPoolExecutor a with small core pool
> but large a max pool size (1000), a virtual thread factory,  a limited size
> LinkedBlockingQueue and the RejectedExecutionHandler uses the spring
> CallerBlocksPolicy which retries to offer() the work to the executor and
> blocks. This causes back pressure and ensure that the service never pulls
> in more work than it can manage, so it no longer OOM. It has enough threads
> to saturate the CPU and by using virtual threads I save a lot of memory vs
> platform threads.
> It has been suggested we use a semaphore to control flow with virtual
> threads instead of a Thread Pool. The best place I can think of for
> controlling the number of worker threads is in the Executor Service itself.
> So, I am investigating creating a ResourceConstrainedThreadPerTaskExecutor
> that will request a resource before launching a new thread. This resource
> could be a semaphore or even a guava ratelimiter, but I have been facing
> some issues. Unlike the ThreadPoolExecutor, the ThreadPerTaskExecutor does
> not allow you to customize it. I can not extend it, I tried to make my own
> ExecutorService that has a ThreadPerTaskExecutor, but the best place to
> check for a resource is the private Thread start(Runnable task)  method
> that I cant wrap. Finally I was going to borrow code from the
> ThreadPerTaskExecutor but it uses several jdk.internal such as
> ThreadContainer.
> So finally my questions:
>    1. I am aware that virtual threads should not be pooled, but is it
>    safe to hold a virtual thread for an extended period of time in a thread
>    pool?
>    2. Is there a perfered method to customize the ThreadPerTaskExecutor?
>    3. If I write my own executor service what is the goal of
>     jdk.internal.ThreadContainer? Other than holding the threads, does it
>    integrate with the JVM on a lower level?
>    4. Can I write an executor service without a ThreadContainer? Or can I
>    use ThreadContainer, is it acceptable to use jdk.internal classes?
> If this is not possible, I can modify Springs, ExecutorChannel to control
> the flow at a level higher, but I felt that it would be useful to have a
> New Thread Per Task Executor that could be throttled or limited.  Thanks
> for any and all comments.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <>

More information about the loom-dev mailing list