Resource Constrained Thread Per Task Executor

Pedro Lamarão pedro.lamarao at
Mon May 13 21:36:05 UTC 2024

This solution generalizes for other kinds of limits too. I have an
application here that does basically the same thing with a token bucket
instead of a semaphore.

Em seg., 13 de mai. de 2024 às 16:47, Robert Engels <rengels at>

> I think you can simply this. You don’t need to pool the virtual threads.
> Just use a semaphore to control how many are running at once. You can put
> this behind and Executor for a standardized api
> On May 13, 2024, at 2:10 PM, Colin Redmond <colin.redmond at>
> wrote:
> Hello,
> I have been experimenting with Virtual Threads in a Spring Integration
> project that is essentially a producer consumer project. it has a small
> number of threads that reads from an external queue (pubsub) and passes the
> received message off to an ThreadPerTaskExecutor executor service. It then
> executes several IO heavy steps.
> Overall Virtual threads have been amazing, before I was running with
> 500-600 platform threads. Switching to virtual threads has enabled me to
> scale 20-30% more for the same resources (CPU, Memory).
> However I am running into one problem. Since this is a spring integration
> project I don't have a lot of control over how the work is propagated
> between threads. The thread that reads from the external queue runs much
> faster than the internal threads processing the work. When running with
> high load the service fails OOM because it can start 100k+ virtual threads,
> and more importantly those virtual threads have 100k+ messages to be
> processed which take up a lot of memory.
> I resolved the issue by using a ThreadPoolExecutor a with small core pool
> but large a max pool size (1000), a virtual thread factory,  a limited size
> LinkedBlockingQueue and the RejectedExecutionHandler uses the spring
> CallerBlocksPolicy which retries to offer() the work to the executor and
> blocks. This causes back pressure and ensure that the service never pulls
> in more work than it can manage, so it no longer OOM. It has enough threads
> to saturate the CPU and by using virtual threads I save a lot of memory vs
> platform threads.
> It has been suggested we use a semaphore to control flow with virtual
> threads instead of a Thread Pool. The best place I can think of for
> controlling the number of worker threads is in the Executor Service itself.
> So, I am investigating creating a ResourceConstrainedThreadPerTaskExecutor
> that will request a resource before launching a new thread. This resource
> could be a semaphore or even a guava ratelimiter, but I have been facing
> some issues. Unlike the ThreadPoolExecutor, the ThreadPerTaskExecutor does
> not allow you to customize it. I can not extend it, I tried to make my own
> ExecutorService that has a ThreadPerTaskExecutor, but the best place to
> check for a resource is the private Thread start(Runnable task)  method
> that I cant wrap. Finally I was going to borrow code from the
> ThreadPerTaskExecutor but it uses several jdk.internal such as
> ThreadContainer.
> So finally my questions:
>    1. I am aware that virtual threads should not be pooled, but is it
>    safe to hold a virtual thread for an extended period of time in a thread
>    pool?
>    2. Is there a perfered method to customize the ThreadPerTaskExecutor?
>    3. If I write my own executor service what is the goal of
>     jdk.internal.ThreadContainer? Other than holding the threads, does it
>    integrate with the JVM on a lower level?
>    4. Can I write an executor service without a ThreadContainer? Or can I
>    use ThreadContainer, is it acceptable to use jdk.internal classes?
> If this is not possible, I can modify Springs, ExecutorChannel to control
> the flow at a level higher, but I felt that it would be useful to have a
> New Thread Per Task Executor that could be throttled or limited.  Thanks
> for any and all comments.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <>

More information about the loom-dev mailing list