Resource Constrained Thread Per Task Executor

[Colin Redmond]

If it is safe to keep virtual threads and pool them, there is still a net benefit to pooling virtual threads. I agree a major benefits of virtual threads is that they are cheap to create, so there is no need to pool them. But the memory and performance increase even pooling is no small matter.

In my case I have 500 platform threads I am limited because my container is small and they start to use a significant portion of the memory. I switch them for 1000 virtual threads, this saves me ~ 500 mb of RAM (maybe less i didn't do the math). Between less need for context switching and more memory for garbage, I am still seeing a 20 - 30% improvements. Which even with pooled virtual threads is a big improvement.

Firstly I was hoping there was an existing solution, I googled it a while but couldn't find any specific open source packages if anyone knows any let me know, I would prefer to import something. If not I may continue my journey of writing a new ExecutorService for my use case.


________________________________
From: Attila Kelemen <attila.kelemen85 at gmail.com>
Sent: May 13, 2024 2:41 PM
To: Colin Redmond <Colin.Redmond at outlook.com>
Cc: masoud parvari <masoud.parvari at gmail.com>; Robert Engels <rengels at ix.netcom.com>; loom-dev at openjdk.org <loom-dev at openjdk.org>
Subject: Re: Resource Constrained Thread Per Task Executor

If you are wrapping another `ExecutorService`, then you don't really need to have anything special for `shutdown` and such. You just have to delegate those calls.

As for "is it safe to keep a virtual thread for an extended period of time": Yes, it is safe, but pooling VT defeats its purpose.

Colin Redmond <Colin.Redmond at outlook.com<mailto:Colin.Redmond at outlook.com>> ezt írta (időpont: 2024. máj. 13., H, 23:34):
Thanks for the idea, the AbstractExecutorService would be a great starting place, and that means I need to implement all the shutdown logic. Which is manageable, but tricky. I guess the moral of the story is there is no easy way out of the box to implement constrained resources with new thread per task executors. I will have to write some code and reimplement some things to get what I want.


But maybe someone can answer me the question: is it safe to keep a virtual thread for an extended period of time? Maybe I can live with my ThreadPoolExecutor solution if it is safe. For my use case it works, but it isn't ideal.

Thanks all for the feedback!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/loom-dev/attachments/20240513/281fc17d/attachment-0001.htm>