Fast graceful shutdown of ThreadPerTaskExecutor (when expected WAITING Threads)

[Rob Bygrave]

*> separate the “idle” and “busy” threads by spawning them in different
ExecutorServices*

That makes perfect sense, thanks!!  I'll look to work this through.

Cheers, Rob.

On Fri, 23 Dec 2022 at 03:29, Ron Pressler <ron.pressler at oracle.com> wrote:

> I am not familiar with Helidon’s internals or with the details of the
> particular issue, but you can separate the “idle” and “busy” threads by
> spawning them in different ExecutorServices. For example, threads that wait
> an accept new requests are spawned in one ES and they, in turn, spawn a
> “request processing” thread in another. To shut down the server you will
> then forcefully shut down the first ES, stopping new requests from being
> accepted, and gracefully shut down the second, giving the requests some
> time to finish.
>
> — Ron
>
> On 21 Dec 2022, at 22:22, Rob Bygrave <robin.bygrave at gmail.com> wrote:
>
> A "solution / fix" to this issue that modifies the internals of *
> java.util.concurrent.ThreadPerTaskExector* is to:
>
>
> *1) Add a new interface InterruptableTask*
>
> public interface InterruptableTask {
>
>     /**
>      * Return true if this is a task that is deemed to be in an idle state
> that can be interrupted.
>      */
>     boolean canInterrupt();
> }
>
>
> *2) Modify java.util.concurrent.ThreadPerTaskExector by:*
>
> *2i)* Changing the threads field from storing just Thread to additionally
> storing the associated task (Runnable or Callable)
>
> // instead of ...
> private final Set<Thread> threads = ConcurrentHashMap.newKeySet();
>
> // use a Map with values being the tasks (Runnable or Callable)
> private final Map<Thread, Object> threadTasks = new ConcurrentHashMap<>();
>
>
>
> *2ii)* Add a new internal method tryStopInterruptableTask():
>
> private void tryStopInterruptableTask() {
>   threadTasks.entrySet().stream()
>        .filter(entry -> entry.getKey().isAlive()) // Thread isAlive
>        .filter(entry -> entry.getKey().getState() == Thread.State.WAITING)
> // Thread WAITING state
>        .forEach(entry -> {
>          if (entry.getValue() instanceof InterruptableTask task) {
>            if (task.canInterrupt()) { // application deems this task to
> NOT be in-flight (e.g. not IO waiting on database response)
>              entry.getKey().interrupt();
>            }
>          }
>        });
> }
>
> *2 iii)* Call this method as part of shutdown() (or perhaps the internal
> tryShutdownAndTerminate())
>
>
>
> With these changes, then the Nima specific ConnectionHandler just needs to
> implement InterruptableTask and then the usual
> shutdown of the ExecutorService operates nicely.
>
> executor.shutdown();
> if (!executor.awaitTermination(gracefulShutdownMillis,
> TimeUnit.MILLISECONDS)) {
>     List<Runnable> running = executor.shutdownNow();
>     if (!running.isEmpty()) {
>         LOGGER.log(INFO, running.size() + " channel tasks did not
> terminate gracefully");
>     }
> }
>
>
>
> Cheers, Rob.
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/loom-dev/attachments/20221223/261e12fa/attachment-0001.htm>