<div dir="ltr">Hey team<br><br>We are experiencing some weird halting issues when scheduling tasks with ScheduledExecutorService and VT.<br><br>We are observing some near deadlock issues / halting of executors which we are not able to reproduce when using platform threads<br><br>This is one example of a near deadlock situation, where for some reason virtual threads are starting to execute a task, and not reaching the finally block nor the catch Throwable block to release the acquired semaphore permit, reaching to drained semaphore.<br><br>Are you familiar with this behavior? I hope we are doing something wrong 😅<br><br>Note - The scheduler is running on a platform thread<br><div><br><div style="padding:0px 2px"><div style="color:rgb(0,0,0);font-family:Consolas;font-size:10pt;white-space:pre"><p style="margin:0px">        ```ScheduledFuture<?> <span style="color:rgb(0,0,192)">deviceFutureTask</span> = <span style="text-decoration-line:underline;text-decoration-style:wavy;text-decoration-color:rgb(255,0,128)">scheduler</span>.scheduleAtFixedRate(() -> {</p><p style="margin:0px">              <span style="color:rgb(127,0,85);font-weight:bold">try</span> {</p><p style="margin:0px">                   logger.debug(<span style="color:rgb(42,0,255)">"[{}] Trying to acquire permit to schedule task [{}] for device. Number of available permits: [{}]"</span>,</p><p style="margin:0px">                                      device, task, semaphore.availablePermits());</p><p style="margin:0px">                    <span style="color:rgb(127,0,85);font-weight:bold">if</span> (semaphore.tryAcquire(waitingTimeout, TimeUnit.MILLISECONDS)) {</p><p style="margin:0px">                              logger.debug(<span style="color:rgb(42,0,255)">"[{}] Acquired permit to schedule [{}] task for device"</span>, device, task);</p><p style="margin:0px">                           Thread.ofVirtual().start(() -> {</p><p style="margin:0px">                                     <span style="color:rgb(127,0,85);font-weight:bold">try</span> {</p><p style="margin:0px">                                           logger.debug(<span style="color:rgb(42,0,255)">"[{}] Scheduling [{}] task for device"</span>, device, task);</p><p style="margin:0px">                                            <span style="color:rgb(63,127,95)">// some I/O intensive work</span></p><p style="margin:0px">                                              logger.debug(<span style="color:rgb(42,0,255)">"[{}] Finished processing [{}] task for device"</span>, device, task);</p><p style="margin:0px">                                   } <span style="color:rgb(127,0,85);font-weight:bold">catch</span> (Exception e) {</p><p style="margin:0px">                                         logger.error(<span style="color:rgb(42,0,255)">"[{}] Failed to process [{}] task for device"</span>, device, task, e);</p><p style="margin:0px">                                  } <span style="color:rgb(127,0,85);font-weight:bold">finally</span> {</p><p style="margin:0px">                                             semaphore.release();</p><p style="margin:0px">                                    }</p><p style="margin:0px">                               });</p><p style="margin:0px">                     } <span style="color:rgb(127,0,85);font-weight:bold">else</span> {</p><p style="margin:0px">                                logger.error(<span style="color:rgb(42,0,255)">"Timed out while waiting for permit to schedule task [{}] for device [{}]"</span>, task, device);</p><p style="margin:0px">                        }</p><p style="margin:0px">               } <span style="color:rgb(127,0,85);font-weight:bold">catch</span> (Throwable t) {</p><p style="margin:0px">                 logger.error(<span style="color:rgb(42,0,255)">"Failed to execute task [{}] for device [{}]"</span>, task, deviceId, t);</p><p style="margin:0px">                        semaphore.release();</p><p style="margin:0px">            }</p><p style="margin:0px">       }, (<span style="color:rgb(127,0,85);font-weight:bold">long</span>) (<span style="text-decoration-line:underline;text-decoration-style:wavy;text-decoration-color:rgb(255,0,128)">entropy</span> * <span style="text-decoration-line:underline;text-decoration-style:wavy;text-decoration-color:rgb(255,0,128)">schedulingInterval</span>), <span style="text-decoration-line:underline;text-decoration-style:wavy;text-decoration-color:rgb(255,0,128)">schedulingInterval</span>, TimeUnit.<span style="color:rgb(0,0,192);font-style:italic;font-weight:bold">MILLISECONDS</span>);```</p><p style="margin:0px"><br></p></div></div><div>We are </div><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div dir="ltr"><div style="font-size:12.8px"><p class="MsoNormal" style="color:rgb(34,34,34);font-size:small;background-image:initial;background-position:initial;background-repeat:initial">Kind regards,<b style="font-size:13.3333px;color:rgb(0,176,240);font-family:Ebrima"><br><br>Yuval Lombard</b><br></p><p class="MsoNormal" style="color:rgb(34,34,34);font-size:small;background-image:initial;background-position:initial;background-repeat:initial"><font color="#3b3838" face="Ebrima"><span style="font-size:12px"><b>Lead Software Engineer</b></span></font></p><p class="MsoNormal" style="color:rgb(34,34,34);font-size:small;background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:9pt;font-family:Ebrima;color:rgb(59,56,56)">+972.50.548.0111</span><span style="font-size:9pt;font-family:Ebrima;color:rgb(64,64,64)"><u></u><u></u></span></p><p class="MsoNormal" style="color:rgb(34,34,34);font-size:small;background-image:initial;background-position:initial;background-repeat:initial"><font color="#0000ff" face="Ebrima"><span style="font-size:12px"><a href="mailto:yuval.l@securithings.com" style="color:rgb(17,85,204)" target="_blank">yuval.l@securithings.com</a></span></font></p><p class="MsoNormal" style="color:rgb(34,34,34);font-size:small;background-image:initial;background-position:initial;background-repeat:initial"><img src="cid:ii_l13cyhga0" alt="logo_black.png" width="200" height="47"></p></div></div></div></div></div></div>