<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">If you look at the thread dump, you will see:<div class=""><br class=""></div><div class=""><div class=""><div class="">"MQTT Snd: test2:con08:13" #891110 [2129] daemon prio=5 os_prio=0 cpu=2772.51ms elapsed=4692.79s tid=0x00007f8a2c028dc0 nid=2129 in Object.wait()  [0x00007f89fe0df000]</div><div class="">   java.lang.Thread.State: WAITING (on object monitor)</div><div class="">        at java.lang.Object.wait0(java.base@23-loom/Native Method)</div><div class="">        - waiting on <no object reference available></div><div class="">        at java.lang.Object.wait(java.base@23-loom/Object.java:389)</div><div class="">        at java.lang.Object.wait(java.base@23-loom/Object.java:347)</div><div class="">        at org.eclipse.paho.client.mqttv3.internal.ClientState.get(ClientState.java:833)</div><div class="">        - locked <0x000000070d880b08> (a java.lang.Object)</div><div class="">        at org.eclipse.paho.client.mqttv3.internal.CommsSender.run(CommsSender.java:129)</div><div class="">        at java.lang.Thread.runWith(java.base@23-loom/Thread.java:1595)</div><div class="">        at java.lang.Thread.run(java.base@23-loom/Thread.java:1582)</div></div><div class=""><br class=""></div><div class="">I am guessing you have library code using monitors. I am concerned about the “no object reference available” above.</div><div class=""><br class=""></div><div class="">I also suggest testing with a lighter weight VT native http server like <a href="http://github.com/robaho/httpserver" class="">github.com/robaho/httpserver</a> to avoid so many framework threads - it should make things easier to diagnose.</div><div class=""><br class=""></div><div class="">Most importantly, use</div><div class=""><br class=""></div><div class=""><span style="caret-color: rgb(84, 84, 84); color: rgb(84, 84, 84); font-family: "Courier New", Courier, monospace; white-space: pre; background-color: rgb(255, 255, 255);" class="">jcmd </span><span class="variable" translate="no" style="margin: 0px; padding: 0px; box-sizing: border-box; font-style: italic; caret-color: rgb(84, 84, 84); color: rgb(84, 84, 84); font-family: "Courier New", Courier, monospace; white-space: pre;"><PID></span><span style="caret-color: rgb(84, 84, 84); color: rgb(84, 84, 84); font-family: "Courier New", Courier, monospace; white-space: pre; background-color: rgb(255, 255, 255);" class=""> Thread.dump_to_file </span><span class="bold" style="margin: 0px; padding: 0px; box-sizing: border-box; font-weight: 700; caret-color: rgb(84, 84, 84); color: rgb(84, 84, 84); font-family: "Courier New", Courier, monospace; white-space: pre;">-format=text</span><span style="caret-color: rgb(84, 84, 84); color: rgb(84, 84, 84); font-family: "Courier New", Courier, monospace; white-space: pre; background-color: rgb(255, 255, 255);" class=""> </span><span class="variable" translate="no" style="margin: 0px; padding: 0px; box-sizing: border-box; font-style: italic; caret-color: rgb(84, 84, 84); color: rgb(84, 84, 84); font-family: "Courier New", Courier, monospace; white-space: pre;"><file></span></div><div class=""><span class="variable" translate="no" style="margin: 0px; padding: 0px; box-sizing: border-box; font-style: italic; caret-color: rgb(84, 84, 84); color: rgb(84, 84, 84); font-family: "Courier New", Courier, monospace; white-space: pre;"><br class=""></span></div><div class=""><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class="">so that the thread dumps contain virtual thread information.</span></div><div><br class=""><blockquote type="cite" class=""><div class="">On Jul 21, 2024, at 9:10 AM, robert engels <<a href="mailto:rengels@ix.netcom.com" class="">rengels@ix.netcom.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html; charset=us-ascii" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Hi. The code looks incorrect (although it is not complete so hard to tell).<div class=""><br class=""></div><div class="">The outer exception catch is releasing the semaphore even though it may have never acquired it.</div><div class=""><br class=""></div><div class="">Since the code is not complete, there is no way to validate what you are seeing.</div><div class=""><br class=""></div><div class="">I suggest trying to provide a standalone reproducible test case.</div><div class=""><br class=""></div><div class=""><div class=""><blockquote type="cite" class=""><div class="">On Jul 21, 2024, at 8:11 AM, Yuval Lombard <<a href="mailto:yuval.l@securithings.com" class="">yuval.l@securithings.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div dir="ltr" class="">Hi Alan,<br class=""><br class="">Attaching the thread dump when the issue occurs (as a txt file)<br class=""><br class="">It might be useful</div></div><br class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, 21 Jul 2024 at 12:41, Yuval Lombard <<a href="mailto:yuval.l@securithings.com" class="">yuval.l@securithings.com</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr" class="">Hi Alan<br class=""><br class="">Thanks for replying!<br class=""><br class="">When the issue occurs the Log you mentioned is not being printed - it is as if (or maybe it is even what is really happening) the virtual thread is not being scheduled (by its internal scheduler)<br class="">I to a thread dump by executing "kill -3 <pid>" and did not see the VT, will 

`java <pid> Thread.dump_to_file <file>` produce a more detailed view of the living threads?<br class=""><br class="">I will try to run the later EA build</div><br class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, 21 Jul 2024 at 11:14, Alan Bateman <<a href="mailto:Alan.Bateman@oracle.com" target="_blank" class="">Alan.Bateman@oracle.com</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On 21/07/2024 08:05, Yuval Lombard wrote:<br class="">
> Hey team<br class="">
><br class="">
> We are experiencing some weird halting issues when scheduling tasks <br class="">
> with ScheduledExecutorService and VT.<br class="">
><br class="">
> We are observing some near deadlock issues / halting of executors <br class="">
> which we are not able to reproduce when using platform threads<br class="">
><br class="">
> This is one example of a near deadlock situation, where for some <br class="">
> reason virtual threads are starting to execute a task, and not <br class="">
> reaching the finally block nor the catch Throwable block to release <br class="">
> the acquired semaphore permit, reaching to drained semaphore.<br class="">
<br class="">
<br class="">
Is "Scheduling .. task for device" logged? Did you take thread dump with <br class="">
`java <pid> Thread.dump_to_file <file>` and try to identify the virtual <br class="">
thread?<br class="">
<br class="">
One of the later mails mentions 23-loom+4-102 and not seeing JFR events. <br class="">
Can you re-run with the latest EA build (currently 24-loom+2-24) as that <br class="">
has changes for the JFR events that may provide some insight here.<br class="">
<br class="">
-Alan<br class="">
</blockquote></div><br clear="all" class=""><div class=""><br class=""></div><span class="gmail_signature_prefix">-- </span><br class=""><div dir="ltr" class="gmail_signature"><div dir="ltr" class=""><div dir="ltr" class=""><div style="font-size:12.8px" class=""><p class="MsoNormal" style="color:rgb(34,34,34);font-size:small;background-image:initial;background-position:initial;background-repeat:initial">Kind regards,<b style="font-size:13.3333px;color:rgb(0,176,240);font-family:Ebrima" class=""><br class=""><br class="">Yuval Lombard</b><br class=""></p><p class="MsoNormal" style="color:rgb(34,34,34);font-size:small;background-image:initial;background-position:initial;background-repeat:initial"><font color="#3b3838" face="Ebrima" class=""><span style="font-size:12px" class=""><b class="">Lead Software Engineer</b></span></font></p><p class="MsoNormal" style="color:rgb(34,34,34);font-size:small;background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:9pt;font-family:Ebrima;color:rgb(59,56,56)" class="">+972.50.548.0111</span><span style="font-size:9pt;font-family:Ebrima;color:rgb(64,64,64)" class=""><u class=""></u><u class=""></u></span></p><p class="MsoNormal" style="color:rgb(34,34,34);font-size:small;background-image:initial;background-position:initial;background-repeat:initial"><font color="#0000ff" face="Ebrima" class=""><span style="font-size:12px" class=""><a href="mailto:yuval.l@securithings.com" style="color:rgb(17,85,204)" target="_blank" class="">yuval.l@securithings.com</a></span></font></p><p class="MsoNormal" style="color:rgb(34,34,34);font-size:small;background-image:initial;background-position:initial;background-repeat:initial"><span id="cid:ii_l13cyhga0" class=""><logo_black.png></span></p></div></div></div></div>
</blockquote></div><br clear="all" class=""><div class=""><br class=""></div><span class="gmail_signature_prefix">-- </span><br class=""><div dir="ltr" class="gmail_signature"><div dir="ltr" class=""><div dir="ltr" class=""><div style="font-size:12.8px" class=""><p class="MsoNormal" style="color:rgb(34,34,34);font-size:small;background-image:initial;background-position:initial;background-repeat:initial">Kind regards,<b style="font-size:13.3333px;color:rgb(0,176,240);font-family:Ebrima" class=""><br class=""><br class="">Yuval Lombard</b><br class=""></p><p class="MsoNormal" style="color:rgb(34,34,34);font-size:small;background-image:initial;background-position:initial;background-repeat:initial"><font color="#3b3838" face="Ebrima" class=""><span style="font-size:12px" class=""><b class="">Lead Software Engineer</b></span></font></p><p class="MsoNormal" style="color:rgb(34,34,34);font-size:small;background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:9pt;font-family:Ebrima;color:rgb(59,56,56)" class="">+972.50.548.0111</span><span style="font-size:9pt;font-family:Ebrima;color:rgb(64,64,64)" class=""><u class=""></u><u class=""></u></span></p><p class="MsoNormal" style="color:rgb(34,34,34);font-size:small;background-image:initial;background-position:initial;background-repeat:initial"><font color="#0000ff" face="Ebrima" class=""><span style="font-size:12px" class=""><a href="mailto:yuval.l@securithings.com" style="color:rgb(17,85,204)" target="_blank" class="">yuval.l@securithings.com</a></span></font></p><p class="MsoNormal" style="color:rgb(34,34,34);font-size:small;background-image:initial;background-position:initial;background-repeat:initial"><span id="cid:ii_l13cyhga0" class=""><logo_black.png></span></p></div></div></div></div>
<span id="cid:f_lyvkt0n11" class=""><thread_dump.txt></span></div></blockquote></div><br class=""></div></div></div></blockquote></div><br class=""></div></body></html>