Java 21 clinit deadlock

[David Holmes]

On 8/09/2023 12:38 am, Simone Bordet wrote:
> Hello,
> 
> We switched the Jetty builds to Java 21 a while ago, and they randomly
> fail with a hard deadlock during class initialization.
> 
> We tried to understand if we were doing something wrong, but the code
> compiles fine, and at first glance there seems to be nothing wrong
> with it, but it may well be a popular Java puzzler that I am not aware
> of :)
> 
> We did not see these failures with Java 20, and we do not see them with Java 17.
> 
> I was wondering if something changed in Java 21 to cause this deadlock?

I've looked at the dump and there is no sign that the MutableHttpFields 
is actively in use. It suggests to me that class initialization has 
failed but the class state has not been correctly updated and the 
waiters released. There were some changes in JDK 21 about how failures 
in this area were handled, so it is possible I/we got something wrong. 
Is it possible to try running this with additional logging enabled e.g.

-Xlog:class+init=debug -Xlog:exceptions=debug

> I hope this is the right mailing list for this problem.
> If not, please let me know.

It may well be a hotspot-runtime-dev issue

Thanks,
David
-----

> I am an OpenJDK author so I can open an issue about this, but I don't
> have a reproducer, since it seems a race condition happening randomly.
> 
> Below you can find more details about this deadlock.
> 
> Thanks!
> 
> ----
> 
> Taking a thread dump we see one thread in this state:
> 
> java.lang.Thread.State: RUNNABLE
> at org.eclipse.jetty.http.HttpFields.build(org.eclipse.jetty.http at 12.0.2-SNAPSHOT/HttpFields.java:76)
> - waiting on the Class initialization monitor for
> org.eclipse.jetty.http.MutableHttpFields
> at org.eclipse.jetty.http.HttpFields.<clinit>(org.eclipse.jetty.http at 12.0.2-SNAPSHOT/HttpFields.java:67)
> "ForkJoinPool-1-worker-1" #23 [1457161] daemon prio=5 os_prio=0
> cpu=320.77ms elapsed=8870.98s allocated=18414K defined_classes=928
> tid=0x00007f6ec0c6a6f0 nid=1457161 waiting on condition
> [0x00007f6e91ffb000]
> 
> And other threads are in this state (or similar -- the bottom frame is
> different but still triggering the clinit of HttpFields):
> 
> java.lang.Thread.State: RUNNABLE
> at org.eclipse.jetty.http.HttpTester.parseResponse(org.eclipse.jetty.http at 12.0.2-SNAPSHOT/HttpTester.java:274)
> - waiting on the Class initialization monitor for
> org.eclipse.jetty.http.HttpFields
> at org.eclipse.jetty.http.HttpTester.parseResponse(org.eclipse.jetty.http at 12.0.2-SNAPSHOT/HttpTester.java:261)
> "ForkJoinPool-1-worker-2" #24 [1457162] daemon prio=5 os_prio=0
> cpu=136.15ms elapsed=8870.96s allocated=3303K defined_classes=340
> tid=0x00007f6e4c01ddb0 nid=1457162 waiting on condition
> [0x00007f6e91efc000]
> 
> Class hierarchy:
> 
> MutableHttpFields implements HttpFields.Mutable
> HttpFields.Mutable extends HttpFields
> 
> Full thread dump:
> https://gist.github.com/olamy/7e883adcf51b2410337abfa775a79a0b
> 
> Code:
> https://github.com/eclipse/jetty.project/blob/jetty-12.0.1/jetty-core/jetty-http/src/main/java/org/eclipse/jetty/http/HttpFields.java
>