Fwd: Class initialization pinning deadlock with Log4j 2
Danny Thomas
dannyt at netflix.com
Mon Nov 3 02:44:52 UTC 2025
Hi folks,
We saw a clinit deadlock with Log4j 2 and virtual threads recently. It's
common for libraries to use a mix of static and instance field loggers, so
seems particularly vulnerable to this kind of deadlock at startup.
I don't recall reading what you have planned for clinit pinning, so thought
I'd pass on this potentially common real-world deadlock and ask what you
have in mind for addressing this limitation.
Cheers,
Danny
Pinned virtual thread holding the clinit lock:
#265 "_internalWorkflowEvaluationQueue_2-workflow-evaluation-worker-0"
virtual
java.base/jdk.internal.misc.Unsafe.park(Native Method)
java.base/java.lang.VirtualThread.parkOnCarrierThread(VirtualThread.java:675)
java.base/java.lang.VirtualThread.park(VirtualThread.java:607)
java.base/java.lang.System$2.parkVirtualThread(System.java:2643)
java.base/jdk.internal.misc.VirtualThreads.park(VirtualThreads.java:54)
java.base/java.util.concurrent.locks.LockSupport.park(LockSupport.java:219)
java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:754)
java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1079)
java.base/java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:738)
org.apache.logging.log4j.core.util.internal.InternalLoggerRegistry.getLogger(InternalLoggerRegistry.java:113)
org.apache.logging.log4j.core.util.internal.InternalLoggerRegistry.computeIfAbsent(InternalLoggerRegistry.java:196)
org.apache.logging.log4j.core.LoggerContext.getLogger(LoggerContext.java:588)
org.apache.logging.log4j.core.LoggerContext.getLogger(LoggerContext.java:561)
org.apache.logging.log4j.core.LoggerContext.getLogger(LoggerContext.java:71)
org.apache.commons.logging.LogAdapter$Log4jLog.<init>(LogAdapter.java:159)
org.apache.commons.logging.LogAdapter$Log4jAdapter.createLog(LogAdapter.java:113)
org.apache.commons.logging.LogAdapter.createLog(LogAdapter.java:95)
org.apache.commons.logging.LogFactory.getLog(LogFactory.java:67)
org.apache.commons.logging.LogFactory.getLog(LogFactory.java:59)
org.springframework.data.redis.connection.convert.Converters.<clinit>(Converters.java:69)
org.springframework.data.redis.connection.DefaultStringRedisConnection.<init>(DefaultStringRedisConnection.java:187)
org.springframework.data.redis.connection.DefaultStringRedisConnection.<init>(DefaultStringRedisConnection.java:171)
Unparked but unmounted virtual thread that's next in line for read lock:
#369 "_internalWorkflowEvaluationQueue_1-workflow-evaluation-worker-11"
virtual
java.base/java.lang.VirtualThread.park(VirtualThread.java:596)
java.base/java.lang.System$2.parkVirtualThread(System.java:2643)
java.base/jdk.internal.misc.VirtualThreads.park(VirtualThreads.java:54)
java.base/java.util.concurrent.locks.LockSupport.park(LockSupport.java:219)
java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:754)
java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1079)
java.base/java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:738)
org.apache.logging.log4j.core.util.internal.InternalLoggerRegistry.getLogger(InternalLoggerRegistry.java:113)
org.apache.logging.log4j.core.util.internal.InternalLoggerRegistry.computeIfAbsent(InternalLoggerRegistry.java:196)
org.apache.logging.log4j.core.LoggerContext.getLogger(LoggerContext.java:588)
org.apache.logging.log4j.core.LoggerContext.getLogger(LoggerContext.java:561)
org.apache.logging.log4j.core.LoggerContext.getLogger(LoggerContext.java:71)
org.apache.commons.logging.LogAdapter$Log4jLog.<init>(LogAdapter.java:159)
org.apache.commons.logging.LogAdapter$Log4jAdapter.createLog(LogAdapter.java:113)
org.apache.commons.logging.LogAdapter.createLog(LogAdapter.java:95)
org.apache.commons.logging.LogFactory.getLog(LogFactory.java:67)
org.apache.commons.logging.LogFactory.getLog(LogFactory.java:59)
org.springframework.data.redis.connection.AbstractRedisConnection.<init>(AbstractRedisConnection.java:38)
With the core workers all taken up by threads blocked on clinit, preventing
the continuation for the unparked thread from running:
"ForkJoinPool-2-worker-1" #266 [1606] daemon prio=5 os_prio=0 cpu=233.52ms
elapsed=2547.44s tid=0x00007fe9f38b4000 nid=1606 waiting on condition
[0x00007fe9ccde6000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base at 21.0.7.0.101/Native Method)
- parking to wait for <0x0000040009f6d730> (a
java.util.concurrent.ForkJoinPool)
--
"ForkJoinPool-2-worker-2" #267 [1607] daemon prio=5 os_prio=0 cpu=393.88ms
elapsed=2547.44s tid=0x00007fe9cdbf8000 [0x00007fe9cc465000]
Carrying virtual thread #357
at jdk.internal.vm.Continuation.run(java.base at 21.0.7.0.101
/Continuation.java:251)
- waiting on the Class initialization monitor for
org.springframework.data.redis.connection.convert.Converters
--
"ForkJoinPool-2-worker-3" #271 [1610] daemon prio=5 os_prio=0 cpu=308.69ms
elapsed=2547.43s tid=0x00007fe9cdbf8700 [0x00007fe9cc2e2000]
Carrying virtual thread #367
at jdk.internal.vm.Continuation.run(java.base at 21.0.7.0.101
/Continuation.java:251)
- waiting on the Class initialization monitor for
org.springframework.data.redis.connection.convert.Converters
--
"ForkJoinPool-2-worker-4" #272 [1611] daemon prio=5 os_prio=0 cpu=428.92ms
elapsed=2547.43s tid=0x00007fe9f5efdf00 [0x00007fe9cc261000]
Carrying virtual thread #297
at jdk.internal.vm.Continuation.run(java.base at 21.0.7.0.101
/Continuation.java:251)
- waiting on the Class initialization monitor for
org.springframework.data.redis.connection.convert.Converters
--
"ForkJoinPool-2-worker-5" #273 [1612] daemon prio=5 os_prio=0 cpu=347.62ms
elapsed=2547.42s tid=0x00007fe9cbef7000 nid=1612 waiting on condition
[0x00007fe9cbee0000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base at 21.0.7.0.101/Native Method)
- parking to wait for <0x0000040009f6d730> (a
java.util.concurrent.ForkJoinPool)
--
"ForkJoinPool-2-worker-6" #316 [1651] daemon prio=5 os_prio=0 cpu=286.41ms
elapsed=2547.04s tid=0x00007fe9db2f2000 [0x00007fe9ca6c5000]
Carrying virtual thread #265
at jdk.internal.vm.Continuation.run(java.base at 21.0.7.0.101
/Continuation.java:251)
at java.lang.VirtualThread.runContinuation(java.base at 21.0.7.0.101
/VirtualThread.java:245)
--
"ForkJoinPool-2-worker-7" #317 [1652] daemon prio=5 os_prio=0 cpu=415.34ms
elapsed=2547.03s tid=0x00007fe9db2f2700 nid=1652 waiting on condition
[0x00007fe9ca645000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base at 21.0.7.0.101/Native Method)
- parking to wait for <0x0000040009f6d730> (a
java.util.concurrent.ForkJoinPool)
--
"ForkJoinPool-2-worker-8" #359 [1675] daemon prio=5 os_prio=0 cpu=108.86ms
elapsed=2546.41s tid=0x00007fe9f58fff00 [0x00007fe9c9a69000]
Carrying virtual thread #270
at jdk.internal.vm.Continuation.run(java.base at 21.0.7.0.101
/Continuation.java:251)
- waiting on the Class initialization monitor for
org.springframework.data.redis.connection.convert.Converters
--
"ForkJoinPool-2-worker-9" #360 [1676] daemon prio=5 os_prio=0 cpu=199.56ms
elapsed=2546.41s tid=0x00007fea12c4ed00 [0x00007fe9c99e8000]
Carrying virtual thread #363
at jdk.internal.vm.Continuation.run(java.base at 21.0.7.0.101
/Continuation.java:251)
- waiting on the Class initialization monitor for
org.springframework.data.redis.connection.convert.Converters
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/loom-dev/attachments/20251103/4a7c3924/attachment.htm>
More information about the loom-dev
mailing list