Impossible (?) code path resulting in IllegalStateException on jdk14+

[David Holmes]

Hi Dawid,

This appears to be a bug in AbstractQueuedSynchronizer and FJP 
interaction, so cc'ing core-libs as this is not a hotspot issue. Also 
cc'ing Martin and Doug as this is a j.u.c bug.

Cheers,
David

On 12/12/2020 12:56 am, Dawid Weiss wrote:
> So, I know what this is. Sort of.
> 
> This isn't a compiler error (sigh of relief). It's a kind of interplay
> between parallel streams (which uses the common ForkJoinPool) and the
> queue's blocking operations.
> 
> In our code streams use an op closure which sends items to an
> ArrayBlockingQueue. This queue is being drained by a separate thread.
> Everything works up until a certain moment when this happens on
> queue.put() -- I've modified JDK 16 source code and recompiled it to
> see what's happening:
> 
> Suppressed: java.util.concurrent.RejectedExecutionException: Thread
> limit exceeded replacing blocked worker
> at java.base/java.util.concurrent.ForkJoinPool.tryCompensate(ForkJoinPool.java:1579)
> at java.base/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3124)
> at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1614)
> at java.base/java.util.concurrent.ArrayBlockingQueue.put(ArrayBlockingQueue.java:371)
> 
> This exception causes the try-finally block in ArrayBlockingQueue to
> hit the finally block unexpectedly (without the attempt to re-acquire
> the lock!), eventually leading to the original odd exception I
> reported:
> 
> Caused by: java.lang.IllegalMonitorStateException
> at java.base/java.util.concurrent.locks.ReentrantLock$Sync.tryRelease(ReentrantLock.java:175)
> at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1006)
> at java.base/java.util.concurrent.locks.ReentrantLock.unlock(ReentrantLock.java:494)
> at java.base/java.util.concurrent.ArrayBlockingQueue.put(ArrayBlockingQueue.java:378)
> 
> This is then propagated up to the caller of queue.put()
> 
> A full simplified repro (without streams) is here:
> https://gist.github.com/dweiss/4787b7aa503ef5702e94d73178c3c842
> 
> When you run it under JDK14+, it'll result in:
> 
> java.lang.IllegalMonitorStateException
>          at java.base/java.util.concurrent.locks.ReentrantLock$Sync.tryRelease(ReentrantLock.java:175)
> ...
> 
> This code works on JDK11, by the way. What I find a bit
> counterintuitive is that the original test (code) doesn't make any
> explicit use of ForkJoinPool - it just collects items from a parallel
> stream and involves throwing random exceptions from ops on that
> stream... This was a bit unexpected.
> 
> Dawid
> 
> 
> On Thu, Dec 10, 2020 at 5:25 PM Dawid Weiss <dawid.weiss at gmail.com> wrote:
>>
>> Hello,
>>
>> I'm scratching my head again over a bug we encountered in randomized
>> tests. The code is quite complex so before I start to try to isolate I
>> thought I'd ask if it rings a bell for anybody.
>>
>> The bug is reproducible for certain seeds but happens only after some
>> VM warmup - the same test is executed a few dozen times, then the
>> problem starts showing up. This only happens on jdk 14 and jdk 15
>> (didn't test on jdk 16 branch). Looks like something related to OSR/
>> compilation races.
>>
>> In the end, we get this exception:
>>
>> java.lang.IllegalMonitorStateException
>> at java.base/java.util.concurrent.locks.ReentrantLock$Sync.tryRelease(ReentrantLock.java:175)
>> at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1006)
>> at java.base/java.util.concurrent.locks.ReentrantLock.unlock(ReentrantLock.java:494)
>> at java.base/java.util.concurrent.ArrayBlockingQueue.put(ArrayBlockingQueue.java:373)
>> [stack omitted]
>>
>> This doesn't seem possible from Java code alone -- it's this snippet
>> in ArrayBlockingQueue:
>>
>> lock.lockInterruptibly();
>> try {
>>      while (count == items.length)
>>          notFull.await();
>>      enqueue(e);
>> } finally {
>>     lock.unlock();     // <<< bam...
>> }
>>
>> If the code entered the lock-protected block it should never throw
>> IllegalMonitorStateException, right?
>>
>> I'll start digging in spare moments but hints at how to isolate this/
>> verify what this can be (other than bisecting git repo) would be very
>> welcome!
>>
>> Dawid