RFR: 8156500: deadlock provoked by new stress test com/sun/jdi/OomDebugTest.java

[Mandy Chung]

> On Jun 28, 2016, at 3:43 PM, Kim Barrett <kim.barrett at oracle.com> wrote:
> 
> Updated webrevs:
> 
> Full:
> http://cr.openjdk.java.net/~kbarrett/8156500/jdk.01/
> http://cr.openjdk.java.net/~kbarrett/8156500/hotspot.01/
> 

The VM change does simplify the implementation a lot.  Does/should the reference processing and enqueue them in the pending list at a safepoint? Or does the enqueuing happen while other Java threads are running except that it holds Heap_lock?

My main concern is that the ReferenceHandler thread now has to make a VM upcall to dequeue a reference object for every iteration.  I believe this is what the original implementation tried to avoid.  Would it be feasible for popReferencePendingList to be intrinsified?  If it is infeasible, we should think through other alternatives for example like Peter’s suggestion  to get the entire pending list at a time.

Comments on Reference.java

 143     /* Atomically pop the next pending reference.  If should_wait is
 144      * true and there are no pending references, block until the
 145      * pending reference list becomes non-empty.  The GC adds
 146      * references to the list, and notifies waiting threads when
 147      * references are added.
 148      */
 149     private static native Reference<Object> popReferencePendingList(boolean should_wait);

The parameter name “should_wait” - I would avoid using the word “should” and `_`.  What about “await”?

The spec needs to specify:

What happens if the thread calling this method is interrupted?  What does it return?  Does it throw InterruptedException?  What happens to the thread’s interrupted status?

This method is significant and can you add @param, @return, @throws to make it a proper javadoc.   

This change is small but the implication is significant.  JDK-7103238 would be more appropriate for this change than JDK-8156400 and its synopsis that does not highlight what the changeset is about.

 
> Still investigating the initialization order for core exceptions.
> 

This would be good to understand what’s going on there.

Mandy