RFR(S): 8008301: G1: guarantee(satb_mq_set.completed_buffers_num() == 0) failure

[John Cuthbertson]

Hi Everyone,

Here's another one associated with overflowing the marking stack. The 
webrev can be found at:

http://cr.openjdk.java.net/~johnc/8008301/webrev.0/

Summary:
In this case the marking stack overflows during the actual remark task - 
when the individual marking tasks are draining the completed SATB 
buffers. As a result we exit the remark task (because of the overflow) 
with some legitimately unprocessed completed SATB buffers and trip the 
guarantee.

The main part of the fix is between line 2575 and 2579 where I relax the 
guarantee to allow for the overflow. Additionally I've also added a heap 
verification for when we restart marking due to overflow. I added it as 
part of the debugging and I think it makes sense to leave it in.

The change between lines 2408 and 2416 is to just skip reference 
processing. While testing I was running into the hang reported in 
8009536 and added this to work around it. An equivalent change will be 
pushed as part of the changes for 8009940.

Testing:
The failing test case in a loop (with and without instrumentation and 
VerifyDuringGC)
The GC test suite with a small mark stack size (2K, 4K)
The Lucene tests for 8009536 to specifically test the verification after 
overflow.

Thanks,

JohnC