RFR: 8157153: TestStressRSetCoarsening fails with OOM

[Stefan Johansson]

Hi,

Please review this testfix for:
https://bugs.openjdk.java.net/browse/JDK-8157153

Webrev:
http://cr.openjdk.java.net/~sjohanss/8157153/hotspot.00/

Summary:
The test sometimes fail due to running out of memory and the reason is 
that the test expects to be able to fit 2 objects of the size 
HeapRegionSize/2 into one heap region. This assumption is ok in theory, 
but if any object at all is allocated between the two allocations the 
second one will end up in a different region. The test calculates how 
many regions it should fill to get 90% of the heap used and then 
allocated twice this number of objects. In many cases this will work 
since nothing else is going on, but in some cases some other subsystem 
might start allocating objects, and this destroys the assumptions 
leading to hitting an OOME.

The fix simply adds a second check to the allocation loop to stop as 
soon as the free memory drops below 10% of the heap, this also requires 
some extra limiting of the indexes used.

Testing:
* Local testing as well as runs through RBT.

Thanks,
Stefan