[8u] backport RFR (XS) 8071534: assert(!failing()) failed: Must not have pending failure. Reason is: out of memory

[Vladimir Kozlov]

8u60 backport request.  Changes were pushed into jdk9 last week, no 
problems were found since then. Changes are applied to 8u cleanly.

https://bugs.openjdk.java.net/browse/JDK-8071534

http://cr.openjdk.java.net/~kvn/8071534/webrev

jdk9 changeset:
http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/25454f0d37d3

Thanks
Vladimir