RFR 8224179: Shenandoah: CTW test failures with traversal GC

[Zhengyu Gu]

The failures were due to nested 
nmethod::oops_do_marking_prologue()/nmethod::oops_do_marking_epilogue() 
calls. This patch is split from early JDK-8224115.

Bug: https://bugs.openjdk.java.net/browse/JDK-8224179
Webrev: http://cr.openjdk.java.net/~zgu/JDK-8224179/webrev.00/

Test:
   hotspot_gc_shenandoah (fastdebug and release)
   applications/ctw/modules/ fastdebug.


Thanks,

-Zhengyu