RFR: 8253361: Shenandoah: runtime barrier does not honor ShenandoahSelfFixing flag

[Zhengyu Gu]

ShenandoahSelfFixing flag is currently only honored by C2, not runtime barrier. Runtime barrier should check the check
too.

-------------

Commit messages:
 - Honor ShenandoahSelfFixing flag in runtime barrier
 - Tighten recusive reserveration handling
 - Fix trailing whitespace
 - Restore assertion
 - JDK-8252921
 - JDK-8252921

Changes: https://git.openjdk.java.net/jdk/pull/254/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=254&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8253361
  Stats: 34 lines in 4 files changed: 18 ins; 13 del; 3 mod
  Patch: https://git.openjdk.java.net/jdk/pull/254.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/254/head:pull/254

PR: https://git.openjdk.java.net/jdk/pull/254