RFR: 8312322: GenShen: Cancelled GCs may become stuck in self-cancellation loop

[William Kemper]

If an old GC is cancelled during its preparation phase, it may not acknowledge the cancellation. This can lead to a situation where the subsequent, interrupting young cycle observes its own cancellation request and does nothing. Which, in turn leads to another request to run a young cycle and so on, ad infinitum.

-------------

Commit messages:
 - Do not transition to bootstrapping if preparation was cancelled
 - Improve handling of cancelled preparation for old gc

Changes: https://git.openjdk.org/shenandoah/pull/298/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=298&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8312322
  Stats: 22 lines in 1 file changed: 4 ins; 10 del; 8 mod
  Patch: https://git.openjdk.org/shenandoah/pull/298.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/298/head:pull/298

PR: https://git.openjdk.org/shenandoah/pull/298