RFR: Allow young collection to suspend marking in old generation [v3]

[Roman Kennke]

On Tue, 23 Feb 2021 08:00:31 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Quick scan
>
> It looks much better now, hotspot_gc_shenandoah is passing. I'll give it a more in-depth review later. I've tried to run some programs with generational mode, and it survives a few cycles, until it crashes with:
> 
> #  Internal Error (/home/rkennke/src/openjdk/shenandoah/src/hotspot/share/gc/shenandoah/shenandoahCollectionSet.cpp:101), pid=6682, tid=6688
> #  assert(!_heap->get_region(index)->is_cset()) failed: should have been cleared before
> 
> Stack: [0x00007fd9529cf000,0x00007fd952acf000],  sp=0x00007fd952acd530,  free space=1017k
> Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code)
> V  [libjvm.so+0x1592374]  ShenandoahCollectionSet::clear()+0x1a4
> V  [libjvm.so+0x15cb5fa]  ShenandoahGeneration::prepare_regions_and_collection_set(bool)+0x20a
> V  [libjvm.so+0x15ac5bb]  ShenandoahDegenGC::op_degenerated()+0x24b
> V  [libjvm.so+0x15ad4e0]  ShenandoahDegenGC::entry_degenerated()+0xf0
> V  [libjvm.so+0x16e391c]  VM_ShenandoahDegeneratedGC::doit()+0x2c
> V  [libjvm.so+0x19126b7]  VM_Operation::evaluate()+0x187
> V  [libjvm.so+0x19372fc]  VMThread::inner_execute(VM_Operation*)+0x30c
> V  [libjvm.so+0x1937c75]  VMThread::loop()+0x255
> V  [libjvm.so+0x1937f1c]  VMThread::run()+0xcc
> V  [libjvm.so+0x183baf8]  Thread::call_run()+0xf8
> V  [libjvm.so+0x13b860e]  thread_native_entry(Thread*)+0x10e
> 
> See also attached hs_err.
> 
> I'll look into this a bit.
> 
> [hs_err_pid6682.log](https://github.com/openjdk/shenandoah/files/6027426/hs_err_pid6682.log)

Hmm, let's step back a little.
Ideally, we would maintain a working state at each milestone or even each commit. Obviously, this is not really possible during (early) development of a large feature like generational. But, as far as I can tell, the code works to some degree, with the exception of certain features (e.g. ref-processing, class-unloading, full-gc, etc). And we can also define what sort of features or behaviours *should* work. Would it be possible to captures this in test programs (ideally in the form of jtreg tests)? This has numerous advantages:
- Ability to continuously verify that what used to work still works after changes
- Documentation: For example, run configuration (e.g. flags) document how to run programs with relevant features disabled
- Testcases can be much narrower in scope than full programs
- We can use WhiteBox API to have better control of the JVM (we newly added GC breakpoint support to be able to 'step through' GC phases in test cases, which can be very useful, see e.g. JDK-8262122)

BTW, the flag to disable ref processing is -XX:-RegisterReferences (somewhat unintuitively)

So, looking at the current state, both 'hotspot_gc_shenandoah' and 'hotspot_gc_shenandoah_generational' are passing, which means we're good in theory. However, I'd think that you added new features and behaviours that go beyond what is currently tested in hotspot_gc_shenandoah_generational, so maybe add a few testcases there?

Also, I am not quite sure which of the not-working-quite-well aspects you intend to fix for this PR (maybe degen and full-GC?), and which you intend to defer to later milestones?

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/19