RFR: Allow young collection to suspend marking in old generation [v3]

[earthling-amzn]

On Tue, 23 Feb 2021 15:25:33 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

>> It looks much better now, hotspot_gc_shenandoah is passing. I'll give it a more in-depth review later. I've tried to run some programs with generational mode, and it survives a few cycles, until it crashes with:
>> 
>> #  Internal Error (/home/rkennke/src/openjdk/shenandoah/src/hotspot/share/gc/shenandoah/shenandoahCollectionSet.cpp:101), pid=6682, tid=6688
>> #  assert(!_heap->get_region(index)->is_cset()) failed: should have been cleared before
>> 
>> Stack: [0x00007fd9529cf000,0x00007fd952acf000],  sp=0x00007fd952acd530,  free space=1017k
>> Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code)
>> V  [libjvm.so+0x1592374]  ShenandoahCollectionSet::clear()+0x1a4
>> V  [libjvm.so+0x15cb5fa]  ShenandoahGeneration::prepare_regions_and_collection_set(bool)+0x20a
>> V  [libjvm.so+0x15ac5bb]  ShenandoahDegenGC::op_degenerated()+0x24b
>> V  [libjvm.so+0x15ad4e0]  ShenandoahDegenGC::entry_degenerated()+0xf0
>> V  [libjvm.so+0x16e391c]  VM_ShenandoahDegeneratedGC::doit()+0x2c
>> V  [libjvm.so+0x19126b7]  VM_Operation::evaluate()+0x187
>> V  [libjvm.so+0x19372fc]  VMThread::inner_execute(VM_Operation*)+0x30c
>> V  [libjvm.so+0x1937c75]  VMThread::loop()+0x255
>> V  [libjvm.so+0x1937f1c]  VMThread::run()+0xcc
>> V  [libjvm.so+0x183baf8]  Thread::call_run()+0xf8
>> V  [libjvm.so+0x13b860e]  thread_native_entry(Thread*)+0x10e
>> 
>> See also attached hs_err.
>> 
>> I'll look into this a bit.
>> 
>> [hs_err_pid6682.log](https://github.com/openjdk/shenandoah/files/6027426/hs_err_pid6682.log)
>
>> It looks much better now, hotspot_gc_shenandoah is passing. I'll give it a more in-depth review later. I've tried to run some programs with generational mode, and it survives a few cycles, until it crashes with:
>> 
>> ```
>> #  Internal Error (/home/rkennke/src/openjdk/shenandoah/src/hotspot/share/gc/shenandoah/shenandoahCollectionSet.cpp:101), pid=6682, tid=6688
>> #  assert(!_heap->get_region(index)->is_cset()) failed: should have been cleared before
>> 
>> Stack: [0x00007fd9529cf000,0x00007fd952acf000],  sp=0x00007fd952acd530,  free space=1017k
>> Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code)
>> V  [libjvm.so+0x1592374]  ShenandoahCollectionSet::clear()+0x1a4
>> V  [libjvm.so+0x15cb5fa]  ShenandoahGeneration::prepare_regions_and_collection_set(bool)+0x20a
>> V  [libjvm.so+0x15ac5bb]  ShenandoahDegenGC::op_degenerated()+0x24b
>> V  [libjvm.so+0x15ad4e0]  ShenandoahDegenGC::entry_degenerated()+0xf0
>> V  [libjvm.so+0x16e391c]  VM_ShenandoahDegeneratedGC::doit()+0x2c
>> V  [libjvm.so+0x19126b7]  VM_Operation::evaluate()+0x187
>> V  [libjvm.so+0x19372fc]  VMThread::inner_execute(VM_Operation*)+0x30c
>> V  [libjvm.so+0x1937c75]  VMThread::loop()+0x255
>> V  [libjvm.so+0x1937f1c]  VMThread::run()+0xcc
>> V  [libjvm.so+0x183baf8]  Thread::call_run()+0xf8
>> V  [libjvm.so+0x13b860e]  thread_native_entry(Thread*)+0x10e
>> ```
>> 
>> See also attached hs_err.
>> 
>> I'll look into this a bit.
>> 
>> [hs_err_pid6682.log](https://github.com/openjdk/shenandoah/files/6027426/hs_err_pid6682.log)
> 
> Looks like that degen into a wrong point, should be _degenerated_updaterefs, instead of _degenerated_outside_cycle.
> 
> Please consider adding generational GC info in event log:
> 
> http://cr.openjdk.java.net/~zgu/shenandoah/gen_event_log.patch
> 
> 
> Thanks,

I'm not sure what a generational specific jtreg test would look like. Would we extend the Whitebox API to tell us the age of objects or regions? I'd like to run the entire hotspot_gc_shenandoah test suite in generational mode while minimizing the number of flags required to maximize the number of passing test (for now, we need `-XX:-RegisterReferences` and `-XX:-UnloadClasses`). I'll figure out what that number of passing tests is today and that will be our proverbial line in the sand. From there we'll ratchet it up. Does that make sense?

Regarding this PR.

Known Issues:

* Full GC's are broken. We understand some of the reasons why (compaction does not update the remembered set). We do not plan to fix this until after the next milestone.
* The `ShenandoahImmediateThreshold` shortcut is disabled (it bypasses some remembered set maintenance during update references). We have a plan and some work in progress to fix this, but it hasn't formally been committed to a milestone yet.
* Degenerated cycles _should_ work. We know they're broken right now and we're working on it. This will be fixed in this PR.
 
Expected Behavior:

With `-XX:-ShenandoahAllowOldMarkingPreemption`:  
  * This _should_ be stable. In this configuration it will run either _young_ or _global_ cycles.
  * Class unloading only happens in global cycles.
  * Concurrent reference processing hasn't been tested in this configuration.

With: `-XX:+ShenandoahAllowOldMarkingPreemption`:
  * It _should_ run complete young cycles and finish marking of the old generation.
  * It _should_ be able to pause old generation marking and run a complete young cycle (and then resume old marking).
  * Class unloading and reference processing must be disabled (will do this when the mode is initialized).
  * Eventually the application will exhaust the heap because it doesn't actually reclaim old regions. 
 
Reclaiming old regions is planned for the next milestone. We will deliver this in smaller incremental changes. 

I will gratefully pull in @zhengyu123 's patch. We have deferred some of the operational improvements while we work on the core features. We expect the JMX beans will need some consideration and we need to add a timing phase for the remembered set scan (at the least).

I hope all that makes sense. Thanks for taking the time to read all that.

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/19