RFR: Allow young collection to suspend marking in old generation [v3]

[Roman Kennke]

On Wed, 24 Feb 2021 01:05:32 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

>> earthling-amzn has updated the pull request incrementally with five additional commits since the last revision:
>> 
>>  - Remove superfluous new line
>>  - Remove redundant is_oop check
>>  - Remove unused parameter, remove unnecessary casting
>>  - Remove debug scaffolding, inline object aging
>>  - Do not allow heuristics to change requested gc cause once a mutator request has been made
>
> Changes requested by zgu (Reviewer).

> I'm not sure what a generational specific jtreg test would look like. Would we extend the Whitebox API to tell us the age of objects or regions? I'd like to run the entire hotspot_gc_shenandoah test suite in generational mode while minimizing the number of flags required to maximize the number of passing test (for now, we need `-XX:-RegisterReferences` and `-XX:-UnloadClasses`). I'll figure out what that number of passing tests is today and that will be our proverbial line in the sand. From there we'll ratchet it up. Does that make sense?

Yes, that sounds sensible. It might even be too much. Pick a few sensible test programs that are known to work with a certain combination of flags, and add those to hotspot_gc_shenandoah_generational configuration. That should be good enough.

> Regarding this PR.
> 
> Known Issues:
> 
>     * Full GC's are broken. We understand some of the reasons why (compaction does not update the remembered set). We do not plan to fix this until after the next milestone.

Ok, good.

>     * The `ShenandoahImmediateThreshold` shortcut is disabled (it bypasses some remembered set maintenance during update references). We have a plan and some work in progress to fix this, but it hasn't formally been committed to a milestone yet.

Ok.

>     * Degenerated cycles _should_ work. We know they're broken right now and we're working on it. This will be fixed in this PR.

Alright. IIRC, you mentioned that degen is problematic when GC is cancelled during e.g. young-gen collection, and we basically need to start from 0 to get the full marking info, or something. If this is the case (I might be mis-remembering it), you could just as well jump into full GC. If, otoh, there is a reasonable way that STW GC could pick up a cancelled young or old GC, then you'll probably need to extend degen machinery to finish the generational young or old cycle. Does that make sense?

> Expected Behavior:
> 
> With `-XX:-ShenandoahAllowOldMarkingPreemption`:
> 
>     * This _should_ be stable. In this configuration it will run either _young_ or _global_ cycles.

Ok, so please test for that with a test program.

>     * Class unloading only happens in global cycles.

This seems reasonable.

>     * Concurrent reference processing hasn't been tested in this configuration.

Sure, disable it for now, and address it in a later milestone.

> With: `-XX:+ShenandoahAllowOldMarkingPreemption`:
> 
>     * It _should_ run complete young cycles and finish marking of the old generation.
> 
>     * It _should_ be able to pause old generation marking and run a complete young cycle (and then resume old marking).

Ok, can you make a test program, that tests with this flag? Not sure if you can test the exact behaviour that you describe, though.

>     * Class unloading and reference processing must be disabled (will do this when the mode is initialized).
> 
>     * Eventually the application will exhaust the heap because it doesn't actually reclaim old regions.

Ok, sure.

> Reclaiming old regions is planned for the next milestone. We will deliver this in smaller incremental changes.

Great!
 
> I will gratefully pull in @zhengyu123 's patch. We have deferred some of the operational improvements while we work on the core features. We expect the JMX beans will need some consideration and we need to add a timing phase for the remembered set scan (at the least).
> 
> I hope all that makes sense. Thanks for taking the time to read all that.

Thanks for explaining that much! That was exactly the information that I needed :-)

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/19