RFR: 8373203: Genshen: Non-strong reference leak in old gen [v8]

[William Kemper]

On Fri, 9 Jan 2026 17:52:35 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> The generational mode for Shenandoah will collect _referents_ for the generation being collected. For example, if we have a young reference pointing to an old referent, that young reference will be processed after we finish marking the old generation. This presents a problem for discovery.
>>   
>> When the young mark _encounters_ a young reference with an old referent, it cannot _discover_ it because old marking hasn't finished. However, if it does not discover it, the old referent will be strongly marked. This, in turn, will prevent the old generation from clearing the referent (if it even reaches it again during old marking).
>>   
>> To solve this, we let young reference processing discover the old reference by having it use the old generation reference processor to do so. This means the old reference processor can have a discovered list that contains young weak references. If any of these young references reside in a region that is collected, old reference processing will crash when it processes such a reference. Therefore, we add a method `heal_discovered_lists` to traverse the discovered lists after young evacuation is complete. The method will replace any forwarded entries in the discovered list with the forwardee.
>> 
>> This PR also extends whitebox testing support for Shenandoah, giving us the ability to trigger young/old collections and interrogate some properties of heaps and regions.
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 23 commits:
> 
>  - Merge remote-tracking branch 'jdk/master' into fix-old-reference-processing
>  - Merge remote-tracking branch 'jdk/master' into fix-old-reference-processing
>  - Heal discovered lists for any young collection coincides with old marking
>  - Configure thread local mark closure on delegated old reference processor
>  - Merge remote-tracking branch 'jdk/master' into fix-old-reference-processing
>  - Fix idiosyncratic white space in whitebox
>    
>    Co-authored-by: Stefan Karlsson <stefan.karlsson at oracle.com>
>  - Sort includes
>  - Heal old discovered lists in parallel
>  - Fix comment
>  - Factor duplicate code into shared method
>  - ... and 13 more: https://git.openjdk.org/jdk/compare/f5fa9e40...abccb8b6

Consider a situation where an old and young reference both point to an old referent:

young_reference       old_reference
              \           /
               \         /
               old_referent

Genshen "bootstraps" old marking with a young collection. During young generation marking, oop iteration will "encounter" `young_reference`, and ask our reference processor to "discover" it. If the reference processor _discovers_ `old_referent`, then the reference processor is responsible for it and the mark thread will _not_ mark through the referent. Conversely, if the reference processor does _not_ discover the referent, then oop iteration will mark the referent and trace through it. This behavior is baked pretty deep into oop iteration and any changes in this code would affect all collectors.

The young reference processor cannot _discover_ old referents because it will not have complete mark information when it comes time to process the discovered list. By not discovering the old referent, the mark thread will mark and trace through the old referent. Following the bootstrap, when old marking encounters `old_reference`, it sees that `old_referent` is already strongly marked and so similarly does not _discover_ it. We could continue to wait for `young_reference` to die or be promoted, but you can see that a single `young_reference` to any `old_referent` can prevent the reference from being cleared.

This PR has the young reference processor defer discovery to the old reference processor. This is simple enough, except that the young references now live in the old reference processor's discovered list. This list runs through the heap so these young references can be evacuated. This requires us to update references for the old processor's discovered list. Note that the list already updates the card table when young references are placed on the discovered list, so they become roots for subsequent young collections by virtue of being in the remembered set.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/28810#issuecomment-3740189396