RFR: 8333697: C2: Hit MemLimit in PhaseCFG::global_code_motion

[Vladimir Kozlov]

On Mon, 13 Jan 2025 14:49:10 GMT, Roland Westrelin <roland at openjdk.org> wrote:

> I investigated the failure from the `Test.java` that's attached to the
> bug. The failure with this test is only reproducible up to 8334060
> (Implementation of Late Barrier Expansion for G1) so experiments I
> describe here are from the source code for the commit right before it.
> 
> Peak malloc memory usage reported by NMT is: 1.3GB
> 
> `PhaseCFG::global_code_motion()`, when `OptoRegScheduling` is true,
> creates a `PhaseIFG` that's, when initialized, allocates `_adjs`: a
> `maxlrg` array of `IndexSet`s that can contain up to `maxlrg`.
> 
> `maxlrg` in this case is 122839. An `IndexSet` is an array of pointers
> to a 256 bit bitset: one `IndexSet` array needs: 
> 
> 
> 122839 / 256 * 8 = 3832
> 
> 
> and there are of 122839:
> 
> 
> 3832 * 122839 = ~470 MB
> 
> 
> It turns out the `PhaseIFG` object when used from
> `PhaseCFG::global_code_motion()` doesn't even use the `_adjs`
> array. So a patch like:
> 
> 
> diff --git a/src/hotspot/share/opto/chaitin.hpp b/src/hotspot/share/opto/chaitin.hpp
> index cf02deb6019..4e5333bf181 100644
> --- a/src/hotspot/share/opto/chaitin.hpp
> +++ b/src/hotspot/share/opto/chaitin.hpp
> @@ -258,7 +258,7 @@ class PhaseIFG : public Phase {
>    VectorSet *_yanked;
>  
>    PhaseIFG( Arena *arena );
> -  void init( uint maxlrg );
> +  void init( uint maxlrg, bool no_adjs = false );
>  
>    // Add edge between a and b.  Returns true if actually added.
>    int add_edge( uint a, uint b );
> diff --git a/src/hotspot/share/opto/gcm.cpp b/src/hotspot/share/opto/gcm.cpp
> index ebdefe597ff..fefd75a88c5 100644
> --- a/src/hotspot/share/opto/gcm.cpp
> +++ b/src/hotspot/share/opto/gcm.cpp
> @@ -1704,7 +1704,9 @@ void PhaseCFG::global_code_motion() {
>      rm_live.reset_to_mark();           // Reclaim working storage
>      IndexSet::reset_memory(C, &live_arena);
>      uint node_size = regalloc._lrg_map.max_lrg_id();
> -    ifg.init(node_size); // Empty IFG
> +    ifg.init(node_size, true); // Empty IFG
>      regalloc.set_ifg(ifg);
>      regalloc.set_live(live);
>      regalloc.gather_lrg_masks(false);    // Collect LRG masks
> diff --git a/src/hotspot/share/opto/ifg.cpp b/src/hotspot/share/opto/ifg.cpp
> index d12698121b9..e42121c2254 100644
> --- a/src/hotspot/share/opto/ifg.cpp
> +++ b/src/hotspot/share/opto/ifg.cpp
> @@ -42,18 +42,24 @@
>  PhaseIFG::PhaseIFG( Arena *arena ) : Phase(Interference_Graph), _arena(arena) {
>  }
>  
> -void PhaseIFG::init( uint maxlrg ) {
> +void PhaseIFG::init( uint maxlrg, bool no_adjs ) {
>    _maxlrg = maxlrg;
>    _yanked = new (_arena) VectorSet(_arena);
>    _is_square = false;
>    // Make uninitialized adjacency lists
> -  ...

Looks good.

-------------

Marked as reviewed by kvn (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/23075#pullrequestreview-2587575240