RFR: 8315916: assert(C->live_nodes() <= C->max_node_limit()) failed: Live Node limit exceeded [v4]

[Christian Hagedorn]

On Mon, 21 Oct 2024 02:39:00 GMT, Dhamoder Nalla <dhanalla at openjdk.org> wrote:

>> In the debug build, the assert is triggered during the parsing (before Code_Gen). In the Release build, however, the compilation bails out at `Compile::check_node_count()` during the code generation phase and completes execution without any issues.
>> 
>> When I commented out the assert(C->live_nodes() <= C->max_node_limit()), both the debug and release builds exhibited the same behavior: the compilation bails out during code_gen after building the ideal graph with more than 80K nodes.
>> 
>> The proposed fix will check the live node count and bail out during compilation while building the graph for scalarization of the elements in the array when the live node count crosses the limit of 80K, instead of unnecessarily building the entire graph and bailing out in code_gen.
>
> Dhamoder Nalla has updated the pull request incrementally with one additional commit since the last revision:
> 
>   change CRLF to LF

src/hotspot/share/opto/macro.cpp line 821:

> 819:     // If scalarize operation is adding too many nodes, bail out
> 820:     if (C->check_node_count(300, "out of nodes while scalarizing object")) {
> 821:       return nullptr;

Would a bailout from this scalarization be enough or do we really require to record the method as non-compilable (which is done with `check_node_count()`? In the latter case, we could also try something like "recompilation without EA" as done, for example, here (i.e. `retry_no_escape_analysis`):

https://github.com/openjdk/jdk/blob/37cfaa8deb4cc15864bb6dc2c8a87fc97cff2f0d/src/hotspot/share/opto/escape.cpp#L3858-L3866

I also suggest to use the `NodeLimitFudgeFactor` instead of `300` to have it controllable.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20504#discussion_r1812029925