RFR: 8340824: C2: Memory for TypeInterfaces not reclaimed by hashcons()

[Roland Westrelin]

On Tue, 24 Sep 2024 15:53:06 GMT, Roland Westrelin <roland at openjdk.org> wrote:

> The list of interfaces for a `TypeInterfaces` is contained in a `GrowableArray` that's allocated in the type arena. When `hashcons()` deletes a `TypeInterfaces` object because an identical one exists, it can't reclaim memory for the object because it can only free the last thing that was allocated and that's the backing store for the `GrowableArray`, not the `TypeInterfaces` object.
> 
> This patch changes the array of interfaces stored in `TypeInterfaces` into a pointer to a `GrowableArray`. `TypeInterfaces::make` calls `hashcons` with a temporary copy of the array of interfaces allocated in the current thread's resource area. This way if `hascons` deletes the `TypeInterfaces`, it is the last thing allocated in the type arena and memory can be reclaimed.  Memory for the `GrowableArray` is freed as well on return from `TypeInterfaces::make`. If the newly allocated `TypeInterfaces` survives `hashcons` then a permanent array of interfaces is allocated in the type arena and linked from the `TypeInterfaces` object.
> 
> I ran into this issue while working on a fix for a separate issue that causes more Type objects to be created. With that prototype fix, TestScalarReplacementMaxLiveNodes uses 1.4GB of memory at peak. With the patch I propose here on top, memory usage goes down to 150-200 MB which is in line with the peak memory usage for TestScalarReplacementMaxLiveNodes when run with current master.
> 
> When I opened the PR initially, the fix I proposed was to let `GrowableArray` try to reclaim memory from an arena when destroyed. With that patch, some gtests failed when a `GrowableArray` is created by a copy constructor and the backing store for the array is shared. As a consequence, I reconsidered the fix and thought it was safer to go with a fix that only affects `TypeInterfaces`.

Converted back to draft while I investigate gtest failures.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21163#issuecomment-2373908023