RFR: 8345265: Minor improvements for LTO across all compilers [v2]

[Julian Waters]

On Fri, 10 Jan 2025 10:42:50 GMT, Kim Barrett <kbarrett at openjdk.org> wrote:

>>> Hi, the workaround 'disable lto in g1ParScanThreadState because of special inlining/flattening used there' is removed , why this works now ?
>> 
>> The issue there was the `-Wattribute-warning` warnings that were being generated.  But this change is suppressing
>> those warnings in the LTO link:
>> https://github.com/openjdk/jdk/blame/9d05cb8eff344fea3c6b9a9686b728ec53963978/make/hotspot/lib/JvmFeatures.gmk#L176C11-L176C11
>> Note that won't work with the new portable forbidding mechanism based on `deprecated` attributes.
>> 
>> I'm trying this new version, and I still get a few other warnings and then seem to have a process hang in lto1-ltrans.
>> The switch from `-flto=auto` to `-flto=$(JOBS)` doesn't seem to have helped in that respect.
>
>> I'm trying this new version, and I still get a few other warnings and then seem to have a process hang in lto1-ltrans. The switch from `-flto=auto` to `-flto=$(JOBS)` doesn't seem to have helped in that respect.
> 
> Turns out I didn't wait long enough.  It does terminate after about 40 minutes, though not successfully.  Instead the
> build crashes with a failed assert:
> 
> #  Internal Error (../../src/hotspot/share/runtime/handles.inline.hpp:76), pid=4017588, tid=4017620
> #  assert(_thread->is_in_live_stack((address)this)) failed: not on stack?
> 
> I've not tried to debug this.  Maybe it's a consequence of one of those problems of bypassing an intentional implicit
> noinline in our code (by ensuring a function definition is in a different TU from all callers), with LTO breaking that.
> Or maybe LTO is breaking us in some other way (such as taking advantage of no-UB constraints that aren't found
> by normal compilation).

I've been thinking this over, and I think the way forward to deal with G1 exploding HotSpot's size is to find a static analysis tool and use it to analyze g1ParScanThreadState.cpp, to find which methods it calls are from within its compilation unit, and which come from outside, from another compilation unit. Alternatively, if anyone knows which methods exactly should be flattened (Force inlined) and which ones should _not_ be, we could possibly fast track the process to making LTO viable on gcc higher than 13. My logic for prioritizing making LTO viable over fixing it for now is that there should be a motivating reason to fix LTO to work properly, and if LTO causes such massive bloat on any compiler, then it is no longer viable and the reason for fixing it is removed. Maybe @kimbarrett could provide some insight on which methods should be inlined and which should not be? If that's not possible, then hopefully someone has a static analysis tool suggestion for seeing the call graphs of
  g1ParScanThreadState.cpp

-------------

PR Comment: https://git.openjdk.org/jdk/pull/22464#issuecomment-2755970938