RFR: 8345265: Minor improvements for LTO across all compilers [v2]

[Julian Waters]

On Fri, 10 Jan 2025 13:09:21 GMT, Magnus Ihse Bursie <ihse at openjdk.org> wrote:

>>> I'm trying this new version, and I still get a few other warnings and then seem to have a process hang in lto1-ltrans. The switch from `-flto=auto` to `-flto=$(JOBS)` doesn't seem to have helped in that respect.
>> 
>> Turns out I didn't wait long enough.  It does terminate after about 40 minutes, though not successfully.  Instead the
>> build crashes with a failed assert:
>> 
>> #  Internal Error (../../src/hotspot/share/runtime/handles.inline.hpp:76), pid=4017588, tid=4017620
>> #  assert(_thread->is_in_live_stack((address)this)) failed: not on stack?
>> 
>> I've not tried to debug this.  Maybe it's a consequence of one of those problems of bypassing an intentional implicit
>> noinline in our code (by ensuring a function definition is in a different TU from all callers), with LTO breaking that.
>> Or maybe LTO is breaking us in some other way (such as taking advantage of no-UB constraints that aren't found
>> by normal compilation).
>
>> Or maybe LTO is breaking us in some other way (such as taking advantage of no-UB constraints that aren't found
> by normal compilation).
> 
> That seems definitely likely. With LTO the linker has a whole lot of room to try many exciting optimizations. I think it is more than likely that this can trigger some UB holes.
> 
> The question is: should we decouple "fixing LTO build" from "fixing a working LTO JVM"? I tend to think so; that the problem for the build system is to be able to build the product, and if it then crashes during use, it's the problem of the component owners instead. OTOH, this fix is relatively trivial, and if the JVM is DOA so we can't even finish the build, then maybe it makes no point to integrate it until that is fixed.

> @magicus In JBS, I see a long conversation about LTO optimization for libraries aiming to cover all use cases. Maybe it's better to start with something smaller? For example, provide a way to enable it per library and per platform, so it can be incrementally adopted. Initial results for some libraries in the java.desktop look promising.

Hi, at the moment this is HotSpot only; We're unfortunately facing a very severe issue in G1 that can't seem to be solved. I'm currently focusing on making it work for HotSpot before introducing this for the native libraries.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/22464#issuecomment-3416555982