Tiered compilation leads to "unloaded signature class" inlining failures in JRuby

[Charles Oliver Nutter]

On Mon, Aug 31, 2020 at 4:58 PM Vladimir Ivanov
<vladimir.x.ivanov at oracle.com> wrote:
> As I can see with the test case, target method is loaded in a separate
> instance of OneShotClassLoader (and, moreover, I see j.l.String loaded
> there!). So, it doesn't mattter whether a class is loaded in a "parent"
> (?) script at all since they are loaded by separate class loaders.

Ok this might be the clue I needed!

Some years ago, in order to avoid conflicting libraries when running
JRuby inside a web container, one of our contributors (with a focus on
JRuby's embedding use cases) modified JRuby's classloading to use a
"self-first" classloader, which always tries to load the classes from
itself *before* trying the parent classloaders.

As it so happens, this also ends up being an ancestor classloader of
the OneShotClassLoader we use to load compiled code.

So, I patched our logic to not use the self-first loader... and the
original reported example now appears to inline properly!

I will need to do more exploration but I think I now have some idea
why this isn't working. Our self-first classloader is likely
"re-homing" some of the core JDK classes, making them appear like
they're the wrong ones, or not properly resolved, or something. That
in turn prevents inlining because the lifecycle and lineage of those
classes doesn't look right.

We will investigate how to do a safer job of self-first classloading
*only* for resources shipped with JRuby, and avoid using it for JDK
libraries that must always come from the system classloaders.

> It's hard to draw a line here. My feeling is JVM can do a better job
> here (but I haven't worked out all the consequences yet). But if you
> want to get rid of this quirk running on 8u, you definitely better fix
> your app (JRuby).

I am also interested to see if there's a way to improve this at the
JVM level... but on my end I will continue to explore how our unusual
classloading can be adjusted to avoid the issue (and perhaps, to get a
more complete understanding of why the current logic breaks inlining
so badly.)

- Charlie