<div dir="ltr"><div>Thanks Vladimir and Ioi for the explanation on custom classloaders.</div><div><br></div><div>If I understand correctly, the problem with the order of loading classes is not </div><div>just limited to classloaders with side-effects, but also with clinit-s with side-effects.</div><div><br></div><div>If the program output depends on the order the classes are loaded,</div><div>irrespective of the classloader involved, the order would need to be preserved, right?</div><div>If it is so, then why do custom classloaders pose a problem and not built-in loaders.</div><div>If having init barriers for built-in loaders is sufficient, why isn't that the case for custom loaders?</div><div>What am I missing?<br></div><div><br></div><div>Thanks,</div><div><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr">- Ashutosh Mehra</div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Sep 7, 2023 at 10:08 PM <<a href="mailto:ioi.lam@oracle.com" target="_blank">ioi.lam@oracle.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On 9/6/23 7:45 PM, Vladimir Ivanov wrote:<br>
><br>
>> There were some experiments with PetClinic on our side before and <br>
>> it was<br>
>> noticed that the application relies on custom loaders which <br>
>> aren't fully<br>
>> supported yet.<br>
>><br>
>><br>
>> Can you please elaborate more about the support required for handling <br>
>> custom classloaders.<br>
>> Do they have an impact on AOT code quality or the training data?<br>
><br>
> As of now, there are a number of implementation-specific constraints <br>
> imposed to simplify prototyping and experiments. I'm not sure about <br>
> training data, but code caching is conservatively disabled for all <br>
> classes loaded by custom loaders. Also, some recent CDS-specific <br>
> enhancements in Leyden repository (like class preloading, constant <br>
> pool entries pre-resolution) are disabled for custom loaders. So, as <br>
> of today, code loaded by custom loaders doesn't benefit much from the <br>
> work in Leyden.<br>
><br>
>> Until proper support for custom loaders is there, I suggest to <br>
>> modify<br>
>> the benchmark so it relies only on existing system loaders.<br>
>><br>
>><br>
>> Is there ongoing work to improve the support for custom loaders?<br>
><br>
> I'll let Ioi to comment on the plans about custom loaders. But I <br>
> wouldn't expect much progress on that front in the short term.<br>
><br>
TL/DR; in the near term, we can probably support only a limited set of <br>
custom class loader types, if at all.<br>
<br>
The main problem with custom class loaders is they can have side effects <br>
that are observable at the Java level, so strictly we can't even change <br>
the order of invocations to ClassLoader.loadClass(). Otherwise we may <br>
change the meaning of the program. E.g.,<br>
<br>
public static class MyLoader extends URLClassLoader {<br>
static int x;<br>
public MyLoader(URL[] urls, ClassLoader parent) {<br>
super(urls, parent);<br>
}<br>
<br>
protected Class<?> loadClass(String name, boolean resolve) <br>
throws ClassNotFoundException {<br>
x <<= 1;<br>
x += name.hashCode();<br>
System.out.println(x);<br>
return super.loadClass(name, resolve);<br>
}<br>
}<br>
<br>
If our optimizations change the order of class loading, you will have <br>
unexpected output in stdout.<br>
<br>
So if we want to time shift some computation that uses code loaded by <br>
MyLoader, how can we authentically preserve all the side effects and <br>
replay them in the production run?<br>
<br>
Even if we remove the I/O from the example above, the value MyLoader.x <br>
is still observable by other Java code. So how do we make sure <br>
MyLoader.x has the correct value at every observation point in the <br>
production run?<br>
<br>
For example, assume we have a "constant" expression that we want to fold:<br>
<br>
class A { static int m() { return 1; }<br>
<br>
class B { static int n() { return 2; }<br>
<br>
class C {<br>
<br>
static final int x = A.m() + B.n();<br>
<br>
}<br>
<br>
We not only need to remember "C.x is constant-folded to 3", but also -- <br>
C.<clinit> will first load A and then B.<br>
<br>
This means we have to keep C.<clinit> around (even though it has an <br>
empty body as we removed the computation itself). We must trigger calls <br>
to C.<clinit> at all static references to C, in order to trigger the <br>
loading of A and B in the correct order.<br>
<br>
As a result, all the AOT code that makes a static reference to C must <br>
take a class initialization barrier.<br>
<br>
So you can see how complexity can quickly get out of hand. For example, <br>
we can't constant fold C.x in the AOT code because we need to preserve <br>
the class init barrier.<br>
<br>
If you pre-compute a set of objects that include instances of C, the <br>
situation becomes even worse. You not only need to remember the values <br>
of these objects, but also the sequence of how they were constructed so <br>
you can replay the correct sequence of class loading ....<br>
<br>
==============<br>
<br>
For loaders that are side-effect free at the Java level <br>
(URLClassLoader??), maybe the story is simpler.<br>
<br>
Thanks<br>
<br>
- Ioi<br>
<br>
<br>
>> Another thing that I want to check is the portability of the AOT code.<br>
>> Do we do anything to ensure the AOT code is portable across <br>
>> microarchitectures,<br>
>> that is, it is not tied to the CPU features of the system where the <br>
>> code is being generated.<br>
>> If we bundle the cached code archive in containers, which I expect <br>
>> would be one of the ways to deploy these archives,<br>
>> then the portability would come into picture.<br>
><br>
> We plan to address that in the future, but, so far, JVM does nothing <br>
> special in that respect. Moreover, CDS archive itself imposes <br>
> constraints on supported JVM modes (e.g., compressed oops and <br>
> compressed class pointer modes should match). But users are free to <br>
> specify any additional constraints during training process. For <br>
> example, if shared code archive is generated with -XX:UseAVX=2, there <br>
> won't be any AVX512 instructions present in the archived code which <br>
> makes it safe to run on any AVX2-capable hardware.<br>
><br>
> Best regards,<br>
> Vladimir Ivanov<br>
><br>
>> On Tue, Sep 5, 2023 at 8:41 PM Vladimir Ivanov <br>
>> <<a href="mailto:vladimir.x.ivanov@oracle.com" target="_blank">vladimir.x.ivanov@oracle.com</a> <mailto:<a href="mailto:vladimir.x.ivanov@oracle.com" target="_blank">vladimir.x.ivanov@oracle.com</a>>> <br>
>> wrote:<br>
>><br>
>> Hi Ashutosh,<br>
>><br>
>> Thanks for giving it a try!<br>
>><br>
>> There were some experiments with PetClinic on our side before and it<br>
>> was<br>
>> noticed that the application relies on custom loaders which aren't<br>
>> fully<br>
>> supported yet. It was the main limiting factor for new <br>
>> optimizations.<br>
>> Until proper support for custom loaders is there, I suggest to <br>
>> modify<br>
>> the benchmark so it relies only on existing system loaders.<br>
>><br>
>> Speaking of peak performance, some loss of performance is expected.<br>
>> Cached code is compiled conservatively (e.g., no constant folding <br>
>> for<br>
>> static final fields) so it can be reused in deployment runs. For <br>
>> now,<br>
>> the intended solution is to eventually recompile cached code online<br>
>> with<br>
>> all the optimizations enabled (have to be explicitly enabled<br>
>> -XX:+UseRecompilation). It's a work-in-progress and our experience<br>
>> using<br>
>> it was mixed: recompilation doesn't always fully restore peak<br>
>> performance.<br>
>><br>
>> But assuming that both CDS and cached code archive are underutilized<br>
>> (due to aforementioned reliance on custom loaders), 10% sounds <br>
>> way too<br>
>> big of a difference. I suggest to experiment with different flag<br>
>> combinations (e.g., turning ReplayTraining and LoadCachedCode on and<br>
>> off<br>
>> independently).<br>
>><br>
>> There's additional diagnostic output JVM produces which may help to<br>
>> observe effects from new optimizations during both training and<br>
>> deployment runs:<br>
>><br>
>> * -XX:+PrintCompilation: compilations satisfied from cached code<br>
>> archive are marked w/ "R";<br>
>><br>
>> * -XX:+CITime: prints information about cached code archive <br>
>> usage;<br>
>><br>
>> * -Xlog:init=info: produces additional information about some<br>
>> startup<br>
>> activities<br>
>><br>
>> * -XX:+PrintSharedArchiveAndExit additionally dumps training data<br>
>> and<br>
>> cached code archive info<br>
>><br>
>> * -Xlog:scc*=info and -Xlog:cds*=info print lots of additional<br>
>> information both during training and deployment<br>
>><br>
>> Hope it helps.<br>
>><br>
>> Best regards,<br>
>> Vladimir Ivanov<br>
>><br>
>> On 9/5/23 13:52, Ashutosh Mehra wrote:<br>
>> > Hi,<br>
>> ><br>
>> > We have been interested in persisting the profiling data in <br>
>> the CDS<br>
>> > archive with the intention of improving the application's warmup<br>
>> time.<br>
>> > And now that the premain branch is here that does save profile <br>
>> data<br>
>> > along with AOT, we started playing with the premain branch to<br>
>> understand<br>
>> > its impact on the performance.<br>
>> ><br>
>> > Our setup uses Springboot Petclinic [0] application and the <br>
>> CDS and<br>
>> > shared code archives are generated in a manner similar to this<br>
>> script [1].<br>
>> > Our training run only covers the application startup phase. That<br>
>> means<br>
>> > at each step we start the application and shut it down without<br>
>> putting<br>
>> > any load on it.<br>
>> ><br>
>> > Using the archives thus generated I have done few experiments <br>
>> on my<br>
>> > local system. In these experiments the application is bound to<br>
>> two cpus.<br>
>> > The baseline for comparing the results is the case where the CDS<br>
>> archive<br>
>> > does not have any profiling data and there is no shared code <br>
>> archive.<br>
>> > The "premain" configuration refers to using a shared code archive<br>
>> and a<br>
>> > CDS archive with training data.<br>
>> ><br>
>> > Here are some initial results:<br>
>> ><br>
>> > 1. Startup: It is heartening to see start-up time improve by<br>
>> almost 11%.<br>
>> ><br>
>> > baseline 10.2s<br>
>> > premain 9.1s<br>
>> ><br>
>> > 2. Warmup:<br>
>> > This test measures the warmup time by applying load using 1 <br>
>> jmeter<br>
>> > thread to get an idea of the ramp-up time to reach the peak<br>
>> throughput.<br>
>> > The load is applied for the duration of 300 seconds. The graph<br>
>> [2] for<br>
>> > aot+profiling configuration shows interesting behavior.<br>
>> > In the initial period premain is ramping up faster than the<br>
>> baseline.<br>
>> > Then the slope of the curve for premain reduces significantly <br>
>> and a<br>
>> > couple of dips are also seen. Finally the throughput stabilizes.<br>
>> > It shows a drastic difference in the warmup time of the<br>
>> application when<br>
>> > running with the "premain" config.<br>
>> ><br>
>> > 3. Peak throughput: Last experiment is to measure peak<br>
>> throughput. It<br>
>> > starts with a warm-up phase of 180 seconds using 1 jmeter thread.<br>
>> After<br>
>> > the warmup phase the load is applied with 10 jmeter threads for a<br>
>> > duration of 5 mins.<br>
>> > Last two minutes of throughput is considered for measurement. The<br>
>> graph<br>
>> > [3] for this test shows almost a 10% drop in the throughput<br>
>> compared to<br>
>> > the baseline.<br>
>> ><br>
>> ><br>
>> > I am sure others would have done similar testing. My <br>
>> questions are:<br>
>> ><br>
>> > 1. Are these results on the expected lines?<br>
>> > 2. Are these tests using the CDS and the shared code (or cached<br>
>> code)<br>
>> > archives in the expected manner.<br>
>> > 3. Warmup time with the premain branch looks pretty bad which is<br>
>> > surprising. Is there any trick I missed in my tests? Is there<br>
>> anything<br>
>> > else that needs to be done to get better warmup time?<br>
>> > 4. What is the point of creating a new static archive? <br>
>> Shouldn't the<br>
>> > applications just create the dynamic archive?<br>
>> > 5. I am also wondering if there is any design doc that can be <br>
>> shared<br>
>> > that explains the AOT compilation strategy adopted in the premain<br>
>> branch?<br>
>> ><br>
>> > I have placed my scripts here [4] in case anyone wants to use<br>
>> them to<br>
>> > run these tests (you need to build the Petclinic app before using<br>
>> these<br>
>> > scripts).<br>
>> ><br>
>> > Please feel free to share your thoughts.<br>
>> ><br>
>> > [0] <a href="https://github.com/spring-projects/spring-petclinic" rel="noreferrer" target="_blank">https://github.com/spring-projects/spring-petclinic</a><br>
>> <<a href="https://urldefense.com/v3/__https://github.com/spring-projects/spring-petclinic__;!!ACWV5N9M2RV99hQ!PpMTtwDD2_k-drLo0lLtZ1pybI_zZkMM7RH-TfRvfAEBwceCBkjYqfi7baTqI_r0e5f-qUZenTYkntGa1ZwDrL4$" rel="noreferrer" target="_blank">https://urldefense.com/v3/__https://github.com/spring-projects/spring-petclinic__;!!ACWV5N9M2RV99hQ!PpMTtwDD2_k-drLo0lLtZ1pybI_zZkMM7RH-TfRvfAEBwceCBkjYqfi7baTqI_r0e5f-qUZenTYkntGa1ZwDrL4$</a>><br>
>> > <<a href="https://github.com/spring-projects/spring-petclinic" rel="noreferrer" target="_blank">https://github.com/spring-projects/spring-petclinic</a><br>
>> <<a href="https://urldefense.com/v3/__https://github.com/spring-projects/spring-petclinic__;!!ACWV5N9M2RV99hQ!PpMTtwDD2_k-drLo0lLtZ1pybI_zZkMM7RH-TfRvfAEBwceCBkjYqfi7baTqI_r0e5f-qUZenTYkntGa1ZwDrL4$" rel="noreferrer" target="_blank">https://urldefense.com/v3/__https://github.com/spring-projects/spring-petclinic__;!!ACWV5N9M2RV99hQ!PpMTtwDD2_k-drLo0lLtZ1pybI_zZkMM7RH-TfRvfAEBwceCBkjYqfi7baTqI_r0e5f-qUZenTYkntGa1ZwDrL4$</a>>><br>
>> > [1]<br>
>> ><br>
>> <a href="https://github.com/openjdk/leyden/blob/d960fb15258cc99a1bf7f0b1e94bd8be06605aad/test/hotspot/jtreg/premain/lib/premain-run.sh#L70-L101" rel="noreferrer" target="_blank">https://github.com/openjdk/leyden/blob/d960fb15258cc99a1bf7f0b1e94bd8be06605aad/test/hotspot/jtreg/premain/lib/premain-run.sh#L70-L101</a> <br>
>> <<a href="https://urldefense.com/v3/__https://github.com/openjdk/leyden/blob/d960fb15258cc99a1bf7f0b1e94bd8be06605aad/test/hotspot/jtreg/premain/lib/premain-run.sh*L70-L101__;Iw!!ACWV5N9M2RV99hQ!PpMTtwDD2_k-drLo0lLtZ1pybI_zZkMM7RH-TfRvfAEBwceCBkjYqfi7baTqI_r0e5f-qUZenTYkntGa4rgAjhQ$" rel="noreferrer" target="_blank">https://urldefense.com/v3/__https://github.com/openjdk/leyden/blob/d960fb15258cc99a1bf7f0b1e94bd8be06605aad/test/hotspot/jtreg/premain/lib/premain-run.sh*L70-L101__;Iw!!ACWV5N9M2RV99hQ!PpMTtwDD2_k-drLo0lLtZ1pybI_zZkMM7RH-TfRvfAEBwceCBkjYqfi7baTqI_r0e5f-qUZenTYkntGa4rgAjhQ$</a>> <br>
>> <<a href="https://github.com/openjdk/leyden/blob/d960fb15258cc99a1bf7f0b1e94bd8be06605aad/test/hotspot/jtreg/premain/lib/premain-run.sh#L70-L101" rel="noreferrer" target="_blank">https://github.com/openjdk/leyden/blob/d960fb15258cc99a1bf7f0b1e94bd8be06605aad/test/hotspot/jtreg/premain/lib/premain-run.sh#L70-L101</a> <br>
>> <<a href="https://urldefense.com/v3/__https://github.com/openjdk/leyden/blob/d960fb15258cc99a1bf7f0b1e94bd8be06605aad/test/hotspot/jtreg/premain/lib/premain-run.sh*L70-L101__;Iw!!ACWV5N9M2RV99hQ!PpMTtwDD2_k-drLo0lLtZ1pybI_zZkMM7RH-TfRvfAEBwceCBkjYqfi7baTqI_r0e5f-qUZenTYkntGa4rgAjhQ$" rel="noreferrer" target="_blank">https://urldefense.com/v3/__https://github.com/openjdk/leyden/blob/d960fb15258cc99a1bf7f0b1e94bd8be06605aad/test/hotspot/jtreg/premain/lib/premain-run.sh*L70-L101__;Iw!!ACWV5N9M2RV99hQ!PpMTtwDD2_k-drLo0lLtZ1pybI_zZkMM7RH-TfRvfAEBwceCBkjYqfi7baTqI_r0e5f-qUZenTYkntGa4rgAjhQ$</a>>> <br>
>><br>
>> > [2]<br>
>> ><br>
>> <a href="https://github.com/ashu-mehra/leyden-perf/blob/main/spring/fd82682/tput-t1.svg" rel="noreferrer" target="_blank">https://github.com/ashu-mehra/leyden-perf/blob/main/spring/fd82682/tput-t1.svg</a> <br>
>> <<a href="https://urldefense.com/v3/__https://github.com/ashu-mehra/leyden-perf/blob/main/spring/fd82682/tput-t1.svg__;!!ACWV5N9M2RV99hQ!PpMTtwDD2_k-drLo0lLtZ1pybI_zZkMM7RH-TfRvfAEBwceCBkjYqfi7baTqI_r0e5f-qUZenTYkntGaB0ccgkk$" rel="noreferrer" target="_blank">https://urldefense.com/v3/__https://github.com/ashu-mehra/leyden-perf/blob/main/spring/fd82682/tput-t1.svg__;!!ACWV5N9M2RV99hQ!PpMTtwDD2_k-drLo0lLtZ1pybI_zZkMM7RH-TfRvfAEBwceCBkjYqfi7baTqI_r0e5f-qUZenTYkntGaB0ccgkk$</a>> <br>
>> <<a href="https://github.com/ashu-mehra/leyden-perf/blob/main/spring/fd82682/tput-t1.svg" rel="noreferrer" target="_blank">https://github.com/ashu-mehra/leyden-perf/blob/main/spring/fd82682/tput-t1.svg</a> <br>
>> <<a href="https://urldefense.com/v3/__https://github.com/ashu-mehra/leyden-perf/blob/main/spring/fd82682/tput-t1.svg__;!!ACWV5N9M2RV99hQ!PpMTtwDD2_k-drLo0lLtZ1pybI_zZkMM7RH-TfRvfAEBwceCBkjYqfi7baTqI_r0e5f-qUZenTYkntGaB0ccgkk$" rel="noreferrer" target="_blank">https://urldefense.com/v3/__https://github.com/ashu-mehra/leyden-perf/blob/main/spring/fd82682/tput-t1.svg__;!!ACWV5N9M2RV99hQ!PpMTtwDD2_k-drLo0lLtZ1pybI_zZkMM7RH-TfRvfAEBwceCBkjYqfi7baTqI_r0e5f-qUZenTYkntGaB0ccgkk$</a>>> <br>
>><br>
>> > [3]<br>
>> ><br>
>> <a href="https://github.com/ashu-mehra/leyden-perf/blob/main/spring/fd82682/tput-t10.svg" rel="noreferrer" target="_blank">https://github.com/ashu-mehra/leyden-perf/blob/main/spring/fd82682/tput-t10.svg</a> <br>
>> <<a href="https://urldefense.com/v3/__https://github.com/ashu-mehra/leyden-perf/blob/main/spring/fd82682/tput-t10.svg__;!!ACWV5N9M2RV99hQ!PpMTtwDD2_k-drLo0lLtZ1pybI_zZkMM7RH-TfRvfAEBwceCBkjYqfi7baTqI_r0e5f-qUZenTYkntGagpmiT9g$" rel="noreferrer" target="_blank">https://urldefense.com/v3/__https://github.com/ashu-mehra/leyden-perf/blob/main/spring/fd82682/tput-t10.svg__;!!ACWV5N9M2RV99hQ!PpMTtwDD2_k-drLo0lLtZ1pybI_zZkMM7RH-TfRvfAEBwceCBkjYqfi7baTqI_r0e5f-qUZenTYkntGagpmiT9g$</a>> <br>
>> <<a href="https://github.com/ashu-mehra/leyden-perf/blob/main/spring/fd82682/tput-t10.svg" rel="noreferrer" target="_blank">https://github.com/ashu-mehra/leyden-perf/blob/main/spring/fd82682/tput-t10.svg</a> <br>
>> <<a href="https://urldefense.com/v3/__https://github.com/ashu-mehra/leyden-perf/blob/main/spring/fd82682/tput-t10.svg__;!!ACWV5N9M2RV99hQ!PpMTtwDD2_k-drLo0lLtZ1pybI_zZkMM7RH-TfRvfAEBwceCBkjYqfi7baTqI_r0e5f-qUZenTYkntGagpmiT9g$" rel="noreferrer" target="_blank">https://urldefense.com/v3/__https://github.com/ashu-mehra/leyden-perf/blob/main/spring/fd82682/tput-t10.svg__;!!ACWV5N9M2RV99hQ!PpMTtwDD2_k-drLo0lLtZ1pybI_zZkMM7RH-TfRvfAEBwceCBkjYqfi7baTqI_r0e5f-qUZenTYkntGagpmiT9g$</a>>> <br>
>><br>
>> > [4] <a href="https://github.com/ashu-mehra/leyden-perf" rel="noreferrer" target="_blank">https://github.com/ashu-mehra/leyden-perf</a><br>
>> <<a href="https://urldefense.com/v3/__https://github.com/ashu-mehra/leyden-perf__;!!ACWV5N9M2RV99hQ!PpMTtwDD2_k-drLo0lLtZ1pybI_zZkMM7RH-TfRvfAEBwceCBkjYqfi7baTqI_r0e5f-qUZenTYkntGaY_ZcbT4$" rel="noreferrer" target="_blank">https://urldefense.com/v3/__https://github.com/ashu-mehra/leyden-perf__;!!ACWV5N9M2RV99hQ!PpMTtwDD2_k-drLo0lLtZ1pybI_zZkMM7RH-TfRvfAEBwceCBkjYqfi7baTqI_r0e5f-qUZenTYkntGaY_ZcbT4$</a>><br>
>> > <<a href="https://github.com/ashu-mehra/leyden-perf" rel="noreferrer" target="_blank">https://github.com/ashu-mehra/leyden-perf</a><br>
>> <<a href="https://urldefense.com/v3/__https://github.com/ashu-mehra/leyden-perf__;!!ACWV5N9M2RV99hQ!PpMTtwDD2_k-drLo0lLtZ1pybI_zZkMM7RH-TfRvfAEBwceCBkjYqfi7baTqI_r0e5f-qUZenTYkntGaY_ZcbT4$" rel="noreferrer" target="_blank">https://urldefense.com/v3/__https://github.com/ashu-mehra/leyden-perf__;!!ACWV5N9M2RV99hQ!PpMTtwDD2_k-drLo0lLtZ1pybI_zZkMM7RH-TfRvfAEBwceCBkjYqfi7baTqI_r0e5f-qUZenTYkntGaY_ZcbT4$</a>>><br>
>> ><br>
>> > Thanks,<br>
>> > - Ashutosh Mehra<br>
>><br>
<br>
</blockquote></div>