RFC: JWarmup precompile java hot methods at application startup

[Ivan Krylov]

Dear all,

I would like to summarize discussions around JWarmup at JVMLS 2019. 
There were many so I will try to collapse into a smaller email.

For the purposes of this discussion I don't represent neither my 
previous employer Azul Systems, maker of ReadyNow, nor my current employer.

The great java warmup problem is a well understood one. This year there 
is probably not a single java conference without a talk on this subject.
Multiple approaches have been developed to tackle this issue, at 
different layers, in the vanilla OpenJDK and in the alternative JDKs. 
These include but not limited to CDS/AppCDS, OpenJ9's JIT Cache / Zing 
Compile Stashing, JWarmUp / ReadyNow, jaotc / graalvm native images, 
etc. etc. Some of this technologies tackle additional aspects like 
memory footprint.

There is an excellent talk [1] at #JVMLS2019 from Mark Stoodly from IBM 
with an overview of a landscape of technologies.

There seems to be an understanding that there is no 1-size-fits-all 
solution. This thread is specifically about an approach of saving and 
using recorded application profiles in subsequent runs of the same 
application.

There are two working solutions that I am aware of: JWarmup in the 
AlibabaJDK and ReadyNow in Zing. There is nothing in the OpenJDK master 
of this variety. At this point there is enough evidence, enough 
production data to say that the approach is beneficial for many large 
applications and microservices. Alibaba folks have proposed a JEP [2] 
and a patch [3] to add this functionality. The discussion seems to 
indicate (and correct me if I am wrong) that as it stands now this patch 
has a low chance of being accepted.

First, reviewers miss a detailed description. What is the user story? 
Can profiles be uses across multiple jvm version? Across multiple 
platforms and ISAs? Across changing library code? Is it a self-updating 
profile or write once kind of? In case of profile on a NFS who's 
responsibility is to ensure proper locking on a profile?

Looking at the internals - there are even more questions? Are compiles 
conservative or speculative? How method profile data is decoded and 
populated to the structures consumes by ci layer /jit ? Are class loads 
infused by a profile reader or done entirely by the application? What 
about class linking and initialization?

What is a success metric? What statistics or monitoring can be used? 
What would constitute regression testing?

On the implementation side It seems to me that the main problem is that 
the protocol between JVM and a profile provider isn't specified. We need 
an API perhaps similar in its spirit to the GC Interface [4]. This would 
allow not to multiplex the regular runtime business and the logic of the 
profile provider. The file format for persisted data also need some 
specification including versioning.

In my view a feature like JWarmup is much bigger than a one patch. 
Bigger than one JEP.

As for the logistics: I have been away from the hotspot aliases for so 
long that I am not sure what is the current practice now. May be 
starting a thread with design discussions is good enough or may be we 
need a dedicated openjdk project and a dedicated mail list. From my side 
I can share my experience and knowledge from building such an assisted 
warmup machinery. There are many lessons that we learned. Some are 
outlined at this excellent talk from Douglas Hawkins[5]. If you scroll 
to the end you'll see why we ended doing essentially two passes of the 
persisted profile application.

Thanks,

Ivan

1 - https://www.youtube.com/watch?v=gx8DVVFPkcQ

2 - https://openjdk.java.net/jeps/8203832

3 - http://cr.openjdk.java.net/~minqi/8220692/webrev-03/

4 - https://openjdk.java.net/jeps/304

5 - https://www.youtube.com/watch?v=AGyctnc1DTA&t=1027s

On 25/04/2019 19:07, yumin qi wrote:

> Hi,
>
>    Apart from comments from compiler professionals can I have comments from
> runtime either? The changes mostly land in runtime area.
>
> Thanks
> Yumin
>
> On Tue, Apr 16, 2019 at 11:27 AM yumin qi <yumin.qi at gmail.com> wrote:
>
>> HI,
>>
>>    Did anyone have comments for this version?
>>
>> Thanks
>> Yumin
>>
>> On Tue, Apr 9, 2019 at 10:36 AM yumin qi <yumin.qi at gmail.com> wrote:
>>
>>> Alan,
>>>    Thanks! Updated in same link:
>>>    http://cr.openjdk.java.net/~minqi/8220692/webrev-02/
>>>
>>>    Removed non-boot loader branch in nativeLookup.cpp.
>>>    Added jdk.jwarmup to boot loader list in make/common/Modules.gmk.
>>>    Tested again to make sure the new changes.
>>>
>>>    Thanks
>>>    Yumin
>>>
>>>
>>> On Tue, Apr 9, 2019 at 4:48 AM Alan Bateman <Alan.Bateman at oracle.com>
>>> wrote:
>>>
>>>> On 09/04/2019 07:10, yumin qi wrote:
>>>>>    Now the registerNatives is found when it looks up for native entry
>>>>> in lookupNative.cpp. I thought the class JWarmUp will be loaded by
>>>>> boot loader like Unsafe or WhiteBox, but I was wrong, it is loaded by
>>>>> app class loader so logic for obtaining its native entry put in both
>>>>> cases, boot loader and non boot loaders.
>>>>>
>>>> make/common/Modules.gmk is where BOOT_MODULES is defined with the list
>>>> of modules mapped to the boot loader.
>>>>
>>>> -Alan
>>>>