<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<div dir="ltr"><span style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; line-height: normal; font-weight: 400;">Having considered the JBS issue [1], we'd like to propose/discuss some additional options, the thought being that while implementation
can be staggered it might be helpful to consider them now to help prioritize/rethink/reject.</span>
<div><br>
</div>
<div><span style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; line-height: normal; font-weight: 400;">In summary we currently have the "stop" action, stopping is immediate (now) and operationalized via the following mechanisms:</span></div>
<div><span style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; line-height: normal; font-weight: 400;"> • Ctrl-Z/D</span></div>
<div><span style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; line-height: normal; font-weight: 400;"> • System.Exit()</span></div>
<div><span style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; line-height: normal; font-weight: 400;"> • Run to completion (normal termination)</span></div>
<div><span style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; line-height: normal; font-weight: 400;"> • Unhandled exception :)</span></div>
<div><br>
</div>
<div><span style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; line-height: normal; font-weight: 400;">And this new ask in JBS [1] is to</span></div>
<div><span style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; line-height: normal; font-weight: 400;"> • Add a mechanism using 'jcmd' and/or a new Leyden API to action the existing "stop" (now) functionality</span></div>
<div><span style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; line-height: normal; font-weight: 400;"> • Add a variant of the "stop" action where training stops when execution enters a java method (optionally after N invocations);
specify method to stop on via Command Line (-XX), JCmd and/or Leyden API</span></div>
<div><br>
</div>
<div><span style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; line-height: normal; font-weight: 400;">We’d like to propose two additional variants to aid with stopping training runs:</span></div>
<div><span style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; line-height: normal; font-weight: 400;"> • Stop training after some time has elapsed</span></div>
<div><span style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; line-height: normal; font-weight: 400;"> • Stop training when some threshold is met</span></div>
<div><br>
</div>
<div><span style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; line-height: normal; font-weight: 400;">The second point comes from considering the definition of 'startup complete' and 'warmup complete'. Considering the developer is already
engaged in AOT training, we should allow the developer to aid the training by:</span></div>
<div><span style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; line-height: normal; font-weight: 400;"> • Indicating when the application has completed startup (ready to work)</span></div>
<div><span style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; line-height: normal; font-weight: 400;"> • Indicate when the application has completed warmup, either by:</span></div>
<div><span style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; line-height: normal; font-weight: 400;"> ○ Calling an API to indicate warmup is done</span></div>
<div><span style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; line-height: normal; font-weight: 400;"> ○ Calling an API when a chunk of work has completed, passing in the 'duration'; this 'duration' is compared to some specified
'threshold' to generate the 'warmup is complete' event</span></div>
<div><br>
</div>
<div><span style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; line-height: normal; font-weight: 400;">Adding a threshold means that training runs can be of optimal length and can handle changes in the environment and/or code. Coupled
with maybe an "abort training" action we now have a training run that targets a threshold and aborts if not reached in time N.</span></div>
<div><br>
</div>
<div><span style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; line-height: normal; font-weight: 400;">Currently we have the command line, jcmd and a new Leyden API to support training runs. Lastly we'd like to propose a fourth method
that being an MxBean (uses the new Leyden API); this would allow the developer to provide the startup and warmup indicators internally or externally (their choice), and would allow for runtime analysis using bespoke production systems or JMC, and offline analysis
via JFR.</span></div>
<div><br>
</div>
<div><span style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; line-height: normal; font-weight: 400;">Cheers</span></div>
<div><span style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; line-height: normal; font-weight: 400;">Mat</span></div>
<div><br>
</div>
<div><span style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; line-height: normal; font-weight: 400;">[1]
</span><a href="https://bugs.openjdk.org/browse/JDK-8335358"><span style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; line-height: normal; font-weight: 400;">https://bugs.openjdk.org/browse/JDK-8335358</span></a><br>
</div>
<div><br>
</div>
<div id="" class="">
<div class="ms-outlook-mobile-reference-message skipProofing"><span style="">
<div style="font-family: Aptos; font-size: 12pt; text-align: left; border-width: 1pt medium medium; border-style: solid none none; border-color: rgb(181, 196, 223) currentcolor currentcolor; padding: 3pt 0in 0in; color: black;">
<span style="font-weight:bold">From: </span>leyden-dev <leyden-dev-retn@openjdk.org> on behalf of John Rose <john.r.rose@oracle.com><br>
<span style="font-weight:bold">Date: </span>Friday, August 16, 2024 at 4:51 PM<br>
<span style="font-weight:bold">To: </span>ioi.lam@oracle.com <ioi.lam@oracle.com><br>
<span style="font-weight:bold">Cc: </span>leyden-dev@openjdk.org <leyden-dev@openjdk.org><br>
<span style="font-weight:bold">Subject: </span>Re: EA feedback<br>
<br>
</div>
<div style="font-family:sans-serif">
<div class="plaintext" style="white-space:normal">
<p dir="auto">Here’s the way I would prefer to think about a “dump command”.</p>
<p dir="auto">The native way that the JVM represents sequential operations is <br>
the method. Talking about methods is therefore a basic way to <br>
specify a condition for injecting a JVM operation like training <br>
dumps. I would like to figure out a good way to tie the training <br>
dump to the invocation of a method, either a single well-known <br>
method, or to a method specified (on the command line) by the <br>
user.</p>
<p dir="auto">In fact, it feels like a breakpoint-like operation would be a <br>
natural way to view the training dump. You don’t need JVMTI <br>
to get it done; you just need a hack in the VM which parallels <br>
the existing breakpoint mechanism, but special-cases it to <br>
drive a training dump.</p>
<p dir="auto">Given such a foundation, jsig could then inject a call to a <br>
method which is appropriately tied to the dump command.</p>
<p dir="auto">Sketch of implementation:</p>
<p dir="auto">When a method is first linked, a list is checked to see if <br>
it has a dump event tied to it, and a bit is set on the method. <br>
The method’s interpreter entry point might be modified, or <br>
perhaps the interpreter just always checks the bit. On entry <br>
to the method, before the first bytecode, an upcall tells <br>
the VM that it’s time to finish the training run.</p>
<p dir="auto">The compilers also check this bit, of course.</p>
<p dir="auto">There is some method deep in the privates of java.base <br>
that is always treated this way. That’s what jcmd reaches. <br>
There is a command line option which lists more methods <br>
to treat this way, something like the CompileOnly command.</p>
<p dir="auto">As a separate option, the upcall to end the training run <br>
might return (allowing the VM to continue) or just exit.</p>
<p dir="auto">As a separate option, allow the user to specify a count N, <br>
so that the training dump happens only after N “hits” on <br>
any marked method(s).</p>
<p dir="auto">I think all this is useful and flexible.</p>
<p dir="auto">On 13 Aug 2024, at 18:22, ioi.lam@oracle.com wrote:</p>
</div>
<blockquote class="embedded" style="margin: 0px 0px 5px; padding-left: 5px; border-left-width: 2px; border-left-style: solid; border-left-color: rgb(119, 119, 119); color: rgb(119, 119, 119);">
<div id="7CDAA009-A4D5-4C2E-964A-A049E31C31D0">
<p><br>
</p>
<div class="moz-cite-prefix">On 8/13/24 12:42 PM, Ashutosh Mehra wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex">
Being able to trigger assembly/verification via jcmd without exiting, would make this far easier for us to support.<br>
</blockquote>
<div> </div>
There is a proposed enhancement for doing exactly this (and exploring other ways to trigger end of training run); see <a href="https://bugs.openjdk.org/browse/JDK-8335358" originalsrc="https://bugs.openjdk.org/browse/JDK-8335358" shash="WN6viZPYZMIofM3eQxhUWPHoOOIg6i1iT0x2yo+IhoH9y8LZ6uTsVBYL5OvOkuXMpPPobR95YyXhZZix/NrtqhsPmSUg9iHe6EKEYi+kQCBEJWvY67ghs8mf+bUVG1x3AZoaUU3bZ5jPA5lWu+6Ind29zOULpIMIuGZd4LFVCP4=" class="moz-txt-link-freetext">https://bugs.openjdk.org/browse/JDK-8335358</a></div>
</blockquote>
<p><br>
</p>
<p>I am working on a prototype for dumping with jcmd. It will be similar to the existing "jcmd VM.cds statoc_dump" command, except that it will also support the dumping of the AOT cache and profile data.</p>
<p><br>
</p>
<p>Thanks</p>
<p>- Ioi<br>
</p>
<p><br>
</p>
<blockquote type="cite">
<div dir="ltr">
<div><br>
</div>
<div>Thanks,<br clear="all">
<div>
<div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature">
<div dir="ltr">- Ashutosh Mehra</div>
</div>
</div>
<br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Fri, Aug 9, 2024 at 4:38 PM Danny Thomas <<a href="mailto:dannyt@netflix.com" class="moz-txt-link-freetext">dannyt@netflix.com</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex">
<div dir="ltr">
<div>I tried 24-leydenpremain+2-8 on a few internal applications, some quick feedback below (good to see you folks at the JVM LS!).<br>
<br>
If a jar has a Class-Path attribute and one or more of those libraries are explicitly on the classpath, it causes the actual and expected classpath to always differ. This is also the case currently with CDS of course, but this feature is sure to be deployed
far more broadly than CDS is currently, so likely something you want to look at:<br>
<br>
[0.057s][info][class,path] non-existent Class-Path entry lib/failureaccess-1.0.1.jar<br>
[0.057s][info][class,path] opened: lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar<br>
[0.057s][info][class,path] library = lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar<br>
</div>
<div><br>
Startup time when training seems to be on par with ArchiveClassesAtExit in JDK 21, but it's about a 3.5x startup time penalty for one of our typical Spring Boot applications. From a back-to-back run on my machine (AMD EPYC 9R14, 32 cores, 123G, Ubuntu 22.04.4
LTS):</div>
<div><br>
Started App in 7.698 seconds (process running for 8.229)<br>
Started App in 26.247 seconds (process running for 29.262) - w/ CacheDataStore, Training Run<br>
Started App in 4.341 seconds (process running for 4.917) - w/ CacheDataStore, Production Run<br>
<br>
I also got a crash on one attempt, I can't remember what I did to cause this unfortunately:<br>
<br>
Stack: [0x00007f3949ab0000,0x00007f3949bb0000], sp=0x00007f3949bae628, free space=1017k<br>
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)<br>
V [libjvm.so+0x42ca30] ArchiveBuilder::get_buffered_addr(unsigned char*) const+0x40<br>
V [libjvm.so+0xce4aa5] VM_PopulateDumpSharedSpace::doit()+0x395<br>
V [libjvm.so+0x100ae69] VM_Operation::evaluate()+0x109<br>
V [libjvm.so+0x100e348] VMThread::evaluate_operation(VM_Operation*)+0xe8<br>
V [libjvm.so+0x10142fb] VMThread::inner_execute(VM_Operation*)+0x35b<br>
V [libjvm.so+0x101460f] VMThread::run()+0x16f<br>
V [libjvm.so+0xf6e5cf] Thread::call_run()+0x9f<br>
V [libjvm.so+0xd74e13] thread_native_entry(Thread*)+0x183<br>
C [libc.so.6+0x98b07]<br>
<br>
siginfo: si_signo: 11 (SIGSEGV), si_code: 1 (SEGV_MAPERR), si_addr: 0x0000000000000030<br>
<br>
</div>
<div>Thinking ahead to operationalizing AOT, while a single-shot/on-exit workflow is great for iterating locally, requiring the VM to exit makes this more difficult to operationalize at scale:<br>
<ol>
<li>We'll perform training and assembly on test, production canary and production instances on behalf of application owners and handle distribution of the archives. Depending on when we're able to perform a training run, it'll have different benefits. i.e.:
<ol>
<li>Test environment will at least improve startup performance, with a mixed benefit for warm up depending on the kind of traffic they take in test</li><li>If an application uses canary deployments we'll have a full production profile prior to the full production deployment, and all instances will come up hot</li><li>If we reach production with only a test environment profile, we'll perform a training run in production, so instances that scale up following that run will come up hot (completely cold instances for an initial deployment is less of a concern, because we
deploy immutably and get a natural warm-up period while we have 200% capacity online for a cluster)</li></ol>
</li><li>It's currently not a problem if a VM doesn't exit completely due to a dangling non-daemon thread or hung shutdown hook</li></ol>
<div>Being able to trigger assembly/verification via jcmd without exiting, would make this far easier for us to support. If the overhead of the instrumentation for CDS can be avoided, being able to take a snapshot at any time on any VM would be better still,
but that wouldn't be an impediment for us: we'll know that the instance will be used for training at boot time.<br>
<br>
We build nightlies of all the currently active OpenJDK projects, so if you land anything on premain between EA builds that you'd like us to try, let us know!<br>
<br>
Cheers,<br>
Danny</div>
</div>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
<div class="plaintext" style="white-space:normal"></div>
</div>
</span></div>
</div>
</div>
</body>
</html>