[External] : Re: Hello from JRuby and here's another crash

Tue Jul 9 21:00:30 UTC 2024

On 7/8/24 9:04 AM, Charles Oliver Nutter wrote:
> The patched build does indeed work, and cuts JRuby's base startup time 
> by 50%! Bravo!
>
> $ time jruby -e 1
> jruby -e 1  3.53s user 0.14s system 276% cpu 1.326 total
> $ time jruby -J-XX:CacheDataStore=jruby.cds -e 1
> jruby -J-XX:CacheDataStore=jruby.cds -e 1  2.38s user 0.19s system 
> 385% cpu 0.667 total
>
> This is a very promising start and the largest single improvement we 
> have seen. Comparison with our previous champion, C1:
>
> $ time jruby --dev -e 1
> jruby --dev -e 1  1.22s user 0.09s system 142% cpu 0.922 total
>
> Interestingly, when I try to use the previously trained CDS dump (via 
> 100.times { org.jruby.Ruby.newInstance }) with a command line that 
> forces C1 (--dev flag passes -XX:TieredStopAtLevel=1) I get a 
> different crash: 
> https://gist.github.com/headius/d3138efa52eed9c678414c8cfe08af27 
> <https://urldefense.com/v3/__https://gist.github.com/headius/d3138efa52eed9c678414c8cfe08af27__;!!ACWV5N9M2RV99hQ!O1UAeB7nD1zEm6bMPw7oDGNPEmvkkh6jK5YVmZJVKQP6Ugi5tSURjYzIpMvg0XmXhF6oaTom1wRsn5A$>
>
I filed https://bugs.openjdk.org/browse/JDK-8336033 . It looks like a 
simple book keeping issue in the compiler code.

> I'm wondering what else I can do to try to train the CDS dump better, 
> so that a larger command (jruby -S gem list) sees more improvement 
> (this loads a bunch more Ruby code into the interpreter):
>
> $ time jruby -S gem list > /dev/null
> jruby -S gem list > /dev/null  6.41s user 0.26s system 236% cpu 2.821 
> total
> $ time jruby -J-XX:CacheDataStore=jruby.cds -S gem list > /dev/null
> jruby -J-XX:CacheDataStore=jruby.cds -S gem list > /dev/null  5.84s 
> user 0.32s system 294% cpu 2.089 total
>

Does this load a lot of dynamically generated classes? The current set 
of Leyden optimizations are mostly for classes loaded in the built-in 
loaders (boot/platform/app).

I wonder how much of the time is spend in one-time activity. If you run 
the above inside a loop (2x, 4x, 8x, 16x, etc), how does it scale?

I think if we see a big drop between 1x vs 2x, then perhaps the current 
set of Leyden optimizations can help (as long as most of the time is 
spent in the built-in loaders).

In any case, dynamic language runtimes like jruby probably have very 
different start-up characteristics than, say, your typical "micro 
service" Java app. More analysis is needed.

Thanks

- Ioi

> - Charlie
>
> On Thu, Jul 4, 2024 at 11:50 PM <ioi.lam at oracle.com> wrote:
>
>     Hi Charlie,
>
>     Thanks for the bug report.
>
>     It turns out some Leyden optimizations can't handle the module
>     options specified by jruby, such as these:
>
>         --add-opens=java.base/java.nio.channels=org.jruby.dist
>         --module-path=/xxxx/jruby-9.4.8.0/lib/jruby.jar
>
>     I filed https://bugs.openjdk.org/browse/JDK-8335735
>
>     I'll work on a proper fix. Meanwhile, I have a work-around in this
>     branch
>
>     https://github.com/iklam/jdk/tree/work-around-8335735-jruby-crash-due-to-lack-of-module-support
>     <https://urldefense.com/v3/__https://github.com/iklam/jdk/tree/work-around-8335735-jruby-crash-due-to-lack-of-module-support__;!!ACWV5N9M2RV99hQ!O1UAeB7nD1zEm6bMPw7oDGNPEmvkkh6jK5YVmZJVKQP6Ugi5tSURjYzIpMvg0XmXhF6oaTomY2MJITU$>
>
>     If you build a JDK from this source branch, you can see the
>     following improvements:
>
>     # [1] No optimizations
>     $ time env JAVA_HOME=$MYJAVA
>     ~/Downloads/ruby/jruby-9.4.8.0/bin/jruby \
>         -e '10.times { org.jruby.Ruby.newInstance }'
>     real   0m3.215s
>     user   0m14.868s
>     sys    0m0.373s
>
>
>     # [2] Leyden
>     $ time env JAVA_HOME=$TBP0 ~/Downloads/ruby/jruby-9.4.8.0/bin/jruby \
>         -J-XX:CacheDataStore=jruby.cds \
>         -e '10.times { org.jruby.Ruby.newInstance }'
>     real   0m1.880s
>     user   0m9.001s
>     sys    0m0.268s
>
>
>     # [3] Compare with CDS in the JDK mainline
>     $ env JAVA_HOME=$MYJAVA ~/Downloads/ruby/jruby-9.4.8.0/bin/jruby \
>          -J-XX:DumpLoadedClassList=jruby.classlist \
>          -e '10.times { org.jruby.Ruby.newInstance }'
>     $ $MYJAVA/bin/java -Xshare:dump \
>     --module-path=/home/iklam/Downloads/ruby/jruby-9.4.8.0/lib/jruby.jar \
>         -XX:SharedClassListFile=jruby.classlist \
>         -XX:SharedArchiveFile=jruby.jsa -Xlog:cds
>     $ time env JAVA_HOME=$MYJAVA
>     ~/Downloads/ruby/jruby-9.4.8.0/bin/jruby \
>         -J-XX:SharedArchiveFile=jruby.jsa \
>         -e '10.times { org.jruby.Ruby.newInstance }'
>     real   0m2.628s
>     user   0m13.007s
>     sys    0m0.428s
>
>     Please try it out and let us know if you run into other problems.
>
>     Thanks
>
>     - Ioi
>
>
>     On 7/4/24 1:06 PM, Charles Oliver Nutter wrote:
>>     Hello friends! Long time lurker, first time poster.
>>
>>     Like others I was excited to hear that the first EA of Leyden had
>>     dropped. Sadly, I have another crash to report.
>>
>>     I see there are two other crashes reported, but mine appears
>>     different (ClassPrelinker::is_indy_resolution_deterministic):
>>
>>     https://gist.github.com/headius/79c6460ed55c1d80c82e1c0209897e24
>>     <https://urldefense.com/v3/__https://gist.github.com/headius/79c6460ed55c1d80c82e1c0209897e24__;!!ACWV5N9M2RV99hQ!O1UAeB7nD1zEm6bMPw7oDGNPEmvkkh6jK5YVmZJVKQP6Ugi5tSURjYzIpMvg0XmXhF6oaTomO4igb9c$>
>>
>>     Perhaps unsurprisingly, the additional flags provided by Vladimir
>>     Kozlov (-XX:+UnlockDiagnosticVMOptions
>>     -XX:-ReduceAllocationMerges) did not appear to change the result.
>>
>>     This is with JRuby's minimally invokedynamic-based mode, which
>>     has given us the shortest startup time in the past (second only
>>     to disabling tiers 2-4 and staying in C1).
>>
>>     I am on MacOS AArch64 Sonoma 14.5, testing against JRuby master,
>>     but reproduction should be as easy as downloading the JRuby
>>     binary tarball, unpacking, and running bin/jruby with the command
>>     line above.
>>
>>     https://www.jruby.org/download
>>     <https://urldefense.com/v3/__https://www.jruby.org/download__;!!ACWV5N9M2RV99hQ!O1UAeB7nD1zEm6bMPw7oDGNPEmvkkh6jK5YVmZJVKQP6Ugi5tSURjYzIpMvg0XmXhF6oaTom4BSxN20$>
>>
>>     I am eager to work with Leyden folks to investigate issues, and I
>>     am planning to be at JVMLS this year to discuss collaborating more!
>>
>>     - Charlie
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/leyden-dev/attachments/20240709/bbba94d5/attachment.htm>