ScopedValue performance declines as number of dynamic bindings increases

[robert engels]

This doesn’t look correct:

        for (int i = 0; i < 100_000; i++) {
            bh.consume(MISSING.orElse("hello world"));
        }

why are you looping 100k times there?

> On Jun 24, 2025, at 11:44 AM, Johan Sjolen <johan.sjolen at oracle.com> wrote:
> 
> Hi Aleksey,
> 
> I re-wrote my benchmarks to use JMH and I used Blackhole::consume instead of the S.o.println. The entire point of the println was to signal to the compiler 'this has to happen'. I kept the 100k iterations, as I didn't want the setup of the 100 dynamic bindings to dominate (or perhaps they are? Not sure).
> 
> I ended up with the following results:
> 
> 
> (This is the test of just doing MISSING.orElse)
> Result "org.openjdk.bench.java.lang.ScopedValues.ScopedValuesAreFasT":
>  815142.530 ±(99.9%) 17764.324 ns/op [Average]
>  (min, avg, max) = (790345.594, 815142.530, 835566.013), stdev = 11750.000
>  CI (99.9%): [797378.206, 832906.854] (assumes normal distribution)
> 
> (This is the test with MISSING.orElse within the 100 dynamic bindings)
> Result "org.openjdk.bench.java.lang.ScopedValues.ScopedValuesAreSlow":
>  18847997.203 ±(99.9%) 587930.099 ns/op [Average]
>  (min, avg, max) = (18176298.357, 18847997.203, 19523500.462), stdev = 388879.358
>  CI (99.9%): [18260067.104, 19435927.301] (assumes normal distribution)
> 
> 
> That's 815142 ns/op vs 18847997ns/op, a pretty large diff still ('contested' case takes 23x longer).
> 
> Am I messing up my benchmarks once again, but in some different way?
> 
> Here's the branch and source: https://github.com/openjdk/jdk/compare/master...jdksjolen:jdk:scopedvaluejmhs
> 
> I just put the benchmark into the pre-existing ScopedValues.java, should just be a copy and paste and be done if you've got a JDK build set up.
> The tests are run on linux-aarch64.
> 
> ________________________________________
> From: Aleksey Shipilev <shipilev at amazon.de>
> Sent: Tuesday, June 24, 2025 18:08
> To: Johan Sjolen; Robert Engels
> Cc: loom-dev at openjdk.org
> Subject: Re: ScopedValue performance declines as number of dynamic bindings increases
> 
> On 24.06.25 18:01, Johan Sjolen wrote:
>> Oooh, that could explain the issue completely. I didn't think about that at all.
>> The issue basically disappears when compiling first, let me see what happens with a JMH benchmark as well.
> 
> I predict JMH would show the same thing.
> 
> javac is not very fast when cold; this is also why we use it as go-to example in AOT cache studies.
> I would also expect compiling deeply nested lambdas (like in ScopedTest.java) makes the javac code
> fairly recursive, which again plays interestingly with JIT compilation. It is still
> actionable/interesting to figure out what can be improved there. Looks fairly weird we spend >3
> seconds compiling a seemingly "easy" source. Submit a tools/javac RFE, let compiler folks have fun
> with it?
> 
> -Aleksey
> 
> 
> 
> Amazon Web Services Development Center Germany GmbH
> Tamara-Danz-Str. 13
> 10243 Berlin
> Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
> Eingetragen am Amtsgericht Charlottenburg unter HRB 257764 B
> Sitz: Berlin
> Ust-ID: DE 365 538 597

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/loom-dev/attachments/20250624/5fc54cf3/attachment-0001.htm>