ScopedValue performance declines as number of dynamic bindings increases

[Johan Sjolen]

Hi Aleksey,

I re-wrote my benchmarks to use JMH and I used Blackhole::consume instead of the S.o.println. The entire point of the println was to signal to the compiler 'this has to happen'. I kept the 100k iterations, as I didn't want the setup of the 100 dynamic bindings to dominate (or perhaps they are? Not sure).

I ended up with the following results:


(This is the test of just doing MISSING.orElse)
Result "org.openjdk.bench.java.lang.ScopedValues.ScopedValuesAreFasT":
  815142.530 ±(99.9%) 17764.324 ns/op [Average]
  (min, avg, max) = (790345.594, 815142.530, 835566.013), stdev = 11750.000
  CI (99.9%): [797378.206, 832906.854] (assumes normal distribution)

(This is the test with MISSING.orElse within the 100 dynamic bindings)
Result "org.openjdk.bench.java.lang.ScopedValues.ScopedValuesAreSlow":
  18847997.203 ±(99.9%) 587930.099 ns/op [Average]
  (min, avg, max) = (18176298.357, 18847997.203, 19523500.462), stdev = 388879.358
  CI (99.9%): [18260067.104, 19435927.301] (assumes normal distribution)


That's 815142 ns/op vs 18847997ns/op, a pretty large diff still ('contested' case takes 23x longer).

Am I messing up my benchmarks once again, but in some different way?

Here's the branch and source: https://github.com/openjdk/jdk/compare/master...jdksjolen:jdk:scopedvaluejmhs

I just put the benchmark into the pre-existing ScopedValues.java, should just be a copy and paste and be done if you've got a JDK build set up.
The tests are run on linux-aarch64.

________________________________________
From: Aleksey Shipilev <shipilev at amazon.de>
Sent: Tuesday, June 24, 2025 18:08
To: Johan Sjolen; Robert Engels
Cc: loom-dev at openjdk.org
Subject: Re: ScopedValue performance declines as number of dynamic bindings increases

On 24.06.25 18:01, Johan Sjolen wrote:
> Oooh, that could explain the issue completely. I didn't think about that at all.
> The issue basically disappears when compiling first, let me see what happens with a JMH benchmark as well.

I predict JMH would show the same thing.

javac is not very fast when cold; this is also why we use it as go-to example in AOT cache studies.
I would also expect compiling deeply nested lambdas (like in ScopedTest.java) makes the javac code
fairly recursive, which again plays interestingly with JIT compilation. It is still
actionable/interesting to figure out what can be improved there. Looks fairly weird we spend >3
seconds compiling a seemingly "easy" source. Submit a tools/javac RFE, let compiler folks have fun
with it?

-Aleksey



Amazon Web Services Development Center Germany GmbH
Tamara-Danz-Str. 13
10243 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 257764 B
Sitz: Berlin
Ust-ID: DE 365 538 597