RFC(S) 8234146: [TESTBUG] compiler/jsr292/ContinuousCallSiteTargetChange.java times out on SPARC

[Reingruber, Richard]

Giving steps how to reproduce the slowdown and test the fixes might be helpful:

* do a release build of the jdk repo without the fix and change into the build directory

* create symbolic link to the jdk repository named jdk-repo

* compile ContinuousCallSiteTargetChange.java

  images/jdk/bin/javac -d classes --source-path jdk-repo/test/lib jdk-repo/test/hotspot/jtreg/compiler/jsr292/ContinuousCallSiteTargetChange.java

* execute PingPongTest with relevant vm options used in ContinuousCallSiteTargetChange::testServer

  time images/jdk/bin/java -server -XX:-TieredCompilation -cp classes 'compiler.jsr292.ContinuousCallSiteTargetChange$PingPongTest' 50

  real    0m2.158s
  user    0m4.068s
  sys     0m0.292s

Linux Only:

    * simulate high load by pinning the test to one cpu

      time taskset 1 images/jdk/bin/java -server -XX:-TieredCompilation -cp classes 'compiler.jsr292.ContinuousCallSiteTargetChange$PingPongTest' 50

      real    0m23.734s
      user    0m23.620s
      sys     0m0.108s

    * Apply http://cr.openjdk.java.net/~rrich/webrevs/8234146/B/webrev.0/jdk.changeset

    * Rebuild and run test again

      time taskset 1 images/jdk/bin/java -server -XX:-TieredCompilation -cp classes 'compiler.jsr292.ContinuousCallSiteTargetChange$PingPongTest' 50

      real    0m2.076s
      user    0m2.024s
      sys     0m0.048s

With higher load (20 cores Xeon(R) CPU E5-2660 v3 @ 2.60GHz):

      wget http://cr.openjdk.java.net/~rrich/webrevs/8234146/CPULoadGenerator.java
      images/jdk/bin/java CPULoadGenerator.java 240

      # without fix
      time images/jdk/bin/java -server -XX:-TieredCompilation -cp classes 'compiler.jsr292.ContinuousCallSiteTargetChange$PingPongTest' 50

      real    1m53.832s
      user    0m30.648s
      sys     0m0.128s

      # with http://cr.openjdk.java.net/~rrich/webrevs/8234146/B/webrev.0/jdk.changeset
      time images/jdk/bin/java -server -XX:-TieredCompilation -cp classes 'compiler.jsr292.ContinuousCallSiteTargetChange$PingPongTest' 50

      real    0m8.241s
      user    0m3.856s
      sys     0m0.100s

Thanks, Richard.

-----Original Message-----
From: hotspot-compiler-dev <hotspot-compiler-dev-bounces at openjdk.java.net> On Behalf Of Reingruber, Richard
Sent: Sonntag, 8. März 2020 22:33
To: hotspot-compiler-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net
Subject: [CAUTION] RFC(S) 8234146: [TESTBUG] compiler/jsr292/ContinuousCallSiteTargetChange.java times out on SPARC

Hi,

I'd like to get comments on two alternative fixes for 

Bug:    https://bugs.openjdk.java.net/browse/JDK-8234146

The timeouts occur when the system is under load. Then the list of dependent nmethods of the dynamic
call site of the PingPongTest gets as long as 1500 nmethods. The list is iterated in
DependencyContext::mark_dependent_nmethods() every time the call site target is changed. And this
takes longer and longer as the list grows. Note that dependents are removed only when they are made
zombie, but this doesn't happen here.

The list of dependent nmethods is virtually empty if the system is idle, because then the target is
changed faster than the method holding the call site gets compiled. When the jit finishes a
compilation it discards the new nmethod, because the call site target was changed already.

To reproduce long runs I'd suggest to run ContinuousCallSiteTargetChange$PingPongTest manually
taking the command from ContinuousCallSiteTargetChange.jtr of a previous run. Under load [1] the
test will take about 10x as long as without load. 30s vs 3s on my linux x86_64 box for the server
config. With the fixes below the test will take under load approximately as long as without load.

There are 2 possible fixes I'd like to get comments for:

Option A: Force the NMethodSweeper using the whitebox api.
Webrev A: http://cr.openjdk.java.net/~rrich/webrevs/8234146/A/webrev.0/

Option B: Specialize DependencyContext::mark_dependent_nmethods() for call site target changes,
i.e. flag list elements that are marked for deoptimization. Upon the next target change only new
dependents are marked for deoptimization. The list iteration is ended as soon as an element is
encountered that is already flagged.
Webrev B: http://cr.openjdk.java.net/~rrich/webrevs/8234146/B/webrev.0/

Personally I like B better, because I think it is simple and straight forward: if the target
changes, then all dependents need to be invalidated. By construction of the list we can stop the
invalidation if we find an element that was invalidated before. This specialization is of course
less generic, and there might be new types of call site dependencies for which it is illegal, but
currently I can't think of any.

Curious about your comments...

Thanks,
Richard.

[1] To simulate load you can pin the jvm process to one cpu (e.g. on Linux with "taskset 1 java ...")
    or
    wget http://cr.openjdk.java.net/~rrich/webrevs/8234146/CPULoadGenerator.java && java CPULoadGenerator.java N
    where N is the number of CPUs of your box