Reflection vs MethodHandle performance in Oracle JDK 7u7

[Ashwin Jayaprakash]

People seem to be skeptical about the micro benchmarks I posted. This is
why I used Caliper (http://code.google.com/p/caliper/).

Caliper runs each test multiple times (indicated by "trials"). Each trial
itself runs the code in a loop with "reps", which loops for 10s of millions
of times. So, I'm uploading the *log files* for your verification. Look for
statements like "running trial with 138668464 reps" in the files. Caliper
does run the test enough times to let the JIT warm up.

*JDK 8:*
 0% Scenario{vm=java, trial=0, benchmark=Reflect, tune=-server -Xmx96M
-Xmx96M, tier=-XX:+TieredCompilation} 19.88 ns; ?=0.90 ns @ 10 trials
10% Scenario{vm=java, trial=0, benchmark=Handle, tune=-server -Xmx96M
-Xmx96M, tier=-XX:+TieredCompilation} 16.06 ns; ?=0.09 ns @ 3 trials
20% Scenario{vm=java, trial=0, benchmark=Direct, tune=-server -Xmx96M
-Xmx96M, tier=-XX:+TieredCompilation} 7.41 ns; ?=0.01 ns @ 3 trials
30% Scenario{vm=java, trial=0, benchmark=Iface, tune=-server -Xmx96M
-Xmx96M, tier=-XX:+TieredCompilation} 7.27 ns; ?=0.06 ns @ 3 trials
40% Scenario{vm=java, trial=0, benchmark=Static, tune=-server -Xmx96M
-Xmx96M, tier=-XX:+TieredCompilation} 7.33 ns; ?=0.01 ns @ 3 trials
50% Scenario{vm=java, trial=0, benchmark=Reflect, tune=-server -Xmx96M
-Xmx96M, tier=-XX:-TieredCompilation} 18.55 ns; ?=1.88 ns @ 10 trials
60% Scenario{vm=java, trial=0, benchmark=Handle, tune=-server -Xmx96M
-Xmx96M, tier=-XX:-TieredCompilation} 15.13 ns; ?=0.06 ns @ 3 trials
70% Scenario{vm=java, trial=0, benchmark=Direct, tune=-server -Xmx96M
-Xmx96M, tier=-XX:-TieredCompilation} 7.21 ns; ?=0.07 ns @ 4 trials
80% Scenario{vm=java, trial=0, benchmark=Iface, tune=-server -Xmx96M
-Xmx96M, tier=-XX:-TieredCompilation} 7.23 ns; ?=0.07 ns @ 9 trials
90% Scenario{vm=java, trial=0, benchmark=Static, tune=-server -Xmx96M
-Xmx96M, tier=-XX:-TieredCompilation} 7.20 ns; ?=0.02 ns @ 3 trials

benchmark                   tier    ns linear runtime
  Reflect -XX:+TieredCompilation 19.88 ==============================
  Reflect -XX:-TieredCompilation 18.55 ===========================
   Handle -XX:+TieredCompilation 16.06 ========================
   Handle -XX:-TieredCompilation 15.13 ======================
   Direct -XX:+TieredCompilation  7.41 ===========
   Direct -XX:-TieredCompilation  7.21 ==========
    Iface -XX:+TieredCompilation  7.27 ==========
    Iface -XX:-TieredCompilation  7.23 ==========
   Static -XX:+TieredCompilation  7.33 ===========
   Static -XX:-TieredCompilation  7.20 ==========

vm: java
trial: 0
tune: -server -Xmx96M -Xmx96M

Writing results to C:\temp\jdk_8_ea_b59.log


*JDK 7:*
 0% Scenario{vm=java, trial=0, benchmark=Reflect, tune=-server -Xmx96M
-Xmx96M, tier=-XX:+TieredCompilation} 16.40 ns; ?=0.16 ns @ 7 trials
10% Scenario{vm=java, trial=0, benchmark=Handle, tune=-server -Xmx96M
-Xmx96M, tier=-XX:+TieredCompilation} 20.89 ns; ?=0.64 ns @ 10 trials
20% Scenario{vm=java, trial=0, benchmark=Direct, tune=-server -Xmx96M
-Xmx96M, tier=-XX:+TieredCompilation} 4.81 ns; ?=0.04 ns @ 3 trials
30% Scenario{vm=java, trial=0, benchmark=Iface, tune=-server -Xmx96M
-Xmx96M, tier=-XX:+TieredCompilation} 4.86 ns; ?=0.05 ns @ 3 trials
40% Scenario{vm=java, trial=0, benchmark=Static, tune=-server -Xmx96M
-Xmx96M, tier=-XX:+TieredCompilation} 4.84 ns; ?=0.04 ns @ 3 trials
50% Scenario{vm=java, trial=0, benchmark=Reflect, tune=-server -Xmx96M
-Xmx96M, tier=-XX:-TieredCompilation} 16.55 ns; ?=0.15 ns @ 4 trials
60% Scenario{vm=java, trial=0, benchmark=Handle, tune=-server -Xmx96M
-Xmx96M, tier=-XX:-TieredCompilation} 20.96 ns; ?=0.59 ns @ 10 trials
70% Scenario{vm=java, trial=0, benchmark=Direct, tune=-server -Xmx96M
-Xmx96M, tier=-XX:-TieredCompilation} 4.79 ns; ?=0.01 ns @ 3 trials
80% Scenario{vm=java, trial=0, benchmark=Iface, tune=-server -Xmx96M
-Xmx96M, tier=-XX:-TieredCompilation} 4.80 ns; ?=0.03 ns @ 3 trials
90% Scenario{vm=java, trial=0, benchmark=Static, tune=-server -Xmx96M
-Xmx96M, tier=-XX:-TieredCompilation} 4.85 ns; ?=0.05 ns @ 7 trials

benchmark                   tier    ns linear runtime
  Reflect -XX:+TieredCompilation 16.40 =======================
  Reflect -XX:-TieredCompilation 16.55 =======================
   Handle -XX:+TieredCompilation 20.89 =============================
   Handle -XX:-TieredCompilation 20.96 ==============================
   Direct -XX:+TieredCompilation  4.81 ======
   Direct -XX:-TieredCompilation  4.79 ======
    Iface -XX:+TieredCompilation  4.86 ======
    Iface -XX:-TieredCompilation  4.80 ======
   Static -XX:+TieredCompilation  4.84 ======
   Static -XX:-TieredCompilation  4.85 ======

vm: java
trial: 0
tune: -server -Xmx96M -Xmx96M

Writing results to C:\temp\jdk_7u7.log


Regards,
Ashwin.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/mlvm-dev/attachments/20121015/ecca8db3/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: jdk_7u7.log
Type: application/octet-stream
Size: 46005 bytes
Desc: not available
Url : http://mail.openjdk.java.net/pipermail/mlvm-dev/attachments/20121015/ecca8db3/jdk_7u7-0001.log 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: jdk_8_ea_b59.log
Type: application/octet-stream
Size: 44817 bytes
Desc: not available
Url : http://mail.openjdk.java.net/pipermail/mlvm-dev/attachments/20121015/ecca8db3/jdk_8_ea_b59-0001.log