Explain why my two benchmarks are performing differently than I expect?

David Karr davidmichaelkarr at gmail.com
Mon Oct 1 18:42:49 UTC 2018


I recently noticed some code in the codebase I work that looks like this
(not with a hardcoded value):

        DecimalFormat df = new DecimalFormat("##0.00");
        new BigDecimal(df.format(47.0));

I was pretty sure that was an inefficient way to do this, which I thought
should be more like:

        new BigDecimal(47.0).setScale(2, BigDecimal.ROUND_HALF_UP);

So, I built a JMH benchmark that compared these two, and I did see that the
throughput numbers for the latter were quite a bit higher than the former
(18x).

I then went on to try to see what the average times were, and I got stuck
for a while trying to figure out the correct annotations to use (and I got
answers for that on this list).  At the same time, I also expanded each of
the two benchmark methods to make more than one call to the BigDecimal
constructor, with a few different values.

When I finally got my average time values, I was a little confused because
the results were showing that the benchmark using DecimalFormat had a lower
average time than the one using "setScale".  I then changed the benchmark
back to show throughput numbers (I've since realized that I can change this
on the command line instead of changing the annotations and rebuilding) and
that showed that the benchmark using DecimalFormat was getting a higher
throughput.

This is the results table that I see:

Benchmark                                         Mode  Cnt
Score      Error  Units
MyBenchmark.createBigDecimalFromDoubleWithScale  thrpt   25  341705.442
1486.779  ops/s
MyBenchmark.createBigDecimalFromFormattedDouble  thrpt   25  490130.005
3239.817  ops/s

The benchmark code and pom file is at
https://github.com/davidmichaelkarr/jmhUnitsIssues . It's not directly
compileable in that state, as the source file is in the root of the repo
(i'm having some connectivity issues right now, so fixing that is awkward).
If you clone it, just move the source file to the package directory, as
specified in the readme.

I'd appreciate any help understanding why this is behaving oppositely of
what I would expect.


More information about the jmh-dev mailing list