Proposal for two new samples
Michael Mirwaldt
michael.mirwaldt at gmail.com
Tue Aug 11 20:32:16 UTC 2015
Hi Aleksey,
you can find my source files in the attachment.
I hope my examples are not flawed too much.
Some remarks:
1) Branch prediction:
a) I use the constant seed 1234 for my Random-Object.
This is quite arbitrary but it must be a constant.
It cannot be the default seed which is dependent on System.nanoTime().
This would lead to completely different results each time you run
the benchmark which is confusing.
I do not think there is a "perfect seed" because how do you want to
measure the degree of happitence
in the unsorted array?
b) I think the effort of incrementing the counter and of the modulo
operation can be neglected.
Can you confirm that?
c) Count ís 1024 ^ 2 which forces int arrays of size 4MB.
The original value 65535 (int arrays of size 256KB) seemed to be
too small to observe anything significant.
d) the decision value 128 seems to have the biggest effect on
branch prediction.
Do not ask me why ;-)
2) Matrix copy
The basic motivation behind the matrix copy benchmark is to make
programmers aware
that two dimensional arrays are technically still one dimensional arrays.
The additional dimensions are only 'mapped' which can lead to cache
misses if arrays are bigger than the caches.
a) I used again the magic number 1024^2 which are 4MB.
Otherwise, all of the vector would fit into the cache making the
benchmark completely useless.
b) The code in the comment of a benchmark method is the intended code
that is expressed by the code in the benchmark method.
The benchmark should actually measure how long it takes to access a
cell.
These explanation should help you follow my thoughts and intentions.
I wish you enjoy my work.
All the best,
Michael
Am 11.08.2015 um 09:04 schrieb Aleksey Shipilev:
> Hi Michael,
>
> OpenJDK Terms of Use mandate you send the patch/files your are about to
> contribute to OpenJDK lists (or any other OpenJDK-affiliated location).
> So, you can send the patches along here.
>
> Thanks,
> -Aleksey
>
> On 08/11/2015 02:01 AM, Michael Mirwaldt wrote:
>> Hi Aleksey,
>> thank you for your reply.
>>
>> Sorry that contributing my code takes so long.
>>
>> I have read http://openjdk.java.net/contribute/ and checked out the
>> project and build it locally.
>> How can I submit my code? Do I just commit it or do I need a new branch?
>>
>> I have not run a "big" test yet because I miss a linux machine at home
>> where I can use "perf".
>> I will have one very soon.
>>
>> Best regards,
>> Michael
>>
>> Am 27.07.2015 um 23:52 schrieb Aleksey Shipilev:
>>> Hi Michael,
>>>
>>> On 07/28/2015 12:22 AM, Michael Mirwaldt wrote:
>>>> I would like to add two samples I miss in the jmh repository.
>>>> They could help jmh users to experience the effect and demonstrate that
>>>> on their lectures.
>>>>
>>>> 1) Branch prediction
>>>> - it demonstrates how branch prediction/misprediction can lead to
>>>> better/worse performance.
>>> Yes, that one is sure missing. This touches on benchmarking methodology
>>> that tells you might need to feed the benchmark with the data arranged
>>> in a way you have it in a real world. Therefore, it is a good addendum
>>> to JMH Samples.
>>>
>>>
>>>> - I could not check whether the branch misses ratio increases.
>>>> If you are interested I will do so and should observe that with the
>>>> command line tool perf on a linux machine.
>>> It would make sense to employ -prof perfnorm or -prof perfasm in the
>>> explanation for the effect.
>>>
>>>
>>>> 2) Matrix copy
>>> The cache behavior example is interesting, but I wonder if we can
>>> somehow construct an example where an innocuously looking *methodology*
>>> omission backfires a lot. Walking the matrix in one way or the other
>>> seems to be a code-under-test implementation issue, not the methodology
>>> quirk?
>>>
>>> For example, could the similar cache unfriendliness be demonstrated on
>>> comparing iteration through HashMap and TreeMap with the same number of
>>> elements (under the misguided assumption both have the same memory
>>> footprint)?
>>>
>>>> Are the results 'significant in numbers' for you?
>>> Yes, they are.
>>>
>>>> How can I submit my sample code so that you can try it out/review them?
>>> Sure. The contributions to JMH are governed by the same rules as
>>> OpenJDK. I see you have the OCA signed and submitted, so we are in
>>> clear. Show us your code!
>>>
>>> Thanks,
>>> -Aleksey
>>>
>
-------------- next part --------------
/*
* Copyright (c) 2015, Oracle America, Inc.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
*
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* * Neither the name of Oracle nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
* THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.openjdk.jmh.samples;
import java.util.Arrays;
import java.util.Random;
import java.util.concurrent.TimeUnit;
import org.openjdk.jmh.annotations.AuxCounters;
import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.BenchmarkMode;
import org.openjdk.jmh.annotations.Fork;
import org.openjdk.jmh.annotations.Level;
import org.openjdk.jmh.annotations.Measurement;
import org.openjdk.jmh.annotations.Mode;
import org.openjdk.jmh.annotations.OutputTimeUnit;
import org.openjdk.jmh.annotations.Scope;
import org.openjdk.jmh.annotations.Setup;
import org.openjdk.jmh.annotations.State;
import org.openjdk.jmh.annotations.Warmup;
import org.openjdk.jmh.infra.Blackhole;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.NANOSECONDS)
@Warmup(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS)
@Measurement(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS)
@Fork(5)
public class JMHSample_36_BranchPrediction {
private final static int COUNT = 1024 * 1024;
private final static int[] sortedArray = new int[COUNT];
private final static int[] unsortedArray = new int[COUNT];
static {
Random random = new Random(1234);
for (int i = 0; i < COUNT; i++) {
int randomInt = random.nextInt() % 256;
sortedArray[i] = randomInt;
unsortedArray[i] = randomInt;
}
Arrays.sort(sortedArray);
}
@AuxCounters
@State(Scope.Thread)
public static class AdditionalCounters {
public int counter;
@Setup(Level.Iteration)
public void clean() {
counter = 0;
}
}
@Benchmark
public void benchmark_sortedArray(AdditionalCounters counters, Blackhole bh) {
final int value = sortedArray[counters.counter % COUNT];
if(value >= 128) {
bh.consume(value);
}
counters.counter++;
}
@Benchmark
public void benchmark_unsortedArray(AdditionalCounters counters, Blackhole bh) {
final int value = unsortedArray[counters.counter % COUNT];
if(value >= 128) {
bh.consume(value);
}
counters.counter++;
}
public static void main(String[] args) throws RunnerException {
Options opt = new OptionsBuilder()
.include(".*" + JMHSample_36_BranchPrediction.class.getSimpleName() + ".*")
.threads(Runtime.getRuntime().availableProcessors()).build();
new Runner(opt).run();
}
}
-------------- next part --------------
/*
* Copyright (c) 2015, Oracle America, Inc.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
*
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* * Neither the name of Oracle nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
* THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.openjdk.jmh.samples;
import java.util.Random;
import java.util.concurrent.TimeUnit;
import org.openjdk.jmh.annotations.AuxCounters;
import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.BenchmarkMode;
import org.openjdk.jmh.annotations.Fork;
import org.openjdk.jmh.annotations.Level;
import org.openjdk.jmh.annotations.Measurement;
import org.openjdk.jmh.annotations.Mode;
import org.openjdk.jmh.annotations.OutputTimeUnit;
import org.openjdk.jmh.annotations.Scope;
import org.openjdk.jmh.annotations.Setup;
import org.openjdk.jmh.annotations.State;
import org.openjdk.jmh.annotations.Warmup;
import org.openjdk.jmh.infra.Blackhole;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.NANOSECONDS)
@Warmup(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS)
@Measurement(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS)
@Fork(5)
public class JMHSample_37_MatrixCopy {
private final static int COUNT = 1024;
private final static int MATRIX_SIZE = COUNT * COUNT;
private final static int[][] matrix = new int[COUNT][COUNT];
static {
Random random = new Random(1234);
for (int i = 0; i < COUNT; i++) {
for (int j = 0; j < COUNT; j++) {
int randomInt = random.nextInt() % 256;
matrix[i][j] = randomInt;
}
}
}
@AuxCounters
@State(Scope.Thread)
public static class AdditionalCounters {
public int counter;
@Setup(Level.Iteration)
public void clean() {
counter = 0;
}
}
/*
* for (int colIndex = 0; colIndex < COUNT; colIndex++) {
* for (int rowIndex = 0; rowIndex < COUNT; rowIndex++) {
* matrixCopy[rowIndex][colIndex];
* }
* }
*/
@Benchmark
public void benchmark_copyColByCol(AdditionalCounters counters, Blackhole bh) {
final int cellIndex = counters.counter % MATRIX_SIZE;
final int colIndex = cellIndex / COUNT;
final int rowIndex = cellIndex % COUNT;
bh.consume(matrix[rowIndex][colIndex]);
counters.counter++;
}
/*
* for (int rowIndex = 0; rowIndex < COUNT; rowIndex++) {
* for (int colIndex = 0; colIndex < COUNT; colIndex++) {
* matrixCopy[rowIndex][colIndex];
* }
* }
*/
@Benchmark
public void benchmark_copyRowByRow(AdditionalCounters counters, Blackhole bh) {
final int cellIndex = counters.counter % MATRIX_SIZE;
final int rowIndex = cellIndex / COUNT;
final int colIndex = cellIndex % COUNT;
bh.consume(matrix[rowIndex][colIndex]);
counters.counter++;
}
public static void main(String[] args) throws RunnerException {
Options opt = new OptionsBuilder()
.include(".*" + JMHSample_37_MatrixCopy.class.getSimpleName() + ".*")
.threads(Runtime.getRuntime().availableProcessors()).build();
new Runner(opt).run();
}
}
More information about the jmh-dev
mailing list