Instructions for setting up multithreaded tests?

[Millies, Sebastian]

Hi there,

not sure if I am doing this right. I want a test that has both global state (shared across benchmark methods) and local ( new for each iteration). I want the test to be multi-threaded, so that each iteration is executed by multiple threads. Thus I will get contention on the local state within each iteration.

Here’s an example. Is this the right way to do it? Are there any docs/instructions beyond the samples included with JMH? I am having particular trouble understanding the interaction between the setup levels and the scopes.


n  Sebastian

@BenchmarkMode({ Mode.AverageTime })
@OutputTimeUnit(TimeUnit.MILLISECONDS)
@State(Scope.Benchmark)
@Warmup(iterations = 10, time = 1000, timeUnit = TimeUnit.MILLISECONDS)
@Measurement(iterations = 20, time = 1000, timeUnit = TimeUnit.MILLISECONDS)
@Fork(1)
public class ConcurrentBenchmark {

    private Object globalState; // want to set this up once for each benchmark method
    private ConcurrentHashMap<String,Integer> contended; // want a new map for each iteration of a benchmark method

    @Setup(Level.Trial)
    public void setUpGlobal() throws InterruptedException {
        globalState = new Object();
    }

    @Setup(Level.Iteration)
    public void setUpLocal() throws InterruptedException {
        contended = new ConcurrentHashMap<>();
    }

    // want contention on CHM in each iteration between all threads executing this method, but independence from test2
    @Benchmark
    @Threads(2)
    public Object test1() {
        return contended.putIfAbsent("x", 1);
    }

    // want contention on CHM in each iteration between all threads executing this method, but independence from test1
    @Benchmark
    @Threads(2)
    public Object test2() {
        return contended.putIfAbsent("y", 2);
    }

    public static void main(String[] args) throws RunnerException {
        Locale.setDefault(Locale.ENGLISH);
        Options opt = new OptionsBuilder().verbosity(VerboseMode.NORMAL)
                .include(".*" + ConcurrentBenchmark.class.getSimpleName() + ".*").build();
        new Runner(opt).run();
    }

}

Software AG – Sitz/Registered office: Uhlandstraße 12, 64297 Darmstadt, Germany – Registergericht/Commercial register: Darmstadt HRB 1562 - Vorstand/Management Board: Karl-Heinz Streibich (Vorsitzender/Chairman), Eric Duffaut, Dr. Wolfram Jost, Arnd Zinnhardt; - Aufsichtsratsvorsitzender/Chairman of the Supervisory Board: Dr. Andreas Bereczky - http://www.softwareag.com