Exception in Benchmark isn't marked as failure in CSV file.
Samuel Bishop
ssb687 at gmail.com
Thu Aug 21 17:52:44 UTC 2014
Hello JMH-mailing list,
I've been reading the provided JMH samples, and I see that in sample 2,
that JMH will stop measuring if an exception is thrown. To test this I
wrote a small benchmark to intentionally throw an exception when given a
particular parameter. I was surprised to see that the CSV file didn't mark
that anything had gone wrong when given my "bad" parameter.
For convenience, I will post my benchmark and CSV output below:
@State(Scope.Thread)
public class BenchmarkException {
@Param({"0","1","2"})
int l;
private BogusItem i;
@Setup
public void setUp()
{
i = new BogusItem();
}
@Benchmark
@BenchmarkMode(Mode.Throughput)
@Warmup(iterations = 2)
@Measurement(iterations = 2)
public void testExcept() throws BogusException {
i.expensiveCalculation(l);
}
private class BogusException extends Throwable {
}
private class BogusItem {
private BogusItem(){
}
private int expensiveCalculation(int l) throws BogusException
{
int answer = 0;
switch (l)
{
case ( 0 ):
answer = answer + l + 10;
break;
case ( 1 ):
answer = answer + l;
break;
case ( 2 ):
throw new BogusException();
default:
answer = 0;
break;
}
return answer;
}
}
}
With the output CSV file of :
"Benchmark","Mode","Threads","Samples","Score","Score Error
(99.9%)","Unit","Param: l"
"benchmark.BenchmarkException.testExcept","thrpt",1,20,8.824403593474023E8,3449877.7951812423,"ops/s","0"
"benchmark.BenchmarkException.testExcept","thrpt",1,20,1.1786938536323738E9,2625966.0416936204,"ops/s","1"
Notice the complete absence of an error for my parameter = 2. Is this a
bug, or intended?
I was expecting something to be there in the file, even if just a string of
"benchmark failed for parameter == 2".
More information about the jmh-dev
mailing list