Benchmark iteration time of SS mode when measurement iteration time is specifed

[Dmitry Vyazelenko]

Actually it seem that execution time is 20 seconds. So overall execution time seems to be based on the annotation setting even pin SingleShotTime case.

Running: time java -jar target/benchmarks.jar .*Jmh.*
Yields this:
# Run complete. Total time: 00:00:20

Benchmark              Mode  Samples      Score  Score error  Units
j.s.JmhExample.test      ss        1  10001.097          NaN     ms

real	0m20.466s
user	0m0.853s
sys	0m0.137s

Best regards,
Dmitry Vyazelenko

On Aug 1, 2014, at 17:14 , Dmitry Vyazelenko <vyazelenko at yahoo.com> wrote:

> Hi Li,
> 
> Actually your benchmark was executed in 10 seconds (i.e. Score reported as 10001.543 ms). When you use SingleShotTime it will ignore time specified and only do a single invocation of the benchmark method (it can actually do more invocation if batchSize is specified). 
> What you see as 21 seconds total time is just a reporting bug. Apparently time element from annotation is used to print that?
> 
> Best regards,
> Dmitry Vyazelenko
> 
> On Aug 1, 2014, at 04:05 , Li Guoyu <Li.Guoyu at blackboard.com> wrote:
> 
>> Hi Everyone,
>> 
>> The benchmark total time of the following dummy test doesn’t make sense to me, the benchmark duration is 21 seconds when the actual benchmark only take 10 seconds, it seems the benchmark duration depends on the measurement time annotation specified, is this expected?
>> 
>> BTW, the benchmark duration is 11 seconds when measurement time annotation is removed, it works pretty work.
>> 
>> I specified the measurement time for ss benchmark because one of my test took more than 10 minutes to complete, and it seems the max time is 10 minutes and otherwise it will be interrupted. Specifying the measurement time to 30 minutes resolved the interrupted exception, but every benchmark took 30 minutes to complete although the actual time needed is 10+ minutes.
>> 
>> 
>> @State( Scope.Benchmark )
>> 
>> @BenchmarkMode( Mode.SingleShotTime )
>> 
>> @Fork( value = 1 )
>> 
>> @OutputTimeUnit( TimeUnit.MILLISECONDS )
>> 
>> @Warmup( iterations = 0 )
>> 
>> @Measurement( iterations = 1, time = 20 )
>> 
>> public class JmhExample
>> 
>> {
>> 
>> 
>> @Setup( Level.Trial )
>> 
>> public void setup()
>> 
>> {
>> 
>>   System.out.println( "+++++++Setup" );
>> 
>> }
>> 
>> 
>> @Benchmark
>> 
>> public void test() throws Exception
>> 
>> {
>> 
>>   System.out.println( "asdf" );
>> 
>>   Thread.sleep( 10 * 1000 );
>> 
>> }
>> 
>> 
>> public static void main( String[] args ) throws Exception
>> 
>> {
>> 
>>   Options opt = new OptionsBuilder().include( ".*" + JmhExample.class.getSimpleName() + ".*" ).build();
>> 
>>   new Runner( opt ).run();
>> 
>> }
>> 
>> }
>> 
>> 
>> # Run complete. Total time: 00:00:21
>> 
>> 
>> Benchmark             Mode   Samples        Score  Score error    Units
>> 
>> b.JmhExample.test       ss         1    10001.543          NaN       ms
>> 
>> 
>> 
>> This email and any attachments may contain confidential and proprietary information of Blackboard that is for the sole use of the intended recipient. If you are not the intended recipient, disclosure, copying, re-distribution or other use of any of this information is strictly prohibited. Please immediately notify the sender and delete this transmission if you received this email in error.
>