Tracking Memory Consumption Metrics With JMH What&How

[Jens Wilke]

On Mittwoch, 5. April 2017 18:56:24 ICT Kirk Pepperdine wrote:
> Hi Jens,
> 
> I will humbly disagree with your dropping run times to 20 seconds and
> restarting JVM’s often. HotSpot warmup can take several minutes even for
> the simplest of benchmarks.

Kirk,

maybe there is a communications hickup. Of course you need warmup.

As described in the blog post:

Within the same VM instance I do 2 warmup iterations of 20 seconds each, and 
then 3 measurement iterations. With 5 VM restarts (forks in JMH lingo) its 10 
measurement iterations altogether.

Since I am doing 3 measurement iterations within the same VM, performance
differences, maybe as a result of insufficient warmup, would be detected 
reliably in the confidence interval.

Findings:

For this scenario, two warmup iterations are needed, after that,  the results 
are very stable. That means the actual measurement starts more than 40 seconds 
after the VM start.

More VM restarts improve the accuracy of the memory metrics

Best,

Jens

-- 
"Everything superfluous is wrong!"

   // Jens Wilke - headissue GmbH - Germany
 \//  https://headissue.com