JMH and continuos integration

[Jean Philippe Bempel]

Hi Leonardo,

We are using JMH & the plugin you mention for jenkins in early stage of
integrating performance non-regression tests
We are still in phase of evaluation to see if we need to build more custom
things around this for our needs.

I agree with Mark Price on not use this for fail/pass test on build
directly. We are using Gerrit for review and it was asked to base the pass
review to the fact there is no regression in perf tests.
However benchmarks take long time to run (as mentioned by Mark). Plus time
for regular unit tests it is a no go to do this.
We are using a nightly run to perform those benchmarks and we need to check
regularly if there is no regression.

Like Mark I witness run to run variance and it is very important to run on
dedicated bare metal hardware.

Cheers

*Jean-Philippe Bempel* *|* Software Architect* |* *ULLINK |* T: +33 1 49 95
10 29
| 23-25 rue de Provence | 75009 Paris | *jpbempel at ullink.com*
<jpbempel at ullink.com> |

On Fri, Nov 11, 2016 at 1:12 AM, Leonardo Gomes <leonardo.f.gomes at gmail.com>
wrote:

> Hi,
>
> I'm looking for feedback on how people use JMH as part of their development
> process.
>
> I've played a bit with https://github.com/blackboard/jmh-jenkins and am
> wondering if people in this discussion list do use any sort of automation
> (Jenkins or other) to detect performance regression on their software, on a
> pull-request basis.
>
> I've seen JMH being used on different open-source projects, but the
> benchmarks seem to have been introduced mostly to compare different
> implementations (when introducing / validating changes) and not as part of
> CI.
>
> Any feedback would be appreciated.
>
> Thank you,
> Leonardo.
>

-- 
*The information contained in or attached to this email is strictly 
confidential. If you are not the intended recipient, please notify us 
immediately by telephone and return the message to us.*