print results with percentile

[Aleksey Shipilev]

Hi,

On 09/15/2015 06:33 PM, WANG Hui wrote:
> Is the latency recorded for each method invocation ?

As documentation says, it *randomly* *samples* the time needed for the
call. We cannot timestamp every single call of arbitrary benchmark due
to time measurement overheads.

> How does automatic freq selection work ?

It basically does an exponential backoff until it fits the desired
number of samples. The desired number of samples is capped at having a
timestamp no more frequently than 500us between the timestamps. If your
workload is larger than that, then every single invocation would be
"sampled".

JMH actually lets you inspect the generated synthetic infra code, look
at target/generated-sources/.../$TestName_jmhTest.java; look for
*_sample_jmhStub method, which would tell how exactly the time sampling
mechanics works.

> Would it be possible for the max latency to be omitted?

If you mean "can we blindly omit a very large outlier", then yes, we
can, due to the nature of randomized sampling. This is practically true
when your workload is nominally very quick but with huge outliers, and
we actually skip the measurements.


> Regarding the overheads of timestamping infrastructure, IMHO, as long as
> we want to observe a system, we must inject some facility to feel the
> system.
> 
> Therefore, the overhead is not avoidable but it should be fine if the
> cost is not comparable with the under-observation system.

The thing is, current scheme only starts to omit samples when you are
allegedly in a case when a payload is too small to compensate for the
timing overheads. If a workload is large enough, then all samples are
recorded. Sound like a win-win to me.


> What are your opinions on the histogram tool which should be rather
> modest in terms of overhead ?

We are using the variant of HDR histogram, see SampleBuffer:
 http://hg.openjdk.java.net/code-tools/jmh/file/f2d57d49d36b/jmh-core/src/main/java/org/openjdk/jmh/util/SampleBuffer.java

It seems to have a low overhead.

Thanks,
-Aleksey