Raw data output

[Petr Stefan]

Hi.

> Depending on the scenario I need to benchmark, I choose the benchmark mode.
> I would find it very limiting to be restricted to one, by some additional
> tooling.
>
> Maybe you want to share some benchmarks and discuss about that?
We are developing a tool, that collects benchmarking data, processes 
them and saves them depending on version (like git commit). Then you can 
specify some SPL formula (if you're interested, look at papers here 
http://d3s.mff.cuni.cz/research/development_awareness/#performance-testing) 
and evaluate it on top of the data. Typical use case is that you don't 
want your code to slow down, so your formula will reflect this 
requirement and our tool will find regressions in your code between 
commits. This is too big piece of code (and probably little 
out-of-topic) to nicely fit into JMH, so we want to get raw data out of 
JMH and process them ourselves.

> Just for the record: I am not the author of JMH, that is Aleksey Shipilev.
> I am a humble user, just as you.
Sure, thanks for your opinion. We're looking forward to hear some 
comments from author, too.

> In my opinion, the current JSON contains lots of different data, so I would be tempted
> to say it is actually meant to include "everything". Extending it in a compatible
> way should be possible.
>
> My personal preference would be to have an output which contains every peace of gathered
> information, so I can archive it and get back later and extract more things as I need it.
>
> Maybe you can show an example JSON output that you currently get from the existing
> output and make a proposal what additional values should be included?
> It is easier to discuss some data output or format with some examples.
It's a big question how to represent the raw data. If you print all the 
values one by one into current JSON file, you'll get huge amount of 
numbers per iteration (like tens of thousands). In our patch we're 
printing the values as pairs of value and quantity, which seems to be 
more space saving. Also, our JSON format has no preatty indentation, 
which adds a lot of spaces per line. However we don't want custom binary 
format, because it's hard to check it by hand.

> For Aleksey beeing able to accept your patches you need to sign the Oracle Contributor
> Agreement. Take a look at http://openjdk.java.net/contribute/
I think we could deal with the author when necessary. Thanks for mentioning.

Cheers,

Petr