JMH feature request: Formal and raw output formats

[Claes Redestad]

Hi,

  to ease integration of JMH output into third-party analysis tools, we 
currently resort to scraping the standard output of JMH. Since output 
isn't formally defined, this integration relies on regexes and other 
quirks and is thus by definition prone to breaking horribly.

   It'd be a great help if it was possible to getting jmh results in a 
machine-readable format, such as json or xml. Also getting raw data 
(iteration scores) would allow better statistical analysis in third 
party tools. The current -of|--outputformat options (pretty, csv, 
silent, textreport) seems insufficient and only alter stdout (which 
might mask issues) and the -o option simply redirects the stdout. A 
better solution I think would be to add a --result flag which only 
outputs the results to the specified file, leaving the stdout intact, or 
implicitly outputting to some file if -rf|--resultFormat is specified.

  Usage examples:
  java -jar microbenchmarks.jar -rf xml [--result jmh.out]
  java -jar microbenchmarks.jar -rf json [--result jmh.out]

  The default json format would look something like this:
  {'benchmark': 
'net.openjdk.micro.crypto.generated.throughput.AESBench.testUseAesIntrinsics', 
'threads': 1, 'count': 4, 'time': 1000, 'mean': 10.700, 'meanError': 
0.472, 'variance': 0.026, 'units': 'ops/sec'}

  To add iteration data, I suggest --raw:
  java -jar microbenchmarks.jar -rf json --raw
  { ..., 'results': [10.617, 10.517, 10.800, 10.867]}

  /Claes