"Nice" report for failed forks/benchmarks

[Aleksey Shipilev]

On 06/03/2015 02:28 PM, Vladimir Sitnikov wrote:
> If at least single measurement iteration succeeds, then we might be
> interested in some of the profiling results.
> Overall benchmark might fail due to "OutOfMemory", however
> "·gc.alloc.rate.norm" is still relevant.

I think it does not worth an effort, trying to breadcrumb the results of
unstable/failing benchmark. There are things that just do not worth the
development and support time.

> What if we add secondary sample that tells user that particular
> benchmark failed at iteration X?

IMO, failing benchmark should be examined by looking at the execution
logs. There, users will see where it failed, and what exception had
occurred. Let's not overcomplicate things.

Also, the scores are implicitly numerical scalars (we are talking
performance metrics here), so there is no clean way to encode an
arbitrary string there. We can still print "<error>" recognizing the
shape of failing result.

Thanks,
-Aleksey.