Gotchas with avoiding benchmarks.jar

[Aleksey Shipilev]

On 08/25/2014 10:08 PM, Brian Harris wrote:
> a) looking at handling of jvm args in Runner the safe thing for me is to
> always explicitly set OptionsBuilder#jvmArgs (perhaps to an empty set,
> even) so that the host JVM args aren't automatically used

Yes, that works.

> b) junit allows concurrent execution with custom @RunWith but by default
> it's always single threaded, so good point tho out of the box behavior
> is safe. to be extra sure, Runner (or our wrapper) could acquire a
> static lock

Static lock does not guard from multiple concurrent JVMs, and many
@JUnit test runners I know of are "isolating" the concurrently running
tests by forking the VMs.

> c) if jmh always generates these two files, Runner could assert they are
> present to prevent this case.

Oh yes, we need to assert these things.

-Aleksey.