Iteration/Warmup Timeout (was: Using jmh.shutdownTimeout)

[Behrooz Nobakht]

Hello again,

A bit of delay from me on this. Sorry.
However, I've been trying to work around this challenge
on and off trying and failing every and each one of ideas.

My last approach is as follows that actually helped to get
into a better situation.

I started to use `-Djmh.executor=CUSTOM` and then
my custom ExecutorService and pass it in via `-Djmh.executor.class`.

As it can be seen from the implementation, it is basically just redundant
implementation at the level of ThreadFactory and ExecutorService extension.
There are two parts in this that help me:

- Custom logic in `CustomThread#interrupt` that first calls a static
singleton
`Runnable` as my "interruption handler" during the benchmarks.

- And, then my customer "interruption handler" which is basically a
singleton
Runnable instance such as:

static final Runnable INTERRUPT_HANDLER = new Runnable() {
  private final AtomicBoolean interrupted = new AtomicBoolean(false);

  @Override
  public void run() {
    try {
      if (interrupted.compareAndSet(false, true)) {
        // DEAL WITH MY HANGING THREADS
      }
    } catch (Exception e) {
      System.err.println(e.getMessage());
    }
  }
}

Now, considering the above, could the following make sense to be added to
JMH:

- Add a new system property as: `jmh.timeout.interrupt.handler`
This can be also be a parameter for @Timeout

- The value for the above should be the FQN of a class implementing
`Runnable`.

- The above means that ThreadFactory implementation at JMH should
provide an extension of Thread that overrides #interrupt().

- When timeout is caught in BenchmarkHandler, and so #interrupt() is
called through BenchmarkTask, the above property comes into the
pictures and "optimistically" handles this timeout by ensuring the
application threads are stopped.

The downside of this approach is that the "interrupt handler" Runnable
should be able to access a static context to handler the hanging threads
during the benchmark.

As another side effect of diving into this, I am not sure but I think this
can be a bug in Java's `ForkJoinWorkerThread`. Basically, unless instances
of this type of thread are not interrupted, shutting Executors at JVM
runtime
does not kill those threads. So, I also had to explicitly interrupt all the
threads
of type ForkJoinWorkerThread so that my JVM can cleanly shut down.

If this makes sense, I'm willing to provide a patch on this for JMH.

Thanks,
Behrooz

P.S. I really wish I could "watch" those JIRA tickets.



On Thu, May 28, 2015 at 10:28 AM, Aleksey Shipilev <
aleksey.shipilev at oracle.com> wrote:

> On 26.05.2015 18:05, Behrooz Nobakht wrote:
> > - Based on the benchmark configuration I expect that each iteration is
> > allowed to be executed at most 1 second.
> > Is this not correct?
>
> Current timeout mechanics only issues the Thread.interrupt to the
> running threads. We can't do much better, since we can't magically stop
> the non-cooperating thread stuck in @Benchmark. We can crash the forked
> VM though, as alluded here:
>   https://bugs.openjdk.java.net/browse/CODETOOLS-7901376
>
> > - If not, what should I use to have control over the iteration/warmup
> > explicit ending?
>
> You nominally have to make sure @Benchmark call finishes in reasonable
> time. When you can't, e.g. it requires coordination for multiple
> @Benchmark calls (mostly in asymmetric scenarios), you have to check for
> interrupts in your @Benchmark code.
>
> > - When I add @Threads(N) to the benchmark configuration, although I
> expect
> > exceptions happen due to shutdown of threads (and catch them at a higher
> > level),
> > yet the benchmark fails to make progress to next iterations/warmups.
> > Any advice?
>
> This might be an instance of:
>  https://bugs.openjdk.java.net/browse/CODETOOLS-7901008
>
> It would be nice to see if you have another case that should be handled.
>
> Thanks,
> -Aleksey
>
>


-- 
-- Behrooz Nobakht