[concurrency-interest] Spin Loop Hint support: Draft JEP proposal

[Gil Tene]

> On Oct 15, 2015, at 11:32 PM, Doug Lea <dl at cs.oswego.edu> wrote:
> 
> On 10/14/2015 11:53 PM, Gil Tene wrote:
>> I agree on the separation between spin-hinting and monitor-like constructs.
>> But not so much on the analogy to or use of the term "yield" to describe what
>> is intended y spin hints.
>> 
> 
> I've been focussing on the spec, which still seems to best support
> this naming. Let's try fleshing out some more (for no-arg version).
> 
>  /**
>   * A hint to the platform that the current thread is momentarily
>   * unable to progress until the occurrence of one or more actions
>   * of one or more other threads. The method is mainly applicable
>   * in spin-then-block constructions entailing a bounded number
>   * of re-checks of a condition, separated by spinYield(), followed
>   * if necessary with use of a blocking synchronization mechanism.
>   */
>  public static void spinYield();

I don't think that this is a good description of the use cases. Yes, the hint is helpful for spin-then-block constructions, but that's just a part of where it can help. In fact, I expect the hint to be very applicable for indefinitely-spinning loops, and I expect the measurable impact there to be much more reliably noticed because such loops are invariably concerned with fast reaction times above all else.

I also don't think that the "…momentarily unable to progress until the occurrence of one or more actions of one or more other threads. " is true: while (!(done || (count++ > threshold))) { spinLoopHint(); } can progress without any action by any other thread.

As noted in my proposed JavaDoc, I see the primary indication of the hint to be that the reaction time to events that would cause the loop to exit (e.g. in nanosecond units) is more important to the caller than the speed at which the loop is executing (e.g. in "number of loop iterations per second" units). So if we are focusing on the spec, here is my suggested (edited to be more specific
) spec:

/**
* Provide the JVM with a hint that this call is made from within a spinning
* loop. The JVM may assume that the reaction time to events that would
* cause the loop to terminate is more important than the speed of executing
* the loop (e.g. in terms of number of loop iterations per second).
* Power savings may also occur, but those are considered incidental to the 
* primary purpose of improving reaction time. The JVM will not slow down
* the loop execution to a point where execution will be delayed indefinitely,
* but other choices of loop execution speed are system-specific. Note that a
* nop is a valid implementation of this hint.
*/

> What should be the response to this hint? When applicable
> and available, the JVM should just issue PAUSE. But on a uniprocessor,
> or when load average is easily detected to be high, or
> on a tightly packed cloud node, a plain yield or something
> along those lines might be a better use of this hint, that
> the spec should not rule out.

Anyone running indefinite spin loops on a uniprocessor deserves whatever they get. Yielding in order to help them out is not mercy. Let Darwin take care of them instead.

But indefinite user-mode spinning on many-core systems is a valid and common use case (see the disruptor link in my previous e-mail). And because a spin loop hint is extremely useful for indefinitely spinning loop situations, and a spin hint is primarily intended to improve the reaction time of spin loops, I would describe any explicit yielding by the JVM at the hint point as mis-behavior. [Not quite an invalid behavior, because we don't want to specify allowed behavior too strongly, but certainly surprising, unexpected, and highly disappointing given the intent expressed by the hint]. Yes, the OS or hypervisor may choose to preempt a thread at any random point in code, including at these hint points, but that's their job and their problem, and not the JVM's. The JVM should not be in the business of voluntarily and implicitly yielding at specific points in code, and especially not at points in code that spins and hints that it wants to improve the performance of that spin.

If what you want is a spin loop that yields, write one: while (!done) {  yield(); }. I don't see how while (!done) { spinYield(); } has any different meaning to the reader. It just reads as something like "yield faster, knowing that you are in a spin".

> Also, I believe that some x86
> hypervisors intercept PAUSE and do something roughly similar
> after repeated invocations.

As to hypervisor choices: preempting a guest OS at a PAUSE instruction is actually higher risk, since the PAUSE instruction could be taken while holding a ciritical kernel resource (e.g. mremap always grabs one spinlock while holding another spinlock). The trick most hypervisors seem to use is to prefer to preempt code in user mode rather than in kernel mode, since user mode code doesn't see preemption as an invalid operation, but kernel code paths (e.g those running under a spinlock) often consider losing the CPU for 200msec in a critical path a surprising thing and a valid cause for panic.

>> While the spinYield() example in your e-mail below can work from a semantic
>> point of view in the same code, IMO the word "yield" suggests the exact
>> opposite of what spnLoopHint() is intending to do or hint at
> 
> Maybe. If you are on a system with load > #cpus, or with
> certain forms of hypervisor,  or without a PAUSE instruction,
> spinYield might not improve responsiveness but might still
> improve system throughput.

In such situations the spinning loop should just be calling yield(), or looping for a very short count (like your magic 64) and then yielding. A "magically choose for me whether reaction time or throughput or being nice to others is more important" call is not a useful hint IMO.

Like in my uniprocessor comment above, any program spinning indefinitely (or for a non-trivial amount of time) with load > # cpus deserves what it gets. Allowing it to live longer/better with it's programmer's mistakes is just polluting the gene pool.

> 
> -Doug
>