[concurrency-interest] Spin Loop Hint support: Draft JEP proposal

[Gil Tene]

This thread seems to have "hopped away" to the concurrency-interest list in mid-Dec-2015. This posting is intended to capture a summary of reasoning and some of the discussion there so that we have it in the record in core-libs-dev. Mostly by including the contents of several posts in the continuations of the original thread.

See thread continuations here: http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/thread.html#14576 and here: http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/thread.html#14580

Summary:

On the reasoning for moving from Thread.SpinLoopHint to Runtime.onSpinWait:

From Gil Tene, on Tue, Dec 1, 2015 at 3:45 PM
> …
> Thread.spinLoopHint() was my first choice as well. But I was swayed by strong arguments against "hint" in the method name. Including the lack of "action" and ambiguity about roles. We looked at various names that were either clear and way too long or way too short or implementation specific (or narrowing), like skip, pause, relax, etc.
> 
> Given the spec we agreed on, the name we were looking for was something that would be equivalent to the obvious expectations from something named as elaborately as:
> 
> maybeYouShouldTryToImproveThePerformanceOfTheSpinWaitLoopMakingThisCall(), with the receiver being either the thread or the runtime.
> 
> The "maybe you should try" part is important because doing nothing is a valid option, and accidentally failing to achieve the goal is probably ok, but consistently working in the opposite direction of the goal would be "surprising behavior". The "...making this call" part is important because of ambiguities around roles and actions (the call is not expected to spin, or wait, it's the caller that is doing those things).
> 
> Given the natural way we'd describe what the options are for the receiver in plain language, it became clear that Runtime fit better: we naturally say "the runtime may..." and "indicate to the runtime...", not "the thread may" or "indicate to the thread...". In addition, some of the implementation possibilities (e.g. switch this thread to spin on a dedicated core) may involve actions that are natural runtime actions but far outside of the scope of what Thread might do.
> 
> With an event delivery paradigm ("I'm in a spin wait loop, you may want to do something about that") Runtime.onSpinWait() fits common naming conventions and roles. It's also  readable enough to understand that the Runtime is being told that a spin wait is going on. And in that sense, it is just as expressive as spinLoopHint(), while actually following a naming convention. We left the "try to improve the performance" to the spec/JavaDoc because it was very hard to fit in the name.

From Martin Thompson, on Tue, Dec 1, 2015 at 9:13 AM:
> ...
> The "on" prefix was suggested as the caller is notifying the runtime that it is in a spin-wait loop. This allows the runtime the option of reacting to the event, or not, and allows flexibility in how it chooses to react to this information. The method is not waiting; the method is notifying that the caller is waiting.
> 
> Yes, but we don't have Runtime.onGC() or Runtime.onRunFinalization(), and both of those are documented as "suggesting" the VM perform those actions.  spinLoopHint() sounded much better than what's proposed here, and carries the suggestion/hint/optionality that's desired.  IMHO, onSpinWait() is the least appealing options proposed thus far.
> 
> System.gc() and Runtime.runFinalizersOnExit(boolean) are clear instructions to the system/runtime to do something in an imperative fashion. The onSpinWait() is a declarative statement about the state of the current thread. I can understand a dislike to a name. We all have personal taste on this but I don't think you are comparing similar semantics.
> 
> Consider the runtime as an object. You send it a message via a method. What exactly is spinLoopHint() telling the Runtime what to do? It is applying an event but not expressing it via any convention. "spinLoopHint()" works for me on a Thread to an extent. We should express intent or declare status for this. Hints don't fit comfortably in a programming model.
> 
> The actual naming does not matter so much as this will only be used by minority of programmers. I'll prepare to be flamed on that :-) However it is desperately needed and anything that makes it slip the date to make Java 9 would be such a let down.



Some discussion about naming choices, aiming the API at the target user, and what we expect people might do across platforms:

From Gil Tene, on Tue, Dec 1, 2015 at 4:35 PM:
> …
> On Dec 1, 2015, at 8:02 AM, Vitaly Davidovich <vitalyd at gmail.com> wrote:
> 
>> See my earlier suggestion about keeping target user of this API in mind and the level of this API (i.e. low).
> 
> In this sense (limited target audience that is very technical) Thread.foobar(), Thread.spinLoopHint(), Thread.relax(), and Runtime.onSpinWait() are all equivalent.
> 
> The naming debate is mostly driven by longer term thinking, wanting to avoid unforeseen consequences, and s wish for readability. Which are the things that seem to drive most naming debates and make naming one of those hard things to do in CS.
> 
> Given that it's already cost a month of elapsed time, I'd like to try to push forward with what we ended up with.
> 
>> As for doing nothing in the implementation, OK.  But I bet if someone using this API finds out it does nothing on their platform they're going to stop using this API and go back to manual hacks that have at least some chance of achieving desired results.  So while I understand the general motivation of not leaking implementation details, I don't agree this API is the appropriate place for that.
> 
> The main driver for this JEP is the fact that in Java today, there are no good manual hack options, and on platforms that do support enhanced-performance spinning, spin wait loops written in Java remain inferior (in performance and reaction time) to ones written in other languages.
> 
> I expect spin-loop writers to naturally use Runtime.onSpinWait() across platforms, accepting that it may do nothing on some (e.g. On platforms where no HW support for better spinning performance exists) and improve spinning performance on others. E.g. I fully expect the various disruptor waitFor() variants will still exist, but half of them will probably end up using Runtime.onSpinWait() for enhanced performance, and would do nothing different (in their Java code) for x86, Aarch64, Aarch32, Power, SPARC, or MIPS.


And some discussions on the term "relax":

From Gil Tene, on Tue, Dec 1, 2015 at 4:19 PM:
> On Dec 1, 2015, at 7:32 AM, Vitaly Davidovich <vitalyd at gmail.com> wrote:
> 
>> On why "relax" (or other "this is how it will be achieved" naming options) are not a good fit IMO:
>> 
>> But relax, although a common term for this scenario, is still vague enough as to not indicate exactly how it does this, so I don't see any leakage of implementation details.  Moreover, the niche target users of this API are likely going to know exactly what they want at the machine instruction level on the platform(s) they care about.  This isn't some widely applicable general purpose API -- it's a low level API, and low level APIs should not be intentionally vague.  I'm almost certain that 99% of people using this API will know, either ahead of time or by researching, exactly what happens under the hood.  When trying to control things at this level, implementation details start to matter.
> 
> The goal of the call is not to relax the CPU. It is to improve the performance of the spin loop.
> 
> This makes goal the opposite of what "relax" suggests. While the spec rules (and clearly says that performance improvement is the goal), we'd probably agree that names like "chill()", "slowDown()", or "dontBotherWithSpeed()" would be a bad fit. "Relax" fits right in with those...
> 
> The fact that the performance-enhancing goal can actually be technically achieved in some specific CPUs by relaxing their execution in specific ways is surprising in itself. It makes for interesting conversation ("wow, who knew that relaxing the aggressiveness of speculation might actually make the reaction time faster?"), but the fact that it's interesting and surprising is a good indication that it is a bad name choice.
> 
> There are various ways to relax CPUs that are wrong to use for the goal of the call, but would be natural for something called "relax". These include deeper cstates in x86, variations of MWAIT with short timeouts on various CPUs, and even instructions named "pause" on some platforms that consistently hurt performance but help save power. These make it clear that if we used "relax" as a name in this spin wait loop performance context, we actually mean "relaxInA2015x86PauseInstructionSense()"...

And what is probably a good summary of the whole back and forth on naming this thing:

From Gil Tene, on Tue, Dec 1, 2015 at 4:38 PM:
> On Dec 1, 2015, at 8:29 AM, Vitaly Davidovich <vitalyd at gmail.com> wrote:
> 
>> I suggested relax because it's a commonly used name for this effect.  With no prior art in this area, I'd agree that relax can be misinterpreted.  But I'm just as happy with your original spinLoopHint on the Thread class, to be honest.
> 
> I liked it too. But was willing to give up on it given the strong arguments and opinions against it, and in the interest of getting something acceptable done.
> 
>> I just don't like onSpinWait() on the Runtime class.
> 
> But can you live with it? Trust me, it grows on you... ;-). Especially when you end up defending it in an e-mail thread.




> On Dec 7, 2015, at 9:34 AM, mark.reinhold at oracle.com wrote:
> 
> 2015/11/30 6:58 -0800, gil at azul.com:
>> Update: After some significant back-and-forth between Doug and I on
>> naming and JavaDoc'ing, and with Martin (Thompson) stepping in to
>> help, we have what we think is a good spec and name selection for this
>> thing. We're proposing to add a new static method to the Runtime
>> class:
>> 
>> class Runtime { /...
>>    /**
>>      * Method signifying that the caller is momentarily unable to
>>      * progress until the occurrence of one or more actions of one or
>>      * more other activities.  When invoked within each iteration, this
>>      * method typically improves performance of spin wait loop
>>      * constructions.
>>      */
>>     public static void onSpinWait() {};
>> }
> 
> I'm glad to see some agreement here, but I'm puzzled by the proposal
> to to place this in the Runtime class.  What's the reasoning for that?
> Wouldn't this more naturally be placed in java.lang.Thread?
> 
> Also, I don't think this single method needs a JEP, but you're welcome
> to do it that way if you really want to.
> 
> - Mark