RFR: 8186670: Implement _onSpinWait() intrinsic for AArch64 [v15]

[Evgeny Astigeevich]

On Thu, 11 Nov 2021 09:36:36 GMT, Evgeny Astigeevich <duke at openjdk.java.net> wrote:

>> This PR is a follow-up on the discussion [“RFC: AArch64: Implementing spin pauses with ISB”](https://mail.openjdk.java.net/pipermail/hotspot-dev/2021-August/054033.html).
>> 
>> It adds DIAGNOSTIC options `OnSpinWaitInst=inst`, where `inst` can be:
>> 
>> - `none`: no implementation for spin pauses. This is the default value.
>> - `nop`: use `nop` instruction for spin pauses.
>> - `isb`: use `isb` instruction for spin pauses.
>> - `yield`: use `yield` instruction for spin pauses.
>> 
>> And  `OnSpinWaitInstCount=count`, where `count` specifies a number of `OnSpinWaitInst` and can be in `1..99` range. It is an error to use `OnSpinWaitInstCount` when `OnSpinWaitInst` is `none`.
>> 
>> The code for the `Thread.onSpinWait` intrinsic is generated based on the values of `OnSpinWaitInst` and `OnSpinWaitInstCount`.
>> 
>> Testing:
>> 
>> - `make test TEST="gtest"`: Passed
>> - `make run-test TEST="tier1"`: Passed
>> - `make run-test TEST="tier2"`: Passed
>> - `make run-test TEST=hotspot/jtreg/compiler/onSpinWait`: Passed
>> 
>> CSR: https://bugs.openjdk.java.net/browse/JDK-8274564
>
> Evgeny Astigeevich has updated the pull request incrementally with one additional commit since the last revision:
> 
>   8275728: Add simple Producer/Consumer microbenchmark for Thread.onSpinWait

`ThreadOnSpinWaitProducerConsumer` is to demonstrate `Thread.onSpinWait` can be used to avoid heavy locks.
The microbenchmark differs from [Gil's original benchmark](https://github.com/giltene/GilExamples/tree/master/SpinWaitTest) and [Dmitry's variations](http://cr.openjdk.java.net/~dchuyko/8186670/yield/spinwait.html). Those benchmarks produce/consume data by incrementing a volatile counter. The latency of such operations is almost zero. They also don't use heavy locks. According to [Gil's SpinWaitTest.java](https://github.com/giltene/GilExamples/blob/master/SpinWaitTest/src/main/java/SpinWaitTest.java):
> This test can be used to measure and document the impact of Runtime.onSpinWait() behavior
>  on thread-to-thread communication latencies. E.g. when the two threads are pinned to
> the two hardware threads of a shared x86 core (with a shared L1), this test will
> demonstrate an estimate the best case thread-to-thread latencies possible on the
> platform

Gil's microbenchmark targets SMT cases (x86 hyperthreading). As not all CPUs support SMT, the microbenchmarks cannot demonstrate benefits of `Thread.onSpinWait`. It is actually opposite. They show `Thread.onSpinWait`  has negative impact on performance.

The microbenchmark from PR uses `BigInteger` to have 100 - 200 ns latencies for producing/consuming data. These latencies can cause either a producer or a consumer to wait each another. Waiting is implemented with `Object.wait`/`Object.notify` which are heavy. `Thread.onSpinWait` can be used in a spin loop to avoid them.

**ARM64 results**:
- No spin loop

Benchmark                               (maxNum)  (spinNum)  Mode  Cnt     Score    Error  Units
ThreadOnSpinWaitProducerConsumer.trial       100          0  avgt   75  1520.448 ± 40.507  us/op

- No `Thread.onSpinWait` intrinsic

Benchmark                               (maxNum)  (spinNum)  Mode  Cnt     Score    Error  Units
ThreadOnSpinWaitProducerConsumer.trial       100        125  avgt   75  1580.756 ± 47.501  us/op

- `ISB`-based `Thread.onSpinWait` intrinsic

Benchmark                               (maxNum)  (spinNum)  Mode  Cnt    Score     Error  Units
ThreadOnSpinWaitProducerConsumer.trial       100        125  avgt   75  617.454 ± 174.431  us/op


**X86_64 results**:
- No spin loop

Benchmark                               (maxNum)  (spinNum)  Mode  Cnt    Score     Error  Units
ThreadOnSpinWaitProducerConsumer.trial      100        125  avgt   75  1417.944 ± 1.691  us/op

- No `Thread.onSpinWait` intrinsic

Benchmark                               (maxNum)  (spinNum)  Mode  Cnt    Score     Error  Units
ThreadOnSpinWaitProducerConsumer.trial      100        125  avgt   75  1410.987 ± 2.093  us/op

- `PAUSE`-based `Thread.onSpinWait` intrinsic

Benchmark                               (maxNum)  (spinNum)  Mode  Cnt    Score     Error  Units
ThreadOnSpinWaitProducerConsumer.trial      100        125  avgt   75  217.054 ± 1.283  us/op

-------------

PR: https://git.openjdk.java.net/jdk/pull/5562