RFR(XL): 8185640: Thread-local handshakes

[Robbin Ehn]

Thanks for looking at this Coleen,

On 2017-10-18 22:44, coleen.phillimore at oracle.com wrote:
> 
> This looks really nice.  A few minor comments.
> 
> http://cr.openjdk.java.net/~rehn/8185640/v0/flat/webrev/src/hotspot/share/runtime/handshake.hpp.html
> 
>    51 // or the JavaThread it self.
> 
> typo, "itself"

Fixed

> 
> Thank you for adding these comments.  I think they're just right in length and detail in the header.
> 
> http://cr.openjdk.java.net/~rehn/8185640/v0/flat/webrev/src/hotspot/share/runtime/handshake.cpp.html
> 
> The protocol in HandshakeState::process_self_inner and cancel_inner is:
> 
>      clear_handshake(thread);
>      if (op != NULL) {
>        op->do_handshake(thread);
>      }
> 
> But in HandshakeState::process_by_vmthread(), the order is reversed. Can you explain why in the comments.
> 
>      _operation->do_handshake(target);
>      clear_handshake(target);
> 
> It looks like the thread can't continue while the handshake operation is in progress, so does the order matter?

The key part here is that must be cleared before signaling the semaphore.
The early clearing is because if the thread is doing it's own operation, the VM thread can quickly skip this thread by looking if it still have an operation.

> 
> http://cr.openjdk.java.net/~rehn/8185640/v0/flat/webrev/test/hotspot/jtreg/runtime/handshake/HandshakeWalkStackNativeTest.java.html
> 
> This has the wrong @test name.  These could use an @comment line about what you expect also.  I don't know what's "Native" about it though, isn't it testing what happens when you use -XX:+ThreadLocalHandshakes?
> 
> http://cr.openjdk.java.net/~rehn/8185640/v0/flat/webrev/test/hotspot/jtreg/runtime/handshake/HandshakeWalkStackFallbackTest.java.html
> 
> This one too an @comment that it's testing the fallback VM operation would be good.
> 
> I don't need to see another webrev for the comment changes.

Here it is, there was inconsistencies in the tests, I think it is better now.

http://cr.openjdk.java.net/~rehn/8185640/v2/Coleen-n-Test-Cleanup-4/webrev/

> 
> Lastly, as I said before, I think putting the safepoint polls in the interpreter at return and backward branches would be a good follow on changeset.

I will let Claes R decided if that is an acceptable approach.

Thanks, Robbin

> 
> Thanks,
> Coleen
> 
> 
> On 10/11/17 9:37 AM, Robbin Ehn wrote:
>> Hi all,
>>
>> Starting the review of the code while JEP work is still not completed.
>>
>> JEP: https://bugs.openjdk.java.net/browse/JDK-8185640
>>
>> This JEP introduces a way to execute a callback on threads without performing a global VM safepoint. It makes it both possible and cheap to stop individual threads and not just all threads or none.
>>
>> Entire changeset:
>> http://cr.openjdk.java.net/~rehn/8185640/v0/flat/
>>
>> Divided into 3-parts,
>> SafepointMechanism abstraction:
>> http://cr.openjdk.java.net/~rehn/8185640/v0/SafepointMechanism-0/
>> Consolidating polling page allocation:
>> http://cr.openjdk.java.net/~rehn/8185640/v0/PollingPage-1/
>> Handshakes:
>> http://cr.openjdk.java.net/~rehn/8185640/v0/Handshakes-2/
>>
>> A handshake operation is a callback that is executed for each JavaThread while that thread is in a safepoint safe state. The callback is executed either by the thread itself or by the VM thread while keeping the thread in a blocked state. The big difference between safepointing and handshaking is that the per thread operation will be performed on all threads as soon as possible and they will continue to execute as soon as it’s own operation is completed. If a JavaThread is known to be running, then a handshake can be performed with that single JavaThread as well.
>>
>> The current safepointing scheme is modified to perform an indirection through a per-thread pointer which will allow a single thread's execution to be forced to trap on the guard page. In order to force a thread to yield the VM updates the per-thread pointer for the corresponding thread to point to the guarded page.
>>
>> Example of potential use-cases:
>> -Biased lock revocation
>> -External requests for stack traces
>> -Deoptimization
>> -Async exception delivery
>> -External suspension
>> -Eliding memory barriers
>>
>> All of these will benefit the VM moving towards becoming more low-latency friendly by reducing the number of global safepoints.
>> Platforms that do not yet implement the per JavaThread poll, a fallback to normal safepoint is in place. HandshakeOneThread will then be a normal safepoint. The supported platforms are Linux x64 and Solaris SPARC.
>>
>> Tested heavily with various test suits and comes with a few new tests.
>>
>> Performance testing using standardized benchmark show no signification changes, the latest number was -0.7% on Linux x64 and +1.5% Solaris SPARC (not statistically ensured). A minor regression for the load vs load load on x64 is expected and a slight increase on SPARC due to the cost of ‘materializing’ the page vs load load.
>> The time to trigger a safepoint was measured on a large machine to not be an issue. The looping over threads and arming the polling page will benefit from the work on JavaThread life-cycle (8167108 - SMR and JavaThread Lifecycle: http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2017-October/024773.html) which puts all JavaThreads in an array instead of a linked list.
>>
>> Thanks, Robbin
>