RFR: 8287325: AArch64: fix virtual threads with -XX:UseBranchProtection=pac-ret

[Hao Sun]

On Thu, 6 Apr 2023 20:32:49 GMT, Dean Long <dlong at openjdk.org> wrote:

>>> Using SP seems like the right way to go. Can't we compute the correct SP value to use in Runtime1::generate_handle_exception()?
>> 
>> Thanks for your question, Dean.
>> Thinking more about it, I guess we can get the expected SP value.
>> 
>> In my local test, I always use `rfp + 16` to authenticate the return address, and test cases under `test/jdk/java/lang/Thread/` and `test/hotspot/jtreg/compiler/c2/` can pass except the `virtual thread` cases. I'm running tier1-3 now.
>> 
>> My concern is that **using absolute SP** is incompatible with **virtual thread**, since PAC re-sign is still needed due to the stack copying process (See Requirement-2 in the commit message). Alternatively, we may want to use **relative SP** as the modifier.
>> 
>> However, I didn't know how to get such an **initial SP** value. Do you have any idea?
>> Thanks.
>
> @shqking, to get a relative SP, I think you would want to subtract SP from thread->last_continuation()->entry_sp().

Thanks @dean-long  for the suggestion of using `relative SP` as the modifier.
Currently I implemented one prototype(See https://github.com/shqking/jdk/commit/bcc15fd09f7dc) but there are still several TODOs to address.

I will continue to fix the TODOs and make this solution ready for official review. 
But I'd like to share the code in advance if you're interested. Any comment/feedback is welcome. Thanks.

### relative SP

Different from [1], this is another implementation of PAC-RET branch protection on Linux/AArch64, i.e. using relative SP as the modifier.

Note that `relative SP = signing SP - initial SP`, where signing SP is the lowest address of the frame record where LR is saved, and initial SP is `thread->last_continuation()->entry_sp()`.
For non-virtual thread, initial SP is zero, and relative SP becomes absolute SP in this case.


  // For virtual thread:

                  |          |
                  |==========|
                  |          |  // Continuation.run
                  |          |
                  |==========|\
                  |          | | ContinuationEntry          Address
                  |          | |                             high
  initial SP ---> |==========|/                                |
                  |          |                                 |
                  | ...      |                                |/
                  |          |                                low
                  |==========|\
                  |  LR      | |
                  |  FP      | | frame record
  signing SP ---> |----------|/
                  |          |
                  |          |


### Implementation

Here lists the key difference from zero modifier [1] in implementation.

1. initial SP: We need access the thread register (i.e. `rthread`) to load the initial SP. However, `rthread` is not assigned yet at the time of stack frame creation for `make_upcall_stub()` and `generate_call_stub()`.

2. initial SP: For shared code, we utilize `Continuation::get_continuation_entry_for_sp()` to find the continuation
which the signing SP belongs to. See the update in `continuationHelper_aarch64.inline.hpp`, `frame_aarch64.cpp` and
`frame_aarch64.inline.hpp`.

3. signing SP: It's easy to compute the signing SP in most cases, but special handling is needed for `patch_callers_callsite()` as `push_CPU_state()` moves the SP a lot.

4. relative SP: One temporary register is needed to save the result. We choose to use `rscratch1`. Note that for the safety consideration, we save/restore it on the stack before/after PAC. See the code in `protect_return_address(Register thread)` and `authenticate_return_address(Register thread)`.

### Test

Ran tier1~3 on Linux/AArch64 w/ and w/o PAC-RET and got only one new failure, i.e. `jdk/internal/vm/Continuation/Fuzz.java`.

### TODOs

1. Fix the jtreg failure: `jdk/internal/vm/Continuation/Fuzz.java`

2. Following [1], we didn't authenticate the return address in the following functons. We should try to enable the authentication.


  ContinuationHelper::return_address_at()
  ContinuationHelper::Frame::real_pc()
  StackChunkFrameStream<frame_kind>::get_pc()


3. Some tests in `stubRoutines.cpp` are disabled, because these tests are conducted during the VM initialization phase and thread register is not assigned yet. We may want to move these tests to some other places.

4. We thought it's the usual convention that any MacroAssembler call can clobber `rscratch1` or `rscratch2` registers. We cannot do that in `protect_return_address()` and `authenticate_return_address()`. Here shows an exception case [2]. We may need revisit all the usages of `rscratch1` and `rscratch2` to check if this convention can be guaranteed.

[1] https://github.com/openjdk/jdk/pull/13322
[2] https://github.com/openjdk/jdk/blob/master/src/hotspot/cpu/aarch64/sharedRuntime_aarch64.cpp#L2936-L2949

-------------

PR Comment: https://git.openjdk.org/jdk/pull/13322#issuecomment-1566347900