RFR: 8328306: AArch64: MacOS lazy JIT "write xor execute" switching [v4]

[David Holmes]

On Sun, 17 Aug 2025 08:27:05 GMT, Andrew Haley <aph at openjdk.org> wrote:

>> In MacOS/AArch64 HotSpot, we have to deal with the fact that a thread must be in one of two modes: it either may write to code cache memory or it may execute (and read) code or data in it. A system call `pthread_jit_write_protect_np(int enabled)` changes from one to the other.
>> 
>> Today, we change mode whenever making a transition from interpreter to VM. This means that we change mode a lot: experiments have shown that during `jshell` startup we change mode 4 million times. Other experiments have shown that we only needed to change mode 45 thousand times.
>> 
>> This "eager" mode switching is perhaps too eager, and we'd be better off switching lazily. While the system call that changes mode is very fast, mode switching still amounts to about 100ms of startup time. Switching eagerly also means that some native calls (e.g. to do arithmetic) are disproportionately expensive, given that they have no need of mode switching at all.
>> 
>> The approach in this PR is to defer transitioning from exec-but-don't-write mode (`WXExec`) to write-but-don't-exec mode (`WXWrite`) until we need to write. Instead of enabling `WXWrite` immediately, we switch to a mode called `WXArmedForWrite`. When in this mode, when we need to write into code memory we call `os_bsd_jit_exec_enabled(false)` to enable writing and then set the current mode to `WXWrite`.
>> 
>> We mark all sites that we know will write to code memory with
>> `MACOS_AARCH64_ONLY(os::thread_wx_enable_write());` Judicious placement of these markers, such as when entering patching code, means that we have a fairly small number of these.
>> 
>> We also keep track (in thread-local storage) of the current state of `pthread_jit_write_protect_np` in order to avoid making the system call unnecessarily.
>> 
>> It is possible that we have missed some sites where we do need to make a transition from write-protected to -enabled. While we haven't seen any in testing, we have a fallback path. An attempt to write into code memory triggers a `SIGILL` signal. A signal handler detects this, and if the current mode `WXArmedForWrite` it changes mode to write-enabled and returns. In addition, the handler "heals" the VM entry point so that next time the same point is entered (and for the rest of the lifetime of the VM) it will immediately transition to `WXWrite`.
>> 
>> One other possibility remains: we could omit all of the `wx_enable_write` markers and use healing instead. We've experimented with this. It works well enough, but is rather crude...
>
> Andrew Haley has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Tmp

I'm not sure I'm really understanding the rules here. Is it the case that the old code would switch between write and exec in a more block structured pattern - changing when needed and then restoring. But the new approach only changes when needed and never restores? If so then I worry that the need to change mode at any given point is a function of the code path to that point, rather than being an obvious property of the code itself. The "healing" allows this to work, but I'm not sure how you would ever explain the placement rules to anyone. ?? For example, I find the placement within the `Assembler` constructor extremely obscure but presumably the code that creates the `Assembler` instance then proceeds to use that instance to write into the code cache?

src/hotspot/share/runtime/interfaceSupport.inline.hpp line 352:

> 350: 
> 351: #define JRT_END                                 \
> 352: }

Please restore this to match `JRT_BLOCK_END`. The newline here is not needed.

src/hotspot/share/runtime/javaThread.hpp line 173:

> 171:   int64_t _monitor_owner_id;
> 172: 
> 173: public:

Please restore the indent of one for the access specifier.

src/hotspot/share/utilities/macros.hpp line 560:

> 558: #define MACOS_AARCH64_ONLY(x) MACOS_ONLY(AARCH64_ONLY(x))
> 559: #if defined(__APPLE__) && defined(AARCH64)
> 560: #define MACOS_W_XOR_X 1

This is just an alias for `MACOS_AARCH64_ONLY` - do we really need it? Especially when, in shared code, we lose the fact that it is AARCH64 only.

-------------

PR Review: https://git.openjdk.org/jdk/pull/26562#pullrequestreview-3131068199
PR Review Comment: https://git.openjdk.org/jdk/pull/26562#discussion_r2284384450
PR Review Comment: https://git.openjdk.org/jdk/pull/26562#discussion_r2284389919
PR Review Comment: https://git.openjdk.org/jdk/pull/26562#discussion_r2284377891