RFR: 8328306: AArch64: MacOS lazy JIT "write xor execute" switching

[Andrew Haley]

In MacOS/AArch64 HotSpot, we have to deal with the fact that a thread must be in one of two modes: it either may write to code cache memory or it may execute (and read) code or data in it. A system call `pthread_jit_write_protect_np(int enabled)` changes from one to the other.

Today, we change mode whenever making a transition from interpreter to VM. This means that we change mode a lot: experiments have shown that during `jshell` startup we change mode 4 million times. Other experiments have shown that we only needed to change mode 45 thousand times.

This "eager" mode switching is perhaps too eager, and we'd be better off switching lazily. While the system call that changes mode is very fast, mode switching still amounts to about 100ms of startup time. Switching eagerly also means that some native calls (e.g. to do arithmetic) are disproportionately expensive, given that they have no need of mode switching at all.

The approach in this PR is to defer transitioning from exec-but-don't-write mode (`WXExec`) to write-but-don't-exec mode (`WXWrite`) until we need to write. Instead of enabling `WXWrite` immediately, we switch to a mode called `WXArmedForWrite`. When in this mode, when we need to write into code memory we call `os_bsd_jit_exec_enabled(false)` to enable writing and then set the current mode to `WXWrite`.

We mark all sites that we know will write to code memory with
`MACOS_AARCH64_ONLY(os::thread_wx_enable_write());` Judicious placement of these markers, such as when entering patching code, means that we have a fairly small number of these.

We also keep track (in thread-local storage) of the current state of `pthread_jit_write_protect_np` in order to avoid making the system call unnecessarily.

It is possible that we have missed some sites where we do need to make a transition from write-protected to -enabled. While we haven't seen any in testing, we have a fallback path. An attempt to write into code memory triggers a `SIGILL` signal. A signal handler detects this, and if the current mode `WXArmedForWrite` it changes mode to write-enabled and returns. In addition, the handler "heals" the VM entry point so that next time the same point is entered (and for the rest of the lifetime of the VM) it will immediately transition to `WXWrite`.

One other possibility remains: we could omit all of the `wx_enable_write` markers and use healing instead. We've experimented with this. It works well enough, but is rather crude, and it's better to be able to see where transitions take place.

-------------

Commit messages:
 - Cleanup
 - More
 - More
 - More
 - Remove debug code
 - More
 - More
 - More
 - Update src/hotspot/share/interpreter/interpreterRuntime.cpp
 - Update src/hotspot/share/runtime/javaThread.cpp
 - ... and 1 more: https://git.openjdk.org/jdk/compare/c239c0ab...b9709363

Changes: https://git.openjdk.org/jdk/pull/26562/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=26562&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8328306
  Stats: 267 lines in 32 files changed: 209 ins; 19 del; 39 mod
  Patch: https://git.openjdk.org/jdk/pull/26562.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/26562/head:pull/26562

PR: https://git.openjdk.org/jdk/pull/26562