RFR: 8259937: guarantee(loc != NULL) failed: missing saved register with native invoker [v3]

[Roland Westrelin]

> We spotted this issue with Shenandoah and I managed to write a simple
> test case that reproduces it reliably with Shenandoah but the issue is
> independent of the GC.
> 
> The loop in the test case calls a native invoker with an oop live in
> rbp. rbp is saved in the native invoker stub's frame. A safepoint is
> triggered from the safepoint check in the native invoker. The stack
> walking code sees that rbp contains an oop but can't find where that
> oop is stored. That's because stack walking updates the caller's frame
> with the location of rbp in the callee on calls to
> frame::sender(). But the current code sets the last java frame to be
> the compiled frame where rbp is live. So there's no call to
> frame::sender() to update the location rbp. The fix I propose is that
> the frame of the native invoker be visible by stack walking. On a
> safepoint, stack walking starts from the native invoker thread, then
> calls frame::sender() to move to the compiled frame. That causes rbp
> to be properly recorded with its location in the native invoker frame.
> 
> Same problem affects both x86 and aarch64. I've tested this patch with:
> 
> make run-test TEST="java/foreign" TEST_VM_OPTS="-Xcomp" JTREG="TIMEOUT_FACTOR=10"
> 
> on both platforms.

Roland Westrelin has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains seven additional commits since the last revision:

 - improved test
 - cleanup
 - Merge branch 'master' into JDK-8259937
 - test & debug
 - broken build
 - whitespaces
 - fix & test

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/2528/files
  - new: https://git.openjdk.java.net/jdk/pull/2528/files/5b9dfff7..cef05b6f

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2528&range=02
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2528&range=01-02

  Stats: 18656 lines in 546 files changed: 11988 ins; 3826 del; 2842 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2528.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2528/head:pull/2528

PR: https://git.openjdk.java.net/jdk/pull/2528