RFR: 8259937: guarantee(loc != NULL) failed: missing saved register with native invoker

[Roland Westrelin]

We spotted this issue with Shenandoah and I managed to write a simple
test case that reproduces it reliably with Shenandoah but the issue is
independent of the GC.

The loop in the test case calls a native invoker with an oop live in
rbp. rbp is saved in the native invoker stub's frame. A safepoint is
triggered from the safepoint check in the native invoker. The stack
walking code sees that rbp contains an oop but can't find where that
oop is stored. That's because stack walking updates the caller's frame
with the location of rbp in the callee on calls to
frame::sender(). But the current code sets the last java frame to be
the compiled frame where rbp is live. So there's no call to
frame::sender() to update the location rbp. The fix I propose is that
the frame of the native invoker be visible by stack walking. On a
safepoint, stack walking starts from the native invoker thread, then
calls frame::sender() to move to the compiled frame. That causes rbp
to be properly recorded with its location in the native invoker frame.

Same problem affects both x86 and aarch64. I've tested this patch with:

make run-test TEST="java/foreign" TEST_VM_OPTS="-Xcomp" JTREG="TIMEOUT_FACTOR=10"

on both platforms.

-------------

Commit messages:
 - whitespaces
 - fix & test

Changes: https://git.openjdk.java.net/jdk/pull/2528/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=2528&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8259937
  Stats: 395 lines in 16 files changed: 264 ins; 53 del; 78 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2528.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2528/head:pull/2528

PR: https://git.openjdk.java.net/jdk/pull/2528