RIP values like 0xffffffff94bf7f80 due to patched NMethod

[Alexander Miloslavskiy]

Hi,

I'm currently investigating a strange JVM problem and decided to ask 
here in hopes someone already saw such problem, or is able to provide clues.

One of our customers report that our java application crashes often for him.
I did quite a bit of debugging and got some facts:
1) Customer used Windows JRE x64 versions 8.0_144-b01, 10.0.1+10, 
11.0.1+13-LTS. All of them crash with the same symptoms.
2) It crashes because java's NMethod executes a wild jmp, resulting in 
crazy RIP values such as 0xffffffff94bf7f80.
3) Example of corrupted jmp:
    00000000`042b2ac0 e9bb549490      jmp     ffffffff`94bf7f80
4) I managed to understand that correct jmp should be:
    00000000`042b2ac0 e9bb5494ff      jmp     00000000`03bf7f80
5) Correct jmp address points to 'RuntimeStub: wrong_method_stub'
6) Just one byte of jmp instruction is corrupted with 0x90. It's always 
0x90, and always the same byte is corrupted.
7) The crash occurs soon after jvm compiles a new NMethod for the same 
Method.
8) The new NMethod is compiled with new optimization settings, usually 
(but not always) it's 'CompLevel_full_profile' --> 
'CompLevel_full_optimization'.
9) The crash always occurs in the old NMethod.
10) The reason why 'jmp' is there is because old NMethod was 
transitioned into 'non_entrant' state by 'nmethod::make_not_entrant()'
11) Customer says he doesn't have any java-specific tools installed such 
as profilers, etc.
12) Customer provided around 20 crash logs and around 10 crash core 
dumps. This is just a portion of his crashes. All of them exhibit the 
same problem.
13) Customer used Windows RAM test and it shown no errors. On the other 
hand, error is too specific to be a hardware problem, I think: the last 
byte of jmp gets corrupted with 0x90 when a new NMethod is compiled...
14) I have verified customer's jvm.dll and it's not corrupted.
15) I have verified (using core dump) the value of 
'SharedRuntime::get_handle_wrong_method_stub()' and it's not corrupted.
16) In every core dump, only a single NMethod is corrupted.