[aarch64-port-dev ] Simple HTTP server app crashes on JDK 11 and 8 on OSv AArch64

[Andrew Haley]

On 3/6/22 05:33, Waldek Kozaczuk wrote:
> Thanks for this hint. I think I have solved this puzzle.
> 
> Once I got hold of the JDK debuginfo files and identified the patching code
> - MacroAssembler::pd_patch_instruction(), I was able to put a breakpoint in
> it and see something very revealing:
> 
> #0  MacroAssembler::pd_patch_instruction_size (branch=0x20000879465c
> "\351\377\237\322\351\377\277\362\351\377\337\362\n\243\352\227\037",
>      target=*0xffffa00042c862e0* "\020zB") at
> src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp:75
> #1  0x0000100000bc13cc in MacroAssembler::pd_patch_instruction (file=0x0,
> line=0, target=0xffffa00042c862e0 "\020zB", branch=<optimized out>)
>      at src/hotspot/cpu/aarch64/macroAssembler_aarch64.hpp:626
> #2  NativeMovConstReg::set_data (this=0x20000879465c, x=-105551995837728)
> at src/hotspot/cpu/aarch64/nativeInst_aarch64.cpp:262
> #3  0x0000100000850bd0 in CompiledIC::set_ic_destination_and_value
> (value=0xffffa00042c862e0,
>      entry_point=0x20000823d290
> "(\b@\271\b\001]\322*\005@\371\037\001\n\353,\001",
> <incomplete sequence \371\200>, this=<optimized out>)
>      at src/hotspot/share/code/compiledIC.hpp:193
> #4  ICStub::finalize (this=<optimized out>) at
> src/hotspot/share/code/icBuffer.cpp:91
> #5  ICStubInterface::finalize (this=<optimized out>, self=<optimized out>)
> at src/hotspot/share/code/icBuffer.cpp:43
> #6  0x0000100000e30958 in StubQueue::stub_finalize
> (this=0xffffa00041555300, s=<optimized out>) at
> src/hotspot/share/code/stubs.hpp:168
> #7  StubQueue::remove_first (this=0xffffa00041555300) at
> src/hotspot/share/code/stubs.cpp:175
> ....
> 
> The corresponding crash value of X9 was this:
> 
> *0x0000a00042c862e0*
> 
> vs the target argument of pd_patch_instruction() (see above in the
> backtrace):
> 
> *0xffffa00042c862e0*
> 
> Now given this comment:
> 
> // Move a constant pointer into r.  In AArch64 mode the virtual
> // address space is 48 bits in size, so we only need three
> // instructions to create a patchable instruction sequence that can
> // reach anywhere.
> 
> and this fragment of pd_patch_instruction() -
> https://github.com/openjdk/jdk17u/blob/6f0f42630eac1febf562062afc523fdf3d2a920a/src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp#L152-L161
> 
> it seems that the code to load x8 register with an address gets patched
> with 0x0000a00042c862e0 instead of 0xffffa00042c862e0. It is interesting
> that this assert -
> https://github.com/openjdk/jdk17u/blob/6f0f42630eac1febf562062afc523fdf3d2a920a/src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp#L77
> - does not get hit.

It won't unless you run a debug build. If I were you I'd do that soon.

> The bottom line is that the valid address 0xffffa00042c862e0 gets truncated
> to 0x0000a00042c862e0 I guess based on the assumption that in Linux all
> userspace addresses are 48-bits long (see
> https://www.kernel.org/doc/html/latest/arm64/memory.html).

That's right.

> In OSv
> unikernel, there is no separation between user space and kernel space, and
> it happens that addresses returned by malloc fall into this range:
> 
> 0xffffa00000000000 - 0xffffafffffffffff
> 
> So I guess the only solution to fix it on the OSv side would be to tweak
> its virtual memory mapping for mallocs and make sure it never uses virtual
> addresses > 48-bits.

Is that possible? AFAIK AArch64 supports two VA subranges,

• The bottom VA range runs up from address 0x0000_0000_0000_0000 . With the maximum
address width of 48 bits this gives a VA range of 0x0000_0000_0000_0000 to
0x0000_FFFF_FFFF_FFFF .

• The top VA subrange runs up to address 0xFFFF_FFFF_FFFF_FFFF . With the maximum address
width of 48 bits this gives a VA range of 0xFFFF_0000_0000_0000 to 0xFFFF_FFFF_FFFF_FFFF .

Do you support both subranges simultaneously in OSy?

> Could you confirm my analysis?
If you can. AArch64 OpenJDK uses 48-bit userspace. We did have a patch to
allow 64-bit addresses, but there wasn't any point at the time. But in any
case, it'd be a really good idea to allow apps designed for 48-bit userspace
to work on OSy.

This might be a good time for us to resurrect the 64-bit address patch,
but that might not help you much, because I guess you'll want older Java
versions to work.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671