[Bug 3533] New: HotSpot generates code with unaligned stack, crashes on SSE operations

[bugzilla-daemon at icedtea.classpath.org]

https://icedtea.classpath.org/bugzilla/show_bug.cgi?id=3533

            Bug ID: 3533
           Summary: HotSpot generates code with unaligned stack, crashes
                    on SSE operations
           Product: IcedTea
           Version: 3.6.0
          Hardware: x86
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: P5
         Component: IcedTea
          Assignee: gnu.andrew at redhat.com
          Reporter: mail at maciej.szmigiero.name
                CC: unassigned at icedtea.classpath.org

At least on 32-bit x86 HotSpot dynamic compiler generates a machine code that
does not keep the stack pointer aligned to a 16-byte boundary (as required by
the i386 psABI).

When such generated code calls back a gcc-compiled C/C++ code (in JVM,
glibc or some other library) this misaligned stack travels along a call chain
until it hits something that assumes a proper stack alignement.
For example, hitting a SSE operation results in a segfault due to a misaligned
address.

Why this didn't occur earlier I don't know - perhaps it worked by pure luck -
but now gcc 7.3.0 does generate a code that triggers this bug even when just
launching java or javac without any arguments.

For example let's consider a code generated by
InterpreterGenerator::generate_native_entry() in
src/cpu/x86/vm/templateInterpreter_x86_32.cpp.
This function explicitly generates a stack-realigning instruction
("__ andptr(rsp, -(StackAlignmentInBytes));"), then uses the
MacroAssembler::call_VM() method with a one register argument in
src/cpu/x86/vm/macroAssembler_x86.cpp to generate a function call.

This method generates a "call" instruction, then a "push" instruction to load
the argument, then the call_VM_base() method in the same class will generate
an additional "push" instruction loading a JavaThread pointer before
finally generating a "call" instruction to
InterpreterRuntime::prepare_native_call().
This will result in total of 3 words being pushed to the stack before the
call, misaligning it.

It is easy to see call paths that call C/C++ code with the stack not aligned
correctly - it turns out that HotSpot already has os::verify_stack_alignment()
method for this.
However, it is unexpectedly ifdef'ed out on 32-bit x86 - I would strongly
recommend enabling it there, too.

I have tried to fix the code generator but gave up - there are simply too
many places there needing a fix (pretty much every generated stack push in
functions that call C/C++ code has to be accounted for) - (preliminary) 
fixes that I have already done are in "icedtea-align-fix-codegen.patch".

In "icedtea-align-fix.patch" there is a workaround using gcc
"force_align_arg_pointer" function attribute that one can attach to C/C++ code
entry points so the compiler will realign the stack pointer automatically.

With this patch applied I was able to compile and run icedtea 3.6.0
successfully
with gcc 7.3.0 on 32-bit x86.
But since I am not a HotSpot expert I don't know if this is a proper solution.

I have also checked the current HotSpot hg repository, the relevant code there
is the same, just files are moved around

This issue is also tracked in Gentoo bug 647954
( https://bugs.gentoo.org/647954 ).

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/distro-pkg-dev/attachments/20180305/502c4be1/attachment.html>