RFR: 8197429: Increased stack guard causes segfaults on x86-32

[Leonid Mesnik]

Hi Andrew

Could you please update bug https://bugs.openjdk.java.net/browse/JDK-8197429
with affected and fixed version of jdk.  It is unclear where you plan to push this fix. I assume that you are going to push into jdk/hs repo. 
(from your webrev: Compare against:http://hg.openjdk.java.net/jdk/hs)

I looked on the tests only. Here are my comments:

1) There are no copyrights for new files.

2) File
http://cr.openjdk.java.net/~aph/8197429-1/test/hotspot/jtreg/runtime/8197429/foo.java.html
 <http://cr.openjdk.java.net/~aph/8197429-1/test/hotspot/jtreg/runtime/8197429/foo.java.html>contains the same code as in T.java and doesn’t seems to be used.


3) The preferable name for test is some meaningful and rather then bug-id.
Would it possible to replace runtime/8197429 with something like runtime/segfaults ?

4) Currently the correct way to use native libs is to compile it during build and use with -nativepath. 
See make examples here:
http://hg.openjdk.java.net/jdk/jdk10/file/tip/make/test/JtregNativeHotspot.gmk
 <http://hg.openjdk.java.net/jdk/jdk10/file/tip/make/test/JtregNativeHotspot.gmk>http://hg.openjdk.java.net/jdk/jdk10/file/tip/test/hotspot/jtreg/runtime/jni/CalleeSavedRegisters <http://hg.openjdk.java.net/jdk/jdk10/file/tip/test/hotspot/jtreg/runtime/jni/CalleeSavedRegisters>
So you might rewrite your test completely on java. So you could use requires tag to filter out unsupported platforms.
Also logic of choosing platform will be slightly different. 

Leonid

> On Feb 9, 2018, at 8:51 AM, Andrew Haley <aph at redhat.com> wrote:
> 
> 32-bit Linux x86 HotSpot allocates an executable memory region just
> below the end of the stack.  This is a workaround for JDK-8023956,
> which in turn relates to a bug in the RHEL 5 & 6 kernels on old
> (pre-NX) CPUs:
> 
>  To summarize: to emulate NX feature on X86_32 code segment is used
>  to limit execution to the highest executable VA. There is a tiny
>  race on SMP MM invalidation code which can cause the lazy CS update
>  code in trap handling to think a general protection fault wasn't
>  cause by itself. This results in sending the JVM a useless SIGSEGV
>  with si_code:SI_KERNEL, results in JVM signal handling forcing a
>  dump.
> 
>  The suggested work around (limited to 32 bit Linux): is to enable
>  execution (PROT_EXEC) on a high address and execute some code.
> 
> To be more precise: on 32-bit Linux kernels the top of the main stack
> is at about 3G (0xC0000000), and HotSpot creates an executable mapping
> of a single page just a little way below the main stack and executes
> an instruction in it.  (It then leaves the region mapped; I do not
> know why.  It could be that the region could be removed at this
> point.)
> 
> Some new Linux kernels by default have a stack guard of a megabyte
> between the main stack and any allocated memory region.  See
> CVE-2017-1000364.  (Note that this megabyte is a default: it can be
> changed at boot time.)
> 
> So, when the stack grows to within a megabyte of the executable region
> HotSpot installed, the process segfaults and is killed.  This only
> happens when we're running on the main stack, and that only happens
> when using the JNI invocation interface.
> 
> 
> I have looked at several ways to fix this.  One was to probe to find
> out what the stack guard size is, and to place the executable mapping
> an appropriate distance from the stack; this can be done, but it is
> complex.  I believe it's also unnecessary, because the workaround for
> JDK-8023956 isn't needed with the newer kernels that have the larger
> stack guard gap.
> 
> The fix I'm proposing here first bangs down the stack to the Java
> stack limit, then tries to map the executable memory region.  On
> systems with a large stack guard gap this mapping attempt will fail,
> and we return and continue.  On older systems which do not have a
> large stack gap it will continue and install the executable memory
> region.
> 
> There are other possible fixes.  Rather than failing, we could loop
> trying to install the executable mapping until we succeed.
> 
> http://cr.openjdk.java.net/~aph/8197429-1/
> 
> -- 
> Andrew Haley
> Java Platform Lead Engineer
> Red Hat UK Ltd. <https://www.redhat.com>
> EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671