RFR(M): 8169373: Work around linux NPTL stack guard error.

[Lindenmaier, Goetz]

Hi,

Please review this change. I please need a sponsor:
http://cr.openjdk.java.net/~goetz/wr16/8169373-ppc-stackFix/webrev.01/

In the Linux NPTL pthread implementation the guard size mechanism is not implemented properly. The posix standard requires to add the size of the guard pages to the stack size, instead Linux takes the space out of 'stacksize'.

The Posix standard http://pubs.opengroup.org/onlinepubs/9699919799/  says "the implementation allocates extra memory at the overflow end of the stack". The linux man page https://linux.die.net/man/3/pthread_attr_setguardsize says "As at glibc 2.8, the NPTL threading implementation includes the guard area within the stack size allocation, rather than allocating extra space at the end of the stack, as POSIX.1 requires".

I encounter this problem in runtime/Thread/TooSmallStackSize.java on ppc with 64K pages. _compiler_thread_min_stack_allowed is 128K on ppc, and ppc specifies two OS guard pages. The VM crashes in pthread creation because there is no usable space in the thread stack after allocating the guard pages.
But TooSmallStackSize.java requires that the VM comes up with the stack size mentioned in the error message.

This fix adapts the requested stack size on Linux by the size of the guard pages to mimick proper behaviour, see change to os_linux.cpp.

The change also streamlines usage of stack_guard_page on linuxppc, linuxppcle, aixppc and linuxs390.

To reproduce the error on linux_x86_64, apply below patch and call the VM with -XX:CompilerThreadStackSize=64.

I'm still exploring why I had to choose such big compiler stacks on ppc to get -version passing, but I wanted to send the RFR now as people obviously looked at the bug I opened (Thanks David!).

Best regards,
  Goetz.



diff -r b7ae012c55c3 src/os_cpu/linux_x86/vm/os_linux_x86.cpp
--- a/src/os_cpu/linux_x86/vm/os_linux_x86.cpp  Mon Nov 07 12:37:28 2016 +0100
+++ b/src/os_cpu/linux_x86/vm/os_linux_x86.cpp  Thu Nov 10 16:52:17 2016 +0100
@@ -701,7 +701,7 @@
size_t os::Linux::default_guard_size(os::ThreadType thr_type) {
   // Creating guard page is very expensive. Java thread has HotSpot
   // guard page, only enable glibc guard page for non-Java threads.
-  return (thr_type == java_thread ? 0 : page_size());
+  return (thr_type == java_thread ? 0 : 64*K);
}

// Java thread: