RFR[9u-dev] 8130425: libjvm crash due to stack overflow in executables with 32k tbss/tdata

[cheleswer sahu]

Hi,

Please review the code changes for 
"https://bugs.openjdk.java.net/browse/JDK-8130425".

Problem summary: When a large TLS (Thread Local Storage) size is set for 
threads,  JVM is throwing StackOverflow exception.

Problem Identified:
As per investigation and a discussion we came to the conclusion that 
issue is not with the JVM but it lies in the way glibc has been 
implemented. When a TLS is declared , it steals the space from threads 
stack size.
So if a thread is created with small stack size, and TLS is set to a 
large value, then it will result in StackOverflow. This is the exact 
case in this bug where reaper thread is allocated a very low stack size 
32768.

Discussion thread: 
http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-December/037558.html 


Solution proposed:
Its expected to get fix in glibc sometime , but for now I propose a 
workaround, a boolean system property "processReaperUseDefaultStackSize" 
using which we can set the stack size for reaper thread to default
instead of fix 32768. This property can be set by the user using "-D" or 
"System.setProperty()". I have tested this fix, it works well with TLS 
size between 32k to 128k.

Fix:
diff -r 5c4530bb9ae6 
src/java.base/share/classes/java/lang/ProcessHandleImpl.jav
--- a/src/java.base/share/classes/java/lang/ProcessHandleImpl.java Fri 
Jan 08 13:06:29 2016 +0800
+++ b/src/java.base/share/classes/java/lang/ProcessHandleImpl.java Tue 
Jan 12 15:55:50 2016 +0530
@@ -83,9 +83,13 @@
                  ThreadGroup systemThreadGroup = tg;

                  ThreadFactory threadFactory = grimReaper -> {
-                    // Our thread stack requirement is quite modest.
-                    Thread t = new Thread(systemThreadGroup, grimReaper,
-                            "process reaper", 32768);
+                   Thread t = null;
+                   if 
(Boolean.getBoolean("processReaperUseDefaultStackSize")) {
+                       t = new Thread(systemThreadGroup, grimReaper, 
"process reaper");
+                    } else {
+                       // Our thread stack requirement is quite modest.
+                       t = new Thread(systemThreadGroup, grimReaper, 
"process reaper", 32768);
+                    }
                      t.setDaemon(true);
                      // A small attempt (probably futile) to avoid 
priority inversion
                      t.setPriority(Thread.MAX_PRIORITY);


Regards,
Cheleswer