RFR[9u-dev] 8130425: libjvm crash due to stack overflow in executables with 32k tbss/tdata

[cheleswer sahu]

Adding core-libs-dev and hotspot-runtime-dev team .

On 1/14/2016 12:24 AM, Martin Buchholz wrote:
> You should include core-libs-dev (and perhaps hotspot-runtime-dev) in
> this thread.  You're changing core library code.
>
> On Wed, Jan 13, 2016 at 2:25 AM, cheleswer sahu
> <cheleswer.sahu at oracle.com> wrote:
>> Hi,
>>
>> Please review the code changes for
>> "https://bugs.openjdk.java.net/browse/JDK-8130425".
>>
>> Problem summary: When a large TLS (Thread Local Storage) size is set for
>> threads,  JVM is throwing StackOverflow exception.
>>
>> Problem Identified:
>> As per investigation and a discussion we came to the conclusion that issue
>> is not with the JVM but it lies in the way glibc has been implemented. When
>> a TLS is declared , it steals the space from threads stack size.
>> So if a thread is created with small stack size, and TLS is set to a large
>> value, then it will result in StackOverflow. This is the exact case in this
>> bug where reaper thread is allocated a very low stack size 32768.
>>
>> Discussion thread:
>> http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-December/037558.html
>>
>> Solution proposed:
>> Its expected to get fix in glibc sometime , but for now I propose a
>> workaround, a boolean system property "processReaperUseDefaultStackSize"
>> using which we can set the stack size for reaper thread to default
>> instead of fix 32768. This property can be set by the user using "-D" or
>> "System.setProperty()". I have tested this fix, it works well with TLS size
>> between 32k to 128k.
>>
>> Fix:
>> diff -r 5c4530bb9ae6
>> src/java.base/share/classes/java/lang/ProcessHandleImpl.jav
>> --- a/src/java.base/share/classes/java/lang/ProcessHandleImpl.java Fri Jan
>> 08 13:06:29 2016 +0800
>> +++ b/src/java.base/share/classes/java/lang/ProcessHandleImpl.java Tue Jan
>> 12 15:55:50 2016 +0530
>> @@ -83,9 +83,13 @@
>>                   ThreadGroup systemThreadGroup = tg;
>>
>>                   ThreadFactory threadFactory = grimReaper -> {
>> -                    // Our thread stack requirement is quite modest.
>> -                    Thread t = new Thread(systemThreadGroup, grimReaper,
>> -                            "process reaper", 32768);
>> +                   Thread t = null;
>> +                   if
>> (Boolean.getBoolean("processReaperUseDefaultStackSize")) {
>> +                       t = new Thread(systemThreadGroup, grimReaper,
>> "process reaper");
>> +                    } else {
>> +                       // Our thread stack requirement is quite modest.
>> +                       t = new Thread(systemThreadGroup, grimReaper,
>> "process reaper", 32768);
>> +                    }
>>                       t.setDaemon(true);
>>                       // A small attempt (probably futile) to avoid priority
>> inversion
>>                       t.setPriority(Thread.MAX_PRIORITY);
>>
>>
>> Regards,
>> Cheleswer