RFR: 8170307: Stack size option -Xss is ignored

[David Holmes]

Thanks for the review Dan. Unfortunately I overlooked one case - see my 
other emails. :)

Cheers,
David

On 30/11/2016 3:57 AM, Daniel D. Daugherty wrote:
> Sorry for being late to this party! Seems like thread stack sizes are
> very much on folks minds lately...
>
>> webrev: http://cr.openjdk.java.net/~dholmes/8170307/webrev/
>
> src/os/linux/vm/os_linux.cpp
>     L936:   // a user-specified value known to be greater than the
> minimum needed.
>         Perhaps: ... known to be at least the minimum needed.
>
>         As enforced by this code in os::Posix::set_minimum_stack_sizes():
>
>         _java_thread_min_stack_allowed =
> MAX2(_java_thread_min_stack_allowed,
> JavaThread::stack_guard_zone_size() +
> JavaThread::stack_shadow_zone_size() +
>                                               (4 * BytesPerWord
> COMPILER2_PRESENT(+ 2)) * 4 * K);
>
>         _java_thread_min_stack_allowed =
> align_size_up(_java_thread_min_stack_allowed, vm_page_size());
>
>         size_t stack_size_in_bytes = ThreadStackSize * K;
>         if (stack_size_in_bytes != 0 &&
>             stack_size_in_bytes < _java_thread_min_stack_allowed) {
>           // The '-Xss' and '-XX:ThreadStackSize=N' options both set
>           // ThreadStackSize so we go with "Java thread stack size" instead
>           // of "ThreadStackSize" to be more friendly.
>           tty->print_cr("\nThe Java thread stack size specified is too
> small. "
>                         "Specify at least " SIZE_FORMAT "k",
>                         _java_thread_min_stack_allowed / K);
>           return JNI_ERR;
>         }
>
>     L939:   // can not do anything to emulate a larger stack than what
> has been provided by
>         Typo: 'can not' -> 'cannot'
>
>     L943:   // Mamimum stack size is the easy part, get it from
> RLIMIT_STACK
>         Typo: 'Mamimum' -> 'Maximum'
>         nit - please add a '.' to the end.
>
>
> Thumbs up!
>
> I don't need to see a new webrev if you decide to make the
> minor edits above.
>
> Dan
>
>
>
> On 11/29/16 5:25 AM, David Holmes wrote:
>> I just realized I overlooked the case where ThreadStackSize=0 and the
>> stack is unlimited. In that case it isn't clear where the guard pages
>> will get inserted - I do know that I don't get a stackoverflow error.
>>
>> This needs further investigation.
>>
>> David
>>
>> On 29/11/2016 9:59 PM, David Holmes wrote:
>>> Hi Thomas,
>>>
>>> On 29/11/2016 8:39 PM, Thomas Stüfe wrote:
>>>> Hi David,
>>>>
>>>> thanks for the good explanation. Change looks good, I really like the
>>>> comment in capture_initial_stack().
>>>>
>>>> Question, with -Xss given and being smaller than current thread stack
>>>> size, guard pages may appear in the middle of the invoking thread
>>>> stack?
>>>> I always thought this is a bit dangerous. If your model is to have the
>>>> VM created from the main thread, which then goes off to do different
>>>> things, and have other threads then attach and run java code, main
>>>> thread later may crash in unrelated native code just because it reached
>>>> the stack depth of the hava threads? Or am I misunderstanding
>>>> something?
>>>
>>> There is no change to the general behaviour other than allowing a
>>> primordial process thread that launches the VM, to now not have an
>>> effective stack limited at 2MB. The current logic will insert guard
>>> pages where ever -Xss states (as long as less than 2MB else 2MB), while
>>> with the fix the guard pages will be inserted above 2MB - as dictated by
>>> -Xss.
>>>
>>> David
>>> -----
>>>
>>>> Thanks, Thomas
>>>>
>>>>
>>>> On Fri, Nov 25, 2016 at 11:38 AM, David Holmes <david.holmes at oracle.com
>>>> <mailto:david.holmes at oracle.com>> wrote:
>>>>
>>>>     Bug: https://bugs.openjdk.java.net/browse/JDK-8170307
>>>>     <https://bugs.openjdk.java.net/browse/JDK-8170307>
>>>>
>>>>     The bug is not public unfortunately for non-technical reasons - but
>>>>     see my eval below.
>>>>
>>>>     Background: if you load the JVM from the primordial thread of a
>>>>     process (not done by the java launcher since JDK 6), there is an
>>>>     artificial stack limit imposed on the initial thread (by sticking
>>>>     the guard page at the limit position of the actual stack) of the
>>>>     minimum of the -Xss setting and 2M. So if you set -Xss to > 2M
>>>> it is
>>>>     ignored for the main thread even if the true stack is, say, 8M.
>>>> This
>>>>     limitation dates back 10-15 years and is no longer relevant today
>>>>     and should be removed (see below). I've also added additional
>>>>     explanatory notes.
>>>>
>>>>     webrev: http://cr.openjdk.java.net/~dholmes/8170307/webrev/
>>>> <http://cr.openjdk.java.net/~dholmes/8170307/webrev/>
>>>>
>>>>     Testing was manually done by modifying the launcher to not run the
>>>>     VM in a new thread, and checking the resulting stack size used.
>>>>
>>>>     This change will only affect hosted JVMs launched with a -Xss value
>>>>     > 2M.
>>>>
>>>>     Thanks,
>>>>     David
>>>>     -----
>>>>
>>>>     Bug eval:
>>>>
>>>>     JDK-4441425 limits the stack to 8M as a safeguard against an
>>>>     unlimited value from getrlimit in 1.3.1, but further constrained
>>>>     that to 2M in 1.4.0 due to JDK-4466587.
>>>>
>>>>     By 1.4.2 we have the basic form of the current problematic code:
>>>>
>>>>     #ifndef IA64
>>>>       if (rlim.rlim_cur > 2 * K * K) rlim.rlim_cur = 2 * K * K;
>>>>     #else
>>>>       // Problem still exists RH7.2 (IA64 anyway) but 2MB is a little
>>>> small
>>>>       if (rlim.rlim_cur > 4 * K * K) rlim.rlim_cur = 4 * K * K;
>>>>     #endif
>>>>
>>>>       _initial_thread_stack_size = rlim.rlim_cur & ~(page_size() - 1);
>>>>
>>>>       if (max_size && _initial_thread_stack_size > max_size) {
>>>>          _initial_thread_stack_size = max_size;
>>>>       }
>>>>
>>>>     This was added by JDK-4678676 to allow the stack of the main thread
>>>>     to be _reduced_ below the default 2M/4M if the -Xss value was
>>>>     smaller than that.** There was no intent to allow the stack size to
>>>>     follow -Xss arbitrarily due to the operational constraints imposed
>>>>     by the OS/glibc at the time when dealing with the primordial
>>>> process
>>>>     thread.
>>>>
>>>>     ** It could not actually change the actual stack size of course,
>>>> but
>>>>     set the guard pages to limit use to the expected stack size.
>>>>
>>>>     In JDK 6, under JDK-6316197, the launcher was changed to create the
>>>>     JVM in a new thread, so that it was not limited by the
>>>>     idiosyncracies of the OS or thread library primordial thread
>>>>     handling. However, the stack size limitations remained in place in
>>>>     case the VM was launched from the primordial thread of a user
>>>>     application via the JNI invocation API.
>>>>
>>>>     I believe it should be safe to remove the 2M limitation now.
>>>>
>>>>
>