RFR: 8170307: Stack size option -Xss is ignored
David Holmes
david.holmes at oracle.com
Wed Dec 14 05:49:04 UTC 2016
Hi Dan,
Thanks for the re-review. I apologize for losing the edits you
previously suggested.
More inline ...
On 14/12/2016 3:12 AM, Daniel D. Daugherty wrote:
> On 12/12/16 9:41 PM, David Holmes wrote:
>> Okay here's the updated webrev complete with nice logging:
>>
>> http://cr.openjdk.java.net/~dholmes/8170307/webrev.v2/
>
> src/os/linux/vm/os_linux.cpp
> L936: // a user-specified value known to be greater than the
> minimum needed.
> Perhaps: ... known to be at least the minimum needed.
Changed.
> L932: // can not do anything to emulate a larger stack than what
> has been provided by
> Typo: 'can not' -> 'cannot'
Changed.
> L936: // Mamimum stack size is the easy part, get it from
> RLIMIT_STACK
> Typo: 'Mamimum' -> 'Maximum'
> nit - please add a '.' to the end.
Fixed.
>
> L1125: SIZE_FORMAT "K, top=" INTPTR_FORMAT
> ", bottom=" INTPTR_FORMAT "\n",
> Does the logging subsystem convert the "\n" into the proper
> output for non-*NIX platforms, e.g., Windows?
No idea :) But that was leftover from when this was a ::printf (I wasn't
sure logging would work this early in VM init - but it does).
Removed.
> L1126: primordial ? "Primordial" : "User",
> max_size/K, _initial_thread_stack_size/K,
> Please add spaces around the div operator.
Changed.
> Any particular reason that "Primordial" and "User" start with
> upper case?
They used to be the first things printed. :) Fixed.
> Thumbs up!
>
> I don't need to see a new webrev if you decide to make the
> minor edits above.
Updated in place for the second reviewer (whomever they may be).
Thanks,
David
-----
> Dan
>
>
>>
>> The stack size will be the smaller of the rlimit stack and the
>> -Xss/ThreadStackSize value. If the rlimit stack is unlimited and
>> ThreadStackSize==0 then we clamp it at 8MB as we do on Solaris. So you
>> can now get whatever primordial thread stack size you want by using
>> ulimit and -Xss appropriately.
>>
>> Thanks,
>> David
>>
>> On 3/12/2016 2:11 PM, David Holmes wrote:
>>> On 3/12/2016 9:12 AM, Daniel D. Daugherty wrote:
>>>> On 12/1/16 10:51 PM, David Holmes wrote:
>>>>> Investigating this further the history is quite complex, especially
>>>>> when we start looking at other platforms. E.g. see
>>>>>
>>>>> https://bugs.openjdk.java.net/browse/JDK-6269555
>>>>>
>>>>> Solaris actually hard-wires an 8MB limit for the primordial thread.
>>>>>
>>>>> I'm very tempted to do the same on Linux.
>>>>
>>>> Vote: yes
>>>
>>> Excellent! Other votes?
>>>
>>>> This latest problem only comes up with -XX:ThreadStackSize=0 when the
>>>> stack is unlimited right?
>>>
>>> Right.
>>>
>>>> When -XX:ThreadStackSize=0 is specified, is taking the smaller of
>>>> 8MB or the ulimit a viable option?
>>>
>>> I think so.
>>>
>>>> Also, it looks like Hui had some things to say about not setting the
>>>> red/yellow zone pages on the primordial thread when we aren't using the
>>>> 'java' launcher because we don't know the environment of the code that
>>>> is using the JNI invocation API...
>>>
>>> Yeah but those comments seem a bit confused to me. They suggest we
>>> shouldn't add guard pages but in fact we do add guard pages. And to me
>>> it is no different in the primordial thread than any other natively
>>> attached thread ie why should the initially attached thread be treated
>>> differently to any other?** I suspect if I keep researching on this I
>>> will find bugs regarding such differences in behaviour (eg the fact that
>>> -Xss wasn't working on the main thread).
>>>
>>> ** There are arguments both ways as to how natively attached threads
>>> should behave. The main argument against guard page insertion is that we
>>> don't know how far down the existing stack we actually are - we could be
>>> past the depth where the guard page would be inserted! The main argument
>>> for (which seems to have won the day) is so that we don't get arbitrary
>>> differences in behaviour between threads created and attached by
>>> application native code; and threads created direct from application
>>> Java code.
>>>
>>> Anyway, simply upping the 2M limit on Linux to 8M seems a simple
>>> solution - assuming it addresses the needs of the folk that ran into
>>> this problem.
>>>
>>> Thanks,
>>> David
>>>
>>>> Dan
>>>>
>>>>
>>>>>
>>>>> David
>>>>> -----
>>>>>
>>>>> On 30/11/2016 6:46 PM, David Holmes wrote:
>>>>>> On 30/11/2016 6:17 PM, Thomas Stüfe wrote:
>>>>>>> On Wed, Nov 30, 2016 at 8:35 AM, David Holmes
>>>>>>> <david.holmes at oracle.com
>>>>>>> <mailto:david.holmes at oracle.com>> wrote:
>>>>>>>
>>>>>>> On 29/11/2016 10:25 PM, David Holmes wrote:
>>>>>>>
>>>>>>> I just realized I overlooked the case where
>>>>>>> ThreadStackSize=0
>>>>>>> and the
>>>>>>> stack is unlimited. In that case it isn't clear where the
>>>>>>> guard
>>>>>>> pages
>>>>>>> will get inserted - I do know that I don't get a
>>>>>>> stackoverflow
>>>>>>> error.
>>>>>>>
>>>>>>> This needs further investigation.
>>>>>>>
>>>>>>>
>>>>>>> So what happens here is that the massive stack-size causes
>>>>>>> stack-bottom to be higher than stack-top! So we will set a
>>>>>>> guard-page goodness knows where, and we can consume the current
>>>>>>> stack until such time as we hit an unmapped or protected
>>>>>>> region at
>>>>>>> which point we are killed.
>>>>>>>
>>>>>>> I'm not sure what to do here. My gut feel is that in such a
>>>>>>> case we
>>>>>>> should not attempt to create a guard page in the initial thread.
>>>>>>> That would require using a sentinel value for the stack-size.
>>>>>>> Though
>>>>>>> it also presents a problem for stack-bottom - which is
>>>>>>> implicitly
>>>>>>> zero. It may also give false positives in the
>>>>>>> is_initial_thread()
>>>>>>> check!
>>>>>>>
>>>>>>> Thoughts? Suggestions?
>>>>>>>
>>>>>>>
>>>>>>> Maybe I am overlooking something, but should
>>>>>>> os::capture_initial_thread() not call pthread_getattr_np() first to
>>>>>>> handle the case where the VM was created on a pthread which is
>>>>>>> not the
>>>>>>> primordial thread and may have a different stack size than what
>>>>>>> getrlimit returns? And fall back to getrlimit only if
>>>>>>> pthread_getattr_np() fails?
>>>>>>
>>>>>> My understanding of the problem (which likely no longer exists) is
>>>>>> that
>>>>>> pthread_getattr_np didn't fail as such but returned bogus values - so
>>>>>> the problem was not detectable and so we just had to not use
>>>>>> pthread_getattr_np.
>>>>>>
>>>>>>> And then we also should handle
>>>>>>> RLIM_INFINITY. For that case, I also think not setting guard pages
>>>>>>> would
>>>>>>> be safest.
>>>>>>>
>>>>>>> We also may just refuse to run in that case, because the workaround
>>>>>>> for
>>>>>>> the user is easy - just set the limit before process start. Note
>>>>>>> that on
>>>>>>> AIX, we currently refuse to run on the primordial thread because it
>>>>>>> may
>>>>>>> have different page sizes than pthreads and it is impossible to get
>>>>>>> the
>>>>>>> exact stack locations.
>>>>>>
>>>>>> I was wondering why the AIX set up seemed so simple in comparison :)
>>>>>>
>>>>>> Thanks,
>>>>>> David
>>>>>>
>>>>>>>
>>>>>>> Thomas
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> David
>>>>>>>
>>>>>>> On 29/11/2016 9:59 PM, David Holmes wrote:
>>>>>>>
>>>>>>> Hi Thomas,
>>>>>>>
>>>>>>> On 29/11/2016 8:39 PM, Thomas Stüfe wrote:
>>>>>>>
>>>>>>> Hi David,
>>>>>>>
>>>>>>> thanks for the good explanation. Change looks
>>>>>>> good, I
>>>>>>> really like the
>>>>>>> comment in capture_initial_stack().
>>>>>>>
>>>>>>> Question, with -Xss given and being smaller than
>>>>>>> current
>>>>>>> thread stack
>>>>>>> size, guard pages may appear in the middle of the
>>>>>>> invoking thread stack?
>>>>>>> I always thought this is a bit dangerous. If your
>>>>>>> model
>>>>>>> is to have the
>>>>>>> VM created from the main thread, which then goes
>>>>>>> off to
>>>>>>> do different
>>>>>>> things, and have other threads then attach and run
>>>>>>> java
>>>>>>> code, main
>>>>>>> thread later may crash in unrelated native code just
>>>>>>> because it reached
>>>>>>> the stack depth of the hava threads? Or am I
>>>>>>> misunderstanding something?
>>>>>>>
>>>>>>>
>>>>>>> There is no change to the general behaviour other than
>>>>>>> allowing a
>>>>>>> primordial process thread that launches the VM, to
>>>>>>> now not
>>>>>>> have an
>>>>>>> effective stack limited at 2MB. The current logic will
>>>>>>> insert guard
>>>>>>> pages where ever -Xss states (as long as less than 2MB
>>>>>>> else
>>>>>>> 2MB), while
>>>>>>> with the fix the guard pages will be inserted above 2MB
>>>>>>> - as
>>>>>>> dictated by
>>>>>>> -Xss.
>>>>>>>
>>>>>>> David
>>>>>>> -----
>>>>>>>
>>>>>>> Thanks, Thomas
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Nov 25, 2016 at 11:38 AM, David Holmes
>>>>>>> <david.holmes at oracle.com
>>>>>>> <mailto:david.holmes at oracle.com>
>>>>>>> <mailto:david.holmes at oracle.com
>>>>>>> <mailto:david.holmes at oracle.com>>> wrote:
>>>>>>>
>>>>>>> Bug:
>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8170307
>>>>>>> <https://bugs.openjdk.java.net/browse/JDK-8170307>
>>>>>>> <https://bugs.openjdk.java.net/browse/JDK-8170307
>>>>>>> <https://bugs.openjdk.java.net/browse/JDK-8170307>>
>>>>>>>
>>>>>>> The bug is not public unfortunately for
>>>>>>> non-technical reasons - but
>>>>>>> see my eval below.
>>>>>>>
>>>>>>> Background: if you load the JVM from the
>>>>>>> primordial
>>>>>>> thread of a
>>>>>>> process (not done by the java launcher since JDK
>>>>>>> 6),
>>>>>>> there is an
>>>>>>> artificial stack limit imposed on the initial
>>>>>>> thread
>>>>>>> (by sticking
>>>>>>> the guard page at the limit position of the
>>>>>>> actual
>>>>>>> stack) of the
>>>>>>> minimum of the -Xss setting and 2M. So if you
>>>>>>> set
>>>>>>> -Xss to > 2M it is
>>>>>>> ignored for the main thread even if the true
>>>>>>> stack
>>>>>>> is, say, 8M. This
>>>>>>> limitation dates back 10-15 years and is no
>>>>>>> longer
>>>>>>> relevant today
>>>>>>> and should be removed (see below). I've also
>>>>>>> added
>>>>>>> additional
>>>>>>> explanatory notes.
>>>>>>>
>>>>>>> webrev:
>>>>>>> http://cr.openjdk.java.net/~dholmes/8170307/webrev/
>>>>>>> <http://cr.openjdk.java.net/~dholmes/8170307/webrev/>
>>>>>>> <http://cr.openjdk.java.net/~dholmes/8170307/webrev/
>>>>>>> <http://cr.openjdk.java.net/~dholmes/8170307/webrev/>>
>>>>>>>
>>>>>>> Testing was manually done by modifying the
>>>>>>> launcher
>>>>>>> to not run the
>>>>>>> VM in a new thread, and checking the resulting
>>>>>>> stack
>>>>>>> size used.
>>>>>>>
>>>>>>> This change will only affect hosted JVMs
>>>>>>> launched
>>>>>>> with a -Xss value
>>>>>>> > 2M.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> David
>>>>>>> -----
>>>>>>>
>>>>>>> Bug eval:
>>>>>>>
>>>>>>> JDK-4441425 limits the stack to 8M as a
>>>>>>> safeguard
>>>>>>> against an
>>>>>>> unlimited value from getrlimit in 1.3.1, but
>>>>>>> further
>>>>>>> constrained
>>>>>>> that to 2M in 1.4.0 due to JDK-4466587.
>>>>>>>
>>>>>>> By 1.4.2 we have the basic form of the current
>>>>>>> problematic code:
>>>>>>>
>>>>>>> #ifndef IA64
>>>>>>> if (rlim.rlim_cur > 2 * K * K) rlim.rlim_cur =
>>>>>>> 2 *
>>>>>>> K * K;
>>>>>>> #else
>>>>>>> // Problem still exists RH7.2 (IA64 anyway)
>>>>>>> but
>>>>>>> 2MB is a little
>>>>>>> small
>>>>>>> if (rlim.rlim_cur > 4 * K * K) rlim.rlim_cur =
>>>>>>> 4 *
>>>>>>> K * K;
>>>>>>> #endif
>>>>>>>
>>>>>>> _initial_thread_stack_size = rlim.rlim_cur &
>>>>>>> ~(page_size() - 1);
>>>>>>>
>>>>>>> if (max_size && _initial_thread_stack_size >
>>>>>>> max_size) {
>>>>>>> _initial_thread_stack_size = max_size;
>>>>>>> }
>>>>>>>
>>>>>>> This was added by JDK-4678676 to allow the
>>>>>>> stack of
>>>>>>> the main thread
>>>>>>> to be _reduced_ below the default 2M/4M if the
>>>>>>> -Xss
>>>>>>> value was
>>>>>>> smaller than that.** There was no intent to
>>>>>>> allow
>>>>>>> the stack size to
>>>>>>> follow -Xss arbitrarily due to the operational
>>>>>>> constraints imposed
>>>>>>> by the OS/glibc at the time when dealing with
>>>>>>> the
>>>>>>> primordial process
>>>>>>> thread.
>>>>>>>
>>>>>>> ** It could not actually change the actual stack
>>>>>>> size of course, but
>>>>>>> set the guard pages to limit use to the expected
>>>>>>> stack size.
>>>>>>>
>>>>>>> In JDK 6, under JDK-6316197, the launcher was
>>>>>>> changed to create the
>>>>>>> JVM in a new thread, so that it was not
>>>>>>> limited by
>>>>>>> the
>>>>>>> idiosyncracies of the OS or thread library
>>>>>>> primordial thread
>>>>>>> handling. However, the stack size limitations
>>>>>>> remained in place in
>>>>>>> case the VM was launched from the primordial
>>>>>>> thread
>>>>>>> of a user
>>>>>>> application via the JNI invocation API.
>>>>>>>
>>>>>>> I believe it should be safe to remove the 2M
>>>>>>> limitation now.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>
>
More information about the hotspot-dev
mailing list