JVM inserting guard pages into stack of initial thread - will that stop with Java 10?

[Thomas Stüfe]

Rereading I found my first answer a bit shortish and I wish to elaborate a
bit on my answer, even though David is probably in a better position to do
so.

We had a discussion about this on hotspot.runtime (
http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2017-October/thread.html#24859),
AFAIK David plans to implement the switch to disable guard stack creation
on the primordial thread.

Both jdk9 and jdk10 already contain David's fix for 8170307, which changes
the clamp-down size for the primordial thread on Linux to 8M.

Latest versions are here:
http://hg.openjdk.java.net/jdk9/hs/hotspot/file/c68024d52834/src/os/linux/vm/os_linux.cpp
http://hg.openjdk.java.net/jdk10/hs/file/1aecd400f2fa/src/hotspot/os/linux/os_linux.cpp


Kind Regards, Thomas

On Tue, Oct 10, 2017 at 11:36 AM, Tomas Kalibera <tomas.kalibera at gmail.com>
wrote:

> Dear hotspot developers,
>
> I am one of the core R developers and what has been causing us a lot of
> trouble is the 2M cap for stack size of the initial thread on Linux. In
> short, R has its own stack overflow checking, the stack size is taken from
> rlimit, and R needs a lot of stack (often more than 2M). Via rJava package,
> R uses Java, so it initializes Java via JNI. As a workaround for some old
> Linux problem, hotspot caps the stack at 2M by inserting guard pages.
> Consequently, R crashes when the recursion gets too deep, and the crash is
> ugly as R's stack overflow checking does not now about the shrinkage of the
> stack. Users have no chance of diagnosing what's wrong and it is causing us
> trouble increasingly more lately, as we have more recursive calls (JIT
> implemented in R) and as newer gcc versions produce bigger frames from our
> interpreter loop. The problem is still present in Java 9.
>
> We're so hopeless that I implemented a workaround for rJava where we fill
> up the stack just enough before initializing the JVM so that
> is_initial_thread returns false even for the initial thread, so the guard
> page is not inserted. This is probably something we will use for now - or
> is there a simpler workaround? (the code is here, but probably you get the
> idea just from the description of it: https://github.com/s-u/rJava/p
> ull/102/files)
>


> As you surely know well, the 2M cap comes from a bug report:
> http://bugs.java.com/view_bug.do?bug_id=4466587
> And there is a cleanup of this code discussed in
> http://openjdk.5641.n7.nabble.com/RFR-8170307-Stack-size-opt
> ion-Xss-is-ignored-td292960.html
>
> If I understand correctly the discussion, this is the new version for Java
> 10 - is that correct?
> http://cr.openjdk.java.net/~dholmes/8170307/webrev.v2/src/os
> /linux/vm/os_linux.cpp.udiff.html
>
> Now, if I am reading this patch correctly, I think that the 2M cap is
> gone. There is a new 8M cap, which however only applies when the rlimit
> stack size is unlimited, which is something we could probably live with. Is
> this a correct interpretation of the code? If so, we should be fine for
> Java 10.
>
> Even though, having an option to initialize the JVM without stack overflow
> checking for the initial thread might be even better... for cases like
> ours, when we have our own stack overflow checking.
>
> Thanks,
> Tomas
>
>