JVM inserting guard pages into stack of initial thread - will that stop with Java 10?

[Doerr, Martin]

Hi everbody,

after reading several mails about guard page problems (and quite some own bad experience), I wonder if they are really worth all the pain. Is the performance benefit from stack banging so great, that it justifies the tremendous complexity?

If not, I'd vote for replacing them by explicit checks. Loading a limit from the Thread and performing a comparison doesn't sound so expensive.

It could be the case that some platforms need banging (e.g. Windows due to on demand committed stack), but probably not all.
Am I missing anything?

Best regards,
Martin


-----Original Message-----
From: hotspot-runtime-dev [mailto:hotspot-runtime-dev-bounces at openjdk.java.net] On Behalf Of Tomas Kalibera
Sent: Dienstag, 10. Oktober 2017 11:36
To: hotspot-runtime-dev at openjdk.java.net
Subject: JVM inserting guard pages into stack of initial thread - will that stop with Java 10?

Dear hotspot developers,

I am one of the core R developers and what has been causing us a lot of 
trouble is the 2M cap for stack size of the initial thread on Linux. In 
short, R has its own stack overflow checking, the stack size is taken 
from rlimit, and R needs a lot of stack (often more than 2M). Via rJava 
package, R uses Java, so it initializes Java via JNI. As a workaround 
for some old Linux problem, hotspot caps the stack at 2M by inserting 
guard pages. Consequently, R crashes when the recursion gets too deep, 
and the crash is ugly as R's stack overflow checking does not now about 
the shrinkage of the stack. Users have no chance of diagnosing what's 
wrong and it is causing us trouble increasingly more lately, as we have 
more recursive calls (JIT implemented in R) and as newer gcc versions 
produce bigger frames from our interpreter loop. The problem is still 
present in Java 9.

We're so hopeless that I implemented a workaround for rJava where we 
fill up the stack just enough before initializing the JVM so that 
is_initial_thread returns false even for the initial thread, so the 
guard page is not inserted. This is probably something we will use for 
now - or is there a simpler workaround? (the code is here, but probably 
you get the idea just from the description of it: 
https://github.com/s-u/rJava/pull/102/files)

As you surely know well, the 2M cap comes from a bug report: 
http://bugs.java.com/view_bug.do?bug_id=4466587
And there is a cleanup of this code discussed in
http://openjdk.5641.n7.nabble.com/RFR-8170307-Stack-size-option-Xss-is-ignored-td292960.html

If I understand correctly the discussion, this is the new version for 
Java 10 - is that correct?
http://cr.openjdk.java.net/~dholmes/8170307/webrev.v2/src/os/linux/vm/os_linux.cpp.udiff.html

Now, if I am reading this patch correctly, I think that the 2M cap is 
gone. There is a new 8M cap, which however only applies when the rlimit 
stack size is unlimited, which is something we could probably live with. 
Is this a correct interpretation of the code? If so, we should be fine 
for Java 10.

Even though, having an option to initialize the JVM without stack 
overflow checking for the initial thread might be even better... for 
cases like ours, when we have our own stack overflow checking.

Thanks,
Tomas