Stack guard pages

[Andrew John Hughes]

On 23 February 2010 16:16, Andrew Haley <aph at redhat.com> wrote:
> I've been working on a bug in OpenOffice where using Java causes a
> later crash.  https://bugzilla.novell.com/show_bug.cgi?id=578802
>
> It's a very strange bug: Java is called, does something, and then the
> thread is detached from the Java VM.  Long after, a segfault occurs.
>
> The reason for this crash is the way that stack guard pages work.
> When DetachCurrentThread() calls
> JavaThread::remove_stack_guard_pages(), the guard pages are not
> removed.  So, if at some point later in the process the stack grows
> beyond the point where the guard pages were mapped, the Linux kernel
> cannot expand the stack any further because the guard pages are in the
> way, and a segfault occurs.
>
> I've attached a reproducer for this bug to the end of this message.
> It crashes on many of the Linux systems I've tried.
>
> The right way to fix this is for remove_stack_guard_pages() to
> munmap() the guard pages.  However, it's essential not to split the
> stack region, so if the stack has already grown beyond the guard
> pages, we have to unmap() it.  Like so:
>
> bool os::create_stack_guard_pages(char* addr, size_t size) {
>  uintptr_t stack_extent, stack_base;
>  // get_stack_bounds() returns the memory extent mapped for the stack
>  // by the kernel.
>  if (get_stack_bounds(&stack_extent, &stack_base)) {
>    if (stack_extent < (uintptr_t)addr)
>      ::munmap((void*)stack_extent, (uintptr_t)addr - stack_extent);
>  }
>
>  return os::commit_memory(addr, size);
> }
>
> bool os::remove_stack_guard_pages(char* addr, size_t size) {
>  return ::munmap(addr, size) == 0;
> }
>
> This fixes the bug.  Unfortunately, there is no os:: interface for
> create_stack_guard_pages() and remove_stack_guard_pages(), so this
> solution doesn't fit well with the current organization of the VM.  I
> can't see any way of making this work only by touching os_linux.?pp .
>
> I could simply patch OpenJDK locally for the Linux distros, but I
> think we should have this discussion first.
>
> Andrew.
>

Segfaults here with the latest OpenJDK7 b84:

$ /mnt/builder/jdk7/j2sdk-image/bin/java -version
openjdk version "1.7.0-internal"
OpenJDK Runtime Environment (build 1.7.0-internal-andrew_2010_02_23_18_45-b00)
OpenJDK 64-Bit Server VM (build 17.0-b09, mixed mode)

$ LD_LIBRARY_PATH=/mnt/builder/jdk7/j2sdk-image/jre/lib/amd64/server ./invoke
Hello
Segmentation fault

I think it would it would be clearer what the suggested fix entails if
you posted a diff or webrev of the suggested change.

Cheers,
-- 
Andrew :-)

Free Java Software Engineer
Red Hat, Inc. (http://www.redhat.com)

Support Free Java!
Contribute to GNU Classpath and the OpenJDK
http://www.gnu.org/software/classpath
http://openjdk.java.net

PGP Key: 94EFD9D8 (http://subkeys.pgp.net)
Fingerprint: F8EF F1EA 401E 2E60 15FA  7927 142C 2591 94EF D9D8