RFR: 8263476: Use reserved memory for stack guard pages [v3]

[Albert Mingkun Yang]

On Fri, 8 Aug 2025 15:20:13 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

> Why we first unmap the stack area to then re-map it again, leading to the potential race I mentioned.

To ensure the reserved stack memory is contiguous, after the stack-guard-pages are removed (unguarded).
The old code also performs `munmap` as well, and the reasoning is the same, covered in the doc right before `os::pd_create_stack_guard_pages` in Linux.

> Sorry, you lost me. What is stack_top in your description? Lowest or highest address?

The lowest-address. This is the C code I used: stack-memory for main-thread is reserved ("stolen") by others and main-thread encounters segfault. Ofc, the more realistic racing is that another thread steals the stack-memory, but this is just to show the idea.

<details>

<summary>C source code</summary>


#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <unistd.h>
#include <sys/mman.h>

int main(void) {
    size_t page_size = sysconf(_SC_PAGESIZE);
    if (page_size == (size_t)-1) {
        perror("sysconf");
        return 1;
    }

    // Current stack address
    void *stack_addr = __builtin_frame_address(0);

    // Pick a page ~1MB below current stack and align down
    uintptr_t target = ((uintptr_t)stack_addr - 1024 * 1024) & ~(page_size - 1);

    // Map one page there — "stealing" it from potential stack growth
    void *m = mmap((void *)target, page_size,
                   PROT_READ | PROT_WRITE,
                   MAP_PRIVATE | MAP_ANONYMOUS | MAP_FIXED, -1, 0);
    if (m == MAP_FAILED) {
        perror("mmap");
        return 1;
    }

    // Now force the stack to grow downward into the stolen page
    // This large local array will overlap the mapping
    char buf[1024 * 1024]; // ~1MB
    buf[0] = 1;

    // If we get here, somehow no crash occurred
    printf("Unexpectedly survived.\n");
    return 0;
}

</details>

> Since we do an unconditional munmap, I assume the underlying pages are guaranteed to belong to this stack.

It's not really "unconditional", we check the memory-range to be munmap belongs to [stack-start, stack-end].

-------------

PR Comment: https://git.openjdk.org/jdk/pull/26571#issuecomment-3169031521