RFR: 8263476: Use reserved memory for stack guard pages [v4]

[David Holmes]

On Wed, 6 Aug 2025 09:05:47 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> Aha, now I understand. The diagram helped a lot!
>
> I don't think we should make this change, since it introduces a subtle bug.
> 
> Apart from being hard to understand, it also opens a time window where the stack guard pages are unmapped; at line 3478, we re-establish MAP_NORESERVE mapping here, but in this time window someone else may have mapped pages in there. Now someone lives in our stack. The reservation will fail, we return false, write a "cannot establish guard pages" warning. If the primordial thread stack grows into this region, it will overwrite that misplaced memory. Errors would be very rare and intermittent.
> 
> Apart from all that, it also means that we add two unnecessary calls to mmap to likely every startup, since the condition at line 3475 is now likely always true. 
> 
> Pre-existing: to be honest, I have no idea why we even do the munmap in the first place. Why would it matter if the primordial thread stack is larger than what we think it should be?

@tstuefe the comment prior to `os::pd_create_stack_guard_pages` is explaining why the munmap is needed.

@albertnetymk  even with the diagram I do not understand this aspect of the change: why do we now need to check `addr+size`?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/26571#discussion_r2289573186