RFR: 8189170: Add option to disable stack overflow checking in primordial thread for use with JNI_CreateJavaJVM

David Holmes david.holmes at oracle.com
Tue Nov 14 10:39:24 UTC 2017

Bug: https://bugs.openjdk.java.net/browse/JDK-8189170

webrev: http://cr.openjdk.java.net/~dholmes/8189170/webrev/

There are three parts to this which made it somewhat more complicated 
than I had envisaged:

1. Add the flag and disable the guards.

This is relatively straight-forward:

  void JavaThread::create_stack_guard_pages() {
    if (!os::uses_stack_guard_pages() ||
        _stack_guard_state != stack_guard_unused ||
        (DisablePrimordialThreadGuardPages && os::is_primordial_thread())) {
        log_info(os, thread)("Stack guard page creation for thread "
                             UINTX_FORMAT " disabled", 

with a tweak in JavaThread::stack_guards_enabled as well.

2. Promote os::XXX::is_initial_thread to os::is_primordial_thread()

We have three platforms that already implement this functionality: 
Linux, Solaris and AIX. For BSD/OSX and Windows we don't attempt to do 
anything different for the primordial thread, and we don't have a way to 
detect the primordial thread - so is_primordial_thread() just returns false.

I also tidied up some comments regarding terminology.

3. Thomas noticed that we unconditionally subtract 2*page_size() from 
the rlimit stack size, without checking it was bigger than 2*page_size() 
to start with. I was unsure how to tackle this. It's no good leaving 
stack_size at zero so I opted to ensure we would have at least one page 
left. Of course in such cases we would hit the bug in libc (if it still 
exists, which seems unlikely but time consuming to establish).

   - Tier 1 (JPRT) testing with a modified launcher that uses the 
primordial thread
     - with guard pages enabled
     - with guard pages disabled
   - Tier 1 (JPRT) normal JDK build (which should be unaffected by this 

The testing was primarily to ensure that disabling the stack guards on 
the primordial thread wasn't totally broken. A number of tests fail:
- setting -Xss32m fails for the primordial thread when guards are 
enabled and the rlimit is 8m
- tests that would hit StackOverflowError crash the VM when guards are 
disabled (as you would expect).
- runtime/execstack/Testexecstack.java crashes when the guards are disabled


More information about the hotspot-runtime-dev mailing list