RFR (S) JDK-7107135 - Stack guard pages becomes writable

[David Holmes]

On 21/02/2013 4:40 AM, Ioi Lam wrote:
> [Moving the discussion to hotspot-runtime-dev at openjdk.java.net]
> Please review:
> http://javaweb.us.oracle.com/~iklam/webrev/7107135/stack_guard_001/
>
> Bug: Stack guard pages are no more protected after loading a shared
>       library with executable stack
> https://jbs.oracle.com/bugs/browse/JDK-7107135

This seems somewhat more complex than necessary.

First dll_open_inner should only ever be called via the VMOperation and 
that will only happen if:
  - LoadExecStackDllInVMThread == true
  - os::uses_stack_guard_pages() == true
  - is_init_completed() == true

so there is no need to check any of these inside dll_open_inner.

Related to that back in dll_load I don't understand why we don't simply 
have:

      if (!LoadExecStackDllInVMThread) {
         result = ::dl_open(..)
      else {
         // use Vm op
      }

???

Also I think you will find that Java threads (reference-processor and 
finalizer threads) have been created before init_completed is set.

David
-----


> Background:
>
>      Recent versions of Linux support Non-Executable Stack
>      protection -- by default, the stack is made non-executable to
>      prevent code injection via overflowing on-stack buffers.
>
>      However, some old Linux DLLs require the stack to be
>      executable. For backwards compatibility, after loading such
>      DLLs, the Linux dynamic loader makes the stack executable.
>
>      Due to a limitation of the Linux system call API, the Linux
>      dynamic loader makes the stack readable/writable as well. This
>      disables Java's stack guard.
>
> Summary of fix:
>
>      1. Check if DLL requires executable stack by inspecting ELF header.
>      2. Enter a Safepoint and load such DLLs in the VM thread.
>         - immediately after loading, change all Java stack guards
>           back to PROT_NONE.
>      3. Leave Safepoint to resume Java execution.
>
>      I also added a global flag LoadExecStackDllInVMThread to load
>      such "bad" DLLs outside of the safepoint (in case the DLL
>      invokes JNI functions inside static constructors, which are
>      executed before dlopen() returns).
>
> Note:
>
>      I got this code from an outside contributor and I don't really
>      understand what this block does. Please comment if it's
>      correct:
>
>      1877:    ThreadInVMfromNative tiv(JavaThread::current());
>      1878:    debug_only(VMNativeEntryWrapper vew;)
>
> Tests executed:
>
>      * JPRT
>      * UTE (vm.quick.testlist)
>      * JTREG (hotspot/tests/runtime, hotspot/tests/closed/runtime)
>
> Thanks,
> Ioi
>