RFR (S) JDK-7107135 - Stack guard pages becomes writable

[Dean Long]

On 2/20/2013 12:26 PM, David Holmes wrote:
> On 21/02/2013 4:52 AM, Ioi Lam wrote:
>>
>>> On 02/20/2013 08:25 AM, Dean Long wrote:
>>>> Which part seems buggy, changing the permissions?  It seems necessary
>>>> as long as executable stacks is
>>>> a supported feature.  It would be nice if dlopen() would just fail to
>>>> load these libraries based on some flag
>>>> in the executable's ELF file.  I think the existing flag means "no
>>>> executable stacks used by this module",
>>>> but "no executable stacks used by this process" would make more sense
>>>> for Java.
>>>>
>>
>> I found that it's possible to forbid dlopen() from loading shared
>> libraries that requires executable stacks. This can be done by calling
>> munmap() on the Java stack guard. Later, when dlopen calls mprotect to
>> make the stack RWX, the mprotect() system call will fail with an "memory
>> unavailable" error. As a result, dlopen would abort the loading.
>
> But dlopen works on all the stacks, so you would have to make sure it 
> fails on the first one it tries, which is the main thread. But now you 
> are depending on internal knowledge of dlopen.
>
As long as it fails for the first Java thread, the side-effect of 
modifying a few non-Java threads might be OK.
However this implementation detail of glibc means that one library 
(libjvm.so) can sabotage dlopen for
other modules in the same process.  Doesn't this mean a browser with an 
embedded JVM would no longer
be able to load certain libraries?

dl

> David
>
>