RFR (S) JDK-7107135 - Stack guard pages becomes writable

[David Holmes]

On 21/02/2013 1:14 PM, Dean Long wrote:
> On 2/20/2013 12:26 PM, David Holmes wrote:
>> On 21/02/2013 4:52 AM, Ioi Lam wrote:
>>>
>>>> On 02/20/2013 08:25 AM, Dean Long wrote:
>>>>> Which part seems buggy, changing the permissions?  It seems necessary
>>>>> as long as executable stacks is
>>>>> a supported feature.  It would be nice if dlopen() would just fail to
>>>>> load these libraries based on some flag
>>>>> in the executable's ELF file.  I think the existing flag means "no
>>>>> executable stacks used by this module",
>>>>> but "no executable stacks used by this process" would make more sense
>>>>> for Java.
>>>>>
>>>
>>> I found that it's possible to forbid dlopen() from loading shared
>>> libraries that requires executable stacks. This can be done by calling
>>> munmap() on the Java stack guard. Later, when dlopen calls mprotect to
>>> make the stack RWX, the mprotect() system call will fail with an "memory
>>> unavailable" error. As a result, dlopen would abort the loading.
>>
>> But dlopen works on all the stacks, so you would have to make sure it
>> fails on the first one it tries, which is the main thread. But now you
>> are depending on internal knowledge of dlopen.
>>
> As long as it fails for the first Java thread, the side-effect of
> modifying a few non-Java threads might be OK.

But there is no way ensure that.

> However this implementation detail of glibc means that one library
> (libjvm.so) can sabotage dlopen for
> other modules in the same process.  Doesn't this mean a browser with an
> embedded JVM would no longer be able to load certain libraries?

I would think so. I don't think messing with dlopen semantics is really 
an option here. I think the best we can do is to narrow the scope of the 
potential problem as Ioi has outlined - load problematic libraries via a 
safepoint in the VMThread so all the Java stacks can be repaired.

There are limits on what the VM can do to account from things broken in 
the OS.

David
-----

>
> dl
>
>> David
>>
>>
>