Request for review [new bug](S): Stack guard pages are no more protected after loading a shared library with executable stack.

[Dmitry Samersoff]

On 2011-10-27 15:58, Lindenmaier, Goetz wrote:
> Hi Dmitry,
> 
> yes, I know that's costly.  But it at most doubles the cost 
> of loading the DSO, as dlopen also reads the file.

OS doesn't actually read DSO at the time of dlopen in most cases because
of caching.

Also we have to parse LD_LIBRARY_PATH and search for file over it.

Also we have to care about LD_PRELOAD, symlinks, permissions -
lots of work - check ld.so sources for details.

i.e.

dlopen("mylib.so")

could end up with

elfread("/slow_nfs_host/lib/mylib.so.1.3")


So I would prefer not to go this way.

> An alternative is to always do a safepoint, and just reprotect
> if SafeFetch failed.  This would have the advantage that
> we are sure we catch all changes of the guard page protection.
> But in that case all threads have to wait for the IO of dlopen.

I'm not sure we have to decrease performance for all clients in order to
workaround a bug in an old library (modern tendency is to treat
executable stack as a security bug).

Also stack overflow is not an everyday situation for well-designed Java
app.

*SS*:
I plan to put more efforts to this problem and check whether we have a
clever solution for this problem or not.

-Dmitry



> 
> Goetz.
> 
> 
> -----Original Message-----
> From: Dmitry Samersoff [mailto:Dmitry.Samersoff at oracle.com] 
> Sent: Mittwoch, 26. Oktober 2011 18:27
> To: Lindenmaier, Goetz
> Cc: Florian Weimer; hotspot-runtime-dev at openjdk.java.net
> Subject: Re: Request for review [new bug](S): Stack guard pages are no more protected after loading a shared library with executable stack.
> 
> Goetz,
> 
>> Yes, that's my understanding, too. The problem is just that there are
>> libs around where the stack is set wrong.  One can simply fix these
>> libs with execstack -c <lib>, but this requires that an application
>> is fixed and updated before the Java user updates a VM or switches
>> to Java 7.
> 
> We definitely have to document it somewhere - so I'll file a bug
> tomorrow (MSD).
> 
> I'm not sure that parsing of DSO's elf header on each dlopen() call is
> the best possible solution.
>   It can significantly decrease startup performance, we should care
> about LD_LIBRARY_PATH, LD_PRELOAD etc.
> 
> Let me bake the problem for some time ...
> 
> -Dmitry
> 
> 
> 
> On 2011-10-26 19:33, Lindenmaier, Goetz wrote:
>> Hi Dmitry and Florian, 
>>
>>> 1. What is the reason to keep an executable stack in the library?
>>>    It still will not work with SELinux config.
>> There are old applications not running on SELinux.  If these contain and 
>> load a jni library without proper stack settings, this issue arises.  They 
>> will not be able to detect stack overflows.
>>
>>> 2. It's seems to me that with RTLD_LAZY kernel change a protection of
>>>    stack guard page not at the time of dl_open call but at the time of
>>>    actual library loading.
>> As I understand,y RTLD_LAZY affects symbol resolution. I could not find
>> documentation telling how ldopen treats stack protection. But I observed
>> in my experiments that the stack protection is lost after dlopen completed.
>> You can also see this in the /proc/self/maps file, where stack access 
>> rights change from rw to rwx. 
>>
>> Florian, I don't think it's an option to change the stack back to 
>> executable.
>>
>>> In most case executable stack is not necessary now days. The only case
>>> in my memory, where executable stack is really necessary - code using
>>> nested functions, as gcc generate trampolines on the stack.
>> Yes, that's my understanding, too. The problem is just that there are
>> libs around where the stack is set wrong.  One can simply fix these
>> libs with execstack -c <lib>, but this requires that an application
>> is fixed and updated before the Java user updates a VM or switches 
>> to Java 7.
>>
>> Best regards,
>>   Goetz.
>>
>>
>>
> 
> 


-- 
Dmitry Samersoff
Java Hotspot development team, SPB04
* There will come soft rains ...