Request for review [7107135](S): Stack guard pages are no more protected after loading a shared library with executable stack.

[Lindenmaier, Goetz]

Hi Dmitry,

I understand that a fix for this issue should not affect 
applications that use properly tagged jni applications.
But I don't think reading the elf file is as problematic
as you indicate.

> > yes, I know that's costly.  But it at most doubles the cost 
> > of loading the DSO, as dlopen also reads the file.
> OS doesn't actually read DSO at the time of dlopen in most cases because
> of caching.
Yes, dlopen must not read the full file. But 
dlopen must know about the required stack protection of the 
library.  To find this out, it must read the GNU_STACK entry,
which is just the part my code reads, too.

> Also we have to parse LD_LIBRARY_PATH and search for file over it.
As far as I see, os::dll_open always gets a full path passed as argument.
So we need not search LD_LIBRARY_PATH.

> Also we have to care about LD_PRELOAD, symlinks, permissions -
> lots of work - check ld.so sources for details.
I think the common case for a jni library isn't that complicated.
Anyways, there is no bulletproof solution, as the loaded library 
can do a mprotect() on the stack.  (Symlinks should be resolved by 
fopen.)  

The other calls to dlopen (like dlopen("librt.so.1", RTLD_LAZY))
are done during initialization, so that the stack protection is 
changed before there are Java threads with guard pages.  So they 
are no issue.

> > An alternative is to always do a safepoint, and just reprotect
> > if SafeFetch failed.  This would have the advantage that
> > we are sure we catch all changes of the guard page protection.
> > But in that case all threads have to wait for the IO of dlopen.
> 
> I'm not sure we have to decrease performance for all clients in order to
> workaround a bug in an old library (modern tendency is to treat
> executable stack as a security bug).
Yep, I think so, too. But I think I should do the reprotection 
of the stacks in a safepoint in any case.  This safepoint will
be needed at most once in the runtime of a VM.

> Also stack overflow is not an everyday situation for well-designed Java
> app.
Well, we recently even had a customer with a stack overflow while
handling an other exception ;).

> I plan to put more efforts to this problem and check whether we have a
> clever solution for this problem or not.
That's good, thank you!

Best regards,
  Goetz.