[patch] Adding stack markings to the x86 assembly for not using executable stack

[Andrew John Hughes]

2009/8/27 Kees Cook <kees at ubuntu.com>:
> Hi Andrew,
>
> On Thu, Aug 27, 2009 at 12:04:07PM +0100, Andrew John Hughes wrote:
>> 2009/8/27 Matthias Klose <doko at ubuntu.com>:
>> > This was reported as https://edge.launchpad.net/bugs/409736
>> >
>> > Java is marked to have an executable stack[1]. This is potentially
>> > dangerous, and is simply an oversight from one of the compiled assembly
>> > files. Adding stack markings to the assembly solves the issue.
>> >
>> > sun/security/ssl/javax/net/ssl/NewAPIs/SessionCacheSizeTests.java passes
>> > both stock and and with non-exec-stack.
>> >
>> > gcc -fstack-protector is the default on Ubuntu. I'd like to see this patch
>> > for the IcedTea 1.6 release as well.
>> >
>> >  Matthias
>> >
>>
>> I've heard about this issue before from Gentoo users and the fix, if
>> it truly is this simple, would be good to have.
>
> The question tends to be one of portability.  In cases were non-gcc is
> used, ifdef's need to be built around the flag line.  I can provide some
> examples, if needed.
>

I don't see an immediate problem, as they only affect x86/linux and
x86_64/linux where the compiler is gcc.

>> Are you sending this patch upstream?  It would be good to have some
>> feedback from the HotSpot developers before we commit this for a
>> release.
>>
>> Does this affect SPARC too?
>
> I'm not familiar with SPARC hardware, but if it supports "execute" memory
> protections, then it is a valuable change there too.  It it doesn't, it
> won't hurt anything, IIUC.
>
> -Kees
>
> --
> Kees Cook
> Ubuntu Security Team
>

Do you have an SCA, either via Ubuntu or personally? A webrev needs to
be prepared against one of the HotSpot forests and posted to
hotspot-dev.  If this is the compiler, hotspot-comp is appropriate and
twisti can review it ;)
-- 
Andrew :-)

Free Java Software Engineer
Red Hat, Inc. (http://www.redhat.com)

Support Free Java!
Contribute to GNU Classpath and the OpenJDK
http://www.gnu.org/software/classpath
http://openjdk.java.net

PGP Key: 94EFD9D8 (http://subkeys.pgp.net)
Fingerprint: F8EF F1EA 401E 2E60 15FA  7927 142C 2591 94EF D9D8