RFR (S) 8023956: Provide a work-around to broken Linux 32 bit "Exec Shield" using CS for NX emulation (crashing with SI_KERNEL)

[Florian Weimer]

On 09/02/2013 02:37 PM, David Simms wrote:

>> This needs at least a compiler barrier between the write and the
>> function call:
>>
>> +  // Some code to exec: the 'ret' instruction
>> +  codebuf[0] = 0xC3;
>> +
>> +  // Call it and ret
>> +  void (*fn)(void) = (void (*)(void)) codebuf;
>> +  fn();
>>
> Noted, originally used __asm__ volatile("call..."), but this was more
> readable. It is specific to IA32, tested and it's works for the
> optimized compile.

If GCC ever learns to treat mmap without MAP_FIXED like malloc, it will 
remove the store, and the call will crash because there's no RET 
instruction.  A compiler barrier à la __asm__(""); should avoid that 
time bomb.

> It's a really small race condition with MM invalidation on SMP, my repro
> (based on ) looks like this:
>
>   * v8 benchmarks from
>     http://v8.googlecode.com/svn/branches/bleeding_edge/benchmarks
>   * Repeatedly run: $JDK8_EA_i586/bin/java -XX:MaxNodeLimit=300000
>     -XX:+BackgroundCompilation -XX:+TieredCompilation
>     -XX:+PrintCodeCache -jar
>     /java/re/jdk/8/promoted/latest/binaries/linux-i586/jre/lib/ext/nashorn.jar
>     run.js
>
> I happened to get lucky and found I could reproduce inside a VirtualBox
> using Ubuntu 10.04 desktop.

Thanks.

-- 
Florian Weimer / Red Hat Product Security Team