Check for PaX during initialization

[Mikael Vidstedt]

Poonam/David,

Thanks for the reviews and feedback. Updated webrev:

http://cr.openjdk.java.net/~mikael/webrevs/portola/paxcheck/webrev.03/hotspot/webrev/ <http://cr.openjdk.java.net/~mikael/webrevs/portola/paxcheck/webrev.03/hotspot/webrev/>

Incremental from webrev.02:

http://cr.openjdk.java.net/~mikael/webrevs/portola/paxcheck/webrev.03.incr/hotspot/webrev/ <http://cr.openjdk.java.net/~mikael/webrevs/portola/paxcheck/webrev.03.incr/hotspot/webrev/>

Cheers,
Mikael

> On May 2, 2017, at 6:36 PM, David Holmes <david.holmes at oracle.com> wrote:
> 
> Seems okay. You may want to add the paxctl instructions as Poonam suggested.
> 
> Thanks,
> David
> 
> On 2/05/2017 7:02 AM, Mikael Vidstedt wrote:
>> 
>> Please review the following change, which adds code to check for the presence of PaX/MPROTECT (https://pax.grsecurity.net/docs/mprotect.txt <https://pax.grsecurity.net/docs/mprotect.txt>) during VM initialization, and prints out a helpful message if PaX is enabled and interfering with the VM. Specifically, the code checks if a writable page can be made executable (mimicking what the JIT would typically do). Since ZERO doesn’t generate code the check is not performed there.
>> 
>> Copy+paste from the actual change:
>> // Some linux distributions (notably: Alpine Linux) include the
>> // grsecurity in the kernel by default. Of particular interest from a
>> // JVM perspective is PaX (https://pax.grsecurity.net/), which adds
>> // some security features related to page attributes. Specifically,
>> // the MPROTECT PaX functionality
>> // (https://pax.grsecurity.net/docs/mprotect.txt) prevents dynamic
>> // code generation by disallowing a (previously) writable page to be
>> // marked as executable. This is, of course, exactly what HotSpot does
>> // for both JIT compiled method, as well as for stubs, adapters, etc.
>> //
>> // Instead of crashing "lazily" when trying to make a page executable,
>> // this code probes for the presence of PaX and reports the failure
>> // eagerly.
>> 
>> http://cr.openjdk.java.net/~mikael/webrevs/portola/paxcheck/webrev.02/hotspot/webrev/
>> 
>> Cheers,
>> Mikael
>>