RFR(XS): 8059868: JVM crashes on attach on Windows when compiled with /RTC1

[Thomas Stüfe]

Hi all,

We saw crashes when connecting to a target VM using
com.sun.tools.attach.WindowsVirtualMachine when injecting VM was compiled
with /RTC1.

The error turned out to be in VirtualMachineImpl.c: the function
"jvm_attach_thread_func" - the one which is injected into the target VM and
used as thread entry point for CreateRemoteThread() - must be compiled with
runtime checks disabled in order to keep the code-to-inject position
independent.

Using /rtc1 will cause the Microsoft compiler to generate relative calls to
a check function ("_RTC_CheckEsp") which will not work if code is planted
in target process at a different address.

This change adds a pragma to locally disable the runtime checks and
re-enable them below the function.

http://cr.openjdk.java.net/~simonis/webrevs/8059868/

Kind regards,

Thomas Stuefe
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20141007/6aa328e7/attachment.html>