Questions about com.sun.tools.attach implementation on windows

[Staffan Larsen]

I completely agree that this code is extremely fragile and quite “hacky” - there is no use defending (except that it works most of the time). If we can come up with a better solution that allows us to attach to a running process, I am all for it. Perhaps shared memory and a shared mutex could be used? Or the windows mailslots API?  

(The mechanism on the non-windows platforms is also full of races and prone to leak resources, so changing those is also welcome.)

/Staffan

On 9 okt 2014, at 11:17, Thomas Stüfe <thomas.stuefe at gmail.com> wrote:

> 
> Hi all,
> 
> I have a question about the implementation of com.sun.tools.attach on Windows.
> 
> when investigating a crashes which were related to VirtualMachineImpl.c (see https://bugs.openjdk.java.net/browse/JDK-8059868), the implementation seemed to me very unusual and fragile:
> 
> We want to call monitoring functions in the target VM.
> 
> To do that, we inject a code stub (compiled by the c++ compiler used to build the injector VM) into the target VM. Then we use CreateRemoteThread() to run this code. 
> 
> As a parameter to this function, we hand over addresses to Win32 APIS taken from the injector-VM. The target VM will call those functions, implicitly assuming the locations to be the same in both processes.
> 
> Problems which may occur (at least as far as I understand it):
> 
> - the code injected from the injector VM into the target VM must be position independent - something which may easily break with new security features Microsoft adds to their compiler (e.g. the /RTC1 crash reported above).
> 
> - kernel32.dll must be always loaded at the same address for the Win32 API calls to work. I am not sure this is always the case now, with address layout randomization.
> 
> - I also could imagine (though I did not see that) virus scanners not being happy and reporting this activity as suspicious.
> 
> Also, if the coding fails - most likely crashes - effects are a bit harsh:
> 
> - the target VM, which is "innocent" and my be stable and years old, will crash if the injector VM was compiled wrong, e.g. in a way which makes the injected code position dependend. This may happen by a new compiler switch or compiler update. 
> 
> - There is no way to get decent error handling, at least for 32bit, because we rely on stack based SEH handling and in the remote thread there is no SEH handler yet set up. The effect is sudden death
> of the target VM with no hs-err file. 
> 
> - The crash is difficult to figure out: first one will analyze the target VM, which is innocent, because it crashes. But we have no matching debug information for the injected stub, so debugging is difficult.  
> 
> I would love to know why we do it this way. I am sure there is a valid reason for it. Maybe Backward compatibility? 
> 
> Thanks! and Kind Regards,
> 
> Thomas
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20141009/fbb5253b/attachment-0001.html>