AttachCurrentThread Crash

[Steve Bohne]

Markus,

Very interesting project.  When your DLL gets unloaded, does jvm.dll 
also get unloaded?  As you mentioned, if jvm.dll is getting loaded 
twice, even sequentially, this might cause problems.  (Yes, it's a known 
limitation of embedding a JVM in another application.  Don't know of any 
current activity to work on a solution for this issue, but submitted 
patches will be carefully considered. :))

Did you try using explicit linking (use LoadLibrary, GetProcAddress, 
etc) for jvm.dll so it won't automatically get unloaded when your DLL does?

Are you able to catch the crash in a debugger and provide a symbolic 
stack trace?  I have a feeling it will reflect the limitation mentioned 
above, so it may not lead to a solution, but it might be educational.

Steve

P.S. Be careful embedding Java (or any single instance runtime) in a 
shell extension: 
http://blogs.msdn.com/junfeng/archive/2005/11/18/494572.aspx.  Many of 
the limitations mentioned here for CLR will also apply to Java.

Markus KARG wrote:

> Hello Hotspot Community,
>
> I experienced a problem when using JNI and since nobody else was able 
> to help me since months, and since it looks like a bug in the jvm.dll, 
> you are my last chance. :-)
>
> In fact I am an experienced JNI user so my question is not about JNI 
> itself but about a very complex case that makes jvm.dll crash down. So 
> I want to discuss with you how we can solve it -- either by working 
> around it in my code, or by adding a fix to the Hotspot VM (which 
> seems to be the cause of the problems).
>
> I am writing a DLL in Windows XP that itself is a plugin to the 
> desktop, a.k.a "Shell Extension". The DLL itself runs really stable 
> (in fact I have drilled down so far that it actually does nothing 
> anymore now, so I am pretty sure there is no bug in it since there is 
> no more "real" code in it besides creating the Java VM). explorer.exe 
> (i. e. the desktop process of Windows) loads the DLL, executes my 
> code, and unloads the DLL then. So far, so good, works pretty well.
>
> But the same Windows process then loads my DLL once more, and executes 
> it again (just as it did before). My DLL, you can imagine, is using 
> JNI to call some Java code. Certainly I was clever enough not to 
> create a VM twice, since I certainly know that both is impossible, 
> either creating two VMs in the same process, or to try to create a VM 
> after the previous VM was deleted (which I think is a bug, but 
> anyways, that is not the point here).
>
> So I am checking first wheter there is a VM already using 
> JNI_GetCreatedJavaVMs which returns the number of 1 and a pointer to 
> the JavaVM. Great. That works pretty well. But if I now try to call 
> AttachCurrentThread to get a pointer to the Env, then the desktop 
> totally crashs down, telling me that I tried to execute code that 
> actually is data (I switched off Data Execution Prevention then, but 
> now it crashs down without ANY further note, so it is not a real 
> help). So why did that happen?
>
> I looked at the source code of the JVM and it seems as if it uses 
> global variables very heavily in the JNI source files. I could imagine 
> that in fact one of those pointers now (after explorer unloaded and 
> reloaded my DLL in the same process) point to a memory page that is 
> marked as "data" but not as "code", so when using the function 
> pointer, windows crashs down (since the memory pointed to is 
> presumably no more part of the process's memory map).
>
> Please help me. I do not have any idea how to fix that. As a 
> workaround I told Windows not to ever unload my DLL (what is working), 
> but that certainly is just a bad hack. The real solution must be to 
> convince jvm.dll to attach the thread without crashing down.
>
> If needed, I can provide an "at most simple" project that proofs the 
> behaviour on any Windows machine (without seriously damaging it).
>
> Thanks!
> Markus
>