[External] : Re: Help with VM Crash using the foreign linker

[Jorn Vernee]

On 16/05/2021 02:27, Duncan McLean wrote:
> Hi Jorn,
>
> Thank you for the suggestion. I've looked into this and the 
> FunctionDescriptor is exactly as you have suggested: 
>  FunctionDescriptor.of(C_DOUBLE, C_POINTER).
Sorry, I think I've misread the stack trace earlier. I initially was 
surprised to see invokeMoves in the stack trace for an intrinsified 
case, but we also use that as a fallback before the JIT compiler kicks in.
>
> I went so far as to hard code FunctionDescriptor call when I was 
> running my TestUsingStructs test case. The FunctionDescriptor is build at:
>
> https://github.com/boulder-on/JPassport/blob/ddaf4482045b83be95000e7fcc240bf720f0b525/jpassport/src/main/java/jpassport/PassportFactory.java#L89 
> <https://urldefense.com/v3/__https://github.com/boulder-on/JPassport/blob/ddaf4482045b83be95000e7fcc240bf720f0b525/jpassport/src/main/java/jpassport/PassportFactory.java*L89__;Iw!!GqivPVa7Brio!JBI3NcMrZhS0vy2j4WYCWeB6uPuulrfmMpdF2nnY0pOBQ8viDh7nsMiHug6gvoSQ$>
>
> For some reason I can only run my test cases from my IDE, maven 
> doesn't like something I'm doing (right now it finds my test classes, 
> but no test cases). However, when run from the IDE, 
> TestUsingStructs crashes the vm every time on the first invokeExact 
> call it encounters. When I comment out the deleting of previously 
> compiled files and the compiling of new files (such that the test runs 
> with the last compiled version) the test case always works.
>
> Line for deleting previous compiles:
> https://github.com/boulder-on/JPassport/blob/ddaf4482045b83be95000e7fcc240bf720f0b525/jpassport/src/main/java/jpassport/PassportWriter.java#L535 
> <https://urldefense.com/v3/__https://github.com/boulder-on/JPassport/blob/ddaf4482045b83be95000e7fcc240bf720f0b525/jpassport/src/main/java/jpassport/PassportWriter.java*L535__;Iw!!GqivPVa7Brio!JBI3NcMrZhS0vy2j4WYCWeB6uPuulrfmMpdF2nnY0pOBQ8viDh7nsMiHuhddCXPS$>
>
> Line for compiling new classes:
> https://github.com/boulder-on/JPassport/blob/ddaf4482045b83be95000e7fcc240bf720f0b525/jpassport/src/main/java/jpassport/PassportWriter.java#L551 
> <https://urldefense.com/v3/__https://github.com/boulder-on/JPassport/blob/ddaf4482045b83be95000e7fcc240bf720f0b525/jpassport/src/main/java/jpassport/PassportWriter.java*L551__;Iw!!GqivPVa7Brio!JBI3NcMrZhS0vy2j4WYCWeB6uPuulrfmMpdF2nnY0pOBQ8viDh7nsMiHuhdBauR8$>
>
> Is there any more detailed logging I can generate?

You could try running with 
-Djdk.internal.foreign.ProgrammableInvoker.DEBUG=true

Looking at the crash closer, I see the EXCEPTION_ACCESS_VIOLATION you're 
getting is due to a data execution prevention. I've been able to 
reproduce the crash here, and running with the above logging option, it 
seems that the address being passed into the native call is incorrect:

Buffer state before:
Next PC: 7ffaea2e10dc
...

And then in the crash log:

siginfo: EXCEPTION_ACCESS_VIOLATION (0xc0000005), data execution 
prevention violation at address 0x00007ffaea2e10dc

Stepping further back in the execution, it seems that this address is 
being loaded straight from the library, I see the 10dc as a virtual 
address for that function in dumpbin:

     ordinal hint RVA      name

           ...
           7    6 000010DC passStructWithArrays = 
@ILT+215(passStructWithArrays)
           ...

So, it seems to be a problem with the library.

After some trial and error, it seems that the library is being unloaded 
during the test execution (try setting a breakpoint in 
LibrariesHelper::tryUnload), which would invalidate the function 
address. Note that the LibraryLookup object needs to be kept strongly 
reachable in order to keep the library loaded, in the version of the 
code you're using (we've changed this recently to be more robust). This 
is normally done by the method handle automatically, because it keeps a 
strong reference to the Symbol you use when linking, which then keeps 
the library alive [1].

However, in PassportFactory, you are discarding the Symbol, and only 
keeping the address:

MethodHandle methodHandle =linker.
         downcallHandle(symb.address(),  <---------------
                 MethodType.methodType(methRet,parameters),
                 fd);

Which means the library will not be kept loaded by the MethodHandle, and 
indeed it is being unloaded.

Symbol implements Addressable already, so there is no need to call 
.address() here. If I remove that call, the crash goes away, and the 
test passes.

Jorn

[1] : 
https://download.java.net/java/early_access/panama/docs/api/jdk.incubator.foreign/jdk/incubator/foreign/LibraryLookup.html

>
> Thanks again,
> Duncan
>
> On Sat, May 15, 2021 at 9:36 AM Jorn Vernee <jorn.vernee at oracle.com 
> <mailto:jorn.vernee at oracle.com>> wrote:
>
>     Hi,
>
>     Looking at the crash log, as well as the test, you seem to be
>     passing a
>     single struct to a native function, but the stack trace in the
>     error log
>     shows that the call is not being intrinsified, which should not be
>     the
>     case if you're only passing a single struct on Windows. Either it
>     fits
>     in a register, or it's passed by reference, both of which cases are
>     intrinsified.
>
>     The most likely explanation for that seems to be that you are not
>     using
>     the right FunctionDescriptor when linking, and a crash occurs when
>     the
>     native code is trying to access what it thinks is a pointer to a
>     struct.
>     This is undefined behavior territory, which could explain why the
>     crash
>     only occurs in certain scenarios.
>
>     My advice would be to check the FunctionDescriptor you're using
>     for the
>     linkage request, and make sure it matches the native function
>     declaration. Looking at [1], the FunctionDescriptor you need is
>     FunctionDescriptor.of(C_DOUBLE, C_POINTER).
>
>     HTH,
>     Jorn
>
>     [1] :
>     https://github.com/boulder-on/JPassport/blob/52bfa8b4390fdde57582722a1c8098958088de08/fl_dll/library.c#L330
>     <https://urldefense.com/v3/__https://github.com/boulder-on/JPassport/blob/52bfa8b4390fdde57582722a1c8098958088de08/fl_dll/library.c*L330__;Iw!!GqivPVa7Brio!JBI3NcMrZhS0vy2j4WYCWeB6uPuulrfmMpdF2nnY0pOBQ8viDh7nsMiHumoug0fD$>
>