GetLastError() (with and without debugger)

[Maurizio Cimadamore]

On 15/08/2022 21:49, Manuel Bleichenbacher wrote:
> I've run a test with jdb, and the outcome is the same: the program 
> behaves incorrectly.
>
> Do you guys know if this is caused by the debugger only, or if it is a 
> general problem that is more likely to occur with the debugger but 
> could also occur if run without debugger?

Hi,
as David pointed out, nothing in the JVM, debuggers, agents etc. takes 
this use case into account - which means that whether errno/LastError is 
preserved is, ultimately, platform/OS-dependent.

In JNI the status quo has probably been good enough: after all with JNI 
you can always insert your errno/LastError check directly into the 
native code, ensuring atomicity.

But when you use some kind of FFI, each Java call corresponds to 
_single_ native call - which means that, even if you want to do:

<call function>
<getLastError>

In reality there are some operations which "might" happen between the 
two calls (after all, the JVM is a complex piece of software, the 
garbage collector might need to allocate more memory, etc.).

This explains why some of the frameworks out there shadow (and save) 
errno/LastError into separate thread-local storage, so that it can be 
read by other FFI calls. What I've seen doesn't seem a particularly 
compelling solution though, because: (a) the FFI support has to add 
platform-dependent logic to mimic what e.g. getLastError and 
setLastError would do, and (b) the FFI support needs to "rewire" calls 
to such native functions, so that they return the shadowed storage 
instead. So you could end up with cases where the native LastError value 
and the FFI value are out of sync.

A more general solution would be to combine multiple downcall method 
handles into one, so that e.g. one could apply a GetLastError filter to 
a downcall method handle, which calls GetLastError after the native 
call, and saves the value into some user-defined variable, and does that 
within the _same_ native execution as the original call (e.g. only one 
Java->native transition, covering both calls). But the Linker API does 
not currently offer such capability (and something like that would be a 
bit on the boundary of what a method handle combinator API is allowed to 
do).

Maurizio


>
> On Fri, Aug 12, 2022 at 3:16 PM Maurizio Cimadamore 
> <maurizio.cimadamore at oracle.com> wrote:
>
>     Hi Manuel,
>     thanks for submitting this issue.
>
>     I think your hunch is probably correct - something inside the JDK
>     is resetting the value of LastError.
>
>     In Hotspot code I see some abstractions to preserve the LastError
>     value (os_windows.cpp):
>
>     // A number of wrappers for more frequently used system calls, to
>     add standard logging. struct PreserveLastError {
>        const DWORD v;
>        PreserveLastError() : v(::GetLastError()) {}
>        ~PreserveLastError() { ::SetLastError(v); }
>     };
>
>
>     And this is used in a number of OS-specific function calls, to
>     avoid polluting the last error value.
>
>     That said, when you are running with a debugger, especially inside
>     an IDE (which might add its own hooks), I think most bets are off,
>     as the debugger might indeed perform additional native calls which
>     might not preserve lastError correctly.
>
>     One experiment would be to try and debugging using jdb - so that
>     at least we'd rule the IDE out, and see if the issue is still
>     there. If that's the case we'll try to reach out to somebody more
>     intimate with architetcure of JPDA, to see if that's something
>     that can be addressed (perhaps in a way similar to what hotspot
>     code seems to be already doing).
>
>     Thanks
>     Maurizio
>
>
>
>
>     On 12/08/2022 10:21, Manuel Bleichenbacher wrote:
>>     Thanks for the work on project Panama. It's an exciting
>>     technology. I'm using it to make native operating system services
>>     available to Java.
>>
>>     On Windows I've run into an issue. This C/C++ Windows code:
>>
>>         BOOL res = WriteFile(INVALID_HANDLE_VALUE, NULL, 0, NULL, NULL);
>>         DWORD err = GetLastError();
>>         printf("WriteFile result: %d, GetLastError result: %d\n",
>>     res, err);
>>
>>     prints (as expected):
>>
>>         WriteFile result: 0, GetLastError result: 6
>>
>>     6 is the value of the constant ERROR_INVALID_HANDLE.
>>
>>     Using panama, I've translated the code to Java. The result is:
>>
>>     Without debugger, the output is the same. Everything is ok.
>>
>>     With the debugger, the result is incorrect:
>>
>>         WriteFile result: 0, GetLastError result: 0
>>
>>     This is incorrect. WriteFile() indicates an error, but
>>     GetLastError() returns 0 (= NO_ERROR). Could it be that the
>>     debugger calls another Windows API function between those two
>>     functions, resetting the last error?
>>
>>     In my project that's a major issue. Since the software behaves
>>     incorrectly with the debugger, the software can no longer be
>>     debugged. This doesn't just affect it when debugging this
>>     particular piece of code but anytime this code is run in a
>>     debugging session.
>>
>>     Is there something I'm not doing incorrectly? Or is there a fix
>>     or workaround?
>>
>>     Here's the Java code:
>>
>>     import java.lang.foreign.*; import java.lang.invoke.MethodHandle; import static java.lang.foreign.MemoryAddress.NULL; import static java.lang.foreign.ValueLayout.ADDRESS; import static java.lang.foreign.ValueLayout.JAVA_INT; public class WinApi {
>>          static final MethodHandleWriteFile$Func; static final MethodHandleGetLastError$Func; static final MemoryAddressINVALID_HANDLE_VALUE = MemoryAddress.ofLong(-1); static {
>>              var linker = Linker.nativeLinker(); var lookup = SymbolLookup.libraryLookup("Kernel32", MemorySession.global()); WriteFile$Func = linker.downcallHandle(
>>                      lookup.lookup("WriteFile").get(), FunctionDescriptor.of(JAVA_INT, ADDRESS, ADDRESS, JAVA_INT, ADDRESS, ADDRESS)
>>              ); GetLastError$Func = linker.downcallHandle(
>>                      lookup.lookup("GetLastError").get(), FunctionDescriptor.of(JAVA_INT)
>>              ); }
>>
>>          public static void main(String[] args) {
>>              var res =WriteFile(INVALID_HANDLE_VALUE, NULL, 0, NULL, NULL); var err =GetLastError(); System.out.printf("WriteFile result: %d, GetLastError result: %d\n", res, err); }
>>
>>          static int WriteFile(MemoryAddress hFile, MemoryAddress lpBuffer, int nNumberOfBytesToWrite, MemoryAddress lpNumberOfBytesWritten, MemoryAddress lpOverlapped) {
>>              try {
>>                  return (int)WriteFile$Func.invokeExact((Addressable)hFile, (Addressable)lpBuffer, nNumberOfBytesToWrite, (Addressable)lpNumberOfBytesWritten, (Addressable)lpOverlapped); }catch (Throwable e) {
>>                  throw new RuntimeException(e); }
>>          }
>>
>>          static int GetLastError() {
>>              try {
>>                  return (int)GetLastError$Func.invokeExact(); }catch (Throwable e) {
>>                  throw new RuntimeException(e); }
>>          }
>>     }
>>
>>     Cheers
>>     Manuel
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/panama-dev/attachments/20220816/3ddf3409/attachment-0001.htm>