[PATCH] Invalid critical JNI function lookup (Windows x86)

[David Holmes]

Hi Ioannis,

I have filed:

https://bugs.openjdk.java.net/browse/JDK-8167408

for this.

If you haven't already, please see:

http://openjdk.java.net/contribute/

on how to contribute. While small simple patches can be accepted from 
anyone, more complex patches require you to sign the OCA.

BTW I don't see anything Windows x86 specific here, as this is shared code.

Thanks,
David

On 10/10/2016 10:34 AM, Ioannis Tsakpinis wrote:
> This patch fixes the lookup of critical JNI functions on Windows x86.
>
> There are two problems with the argument size calculation in the
> lookup_critical_entry function:
>
> 1) Critical natives do not have a JNIEnv parameter. Critical natives are
> always static, but do not have a jclass parameter. The current code assumes
> that both parameters exist and counts them against the total argument size.
>
> 2) For each Java array parameter, the critical native gets an additional
> length parameter for that array. The current code does not count them.
>
> On the 32-bit VM, the argument size is used to apply stdcall decorations to
> the function name. A wrong size is calculated with the current code, so the
> name used for the lookup is invalid (unless the function happens to have
> exactly two array parameters).
>
> diff -r fec31089c2ef src/share/vm/prims/nativeLookup.cpp
> --- a/src/share/vm/prims/nativeLookup.cpp       Thu Oct 06 18:05:53 2016 -0700
> +++ b/src/share/vm/prims/nativeLookup.cpp       Sun Oct 09 22:44:54 2016 +0300
> @@ -293,10 +293,12 @@
>    char* critical_name = critical_jni_name(method);
>
>    // Compute argument size
> -  int args_size = 1                             // JNIEnv
> -                + (method->is_static() ? 1 : 0) // class for static methods
> -                + method->size_of_parameters(); // actual parameters
> -
> +  int args_size = method->size_of_parameters(); // actual parameters
> +  for (SignatureStream ss(signature); !ss.at_return_type(); ss.next()) {
> +    if (ss.is_array()) {
> +        args_size += T_INT_size; // array length parameter
> +    }
> +  }
>
>    // 1) Try JNI short style
>    entry = lookup_critical_style(method, critical_name, "",
> args_size, true);
>
> In steps 3 and 4 the function lookup is done without a prefix/suffix, so a
> workaround is available. On msvc JNI functions can be exported without
> decorations, but it's not without pain: it requires pragmas or a .DEF file.
>
> Regards,
> Ioannis
>