[PATCH] Windows 32-bit DLL name decoration

[Ali İnce]

Hi Alexey,

Thanks for your reply.

I will definitely give it a try though I’m not sure if that’ll be a breaking change. This is because JNICALL modifier is defined to be __stdcall on windows which is both specified on jdwpTransport_OnLoad exported functions both for dt_socket.dll and dt_shmem.dll. 

The __stdcall is ignored on x64 platforms and also name decoration is not applied (https://docs.microsoft.com/en-gb/cpp/build/reference/decorated-names?view=vs-2017). I believe that’s why we don’t experience any problems on x64 builds.

Removing JNICALL (thus __stdcall) modifier (apparently only applies to x86 builds) will result in the calling convention to be changed, and thus will change how parameters are ordered and also the stack cleanup rules. I think this may result in problems in programs that are designed to load shared libraries and its exported functions at runtime (not bound by link time which probably won’t cause any issues - assuming that they use the shared function signature) - as in https://github.com/AdoptOpenJDK/openjdk-jdk11u/blob/5f01925b80ed851b133ee26fbcb07026ac04149e/src/jdk.jdwp.agent/share/native/libjdwp/transport.c#L99.

Any thoughts?

Regards,

Ali

> On 15 Nov 2018, at 22:14, Alexey Ivanov <alexey.ivanov at oracle.com> wrote:
> 
> Hi Ali,
> 
> Can the issue be resolved by using different call modifiers on the affected functions?
> 
> Some build / link issues were resolved by adding / removing JNICALL modifier, see
> https://bugs.openjdk.java.net/browse/JDK-8201226
> http://mail.openjdk.java.net/pipermail/build-dev/2018-April/021553.html
> 
> https://bugs.openjdk.java.net/browse/JDK-8202476
> http://mail.openjdk.java.net/pipermail/build-dev/2018-May/021913.html
> 
> Regards,
> Alexey
> 
> On 15/11/2018 21:43, Ali İnce wrote:
>> Hi,
>> 
>> An issue (https://github.com/AdoptOpenJDK/openjdk-build/issues/709) raised against AdoptOpenJDK jdk11 32-bit windows builds led us to the name decoration problem on built 32-bit dlls - specifically dt_socket.dll and dt_shmem.dll. Whereas 64-bit MSVC builds don’t apply any name decorations on exported functions, 32-bit builds apply them - which requires either def files or /export flags to be specified on the linker arguments.
>> 
>> Although the expected linker arguments were present on previous jdk makefiles, they were removed in jdk11 makefiles.
>> 
>> Below is the proposed patch, which can also be browsed at https://github.com/AdoptOpenJDK/openjdk-jdk11u/pull/1.
>> 
>> Would you please have a look and let me know of any points I may have missed (I don’t have any background information about why the export flags were removed in jdk11)?
>> 
>> Regards,
>> 
>> Ali
>> 
>> diff --git a/make/lib/Lib-jdk.jdi.gmk b/make/lib/Lib-jdk.jdi.gmk
>> index 197b95c2e2..ac6ebf5591 100644
>> --- a/make/lib/Lib-jdk.jdi.gmk
>> +++ b/make/lib/Lib-jdk.jdi.gmk
>> @@ -37,6 +37,7 @@ ifeq ($(OPENJDK_TARGET_OS), windows)
>>            jdk.jdwp.agent:include \
>>            jdk.jdwp.agent:libjdwp/export, \
>>        LDFLAGS := $(LDFLAGS_JDKLIB), \
>> +      LDFLAGS_windows := -export:jdwpTransport_OnLoad, \
>>        LIBS := $(JDKLIB_LIBS), \
>>    ))
>> 
>> diff --git a/make/lib/Lib-jdk.jdwp.agent.gmk b/make/lib/Lib-jdk.jdwp.agent.gmk
>> index 0bc93e0d35..0382e55325 100644
>> --- a/make/lib/Lib-jdk.jdwp.agent.gmk
>> +++ b/make/lib/Lib-jdk.jdwp.agent.gmk
>> @@ -37,6 +37,7 @@ $(eval $(call SetupJdkLibrary, BUILD_LIBDT_SOCKET, \
>>          libjdwp/export, \
>>      LDFLAGS := $(LDFLAGS_JDKLIB) \
>>          $(call SET_SHARED_LIBRARY_ORIGIN), \
>> +    LDFLAGS_windows := -export:jdwpTransport_OnLoad, \
>>      LIBS_linux := -lpthread, \
>>      LIBS_solaris := -lnsl -lsocket, \
>>      LIBS_windows := $(JDKLIB_LIBS) ws2_32.lib, \
>