Making the foreign liner a required API

[Tyler Steele]

Hi Jorn,

Thanks for you email a few weeks back. That was very helpful in getting started.

Now that I've debugged my libffi build on AIX, I have been able to configure OpenJDK successfully with the options you mentioned in your previous email.
--enable-fallback-linker
--with-libffi=libffi # a symbolicly linked dir containing lib and include
--enable-libffi-bundling
I am now vexed by the following error which occurs during make images (for OpenJDK):

gmake[3]: *** No rule to make target 'libffi/lib/libffi.so.8', needed by '/home/hotspot/openjdk/jdk-tyler/build/aix-ppc64-server-fastdebug/support/modules_libs/java.base/libffi.so.8'.  Stop.

I see java.base/Copy.gmk:238 ought to be bringing that file into the build. I can also see by the error message that it has found the location of the correct lib. Did you face any similar issue in getting your bundle to work?

Thanks
Tyler
________________________________
From: Jorn Vernee <jorn.vernee at oracle.com>
Sent: June 9, 2023 16:27
To: Tyler Steele <Tyler.Steele at ibm.com>
Cc: porters-dev at openjdk.org <porters-dev at openjdk.org>
Subject: [EXTERNAL] Re: Making the foreign liner a required API

Hello Tyler, First of all, thanks for trying this out! To answer some of your questions: > Does this createLibffiBundle. sh get called programmatically during the build process (I can't seem to find where that would happen currently), or
ZjQcmQRYFpfptBannerStart
This Message Is From an External Sender
This message came from outside your organization.

ZjQcmQRYFpfptBannerEnd

Hello Tyler,


First of all, thanks for trying this out!


To answer some of your questions:

> Does this createLibffiBundle.sh get called programmatically during the build process (I can't seem to find where that would happen currently), or does each platform check in a bundle with a compiled version of the library to source control (and the build phase assumes it's present for those platforms that support it)?


I pointed to the createLibffiBundle.sh script as an example, so you could get a rough idea of the steps I took. The script takes a devkit as input, which is an artifact we use internally for our CI builds. These bundles (devkit and libffi) are not distributed.


The libffi library is either automatically detected by the build system if it is installed in a well-known system location, or it can be provided manually using the --with-libffi configure option (I've used the latter myself). I you'd like to know how the build works, I suggest taking a look at the ./make/autoconf/lib-ffi.m4 file, which is responsible for pulling the contents of the --with-libffi option into the build system. It basically sets up several make variables with compiler and linker options. You could grep through the rest of the ./make folder for some of the LIBFFI_* variables that are 'exported' at the end of the lib-ffi.m4 file using AC_SUBST. For the fallback linker you should find the relevant usages in the ./make/modules/java.base/lib.gmk file (near the end).


To incorporate the library (.dll/.so/.dylib) into the JDK, it is essentially just copied into the right folder of the built JDK image. Automatic copying can be turned on using the --enable-libffi-bundling configuration flag. In practice, I'm using `--with-libffi=<path to extracted bundle> --enable-libffi-bundling --enable-fallback-linker` to create a working JDK image.


> At present I'm getting an error at build time after enabling the fallback linker on aix.

This is likely caused by an outdated version of libffi. The ffi_get_struct_offsets function was added in version 3.3, so that one or a later version should work. I've done all my testing with version 3.4.2, so I recommend going with that.


Also, note that we decided to postpone making the linker API required for now. Given the deprecation of the 32-bit Windows x86 port in JDK 21, making the linker API required after that port is removed would avoid the need of having to amend the public API and implementation to allow selecting one of the 2 major ABIs on that platform (cdecl or stdcall).


HTH,
Jorn


On 09/06/2023 19:02, Tyler Steele wrote:
Hello Jorn and fellow porters,

I am following up with this message from Jorn from a few months ago [1]. I am generally supportive of their proposal, but I am not certain how to implement it. I have successfully built and bundled libffi on AIX, but I didn't use the createLibffiBundle.sh script provided by the PR.

What I'm feeling confused about is how libffi will be incorporated into the jvm. Does this createLibffiBundle.sh get called programmatically during the build process (I can't seem to find where that would happen currently), or does each platform check in a bundle with a compiled version of the library to source control (and the build phase assumes it's present for those platforms that support it)? At present I'm getting an error at build time after enabling the fallback linker on aix.

src/java.base/share/native/libfallbackLinker/fallbackLinker.c:66:10: warning: implicit declaration of function 'ffi_get_struct_offsets' is invalid in C99 [-Wimplicit-function-declaration]
  return ffi_get_struct_offsets((ffi_abi) abi, jlong_to_ptr(type), jlong_to_ptr(offsets));
         ^
Which is not totally surprising given that I haven't figured out how to link it yet.

Thanks in advance,
Tyler

[1] https://mail.openjdk.org/pipermail/porters-dev/2023-March/000753.html<https://mail.openjdk.org/pipermail/porters-dev/2023-March/000753.html>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/porters-dev/attachments/20230622/4786b9aa/attachment-0001.htm>