How to dispatch the same method call on different platforms?

[Maurizio Cimadamore]

Hi,
Note that here the problem is not the downcall method handle, which 
returns just a pointer, so that's portable.

The problem is that the pointer points to a struct whose layout might 
vary depending on the platform.

The way one might attack this could be to wrap a bunch of "shared" 
accessors for "passwd" inside a class, where the static initializer for 
the class determines which platforms we're in, and sets up the layout 
accordingly.

For that, I agree that some form of [1] is needed. Note that we 
currently have this internal class:

https://github.com/openjdk/jdk/blob/master/src/java.base/share/classes/jdk/internal/util/Architecture.java

We have started using this internally more and more, and adding stuff we 
know the JDK needs. Ideally, this class will be added in some shape or 
form to the public API, because I think that it could be useful to 
regularize a lot of ad-hoc code that is written by 3rd parties (e.g. 
using system properties and what not). I think that, realistically, 
there will be small follow ups such as this: FFM is a relatively small 
API, but one that comes with a relatively big conceptual shift. As some 
layouts and method handles are now platform dependent, of course that 
now puts more pressure on ways in which to detect said platform 
differences (and adapt method/var handles as required).

P.S.

While what you say re. System::loadLibrary is true (e.g. it is not fully 
aware of dynamic linker paths), FFM provides a different "blessed" way 
of looking up libraries that is very well aware of what the dynamic 
linker does. In fact, it's nothing but a thin wrapper around dlopen:

https://download.java.net/java/early_access/jdk22/docs/api/java.base/java/lang/foreign/SymbolLookup.html#libraryLookup(java.lang.String,java.lang.foreign.Arena)

On Linux, you can do `loaderLookup("libc.so.6")` and that works fine - 
no custom logic is needed. The problem of shipping native libraries into 
maven modules, on the other hand, is one that still requires manual 
workarounds. But I believe that to be more a problem of the tool we're 
"forced" to work with (Maven/Gradle): there's _no sane way_ to load a 
shared library from a zip file (or from some random memory address). 
Meaning that, to do that, you need to copy the library _somewhere_ 
before loading, which can come with its own set of issues (e.g. no write 
access, not even to "/tmp"). A qualitatively better solution would be to 
use jmods and custom JDK images (with jlink) which can deal with native 
libraries just fine, but unfortunately these things are not supported by 
the build tools we have today. This is another of these follow-up 
activity that will likely fall out from the initial FFM addition.

Cheers
Maurizio

On 17/02/2024 06:53, tison wrote:
> For example, getpwnam returns passwd struct, but the struct layout is 
> various on different platform, making the code snippet below unportable:
>
>         final Linker linker = Linker.nativeLinker();
>         final SymbolLookup lookup = linker.defaultLookup();
>         final MemorySegment getpwnam = 
> lookup.find("getpwnam").orElseThrow();
>         final MethodHandle getpwnamFn = 
> linker.downcallHandle(getpwnam, 
> FunctionDescriptor.of(ValueLayout.ADDRESS, ValueLayout.ADDRESS));
>         final StructLayout passwdLayout = MemoryLayout.structLayout(
>                 ValueLayout.ADDRESS.withName("pw_name"),
>                 ValueLayout.ADDRESS.withName("pw_passwd"),
>                 ValueLayout.JAVA_INT.withName("pw_uid"),
>                 ValueLayout.JAVA_INT.withName("pw_gid"),
>                 ValueLayout.JAVA_LONG.withName("pw_change"),
>                 ValueLayout.ADDRESS.withName("pw_class"),
>                 ValueLayout.ADDRESS.withName("pw_gecos"),
>                 ValueLayout.ADDRESS.withName("pw_dir"),
>                 ValueLayout.ADDRESS.withName("pw_shell"),
>                 ValueLayout.JAVA_LONG.withName("pw_expire"),
>                 ValueLayout.JAVA_INT.withName("pw_fields"));
>         try (final Arena arena = Arena.ofConfined()) {
>             final MemorySegment username = arena.allocateUtf8String(user);
>             final MemorySegment passwd = ((MemorySegment) 
> getpwnamFn.invoke(username)).reinterpret(Long.MAX_VALUE);
>             final MemorySegment dir = passwd.get(
>                     ValueLayout.ADDRESS,
> passwdLayout.byteOffset(MemoryLayout.PathElement.groupElement("pw_dir")));
> System.out.println(dir.reinterpret(Long.MAX_VALUE).getUtf8String(0));
>         }
>
> This code snippet only works on macOS because the layout differs on 
> other platforms.
>
> In Rust, we can use #[target(os = ..)] to switch the manner, and in 
> Java, perhaps we can use inheritance or interfaces, but it still lacks:
>
> * A unified way to determine current os, arch, toolchain, etc. I made 
> [1] that can help but it's no more than another incomplete slang.
> * A compile-time dispatch decision. Maybe with static initialization 
> block it can helps by generating 'static final' 
> MethodHandle/VarHandle. I don't know.
>
> Best,
> tison.
>
> [1] github.com/tisonspieces/os-detector 
> <http://github.com/tisonspieces/os-detector>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/panama-dev/attachments/20240218/ca492305/attachment-0001.htm>