RFR: 8268129: LibraryLookup::ofDefault leaks symbols from loaded libraries

[Cheng Jin]

On Fri, 8 Oct 2021 21:29:19 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> Hi @mcimadamore,
>> 
>> As you mentioned at https://github.com/openjdk/jdk/pull/4316#issuecomment-853238872, the system lookup is supported by the underlying `NativeLibraries` which also works on OpenJDK16 to support `LibraryLookup::ofDefault`.
>> 
>> But my finding is that it `LibraryLookup::ofDefault` works good on AIX (with `libc.a`) in OpenJDK16 but `CLinker.systemLookup()` ended up with  `NoSuchElementException` after changing the code in `SystemLookup.java` and `CABI.java` as follows:
>> 
>>     private static final SymbolLookup syslookup = switch (CABI.current()) {
>>         case SysV, LinuxAArch64, MacOsAArch64, AIX -> libLookup(libs -> libs.loadLibrary("syslookup"));
>>         case Win64 -> makeWindowsLookup(); // out of line to workaround javac crash
>>     };
>> 
>> with a simple test & output:
>> 
>> import jdk.incubator.foreign.CLinker;
>> import static jdk.incubator.foreign.CLinker.*;
>> import jdk.incubator.foreign.SymbolLookup;
>> import jdk.incubator.foreign.Addressable;
>> 
>> public class Test {
>>         private static CLinker clinker = CLinker.getInstance();
>>         private static final SymbolLookup defaultLibLookup = CLinker.systemLookup();
>> 
>>         public static void main(String args[]) throws Throwable {
>>                 Addressable strlenSymbol = defaultLibLookup.lookup("strlen").get();
>>         }
>> }
>> 
>> bin/java  --enable-native-access=ALL-UNNAMED  --add-modules jdk.incubator.foreign -Dforeign.restricted=permit Test
>> WARNING: Using incubator modules: jdk.incubator.foreign
>> Exception in thread "main" java.util.NoSuchElementException: No value present <-----
>>         at java.base/java.util.Optional.get(Optional.java:143)
>>         at Test.main(Test.java:11)
>> 
>> 
>> So I am wondering what happened to the system lookup in such case given there should be no fundamental difference in leveraging `NativeLibraries` (I assume the library loading in OpenJDK16 & 17 is the same at this point) unless there is something else new in OpenJDK17 I am unaware of (e.g. the changes in `Lib.gmk` or `lib-std.m4`, or a custom system lookup is required on AIX, etc).  Thanks.
>
>> So I am wondering what happened to the system lookup in such case given there should be no fundamental difference in leveraging `NativeLibraries` (I assume the library loading in OpenJDK16 & 17 is the same at this point) unless there is something else new in OpenJDK17 I am unaware of (e.g. the changes in `Lib.gmk` or `lib-std.m4`, or a custom system lookup is required on AIX, etc). Thanks.
> 
> In 17, SystemLookup loads a specific library that is generated at build time - which contains all the c stdlib symbols. That's what the Lib.gmk changes are for. What I suspect is that the library is not generated at all, or not correctly on AIX, which then causes the system lookup to misbehave.
> 
> I would start by double checking that you have a file like this:
> 
> <JDK_ROOT>/lib/libsyslookup.so
> 
> And then, if the library exists, check that it has the right dependency; on my linux it's an empty library, but which depends on libc, libm and libdl:
> 
> 
> ldd lib/libsyslookup.so  
> 	linux-vdso.so.1 (0x00007fff2bdf7000)
> 	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f35f1def000)
> 	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f35f1ca0000)
> 	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f35f1c9a000)

Hi @mcimadamore,

Here's output of `libsyslookup.so` on AIX:

$ ldd  ./lib/libsyslookup.so
./lib/libsyslookup.so needs:  <--- nothing here

$ whereis libc
libc: /usr/lib/libc.a /usr/lib/libc128.a /usr/ccs/lib/libc.a

So it is high likely that there were no dependencies in this generated library.

> perhaps on AIX something similar to what we did for Windows [1] would be better.

That's what I thought to be the only way around but might need to figure out the specifics on AIX.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4316