Integrated: 7903654: Jextract should use library lookup

[Maurizio Cimadamore]

Uhm... I'm not sure I get all the moving parts here.

The jextract code here is loading "libssl.dylib". E.g. Jextract is not 
trying to directly load an unversioned "libcrypto.dylib". Correct?

If that's correct, the problem seems to be in _which_ libssl.dylib ends 
up being loaded by jextract. E.g. perhaps the version picked up by 
jextract has a dpeendency on an unversioned libcrypto.dylib, which then 
results in the error you are seeing.

But, jextract (and the library lookup) are merely asking the dynamic 
linker what to load, given a library name. That is, the code jextract 
generates will result in "dlopen("libssl.dylib")". Does it means that, 
on MacOS, that dlopen command leads to a crash? That seems surprising 
(as I'd expect the dynamic linker to be configured in a way that makes 
sense), but I see something similar documented e.g. here:

https://github.com/sergot/openssl/issues/81

It seems like the solution is to pass a versioned library name to dlopen 
(e.g. using jextract option `-l :|libssl.46.dylib`). E.g. I don't think 
you can use the same library lookup for both Linux and MacOS. So, if you 
want to share code between MacOS and Linux you would have to manually 
tweak the generated code to do different things depending on the 
platform. Or, you could keep using -Djava.library.path (e.g. by passing 
--use-system-load-library to jextract).
|

Maurizio

On 13/02/2024 19:08, Rémy Maucherat wrote:
> On Thu, Feb 8, 2024 at 4:00 PM Maurizio Cimadamore
> <mcimadamore at openjdk.org>  wrote:
>> On Wed, 7 Feb 2024 16:18:45 GMT, Maurizio Cimadamore<mcimadamore at openjdk.org>  wrote:
>>
>>> Historically, jextract has always used `System::loadLibrary` and `System::load` to load libraries specified via the `-l` option. This is suboptimal, as in the most common case, such libraries will be installed using e.g. `apt`, in which case `System::load/loadLibrary` is of little use, as it cannot find libraries in path known to the dynamic linker.
>>>
>>> This patch changes the default for library loading to use `SymbolLookup::libraryLookup` instead. That is, in absence of any other options, each library provided via `-l` will create a new `SymbolLookup::loaderLookup`, associated with the library arena.
>>>
>>> The `-l` option also supports a new disambiguation mechanism, borrowed from the gcc linker options - that is, if the library name starts with `:`, what follows is parsed as a path. Otherwise, we assume it's a library name, and we add prefix/suffix, using `System::mapLibraryName` before creating the lookup.
>>>
>>> In cases where the old behavior is preferred (e.g. if libraries to be loaded sit in `java.library.path`) a fallback option has been provided, namely `--use-system-load-library`.
>>>
>>> Regardless of the options used, the header class will feature a composite lookup. Inside this lookup we find:
>>>
>>> 1. one or more library lookup (one for each `-l` option)
>>> 2. the loader lookup
>>> 3. the native linker's default lookup
>>>
>>> When `--use-system-load-library` is used, (1) is empty, and a static initializer block with the corresponding calls to `System::load`, `System::loadLibrary` is added instead.
>> This pull request has now been integrated.
>>
>> Changeset: 456632df
>> Author:    Maurizio Cimadamore<mcimadamore at openjdk.org>
>> URL:https://git.openjdk.org/jextract/commit/456632df2e0b00a3f0fd5714b020d00f90c41804
>> Stats:     176 lines in 32 files changed: 91 ins; 18 del; 67 mod
>>
>> 7903654: Jextract should use library lookup
> With that change in place, the code to load OpenSSL now looks like:
>      static final SymbolLookup SYMBOL_LOOKUP =
> SymbolLookup.libraryLookup(System.mapLibraryName("ssl"),
> LIBRARY_ARENA)
>              .or(SymbolLookup.loaderLookup())
>              .or(Linker.nativeLinker().defaultLookup());
>
> This works very well on Linux, where it improves things in some setups
> (like the Tomcat CI runs which are now able to load OpenSSL).
> However, on MacOS this seems to fail with:
> "WARNING: <path to java executable> is loading libcrypto in an unsafe way"
> And the JVM exits.
>
> Given this:https://github.com/cl-plus-ssl/cl-plus-ssl/issues/114#issuecomment-770370592
> It would seem it is trying to load the wrong "library" on that platform.
>
> Rémy
>
>> Reviewed-by: jvernee
>>
>> -------------
>>
>> PR:https://git.openjdk.org/jextract/pull/201
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/jextract-dev/attachments/20240214/970fe31e/attachment-0001.htm>