Linking to macOS frameworks

[Maurizio Cimadamore]

I put together a quick Gist to show a possible approach:

https://gist.github.com/mcimadamore/0883ea6f4836ae0c1d2a31c48197da1a

This allows you to obtain a dlopen-based symbol lookup (see 
DlOpen::lookup). To open the library, the flag RTLD_LAZY is used. The 
provided scope is used to determine the library lifecycle - that is, 
when the scope is closed, the library will be unloaded.

This is a fairly simple implementation, but it should allow you to "just 
use" dlopen/dlsym to load libraries and lookup sym.

Maurizio

On 22/05/2021 22:12, Maurizio Cimadamore wrote:
> Hi,
> in the new API, you could in principle use the linker API to create a 
> wrapper around dlopen/dlsym, and then wrap a SymbolLookup around that.
>
> You would have to be careful not to unload the library _while_ a 
> native function is being invoked (which is why we didn't provide such 
> an alternative out of the box - yet).
>
> We plan to have a more direct wrapper around dlopen in the future - 
> but we have to address this first:
>
> https://bugs.openjdk.java.net/browse/JDK-8267451
>
> to allow for safe unloading in the presence of multiple threads.
>
> Maurizio
>
> On 22/05/2021 20:02, Clayton Walker wrote:
>> I've since updated to the most recent panama build, panama+3-167.
>>
>> Under Mojave I can load the IOKit library via
>> System.load("/System/Library/Frameworks/IOKit.framework/Versions/A/IOKit"); 
>>
>> and then lookup the symbol "IORegistryGetRootEntry" via loaderLookup,
>> however under Big Sur this code no longer works. While I can call
>> dlopen manually under Big Sur, that doesn't make the symbols in the
>> loaded library available to loaderLookup. This makes it frustrating
>> trying to bind to native libraries on macOS.
>>   I guess a workaround might be to create a dylib that links to a
>> framework, then load that? But then you're back shipping native
>> library shims.
>>
>> At least during testing, removing the file.exists() check from
>> NativeLibaries.loadLibrary allowed the code to properly work under Big
>> Sur.
>>
>> On Wed, May 19, 2021 at 3:12 PM Maurizio Cimadamore
>> <maurizio.cimadamore at oracle.com> wrote:
>>> I see.
>>>
>>> Obviously some kind of workaround is happening behind the scene when
>>> dlopen is invoked with that library name, as a compatibility measure.
>>>
>>> If you load this:
>>>
>>> /usr/lib/libSystem.B.dylib
>>>
>>> Do you, by any change, get access to the symbols you want?
>>>
>>> Anyway - to your original question, LibraryLookup.ofPath works with a
>>> library file in the file system. Here it seems a case where libraries
>>> are not in the file system, but in some other cache, and dlopen has 
>>> been
>>> instructed to "work" when asked with certain well-known names.
>>>
>>> In this case, if the above trick (libSystem) doesn't work, the only
>>> thing left to you might well be calling dlopen directly.
>>>
>>> Maurizio
>>>
>>> On 19/05/2021 20:31, Clayton Walker wrote:
>>>> Hi Maurizio,
>>>>
>>>> Yes, UserNotification is a library, but unfortunately
>>>> UserNotification.dylib is no longer in that folder.
>>>> Per this page 
>>>> https://urldefense.com/v3/__https://developer.apple.com/documentation/macos-release-notes/macos-big-sur-11_0_1-release-notes__;!!GqivPVa7Brio!PxJkXokERrp3NAcaRqK7q1eIHPqhs9AWyxmbu4_AfBKvGVbf2OcqtR2HUvwImxChbwHDpmE$ 
>>>> ,
>>>> Apple's release notes state
>>>> `copies of dynamic libraries are no longer present on the filesystem`.
>>>> So while previously they did exist on the filesystem, now they're not.
>>>> dlopen still handles the path on macOS Big Sur,
>>>> dlopen("/System/Library/Frameworks/UserNotifications.framework/Versions/Current/UserNotifications"); 
>>>>
>>>> works.
>>>>
>>>>
>>>> On Wed, May 19, 2021 at 11:03 AM Maurizio Cimadamore
>>>> <maurizio.cimadamore at oracle.com> wrote:
>>>>> Hi Clayton,
>>>>> Is UserNotification a library? E.g. is there a file
>>>>> UserNotification.dylib in that folder?
>>>>>
>>>>> If so, just use "Library.ofPath" and include the suffix - as that 
>>>>> method
>>>>> wants the library absolute path.
>>>>>
>>>>> Maurizio
>>>>>
>>>>> On 19/05/2021 17:46, Clayton Walker wrote:
>>>>>> Hello,
>>>>>>
>>>>>> I'm working on a project that would let you call objective-c 
>>>>>> functions from
>>>>>> java, without first needing to wrap the api in a shared library. 
>>>>>> While this
>>>>>> works for symbols that are already available
>>>>>> (i.e.LibraryLookup.ofDefault().lookup("objc_getClass")), it 
>>>>>> doesn't work
>>>>>> for symbols that are in another framework.
>>>>>>
>>>>>> For example, I would like to be able to write something like this:
>>>>>> LibraryLookup.ofLibrary("/System/Library/Frameworks/UserNotifications.framework/Versions/Current/UserNotifications"); 
>>>>>>
>>>>>> // or
>>>>>> LibraryLookup.ofPath(Path.of("/System/Library/Frameworks/UserNotifications.framework/Versions/Current/UserNotifications")); 
>>>>>>
>>>>>> however this currently doesn't work. The first call throws with 
>>>>>> "Directory
>>>>>> separator should not appear in library name", second one throws with
>>>>>> "Library not found".
>>>>>>
>>>>>> I've been able to work around this by using dlopen
>>>>>> MemoryAddress l1;
>>>>>> try (var c =
>>>>>> CLinker.toCString("/System/Library/Frameworks/UserNotifications.framework/Versions/Current/UserNotifications")) 
>>>>>>
>>>>>> {
>>>>>>        l1 = (MemoryAddress) dlopen.invokeExact(c.address(), 1);
>>>>>> }
>>>>>> but this seems rather strange. It looks like LibraryLookup.ofPath 
>>>>>> requires
>>>>>> the path to exist, whereas in Big Sur the files no longer exist (via
>>>>>> https://urldefense.com/v3/__https://developer.apple.com/forums/thread/657267__;!!GqivPVa7Brio!PxJkXokERrp3NAcaRqK7q1eIHPqhs9AWyxmbu4_AfBKvGVbf2OcqtR2HUvwImxChS2opDZY$ 
>>>>>> ).