Project Panama & dynamic library loading

[Ty Young]

On 9/6/19 8:12 PM, Henry Jen wrote:
>
>> On Sep 6, 2019, at 4:58 PM, Ty Young <youngty1997 at gmail.com> wrote:
>>
>>
>> On 9/6/19 6:36 PM, Ty Young wrote:
>>>
>>> On Fri, Sep 6, 2019 at 6:34 PM, Maurizio Cimadamore <maurizio.cimadamore at oracle.com> wrote:
>>>> Hi Ty, you are not forced to save the library path in the generated extracted jar with the --record-library-path  option. If you can also omit that option, and then set java.library.path (or LD_LIBRARY_PATH) accordingly when it's time to run your application.
>>
>> Can java.library.path be modified in code or does it have to be a launch option to work? I've always assumed it was a launch option only and haven't thought much of it.
>>
> It’s a launch option, and thus can adapt to your deployment environment.


There is no specific "deployment environment". If you were to create 
bindings in Arch Linux you'd need to specify the pathing for every other 
Linux distro and even Windows, resulting in not working on more obscure 
Linux distros and two different builds between Windows and Linux. If you 
could just dynamically add the paths that'd make this so much easier...


>
>> It would really be nice if the whole thing was dynamic with zero hardcoded values, even if startup time was sacrificed due to library searching. With how fragmented the Linux ecosystem specifically is, there is never any guarantee that any hardcoded values(s) will work for all Linux distros.
>>
> Dynamic loading is supported since beginning, typical code snippet is like following.
>
> Library lib = Libraries.loadLibrary(MethodHandles.lookup(), "tensorflow");
> libTF = Libraries.bind(c_api_h.class, lib);
>
> As long as you define java.library.path to include correct search path for the native library, it should work just fine.


The search path being static is the problem here. Without a dynamic way 
to add libraries easily things will blowup real badly.


>
> The —record-library-path is added to provide a default search path for the static-forwarder class. It’s not an required options, but was added to search path provided by JVM, which include the path defined in  java.library.path.
>
> HTH,
> Henry
>
>
>>>> One thing that we'd also like to improve longer term is how the default value of the java.library.path is populated across the various OS/distros. The values under linux e.g. do not make a lot of sense (apart from /usr/lib) - and we need a better way to talk to the system linker.
>>
>> Sounds like a fun game of wackamole.
>>
>>
>>>> Maurizio On 07/09/2019 00:24, Ty Young wrote:
>>>>
>>>>     Hi, I mentioned this before in a linked article I did, but
>>>>     another major issue with current jextract/java library pathing
>>>>     is that it is static instead of dyanamic(that is, a specific
>>>>     path). This causes huge problems not only with cross-platform
>>>>     libraries but also in the heavily fragmented Linux ecosystem.
>>>>     NVML(Nvidia Management Library) for example exists in entirely
>>>>     different locations in /usr/lib in Ubuntu, Arch, and Fedora as
>>>>     well as in Windows. As a result, bindings have to be generated
>>>>     multiple times even on the same platform in order to work
>>>>     correctly. Can a way to dynamically specify the library path from
>>>>     within Java be added?