Project Panama & dynamic library loading

[Maurizio Cimadamore]

Another thing to check is library dependencies, with 'ldd <filename'.

I've seen cases of "weirdly" packaged libraries which failed to specify 
a dependency on other required libraries. Opening such libraries would 
lead to missing symbols, even when using a low-level 'dlopen', not much 
that can be done in that case.

I wonder if in the Arch case, libnvidia-ml.so depends on 
libnvidia-ml.so.<DRIVER_VERSION>, whereas in Ubuntu it does not.

Also, there seem to be some issues with nvidia libraries and their 
integration with the host system:

https://github.com/NVIDIA/nvidia-docker/issues/854

And even weirder interaction between Nvidia drivers and bumblebee 
causing wrong library to be loaded

https://askubuntu.com/questions/833862/nvidia-smi-couldnt-find-libnvidia-ml-so-library

In other words, _something_ seems not to be set up correctly in your 
Ubuntu install, and the jextract/VM issues with library loading are 
likely just symptoms, not the cause.

Maurizio

On 16/09/2019 10:53, Maurizio Cimadamore wrote:
> I agree with this analysis.
>
> One addition - it's not really that jextract cares about sonames - 
> it's just that if you call
>
> System::loadLibrary("foo")
>
> and you are on Linux, the VM will try to load:
>
> libfoo.so
>
> on all the paths that are part of the native library path.
>
> If you want something different, then there's the more general 
> System::load which takes a full _patth_ (not a libname), which can of 
> course contain sonames and all.
>
> Jexract/binder is relying on libnames and System::loadLibrary - so 
> it's expecting that the _right_ library is available at PATH/libfoo.so 
> (I've seen cases where this path, despite pointing to a shared object, 
> it pointed to one that did NOT contain any symbols - on Linux 
> x64/Ubuntu).
>
> At the moment jextract doesn't have a way to specify a full library 
> path - but that could be added. In the longer term, I think that, as 
> part of the 'cleanup' I was mentioning few emails ago, it would also 
> be nice, if, in addition of honoring the right system native paths 
> (e.g. ld paths on linux), the VM would also be smarter about sonames 
> when deriving a library path from a library name.
>
> Maurizio
>
> On 15/09/2019 14:00, Samuel Audet wrote:
>> The soname that the OS cares about is in the file itself, not the 
>> filename. Check with `readelf -d`, or `objdump -p`, etc to make sure 
>> you're actually loading what you think you're loading. The soname 
>> might have a version and that's what the OS will consider as the 
>> library name. The runtime will search the path for files that have 
>> that as filename, but once loaded, whether it matches the filename or 
>> not doesn't matter. If jextract cares about the filename on Linux, 
>> you'll run into problems when there's a mismatch between the two. 
>> Windows is different in that regard...
>>
>> On 9/15/19 9:13 PM, Ty Young wrote:
>>>
>>> On 9/15/19 6:59 AM, Maurizio Cimadamore wrote:
>>>>
>>>> On 15/09/2019 12:27, Ty Young wrote:
>>>>> The only difference between the two distros is a ".0" soname at 
>>>>> the end on Ubuntu vs. none on Arch Linux*. Could that be the reason? 
>>>>
>>>> I had issues with soname in the past. Would it be possible, as a 
>>>> means of an experiment to try and copy the library stripped of the 
>>>> .0 suffix?
>>>>
>>>
>>> Yep. Tried it on a USB live boot of Pop!_OS 19.10 and it stops 
>>> complaining about NvXCtrl.
>>>
>>>
>>> ...Then it starts complaining about nvidia-ml not existing but the 
>>> library names are 100% the same with paths only being different. 
>>> There is libnvidia-ml.so, libnvidia-ml.so.0, and 
>>> libnvidia-ml.so.<DRIVER_VERSION> on both Ubuntu and Arch.
>>>
>>>
>>>> Maurizio
>>>>