Using SymbolLookup#libraryLookup with fallback of SymbolLookup#loaderLookup

[some-java-user-99206970363698485155 at vodafonemail.de]

Thanks a lot for both your answers! Yes Maurizio you are right, I was 
asking this in the context of tree-sitter / jtreesitter ;)
However, I think this generally applies to all native libraries which 
want to provide 'official' Java bindings through the foreign API, for 
example also database drivers or similar.

> A user can simply amend the OS dependent library search locations. 
> Typically through amending PATH (Windows), LD_LIBRARY_PATH (Linux), or 
> DYLD_LIBRARY_PATH (mac), but there are other options. This will work 
> for a library's dependencies as well, and is more or less as much 
> effort as setting java.library.path, I think?

I think this is part of the problem: If you are maintaining Java 
bindings for native libraries, and maybe even provide prebuilt binaries 
for certain targets, then users can't just place the native libraries at 
one place. Instead you need to tell your users to lookup where their OS 
loads libraries from (and possibly you cannot assist because you are not 
familiar with their OS). And similarly if users want to write a start or 
install script for their application, they can't just copy the library 
to a fixed location, instead they need to write OS specific code (and 
possibly even code for detection of the OS) to place the libraries at 
the correct location respectively adjust the correct environment 
variable. This can be cumbersome to users who are not very familiar with 
native development.
And it can lead to situations similar to "works on my machine", for 
example: You develop on Windows and place the Windows native library in 
the working directory (= included in PATH) and it works, then you try to 
set up CI on Linux but placing the Linux native library in the working 
directory does not work (because the working directory is not included 
in LD_LIBRARY_PATH by default).

Maybe this can be solved by using a different `--library` option of 
jextract, but it might not be obvious what the correct / best choice for 
binding authors is.


>> Ideally this would be solved with a user-provided SymbolLookup (see 
>> also [1] and CODETOOLS-7903186), but that might not be that easily 
>> achievable or complicate the API of jextract-based bindings.
> Yes, this is the general solution to this kind of problems.
My concern here is that this mostly works for bindings which expose 
native methods as static Java methods, e.g. OpenGL [^1]. However, for 
bindings such as jtreesitter this might require larger API changes 
because it currently uses constructors to create binding objects (e.g. 
`new Parser(...)`). If jextract supported providing a custom 
SymbolLookup, then it would require a factory class and factory methods 
which use that lookup, e.g. `var jtreesitter = new 
JTreesitter(symbolLookup); var parser = jtressitter.newParser(...)`. 
That might be the cleanest approach but would require some (larger) 
refactoring for existing code.
An alternative might be a `static volatile SymbolLookup` field which is 
lazily initialized and can be overwritten (but only if not overwritten / 
initialized yet).


These are just some of my thoughts on this. As mentioned in the other 
thread, I am only a user of jtreesitter and am not directly writing Java 
bindings using the foreign API.
I am also not sure what the best solution here would be, maybe other 
users will come up with better solutions over time and highlight 
different aspects of this, or it would be worth having a look at how 
other projects using the foreign API currently solve this.


> I want to point out another issue with 
> System::load(Library)/java.library.path. It doesn't handle library 
> dependencies, which are instead handled automatically by the OS loader.
Good point, this will probably lead to confusing 'surprises' in some 
cases. I had created JDK-8339367 to make it clearer how 
`java.library.path` works and where it has an effect.
Though if authors of bindings know that their library has no specific 
dependencies, then maybe at least in these cases this is not an issue.


> [The proposal to ignore the exception and use loaderLookup as 
> fallback] would allow jextract to “keep going” even in the face of 
> library loading errors. My question is whether this behavior would be 
> useful
Also good point, yes that might make troubleshooting more difficult, 
especially if the library file exists but could not be loaded for 
another reason. And it would not be as clear anymore where a symbol was 
actually loaded from.


Kind regards


[^1]: Similar to what you Maurizio mentioned previously with
```
final static OpenGL GL = new OpenGL(mylookup);
GL.glVersion();
```