Versioned Symbols in Shared Libraries

[Maurizio Cimadamore]

Hi,
I thought about this in the past.

IMHO jextract should have a way not to generate the usual common 
classes. This also comes up when there are multiple extractions that end 
up re-generating the same RuntimeHelper etc.

Jextract should have a way to say "please find runtime helper in that 
path". With that, I think your first problem would go away.

I think (2) looks like a bug - in the sense that we should use same 
logic we use for native calls (e.g. stash method handles in constant 
classes) - but perhaps this was done to reuse e.g. a global segment. But 
it is still questionable, so, a bug.

On (3), my feeling is that, if you can plug in your own SymbolLookup 
(see above), then you can initialize the SymbolLookup using some 
configuration in some file, which tells it which version it should look 
for, given a symbol name.

That way, jextract would lookup "foo" and your lookup would say "ah, ok, 
I know I need foo@@1 here".

Maurizio

On 30/09/2022 11:49, Sebastian Stenzel wrote:
> Thanks for the suggestion, a quick test [1] shows that a custom SymbolLookup based on dlvsym does in fact work (hooray!)
>
> However, this isn't (yet?) very practical with jextract, because:
> 1. afaik there is no way to tell it to use a custom SymbolLookup
> 2. jextract'ed code defines method handles eagerly (static constants) - while the lib may not yet be opened during class loading
> 2. we have to add versions to symbol names for all jextracted code manually
>
> All this can be solved manually, but re-jextract-ing isn't much fun, eliminating one of its benefits, which (at least to me) is rapidly adjusting Java code to upstream native library changes.
>
> [1] https://urldefense.com/v3/__https://github.com/cryptomator/jfuse/commit/85b296c1d361d3756b6fa842b2827fe63700e535__;!!ACWV5N9M2RV99hQ!PjmSLHFotOUgXUIjUUoklhj3z1pUzJDkDOMoUJdOBPrKk1tDGA4JeJV9wNPYz6YusTY8rGXsSqUO22S5bLn3VeNR8kRRxN_xAw$
>
>> On 29. Sep 2022, at 16:58, Maurizio Cimadamore <maurizio.cimadamore at oracle.com> wrote:
>>
>> Hi Sebastian,
>> Sadly, `dlopen` only does unversioned lookups (unless you use some tricks, like the RTLD_NEXT pseudo handles, which we don't use). On *nix systems another primitive for looking up symbols exists, namely `dlvsym` which takes an extra string pointer (the version).
>>
>> So, I think your best bet would to create your own SymbolLookup which can be initialized with a given version number, which is then passed to calls to `dlvsym`.
>>
>> Since this capability doesn't make sense on platforms other than GCC (I believe), adding support for versioning in SymbolLookup itself would seem a bit odd.
>>
>> Maurizio
>>
>> On 29/09/2022 12:49, Sebastian Stenzel wrote:
>>> Hi,
>>>
>>> Today I learned about versioned symbols when investigating a downwards compatibility problem in a jextract'ed API. See discussion [1]
>>>
>>> The concept is new to me, but apparently broadly used to make sure to invoke the exact function that was referenced at compile time. Since jextract only knows the header files, I suppose it isn't aware of such versions? Does the SymbolLookup API provide any mechanism to lookup a specific version?
>>>
>>> I hope you can make sense of what I'm asking as this is all new to me and I'm not sure I'm asking the right questions.
>>>
>>> Thank you!
>>>
>>> [1]: https://urldefense.com/v3/__https://github.com/libfuse/libfuse/discussions/701__;!!ACWV5N9M2RV99hQ!PjmSLHFotOUgXUIjUUoklhj3z1pUzJDkDOMoUJdOBPrKk1tDGA4JeJV9wNPYz6YusTY8rGXsSqUO22S5bLn3VeNR8kQlfFHH4Q$