[foreign] RFC: Jextract -l options ignored by SymbolFilter when -L is not specified

Henry Jen henry.jen at oracle.com
Wed Jan 23 02:49:00 UTC 2019 

What we are suggesting is that, -L is supplement to java.library.path. that -L is optional while -l is required for validating again the shared library.

In fact, -L is pretty much like a shortcut to prepend more search path for java.library.path.

Cheers,
Henry

> On Jan 22, 2019, at 6:01 PM, Sundararajan Athijegannathan <sundararajan.athijegannathan at oracle.com> wrote:
> 
> java.library.path contains only the JRE's own directories + system specific java dirs. This does not include directories like /usr/lib or /usr/local/lib (where you're likely to find shared objects of C libraries that are installed).  In other words, the default value of java.library.path system property does not get you too far.
> 
> If user has to pass java.library.path for jextract, s/he has to use
> 
>    jextract -J-Djava.library.path=...
> 
> -J is needed to set pass JVM option to jdk/bin tools other than "java". That is not super friendly. S/he might as well use -L. So I'm not sure I agree with using "java.library.path" as fallback.
> 
> -Sundar
> 
> On 23/01/19, 12:50 AM, Jorn Vernee wrote:
>> This also seems the most natural to me, since it follows what the linker flags do.
>> 
>> -L is to specify additional linker directories. We would consider java.library.path to be the "default"/"system" directories.
>> 
>> This is also what I tried to do in the patch [1] (minor update). With the addition of emitting a warning that symbol filtering is disabled when -l is used but no library paths are available (either in -L options or in java.library.path).
>> 
>> That said, I think having an extra option to explicitly turn on, or off, the symbol checking is a good idea as well.
>> 
>> Jorn
>> 
>> [1]: http://cr.openjdk.java.net/~jvernee/panama/webrevs/jlibpath/webrev.02/
>> 
>> Henry Jen schreef op 2019-01-22 17:40:
>>> It is preferred to keep options compatible with cc when applicable.
>>> 
>>> -L is only to provide the path for library at tooling time, that’s jextract.
>>> —infer-path is similar to -R, will record the path for searching at
>>> runtime, that’s is, the path specified with -L will be added into
>>> search path of library.
>>> 
>>> As symbol check, it should be enabled with -l. -L is simply provide
>>> extra path to search for the library, without -L, it will simply
>>> search in java.library.path.
>>> 
>>> Cheers,
>>> Henry
>>> 
>>> 
>>>> On Jan 22, 2019, at 4:31 AM, Maurizio Cimadamore <maurizio.cimadamore at oracle.com> wrote:
>>>> 
>>>> 
>>>> On 22/01/2019 12:09, Jorn Vernee wrote:
>>>>> This sounds good, I really like the idea of a separate option to enable the symbol filtering. But can you share what you think the role of java.library.path should be as well?
>>>> 
>>>> I think using java.library.path as a default for the missing symbol check could be ok. But I don't think it would be ok to use it as a basis for infer-rpath. That is, I don't want static properties (e.g. valid at extraction time) to spill onto the runtime. If the user really wants to set some dynamic property, it has to use an explicit flag to do so (e.g. -L).
>>>> 
>>>> Maurizio
>>>> 
>>>>> 
>>>>> Jorn
>>>>> 
>>>>> Maurizio Cimadamore schreef op 2019-01-22 12:58:
>>>>>> Looking at this, I remember being confused about this too.
>>>>>> 
>>>>>> Let me try to see if we can find a better stacking for the existing
>>>>>> options - as Sundar said, we currently have:
>>>>>> 
>>>>>> * -l
>>>>>> 
>>>>>> This option is used to specify library _names_.
>>>>>> 
>>>>>> The main goal of this option is to alter the contents of the
>>>>>> @NativeHeader annotation (by adding the library name) but there are,
>>>>>> as we shall see, other subtle side-effects.
>>>>>> 
>>>>>> * -L + -l
>>>>>> 
>>>>>> When both -L and -l are specified, the so called "missing symbols
>>>>>> check" will kick in,  that is, jextract will check that all symbols in
>>>>>> the library are indeed defined in the header files being extracted. A
>>>>>> subtle side-effect of that check, is that when -l and -L are specified
>>>>>> together, and the missing symbol check is enabled, jextract will warn
>>>>>> for symbols not found and _it will exclude them_ from the extracted
>>>>>> classfile (w/o need for --include-symbols or --exclude-symbols).
>>>>>> 
>>>>>> * -L + -l + -infer-rpath
>>>>>> 
>>>>>> When -L and -l are used together, and the -infer-rpath option is
>>>>>> given, a runtime library path will be inferred from the contents of
>>>>>> -L, and will be stored in @NativeHeader, so that the binder can use
>>>>>> it.
>>>>>> 
>>>>>> 
>>>>>> I think the status quo is a bit confusing - because -L has multiple
>>>>>> functions (it serves up the library paths to be used as inferred
>>>>>> rpaths, and it also serves up the library paths to be used for the
>>>>>> missing symbol check). I think a more consistent stacking could be
>>>>>> something like this:
>>>>>> 
>>>>>> -l --> used to specify library _names_; only side-effect is contents
>>>>>> of @NativeHeader
>>>>>> 
>>>>>> -L --> used to specify _custom_ library _paths_; no side-effects
>>>>>> 
>>>>>> -exclude-missing -> must be used in conjunction with -l and -L ;
>>>>>> enables the missing symbol check and auto-exclusion
>>>>>> 
>>>>>> -infer-rpath -> must be used in conjunction with -l and -L ; enables
>>>>>> rpath inference (rpath inferred with paths specified in -L)
>>>>>> 
>>>>>> 
>>>>>> Thoughts?
>>>>>> 
>>>>>> Maurizio
>>>>>> 
>>>>>> 
>>>>>> On 22/01/2019 05:41, Sundararajan Athijegannathan wrote:
>>>>>>> I don't think it is a bug - afaik it is as per design. The primary use of "-l" is to record the library in annotation of the generated jar - so that binder can auto-load the library (either from java.library.path configuration or -rpath value recorded in annotation).  It is okay to record name of the shared object alone and leave the library path configuration to java.library.path setting.
>>>>>>> 
>>>>>>> "-L" option is added feature to perform missing symbols checking. "-rpath" option is to add a path for library search - so that binder can locate the shared object in the specific directory. If no -rpath is specified, "-L" is used for runtime search as well.
>>>>>>> 
>>>>>>> -Sundar
>>>>>>> 
>>>>>>> On 22/01/19, 12:01 AM, Jorn Vernee wrote:
>>>>>>>> Hi,
>>>>>>>> 
>>>>>>>> I've recently updated the instructions for using libraries on Windows. For python the jextract example I gave was:
>>>>>>>> 
>>>>>>>>    jextract -l python27 -o "python.jar" -t "org.python" C:\Python27\include\Python.h
>>>>>>>> 
>>>>>>>> I'm lacking an `-L` option here (for specifying library directories) since the contents of PATH seems to be added to java.library.path by default, and this is presumably also how jextract is able to load the library. But, since I'm not using an `-L` option, SymbolFilter is not checking if the symbols are in the python27.dll [1]
>>>>>>>> 
>>>>>>>>    private void initSymChecker(List<String> linkCheckPaths) {
>>>>>>>>        if (!libraryNames.isEmpty() && !linkCheckPaths.isEmpty()) {
>>>>>>>>            // ... init symChecker
>>>>>>>>        } else {
>>>>>>>>            symChecker = null;
>>>>>>>>        }
>>>>>>>>    }
>>>>>>>> 
>>>>>>>> (linkCheckPaths comes from the -L option values)
>>>>>>>> 
>>>>>>>> This behaviour is somewhat unexpected. At least a warning that missing an `-L` option will turn off symbol checking would be nice.
>>>>>>>> 
>>>>>>>> We could also add the paths in `java.library.path` to the list of link check paths in jextract [2]. That would mean that the symbol checker would run for the example command.
>>>>>>>> 
>>>>>>>> What do you think?
>>>>>>>> 
>>>>>>>> Jorn
>>>>>>>> 
>>>>>>>> [1] : http://hg.openjdk.java.net/panama/dev/file/eaca2d16b80b/src/jdk.jextract/share/classes/com/sun/tools/jextract/SymbolFilter.java#l89
>>>>>>>> [2] : http://cr.openjdk.java.net/~jvernee/panama/webrevs/jlibpath/webrev.01/

More information about the panama-dev mailing list