[foreign] RFC: Jextract -l options ignored by SymbolFilter when -L is not specified

Tue Jan 22 19:20:43 UTC 2019

This also seems the most natural to me, since it follows what the linker 
flags do.

-L is to specify additional linker directories. We would consider 
java.library.path to be the "default"/"system" directories.

This is also what I tried to do in the patch [1] (minor update). With 
the addition of emitting a warning that symbol filtering is disabled 
when -l is used but no library paths are available (either in -L options 
or in java.library.path).

That said, I think having an extra option to explicitly turn on, or off, 
the symbol checking is a good idea as well.

Jorn

[1]: 
http://cr.openjdk.java.net/~jvernee/panama/webrevs/jlibpath/webrev.02/

Henry Jen schreef op 2019-01-22 17:40:
> It is preferred to keep options compatible with cc when applicable.
> 
> -L is only to provide the path for library at tooling time, that’s 
> jextract.
> —infer-path is similar to -R, will record the path for searching at
> runtime, that’s is, the path specified with -L will be added into
> search path of library.
> 
> As symbol check, it should be enabled with -l. -L is simply provide
> extra path to search for the library, without -L, it will simply
> search in java.library.path.
> 
> Cheers,
> Henry
> 
> 
>> On Jan 22, 2019, at 4:31 AM, Maurizio Cimadamore 
>> <maurizio.cimadamore at oracle.com> wrote:
>> 
>> 
>> On 22/01/2019 12:09, Jorn Vernee wrote:
>>> This sounds good, I really like the idea of a separate option to 
>>> enable the symbol filtering. But can you share what you think the 
>>> role of java.library.path should be as well?
>> 
>> I think using java.library.path as a default for the missing symbol 
>> check could be ok. But I don't think it would be ok to use it as a 
>> basis for infer-rpath. That is, I don't want static properties (e.g. 
>> valid at extraction time) to spill onto the runtime. If the user 
>> really wants to set some dynamic property, it has to use an explicit 
>> flag to do so (e.g. -L).
>> 
>> Maurizio
>> 
>>> 
>>> Jorn
>>> 
>>> Maurizio Cimadamore schreef op 2019-01-22 12:58:
>>>> Looking at this, I remember being confused about this too.
>>>> 
>>>> Let me try to see if we can find a better stacking for the existing
>>>> options - as Sundar said, we currently have:
>>>> 
>>>> * -l
>>>> 
>>>> This option is used to specify library _names_.
>>>> 
>>>> The main goal of this option is to alter the contents of the
>>>> @NativeHeader annotation (by adding the library name) but there are,
>>>> as we shall see, other subtle side-effects.
>>>> 
>>>> * -L + -l
>>>> 
>>>> When both -L and -l are specified, the so called "missing symbols
>>>> check" will kick in,  that is, jextract will check that all symbols 
>>>> in
>>>> the library are indeed defined in the header files being extracted. 
>>>> A
>>>> subtle side-effect of that check, is that when -l and -L are 
>>>> specified
>>>> together, and the missing symbol check is enabled, jextract will 
>>>> warn
>>>> for symbols not found and _it will exclude them_ from the extracted
>>>> classfile (w/o need for --include-symbols or --exclude-symbols).
>>>> 
>>>> * -L + -l + -infer-rpath
>>>> 
>>>> When -L and -l are used together, and the -infer-rpath option is
>>>> given, a runtime library path will be inferred from the contents of
>>>> -L, and will be stored in @NativeHeader, so that the binder can use
>>>> it.
>>>> 
>>>> 
>>>> I think the status quo is a bit confusing - because -L has multiple
>>>> functions (it serves up the library paths to be used as inferred
>>>> rpaths, and it also serves up the library paths to be used for the
>>>> missing symbol check). I think a more consistent stacking could be
>>>> something like this:
>>>> 
>>>> -l --> used to specify library _names_; only side-effect is contents
>>>> of @NativeHeader
>>>> 
>>>> -L --> used to specify _custom_ library _paths_; no side-effects
>>>> 
>>>> -exclude-missing -> must be used in conjunction with -l and -L ;
>>>> enables the missing symbol check and auto-exclusion
>>>> 
>>>> -infer-rpath -> must be used in conjunction with -l and -L ; enables
>>>> rpath inference (rpath inferred with paths specified in -L)
>>>> 
>>>> 
>>>> Thoughts?
>>>> 
>>>> Maurizio
>>>> 
>>>> 
>>>> On 22/01/2019 05:41, Sundararajan Athijegannathan wrote:
>>>>> I don't think it is a bug - afaik it is as per design. The primary 
>>>>> use of "-l" is to record the library in annotation of the generated 
>>>>> jar - so that binder can auto-load the library (either from 
>>>>> java.library.path configuration or -rpath value recorded in 
>>>>> annotation).  It is okay to record name of the shared object alone 
>>>>> and leave the library path configuration to java.library.path 
>>>>> setting.
>>>>> 
>>>>> "-L" option is added feature to perform missing symbols checking. 
>>>>> "-rpath" option is to add a path for library search - so that 
>>>>> binder can locate the shared object in the specific directory. If 
>>>>> no -rpath is specified, "-L" is used for runtime search as well.
>>>>> 
>>>>> -Sundar
>>>>> 
>>>>> On 22/01/19, 12:01 AM, Jorn Vernee wrote:
>>>>>> Hi,
>>>>>> 
>>>>>> I've recently updated the instructions for using libraries on 
>>>>>> Windows. For python the jextract example I gave was:
>>>>>> 
>>>>>>     jextract -l python27 -o "python.jar" -t "org.python" 
>>>>>> C:\Python27\include\Python.h
>>>>>> 
>>>>>> I'm lacking an `-L` option here (for specifying library 
>>>>>> directories) since the contents of PATH seems to be added to 
>>>>>> java.library.path by default, and this is presumably also how 
>>>>>> jextract is able to load the library. But, since I'm not using an 
>>>>>> `-L` option, SymbolFilter is not checking if the symbols are in 
>>>>>> the python27.dll [1]
>>>>>> 
>>>>>>     private void initSymChecker(List<String> linkCheckPaths) {
>>>>>>         if (!libraryNames.isEmpty() && !linkCheckPaths.isEmpty()) 
>>>>>> {
>>>>>>             // ... init symChecker
>>>>>>         } else {
>>>>>>             symChecker = null;
>>>>>>         }
>>>>>>     }
>>>>>> 
>>>>>> (linkCheckPaths comes from the -L option values)
>>>>>> 
>>>>>> This behaviour is somewhat unexpected. At least a warning that 
>>>>>> missing an `-L` option will turn off symbol checking would be 
>>>>>> nice.
>>>>>> 
>>>>>> We could also add the paths in `java.library.path` to the list of 
>>>>>> link check paths in jextract [2]. That would mean that the symbol 
>>>>>> checker would run for the example command.
>>>>>> 
>>>>>> What do you think?
>>>>>> 
>>>>>> Jorn
>>>>>> 
>>>>>> [1] : 
>>>>>> http://hg.openjdk.java.net/panama/dev/file/eaca2d16b80b/src/jdk.jextract/share/classes/com/sun/tools/jextract/SymbolFilter.java#l89
>>>>>> [2] : 
>>>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/jlibpath/webrev.01/