[foreign] RFC: Jextract -l options ignored by SymbolFilter when -L is not specified

Sundararajan Athijegannathan sundararajan.athijegannathan at oracle.com
Wed Jan 23 02:01:41 UTC 2019 

java.library.path contains only the JRE's own directories + system 
specific java dirs. This does not include directories like /usr/lib or 
/usr/local/lib (where you're likely to find shared objects of C 
libraries that are installed).  In other words, the default value of 
java.library.path system property does not get you too far.

If user has to pass java.library.path for jextract, s/he has to use

     jextract -J-Djava.library.path=...

-J is needed to set pass JVM option to jdk/bin tools other than "java". 
That is not super friendly. S/he might as well use -L. So I'm not sure I 
agree with using "java.library.path" as fallback.

-Sundar

On 23/01/19, 12:50 AM, Jorn Vernee wrote:
> This also seems the most natural to me, since it follows what the 
> linker flags do.
>
> -L is to specify additional linker directories. We would consider 
> java.library.path to be the "default"/"system" directories.
>
> This is also what I tried to do in the patch [1] (minor update). With 
> the addition of emitting a warning that symbol filtering is disabled 
> when -l is used but no library paths are available (either in -L 
> options or in java.library.path).
>
> That said, I think having an extra option to explicitly turn on, or 
> off, the symbol checking is a good idea as well.
>
> Jorn
>
> [1]: 
> http://cr.openjdk.java.net/~jvernee/panama/webrevs/jlibpath/webrev.02/
>
> Henry Jen schreef op 2019-01-22 17:40:
>> It is preferred to keep options compatible with cc when applicable.
>>
>> -L is only to provide the path for library at tooling time, that’s 
>> jextract.
>> —infer-path is similar to -R, will record the path for searching at
>> runtime, that’s is, the path specified with -L will be added into
>> search path of library.
>>
>> As symbol check, it should be enabled with -l. -L is simply provide
>> extra path to search for the library, without -L, it will simply
>> search in java.library.path.
>>
>> Cheers,
>> Henry
>>
>>
>>> On Jan 22, 2019, at 4:31 AM, Maurizio Cimadamore 
>>> <maurizio.cimadamore at oracle.com> wrote:
>>>
>>>
>>> On 22/01/2019 12:09, Jorn Vernee wrote:
>>>> This sounds good, I really like the idea of a separate option to 
>>>> enable the symbol filtering. But can you share what you think the 
>>>> role of java.library.path should be as well?
>>>
>>> I think using java.library.path as a default for the missing symbol 
>>> check could be ok. But I don't think it would be ok to use it as a 
>>> basis for infer-rpath. That is, I don't want static properties (e.g. 
>>> valid at extraction time) to spill onto the runtime. If the user 
>>> really wants to set some dynamic property, it has to use an explicit 
>>> flag to do so (e.g. -L).
>>>
>>> Maurizio
>>>
>>>>
>>>> Jorn
>>>>
>>>> Maurizio Cimadamore schreef op 2019-01-22 12:58:
>>>>> Looking at this, I remember being confused about this too.
>>>>>
>>>>> Let me try to see if we can find a better stacking for the existing
>>>>> options - as Sundar said, we currently have:
>>>>>
>>>>> * -l
>>>>>
>>>>> This option is used to specify library _names_.
>>>>>
>>>>> The main goal of this option is to alter the contents of the
>>>>> @NativeHeader annotation (by adding the library name) but there are,
>>>>> as we shall see, other subtle side-effects.
>>>>>
>>>>> * -L + -l
>>>>>
>>>>> When both -L and -l are specified, the so called "missing symbols
>>>>> check" will kick in,  that is, jextract will check that all 
>>>>> symbols in
>>>>> the library are indeed defined in the header files being extracted. A
>>>>> subtle side-effect of that check, is that when -l and -L are 
>>>>> specified
>>>>> together, and the missing symbol check is enabled, jextract will warn
>>>>> for symbols not found and _it will exclude them_ from the extracted
>>>>> classfile (w/o need for --include-symbols or --exclude-symbols).
>>>>>
>>>>> * -L + -l + -infer-rpath
>>>>>
>>>>> When -L and -l are used together, and the -infer-rpath option is
>>>>> given, a runtime library path will be inferred from the contents of
>>>>> -L, and will be stored in @NativeHeader, so that the binder can use
>>>>> it.
>>>>>
>>>>>
>>>>> I think the status quo is a bit confusing - because -L has multiple
>>>>> functions (it serves up the library paths to be used as inferred
>>>>> rpaths, and it also serves up the library paths to be used for the
>>>>> missing symbol check). I think a more consistent stacking could be
>>>>> something like this:
>>>>>
>>>>> -l --> used to specify library _names_; only side-effect is contents
>>>>> of @NativeHeader
>>>>>
>>>>> -L --> used to specify _custom_ library _paths_; no side-effects
>>>>>
>>>>> -exclude-missing -> must be used in conjunction with -l and -L ;
>>>>> enables the missing symbol check and auto-exclusion
>>>>>
>>>>> -infer-rpath -> must be used in conjunction with -l and -L ; enables
>>>>> rpath inference (rpath inferred with paths specified in -L)
>>>>>
>>>>>
>>>>> Thoughts?
>>>>>
>>>>> Maurizio
>>>>>
>>>>>
>>>>> On 22/01/2019 05:41, Sundararajan Athijegannathan wrote:
>>>>>> I don't think it is a bug - afaik it is as per design. The 
>>>>>> primary use of "-l" is to record the library in annotation of the 
>>>>>> generated jar - so that binder can auto-load the library (either 
>>>>>> from java.library.path configuration or -rpath value recorded in 
>>>>>> annotation).  It is okay to record name of the shared object 
>>>>>> alone and leave the library path configuration to 
>>>>>> java.library.path setting.
>>>>>>
>>>>>> "-L" option is added feature to perform missing symbols checking. 
>>>>>> "-rpath" option is to add a path for library search - so that 
>>>>>> binder can locate the shared object in the specific directory. If 
>>>>>> no -rpath is specified, "-L" is used for runtime search as well.
>>>>>>
>>>>>> -Sundar
>>>>>>
>>>>>> On 22/01/19, 12:01 AM, Jorn Vernee wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> I've recently updated the instructions for using libraries on 
>>>>>>> Windows. For python the jextract example I gave was:
>>>>>>>
>>>>>>>     jextract -l python27 -o "python.jar" -t "org.python" 
>>>>>>> C:\Python27\include\Python.h
>>>>>>>
>>>>>>> I'm lacking an `-L` option here (for specifying library 
>>>>>>> directories) since the contents of PATH seems to be added to 
>>>>>>> java.library.path by default, and this is presumably also how 
>>>>>>> jextract is able to load the library. But, since I'm not using 
>>>>>>> an `-L` option, SymbolFilter is not checking if the symbols are 
>>>>>>> in the python27.dll [1]
>>>>>>>
>>>>>>>     private void initSymChecker(List<String> linkCheckPaths) {
>>>>>>>         if (!libraryNames.isEmpty() && !linkCheckPaths.isEmpty()) {
>>>>>>>             // ... init symChecker
>>>>>>>         } else {
>>>>>>>             symChecker = null;
>>>>>>>         }
>>>>>>>     }
>>>>>>>
>>>>>>> (linkCheckPaths comes from the -L option values)
>>>>>>>
>>>>>>> This behaviour is somewhat unexpected. At least a warning that 
>>>>>>> missing an `-L` option will turn off symbol checking would be nice.
>>>>>>>
>>>>>>> We could also add the paths in `java.library.path` to the list 
>>>>>>> of link check paths in jextract [2]. That would mean that the 
>>>>>>> symbol checker would run for the example command.
>>>>>>>
>>>>>>> What do you think?
>>>>>>>
>>>>>>> Jorn
>>>>>>>
>>>>>>> [1] : 
>>>>>>> http://hg.openjdk.java.net/panama/dev/file/eaca2d16b80b/src/jdk.jextract/share/classes/com/sun/tools/jextract/SymbolFilter.java#l89
>>>>>>> [2] : 
>>>>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/jlibpath/webrev.01/ 
>>>>>>>

More information about the panama-dev mailing list