[External] : Re: Feedback / query on jextract for Windows 10

Tue Jan 26 16:54:30 UTC 2021

I'll take a look with the latest version - it might be that something is 
interacting with the lazy constant loading process.

Thanks for reporting this - minimizing startup of jextracted libraries 
is a pretty important use case for us.

Cheers
Maurizio

On 26/01/2021 15:43, Duncan Gittins wrote:
> I don't know what causes the speed difference, which may be down to 
> the jar size as you suggest. I'll re-test after the time that 
> "--source" generates same code as seen in the .class files when it may 
> be easier to understand why there is a difference. I am using 
> 16-panama+3-385.
>
> This isn't a very robust test, but I also tried deleted all the 
> un-necessary .class files from the 11MB jextract generated classes 
> (not produced using --source) down to 2MB jar, then I moved my 
> handcoded method into a separate class and added a dependency on a 
> different 2MB jar. Neither of those changes affected the typical 
> speeds of first call times - I still consistently see that the hand 
> coded method handle version of my app is ~300-500ms faster on first 
> call, and no difference for 2nd+ calls.
>
> Duncan
>
> On 25/01/2021 16:18, Maurizio Cimadamore wrote:
>>
>> On 25/01/2021 16:13, Duncan Gittins wrote:
>>> Thanks for the information.  As you suspected that bug must be 
>>> related and there is a minor difference in the --source output 
>>> because the problem does go away when removing the jextract 
>>> "--source" flag and making jar directly from generated classes 
>>> rather than just compiling from the generated source code.
>>>
>>> My code samples using jextract bindings are ~300ms slower at startup 
>>> time (but same speed runtime). I'm not certain if this overhead is 
>>> down to initialisation of static constants or simply to be expected 
>>> when adding 11MB jar footprint (compared to 56k jar for my 
>>> handwritten MH bindings). I will follow-up if I find out more.
>> If you see the startup slowdown even when omitting --source, then the 
>> issue is with loading the jar, rather than initializing - as 
>> explaining no initializing should happen outside for stuff that is 
>> actually used by your code - so I'd expect that initialization 
>> overhead for jextract code and your manually written versions should 
>> be roughly the same.
>>>
>>> Looking at the generated code for "xxx$constants$nn.java" there 
>>> seems to be a degree of unnecessary initialisation per 
>>> xxx$constants$nn class as it contains static variables for 
>>> FunctionDescriptor / MemoryLayout which would be initialised every 
>>> time that particular class was loaded and not used in my code. I 
>>> felt that my own jar was slow starting so switched foreign linker 
>>> references to a lazy load Supplier for every library / symbol / mh / 
>>> vh so that only the referenced types get created.
>>
>> Sure - but that's only when --source is used - when you generated 
>> classes directly, everything is lazy (and in a more general way than 
>> using Suppliers).
>>
>> So, can you please confirm, to be clear, that you still see startup 
>> regression even when NOT using "--source" ?
>>
>> Thanks!
>> Maurizio
>>
>>>
>>> I will have a look at future builds - this project is extremely 
>>> useful so I hope it gets into a full JDK release soon. Unfortunately 
>>> my issue with a huge jar in Eclipse means I can't easily develop 
>>> using the generated code variant right now, but in the meantime the 
>>> jextract code is still very helpful to work out how to call Windows 
>>> APIs correctly.
>>>
>>> Duncan
>>>
>>> On 25/01/2021 10:20, Maurizio Cimadamore wrote:
>>>>
>>>> On 23/01/2021 18:34, Duncan Gittins wrote:
>>>>> I've been using Panama on Windows 10 to eliminate my C# apps which 
>>>>> I would normally call via ProcessBuilder. I'm very impressed so 
>>>>> far and looking forward to newer versions: everything I have 
>>>>> written in Windows C# is now replaced with foreign memory / linker 
>>>>> calls to native code using the latest Java 16 EA.  I have examples 
>>>>> for various Win32 API and COM objects, HWND objects and 
>>>>> screensaver. However I'm struggling to get the code running with 
>>>>> output from jextract build. Perhaps I overlooked something very 
>>>>> obvious in how to setup?
>>>>>
>>>>> One example app is attached - this calls SystemParametersInfoA to 
>>>>> set the Windows desktop background. There are two implementations 
>>>>> of setImage: one with hardwired MethodHandle works fine, but the 
>>>>> jextract version gives UnsupportedOperationException for layout 
>>>>> unrelated to SystemParametersInfoA. If I edit the generated code 
>>>>> to comment out all lines unrelated to SystemParametersInfoA then 
>>>>> both versions of setImage work.
>>>>
>>>> I think the culprit is this:
>>>>
>>>> ```
>>>> Caused by: java.lang.UnsupportedOperationException: Invalid 
>>>> alignment requirements for layout 
>>>> b64(ArbitraryUserPointer)[abi/kind=POINTER,layout/name=ArbitraryUserPointer]
>>>>         at 
>>>> jdk.incubator.foreign/jdk.internal.foreign.LayoutPath.checkAlignment(LayoutPath.java:273)
>>>>         at 
>>>> jdk.incubator.foreign/jdk.internal.foreign.LayoutPath.dereferenceHandle(LayoutPath.java:159)
>>>>         at 
>>>> jdk.incubator.foreign/jdk.incubator.foreign.MemoryLayout.lambda$varHandle$2(MemoryLayout.java:488)
>>>>         at 
>>>> jdk.incubator.foreign/jdk.incubator.foreign.MemoryLayout.computePathOp(MemoryLayout.java:534)
>>>>         at 
>>>> jdk.incubator.foreign/jdk.incubator.foreign.MemoryLayout.varHandle(MemoryLayout.java:488)
>>>>         at 
>>>> duncan.win32.shobjidl_core.shlobj_core_h$constants$16.<clinit>(shlobj_core_h$constants$16.java:1844)
>>>> ```
>>>>
>>>> This seems a duplicate of another issue that has been reported last 
>>>> week:
>>>>
>>>> https://bugs.openjdk.java.net/browse/JDK-8259832
>>>>
>>>> E.g. jextract ignoring alignment requirements. Note that jextract 
>>>> generates lots of constants, and it takes only one bad one to make 
>>>> initialization fail.
>>>>
>>>>>
>>>>> The jextract command on Windows 10 (with fresh Vis Studio 
>>>>> installation) expands shlobj_core.h because I my app also uses 
>>>>> IShellLink.
>>>>>
>>>>>      set "WINKIT=c:\Program Files (x86)\Windows 
>>>>> Kits\10\Include\10.0.18362.0"
>>>>>      jextract --source -t duncan.win32.shobjidl_core -d 
>>>>> shlobj_core.h/java     "%WINKIT%/um/shlobj_core.h"
>>>>>
>>>>> Another issue is that this single windows header file gives rise 
>>>>> to 11MB jar and is unusable inside Eclipse (on my slow PC), and is 
>>>>> slow startup at runtime. Have you considered changing the 
>>>>> generated code to use lazy Supplier<MethodHandle> instead of 
>>>>> MethodHandle to ensure only required handles are created on-demand?
>>>>
>>>> Uhm... we are already using dynamic constants, so you should only 
>>>> pay for the MHs and VHs you use - e.g. in fact I'm even surprised 
>>>> you get the failure inside the <clinit> above - since that constant 
>>>> should not be initialized unless it's used. I wonder if there's a 
>>>> regression here... are you using the `--source` flag? With that 
>>>> flag we revert back to a monolithic header, so that will explain 
>>>> both failures. If so, try removing that flag - and see what happens 
>>>> (if everything is working as it should I don't think you should 
>>>> even see the error above, assuming you never use that layout).
>>>>
>>>> Maurizio
>>>>
>>>>>
>>>>> Kind regards
>>>>>
>>>>> Duncan
>>>>>
>>>>>
>>>
>