[External] : Re: Feedback / query on jextract for Windows 10

[Duncan Gittins]

I don't know what causes the speed difference, which may be down to the 
jar size as you suggest. I'll re-test after the time that "--source" 
generates same code as seen in the .class files when it may be easier to 
understand why there is a difference. I am using 16-panama+3-385.

This isn't a very robust test, but I also tried deleted all the 
un-necessary .class files from the 11MB jextract generated classes (not 
produced using --source) down to 2MB jar, then I moved my handcoded 
method into a separate class and added a dependency on a different 2MB 
jar. Neither of those changes affected the typical speeds of first call 
times - I still consistently see that the hand coded method handle 
version of my app is ~300-500ms faster on first call, and no difference 
for 2nd+ calls.

Duncan

On 25/01/2021 16:18, Maurizio Cimadamore wrote:
>
> On 25/01/2021 16:13, Duncan Gittins wrote:
>> Thanks for the information.  As you suspected that bug must be 
>> related and there is a minor difference in the --source output 
>> because the problem does go away when removing the jextract 
>> "--source" flag and making jar directly from generated classes rather 
>> than just compiling from the generated source code.
>>
>> My code samples using jextract bindings are ~300ms slower at startup 
>> time (but same speed runtime). I'm not certain if this overhead is 
>> down to initialisation of static constants or simply to be expected 
>> when adding 11MB jar footprint (compared to 56k jar for my 
>> handwritten MH bindings). I will follow-up if I find out more.
> If you see the startup slowdown even when omitting --source, then the 
> issue is with loading the jar, rather than initializing - as 
> explaining no initializing should happen outside for stuff that is 
> actually used by your code - so I'd expect that initialization 
> overhead for jextract code and your manually written versions should 
> be roughly the same.
>>
>> Looking at the generated code for "xxx$constants$nn.java" there seems 
>> to be a degree of unnecessary initialisation per xxx$constants$nn 
>> class as it contains static variables for FunctionDescriptor / 
>> MemoryLayout which would be initialised every time that particular 
>> class was loaded and not used in my code. I felt that my own jar was 
>> slow starting so switched foreign linker references to a lazy load 
>> Supplier for every library / symbol / mh / vh so that only the 
>> referenced types get created.
>
> Sure - but that's only when --source is used - when you generated 
> classes directly, everything is lazy (and in a more general way than 
> using Suppliers).
>
> So, can you please confirm, to be clear, that you still see startup 
> regression even when NOT using "--source" ?
>
> Thanks!
> Maurizio
>
>>
>> I will have a look at future builds - this project is extremely 
>> useful so I hope it gets into a full JDK release soon. Unfortunately 
>> my issue with a huge jar in Eclipse means I can't easily develop 
>> using the generated code variant right now, but in the meantime the 
>> jextract code is still very helpful to work out how to call Windows 
>> APIs correctly.
>>
>> Duncan
>>
>> On 25/01/2021 10:20, Maurizio Cimadamore wrote:
>>>
>>> On 23/01/2021 18:34, Duncan Gittins wrote:
>>>> I've been using Panama on Windows 10 to eliminate my C# apps which 
>>>> I would normally call via ProcessBuilder. I'm very impressed so far 
>>>> and looking forward to newer versions: everything I have written in 
>>>> Windows C# is now replaced with foreign memory / linker calls to 
>>>> native code using the latest Java 16 EA.  I have examples for 
>>>> various Win32 API and COM objects, HWND objects and screensaver. 
>>>> However I'm struggling to get the code running with output from 
>>>> jextract build. Perhaps I overlooked something very obvious in how 
>>>> to setup?
>>>>
>>>> One example app is attached - this calls SystemParametersInfoA to 
>>>> set the Windows desktop background. There are two implementations 
>>>> of setImage: one with hardwired MethodHandle works fine, but the 
>>>> jextract version gives UnsupportedOperationException for layout 
>>>> unrelated to SystemParametersInfoA. If I edit the generated code to 
>>>> comment out all lines unrelated to SystemParametersInfoA then both 
>>>> versions of setImage work.
>>>
>>> I think the culprit is this:
>>>
>>> ```
>>> Caused by: java.lang.UnsupportedOperationException: Invalid 
>>> alignment requirements for layout 
>>> b64(ArbitraryUserPointer)[abi/kind=POINTER,layout/name=ArbitraryUserPointer]
>>>         at 
>>> jdk.incubator.foreign/jdk.internal.foreign.LayoutPath.checkAlignment(LayoutPath.java:273)
>>>         at 
>>> jdk.incubator.foreign/jdk.internal.foreign.LayoutPath.dereferenceHandle(LayoutPath.java:159)
>>>         at 
>>> jdk.incubator.foreign/jdk.incubator.foreign.MemoryLayout.lambda$varHandle$2(MemoryLayout.java:488)
>>>         at 
>>> jdk.incubator.foreign/jdk.incubator.foreign.MemoryLayout.computePathOp(MemoryLayout.java:534)
>>>         at 
>>> jdk.incubator.foreign/jdk.incubator.foreign.MemoryLayout.varHandle(MemoryLayout.java:488)
>>>         at 
>>> duncan.win32.shobjidl_core.shlobj_core_h$constants$16.<clinit>(shlobj_core_h$constants$16.java:1844)
>>> ```
>>>
>>> This seems a duplicate of another issue that has been reported last 
>>> week:
>>>
>>> https://bugs.openjdk.java.net/browse/JDK-8259832
>>>
>>> E.g. jextract ignoring alignment requirements. Note that jextract 
>>> generates lots of constants, and it takes only one bad one to make 
>>> initialization fail.
>>>
>>>>
>>>> The jextract command on Windows 10 (with fresh Vis Studio 
>>>> installation) expands shlobj_core.h because I my app also uses 
>>>> IShellLink.
>>>>
>>>>      set "WINKIT=c:\Program Files (x86)\Windows 
>>>> Kits\10\Include\10.0.18362.0"
>>>>      jextract --source -t duncan.win32.shobjidl_core -d 
>>>> shlobj_core.h/java     "%WINKIT%/um/shlobj_core.h"
>>>>
>>>> Another issue is that this single windows header file gives rise to 
>>>> 11MB jar and is unusable inside Eclipse (on my slow PC), and is 
>>>> slow startup at runtime. Have you considered changing the generated 
>>>> code to use lazy Supplier<MethodHandle> instead of MethodHandle to 
>>>> ensure only required handles are created on-demand?
>>>
>>> Uhm... we are already using dynamic constants, so you should only 
>>> pay for the MHs and VHs you use - e.g. in fact I'm even surprised 
>>> you get the failure inside the <clinit> above - since that constant 
>>> should not be initialized unless it's used. I wonder if there's a 
>>> regression here... are you using the `--source` flag? With that flag 
>>> we revert back to a monolithic header, so that will explain both 
>>> failures. If so, try removing that flag - and see what happens (if 
>>> everything is working as it should I don't think you should even see 
>>> the error above, assuming you never use that layout).
>>>
>>> Maurizio
>>>
>>>>
>>>> Kind regards
>>>>
>>>> Duncan
>>>>
>>>>
>>