Panama unresolved error when instantiating wayland struct...
Mark Hammons
mark.hammons at inaf.cnrs-gif.fr
Fri Feb 15 13:25:37 UTC 2019
That would help a ton! I was searching like crazy in LayoutType and
Layout for this kind of functionality!
On 15/02/2019 14:18, Jorn Vernee wrote:
> Response inline...
>
> Mark Hammons schreef op 2019-02-15 13:34:
>> It’s not a problem. I started this project aware that foreign is still
>> in heavy development. I’m actually impressed it works as well as it
>> does, and I’ve made more progress using foreign to bind to wayland
>> than I did with JNR. Foreign is shaping up to be very good and I hope
>> that I can finish my program (implementing the wayland mcwayface
>> beginner’s window manager in scala) and have a semi-real world example
>> of using it to make a program.
>>
>> Two last questions while I have your attention:
>>
>> When I allocate a struct using the naive scope, can I control when
>> it’s freed? Do I just use the c free method on the pointer?
>>
>> Also, wayland loves the concept of doubly linked lists that use the
>> list nodes as pointers to elements within structs. In the wl_listener
>> struct I showed, there is a link member of wl_list. A pointer to that
>> member is what is inserted into the list with wl_list_insert. When it
>> comes time to traverse the list I take the list node pointer, declare
>> I think it’s part of wl_listener, find the offset from the link member
>> pointer of a wl_listener and a pointer to a wl_listener itself, and
>> apply that offset to the link. Below is the implementation of this in
>> scala:
>>
>> type HellType[T] = {
>> def ptr(): Pointer[T]
>> def link$ptr(): Pointer[wl_list]
>> }
>>
>> def extractFrom[T <: Struct[T] with HellType[T]](listItem: wl_list,
>> clazz: Class[T]) = {
>> val ev = s.allocateStruct(clazz)
>> val offset = ev.ptr.addr() - ev.link$ptr().addr()
>> listItem.ptr().cast(NativeTypes.VOID).cast(NativeTypes.INT8).offset(offset).cast(NativeTypes.VOID).cast(LayoutType.ofStruct(clazz))
>>
>> }
>>
>> Where HellType[T] is a structural type that is a supertype of anything
>> that has a Pointer<T> ptr() method and a Pointer<wl_list> link$ptr().
>> Is this the appropriate way to do this with foreign?
>
> Yes, this looks good to me.
>
> It looks like we need to add some sort of static offsetOf utility API
> so that you don't have to allocate a struct to find the offset of a
> field. Maybe something like (Java):
>
> public static <T extends Struct<T>> long offsetOf(Class<T>
> structClass, String fieldName) {
> ...
> }
>
> Would that help?
>
>> It seems to work
>> for me (and when I intentionally botch the offset, I get an error as
>> expected), but I've only used it with things I've allocated via the
>> scope and I've not tested using this with wl_list nodes that have been
>> allocated by C.
>
> That should behave exactly the same, unless the C code maintains a
> different layout (ABI) then is standard. jextract derives the layout
> from the C ABI of the platform it runs on.
>
> Cheers,
> Jorn
>
>> Thanks,
>>
>> Mark
>>
>> On 2/15/19 11:22 AM, Maurizio Cimadamore wrote:
>>>
>>> On 15/02/2019 01:33, Mark Hammons wrote:
>>>> I was wrong in my previous email. The issue is still fixed for me,
>>>> but the fix was not because of a change in jextract usage, but
>>>> rather the inclusion of this in my code:
>>>>
>>>> val lib = Libraries.bind(MethodHandles.lookup,
>>>> classOf[wayland.wayland_server_core])
>>>
>>> I was just about to suggest doing that (I came upon the same trick
>>> overnight) - but I wanted to try with your specific case first!
>>>
>>> I'm super happy that this trick works for you.
>>>
>>> Binding the library gives a couple of kicks to the resolution logic,
>>> so that it performs as it should.
>>>
>>> In other words, in the current state, if you want to allocate a
>>> struct defined in some library, it's always better to bind the whole
>>> library first (even if one doesn't plan to use it).
>>>
>>> I'll add this workaround to the JBS entry.
>>>
>>> Thanks again for the report - and, yes, we'll fix this in the next
>>> EA, as this is very frustrating/confusing (sorry!).
>>>
>>> Maurizio
>>>
>>>>
>>>> I never use lib, but if I remove that line the allocation of
>>>> wl_listener starts failing again.
>>>>
>>>> Mark
>>>>
>>>> On 2/15/19 2:14 AM, Maurizio Cimadamore wrote:
>>>>> Thanks Jorn - clever approach; I'll give this some though to make
>>>>> sure it covers all the bases.
>>>>>
>>>>> Maurizio
>>>>>
>>>>> On 15/02/2019 01:06, Jorn Vernee wrote:
>>>>>> FWIW, I've previously used the following fix to work around a
>>>>>> similar issue (also involving a linked lists).
>>>>>>
>>>>>> (Rough) Webrev:
>>>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8219042/webrev.00/
>>>>>>
>>>>>>
>>>>>> Cheers,
>>>>>> Jorn
>>>>>>
>>>>>> Maurizio Cimadamore schreef op 2019-02-15 01:14:
>>>>>>> Here's the bug reference I've created:
>>>>>>>
>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8219042
>>>>>>>
>>>>>>> unfortunately, I tried allocating the structs in different order
>>>>>>> and
>>>>>>> the problem cannot be resolved at the client side.
>>>>>>>
>>>>>>> Maurizio
>>>>>>>
>>>>>>> On 15/02/2019 00:06, Mark Hammons wrote:
>>>>>>>> I previously allocated a wl_list in my code. I'm still new to
>>>>>>>> the foreign interfaces, so I'm not aware if there's a way to
>>>>>>>> allocate the wl_listener using a pre-allocated wl_list.
>>>>>>>>
>>>>>>>> Mark
>>>>>>>>
>>>>>>>> On 2/15/19 12:49 AM, Maurizio Cimadamore wrote:
>>>>>>>>>
>>>>>>>>> On 14/02/2019 23:38, Mark Hammons wrote:
>>>>>>>>>> Hi Maurizio,
>>>>>>>>>>
>>>>>>>>>> No, wl_list is defined in wayland_utils.h while wl_listener
>>>>>>>>>> is in wayland_server_core.h. I am currently looking through
>>>>>>>>>> the issues on the openjdk tracker and seeing if there's a
>>>>>>>>>> mitigation for this.
>>>>>>>>>
>>>>>>>>> Right - you beat me to this:
>>>>>>>>>
>>>>>>>>> https://people.freedesktop.org/~whot/wayland-doxygen/wayland/Server/structwl__listener.html
>>>>>>>>> and
>>>>>>>>>
>>>>>>>>> https://people.freedesktop.org/~whot/wayland-doxygen/wayland/Server/structwl__list.html
>>>>>>>>> Unfortunately this issue is not easy to workaround. I'll make
>>>>>>>>> sure to create a JBS entry for it (we do have one, but it's
>>>>>>>>> probably not visible outside).
>>>>>>>>>
>>>>>>>>> I'll also try to play with this a bit to see what can be done
>>>>>>>>> - with this issue sometimes it helps to allocate the inner
>>>>>>>>> struct first (e.g. wl_list), and then the one that depends on
>>>>>>>>> it (e.g. wl_listener).
>>>>>>>>>
>>>>>>>>> Maurizio
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ~Mark
>>>>>>>>>>
>>>>>>>>>> On 2/15/19 12:30 AM, Maurizio Cimadamore wrote:
>>>>>>>>>>> Hi Mark,
>>>>>>>>>>> thanks for the report - from the looks of it, it seems an
>>>>>>>>>>> issue with cross-header layout resolution, which is listed
>>>>>>>>>>> in the 'known issues' in the EA page:
>>>>>>>>>>>
>>>>>>>>>>> "Dynamic layout resolution doesn't work across multiple
>>>>>>>>>>> headers."
>>>>>>>>>>>
>>>>>>>>>>> I will check in more details tomorrow, and confirm, one way
>>>>>>>>>>> or another.
>>>>>>>>>>>
>>>>>>>>>>> Quick check: are wl_list and wl_listener defined in the same
>>>>>>>>>>> header file? If not that's likely the issue here.
>>>>>>>>>>>
>>>>>>>>>>> I think Pointer<?> is the correct type - jextract tries to
>>>>>>>>>>> insert as more general types as possible when inserting
>>>>>>>>>>> Pointer in argument position; if it generated Pointer<Void>,
>>>>>>>>>>> and that was an ordinary function call, you could only call
>>>>>>>>>>> it with another Pointer<Void> - if the argument type is
>>>>>>>>>>> Pointer<?> you can pass _any_ pointer - e.g. Pointer<Byte>,
>>>>>>>>>>> Pointer<Integer> which is kind of close to what you can do
>>>>>>>>>>> in C.
>>>>>>>>>>>
>>>>>>>>>>> Maurizio
>>>>>>>>>>>
>>>>>>>>>>> On 14/02/2019 22:23, Mark Hammons wrote:
>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>
>>>>>>>>>>>> I decided to try to take the dive on project panama,
>>>>>>>>>>>> starting with making a binding to linux's wayland server. I
>>>>>>>>>>>> used the following command: ~/bin/jdk-13/bin/jextract
>>>>>>>>>>>> /usr/include/wayland/wayland-server-core.h
>>>>>>>>>>>> /usr/include/wayland/wayland-server.h
>>>>>>>>>>>> /usr/include/wayland/wayland-util.h
>>>>>>>>>>>> /usr/include/wayland/wayland-version.h
>>>>>>>>>>>> /usr/include/wayland/wayland-server-protocol.h -I
>>>>>>>>>>>> /usr/include/wayland -L /usr/lib64/ --record-library-path
>>>>>>>>>>>> -l wayland-server -t wayland -o wayland_server.jar
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> When I try to allocate a wl_listener struct, I get the
>>>>>>>>>>>> following error:
>>>>>>>>>>>>
>>>>>>>>>>>> [error] Exception in thread "main"
>>>>>>>>>>>> java.lang.UnsupportedOperationException: bitsSize on
>>>>>>>>>>>> Unresolved
>>>>>>>>>>>> [error] at
>>>>>>>>>>>> java.base/java.foreign.layout.Unresolved.bitsSize(Unresolved.java:76)
>>>>>>>>>>>> [error] at
>>>>>>>>>>>> java.base/java.util.stream.ReferencePipeline$5$1.accept(ReferencePipeline.java:229)
>>>>>>>>>>>> [error] at
>>>>>>>>>>>> java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
>>>>>>>>>>>> [error] at
>>>>>>>>>>>> java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
>>>>>>>>>>>> [error] at
>>>>>>>>>>>> java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
>>>>>>>>>>>> [error] at
>>>>>>>>>>>> java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
>>>>>>>>>>>> [error] at
>>>>>>>>>>>> java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
>>>>>>>>>>>> [error] at
>>>>>>>>>>>> java.base/java.util.stream.LongPipeline.reduce(LongPipeline.java:474)
>>>>>>>>>>>> [error] at
>>>>>>>>>>>> java.base/java.util.stream.LongPipeline.sum(LongPipeline.java:432)
>>>>>>>>>>>> [error] at
>>>>>>>>>>>> java.base/java.foreign.layout.Group.bitsSize(Group.java:119)
>>>>>>>>>>>> [error] at
>>>>>>>>>>>> java.base/java.foreign.memory.LayoutType.bytesSize(LayoutType.java:49)
>>>>>>>>>>>> [error] at
>>>>>>>>>>>> java.base/jdk.internal.foreign.ScopeImpl.allocateInternal(ScopeImpl.java:66)
>>>>>>>>>>>> [error] at
>>>>>>>>>>>> java.base/jdk.internal.foreign.ScopeImpl.allocate(ScopeImpl.java:92)
>>>>>>>>>>>> [error] at
>>>>>>>>>>>> java.base/jdk.internal.foreign.ScopeImpl.allocateStruct(ScopeImpl.java:98)
>>>>>>>>>>>> [error] at
>>>>>>>>>>>> TestApp$.delayedEndpoint$TestApp$1(TestApp.scala:22)
>>>>>>>>>>>> [error] at
>>>>>>>>>>>> TestApp$delayedInit$body.apply(TestApp.scala:13)
>>>>>>>>>>>> [error] at
>>>>>>>>>>>> scala.Function0.apply$mcV$sp(Function0.scala:39)
>>>>>>>>>>>> [error] at
>>>>>>>>>>>> scala.Function0.apply$mcV$sp$(Function0.scala:39)
>>>>>>>>>>>> [error] at
>>>>>>>>>>>> scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
>>>>>>>>>>>> [error] at scala.App.$anonfun$main$1$adapted(App.scala:80)
>>>>>>>>>>>> [error] at
>>>>>>>>>>>> scala.collection.immutable.List.foreach(List.scala:392)
>>>>>>>>>>>> [error] at scala.App.main(App.scala:80)
>>>>>>>>>>>> [error] at scala.App.main$(App.scala:78)
>>>>>>>>>>>> [error] at TestApp$.main(TestApp.scala:13)
>>>>>>>>>>>> [error] at TestApp.main(TestApp.scala)
>>>>>>>>>>>>
>>>>>>>>>>>> Looking at other bugs involving this kind of error message,
>>>>>>>>>>>> it appears that unresolved is a type for when there's not
>>>>>>>>>>>> enough layout information? In any case, here's the struct
>>>>>>>>>>>> in question:
>>>>>>>>>>>>
>>>>>>>>>>>> struct wl_listener {
>>>>>>>>>>>> struct wl_list link;
>>>>>>>>>>>> wl_notify_func_t notify;
>>>>>>>>>>>> };
>>>>>>>>>>>>
>>>>>>>>>>>> and the definition of the elements:
>>>>>>>>>>>>
>>>>>>>>>>>> typedef void (*wl_notify_func_t)(struct wl_listener
>>>>>>>>>>>> *listener, void *data);
>>>>>>>>>>>>
>>>>>>>>>>>> struct wl_list {
>>>>>>>>>>>> /** Previous list element */
>>>>>>>>>>>> struct wl_list *prev;
>>>>>>>>>>>> /** Next list element */
>>>>>>>>>>>> struct wl_list *next;
>>>>>>>>>>>> };
>>>>>>>>>>>>
>>>>>>>>>>>> I'm fairly certain the issue lies with the function pointer
>>>>>>>>>>>> notify. When I looked at the decompiled source,
>>>>>>>>>>>> wl_notify_func_t is defined as:
>>>>>>>>>>>>
>>>>>>>>>>>> @FunctionalInterface
>>>>>>>>>>>> @NativeCallback("(u64:${wl_listener}u64:v)v")
>>>>>>>>>>>> public interface FI5 {
>>>>>>>>>>>> void fn(Pointer<wayland_server_core.wl_listener>
>>>>>>>>>>>> var1, Pointer<?> var2);
>>>>>>>>>>>> }
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> which seems suspicious to me. var2 should be a
>>>>>>>>>>>> Pointer<Void> I would think. It's a type I see elsewhere in
>>>>>>>>>>>> the source for this file, so it seems suspect that var2 is
>>>>>>>>>>>> a Pointer<?>.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Is this a bug? Am I just using jextract wrong?
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks for your help,
>>>>>>>>>>>>
>>>>>>>>>>>> Mark Hammons
>>>>>>>>>>>>
More information about the panama-dev
mailing list