Panama unresolved error when instantiating wayland struct...
Jorn Vernee
jbvernee at xs4all.nl
Fri Feb 15 13:18:43 UTC 2019
Response inline...
Mark Hammons schreef op 2019-02-15 13:34:
> It’s not a problem. I started this project aware that foreign is still
> in heavy development. I’m actually impressed it works as well as it
> does, and I’ve made more progress using foreign to bind to wayland
> than I did with JNR. Foreign is shaping up to be very good and I hope
> that I can finish my program (implementing the wayland mcwayface
> beginner’s window manager in scala) and have a semi-real world example
> of using it to make a program.
>
> Two last questions while I have your attention:
>
> When I allocate a struct using the naive scope, can I control when
> it’s freed? Do I just use the c free method on the pointer?
>
> Also, wayland loves the concept of doubly linked lists that use the
> list nodes as pointers to elements within structs. In the wl_listener
> struct I showed, there is a link member of wl_list. A pointer to that
> member is what is inserted into the list with wl_list_insert. When it
> comes time to traverse the list I take the list node pointer, declare
> I think it’s part of wl_listener, find the offset from the link member
> pointer of a wl_listener and a pointer to a wl_listener itself, and
> apply that offset to the link. Below is the implementation of this in
> scala:
>
> type HellType[T] = {
> def ptr(): Pointer[T]
> def link$ptr(): Pointer[wl_list]
> }
>
> def extractFrom[T <: Struct[T] with HellType[T]](listItem: wl_list,
> clazz: Class[T]) = {
> val ev = s.allocateStruct(clazz)
> val offset = ev.ptr.addr() - ev.link$ptr().addr()
> listItem.ptr().cast(NativeTypes.VOID).cast(NativeTypes.INT8).offset(offset).cast(NativeTypes.VOID).cast(LayoutType.ofStruct(clazz))
> }
>
> Where HellType[T] is a structural type that is a supertype of anything
> that has a Pointer<T> ptr() method and a Pointer<wl_list> link$ptr().
> Is this the appropriate way to do this with foreign?
Yes, this looks good to me.
It looks like we need to add some sort of static offsetOf utility API so
that you don't have to allocate a struct to find the offset of a field.
Maybe something like (Java):
public static <T extends Struct<T>> long offsetOf(Class<T>
structClass, String fieldName) {
...
}
Would that help?
> It seems to work
> for me (and when I intentionally botch the offset, I get an error as
> expected), but I've only used it with things I've allocated via the
> scope and I've not tested using this with wl_list nodes that have been
> allocated by C.
That should behave exactly the same, unless the C code maintains a
different layout (ABI) then is standard. jextract derives the layout
from the C ABI of the platform it runs on.
Cheers,
Jorn
> Thanks,
>
> Mark
>
> On 2/15/19 11:22 AM, Maurizio Cimadamore wrote:
>>
>> On 15/02/2019 01:33, Mark Hammons wrote:
>>> I was wrong in my previous email. The issue is still fixed for me,
>>> but the fix was not because of a change in jextract usage, but rather
>>> the inclusion of this in my code:
>>>
>>> val lib = Libraries.bind(MethodHandles.lookup,
>>> classOf[wayland.wayland_server_core])
>>
>> I was just about to suggest doing that (I came upon the same trick
>> overnight) - but I wanted to try with your specific case first!
>>
>> I'm super happy that this trick works for you.
>>
>> Binding the library gives a couple of kicks to the resolution logic,
>> so that it performs as it should.
>>
>> In other words, in the current state, if you want to allocate a struct
>> defined in some library, it's always better to bind the whole library
>> first (even if one doesn't plan to use it).
>>
>> I'll add this workaround to the JBS entry.
>>
>> Thanks again for the report - and, yes, we'll fix this in the next EA,
>> as this is very frustrating/confusing (sorry!).
>>
>> Maurizio
>>
>>>
>>> I never use lib, but if I remove that line the allocation of
>>> wl_listener starts failing again.
>>>
>>> Mark
>>>
>>> On 2/15/19 2:14 AM, Maurizio Cimadamore wrote:
>>>> Thanks Jorn - clever approach; I'll give this some though to make
>>>> sure it covers all the bases.
>>>>
>>>> Maurizio
>>>>
>>>> On 15/02/2019 01:06, Jorn Vernee wrote:
>>>>> FWIW, I've previously used the following fix to work around a
>>>>> similar issue (also involving a linked lists).
>>>>>
>>>>> (Rough) Webrev:
>>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8219042/webrev.00/
>>>>>
>>>>> Cheers,
>>>>> Jorn
>>>>>
>>>>> Maurizio Cimadamore schreef op 2019-02-15 01:14:
>>>>>> Here's the bug reference I've created:
>>>>>>
>>>>>> https://bugs.openjdk.java.net/browse/JDK-8219042
>>>>>>
>>>>>> unfortunately, I tried allocating the structs in different order
>>>>>> and
>>>>>> the problem cannot be resolved at the client side.
>>>>>>
>>>>>> Maurizio
>>>>>>
>>>>>> On 15/02/2019 00:06, Mark Hammons wrote:
>>>>>>> I previously allocated a wl_list in my code. I'm still new to the
>>>>>>> foreign interfaces, so I'm not aware if there's a way to allocate
>>>>>>> the wl_listener using a pre-allocated wl_list.
>>>>>>>
>>>>>>> Mark
>>>>>>>
>>>>>>> On 2/15/19 12:49 AM, Maurizio Cimadamore wrote:
>>>>>>>>
>>>>>>>> On 14/02/2019 23:38, Mark Hammons wrote:
>>>>>>>>> Hi Maurizio,
>>>>>>>>>
>>>>>>>>> No, wl_list is defined in wayland_utils.h while wl_listener is
>>>>>>>>> in wayland_server_core.h. I am currently looking through the
>>>>>>>>> issues on the openjdk tracker and seeing if there's a
>>>>>>>>> mitigation for this.
>>>>>>>>
>>>>>>>> Right - you beat me to this:
>>>>>>>>
>>>>>>>> https://people.freedesktop.org/~whot/wayland-doxygen/wayland/Server/structwl__listener.html
>>>>>>>> and
>>>>>>>>
>>>>>>>> https://people.freedesktop.org/~whot/wayland-doxygen/wayland/Server/structwl__list.html
>>>>>>>> Unfortunately this issue is not easy to workaround. I'll make
>>>>>>>> sure to create a JBS entry for it (we do have one, but it's
>>>>>>>> probably not visible outside).
>>>>>>>>
>>>>>>>> I'll also try to play with this a bit to see what can be done -
>>>>>>>> with this issue sometimes it helps to allocate the inner struct
>>>>>>>> first (e.g. wl_list), and then the one that depends on it (e.g.
>>>>>>>> wl_listener).
>>>>>>>>
>>>>>>>> Maurizio
>>>>>>>>
>>>>>>>>>
>>>>>>>>> ~Mark
>>>>>>>>>
>>>>>>>>> On 2/15/19 12:30 AM, Maurizio Cimadamore wrote:
>>>>>>>>>> Hi Mark,
>>>>>>>>>> thanks for the report - from the looks of it, it seems an
>>>>>>>>>> issue with cross-header layout resolution, which is listed in
>>>>>>>>>> the 'known issues' in the EA page:
>>>>>>>>>>
>>>>>>>>>> "Dynamic layout resolution doesn't work across multiple
>>>>>>>>>> headers."
>>>>>>>>>>
>>>>>>>>>> I will check in more details tomorrow, and confirm, one way or
>>>>>>>>>> another.
>>>>>>>>>>
>>>>>>>>>> Quick check: are wl_list and wl_listener defined in the same
>>>>>>>>>> header file? If not that's likely the issue here.
>>>>>>>>>>
>>>>>>>>>> I think Pointer<?> is the correct type - jextract tries to
>>>>>>>>>> insert as more general types as possible when inserting
>>>>>>>>>> Pointer in argument position; if it generated Pointer<Void>,
>>>>>>>>>> and that was an ordinary function call, you could only call it
>>>>>>>>>> with another Pointer<Void> - if the argument type is
>>>>>>>>>> Pointer<?> you can pass _any_ pointer - e.g. Pointer<Byte>,
>>>>>>>>>> Pointer<Integer> which is kind of close to what you can do in
>>>>>>>>>> C.
>>>>>>>>>>
>>>>>>>>>> Maurizio
>>>>>>>>>>
>>>>>>>>>> On 14/02/2019 22:23, Mark Hammons wrote:
>>>>>>>>>>> Hi all,
>>>>>>>>>>>
>>>>>>>>>>> I decided to try to take the dive on project panama, starting
>>>>>>>>>>> with making a binding to linux's wayland server. I used the
>>>>>>>>>>> following command: ~/bin/jdk-13/bin/jextract
>>>>>>>>>>> /usr/include/wayland/wayland-server-core.h
>>>>>>>>>>> /usr/include/wayland/wayland-server.h
>>>>>>>>>>> /usr/include/wayland/wayland-util.h
>>>>>>>>>>> /usr/include/wayland/wayland-version.h
>>>>>>>>>>> /usr/include/wayland/wayland-server-protocol.h -I
>>>>>>>>>>> /usr/include/wayland -L /usr/lib64/ --record-library-path -l
>>>>>>>>>>> wayland-server -t wayland -o wayland_server.jar
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> When I try to allocate a wl_listener struct, I get the
>>>>>>>>>>> following error:
>>>>>>>>>>>
>>>>>>>>>>> [error] Exception in thread "main"
>>>>>>>>>>> java.lang.UnsupportedOperationException: bitsSize on
>>>>>>>>>>> Unresolved
>>>>>>>>>>> [error] at
>>>>>>>>>>> java.base/java.foreign.layout.Unresolved.bitsSize(Unresolved.java:76)
>>>>>>>>>>> [error] at
>>>>>>>>>>> java.base/java.util.stream.ReferencePipeline$5$1.accept(ReferencePipeline.java:229)
>>>>>>>>>>> [error] at
>>>>>>>>>>> java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
>>>>>>>>>>> [error] at
>>>>>>>>>>> java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
>>>>>>>>>>> [error] at
>>>>>>>>>>> java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
>>>>>>>>>>> [error] at
>>>>>>>>>>> java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
>>>>>>>>>>> [error] at
>>>>>>>>>>> java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
>>>>>>>>>>> [error] at
>>>>>>>>>>> java.base/java.util.stream.LongPipeline.reduce(LongPipeline.java:474)
>>>>>>>>>>> [error] at
>>>>>>>>>>> java.base/java.util.stream.LongPipeline.sum(LongPipeline.java:432)
>>>>>>>>>>> [error] at
>>>>>>>>>>> java.base/java.foreign.layout.Group.bitsSize(Group.java:119)
>>>>>>>>>>> [error] at
>>>>>>>>>>> java.base/java.foreign.memory.LayoutType.bytesSize(LayoutType.java:49)
>>>>>>>>>>> [error] at
>>>>>>>>>>> java.base/jdk.internal.foreign.ScopeImpl.allocateInternal(ScopeImpl.java:66)
>>>>>>>>>>> [error] at
>>>>>>>>>>> java.base/jdk.internal.foreign.ScopeImpl.allocate(ScopeImpl.java:92)
>>>>>>>>>>> [error] at
>>>>>>>>>>> java.base/jdk.internal.foreign.ScopeImpl.allocateStruct(ScopeImpl.java:98)
>>>>>>>>>>> [error] at
>>>>>>>>>>> TestApp$.delayedEndpoint$TestApp$1(TestApp.scala:22)
>>>>>>>>>>> [error] at
>>>>>>>>>>> TestApp$delayedInit$body.apply(TestApp.scala:13)
>>>>>>>>>>> [error] at
>>>>>>>>>>> scala.Function0.apply$mcV$sp(Function0.scala:39)
>>>>>>>>>>> [error] at
>>>>>>>>>>> scala.Function0.apply$mcV$sp$(Function0.scala:39)
>>>>>>>>>>> [error] at
>>>>>>>>>>> scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
>>>>>>>>>>> [error] at
>>>>>>>>>>> scala.App.$anonfun$main$1$adapted(App.scala:80)
>>>>>>>>>>> [error] at
>>>>>>>>>>> scala.collection.immutable.List.foreach(List.scala:392)
>>>>>>>>>>> [error] at scala.App.main(App.scala:80)
>>>>>>>>>>> [error] at scala.App.main$(App.scala:78)
>>>>>>>>>>> [error] at TestApp$.main(TestApp.scala:13)
>>>>>>>>>>> [error] at TestApp.main(TestApp.scala)
>>>>>>>>>>>
>>>>>>>>>>> Looking at other bugs involving this kind of error message,
>>>>>>>>>>> it appears that unresolved is a type for when there's not
>>>>>>>>>>> enough layout information? In any case, here's the struct in
>>>>>>>>>>> question:
>>>>>>>>>>>
>>>>>>>>>>> struct wl_listener {
>>>>>>>>>>> struct wl_list link;
>>>>>>>>>>> wl_notify_func_t notify;
>>>>>>>>>>> };
>>>>>>>>>>>
>>>>>>>>>>> and the definition of the elements:
>>>>>>>>>>>
>>>>>>>>>>> typedef void (*wl_notify_func_t)(struct wl_listener
>>>>>>>>>>> *listener, void *data);
>>>>>>>>>>>
>>>>>>>>>>> struct wl_list {
>>>>>>>>>>> /** Previous list element */
>>>>>>>>>>> struct wl_list *prev;
>>>>>>>>>>> /** Next list element */
>>>>>>>>>>> struct wl_list *next;
>>>>>>>>>>> };
>>>>>>>>>>>
>>>>>>>>>>> I'm fairly certain the issue lies with the function pointer
>>>>>>>>>>> notify. When I looked at the decompiled source,
>>>>>>>>>>> wl_notify_func_t is defined as:
>>>>>>>>>>>
>>>>>>>>>>> @FunctionalInterface
>>>>>>>>>>> @NativeCallback("(u64:${wl_listener}u64:v)v")
>>>>>>>>>>> public interface FI5 {
>>>>>>>>>>> void fn(Pointer<wayland_server_core.wl_listener>
>>>>>>>>>>> var1, Pointer<?> var2);
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> which seems suspicious to me. var2 should be a Pointer<Void>
>>>>>>>>>>> I would think. It's a type I see elsewhere in the source for
>>>>>>>>>>> this file, so it seems suspect that var2 is a Pointer<?>.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Is this a bug? Am I just using jextract wrong?
>>>>>>>>>>>
>>>>>>>>>>> Thanks for your help,
>>>>>>>>>>>
>>>>>>>>>>> Mark Hammons
>>>>>>>>>>>
More information about the panama-dev
mailing list