Panama unresolved error when instantiating wayland struct...
Jorn Vernee
jbvernee at xs4all.nl
Fri Feb 15 13:35:19 UTC 2019
FWIW, here is a quick prototype that you could use as a utility method
until then:
public static long offsetOf(Class<? extends Struct<?>> cls, String
fieldName) {
Group g = (Group)
Layout.of(cls.getAnnotation(NativeStruct.class).value());
long offset = 0;
for(Layout l : g.elements()) {
Optional<String> name = l.name();
if(name.isPresent() && name.get().equals(fieldName)) {
return offset / 8; // convert to bytes
}
offset += l.bitsSize();
}
throw new IllegalArgumentException("Not a valid field name: " +
fieldName);
}
Cheers,
Jorn
Mark Hammons schreef op 2019-02-15 14:25:
> That would help a ton! I was searching like crazy in LayoutType and
> Layout for this kind of functionality!
>
> On 15/02/2019 14:18, Jorn Vernee wrote:
>> Response inline...
>>
>> Mark Hammons schreef op 2019-02-15 13:34:
>>> It’s not a problem. I started this project aware that foreign is
>>> still
>>> in heavy development. I’m actually impressed it works as well as it
>>> does, and I’ve made more progress using foreign to bind to wayland
>>> than I did with JNR. Foreign is shaping up to be very good and I hope
>>> that I can finish my program (implementing the wayland mcwayface
>>> beginner’s window manager in scala) and have a semi-real world
>>> example
>>> of using it to make a program.
>>>
>>> Two last questions while I have your attention:
>>>
>>> When I allocate a struct using the naive scope, can I control when
>>> it’s freed? Do I just use the c free method on the pointer?
>>>
>>> Also, wayland loves the concept of doubly linked lists that use the
>>> list nodes as pointers to elements within structs. In the wl_listener
>>> struct I showed, there is a link member of wl_list. A pointer to that
>>> member is what is inserted into the list with wl_list_insert. When it
>>> comes time to traverse the list I take the list node pointer, declare
>>> I think it’s part of wl_listener, find the offset from the link
>>> member
>>> pointer of a wl_listener and a pointer to a wl_listener itself, and
>>> apply that offset to the link. Below is the implementation of this in
>>> scala:
>>>
>>> type HellType[T] = {
>>> def ptr(): Pointer[T]
>>> def link$ptr(): Pointer[wl_list]
>>> }
>>>
>>> def extractFrom[T <: Struct[T] with HellType[T]](listItem: wl_list,
>>> clazz: Class[T]) = {
>>> val ev = s.allocateStruct(clazz)
>>> val offset = ev.ptr.addr() - ev.link$ptr().addr()
>>> listItem.ptr().cast(NativeTypes.VOID).cast(NativeTypes.INT8).offset(offset).cast(NativeTypes.VOID).cast(LayoutType.ofStruct(clazz))
>>> }
>>>
>>> Where HellType[T] is a structural type that is a supertype of
>>> anything
>>> that has a Pointer<T> ptr() method and a Pointer<wl_list> link$ptr().
>>> Is this the appropriate way to do this with foreign?
>>
>> Yes, this looks good to me.
>>
>> It looks like we need to add some sort of static offsetOf utility API
>> so that you don't have to allocate a struct to find the offset of a
>> field. Maybe something like (Java):
>>
>> public static <T extends Struct<T>> long offsetOf(Class<T>
>> structClass, String fieldName) {
>> ...
>> }
>>
>> Would that help?
>>
>>> It seems to work
>>> for me (and when I intentionally botch the offset, I get an error as
>>> expected), but I've only used it with things I've allocated via the
>>> scope and I've not tested using this with wl_list nodes that have
>>> been
>>> allocated by C.
>>
>> That should behave exactly the same, unless the C code maintains a
>> different layout (ABI) then is standard. jextract derives the layout
>> from the C ABI of the platform it runs on.
>>
>> Cheers,
>> Jorn
>>
>>> Thanks,
>>>
>>> Mark
>>>
>>> On 2/15/19 11:22 AM, Maurizio Cimadamore wrote:
>>>>
>>>> On 15/02/2019 01:33, Mark Hammons wrote:
>>>>> I was wrong in my previous email. The issue is still fixed for me,
>>>>> but the fix was not because of a change in jextract usage, but
>>>>> rather the inclusion of this in my code:
>>>>>
>>>>> val lib = Libraries.bind(MethodHandles.lookup,
>>>>> classOf[wayland.wayland_server_core])
>>>>
>>>> I was just about to suggest doing that (I came upon the same trick
>>>> overnight) - but I wanted to try with your specific case first!
>>>>
>>>> I'm super happy that this trick works for you.
>>>>
>>>> Binding the library gives a couple of kicks to the resolution logic,
>>>> so that it performs as it should.
>>>>
>>>> In other words, in the current state, if you want to allocate a
>>>> struct defined in some library, it's always better to bind the whole
>>>> library first (even if one doesn't plan to use it).
>>>>
>>>> I'll add this workaround to the JBS entry.
>>>>
>>>> Thanks again for the report - and, yes, we'll fix this in the next
>>>> EA, as this is very frustrating/confusing (sorry!).
>>>>
>>>> Maurizio
>>>>
>>>>>
>>>>> I never use lib, but if I remove that line the allocation of
>>>>> wl_listener starts failing again.
>>>>>
>>>>> Mark
>>>>>
>>>>> On 2/15/19 2:14 AM, Maurizio Cimadamore wrote:
>>>>>> Thanks Jorn - clever approach; I'll give this some though to make
>>>>>> sure it covers all the bases.
>>>>>>
>>>>>> Maurizio
>>>>>>
>>>>>> On 15/02/2019 01:06, Jorn Vernee wrote:
>>>>>>> FWIW, I've previously used the following fix to work around a
>>>>>>> similar issue (also involving a linked lists).
>>>>>>>
>>>>>>> (Rough) Webrev:
>>>>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8219042/webrev.00/
>>>>>>> Cheers,
>>>>>>> Jorn
>>>>>>>
>>>>>>> Maurizio Cimadamore schreef op 2019-02-15 01:14:
>>>>>>>> Here's the bug reference I've created:
>>>>>>>>
>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8219042
>>>>>>>>
>>>>>>>> unfortunately, I tried allocating the structs in different order
>>>>>>>> and
>>>>>>>> the problem cannot be resolved at the client side.
>>>>>>>>
>>>>>>>> Maurizio
>>>>>>>>
>>>>>>>> On 15/02/2019 00:06, Mark Hammons wrote:
>>>>>>>>> I previously allocated a wl_list in my code. I'm still new to
>>>>>>>>> the foreign interfaces, so I'm not aware if there's a way to
>>>>>>>>> allocate the wl_listener using a pre-allocated wl_list.
>>>>>>>>>
>>>>>>>>> Mark
>>>>>>>>>
>>>>>>>>> On 2/15/19 12:49 AM, Maurizio Cimadamore wrote:
>>>>>>>>>>
>>>>>>>>>> On 14/02/2019 23:38, Mark Hammons wrote:
>>>>>>>>>>> Hi Maurizio,
>>>>>>>>>>>
>>>>>>>>>>> No, wl_list is defined in wayland_utils.h while wl_listener
>>>>>>>>>>> is in wayland_server_core.h. I am currently looking through
>>>>>>>>>>> the issues on the openjdk tracker and seeing if there's a
>>>>>>>>>>> mitigation for this.
>>>>>>>>>>
>>>>>>>>>> Right - you beat me to this:
>>>>>>>>>>
>>>>>>>>>> https://people.freedesktop.org/~whot/wayland-doxygen/wayland/Server/structwl__listener.html
>>>>>>>>>> and
>>>>>>>>>>
>>>>>>>>>> https://people.freedesktop.org/~whot/wayland-doxygen/wayland/Server/structwl__list.html
>>>>>>>>>> Unfortunately this issue is not easy to workaround. I'll make
>>>>>>>>>> sure to create a JBS entry for it (we do have one, but it's
>>>>>>>>>> probably not visible outside).
>>>>>>>>>>
>>>>>>>>>> I'll also try to play with this a bit to see what can be done
>>>>>>>>>> - with this issue sometimes it helps to allocate the inner
>>>>>>>>>> struct first (e.g. wl_list), and then the one that depends on
>>>>>>>>>> it (e.g. wl_listener).
>>>>>>>>>>
>>>>>>>>>> Maurizio
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> ~Mark
>>>>>>>>>>>
>>>>>>>>>>> On 2/15/19 12:30 AM, Maurizio Cimadamore wrote:
>>>>>>>>>>>> Hi Mark,
>>>>>>>>>>>> thanks for the report - from the looks of it, it seems an
>>>>>>>>>>>> issue with cross-header layout resolution, which is listed
>>>>>>>>>>>> in the 'known issues' in the EA page:
>>>>>>>>>>>>
>>>>>>>>>>>> "Dynamic layout resolution doesn't work across multiple
>>>>>>>>>>>> headers."
>>>>>>>>>>>>
>>>>>>>>>>>> I will check in more details tomorrow, and confirm, one way
>>>>>>>>>>>> or another.
>>>>>>>>>>>>
>>>>>>>>>>>> Quick check: are wl_list and wl_listener defined in the same
>>>>>>>>>>>> header file? If not that's likely the issue here.
>>>>>>>>>>>>
>>>>>>>>>>>> I think Pointer<?> is the correct type - jextract tries to
>>>>>>>>>>>> insert as more general types as possible when inserting
>>>>>>>>>>>> Pointer in argument position; if it generated Pointer<Void>,
>>>>>>>>>>>> and that was an ordinary function call, you could only call
>>>>>>>>>>>> it with another Pointer<Void> - if the argument type is
>>>>>>>>>>>> Pointer<?> you can pass _any_ pointer - e.g. Pointer<Byte>,
>>>>>>>>>>>> Pointer<Integer> which is kind of close to what you can do
>>>>>>>>>>>> in C.
>>>>>>>>>>>>
>>>>>>>>>>>> Maurizio
>>>>>>>>>>>>
>>>>>>>>>>>> On 14/02/2019 22:23, Mark Hammons wrote:
>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I decided to try to take the dive on project panama,
>>>>>>>>>>>>> starting with making a binding to linux's wayland server. I
>>>>>>>>>>>>> used the following command: ~/bin/jdk-13/bin/jextract
>>>>>>>>>>>>> /usr/include/wayland/wayland-server-core.h
>>>>>>>>>>>>> /usr/include/wayland/wayland-server.h
>>>>>>>>>>>>> /usr/include/wayland/wayland-util.h
>>>>>>>>>>>>> /usr/include/wayland/wayland-version.h
>>>>>>>>>>>>> /usr/include/wayland/wayland-server-protocol.h -I
>>>>>>>>>>>>> /usr/include/wayland -L /usr/lib64/ --record-library-path
>>>>>>>>>>>>> -l wayland-server -t wayland -o wayland_server.jar
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> When I try to allocate a wl_listener struct, I get the
>>>>>>>>>>>>> following error:
>>>>>>>>>>>>>
>>>>>>>>>>>>> [error] Exception in thread "main"
>>>>>>>>>>>>> java.lang.UnsupportedOperationException: bitsSize on
>>>>>>>>>>>>> Unresolved
>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>> java.base/java.foreign.layout.Unresolved.bitsSize(Unresolved.java:76)
>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>> java.base/java.util.stream.ReferencePipeline$5$1.accept(ReferencePipeline.java:229)
>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>> java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>> java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>> java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>> java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>> java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>> java.base/java.util.stream.LongPipeline.reduce(LongPipeline.java:474)
>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>> java.base/java.util.stream.LongPipeline.sum(LongPipeline.java:432)
>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>> java.base/java.foreign.layout.Group.bitsSize(Group.java:119)
>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>> java.base/java.foreign.memory.LayoutType.bytesSize(LayoutType.java:49)
>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>> java.base/jdk.internal.foreign.ScopeImpl.allocateInternal(ScopeImpl.java:66)
>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>> java.base/jdk.internal.foreign.ScopeImpl.allocate(ScopeImpl.java:92)
>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>> java.base/jdk.internal.foreign.ScopeImpl.allocateStruct(ScopeImpl.java:98)
>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>> TestApp$.delayedEndpoint$TestApp$1(TestApp.scala:22)
>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>> TestApp$delayedInit$body.apply(TestApp.scala:13)
>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>> scala.Function0.apply$mcV$sp(Function0.scala:39)
>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>> scala.Function0.apply$mcV$sp$(Function0.scala:39)
>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>> scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>> scala.App.$anonfun$main$1$adapted(App.scala:80)
>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>> scala.collection.immutable.List.foreach(List.scala:392)
>>>>>>>>>>>>> [error] at scala.App.main(App.scala:80)
>>>>>>>>>>>>> [error] at scala.App.main$(App.scala:78)
>>>>>>>>>>>>> [error] at TestApp$.main(TestApp.scala:13)
>>>>>>>>>>>>> [error] at TestApp.main(TestApp.scala)
>>>>>>>>>>>>>
>>>>>>>>>>>>> Looking at other bugs involving this kind of error message,
>>>>>>>>>>>>> it appears that unresolved is a type for when there's not
>>>>>>>>>>>>> enough layout information? In any case, here's the struct
>>>>>>>>>>>>> in question:
>>>>>>>>>>>>>
>>>>>>>>>>>>> struct wl_listener {
>>>>>>>>>>>>> struct wl_list link;
>>>>>>>>>>>>> wl_notify_func_t notify;
>>>>>>>>>>>>> };
>>>>>>>>>>>>>
>>>>>>>>>>>>> and the definition of the elements:
>>>>>>>>>>>>>
>>>>>>>>>>>>> typedef void (*wl_notify_func_t)(struct wl_listener
>>>>>>>>>>>>> *listener, void *data);
>>>>>>>>>>>>>
>>>>>>>>>>>>> struct wl_list {
>>>>>>>>>>>>> /** Previous list element */
>>>>>>>>>>>>> struct wl_list *prev;
>>>>>>>>>>>>> /** Next list element */
>>>>>>>>>>>>> struct wl_list *next;
>>>>>>>>>>>>> };
>>>>>>>>>>>>>
>>>>>>>>>>>>> I'm fairly certain the issue lies with the function pointer
>>>>>>>>>>>>> notify. When I looked at the decompiled source,
>>>>>>>>>>>>> wl_notify_func_t is defined as:
>>>>>>>>>>>>>
>>>>>>>>>>>>> @FunctionalInterface
>>>>>>>>>>>>> @NativeCallback("(u64:${wl_listener}u64:v)v")
>>>>>>>>>>>>> public interface FI5 {
>>>>>>>>>>>>> void fn(Pointer<wayland_server_core.wl_listener>
>>>>>>>>>>>>> var1, Pointer<?> var2);
>>>>>>>>>>>>> }
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> which seems suspicious to me. var2 should be a
>>>>>>>>>>>>> Pointer<Void> I would think. It's a type I see elsewhere in
>>>>>>>>>>>>> the source for this file, so it seems suspect that var2 is
>>>>>>>>>>>>> a Pointer<?>.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Is this a bug? Am I just using jextract wrong?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks for your help,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Mark Hammons
>>>>>>>>>>>>>
More information about the panama-dev
mailing list