Panama unresolved error when instantiating wayland struct...
Maurizio Cimadamore
maurizio.cimadamore at oracle.com
Fri Feb 15 13:53:08 UTC 2019
This looks good - I would stick with LayoutType.ofStruct().layout()
since that will apply all necessary resolution - otherwise the method
will end up calling bitsSize on unresolved
Maurizio
On 15/02/2019 13:35, Jorn Vernee wrote:
> FWIW, here is a quick prototype that you could use as a utility method
> until then:
>
> public static long offsetOf(Class<? extends Struct<?>> cls, String
> fieldName) {
> Group g = (Group)
> Layout.of(cls.getAnnotation(NativeStruct.class).value());
>
> long offset = 0;
> for(Layout l : g.elements()) {
> Optional<String> name = l.name();
> if(name.isPresent() && name.get().equals(fieldName)) {
> return offset / 8; // convert to bytes
> }
> offset += l.bitsSize();
> }
>
> throw new IllegalArgumentException("Not a valid field name: "
> + fieldName);
> }
>
> Cheers,
> Jorn
>
> Mark Hammons schreef op 2019-02-15 14:25:
>> That would help a ton! I was searching like crazy in LayoutType and
>> Layout for this kind of functionality!
>>
>> On 15/02/2019 14:18, Jorn Vernee wrote:
>>> Response inline...
>>>
>>> Mark Hammons schreef op 2019-02-15 13:34:
>>>> It’s not a problem. I started this project aware that foreign is still
>>>> in heavy development. I’m actually impressed it works as well as it
>>>> does, and I’ve made more progress using foreign to bind to wayland
>>>> than I did with JNR. Foreign is shaping up to be very good and I hope
>>>> that I can finish my program (implementing the wayland mcwayface
>>>> beginner’s window manager in scala) and have a semi-real world example
>>>> of using it to make a program.
>>>>
>>>> Two last questions while I have your attention:
>>>>
>>>> When I allocate a struct using the naive scope, can I control when
>>>> it’s freed? Do I just use the c free method on the pointer?
>>>>
>>>> Also, wayland loves the concept of doubly linked lists that use the
>>>> list nodes as pointers to elements within structs. In the wl_listener
>>>> struct I showed, there is a link member of wl_list. A pointer to that
>>>> member is what is inserted into the list with wl_list_insert. When it
>>>> comes time to traverse the list I take the list node pointer, declare
>>>> I think it’s part of wl_listener, find the offset from the link member
>>>> pointer of a wl_listener and a pointer to a wl_listener itself, and
>>>> apply that offset to the link. Below is the implementation of this in
>>>> scala:
>>>>
>>>> type HellType[T] = {
>>>> def ptr(): Pointer[T]
>>>> def link$ptr(): Pointer[wl_list]
>>>> }
>>>>
>>>> def extractFrom[T <: Struct[T] with HellType[T]](listItem: wl_list,
>>>> clazz: Class[T]) = {
>>>> val ev = s.allocateStruct(clazz)
>>>> val offset = ev.ptr.addr() - ev.link$ptr().addr()
>>>> listItem.ptr().cast(NativeTypes.VOID).cast(NativeTypes.INT8).offset(offset).cast(NativeTypes.VOID).cast(LayoutType.ofStruct(clazz))
>>>> }
>>>>
>>>> Where HellType[T] is a structural type that is a supertype of anything
>>>> that has a Pointer<T> ptr() method and a Pointer<wl_list> link$ptr().
>>>> Is this the appropriate way to do this with foreign?
>>>
>>> Yes, this looks good to me.
>>>
>>> It looks like we need to add some sort of static offsetOf utility
>>> API so that you don't have to allocate a struct to find the offset
>>> of a field. Maybe something like (Java):
>>>
>>> public static <T extends Struct<T>> long offsetOf(Class<T>
>>> structClass, String fieldName) {
>>> ...
>>> }
>>>
>>> Would that help?
>>>
>>>> It seems to work
>>>> for me (and when I intentionally botch the offset, I get an error as
>>>> expected), but I've only used it with things I've allocated via the
>>>> scope and I've not tested using this with wl_list nodes that have been
>>>> allocated by C.
>>>
>>> That should behave exactly the same, unless the C code maintains a
>>> different layout (ABI) then is standard. jextract derives the layout
>>> from the C ABI of the platform it runs on.
>>>
>>> Cheers,
>>> Jorn
>>>
>>>> Thanks,
>>>>
>>>> Mark
>>>>
>>>> On 2/15/19 11:22 AM, Maurizio Cimadamore wrote:
>>>>>
>>>>> On 15/02/2019 01:33, Mark Hammons wrote:
>>>>>> I was wrong in my previous email. The issue is still fixed for
>>>>>> me, but the fix was not because of a change in jextract usage,
>>>>>> but rather the inclusion of this in my code:
>>>>>>
>>>>>> val lib = Libraries.bind(MethodHandles.lookup,
>>>>>> classOf[wayland.wayland_server_core])
>>>>>
>>>>> I was just about to suggest doing that (I came upon the same trick
>>>>> overnight) - but I wanted to try with your specific case first!
>>>>>
>>>>> I'm super happy that this trick works for you.
>>>>>
>>>>> Binding the library gives a couple of kicks to the resolution
>>>>> logic, so that it performs as it should.
>>>>>
>>>>> In other words, in the current state, if you want to allocate a
>>>>> struct defined in some library, it's always better to bind the
>>>>> whole library first (even if one doesn't plan to use it).
>>>>>
>>>>> I'll add this workaround to the JBS entry.
>>>>>
>>>>> Thanks again for the report - and, yes, we'll fix this in the next
>>>>> EA, as this is very frustrating/confusing (sorry!).
>>>>>
>>>>> Maurizio
>>>>>
>>>>>>
>>>>>> I never use lib, but if I remove that line the allocation of
>>>>>> wl_listener starts failing again.
>>>>>>
>>>>>> Mark
>>>>>>
>>>>>> On 2/15/19 2:14 AM, Maurizio Cimadamore wrote:
>>>>>>> Thanks Jorn - clever approach; I'll give this some though to
>>>>>>> make sure it covers all the bases.
>>>>>>>
>>>>>>> Maurizio
>>>>>>>
>>>>>>> On 15/02/2019 01:06, Jorn Vernee wrote:
>>>>>>>> FWIW, I've previously used the following fix to work around a
>>>>>>>> similar issue (also involving a linked lists).
>>>>>>>>
>>>>>>>> (Rough) Webrev:
>>>>>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8219042/webrev.00/
>>>>>>>> Cheers,
>>>>>>>> Jorn
>>>>>>>>
>>>>>>>> Maurizio Cimadamore schreef op 2019-02-15 01:14:
>>>>>>>>> Here's the bug reference I've created:
>>>>>>>>>
>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8219042
>>>>>>>>>
>>>>>>>>> unfortunately, I tried allocating the structs in different
>>>>>>>>> order and
>>>>>>>>> the problem cannot be resolved at the client side.
>>>>>>>>>
>>>>>>>>> Maurizio
>>>>>>>>>
>>>>>>>>> On 15/02/2019 00:06, Mark Hammons wrote:
>>>>>>>>>> I previously allocated a wl_list in my code. I'm still new to
>>>>>>>>>> the foreign interfaces, so I'm not aware if there's a way to
>>>>>>>>>> allocate the wl_listener using a pre-allocated wl_list.
>>>>>>>>>>
>>>>>>>>>> Mark
>>>>>>>>>>
>>>>>>>>>> On 2/15/19 12:49 AM, Maurizio Cimadamore wrote:
>>>>>>>>>>>
>>>>>>>>>>> On 14/02/2019 23:38, Mark Hammons wrote:
>>>>>>>>>>>> Hi Maurizio,
>>>>>>>>>>>>
>>>>>>>>>>>> No, wl_list is defined in wayland_utils.h while wl_listener
>>>>>>>>>>>> is in wayland_server_core.h. I am currently looking through
>>>>>>>>>>>> the issues on the openjdk tracker and seeing if there's a
>>>>>>>>>>>> mitigation for this.
>>>>>>>>>>>
>>>>>>>>>>> Right - you beat me to this:
>>>>>>>>>>>
>>>>>>>>>>> https://people.freedesktop.org/~whot/wayland-doxygen/wayland/Server/structwl__listener.html
>>>>>>>>>>> and
>>>>>>>>>>>
>>>>>>>>>>> https://people.freedesktop.org/~whot/wayland-doxygen/wayland/Server/structwl__list.html
>>>>>>>>>>> Unfortunately this issue is not easy to workaround. I'll
>>>>>>>>>>> make sure to create a JBS entry for it (we do have one, but
>>>>>>>>>>> it's probably not visible outside).
>>>>>>>>>>>
>>>>>>>>>>> I'll also try to play with this a bit to see what can be
>>>>>>>>>>> done - with this issue sometimes it helps to allocate the
>>>>>>>>>>> inner struct first (e.g. wl_list), and then the one that
>>>>>>>>>>> depends on it (e.g. wl_listener).
>>>>>>>>>>>
>>>>>>>>>>> Maurizio
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> ~Mark
>>>>>>>>>>>>
>>>>>>>>>>>> On 2/15/19 12:30 AM, Maurizio Cimadamore wrote:
>>>>>>>>>>>>> Hi Mark,
>>>>>>>>>>>>> thanks for the report - from the looks of it, it seems an
>>>>>>>>>>>>> issue with cross-header layout resolution, which is listed
>>>>>>>>>>>>> in the 'known issues' in the EA page:
>>>>>>>>>>>>>
>>>>>>>>>>>>> "Dynamic layout resolution doesn't work across multiple
>>>>>>>>>>>>> headers."
>>>>>>>>>>>>>
>>>>>>>>>>>>> I will check in more details tomorrow, and confirm, one
>>>>>>>>>>>>> way or another.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Quick check: are wl_list and wl_listener defined in the
>>>>>>>>>>>>> same header file? If not that's likely the issue here.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I think Pointer<?> is the correct type - jextract tries to
>>>>>>>>>>>>> insert as more general types as possible when inserting
>>>>>>>>>>>>> Pointer in argument position; if it generated
>>>>>>>>>>>>> Pointer<Void>, and that was an ordinary function call, you
>>>>>>>>>>>>> could only call it with another Pointer<Void> - if the
>>>>>>>>>>>>> argument type is Pointer<?> you can pass _any_ pointer -
>>>>>>>>>>>>> e.g. Pointer<Byte>, Pointer<Integer> which is kind of
>>>>>>>>>>>>> close to what you can do in C.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Maurizio
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 14/02/2019 22:23, Mark Hammons wrote:
>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I decided to try to take the dive on project panama,
>>>>>>>>>>>>>> starting with making a binding to linux's wayland server.
>>>>>>>>>>>>>> I used the following command: ~/bin/jdk-13/bin/jextract
>>>>>>>>>>>>>> /usr/include/wayland/wayland-server-core.h
>>>>>>>>>>>>>> /usr/include/wayland/wayland-server.h
>>>>>>>>>>>>>> /usr/include/wayland/wayland-util.h
>>>>>>>>>>>>>> /usr/include/wayland/wayland-version.h
>>>>>>>>>>>>>> /usr/include/wayland/wayland-server-protocol.h -I
>>>>>>>>>>>>>> /usr/include/wayland -L /usr/lib64/ --record-library-path
>>>>>>>>>>>>>> -l wayland-server -t wayland -o wayland_server.jar
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> When I try to allocate a wl_listener struct, I get the
>>>>>>>>>>>>>> following error:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [error] Exception in thread "main"
>>>>>>>>>>>>>> java.lang.UnsupportedOperationException: bitsSize on
>>>>>>>>>>>>>> Unresolved
>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>> java.base/java.foreign.layout.Unresolved.bitsSize(Unresolved.java:76)
>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>> java.base/java.util.stream.ReferencePipeline$5$1.accept(ReferencePipeline.java:229)
>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>> java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>> java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>> java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>> java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>> java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>> java.base/java.util.stream.LongPipeline.reduce(LongPipeline.java:474)
>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>> java.base/java.util.stream.LongPipeline.sum(LongPipeline.java:432)
>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>> java.base/java.foreign.layout.Group.bitsSize(Group.java:119)
>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>> java.base/java.foreign.memory.LayoutType.bytesSize(LayoutType.java:49)
>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>> java.base/jdk.internal.foreign.ScopeImpl.allocateInternal(ScopeImpl.java:66)
>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>> java.base/jdk.internal.foreign.ScopeImpl.allocate(ScopeImpl.java:92)
>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>> java.base/jdk.internal.foreign.ScopeImpl.allocateStruct(ScopeImpl.java:98)
>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>> TestApp$.delayedEndpoint$TestApp$1(TestApp.scala:22)
>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>> TestApp$delayedInit$body.apply(TestApp.scala:13)
>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>> scala.Function0.apply$mcV$sp(Function0.scala:39)
>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>> scala.Function0.apply$mcV$sp$(Function0.scala:39)
>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>> scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>> scala.App.$anonfun$main$1$adapted(App.scala:80)
>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>> scala.collection.immutable.List.foreach(List.scala:392)
>>>>>>>>>>>>>> [error] at scala.App.main(App.scala:80)
>>>>>>>>>>>>>> [error] at scala.App.main$(App.scala:78)
>>>>>>>>>>>>>> [error] at TestApp$.main(TestApp.scala:13)
>>>>>>>>>>>>>> [error] at TestApp.main(TestApp.scala)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Looking at other bugs involving this kind of error
>>>>>>>>>>>>>> message, it appears that unresolved is a type for when
>>>>>>>>>>>>>> there's not enough layout information? In any case,
>>>>>>>>>>>>>> here's the struct in question:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> struct wl_listener {
>>>>>>>>>>>>>> struct wl_list link;
>>>>>>>>>>>>>> wl_notify_func_t notify;
>>>>>>>>>>>>>> };
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> and the definition of the elements:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> typedef void (*wl_notify_func_t)(struct wl_listener
>>>>>>>>>>>>>> *listener, void *data);
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> struct wl_list {
>>>>>>>>>>>>>> /** Previous list element */
>>>>>>>>>>>>>> struct wl_list *prev;
>>>>>>>>>>>>>> /** Next list element */
>>>>>>>>>>>>>> struct wl_list *next;
>>>>>>>>>>>>>> };
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I'm fairly certain the issue lies with the function
>>>>>>>>>>>>>> pointer notify. When I looked at the decompiled source,
>>>>>>>>>>>>>> wl_notify_func_t is defined as:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> @FunctionalInterface
>>>>>>>>>>>>>> @NativeCallback("(u64:${wl_listener}u64:v)v")
>>>>>>>>>>>>>> public interface FI5 {
>>>>>>>>>>>>>> void fn(Pointer<wayland_server_core.wl_listener>
>>>>>>>>>>>>>> var1, Pointer<?> var2);
>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> which seems suspicious to me. var2 should be a
>>>>>>>>>>>>>> Pointer<Void> I would think. It's a type I see elsewhere
>>>>>>>>>>>>>> in the source for this file, so it seems suspect that
>>>>>>>>>>>>>> var2 is a Pointer<?>.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Is this a bug? Am I just using jextract wrong?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks for your help,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Mark Hammons
>>>>>>>>>>>>>>
More information about the panama-dev
mailing list