Panama unresolved error when instantiating wayland struct...
Maurizio Cimadamore
maurizio.cimadamore at oracle.com
Fri Feb 15 14:01:44 UTC 2019
Or, even better - just use the LayoutPaths internal API, which is what
the binder uses:
public static long offsetOf(Class<? extends Struct<?>> cls, String
fieldName) {
Layout l = LayoutType.ofStruct(cls).layout();
LayoutPath path = LayoutPaths.lookup(l, el ->
el.name().equals(fieldName)).findFirst().orElseThrow(IllegalArgumentException::new);
return path.offset();
}
Maurizio
On 15/02/2019 13:53, Maurizio Cimadamore wrote:
> This looks good - I would stick with LayoutType.ofStruct().layout()
> since that will apply all necessary resolution - otherwise the method
> will end up calling bitsSize on unresolved
>
> Maurizio
>
> On 15/02/2019 13:35, Jorn Vernee wrote:
>> FWIW, here is a quick prototype that you could use as a utility
>> method until then:
>>
>> public static long offsetOf(Class<? extends Struct<?>> cls,
>> String fieldName) {
>> Group g = (Group)
>> Layout.of(cls.getAnnotation(NativeStruct.class).value());
>>
>> long offset = 0;
>> for(Layout l : g.elements()) {
>> Optional<String> name = l.name();
>> if(name.isPresent() && name.get().equals(fieldName)) {
>> return offset / 8; // convert to bytes
>> }
>> offset += l.bitsSize();
>> }
>>
>> throw new IllegalArgumentException("Not a valid field name: "
>> + fieldName);
>> }
>>
>> Cheers,
>> Jorn
>>
>> Mark Hammons schreef op 2019-02-15 14:25:
>>> That would help a ton! I was searching like crazy in LayoutType and
>>> Layout for this kind of functionality!
>>>
>>> On 15/02/2019 14:18, Jorn Vernee wrote:
>>>> Response inline...
>>>>
>>>> Mark Hammons schreef op 2019-02-15 13:34:
>>>>> It’s not a problem. I started this project aware that foreign is
>>>>> still
>>>>> in heavy development. I’m actually impressed it works as well as it
>>>>> does, and I’ve made more progress using foreign to bind to wayland
>>>>> than I did with JNR. Foreign is shaping up to be very good and I hope
>>>>> that I can finish my program (implementing the wayland mcwayface
>>>>> beginner’s window manager in scala) and have a semi-real world
>>>>> example
>>>>> of using it to make a program.
>>>>>
>>>>> Two last questions while I have your attention:
>>>>>
>>>>> When I allocate a struct using the naive scope, can I control when
>>>>> it’s freed? Do I just use the c free method on the pointer?
>>>>>
>>>>> Also, wayland loves the concept of doubly linked lists that use the
>>>>> list nodes as pointers to elements within structs. In the wl_listener
>>>>> struct I showed, there is a link member of wl_list. A pointer to that
>>>>> member is what is inserted into the list with wl_list_insert. When it
>>>>> comes time to traverse the list I take the list node pointer, declare
>>>>> I think it’s part of wl_listener, find the offset from the link
>>>>> member
>>>>> pointer of a wl_listener and a pointer to a wl_listener itself, and
>>>>> apply that offset to the link. Below is the implementation of this in
>>>>> scala:
>>>>>
>>>>> type HellType[T] = {
>>>>> def ptr(): Pointer[T]
>>>>> def link$ptr(): Pointer[wl_list]
>>>>> }
>>>>>
>>>>> def extractFrom[T <: Struct[T] with HellType[T]](listItem: wl_list,
>>>>> clazz: Class[T]) = {
>>>>> val ev = s.allocateStruct(clazz)
>>>>> val offset = ev.ptr.addr() - ev.link$ptr().addr()
>>>>> listItem.ptr().cast(NativeTypes.VOID).cast(NativeTypes.INT8).offset(offset).cast(NativeTypes.VOID).cast(LayoutType.ofStruct(clazz))
>>>>> }
>>>>>
>>>>> Where HellType[T] is a structural type that is a supertype of
>>>>> anything
>>>>> that has a Pointer<T> ptr() method and a Pointer<wl_list> link$ptr().
>>>>> Is this the appropriate way to do this with foreign?
>>>>
>>>> Yes, this looks good to me.
>>>>
>>>> It looks like we need to add some sort of static offsetOf utility
>>>> API so that you don't have to allocate a struct to find the offset
>>>> of a field. Maybe something like (Java):
>>>>
>>>> public static <T extends Struct<T>> long offsetOf(Class<T>
>>>> structClass, String fieldName) {
>>>> ...
>>>> }
>>>>
>>>> Would that help?
>>>>
>>>>> It seems to work
>>>>> for me (and when I intentionally botch the offset, I get an error as
>>>>> expected), but I've only used it with things I've allocated via the
>>>>> scope and I've not tested using this with wl_list nodes that have
>>>>> been
>>>>> allocated by C.
>>>>
>>>> That should behave exactly the same, unless the C code maintains a
>>>> different layout (ABI) then is standard. jextract derives the
>>>> layout from the C ABI of the platform it runs on.
>>>>
>>>> Cheers,
>>>> Jorn
>>>>
>>>>> Thanks,
>>>>>
>>>>> Mark
>>>>>
>>>>> On 2/15/19 11:22 AM, Maurizio Cimadamore wrote:
>>>>>>
>>>>>> On 15/02/2019 01:33, Mark Hammons wrote:
>>>>>>> I was wrong in my previous email. The issue is still fixed for
>>>>>>> me, but the fix was not because of a change in jextract usage,
>>>>>>> but rather the inclusion of this in my code:
>>>>>>>
>>>>>>> val lib = Libraries.bind(MethodHandles.lookup,
>>>>>>> classOf[wayland.wayland_server_core])
>>>>>>
>>>>>> I was just about to suggest doing that (I came upon the same
>>>>>> trick overnight) - but I wanted to try with your specific case
>>>>>> first!
>>>>>>
>>>>>> I'm super happy that this trick works for you.
>>>>>>
>>>>>> Binding the library gives a couple of kicks to the resolution
>>>>>> logic, so that it performs as it should.
>>>>>>
>>>>>> In other words, in the current state, if you want to allocate a
>>>>>> struct defined in some library, it's always better to bind the
>>>>>> whole library first (even if one doesn't plan to use it).
>>>>>>
>>>>>> I'll add this workaround to the JBS entry.
>>>>>>
>>>>>> Thanks again for the report - and, yes, we'll fix this in the
>>>>>> next EA, as this is very frustrating/confusing (sorry!).
>>>>>>
>>>>>> Maurizio
>>>>>>
>>>>>>>
>>>>>>> I never use lib, but if I remove that line the allocation of
>>>>>>> wl_listener starts failing again.
>>>>>>>
>>>>>>> Mark
>>>>>>>
>>>>>>> On 2/15/19 2:14 AM, Maurizio Cimadamore wrote:
>>>>>>>> Thanks Jorn - clever approach; I'll give this some though to
>>>>>>>> make sure it covers all the bases.
>>>>>>>>
>>>>>>>> Maurizio
>>>>>>>>
>>>>>>>> On 15/02/2019 01:06, Jorn Vernee wrote:
>>>>>>>>> FWIW, I've previously used the following fix to work around a
>>>>>>>>> similar issue (also involving a linked lists).
>>>>>>>>>
>>>>>>>>> (Rough) Webrev:
>>>>>>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8219042/webrev.00/
>>>>>>>>> Cheers,
>>>>>>>>> Jorn
>>>>>>>>>
>>>>>>>>> Maurizio Cimadamore schreef op 2019-02-15 01:14:
>>>>>>>>>> Here's the bug reference I've created:
>>>>>>>>>>
>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8219042
>>>>>>>>>>
>>>>>>>>>> unfortunately, I tried allocating the structs in different
>>>>>>>>>> order and
>>>>>>>>>> the problem cannot be resolved at the client side.
>>>>>>>>>>
>>>>>>>>>> Maurizio
>>>>>>>>>>
>>>>>>>>>> On 15/02/2019 00:06, Mark Hammons wrote:
>>>>>>>>>>> I previously allocated a wl_list in my code. I'm still new
>>>>>>>>>>> to the foreign interfaces, so I'm not aware if there's a way
>>>>>>>>>>> to allocate the wl_listener using a pre-allocated wl_list.
>>>>>>>>>>>
>>>>>>>>>>> Mark
>>>>>>>>>>>
>>>>>>>>>>> On 2/15/19 12:49 AM, Maurizio Cimadamore wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> On 14/02/2019 23:38, Mark Hammons wrote:
>>>>>>>>>>>>> Hi Maurizio,
>>>>>>>>>>>>>
>>>>>>>>>>>>> No, wl_list is defined in wayland_utils.h while
>>>>>>>>>>>>> wl_listener is in wayland_server_core.h. I am currently
>>>>>>>>>>>>> looking through the issues on the openjdk tracker and
>>>>>>>>>>>>> seeing if there's a mitigation for this.
>>>>>>>>>>>>
>>>>>>>>>>>> Right - you beat me to this:
>>>>>>>>>>>>
>>>>>>>>>>>> https://people.freedesktop.org/~whot/wayland-doxygen/wayland/Server/structwl__listener.html
>>>>>>>>>>>> and
>>>>>>>>>>>>
>>>>>>>>>>>> https://people.freedesktop.org/~whot/wayland-doxygen/wayland/Server/structwl__list.html
>>>>>>>>>>>> Unfortunately this issue is not easy to workaround. I'll
>>>>>>>>>>>> make sure to create a JBS entry for it (we do have one, but
>>>>>>>>>>>> it's probably not visible outside).
>>>>>>>>>>>>
>>>>>>>>>>>> I'll also try to play with this a bit to see what can be
>>>>>>>>>>>> done - with this issue sometimes it helps to allocate the
>>>>>>>>>>>> inner struct first (e.g. wl_list), and then the one that
>>>>>>>>>>>> depends on it (e.g. wl_listener).
>>>>>>>>>>>>
>>>>>>>>>>>> Maurizio
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> ~Mark
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 2/15/19 12:30 AM, Maurizio Cimadamore wrote:
>>>>>>>>>>>>>> Hi Mark,
>>>>>>>>>>>>>> thanks for the report - from the looks of it, it seems an
>>>>>>>>>>>>>> issue with cross-header layout resolution, which is
>>>>>>>>>>>>>> listed in the 'known issues' in the EA page:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> "Dynamic layout resolution doesn't work across multiple
>>>>>>>>>>>>>> headers."
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I will check in more details tomorrow, and confirm, one
>>>>>>>>>>>>>> way or another.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Quick check: are wl_list and wl_listener defined in the
>>>>>>>>>>>>>> same header file? If not that's likely the issue here.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I think Pointer<?> is the correct type - jextract tries
>>>>>>>>>>>>>> to insert as more general types as possible when
>>>>>>>>>>>>>> inserting Pointer in argument position; if it generated
>>>>>>>>>>>>>> Pointer<Void>, and that was an ordinary function call,
>>>>>>>>>>>>>> you could only call it with another Pointer<Void> - if
>>>>>>>>>>>>>> the argument type is Pointer<?> you can pass _any_
>>>>>>>>>>>>>> pointer - e.g. Pointer<Byte>, Pointer<Integer> which is
>>>>>>>>>>>>>> kind of close to what you can do in C.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Maurizio
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 14/02/2019 22:23, Mark Hammons wrote:
>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I decided to try to take the dive on project panama,
>>>>>>>>>>>>>>> starting with making a binding to linux's wayland
>>>>>>>>>>>>>>> server. I used the following command:
>>>>>>>>>>>>>>> ~/bin/jdk-13/bin/jextract
>>>>>>>>>>>>>>> /usr/include/wayland/wayland-server-core.h
>>>>>>>>>>>>>>> /usr/include/wayland/wayland-server.h
>>>>>>>>>>>>>>> /usr/include/wayland/wayland-util.h
>>>>>>>>>>>>>>> /usr/include/wayland/wayland-version.h
>>>>>>>>>>>>>>> /usr/include/wayland/wayland-server-protocol.h -I
>>>>>>>>>>>>>>> /usr/include/wayland -L /usr/lib64/
>>>>>>>>>>>>>>> --record-library-path -l wayland-server -t wayland -o
>>>>>>>>>>>>>>> wayland_server.jar
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> When I try to allocate a wl_listener struct, I get the
>>>>>>>>>>>>>>> following error:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> [error] Exception in thread "main"
>>>>>>>>>>>>>>> java.lang.UnsupportedOperationException: bitsSize on
>>>>>>>>>>>>>>> Unresolved
>>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>>> java.base/java.foreign.layout.Unresolved.bitsSize(Unresolved.java:76)
>>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>>> java.base/java.util.stream.ReferencePipeline$5$1.accept(ReferencePipeline.java:229)
>>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>>> java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
>>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>>> java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
>>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>>> java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
>>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>>> java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
>>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>>> java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
>>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>>> java.base/java.util.stream.LongPipeline.reduce(LongPipeline.java:474)
>>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>>> java.base/java.util.stream.LongPipeline.sum(LongPipeline.java:432)
>>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>>> java.base/java.foreign.layout.Group.bitsSize(Group.java:119)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>>> java.base/java.foreign.memory.LayoutType.bytesSize(LayoutType.java:49)
>>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>>> java.base/jdk.internal.foreign.ScopeImpl.allocateInternal(ScopeImpl.java:66)
>>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>>> java.base/jdk.internal.foreign.ScopeImpl.allocate(ScopeImpl.java:92)
>>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>>> java.base/jdk.internal.foreign.ScopeImpl.allocateStruct(ScopeImpl.java:98)
>>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>>> TestApp$.delayedEndpoint$TestApp$1(TestApp.scala:22)
>>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>>> TestApp$delayedInit$body.apply(TestApp.scala:13)
>>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>>> scala.Function0.apply$mcV$sp(Function0.scala:39)
>>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>>> scala.Function0.apply$mcV$sp$(Function0.scala:39)
>>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>>> scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
>>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>>> scala.App.$anonfun$main$1$adapted(App.scala:80)
>>>>>>>>>>>>>>> [error] at
>>>>>>>>>>>>>>> scala.collection.immutable.List.foreach(List.scala:392)
>>>>>>>>>>>>>>> [error] at scala.App.main(App.scala:80)
>>>>>>>>>>>>>>> [error] at scala.App.main$(App.scala:78)
>>>>>>>>>>>>>>> [error] at TestApp$.main(TestApp.scala:13)
>>>>>>>>>>>>>>> [error] at TestApp.main(TestApp.scala)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Looking at other bugs involving this kind of error
>>>>>>>>>>>>>>> message, it appears that unresolved is a type for when
>>>>>>>>>>>>>>> there's not enough layout information? In any case,
>>>>>>>>>>>>>>> here's the struct in question:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> struct wl_listener {
>>>>>>>>>>>>>>> struct wl_list link;
>>>>>>>>>>>>>>> wl_notify_func_t notify;
>>>>>>>>>>>>>>> };
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> and the definition of the elements:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> typedef void (*wl_notify_func_t)(struct wl_listener
>>>>>>>>>>>>>>> *listener, void *data);
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> struct wl_list {
>>>>>>>>>>>>>>> /** Previous list element */
>>>>>>>>>>>>>>> struct wl_list *prev;
>>>>>>>>>>>>>>> /** Next list element */
>>>>>>>>>>>>>>> struct wl_list *next;
>>>>>>>>>>>>>>> };
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I'm fairly certain the issue lies with the function
>>>>>>>>>>>>>>> pointer notify. When I looked at the decompiled source,
>>>>>>>>>>>>>>> wl_notify_func_t is defined as:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> @FunctionalInterface
>>>>>>>>>>>>>>> @NativeCallback("(u64:${wl_listener}u64:v)v")
>>>>>>>>>>>>>>> public interface FI5 {
>>>>>>>>>>>>>>> void fn(Pointer<wayland_server_core.wl_listener>
>>>>>>>>>>>>>>> var1, Pointer<?> var2);
>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> which seems suspicious to me. var2 should be a
>>>>>>>>>>>>>>> Pointer<Void> I would think. It's a type I see elsewhere
>>>>>>>>>>>>>>> in the source for this file, so it seems suspect that
>>>>>>>>>>>>>>> var2 is a Pointer<?>.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Is this a bug? Am I just using jextract wrong?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks for your help,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Mark Hammons
>>>>>>>>>>>>>>>
More information about the panama-dev
mailing list