Panama unresolved error when instantiating wayland struct...
Maurizio Cimadamore
maurizio.cimadamore at oracle.com
Fri Feb 15 12:51:20 UTC 2019
On 15/02/2019 12:34, Mark Hammons wrote:
> It’s not a problem. I started this project aware that foreign is still
> in heavy development. I’m actually impressed it works as well as it
> does, and I’ve made more progress using foreign to bind to wayland
> than I did with JNR. Foreign is shaping up to be very good and I hope
> that I can finish my program (implementing the wayland mcwayface
> beginner’s window manager in scala) and have a semi-real world example
> of using it to make a program.
Thanks!!!
>
> Two last questions while I have your attention:
>
> When I allocate a struct using the naive scope, can I control when
> it’s freed? Do I just use the c free method on the pointer?
If you allocate the struct using a scope with a try-with-resource, as in:
try (Scope sc = Scope.newNativeScope()) {
sc.allocateStruct(Foo.class);
} //everything freed here
That is, everything will automatically be freed at the end of the try block.
Or, you go the manual way:
Scope sc = Scope.newNativeScope()
...
sc.allocateStruct(Foo.class);
...
sc.close(); //this frees memory
But you have to remember calling Scope::close or there will be a memory
leak.
>
> Also, wayland loves the concept of doubly linked lists that use the
> list nodes as pointers to elements within structs. In the wl_listener
> struct I showed, there is a link member of wl_list. A pointer to that
> member is what is inserted into the list with wl_list_insert. When it
> comes time to traverse the list I take the list node pointer, declare
> I think it’s part of wl_listener, find the offset from the link member
> pointer of a wl_listener and a pointer to a wl_listener itself, and
> apply that offset to the link. Below is the implementation of this in
> scala:
>
> type HellType[T] = {
> def ptr(): Pointer[T]
> def link$ptr(): Pointer[wl_list]
> }
>
> def extractFrom[T <: Struct[T] with HellType[T]](listItem: wl_list,
> clazz: Class[T]) = {
> val ev = s.allocateStruct(clazz)
> val offset = ev.ptr.addr() - ev.link$ptr().addr()
> listItem.ptr().cast(NativeTypes.VOID).cast(NativeTypes.INT8).offset(offset).cast(NativeTypes.VOID).cast(LayoutType.ofStruct(clazz))
>
> }
>
> Where HellType[T] is a structural type that is a supertype of anything
> that has a Pointer<T> ptr() method and a Pointer<wl_list> link$ptr().
> Is this the appropriate way to do this with foreign? It seems to work
> for me (and when I intentionally botch the offset, I get an error as
> expected), but I've only used it with things I've allocated via the
> scope and I've not tested using this with wl_list nodes that have been
> allocated by C.
I'm not sure I get this - your extractFrom allocates a struct (which
will be empty) and then you do some offset computation, but it seems to
me that ev.link$ptr() will always be null? (address 0) ?
Can you expand a bit on what extractFrom() is expected to do?
Maurizio
>
> Thanks,
>
> Mark
>
> On 2/15/19 11:22 AM, Maurizio Cimadamore wrote:
>>
>> On 15/02/2019 01:33, Mark Hammons wrote:
>>> I was wrong in my previous email. The issue is still fixed for me,
>>> but the fix was not because of a change in jextract usage, but
>>> rather the inclusion of this in my code:
>>>
>>> val lib = Libraries.bind(MethodHandles.lookup,
>>> classOf[wayland.wayland_server_core])
>>
>> I was just about to suggest doing that (I came upon the same trick
>> overnight) - but I wanted to try with your specific case first!
>>
>> I'm super happy that this trick works for you.
>>
>> Binding the library gives a couple of kicks to the resolution logic,
>> so that it performs as it should.
>>
>> In other words, in the current state, if you want to allocate a
>> struct defined in some library, it's always better to bind the whole
>> library first (even if one doesn't plan to use it).
>>
>> I'll add this workaround to the JBS entry.
>>
>> Thanks again for the report - and, yes, we'll fix this in the next
>> EA, as this is very frustrating/confusing (sorry!).
>>
>> Maurizio
>>
>>>
>>> I never use lib, but if I remove that line the allocation of
>>> wl_listener starts failing again.
>>>
>>> Mark
>>>
>>> On 2/15/19 2:14 AM, Maurizio Cimadamore wrote:
>>>> Thanks Jorn - clever approach; I'll give this some though to make
>>>> sure it covers all the bases.
>>>>
>>>> Maurizio
>>>>
>>>> On 15/02/2019 01:06, Jorn Vernee wrote:
>>>>> FWIW, I've previously used the following fix to work around a
>>>>> similar issue (also involving a linked lists).
>>>>>
>>>>> (Rough) Webrev:
>>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8219042/webrev.00/
>>>>>
>>>>> Cheers,
>>>>> Jorn
>>>>>
>>>>> Maurizio Cimadamore schreef op 2019-02-15 01:14:
>>>>>> Here's the bug reference I've created:
>>>>>>
>>>>>> https://bugs.openjdk.java.net/browse/JDK-8219042
>>>>>>
>>>>>> unfortunately, I tried allocating the structs in different order and
>>>>>> the problem cannot be resolved at the client side.
>>>>>>
>>>>>> Maurizio
>>>>>>
>>>>>> On 15/02/2019 00:06, Mark Hammons wrote:
>>>>>>> I previously allocated a wl_list in my code. I'm still new to
>>>>>>> the foreign interfaces, so I'm not aware if there's a way to
>>>>>>> allocate the wl_listener using a pre-allocated wl_list.
>>>>>>>
>>>>>>> Mark
>>>>>>>
>>>>>>> On 2/15/19 12:49 AM, Maurizio Cimadamore wrote:
>>>>>>>>
>>>>>>>> On 14/02/2019 23:38, Mark Hammons wrote:
>>>>>>>>> Hi Maurizio,
>>>>>>>>>
>>>>>>>>> No, wl_list is defined in wayland_utils.h while wl_listener is
>>>>>>>>> in wayland_server_core.h. I am currently looking through the
>>>>>>>>> issues on the openjdk tracker and seeing if there's a
>>>>>>>>> mitigation for this.
>>>>>>>>
>>>>>>>> Right - you beat me to this:
>>>>>>>>
>>>>>>>> https://people.freedesktop.org/~whot/wayland-doxygen/wayland/Server/structwl__listener.html
>>>>>>>> and
>>>>>>>>
>>>>>>>> https://people.freedesktop.org/~whot/wayland-doxygen/wayland/Server/structwl__list.html
>>>>>>>> Unfortunately this issue is not easy to workaround. I'll make
>>>>>>>> sure to create a JBS entry for it (we do have one, but it's
>>>>>>>> probably not visible outside).
>>>>>>>>
>>>>>>>> I'll also try to play with this a bit to see what can be done -
>>>>>>>> with this issue sometimes it helps to allocate the inner struct
>>>>>>>> first (e.g. wl_list), and then the one that depends on it (e.g.
>>>>>>>> wl_listener).
>>>>>>>>
>>>>>>>> Maurizio
>>>>>>>>
>>>>>>>>>
>>>>>>>>> ~Mark
>>>>>>>>>
>>>>>>>>> On 2/15/19 12:30 AM, Maurizio Cimadamore wrote:
>>>>>>>>>> Hi Mark,
>>>>>>>>>> thanks for the report - from the looks of it, it seems an
>>>>>>>>>> issue with cross-header layout resolution, which is listed in
>>>>>>>>>> the 'known issues' in the EA page:
>>>>>>>>>>
>>>>>>>>>> "Dynamic layout resolution doesn't work across multiple
>>>>>>>>>> headers."
>>>>>>>>>>
>>>>>>>>>> I will check in more details tomorrow, and confirm, one way
>>>>>>>>>> or another.
>>>>>>>>>>
>>>>>>>>>> Quick check: are wl_list and wl_listener defined in the same
>>>>>>>>>> header file? If not that's likely the issue here.
>>>>>>>>>>
>>>>>>>>>> I think Pointer<?> is the correct type - jextract tries to
>>>>>>>>>> insert as more general types as possible when inserting
>>>>>>>>>> Pointer in argument position; if it generated Pointer<Void>,
>>>>>>>>>> and that was an ordinary function call, you could only call
>>>>>>>>>> it with another Pointer<Void> - if the argument type is
>>>>>>>>>> Pointer<?> you can pass _any_ pointer - e.g. Pointer<Byte>,
>>>>>>>>>> Pointer<Integer> which is kind of close to what you can do in C.
>>>>>>>>>>
>>>>>>>>>> Maurizio
>>>>>>>>>>
>>>>>>>>>> On 14/02/2019 22:23, Mark Hammons wrote:
>>>>>>>>>>> Hi all,
>>>>>>>>>>>
>>>>>>>>>>> I decided to try to take the dive on project panama,
>>>>>>>>>>> starting with making a binding to linux's wayland server. I
>>>>>>>>>>> used the following command: ~/bin/jdk-13/bin/jextract
>>>>>>>>>>> /usr/include/wayland/wayland-server-core.h
>>>>>>>>>>> /usr/include/wayland/wayland-server.h
>>>>>>>>>>> /usr/include/wayland/wayland-util.h
>>>>>>>>>>> /usr/include/wayland/wayland-version.h
>>>>>>>>>>> /usr/include/wayland/wayland-server-protocol.h -I
>>>>>>>>>>> /usr/include/wayland -L /usr/lib64/ --record-library-path -l
>>>>>>>>>>> wayland-server -t wayland -o wayland_server.jar
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> When I try to allocate a wl_listener struct, I get the
>>>>>>>>>>> following error:
>>>>>>>>>>>
>>>>>>>>>>> [error] Exception in thread "main"
>>>>>>>>>>> java.lang.UnsupportedOperationException: bitsSize on Unresolved
>>>>>>>>>>> [error] at
>>>>>>>>>>> java.base/java.foreign.layout.Unresolved.bitsSize(Unresolved.java:76)
>>>>>>>>>>>
>>>>>>>>>>> [error] at
>>>>>>>>>>> java.base/java.util.stream.ReferencePipeline$5$1.accept(ReferencePipeline.java:229)
>>>>>>>>>>> [error] at
>>>>>>>>>>> java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
>>>>>>>>>>> [error] at
>>>>>>>>>>> java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
>>>>>>>>>>> [error] at
>>>>>>>>>>> java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
>>>>>>>>>>> [error] at
>>>>>>>>>>> java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
>>>>>>>>>>> [error] at
>>>>>>>>>>> java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
>>>>>>>>>>> [error] at
>>>>>>>>>>> java.base/java.util.stream.LongPipeline.reduce(LongPipeline.java:474)
>>>>>>>>>>>
>>>>>>>>>>> [error] at
>>>>>>>>>>> java.base/java.util.stream.LongPipeline.sum(LongPipeline.java:432)
>>>>>>>>>>>
>>>>>>>>>>> [error] at
>>>>>>>>>>> java.base/java.foreign.layout.Group.bitsSize(Group.java:119)
>>>>>>>>>>> [error] at
>>>>>>>>>>> java.base/java.foreign.memory.LayoutType.bytesSize(LayoutType.java:49)
>>>>>>>>>>> [error] at
>>>>>>>>>>> java.base/jdk.internal.foreign.ScopeImpl.allocateInternal(ScopeImpl.java:66)
>>>>>>>>>>> [error] at
>>>>>>>>>>> java.base/jdk.internal.foreign.ScopeImpl.allocate(ScopeImpl.java:92)
>>>>>>>>>>>
>>>>>>>>>>> [error] at
>>>>>>>>>>> java.base/jdk.internal.foreign.ScopeImpl.allocateStruct(ScopeImpl.java:98)
>>>>>>>>>>> [error] at
>>>>>>>>>>> TestApp$.delayedEndpoint$TestApp$1(TestApp.scala:22)
>>>>>>>>>>> [error] at TestApp$delayedInit$body.apply(TestApp.scala:13)
>>>>>>>>>>> [error] at scala.Function0.apply$mcV$sp(Function0.scala:39)
>>>>>>>>>>> [error] at
>>>>>>>>>>> scala.Function0.apply$mcV$sp$(Function0.scala:39)
>>>>>>>>>>> [error] at
>>>>>>>>>>> scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
>>>>>>>>>>> [error] at scala.App.$anonfun$main$1$adapted(App.scala:80)
>>>>>>>>>>> [error] at
>>>>>>>>>>> scala.collection.immutable.List.foreach(List.scala:392)
>>>>>>>>>>> [error] at scala.App.main(App.scala:80)
>>>>>>>>>>> [error] at scala.App.main$(App.scala:78)
>>>>>>>>>>> [error] at TestApp$.main(TestApp.scala:13)
>>>>>>>>>>> [error] at TestApp.main(TestApp.scala)
>>>>>>>>>>>
>>>>>>>>>>> Looking at other bugs involving this kind of error message,
>>>>>>>>>>> it appears that unresolved is a type for when there's not
>>>>>>>>>>> enough layout information? In any case, here's the struct in
>>>>>>>>>>> question:
>>>>>>>>>>>
>>>>>>>>>>> struct wl_listener {
>>>>>>>>>>> struct wl_list link;
>>>>>>>>>>> wl_notify_func_t notify;
>>>>>>>>>>> };
>>>>>>>>>>>
>>>>>>>>>>> and the definition of the elements:
>>>>>>>>>>>
>>>>>>>>>>> typedef void (*wl_notify_func_t)(struct wl_listener
>>>>>>>>>>> *listener, void *data);
>>>>>>>>>>>
>>>>>>>>>>> struct wl_list {
>>>>>>>>>>> /** Previous list element */
>>>>>>>>>>> struct wl_list *prev;
>>>>>>>>>>> /** Next list element */
>>>>>>>>>>> struct wl_list *next;
>>>>>>>>>>> };
>>>>>>>>>>>
>>>>>>>>>>> I'm fairly certain the issue lies with the function pointer
>>>>>>>>>>> notify. When I looked at the decompiled source,
>>>>>>>>>>> wl_notify_func_t is defined as:
>>>>>>>>>>>
>>>>>>>>>>> @FunctionalInterface
>>>>>>>>>>> @NativeCallback("(u64:${wl_listener}u64:v)v")
>>>>>>>>>>> public interface FI5 {
>>>>>>>>>>> void fn(Pointer<wayland_server_core.wl_listener>
>>>>>>>>>>> var1, Pointer<?> var2);
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> which seems suspicious to me. var2 should be a Pointer<Void>
>>>>>>>>>>> I would think. It's a type I see elsewhere in the source for
>>>>>>>>>>> this file, so it seems suspect that var2 is a Pointer<?>.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Is this a bug? Am I just using jextract wrong?
>>>>>>>>>>>
>>>>>>>>>>> Thanks for your help,
>>>>>>>>>>>
>>>>>>>>>>> Mark Hammons
>>>>>>>>>>>
More information about the panama-dev
mailing list