Anonymous structs have strange issues with scala and intellij in ea+44
Maurizio Cimadamore
maurizio.cimadamore at oracle.com
Thu Mar 7 00:31:33 UTC 2019
On 07/03/2019 00:08, Jorn Vernee wrote:
> Thanks, Mark,
>
> Through some debugging, the stack trace that you're seeing seems to be
> caused by the option parser we are using automatically trying to
> recognize the removed -m option as an abbreviation of the
> --missing-symbols option, which leads to a cryptic error message.
>
> We should probably disable the automatic abbreviations. After doing
> this I instead get the error
> `jdk.internal.joptsimple.UnrecognizedOptionException: m is not a
> recognized option`, which seems much more informative. (filed
> https://bugs.openjdk.java.net/browse/JDK-8220255)
+1
Automatic abbreviations can be handy, but not much in JDK style
Maurizio
>
> Thanks for the report!
> Jorn
>
> Mark Hammons schreef op 2019-03-07 01:02:
>> Replacing -m with --package-map seems to fix the error I was getting,
>> so there's that at least. Dotty is failing to compile when it comes to
>> the new jextract jars, but it might just be a bug in dotty. The new
>> jextract jars work perfectly with intellij and scala now. I'm trying
>> to create a minimal version of the jextract jar that crashes dotty to
>> report to them.
>>
>>> Hi Mark,
>>> we'll try to get a new build in next week or so.
>>>
>>> Maurizio
>>
>> Great, glad to hear it. Thankfully it turns out my problems were
>> because the arguments to jextract changed, but otherwise everything
>> seems to be working with the nightly build now.
>>
>> ~Mark
>>
>> On 3/7/19 12:11 AM, Mark Hammons wrote:
>>> Here you go:
>>>
>>> [info] issuing command jextract /usr/include/wlr/types/wlr_matrix.h
>>> /usr/include/wlr/types/wlr_surface.h
>>> /usr/include/wlr/types/wlr_box.h
>>> /usr/include/wlr/types/wlr_gamma_control.h
>>> /usr/include/wlr/types/wlr_idle.h /usr/include/wlr/backend.h
>>> /usr/include/wlr/types/wlr_output.h
>>> /usr/include/wlr/types/wlr_primary_selection_v1.h
>>> /usr/include/wlr/types/wlr_screenshooter.h
>>> /usr/include/wlr/types/wlr_compositor.h
>>> /usr/include/wlr/types/wlr_xdg_shell_v6.h
>>> /usr/include/wlr/render/wlr_renderer.h -m
>>> /usr/include/wlr/backend=wlroots.backend_headers -m
>>> /usr/include/bits/types=usr.include.bits.type_headers -I
>>> /usr/include/wayland -I /usr/include/wlr -I /usr/include/pixman-1 -I
>>> /usr/include/libxkbcommon -I
>>> /home/mhammons/Documents/MyProjects/waylandTest/include -C
>>> -DWLR_USE_UNSTABLE -L /usr/lib64 --record-library-path -l wlroots -t
>>> wlroots -o
>>> /home/mhammons/Documents/MyProjects/waylandTest/lib/wlroots.jar
>>>
>>> I'll try replacing the -m option with package-map.
>>>
>>> Mark
>>>
>>> On 3/7/19 12:10 AM, Jorn Vernee wrote:
>>>> Hi Mark,
>>>>
>>>> Can you provide the command that is producing that stack trace?
>>>>
>>>> Please note that the `-m` options was removed, and should be
>>>> replaced by the `--package-map` option.
>>>>
>>>> Thanks,
>>>> Jorn
>>>>
>>>> Mark Hammons schreef op 2019-03-06 23:05:
>>>>> Hi Maurizio,
>>>>>
>>>>> Do you know when an updated ea build will be released with the fix
>>>>> for
>>>>> this? I tried to port my application to dotty (a very early,
>>>>> experimental version of scala 3), but it seems that dotty is even
>>>>> more
>>>>> allergic to the inner classes that jextract is producing than scala 2
>>>>> is. For example, it insists that
>>>>> usr.include.wayland.wayland_util.wl_list is actually
>>>>> usr.include.wayland.wayland_util$wl_list and vice-versa. I've tried
>>>>> using one of Shipilev's nightly builds, but it appears jextract is
>>>>> broken in that:
>>>>>
>>>>> [error] jdk.internal.joptsimple.MultipleArgumentsForOptionException:
>>>>> Found multiple arguments for option missing-symbols, but you asked
>>>>> for
>>>>> only one
>>>>> [error] at
>>>>> jdk.internal.opt/jdk.internal.joptsimple.OptionSet.valueOf(OptionSet.java:209)
>>>>> [error] at
>>>>> jdk.internal.opt/jdk.internal.joptsimple.OptionSet.valueOf(OptionSet.java:183)
>>>>> [error] at
>>>>> jdk.jextract/com.sun.tools.jextract.Main.createOptions(Main.java:180)
>>>>> [error] at
>>>>> jdk.jextract/com.sun.tools.jextract.Main.runInternal(Main.java:268)
>>>>> [error] at
>>>>> jdk.jextract/com.sun.tools.jextract.Main.run(Main.java:207)
>>>>> [error] at
>>>>> jdk.jextract/com.sun.tools.jextract.Main$JextractToolProvider.run(Main.java:363)
>>>>> [error] at
>>>>> $a0b469e0da23c15d2805$.$anonfun$runTool$1(build.sbt:85)
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Mark
>>>>>
>>>>> On 2/20/19 4:05 PM, Maurizio Cimadamore wrote:
>>>>>> Well, not solved, but at least isolated :-)
>>>>>>
>>>>>> Probably the fix will come in steps - as this issue goes pretty
>>>>>> deep - however, to just fix the problem with anon structs we
>>>>>> should be able to come up with something quick.
>>>>>>
>>>>>> Maurizio
>>>>>>
>>>>>> On 20/02/2019 12:52, Mark Hammons wrote:
>>>>>>> I’m glad to hear the problem has been solved. I’ll continue the
>>>>>>> project with the workaround I found last night (using java
>>>>>>> written helpers to handle these classes with anonymous inner
>>>>>>> structs since regular inner structs are fine with scala and
>>>>>>> IntelliJ as you mentioned above) and let you know if I hit any
>>>>>>> other bugs.
>>>>>>>
>>>>>>> Thanks for the quick work,
>>>>>>> Mark
>>>>>>>
>>>>>>> Mark Edgar Hammons II - Ingénieur d'études at BioEmergences
>>>>>>> 0603695656
>>>>>>>
>>>>>>>> On 20 Feb 2019, at 12:03, Maurizio Cimadamore
>>>>>>>> <maurizio.cimadamore at oracle.com> wrote:
>>>>>>>>
>>>>>>>> Btw, anon structs are not relevant - this code:
>>>>>>>>
>>>>>>>> struct outer {
>>>>>>>> struct inner {
>>>>>>>> int x;
>>>>>>>> } y;
>>>>>>>> };
>>>>>>>>
>>>>>>>> already has a missing InnerClasses attribute entry in outer.class.
>>>>>>>>
>>>>>>>> Only, in this case the IDE manages to recover the situation,
>>>>>>>> and that has to do with the absence of "$" characters. So, all
>>>>>>>> nested structs (structs inside other structs) seem to be
>>>>>>>> affected by this - but most of the times the IDE is happy (but
>>>>>>>> the bug should be fixed nevertheless).
>>>>>>>>
>>>>>>>> Maurizio
>>>>>>>>
>>>>>>>>
>>>>>>>>> On 20/02/2019 10:52, Maurizio Cimadamore wrote:
>>>>>>>>> I found the issue. Consider this example:
>>>>>>>>>
>>>>>>>>> $ cat test.h
>>>>>>>>>
>>>>>>>>> struct server {
>>>>>>>>> struct {
>>>>>>>>> int x;
>>>>>>>>> int y;
>>>>>>>>> } events;
>>>>>>>>> };
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> in the classfile for test$server.class, jextract is omitting
>>>>>>>>> an inner class attribute which lists the anon class. That is,
>>>>>>>>> a regularly compiled class (with javac) will feature an inner
>>>>>>>>> class attribute like this:
>>>>>>>>>
>>>>>>>>> InnerClasses:
>>>>>>>>> public static #6= #5 of #15; //
>>>>>>>>> anon$test_h$19=class test$anon$test_h$19 of class test
>>>>>>>>> public static #10= #1 of #15; // server=class
>>>>>>>>> test$server of class test
>>>>>>>>>
>>>>>>>>> Whereas the jextracted class will be like this:
>>>>>>>>>
>>>>>>>>> InnerClasses:
>>>>>>>>> public static #20= #2 of #19; // server=class
>>>>>>>>> test$server of class test
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> In other words, jextract is failing to mention, in the
>>>>>>>>> InnerClasses attributes, all _uses_ of nested classes - which
>>>>>>>>> creates problem to all tools. The classfile generated by
>>>>>>>>> jextract does not behave according to the JVM specification,
>>>>>>>>> which mandates that whenever there's a constant pool entry
>>>>>>>>> modelling an inner class, a corresponding entry to the
>>>>>>>>> InnerClasses attribute should be added.
>>>>>>>>>
>>>>>>>>> Thanks for the report.
>>>>>>>>>
>>>>>>>>> Maurizio
>>>>>>>>>
>>>>>>>>>> On 20/02/2019 10:19, Maurizio Cimadamore wrote:
>>>>>>>>>> Hi Mark,
>>>>>>>>>> if my hunch is correct, I believe attempting to reproduce
>>>>>>>>>> with Java code might be impossible.
>>>>>>>>>>
>>>>>>>>>> The problem has to do with the naming scheme AND something in
>>>>>>>>>> the bytecode that jextract generates which seems to confuse
>>>>>>>>>> IDE and other tools.
>>>>>>>>>>
>>>>>>>>>> Redundant signature attributes (such as the one pointed out
>>>>>>>>>> in my other email) might be a part of the puzzle.
>>>>>>>>>>
>>>>>>>>>> Rather than attempting to reproduce with Java - it would be
>>>>>>>>>> more helpful, I think, to try to come up with a simple header
>>>>>>>>>> which exhibits the same issues when programming against it. I
>>>>>>>>>> plan to do that next and see if I'm successful. Hopefully I
>>>>>>>>>> just need an anon struct.
>>>>>>>>>>
>>>>>>>>>> Maurizio
>>>>>>>>>>
>>>>>>>>>>> On 20/02/2019 01:59, Mark Hammons wrote:
>>>>>>>>>>> Thanks Maurizio,
>>>>>>>>>>>
>>>>>>>>>>> I have been trying to reproduce this on my own and I cannot
>>>>>>>>>>> get the same behavior. An inner class of a struct with a
>>>>>>>>>>> name like anon$backend_h$555 works fine. In the case of
>>>>>>>>>>> every anonymous class in this code, scala can't handle it,
>>>>>>>>>>> nor can intellij. I've attached the javap of the other
>>>>>>>>>>> anonymous struct bindings I've found as well as their host
>>>>>>>>>>> classes. I hope this helps determine the issue.
>>>>>>>>>>>
>>>>>>>>>>> One thing I've noticed in the class files I've given you is
>>>>>>>>>>> they all end with something like this:
>>>>>>>>>>>
>>>>>>>>>>> Signature: #3 //
>>>>>>>>>>> Ljava/lang/Object;Ljava/foreign/memory/Struct<Lwlroots/backend_headers/session$anon$session_h$751;>;
>>>>>>>>>>> RuntimeVisibleAnnotations:
>>>>>>>>>>> 0: #8(#9=s#10,#11=I#12,#13=I#14)
>>>>>>>>>>> java.foreign.annotations.NativeLocation(
>>>>>>>>>>> file="/usr/include/wlr/backend/session.h"
>>>>>>>>>>> line=43
>>>>>>>>>>> column=2
>>>>>>>>>>> )
>>>>>>>>>>> 1: #15(#16=s#17)
>>>>>>>>>>> java.foreign.annotations.NativeStruct(
>>>>>>>>>>> value="[${wl_signal}(destroy)](anon$session_h$751)"
>>>>>>>>>>> )
>>>>>>>>>>>
>>>>>>>>>>> I dunno if that's the issue or not, I'm not familiar enough
>>>>>>>>>>> with bytecode to know that. However, I haven't been able to
>>>>>>>>>>> create an innerclass with the anon$backend_h$114 naming
>>>>>>>>>>> scheme that has that kind of ending information. I'll try
>>>>>>>>>>> some more tomorrow and see if I can't produce a java example
>>>>>>>>>>> that produces the same issues.
>>>>>>>>>>>
>>>>>>>>>>> Mark
>>>>>>>>>>>
>>>>>>>>>>>> On 2/20/19 2:14 AM, Maurizio Cimadamore wrote:
>>>>>>>>>>>> The signature attributes look in order - actually there are
>>>>>>>>>>>> some attributes which are not even needed, as in this case:
>>>>>>>>>>>>
>>>>>>>>>>>> public abstract wlroots.backend$anon$backend_h$444
>>>>>>>>>>>> events$get();
>>>>>>>>>>>> descriptor: ()Lwlroots/backend$anon$backend_h$444;
>>>>>>>>>>>> flags: (0x0401) ACC_PUBLIC, ACC_ABSTRACT
>>>>>>>>>>>> Signature: #36 // ()Lwlroots/backend$anon$backend_h$444;
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> It seems like the code generator always emits a signature...
>>>>>>>>>>>>
>>>>>>>>>>>> What I suspect is that the tools are confused by the '$'
>>>>>>>>>>>> sign, and they are tricked into thinking that "444" is an
>>>>>>>>>>>> inner class of "backend_h".
>>>>>>>>>>>>
>>>>>>>>>>>> I will do some more experiments of this though.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks
>>>>>>>>>>>> Maurizio
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> On 20/02/2019 00:28, Mark Hammons wrote:
>>>>>>>>>>>>> Sure,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I'd like to note one thing quickly. Before I was
>>>>>>>>>>>>> extracting the jar with an archive extracter and calling
>>>>>>>>>>>>> javap on the raw classfiles. Just now I tried "javap -p -v
>>>>>>>>>>>>> -cp wlroots.jar wlroots.backend.anon$backend_h$444" and it
>>>>>>>>>>>>> couldn't find the class, but "javap -p -v -cp wlroots.jar
>>>>>>>>>>>>> wlroots.backend.anon.backend_h.444" had it able to find
>>>>>>>>>>>>> the class in question. Ditto for "javap -p -v -cp
>>>>>>>>>>>>> wlroots.jar
>>>>>>>>>>>>> wlroots.backend_headers.session.anon.session_h.751". I'm
>>>>>>>>>>>>> attaching my wlroots.jar along with the javap of
>>>>>>>>>>>>> wlr_backend.class.
>>>>>>>>>>>>>
>>>>>>>>>>>>> This seems to be a recurring issue with the anonymous
>>>>>>>>>>>>> structs generated by jextract.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Mark
>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 2/20/19 1:04 AM, Maurizio Cimadamore wrote:
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> can you please also post the dump of the 'wlr_backend'
>>>>>>>>>>>>>> struct?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Maurizio
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 19/02/2019 23:56, Mark Hammons wrote:
>>>>>>>>>>>>>>> Here's the javap dump. Tell me if there's anything else
>>>>>>>>>>>>>>> I can get you.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Mark
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 2/20/19 12:44 AM, Maurizio Cimadamore wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On 19/02/2019 23:29, Jorn Vernee wrote:
>>>>>>>>>>>>>>>>> I tried to reproduce the problem without using
>>>>>>>>>>>>>>>>> jextract as well, but was not able to, so the bug
>>>>>>>>>>>>>>>>> seems to be up to a particularity of the classes
>>>>>>>>>>>>>>>>> generated by jextract. I'm not sure if the bug should
>>>>>>>>>>>>>>>>> be attributed to jextract or IntelliJ though.
>>>>>>>>>>>>>>>> I wonder if this could be due to some missing (or, most
>>>>>>>>>>>>>>>> likely, just bad) Signature attributes in the resulting
>>>>>>>>>>>>>>>> classfile?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Would you be able to post a 'javap -v -p' dump of the
>>>>>>>>>>>>>>>> classfile, so that we can take a look at it and look
>>>>>>>>>>>>>>>> for anomalies?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Maurizio
>>>>>>>>>>>>>>>>
More information about the panama-dev
mailing list