Finished part 1 of the Wayland McWayface tutorial of Drew DeVault - Issue with spi toolprovider interface and jextract

Mon Feb 25 00:40:52 UTC 2019

I just retried with the debug option and still nothing. I rewrote my 
runTool function to look like this:

def runTool(name: String, arguments: Seq[String]): Either[String,String] = {
   System.setProperty("jextract.debug", "true")

   val maybeTool: Option[ToolProvider] = {
     val _tool = ToolProvider.findFirst(name)
     if(_tool.isPresent) {
       Some(_tool.get())
     } else {
       None
     }
   }

   val result = for(tool <- maybeTool) yield {
     println(s"running ${tool.name()}")
     val stdOut = new ByteArrayOutputStream()
     val errOut = new ByteArrayOutputStream()
     val code = tool.run(new PrintWriter(System.out), new 
PrintWriter(System.err), arguments: _*)
     (code, new String(stdOut.toByteArray), new String(errOut.toByteArray))
   }

   result
     .toRight(s"Could not find tool $name in your java development 
environment")
     .flatMap{ case (code,ret,err) =>
       if(ret.contains("Error:") || err.nonEmpty || code != 0) {
         Left(s"failure with code $code: ${ret + err}")
       } else {
         println(s"return value: $ret")
         println(s"error value: $ret")
         Right(ret -> "")
       }
     }
     .map(_._1)
}

And this is the result I get now:

[info] generating java bindings for wlroots using jextract
running jextract
[info] issuing command jextract /usr/include/wlr/types/wlr_output.h 
/usr/include/wlr/backend.h /usr/include/wlr/render/wlr_renderer.h -m 
/usr/include/wlr/backend=wlroots.backend_headers -m 
/usr/include/bits/types=usr.include.bits.type_headers -I 
/usr/include/wlr -I /usr/include/wayland -I /usr/include/pixman-1 -C 
"-DWLR_USE_UNSTABLE" -L /usr/lib64 --record-library-path -l wlroots -t 
wlroots -o /home/mhammons/Documents/MyProjects/waylandTest/lib/wlroots2.jar
[error] java.lang.RuntimeException: failure with code 4:

So at least I know the command is failing, but still no output as to why.

On 2/25/19 1:27 AM, Mark Hammons wrote:
> Yeah, I put System.out/System.err in there to see if maybe I was 
> filtering out error output or something accidentally. I'll try the 
> jextract debug property.
>
> Thanks for the tip on that function. I didn't realize that was for 
> passing in arrays thanks to the function signature.
>
> Mark
>
> On 2/24/19 10:20 PM, Jorn Vernee wrote:
>> Hi Mark,
>>
>>> 3) Yeah, I allocated a c array using the scope and filled it with my
>>> data. It just feels like something that should be achievable by
>>> passing in a java array to my scope.
>>
>> We have Scope::allocateArray(LayoutType<X>, Object); for that. e.g.:
>>
>> ```
>>     try (Scope scope = Scope.globalScope().fork()) {
>>         Array<Integer> nativeArr = 
>> scope.allocateArray(NativeTypes.INT32, new int[]{ 1, 2, 3 });
>> nativeArr.iterate().map(Pointer::get).forEach(out::println); // 1 2 3
>>     }
>> ```
>>
>>> 4) That code looks like it's testing jextract. I'm not sure what''s
>>> going on then, cause when I ran my tool runner method with jextract
>>> and the appropriate arguments (and I tested the generated arguments by
>>> copy pasting them to a standard command line), jextract returned
>>> nothing and exited very quickly.
>>
>> I'm not sure what's going on there. I can use the ToolProvider 
>> interface to run jextract successfully e.g.:
>>
>> ```
>>     private static final ToolProvider JEXTRACT = 
>> ToolProvider.findFirst("jextract").get();
>>
>>     public static void main(String[] args) {
>>         test();
>>     }
>>
>>     public static void test() {
>>         Path dir = Paths.get("J:\\WS\\jextract\\test");
>>         Path jar = Paths.get("Test.jar");
>>         Path testH = dir.resolve("test1.h");
>>
>>         System.setProperty("jextract.debug", "true");
>>         int result = JEXTRACT.run(System.out, System.err,
>>                 "-o", jar.toString(),
>>                 "--no-locations",
>>                 "-t", "org",
>>                 "--",
>>                 testH.toString());
>>         System.out.println(result);
>>     }
>> ```
>>
>> Which still works today and produces a jar.
>>
>> The only thing I could say looking at your build script [1] is that 
>> you seem to be printing out the jextract results to System.out and 
>> System.err but then returning 2 empty Strings instead:
>>
>> ```
>>     val result = for(tool <- maybeTool) yield {
>>         println(s"running ${tool.name()}")
>>         val stdOut = new ByteArrayOutputStream()
>>         val errOut = new ByteArrayOutputStream()
>>         tool.run(new PrintWriter(System.out), new 
>> PrintWriter(System.err), arguments: _*)
>>         (new String(stdOut.toByteArray), new String(errOut.toByteArray))
>>     }
>> ```
>>
>> But, I suppose that is just for testing?
>>
>> Also, like in my snippet, you could try enabling jextract debug 
>> output with `System.setProperty("jextract.debug", "true");` and see 
>> if anything is printed then.
>>
>> Jorn
>>
>> [1] : 
>> https://github.com/markehammons/Wayland-McWayface_JVM-edition/blob/master/build.sbt
>>
>> Mark Hammons schreef op 2019-02-24 19:31:
>>> Hi Maurizio,
>>>
>>> Regarding 1) I would try to fork the global scope to create pointers
>>> for my callbacks. However, since the object that was taking the
>>> callbacks was created by a wlr_backend_autocreate foreign function
>>> call, I guess the scopes were incompatible and it would not accept the
>>> pointers from the forked scope. You can give it a shot with my
>>> repository. Just pass in scope.fork to anywhere I'm passing in a scope
>>> and you should see the access violation messages.
>>>
>>> 2) I'm glad to hear this has been completely fixed, and I look forward
>>> to the next release. Will that be coming soon?
>>>
>>> 3) Yeah, I allocated a c array using the scope and filled it with my
>>> data. It just feels like something that should be achievable by
>>> passing in a java array to my scope.
>>>
>>> 4) That code looks like it's testing jextract. I'm not sure what''s
>>> going on then, cause when I ran my tool runner method with jextract
>>> and the appropriate arguments (and I tested the generated arguments by
>>> copy pasting them to a standard command line), jextract returned
>>> nothing and exited very quickly.
>>>
>>> Thanks,
>>>
>>> Mark
>>>
>>> On 2/24/19 1:30 AM, Maurizio Cimadamore wrote:
>>>>
>>>> On 23/02/2019 23:26, Mark Hammons wrote:
>>>>> Hi all,
>>>>>
>>>>> I've finally written a working implementation of the first part of 
>>>>> Drew DeVault's tutorial, and I wanted to put it here first to get 
>>>>> your feedback
>>>>>
>>>>> https://github.com/markehammons/Wayland-McWayface_JVM-edition/tree/Part1 
>>>>>
>>>> Thanks! I'll look into that in more details.
>>>>>
>>>>> I've tried using the new forked scopes in the recent release, but 
>>>>> I frequently hit issues of being unable to use pointers I need to 
>>>>> use. This usually happens with the allocated callbacks, so I 
>>>>> assume I could merge said callback's pointers into the scope of 
>>>>> what they're being assigned to, but I haven't tried that yet. The 
>>>>> struct issue I reported earlier continues to plague me, and has 
>>>>> resulted in me writing a few workaround classes in java. Also, I 
>>>>> allocated a c_array to communicate with a function call but is 
>>>>> there no way to just pass in a regular java array at this time?
>>>>
>>>> So:
>>>>
>>>> 1) I'd like to know more about the issues with pointer/callback - 
>>>> e.g. what issues have you encountered; if it mostly happens with 
>>>> callbacks, there could be some lurking issue with scoping of these 
>>>> callback pointers?
>>>>
>>>> 2) the struct issue has been fixed - you'll see that situation 
>>>> improved on the next binary build
>>>>
>>>> 3) if you want to pass an array to a native function you have to 
>>>> create a native array; while in the future we might provide 
>>>> 'civilization' options to emit signatures closer to what a Java 
>>>> developer would expect, that's not the goal now. You should be able 
>>>> to allocate a native array from Java array with relative ease.
>>>>
>>>>>
>>>>> Finally, and this is the biggest issue, I cannot get jextract 
>>>>> working via the spi.ToolProvider interface. If you look at the 
>>>>> build.sbt in the root of my project, I have defined a task binding 
>>>>> for jextract to be called and configured by sbt without having to 
>>>>> call outside of the JVM. When I tried this with jlink 
>>>>> (https://gist.github.com/markehammons/42d75709e060625f1a663b442842b461) 
>>>>> it worked fine, and spi.ToolProvider says it's finding jextract, 
>>>>> jextract is just not doing anything. I'm guessing the jextract 
>>>>> tool isn't hooked into ToolProvider yet?
>>>>
>>>> This surprises me - I'll leave this to Sundar - but I seem to 
>>>> recall that jextract was indeed hooked up and that we're even using 
>>>> that capability for testing? The following was pushed last year
>>>>
>>>> http://cr.openjdk.java.net/~sundar/jextract_tool_provider_testng/webrev.00/raw_files/new/test/jdk/com/sun/tools/jextract/JextractToolProviderTest.java 
>>>> Maurizio
>>>>
>>>>
>>>>>
>>>>> Anyway, tell me if you have any suggestions to improve my usage of 
>>>>> the foreign APIs in this project. And especially tell me if I can 
>>>>> get jextract working through the ToolProvider interface. I'd love 
>>>>> to start developing sbt and mill plugins for projects to bind 
>>>>> native code with, but I can't till that gets worked out.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Mark
>>>>>
>>>>>