hsail gate commands

Wed Jan 15 13:11:21 PST 2014

Hi Tom,

A quick glance through the HSAIL C++ layer seems to indicate that it should be able to lazily initialize the HSAIL library linkage (currently done in gnu::Hsail::probe_linkage()). This means it should be deferrable until after the Java part of the Okra library has had a chance to extract the native library file (and hsailasm executable) into the file system. Then you would call System.load() to load the library. There’s still the problem of how to communicate the absolute path of the native library to the os::dll_load call. There may be a more elegant solution, but I’d suggest using the JNI_OnLoad[1] mechanism. Inside the JNI_OnLoad function in the Okra native library, you can set an environment variable (e.g., OKRA_DLL_PATH) which gnu::Hsail::probe_linkage can then use. Note that the environment variable has to be set this way as there’s no way to set an environment variable from Java.

By the time you finish all this hackery, you may start thinking that simply porting the HSAIL simulator/toolchain to Java might be easier ;-)

[1] http://docs.oracle.com/javase/1.5.0/docs/guide/jni/spec/invocation.html#JNI_OnLoad

On Jan 15, 2014, at 9:05 PM, Deneau, Tom <tom.deneau at amd.com> wrote:

> Doug --
> 
> One complication I saw here.
> The native library file is used both from the java side (using loadLibrary)
> and from the hotspot side (using os::dll_load followed by looking up the specific functions it needs).
> And the hotspot attempt to load happens first (from gpu::init()).
> 
> So anything the java side would have done with resources in the .jar file and temp directories
> wouldn't have been done yet.
> 
> -- Tom
> 
> 
>> -----Original Message-----
>> From: Doug Simon [mailto:doug.simon at oracle.com]
>> Sent: Tuesday, January 14, 2014 3:27 AM
>> To: Deneau, Tom
>> Cc: graal-dev at openjdk.java.net
>> Subject: Re: hsail gate commands
>> 
>> 
>> On Jan 13, 2014, at 8:52 PM, Deneau, Tom <tom.deneau at amd.com> wrote:
>> 
>>> Doug --
>>> 
>>> Using the techniques in
>>> http://frommyplayground.com/how-to-load-native-jni-library-from-jar/
>>> that you sent, would it be acceptable to leave hsailasm as a separate
>> executable which gets unjarred into the system temporary directory and
>> then executed from there using an absolute pathname?
>> 
>> Sounds feasible and reasonable to me.
>> 
>>> 
>>> -- Tom
>>> 
>>> 
>>> 
>>>> -----Original Message-----
>>>> From: Doug Simon [mailto:doug.simon at oracle.com]
>>>> Sent: Sunday, January 12, 2014 2:29 PM
>>>> To: Deneau, Tom
>>>> Subject: Re: hsail gate commands
>>>> 
>>>> 
>>>> On Jan 9, 2014, at 9:50 PM, Deneau, Tom <tom.deneau at amd.com> wrote:
>>>> 
>>>>> Doug --
>>>>> 
>>>>> We'll take a look at your suggestion.
>>>> 
>>>> Great.
>>>> 
>>>> BTW, you guys may want to bring
>>>> https://wiki.openjdk.java.net/display/Sumatra/The+HSAIL+Simulator
>>>> more up to date at some point. At least you can remove the "(webrev
>>>> under review)" qualifications.
>>>> 
>>>> -Doug
>>>> 
>>>>>> -----Original Message-----
>>>>>> From: Doug Simon [mailto:doug.simon at oracle.com]
>>>>>> Sent: Thursday, January 09, 2014 1:38 PM
>>>>>> To: Deneau, Tom
>>>>>> Cc: graal-dev at openjdk.java.net
>>>>>> Subject: Re: hsail gate commands
>>>>>> 
>>>>>> 
>>>>>> On Jan 9, 2014, at 6:51 PM, Tom Rodriguez
>>>>>> <tom.rodriguez at oracle.com>
>>>>>> wrote:
>>>>>> 
>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> It would still be great if there would be some way to
>>>>>>>>>> incorporate
>>>>>> the hsail simulator into the gate...
>>>>>>>>> 
>>>>>>>>> A good start would be if you could provide us with a patch for
>>>>>>>>> `mx',
>>>>>> that adds a command which sets up the HSAIL simulator properly.
>>>>>> Currently our gate is running on Linux, but this might change in
>>>>>> the future, so keep in mind that this should potentially work on
>>>>>> all platforms (Linux, MacOSX, Windows).
>>>>>>>>> 
>>>>>>>>> Our gate server is basically executing `mx gate' to validate
>>>>>> changes. That includes building, bootstrap tests, unittest, etc.
>>>>>> If that command was successful on our gate server, the changes are
>>>>>> accepted and pushed to the repository.  Having the `mx gate'
>>>>>> command, one can reproduce the gate process on his/her local
>> machine.
>>>>>> Therefore it would be important that one can easily and reliable
>>>>>> set up the HSAIL environment if it is part of the gate process.
>>>>>>>> 
>>>>>>>> Given the dependencies for building the simulator[1] and the fact
>>>>>>>> it
>>>>>> only currently runs on linux, I don't think it will be that easy to
>>>>>> come up with an mx command.
>>>>>>> 
>>>>>>> What precompiling an okra distribution for linux that mx can
>>>> download?
>>>>>> 
>>>>>> If Okra used a self contained JNI library, that might just work.
>>>>>> However, the way it is deployed now involves having the hsailasm
>>>>>> executable on your PATH. Also, LD_LIBRARY_PATH has to be setup to
>>>>>> find the JNI library itself.
>>>>>> 
>>>>>> @AMD guys: How hard would it be to make the Okra JNI be self
>>>>>> contained library and include the hsailasm functionality? If that's
>>>>>> possible, you could bundle the JNI library inside the Okra jar
>>>>>> using a technique like
>>>>>> http://frommyplayground.com/how-to-load-native-jni-library-from-jar
>>>>>> /
>>>> to avoid having to set up PATH or LD_LIBRARY_PATH.
>>>>>> 
>>>>>> -Doug
>>>>>> 
>>>>>>>> [1]
>>>>>>>> https://github.com/HSAFoundation/Okra-Interface-to-HSAIL-Simulato
>>>>>>>> r#
>>>>>>>> ok
>>>>>>>> ra-interface-to-hsail-simulator
>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>> From: graal-dev-bounces at openjdk.java.net [mailto:graal-dev-
>>>>>>>>>>> bounces at openjdk.java.net] On Behalf Of Deneau, Tom
>>>>>>>>>>> Sent: Monday, December 16, 2013 11:24 AM
>>>>>>>>>>> To: Doug Simon
>>>>>>>>>>> Cc: graal-dev at openjdk.java.net
>>>>>>>>>>> Subject: RE: hsail gate commands
>>>>>>>>>>> 
>>>>>>>>>>> OK, I see why there is no error when running mx --vm server
>>>>>>>>>>> unittest hsail
>>>>>>>>>>> 
>>>>>>>>>>> At some point (I don't recall why, maybe at Oracle's
>>>>>>>>>>> request??), we put in some code in KernelTester that if it
>>>>>>>>>>> could not find the okra simulator files would just silently
>>>>>>>>>>> not run the tests (which I guess counts as a pass).
>>>>>>>>>>> 
>>>>>>>>>>> No the hsail simulator is not a java app at all.
>>>>>>>>>>> The page at
>>>>>>>>>>> https://wiki.openjdk.java.net/display/Sumatra/The+HSAIL+Simula
>>>>>>>>>>> to r should describe how to build it and what environment
>>>>>>>>>>> variables to set up to use it...
>>>>>>>>>>> 
>>>>>>>>>>> -- Tom
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>> From: Doug Simon [mailto:doug.simon at oracle.com]
>>>>>>>>>>>> Sent: Monday, December 16, 2013 11:02 AM
>>>>>>>>>>>> To: Deneau, Tom
>>>>>>>>>>>> Cc: graal-dev at openjdk.java.net
>>>>>>>>>>>> Subject: Re: hsail gate commands
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> On Dec 16, 2013, at 5:48 PM, Deneau, Tom <tom.deneau at amd.com>
>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> Doug --
>>>>>>>>>>>>> 
>>>>>>>>>>>>> I see.
>>>>>>>>>>>>> I don't understand why the tests would run without error if
>>>>>>>>>>>>> the simulator and associated assembler is missing but...
>>>>>>>>>>>> 
>>>>>>>>>>>> What error are you seeing?
>>>>>>>>>>>> 
>>>>>>>>>>>>> Is there a way we can get the HSAIL simulator into the gate
>>>>>>>>>>>> infrastructure?
>>>>>>>>>>>> 
>>>>>>>>>>>> Is it a pure Java app yet? That would certainly make it
>>>> trivial.
>>>>>>>>>>>> In any case, I'll defer to Bernhard or Gilles to answer this
>>>>>>>>>>>> since they are the maintainers of this infrastructure. Can
>>>>>>>>>>>> you please send instructions on how to install/use/configure
>>>>>>>>>>>> the
>>>>>> simulator.
>>>>>>>>>>>> 
>>>>>>>>>>>> -Doug
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>> From: Doug Simon [mailto:doug.simon at oracle.com]
>>>>>>>>>>>>>> Sent: Monday, December 16, 2013 10:36 AM
>>>>>>>>>>>>>> To: Deneau, Tom
>>>>>>>>>>>>>> Cc: graal-dev at openjdk.java.net
>>>>>>>>>>>>>> Subject: Re: hsail gate commands
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Dec 16, 2013, at 5:21 PM, Deneau, Tom
>>>>>>>>>>>>>> <tom.deneau at amd.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Doug --
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> I noticed in syncing with the trunk as of last Friday that
>>>>>>>>>>>>>>> all of our
>>>>>>>>>>>>>> HSAIL test cases broke.  The cause was some imperfect code
>>>>>>>>>>>>>> in our HSAILAssembler for compare instructions which when
>>>>>>>>>>>>>> presented with an unordered compare could generate code
>>>>>>>>>>>>>> that would not assemble, and was easy to fix.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> But I was wondering how this made it past the gate.
>>>>>>>>>>>>>>> Can you describe what gate commands are used regarding
>>>> hsail?
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> We simply run 'mx gate' which will include running all the
>>>>>>>>>>>>>> HSAIL unit tests.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On my Mac with the latest bits:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> $ mx --vm server unittest hsail executing junit tests
>> now...
>>>>>>>>>>>>>> (107 test classes) JUnit version 4.8
>>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>> 
>>>> 
>> ........................................................................
>>>>>>>>>>>>>> ...........I................I......
>>>>>>>>>>>>>> Time: 5.525
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> OK (105 tests)
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Keep in mind I (and the gate infrastructure) don't have the
>>>>>>>>>>>>>> HSAIL simulator.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> -Doug
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>> 
>>> 
>> 
> 
> 