hsail gate commands

Doug Simon doug.simon at oracle.com
Wed Jan 15 14:07:49 PST 2014 

On Jan 15, 2014, at 10:46 PM, Deneau, Tom <tom.deneau at amd.com> wrote:

> Doug --
> 
> how does the JNI_OnLoad know what path it was loaded from?
> or would the java side have to pass this information down?

The Java side has to pass this down. One way to do it would be to define a static String field in some class in the Okra Java library and assign it the path before calling System.load(). Inside JNI_OnLoad, the JNI interface can be used to find the class and read the field.

-Doug

>> -----Original Message-----
>> From: Doug Simon [mailto:doug.simon at oracle.com]
>> Sent: Wednesday, January 15, 2014 3:11 PM
>> To: Deneau, Tom
>> Cc: graal-dev at openjdk.java.net
>> Subject: Re: hsail gate commands
>> 
>> Hi Tom,
>> 
>> A quick glance through the HSAIL C++ layer seems to indicate that it
>> should be able to lazily initialize the HSAIL library linkage (currently
>> done in gnu::Hsail::probe_linkage()). This means it should be deferrable
>> until after the Java part of the Okra library has had a chance to
>> extract the native library file (and hsailasm executable) into the file
>> system. Then you would call System.load() to load the library. There's
>> still the problem of how to communicate the absolute path of the native
>> library to the os::dll_load call. There may be a more elegant solution,
>> but I'd suggest using the JNI_OnLoad[1] mechanism. Inside the JNI_OnLoad
>> function in the Okra native library, you can set an environment variable
>> (e.g., OKRA_DLL_PATH) which gnu::Hsail::probe_linkage can then use. Note
>> that the environment variable has to be set this way as there's no way
>> to set an environment variable from Java.
>> 
>> By the time you finish all this hackery, you may start thinking that
>> simply porting the HSAIL simulator/toolchain to Java might be easier ;-)
>> 
>> [1]
>> http://docs.oracle.com/javase/1.5.0/docs/guide/jni/spec/invocation.html#
>> JNI_OnLoad
>> 
>> On Jan 15, 2014, at 9:05 PM, Deneau, Tom <tom.deneau at amd.com> wrote:
>> 
>>> Doug --
>>> 
>>> One complication I saw here.
>>> The native library file is used both from the java side (using
>>> loadLibrary) and from the hotspot side (using os::dll_load followed by
>> looking up the specific functions it needs).
>>> And the hotspot attempt to load happens first (from gpu::init()).
>>> 
>>> So anything the java side would have done with resources in the .jar
>>> file and temp directories wouldn't have been done yet.
>>> 
>>> -- Tom
>>> 
>>> 
>>>> -----Original Message-----
>>>> From: Doug Simon [mailto:doug.simon at oracle.com]
>>>> Sent: Tuesday, January 14, 2014 3:27 AM
>>>> To: Deneau, Tom
>>>> Cc: graal-dev at openjdk.java.net
>>>> Subject: Re: hsail gate commands
>>>> 
>>>> 
>>>> On Jan 13, 2014, at 8:52 PM, Deneau, Tom <tom.deneau at amd.com> wrote:
>>>> 
>>>>> Doug --
>>>>> 
>>>>> Using the techniques in
>>>>> http://frommyplayground.com/how-to-load-native-jni-library-from-jar/
>>>>> that you sent, would it be acceptable to leave hsailasm as a
>>>>> separate
>>>> executable which gets unjarred into the system temporary directory
>>>> and then executed from there using an absolute pathname?
>>>> 
>>>> Sounds feasible and reasonable to me.
>>>> 
>>>>> 
>>>>> -- Tom
>>>>> 
>>>>> 
>>>>> 
>>>>>> -----Original Message-----
>>>>>> From: Doug Simon [mailto:doug.simon at oracle.com]
>>>>>> Sent: Sunday, January 12, 2014 2:29 PM
>>>>>> To: Deneau, Tom
>>>>>> Subject: Re: hsail gate commands
>>>>>> 
>>>>>> 
>>>>>> On Jan 9, 2014, at 9:50 PM, Deneau, Tom <tom.deneau at amd.com> wrote:
>>>>>> 
>>>>>>> Doug --
>>>>>>> 
>>>>>>> We'll take a look at your suggestion.
>>>>>> 
>>>>>> Great.
>>>>>> 
>>>>>> BTW, you guys may want to bring
>>>>>> https://wiki.openjdk.java.net/display/Sumatra/The+HSAIL+Simulator
>>>>>> more up to date at some point. At least you can remove the "(webrev
>>>>>> under review)" qualifications.
>>>>>> 
>>>>>> -Doug
>>>>>> 
>>>>>>>> -----Original Message-----
>>>>>>>> From: Doug Simon [mailto:doug.simon at oracle.com]
>>>>>>>> Sent: Thursday, January 09, 2014 1:38 PM
>>>>>>>> To: Deneau, Tom
>>>>>>>> Cc: graal-dev at openjdk.java.net
>>>>>>>> Subject: Re: hsail gate commands
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Jan 9, 2014, at 6:51 PM, Tom Rodriguez
>>>>>>>> <tom.rodriguez at oracle.com>
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> It would still be great if there would be some way to
>>>>>>>>>>>> incorporate
>>>>>>>> the hsail simulator into the gate...
>>>>>>>>>>> 
>>>>>>>>>>> A good start would be if you could provide us with a patch for
>>>>>>>>>>> `mx',
>>>>>>>> that adds a command which sets up the HSAIL simulator properly.
>>>>>>>> Currently our gate is running on Linux, but this might change in
>>>>>>>> the future, so keep in mind that this should potentially work on
>>>>>>>> all platforms (Linux, MacOSX, Windows).
>>>>>>>>>>> 
>>>>>>>>>>> Our gate server is basically executing `mx gate' to validate
>>>>>>>> changes. That includes building, bootstrap tests, unittest, etc.
>>>>>>>> If that command was successful on our gate server, the changes
>>>>>>>> are accepted and pushed to the repository.  Having the `mx gate'
>>>>>>>> command, one can reproduce the gate process on his/her local
>>>> machine.
>>>>>>>> Therefore it would be important that one can easily and reliable
>>>>>>>> set up the HSAIL environment if it is part of the gate process.
>>>>>>>>>> 
>>>>>>>>>> Given the dependencies for building the simulator[1] and the
>>>>>>>>>> fact it
>>>>>>>> only currently runs on linux, I don't think it will be that easy
>>>>>>>> to come up with an mx command.
>>>>>>>>> 
>>>>>>>>> What precompiling an okra distribution for linux that mx can
>>>>>> download?
>>>>>>>> 
>>>>>>>> If Okra used a self contained JNI library, that might just work.
>>>>>>>> However, the way it is deployed now involves having the hsailasm
>>>>>>>> executable on your PATH. Also, LD_LIBRARY_PATH has to be setup to
>>>>>>>> find the JNI library itself.
>>>>>>>> 
>>>>>>>> @AMD guys: How hard would it be to make the Okra JNI be self
>>>>>>>> contained library and include the hsailasm functionality? If
>>>>>>>> that's possible, you could bundle the JNI library inside the Okra
>>>>>>>> jar using a technique like
>>>>>>>> http://frommyplayground.com/how-to-load-native-jni-library-from-j
>>>>>>>> ar
>>>>>>>> /
>>>>>> to avoid having to set up PATH or LD_LIBRARY_PATH.
>>>>>>>> 
>>>>>>>> -Doug
>>>>>>>> 
>>>>>>>>>> [1]
>>>>>>>>>> https://github.com/HSAFoundation/Okra-Interface-to-HSAIL-Simula
>>>>>>>>>> to
>>>>>>>>>> r#
>>>>>>>>>> ok
>>>>>>>>>> ra-interface-to-hsail-simulator
>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>> From: graal-dev-bounces at openjdk.java.net [mailto:graal-dev-
>>>>>>>>>>>>> bounces at openjdk.java.net] On Behalf Of Deneau, Tom
>>>>>>>>>>>>> Sent: Monday, December 16, 2013 11:24 AM
>>>>>>>>>>>>> To: Doug Simon
>>>>>>>>>>>>> Cc: graal-dev at openjdk.java.net
>>>>>>>>>>>>> Subject: RE: hsail gate commands
>>>>>>>>>>>>> 
>>>>>>>>>>>>> OK, I see why there is no error when running mx --vm server
>>>>>>>>>>>>> unittest hsail
>>>>>>>>>>>>> 
>>>>>>>>>>>>> At some point (I don't recall why, maybe at Oracle's
>>>>>>>>>>>>> request??), we put in some code in KernelTester that if it
>>>>>>>>>>>>> could not find the okra simulator files would just silently
>>>>>>>>>>>>> not run the tests (which I guess counts as a pass).
>>>>>>>>>>>>> 
>>>>>>>>>>>>> No the hsail simulator is not a java app at all.
>>>>>>>>>>>>> The page at
>>>>>>>>>>>>> https://wiki.openjdk.java.net/display/Sumatra/The+HSAIL+Simu
>>>>>>>>>>>>> la to r should describe how to build it and what environment
>>>>>>>>>>>>> variables to set up to use it...
>>>>>>>>>>>>> 
>>>>>>>>>>>>> -- Tom
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>> From: Doug Simon [mailto:doug.simon at oracle.com]
>>>>>>>>>>>>>> Sent: Monday, December 16, 2013 11:02 AM
>>>>>>>>>>>>>> To: Deneau, Tom
>>>>>>>>>>>>>> Cc: graal-dev at openjdk.java.net
>>>>>>>>>>>>>> Subject: Re: hsail gate commands
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Dec 16, 2013, at 5:48 PM, Deneau, Tom
>>>>>>>>>>>>>> <tom.deneau at amd.com>
>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Doug --
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> I see.
>>>>>>>>>>>>>>> I don't understand why the tests would run without error
>>>>>>>>>>>>>>> if the simulator and associated assembler is missing
>> but...
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> What error are you seeing?
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Is there a way we can get the HSAIL simulator into the
>>>>>>>>>>>>>>> gate
>>>>>>>>>>>>>> infrastructure?
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Is it a pure Java app yet? That would certainly make it
>>>>>> trivial.
>>>>>>>>>>>>>> In any case, I'll defer to Bernhard or Gilles to answer
>>>>>>>>>>>>>> this since they are the maintainers of this infrastructure.
>>>>>>>>>>>>>> Can you please send instructions on how to
>>>>>>>>>>>>>> install/use/configure the
>>>>>>>> simulator.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> -Doug
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>>>> From: Doug Simon [mailto:doug.simon at oracle.com]
>>>>>>>>>>>>>>>> Sent: Monday, December 16, 2013 10:36 AM
>>>>>>>>>>>>>>>> To: Deneau, Tom
>>>>>>>>>>>>>>>> Cc: graal-dev at openjdk.java.net
>>>>>>>>>>>>>>>> Subject: Re: hsail gate commands
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On Dec 16, 2013, at 5:21 PM, Deneau, Tom
>>>>>>>>>>>>>>>> <tom.deneau at amd.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Doug --
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> I noticed in syncing with the trunk as of last Friday
>>>>>>>>>>>>>>>>> that all of our
>>>>>>>>>>>>>>>> HSAIL test cases broke.  The cause was some imperfect
>>>>>>>>>>>>>>>> code in our HSAILAssembler for compare instructions which
>>>>>>>>>>>>>>>> when presented with an unordered compare could generate
>>>>>>>>>>>>>>>> code that would not assemble, and was easy to fix.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> But I was wondering how this made it past the gate.
>>>>>>>>>>>>>>>>> Can you describe what gate commands are used regarding
>>>>>> hsail?
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> We simply run 'mx gate' which will include running all
>>>>>>>>>>>>>>>> the HSAIL unit tests.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On my Mac with the latest bits:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> $ mx --vm server unittest hsail executing junit tests
>>>> now...
>>>>>>>>>>>>>>>> (107 test classes) JUnit version 4.8
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>> 
>>>>>> 
>>>> 
>> ........................................................................
>>>>>>>>>>>>>>>> ...........I................I......
>>>>>>>>>>>>>>>> Time: 5.525
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> OK (105 tests)
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Keep in mind I (and the gate infrastructure) don't have
>>>>>>>>>>>>>>>> the HSAIL simulator.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> -Doug
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>> 
>>> 
>> 
> 
>

More information about the graal-dev mailing list