hsail gate commands

Wed Jan 15 14:15:05 PST 2014

dladdr can also be used to figure it out directly.

tom

On Jan 15, 2014, at 2:07 PM, Doug Simon <doug.simon at oracle.com> wrote:

> 
> On Jan 15, 2014, at 10:46 PM, Deneau, Tom <tom.deneau at amd.com> wrote:
> 
>> Doug --
>> 
>> how does the JNI_OnLoad know what path it was loaded from?
>> or would the java side have to pass this information down?
> 
> The Java side has to pass this down. One way to do it would be to define a static String field in some class in the Okra Java library and assign it the path before calling System.load(). Inside JNI_OnLoad, the JNI interface can be used to find the class and read the field.
> 
> -Doug
> 
>>> -----Original Message-----
>>> From: Doug Simon [mailto:doug.simon at oracle.com]
>>> Sent: Wednesday, January 15, 2014 3:11 PM
>>> To: Deneau, Tom
>>> Cc: graal-dev at openjdk.java.net
>>> Subject: Re: hsail gate commands
>>> 
>>> Hi Tom,
>>> 
>>> A quick glance through the HSAIL C++ layer seems to indicate that it
>>> should be able to lazily initialize the HSAIL library linkage (currently
>>> done in gnu::Hsail::probe_linkage()). This means it should be deferrable
>>> until after the Java part of the Okra library has had a chance to
>>> extract the native library file (and hsailasm executable) into the file
>>> system. Then you would call System.load() to load the library. There's
>>> still the problem of how to communicate the absolute path of the native
>>> library to the os::dll_load call. There may be a more elegant solution,
>>> but I'd suggest using the JNI_OnLoad[1] mechanism. Inside the JNI_OnLoad
>>> function in the Okra native library, you can set an environment variable
>>> (e.g., OKRA_DLL_PATH) which gnu::Hsail::probe_linkage can then use. Note
>>> that the environment variable has to be set this way as there's no way
>>> to set an environment variable from Java.
>>> 
>>> By the time you finish all this hackery, you may start thinking that
>>> simply porting the HSAIL simulator/toolchain to Java might be easier ;-)
>>> 
>>> [1]
>>> http://docs.oracle.com/javase/1.5.0/docs/guide/jni/spec/invocation.html#
>>> JNI_OnLoad
>>> 
>>> On Jan 15, 2014, at 9:05 PM, Deneau, Tom <tom.deneau at amd.com> wrote:
>>> 
>>>> Doug --
>>>> 
>>>> One complication I saw here.
>>>> The native library file is used both from the java side (using
>>>> loadLibrary) and from the hotspot side (using os::dll_load followed by
>>> looking up the specific functions it needs).
>>>> And the hotspot attempt to load happens first (from gpu::init()).
>>>> 
>>>> So anything the java side would have done with resources in the .jar
>>>> file and temp directories wouldn't have been done yet.
>>>> 
>>>> -- Tom
>>>> 
>>>> 
>>>>> -----Original Message-----
>>>>> From: Doug Simon [mailto:doug.simon at oracle.com]
>>>>> Sent: Tuesday, January 14, 2014 3:27 AM
>>>>> To: Deneau, Tom
>>>>> Cc: graal-dev at openjdk.java.net
>>>>> Subject: Re: hsail gate commands
>>>>> 
>>>>> 
>>>>> On Jan 13, 2014, at 8:52 PM, Deneau, Tom <tom.deneau at amd.com> wrote:
>>>>> 
>>>>>> Doug --
>>>>>> 
>>>>>> Using the techniques in
>>>>>> http://frommyplayground.com/how-to-load-native-jni-library-from-jar/
>>>>>> that you sent, would it be acceptable to leave hsailasm as a
>>>>>> separate
>>>>> executable which gets unjarred into the system temporary directory
>>>>> and then executed from there using an absolute pathname?
>>>>> 
>>>>> Sounds feasible and reasonable to me.
>>>>> 
>>>>>> 
>>>>>> -- Tom
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>> -----Original Message-----
>>>>>>> From: Doug Simon [mailto:doug.simon at oracle.com]
>>>>>>> Sent: Sunday, January 12, 2014 2:29 PM
>>>>>>> To: Deneau, Tom
>>>>>>> Subject: Re: hsail gate commands
>>>>>>> 
>>>>>>> 
>>>>>>> On Jan 9, 2014, at 9:50 PM, Deneau, Tom <tom.deneau at amd.com> wrote:
>>>>>>> 
>>>>>>>> Doug --
>>>>>>>> 
>>>>>>>> We'll take a look at your suggestion.
>>>>>>> 
>>>>>>> Great.
>>>>>>> 
>>>>>>> BTW, you guys may want to bring
>>>>>>> https://wiki.openjdk.java.net/display/Sumatra/The+HSAIL+Simulator
>>>>>>> more up to date at some point. At least you can remove the "(webrev
>>>>>>> under review)" qualifications.
>>>>>>> 
>>>>>>> -Doug
>>>>>>> 
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: Doug Simon [mailto:doug.simon at oracle.com]
>>>>>>>>> Sent: Thursday, January 09, 2014 1:38 PM
>>>>>>>>> To: Deneau, Tom
>>>>>>>>> Cc: graal-dev at openjdk.java.net
>>>>>>>>> Subject: Re: hsail gate commands
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On Jan 9, 2014, at 6:51 PM, Tom Rodriguez
>>>>>>>>> <tom.rodriguez at oracle.com>
>>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> It would still be great if there would be some way to
>>>>>>>>>>>>> incorporate
>>>>>>>>> the hsail simulator into the gate...
>>>>>>>>>>>> 
>>>>>>>>>>>> A good start would be if you could provide us with a patch for
>>>>>>>>>>>> `mx',
>>>>>>>>> that adds a command which sets up the HSAIL simulator properly.
>>>>>>>>> Currently our gate is running on Linux, but this might change in
>>>>>>>>> the future, so keep in mind that this should potentially work on
>>>>>>>>> all platforms (Linux, MacOSX, Windows).
>>>>>>>>>>>> 
>>>>>>>>>>>> Our gate server is basically executing `mx gate' to validate
>>>>>>>>> changes. That includes building, bootstrap tests, unittest, etc.
>>>>>>>>> If that command was successful on our gate server, the changes
>>>>>>>>> are accepted and pushed to the repository.  Having the `mx gate'
>>>>>>>>> command, one can reproduce the gate process on his/her local
>>>>> machine.
>>>>>>>>> Therefore it would be important that one can easily and reliable
>>>>>>>>> set up the HSAIL environment if it is part of the gate process.
>>>>>>>>>>> 
>>>>>>>>>>> Given the dependencies for building the simulator[1] and the
>>>>>>>>>>> fact it
>>>>>>>>> only currently runs on linux, I don't think it will be that easy
>>>>>>>>> to come up with an mx command.
>>>>>>>>>> 
>>>>>>>>>> What precompiling an okra distribution for linux that mx can
>>>>>>> download?
>>>>>>>>> 
>>>>>>>>> If Okra used a self contained JNI library, that might just work.
>>>>>>>>> However, the way it is deployed now involves having the hsailasm
>>>>>>>>> executable on your PATH. Also, LD_LIBRARY_PATH has to be setup to
>>>>>>>>> find the JNI library itself.
>>>>>>>>> 
>>>>>>>>> @AMD guys: How hard would it be to make the Okra JNI be self
>>>>>>>>> contained library and include the hsailasm functionality? If
>>>>>>>>> that's possible, you could bundle the JNI library inside the Okra
>>>>>>>>> jar using a technique like
>>>>>>>>> http://frommyplayground.com/how-to-load-native-jni-library-from-j
>>>>>>>>> ar
>>>>>>>>> /
>>>>>>> to avoid having to set up PATH or LD_LIBRARY_PATH.
>>>>>>>>> 
>>>>>>>>> -Doug
>>>>>>>>> 
>>>>>>>>>>> [1]
>>>>>>>>>>> https://github.com/HSAFoundation/Okra-Interface-to-HSAIL-Simula
>>>>>>>>>>> to
>>>>>>>>>>> r#
>>>>>>>>>>> ok
>>>>>>>>>>> ra-interface-to-hsail-simulator
>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>> From: graal-dev-bounces at openjdk.java.net [mailto:graal-dev-
>>>>>>>>>>>>>> bounces at openjdk.java.net] On Behalf Of Deneau, Tom
>>>>>>>>>>>>>> Sent: Monday, December 16, 2013 11:24 AM
>>>>>>>>>>>>>> To: Doug Simon
>>>>>>>>>>>>>> Cc: graal-dev at openjdk.java.net
>>>>>>>>>>>>>> Subject: RE: hsail gate commands
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> OK, I see why there is no error when running mx --vm server
>>>>>>>>>>>>>> unittest hsail
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> At some point (I don't recall why, maybe at Oracle's
>>>>>>>>>>>>>> request??), we put in some code in KernelTester that if it
>>>>>>>>>>>>>> could not find the okra simulator files would just silently
>>>>>>>>>>>>>> not run the tests (which I guess counts as a pass).
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> No the hsail simulator is not a java app at all.
>>>>>>>>>>>>>> The page at
>>>>>>>>>>>>>> https://wiki.openjdk.java.net/display/Sumatra/The+HSAIL+Simu
>>>>>>>>>>>>>> la to r should describe how to build it and what environment
>>>>>>>>>>>>>> variables to set up to use it...
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> -- Tom
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>>> From: Doug Simon [mailto:doug.simon at oracle.com]
>>>>>>>>>>>>>>> Sent: Monday, December 16, 2013 11:02 AM
>>>>>>>>>>>>>>> To: Deneau, Tom
>>>>>>>>>>>>>>> Cc: graal-dev at openjdk.java.net
>>>>>>>>>>>>>>> Subject: Re: hsail gate commands
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On Dec 16, 2013, at 5:48 PM, Deneau, Tom
>>>>>>>>>>>>>>> <tom.deneau at amd.com>
>>>>>>>>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Doug --
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> I see.
>>>>>>>>>>>>>>>> I don't understand why the tests would run without error
>>>>>>>>>>>>>>>> if the simulator and associated assembler is missing
>>> but...
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> What error are you seeing?
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Is there a way we can get the HSAIL simulator into the
>>>>>>>>>>>>>>>> gate
>>>>>>>>>>>>>>> infrastructure?
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Is it a pure Java app yet? That would certainly make it
>>>>>>> trivial.
>>>>>>>>>>>>>>> In any case, I'll defer to Bernhard or Gilles to answer
>>>>>>>>>>>>>>> this since they are the maintainers of this infrastructure.
>>>>>>>>>>>>>>> Can you please send instructions on how to
>>>>>>>>>>>>>>> install/use/configure the
>>>>>>>>> simulator.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> -Doug
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>>>>> From: Doug Simon [mailto:doug.simon at oracle.com]
>>>>>>>>>>>>>>>>> Sent: Monday, December 16, 2013 10:36 AM
>>>>>>>>>>>>>>>>> To: Deneau, Tom
>>>>>>>>>>>>>>>>> Cc: graal-dev at openjdk.java.net
>>>>>>>>>>>>>>>>> Subject: Re: hsail gate commands
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On Dec 16, 2013, at 5:21 PM, Deneau, Tom
>>>>>>>>>>>>>>>>> <tom.deneau at amd.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Doug --
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> I noticed in syncing with the trunk as of last Friday
>>>>>>>>>>>>>>>>>> that all of our
>>>>>>>>>>>>>>>>> HSAIL test cases broke.  The cause was some imperfect
>>>>>>>>>>>>>>>>> code in our HSAILAssembler for compare instructions which
>>>>>>>>>>>>>>>>> when presented with an unordered compare could generate
>>>>>>>>>>>>>>>>> code that would not assemble, and was easy to fix.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> But I was wondering how this made it past the gate.
>>>>>>>>>>>>>>>>>> Can you describe what gate commands are used regarding
>>>>>>> hsail?
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> We simply run 'mx gate' which will include running all
>>>>>>>>>>>>>>>>> the HSAIL unit tests.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On my Mac with the latest bits:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> $ mx --vm server unittest hsail executing junit tests
>>>>> now...
>>>>>>>>>>>>>>>>> (107 test classes) JUnit version 4.8
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>> 
>>>>> 
>>> ........................................................................
>>>>>>>>>>>>>>>>> ...........I................I......
>>>>>>>>>>>>>>>>> Time: 5.525
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> OK (105 tests)
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Keep in mind I (and the gate infrastructure) don't have
>>>>>>>>>>>>>>>>> the HSAIL simulator.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> -Doug
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>>>> 
>>> 
>> 
>> 
> 