hsail gate commands

Wed Jan 15 13:46:26 PST 2014

Doug --

how does the JNI_OnLoad know what path it was loaded from?
or would the java side have to pass this information down?

> -----Original Message-----
> From: Doug Simon [mailto:doug.simon at oracle.com]
> Sent: Wednesday, January 15, 2014 3:11 PM
> To: Deneau, Tom
> Cc: graal-dev at openjdk.java.net
> Subject: Re: hsail gate commands
> 
> Hi Tom,
> 
> A quick glance through the HSAIL C++ layer seems to indicate that it
> should be able to lazily initialize the HSAIL library linkage (currently
> done in gnu::Hsail::probe_linkage()). This means it should be deferrable
> until after the Java part of the Okra library has had a chance to
> extract the native library file (and hsailasm executable) into the file
> system. Then you would call System.load() to load the library. There's
> still the problem of how to communicate the absolute path of the native
> library to the os::dll_load call. There may be a more elegant solution,
> but I'd suggest using the JNI_OnLoad[1] mechanism. Inside the JNI_OnLoad
> function in the Okra native library, you can set an environment variable
> (e.g., OKRA_DLL_PATH) which gnu::Hsail::probe_linkage can then use. Note
> that the environment variable has to be set this way as there's no way
> to set an environment variable from Java.
> 
> By the time you finish all this hackery, you may start thinking that
> simply porting the HSAIL simulator/toolchain to Java might be easier ;-)
> 
> [1]
> http://docs.oracle.com/javase/1.5.0/docs/guide/jni/spec/invocation.html#
> JNI_OnLoad
> 
> On Jan 15, 2014, at 9:05 PM, Deneau, Tom <tom.deneau at amd.com> wrote:
> 
> > Doug --
> >
> > One complication I saw here.
> > The native library file is used both from the java side (using
> > loadLibrary) and from the hotspot side (using os::dll_load followed by
> looking up the specific functions it needs).
> > And the hotspot attempt to load happens first (from gpu::init()).
> >
> > So anything the java side would have done with resources in the .jar
> > file and temp directories wouldn't have been done yet.
> >
> > -- Tom
> >
> >
> >> -----Original Message-----
> >> From: Doug Simon [mailto:doug.simon at oracle.com]
> >> Sent: Tuesday, January 14, 2014 3:27 AM
> >> To: Deneau, Tom
> >> Cc: graal-dev at openjdk.java.net
> >> Subject: Re: hsail gate commands
> >>
> >>
> >> On Jan 13, 2014, at 8:52 PM, Deneau, Tom <tom.deneau at amd.com> wrote:
> >>
> >>> Doug --
> >>>
> >>> Using the techniques in
> >>> http://frommyplayground.com/how-to-load-native-jni-library-from-jar/
> >>> that you sent, would it be acceptable to leave hsailasm as a
> >>> separate
> >> executable which gets unjarred into the system temporary directory
> >> and then executed from there using an absolute pathname?
> >>
> >> Sounds feasible and reasonable to me.
> >>
> >>>
> >>> -- Tom
> >>>
> >>>
> >>>
> >>>> -----Original Message-----
> >>>> From: Doug Simon [mailto:doug.simon at oracle.com]
> >>>> Sent: Sunday, January 12, 2014 2:29 PM
> >>>> To: Deneau, Tom
> >>>> Subject: Re: hsail gate commands
> >>>>
> >>>>
> >>>> On Jan 9, 2014, at 9:50 PM, Deneau, Tom <tom.deneau at amd.com> wrote:
> >>>>
> >>>>> Doug --
> >>>>>
> >>>>> We'll take a look at your suggestion.
> >>>>
> >>>> Great.
> >>>>
> >>>> BTW, you guys may want to bring
> >>>> https://wiki.openjdk.java.net/display/Sumatra/The+HSAIL+Simulator
> >>>> more up to date at some point. At least you can remove the "(webrev
> >>>> under review)" qualifications.
> >>>>
> >>>> -Doug
> >>>>
> >>>>>> -----Original Message-----
> >>>>>> From: Doug Simon [mailto:doug.simon at oracle.com]
> >>>>>> Sent: Thursday, January 09, 2014 1:38 PM
> >>>>>> To: Deneau, Tom
> >>>>>> Cc: graal-dev at openjdk.java.net
> >>>>>> Subject: Re: hsail gate commands
> >>>>>>
> >>>>>>
> >>>>>> On Jan 9, 2014, at 6:51 PM, Tom Rodriguez
> >>>>>> <tom.rodriguez at oracle.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> It would still be great if there would be some way to
> >>>>>>>>>> incorporate
> >>>>>> the hsail simulator into the gate...
> >>>>>>>>>
> >>>>>>>>> A good start would be if you could provide us with a patch for
> >>>>>>>>> `mx',
> >>>>>> that adds a command which sets up the HSAIL simulator properly.
> >>>>>> Currently our gate is running on Linux, but this might change in
> >>>>>> the future, so keep in mind that this should potentially work on
> >>>>>> all platforms (Linux, MacOSX, Windows).
> >>>>>>>>>
> >>>>>>>>> Our gate server is basically executing `mx gate' to validate
> >>>>>> changes. That includes building, bootstrap tests, unittest, etc.
> >>>>>> If that command was successful on our gate server, the changes
> >>>>>> are accepted and pushed to the repository.  Having the `mx gate'
> >>>>>> command, one can reproduce the gate process on his/her local
> >> machine.
> >>>>>> Therefore it would be important that one can easily and reliable
> >>>>>> set up the HSAIL environment if it is part of the gate process.
> >>>>>>>>
> >>>>>>>> Given the dependencies for building the simulator[1] and the
> >>>>>>>> fact it
> >>>>>> only currently runs on linux, I don't think it will be that easy
> >>>>>> to come up with an mx command.
> >>>>>>>
> >>>>>>> What precompiling an okra distribution for linux that mx can
> >>>> download?
> >>>>>>
> >>>>>> If Okra used a self contained JNI library, that might just work.
> >>>>>> However, the way it is deployed now involves having the hsailasm
> >>>>>> executable on your PATH. Also, LD_LIBRARY_PATH has to be setup to
> >>>>>> find the JNI library itself.
> >>>>>>
> >>>>>> @AMD guys: How hard would it be to make the Okra JNI be self
> >>>>>> contained library and include the hsailasm functionality? If
> >>>>>> that's possible, you could bundle the JNI library inside the Okra
> >>>>>> jar using a technique like
> >>>>>> http://frommyplayground.com/how-to-load-native-jni-library-from-j
> >>>>>> ar
> >>>>>> /
> >>>> to avoid having to set up PATH or LD_LIBRARY_PATH.
> >>>>>>
> >>>>>> -Doug
> >>>>>>
> >>>>>>>> [1]
> >>>>>>>> https://github.com/HSAFoundation/Okra-Interface-to-HSAIL-Simula
> >>>>>>>> to
> >>>>>>>> r#
> >>>>>>>> ok
> >>>>>>>> ra-interface-to-hsail-simulator
> >>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> -----Original Message-----
> >>>>>>>>>>> From: graal-dev-bounces at openjdk.java.net [mailto:graal-dev-
> >>>>>>>>>>> bounces at openjdk.java.net] On Behalf Of Deneau, Tom
> >>>>>>>>>>> Sent: Monday, December 16, 2013 11:24 AM
> >>>>>>>>>>> To: Doug Simon
> >>>>>>>>>>> Cc: graal-dev at openjdk.java.net
> >>>>>>>>>>> Subject: RE: hsail gate commands
> >>>>>>>>>>>
> >>>>>>>>>>> OK, I see why there is no error when running mx --vm server
> >>>>>>>>>>> unittest hsail
> >>>>>>>>>>>
> >>>>>>>>>>> At some point (I don't recall why, maybe at Oracle's
> >>>>>>>>>>> request??), we put in some code in KernelTester that if it
> >>>>>>>>>>> could not find the okra simulator files would just silently
> >>>>>>>>>>> not run the tests (which I guess counts as a pass).
> >>>>>>>>>>>
> >>>>>>>>>>> No the hsail simulator is not a java app at all.
> >>>>>>>>>>> The page at
> >>>>>>>>>>> https://wiki.openjdk.java.net/display/Sumatra/The+HSAIL+Simu
> >>>>>>>>>>> la to r should describe how to build it and what environment
> >>>>>>>>>>> variables to set up to use it...
> >>>>>>>>>>>
> >>>>>>>>>>> -- Tom
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>> -----Original Message-----
> >>>>>>>>>>>> From: Doug Simon [mailto:doug.simon at oracle.com]
> >>>>>>>>>>>> Sent: Monday, December 16, 2013 11:02 AM
> >>>>>>>>>>>> To: Deneau, Tom
> >>>>>>>>>>>> Cc: graal-dev at openjdk.java.net
> >>>>>>>>>>>> Subject: Re: hsail gate commands
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Dec 16, 2013, at 5:48 PM, Deneau, Tom
> >>>>>>>>>>>> <tom.deneau at amd.com>
> >>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Doug --
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I see.
> >>>>>>>>>>>>> I don't understand why the tests would run without error
> >>>>>>>>>>>>> if the simulator and associated assembler is missing
> but...
> >>>>>>>>>>>>
> >>>>>>>>>>>> What error are you seeing?
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Is there a way we can get the HSAIL simulator into the
> >>>>>>>>>>>>> gate
> >>>>>>>>>>>> infrastructure?
> >>>>>>>>>>>>
> >>>>>>>>>>>> Is it a pure Java app yet? That would certainly make it
> >>>> trivial.
> >>>>>>>>>>>> In any case, I'll defer to Bernhard or Gilles to answer
> >>>>>>>>>>>> this since they are the maintainers of this infrastructure.
> >>>>>>>>>>>> Can you please send instructions on how to
> >>>>>>>>>>>> install/use/configure the
> >>>>>> simulator.
> >>>>>>>>>>>>
> >>>>>>>>>>>> -Doug
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>>> -----Original Message-----
> >>>>>>>>>>>>>> From: Doug Simon [mailto:doug.simon at oracle.com]
> >>>>>>>>>>>>>> Sent: Monday, December 16, 2013 10:36 AM
> >>>>>>>>>>>>>> To: Deneau, Tom
> >>>>>>>>>>>>>> Cc: graal-dev at openjdk.java.net
> >>>>>>>>>>>>>> Subject: Re: hsail gate commands
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Dec 16, 2013, at 5:21 PM, Deneau, Tom
> >>>>>>>>>>>>>> <tom.deneau at amd.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Doug --
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I noticed in syncing with the trunk as of last Friday
> >>>>>>>>>>>>>>> that all of our
> >>>>>>>>>>>>>> HSAIL test cases broke.  The cause was some imperfect
> >>>>>>>>>>>>>> code in our HSAILAssembler for compare instructions which
> >>>>>>>>>>>>>> when presented with an unordered compare could generate
> >>>>>>>>>>>>>> code that would not assemble, and was easy to fix.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> But I was wondering how this made it past the gate.
> >>>>>>>>>>>>>>> Can you describe what gate commands are used regarding
> >>>> hsail?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> We simply run 'mx gate' which will include running all
> >>>>>>>>>>>>>> the HSAIL unit tests.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On my Mac with the latest bits:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> $ mx --vm server unittest hsail executing junit tests
> >> now...
> >>>>>>>>>>>>>> (107 test classes) JUnit version 4.8
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>
> >>>>
> >>
> ........................................................................
> >>>>>>>>>>>>>> ...........I................I......
> >>>>>>>>>>>>>> Time: 5.525
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> OK (105 tests)
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Keep in mind I (and the gate infrastructure) don't have
> >>>>>>>>>>>>>> the HSAIL simulator.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> -Doug
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>>
> >>
> >
> >
> 