From jules_gosnell at yahoo.com Sun Apr 6 14:22:19 2014 From: jules_gosnell at yahoo.com (Jules Gosnell) Date: Sun, 06 Apr 2014 15:22:19 +0100 Subject: Graal and Clojure In-Reply-To: References: <1394063789.8387.YahooMailNeo@web122406.mail.ne1.yahoo.com> <5326354D.8010204@yahoo.com> <53277532.3030807@amd.com>, <532B5D35.4010108@yahoo.com> Message-ID: <5341631B.1090207@yahoo.com> So, Guys, Here is the promised update on Clumatra - an effort to get Clojure up and running on top of Graal/Okra/HSA. - I now have a working Ubuntu/AMD HSA box. - I have a Jenkins nightly build of JDK8 / Graal : http://ouroboros.dyndns-free.com/ci/view/clumatra/job/graal/ - if useful, you can download the latest linux/64 tgz here (it is 300mb - I'm going to look at shrinking it: http://ouroboros.dyndns-free.com/clumatra/jdk1.8.0-graal.tgz - I have builds of Clumatra without, with simulated and with hardware based HSA : http://ouroboros.dyndns-free.com/ci/view/clumatra/ - The clumatra testsuite is growing - results have been a bit unstable recently as I learnt which tests completely take out or hang the jdk8/graal jvm and which can be trusted to succeed / fail in a well behaved fashion: http://ouroboros.dyndns-free.com/ci/view/clumatra/job/clumatra-hardware-acceleration/ I am working on a reflection/macro based solution, which will automagically create tests for all methods of all classes that I want tests for. When I bring this online in a couple of weeks the number of tests should jump dramatically. Please feel free to inspect test results etc on my Jenkins and let me know whether they are helpful and whether they can be made more useful in any way. currently some of them seem to indicate problems in the okra backend. This is when graal seems to run happily but I get back the wrong result from my test: e.g. http://ouroboros.dyndns-free.com/ci/view/clumatra/job/clumatra-hardware-acceleration/83/testReport/clumatra/core-test/long_unchecked_inc_test/ I run the test via graal/okra twice and once in pure clojure - so you see three results. The last one is the expected one, the first two the ones returned from the GPU. Other tests let me know that graal is not yet implementing various features of jvm bytecode - e.g. http://ouroboros.dyndns-free.com/ci/view/clumatra/job/clumatra-hardware-acceleration/83/testReport/clumatra/core-test/long_my_static_inc_test/ Unfortunately the test report does not seem to be capturing the HSAIL logging that you can see in the console output... My guess is that it is captured by redirecting clojure's usual output stream, but this is not the same one as being used by Graal... http://ouroboros.dyndns-free.com/ci/view/clumatra/job/clumatra-hardware-acceleration/83/console That's enough for now. I'll post with more progress as and when. Jules On 31/03/14 23:40, Caspole, Eric wrote: > Hi Jules, > I updated everything and got Clumatra running on the real hardware today. > > In project.clj, I used this: > > :jvm-opts ["-server" "-ea" "-esa" "-Xms1g" "-Xmx1g" > "-verbosegc" "-G:Log=CodeGen" > "-XX:+GPUOffload" "-XX:+TraceGPUInteraction"] > > Then make sure your /path/to/okra/dist/bin is in the PATH and LD_LIBRARY_PATH. > > When it is using the simulator, you will see this: > [HSAIL] using _OKRA_SIM_LIB_PATH_=/tmp/okraresource.dir_2595501614013009358/libokra_x86_64.so > > With hardware or external okra you will see this: > > [HSAIL] library is libokra_x86_64.so > > Regards, > Eric > > > ________________________________________ > From: Jules Gosnell [jules_gosnell at yahoo.com] > Sent: Thursday, March 20, 2014 5:27 PM > To: Caspole, Eric; graal-dev at openjdk.java.net; sumatra-dev at openjdk.java.net > Subject: Re: Graal and Clojure > > Eric, > > I just checked support to Clumatra for running individual unit tests > from the CLI - should be enough to get you set up with your debugger. > > All details at the bottom of the README. > > https://github.com/JulesGosnell/clumatra > > I'm afraid you will need to install maven if you don't already have it - > for some reason lein does not seem to have a test-jar target. > > It is still a little rough around the edges, but better than nothing :-) > > I also just bought my HSA h/w - should be up and running next week > sometime. Did you figure out how to enable native-mode Okra ? > > catch you later, > > > Jules > > > > On 17/03/14 22:20, Eric Caspole wrote: >> Hi everybody, >> I was setting up a new HSA system today and I installed Clumatra and got >> it working under the simulator. I have a real HSA system not too >> different from this: >> >> http://code.google.com/p/aparapi/wiki/SettingUpLinuxHSAMachineForAparapi >> >> In the clojure/lein/maven based system there is so much harness I can't >> immediately see how to switch in the real hardware version of Okra >> instead of the simulator Okra that gets installed into maven and >> exploded in /tmp when the tests run. >> >> Jules, could you show how to bypass the harness etc to switch the Okra? >> Can I just run 1 test with a simple java command line? This makes it a >> lot easier to use a debugger which I will definitely want to do. >> >> Otherwise this is a cool project and we already found 1 or 2 issues we >> can fix in HSAIL Graal related to your clojure tests. >> Regards, >> Eric >> >> >> >> On 03/17/2014 06:43 AM, Doug Simon wrote: >>> Hi Julian, >>> >>> In terms of what to test, I?ll leave it up to the Sumatra developers >>> at AMD and Oracle to comment on what may be interesting. >>> >>> As far as I know, no one in the Graal team is that familiar with >>> Clojure. This actually makes the value of having Clojure tests >>> integrated into our gate system somewhat problematic - >>> debugging/fixing failures may be difficult. I?d suggest you report >>> failures your discover on the sumatra-dev and graal-dev lists and we >>> can resolve them that way for now. >>> >>> In addition, for Graal we?re always interested in benchmark suites for >>> JVM hosted languages. Can you recommend any well known and trusted >>> Clojure benchmarks we should consider? >>> >>> -Doug >>> >>> On Mar 17, 2014, at 12:35 AM, Jules Gosnell >>> wrote: >>> >>>> Doug, >>>> >>>> Sorry it has taken me a while to get back - I've been busy playing >>>> with code. >>>> >>>> I currently have a small clojure testsuite up and running. I plan to >>>> broaden this and then think about providing it in a junit-compatible >>>> way to your project. >>>> >>>> I still haven't made up my mind exactly what I should be testing. It >>>> seems unnecessary to duplicate all your tests in Clojure, since you >>>> are already guarding against regression in these areas - I think I >>>> will start out with a fairly general set of tests as I probe for >>>> particular problem areas for Clojure, into which I may dive for more >>>> detail - time will tell... if you have any ideas, please let me know. >>>> >>>> I've checked my stuff in at github, if anyone is interested - here is >>>> the project: >>>> >>>> https://github.com/JulesGosnell/clumatra >>>> >>>> here is some clojure that I reverse engineered from GraalKernelTester: >>>> >>>> https://github.com/JulesGosnell/clumatra/blob/master/src/clumatra/core.clj >>>> >>>> >>>> and here are the first successful tests: >>>> >>>> https://github.com/JulesGosnell/clumatra/blob/master/test/clumatra/core_test.clj >>>> >>>> >>>> not very pretty yet - I am still feeling my way. >>>> >>>> cheers >>>> >>>> >>>> Jules >>>> >>>> >>>> >>>> >>>> On 06/03/14 10:24, Doug Simon wrote: >>>>> Hi Julian, >>>>> >>>>> This looks very interesting and will be an good alternative testing >>>>> vector for the Sumatra work as it matures. If Clojure tests can >>>>> somehow be made to run from Junit, then I?m sure we can try >>>>> integrating it into our testing. >>>>> >>>>> -Doug >>>>> >>>>> On Mar 6, 2014, at 12:56 AM, Julian Gosnell >>>>> wrote: >>>>> >>>>>> Guys, >>>>>> >>>>>> I just built the Java8 / Graal / Okra stack and managed to run some >>>>>> very simple Clojure copying the contents of one int array into >>>>>> another one on Okra. >>>>>> >>>>>> https://groups.google.com/forum/#!topic/clojure/JpjK__NTR5Y >>>>>> >>>>>> >>>>>> I find the ramifications of this very exciting :-) >>>>>> >>>>>> I understand that fn-ality is limited at the moment - but I am keen >>>>>> to keep testing and to help ensure early visibility of Clojure >>>>>> related issues to both communities - perhaps even the submission of >>>>>> a Clojure testsuite if Graal developers thought that useful. >>>>>> >>>>>> I'd be very interested in your thoughts on Graal / Clojure. >>>>>> >>>>>> regards, >>>>>> >>>>>> >>>>>> Jules >>>>>> >>>>> >>>>> >>>> >>> >>> >> >> > > > > From jules_gosnell at yahoo.com Mon Apr 7 06:20:02 2014 From: jules_gosnell at yahoo.com (Jules Gosnell) Date: Mon, 07 Apr 2014 07:20:02 +0100 Subject: Graal and Clojure In-Reply-To: <5341631B.1090207@yahoo.com> References: <1394063789.8387.YahooMailNeo@web122406.mail.ne1.yahoo.com> <5326354D.8010204@yahoo.com> <53277532.3030807@amd.com>, <532B5D35.4010108@yahoo.com> <5341631B.1090207@yahoo.com> Message-ID: <53424392.8060700@yahoo.com> Good news - I reapplied the -XX:-UseHSAILDeoptimization flag (last time I tried it the code had not yet made it into my jdk8/graal build) and a lot of the instability mentioned in my last mail disappeared - particularly in the simulator. Many more tests also passed :-) http://ouroboros.dyndns-free.com/ci/job/clumatra-hardware-acceleration/ http://ouroboros.dyndns-free.com/ci/view/clumatra/job/clumatra-simulated-acceleration/ Jules On 06/04/14 15:22, Jules Gosnell wrote: > So, Guys, > > Here is the promised update on Clumatra - an effort to get Clojure up > and running on top of Graal/Okra/HSA. > > - I now have a working Ubuntu/AMD HSA box. > > - I have a Jenkins nightly build of JDK8 / Graal : > > http://ouroboros.dyndns-free.com/ci/view/clumatra/job/graal/ > > - if useful, you can download the latest linux/64 tgz here (it is 300mb > - I'm going to look at shrinking it: > > http://ouroboros.dyndns-free.com/clumatra/jdk1.8.0-graal.tgz > > - I have builds of Clumatra without, with simulated and with hardware > based HSA : > > http://ouroboros.dyndns-free.com/ci/view/clumatra/ > > - The clumatra testsuite is growing - results have been a bit unstable > recently as I learnt which tests completely take out or hang the > jdk8/graal jvm and which can be trusted to succeed / fail in a well > behaved fashion: > > http://ouroboros.dyndns-free.com/ci/view/clumatra/job/clumatra-hardware-acceleration/ > > > I am working on a reflection/macro based solution, which will > automagically create tests for all methods of all classes that I want > tests for. When I bring this online in a couple of weeks the number of > tests should jump dramatically. > > Please feel free to inspect test results etc on my Jenkins and let me > know whether they are helpful and whether they can be made more useful > in any way. currently some of them seem to indicate problems in the okra > backend. This is when graal seems to run happily but I get back the > wrong result from my test: > > e.g. > > http://ouroboros.dyndns-free.com/ci/view/clumatra/job/clumatra-hardware-acceleration/83/testReport/clumatra/core-test/long_unchecked_inc_test/ > > > I run the test via graal/okra twice and once in pure clojure - so you > see three results. The last one is the expected one, the first two the > ones returned from the GPU. > > Other tests let me know that graal is not yet implementing various > features of jvm bytecode - e.g. > > http://ouroboros.dyndns-free.com/ci/view/clumatra/job/clumatra-hardware-acceleration/83/testReport/clumatra/core-test/long_my_static_inc_test/ > > > Unfortunately the test report does not seem to be capturing the HSAIL > logging that you can see in the console output... My guess is that it is > captured by redirecting clojure's usual output stream, but this is not > the same one as being used by Graal... > > http://ouroboros.dyndns-free.com/ci/view/clumatra/job/clumatra-hardware-acceleration/83/console > > > > That's enough for now. I'll post with more progress as and when. > > > > Jules > > > > On 31/03/14 23:40, Caspole, Eric wrote: >> Hi Jules, >> I updated everything and got Clumatra running on the real hardware today. >> >> In project.clj, I used this: >> >> :jvm-opts ["-server" "-ea" "-esa" "-Xms1g" "-Xmx1g" >> "-verbosegc" "-G:Log=CodeGen" >> "-XX:+GPUOffload" "-XX:+TraceGPUInteraction"] >> >> Then make sure your /path/to/okra/dist/bin is in the PATH and >> LD_LIBRARY_PATH. >> >> When it is using the simulator, you will see this: >> [HSAIL] using >> _OKRA_SIM_LIB_PATH_=/tmp/okraresource.dir_2595501614013009358/libokra_x86_64.so >> >> >> With hardware or external okra you will see this: >> >> [HSAIL] library is libokra_x86_64.so >> >> Regards, >> Eric >> >> >> ________________________________________ >> From: Jules Gosnell [jules_gosnell at yahoo.com] >> Sent: Thursday, March 20, 2014 5:27 PM >> To: Caspole, Eric; graal-dev at openjdk.java.net; >> sumatra-dev at openjdk.java.net >> Subject: Re: Graal and Clojure >> >> Eric, >> >> I just checked support to Clumatra for running individual unit tests >> from the CLI - should be enough to get you set up with your debugger. >> >> All details at the bottom of the README. >> >> https://github.com/JulesGosnell/clumatra >> >> I'm afraid you will need to install maven if you don't already have it - >> for some reason lein does not seem to have a test-jar target. >> >> It is still a little rough around the edges, but better than nothing :-) >> >> I also just bought my HSA h/w - should be up and running next week >> sometime. Did you figure out how to enable native-mode Okra ? >> >> catch you later, >> >> >> Jules >> >> >> >> On 17/03/14 22:20, Eric Caspole wrote: >>> Hi everybody, >>> I was setting up a new HSA system today and I installed Clumatra and got >>> it working under the simulator. I have a real HSA system not too >>> different from this: >>> >>> http://code.google.com/p/aparapi/wiki/SettingUpLinuxHSAMachineForAparapi >>> >>> In the clojure/lein/maven based system there is so much harness I can't >>> immediately see how to switch in the real hardware version of Okra >>> instead of the simulator Okra that gets installed into maven and >>> exploded in /tmp when the tests run. >>> >>> Jules, could you show how to bypass the harness etc to switch the Okra? >>> Can I just run 1 test with a simple java command line? This makes it a >>> lot easier to use a debugger which I will definitely want to do. >>> >>> Otherwise this is a cool project and we already found 1 or 2 issues we >>> can fix in HSAIL Graal related to your clojure tests. >>> Regards, >>> Eric >>> >>> >>> >>> On 03/17/2014 06:43 AM, Doug Simon wrote: >>>> Hi Julian, >>>> >>>> In terms of what to test, I?ll leave it up to the Sumatra developers >>>> at AMD and Oracle to comment on what may be interesting. >>>> >>>> As far as I know, no one in the Graal team is that familiar with >>>> Clojure. This actually makes the value of having Clojure tests >>>> integrated into our gate system somewhat problematic - >>>> debugging/fixing failures may be difficult. I?d suggest you report >>>> failures your discover on the sumatra-dev and graal-dev lists and we >>>> can resolve them that way for now. >>>> >>>> In addition, for Graal we?re always interested in benchmark suites for >>>> JVM hosted languages. Can you recommend any well known and trusted >>>> Clojure benchmarks we should consider? >>>> >>>> -Doug >>>> >>>> On Mar 17, 2014, at 12:35 AM, Jules Gosnell >>>> wrote: >>>> >>>>> Doug, >>>>> >>>>> Sorry it has taken me a while to get back - I've been busy playing >>>>> with code. >>>>> >>>>> I currently have a small clojure testsuite up and running. I plan to >>>>> broaden this and then think about providing it in a junit-compatible >>>>> way to your project. >>>>> >>>>> I still haven't made up my mind exactly what I should be testing. It >>>>> seems unnecessary to duplicate all your tests in Clojure, since you >>>>> are already guarding against regression in these areas - I think I >>>>> will start out with a fairly general set of tests as I probe for >>>>> particular problem areas for Clojure, into which I may dive for more >>>>> detail - time will tell... if you have any ideas, please let me know. >>>>> >>>>> I've checked my stuff in at github, if anyone is interested - here is >>>>> the project: >>>>> >>>>> https://github.com/JulesGosnell/clumatra >>>>> >>>>> here is some clojure that I reverse engineered from GraalKernelTester: >>>>> >>>>> https://github.com/JulesGosnell/clumatra/blob/master/src/clumatra/core.clj >>>>> >>>>> >>>>> >>>>> and here are the first successful tests: >>>>> >>>>> https://github.com/JulesGosnell/clumatra/blob/master/test/clumatra/core_test.clj >>>>> >>>>> >>>>> >>>>> not very pretty yet - I am still feeling my way. >>>>> >>>>> cheers >>>>> >>>>> >>>>> Jules >>>>> >>>>> >>>>> >>>>> >>>>> On 06/03/14 10:24, Doug Simon wrote: >>>>>> Hi Julian, >>>>>> >>>>>> This looks very interesting and will be an good alternative testing >>>>>> vector for the Sumatra work as it matures. If Clojure tests can >>>>>> somehow be made to run from Junit, then I?m sure we can try >>>>>> integrating it into our testing. >>>>>> >>>>>> -Doug >>>>>> >>>>>> On Mar 6, 2014, at 12:56 AM, Julian Gosnell >>>>>> wrote: >>>>>> >>>>>>> Guys, >>>>>>> >>>>>>> I just built the Java8 / Graal / Okra stack and managed to run some >>>>>>> very simple Clojure copying the contents of one int array into >>>>>>> another one on Okra. >>>>>>> >>>>>>> https://groups.google.com/forum/#!topic/clojure/JpjK__NTR5Y >>>>>>> >>>>>>> >>>>>>> I find the ramifications of this very exciting :-) >>>>>>> >>>>>>> I understand that fn-ality is limited at the moment - but I am keen >>>>>>> to keep testing and to help ensure early visibility of Clojure >>>>>>> related issues to both communities - perhaps even the submission of >>>>>>> a Clojure testsuite if Graal developers thought that useful. >>>>>>> >>>>>>> I'd be very interested in your thoughts on Graal / Clojure. >>>>>>> >>>>>>> regards, >>>>>>> >>>>>>> >>>>>>> Jules >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>>> >>> >>> >> >> >> >> > From thomas.wuerthinger at oracle.com Wed Apr 16 11:13:49 2014 From: thomas.wuerthinger at oracle.com (Thomas Wuerthinger) Date: Wed, 16 Apr 2014 13:13:49 +0200 Subject: Graal and Clojure In-Reply-To: <53424392.8060700@yahoo.com> References: <1394063789.8387.YahooMailNeo@web122406.mail.ne1.yahoo.com> <5326354D.8010204@yahoo.com> <53277532.3030807@amd.com>, <532B5D35.4010108@yahoo.com> <5341631B.1090207@yahoo.com> <53424392.8060700@yahoo.com> Message-ID: <4481FFF5-F9FD-4318-9528-96005AF40056@oracle.com> Cool! It is very exciting to see so much progress with Clojure running on the GPU via Graal. - thomas On 07 Apr 2014, at 08:20, Jules Gosnell wrote: > Good news - I reapplied the -XX:-UseHSAILDeoptimization flag (last time I tried it the code had not yet made it into my jdk8/graal build) and a lot of the instability mentioned in my last mail disappeared - particularly in the simulator. Many more tests also passed :-) > > http://ouroboros.dyndns-free.com/ci/job/clumatra-hardware-acceleration/ > http://ouroboros.dyndns-free.com/ci/view/clumatra/job/clumatra-simulated-acceleration/ > > > Jules > > On 06/04/14 15:22, Jules Gosnell wrote: >> So, Guys, >> >> Here is the promised update on Clumatra - an effort to get Clojure up >> and running on top of Graal/Okra/HSA. >> >> - I now have a working Ubuntu/AMD HSA box. >> >> - I have a Jenkins nightly build of JDK8 / Graal : >> >> http://ouroboros.dyndns-free.com/ci/view/clumatra/job/graal/ >> >> - if useful, you can download the latest linux/64 tgz here (it is 300mb >> - I'm going to look at shrinking it: >> >> http://ouroboros.dyndns-free.com/clumatra/jdk1.8.0-graal.tgz >> >> - I have builds of Clumatra without, with simulated and with hardware >> based HSA : >> >> http://ouroboros.dyndns-free.com/ci/view/clumatra/ >> >> - The clumatra testsuite is growing - results have been a bit unstable >> recently as I learnt which tests completely take out or hang the >> jdk8/graal jvm and which can be trusted to succeed / fail in a well >> behaved fashion: >> >> http://ouroboros.dyndns-free.com/ci/view/clumatra/job/clumatra-hardware-acceleration/ >> >> >> I am working on a reflection/macro based solution, which will >> automagically create tests for all methods of all classes that I want >> tests for. When I bring this online in a couple of weeks the number of >> tests should jump dramatically. >> >> Please feel free to inspect test results etc on my Jenkins and let me >> know whether they are helpful and whether they can be made more useful >> in any way. currently some of them seem to indicate problems in the okra >> backend. This is when graal seems to run happily but I get back the >> wrong result from my test: >> >> e.g. >> >> http://ouroboros.dyndns-free.com/ci/view/clumatra/job/clumatra-hardware-acceleration/83/testReport/clumatra/core-test/long_unchecked_inc_test/ >> >> >> I run the test via graal/okra twice and once in pure clojure - so you >> see three results. The last one is the expected one, the first two the >> ones returned from the GPU. >> >> Other tests let me know that graal is not yet implementing various >> features of jvm bytecode - e.g. >> >> http://ouroboros.dyndns-free.com/ci/view/clumatra/job/clumatra-hardware-acceleration/83/testReport/clumatra/core-test/long_my_static_inc_test/ >> >> >> Unfortunately the test report does not seem to be capturing the HSAIL >> logging that you can see in the console output... My guess is that it is >> captured by redirecting clojure's usual output stream, but this is not >> the same one as being used by Graal... >> >> http://ouroboros.dyndns-free.com/ci/view/clumatra/job/clumatra-hardware-acceleration/83/console >> >> >> >> That's enough for now. I'll post with more progress as and when. >> >> >> >> Jules >> >> >> >> On 31/03/14 23:40, Caspole, Eric wrote: >>> Hi Jules, >>> I updated everything and got Clumatra running on the real hardware today. >>> >>> In project.clj, I used this: >>> >>> :jvm-opts ["-server" "-ea" "-esa" "-Xms1g" "-Xmx1g" >>> "-verbosegc" "-G:Log=CodeGen" >>> "-XX:+GPUOffload" "-XX:+TraceGPUInteraction"] >>> >>> Then make sure your /path/to/okra/dist/bin is in the PATH and >>> LD_LIBRARY_PATH. >>> >>> When it is using the simulator, you will see this: >>> [HSAIL] using >>> _OKRA_SIM_LIB_PATH_=/tmp/okraresource.dir_2595501614013009358/libokra_x86_64.so >>> >>> >>> With hardware or external okra you will see this: >>> >>> [HSAIL] library is libokra_x86_64.so >>> >>> Regards, >>> Eric >>> >>> >>> ________________________________________ >>> From: Jules Gosnell [jules_gosnell at yahoo.com] >>> Sent: Thursday, March 20, 2014 5:27 PM >>> To: Caspole, Eric; graal-dev at openjdk.java.net; >>> sumatra-dev at openjdk.java.net >>> Subject: Re: Graal and Clojure >>> >>> Eric, >>> >>> I just checked support to Clumatra for running individual unit tests >>> from the CLI - should be enough to get you set up with your debugger. >>> >>> All details at the bottom of the README. >>> >>> https://github.com/JulesGosnell/clumatra >>> >>> I'm afraid you will need to install maven if you don't already have it - >>> for some reason lein does not seem to have a test-jar target. >>> >>> It is still a little rough around the edges, but better than nothing :-) >>> >>> I also just bought my HSA h/w - should be up and running next week >>> sometime. Did you figure out how to enable native-mode Okra ? >>> >>> catch you later, >>> >>> >>> Jules >>> >>> >>> >>> On 17/03/14 22:20, Eric Caspole wrote: >>>> Hi everybody, >>>> I was setting up a new HSA system today and I installed Clumatra and got >>>> it working under the simulator. I have a real HSA system not too >>>> different from this: >>>> >>>> http://code.google.com/p/aparapi/wiki/SettingUpLinuxHSAMachineForAparapi >>>> >>>> In the clojure/lein/maven based system there is so much harness I can't >>>> immediately see how to switch in the real hardware version of Okra >>>> instead of the simulator Okra that gets installed into maven and >>>> exploded in /tmp when the tests run. >>>> >>>> Jules, could you show how to bypass the harness etc to switch the Okra? >>>> Can I just run 1 test with a simple java command line? This makes it a >>>> lot easier to use a debugger which I will definitely want to do. >>>> >>>> Otherwise this is a cool project and we already found 1 or 2 issues we >>>> can fix in HSAIL Graal related to your clojure tests. >>>> Regards, >>>> Eric >>>> >>>> >>>> >>>> On 03/17/2014 06:43 AM, Doug Simon wrote: >>>>> Hi Julian, >>>>> >>>>> In terms of what to test, I?ll leave it up to the Sumatra developers >>>>> at AMD and Oracle to comment on what may be interesting. >>>>> >>>>> As far as I know, no one in the Graal team is that familiar with >>>>> Clojure. This actually makes the value of having Clojure tests >>>>> integrated into our gate system somewhat problematic - >>>>> debugging/fixing failures may be difficult. I?d suggest you report >>>>> failures your discover on the sumatra-dev and graal-dev lists and we >>>>> can resolve them that way for now. >>>>> >>>>> In addition, for Graal we?re always interested in benchmark suites for >>>>> JVM hosted languages. Can you recommend any well known and trusted >>>>> Clojure benchmarks we should consider? >>>>> >>>>> -Doug >>>>> >>>>> On Mar 17, 2014, at 12:35 AM, Jules Gosnell >>>>> wrote: >>>>> >>>>>> Doug, >>>>>> >>>>>> Sorry it has taken me a while to get back - I've been busy playing >>>>>> with code. >>>>>> >>>>>> I currently have a small clojure testsuite up and running. I plan to >>>>>> broaden this and then think about providing it in a junit-compatible >>>>>> way to your project. >>>>>> >>>>>> I still haven't made up my mind exactly what I should be testing. It >>>>>> seems unnecessary to duplicate all your tests in Clojure, since you >>>>>> are already guarding against regression in these areas - I think I >>>>>> will start out with a fairly general set of tests as I probe for >>>>>> particular problem areas for Clojure, into which I may dive for more >>>>>> detail - time will tell... if you have any ideas, please let me know. >>>>>> >>>>>> I've checked my stuff in at github, if anyone is interested - here is >>>>>> the project: >>>>>> >>>>>> https://github.com/JulesGosnell/clumatra >>>>>> >>>>>> here is some clojure that I reverse engineered from GraalKernelTester: >>>>>> >>>>>> https://github.com/JulesGosnell/clumatra/blob/master/src/clumatra/core.clj >>>>>> >>>>>> >>>>>> >>>>>> and here are the first successful tests: >>>>>> >>>>>> https://github.com/JulesGosnell/clumatra/blob/master/test/clumatra/core_test.clj >>>>>> >>>>>> >>>>>> >>>>>> not very pretty yet - I am still feeling my way. >>>>>> >>>>>> cheers >>>>>> >>>>>> >>>>>> Jules >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On 06/03/14 10:24, Doug Simon wrote: >>>>>>> Hi Julian, >>>>>>> >>>>>>> This looks very interesting and will be an good alternative testing >>>>>>> vector for the Sumatra work as it matures. If Clojure tests can >>>>>>> somehow be made to run from Junit, then I?m sure we can try >>>>>>> integrating it into our testing. >>>>>>> >>>>>>> -Doug >>>>>>> >>>>>>> On Mar 6, 2014, at 12:56 AM, Julian Gosnell >>>>>>> wrote: >>>>>>> >>>>>>>> Guys, >>>>>>>> >>>>>>>> I just built the Java8 / Graal / Okra stack and managed to run some >>>>>>>> very simple Clojure copying the contents of one int array into >>>>>>>> another one on Okra. >>>>>>>> >>>>>>>> https://groups.google.com/forum/#!topic/clojure/JpjK__NTR5Y >>>>>>>> >>>>>>>> >>>>>>>> I find the ramifications of this very exciting :-) >>>>>>>> >>>>>>>> I understand that fn-ality is limited at the moment - but I am keen >>>>>>>> to keep testing and to help ensure early visibility of Clojure >>>>>>>> related issues to both communities - perhaps even the submission of >>>>>>>> a Clojure testsuite if Graal developers thought that useful. >>>>>>>> >>>>>>>> I'd be very interested in your thoughts on Graal / Clojure. >>>>>>>> >>>>>>>> regards, >>>>>>>> >>>>>>>> >>>>>>>> Jules >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>> >>>> >>> >>> >>> >>> >> > From thomas.wuerthinger at oracle.com Wed Apr 16 11:13:49 2014 From: thomas.wuerthinger at oracle.com (Thomas Wuerthinger) Date: Wed, 16 Apr 2014 13:13:49 +0200 Subject: Graal and Clojure In-Reply-To: <53424392.8060700@yahoo.com> References: <1394063789.8387.YahooMailNeo@web122406.mail.ne1.yahoo.com> <5326354D.8010204@yahoo.com> <53277532.3030807@amd.com>, <532B5D35.4010108@yahoo.com> <5341631B.1090207@yahoo.com> <53424392.8060700@yahoo.com> Message-ID: <8F99464C-9597-4660-B09B-011036A81553@oracle.com> Cool! It is very exciting to see so much progress with Clojure running on the GPU via Graal. - thomas On 07 Apr 2014, at 08:20, Jules Gosnell wrote: > Good news - I reapplied the -XX:-UseHSAILDeoptimization flag (last time I tried it the code had not yet made it into my jdk8/graal build) and a lot of the instability mentioned in my last mail disappeared - particularly in the simulator. Many more tests also passed :-) > > http://ouroboros.dyndns-free.com/ci/job/clumatra-hardware-acceleration/ > http://ouroboros.dyndns-free.com/ci/view/clumatra/job/clumatra-simulated-acceleration/ > > > Jules > > On 06/04/14 15:22, Jules Gosnell wrote: >> So, Guys, >> >> Here is the promised update on Clumatra - an effort to get Clojure up >> and running on top of Graal/Okra/HSA. >> >> - I now have a working Ubuntu/AMD HSA box. >> >> - I have a Jenkins nightly build of JDK8 / Graal : >> >> http://ouroboros.dyndns-free.com/ci/view/clumatra/job/graal/ >> >> - if useful, you can download the latest linux/64 tgz here (it is 300mb >> - I'm going to look at shrinking it: >> >> http://ouroboros.dyndns-free.com/clumatra/jdk1.8.0-graal.tgz >> >> - I have builds of Clumatra without, with simulated and with hardware >> based HSA : >> >> http://ouroboros.dyndns-free.com/ci/view/clumatra/ >> >> - The clumatra testsuite is growing - results have been a bit unstable >> recently as I learnt which tests completely take out or hang the >> jdk8/graal jvm and which can be trusted to succeed / fail in a well >> behaved fashion: >> >> http://ouroboros.dyndns-free.com/ci/view/clumatra/job/clumatra-hardware-acceleration/ >> >> >> I am working on a reflection/macro based solution, which will >> automagically create tests for all methods of all classes that I want >> tests for. When I bring this online in a couple of weeks the number of >> tests should jump dramatically. >> >> Please feel free to inspect test results etc on my Jenkins and let me >> know whether they are helpful and whether they can be made more useful >> in any way. currently some of them seem to indicate problems in the okra >> backend. This is when graal seems to run happily but I get back the >> wrong result from my test: >> >> e.g. >> >> http://ouroboros.dyndns-free.com/ci/view/clumatra/job/clumatra-hardware-acceleration/83/testReport/clumatra/core-test/long_unchecked_inc_test/ >> >> >> I run the test via graal/okra twice and once in pure clojure - so you >> see three results. The last one is the expected one, the first two the >> ones returned from the GPU. >> >> Other tests let me know that graal is not yet implementing various >> features of jvm bytecode - e.g. >> >> http://ouroboros.dyndns-free.com/ci/view/clumatra/job/clumatra-hardware-acceleration/83/testReport/clumatra/core-test/long_my_static_inc_test/ >> >> >> Unfortunately the test report does not seem to be capturing the HSAIL >> logging that you can see in the console output... My guess is that it is >> captured by redirecting clojure's usual output stream, but this is not >> the same one as being used by Graal... >> >> http://ouroboros.dyndns-free.com/ci/view/clumatra/job/clumatra-hardware-acceleration/83/console >> >> >> >> That's enough for now. I'll post with more progress as and when. >> >> >> >> Jules >> >> >> >> On 31/03/14 23:40, Caspole, Eric wrote: >>> Hi Jules, >>> I updated everything and got Clumatra running on the real hardware today. >>> >>> In project.clj, I used this: >>> >>> :jvm-opts ["-server" "-ea" "-esa" "-Xms1g" "-Xmx1g" >>> "-verbosegc" "-G:Log=CodeGen" >>> "-XX:+GPUOffload" "-XX:+TraceGPUInteraction"] >>> >>> Then make sure your /path/to/okra/dist/bin is in the PATH and >>> LD_LIBRARY_PATH. >>> >>> When it is using the simulator, you will see this: >>> [HSAIL] using >>> _OKRA_SIM_LIB_PATH_=/tmp/okraresource.dir_2595501614013009358/libokra_x86_64.so >>> >>> >>> With hardware or external okra you will see this: >>> >>> [HSAIL] library is libokra_x86_64.so >>> >>> Regards, >>> Eric >>> >>> >>> ________________________________________ >>> From: Jules Gosnell [jules_gosnell at yahoo.com] >>> Sent: Thursday, March 20, 2014 5:27 PM >>> To: Caspole, Eric; graal-dev at openjdk.java.net; >>> sumatra-dev at openjdk.java.net >>> Subject: Re: Graal and Clojure >>> >>> Eric, >>> >>> I just checked support to Clumatra for running individual unit tests >>> from the CLI - should be enough to get you set up with your debugger. >>> >>> All details at the bottom of the README. >>> >>> https://github.com/JulesGosnell/clumatra >>> >>> I'm afraid you will need to install maven if you don't already have it - >>> for some reason lein does not seem to have a test-jar target. >>> >>> It is still a little rough around the edges, but better than nothing :-) >>> >>> I also just bought my HSA h/w - should be up and running next week >>> sometime. Did you figure out how to enable native-mode Okra ? >>> >>> catch you later, >>> >>> >>> Jules >>> >>> >>> >>> On 17/03/14 22:20, Eric Caspole wrote: >>>> Hi everybody, >>>> I was setting up a new HSA system today and I installed Clumatra and got >>>> it working under the simulator. I have a real HSA system not too >>>> different from this: >>>> >>>> http://code.google.com/p/aparapi/wiki/SettingUpLinuxHSAMachineForAparapi >>>> >>>> In the clojure/lein/maven based system there is so much harness I can't >>>> immediately see how to switch in the real hardware version of Okra >>>> instead of the simulator Okra that gets installed into maven and >>>> exploded in /tmp when the tests run. >>>> >>>> Jules, could you show how to bypass the harness etc to switch the Okra? >>>> Can I just run 1 test with a simple java command line? This makes it a >>>> lot easier to use a debugger which I will definitely want to do. >>>> >>>> Otherwise this is a cool project and we already found 1 or 2 issues we >>>> can fix in HSAIL Graal related to your clojure tests. >>>> Regards, >>>> Eric >>>> >>>> >>>> >>>> On 03/17/2014 06:43 AM, Doug Simon wrote: >>>>> Hi Julian, >>>>> >>>>> In terms of what to test, I?ll leave it up to the Sumatra developers >>>>> at AMD and Oracle to comment on what may be interesting. >>>>> >>>>> As far as I know, no one in the Graal team is that familiar with >>>>> Clojure. This actually makes the value of having Clojure tests >>>>> integrated into our gate system somewhat problematic - >>>>> debugging/fixing failures may be difficult. I?d suggest you report >>>>> failures your discover on the sumatra-dev and graal-dev lists and we >>>>> can resolve them that way for now. >>>>> >>>>> In addition, for Graal we?re always interested in benchmark suites for >>>>> JVM hosted languages. Can you recommend any well known and trusted >>>>> Clojure benchmarks we should consider? >>>>> >>>>> -Doug >>>>> >>>>> On Mar 17, 2014, at 12:35 AM, Jules Gosnell >>>>> wrote: >>>>> >>>>>> Doug, >>>>>> >>>>>> Sorry it has taken me a while to get back - I've been busy playing >>>>>> with code. >>>>>> >>>>>> I currently have a small clojure testsuite up and running. I plan to >>>>>> broaden this and then think about providing it in a junit-compatible >>>>>> way to your project. >>>>>> >>>>>> I still haven't made up my mind exactly what I should be testing. It >>>>>> seems unnecessary to duplicate all your tests in Clojure, since you >>>>>> are already guarding against regression in these areas - I think I >>>>>> will start out with a fairly general set of tests as I probe for >>>>>> particular problem areas for Clojure, into which I may dive for more >>>>>> detail - time will tell... if you have any ideas, please let me know. >>>>>> >>>>>> I've checked my stuff in at github, if anyone is interested - here is >>>>>> the project: >>>>>> >>>>>> https://github.com/JulesGosnell/clumatra >>>>>> >>>>>> here is some clojure that I reverse engineered from GraalKernelTester: >>>>>> >>>>>> https://github.com/JulesGosnell/clumatra/blob/master/src/clumatra/core.clj >>>>>> >>>>>> >>>>>> >>>>>> and here are the first successful tests: >>>>>> >>>>>> https://github.com/JulesGosnell/clumatra/blob/master/test/clumatra/core_test.clj >>>>>> >>>>>> >>>>>> >>>>>> not very pretty yet - I am still feeling my way. >>>>>> >>>>>> cheers >>>>>> >>>>>> >>>>>> Jules >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On 06/03/14 10:24, Doug Simon wrote: >>>>>>> Hi Julian, >>>>>>> >>>>>>> This looks very interesting and will be an good alternative testing >>>>>>> vector for the Sumatra work as it matures. If Clojure tests can >>>>>>> somehow be made to run from Junit, then I?m sure we can try >>>>>>> integrating it into our testing. >>>>>>> >>>>>>> -Doug >>>>>>> >>>>>>> On Mar 6, 2014, at 12:56 AM, Julian Gosnell >>>>>>> wrote: >>>>>>> >>>>>>>> Guys, >>>>>>>> >>>>>>>> I just built the Java8 / Graal / Okra stack and managed to run some >>>>>>>> very simple Clojure copying the contents of one int array into >>>>>>>> another one on Okra. >>>>>>>> >>>>>>>> https://groups.google.com/forum/#!topic/clojure/JpjK__NTR5Y >>>>>>>> >>>>>>>> >>>>>>>> I find the ramifications of this very exciting :-) >>>>>>>> >>>>>>>> I understand that fn-ality is limited at the moment - but I am keen >>>>>>>> to keep testing and to help ensure early visibility of Clojure >>>>>>>> related issues to both communities - perhaps even the submission of >>>>>>>> a Clojure testsuite if Graal developers thought that useful. >>>>>>>> >>>>>>>> I'd be very interested in your thoughts on Graal / Clojure. >>>>>>>> >>>>>>>> regards, >>>>>>>> >>>>>>>> >>>>>>>> Jules >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>> >>>> >>> >>> >>> >>> >> > From cmaniak at o2.pl Wed Apr 16 12:27:08 2014 From: cmaniak at o2.pl (=?UTF-8?Q?cmaniak?=) Date: Wed, 16 Apr 2014 14:27:08 +0200 Subject: =?UTF-8?Q?Sumatra_on_i5_cpu?= Message-ID: <323a6dc7.7f3fb3e9.534e771c.1bf82@o2.pl> Hello, I'm fascinated with idea of Project Sumatra and want to run it on my laptop with intel i5 cpu. Please correct me if I'm wrong 1)When I use cpu without HSA functionality (for example i5) should I use Okra-Interface-to-HSAIL-Simulator? 2)Are there any benefits in performance using Sumatra with Okra-Interface-to-HSAIL-Simulator? I know that I can master new syntax with lambdas, but I'm rather interested in performance. Are there any differences in pure jdk8 and Sumatra build with okra simulator? 3)What are your expieriences with nvidia graphic card? I have nvidia geforce 660m in my laptop. Can I use it with Sumatra? 4)What for a project Graal is? Is it required to run with Sumatra? Regards, Leszek Korpowski From duboscq at ssw.jku.at Wed Apr 16 14:46:38 2014 From: duboscq at ssw.jku.at (Gilles Duboscq) Date: Wed, 16 Apr 2014 16:46:38 +0200 Subject: Nodes in Graal: ParameterNode In-Reply-To: <1395238578.29018.44.camel@gofio> References: <1395163053.29018.17.camel@gofio> <1395238578.29018.44.camel@gofio> Message-ID: Hello Juan, I don't think you should try to modify the argument at the start of the compilation, i would rather do the transformation at the end of the compilation. In order to do your transformation, you could do something similar to what was done by AMD for the Sumatra HSAIL backend: You have a method such as: int[] apply(int[] x, int[]y, IntBinaryOperator op) that can be implemented with int[] apply(int[] x, int[]y, IntBinaryOperator op) { assert x.length == y.length; int[] result = new int[x.length]; for (int i = 0; i < x.length; i++) { result[i] = op.applyAsInt(x[i], y[i]); } return result; } However, it can also be implemented as: int[] apply(int[] x, int[]y, IntBinaryOperator op) { assert x.length == y.length; ResolvedJavaType type = meta.lookupJavaType(op.getClass()); ResolvedJavaMethod concrete = type.resolveMethod(applyAsIntMethod); Kernel k = openclBackend.compileAndInstallKernel(concrete); k.execute(x.length, op, x, y) } This means you are actually compiling a method with the int IntBinaryOperator.applyAsInt(int, int) signature. Maybe other can comment on how this is achieved in HSAIL but i suppose this int[] -> int transform is done during the prolog/epilog which is emitted for argument handling. You can know which arguments should actually be pulled from arrays an add some code to extract them. For example with: int[] x = ... int[] y = ... apply(x, y, (a, b) -> a + b); You would end up compiling something like: int applyAsInt(int a, int b) { return a +b; } which in OpenCL probably will look very similar: ret = a + b; (Notice how the ReturnNode is lowered to a variable assignment rather than an actuall return) and you can emit the right prolog and epilog code: __kernel void applyAsInt (__global const int* a_array, __global const int* b_array, __global int* res_array) { const int gid = get_global_id(0); int a = a_array[gid]; int b = b_array[gid]; ret = a + b; res_array[gid] = ret; } This would typically be done in the com.oracle.graal.compiler.target.Backend.emitCode implementation. For example for HSAIL in com.oracle.graal.hotspot.hsail.HSAILHotSpotBackend.emitCode. -Gilles On Wed, Mar 19, 2014 at 3:16 PM, Juan Jose Fumero wrote: > Hello, > Thanks Gilles for your explanation. As you said, map in stream.util is > IntUnaryOperator. What I am using now is BiFunction. Maybe I > should change to a specific one as IntBinaryOperator. > > More specifically I am trying is this (this is an example): > > // Vector Addition > Stream stream = new Stream<>(); > stream.map((x,y) -> x + y).run(input); > > This is now running in CPU with my library. This map does not have side > effects and each operation returns a new Stream. When the run is called, > the pipeline is executed. What I am trying is to generate OpenCL code > from the map lambda expression, but I need to "transform" or replace > Parameters x and y to Arrays for example. > > Is there other ways available in Graal? > > Thanks > Juanjo > > On Wed, 2014-03-19 at 11:49 +0100, Gilles Duboscq wrote: >> Hello, >> >> The graph you present is the code for: >> >> Integer foo(Integer x, Integer y) { >> return x + y; >> } >> >> There are no arrays involved and you can not force the parameters to >> be arrays, that would just not work. >> >> If you want to work with integer streams you can use IntStream which >> will allow you to work without the boxing (even for the lambdas). >> I'm not sure what the example should do exactly since map seems to >> only accept unary functions. >> In any case, this lambda would be the kernel which is the "what to >> run" and thus does not/can not contain any information about "what to >> run *on*". >> This information can only be available in the code which is applying >> this lambda to some specific data. >> >> Maybe you can give a small example of what you would like to achieve >> from java code to OpenCL code? >> >> -Gilles >> >> On Tue, Mar 18, 2014 at 6:17 PM, Juan Jose Fumero wrote: >> > Hello, >> > I am working with lambda expression on JDK8 and Graal. My question is >> > related with the creation of new nodes in the Graph to update or change >> > the information. >> > >> > >> > Given this lambda expression: >> > >> > stream.map((x,y) -> x + y); >> > >> > The StructuredGraph is the following: >> > >> > 0|StartNode >> > 1|Parameter(0) >> > 2|Parameter(1) >> > 3|FrameState at 0 >> > 4|MethodCallTarget >> > 5|Invoke#intValue >> > 6|FrameState at 4 >> > 8|MethodCallTarget >> > 9|Invoke#intValue >> > 10|FrameState at 8 >> > 12|+ >> > 13|MethodCallTarget >> > 14|Invoke#valueOf >> > 15|FrameState at 12 >> > 17|Return >> > 13|Invoke#valueOf >> > >> > >> > Parameter(0) and Parameter(1) are Objects in the moment that I get the >> > lambda expression. But later on, I know that could be Integer[], >> > Double[], etc. >> > >> > I would like to rewrite this part of the Graph with the new information. >> > Is there any utility in Graal to do that? >> > >> > >> > For instance: if I get the parameterNode from the previous graph: >> > >> > if (((ObjectStamp) parameterNode.stamp()).type().isArray()) { >> > ... >> > } >> > >> > >> > The nodes Parameter(0) and Parameter(1) in the lambda expression are not >> > arrays. What I want to do is to change or update these nodes. What I am >> > using now is a new node (paramNode): >> > >> > >> > IntegerStamp integerStamp = new IntegerStamp(Kind.Int); >> > ParameterNode paramNode = new ParameterNode(index, integerStamp); >> > newGraph.unique(paramNode); >> > >> > But I need also to store the array information (size and dimension). The >> > aim is facilitates the OpenCL code generation from this expression. >> > >> > >> > Many thanks >> > Juanjo >> > >> > >> > -- >> > The University of Edinburgh is a charitable body, registered in >> > Scotland, with registration number SC005336. >> > >> > > > > -- > The University of Edinburgh is a charitable body, registered in > Scotland, with registration number SC005336. > From Gary.Frost at amd.com Wed Apr 16 15:38:46 2014 From: Gary.Frost at amd.com (Frost, Gary) Date: Wed, 16 Apr 2014 15:38:46 +0000 Subject: Sumatra on i5 cpu In-Reply-To: <323a6dc7.7f3fb3e9.534e771c.1bf82@o2.pl> References: <323a6dc7.7f3fb3e9.534e771c.1bf82@o2.pl> Message-ID: Hi Leszek Some answers to some of your questions. 1)When I use cpu without HSA functionality (for example i5) should I use Okra-Interface-to-HSAIL-Simulator? Yes. Unless you have a HSA enabled APU you will need to use the HSAIL simulator. 2)Are there any benefits in performance using Sumatra with Okra-Interface-to-HSAIL-Simulator? I know that I can master new syntax with lambdas, but I'm rather interested in performance. Are there any differences in pure jdk8 and Sumatra build with okra simulator? You should not expect code executing on the HSAILSimulator to be more performant than code running on a standard Java 8 JVM. The HSAILSimulator is so that we can check functionality and to help debug our code. 3)What are your expieriences with nvidia graphic card? I have nvidia geforce 660m in my laptop. Can I use it with Sumatra? I am aware that there is indeed some work going into the Graal project to support NVidia devices. Sumatra currently uses Graal for code generation, but the current code base for Sumatra does not (for example) dispatch IntStream/ObjectStream lambdas to NVidia hardware as far as I am aware. Maybe others on the list more familiar with the NVidia progress can comment further here. 4)What for a project Graal is? Is it required to run with Sumatra? As I hinted above Sumatra uses Graal to generate HSAIL from Java bytecode. HSAIL is an intermediate form for HSAIL compatible devices. So Sumatra uses Graal to convert bytecode (at runtime) into HSAIL for execution on either real hardware or on the HSAILSimulator. Hope this helps. Gary -----Original Message----- From: sumatra-dev [mailto:sumatra-dev-bounces at openjdk.java.net] On Behalf Of cmaniak Sent: Wednesday, April 16, 2014 7:27 AM To: sumatra-dev at openjdk.java.net Subject: Sumatra on i5 cpu Hello, I'm fascinated with idea of Project Sumatra and want to run it on my laptop with intel i5 cpu. Please correct me if I'm wrong 1)When I use cpu without HSA functionality (for example i5) should I use Okra-Interface-to-HSAIL-Simulator? 2)Are there any benefits in performance using Sumatra with Okra-Interface-to-HSAIL-Simulator? I know that I can master new syntax with lambdas, but I'm rather interested in performance. Are there any differences in pure jdk8 and Sumatra build with okra simulator? 3)What are your expieriences with nvidia graphic card? I have nvidia geforce 660m in my laptop. Can I use it with Sumatra? 4)What for a project Graal is? Is it required to run with Sumatra? Regards, Leszek Korpowski From eric.caspole at amd.com Mon Apr 21 16:20:19 2014 From: eric.caspole at amd.com (eric.caspole at amd.com) Date: Mon, 21 Apr 2014 16:20:19 +0000 Subject: hg: sumatra/sumatra-dev/jdk: Summary: This change will avoid possibly running the lambda twice in the event of an HSAIL deoptimization Message-ID: <201404211620.s3LGKPbj004257@aojmv0008> Changeset: 11adb0eba927 Author: ecaspole Date: 2014-04-21 12:19 -0400 URL: http://hg.openjdk.java.net/sumatra/sumatra-dev/jdk/rev/11adb0eba927 Summary: This change will avoid possibly running the lambda twice in the event of an HSAIL deoptimization ! src/share/classes/java/util/stream/PipelineInfo.java From jules_gosnell at yahoo.com Mon Apr 21 20:59:24 2014 From: jules_gosnell at yahoo.com (Jules Gosnell) Date: Mon, 21 Apr 2014 21:59:24 +0100 Subject: Graal and Clojure In-Reply-To: <4481FFF5-F9FD-4318-9528-96005AF40056@oracle.com> References: <1394063789.8387.YahooMailNeo@web122406.mail.ne1.yahoo.com> <5326354D.8010204@yahoo.com> <53277532.3030807@amd.com>, <532B5D35.4010108@yahoo.com> <5341631B.1090207@yahoo.com> <53424392.8060700@yahoo.com> <4481FFF5-F9FD-4318-9528-96005AF40056@oracle.com> Message-ID: <535586AC.1080408@yahoo.com> Thanks for your interest, Thomas, Progress is currently in the form of tests - currently around 500 - most are generated by a huge and hairy clojure macro ! These are run on a nightly build of graal/jdk8 against a pure clojure control, okra simulator and okra with HSA hardware. The tests are totally clojure-centric - I didn't see any point in replicating the graal unit test suite. I currently have a small number of tests which actually crash the okra simulator. I can provide more detail if anyone is interested,... The tests exercise some of the java components of the clojure runtime. In order to start testing actual Clojure code I need support for everything required to compile a Clojure Function. Some part[s] of this is/are currrently still "unimplemented". I would be very interested in pursuing this with an interested Graal developer. Meanwhile, I will keep extending my testsuite and hoping to see more and more tests passing as graal/okra development continues. Thanks guys, Jules P.S. I would repost all the relevant links, but apparently the dyndns service that I use tends to cause my posts to be filtered...You can find them back up this thread. On 16/04/14 12:13, Thomas Wuerthinger wrote: > Cool! It is very exciting to see so much progress with Clojure running on the GPU via Graal. - thomas > > On 07 Apr 2014, at 08:20, Jules Gosnell wrote: > >> Good news - I reapplied the -XX:-UseHSAILDeoptimization flag (last time I tried it the code had not yet made it into my jdk8/graal build) and a lot of the instability mentioned in my last mail disappeared - particularly in the simulator. Many more tests also passed :-) >> >> http://ouroboros.dyndns-free.com/ci/job/clumatra-hardware-acceleration/ >> http://ouroboros.dyndns-free.com/ci/view/clumatra/job/clumatra-simulated-acceleration/ >> >> >> Jules >> >> On 06/04/14 15:22, Jules Gosnell wrote: >>> So, Guys, >>> >>> Here is the promised update on Clumatra - an effort to get Clojure up >>> and running on top of Graal/Okra/HSA. >>> >>> - I now have a working Ubuntu/AMD HSA box. >>> >>> - I have a Jenkins nightly build of JDK8 / Graal : >>> >>> http://ouroboros.dyndns-free.com/ci/view/clumatra/job/graal/ >>> >>> - if useful, you can download the latest linux/64 tgz here (it is 300mb >>> - I'm going to look at shrinking it: >>> >>> http://ouroboros.dyndns-free.com/clumatra/jdk1.8.0-graal.tgz >>> >>> - I have builds of Clumatra without, with simulated and with hardware >>> based HSA : >>> >>> http://ouroboros.dyndns-free.com/ci/view/clumatra/ >>> >>> - The clumatra testsuite is growing - results have been a bit unstable >>> recently as I learnt which tests completely take out or hang the >>> jdk8/graal jvm and which can be trusted to succeed / fail in a well >>> behaved fashion: >>> >>> http://ouroboros.dyndns-free.com/ci/view/clumatra/job/clumatra-hardware-acceleration/ >>> >>> >>> I am working on a reflection/macro based solution, which will >>> automagically create tests for all methods of all classes that I want >>> tests for. When I bring this online in a couple of weeks the number of >>> tests should jump dramatically. >>> >>> Please feel free to inspect test results etc on my Jenkins and let me >>> know whether they are helpful and whether they can be made more useful >>> in any way. currently some of them seem to indicate problems in the okra >>> backend. This is when graal seems to run happily but I get back the >>> wrong result from my test: >>> >>> e.g. >>> >>> http://ouroboros.dyndns-free.com/ci/view/clumatra/job/clumatra-hardware-acceleration/83/testReport/clumatra/core-test/long_unchecked_inc_test/ >>> >>> >>> I run the test via graal/okra twice and once in pure clojure - so you >>> see three results. The last one is the expected one, the first two the >>> ones returned from the GPU. >>> >>> Other tests let me know that graal is not yet implementing various >>> features of jvm bytecode - e.g. >>> >>> http://ouroboros.dyndns-free.com/ci/view/clumatra/job/clumatra-hardware-acceleration/83/testReport/clumatra/core-test/long_my_static_inc_test/ >>> >>> >>> Unfortunately the test report does not seem to be capturing the HSAIL >>> logging that you can see in the console output... My guess is that it is >>> captured by redirecting clojure's usual output stream, but this is not >>> the same one as being used by Graal... >>> >>> http://ouroboros.dyndns-free.com/ci/view/clumatra/job/clumatra-hardware-acceleration/83/console >>> >>> >>> >>> That's enough for now. I'll post with more progress as and when. >>> >>> >>> >>> Jules >>> >>> >>> >>> On 31/03/14 23:40, Caspole, Eric wrote: >>>> Hi Jules, >>>> I updated everything and got Clumatra running on the real hardware today. >>>> >>>> In project.clj, I used this: >>>> >>>> :jvm-opts ["-server" "-ea" "-esa" "-Xms1g" "-Xmx1g" >>>> "-verbosegc" "-G:Log=CodeGen" >>>> "-XX:+GPUOffload" "-XX:+TraceGPUInteraction"] >>>> >>>> Then make sure your /path/to/okra/dist/bin is in the PATH and >>>> LD_LIBRARY_PATH. >>>> >>>> When it is using the simulator, you will see this: >>>> [HSAIL] using >>>> _OKRA_SIM_LIB_PATH_=/tmp/okraresource.dir_2595501614013009358/libokra_x86_64.so >>>> >>>> >>>> With hardware or external okra you will see this: >>>> >>>> [HSAIL] library is libokra_x86_64.so >>>> >>>> Regards, >>>> Eric >>>> >>>> >>>> ________________________________________ >>>> From: Jules Gosnell [jules_gosnell at yahoo.com] >>>> Sent: Thursday, March 20, 2014 5:27 PM >>>> To: Caspole, Eric; graal-dev at openjdk.java.net; >>>> sumatra-dev at openjdk.java.net >>>> Subject: Re: Graal and Clojure >>>> >>>> Eric, >>>> >>>> I just checked support to Clumatra for running individual unit tests >>>> from the CLI - should be enough to get you set up with your debugger. >>>> >>>> All details at the bottom of the README. >>>> >>>> https://github.com/JulesGosnell/clumatra >>>> >>>> I'm afraid you will need to install maven if you don't already have it - >>>> for some reason lein does not seem to have a test-jar target. >>>> >>>> It is still a little rough around the edges, but better than nothing :-) >>>> >>>> I also just bought my HSA h/w - should be up and running next week >>>> sometime. Did you figure out how to enable native-mode Okra ? >>>> >>>> catch you later, >>>> >>>> >>>> Jules >>>> >>>> >>>> >>>> On 17/03/14 22:20, Eric Caspole wrote: >>>>> Hi everybody, >>>>> I was setting up a new HSA system today and I installed Clumatra and got >>>>> it working under the simulator. I have a real HSA system not too >>>>> different from this: >>>>> >>>>> http://code.google.com/p/aparapi/wiki/SettingUpLinuxHSAMachineForAparapi >>>>> >>>>> In the clojure/lein/maven based system there is so much harness I can't >>>>> immediately see how to switch in the real hardware version of Okra >>>>> instead of the simulator Okra that gets installed into maven and >>>>> exploded in /tmp when the tests run. >>>>> >>>>> Jules, could you show how to bypass the harness etc to switch the Okra? >>>>> Can I just run 1 test with a simple java command line? This makes it a >>>>> lot easier to use a debugger which I will definitely want to do. >>>>> >>>>> Otherwise this is a cool project and we already found 1 or 2 issues we >>>>> can fix in HSAIL Graal related to your clojure tests. >>>>> Regards, >>>>> Eric >>>>> >>>>> >>>>> >>>>> On 03/17/2014 06:43 AM, Doug Simon wrote: >>>>>> Hi Julian, >>>>>> >>>>>> In terms of what to test, I?ll leave it up to the Sumatra developers >>>>>> at AMD and Oracle to comment on what may be interesting. >>>>>> >>>>>> As far as I know, no one in the Graal team is that familiar with >>>>>> Clojure. This actually makes the value of having Clojure tests >>>>>> integrated into our gate system somewhat problematic - >>>>>> debugging/fixing failures may be difficult. I?d suggest you report >>>>>> failures your discover on the sumatra-dev and graal-dev lists and we >>>>>> can resolve them that way for now. >>>>>> >>>>>> In addition, for Graal we?re always interested in benchmark suites for >>>>>> JVM hosted languages. Can you recommend any well known and trusted >>>>>> Clojure benchmarks we should consider? >>>>>> >>>>>> -Doug >>>>>> >>>>>> On Mar 17, 2014, at 12:35 AM, Jules Gosnell >>>>>> wrote: >>>>>> >>>>>>> Doug, >>>>>>> >>>>>>> Sorry it has taken me a while to get back - I've been busy playing >>>>>>> with code. >>>>>>> >>>>>>> I currently have a small clojure testsuite up and running. I plan to >>>>>>> broaden this and then think about providing it in a junit-compatible >>>>>>> way to your project. >>>>>>> >>>>>>> I still haven't made up my mind exactly what I should be testing. It >>>>>>> seems unnecessary to duplicate all your tests in Clojure, since you >>>>>>> are already guarding against regression in these areas - I think I >>>>>>> will start out with a fairly general set of tests as I probe for >>>>>>> particular problem areas for Clojure, into which I may dive for more >>>>>>> detail - time will tell... if you have any ideas, please let me know. >>>>>>> >>>>>>> I've checked my stuff in at github, if anyone is interested - here is >>>>>>> the project: >>>>>>> >>>>>>> https://github.com/JulesGosnell/clumatra >>>>>>> >>>>>>> here is some clojure that I reverse engineered from GraalKernelTester: >>>>>>> >>>>>>> https://github.com/JulesGosnell/clumatra/blob/master/src/clumatra/core.clj >>>>>>> >>>>>>> >>>>>>> >>>>>>> and here are the first successful tests: >>>>>>> >>>>>>> https://github.com/JulesGosnell/clumatra/blob/master/test/clumatra/core_test.clj >>>>>>> >>>>>>> >>>>>>> >>>>>>> not very pretty yet - I am still feeling my way. >>>>>>> >>>>>>> cheers >>>>>>> >>>>>>> >>>>>>> Jules >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 06/03/14 10:24, Doug Simon wrote: >>>>>>>> Hi Julian, >>>>>>>> >>>>>>>> This looks very interesting and will be an good alternative testing >>>>>>>> vector for the Sumatra work as it matures. If Clojure tests can >>>>>>>> somehow be made to run from Junit, then I?m sure we can try >>>>>>>> integrating it into our testing. >>>>>>>> >>>>>>>> -Doug >>>>>>>> >>>>>>>> On Mar 6, 2014, at 12:56 AM, Julian Gosnell >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Guys, >>>>>>>>> >>>>>>>>> I just built the Java8 / Graal / Okra stack and managed to run some >>>>>>>>> very simple Clojure copying the contents of one int array into >>>>>>>>> another one on Okra. >>>>>>>>> >>>>>>>>> https://groups.google.com/forum/#!topic/clojure/JpjK__NTR5Y >>>>>>>>> >>>>>>>>> >>>>>>>>> I find the ramifications of this very exciting :-) >>>>>>>>> >>>>>>>>> I understand that fn-ality is limited at the moment - but I am keen >>>>>>>>> to keep testing and to help ensure early visibility of Clojure >>>>>>>>> related issues to both communities - perhaps even the submission of >>>>>>>>> a Clojure testsuite if Graal developers thought that useful. >>>>>>>>> >>>>>>>>> I'd be very interested in your thoughts on Graal / Clojure. >>>>>>>>> >>>>>>>>> regards, >>>>>>>>> >>>>>>>>> >>>>>>>>> Jules >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>> >>>> >>>> >>>> >>> >> > >